- in Blog by David Wilkinson
The Enablers of Evidence-Based Management – Research Interview
Moving into a more evidence-based approach to management within organisations is often a noble cause, but the actual process is often fraught with a range of problems and issues. Understanding what enables an evidence-based management approach within an organisation is a key research issue.
In this podcast researcher Christian Criado-Perez from The University of New South Wales, one of the authors of a recent research paper “Enablers of evidence-based management: Clues from the absorptive capacity literature” looks at the role of absorptive capacity or the capability or capacity of individuals and organisations to learn.
In the interview Christian explores what absorptive capacity is and why it has an impact on an organisation’s efforts to develop evidence-based management practices.
Christian Criado-Perez
Podcast
Transcript
Evidence-Based Management
– Hello, and welcome back to the podcast. I’m David Wilkinson, the editor of the Oxford Review. And today, we have Christian. Do you want to just pronounce your surname?
– Yes, it’s a Spanish surname, rather long Creado-Perez, I guess would be the English way of pronouncing it.
– Got it, Creado-Perez. Right, brilliant, thanks Christian. So what we’re going, Christian’s together with some colleagues from New South Wales University, have published a paper called “Enablers of Evidence-based Management: “Clues From Absorptive Capacity Literature,” which I find particularly interesting, because I’ve got quite a big interest in learning, but also absorptive capacity. So welcome, Christian. And I just wondered if you could just kind of kick us off by telling us a little bit about yourself, kind of your background, your research history and interests, and we’ll take it from there.
– Sure. So I began my career in the aerospace industry, as an engineer, and I worked in manufacturing, and project management for several years. And during that time I studied an MBA, which gave me my first exposure to things like journal articles, case studies, scientific research in general. And it was eyeopening for me really, to realize how much scientific research could actually help us understand management problems. But also the lack of awareness, and lack of adoption of these practices. It was as if all the scientific research just did not exist in my work environment. And at the time, I bumped into the Center for Evidence-based Management, called CEBMa, and the concept itself of evidence-based practice. And I just thought, you know, I want to make this happen around me. How come we’re okay with this not being the norm? I also suddenly realized, you know, am I just blindly following management fads too? So since then, I quit, my job, went back to studying to pursue my PhD here at UNSW. And I’ve been doing research on evidence-based management and decision-making in general for the past four to five years.
– Okay, and what would you PhD on?
– So it’s in the school of management, focusing on the enablers and barriers of evidence-based management.
– Okay, right. Cool. Brilliant. Okay, so enablers of evidence-based management. Can we just kind of, what are we talking about here when we’re talking about evidence-based management, and why did it kind of grab you?
– Well, as I said, it was surprising to me how all of the these, all of his knowledge from scientific research had been around for ages. And although it was rather old stuff, the thing, the content I was learning in my MBA, the managers around me were completely unaware of of this body of knowledge. And so when I started looking into the, I started looking into the concept of evidence-based management. And perhaps I’ll give a general definition of what I mean by that. So evidence-based management is a framework, really, for making decisions. And it focuses on relying on the best available evidence from multiple sources. So as such, it emphasizes firstly, the need to focus on the quality of the evidence that you’re relying on, but also to collect evidence from multiple sources, and it lists four sources as key sources you can rely on, one being the expertise off the decision makers themselves. Also, organizational data that might be relevant, or then the concerns of the stakeholders involved. And lastly, the body of scientific research that might be relevant. And evidence-based management will propose us to rely on those four sources, and follow six steps that lead to increasing the likelihood of the desired outcome. Those six steps being well first, identifying the problem, and translating it into an answerable question. So in a way, like setting up your hypothesis. Then also acquiring the evidence, appraising the quality of that evidence, to later aggregate it properly and apply it to your decision. And finally, assessing the outcomes to learn from the application of that evidence, and to inform future decisions. So that’s what we, that whole framework is what we’re referring to when we talk about evidence-based management.
– Yeah, yeah. And it, I’m like you, I kind of find it incredible that there’s this massive body of evidence that the vast majority of practitioners A, don’t know about, and B, don’t have access to, and just generally tends not to get used at all within kind of, decision-making not only within management, but anything to do with kind of organizations, which is one of the reasons why we’re here, I suppose. So your paper, what I found interesting, and when I first kind of scanned it right at the beginning, I went, “Oh, absorptive capacity,” is kind of interesting putting those two things together. So do you just want to explain a little bit about absorptive capacity for the listeners, what it is, and why it’s important?
– Sure. So yeah, absorptive capacity is, it refers to the ability to identify valuable knowledge from your external environment, but also to, for a firm to assimilate it, and exploit that knowledge. So different researchers have separated absorptive capacity into different dimensions in a number of ways. But the overall idea is that absorptive capacity allows firms to learn, innovate, and as a result, increase firm’s performance. And, well I mean, it’s obvious, beside the obvious fact that it’s important for an organization to be able to learn from an external environment, we do have recent meta-analyses that provide supporting evidence that it is, absorptive capacity does lead to both higher levels of innovation, and firm performance.
– Yes, yeah. And that we can start to look at organizations and see whether they’ve got higher levels of absorptive capacity, or lower levels of absorptive capacity in terms of their ability to be able to, I suppose, learn, generally, and use materials, particularly from the outside, as you were saying, and learning from outside. And that, one of the things that intrigues me, and I don’t know, and this is kind of like, it’s not part of the interview that I have asked you, but it was something that I was thinking about a little bit earlier on. Because we’re describing it as a capacity. So it’s that capacity to be able to learn. And how do we go about evaluating an organization’s capacity to learn? So what are the kind of factors that might be involved in that?
– Yeah, so I guess it’s a, you’ve got different research have measured that in a whole range of ways. And also depending on the ways in which they define each of those dimensions. But for example, there are extremes that, so to give you an example that I don’t personally think are as good, for example, looking at how much a firm is investing in R and D. And some papers have used that as a proxy of absorptive capacity. I don’t think that’s a very, I don’t think it’s an ideal measure of absorptive capacity. And then you also have many, many, a lot of research pretty much just surveying the senior management off the team to try to capture to what degree they look outside their firm for innovation, and for new ideas. And also look into the process with which they try to assimilate that knowledge, and disseminate it throughout the organization to be able to use it, and come up with innovative solutions. So I guess there are a lot of things that can get into the way of that, right. I mean, first they, like we were discussing before, very often managers are looking at their immediate environment, but not necessarily outside their organizations. We’re also not necessarily good at evaluating the value of new knowledge, and the quality of the evidence that we are exposed to. And even if we do, of course, bring in this new knowledge into a large organization, and integrating it within the products and the processes of that organization is not an easy task. You’re surely gonna face a lot of resistance and miscommunications, right.
– Yeah, and there are a lot of barriers in organizations to that kind of thing, particularly, kind of department silos, and things like that, the culture, whether there’s a learning culture, or a knowing culture, all those kinds of things as well. So do you just want to kind of talk us through the paper a little bit in terms of what you found from your meta-analysis?
– Sure. So we pretty much look at all, we looked for empirical research that had examined what might be a predictor or an antecedent of absorptive capacity. And we found, to summarize, perhaps I can highlight three key factors that we found that were associated with absorptive capacity at the firm level, and three key factors at the individual level. So at the firm level, our meta-analysis found that the strongest factor that was associated with it was information systems capabilities. So that is the capability of a firm to collect, or of their information system, to collect and store data, making it easily accessible across the organization was a key factor. And I think that also explains why firms are investing very very heavily in IT. We also found that transformational leadership was particularly important. Also, and a degree of slack resources. So what I mean by that is, they need to create a working environment that encourages employees to question their assumptions, that fosters inquiry and exploration, but also to have the time and the resources to do so, right. And then thirdly, there were a whole range of constructs surrounding the idea of collaboration that were key for absorptive capacity. Collaboration across functions within the company, but also with stakeholders beyond the organization. So that reflects the importance of what you were mentioning of breaking down those silos within their organization, but also to interact with clients, our customers, and maybe even competitors, right. Then at the individual level, we found that managers had to have the ability, the motivation, and the opportunity to learn from external environment. And by that, I mean, well, firstly they had to have a certain level of expertise in the domain. And this is necessary to understand and appreciate the value of that knowledge. A motivation to engage in learning tasks. So like you were saying, yeah, the motivation to learn. So, it can be through learning goals. And lastly, to have employees that are empowered to actually do so, to go out and seek for new solutions, right, which is not always the case.
– No, lots of organizations are very inward-looking, and really don’t encourage people A, to go out and have a look at other things outside or bring them in. And that’s, they kind of compartmentalize the knowledge in as much as, you know, IT knowledge depends, you know, goes to the IT department, and management knowledge don’t interfere in those, and don’t start bringing in new ideas about something that’s not your area. So we see that a lot in organizations, which is one huge barrier to organizational learning.
– Yep, that’s right.
– Interesting. Can I, I’m just gonna kinda, you published another paper a little bit earlier, “Beyond an Informed Opinion: “Evidence-based Practice in the Built Environment.” What I’m interested in here is, and this kind of connects to the learning thing, is, can you just explain a little bit about what you mean by beyond an informed opinion?
– Yeah, so that title was inspired from the conversations that I had with senior managers in the built environment from another research, well, from that paper. And when I was asking them about their levels of evidence-based management adoption, they often looked surprised by the question, and answered things like, “Well of course our decisions are evidenced-based,” you know, how could they not be? But by looking a little closer, and asking a few more questions, it was, it became very clear that they had a different definition in mind when we were talking about evidence-based management. So they were often referring to the use of their judgment and expertise, which was informed of course, by their experience. And very often feedback that they were getting from previous projects, or previous clients. The problem was that they were often relying on anecdotal feedback, on very low-quality evidence. And they were often ignoring evidence from a wide number of key sources that they could have used. So it’s one thing to have an opinion based on personal experience or, you know, based on your team of experts, which is valuable. But if you don’t go beyond that, you’re turning a blind eye to a lot of potentially strong and relevant evidence.
– Yes.
– Yeah, so that’s, I guess the difference that we were trying to imply through that type.
– Yeah, that’s interesting. And certainly this, so one of the concepts that we talk about in psychology is this idea of cognitive tunneling, and that the lack of capacity, at times, this tends to happen particularly when we’re stressed, but some people seem to make more a habit of it, I suppose, of not being able to notice, and take in peripheral information, and deal with things that are kind of outside, which is what we’re talking about with absorptive capacity, I suppose, is, is that capacity to learn is more than just the process of learning. It also includes that process of being open to new information from disparate sources, and information that comes at us in disparate ways, you know, I suppose, in different types of formats. And that idea of absorptive capacity involves that, that wider ability to be able to see things that are on the periphery, that aren’t part of the core, but actually inform it, or are a different mode of knowledge, a different mode of information, like research information. So, and we see in organizations, quite a lot of people it’s like, it’s almost too difficult. And I don’t think academics make it any easier because of the way that we write, and things like that. You know, I think we make it more opaque than anything else for practitioners. But I think it’s becoming, the whole idea of absorptive capacity, is kind of getting to a, kind of a critical point for organizations, particularly as we’re kind of moving more and more into industry 4.0, and people are having to work with, well, robots, and, but with systems that can actually provide a wide range of information. And quite often, we’re limited by our own capacity to both see it, and think about it. So it’s not just having the information, it’s those kind of thinking skills, the critical thinking skills that come into being as well. I don’t know, I don’t know what your thoughts are about that.
– Yeah yeah, absolutely. I mean, there’s definitely a number of abilities, of skills required to be able to foster absorptive capacity, or to engage in evidence-based management. And for example, one of them, which you briefly touched on was, If you’re gonna be drawing from multiple sources, you’re going to find contradictory evidence. And it’s, you’re gonna increase the complexity in a way, because you’re gonna have to deal with a very different kind of evidence that may be pulling into different directions. And I can understand some managers being averse to that idea, and being scared of just making their problems even bigger. But it’s not necessarily about, you know, being paralyzed until you find that certainty. Because there probably isn’t a hundred percent certainty. It’s simply about being a little more informed, right, and seeing a little more of the bigger picture.
– Yeah, so if you’ve got better evidence for making decisions, then the decisions are becoming increasingly better.
– Exactly, exactly. And we have plenty of evidence showing that, you know, this leads to better outcomes. And regarding that tunnel vision that you were saying, in fact that’s one area that I’m really interested in, and I have some ongoing work on it, which is looking at the influence of emotions too. So if you have employees that are, you know, under a lot of stress, that’s definitely gonna influence the way they, the degree to which they engage in evidence-based management, for instance.
– Yes definitely, yes. Yeah, there’s quite a bit of evidence around that, and particularly around cognitive tunneling and emotion. There’s been quite a lot of work done on those kinds of areas. And also that, the capacity for dealing with paradoxes. There was a lovely study done about two or three years ago on the, actually, it might’ve been longer than that. Time, huh? That was done on the Finnish police, looking at, so after 2008, like many countries, Finland ended up with a stereotype. And what occurred was, and again, this was a common thing that happened as a result, was that policy decisions were made at governmental level that actually set up a series of paradoxes without anybody realizing. So they would, they said, right, okay, the Finnish police, you’re a whole group of different police forces. You’ve all got different uniforms, you’ve all got different processes and standards. Whilst it’s the same law, each force is operating individually. So we want you to combine all that, have the same uniform, have the same policies and procedures, have the same standards. Well that’s all right, until you start realizing that policing in Helsinki is very different to policing in the Arctic Circle. You know, where there are more reindeer than human beings. So it’s, you know, having the same uniform actually doesn’t start to make sense. What was interesting was how the police dealt with the issue. Firstly, the researchers discovered that they didn’t recognize the paradoxes. There was nobody in the whole system said, “Ah, we set up a paradox here.” So that limits your capacity to be able to actually deal with it. And then what they found was, that the police forces tend to go from one side of the paradox to the other, right to the extremes. So they’d set up a collective process. And then what they’d do is they’d say, “Ah, hang on a minute, we need a little additions to this.” And then they’d go back to. So there was nobody managing that process in terms of decision-making of going, “Whoa, hang on a minute.” So there’s like this lack of metacognition, if you want. An overview of the whole kind of decision-making process, which I found interesting. And that comes back to this cognitive tunneling, is that what it was doing was it was pushing people in one direction or the other to the extremes without the whole process being managed. And,
– Yeah. And I think, I mean, I’ve definitely seen that in my personal work experience, but I’m sure a lot of people have gone through the same thing, but we, there’s this new, you know, you’re presented with this new approach or methodology. For me, it was the lean manufacturing and the assembly line. But there are many, many others. And you, I mean typically, we don’t stop and question it enough, right. It’s more like, okay, let’s adopt it and go with it. It’s more about the efficiency of the implementation rather than actually questioning the assumptions of, that that system is gonna work.
– Yes, yeah. And it’s that kind of thinking, of being able to get on top of it, and get an overview of what we’re doing. You know, at least having somebody who’s doing that. Yeah. Interesting. So, okay Christian. So if you were to pull out the main takeaways from your study for practitioners, what would they be?
– So are we talking about the meta-analysis and absorptive capacity, or?
– Yeah sorry, yes, the meta-analysis.
– No sure, that’s fine. Well, as I listed before, I mean, there were a whole range of factors associated with it. Now as a cautionary note, this was, we found that most of the empirical studies that we identified and we were drawing evidence from were correlational, or cross-sectional sorry. Now that said, knowing that there are these factors, both at a firm, and at an individual level, that are associated with those sorts of capacity, there are a whole range of things that managers can do, and depending on their level of seniority, and place in the organization, right? So in the paper, we provide a checklist with different things that they can do. But for instance, you know, we would encourage managers to again, make sure that they’re, they have good information system capabilities, that they’re fostering that culture of questioning, and inquiry among their employees. That they’re doing all the efforts they can to break down those silos, whether it’s in, through multidisciplinary projects, or other ways in which you increase collaboration between different departments. And then keeping an eye, at the individual level, keeping, you know, valuing the ability of employees to be able to engage with this kind of task, right, of well, having the expertise to be able to identify valuable knowledge, but also making sure that they have the motivation and the opportunity to do so, the time to do so. And one thing that is really gonna influence all of that very heavily, and this we found in the meta-analysis, but also in the other study you mentioned, that we published before, was the strong influence of social norms. If an employee doesn’t perceive that the organization is encouraging and supporting them to go out of their way to make sure they’re relying on the best available evidence, to actually question the way things are done, they’re not gonna do it, right. Why would they run the risk? They’re just gonna go with what is commonly accepted in their firm. So those social norms are really, really important to make sure that the employee perceives that that is supported. And this aligns a lot with research that we had from evidence-based medicine, where nurses and doctors were talking about the importance of what’s called psychological safety within their teams, to be able to do that, to question assumptions, and to say, “Hey, you know, there might be a better way, “despite the fact that we’ve been doing this “like this for the past 10 years.”
– Yes. Yeah. And having, I suppose, thinking about as a manager, developing a learning and innovation culture, I suppose, or an orientation towards those things. One of the things that I have seen, in fact it was a company in the Middle East, where the manager every month, when they were having their, it’s a kind of a global virtual team. And he had this section in his meetings about what they’d seen from outside that would be useful inside, which was really cool actually, because he set it up as part of the meeting. And it became an expectation that you would find something and say, “Actually, they’re doing this. “Maybe, you know, maybe it was a good idea. “Maybe we can test it here.” And I thought that was a, like a really positive step forward. And even if they didn’t, you know, he would give them license to give it a go, and say, “Well, let’s have a go at it, let’s see what happens.” And that was making a huge headway actually, in that group. Massive.
– Yeah, that seems like an excellent routine to put in place, to nudge people to look beyond the organization. And, one challenge I hadn’t mentioned that I also think is stopping evidence-based management from happening too is, that employees and managers don’t necessarily have the skills to evaluate the quality of the evidence they’re seeing, right. So I think they appreciate, and actually from surveys that we have done, we have evidence to show that, supporting that they appreciate the value of consulting with evidence beyond the organization. But if you give them scientific research from academic journals, well, like you said, it’s not like we write them in a way that’s easy for them to digest, right. Where I think it might, it’s gonna be a little easier potentially is with, you know, all of the data that we are getting from the advances in technology. So in AI, and big data, and so forth. The organizations are gonna be able to tap into a lot more relevant evidence from within the organization, but also to understand their customers much better. All of these are, you know, pieces of information that can be really, really helpful for managers.
– Yes, and turning to aggregators and things like that. So that, you know, there are people who kind of sit in that space of both translating the research, but can also help with understanding what the quality of the research is, and how applicable it is in this set of circumstances. And it’s that, I suppose, you know, we would term in academia, you know, from our point of view, that’s what critical thinking is part, or that’s part of the critical thinking process, is being able to weigh up the evidence, and work out A, how relevant it is to this particular situation, but how, you know, how valid and reliable it is. But then we start using those kinds of terms. And, you know, anybody who’s not an academic starts to glaze over, it’s like, what? So, and it is having those translators. And I’ve seen one or two organizations have started to employ scientists to do just that. They set in, they’re kind of knowledge processes. Their job is to go and find stuff that’s relevant to the organization, and turn it into a form that is actually, I suppose, absorbable, if that’s such a word, that that can be readily absorbed within the organization in a way that people can actually understand it. Because one of the things that’s come out of a whole load of previous research about different sectors, is that they’ve got different ways of thinking. And those different ways of thinking about things, and seeing things, perceiving things from a, you know, so engineers see things in a very different way to psychologists do, to medics do, for example. And there’s some evidence that whilst cognitively, they’re not learning in a different way, they are learning in a different way through kind of language and what they’re looking for. And it’s making that kind of translation piece that I think is important to kind of, I suppose, increase the absorptive capacity within organizations. But also get people to engage with those four areas of evidence-based practice a bit more, so it’s a little bit more balanced rather than it being, as you were saying, anecdotal. This has been really interesting, actually.
– Yeah, yeah. But I agree with, I also see that trend of companies hiring people to help them digest that, that body of knowledge. I’ve seen it in the built environment, and also particularly tech companies, I think, are good at you know, I think more and more of them seem to be building their own research teams within their organization, to even do their own research in a more scientific way. So yeah, I think there’s quite a bit of progress. I still think we have a long way to go. Evidence-based management is definitely not the norm. It is lacking in most organizations. But yeah, there is some progress, definitely.
– Yeah, brilliant. Thank you, Christian, I really appreciate it. And I appreciate your time as well. Can I just ask, you know, how can our members, how can people contact you if they want to do so, you know, kind of if you’ve got a website, profile pages and things like that, and then what we’ll do is we’ll link to them.
– Yeah, sure. So, I mean, I have a profile website at the UNSW business school website. But of course they could contact me through my email, which is listed in that paper from the Australian Journal of Management. So my UNSW email, which is publicly available. And of course, through, you know, all the social networks that we use: LinkedIn, ResearchGate, Academia, and so forth. So I would be happy to discuss.
– Fantastic, I’ll put those on the show notes, and I’ll send you a link as well. It’s been fascinating. Thank you very much, Christian. I really appreciate it. And I appreciate your time.
– [Christian] Sure, thanks. Thanks for having me.
– [David] Great.
Be impressively well informed
Get the very latest research intelligence briefings, video research briefings, infographics and more sent direct to you as they are published
Be the most impressively well-informed and up-to-date person around...
