The evidence-based practice problem…
There is a big problem with evidence-based practice and why evidence-based practice isn’t worth it for most people…
As a pedlar of evidence based practice this post hasn’t been an easy one to write but after years of work in this area, and having a service based around evidence based practice, I have come to a conclusion: There is a really big evidence based practice problem that people outside of health and technology haven’t acknowledged.
In short there is a big problem with evidence-based practice.
People don’t want evidence-based practice
The problem with evidence based practice is that outside of areas like health care and aviation / technology is that most people in organisations don’t care about having research evidence for almost anything they do. That doesn’t mean they are not interesting in research but they are just not that interested in using the research to change how they do things – period.
Be impressively well-informed
Get your FREE organizational and people development research briefings, infographics, video research briefings, a free copy of The Oxford Review and more...
Evidence-based practice is a bit of a fail
Ok so lets look at what people at the top of the EBP tree say. Some recognise the problem…
Professor Rob Briner -Professor of Organizational Psychology at Queen Mary University London has a hobby horse and that is that almost every conference he attends is populated by evidence-based practice converts, people with skin the the game. People like researchers, or organisational development professionals or academics or practitioners turned academics.
Very few conferences have significant numbers of managers or workers from the ‘shop floor’ or operations’.
Attendance at such events and engagement generally appears to be largely confined to the academically minded converts outside of health and technology.
So why?
Considering the point of EBP is to engage operational people in organisations, to bring the research to them to help them make better decisions and work better, this appear to be a bit of a fail.
To underline that have a look at this…
I posted this: https://www.linkedin.com/pulse/6-important-factors-ensure-change-success-david-wilkinson on LinkedIn
At this moment in time to post has had 1 like, 2 comments and 1 share. Ok not great figures.
The first comment was this:
As you can see, not exactly a ringing endorsement for evidence based practice!
… and it got more likes than my original post!!
So who does engage with EBP?
Well largely, but not wholly occupations that have close ties with academia. I lecture in the Medical Sciences Division at the University of Oxford and a fair bit of my teaching is conducted in the hospitals in and around Oxford.
The professors and many of the lecturers are mainly, but not wholly, practicing medics. The University and the hospital, particularly the John Radcliffe or JR is actually part of the Oxford University Hospitals group. The same applies at the other university in Oxford, Oxford Brookes University that looks after nursing which is also heavily evidence-based.
Close academic – practice connection – really close
In fact they are so closely allied that it is impossible to distinguish between the academic side and the practice side. This I believe is an important aspect of this. The practitioners are the researchers and teachers and the researchers and teachers are often practitioners to ex-practitioners. It is hard to slide a sheet of paper between the two.
My lectures and session are in the operational context. My students are often on duty. They are doing the do, as they say. The people I teach are the next batch of lecturers and researchers, but they are the practitioners. Little separation.
Now this doesn’t mean that we don’t have partitioners who don’t teach and we don’t have researchers and lecturers who don’t practice. We do. Lots in fact.
But, and this is important –
The teachers and researchers are embedded in the culture of practice
and
the practitioners and operational staff are embedded in a culture of evidence and research.
So is it just a question of culture?
Well there is something else…
So what about the sectors, industries and organisations where EBP isn’t taking off?
Part of the the problem in the areas, sectors, industries and organisations where EBP isn’t taking off is this…
Who cares about using evidence for anything?
How to
So before I start on this let me first make an apology…. to Rob Briner. What you see here is a symptom of the cause of a lack of engagement.
Ok so let me start by saying I am not critiquing Rob or his message. I actually agree with most (not all) of what he says. Hey this is academia. Its all about the argument..
I get this in terms of rigour. It’s not without it’s problems – more of that later… but this roughly represents state of rigour in EBP.
But compared to porn and the weather it’s a bit freaking boring… no actually it’s a lot freaking boring unless you are an active researcher
and that is part of the problem – but before I advance this lets look at something…
The point Rob makes with this is that the lower we go down the hierarchy we go the more bias… well thats not quite true actually its an increased chance or risk of bias.
As we go up the hierarchy the better the chance or risk of finding the factors involved in causality also increases:
Ok all good yes?
So why is this a pyramid? Why is there more of the opinion anecdotes etc and fewer systematic reviews?
Well a couple of reasons…
Research dependencies
The first thing is that the research at the top depends on the stuff at the bottom.
You cant do a systematic review if there isn’t anything to review. Research works up the hierarchy, So, someone notices something and says thats odd or thats interesting and and mention it to someone else. They do a case study or write a blog. That gets picked up by someone doing research who says lets see what evidence there is for this and so on up the ladder.
In effect the things at the top DEPEND on the flow up the hierarchy from below them. As I said you cant have a systematic review if you haven’t got anything to review. So look at this…
How people make decisions at work
This comes from the CIPD HR Outlook Winter 2016/17 Report and is the result of a survey of 629 senior HR professionals and looks at how frequently they use what source of evidence when making business decisions.
80% of operational decision-making
All of these …. 80% of this chart are the very things that are on the very bottom rung of the Briner’s Hierarchy of evidence.
Research based decisions
And then you look at the numbers of people using these…
The figures tell a powerful story…
The figures speak for themselves…
At the bottom 22% using research always or often and
at the top 77% are using personal experience always or often
Its not even close…
Intuition beats evidence-based practice into a cocked hat…
Even intuition gets a better press than research!
But why? Are these uneducated, ignorant people?
I don’t think so. Many will have degrees and higher degrees like a Masters. Remember this survey was with managers and leaders….
It’s not a lack of education…
But I think it may be something else…
The world’s oldest profession
The world’s oldest profession…. and it’s not what you are thinking…
It’s the biggest industry on earth – bar non.
Story telling
Every book,
every film,
every news item,
and just about every Facebook post and every tweet is a story.
Social media are all about stories…
We live in a world of anecdotes and stories
What do you notice at the bottom?
Stories…
We as humans are preprogrammed to tell and listen to stories. You probably spend more time every single day of your life day listening to, creating and telling stories than all the other activities put together.
So why are we surprised that this is the number 1 source of evidence for work based decisions?
So if we want to get people interested in Evidence Based Practice hitting people over the head, telling them their evidence is crap, or that there is no evidence for what they are doing isn’t really going to win many friends or influence them…….,
maybe just maybe we should be flipping this on its head…
Its a funnel. A funnel of engagement. In the entrepreneurial side of my world there are things called marketing funnels…
Awareness, interest, consideration, intent, evaluation, purchase… all that has to happen before anyone will buy your product or
idea
People aren’t going to buy Evidence based practice until they are convinced it will solve their problems
But and its a BIG but….. You are not selling evidence based-practice though… remember this?
You are better off talking about porn or the weather rather than evidence-based anything…
PEOPLE AREN’T INTERESTED in EBP!
So don’t even try to sell it. There is no point….
…at first at least.
But we do know what they want,
what they really really want…
…inform business decisions…
Even more than porn and the weather…
Stuff like… How to make good decisions!
STOP beating people up and for the love of …
STOP selling Evidence-Based Practice
Use these…
use stories, case studies and anecdotes to capture people.
Get their interest first.
Tell stories, capture interest, stop being stuck up about systematic reviews, this method or that method… thats for later…
Research methods for most people
is boring
and
most people don’t care!
You ain’t going to capture anyone like that.
Thats for the people who buy evidence-based practice…
but before then…
go out and have some fun,
slay a few dragons,
help people to do stuff bit by bit.
Slowly slowly draw people in.
That’s it! I hope this has been interesting but more than anything I hope it has sold the bottom of the hierarchy, the vital importance of story telling, anecdotes and case studies in selling Evidence-Based Practice.
Be impressively well informed
Get the very latest research intelligence briefings, video research briefings, infographics and more sent direct to you as they are published
Be the most impressively well-informed and up-to-date person around...
3 comments
Yup agree with everything! in fact, to most people, an anecdote is ‘evidence-based’ since it actually happened and to someone I know or known to the person telling me the anecdote to boot….. It doesn’t matter that it only happened once in a blue moon! if it worked like that for them it’ll work like that for me and I’m willing to give it a try! it’s what most people use when arguing a point in any social setting – anecdotes!
In fact just turn Fig. 10 over and there’s your pyramid!
and don’t get me started on correlation vs. cause and effect!
I enjoyed this article. I like the vote for storytelling. I would add that, in my experience, involving the operations folks deeply in any study is important. Putting them in charge of data collection and analysis and in forming conclusions arising from this. Much easier than “selling” in research findings afterwards.
An idea combining both storytelling and line involvement is a great method called “most significant change”. As well as promoting involvement it also lends itself to a strengths based approach as opposed to the common deficit orientated research models.
I like what you’re saying and largely agree. However even EVP is a academically coined phrase. Of course people operate on evidence, (as you actually address) our beliefs and values are formed through evidence. The issue is that it is not the sort of evidence that academics typically like. I often despair of articles that get to the end of a ‘good science’ article and chuck in some lame one liner at the end that says something like ‘and *insert name of practitioner that the researcher thinks they’re writing for but have actually got bugger all clue about the reality of* should therefore *insert some patronising massively decontextualized suggestion*.
I and my colleagues do seem to get some success working with ‘practitioners’ in trying to add ‘criticalness’ to their judgement and decision making. I believe we get this success for 3 core reasons
1.We try to understand the world they operate in and the challenges they face
2. We acknowledge they have got access to far more evidence based on their experiences than we can ever point them to in the literature
3. We offer theories which we think relate to their field, have some evidence to support their connection to that field but then challenge practitioners to see if these theories help them make ‘better’ sense of their evidence (experiences) in order to encourage them to think more clearly and deeply about what they are doing
4. We offer a schematic that we think helps them see the wood for the trees and therefore bring some coherence to their process and practice.
Funnily enough we don’t yet have ‘evidence’ to support or challenge these assertions yet but hopefully will soon.