The Essential (Oxford Review) Guide to Evidence-Based Practice

The Essential Guide to Evidence-Based Practice – 2019

Evidence-based-practice-guide

(Evidence-based practice - Update November 2019) 

Your indispensable guide to evidence-based practice (EBP) 

 

Evidence-based practice is becoming a real force in a rapidly growing number of organisations and public services. Even traditional organisations like the police have a growing body of evidence-based practitioners and functions to support the promotion and embedding of evidence-based practice as well as the development of skills and expertise and allocating resources to EBP. This guide will take you through all the essentials of evidence-based practice and point you in the direction of additional resources, sources and show you the relevant research evidence.

NB. Please note this is a living, continually updated document. As a result it changes and is added to frequently as new research and practice-based evidence become available. If there is anything missing or needs correcting please do let us know.



What is Evidence-Based Practice?


Essentially evidence-based practice is about improving our decision-making by using clear, well-researched andevidencedjustifications for why we do things in certain ways, with the ultimate goal of delivering continual improvements, learning and excellence in our organisation or business. In short, it is about developing and fostering best practice and thinking by looking at and critically considering the real evidence about the issue.

Probably the most avid users of EBP are the health care and aviation industries. Evidence-based practice has revolutionised the way health care is undertaken in many countries around the world and has improved health care outcomes and impact for patients as a direct result.

using evidence-based practice

In the aviation industry evidence-based practice has dramatically increased safety, reduced costs and increased operational effectiveness across the board in what is one of the most heavily regulated industries on the planet.

We will be looking at examples from both of these industries use of evidence-based practice throughout this guide.

Increasingly, clinicians, engineers and pilots in health care and aviation are basing most of their operational work on EBP these days, placing an ever increasing emphasis on using and creating good evidence as a basis for their work and decisions. As a result clinical expertise has increased alongside better patient outcomes and in the aviation industries better safety, working practices, use of materials and construction methods have delivered greater efficiencies, profitability and productivity as well as better customer outcomes.


Definition of Evidence Based Practice

 

One of the more widely-accepted definitions of evidence-based practice is provided by Dr. David Sackett (who was an American-Canadian medical doctor and a pioneer in evidence-based medicine who died in 2015) defines it as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research.”1

Derivatives of this definition have extensively been used and acknowledged in many contexts from medicine, policing, the military, construction, engineering, to housing and social care.

In essence organisational evidence-based practice (EBP) refers to the systematic process where-by decisions are made and actions or activities are undertaken using the best evidence available. The aim of evidence-based practice is to remove as far as possible, subjective opinion, unfounded beliefs, or bias from decisions and actions in organisations in order to achieve to goals of the organisation.

research about evidence-based practice

The four sources of evidence for evidence-based practice


There are four sources of evidence for EBP:

  1. Research evidence - preferably peer reviewed research / scientific literature
  2. Work-based research (trial and error testing)
  3. Feedback from the organisation and customers / clients / stakeholders
  4. Practitioner experience and expertise (See The role of experience in evidence-based practice)
Evidence-based practice sources

 

The point of this is that evidence-based practice is based on evidence from across all four sources, in order to inform decisions and not just one source. The idea is to have the 'best evidence' available to inform practice. The term best evidence refers to evidence from a number of sources including research or scientific studies. The evidence from these sources don't make the decision, rather they inform the decision and help guide the best course of action for the practitioner who used their experience and skills along side the thinking and experience of other practitioners, data from the organisation, academic research and feedback from stakeholders, clients and customers. In short evidence-based practice is founded on multiple sources of evidence and data (They are not the same thing - see this article - What's the difference between evidence and data)


Knowledge management and evidence-based practice

Evidence-based practice is essentially a form of knowledge management13Knowledge management refers to " the systematic management of an organization's knowledge assets for the purpose of creating value and meeting tactical & strategic requirements; it consists of the initiatives, processes, strategies, and systems that sustain and enhance the storage, assessment, sharing, refinement, and creation of knowledge." 14

Types or forms of knowledge

There are considered to be two types of knowledge that inform decision-making in organisations:

  1. Explicit
  2. Tacit

Explicit knowledge refers to the codified, formal information that can be easily shared such as this guide, instructions, book contents etc Tacit knowledge on the other hand refers to the type of information that is difficult to transfer to others such as our subjective and personal intuitive understanding about how we sense or feel things, or how we know how to do something. They are often referred to know-what (explicit knowledge) and know-how (tacit knowledge).


The difference between evidence-based practice and EST's


In 2006  an American Presidential Taskforce on Evidence-Based Practice 2 made an interesting distinction between EBP and what are known as 'empirically supported treatments' or ESTs. A treatment in this context refers to any solution being proposed to deal with a problem. In essence, an empirically supported treatment is an evaluation of the evidence of what effect or impact a particular solution to an issue is likely to have in a given situation. Whilst empirically supported treatments may form part of the broader application of evidence-based practice, it is not on it's own sufficient to be considered to be EBP. Empirically supported treatments start with the the treatment and asks whether it will work in the situation at hand. Evidence-based practice on the other hand starts with the customer, client or patient and asks what research evidence will assist in achieving the best outcome for them. The goal with EBP isn't just to solve a particular problem but to find out what we can learn to obtain the best outcome for the enduser.

In terms of a definition of evidence-based practice this helps to clarify the broader scope of the general and habitual practice of using best evidence to find the best outcome rather than simply evaluating the potential effectiveness of a particular solution.

99% of everything you are trying to do...

Reinventwheel

...has already been done by someone else, somewhere - and meticulously researched.

Get the latest research briefings, infographics and more from The Oxford Review - Free.

We won't send you spam. Unsubscribe at any time. Powered by ConvertKit

Decisions based on best evidence


Evidence-based practice requires that operational decisions in the organisation are based on the best available, current, valid and reliable evidence. Now I emphasis the word 'evidence' as there is a critical distinction between evidence and data which people often confuse...

The difference between evidence and data: –
See this article about the difference between evidence and data


Opinion and Evidence based decisions


In order to achieve evidence-based practice, we have to be informed by a more objective evidence base. The traditional approach for leaders, managers, practitioners, coaches and consultants has been to make decisions on a more subjective basis, using:

These approaches however can frequently lead to misguided and biased decisions, yet they are frequently still regarded as the best method by many individuals and organisations.

The devastating impact of opinion-based decisions - See this article about the devastating effects of opinion based decision-making.

The application of evidence-based practice does not mean that experience and judgment are no longer applicable. On the contrary, they remain vital, but only where they are used in conjunction with evidence to consider the facts about what approach to take or decision to make, to reduce the likelihood of bias, habit or cultural preference swaying the results.

Opinion v evidence-based

The aim with the application of EBP and personal judgement is to apply them in a critical manner, avoiding or at least being aware of the assumptions being made.

The point here is that when thinking about or tackling a  problem or intervention, it is important that our actions take account of what has been learned over time by previous practitioners and researchers, preferably using the evidence from high-quality research studies.4


Focus on the practical

 

Evidence-based practice is meant to focus on the practical to help people understand what evidence there is, or is not, for their work practices, polices, procedures and decisions. Just because something appears to work doesn’t mean it does to the extent people believe it does and there may be unseen side effects or consequences that a more fully explored and evidenced approach may well discover.

In medicine, EBP is referred to as a process that allows physicians and clinicians to assess research and guidelines on clinical procedures from highly trusted findings and then apply them in their daily practice.

So placed into the context of organisational evidence-based practice this means that EBP is a set of processes that allows practitioners (employees), managers and leaders to assess research, guidelines, policies, procedures, working practices and decisions from trusted research findings and apply them in their daily practice, with an emphasis ondaily practice.

Focus on the practical

This focus on the practical is really important. It is vital that individuals and groups in the organisation are motivated to implement changes and that they regard the introduction of EBP as useful, valuable and important. If many individuals feel that the existing way of doing things is still the best approach, then there may be resistance to adopt and use EBP. In short, a change of culture may have to take place in the organisation to facilitate EBP in practice, but more of this in the implementation section of this guide.

However, to convince individuals, there has to be reliable, rigorous and robust evidence that the interventions have worked in similar contexts and will be applicable to the given organisation. If it can be clearly shown that EBP will lead to more efficiency, profit, better working practices and outcomes attitudes can change quickly and lead to a positive, favourable consensus to engage with evidence-based practice. Overall, as long as the focus remains on operational practice rather than turning employees and managers into academic researchers, it is significantly more likely that people in organisations will be able to discern its value and therefore accept and engage with EBP.5


The Principles of Evidence-Based Practice


Evidence-based practice is based on two principles:

  1. Understanding that scientific evidence alone is insufficient to guide decision-making, and that
  2. Within all the available sources of evidence, hierarchies exists - some types of evidence are seen as more valid or carry more weight than others (see the section below: Hierarchy of Evidence)6.

The basic 5 stages or steps of evidence-based practice  - what's involved?

Evidence-based practice in 5 steps

Over the past 20 years of so, the research into the foundational principals of EBP has developed into a coherent framework of steps or stages for EBP.7 They are as follows:

  1. First of all, EBP requires an assessment of the given situation and formulation of the basic question or questions (hypotheses) that need to be answered in order to address the problem.
  2. Secondly, the evidence should be acquired through a systematic search of online resources, books and, in many cases, specialist journals/databases in the field. There is a convention as to what is considered to be the best evidence - see the next section of this guide.
  3. An assessment and appraisal of the materials/evidence needs to take place to consider the applicability and validity of the given evidence about what type of action or intervention to pursue and to inform a final decision.
  4. The next stage involves integrating the new knowledge into the organisation and applying it in practice.
  5. Finally, an evaluation of performance takes place through mechanisms, such as feedback from relevant stakeholders.


The 5 A's of Evidence-based practice


These 5 steps are also known as the 5 A's

  • Asking a critical question
  • Acquiring the evidence
  • Appraising and assessing the evidence
  • Applying the best evidence to make an action plan and take action
  • Assessing the outcomes of the action taken

Hierarchy or Levels of Evidence


Not all evidence is created equal and tends to be graded a number of factors:

  • Generalisability - how applicable are the findings to a wide range of scenarios or do these finding only really relate to a very specific set of circumstances or conditions?
  • Reliability - how stable and consistent the results of a study are over time? If we do the study again are we going to get the same results?
  • Replicability - can the study be repeated and retested? If not how can we make any claims of reliability?
  • Validity - do the methods and design of the study actually measure or describe what the study is purporting to show? In terms of validity there are two forms:
    • Internal validity - do the methods and instruments used actually measure or describe what we think they are or are they measuring or describing something else?  Examples of factors that can change the validity of a study include:
      • Subject variability - you have a particularly un-representative subject pool. They are all university students and don't represent the general population for example.
      • Size of the subject population. Having a study based on looking at 5 people is quite different and will effect the validity of a study differently from a study of thousands of people.
      • Time - was the study conducted once in a short space of time or over a longer period of time.
    • External validity - This refers to whether the study in question is applicable to the situation you want to apply it to. A results of a study conducted on pre-school children might not have much external validity when applied to a group of combat soldiers for example.
  • Authority - has the study come from a well known and respected academic/university or a consultancy selling a product or service for example?
  • Peer-reviewed - has the study or paper been through a process of checking and critique by fellow researchers or is it self published?

Evidential sources


In general terms there are two forms of sources of evidence:

  1. Primary sources - this refers to data that has come direct from the source in question. For example eye witness accounts, legal or historical documents, raw data, the direct results of experiments or research.
  2. Secondary sources which include second party analysis, summaries, evaluations, opinions, comments, articles etc that do not have first hand experience or knowledge of the issue or event.

Hierarchy of research 

In terms of confidence about the findings all research sits in a hierarchy:

Causality

As we go up the pyramid there is more likelihood of greater levels of reliability, validity, generalisability and confidence and as we go down there is an increased risk of bias, and potentially lower levels of reliability, validity, generalisability and confidence. This however does not mean that case studies have no usefulness. It just means that they are situated in a particular context that may or may not apply to your situation.

The evidence-based practitioner needs to be able to understand and evaluate the nature of the evidence they are dealing in, and with reference to the context within which they are going to use the evidence, before using it.


The History of Evidence-Based Practice


The Ancient Greeks - Pedanius Dioscorides and Galen of PergamonWhilst the history of Evidence Based Practice can actually be traced back the ancient Greeks, in particular Pedanius Dioscorides (AD 40-90)  was a Greek physician, pharmacologist, botanist, and author of De Materia Medica a 5-volume Greek encyclopedia about herbal medicine and related medicinal substances (known as a pharmacopeia), that was widely read for more than 1,500 years. He was later employed as a medic in the Roman army.

Dioscorides

Dioscorides

 

Later Galen of Pergamon (AD 129-216) who also served the Romans the most accomplished of all medical researchers of antiquity, Galen influenced the development of various scientific disciplines, including anatomy, physiology, pathology, pharmacology, and neurology, as well as philosophy and logic.

 

Archie Cochrane and modern day evidence-based practice. However modern day connections of evidence-based practice really stem from the work of Archie Cochrane a Scottish doctor noted for his book Effectiveness and Efficiency: Random Reflections on Health Services.6

Archie Cochrane

Archie Cochrane

This book argued for a more systematic and rigorous approach to medicine and championed the use of randomised control trials to make medicine more effective, efficient and reliable. This campaign for the use of randomised controlled trials eventually led to the development of the Cochrane Library database of systematic reviews (more about these below), the establishment of the UK Cochrane Centre in Oxford (http://uk.cochrane.org) and the international Cochrane Collaboration, a collection of 14 evidence-based medicinal research centres. The tag line of the Cochrane centres is "Trusted evidence. Informed decisions. Better health." 7 which sums its mission up nicely.

Cochrane is known as one of the fathers of modern clinical Evidence-Based Medicine and is considered to be the originator of the idea of Evidence-Based Medicine.

 He was known for arguing that medical strategies should be thoroughly and rigorously tested before being used on human beings. This advocacy for the use of evidence before practice laid the foundation of evidence-based medicine. He is particularly known for emphasising the need for random studies to be carried in different places to guarantee the accuracy of the evidence and the reduction of bias the tendency of medical practitioners to push their favourite home grown theories about best practice.

This shift from the personal experience-based intuitive and deductive reasoning of individual medics into a systematic, scientific and rigorous process was really the foundation of all modern evidence-based practice.

This approach to the practice of medicine rapidly gained ground as the results and outcomes on health care became measurable apparent. In the early stages of evidence-based medicine, some practitioners railed against it, stating that it was too slow and negated the vast experience of many professional medics.


The Four Elements of Evidence-Based Practice


Recently however evidence-based medicine has become seen as a conflation or union of four elements:

  1. The best research evidence available
  2. The expertise and experience of the clinician (See this article about the role of experience in evidence-based practice)
  3. The patient's or customers values and preferences
  4. As well as the environment or background existing at the moment.


This last element is important, as practicing medicine in a well-staffed well resourced hospital in the west is often entirely different to doing so in the middle of a battlefield in a developing country where resources are few and far between. This idea of evidence-based medicine really came out of the work and efforts of Dr. David Sackett who argued in 1996 that evidence-based practice is not solely based on research 8 but that also embraces medical expertise too. He further explained that patients’ needs and preferences could be taken into account by the evidence-based practice and this is what revolutionized evidence-based practice.


The model of evidence-based practice


This model of practice was been embraced in several sectors of medicine such as nursing, surgery, dentistry as well as aviation and organisational evidence-based practice and looked like this:

old version of evidence-based-practice

The latest Transdisciplinary Model


As a result of more recent research (9) however the model has now become a transdisciplinary model based on the recognition that most problem solving is done with reference to other disciplines and areas of expertise and the fact that decisions frequently span a wider arc than purely in the centre of the three areas of influence. The accepted model now looks more like this:

The Transdisciplinary Evidence-based practice model

The Transdisciplinary Model model displays the components of EBP and how they can be applied to larger organizational levels such as in medical facilities and any organisation. The intention here is to show that evidence-based decision-making spans a range of sources and is not purely based on academic research and incorporates more operational and experiential elements of expertise.


The Iowa Model of Evidence-Based Practice

In 2017 researchers and nursing EBP practitioners based at Iowa University tested and published the Iowa Model of Evidence-Based Practice (16). This is a decision making matrix with the  aim of helping practitioners navigate the evidence-based practice process:

  1. Identify the triggering issues or opportunities
  2. State the question or purpose
  3. Is this topic a priority? - If not do not proceed
  4. If yes - Form a team
  5. Collate, analyse and synthesise the body of published evidence
    1. Conduct systematic research /search
    2. Weigh quality, consistency and quantity of studies with the risk
  6. Is the sufficient evidence to act? - if not conduct research to provide the evidence
  7. If there is sufficient evidence - Design and pilot changes
  8. Is the change robust enough and appropriate for adoption of the changes into routine practice? - If not consider alternatives and redesign action and test
  9. If yes, integrate and embed into practice 
  10. Disseminate results and enter process and results into the organisational knowledge management system

Problems faced with implementing evidence-based practice

 

A number of studies (10) show that whilst many specialists look upon the idea of EBP in a favourable way, there are barriers in understanding how to translate evidence and apply it to practice. Many different fields, from business to sport, need good research to be properly disseminated and implemented in the organisation if it is going to lead to real, practical improvements. However a series of problems do cause barriers to implementing evidence-based practice. The four major barriers to successful implementation have been found to be:

 

Access to resources has been cited as a primary problem for many individuals and organisations. Access to professional resources required to develop EBP is often quite difficult and time consuming. In some cases this just means the use of textbooks and Web sites. However, some areas of practice may require access to electronic online journals or databases with specialist research and this is often limited. In some cases, the specialist knowledge required for the organisation may require an investment into resource access and the skills to translate the sources into useful material. Nonetheless, improving knowledge of how to use freely available online resources may be sufficient to allow for the rapid development of EBP.11

 

Time constraints are regularly cited in every profession as being a key barrier to the application of EBP. For instance, in education, research indicates that teachers and instructors suggest that they are generally pushed to the limit and that the time needed to do research and introduce best practice is difficult to find. This is often because, in an ever-demanding workplace, specialists in most fields find that they often have to “wear multiple hats”, leading to legitimate concerns about time availability. 12 A number of studies have found that time constraints to source, obtain and translate research studies into something that is practical is the number one barrier to implementing and maintaining evidence-based practice across all professions.

Time constraints as a barrier to evidence-based practice

Another key issue is a  lack of support at organisational and managerial levels to give people the time to engage with the research needed to implement EBP properly. This can be overcome through the introduction of reliable administrative support in the organisation and a careful consideration of priorities when the application of EBP is likely to bring substantial transformative benefits or by using specialist research translators and providers like The Oxford Review.

 At the bottom of all of the barriers to implementation of evidence-based practice is just a simple lack of knowledge and understanding. Individuals and organisations are often completely unaware of the new developments in their field, or the scientific evidence available to them that could completely revolutionise their organisation, outcomes and thinking. In addition, many specialists lack the basic skills required to assess the value and applicability of research, as well as its trustworthiness. Educational workshops and training within an organisation can be vital to help understand and integrate new practices in to everyday operations. Employees may benefit from training sessions led by external experts. The building of knowledge and understanding of how to use EBP can give specialists the confidence to go beyond a sole reliance on poor-quality subjective approaches.12

A 2019 study  (17) found that new practitioners tend not to transfer their learning of evidence-based practice  into the workplace for for a number of specific reasons:

There tends to be an essential disconnection in practitioners minds between evidence and research. This backs up a 2017 study which found that practitioners often only have a shallow understanding of evidence and fail to understand the importance of systematic and valid research evidence to inform practitioner experience and the immediate evidence of the situation.

Additionally the study found that:

1.Practitioners know that they should be using research evidence as part of the evidence-based mix but recognise they aren’t.
2.This is appears, is because they move form a culture of research centrality (at university) into a culture of practice, immediacy and reduced time.
3.From a student’s and practitioner’s point of view, research is someone else’s job.
4.Practitioners and particularly new practitioners who usually work within a hierarchy and usually at the bottom levels of the hierarchy feel that their voice, opinion and knowledge of the research doesn’t carry much weight.
5.In the intensity of daily work and practical concerns, knowledge production and study tend to get seen as situated in the college or university and not the work environment.

However, the study found that the culture of the workplace appeared to be a prime indicator of whether or not new practitioners engage with evidence-based practice.

One of the biggest issues and barriers is how evidence-based practice is promoted and people are introduced to it.

See this article about Why evidence-based practice probably isn’t worth it


Developing evidence-based practice in organisations


There are a number of key issues that need to be addressed when you are trying to get evidence-based practice going in an organisation:

  • Developing the skills, attitudes, behaviours and knowledge required of evidence-based:
    • Practitioners
    • Supervisors
    • Mentors/trainers/educators
    • Managers, and
    • Leaders
  • Creating, developing and maintaining the systems and procedures needed
  • Obtaining, creating and developing the resources needed


Developing the skills, attitudes, behaviours and knowledge required of evidence-based


A systematic review of post-graduate development of medical evidence-based practitioners(11) found that whilst standalone classroom-based teaching improved knowledge it does not develop but not skills, attitudes, or behaviour. However the research shows that clinically integrated teaching (actually using and being mentored / coached in the context the practice is going to be used) improved knowledge, skills, attitudes, and behaviour.

 

The skills required of evidence based practitioners


There are a series of six distinct skill sets EBP practitioners need such as: "

  1. Asking: translating a practical issue or problem into an answerable question
  2. Acquiring: systematically searching for and retrieving the evidence
  3. Appraising: critically judging the trustworthiness and relevance of the evidence
  4. Aggregating: weighing and pulling together the evidence
  5. Applying: incorporating the evidence into the decision-making process
  6. Assessing: evaluating the outcome of the decision taken

to increase the likelihood of a favorable outcome." 3.


Evidence-based practice competencies

A 2014 study was able to discern that there are 24 evidence-based practice competencies that contribute to a proficient practitioner. 13 are basic level competencies and the remaining 11 are advanced level competencies:

    24 evidence-based practice competencies

  1. Questions operational practices for the purpose of improving the quality operations.
  2. Describes problems using internal evidence (internal evidence* = evidence generated internally using internal assessment data, outcomes management, and quality improvement data).
  3. Participates in the formulation of critical questions using PICOT* format (*PICOT = Population; Intervention or area of interest; Comparison intervention or group; Outcome; Time).
  4. Searches for external evidence to answer focused questions (external evidence = evidence generated from research).
  5. Participates in critical appraisal of pre-appraised evidence (such as practice guidelines, evidence‐based policies and procedures, and evidence syntheses).
  6. Participates in the critical appraisal of published research studies to determine their strength and applicability to operational practice.
  7. Participates in the evaluation and synthesis of a body of evidence gathered to determine its strength and applicability to practice.
  8. Collects practice data (e.g. individual patient/customer data, quality improvement data) systematically as internal evidence for decision making in the care of individuals, groups, and populations. 
  9. Integrates evidence gathered from external and internal sources in order to plan evidence‐based practice changes.
  10. Implements practice changes based on evidence and clinical expertise and patient preferences to improve care processes and customer outcomes.
  11. Evaluates outcomes of evidence‐based decisions and practice changes for individuals, groups and populations to determine best practices.
  12. Disseminates best practices supported by evidence to improve quality of service and customer outcomes.
  13. Participates in strategies to sustain an evidence‐based practice culture.

    Advanced evidence-based competencies


  1. Systematically conducts an exhaustive search for external evidence to answer critical questions. 
  2. Critically appraises relevant pre-appraised evidence (i.e. guidelines, summaries, synopses, syntheses of relevant external evidence) and primary studies, including evaluation and synthesis.
  3. Integrates a body of external evidence from research and related fields with internal evidence* in making decisions about patient care. 
  4. Leads transdisciplinary teams in applying synthesised evidence to initiate clinical decisions and practice changes.
  5. Generates internal evidence through outcomes management and EBP implementation projects for the purpose of integrating best practices.
  6. Measures processes and outcomes of evidence‐based critical decisions.
  7. Formulates evidence‐based policies and procedures.
  8. Participates in the generation of external evidence with other professionals.
  9. Mentors others in evidence‐based decision making and the EBP process.
  10. Implements strategies to sustain an EBP culture.
  11. Communicates best evidence to individuals, groups, colleagues and policy makers.


The role of experience in evidence-based practice


Evidence-based practice it is not solely focused on being dependent on peer-reviewed research. Experience plays a vital role in evidence-based practice, only it’s not any old experience that counts here. There is experience and experience. Understanding the role of experience in evidence based practice, and what types of experience that is matters.

See the rest of this article:  The role of experience in evidence-based practice


So, how can you tell how good a piece of research is and what do we mean by good?

In evidence-based practice circles, the accuracy and reliability of the research is an important factor when deciding to include a study in the evidence-base for a decision, especially clinical or engineering decisions where people’s lives are on the line. 

GRADE

One way that evidence-based practitioners judge the quality of research is to use the GRADE system or framework (18). GRADE stands for Grading of Recommendations, Assessment, Development and Evaluations, and is used extensively in medical scenarios. 

 GRADE is now being used in engineering, organisational evidence-based practice and is the basis of many systematic reviews.  

The 4 levels of evidence

When choosing studies to include in a decision GRADE has four levels of evidence, or four levels of certainty, in the quality of the evidence/study:

  1. Very low – The real effect is probably very different from the reported findings.
  2. Low – The true effect is quite likely to be different from the findings of this study.
  3. Moderate – The true effect is probably close to the findings of this study
  4. High – A high level of confidence that, the findings represent the true effect / represents reality

The 5 GRADE Domains 

There are five overall factors which are used to help a practitioner work out what the level of a particular study:

  1. Risk of bias. In 2005 The Cochrane Institute in Oxford produced a tool that has become the standard bias risk assessment tool – The Cochrane Collaboration’s Tool for Assessing Risk of Bias - and looks specifically at a range of different biases that can affect a study’s findings. The tool looks for whether the study being looked at:
    1. Uses quality scales. Quality scales are often inherently biased and based on opinions.
    2. Looks at the internal validity of the study. What this means is that the researchers extensively look for and report any potential biases the method or study might have. In other words, are the methods used in the study likely to lead to bias or less so?
    3. Has actively looked for sources of bias or influence in their results. Double blind trials, where neither the participants (subjects) nor the researchers know which subject is getting which treatment or is in which group are considered to be at the least risk of bias.
    4. That the assessor / evaluator understands the methods used and can make a good judgement about the methods used.
    5. Is there a risk of bias in the way the data is being presented or represented?
    6. Does the use or group to which you are putting the study introduce a risk of bias not associated to the study?


The Cochrane Collaboration’s Tool for Assessing Risk of Bias usually uses a 3-point grading system for judging bias:

  1. Low risk of bias
  2. High risk of bias
  3. Unclear risk of bias 

2. Imprecision. How precise are the data and methods applied?

3. Inconsistency. Does the study show inconsistences openly or do they try to mask them? Are there inherent inconsistencies that haven’t been reported?

4. Indirectness. How close is the situation / population of the study being looked at to the one it is being applied to? Is this study talking directly into and about the organisation or population of the study, or is this being used in a more indirect way?

5. Publication bias. Does the publisher have a stake in the findings? For example, is this a study by a consultancy attempting to show how good its own methods are? 

 All of these potential biases downgrade the quality of the study.


References


  1. Sackett, D., Rosenberg, W., Gray, J., et al. (1996). Evidence based medicine: what it is and what       it isn't: It's about integrating individual clinical expertise and the best external evidence. BMJ, 312, 71-72. Doi: http://dx.doi.org/10.1136/bmj.312.7023.71
  2. APA Presidential Task Force on Evidence-Based Practice. "Evidence-based practice in psychology." The American Psychologist 61.4 (2006): 271.
  3. Barends, E., Rousseau, D. M., & Briner, R. B. (2014). Evidence-based management: The basic principles. - https://www.cebma.org/wp-content/uploads/Evidence-Based-Practice-The-Basic-Principles-vs-Dec-2015.pdf
  4. A beginner's guide to evidence-based practice in health and social care professions (2nd ed.) Helen Aveyard (Milton Keynes: Open University Press, 2013). PACER center, Evidence-Based Practices at School: A Guide for Parents, http://www.readingrockets.org/article/evidence-based-practices-school-guide-parents)
  5. Gill Harvey, Alison Kitson,Implementing Evidence-Based Practice in Healthcare: A Facilitation Guide(London: Routledge, 2014), pp. 47-65.
  6. Murphy, M., MacCarthy, M. J., McAllister, L., & Gilbert, R. (2014). Application of the principles of evidence-based practice in decision making among senior management in Nova Scotia’s addiction services agencies.Substance abuse treatment, prevention, and policy,9(1), 47.
  7. Straus, S. E., Glasziou, P., Richardson, W. S., & Haynes, R. B. (2018). Evidence-Based Medicine E-Book: How to Practice and Teach EBM. Elsevier Health Sciences.
  8. Satterfield, J. M., Spring, B., Brownson, R. C., Mullen, E. J., Newhouse, R. P., Walker, B. B., & Whitlock, E. P. (2009). Toward a transdisciplinary model of evidence‐based practice.The Milbank Quarterly,87(2), 368-390.
  9. AnBussières, A. E., Al Zoubi, F., Stuber, K., French, S. D., Boruff, J., Corrigan, J., & Thomas, A. (2016). Evidence-based practice, research utilization, and knowledge translation in chiropractic: a scoping review.BMC complementary and alternative medicine,16(1), 216.
  10. W. McCarty, C., Hankemeier, D. A., Walter, J. M., Newton, E. J., & Van Lunen, B. L. (2013). Use of evidence-based practice among athletic training educators, clinicians, and students, part 2: attitudes, beliefs, accessibility, and barriers.Journal of athletic training,48(3), 405-415.
  11. Manspeaker, S., & Van Lunen, B. (2011). Overcoming barriers to implementation of evidence-based practice concepts in athletic training education: perceptions of select educators.
  12. Coomarasamy, Arri, and Khalid S. Khan. "What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review." Bmj329.7473 (2004): 1017.
  13. Hübner, Lena. "Reflections on knowledge management and evidence-based practice in the personal social services of Finland and Sweden." Nordic Social Work Research 6.2 (2016): 114-125.
  14. Knowledge Management Tools at https://www.knowledge-management-tools.net/knowledge-management-definition.html accessed 8th April 2018
  15. Melnyk, B. M., Gallagher‐Ford, L., Long, L. E., & Fineout‐Overholt, E. (2014). The establishment of evidence‐based practice competencies for practicing registered nurses and advanced practice nurses in real‐world clinical settings: Proficiencies to improve healthcare quality, reliability, patient outcomes, and costs. Worldviews on EvidenceBased Nursing11(1), 5-15.
  16. Iowa Model Collaborative. (2017). Iowa model of evidence-based practice: Revisions and validation. Worldviews on Evidence-Based Nursing, 14(3), 175-182. doi:10.1111/wvn.12223
  17. Moore, F., & Tierney, S. (2019). What and how… but where does the why fit in? The disconnection between practice and research evidence from the perspective of UK nurses involved in a qualitative study. Nurse education in practice34, 90-96
  18. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical research ed). 2008;336(7650):924-6.
    Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. Journal of clinical epidemiology. 2011;64(4):383-94.
    Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. Journal of clinical epidemiology. 2011;64(4):395-400.
    Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3. Rating the quality of evidence. Journal of clinical epidemiology. 2011;64(4):401-6.
    Guyatt, G. H., Oxman, A. D., Kunz, R., Vist, G. E., Falck-Ytter, Y., & Schünemann, H. J. (2008). What is “quality of evidence” and why is it important to clinicians?. Bmj336(7651), 995-998.
    BMJ Best practice series: What is GRADE? Accessed at: https://bestpractice.bmj.com/info/toolkit/learn-ebm/what-is-grade/ on 3rd November 2019
    Guyatt, G., Oxman, A. D., Akl, E. A., Kunz, R., Vist, G., Brozek, J., ... & Jaeschke, R. (2011). GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. Journal of clinical epidemiology64(4), 383-394.

 

Evidence-based practice in core work functions

Not only is there is there a lot of research evidence about evidence-practice but over the last 10-20 years or so there has been a significant expansion into specific applications of evidence-based practice. As mentioned above the oldest and most established area of EBP is medicine followed by aviation. The police in many countries are now developing a serious interest in the topic as well for many of their roles and functions. In the broader organisational and human development space there are also significant developments on this front.

Rather than including all of the different work functions in this guide we are building a repository of knowledge for each area. See each section below for the relevant guide and resources:

The essential guide to evidence-based leadership

The essential guide to evidence based management (coming soon)

The essential guide to evidence-based Organisational development, change and transformation (coming soon)

The essential guide to evidence-based HRM (coming soon)

The essential guide to evidence-based learning (coming soon)

The essential guide to evidence-based coaching (coming soon)

The essential guide to evidence-based decision-making (coming soon)

 

 

 

 

 

 

 

 

 

 

 

 

 

References

  1. Sackett, D., Rosenberg, W., Gray, J., et al. (1996). Evidence based medicine: what it is and what       it isn’t: It’s about integrating individual clinical expertise and the best external evidence. BMJ, 312, 71-72. Doi: http://dx.doi.org/10.1136/bmj.312.7023.71
  2. APA Presidential Task Force on Evidence-Based Practice. “Evidence-based practice in psychology.” The American Psychologist 61.4 (2006): 271.
  3. Barends, E., Rousseau, D. M., & Briner, R. B. (2014). Evidence-based management: The basic principles. – https://www.cebma.org/wp-content/uploads/Evidence-Based-Practice-The-Basic-Principles-vs-Dec-2015.pdf
  4. A beginner’s guide to evidence-based practice in health and social care professions (2nd ed.) Helen Aveyard (Milton Keynes: Open University Press, 2013). PACER center, Evidence-Based Practices at School: A Guide for Parents, http://www.readingrockets.org/article/evidence-based-practices-school-guide-parents)
  5. Gill Harvey, Alison Kitson,Implementing Evidence-Based Practice in Healthcare: A Facilitation Guide(London: Routledge, 2014), pp. 47-65.
  6. Murphy, M., MacCarthy, M. J., McAllister, L., & Gilbert, R. (2014). Application of the principles of evidence-based practice in decision making among senior management in Nova Scotia’s addiction services agencies.Substance abuse treatment, prevention, and policy,9(1), 47.
  7. Straus, S. E., Glasziou, P., Richardson, W. S., & Haynes, R. B. (2018). Evidence-Based Medicine E-Book: How to Practice and Teach EBM. Elsevier Health Sciences.
  8. Satterfield, J. M., Spring, B., Brownson, R. C., Mullen, E. J., Newhouse, R. P., Walker, B. B., & Whitlock, E. P. (2009). Toward a transdisciplinary model of evidence‐based practice.The Milbank Quarterly,87(2), 368-390.
  9. AnBussières, A. E., Al Zoubi, F., Stuber, K., French, S. D., Boruff, J., Corrigan, J., & Thomas, A. (2016). Evidence-based practice, research utilization, and knowledge translation in chiropractic: a scoping review.BMC complementary and alternative medicine,16(1), 216.
  10. W. McCarty, C., Hankemeier, D. A., Walter, J. M., Newton, E. J., & Van Lunen, B. L. (2013). Use of evidence-based practice among athletic training educators, clinicians, and students, part 2: attitudes, beliefs, accessibility, and barriers.Journal of athletic training,48(3), 405-415.
  11. Manspeaker, S., & Van Lunen, B. (2011). Overcoming barriers to implementation of evidence-based practice concepts in athletic training education: perceptions of select educators.
  12. Coomarasamy, Arri, and Khalid S. Khan. “What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review.” Bmj329.7473 (2004): 1017.
  13. Hübner, Lena. “Reflections on knowledge management and evidence-based practice in the personal social services of Finland and Sweden.” Nordic Social Work Research 6.2 (2016): 114-125.
  14. Knowledge Management Tools at https://www.knowledge-management-tools.net/knowledge-management-definition.html accessed 8th April 2018
  15. Melnyk, B. M., Gallagher‐Ford, L., Long, L. E., & Fineout‐Overholt, E. (2014). The establishment of evidence‐based practice competencies for practicing registered nurses and advanced practice nurses in real‐world clinical settings: Proficiencies to improve healthcare quality, reliability, patient outcomes, and costs. Worldviews on EvidenceBased Nursing11(1), 5-15.

 

 

 

 

 

Be impressively well informed

Librarysmileherobg

Get the very latest research intelligence briefings, video research briefings, infographics and more sent direct to you as they are published

Be the most impressively well-informed and up-to-date person around...

Powered by ConvertKit
Like what you see? Help us spread the word