Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Evidence-Based Practice: APPRAISE - Critical Appraisal

This guide is designed to assist health care professionals to become effective and efficient users of the medical and nursing literature.

How are resources evaluated?

Appraisal is the third step in the Evidence Based Practice process.  It requires that the evidence found be evaluated for its validity and clinical usefulness. 

Definition:

  • Critical Appraisal is the systematic evaluation of clinical research papers in order to establish the validity of the methodology, results, and applicability to patient care.  (from CEBM)

Ask 3 Questions:

  • Is there a high risk of bias in the methodology?
  • Are the results valid?
  • Can the results be applied to my patient?

Helpful Books

Rough Guide to Spotting Bad Science

undefined

From Compound Chem:  "This graphic looks at the different factors that can contribute towards ‘bad’ science – it was inspired by the research I carried out for [a different project,] where many articles linked the compound to causing breast cancer, referencing scientific research which drew questionable conclusions from their results."  Read more ...

Sketchy EBM Video : How I Read a Paper

Evaluating a Study - Parachute Example

Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial.

Yeh RW, Valsdottir LR, Yeh MW, Shen C, Kramer DB, Strom JB, Secemsky EA, Healy JL, Domeier RM, Kazi DS, Nallamothu BK; PARACHUTE Investigators.

BMJ. 2018 Dec 13;363:k5094. doi: 10.1136/bmj.k5094. Erratum in: BMJ. 2018 Dec 18;363:k5343.

 

Sample questions for evaluating a study:

  • Has the study's aim been clearly stated?
  • Does the sample accurately reflect the population?
  • Has the sampling method and size been described and justified?
  • Have exclusions been stated?
  • Is the control group easily identified?
  • Is the loss to follow-up detailed?
  • Can the results be replicated?
  • Are there confounding factors?
  • Are the conclusions logical?
  • Can the results be extrapolated to other populations?
  • Does it make sense?

How to Read a Paper by Trisha Greenhalgh

Reading a Scientific Paper - In Brief

Read the Abstract

Skim the Introduction

Skip the Methods for now

Read Results

Read Introduction

Read Discussion

Read Methods - If the methods has a high risk of bias, find a different article.  


"Quick & Dirty" Review of a Medical Study

1) Identify the Working Hypothesis

2) Identify the Type of Study

3) State the Conditional Prediction

          If [working hypothesis] is true, then in this type of study [prediction].

4) Look at the results (dat) to see if it bears out the prediction.

5) Assess the validity of the author's conclusions.

Protocols and Checklists for SR/MAs

The following protocols and checklists can help you ascertain if a systematic review or metanalysis was done according to best practices.

Critical Appraisal of Practice Guidelines

While many groups develop practice guidelines, they do not necessarily reach the same conclusions and recommendations for practice. The following journal article and document provide guidance for the critical appraisal of practice guidelines.

 

Even RCTs Can Be Biased

In recent years, the drawbacks to randomized studies have become increasingly recognized. 

  • Patients who are enrolled in such studies typically represent a subset of the general population, compromising “generalizability.”
  • Many patients (and their referring physicians) may not wish to have their treatment randomized, particularly when the outcome difference might be existential.
  • As the inclusion rate drops, so does the reliability.
  • Commercial bias is always a consideration.
  • Investigators are more likely to find in favor of a commercial device if it is provided free of charge for their study. They may feel indebted to the company, and may depend on the company for supplies after the trial.

 

Swanson E. Levels of evidence in cosmetic surgery: Analysis and recommendations using a new CLEAR classification. Plast Reconstr Surg Glob Open. 2013;1:e66

Ascension Wisconsin Associate Intranet CSM Library Intranet MHC Library Intranet WFHC Library Intranet What's New with Ascension Wisconsin Library Services Blog