MS3 - Evidence-based practice

7/2006
Bob Badgett

If you are viewing a printed copy of this handout, the most current version is at http://medinformatics.uthscsa.edu/ms3

This handout is divided into the following sections

Assessing information
Applying evidence - numeracy
Appendices
- Presentaton assignment
- Information sheet about research data collected during this course is available at http://medinformatics.uthscsa.edu/ms3/

Assessing information

Original research

Details of assessing information are at JAMA Users' Guide Series (search "users[title] AND guide*[title] AND JAMA[journal]" at PubMed). Free access of pre-publication versions at Centre for Health Evidence.

First, assess the "Level of Evidence" of an article.

Level I
- A: Significant, homogeneous systematic review of randomized controlled trials (RCTs)
- B: Significant single RCT
- C: "All or none study"
Level II

A: Significant, homogeneous systematic review of cohort studies
B: Significant single cohort study

Level III

A: Significant, homogeneous systematic review of case-control studies
B: Significant single case-control study

Level IV
- Case series
Level V
- Case report

Take home point: be careful about changing your medical practice after reading a case-control or lesser study.
Question: Can these levels be applied to a diagnostic test?
From: http://www.cebm.net/levels_of_evidence.asp#levels

Second, use the Pico items:

When comparing articles, how do they compare on the PICO iterms?
Relevant articles should match your clinical setting using the PICO structure (see PICO below).

Systematic reviews

Can you interpret this meta-analysis?

Good search strategy
Good criteria on studies to include
At least 2 careful, independent readings
Homogenous groupings
Include at least some studies that are sizable - less risk of publication bias
Individual patient meta-analysis - best but uncommonly done

Texts - books, websites, CD-ROMs

Are they 'systematic'?

Attribution
Peer review
Explicit grading of evidence
Good update mechanism, explicitly stated
Each chapter is a systematic review, ideally

Applying Evidence - Numeracy

This is very difficult, why do this?

Clinicians vary much in probabilities they assign to adverbs such as 'frequently', 'rarely', etc (PMID: 6859055)
Display results with relative numbers increases reactions by clinicians as compared to displaying absolute numbers (references)
Teaching about probabilistic reasoning improves the efficiency of test ordering (PMID: 2492066)
Some important practice guidelines now based decision on probability

Getting numbers out of original articles

Calculators
- Online calculator: Center for Evidence-based medicine
- Palm OS application available at either:
  - Clinical.UTHSCSA.edu/PDA
  - Medinformatics.UTHSCSA.edu/MS3
Glossaries
- Clinical Evidence (most detailed)
- Oxford-Centre for Evidence Based Medicine
- ACP Journal Club
  - On UTHSCSA network: http://www.acpjc.org/shared/glossary.htm
  - Not on UTHSCSA network: EZProxy (PIN required)
- More detail - check out of the Library either of the books by David Sackett

Diagnosis

More information:

Jaeschke R, Guyatt GH, Sackett DL. Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group. JAMA 1994;271:703-7. PMID: 8309035

PubMed:
Abstract - Related articles Fulltext options:
EBSCO - OVID - Centre for Health Evidence (free access)

Summary of Bayes theory:

Rarely is a test perfect with both sensitivity & specificity equal 100%
If a test result is unexpected, it may be incorrect
SPPIN and SNNOUT (more from CEBM )
- For a test to be useful in isolation, needs:
  - sn or sp>90 to 95%
    - or -
    LR+ > 5-10 or LR- < 0.2 to 0.1 (Likelihood ratios)
  - SPPIN - highly SPecific test, if Positive, helps rule IN
  - SNNOUT - highly SNecific test, if Negative, helps rule OUT
Blank two by two table

Treatment

More information:

Wen L, Badgett R, Cornell J. Number needed to treat: a descriptor for weighing therapeutic options. Am J Health Syst Pharm 2005;62:2031-6. PMID: 16174840

PubMed:
Abstract - Related articles Fulltext options:
JAMA - CrossRef - DOI - EBSCO - OVID

Use NNT:

Probably better for understanding complex information

Limitations of NNT:

Difficult math
Need to specify length of treatment
Does not generalize across different risks of outcome

Appendix - Assignments

1. Oral presentation (this is part of your grade)

Requirements:

Presentation is up to 10 minutes (tip: practice and time talk once ahead of time)
State your question in the PICO format.
You must create an evidence table (see below) that listeners can refer to during your talk. If you do so, a suggested template is at http://medinformatics.uthscsa.edu/ms3/
You must provide a printed evidence table and short reference (approximately five references) list for the faculty member who hears your presentation. This list should include the citations to the studies in your evidence table (tip: during talk, help your listeners by refering to studies by their name and note what study design they used, eg RCT, cohort, systematic review of RCTs, etc).
Consider using numerics as taught in class. This is especially helpful if you conclude that the answer to your controversy is a "toss-up" and so you want to share with the patient the pros/cons. To do this:
1. Fill out the four cells
2. Provide key numbers
  - If diagnostic test -> sensitivity / specificity or Likelihood ratios
  - If intervention -> absolute risk reduction and number needed to treat (NNT). Click for more information.
Consider using PowerPoint, OpenOffice (http://www.openoffice.org/) or similar program to make slides to help your users. If you do so, a suggested template is at http://medinformatics.uthscsa.edu/ms3/.
End your talk with your clinical recommendations. If there was conflict among your guidelines, or among your studies, try to explain the conflict so your recommendations are the best of all.

Constructing your question

First, conciously formulate your question with the PICO format. This helps:

Dictates search terms and also what to look for in comparing articles
Directs assessment of validity (see assessing articles - above)

Traditional components of a clinical question.	Is this component important when appraising validity of articles?	But do you need to include this component in your search terms?
Patient, population	Yes	almost always
Intervention (Treatment or test)	Yes	usually, but sometimes not needed if only a few articles remain after combining patient/population with filter
Comparison	Yes	rarely
Outcome	Yes	sometimes

Organize your evidence - sample evidence table

Creating an evidence table may help your organize your thinking and help listeners follow your presentation. Below is a suggested format:

Study, year	Study design	Intervention	Outcome measured	Results	Comments
Annane, 2002	RCT (300 pts, all ventilated)	hydrocortisone (50 mg q 6 hrs) & fludrocortisone (50 mcg qd)	mortality	Overall: steroid group 55% placebo 61% (P=.09) Adrenal normal: steroid group 61% placebo 53% (P=.02) Adrenal insufficient: steroid group 53% placebo 63% (P=.02)	First study with sufficient power. Cotreated with fludrocortisone. 76% with adrenal insuff (<9mcg cortisol increase after corticotropin test).
Briegel, 1999	RCT (40 pts)	hydrocortisone 100 mg bolus, 0.18 mg/kg/hr (100 kg patient would receive 432 mg/d)	mortality shock reversal	Mortality: steroids: 20% placebo 30%(insig) Shock reversal: steroids:90% placebo: 80%(insig)	Small size limits power
Bollaert, 1998	RCT (41 patients requiring pressors >48 hrs)	hydrocortisone 100 mg tid x5d	mortality	Overall: steroids: 32% placebo: 63% (insig) Adrenal normal (n=29): steroids 33% placebo 64% Adrenal insufficient (n=12): steroids 25% placebo 63%	Introduced role of corticotropin testing - which did not predict response to steroids. 29% were adrenal insufficient (<6mcg cortisol increase after corticotropin test)
Cronin, 1995	Systematic review of RCTs (730 patients with septic shock in 6 studies)	varying regimens	mortality	RR=1.07 (95% CI 0.91, 1.26)	Much heterogeneity
Lefering, 1995	Systematic review of RCTs (1329 patients in 10 studies)	varying regimens	mortality	ARR=-0.2% (CI: -9.2, 8.8)	Much heterogeneity No differences between low - vs. high-dose or type of corticosteroid. The Gram-negative group demonstrated better outcome (-5.6% vs. 1.8% for gram positive).

Conclusion in this example:
The best evidence suggests that in this clinical controversy, the risk of death is reduced, but only in patients with relative adrenal insufficiency and if steroids are combined with a mineralocorticoid.
Notes:
You are not expected to do a formal systematic review of every study of your topic. Just include in your evidence table the 4-5 most important studies.

Grading

Pass/fail

Click here to rotate table