Journal Club 4/5/12: Making Errors in Diagnosis

This week's Journal Club is a simple study from JAMA on the effects of bias on making diagnostic errors. The findings may or may not have implications for how we think in the ED. Please follow the link and join us on Thursday at 9 am to discuss.

Reference:
Mamede S, van Gog T, van den Berge K, et al. "Effect of availability bias and reflective reasoning on diagnostic accuracy among internal medicine residents." JAMA. 304(11); 1198-1203: 2010.


Post Conference Summary
-Most common cause of a diagnostic error in internal medicine: failure to synthesize available data. Knowledge deficits were relatively rare. 
-Most common cause of closed pediatric malpractice claims originating in the ED is diagnostic error.

This study asked 1^st and 2^nd year internal medicine residents to make diagnoses after being exposed to diagnoses that had similar presentations. Did this exposure bias the resident to make the wrong diagnosis? Is this bias more pronounced in more experienced clinicians? And if so, could more thoughtful reflection of the case overcome this bias?

Valid?

Study Design Prospective Cohort Study

Comparison Groups

-1^st and 2^nd year internal medicine residents

-Exposure to biasing cases and non-biasing cases

Intervention:

-Reflective analysis of cases where residents are asked to systematically come up with differential and supporting and refuting evidence before committing to diagnosis

Important?

Outcomes

-Exposure to similar cases negatively impacted diagnostic abilities of the second year residents. The same effects were not seen in first year residents.

Strength and precision of effect

-We are unable to judge the strength of the effect because we were not given risk ratios. The precision is good, with narrow confidence intervals.

Applicable?

Population

-Only studied residents in internal medicine in the Netherlands.

-Most did agree that availability bias probably does negatively affect our diagnostic abilities. It’s not clear if more clinical experience helps or hurts though

Limitations

-power?

-difficult to extrapolate to faculty and fellows because study does not address the long-term relationship between bias leading to error and experience

-test booklet scenario unrealistic

-potential misclassification bias of study participants because we are assuming predictable differences between 1^st and 2^nd year residents

Two statistical pearls discussed

1. Confidence intervals in this study overlapped. If confidence intervals do not overlap, the means in a normally distributed sample are necessarily significant. HOWEVER, if they do overlap, they can still be statistically significant.
2. When to use which tests:

Continuous variables

2 groups: T-test

>2 groups: ANOVA, linear regression

Categorical variables

2 groups: Chi-square

>2 groups: Logistic regression


Slides HERE