A systematic review in the BMJ examined how well health professionals understand diagnostic tests. Using “statistics”, “healthcare”, and “accuracy” keywords the reviewers searched EMBASE, PsycINFO, and MEDLINE databases identifying 4818 hits. 74 were reviewed as potentially relevant based on title and abstract, with 28 meeting full inclusion criteria. Due to the heterogeneity of studies the systematic review was limited to presenting its findings in narrative format. The authors categorized the findings into four themes: self-rating, accuracy of definition, Bayesian reasoning, and presentation format categories.
Two studies examined clinicians’ self-reported familiarity of statistical measures. In one study 13/50 clinicians reported understanding sensitivity, specificity, and positive predictive value; although only one was able to provide a correct definition when asked to do so. The other study found that only 58% of clinicians actually used statistical measures in their practice (although 82% claimed to do so).
6 studies examined clinicians’ understanding of statistical definitions; they found that on average clinicians could provide a correct definition for sensitivity 76-88% of the time, specificity 80-88% of the time, but only 17% could correctly identify the correct definition for likelihood ratios.
22 studies examined how well clinicians’ were able to use pre-test probability and test accuracy to determine post-test probability (Bayesian reasoning). The studies found that in general clinicians had a poor understanding of Bayesian reasoning and were unable to determine post-test likelihood the majority of the time (0-61% success), clinicians also tended to overestimate post-test probability by 46-73%, and in one study clinicians inverted the likelihoods incorrectly interpreting patients with negative results as having a higher post-test likelihood of having a disease.
The 5 studies that examined presentation format found that healthcare providers were more accurate in their post-test estimates if the findings were expressed as natural frequencies (50 out of 100) rather than as probabilities (50%). The use of graphical aides improved clinician post-test accuracy to 73% compared to 48% when natural frequencies alone were used to communicated test power, or 23% when probabilistic language was used to describe tests.
This systematic review suggests that not only are we as healthcare providers poor at using probabilistic reasoning; but that we’re also oblivious to our weakness in this area. As a nurse I know that the preparation I received in school to interpret and use statistics was severely lacking. This research, which focused primarily on physicians, would suggest that this is a common area of weakness. This research highlights how little we all know about the tools we use on a daily basis, it may also shed some light on why so few of our colleagues engage with research, and I would suggest that most importantly it empirically shows that we could all use some brushing up on how to use statistics. I’ve covered some great resources to help with this is a few previous weekly reviews (#4 & #6) and would suggest that anyone looking for a brief intro into using Bayesian statistics take a look at the first few chapters of this online book.
Steve Mathieu reviewed the HEAT trial of acetaminophen for fever in critically ill patients on The Bottom Line Review. This study was a double blind RCT of 700 patients that sought to determine if the administration of paracetamol to critically ill patients had any effect on: ICU free days, mortality, length of stay (LOS), number of days on organ support, and its effect on lab values and temperature. Patients were block randomized to receive either 1g IV paracetamol, or IV D5W every 6hrs for 28 days or until: ICU discharge, fever resolution, cessation of antibiotics, death, or contraindication. The study found no statistical difference in mortality, LOS, ICU free days, or organ support, although patients who received paracetamol had a lower (0.25C) average temperature. This research is contrary to a retrospective study published earlier this year that found a mortality benefit associated with paracetamol administration, it has a much higher quality design, and is one that I've been waiting to see published. Mathieu's summary of this research is concise, easy to interpret, and very timely; the summaries on this site are consistently of very high quality, this review and the site in general are a fantastic resource.http://www.wessexics.com/The_Bottom_Line/Review/index.php?id=7257804966227311886
A retrospective analysis of tourniquet use in the
pre-hospital setting was published in the Journal of Acute Care Surgery this
month. Ode et al., examined the EMS use of tourniquets in a metro North
Carolina ambulance service during 2012-2013. They examined patients with
uncontrolled hemorrhage to determine the frequency of “correct” tourniquet (Tk)
application, the efficacy of Tk as a treatment, and the frequency of Tk related
adverse outcomes. 98 patients met the inclusion criteria (uncontrolled
hemorrhage), 42 were excluded because they were treated outside of the metro
area. Of the remaining 56 patient: 24 received a Tk (19 Combat Application Tourniquet,
5 improvised), although 5 were deemed unnecessary (the patients weren’t in
shock - SBP>80mmHg) and 4 were delayed. Of the 32 who didn’t receive
tourniquets three were indicated, but did not receive treatment.
The patients treated with a Tk, compared to those not, had
significantly higher rates of: shock (50 vs 12.5%), vascular injury (69.6 vs
25.8%), blood transfusion (37.5 vs 9.4%), rates of admission (77.3 vs 38.7%),
and mortality (8.3 vs 3.1%). None of these findings are surprising given that
the protocol for application of a Tk was quiet conservative, requiring patients
to be in shock, and therefore significantly sicker. Secondary analyses showed
that patients who were indicated to receive a Tk but didn’t had higher incidence
of shock compared to those who did (85.7 vs 60%), and that those who weren’t
indicated (not in shock) but did receive a treatment by tourniquet had no
adverse outcome as a result.
Due to the overly conservative treatment protocols, the
small sample size, and the lack of an equivalent control arm the primary
results of this study have little to contribute to the overall body of evidence
for Tk use in civilian trauma. The secondary analysis does show worse outcomes for
patients with missed Tk’s, and no complications associated with liberal Tk (non-indicated)
use; although the numbers (n=22) are too small to reach statistical
significance. From a clinical perspective this research would seem, in a small
way, to support that liberal Tk use isn’t associated with worse outcomes, and
that even extremely conservative (only once shock becomes apparent) Tk use
imparts benefit. From a practical perspective this research provides weak
evidence to support Tk use in civilian trauma, it also suggests that the
current military research may be generalizable to the civilian population, and
indirectly it would seem to suggest the need for a liberalization of Tk protocols
among EMS services.
http://www.ncbi.nlm.nih.gov/pubmed/26402532
Using the current VW emission scandal as a comparison piece Richard Smith offered a critique of scientific misconduct on his BMJ blog, and suggests that scientists should face criminal charges if found guilty. Criminal charges have been used to punish scientific misconduct before, as was the case with June Dong-Pyou Hon’s faking of HIV immunization results; and Smith offers three additional reasons why misconduct should be investigated criminally: Because inappropriate use of research funding is financial fraud, because universities are poorly equipped to conduct investigations, and because investigations by the university would be a conflict of interest. I would also suggest that there is a basis for criminal charge based on harm to the patient, for example the intentional non-disclosure of the increased risk for suicide when paroxetine is used to treat pediatric depression, and Andrew Wakefield's fabrication of evidence that immunizations result in autism. A thoughtful and timely piece by Smith, maybe one that will find increased traction.http://blogs.bmj.com/bmj/2015/09/28/richard-smith-if-volkswagen-staff-can-be-criminally-charged-so-should-fraudulent-scientists/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+bmj%2Fblogs+%28Latest+BMJ+blogs%29&g=w_blogs_bmj-com
Mark Culver discussed the differences between intermittent and continuous PPI therapy for UGIB on Emergency Medicine PharmD. There has been a fair bit of research into this topic lately, and UpToDate had suggested changing practice from continuous to intermittent PPI therapy last year; however the practice remains widespread. This post offers a great review of the research behind the change in recommendations. The cost savings, and reduction in nurse time this practice change could result in make this post well worth the read.
There was a podcast review of fluid resuscitation in hemorrhagic shock on HEFTEMCAST. This podcast reviews the key evidence and discusses the concepts of: damage control resuscitation: permissive hypotension, hemostatic resuscitation, and damage control surgery. The review includes seminal work from the military as well as evidence from civilian trauma, it nicely summarizes the key literature, and provides links to the source research. This 16 minute review is well worth a review for anyone working in an emergency settings.
Josh Farkas offered a well balanced review of the SPLIT trial on PulmCrit. The SPLIT trial examined the differences between PlasmaLyte and Saline in patients admitted to the ICU, and found that there was no significant difference between the two fluids. Farkas acknowledges that the findings are valid among the patients reviewed, but critiques the external validity of the study; he points out that the small volumes of fluid received, the admission reason (elective post-operative), and the low illness severity observed among the patients in this cohort are not generalizable to the typical ICU population.
No comments:
Post a Comment