The Big Statistic in Evidence-Based Medicine

When you’re a doctor who has to get kids to snooze through an operation you need a few skills. You need a repertoire of kids’ songs and a few stories to make having a mask hover over your face seem vaguely logical. You need to be able to tolerate a wide range of musical tastes amongst your colleagues. Even Phil Collins megamixes. It is also useful to be able to apply a knowledge of related bits of medicine, evidence particular to the area you work in and an ability to apply that to the needs of the patient. That is, after all, the core medicine bit.

Successfully integrating evidence into practice is defining for a doctor’s career and the way they view themselves as a practitioner. From the first bright-eyed lecture of my training, evidence-based medicine (EBM) has been the driving philosophy for doctors. As a student at first this seems pretty simple. You have a patient, they have a thing that is going on. That thing has been explored by people who know about things. They have suggestions for what you do when that thing is the problem. You apply those suggestions. You have done evidence-based medicine [high five for the super rational doctor].

It doesn’t take long to realise it is much harder than that. The broader state of play with evidence-based medicine is constantly under discussion (as in this excellent article by Greenhalgh et al).  The most prominent early definition of evidence-based medicine from David Sackett et al. described a practice of “integrating clinical expertise with the best available external clinical evidence from systematic research”. It also was noted to be about applying the evidence to the care of individual patients.

The trick with those patients is that they are so very individual. Sometimes this means the evidence that’s out there is a bit of a round hole for the triangular patient peg.  There’s also another obvious bit of variability there. The ‘clinical expertise’ bit. That relies entirely on the individual doctor and doctors are people which equals chaos in so many ways.

In that article by Greenhalgh and colleagues, they point to 5 big problems with EBM. They rightly point out that EBM has been distorted by those looking to make a buck or push a point. The penetration of EBM has also led to such a vast amount of evidence, practitioners can be overwhelmed by such vast amounts of information it can feel like being washed off with a water cannon. Primacy of statistics can lead to results being pushed which look pretty in a statistical analysis and meaningless in the clinical practice. Then even where the evidence may be good, it is not always applied in a way that is patient-centred, or the evidence doesn’t adequately address those with lots of other health issues at the same time.

Those are all true. The group who gets off lightly though are the doctors and an examination of the most influential of all statistical relationships: n = 1.


Would an encounter with a tiger change anything? [via fPat Murray under CC]

Would one encounter with a tiger change anything? [via fPat Murray under CC]

The Personal is Generalisable

As I’m doing a PhD, I am having to learn more statistics than I ever thought I would retain beyond an actual exam. I even know that bootstrapping is a legitimate method in statistics and not just a preparatory step for competitive line dancing. This puts me ahead of plenty of doctors who evidently struggle a bit with statistical concepts highly relevant to their everyday practice.

The association that does have a lot of sway for doctors is n = 1. This represents the number of personal experiences of a particular outcome a doctor must encounter before it dwarfs the impact of the  values in any given paper. The number is 1. I’d argue that a personal experience in a doctor’s practice drives the way they interpret the research they read more than dry discussions of EBM allow. This is where we need more discussion.


Air Goes In and Out. Preferably.

Here’s an example. When kids have an anaesthetic they will sometimes have things happen that could be broadly termed “breathing events”. This might be coughing, breath-holding and occasionally they’ll have a reflex movement that causes the structures around the vocal cords to contract tightly. This stops air going in and out (we consider that not great) and is called laryngospasm.

On the few occasions it does happen it is generally easy to get it to resolve. Very occasionally, it is not. As the air stops going in and out the body uses up the oxygen that’s handy and more and more of the blood flowing around shifts to the colour it adopts when it’s unburdened by that oxygen. This is at the bluish end of the spectrum. If it goes on a bit longer that colour shift becomes more apparent and it can be followed by the heart beating slower. At that point you rapidly discover that there are quite a few shades between blue something malignantly darker that you never wanted to see. Having seen it a handful of times, it’s not a memory I’m looking to recreate.

We have evidence that if you’ve had a recent bug that has given you a cold your airways get a bit ticklish. It’s a little like when passing drifts of pollen make you cough for weeks afterwards. Assuming the operation can’t wait we have a series of choices to make about how to make sure we can get air in and out during the anaesthetic, be it the sort of breathing tube they habitually misplace in television shows that passes all the way down to the windpipe (an endotracheal tube), or an alternate kind that sits above the structures in the larynx (a supraglottic airway).

We know that the latter one is less likely to trigger one of that broad group of “breathing events” than the endotracheal tube. As a younger anaesthetist I used to tend to avoid the tube for this reason. A quick vox pop of the more experienced heads showed that plenty more of them did exactly the reverse. And when I chatted about it often at the bottom of that decision was “well I saw this kid go black once with a laryngospasm. Can’t laryngospasm strong enough to stop air going through the tube.”

This anecdote could be repeated in any of a number of areas. The one experience that sticks in the mind, particularly if associated with either the rush of a great outcome or the horror of a bad one, assumes huge importance. It is also fair to say that the evidence you already know gets reinterpreted to make sense of your new experience. All evidence is malleable to support a new philosophy.

No Assumptions

All the points covered in that discussion of crisis by Greenhalgh and colleagues are entirely valid. The solutions make lots of sense. Perhaps what we need to remember is that trying to make clinical decisions governed by reason will be hard if we don’t also discuss the irrational influences on what doctors do. There’s work out there on just this topic as covered in this excellent general practice perspective (or if you like, here’s one example of a review on this here).

The last thing we should do is assume that the individual can be forgotten in trying to make EBM as robust as possible. Individuals are the ones making plans with their patients  so it’s vital not to forget their values and foibles in the process. It’s a lot more practical to factor that in than to assume that bootstrapping statistics will become part of the hipster doctor future.



One thought on “The Big Statistic in Evidence-Based Medicine

  1. Pingback: The Big Statistic in Evidence-Based Medicine – The Flying PHD | Emergency Live

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s