The number in the “stat for that” isn’t 42 either …

There’s an awful lot of numbers in medicine. Numbers of patients. Numbers related to all the years of training and numbers of this and that case. There are numbers for all the machines that go bing. There is a finite number of times you can try to look like you find a senior doctor’s one standard joke funny.

There’s also lots of numbers to attach to diseases and treatments. They’re the ones you’re forced to learn along the way – incidence and prevalence, mortality rates, complication rates. Then there’s the number that’s been getting a little coverage of late – the NNT or number needed to treat.

 

Magic Numbers

The headlines of recent pieces are a little breathless – “Can This Treatment Help Me? There’s a Statistic for That” or “This Man’s Simple System Could Transform American Medicine”. Part of you clicks on these links expecting the magic number to come with a special one-time limited bonus edition blender to turn the printout of your number into an easily digestible health drink.

And it would be pretty great to have the sort of number that would tell you whether you needed a pill for your complaint. Or surgery. Or acupressure delivered by a particularly well trained capybara. (Yeah, you spotted the odd one out. Does anyone ever really “need” surgery?)

Do you need the shelled sidekicks for acupressure? [via the excellent @emergency_capy Twitter account]

Do you need the shelled sidekicks for acupressure? [via the excellent @emergency_capy Twitter account]

 The thing is that amongst the text (especially the NY Times one) is a pretty sensible acknowledgement – of course it’s not a magic statistic for everything. In fact I know plenty who would say you can’t use it to give any more than a vague “yeah, it’s probably pretty good.” It’s certainly not useful enough to hand over all future medical decisions to Wayne your local shopping centre tax accountant any time in the future.

 

You Need More Numbers

If you actually go on to read the articles in a bit more detail (assuming you can get past the little “segregation animations” on the NYT version) they make some points that are needed to put this in perspective.

The first is that your particular NNT only applies to that particular situation. So your 1 in 2000 aspirin preventing heart attack stat applies to that particular prevention scenario. But if you apply aspirin to the patient who has had a heart attack before, the NNT is more like 1 in 77 (to prevent a non-fatal heart attack). So it’s kind of important to know that the NNT you’re talking about actually applies to the particular situation you’re assessing.

More importantly, the NNT is based on research in specific patient populations. One of the biggest challenges in medicine applying clinical research to the patient in front of you. And there’s plenty of patient’s who don’t fit the clinical trial box. A little too old. The wrong extra diseases. The trial was done in an entirely different socio-economic group. The patient doesn’t quite fit? Then the NNT isn’t quite the NNT.

On top of that research studies tend to idealise conditions. In particular, compliance with the intervention is pretty likely to be higher than the real world. Not taking the medications as much as in the trial? The NNT just got weaker again.

And if you’re looking at numbers you don’t get to do it in isolation. You need to look at the potential for bad stuff to happen from the treatment – in this case the number needed to harm (NNH). If you go back to the aspirin example, 1 in 3333 have a major bleeding event when being given the little white pill. And each time there’s the NNT, there’s the NNH you have to consider to weigh up whether the treatment is worth it. (Happily, the NY Times article which really is good mentions they’ll get to that in the next article).

So if your kid has an acute middle ear infection you’d like antibiotics right? Well 1 in 20 do get a reduction in pain once more than 24 hours have passed but there is no change in infection complications or recurrence. 1 in 9 get diarrhoea. Still keen?

 

There are Some Things Not About the Number At All

There’s another issue that’s pretty central to the practice of medicine too. It’s sort of about patients. And using evidence-based medicine isn’t just about applying numbers and statistics. Evidence-based medicine is a process that seeks the best available information to guide the best course of action for the particular patient being considered, within the context they find themselves.

It’s worth taking the time to consider a broader view of what is required for good practice of evidence-based medicine from this recent piece in the BMJ by Trisha Greenhalgh. In it she highlights the need for the patient to be involved in “real shared decision making” (amongst lots of other issues). This requires the clinician to find out “what matters to the patient – what is at stake for them”. Only in doing that can the dialogue about treatment be appropriately informed.

That’s why the idea of a “stat for that” doesn’t quite work out so simply. There might be numbers that provide information about whether you should adopt a treatment. But there’s not an easily accessible number that says much about what is at stake for the patient. That’s where the people working in health come in.

And maybe capybaras if there’s actually a way to practice evidence-based acupressure.

 

More? The actual site that provides summaries of lots and lots of NNTs is www.thennt.com 

Also, just after I posted this the follow-up NY Times piece on NNH appeared (and chose some of the same numbers as their examples coincidentally). Here’s the link and it is a good read just like the first NNT one that triggered this post. 

The Big Statistic in Evidence-Based Medicine

When you’re a doctor who has to get kids to snooze through an operation you need a few skills. You need a repertoire of kids’ songs and a few stories to make having a mask hover over your face seem vaguely logical. You need to be able to tolerate a wide range of musical tastes amongst your colleagues. Even Phil Collins megamixes. It is also useful to be able to apply a knowledge of related bits of medicine, evidence particular to the area you work in and an ability to apply that to the needs of the patient. That is, after all, the core medicine bit.

Successfully integrating evidence into practice is defining for a doctor’s career and the way they view themselves as a practitioner. From the first bright-eyed lecture of my training, evidence-based medicine (EBM) has been the driving philosophy for doctors. As a student at first this seems pretty simple. You have a patient, they have a thing that is going on. That thing has been explored by people who know about things. They have suggestions for what you do when that thing is the problem. You apply those suggestions. You have done evidence-based medicine [high five for the super rational doctor].

It doesn’t take long to realise it is much harder than that. The broader state of play with evidence-based medicine is constantly under discussion (as in this excellent article by Greenhalgh et al).  The most prominent early definition of evidence-based medicine from David Sackett et al. described a practice of “integrating clinical expertise with the best available external clinical evidence from systematic research”. It also was noted to be about applying the evidence to the care of individual patients.

The trick with those patients is that they are so very individual. Sometimes this means the evidence that’s out there is a bit of a round hole for the triangular patient peg.  There’s also another obvious bit of variability there. The ‘clinical expertise’ bit. That relies entirely on the individual doctor and doctors are people which equals chaos in so many ways.

In that article by Greenhalgh and colleagues, they point to 5 big problems with EBM. They rightly point out that EBM has been distorted by those looking to make a buck or push a point. The penetration of EBM has also led to such a vast amount of evidence, practitioners can be overwhelmed by such vast amounts of information it can feel like being washed off with a water cannon. Primacy of statistics can lead to results being pushed which look pretty in a statistical analysis and meaningless in the clinical practice. Then even where the evidence may be good, it is not always applied in a way that is patient-centred, or the evidence doesn’t adequately address those with lots of other health issues at the same time.

Those are all true. The group who gets off lightly though are the doctors and an examination of the most influential of all statistical relationships: n = 1.

 

Would an encounter with a tiger change anything? [via fPat Murray under CC]

Would one encounter with a tiger change anything? [via fPat Murray under CC]

The Personal is Generalisable

As I’m doing a PhD, I am having to learn more statistics than I ever thought I would retain beyond an actual exam. I even know that bootstrapping is a legitimate method in statistics and not just a preparatory step for competitive line dancing. This puts me ahead of plenty of doctors who evidently struggle a bit with statistical concepts highly relevant to their everyday practice.

The association that does have a lot of sway for doctors is n = 1. This represents the number of personal experiences of a particular outcome a doctor must encounter before it dwarfs the impact of the  values in any given paper. The number is 1. I’d argue that a personal experience in a doctor’s practice drives the way they interpret the research they read more than dry discussions of EBM allow. This is where we need more discussion.

 

Air Goes In and Out. Preferably.

Here’s an example. When kids have an anaesthetic they will sometimes have things happen that could be broadly termed “breathing events”. This might be coughing, breath-holding and occasionally they’ll have a reflex movement that causes the structures around the vocal cords to contract tightly. This stops air going in and out (we consider that not great) and is called laryngospasm.

On the few occasions it does happen it is generally easy to get it to resolve. Very occasionally, it is not. As the air stops going in and out the body uses up the oxygen that’s handy and more and more of the blood flowing around shifts to the colour it adopts when it’s unburdened by that oxygen. This is at the bluish end of the spectrum. If it goes on a bit longer that colour shift becomes more apparent and it can be followed by the heart beating slower. At that point you rapidly discover that there are quite a few shades between blue something malignantly darker that you never wanted to see. Having seen it a handful of times, it’s not a memory I’m looking to recreate.

We have evidence that if you’ve had a recent bug that has given you a cold your airways get a bit ticklish. It’s a little like when passing drifts of pollen make you cough for weeks afterwards. Assuming the operation can’t wait we have a series of choices to make about how to make sure we can get air in and out during the anaesthetic, be it the sort of breathing tube they habitually misplace in television shows that passes all the way down to the windpipe (an endotracheal tube), or an alternate kind that sits above the structures in the larynx (a supraglottic airway).

We know that the latter one is less likely to trigger one of that broad group of “breathing events” than the endotracheal tube. As a younger anaesthetist I used to tend to avoid the tube for this reason. A quick vox pop of the more experienced heads showed that plenty more of them did exactly the reverse. And when I chatted about it often at the bottom of that decision was “well I saw this kid go black once with a laryngospasm. Can’t laryngospasm strong enough to stop air going through the tube.”

This anecdote could be repeated in any of a number of areas. The one experience that sticks in the mind, particularly if associated with either the rush of a great outcome or the horror of a bad one, assumes huge importance. It is also fair to say that the evidence you already know gets reinterpreted to make sense of your new experience. All evidence is malleable to support a new philosophy.

No Assumptions

All the points covered in that discussion of crisis by Greenhalgh and colleagues are entirely valid. The solutions make lots of sense. Perhaps what we need to remember is that trying to make clinical decisions governed by reason will be hard if we don’t also discuss the irrational influences on what doctors do. There’s work out there on just this topic as covered in this excellent general practice perspective (or if you like, here’s one example of a review on this here).

The last thing we should do is assume that the individual can be forgotten in trying to make EBM as robust as possible. Individuals are the ones making plans with their patients  so it’s vital not to forget their values and foibles in the process. It’s a lot more practical to factor that in than to assume that bootstrapping statistics will become part of the hipster doctor future.