Early Excellence in Failure

There is a point at which I hope to do research which doesn’t fail. There’s probably a way to write code that would tell me when that point might be. Of course, I don’t code. I fail at that too.

At this point I can see a giant set of sparkling white self-help teeth spouting horrible affirmations about our failures being the first sunbeams of success (damn late night TV).  Not quite hitting the expected mark in research is clearly very much part of the standard story, even though it feels as graceful as this …

I remember feeling like this for about  a year. [via hdwpapers.com]

I remember feeling like this for about a year. [via hdwpapers.com]

I was part of a research team that undertook a failed bit of research. When I look back at it, it reeks of inexperience as a researcher. And I have to confess that the failure was a ‘valuable learning process’ . Yes, that hurt as much to write as it did to read. Of course, even research that looks implausible isn’t really a failure as there’s value to be found everywhere. Possibly though, before sharing the myriad ways we got it wrong a little background might help.

The Project

The trigger for the whole thing was a worthy question about burns care. In anaesthesia we focus a lot on managing symptoms. When most people think burns, they think of the pain. Which is fair enough because it’s a big issue but we have an arsenal of pain management options. The other really annoying question is what to do about the itch.

Somewhere north of 80% of people with sizeable burns (which we’ll define as more than 10% of their body surface area) will report itch at some point and it’s often severe. A variety of cells involved in the healing process tend to release substances which, while important to the healing, stimulate pathways in the nervous system that trigger that annoying bug-crawling feeling we’ve all cursed.

The itch caused by burns often seems to turn up a week or so down the track as healing gets into swing. Not only is it an issue because feeling like you want to scratch your own insides out with nothing more than a sweet potato. The itch is classically more disturbing at night (and if you’ve ever tried to sleep in a hospital, you’ll understand distractions aren’t welcome) and causes lots of distress. It can even lead to failure of wound healing or disruption of skin grafting as patients scrabble away at their dressings.

In addition to all of that, a lot of available early treatments are as effective as an old cattle dog trying to catch flies. So we set out to see if we could make something different work – a drug originally introduced to help with seizures, gabapentin, also helps with pain relief and in some cases itch. Having decided to try and establish its role for burns itch a little better we figured that even better than stopping itch would be preventing it ever happening. So we embarked upon a study to test it out. Just a little double-blind randomised controlled trial to test it all. That would be easy, right? Actually that’s where our problems started and we ended with incomplete data that was really hard to work with. Each one of our missteps has informed subsequent work though with some really obvious big themes for clinical research.

1. Know your area

A wide-ranging group of clinicians were involved in setting the trial up. They provided lots of useful input on things like how severe itch was, how many kids put up with it and crucially, how many eligible patients we’d have. Except those experienced clinicians were wrong. In particular, the ones saying how many kids were admitted on a month to month basis were wrong by a factor of ten. ‘Feelings’ about an area are no help where data is required. Which leads on to …

2. Make no assumptions

Rose-hued anecdotes are no substitute for numbers when trying to plan. To get onto the heart of the project, have actual numbers or pilot work to inform every bit of the later planning. The only acceptable assumption is that your gut is wrong. This also leads to …

3. Question everything and everyone  …

Accepting no assumptions means not allowing any of the little things through without consideration. You may end up having to accept a consensus view on an item of the plan, but do this with eyes open, not through an oversight. The follow-up is that you can’t assume people you work with are clever at this bit of research just because they’re clever at other things or other research. One of our fatal assumptions was that those with a bit more experience knew the numbers accurately or, in some cases, knew how to make this research work best. Some of the time they were working off a bit of first principle logic themselves.

Even as a junior researcher, question them. Not in an  “I want the truth!”/”You can’t handle the truth!” style, but to test their reasoning and clarify the planning. There’s only two outcomes from those discussions: you highlight a potential flaw or you learn from the beauty of their explanation.

4. Resources

Research needs actual resources not an assumption that your effort, enthusiasm and goodwill can get you there. All that stuff will wane. Other realities of life will impose themselves. The effort at the start doesn’t allow autopilot later. There is only continuous work so plan it like that. This also relates to …

5. Love isn’t universal

No one will love your project like you do. You won’t even love your project like you do when you start. If you set up a system (like we did) where a variety of people will be involved in the trial assume that all of them will have, at best, a passing interest in the work. One assumption is fair – they will probably devote as much brain space focussing on it as a night security guard with the TV remote in hand.

As a group, we learnt lots from undertaking the study. We ended up with better resourcing for research generally. It still didn’t poison us against future work and I’m now involved in work where we searched hard for actual numbers to inform our planning and we’ve spent way longer asking questions, preparing the logistics and working on having enough people continuously devoted to it.

Here’s hoping this latest project looks a fair bit more elegant. Or at least provides more refined failures.

I bet the second time they did a Tupac unicorn tattoo, it was awesome. [via the4sp.com]

I bet the second time they did a Tupac unicorn tattoo, it was awesome. [via the4sp.com]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s