Scores that Count

I should start with a disclosure. I don’t dig scorecards. Not so much scorecards which are sort of objective like on a football field. I’m more troubled by the scorecards you get where they’re trying to capture the ‘vibe’ of the thing.

That’s probably because I made the mistake of watching the equestrian during the Olympics once. Or it’s the result of struggling with the judges’ scores in the gymnastics where they sift out the form on the ‘flat bags’.  It’s all a bit unsatisfactory.

Then when I think from the other angle, I can see that it’s probably pretty hard to separate two horses trotting diagonally. How do you decide which horse is doing the best job of moving unnaturally?

At least it would be pretty easy to hand out scores for doctors, right? It’d have to be easier than grading horses, gymnasts or even sports that involve nose clips, sequins and underwater breath-holding.


Surgeons and Artistic Merit

Well recently in the US Propublica publicised a “Surgeons Scorecard” (you can see the entry page here and the accompanying story here). They took (US) Medicare billing records for inpatient hospital stays for a variety of common elective procedures from 2009-2013. They then compiled a list of complications you might expect related to those operations and cross-checked against individual surgeons’ results.

Then they put the complication rates out there. For 16,827 surgeons.

That’s good, right? Having information out there should help. Knowing complication rates would give patients data that’s important to their health care. Having information published should give surgeons an incentive to improve practice.

Well not everyone is excited by these scorecards.

In fact there are those who suggest that reporting on surgical complication rates to improve patient health care is a good way to erode good patient health care.

Does it make sense yet?


Because not only does this Monarch caterpillar struggle to follow this risk stuff, it looks cool.

Because not only does this Monarch caterpillar struggle to follow this risk stuff, it looks cool.

Dropping the Technical Difficulty

The counter argument is pretty much summed up in this piece from the New York Times and this commentary from Medscape. One argument leveled against scoring systems is that to avoid bad ratings surgeons or other proceduralists are driven to sift out the risky cases. Walking past those cases is perceived to be a way to walk to a better rating. Higher risk cases on the other hand guarantees more complications and a worse rating. Healthcare distortion by measurement.

It’s not that those with the texta and white cardboard in hand don’t recognise there’s different risks in different situations. ProPublica have a discussion of their methodology posted alongside the other pages. Nobody disputes they’ve made an effort, but procedualists don’t always feel that enough allowance is made for all the other factors that might impact on complication rates.

It’s not just an overseas phenomenon either. The NSW Bureau of Health Information has recently released a report in its spotlight series looking at readmission to acute care occurring after any one of a range of initial health admissions. These were conditions such as heart attacks, strokes, heart failure, pneumonia and different varieties of orthopaedic surgery. The authors have attempted to factor in all the other things that might impact on patient health and lead to a need for readmission. Age, sex, comorbidities, private vs public hospital and socioeconomic status all get a look in. Just listing those gives some sense that it’s complex to measure. Whether they’ve got those relative risk considerations right will no doubt be up for scrutiny.

Even more significant is the use of data that is incomplete or incorrect. The Medscape piece even refers to the fact that the author’s own hospital has a scorecard entry for a cardiologist performing a knee replacement. This is not a thing. It’s hard enough to balance contributors to risk well but surely it’s impossible when you’re working off data that isn’t actual data.


Who gets the score right?

So are scorecards the way forward or do they just leave the highest risk patients at risk of not getting care? Should this reporting happen or not?

Well of course open reporting has to happen. Somehow. The balance might be hard to get right but reporting and understanding things that can be done better is kind of what healthcare is about. And for all the astonishment that comes with medical advances some of the biggest advances are actually from doing simple things better.

Here’s an example. Infections in the sorts of cannulas that go into the larger veins of the body are a major potential source of infection. That sort of infection can cause big problems. They can be associated with patient death. Over the last couple of decades it has become apparent these infections aren’t something we have to put up with either.

If you read through this Vox piece (based on US data and stories) one of the more interesting points is that sometimes when you examine a problem the solution can be pretty simple – a five point checklist instead of a list of 90 things to consider to mitigate. Giving health facilities an incentive to introduce those simple steps and watch their complication rates with public scrutiny has been part of reducing these infections. Public reporting can be part of good changes.


Better Scoring

It still has to be the right information though. The answer is not to axe the idea of scorecards but report it in a way that is accurate and doesn’t influence care in the wrong way. And it seems like too often the reporting is confined to complications not success rates at pulling off good results. Is it because positive health statistics take more effort to track?

Then there’s the other big story in tracking health information. For frontline workers it doesn’t always seem like we have the space to prioritise information gathering in our day to day work. We don’t necessarily get the support to design the reporting systems we’d like. Every time health hits the political pages you expect to see something about “frontline staff”. Apparently you can never threaten the numbers of frontline staff. If you work in an office you are fair game.

The thing is frontline staff need adequate time and support to do the other work – it’s the work that is spent doing stuff other than direct patient care that allows us to look at information, update practice, get education done and fix up policies.

Caring for patients better doesn’t always involve the magic paddles with the shouting of “clear” or machines that go bing. And every time someone in support is taken away clinicians take up general admin work and have less time to do the other stuff. Less time to look at the markers that matter.

Better measurement needs better support. When do we introduce a scorecard for whether healthcare facilities do that?


The image today is from The Commons on and was posted by Vicki De Loach (with no modifications made). 


Rwanda Day 4: We didn’t bring a robot

Day 4 and we’re 10 operations in. A few challenges along the way but that’s sort of the nature of the game. Although it’s been easy enough to mention the stuff about all the fancy gear and big machines, it’s actually probably more important that we brought people.

There’s more than just those you might have guessed as well. In addition to anaesthetists, physicians, nurses and physios, there is someone just to look after all the technical equipment, someone to manage pathology, a pharmacist and someone providing all that excellent info over at

They’re all vital and none of them could be replaced by machines. I wouldn’t even trade in the surgeon for a robot.

But Robots Are Cool

Well, robots are sort of cool. Or sometimes freaky. This self-folding origami one is kind of freaky.

In medicine though they’re getting popular and that includes in surgery. For an aid trip they are a bit big and cumbersome and expensive (like millions of dollars too expensive). We’re also not talking about fully autonomous humanoid things with temperament chips that happen to cut. The robots involved in surgery are mostly of the variety where a human still operates them.

The advantage that gets mentioned is that superior control for the surgeon can mean less tissue trauma to impose on the patient and that means less pain and better recovery for the patient. Sounds good, right?

I’ve mentioned before that I’m not convinced robots have found their niche yet. Part of the reason is that for every clinical situation where evidence appears to be mounting that they are of benefit, as in prostate surgery there are other areas where people have been overzealous. Situations like some forms of gynaecological surgery (see also here), or where there is evidence that lots of complications from robot-assisted surgery go underreported.

There are people already using these machines for cardiac surgery and examples of researchers trying to figure out if there might be benefits. What I am fairly confident of is that we are a long way off robots being able to do much without the input of the human brain (not that many are suggesting that really).

The Upsides of the Human Brain

You’ll have to gather that I am effectively whispering this online, as it’s not traditional to spend too much time telling the surgeons they are impressive at what they do. And having humans involved in stuff comes with issues with the way humans think and act in challenging situations. On this trip I’ve even heard a couple of stories of people behaving colourfully in healthcare situations that left me with this look…

Yep, pretty wide-eyed.  [Via]

Yep, pretty wide-eyed. [Via]

The things I find super impressive about the human brain are demonstrated in the way the surgeons repair the heart.

For some of the repairs there might be holes in the heart to fix, or valves to repair or replace (we’re not doing so much in the way of replacements here). When the surgeon is doing that though they’ve already put the patient onto the bypass machine, drained it of blood and given a dose of a solution to stop the heart from beating at all.

So surgeons work in an empty, blood-free heart to produce a repair that has to work in a heart that is pumping away and full of blood. In the meantime they also fundamentally alter the way the heart has been working.

That’s partly a testament to the impressive ability of the heart to adapt to the new state of affairs (in most cases pretty easily). The surgeon though has to judge how to do their repair to deal with an entirely different set of conditions once they get that heart going again. It’s a bit about experience and a bit about the ability to see the structure in front of them and visualise it under a different future reality and a lot about judgment.

So we’ll all be saddled with old jokes, occasional interpersonal challenges and some fairly ordinary musical choices for a while before robots can do all that.

And most of the time we’ll be quietly impressed.

But definitely not mention it.

Honestly sometimes I think it's better not to ask.

Honestly sometimes I think it’s better not to ask.