The False Science of Picking Winners

I have nothing specific against Michel de Nostredame. I’d probably happily share a game of petanque with him if the chance came up. Having said that, a condition of us hanging out would probably be a strict “no prophecies” rule (yep, if you missed it, he’s otherwise known as Nostradamus). This is partly because quatrains are a fairly irritating means of conducting a conversation. The other thing, though, is that predictions of such a nature are fairly tiresome when founded on not much more than an incantation inspired by the direction of the breeze rather than robust examination of real data.

The flawed nature of such predictions came to mind this week when I saw this coverage. Let’s put to one side the bizarre situation that sees a climate change denying legislator heading the House of Reps science committee. The more substantive issue is the fact that such a lawmaker is seeking to supplant the whole peer review process. Before you say this is an “only in America” issue, concerns have been raised within the last 12 months by some members of the Australian parliament regarding the appropriateness of some of the funding decisions of the NHMRC and ARC, as well as the need to ensure research is in areas with obvious potential for driving innovation and growth of the nation (bit of a rough cut at the humanities).

The stated reason is always to ensure that “public money is being spent wisely”. They will insist that there must be a clear sense of almost immediate payoff. There are a few obvious problems with this. The first is that peer review is a rigorous process that’s unlikely to be replaced effectively by a committee of legislators. Those charged with peer review of a grant application take their job seriously, wading through large amounts of submitted paperwork, reading additional information and drawing on their pre-existing expertise. I’ve had the chance to be involved on both sides of the ledger, and it’s exhaustive.

The next problem is with the described preconditions being proposed. Let’s examine those. The conditions include that the research must be in the interests of advancing the national health of the US. You also have to make sure your research is groundbreaking and of the finest quality, answering questions of the utmost importance to society. Oh, and you can’t be duplicative.

The problem is that basic research can be a little shy about staking its claim. History is riddled with publications whose importance was only appreciated over subsequent decades. Just one example lies in the works of Max Planck, who released the initial work upon which quantum physics is built at the turn of the 20th century. Humbly called Thermodynamik (1897) and Theorie der Wärmestrahlung (Theory of Heat Radiation) (1906), in describing the relationship between energy and the frequency of radiation, he made a huge leap inundertanding many observeddiscrepancies in existing theories within the field of physics. You might not have guessed that from those titles.

Another great example lies in the comments of the linked article above. Consider for a moment the 1978 publication of Limits of Cosmic Radio Bursts with Microsecond Time Scales”. I’m sure the link to the subsequent innovation is really obvious. WiFi of course .In fact this is the original name of that paper. Every WiFi device on earth was built thanks to the sharing of that research.

Is it any surprise that we don’t do well at predictions (and I mean all of us, but now consider how that looks when we only consider the politicians)? Need further proof, how about these quotes (which I duly acknowledge may represent the clunkers that got written down, while lots of other voices were raised in opposition)?

– “Louis Pasteur’s theory of germs is ridiculous fiction.” Pierre Pachet, Prof of Physiology at Toulouse, 1872.

– “We don’t like their sound, and guitar music is on the way out.” – Decca Recording Co. rejecting the Beatles, 1962.

– “Who the hell wants to hear actors talk?” – HM Warner, Warner Brothers 1927. 

– “This ‘telephone’ has too many shortcomings to be seriously considered as a means of communication. The device is inherently of no value to us.” – Western Union internal memo, 1876. 

– “There is no reason anyone would want a computer in their home.” – Ken Olson, president, chairman and founder of Digital Equipment Corp., 1977. 

“I think there is a world market for maybe five computers.” – Thomas Watson, chairman of IBM, 1943.

The problem with predictions is (at least) two-fold – we are so often blessed with a total lack of imagination as to assume that only today’s applications make sense when a new factoid is discovered. The other problem is that we can’t measure the value of such discoveries within the short-term currency of the political cycle. Look at the lag between that paper in 1978 and WiFi becoming standard. We’re all glad they looked at cosmic microbursts now though, aren’t we?

The other big problem inherent in imposing the proposed standards from Lamar Smith is with the requirement to be “groundbreaking”. As much as I quite like the idea that all science types are quietly diligent superheroes, shattering the falsely constructed frontiers limiting the pursuit of knowledge and wisdom, the truth is a little more low key than that. Research pursues small steps, not massive strides in most instances. Most researchers will add a very particular tile in the mosaic they’re working on, while others working in the same field add their own multicoloured irregularity.  Research rarely breaks the ground, it slowly claims it after the model of the inchworm.

So let the experts review the grants and leave the politicians playing out their small time dramatisations after Machiavelli. There is no tougher judge of an application than can be found in the peer review (whether it’s too onerous is for another day). Once we start demanding results delivered by bedtime for the MTV generation, or titles we can decipher the future from, we erode the ability of the journey of research to deliver unforeseen possibilities (not to mention the struggle of the humanities, where it’s harder to measure the market benefit derived from your work). And if the politicians must try to tell the future, let’s verse them in obscure literary forms (haiku, anyone?) and suggest they write down those predictions in a vague poetic style that leaves room for numerous interpretations. That way, they might just have people chatting about them in a few centuries time . Then let the researchers get on with the innovation that will guide all of our futures.


2 thoughts on “The False Science of Picking Winners

  1. Pingback: Failure Fetish and Aardvarks in Research | The Flying PhD

  2. Pingback: Research and a New Big Brother | The Flying PhD

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s