"There are only two ways of telling the complete truth—anonymously and posthumously."Thomas Sowell

Friday, February 10, 2006

Consensus Nonsense

In a space of less than 24 hours, both "consensus science" and "evidence based medicine" have been mentioned on the Reform Club, and I gather from the comments sections that neither of these terms is being precisely understood.

"Consensus science" does not refer to the perfectly organic, Kuhnian process of scientific progress through hypothesis testing, replication, and peer review. While this process often does produce what might be termed 'consensus' on a body of theory, this is not what the term means. Consensus science is a particular process of reaching conclusions by committee. The rise of the scientific bureaucracies and the vastly increased interconnections between academic science and government that have grown up in the post World War II period have provided the culture medium that consensus science has colonized.

It's only important to produce a consensus if money is being handed out or regulations passed or actions forbidden on the basis of that consensus. From the 1950s through the late 80s these actions were typically intranational, of localized import -- the FDA allows a new drug on the market, NIH gives twenty million dollars to Johns Hopkins, that sort of thing. This changed in 1987 with the negotiation of the Montreal Protocol, an international agreement to phase out the production of chemicals that were thought to deplete stratospheric ozone.

The Montreal Protocol opened the door for similar treaty negotiations on the issue of global climate change. This happened despite the fact that the science underpinning Montreal had much more in common with the limited issues that had typified earlier consensus science than it did with the scientifically immature discipline of global climatology.

Evidence-based medicine is the clinical equivalent of consensus science: it's mass-produced medical treatment by anonymous committee. I am ashamed to admit that while my only role in consensus science has been mid-level onlooker (I was a GS-11 policy analyst at NSF for three years under Reagan and Bush 41) I have played an active, if minor, role in inflicting evidence-based medicine on American citizens. Evidence-based medicine, which sometimes travels under the alias 'best practices' is a centralized medical bureaucracy's attempt to ration care by only allowing those procedures approved by a committee of physicians. Paired with 'computerized medical records' -- another current bugabear that I was alarmed to hear mentioned in the SOTU speech -- you are moving towards having a computer tell your doctor what tests and treatments you should, may, and may not receive, based on running your computerized data through some algorithms coded by a couple of white coated eggheads at McMaster University. I know because I helped write some of them. I also helped design one of the main database applications that makes computerized medical results possible. I'm sorry. If it makes you feel any better, they didn't pay me very well.


Hunter Baker said...

Great stuff, Kathy. This is exactly what I was talking about earlier. I love it when someone comes to the ballgame with some experience or other advanced knowledge of a topic. And to think I get all that payback just for re-posting something from another blog.

Tom Van Dyke said...

Your work on the computer program reminds me of the tour of the engine room in The Caine Mutiny: it's "designed by geniuses to be run by idiots."

Having headhunted doctors for awhile and getting into their heads (shudder), I'm not sure that it's a bad thing to apply such rigid protocols to diagnosis. Older doctors quickly get out of touch, and young ones get addicted to the latest and greatest.

Systems are good when humans are involved, because even though we're created equal, we don't end up that way. The best and brightest are designing the system for the highest achievable mean: great doctors will find a way to stray from the protocol to save patients, while poor doctors will kill fewer of theirs by following it.

(Although the best and brightest make mistakes too, one would think that a protocol that kills more than it saves won't last long.)

I think you're right about the Montreal Protocol, in that getting rid of CFCs and the like had a modest price, but with a priceless (possible, albeit not definitively proven) benefit, the preservation of the ozone layer. So everybody said, what the hell, let's do it.

The same logic is being applied to global warming via Kyoto, but the costs are astronomical, and the benefits perhaps even more uncertain.

Like many proposed solutions to the current health care problem, no price is too high for those who don't intend to pay it. (As if "corporations" or governments actually eat such costs.)

There may be consensus science that is the province of scientists, but in policy, everybody gets a vote, including those who stand to pay for it.

Tlaloc said...

""Consensus science" does not refer to the perfectly organic, Kuhnian process of scientific progress through hypothesis testing, replication, and peer review. While this process often does produce what might be termed 'consensus' on a body of theory, this is not what the term means"

Well, when talking about Global Warming, it is in fact exactly what the term means. There is a global consensus among climatologists about climate change. Any refernce to "consensus science" about global warming MUST take into account the very real consensus on the issue.