The scientific devil is always in the details and those details typically involve math and simple, but hard, thinking. To illustrate these perils, consider this expert report that recommends universal cholesterol screening for children 9 to 11 (page S27 in this pdf). You need to read the report carefully to see the scientific science despite the fact that virtually everyone involved in the report would call themselves scientists. When you see the bad science, you then can begin to see how persuasion principles explain the outcome better than math and simple, but hard, thinking.
First, read the damn paper, most particularly Section 9 on Cholesterol. Think carefully and skeptically about it. In other words, think like a scientist and not a member of a high prestige group. Meanwhile consider this quotation from an expert not on this particular expert panel.
Rita Redberg, a cardiologist at University of California, San Francisco, expressed skepticism. “I don’t know of any data that screening children ages 9 to 11 is of any benefit to them,” she said. “We don’t need to do cholesterol tests to advise children to eat fruits and vegetables, watch their weight and get regular physical activity.”
Or this from an expert from a different expert panel on the same topic of cholesterol screening.
. . . the U.S. Preventive Services Task Force, concluded in 2007 that not enough is known about the possible benefits and risks to recommend for or against cholesterol screening for children and teens. One of its leaders, Dr. Michael LeFevre, a family medicine specialist at the University of Missouri, said that for the task force to declare screening beneficial there must be evidence that treatment improves health, such as preventing heart attacks, rather than just nudging down a number — the cholesterol score.
Maybe Redberg and LeFevre are idiots who don’t read the relevant research literature. They are in that terrible position of claiming there is no needle in the haystack. Asserting Nothing against a group that asserts Something is a losing persuasion proposition. Of course, it is easy to prove Redberg or LeFevre the idiot. Just produce the evidence of science. You have to do that heavy lifting for yourself.
Still, those two quotations permit a shortcut if you don’t want to read the report and figure it out for yourself. Some experts claim there is no good scientific evidence to support the recommendation for universal screening with 9 to 11 year olds. Yet, this NIH panel of experts made such a recommendation. How is this possible?
I think persuasion answers the question better than science.
Realize the social context of the panel. Appointment to the panel is a huge status hit; you have to be well connected to get there and you are a big deal to earn this merit badge. The panel has no power; recommend at your pleasure and nothing will change as a result, a case of the politics of the unelectable or actions without consequences. There are no agreed upon standards of judgment; bring your expertise to the Table of Brother and Sister Hood. Majority voting applies, so coalitions are more important than, say, scientific evidence.
If you read section 9 on cholesterol screening you see an incredible amount of detail, expert quotation or citation, elaborate decision trees, and tables and tables of topics, criteria, and evidence summaries. The report looks serious, scientific, and objective when it is only the majority report of a faction with self interest, exactly what the Federalists warned about in their Papers. It is the text version of a high color figure of a brain scan: Lots of misleading detail that hides more than it reveals.
Now, make the contrast between this process and the one involved in the US Task Force on Preventative Health for prostate cancer. You might recall that another group of health experts reviewed the scientific literature on prostate screening and recommended against it. Other experts howled. Why the difference?
Realize that the US Task Force operates more like a jury with a predefined set of rules and procedures. They operate under the same standards for all issues. Now, compare this to the operation of this NIH panel on screening kids for cholesterol and the howling prostate cancer experts. The other experts employ something quite different from that jury metaphor. They review and evaluate different kinds of evidence with different kinds of standards, then make a public declaration that has absolutely no legal or practical impact.
You see the same kind of outcomes in the peer review literature. Even if you are not that kind of researcher or scientist, you’ve read enough examples in the Persuasion Blog to know that peer review does not ensure any kind of Truth much less consistency. Standards of judgment vary with the reviewers, the writing style, the trendiness of the topic.
Thus, you are always still dealing with Experts, but how they decide is very different. A process like the US Task Force on Preventative Services operates under a very different kind of WATTage compared to the other Expert groups whether in panels, committees, associations, or just individuals trying to get published in peer review. Everyone always has some kind of Bias operating on them. That’s why external rules, procedures, and standards are so useful and effective. The problem isn’t with the science – it’s with the scientists.
And, yeah, just to stay consistent . . . see how this inflates the Bubble.