Priming Your Way to Good Mental Health

The September issue (September 2011, v.6, no.5) of Perspectives on Psychological Science contains a series of essays on new models of delivery for fundamental mental health services.  While we’ve got a pretty good science for clinical psychology, getting it out to the Other Guys is expensive and difficult.  The essays all address large scale public delivery of effective clinical treatments.  That means anything other than the proven method of face-to-face interaction with a therapist in controlled settings with either individual or group meetings.  One proposed large scale delivery intervention on priming raises a couple of interesting persuasion points.

Shalev and Bargh suggest that priming could be a cheap, easy, and effective clinical modality that could apply in a wide variety of specific cases.  They outline a series of priming examples that have been proven in the lab to produce positive emotional and cognitive changes in people that are at least as effective as many individualized clinical treatments.  In terms of faster, cheaper, better, Shalev and Bargh make the case for public mental health priming.

While Shalev and Bargh don’t cite this, the subliminal work from Lloyd Silverman provides compelling evidence of the clinical effectiveness for priming.  You can read more about it in the Primer chapter on Subliminals, but very briefly, Silverman conducted well controlled studies with seriously ill people using “Mommy and I are one” as a key subliminal message.  It worked.  As a 1990 meta analysis from Hardaway in Psychological Bulletin reported, the average effect for this treatment is a Smallish Windowpane, about a 40/60 effect.  So even past Shalev and Bargh, there’s some good evidence that priming does work.


Consider two persuasion problems.

First, think about Box and Play.  The Priming Play is proven to work in the Box called Experimental Lab Settings.  I don’t know of any large scale field testing of priming in natural settings.  We’ve looked at this with the Queen of Tomorrow and subliminal applications in marketing, sales, and, cue the creepy music, Big Brother politics.  While we are getting closer to priming in public, the technical demands of delivering effective subliminals or primes defeat practical application.  You’ve got to control the visual or auditory field of the Other Guy who is moving through life.  That’s extremely hard to do.  So, the jump from the Box called the Lab to the Box called Life is a major persuasion problem.

And, even the previously noted Silverman work with “Mommy and I are one” interventions took place under controlled conditions and not as part of a field study.  And often times, the participants in the intervention were specifically misled about the subliminal training and given a cover story to explain the computer apparatus and the procedures with it.  People knew they were in a “study,” but they were clueless about the specific priming intervention aimed at their mental health in most cases.

To summarize the first persuasion point, the good evidence in support of priming for mental health is restricted to a particular and unique Box and Play.  We don’t know whether it generalizes to other Boxes without that obvious experimenter/therapist presence and control.

The second problem is awareness.  Even past the impact that informed consent would have on this application, all the research I’ve seen on priming and subliminals indicates that if you have conscious awareness that the Prime is out there then the impact of that Prime is strongly reduced.  Thus, in the standard kind of public mental health intervention, you’d have to tell people about priming and subliminals (informed consent), but then you’d have to make sure you deliver the intervention without their awareness that you are doing it.

Think of this through the movie, The Truman Show.  A baby named Truman is raised in an environment that looks like normal life to Truman, but is actually a Hollywood sound stage.  Truman’s life is just a reality TV show.  As long as Truman doesn’t realize he’s in this controlled environment, he thinks he’s the Captain of His Ship, the Master of His Fate when he living in a When-Do-Get machine controlled by the TV director.

The Priming Box and Play proposed here for public mental health in essence puts Other Guys in The Truman Show.  They sign off with informed consent for the un- or sub-conscious primes that will deliver better mental health.  So, they know they are in the movie.  Will the priming still work under these conditions of awareness?  Just think of the attributional impact of this and how it could mess with your head.

You’re feeling lousy and your psychologist suggests that priming could help you.  You sign up, informed consent and all, then wait.  Some days you feel better than usual.  How do you attribute it?  Is it the priming or something else?  How will you know not only that you feel better, but that you feel better because of those unconscious primes because you cannot ever know when the prime is On?  Yet, if anyone would propose doing this to you without your knowledge, we’d have serious legal and ethical issues.

To summarize the second persuasion problem, awareness of priming tends to reduce the effectiveness of the priming.  In experiments we can design cover stories that deflect awareness, then do a debriefing.  In a large scale field application – priming on the Web – people would have to initiate the intervention on their own accord, which means they would have some kind of awareness with all those attendant effectiveness and attributional problems.

If ever there was a case for More Research, this is it.  Can you deliver clinical priming through a website and still get the effects from the Lab Box?  And, how do you handle the informed consent, awareness, and attributional problems inherent in this situation?

Finally and just to raise the creepy specter of Big Brother again, imagine that therapeutic priming on the web can be done efficiently and effectively.  Assume that More Research answers my two persuasion problems and we’ve got Dr. Feelgood online all the time.  That means we’ve found a socially acceptable way to change the way Other Guys think, feel, and act in practical, clinically significant, not just statistically significant, ways . . . without the Other Guys knowing this happened or how.

The history of the 20th century documents that many smart and well meaning people thought that government should be used to shape the New Man, the New Soviet, the New Fascist, the New Nazi all in the service of a better world, the perfection of humanity as the inevitable product of evolution’s assumed march to excellence, the Fittest of the Fit.  If we’ve got Feelgood Online why not bring it into government to solve intractable social problems like ignorance, poverty, disease, and crime?  The rest is just the persuasion gravity of falling off the log.

Persuasion affords such imaginings.  Perhaps we can.  Persuasion theory and research points the way.

Perhaps we should?  Persuasion says nothing about that.

The Lowball Box

This WSJ article nicely details the Persuasion Box you need to work the Persuasion Play called the Lowball. The Lowball is just what it sounds like: A bad pitch out of the strike zone that a batter nonetheless chases. In economic terms this metaphor translates into a potential buyer making a low offer that a seller willingly takes. The trick to the Lowball is less about the actual Pitch (how low and away) and more about finding the right Box for throwing it.

The WSJ article shows how to throw the Lowball in real estate deals. It identifies 6 elements for the right Box.

1. Understand the market
2. Pick the right real-estate agent
3. Back up your price
4. Know what you’re willing to pay
5. Make a clean, easy offer
6. Be smart about a cash deal

Without understanding the details from these headline descriptors you can see the key elements in the Box (also understood as the Local). Market conditions define the range of prices available for comparison. The right agent is someone who understands persuasion. Backing up your price means providing reasonable arguments for the Lowball. Your limit means knowing when to walk away. A clean and easy offer is obvious and that cash deal thing means don’t think an all cash offer is sufficient for a good Lowball.

Now, if you’re thinking, you realize the Lowball Box and Play requires both a buyer and a seller and all of these elements operate with both parties. The article aims at talking to buyers, but if you’re a seller in a bad real estate market, realize this information helps you, too. See that many buyers think they’ve got you over a barrel and will try the Lowball. If you understand how they will be coming at you with those 6 Box elements, you know how to turn it around in your favor during a negotiation. Consider this.

A Lowball offer appears to favor the buyer, but that offer can also work as the unintended First Step in a Two Step Persuasion Cue called Sequential Requests. That buyer is making a move towards buying which is also known as making a Choice and sometimes as making a Commitment. Sure, it’s a Lowball, but if you’re a smart seller with persuasion skill, you’ve now got a buyer on the hook. Of course that buyer wants the Lowball, but if you persuade properly you can move the buyer into a position more favorable to you. Maybe the buyer included an All Cash Deal in the offer (element 6 in the article) thinking it would be decisive. You accept the Lowball All Cash offer and title to the car the buyer drove up in or offer to include your car in the deal (at a higher price, of course).

People tend to think of persuasion as a one way transaction, as something they are doing to the Other Guy. Realize that Other Guys are your persuasion receiver, but at the same time They may also see Themselves as a persuasion source with you as their target. If you can’t see beyond your own persuasion effort, you make yourself even more vulnerable to the plays the Other Guys spring on you. Remember the Rules.

Persuasion Is Strategic Or It Is Not.

If You Can’t Succeed, Don’t Try.

Science in Doom or Monkey Wins Nobel!

Shall I be plain, I want my science pure!

At least when Blogging wherein I will defend Science as the one and true faith against all pretenders, contenders, and doubters.  But what I can do with this?

People who play computer games solved a difficult science problem (pdf) using something pretty much like a Doom multiplayer game!  Sure, I simplify, but not overly.  If this breakthrough leads to a major discovery, will the winning shooter squad share the Nobel?  Quel dommage!

Jeepers if slackers are solving . . .

. . . real-world modeling problems with more complex deviations from native structures, leading to the solution of a long-standing protein crystal structure problem.

. . . imagine what they can do with persuasion . . . tipping points . . . Nudge . . . Health Beliefs Model . . . Message Framing . . . Graphing Towards Truth?

P.S. Isn’t the Infinite Monkey Theorem just another variation on Nietzsche’s Eternal Recurrence? Whether creating Protein Fold Solutions or Shakespeare or Souls, it’s only Forever Random.


on the Yellow Brick Road of Truth with Graphical Statistics

What do you do with this?

It’s long been my pet opinion that the ability to quickly produce and disseminate simple graphs is one of the huge gains of the digital media era. And now from Brendan Nyhan & Jason Reifler comes some empirical research to confirm my pre-existing biases.

Matthew Yglesias, public intellectual writing at the Atlantic, then extensively quotes the easy to read text from a poly sci paper that does indeed demonstrate how people follow graphic presentations of data to conclusions.  Yglesias trumpets.

In other words, if someone’s wrong about something, it’s easier to persuade them to change their mind with a graphical presentation than a textual one.

The paper uses perceptions of US casualities in Iraq following the Surge, Global Warming, and Obama Jobs Creation as the key issues for three quasi-experimental studies that manipulate either text or graphic presentation of data (along with an affirmation manipulation). Yglesia believes there is a 2+2=4 kind of truth to be found on these political issues which is both sweet and sincere. And, if you are wrong on War, Environment, or Jobs, Ygelsias knows how to straighten you out. Graphs!

Persuasion mavens and regular readers of the Persuasion Blog spot the raccoon here.  Yeah, pretty pictures for persuading Low WATTers kicking Cues down the Peripheral Route.  And, Yglesias believes that this is a triumph for the digital media age!?!  As I’ve documented numerous times here, pictures ain’t no place for thoughtful consideration of well constructed arguments.  The Digital Age is the great magic trick that fools even magna cum laude graduates of Harvard into accepting the weaker for the stronger.

Might wait to see if this conference paper survives peer review for publication.  Might actually read the paper (pdf) and realize the results are weak, inconsistent, and contradictory to theory and the literature.  Might just peruse the paper’s graphs (although Never Make Decisions Based On Graphical Data! with appropriate head nods to Dr. Tufte.).  Might, sigh, read the probit results tables.

But if we close our eyes and click our ruby slippers, we’ll find ourselves slouching back to Cambridge and the Truth.

Function not Content with Beauty (and All Persuasion)

Note from Steve: I based this post on a peer review research study that is retracted as fraudulent (July 2012). You can read more about the investigation into Diederik Stapel for details.

The general form of the post is still reasonable since it describes basic ELM theory. I used the study to illustrate theory principles and the bad research broke no new ground. It simply extended what we already knew from past work. ELM or any other dual process theory is not disproven as a result of this fraud. In fact it appears that Stapel was able to perpetrate his unethical behavior because his work generally confirmed well established theory and literature and merely moved it into domains that were not tested.

I am also reluctant to delete the post because that strikes me as a confusing action that looks both self-serving and totalitarian like the infamous examples from Stalin’s Soviet Union where old photos would be gathered up and old comrades now disgraced in show trials would be erased. Stapel’s work fooled me enough to use it as an example. And, the fundamental point of the post is still valid even if I was dumb enough to fall for a fake example.

Thus, the main point of this post is valid: the “same” persuasion variable, in this case a Hot Blonde like Marilyn Monroe, can have multiple functions in persuasion, once a Cue, then an Argument, and then a WATTage switch. You know a persuasion variable not for its appearance, but for its function.

Which explains in part how Stapel was able to run a fraud for awhile. People saw the appearance and missed the function.

Now, back to the Opera!

You can separate the mavens from the muggles on the difference between function and content.  Muggles expect persuasion cookbooks, Magic Words, and any ritual, incantation, or wink ‘n nod they can find which means they look at the content of persuasion or how persuasion appears.  Mavens, by contrast, know it’s all in the function, the how and not the what, the doing and not the deed, the verb, not the noun.  Function for mavens, content for muggles.  My favorite illustration of function is with Marilyn Monroe.  Consider these three photos.  First, MM and a ‘Vette.

Second, MM and shampoo.

Last, MM and Joltin’ Joe.

All and always that beautiful woman, but each and every one different.  Same content, but different function.  You’d think the Original Blonde Bombshell would always and only be that Beautiful Cue, just a Ding-Dong to sell any persuasion product, proposition, or proposal, but you’d be wrong.

And, this great series of lab experiments from the Journal of Consumer Research proves it.  Janne van Doorn and Diederik Stapel take the functional nature of Beauty and run it through the wringer to demonstrate that the same content can produce radically different persuasion outcomes depending upon function.  I will not detail the four experiments but simply describe.

In each study van Doorn and Stapel present a print ad with a Beauty and Something Else.  And that Something Else shapes the Beauty to sell as an Argument or a Cue or a Prime (they don’t employ Beauty as an Elaboration Moderator or WATTage switch).  If you want to see a great illustration of proving the function versus content persuasion revelation, check out this paper.

Janne van Doorn & Diederik Stapel.  When and how beauty sells:  Priming, conditioning, and persuasion processes.  Journal of Consumer Research.  Published electronically, June 6, 2011. 

DOI: 10.1086/660700.

Note from Steve: I based this post on a peer review research study that is retracted as fraudulent (July 2012). You can read more about the investigation into Diederik Stapel for details.

The general form of the post is still reasonable since it describes basic ELM theory. I used the study to illustrate theory principles and the bad research broke no new ground. It simply extended what we already knew from past work. ELM or any other dual process theory is not disproven as a result of this fraud. In fact it appears that Stapel was able to perpetrate his unethical behavior because his work generally confirmed well established theory and literature and merely moved it into domains that were not tested.

I am also reluctant to delete the post because that strikes me as a confusing action that looks both self-serving and totalitarian like the infamous examples from Stalin’s Soviet Union where old photos would be gathered up and old comrades now disgraced in show trials would be erased. Stapel’s work fooled me enough to use it as an example. And, the fundamental point of the post is still valid even if I was dumb enough to fall for a fake example.

Thus, the main point of this post is valid: the “same” persuasion variable, in this case a Hot Blonde like Marilyn Monroe, can have multiple functions in persuasion, once a Cue, then an Argument, and then a WATTage switch. You know a persuasion variable not for its appearance, but for its function.

Which explains in part how Stapel was able to run a fraud for awhile. People saw the appearance and missed the function.

Succeeding By Failing

Consider this persuasion science.

“Great by Choice” is a sequel to Jim Collins’s best-selling “Good to Great” (2001), which identified seven characteristics that enabled companies to become truly great over an extended period of time. Never mind that one of the 11 featured companies is now bankrupt (Circuit City) and another is in government receivership (Fannie Mae). Mr. Collins has a knack for analysis that business readers find compelling.

What? Two of eleven good-to-great companies are bankrupt in less than ten years? That’s great? And how good can your science be when it declares 11 marvels that then quickly reduces to 9? And, if you’ve read the Halo Effect by Phil Rosenzweig, you know the problem is worse than that. Rosenzweig employed the Collins science both before and after the time period Collins studied to find good-to-great companies and found different companies making the list at different time periods. In other words, these great companies appear to be great for just a moment in time. A better title would be Not To Hot To Not Again. So much for the business science of Jim Collins.

Yet, the newest Collins book is greeted with acclamation. How can you be wrong with your science, and provably so, yet remain acclaimed?


All Bad Science Is Persuasive.

If You Can’t Count It, You Can Change It.

You Can’t Persuade A Falling Apple.

Irrelevant Persuasion

Recall the fall of 2008.  An economy and stock market in free fall.  Two long, hot wars.  A reviled President leaving the White House.  A tired old Republican asking for the job.

If we can’t win in this environment, we have to question the whole premise of the party.  James Carville, 2008

This from one of the best Democrat political consultants of his generation.  Carville’s worry in front of that election proved superfluous.  Mr. Obama won easily.  But, Carville’s point is worth translating.

Persuasion didn’t matter in 2008.

Sure, you’ve got freely choosing Other Guys.  Sure, there is controversy, ambiguity, possibility.  But, in 2008, if any Democrat couldn’t win, the Democratic Party was deader than the Bull Moose Party, and no persuasion could have or should have mattered.

And, it’s entirely possible that you will be reading a quote from a smart and experienced Republican political consultant making Carville’s assertion, but for the GOP candidate in 2012; Obama is in that much trouble right now.

Sometimes persuasion doesn’t matter even in a persuasion situation.

Ding-Dong Still Works, but Just Not Practically

Ding-Dong is a persuasion fact of life that even common sense can outline. Repeatedly pair two things in time and they form a bond in memory. And, better than mere memory, this association can also bring affect and attitude into the equation. We can learn not only Ding-Dong, but also hate, fear, and disgust. If you read the Primer chapter on Classical Conditioning you can see the earliest examples of Ding-Donging for Persuasion. It works. Consider a 2011 example.

Participants were asked to watch a slideshow that featured 100 images (five snack images shown 20 times in a random order). Each of the 100 trials lasted 2.5 s in total, with the snack image appearing for one second, followed by the presentation of a either a blank screen (control condition) or the presentation of one of five aversive bodily images (intervention condition) for one second. To ensure that participants concentrated fully on the images, they were instructed to respond to the brief random appearance of a white circle at five intertrial points throughout the slideshow, by pressing a given key. The intertrial interval was 500 ms.

Here’s an example of a treatment trial.



Pair the delicious chocolate with the somewhat less attractive rotund belly. This is called Evaluative Conditioning, which seems to be Classical Conditioning with an attitude. The authors argue that participants in the Treatment condition will develop negative attitudes about chocolate and further will show immediate behavior change in the form of preferring fruit to candy when given a choice after their participation.

Now, we already know from the extensive peer review literature on Classical Conditioning that such training will produce changes in attitude and behavior. Again, just check the Primer (look for Razran) or hit Google Scholar or a good scientific search engine to confirm this.

To this we add the attitude measurement task. The researchers will employ two types, explicit and implicit. Explicit attitudes are ones that are measured directly on self report: You know you’ve got it and you report it. Here’s the measure.

Explicit attitudes toward fruits and snacks were assessed separately (“For me, eating fruit/snacks is”) along five 7-point semantic differential scales (not at all healthy–healthy; bad–not at all bad; not at all enjoyable–enjoyable; not at all unpleasant–unpleasant; good–not at all good) and reverse scored where appropriate. Composite scales (α = .70 and .69, respectively) were produced, and in line with previous research that uses a fruit–snack comparison (Perugini, 2005), summed explicit attitude scores for snacks were subtracted from those for fruits to give an overall explicit attitude score, with positive scores indicating relative preference for fruit.

Implicit attitudes are attitude that are measured indirectly through an incompatible action task. This is trickier and you need to read the following description carefully to get it.

Implicit attitudes were assessed using the IAT (Greenwald et al., 1998). The critical tasks of the IAT require participants to sort words that relate to one of four categories to one of two response keys. The assumption is that responses to categories that are highly associated in memory are assumed to be faster when they share the same response key, relative to the response to categories not associated in memory. The four categories comprised two targets (fruits, with the exemplars fruits, apple, banana, grapes, orange; and snacks, with the exemplars, chocolate, biscuits, cake, snacks, crisps) and two attributes (pleasant, with the exemplars, rainbow, happy, smile, joy, peace; and unpleasant, with the exemplars, pain, death, poison, agony, sickness). In the critical tasks, stimuli from all four categories were presented in a random order, and participants assigned them to one of two combined category-attribute pairs (e.g., left computer key for fruits–pleasant and right key for snacks–unpleasant). A second combined task switched the targets (e.g., left computer key for snacks–pleasant and right key for fruits–unpleasant). The category headings were displayed throughout in the left- and right-hand corners of the screen. Errors in sorting were pointed out, with an accurate response being necessary to move on to the next presentation. Participants were required to respond by pressing the E or I keys. The order of the block assignment within the IAT was counterbalanced, although no order effects were observed.

Do you see the trick here. The classification task (pressing the E or I key) is sometimes arranged so that your “true” attitude is easy to express with a click of the E key, but then on other trials, you must press the I key. Speed differences in key pressing express attitude differences.

So, what happens here?

In the behavioral food choice task, participants in the intervention condition chose fruit (as opposed to snacks) more often than those in the control condition (respective means of 1.33 vs. 0.17 on the food choice index, indicating the number of choices of fruit made, see Table 2). This effect was significant, F(1, 128) = 25.20, p < .001, r = .40, supporting Hypothesis 1.

Hubba-Hubba. We get the predictable behavior change. And, it is driven by change in attitude. But not explicit attitudes. That standard self report shows no difference in attitude towards the food between Treatment (M = 11.94) and Control (M = 11.30), t < 1.0, d effect = .125. But for the implicit attitude, the differences are there.

In relation to Hypothesis 2a, after controlling for baseline IAT scores (plus age and gender), intervention condition marginally predicted post-intervention IAT scores for the complete sample, F(1, 127) = 3.62, p = .06, r = .17 (although this effect was statistically significant, p = .033, if we did not control for age and gender).

I’m not sure why they adjusted from age and gender and as they parenthetically report, the effect is significant at p < .033 without the adjustments. I can think of no good theoretical reason for this and suspect that this is due as much to the review process as anything else.

Thus, we get a Medium Plus Windowpane effect of 30/70 for the behavior choice task (pick fruit or snack) and a Small Plus Windowpane, 41/59, for implicit attitude.

So, in 2011 we confirm again the basic operation of the Ding-Dong in the application of getting people to hate good food, all those tasty snacks like chocolate, cupcakes, éclairs, cherry pie, and on and on in an ecstasy of eating. Of course, left unsaid and untested in this study is whether the attitudes, implicit or explicit will persist over time. My best guess is all the participants took the Peripheral Route here and their attitude change was decaying rapidly after they left the lab. If someone had instead approached them 100 feet later, say down a public hallway outside of the lab with the press of pedestrian traffic around them, with a choice of fruit or chocolate, the behavioral results would have been very different.

And, this is the practical challenge for the external validity of this study. Sure, Classical Conditioning changes people. Read the Primer or just think about all that brand building you see in corporate advertising. But, if you want people to avoid high calorie, high fat, and, more importantly, high impulse food, how are you going to deliver this Treatment out in the wild? Hey, how about plastering posters like this all around their workplace.


Folks, that’s called Warning Labels and we know those don’t work despite the frequent and noisy parades from the Lifestyle Drum and Bugle Corps. The challenging part of this paper is not the mere repetition of Classical Conditioning, but in rather trying find ways to make it work on the Other Guys. Some readers will be tempted to view this as support for Warning Labels, but think again. The researchers did not put these pictures on candy wrappers, as the fat equivalent of tobacco products. They merely ran a Ding-Dong and replicated what we’ve known since Pavlov.

Hollands, G. J., Prestwich, A., & Marteau, T. M. (2011). Using aversive images to enhance healthy food choices and implicit attitudes: An experimental test of evaluative conditioning. Health Psychology, 30(2), 195-203.


Measuring Yelp’s Impact

The social web captures the wisdom of the crowds and provides aggregate information about products and services, events and ideas for anyone with a web addressed device. Yelp is a great example with millions of sourced reviews on thousands of restaurants over many years. Go to Yelp, type in an eatery, and odds are great you’ll find several detailed reviews on it for your attitudinal consideration. Man, the mind reels at the persuasion possibilities. It must make a difference for those restaurants, right?

According to this unpublished econometrics paper as noted at the Wall Street Journal, Yelp makes a difference to the bottom line. Each star a restaurant gains on a 5 star scale is worth . . . what do you think? On a percent basis how much more money does a restaurant with 5 stars earn compared to a restaurant with 4 stars? Or 4 versus 3?

100%? 200%? 20%? 50%?

Michael Luca crunches the numbers for Seattle restaurants from 2003 to 2009 and finds, drum roll, an average gain of 7% (plus or minus 2%) in annual revenue. Thus, if 4 star restaurants earn $1,000,000 a year, 5 star restaurants earn $1,070,000. Expressed in Windowpane terms, a Small Windowpane would be 10% so this 7% difference is 30% smaller than a Small Effect. That’s pretty small and pretty close to a zero effect, but because Luca has a huge sample of observations, even something this small is statistically significant and thus reportable in a paper.

Realize too that this is another example from the Observational Tooth Fairy. We have no control over any variable in this mess of life study. First, people self select into writing the Yelp reviews and then haphazardly provide reviews, sometimes providing a review after a meal other times not. Second, as Luca notes, 69% of Seattle eateries are reviewed on Yelp which means that 31% are not. Just think about all the potential sources of bias in these two facts. Couldn’t they account for that very small effect on revenue?

Please note that I am not saying Luca did anything wrong in his analysis. It appears to be a competent example of this kind of observational research this time with a focus on money rather than health. What I am saying is that the effect size is so piddling and that Rival Explanations are so obvious, that these data provide no comfort to that Wisdom of the Crowds social media claim. Think about it.

In persuasion theory, Yelp is best understood from a dual process perspective like the Elaboration Likelihood Model. Reviews can provide both Arguments and Cues to readers about their various choices. The Wisdom of the Crowd concept suggests that regardless of whether you prefer an Argument (with the Long Conversation in the Head) or a Cue (following the shiny spangles and dazzles), as long as it is a Strong Argument or a Positive Cue, it’s all good and those star ratings tell you which places are Better, whether on the Central or Peripheral Route. Those stars are killer persuasion tools.

Yet, realize the behavioral impact of those stars. Stated another way, the TACT here is revenue for the restaurant, and in this study, the revenue impact of those stars is almost zero. If social media like Yelp were actually collecting and providing true Wisdom from the Crowd, wouldn’t you expect a much larger impact?

The failure here stems from several independent and interacting factors; consider just two. First, reviewers are wildly inconsistent in when they chose to review and how they review. Thus, Yelp is collecting a lot of information that is biased in many different directions. Combining all that bias into a huge dataset does not make the data consistent, smooth, systematic; it’s just a humongous biased data source. Second, readers are wildly inconsistent in when they go to Yelp and what they select to read on Yelp and what they weigh as important. Thus, even if Yelp contained extremely reliable and valid data on restaurants, our own variable human nature, or just plain cussedness, would cause us to behave in wildly inconsistent ways. Now, combine those two factors and the persuasive impact of Yelp on anything – making a reservation, spending more or less, writing your own following Yelp review – is simply the mess of life. All data, no wisdom.

The web has become a giant data machine that generates more bytes of information than anything in history of human civilization. Many people have a naïve faith in both data and size as if these stupendous databases contain the wisdom of the current ages, when all they contain is digital dust.

All Bad Science Is Persuasive with The Lancet and 9-11 Cancer

Hitting the headlines in major media sources during the 10th anniversary of the 9-11 attacks is a scientific report published in the scientific journal, The Lancet, which asserts a link between New York City fire fighters exposed at the WTC disaster and higher cancer rates.  The headline number you are reading is 19% greater risk of all cancers among exposed fire fighters.  This, of course, leads to calls for greater funding for fire fighter health programs.  I’d like to raise two observations.

First, if you read about this report in major media sources like broadcast TV news, large city newspapers, and even major cable outlets, you get that 19% Greater Cancer Risk Headline and calls for More Funding almost uniformly.  Here’s the WSJ.

Firefighters who worked at Ground Zero are 19% more likely to have cancer than their colleagues who did not work at the site, according to newly published research that could pave the way for government payments to those suffering from some types of cancer.  The research marks the first substantive findings on the difficult question of whether working at the World Trade Center site increased cancer risk.

Now, consider the actions taken by my former lead agency, the National Institute for Occupational Safety and Health who looked at the same data.  (Indeed, they are the funding source for this research.)

Federal officials had announced Tuesday that those with cancer will continue to be excluded from federal help for people with illnesses linked to the Sept. 11 attacks. There was too little scientific evidence linking cancer to time spent amid the dust and wreckage, a review by the National Institute for Occupational Safety and Health determined.

What gives?  How can intelligent people come to different conclusions about the same scientific data?  I’d argue the answer to that question is, Persuasion.  To see this you need to consider my second observation.

If you actually read the results and ignore the study author’s rhetoric, you realize that only one of the statistical tests on any cancers is even remotely close to the lowest standard of science, statistical significance, expressed here at 95% Confidence Intervals on various ratios.  I’ll reproduce Table 3 which provides all the raw data for all the cancers.

First, realize there are no tests of statistical significance reported anywhere in this Table.  None.  Instead the authors hide that absence behind 95% confidence intervals and ratios of those confidence intervals.  This is no substitute for testing statistical significance and is indeed an Artful Dodger.  Why reviewers and editors at The Lancet accepted this, I cannot explain; however, you will recall that The Lancet published that infamous vaccine study by the even more infamous Dr. Wakefield on childhood autism.

Even if you are foolish enough to accept this as any kind of science, then let’s look more carefully at your foolishness.  You’ll note that each category of cancer is expressed in three lines, Exposed, Non-Exposed, and SIR ratio.  The key line to check in each case is the third line, the SIR ratio.  We’ve already looked at the joys of forming ratios of ratios in a prior post and we see that magic trick again here.  You actually have to read and think about numbers to see the fantasy here.

Excluding the entry for All Sites (uncorrected) and for Thyroid Cancers, as you look at the SIR ratio line you will observe one constant outcome.  All of these made up ratios always include 1.00 in the range.  That proves all cancer comparisons between exposed and non-exposed fire fighters are within the 95% confidence limit; they are not different from each other even though there are magnitude differences.  Those differences are within the expectation of random variation.

In essence we are throwing a fair red die for the exposed group and a fair green die for the nonexposed group 10 to 40 times and then comparing the frequencies of 1, 2, 3, 4, 5, and 6.  And while there are differences between the red die and the green die for the number of 1′s or 4′s that came up in our 10 to 40 throws, the difference is within the random variation of throwing fair dice a limited number of times.

You also need to realize that there is an enormous amount of adjusting and moderating and transforming with the data.  The researchers add or subtract quantities in cancer rates to adjust the data for race or ethnicity.  And, of course, realize that this is observational research.  No random assignment, no controlled conditions, no hard headed comparison, and a lot of dipsy doodle counting.  This is not science, but just another example of bad science hidden in persuasion.

I was hired into the Federal government in part to work on a national, Congressionally-mandated, safety campaign for first responders.  The leading cause of death among fire fighters is death from structural collapse, the very event that killed nearly 350 of my clients, customers, stakeholders, and colleagues on 9-11.  I watched the collapse of the WTC buildings in awestruck horror, worrying about whether FDNY was following the recommendations from that Congressionally mandated safety campaign.  I then watched the recovery and cleanup at Ground Zero with another kind of horror, observing the fire fighters ignore basic safety rules about exposure to airborne hazards, refusing to wear safety masks and respirators despite the warnings from all the safety experts.  I know what the disaster was and the personal impact of it.  I have nothing but love and respect for everyone who faced it and did the best they could under those terrible circumstances.

But, pretending to do science in a transparent attempt to get more money is not how I would choose to help.  The study authors and The Lancet have clearly sold their scientific credibility for empathy and emotion.  I strongly support my former agency, NIOSH, and their commitment to basic science.  They have reviewed the same evidence this study reports in The Lancet and have rejected it as proof of a link between WTC exposure and cancer.  That’s a tough public step, but it distinguishes the difference between science and persuasion.