Friday, September 30, 2016

To Retract or Not to Retract, That is the Question. Whether Tis Nobler...

Sometimes as we strive to understand the world in which we live, we are handed a moment of clarity. We need to accept these gifts and learn all we can. We had two major retractions in wildly different fields recently - the power pose renunciation by study co-author Dana Carney and the birther retraction by candidate Donald Trump. But first, the power pose.

Amy Cuddy is a social scientist who has found semi-renown with a current New York Times best seller Presence. In 2010 she and her colleagues published results of a study on the effectiveness of "power poses" in building confidence, as measured by levels of cortisol and testosterone. She famously advocated "fake it till you make it" and now it appears that is what she may have done, or at least the statistical techniques were applied sloppily.

Cuddy's co-author of the study, Dana Carney, recently renounced the findings of the original study, which researchers have been unable to replicate. That renunciation provides a moment of clarity because the arguments about the validity of the results can stop, we can all agree there was a problem, and we can turn our attention to understanding better what went wrong and how do we fix it.
Cuddy's Ted Talk available on Youtube is entertaining.
Dana Carney, who went from Columbia to Berkeley after the paper was published, has a special comment on the power pose study which is a clear renunciation.
Retraction Watch has a post on the study. Carney says the paper should not be retracted. We will likely see more activity in the weeks ahead.

When I originally watched Cuddy perform, I noticed that her ways of thinking about herself and the world often fundamentally differ from mine. She concludes that doctors who have a warm social interaction with their patients are sued less often, regardless of their competence. (I am not familiar with that study or how competence was measured). I lean toward the belief that doctors (and really other professionals) who are the most competent are the most confident -they have every reason to be - and therefore the least defensive, so they are able to engage their patients more directly.

So she looks for mechanisms that do not offer intuitive appeal to me. For example, she posits ways to build confidence for job interviews and chooses the example of Usain Bolt thrusting his arms in the air after the race as a natural "power pose". But that is backwards! Usain throws his arms up after he wins, after the validation of winning, not before the race. In the past, before a client meeting - or an interview - I would don the suit and tie and stare at myself in the mirror for 5 seconds taking myself in as a stranger would. It was almost a way of saying to myself - see, if I saw that person on the other side of the table, I would assume they are capable and belong there. I thought of my little ritual as getting psyched for game day. Frankly I never saw this kind of self-help as requiring scientific study. Isn't this just the "ruby slippers" effect in the Wizard of Oz? The cowardly lion had reason to feel brave because he had acted bravely. The scarecrow made wise recommendations, so he really did have a brain. And the Wizard really was a wizard because he helped the others understand they had every reason to feel confident. Cuddy leaves open the question of how a person lacking in competence can feel confident in areas beyond their expertise and whether that is a good thing or a bad thing.

The power pose results failed the common sense test (for many, but not all of us). The primary goal of at least one of the authors seemed to be to help women feel more confident and to be able to validate the recommendations by pointing to "science proves it". Almost as if to say - "See you can really feel more confident if you use this technique, and you can feel confident in this recommendation because science backs me up on this.

Dana Carney's rejection of the findings is big news. Many statistical studies have failed to replicate in recent years, especially in social psychology. Sometimes a study can not be replicated because the data is kept confidential (not good), but the findings fly in the face of common sense. For example, I thought I understood the most important mechanisms for unruly passengers -- access to overhead storage, access to under-the-seat storage in front of you, seat tilting and other confined quarters conflicts, timely access to a clean restroom, emotions sometimes fueled by alcohol, which happens to be offered free in first class, usually at the start of the flight. But a recent study found that unruly passenger disruptions were more common in flights with economy passengers walking through first class than with economy passengers entering the flight behind first class. Maybe, but it will take more for me to believe it. Passengers in first class are not always a rarefied elite. The same persons fly economy, but are bumped up with rewards points to first or business class. Studies like this are potentially dangerous because the airlines could start building more planes with middle entry instead of addressing the real problems - plenty of overhead space for luggage, timely access to restrooms, adequate seating room - all because legitimate concerns about income inequality morphed in popular culture into concerns about elites being cordoned off from the masses on planes, boats, and trains as well as stadiums.

On power pose, this afternoon Amy Cuddy issued her response to critiques of her research, which is actually a response to the Dana Carney's clearly backing away from the original research findings.
We will wait for the response from the experts on statistical methods. Suffice it to say that Cuddy decries the "chilling effect on science. In the last six months, three labs have contacted me to let me know that they have conducted high-powered studies on expansive postures or “power posing,” but that they are reluctant to submit them because they fear being “targeted” for doing research in this area. Open science must be inclusive." I was not aware that scientists are such cowards.

An upcoming post will compare and contrast this evolving situation with the Trump retraction on birtherism and the press treatment of science and politics. As a start, it is interesting that in Trump's retraction, he blamed Hillary Clinton for birtherism. Cuddy lashed back at her co-author for the failures -" I also cannot contest the first author’s recollections of how the data were collected and analyzed, as she led both. By today’s improved methodological standards, the studies in that paper — which was peer-reviewed — were “underpowered,” meaning that they should have included more participants. I wish we had conducted those studies with the rigor of today’s methodological standards, which I firmly believe are moving our field in the right direction."

In the end, the story is not about individuals, but about applying standards of rigor and fairness to produce accurate results and reporting. More to come.

No comments:

Post a Comment