Friday, June 27, 2014

Really An Important Article About The Problem of Fraud in Science

The April 28, 2014 Pacific Standard has a lengthy article well worth reading in full about the problems of fraud and non-replicability in the social sciences.  A couple of disturbing excerpts:
Another recent incident was unsettling precisely because it could not simply be dismissed as deviant. Around the same time that Simmons published his tour de force, a paper by the respected Cornell psychologist Daryl Bem claimed to have found evidence that some people can react to events that are about to occur in the near future—a finding as ludicrous-sounding as Simmons’, but one that has been presented by its author as completely legitimate. Bem’s paper set off a frenzy of efforts within the field to debunk his findings. His colleagues’ concern wasn’t just that his paper seemed unbelievable, but that it threatened the whole enterprise of social psychology. After all, if you can follow all the methods and protocols of science and end up with an impossible result, perhaps there is something wrong with those methods and protocols in the first place.
Something unprecedented has occurred in the last couple of decades in the social sciences. Overlaid on the usual academic incentives of tenure, advancement, grants, and prizes are the glittering rewards of celebrity, best-selling books, magazine profiles, TED talks, and TV appearances. A whole industry has grown up around marketing the surprising-yet-oddly-intuitive findings of social psychology, behavioral economics, and related fields. The success of authors who popularize academic work—Malcolm Gladwell, the Freakonomics guys, and the now-disgraced Jonah Lehrer—has stoked an enormous appetite for usable wisdom from the social sciences. And the whole ecosystem feeds on new, dramatic findings from the lab. “We are living in an age that glorifies the single study,” says Nina Strohminger, a Duke post-doc in social psychology. “It’s a folly perpetuated not just by scientists, but by academic journals, the media, granting agencies—we’re all complicit in this hunger for fast, definitive answers.”
This article focuses on social psychology -- a field where, to be blunt, leftist politics is pretty darn thick, as some of the examples of non-reproducible experiments suggest.  I suppose if I could have some confidence that other points of view were widely represented (even 25% of the time), I would not be so darn skeptical of a lot of the stuff that comes out of the field.  The political monoculture of many academic disciplines makes them just too easy for both fraud and self-delusion to take over.

I am also pleased to see that part of the Reproducibility Project is demanding that studies supply the data pre-publication, as a way to more rapidly discover error, non-reproducibility, or fraud.  This has long been one of the problems with the gun control ideologues -- too often, "researchers" in this field will not share data.

1 comment:

Jeff Dege said...

I'd be much more complacent about the future if this sort of thing was only found in the social sciences.

But if John Ioannidis is right, most published medical research is simply wrong.