Failure to replicate earlier study causes original author to have a hissy fit.

[Kai]

Full summary over here: http://blogs.discovermagazine.com/notrocketscience/2012/03/10/failed-replication-bargh-psychology-study-doyen/

Basically, John Bargh, a researcher at Yale, conducted a psychological experiment to see if by talking about old age he could prime people to feel the effects of age (aka walking slower). This was back in 1996.

Just recently, Stephane Doyen attempted to replicate this study, but with blinding in some units, something that the original study did not have. Which ended up showing that the individuals walked slower "only when they were tested by experimenters who expected them to move slowly". A basic case of blinding revealing unconscious bias.

Bargh then wrote a blog post where he proceeded to throw a fit. Mind you, it's the sort of gentlemanly like fit that you see in academic circles, but most obviously a case of a grown man loosing it because he wasn't right. He attacks the journal, the authors, and Ed Yong as well. The last was a particularly dumb move, since Yong is probably the best damn science journalist out there, and wasn't exactly going to let it go at that.

This all illustrates that scientists aren't really different than other people when it comes to territoriality, and that replication, /published/ replication studies, are needed now more than ever. Until a study is replicated, it's a sample size one.

[Nephew Twiddleton]

Thats actually a really good idea. Any particular reason why follow up studies arent published?

[Kai]

Thats actually a really good idea. Any particular reason why follow up studies arent published?

Because there's this idiotic taboo against publishing negative results. The whole "you can't prove a negative" is deeply ingrained.

Commoner, however, is the failure of an experiment to demonstrate something because the exact conditions necessary
are not known, such as Faraday's early repeated failures to obtain an electric current by means of a magnet. Such experiments demonstrate the well-known difficulty of proving a negative proposition, and the folly of drawing definite conclusions from them is usually appreciated by scientists. It is said that some research institutes deliberately destroy records of " negative
experiments ", and it is a commendable custom usually not to publish investigations which merely fail to substantiate the hypothesis they were designed to test.

Now, okay, it's not all stupid. There are good reasons not to publish an experiment that failed to substantiate your personal hypothesis. But in the case of replicating someone else's work, a negation is even more important than an affirmation. Results that do not point to one hypothesis will point to another, even if that hypothesis is "our experiment was faulty".

But since it's already drilled into scientists (and therefore journal editors) not to publish negative results, they don't get published usually.

[Nephew Twiddleton]

Seems counterproductive if you cant publish follow up results that negate a previous hypothesis or at least throwit into question.

[Mesozoic Mister Nigel]

I understand the reasoning, I just don't agree with it... not in the age of internet publishing, where it need not be prohibitively expensive to publish studies that essentially just serve to say "Yo, previous study is not to be relied on".

In addition, I think there is an interesting double standard, in that studies that prove, for instance, that there is no link between Atrazine and cancer have no problem getting published.

And I also find the high drama in the scientific community hilarious. 

I swear, I should start a journal called "The Journal of Negative Findings". It would be SO popular.

[Oysters Rockefeller]

Seems counterproductive if you cant publish follow up results that negate a previous hypothesis or at least throwit into question.

Agreed.

"Yo, previous study is not to be relied on".

Man, it's not even like the reactor reached critical mass or some shit.

[Mesozoic Mister Nigel]

Also, I realize that behavioral sciences are a bit different, but I really took issue with the "There are many reasons for an experiment not to work" because findings that don't confirm your hypothesis does not equal "not working". How many fucking times did Bacall and Davis re-run their numbers and very expensive experiment? It was working FINE, it just didn't yield the results they were expecting, which led to an even more significant discovery.

[Q. G. Pennyworth]

I swear, I should start a journal called "The Journal of Negative Findings". It would be SO popular.

This should be a thing. Let's find some scientists and make it a thing.

[Mesozoic Mister Nigel]

I swear, I should start a journal called "The Journal of Negative Findings". It would be SO popular.

This should be a thing. Let's find some scientists and make it a thing.

I'm gonna try to do it. Seriously. If I don't forget.

[Kai]

I understand the reasoning, I just don't agree with it... not in the age of internet publishing, where it need not be prohibitively expensive to publish studies that essentially just serve to say "Yo, previous study is not to be relied on".

In addition, I think there is an interesting double standard, in that studies that prove, for instance, that there is no link between Atrazine and cancer have no problem getting published.

And I also find the high drama in the scientific community hilarious. 

I swear, I should start a journal called "The Journal of Negative Findings". It would be SO popular.

Well, there's already a Journal of Irreproducible Results, so it should go over well.

[Mesozoic Mister Nigel]

I understand the reasoning, I just don't agree with it... not in the age of internet publishing, where it need not be prohibitively expensive to publish studies that essentially just serve to say "Yo, previous study is not to be relied on".

In addition, I think there is an interesting double standard, in that studies that prove, for instance, that there is no link between Atrazine and cancer have no problem getting published.

And I also find the high drama in the scientific community hilarious. 

I swear, I should start a journal called "The Journal of Negative Findings". It would be SO popular.

Well, there's already a Journal of Irreproducible Results, so it should go over well.

The JIR is always entertaining. Nerdery in general is always entertaining, especially when nerds quibble.

This morning I was reading what may have been the pettiest debate ever, about the best longhand method of finding a square root. There was name-calling. It was amazing. I think that a journal that published nothing but negative findings would become a major locus for science drama.

[Mesozoic Mister Nigel]

I also really enjoy the hell out of the Annals of Improbable Research, which most people are probably familiar with thanks to the Ig-Nobels and the Luxuriant Flowing Hair Club for Scientists (which I am looking forward to joining, once I am a scientist).

[Kai]

I also really enjoy the hell out of the Annals of Improbable Research, which most people are probably familiar with thanks to the Ig-Nobels and the Luxuriant Flowing Hair Club for Scientists (which I am looking forward to joining, once I am a scientist).

I have yet to join the LFHCS. This is a moral imperative.

[Mesozoic Mister Nigel]

I also really enjoy the hell out of the Annals of Improbable Research, which most people are probably familiar with thanks to the Ig-Nobels and the Luxuriant Flowing Hair Club for Scientists (which I am looking forward to joining, once I am a scientist).

I have yet to join the LFHCS. This is a moral imperative.

I can't even believe that you're not already a member! I kind of just assumed that you were.

[hirley0]

  ~o~ v read top down v
SO: 2 sepant / 21March N. i'poise the prime problem R thumbnails
7:47Ab while its true i do NOT know what to do HERE is A link Anyway
3/21 ? thumbnail photo ?

http://sf0.org/SFmedia/badges/newplayer.gif & 18
  {18X18 piX

http://media.turnofspeed.com/media/hub/square_26101659.jpg
{60x60

http://media.turnofspeed.com/media/hub/main_26101659.jpg
{400x320

640x480
http://photos3.meetupstatic.com/photos/member/4/f/f/d/highres_12620477.jpeg