News:

If you really want to hurt your parents, and you don't have the nerve to be a homosexual, the least you can do is go into the arts. But do not use semicolons. They are transvestite hermaphrodites, standing for absolutely nothing. All they do is show you've been to college.

Main Menu

And this is why peer review is a joke.

Started by Kai, January 05, 2012, 07:18:28 PM

Previous topic - Next topic

Triple Zero

Additionally, I wrote this as my own very nitpicky review of that paper this afternoon when I was offline, it's kind of late to the party but here it is anyway:

Quote from: ZL 'Kai' Burington, M.S. on January 08, 2012, 01:52:54 AMAlso, may I just say that I really hate how they seem to minimize statistics in this paper by making them all "x10-3; it's very misleading.

That should have been refused for publications on stylistic reasons alone!

For those that didn't click, the (partial) table sort of looks like this:
Year
Population x103
198029,300
198130,200
198231,100
198332,100
198433,200
198534,300
198635,100
......
200447,000
......

As you see, every population figure ends with two zeros. If I were to take this table at face value, I'd have to assume this was some sort of amazing coincidence, since the usage of scientific notation (that's what you call the x103 stuff) indicates that you're paying attention to the number of significant digits in your data.

Except that they're not. Because probably the zeros mean the numbers are only significant up to three digits. Which means whoever reviewed this should have sent it back after simply paging it through with a note "I'm going to stop reading now, please fix your article in accordance with generally accepted scientific writing style, so's your tables look like this:"

Year
Population x106
198029.3
198130.2
198231.1
198332.1
198433.2
198534.3
198635.1
......
200447.0
......

Why is this important? Well because you can't just look at the number of trailing zeroes to determine the number of significant digits. That's why I included 2004 there, as an example. Because I don't suppose they suddenly measured a whole degree of magnitude less accurate that year, it's probably that the number just happened to end in a zero.

There's nothing in the original table that indicates this. Except for the assumption "well it would be an amazing coincidence if it was significant up to, say, four digits that all just happen to end in a zero", which is of course a very reasonable assumption, but it's incredibly sloppy to leave it up to the reader to have to make it.

As an added bonus, shifting the decimal point like this makes the table x106, which is millions of people, a convenient and familiar unit of magnitude when dealing with human population sizes.

And that's before the reviewer would have to deal with the actual content of the article, which is so full of opinionated bullshit, the only thing that makes it seem "scientific" is the way it's typeset. It would have a better place as an opinion piece in some pseudo-intellectual conservative magazine.

Proper scientific style and tone of writing are both perfectly fine reasons to reject a paper for publication. By tone of writing I mean the passive-aggressive "quoted" words when referring to other research and loaded language, scientific publication needs to be objective and clear. That doesn't mean "dry" or "boring", it just means that if you want to discredit something you state it in a factual manner.
Ex-Soviet Bloc Sexual Attack Swede of Tomorrow™
e-prime disclaimer: let it seem fairly unclear I understand the apparent subjectivity of the above statements. maybe.

INFORMATION SO POWERFUL, YOU ACTUALLY NEED LESS.

Kai

Quote from: Triple Zero on January 08, 2012, 05:08:01 PM
Quote from: The Good Reverend Roger on January 08, 2012, 02:37:45 PMIt was always my understanding that "peer review" meant that you published an article, and everyone else got to take a crack at it via independent confirmation.  If you're right, then it's been tested.  If you're pulling a Fleischman & Pons, you get slapped down.  If you're mistaken, it gets pointed out.

I can understand the need for "gatekeepers", but if being published in a peer-reviewed journal now implies that the idea is accepted, then it serves no purpose.

They're just supposed to maintain some measure of scientific quality, in a sense. Being published in a peer-reviewed journal should mean the logic is sound, the tone is factual, and it's not deliberately ignoring most of the other published research on the topic.

With that, you can assume the conclusion at least has some merit, although that doesn't mean it's accepted as fact, and it can still be picked apart or refuted by new research.

Thing is, peer-reviewed publications are also getting cited by other research papers, so yes it is necessary that the peer-review process guarantees at least some level of accuracy.

In this case, that obviously failed.

Which brings me to Nigel's "career suicide" remark. I wonder, is peer-review anonymous? Could we find out who approved this poop?

And one more remark about papers being or not being freely available to the public, I was going to elaborate on that but Kai already said everything I wanted to say, except to point out that if, as a Muggle, you want to order just a single publication, that's 2-10 pages or so, expect to pay about $30 for it. That's ridiculous, because the journals don't make that kind of costs, they don't even pay commission for the content (because getting published is reward of its own--researcher is more likely to get funding), there's a minor cost of printing, and I dunno what the peer-reviewing costs, but it can't be that much. Especially not when you consider what universities all over the world must already be paying to get on the blanket access IP whitelist.

Which brings me to Kai's remark about Creative Commons, I don't understand what you mean? Which scientific information is licensed as CC? (also, which particular CC version?) Surely not all of it. Most often the copyright automatically defers to the university or faculty.

We know who peer reviewed this article because the reviewers made themselves public in the press release in the opening link. Pre-publication review (except in the case of the editor) is always anonymous unless the reviewers wish to reveal their identity. Though in many cases you generally have some idea who would be reviewing your paper because you know the experts in your field, and the editor usually asks for the names and addresses of potential reviewers when you first send the manuscript.

Peer review is free in the sense that no one gets paid for it. It is expected that, if you are a scientist and you publish papers, eventually you will be asked to pre-pub review someone's article. The really big time experts get asked to review all the time. You are of course allowed to decline, but if you do, the editors of that journal may not be interested in publishing your articles in the future. It's a lot like jury duty. No one really wants to do it, but the benefits are the reason for the obligation.

I meant it in a figurative sense. There are some journals which explicitly state they are creative commons, but due to the nature of scientific information and research, and to copyright law in terms of published ideas and data (and to making copies for research and individual use), all scientific information tends to be used like a CC non-commercial attribution derivative-allowed license with few exceptions. Those being illustrations and photographs.
If there is magic on this planet, it is contained in water. --Loren Eisley, The Immense Journey

Her Royal Majesty's Chief of Insect Genitalia Dissection
Grand Visser of the Six Legged Class
Chanticleer of the Holometabola Clade Church, Diptera Parish

Kai

#77
Quote from: Triple Zero on January 08, 2012, 05:09:01 PM
Additionally, I wrote this as my own very nitpicky review of that paper this afternoon when I was offline, it's kind of late to the party but here it is anyway:

Quote from: ZL 'Kai' Burington, M.S. on January 08, 2012, 01:52:54 AMAlso, may I just say that I really hate how they seem to minimize statistics in this paper by making them all "x10-3; it's very misleading.

That should have been refused for publications on stylistic reasons alone!

For those that didn't click, the (partial) table sort of looks like this:
Year
Population x103
198029,300
198130,200
198231,100
198332,100
198433,200
198534,300
198635,100
......
200447,000
......

As you see, every population figure ends with two zeros. If I were to take this table at face value, I'd have to assume this was some sort of amazing coincidence, since the usage of scientific notation (that's what you call the x103 stuff) indicates that you're paying attention to the number of significant digits in your data.

Except that they're not. Because probably the zeros mean the numbers are only significant up to three digits. Which means whoever reviewed this should have sent it back after simply paging it through with a note "I'm going to stop reading now, please fix your article in accordance with generally accepted scientific writing style, so's your tables look like this:"

Year
Population x106
198029.3
198130.2
198231.1
198332.1
198433.2
198534.3
198635.1
......
200447.0
......

Why is this important? Well because you can't just look at the number of trailing zeroes to determine the number of significant digits. That's why I included 2004 there, as an example. Because I don't suppose they suddenly measured a whole degree of magnitude less accurate that year, it's probably that the number just happened to end in a zero.

There's nothing in the original table that indicates this. Except for the assumption "well it would be an amazing coincidence if it was significant up to, say, four digits that all just happen to end in a zero", which is of course a very reasonable assumption, but it's incredibly sloppy to leave it up to the reader to have to make it.

As an added bonus, shifting the decimal point like this makes the table x106, which is millions of people, a convenient and familiar unit of magnitude when dealing with human population sizes.

And that's before the reviewer would have to deal with the actual content of the article, which is so full of opinionated bullshit, the only thing that makes it seem "scientific" is the way it's typeset. It would have a better place as an opinion piece in some pseudo-intellectual conservative magazine.

Proper scientific style and tone of writing are both perfectly fine reasons to reject a paper for publication. By tone of writing I mean the passive-aggressive "quoted" words when referring to other research and loaded language, scientific publication needs to be objective and clear. That doesn't mean "dry" or "boring", it just means that if you want to discredit something you state it in a factual manner.

Thanks Zero! I knew there was something very strange and wrong about those tables. I also thought that a title with 'in millions' (or in your case, 106) with the statistics in decimal millions would have been so much more straightforward, but I did not consider significant figures.

And yeah, the way Duesberg referred to the Chigwedere et al. article was an intentional jab.

Quote from: Duesberg 2011Recently, however, a new study by Chigwedere et al. from Harvard University "estimated" that from 2000 to 2005 1.8 million South Africans were killed by HIV at a steady rate of 300,000 per year (Chigwedere et al., 2008). These estimates were based on information from the World Health Organization (WHO) (World Health Organization, UNAIDS, UNICEF, 2008a). Chigwedere et al. (2008) further claimed, based on "modeling" the South African epidemic, that anti-HIV drugs could have prevented at least 330,000 of those 1.8 million estimated deaths.

Not only are the statistics altered (the 2008 paper estimated 340 thousand deaths in South Africa between 2000 and 2005, not that many a year), the quotes were sarcastic jabs meant to imply that they thought the paper's methods were crap without actually showing that they were crap.
If there is magic on this planet, it is contained in water. --Loren Eisley, The Immense Journey

Her Royal Majesty's Chief of Insect Genitalia Dissection
Grand Visser of the Six Legged Class
Chanticleer of the Holometabola Clade Church, Diptera Parish


Kai

That second link is living evidence of why it is dangerous and unethical to publish bad science, to allow it to be published, or to give it any time in the spotlight. It has nothing to do with controversial ideas, and everything to do with public safety.
If there is magic on this planet, it is contained in water. --Loren Eisley, The Immense Journey

Her Royal Majesty's Chief of Insect Genitalia Dissection
Grand Visser of the Six Legged Class
Chanticleer of the Holometabola Clade Church, Diptera Parish

Bruno

I don't suppose Kary Mullis has come around on any of his outlandish theories yet, has he?

How can someone that smart be that dumb?
Formerly something else...

Mesozoic Mister Nigel

"I'm guessing it was January 2007, a meeting in Bethesda, we got a bag of bees and just started smashing them on the desk," Charles Wick said. "It was very complicated."


Mesozoic Mister Nigel

One positive benefit of it is that it seems to be creating a schism among anti-gay bigots. I was starting to wonder whether any of these guys had a line they wouldn't cross.
"I'm guessing it was January 2007, a meeting in Bethesda, we got a bag of bees and just started smashing them on the desk," Charles Wick said. "It was very complicated."


Kai

Quote from: Emo Howard on January 27, 2012, 08:36:48 PM
I don't suppose Kary Mullis has come around on any of his outlandish theories yet, has he?

How can someone that smart be that dumb?

Example: Lynn Margulis.
If there is magic on this planet, it is contained in water. --Loren Eisley, The Immense Journey

Her Royal Majesty's Chief of Insect Genitalia Dissection
Grand Visser of the Six Legged Class
Chanticleer of the Holometabola Clade Church, Diptera Parish

ñͤͣ̄ͦ̌̑͗͊͛͂͗ ̸̨̨̣̺̼̣̜͙͈͕̮̊̈́̈͂͛̽͊ͭ̓͆ͅé ̰̓̓́ͯ́́͞

Is there some sort of economic rationale that argues for the current fees required to access peer-reviewed articles?

Who profits the most off of these fees?
P E R   A S P E R A   A D   A S T R A

Triple Zero

Ex-Soviet Bloc Sexual Attack Swede of Tomorrow™
e-prime disclaimer: let it seem fairly unclear I understand the apparent subjectivity of the above statements. maybe.

INFORMATION SO POWERFUL, YOU ACTUALLY NEED LESS.

Kai

#86
The publishers. Example: Elsevier. And they are catching heavy flack right now.

In news related to the OP, one of IJAE editors has resigned over this paper.

QuoteKlaudia Brix, a cell biologist at Jacobs University in Bremen, Germany, says that she tendered her resignation from the board of the Italian Journal of Anatomy and Embryology (IJAE) because she felt that it was important for a journal to function within its scientific "scope".

Others on the 13-member board have also raised concerns. Hanne Mikkelsen, associate professor of molecular medicine at the University of Copenhagen, Denmark, says that she too is considering resigning her position.

ETA: When journals self publish, their societies tend to do well enough off the membership and subscription fees. Publishing print is expensive when the publisher's prices are exorbitant.
If there is magic on this planet, it is contained in water. --Loren Eisley, The Immense Journey

Her Royal Majesty's Chief of Insect Genitalia Dissection
Grand Visser of the Six Legged Class
Chanticleer of the Holometabola Clade Church, Diptera Parish

Kai

You know what? We are awesome.

QuoteThis version, like the original, attempts to challenge estimates of HIV–AIDS death-tolls in South Africa put forward in a study3 led by AIDS epidemiologist Max Essex of the Harvard School of Public Health in Boston, Massachusetts, and questions the effectiveness of antiretroviral (ARV) drugs. "There is no evidence for a new fatal HIV-AIDS epidemic in Africa," write the authors. "We deduce...that HIV is not a new killer virus," they add, and propose a "reevaluation of the HIV–AIDS hypothesis".

But AIDS researchers consulted by Nature say that the new paper uses the same arguments and data as the original version. Both papers, in their view, use flawed methods and selective evidence, they say. Given the body of available evidence, it is "ridiculous" to deny the link between HIV and AIDS, says Essex.

The previous referee reports — obtained by Nature — apply to the new paper "in almost their entirety", at least as far as the demographic anaylsis is concerned says Ian Timaeus, a professor of demography at the London School of Hygiene and Tropical Medicine, who studies the impact of the HIV–AIDS epidemic in South Africa.

One problem that remains unaddressed in the new paper, says Timaeus, is the use of estimates of AIDS deaths in South Africa based on cause-of-death data, which are notoriously unreliable. Another is the claim that South Africa's population is increasing, so large numbers of people cannot be dying of HIV–AIDS, an argument a previous reviewer described as "completely fatuous". There is no reason why South Africa's population can't grow in the presence of AIDS given, for example, its moderately high birth rate and fairly low infant and child mortality from other causes, Timaeus says.

We figured out all of this on our own, with minimal knowledge of the field. Pat yourselves on the back, people, YOU are how peer review should work. I broke into smile when I saw the bolded lines.
If there is magic on this planet, it is contained in water. --Loren Eisley, The Immense Journey

Her Royal Majesty's Chief of Insect Genitalia Dissection
Grand Visser of the Six Legged Class
Chanticleer of the Holometabola Clade Church, Diptera Parish

Mesozoic Mister Nigel

Quote from: ZL 'Kai' Burington, M.S. on January 31, 2012, 11:29:30 PM
The publishers. Example: Elsevier. And they are catching heavy flack right now.

In news related to the OP, one of IJAE editors has resigned over this paper.

QuoteKlaudia Brix, a cell biologist at Jacobs University in Bremen, Germany, says that she tendered her resignation from the board of the Italian Journal of Anatomy and Embryology (IJAE) because she felt that it was important for a journal to function within its scientific "scope".

Others on the 13-member board have also raised concerns. Hanne Mikkelsen, associate professor of molecular medicine at the University of Copenhagen, Denmark, says that she too is considering resigning her position.

ETA: When journals self publish, their societies tend to do well enough off the membership and subscription fees. Publishing print is expensive when the publisher's prices are exorbitant.

I am unsurprised about the resignation, and would expect that most scientists will decline to review submissions in the future. If they published that piece of junk in order to increase their visibility, they certainly accomplished that, if they are OK with "visibility" = "on a lot of shit lists".

I don't see any reason they should have expected publishing bad science to get them any more subscriptions. There is little or no reason for any university to pay for a journal that publishes shoddy and unscientific work.
"I'm guessing it was January 2007, a meeting in Bethesda, we got a bag of bees and just started smashing them on the desk," Charles Wick said. "It was very complicated."


ñͤͣ̄ͦ̌̑͗͊͛͂͗ ̸̨̨̣̺̼̣̜͙͈͕̮̊̈́̈͂͛̽͊ͭ̓͆ͅé ̰̓̓́ͯ́́͞

Quote from: ZL 'Kai' Burington, M.S. on January 31, 2012, 11:29:30 PM
The publishers. Example: Elsevier. And they are catching heavy flack right now.

ETA: When journals self publish, their societies tend to do well enough off the membership and subscription fees. Publishing print is expensive when the publisher's prices are exorbitant.

Thanks for the info, Kai.

I've only read one of those links so far, but I'm completely infuriated and disgusted.

How long has it been this way?
P E R   A S P E R A   A D   A S T R A