Do You Trust Research Findings? Perhaps You Shouldn't!

[MylesBAstor]

For those who actually did research rather than speculate and take things always at face value. Another words, the alarming loss of the ability to critical appraise a research paper:

http://www.economist.com/news/brief...elf-correcting-alarming-degree-it-not-trouble

 

[Kal Rubinson]

For those who actually did research rather than speculate and take things always at face value. Another words, the alarming loss of the ability to critical appraise a research paper:

http://www.economist.com/news/brief...elf-correcting-alarming-degree-it-not-trouble

Old news but always worth a reminder to those who actually read research papers. Nonetheless, it is hard to get medical students to put as much effort into biostatistics as they do into, say, gross anatomy. The former will be of more enduring importance to them in their professional lives.

(Those who read only headlines are at the mercy of the press, most of whom cannot make a valid assessment.)

 

[marty]

Myles, thanks for this. Its especially gratifying to see an Economist article posted here. However, while there are many interesting points raised, this article does just what it accuses scientists of doing. It “cherry picks” some of the data. Notice that all discussion of statistics assumes that everyone’s data is significant at the 0.05 level. It then goes on to show how many false positive and false negative results will be obtained. There is no consideration of the fact that most data are significant at much more “solid” levels, like 0.001. I’ve read many discussion of “false science” that make this fundamental and erroneous assumption.

 

[jkeny]

Yesm here's another interesting "mistake" in a published paper http://businessetc.thejournal.ie/austerity-paper-excel-error-873774-Apr2013/
Unfortunately, this paper was often cited in the "austerity" policy foisted on many European countries with debt issues.

 

[j_j]

Indeed, some stuff does leak through peer review, in some cases, way too much. At the same time, some stuff, rather than leaking through, is blocked because a reviewer doesn't like evidence contrary to their prior understanding.

The system is imperfect.

This is no reason to write off the whole system, rather a reason to ask for replication.

I find it interesting that high-end audio is infested with things like the right-wing creationists, there is a smallish, but rabidly vocal group, who wants to reject science and denigrates it at every opportunity.

 

[j_j]

Yesm here's another interesting "mistake" in a published paper http://businessetc.thejournal.ie/austerity-paper-excel-error-873774-Apr2013/
Unfortunately, this paper was often cited in the "austerity" policy foisted on many European countries with debt issues.

Hence the need for replication. And the irony is that that lame paper is the only excuse for austerity economics beyond "I got mine, screw you".

 

[jkeny]

Indeed, some stuff does leak through peer review, in some cases, way too much. At the same time, some stuff, rather than leaking through, is blocked because a reviewer doesn't like evidence contrary to their prior understanding.

The system is imperfect.

This is no reason to write off the whole system, rather a reason to ask for replication.

I find it interesting that high-end audio is infested with things like the right-wing creationists, there is a smallish, but rabidly vocal group, who wants to reject science and denigrates it at every opportunity.

I don't see evidence on this forum for what you are saying about this rabidly vocal group. Are you exaggerating?

 

[jkeny]

Hence the need for replication. And the irony is that that lame paper is the only excuse for austerity economics beyond "I got mine, screw you".

But what percentage of papers have their results tested to a deep level i.e replicated & what percentage lead to "received knowledge" due to many reasons, psychological, economic, political, academic pressure, etc.?

I believe this flaw was pointed out to you before about the human frailties in the application of the scientific principles.

 

[MylesBAstor]

Indeed, some stuff does leak through peer review, in some cases, way too much. At the same time, some stuff, rather than leaking through, is blocked because a reviewer doesn't like evidence contrary to their prior understanding.

The system is imperfect.

This is no reason to write off the whole system, rather a reason to ask for replication.

I find it interesting that high-end audio is infested with things like the right-wing creationists, there is a smallish, but rabidly vocal group, who wants to reject science and denigrates it at every opportunity.

Correction. Badly applied science where ego gets in the way. Remember the phrase great men think alike but fools seldom differ.

 

[Gregadd]

"I find it interesting that high-end audio is infested with things like the right-wing creationists, there is a smallish, but rabidly vocal group, who wants to reject science and denigrates it at every opportunity." j.j.

A decidedly unscientific statement.

 

[Bruce B]

I've written 3 grants this year and "usually" they get approved by the same committee at NIH depending on the success of the previous experiment. It has gotten so bad to where you need to funnel your request to the right people or it gets flatly refused, either by ego or animosity!

 

[j_j]

But what percentage of papers have their results tested to a deep level i.e replicated & what percentage lead to "received knowledge" due to many reasons, psychological, economic, political, academic pressure, etc.?

I believe this flaw was pointed out to you before about the human frailties in the application of the scientific principles.

Your alleged flaw is refuted, time and again, by the clear evidence of self-correction in the scientific community.

The fact that you allege extensive, nearly universal misconduct among scientists is nothing more or less than a conspiracy theory, and one that is refuted by your stereo, cell phone, TV, radio, automobile, doctor, medicine, computer ...

 

[j_j]

I've written 3 grants this year and "usually" they get approved by the same committee at NIH depending on the success of the previous experiment. It has gotten so bad to where you need to funnel your request to the right people or it gets flatly refused, either by ego or animosity!

This is true, sadly.

I've seen the same in being an editor for journals. The problem is usually not what IS published, but rather what is NOT published.

 

[j_j]

I don't see evidence on this forum for what you are saying about this rabidly vocal group. Are you exaggerating?

Go to the mirror, boy! - The Who

 

[j_j]

"I find it interesting that high-end audio is infested with things like the right-wing creationists, there is a smallish, but rabidly vocal group, who wants to reject science and denigrates it at every opportunity." j.j.

A decidedly unscientific statement.

Your claim, your evidence. The evidence for my claim is in public view and we both know it.

 

[jkeny]

Your alleged flaw is refuted, time and again, by the clear evidence of self-correction in the scientific community.

Ok, give us the evidence for your claim - again I ask what percentage of papers have their findings tested by repeating the experiment?

The fact that you allege extensive, nearly universal misconduct among scientists is nothing more or less than a conspiracy theory, and one that is refuted by your stereo, cell phone, TV, radio, automobile, doctor, medicine, computer ...

This is a gross misrepresentation of my position. I said scientists are flawed & therefore are subject to all the common frailties of humans. Now all you have to do is present the evidence which shows how the scientific machine corrects these flaws. You can start with how research grants are awarded & end with the evidence I asked for above

 

[jkeny]

This is true, sadly.

I've seen the same in being an editor for journals. The problem is usually not what IS published, but rather what is NOT published.

Well, yes this is very much a part of what I'm talking about but you have a knee-jerk reaction to my posts.

 

[jkeny]

Go to the mirror, boy! - The Who

Withdraw that, JJ - it's a complete fabrication of your mind. Try to read what I write for a change

 

[jkeny]

Your claim, your evidence. The evidence for my claim is in public view and we both know it.

Are you attempting to refer to me again, JJ? Let's see if Gregadd agrees with you?

 

[j_j]

Withdraw that, JJ - it's a complete fabrication of your mind. Try to read what I write for a change

You mean like how you're scolding me for carefully making a distinction between domains and convertors?

There's a reason for these distinctions: It matters.

As to rejecting plain science, there really is no doubt about what kind of measurements can be made, in terms of accuracy. There is no "hidden distortion" present in a competent measurement, which is, just so we're clear, not a 1kHz ThD measurement (which is 'mostly harmless' until somebody tries to use it), but you have insisted over in the other thread that you want evidence for this and that in regard to accuracy.

So, you are AN example. Hardly the only one.