errors in material & methods sections of papers - by chance or not (Aug/16/2006 )
In the last months we tried to reproduce a relatively long protocol, published in a good, peer-reviewd journal (impact factor >2). We found serveral errors and it took long time until it worked.
As I asked around, other working groups here told me similar experiences. And the general tendency of all scientists was: Material & methods sections in all papers do not trust. There is at least one "error" included with the purpose that it is not easy/fast to reproduce ...
Did you make similar experiences? Or even write papers in this style? For me it was new, since before I did not use papers as method source.
Or is it just bad belief coming from occasional errors in the papers plus typical lab problems, if you adopt the method to your own requirements?
Please report your opinions and experiences.
It's true, it's very difficult to follow a method out of a paper.
Often it's just due to lack of information. I don't know if it's deliberate or due to omissions. I had to take 5 papers to get a complete method for EMSA !
When I take primers out of papers, I always check, because it's always wrong. sometime it's written from 3' to 5' instead of 5 to 3, sometime it's mispelling.
I wanted to use a nice paper, showing how to mesurre the cholesterol on the surface of the cells, by using a chemical and flow cytometer with a special laser. by chance i had everything needed/ Actually, I realised that the signal was due to cell death, due to the chemical ! (and I used the same cell type than in the paper).
some time you gain so much time with a new technique published, and sometime you lose so much time that I never know what to do !
I know this kind of experience as well and I already sometimes thought that things said in the MM section can´t be true regarding the stuff authors claimes in the Result section....never ever... .
However, normally it gives you quiet a good guide line... .
Anf by the way ...f**k impact factors regarding the qualitiy of something. They tell you nothing about well written manuscripts. There are papers out there with an uncountable impact factor but things worked out just brilliant in terms of trying to reproduce results... .
Many times, authors just copy paste their methods and materials without even checking it properly. And some poor person keeps trying the protocol and repeating it and after a couple of days would give up and look up at someother protocol. So much for truthfulness.
Is it some sort of conspiracy ?
May be the authors does not want other researchers to repeat the experiment in a easy way
because the authors had done it the hard way .
I've got similar experiences with published primer sequences - sometimes there's a nucleotide missing or a typo.
I always BLAST the sequence first.