Friday, February 15, 2008

How Science Works (1): Reproducibility

You may think there is something wrong happening with scientists refuting and criticizing each other’s studies all the time. In fact, that is just the way it is supposed to be; that is how science works.

A researcher publishes his or her observations in a scientific journal, explaining why and how the study was done, and reaching a conclusion—which can be a theory—based on the results of this and other studies. Then, other researchers try to understand the methodology of the study and replicate or reproduce the results. This is known as reproducibility. If other researchers cannot obtain similar results using the methodology described in the original paper, the study is considered irreproducible—thus, it does not provide useful evidence of causality. On the other hand, study results that are reproducible have more credibility.

For instance, a few of years ago, Andrew Wakefield (the guy that started the MMR-autism scare) described a new syndrome called "autistic enterocolitis."

In a letter to the editors of Molecular Pathology, Eric Fombonne, a researcher from McGill University in Montreal, said about Wakefield's study:
The study [...] did not contain a proper Subjects section providing the necessary background clinical characteristics which are required both for a proper evaluation of the significance of the results and for allowing replication by independent research groups.” Fombonne also said, “Earlier reports by Wakefield and colleagues have consistently lacked basic descriptive clinical information on the research subjects. Thus, the methods used to arrive at the diagnosis of pervasive developmental disorders in these reports were unstandardized, were not uniform across subjects, and no reliability data were ever provided.
In 2006 Fombonne and his colleagues at McGill published a study describing how they repeated the experiments by Wakefield’s group. They found no evidence that measles virus persisted in autistic children. Wakefield’s results were not reproducible.


Linda said...

Interesting and very well explained. This is something the public should be made aware of. Science is not as scary as it sometimes appears to be.

Anonymous said...

They have indeed been reproduced - blame special interests for keeping it under wraps. Here's the information on the studies:


American scientists claim to have confirmed link between MMR vaccine and autism

NEW YORK, USA: American scientists claim to have confirmed the controversial link between the triple MMR vaccine and autism.

The findings corroborate research by the British gastroenterologist, Dr Andrew Wakefield, discredited by the Department of Health in the United Kingdom, for suggesting the combined measles, mumps and rubella jab may have contributed to rises in the disorder.

The new study, led by Dr Arthur Krigsman, a child gastroenterologist from New York University School of Medicine

Dr Stephen Walker, of Wake Forest University Medical Centre in North Carolina, said the findings replicated Dr Wakefield's in establishing a strong association between measles and autistic children with bowel disease.

A discussion of this issue, with supporting studies, and criticism of studies purported to prove there is no link on this site: