The System of Science

With great frequency these days the response to something new and revolutionary is “Ah, but is it scientifically proven".  Not a lot of thought goes into how something gets to be "scientifically proven" but the consensus feeling is that those who are more intelligent than the average man in the street have checked it out and found it to be so -- so it must be right therefore I don't have to think about it, I can go and watch the footy.


In a nutshell science is about observing, some people are good at observing while others are definitely not, this applies across all industries scientists included.  Taking a hypothetical case, say a scientist conducts an expedition into the jungle with a view to making a meaningful discovery.  He happens upon a shrub called XYZ and he observes that most of the mammals in the area eat its leaves.  He notices also that all said mammals are extremely fit and healthy.  Aha  he says, this could be in mankind's elixir of life so he takes some plants home and conducts a trial on 100 people.  At the end of the trial it was found that 90% of the participants had their hair fall out and the other 10% were okay.  So now it is scientifically proven that humans should not eat the leaves from shrub XYZ.


Later another scientist decides to check the findings.  He interviews all the participants on various matters including diet.  He finds that the 10% who kept their hair had one thing in common, they did not drink milk whereas the other 90% did.  So just what has been scientifically proven here? Might it not show that adult humans should not drink milk (that much is certainly true, we are the only species on the planet that drinks milk after we are weaned.) If one or more eminent scientists concur with the first finding it is almost impossible for the second finding to see the light of day. Strike two against it is that if scientists advise the government that drinking milk is bad for public health, it still would not see the light of day because the government just wouldn't allow such a massive disruption to the economy.


The message from all this is that we cannot look at one thing in isolation, for every single thing is interdependent on all the other things in and around its sphere of existence.


Another unfortunate development in modern day science is that our scientists are not always on a quest seeking something that will be to our benefit (who is going to fund that?) but instead devote their time in a largely biased way, proving that this drug is safe or this compound improves the symptoms of XYZ. If a mere 5% of people participating in a trial perceive a benefit, then it becomes 'scientifically proven'.


How about this little gem :-

French research published in 2002 in the British Medical Journal followed 1,674 elderly residents of southern France for seven years, studying their consumption of meat versus seafood and the presence of dementia symptoms. The conclusion was that people who ate fish at least once a week had a significantly lower risk of being diagnosed with dementia over a seven-year period. However the study was unclear as to whether fish consumption protected against dementia, or if dementia prevented the participants from wanting more fish. There was also a possible confounding factor in that individuals with higher education have both a lower risk of dementia and higher consumption of fish


What does this do for your confidence in statistical 'proof' :-

In statistics, a confounding variable (also confounding factor, hidden variable, lurking variable, a confound, or confounder) is an extraneous variable in a statistical model that correlates (positively or negatively) with both the dependent variable and the independent variable. The methodologies of scientific studies therefore need to account for these variables - either through experimental design, in which case, one achieves control, or through statistical means, in which case we are said to account for them - to avoid a false positive (Type I) error; an erroneous conclusion that the dependent variables are in a causal relationship with the independent variable. Such a relation between two observed variables is termed a spurious relationship. Thus, confounding is a major threat to the validity of inferences made about cause and effect, i.e. internal validity, as the observed effects should be attributed to the independent variable rather than the confounder.

In the case of risk assessments evaluating the magnitude and nature of risk to human health, it is important to control for confounding to isolate the effect of a particular hazard such as a food additive, pesticide, or new drug. For prospective studies, it is difficult to recruit and screen for volunteers with the same background (age, diet, education, geography, etc.), and in historical studies, there can be similar variability. Due to the inability to control for variability of volunteers and human studies, confounding is a particular challenge.