Scientific Studies

From an online debate on

ThePhiant wrote:

as you probably know, any study or data can be used to prove anybody’s point. bunching a number of study’s together and coming to a conclusion is always subjective. having said that my main point is that generalizations however subject-ive they may be, don’t mean a thing in the real world. Your partner in life or at work is more than likely the opposite of what those studies are telling you. Just because your studies say you are just as good at math as boys, that doesn’t imply that YOU are good at math. In the real world we are dealing with individuals, not statistics. Talk about YOUR experience,

Now wait a minute here. You may have your own views about the value of scientific study, but what you say is simply not true. Studies cannot legitimately be used to support any point of view, and they are certainly not subjective. In fact studies are very carefully designed to be objective, and where this is not possible, this fact is pointed out. A properly presented study includes an estimate of confidence; the authors’ estimate of likelihood that their conclusions are correct. Science includes in its procedures mechanisms for the discovery of error, and acknowledges that some error will always be present. 

And what exactly do you mean by the “real world?” Is your world any more real than that of thousands of dedicated and hard-working scientists (plus, of course, the normal admixture of charlatans, thieves and cheats; a minority in this as in most pursuits)? To say that generalizations mean nothing is wrong; they are valuable evidence, and to simply dismiss them as valueless is a waste of a precious resource, knowledge. Certainly a study incompletely reported can be used to support a point of view contrary to its true meaning, but this is true of any piece of infomation. It is up to the consumer to decide what is good information and what is bad. Go and read the underlying data and make up you own mind.

As for the contention that “your partner in life or at work is more than likely the opposite of what those studies are telling you”: since the precise purpose of studies is estimating likelihoods, by definition you cannot be more likely to contravene the generalization than follow it.

ThePhiant wrote:

Since when is a study’ objective? Do you mean to say that studies are chosen at random. Studies are chosen to prove or disprove something, and the way to go about is to test certain variables that are CHOSEN not objectively but subjectively. Just like the 11 blind men studying an elephant. They can and want to see only what is in front of them, subjectively. Antioxidants were supposed to be a lifesaver, turns out they actually can shorten someone’s life. Which scientist do you believe?

I will respond, not because I hope to persuade you of anything, but because I think it is important to supply the contrary point of view for others who might be reading this.

Scientific studies, in order to claim that designation, must conform to certain criteria. Among them are the requirements that both the initial assumptions and the methodology used are incorporated as part of the report. These results are studied by others, who are free to critique them. 

Yes, many studies are commissioned by interested parties hoping to prove something, but this does not always succeed; many studies go the other way. Yes, this often results in the suppression of those particular studies, and that is unfortunate, but is part and parcel of the capitalist system. I do not believe that there is a significant number of studies that are falsified for financial gain. Scientific hoaxes have certainly happened, but generally in important cases a lot of scrutiny is brought to bear, and it is hard to sustain a lie. There is a great deal of rivalry in science, and there are a great many scientists who are genuinely motivated by a search for verifiable and useful truths. They are constantly checking each others’ work in hopes of finding errors.

So it would seem extreme to simply reject all scientific studies categorically as unreliable. In a previous post you commented that “bunching a number of study’s together and coming to a conclusion is always subjective”, but i would submit that the more studies agree on a subject the more likely they can be relied on. One study might be a good indicator, especially if its methodology passes scrutiny; five studies that show similar results show a strong likelihood of truth.

Let’s face it; studies are like hamburgers and television sets and cars; some are very good and some are very bad. Most are in the acceptable range. Unlike hamburgers and television sets and cars, however, the way they are constructed can be examined by the user because the map is part of the product.

This, as I see it, is the case for the defense of science; no doubt there are other points that would be helpful, but someone else will have to supply them.

ThePhiant wrote:

Unfortunately you are not supplying the contrary point but actually, you are extrapolating my point that science is not as reliable as it pertains to be. Good science, bad science?  Science is only as good as the next study who proves otherwise. Case in point; antioxidants! btw; I have neither hamburgers, nor tv’s nor cars in my life to examine

Well, as I said, I had no ambition to persuade you from your views; I can see that you twist whatever input you receive to feed your preconceptions. If I am correctly understanding your somewhat idiosyncratic way of using words (extrapolating, pertains?) then once again you state the exact opposite of the case. Since scientific studies include estimates of the likelihood of error, scientific studies are in fact pretty much as reliable as they claim to be!

Maybe you are confusing the studies themselves with the way they are used by people who do not necessarily even understand them, and are all too often willing to omit whatever parts do not suit their arguments.

I know it can be confusing when studies show that something that has been previously thought to be useful also has dangers under some circumstances (antioxidants, for instance). That is where clear thought and judgment come in; my own observation is that pretty much everything in life has a medicinal dose and a toxic dose, and it is up to the consumer to determine from the information provided what those doses are.

Making sense of these kinds of complex issues, where conflicting evidence needs to be properly weighed and evaluated, is greatly facilitated by the use of critical thinking skills. Critical thinking was developed for exactly this purpose; to provide a tool for examining controversial questions and coming to the conclusions that are best supported by the available evidence. Critical thinking does not tell you the answer, any more than a good sharp knife makes you a good cook. If the evidence in fact supports your point of view, then you can use critical thinking to establish that fact. As it is, even though you may in fact be correct about some of your conclusions, your ways of arguing them, and your insistence on making statements that are at the very least highly misleading, do more to conceal that possibility than to reveal it.

I think at this point that I have said pretty much everything I have to say on the subject, and for me at least this discussion is reaching its toxic dose!