A NY Times article asking “Do Clinical Trials Work?” stirred up lots of questions about the challenges of using experimental designs. Read here Clinical trials (also called experimental designs) are often referred to as “the gold standard” of research. In social science, they are hard to do; when they are used, the results are often inconclusive. Somehow, I thought science might be different, but at least in terms of health research on drugs, the results are also often disappointing, according to the author Clifton Leaf.
Among the evidence presented, he reported, “In a famous 2005 paper published in The Journal of the American Medical Association, Dr. Ioannidis, an authority on statistical analysis, examined nearly four dozen high-profile trials that found a specific medical intervention to be effective. Of the 26 randomized, controlled studies that were followed up by larger trials (examining the same therapy in a bigger pool of patients), the initial finding was wholly contradicted in three cases (12 percent). And in another 6 cases (23 percent), the later trials found the benefit to be less than half of what was first reported.” see Ioannidis article here Lots of comments were generated by the article, followed by several letters to the editor–In defense of clinical trials.Read here The author suggests that perhaps people are too individual: “Researchers are coming to understand just how individualized human physiology and human pathology really are. On a genetic level, the tumors in one person with pancreatic cancer almost surely won’t be identical to those of any other. Even in a more widespread condition like high cholesterol, the variability between individuals can be great, meaning that any two patients may have starkly different reactions to a drug.” But there are other issues as well, including that the studies tend to be small and while volunteers may be randomly assigned to treatment and control groups, they tend to not be fully representative of the larger population. There is also a question of bias, or at least the potential for bias, since drug companies do the testing and clearly have an incentive to see positive results. Some have raised the issue about the short-term nature of clinical trials; sometimes it is only over the longer term that some drugs are seen to be ineffective for most people, or worse, are harmful to some people. In science, there is as much to be learned from failure as from success. However, the research process becomes problematical when people want clear results to guide action: whether it is drug companies looking to manufacture and market a particular drug to make a profit, authors looking to tout “the answer” in the next diet best-seller, or for individuals seeking a cure for disease. What I take away from this debate is that the human body is far more complex than we like to believe and only yields it secrets through a very slow process of research. Theory is not fact, but perhaps we lack the patience to work through the scientific process to really figure out what is really true.