Bad Questions, Bad Science
By Nancy Hall at http://www.cfidsreport.com
I was sitting in the Grassroots Advocacy session at the Seventh International Conference of the AACFS, when I heard a prominent ME/CFS researcher and clinician make the observation that patients are not always happy with research outcomes. She went on to say, in essence, that good science may lead to unexpected outcomes. She seemed to be suggesting that we should be more accepting of these outcomes, even when they are not what we had hoped for.
At the time, I felt that a response might be in order but I rummaged around in my ME/CFS addled and exhausted brain for a minute or two and couldn’t find one. The moment passed, but the comment lingered in my mind until, several days later, a response took shape. Here is what I should have said.
The scientific process is circular. It starts with a question or hypothesis, the answer to which will add to the body of knowledge related to a particular phenomenon or subject area. In the case of ME/CFS, this knowledge generally relates to information about what causes it, how it affects our bodies, or how to treat it. The process of testing the hypothesis leads to an outcome, which may generate more questions.
The outcome is dependent on a number of factors, not the least of which is the significance and validity of the original question. Some questions beg to be answered and some do not, perhaps because they have been answered many times by other scientists in earlier studies or maybe simply because they are pointless. In the case of ME/CFS, these are either questions that have been asked and answered enough times to establish validity, or they are questions that do not help to define the illness.
Another problem with many hypotheses is that they may seem reasonable and important at face value, but they are based on assumptions that have never been proven. Basing hypotheses on faulty assumptions can be a sneaky but effective way for a researcher to create the illusion that his or her personal beliefs are scientific fact without ever having to subject these assumptions to rigorous scientific testing.
This strategy can be expedient in helping achieve personal or political goals; but it is also, ultimately, dishonest and manipulative. The results of these studies may advance a particular researcher’s agenda or give the appearance of validating his or her belief system, but they do nothing to advance knowledge or understanding of the object of the research in question.
Bad questions make for bad science and it seems, to me, that we are seeing an epidemic of bad questions in ME/CFS research these days. Was it really all that critical, for example, for the CDC to spend our dwindling research dollars searching for links between CFS and the terrorist attacks on U.S. targets on September 11, 2002? How many more studies do we need examining the effects of stressful life events on people with ME/CFS, or on the benefits of CBT and other forms of psychotherapy for people who are struggling to cope with a life altering illness? Does the knowledge that a small percentage of adults with ME/CFS were sedentary as children lead us closer to knowing what causes ME/CFS and how it affects our bodies?
These are a few examples either of recent, government funded research projects or of general themes in ME/CFS research over the past few years. What they seem to reflect is a focus more on knowing about how the experience of having ME/CFS affects the thoughts and behavior of people who have the illness than on knowing more about the illness itself. These kinds of studies often, also, reflect bias in that they are based on the unproven assumption that stress and/or emotional problems are at the root of ME/CFS.
Questions about emotional and cognitive influences or lifestyle may be asked about people with diseases like breast cancer or arthritis, but they are not the primary focus of research into these illnesses. They are tangential, at best, and placing questions about perception of illness or catastrophic thinking or attitudes about exercise at the center of ME/CFS research is irresponsible, particularly given the scarcity of ME/CFS research funds in recent years.
We, as citizens and taxpayers with ME/CFS, cannot afford to continue funding research that asks irrelevant questions. These soft, highly subjective sociological and behavioral studies do nothing to help pinpoint the ways in which the illness changes the cells, and tissues, and chemicals that make our bodies function or, in the case of people with ME/CFS, malfunction.
If the AACFS conference demonstrated nothing else, it is that there is good quality biomedical ME/CFS research going on in laboratories throughout the world and that it is yielding a picture not of a somatoform disorder, but of an illness that originates in the bodies of the people who are afflicted. This is the kind of research that we need to encourage and support.
Members of the ME/CFS community need to unite in demanding that funding be directed toward biomedical research and that the focus of this research shift from “medically unexplained fatigue”, and back to the distinct entity that we know as ME/CFS. Meaningless research into the emotional and behavioral consequences of the illness needs to be halted or, at the very least, placed on the back burner until a great deal more is known about the chemistry and the physiology of ME/CFS.
When we allow this kind of fuzzy research to continue unchallenged, we expose ourselves to risk on several levels. The most obvious is that there will not be enough resources available for substantive biomedical research if the scant research dollars allocated for investigation of ME/CFS are squandered on more behavioral studies.
Historically, the bulk of independent academic research in the U.S. is funded through the National Institutes of Health. While this may be shifting as more and more research funding comes form the private sector, often with strings attached, the NIH remains the major player in this arena.
It is estimated that the federal government has allocated six million dollars a year for CFS research, compared with 389 million for arthritis for 2004, 97 million for lupus, 101 million for multiple sclerosis, and a staggering 4 billion, 284 million for HIV/AIDS research for 2004. The more of our tiny piece of the pie the federal government wastes on behavioral research, the less there will be remaining for biomedical research
Another, more insidious risk, is that the proliferation of behavioral research will result in a definition of the illness based on a behavioral interpretation. This brings us back to the question of hypotheses and outcomes. If the only research that gets funded is research based on behavioral questions or hypotheses, then the only outcomes appearing in medical journals and databases will be behavioral outcomes.
Eventually, this will lead to a view of the illness based entirely on the psychological and behavioral characteristics of the people who have ME/CFS. If there are no outcomes based on biomedical research reported in the literature because there has been little if any biomedical research, then there will be no biomedical data to help define the illness and it will become a behavioral illness by default.
If the behavioral interpretation finds its way into updated versions of the CDC’s clinical case definition, then we may well find ourselves fighting losing battles on every front. If ME/CFS is categorized a behavioral illness, this will affect public policy, disability benefits, health insurance codes and reimbursement, and treatment. We must do whatever we can to insure that the behavioral view does not prevail. This should be the focus of our advocacy efforts.
Nancy Hall ATR-BC, CRC, WATR-L
Board Certified Art Therapist
Certified Rehabilitation Counselor