Scientific Review, CFS, and the NIH--- The CFS Special Emphasis Panel
By Craig Maupin at http://www.cfidsreport.com
(Sept 2005) Since 1999, extramural research funding for CFS failed to keep pace with increases in funding for other illnesses. According to the NIH, funding fell from 7 million in 1999 (2000 GAO) to 6 million in 2004. This stagnation occurred on the heels of a doubling of the NIH budget by Congress.
The low levels of funding for CFS research have focused many CFS advocates' attention on the scientific review process. In response to advocate concerns over this issue, the CFIDS Association of America (CAA) has asked the Health and Human Services CFS Advisory Committee to examine the process "to ensure that the panels have appropriate expertise and that the review has been fair and balanced."
Scientific Review for CFS -- Why the Controversy?
Why is CFS and scientific review receiving more attention? Perhaps it is because what constitutes "scientifically sound" research for chronic fatigue syndrome may be subject to more disagreement. In a controversial 1996 Royal Colleges Report, psychiatrists claimed CFS research efforts should be integrated with a class of emerging illnesses that predominantly affect women. They also claimed CFS research should also be steered to specific research areas focusing on the brain, HPA axis, neurochemistry,and behavioral dysfunction.
Conversely, another set of researchers may be more likely to see grant proposals that treat CFS as a systemic and biological illness as "technically and scientifically relevant". These researchers may focus on exclusive research findings. This research may include low molecular-weight Rnase L, circulatory dysfunction, and metabolic irregularities, and coping strategies.
Both the psychiatric and systemic models for CFS blend behavioral and biological approaches. However, the issue that has often sparked conflict has been which biological research, and which behavioral treatments, should be funded. Gender attitudes, cultural and commercial interests, and medical specialties may play a strong role in how a reviewer perceives "good science" for an emerging illness such as CFS, making this illness unique.
The CFIDS Report took a look at some of the controversy surrounding scientific review for CFS at the NIH. We spoke with advocates who say the system is failing. We also took time to speak with officials at the NIH to gain a better understanding of the grant review process. Dr. Terrell Hoffeld, who is the administrator for the CFS Special Emphasis Panel (SEP) at the Center for Scientific Review (CSR), stepped up to explain how the NIH handles grant proposals for CFS.
The CFS Special Emphasis Panel
Grant applications submitted to the NIH for research pertaining to CFS are sent to the Musculoskeletal, Oral, and Skin Sciences Study Section (MOSS) at the Center for Scientific Review. Once there, Dr. Hoffield sets up review panel to assess the proposals. These panels meet three times a year.
Each panel gives each grant application a ranking. Then, proposals are sent to an appropriate institute possible funding. Those proposals ranked in the top 30% have a solid chance of being funded. Those in the top 15% percentile are extremely likely to receive funding.
The CFS Special Emphasis Panel (SEP) is ad hoc. This means, it is not a chartered study section. According to Dr. Hoffeld, "Being a SEP, it has some core members but no set roster," he says, " so it has greater flexibility in putting together the best reviewers for each round. This is a definite advantage since the range of scientific disciplines involved in the applications varies widely from round to round."
Historically, Hoffeld says the CFS SEP was formed "in response to the need consolidate CFS reviews at the time when they were handled in various study sections. There was a request in one of the annual budget write-ups that Congress thought it would be a nice idea if we could track the applications. The easiest way to do that is to have it organizationally in the same place. This actually allows a lot more fairness in the review of applications. "
Classification of CFS Review - Polysystemic Morbidity Syndromes
The term polysystemic morbidity syndrome has rarely been used in scientific literature. However, the NIH has used the term to describe a group of illnesses that they integrated into the CFS scientific review process. The CFS SEP's formal responsibility is to look for the "causes, manifestations and treatments of the Chronic Fatigue Syndrome, the Fibromyalgia Syndrome and other chronic polysystemic morbidity syndromes."
According to Hoffeld, "This was recognition that there are a lot of similarities in the conditions. There are certainly differences too, but [they are integrated] because neither is a specific disease, each is a syndrome."
Hoffeld claims that science is driving the integrative aspects of CFS and other emerging illnesses. "The scientific community has, through their feedback, said that there are certainly similarities and there are differences, and a lot of that has to do with the definition and the diagnosis of each of the two conditions. [CFS and Fibromyalgia] "
Hoffeld says he is concerned that there is a misperception that the integrated review system is diluting the funding of what he says are "CFS-related" grants. "It is important to note that reviewers on the CFS SEP are instructed to evaluate each application in the context of the current status of the particular scientific field and in terms of the likelihood it will be successful. Applications reviewed in SEPs are not compared to one another but to all the other applications reviewed in CSR. It's possible that every application reviewed in a SEP could receive a very favorable percentile score."
"In reality, CFS applications do about as well as others do. Over the past two years, nearly 30% of all CFS-related R01 applications reviewed by the CFS SEP had percentiles of 30 or better. The same was true for the top 15% of CFS-related R01applications."
Hoffeld claims that research for emerging illnesses like CFS and Fibromyalgia could reap advantages from an integrative review process. "To the extent to which the scientific research into one can benefit the scientific research into the treatment of patients with either condition, it's to everybody's advantage to have them all considered together..." Hoffeld says if technology "...is brand new to being applied to this particular syndrome, then we need to have people who have used it in related conditions."
Who is reviewing CFS grant proposals?
According to Hoffeld, the reviewers chosen should reflect the specific types of grant proposals received each round. "That is one of my responsibilities; to determine who has the appropriate scientific background to cover the various disciplines."
Hoffeld also says the individual institutes may, and do, inject their own input as to possible selections. "Any of the programs from any of the institutes can make recommendations in regard to people who would be appropriate study section members. ....I often consult with other scientific review administrators whose areas of expertise overlap into some of the scientific disciplines that are involved in some of the research grant applications."
Since CFS was moved to the Office of the Director in 2000, some advocates feel the makeup of review panels began to be heavily weighted toward psychiatric biology and behavioral interests, locking out researchers who propose systemic and organic models. Grant proposals submitted to the NIH are not available to the public. However, the NIH posts lists of the reviewers selected by review administrators each round. For the last 5 years, few of these reviewers have shown any previous interest in CFS. According the NIH guidelines, reviewers must be, "recognized authorities in their field" and the "principal investigator on a project comparable to those being reviewed". The NIH also encourages balanced review panels that are said to be "multidisciplinary".
The CFIDS Report evaluated the rosters of the CFS SEP in 2004, compiling the overlapping interests and professional expertise represented during the year. Out of approximately 30-40 different reviewers, six reviewers displayed some -- but often minimal -- interest in CFS. Of those, three researchers focused on behavioral aspects of the disease, one on clinical aspects, one on physiology, and another on circulatory/hematological research. Three reviewers in 2004 were evaluated to have CFS research as their dominant interest alone.
According to CFS advocate Jill McLaughlin, the balance of reviewers raises questions. "Effective peer review should include researchers who have some expertise in the field, and very few reviewers have published or worked in the area of CFS. Would AIDS advocates be satisfied with this revolving group of reviewers composed of dentists, veterinarians, neurologists, and psychiatrists etc. reviewing AIDS grants?" McLaughlin says the panels aren't "multidisciplinary" but are "imbalanced".
Kate Duprey, a journalist and CFS advocate from South Carolina, is concerned about the long-term effect the makeup of review panels will have on CFS research. Duprey says the NIH is "pretending that people don't have commercial or professional bias. Commercial interests are playing a large role in what is being deemed 'scientifically significant' for CFS. You cannot assume that a group of people with any predominant interest is value neutral." Duprey says it is not the "number of grants" related to CFS given favorable scores by the SEP, but "which grants are given favorable scores" that is her primary concern.
Economic Demand and Societal Focus
The integration of TMJ and Fibromyalgia with CFS could have a tremendous impact on funding patterns for CFS, placing a predominance of dentistry and pain control experts on review panels for CFS proposals. Another concern is that the NIH decision to place controversial and emerging women's illnesses on one SEP, concentrates reviewers who espouse the same "perception-based" psychiatric theories across these syndromes.
Yet, Hoffeld says the makeup of the CFS SEP panels is being driven by the types of grant applications received, as well as scientific trends that, he says, the NIH has followed closely. "The makeup of the panel is reflecting the type of grant proposals the NIH is seeing. If I stuck with people who were interested in infectious disease and we had things that come in for pain control, could we say that they were being treated fairly? Certainly not."
"Everybody was looking for an infectious agent. A lot has moved on in terms of more palliative directions, ways in which patients can be made to feel more comfortable, experience less pain, less anxiety. There has been an increase over the last decade in terms of the neurological science and looking in that area."
Lack of CFS Expertise
Pat Fero, who heads the Wisconsin CFS Association, has been a long-time critic of the system of review for CFS. "In looking at the NIH SEP reviewers for 2004, all appear to be experts in their respective fields but, few have CFS interest, experience and training. No matter how you look at it, bias against certain CFS proposals will exist until this situation is changed." Fero believes that since CFS was moved to the Office of the Director in 2000, the way review has been handled has changed. She notes that the restructuring of how CFS is reviewed led to a collapse in CFS funding, particularly among researchers who espoused a biological model for CFS.
But Hoffeld claims CFS research is a broad field. He also says much of the science is being applied to CFS for the first time, "It depends upon your definition of 'familiarity with CFS'. Certainly, some of the people who come in don't have a reputation in the community as working in that area [CFS]. And, a lot of that may be simply because the scientific disciplines of the individual applications under review are for the very first time being applied to patients with chronic fatigue syndrome."
Hoffeld believes scientific knowledge for CFS will benefit from a general, rather than a specific, approach. "The generalist can sort of look at things. If an area of science is brought in that has a lot of critical variables; it can make or break the quality of the research. If it is brand new to being applied to this particular syndrome, then we need to have people who have used it in related conditions.... There are certainly always people in the room who have hands on experience with the disease [CFS]." For Hoffeld and NIH officials, viewing CFS as one of many related conditions is a key to scientific success.
Waiting for "Good Grants"
NIH officials have told the CFIDS Report they want to receive "good" grant proposals that ask the "right questions". Officials did not choose to elaborate on what those "right questions" were. They point to the 2003 conference and the 2005 RFA as indicators of an effort to build up a "new" group of scientists who are looking into, what they term as, "productive areas". And, if the review panels are following the types of grants coming in, the NIH is seeing grants focusing on psychiatric biology and behavior.
A 2003 NIH-sponsored Conference for CFS sponsored by the NIH was titled "Neuro-immune Mechanisms for CFS", suggesting a broad focus. However, some advocates felt conference output focused on brain biology and mind-body interactions that align with the psychiatric models of the two co-chairs - Dedra Buchwald and Leslie Crofford. Advocates noted that the two chairs of the conference -- both espousing a model of hyper-perception for emerging women's illnesses -- did not represent a full representation of the scientific community's views on CFS.
According to Kate Duprey, the conference showed that the NIH is "getting what they are looking for [in terms of the balance of grant submissions], but blaming external and societal beliefs about CFS." Duprey says the NIH has always had a discretionary option to place reviewers who espouse systemic or organic models of CFS on review panels. According to Duprey, this is an option NIH officials have chosen not to exercise.
But Hoffeld says there is no bias against microbiological research for CFS. He points to the recent 2005 program announcement. "If you look at that program announcement, those are the areas that are particularly being sought."
Hoffeld says a scarcity of new breakthroughs has driven the problem. According to Hoffeld, changes in what was submitted for CFS occurred after the CFS SEP was reformed: "My study section can only evaluate the applications being submitted. And, the applications being submitted are emphasizing palliative care more than they did in the past, and that is simply because that is what people are thinking about these days. They are sort of waiting for the next breakthrough in the technology to be able to study new things like biomarkers or diagnostic approaches."
Fairness the Goal?
Hoffeld emphasizes that societal or commercial battles over how emerging women's illnesses should be handled will, and should not, play a role scientific review at the NIH. Hoffeld says the Center for Scientific Review is proud of its record. "We at the Center for Scientific Review take pride in the fact that our work is focused on fairness and the future, not only by reviewing the cutting-edge of science, but also by advancing the efforts of new researchers with promise and ultimately helping the best scientists develop new ways to treat, cure and prevent disease."
Next Week: CFS and the NIH: Advocate's Concerns
Craig Maupin can be reached firstname.lastname@example.org