Center for Integrative Medicine, University of Maryland School of Medicine, 2200 Kernan Drive, Kernan Hospital Mansion, Baltimore MD 21207
Email : firstname.lastname@example.org
During the past decade, the Cochrane Collaboration has been an increasingly important source of information on complementary and alternative medicine (CAM) therapies. From 2007 to 2008, the Cochrane CAM Field developed a topics list that allowed us to categorize all 396 Cochrane reviews related to CAM (as of The Cochrane Library, Issue 4, 2009). This topics list is an advance in making Cochrane reviews on CAM topics accessible to the public. In this article, we discuss challenges in developing the topics list, including developing an operational efinition of CAM, deciding which reviews should be included within the CAM Field's scope, developing the structured list of CAM Field-specific topics, and determining where in the topics list the reviews should be placed. Although aspects of our operational definition of CAM are open to revision, a standardized definition provides us with an objective, reproducible, and systematic method for defining and classifying CAM therapies.
During the past decade the Cochrane Collaboration has been an increasingly important source of information on complementary and alternative medicine (CAM) therapies.1,2 Because CAM therapies are relevant to multiple health care conditions, members of the Cochrane CAM Field felt that it might be useful to develop a convenient way for users of The Cochrane Library (researchers, clinicians, consumers) to identify Cochrane reviews that are CAM-related, and to find Cochrane reviews on specific CAM therapies. From 2007 to 2008 the Cochrane CAM Field developed a Field topics list categorizing all 396 CAM-related Cochrane reviews (as of The Cochrane Library, Issue 4, 2009). This topics list, which is part of the internal Cochrane administrative database and facilitates internal Field operations and communications among Cochrane colleagues, is also an advance in making Cochrane reviews on CAM topics accessible to the public. The topics list, which is freely available online at http://www.cochrane.org/reviews/en/topics/22_reviews.html, provides links to the titles, abstracts and plain language summaries of all Cochrane reviews on CAM therapies. In addition, the Field topics list clarifies, to internal and external audiences, the scope of the CAM Field. This paper describes the rationale for producing the Field topics list and reviews some of the obstacles we encountered in operationalizing and classifying CAM therapies in our Field topics list. We conclude with a discussion of ongoing challenges and opportunities related to the Cochrane CAM Field topics list, including ways in which the Field topics list may assist in identifying gaps in Cochrane systematic review coverage of CAM therapies.
We encountered three major challenges in developing the CAM Field topics list. Our first challenge was to develop an operational definition of CAM so that we knew which reviews to classify as CAM-related. Our second challenge was to apply this operational definition. That is, what methods should we use to classify individual reviews on The Cochrane Library as meeting or not meeting the operational definition eligibility criteria? Our third challenge was to develop a classification tree structure to organize the CAM-related Cochrane reviews into sub-categories in a list, so that individual reviews on specific therapies could be easily found.
1. Developing an operational definition of CAM
The set of CAM interventions is comprised of multiple therapies. During the last decade and a half, some of these therapies have been integrated into medical school curriculums and provided alongside conventional therapies in physician offices, clinics and hospitals.3,4 CAM therapies have also been the focus of randomized controlled trials and systematic reviews, and some authorities have suggested distinguishing between unproven versus proven treatments, rather than conventional versus unconventional therapies.5 Despite this increased mainstream openness to and acceptance of CAM, there is a shared sense among practitioners, researchers and consumers that there remains a group of therapies that are in some sense outside the mainstream medical model, and that these therapies are appropriate to group together under the CAM label. However, while some therapies would be accepted by virtually everyone as CAM (e.g., acupuncture) other therapies would be accepted as CAM by some people and not by others (e.g., vitamin supplements). The therapies that are members of the CAM group therefore vary, and this lack of an existing set of identified CAM therapies was the first challenge we encountered in developing our Field topics list.
Operational versus theoretical definition
As noted above, in order to decide whether or not specific Cochrane reviews fell within the scope of the CAM Field, we needed to develop an operational definition of CAM. In contrast to a theoretical definition, which characterizes the fundamental nature of a construct, an operational definition tests whether a specific instance is or is not a member of the construct through a series of criteria or tests. A theoretical construct may have many different possible operational definitions. For example, in a randomized trial of depression medication, the theoretical construct of the disease being treated is ‘depression’, but the operational definition may be, for example, score of 20 or more on the Beck Depression Inventory or answering ‘yes’ to the question ‘are you depressed’. Without specific operational criteria the theoretical definition is of limited practical use.
The definition of CAM has been much debated in recent years as CAM has increasingly become a focus of public and academic attention. While there have been many theoretical definitions of CAM6,7 there are no comprehensive operational definitions of what should be considered CAM.
The need for an operational definition of CAM to construct a set of CAM-related Cochrane reviews
Because there is no accepted operational definition of CAM, we found that each time we tried to classify Cochrane reviews as CAM or not CAM we would end up with overlapping but slightly different sets of reviews. Because of this difficulty, we decided that we had to develop and implement an operational definition in order to build the Cochrane CAM Field topics list. This is not to say that the operational definition we developed is the ‘right’ one, only that we needed to create explicit and transparent criteria in order to clarify the scope of the CAM field and build our list of CAM-related Cochrane reviews. Otherwise we would have no objective, reproducible and systematic criteria for including or excluding individual Cochrane reviews from our scope and we would continue to have the problem of including different sets of Cochrane reviews as CAM each time we populated our list.
The need for an operational definition of CAM to facilitate and harmonize research
Standardization of definitions of CAM also means that different groups working independently on database development can collaborate more effectively and build on each others’ work. For example, not only the Cochrane CAM Field, but also the UK National Health Service is developing a database of CAM studies; if we use the same definitions, then we can collaborate more easily, avoid duplication of effort, and promote harmonization of processes.
Standardization is also useful in promoting and comparing research in different CAM areas. For example, defining CAM operational criteria allows survey results on CAM prevalence to be comparable across time and across investigators. Depending on how CAM is defined, prevalence surveys on CAM use vary from 10% to 100%.8 It seems like an enormous waste of research resources to not allow for the results of these different studies to be put side by side. If operational criteria are consistent, then studies can be directly compared. If operational criteria are transparent, then even if there is inconsistency between different sets of criteria there is some basis for comparison.9
As another example, standardized operational criteria allow examination of evidence for benefits or adverse effects of CAM. That is, often people claim that there is no evidence for CAM therapies. However, this statement is not interpretable if there is no clear agreement on what is CAM. What if one were to wish to compare the evidence base for CAM versus the evidence base for conventional medicine? If there is no clear definition of what CAM is, how could we do this? Indeed, it is misleading to even use the word CAM, if different people would define it in different ways.
Using a theoretical definition of CAM for clues to develop an operational definition
To begin to develop an operational definition, we started by looking at the theoretical definitions of CAM. The widely-accepted theoretical definition of CAM (see Box 1) was arrived at by the Office of Alternative Medicine (OAM) expert panel at the Conference on CAM Research methodology in April 1995. (The Office of Alternative Medicine later became the US National Center for Complementary and Alternative Medicine (NCCAM).)
"Complementary and alternative medicine (CAM) is a broad domain of healing resources that encompasses all health systems, modalities, and practices and their accompanying theories and beliefs, other than those intrinsic to the politically dominant health system of a particular society or culture in a given historical period."[Institute of Medicine, 2005]
This definition is not an operational definition because it does not tell you whether or not, for example, acupuncture or relaxation therapy or omega-3 supplementation are CAM therapies. You could think about whether or not an individual therapy meets the definition above, but the definition does not provide concrete tests to tell you ‘yes’ or ‘no’ about any specific therapy. However, the theoretical definition does provide a major criterion we needed to consider in developing our operational criteria. We needed to take into account whether the therapies are founded upon the theories of disease and healing that are fundamental to the dominant health system in the culture in which the Cochrane CAM field is situated – the conventional Western medical model as it is practiced in the early twenty-first century. That is, therapies such as acupuncture or homeopathy that are based on non-allopathic theories of disease and healing are clearly CAM. One path to developing an operational definition of CAM could therefore involve identifying and listing therapies that rely upon non-allopathic models of health.
The major limitation to considering this criterion, and solely this criterion, in developing an operationalization of CAM is that the dominant medical model changes over time. As we have seen above, despite a common understanding that acupuncture is indeed ‘CAM’, many CAM therapies, including acupuncture, have experienced increased inclusion in Western medical contexts during the past several years. If one were to take the view that whether or not something is CAM depends on the evidence accepted by the dominant system, then defining what is CAM would require a periodic re-evaluation as the evidence changes over time, and the politically dominant health system incorporates the therapy not only in practice but also in principle. For example, what if acupuncture becomes explicable in terms of allopathic mechanisms and is therefore completely accepted within conventional medicine? What if acupuncture is proven highly effective for some indications and thus becomes an accepted treatment even though the allopathic model does not provide a plausible mechanism of action? What if the allopathic model itself evolves to include the concept of Qi? In each of these cases we would be forced to ask whether acupuncture is no longer CAM, despite its origins outside conventional medicine, and if it is no longer CAM then what was the defining moment when it lost its CAM status. Furthermore, it is possible to imagine places or times in which multiple medical paradigms might coexist without any single model achieving clear social or political dominance. For now, we considered these questions but decided to concentrate on defining the parameters of CAM from our current historical and geographical perspective of a dominant allopathic model. However, we felt that we needed to go beyond the non-allopathic criterion in operationalizing CAM.
Criteria considered in developing CAM operational definition
In expanding and refining our CAM operational definition, we considered several criteria. First we considered whether the historical notion of the therapy was CAM or conventional. As suggested above in the theoretical model of CAM, if the therapy was based upon the theories of a medical system outside the Western allopathic medical model, then (from the current perspective of the US and Europe, anyway) it would be labeled alternative medicine, or CAM. We therefore included therapies such as Chinese and Japanese traditional medicine, Ayurvedic medicine, homeopathy, chiropractic, naturopathy and Reiki, among others, as CAM, and we did so regardless of their current level of acceptance by the allopathic system.
For therapies that did not clearly originate outside the theories or beliefs of the allopathic medical system, we next considered whether the use of the therapy for a particular condition is currently considered to be a standard treatment within the dominant medical system. If something is currently a standard, accepted therapy, then it is not likely to be widely considered as CAM. Indicators that a therapy is accepted include government licensing of practitioners, coverage by health insurance, statements of approval by government agencies, and recommendation as part of a practice guideline. For example, the US Federal Drug Administration (FDA) has approved a device for administering transcranial magnetic stimulation (TMS) in treatment resistant depression (http://www.medinewsdirect.com/?p=593). TMS for depression may therefore be questionable as a CAM therapy. While following FDA guidelines would only be helpful for a limited number of therapies and would be exclusive to the US, we considered this to be one source of information [not definitive] on whether or not a specific therapy is CAM. Using the criterion of acceptability to the dominant health system meant that we would have to consider treatments and conditions together, since a treatment might be standard for one condition but alternative for another medical condition. For example, TMS is not approved by the FDA as a treatment for Parkinson Disease. As we stated above, in cases where we had decided that the therapy was CAM based upon its origins outside the allopathic system, we did not change our assessment if the therapy was accepted by insurance or practice guidelines. For example, some health insurance plans in the US and Europe cover acupuncture or chiropractic treatment, but we nevertheless considered these therapies to be CAM.
Third, for therapies that did not clearly originate outside of the allopathic system, we also considered the setting in which the therapy is delivered. Therapies that are self-care or delivered by alternative practitioners are more likely to be widely considered as CAM, while therapies that are delivered exclusively by conventionally credentialed medical personnel or exclusively within hospital settings are much less likely to be considered as CAM. Thus, nutritional therapies, including special diets and dietary supplements, are likely to be widely considered as CAM, while surgery is not considered CAM. Some therapies would be classified differently according to this consideration versus that of the previous criterion. For example, a self-delivered nutritional therapy that is supported by FDA recommendations might be considered as CAM by some persons and as non-CAM by others.
Finally, we should mention that we did not consider evidence of efficacy (or lack of evidence) as a test for identifying a CAM therapy. This is because there are many therapies that are not currently accepted as efficacious, but not all of them would be necessarily considered CAM. For example, a new synthetic chemotherapy agent would not be considered CAM, even if it has not been proven to be efficacious, while an herbal therapy for cancer would generally be considered CAM, even where it had trial evidence of efficacy. As with CAM therapies, many conventional therapies also do not have convincing evidence of benefit. An earlier assessment of reviews from the Cochrane Collaboration found that less than 25% of Cochrane reviews on conventional biomedical interventions resulted in significant evidence of benefit.10 Therefore, lack of proven efficacy is not an appropriate test for CAM.
With these criteria in mind, and a beginning set of CAM therapies, we looked for sources that would give us additional guidance in developing a list of specific therapies for our operational definition.
Sources considered in developing the operational definition of CAM
We examined several existing sources in developing the CAM operational definition. First, we looked at two sources within the US National Library of Medicine’s PubMed database: 1) the MeSH (Medical Subject Headings) definition of complementary therapies and 2) the Complementary Medicine subset search strategy (also called CAM on PubMed). The MeSH definition of complementary therapies is “Therapeutic practices which are not currently considered an integral part of conventional allopathic medical practice.” This is a theoretical rather than an operational definition, and therefore was of limited use for our purposes. The CAM on PubMed search strategy conducts a complex search within MEDLINE, with the terms in the search strategy representing different CAM therapies and CAM journals.11 It is a form of an operational definition. However, the subset search strategy is not precise as to therapies/conditions pairings, and it is not sensitive or specific enough.
We next considered conducting a Delphi survey, which is a structured series of surveys consisting of rounds of opinion collection and feedback, to arrive at an operational definition of CAM. However, we decided against it, primarily because of time and resource limitations. We did consult recent research utilizing Delphi surveys, and confirm that we did not omit any CAM therapies identified by this strategy from our own operational definition.9
CAM operational definition – list of therapies included
Based upon our review of the sources described above, we identified CAM therapies as therapies used in treating or preventing disease that were captured by the 70 different terms or combination of terms listed in Table 1.
|Acupuncture (e.g., needle acupuncture, electroacupuncture)|
|Arts therapy (e.g., dance therapy, drama therapy, music therapy)|
|Ayurvedic traditional medicine (Ayurveda)|
|Bee products (eg, honey, pollen, propolis, royal jelly, venom)|
|Chinese traditional medicine|
|Chiropractic (i.e., spinal manipulation)|
|Color therapy (i.e., chromotherapy)|
|Dietary supplements (non-herbal)† (e.g., vitamins, hormones, amino acids)|
|Diet therapy† (e.g. low fat diets, vegan diets)|
|Electric stimulation therapy† (eg, transcutaneous electrical nerve stimulation)|
|Eye Movement Desensitization and Reprocessing (EMDR)|
|Herbal supplements (eg, echinacea, garlic)|
|Imagery (i.e., visualization techniques)|
|Light therapy† (phototherapy)|
|Magnetic field therapy† (eg, transcranial magnetic stimulation)|
|Spiritual healing (eg, prayer)|
|Traditional healers and healing practices (other than Chinese) (eg, Kampo, Shamanism)|
We attempted to be comprehensive in our operationalization of CAM therapies, even including multiple terms that could be considered to overlap (e.g., Chinese traditional medicine and acupuncture). However the number of individual CAM therapies within some categories (e.g., individual herbs within the herbal therapies category) was impossible to capture in a single table suitable for publication. This operationalization is also biased towards inclusion of therapies that have been the subject of randomized controlled trials, and there may be other CAM therapies of which we are unaware. This operationalization therefore cannot be considered to be exhaustive, and is subject to expansion over time.
For many of these categories, the operationalization is clear-cut and everything described with the term would be considered to be CAM. For example, any therapy described as ‘homeopathy’ or ‘homeopathic’ would be considered complementary and alternative. In some cases, however, as described above in our discussion of the criteria for CAM, the context of the therapy determines whether the therapy would be considered CAM or not, and we needed to be specific about the therapy/condition pairing because the therapy was conventional and accepted for some conditions but complementary and alternative for others. For example, hyperbaric oxygenation is a standard treatment for carbon monoxide poisoning, but is an alternative treatment for multiple sclerosis. We have omitted the therapy/condition pairings from Table 1 for reasons of space, although we have indicated with a note those therapies that are considered conventional in some situations.
The category in which we had to be most concerned about therapy/condition pairings was dietary supplements. Our first exclusion in this category was supplements that are administered parenterally in hospital settings (eg, intravenous magnesium for acute traumatic brain injury). Our rationale for this was that both the setting and the route of administration are so embedded in the dominant health care system that most persons would not consider the therapy to be CAM. A second category of exclusion was dietary supplementation for treatment or prevention of medically diagnosed deficiency states and disorders (e.g. iodine supplementation for preventing iodine deficiency disorders in children), and a third major exclusion was vitamin treatments used for preventing or treating disease in countries where vitamin deficiency is widespread (e.g. Vitamin A for treating measles in children in Niger). We excluded these last two categories of reviews because the therapy is provided in the context of a conventional determination of deficiency, and we therefore believe that almost all researchers, and probably most consumers, would not consider these to be examples of CAM. It was more difficult to decide whether to include as CAM dietary therapies that are accepted for prevention or treatment of specific disorders. For example, we debated whether to include the review of folic acid for neural tube defects as CAM-related. Many would not consider this treatment-condition pairing to be CAM-related because it has strong supportive evidence from randomized controlled trials which has resulted in its being integrated into the dominant healthcare system, such that folic acid supplementation is approved by the FDA for prevention of neural tube deficits. Because dietary supplements and therapies are generally self-administered and not dependent on medical professionals, however, and because we believed that users of our Topic List would expect to see folic acid reviews listed under the “Vitamins” subheading, we decided to classify the folic acid review as CAM-related. In general, we decided that we should be over-inclusive rather than under-inclusive with nutritional therapies, aside from the major exclusions detailed above.
Our last categories of exclusion from the CAM operationalization were not based on therapy/condition pairings. We decided to exclude reviews of exercise therapies, with the exception of mind body exercise (e.g. tai chi, yoga), and psychotherapy, with the exception of unconventional psychotherapies (e.g. Morita therapy).
We have posted our complete operational definition of CAM, with exclusions for each therapy indication where appropriate, on our Field website at http://www.compmed.umm.edu/Camdef.asp, where it can be updated when necessary and referred to by those interested in the scope of the Cochrane CAM Field.
2. Applying the operational definition to identify relevant Cochrane reviews
The second challenge in developing a topics list was deciding how to apply the operational definition to searching for and identifying relevant Cochrane reviews. We needed to set out these rules in order to make the selection of reviews transparent, consistent, and reproducible. In deciding how to apply the operational definition to identification of relevant reviews we had to ask ourselves a series of questions.
First, which text do you examine for each review on The Cochrane Library to select which reviews meet your operational definition eligibility criteria? We decided that we would search the title and abstract text rather than read the full review. This strategy would identify nearly all relevant reviews, and all reviews for which a CAM therapy is an important focus of the review, and would be the most efficient approach.
Second, should retrieval be limited to reviews that find a trial with a CAM therapy? That is, what if a trial of the CAM therapy is searched for but no trials are found by the reviewer? We decided that we would include the review as long as the abstract explicitly mentioned a CAM therapy was an eligible intervention. So if the Selection Criteria in the abstract of a review specified that acupuncture trials were searched for, we would include the review as CAM whether or not any acupuncture trials were found. This would identify not only reviews in which trials had been found, but also reviews in which trials had been sought but not found, and would therefore assist in detecting gaps in the literature.
Third, should we only include reviews that explicitly search for therapies that we have defined as CAM? For example, in the review ‘Wound cleansing for pressure ulcers’, the SELECTION CRITERIA was as follows: “Randomised controlled trials (RCTs) comparing wound cleansing with no wound cleansing, or different wound cleansing solutions, or different cleansing techniques, were eligible for inclusion if they reported an objective measure of pressure ulcer healing.” Note that this does not explicitly specify any CAM intervention. However, the Results section of the abstract includes an RCT of a saline spray containing Aloe Vera, which is an herbal therapy and therefore eligible as CAM. We decided that even if the review did not explicitly search for CAM therapies, the mention of a CAM therapy in the Results section would qualify the review for inclusion as a CAM review. In sum, the specific mention of a CAM therapy in the title or abstract is required for us to consider the review as CAM, and any mention of searching for or retrieving a trial of a CAM therapy requires us to include the review.
3. Deciding on the classification tree structure for the topics list
How to organize the list of therapies? Alphabetical or grouped conceptually?
The final challenge in developing a topics list was deciding on the classification tree structure for organizing the therapies. One possible organizing principle was simply to list the therapies alphabetically. An advantage of this organization method is that it is easy to find the therapy you are looking for in the alphabetical list. However, reviewing all therapies in the list is time-consuming. There may also be some confusion about where to find therapies that could be identified with more than one name. For example, one would have to search under ‘L’ for light therapy and ‘P’ for phototherapy. Neither term is more correct than the other, and this could result in confusion if only one term is listed, and unacceptable length and opportunity for error if both terms are listed. In addition, for many of the therapies in an alphabetical list, we would have to consider whether or not to list additional subcategories. We therefore decided that a purely alphabetical approach would be insufficient to organize the topics list.
Classification tree structures present in existing electronic resources of CAM
At this point, we decided to take a step back and look at borrowing from existing structures to develop our own classification tree. We examined the structures of the following three classification systems that already exist online in CAM: the UK NHS Evidence-CAM specialist collection (NHS Evidence-CAM collection), the US National Library of Medicine (NLM) and its Medical Subject Headings (or MeSH), and the categories of CAM delineated by the US National Institutes of Health Center for Complementary and Alternative Medicine (NCCAM).
We began by looking at the NHS Evidence-CAM collection, which is an extensive online library of evidence, education, and patient information on CAM. The portal for the NHS Evidence-CAM collection lists the following categories of CAM therapies alphabetically: “acupuncture, aromatherapy, chiropractic, dietary and nutritional therapies, herbal medicine, homeopathy, hypnosis, massage, meditation, osteopathy, reflexology, yoga, other therapies or medical systems”. Clicking on “other therapies or medical systems” brings up a list of 15 additional heterogeneous therapies. We decided to use all of the major categories from the NHS Evidence-CAM collection as major categories in our CAM Field scheme. Therefore the two schemes would be compatible, leaving open the possibility of future collaboration. However, we decided to look further and see whether we could construct a tree structure based on a more conceptual rationale and without relying on putting multiple therapies under the category of “other”, thinking that a more specific structure might be easier to navigate.
We then looked at NLM’s MeSH heading structure for the MeSH term Complementary Therapies. This list of MeSH headings is organized alphabetically, but there are many differences between the Complementary Therapies MeSH tree and the NHS Evidence-CAM collection structure described above. As one example, in the MeSH headings acupressure is listed as its own major heading while in the NHS Evidence-CAM collection acupressure was listed under the category ‘other’. The MeSH placement of acupressure might make it easier to locate quickly, and the MeSH structure does not include an ‘other’ category. However, there are some anomalies in the MeSH structure. For example, some of the major headings are of relatively minor therapies. Anthroposophy, which derives from the philosophy of Rudolph Steiner, has only 134 citations in all of Medline/PubMed and probably should not be a major heading. Also, the MeSH headings are not always organized in an intuitively obvious or logical fashion. For example, the alternative medical systems of Homeopathy and Naturopathy are major headings but non-western alternative medical systems (e.g., Ayurvedic Medicine) are grouped together as “traditional medicine”. We thought that perhaps we could build on aspects of the MeSH structure but look elsewhere for an overarching organizational scheme and thus avoid some of the limitations we saw here.
The last structure we looked at was that developed at the US National Institute of Health’s NCCAM. NCCAM groups CAM practices into four domains, or types of therapies, recognizing there can be some overlap. In addition, NCCAM has a category of CAM whole medical systems, which cut across all domains.
The NCCAM categories of therapies are as follows 12:
Mind-Body Medicine, which uses a variety of techniques to enhance the mind's capacity to affect bodily function and symptoms.
Natural Product Based Therapies, which use substances found in nature to promote health.
Manipulative and Body-Based Practices, which are based on manipulation and/or movement of parts of the body.
Energy Medicine, which involves the use of energy fields, either the unconventional use of electromagetic fields, or the manipulation of energy fields that purportedly surround and penetrate the human body.
Whole Medical Systems, which are complete systems of theory and practice outside the conventional allopathic model.
The great appeal of this way of organizing CAM is that there are overarching principles by which the therapies can be organized. A possible problem with this system is that someone who does not understand the language and principles of CAM might have some difficulty in knowing what category a specific therapy would fall within. In some cases, even someone who is familiar with CAM would have several valid options for where a therapy might be classified.
Classification suggestions from the CAM literature
Finally, we reviewed the CAM literature to identify additional proposals for how to organize and classify CAM therapies. Suggestions for organizational schema included the following: epistemological perspective and type of healing method,13 a taxonomy of five sectors of CAM,14 four paradigms of health and disease,15 and modes of therapeutic action.16 While each of these classification schema had merit and each provided valuable insights into the parameters of CAM and the underpinnings of CAM practices, we did not feel that any one of them was sufficiently well-known or widely used to adopt for the Cochrane Field topics list.
Final classification scheme
We decided to follow the NCCAM model because of its logical CAM therapy classification system, and its prevalence online and in the literature. We found additional reputable online resources on CAM (eg, the Mayo clinic website at www.mayoclinic.com/health/alternative-medicine/PN00001), books17 and journal articles18 that discussed CAM therapies by using the NCCAM framework (sometimes with minor modifications). Because the NCCAM is a large funding organization and source of government information to researchers and consumers of CAM, we felt that researchers and consumers, especially in the US, would be most familiar with the NCCAM classification scheme and therefore this organization of topics would be the most accessible.
After deciding on this classification system, we placed each of the CAM therapies into one of the five major topic categories corresponding to the five NCCAM categories. Although there is some overlap between the categories, we put each therapy into only one place in the classification tree. For example, acupuncture could belong to the “whole medical systems” category because it is part of Chinese Traditional Medicine, or it could belong to the “energy medicine” category because it relies upon the principles of energy flow. Based on consensus among the authors, we chose to put it in only one of those places (i.e., we decided to classify it in the “energy medicine” category). Therapies were put in only one place in the tree for practical reasons, as the Cochrane topics list software is not designed for setting up classifications and links in which therapies can be listed once and appear under multiple headings. Manually placing therapies in multiple places in the topics list would require additional resources, and present opportunities for error that would require time and energy to guard against. As mentioned above, it might be difficult for some people to guess that acupuncture is in the “energy medicine” category rather than the “whole medical systems” category. Some topics list users might even guess that acupuncture is in the “mind-body medicine” category or the “manipulative and body-based practices” category. But with only five broad categories to look under, it should become quickly apparent where each individual therapy is classified. When we were unsure which area to classify a therapy under, we looked first at whether NCCAM had explicitly included a therapy under a specific category. If this was unclear, we looked at MeSH headings and NHS Evidence-CAM collection classifications for options.
Table 2 shows the fully expanded CAM Field topics list, which displays the topics and sub-topics, and the number of Cochrane reviews associated with each topic. To create subtopics, we relied upon some of the hierarchies we had observed in the NHS Evidence-CAM collection and the NLM MeSH headings. We tried to strike a balance between being overly broad (which would require the topics list user to search through too many reviews within a single topics heading to find the review of interest) and being overly specific (which would increase the possibility of misclassification of reviews and require duplicate entry of some reviews). For example, under “Natural Product Based Therapies” there is a heading for “Nutrition therapy”, and under that is the subcategory of “Dietary Supplements”. These dietary supplements have been categorized into types of supplements. “Vitamins” is one of these subheadings. We did not create further categories of Vitamin A, Vitamin B, Vitamin C etc, because if we had done so, we would have had to enter the review “Vitamin supplementation for preventing miscarriage”, which examined the effects of several different vitamin supplements, under multiple vitamin categories, or created the category ‘Other vitamins’ or ‘Multiple vitamins’. In contrast, most (though not all) herbal medicines are studied individually in Cochrane reviews, and therefore we created a discrete category for each commonly used herb for which a Cochrane review exists, and an ‘Other plants or plant extracts’ category for Cochrane reviews examining other herbs. Finally, systematic reviews that encompass multiple CAM therapies are classified under the subtopic of each therapy reviewed. For example, the review ‘Herbal medicine for low back pain’ reviews trials of devil’s claw, white willow bark, and cayenne, and therefore this review was classified under all three of these “herbal medicine” sub-topics. We hope our tree structure succeeds in striking the balance between breadth and specificity, and that it is comprehensive and user-friendly.
|▼Alternative Medical Systems (107)||►Other plants or plant extracts (29)|
|► Ayurvedic Medicine (4)||►Prolotherapy (1)|
|▼ Chinese Traditional Medicine (87)||►Speleotherapy (1)|
|► Chinese herbal drugs (87)||►Topical therapies (5)|
|►Homeopathy (12)||▼Unconventional synthetic drugs (1)|
|►Japanese traditional medicine(1)||►Laetrile|
|►Naturopathy (1)||►Procaine (1)|
|►Tibetan traditional medicine (2)|
|▼Energy Therapies (154)|
|▼Natural Product Based Therapies (354)||▼Acupuncture therapy (87)|
|►Chelation therapy (1)||►Acupressure (5)|
|►Hydrotherapy (3)||►Acupuncture (69)|
|▼Nutrition therapy (280)||►Electroacupuncture (6)|
|▼Diet therapy (22)||►Laser acupuncture (6)|
|►Calorie control or calorie restriction (2)||►Moxibustion (1)|
|►Carbohydrate-restricted diet (1)||▼Breathing exercises (0)|
|►Casein-free diets (1)||Qi gong (0)|
|►Fat-restricted diet (2)||►Distant healing (1)|
|►Gluten-free diet (1)||►Electric stimulation therapy (32)|
|►High-fiber diet (2)||►Magnetic therapy (11)|
|►Low glycemic-index diet (4)||►Phototherapy (7)|
|►Protein-restricted diet (3)||►Reiki therapy (2)|
|►Sodium-restricted diet (3)||►Therapeutic touch (3)|
|►Vegetarian or vegan diet (1)||►Ultrasonic therapy (11)|
|►Other diet therapies (2)|
|▼Dietary supplements (258)||▼Manipulative and Body-Based Methods (21)|
|►Amino acids (20)||►Alexander Technique (1)|
|►Enzymes and coenzymes (7)||►Chiropractic Manipulation/Spinal Manipulation (8)|
|►Fats (26)||Craniosacral Massage (0)|
|►Hormones (10)||Feldenkrais Method (0)|
|► Minerals (56)||►Massage (10)|
|►Probiotics (23)||Osteopathic Manipulation (0)|
|►Vitamins (76)||►Reflexology (2)|
|►Other supplements (39)|
|►Oxygen therapy (5)|
|►Ozone therapy (1)||▼Mind-Body Interventions (54)|
|▼Herbal Medicine (56)||►Biofeedback (3)|
|►African prune (1)||►Hypnosis (7)|
|►Artichoke leaf (1)||Imagery (0)|
|►Cayenne (1)||►Meditation (2)|
|►Cranberry (2)||►Play therapy (1)|
|►Devil's claw (1)||►Relaxation techniques (7)|
|►Echinacea (1)||▼Sensory art therapies (24)|
|►Feverfew (1)||►Aromatherapy (5)|
|►Garlic (5)||►Art therapy (1)|
|►Ginkgo biloba (6)||Color therapy (0)|
|►Horse chestnut (1)||►Dance therapy (2)|
|►Kava (1)||►Drama therapy (1)|
|►Milk thistle (1)||►Music therapy (14)|
|►Passiflora (1)||►Other sensory therapies (1)|
|►Saw palmetto (1)||►Tai Chi (5)|
|►St. John's wort (1)||▼Unconventional psychotherapies (1)|
|►Valerian (1)||►Morita therapy (1)|
|►White willow (1)||►Yoga (4)|
II. Remaining Challenges and Opportunities
As described above, we have included some nutrition and vitamin reviews (e.g., folic acid for neural tube defects) even though some people may consider these not to be CAM-related. Ideally we would have some way to mark such reviews, within the topics list, as possibly non-CAM, so that topics list users would be alerted to possible over-inclusiveness. Currently, however, this is not an option of the Cochrane topics list software. We hope that our operational definition, available on our website, will alert users to our desire to be over-inclusive rather than under-inclusive. We are open to revising our policies about nutrition and dietary supplement reviews in the future.
Placing therapies in more than one place in the topics list
We recognize that having therapies appear in only one place in the topics list might cause slight difficulty for users when they first approach the Field topics list and may have to look in more than one place before they find the therapy they are looking for. For example, under the heading ‘Chinese traditional medicine’, the only sub-topic listed is ‘Chinese herbal drugs’, but users may also expect to find reviews of acupuncture and tai chi listed under this heading. The addition of a scope note under ‘Chinese traditional medicine’ to indicate the location in the classification tree for acupuncture (i.e. under Energy therapies) and tai chi (i.e. under ‘Mind-body interventions’) would be helpful to include, once the Cochrane topics list software makes this possible. Alternatively, if the Cochrane topics list software could be automated to place a set of reviews in multiple places in the topics list, without reviews having to be manually added to each of the multiple locations, our concerns about errors and resource utilization would be eased, and we could easily see placing therapies under multiple categories in the topics list. For the present, we have decided to keep therapies in single locations, but concerns about usability make us open to revisiting the issue of placing individual therapies in multiple locations in the Field topics list in the future, even in the absence of changes to the topics list software.
We continue to debate how narrowly to define topics list subheadings. For example, we are unsure whether we should have subheadings for widely used herbal therapies even when there are no Cochrane reviews for the therapy. It might be frustrating for people to go to a category (e.g., a particular herb for which there are no reviews) and find that it is empty of reviews. If we do include empty therapy headings, how should we decide on which therapies are important to include in the list even without reviews? As noted above, for herbal medicine, we currently include only categories where there is at least one review. In other cases (e.g., Qi gong), we have listed categories that are empty because they are important classes of therapies rather than individual therapeutic agents. We tried to strike a balance, showing that we are aware of important therapies even in the absence of Cochrane reviews, and yet omitting multiple empty headings for individual therapies within other categories of therapy. However, we are willing to revisit this decision in the future, depending upon user feedback.
Using the Field topics list to identify gaps in Cochrane review evidence
A standardized operational definition of CAM and the development of a topics list of CAM-related Cochrane reviews present us with an opportunity to identify areas where Cochrane reviews are needed. That is, by comparing the lists of current Cochrane CAM reviews against the contents of the Cochrane CAM Field database of controlled trials we can spot condition/treatment pairings for which randomized trials have been published but no Cochrane review has yet been completed. As one example, there are currently only five Cochrane reviews on yoga in the topics list, but a search of the Cochrane database of controlled trials identifies 241 yoga controlled trials. We are beginning a project to identify therapies with randomized controlled trials but no reviews, and hope to develop a system that may be used to contribute to prioritizing future Cochrane reviews.
We do not believe that the Cochrane CAM Field operational definition of CAM is definitive. Indeed, we question whether it is possible to arrive upon a definitive set of therapies that are universally agreed upon as CAM. We suspect that there will never be universal agreement upon CAM aside from a core set of therapies, and that even this agreement will be susceptible to change over time. However, we are satisfied that our operational definition is transparent, and we have posted our operational criteria on the CAM Field website to maintain this transparency.
We are also satisfied with our methods for identifying CAM-related Cochrane reviews. Like our operationalization, it is a transparent method that tends to err on the side of over-inclusiveness rather than under-inclusiveness. We would rather provide information that not all persons consider to be CAM-related than omit information that a consumer may be seeking from our Field.
We are also satisfied with our identification of a popular and usable classification scheme, although we would like additional flexibility in placing reviews in multiple places and including scope notes as we feel that would further improve the usability of the topics list. Finally, we look forward to using the topics list together with our Field database of controlled trials to identify gaps in Cochrane reviews. We believe that the development and publishing of the Cochrane CAM Field topics list will both contribute to the identification and dissemination of the systematic review evidence on CAM and lead to identifying opportunities for advancing that evidence base.