General Guidelines for Methodologies on Research and Evaluation of Traditional Medicine
(2000; 80 pages) [French] [Spanish] View the PDF document
Table of Contents
View the documentAcknowledgements
View the documentForeword
Open this folder and view contentsIntroduction
Open this folder and view contents1. Methodologies for Research and Evaluation of Herbal Medicines
Open this folder and view contents2. Methodologies for Research and Evaluation of Traditional Procedure-Based Therapies
Open this folder and view contents3. Clinical Research
Open this folder and view contents4. Other Issues and Considerations
View the documentReferences
Open this folder and view contentsAnnexes
Open this folder and view contentsAnnex I. Guidelines for the Assessment of Herbal Medicinesa
Open this folder and view contentsAnnex II. Research Guidelines for Evaluating the Safety and Efficacy of Herbal Medicinesa
Open this folder and view contentsAnnex III. Report of a WHO Consultation on Traditional Medicine and AIDS: Clinical Evaluation of Traditional Medicines and Natural Productsa
Open this folder and view contentsAnnex IV. Definition of Levels of Evidence and Grading of Recommendationa
Open this folder and view contentsAnnex V. Guidelines for Levels and Kinds of Evidence to Support Claims for Therapeutic Goodsa
Open this folder and view contentsAnnex VI. Guidelines for Good Clinical Practice (GCP) for Trials on Pharmaceutical Productsa
Close this folderAnnex VII. Guidance for Industry: Significant Scientific Agreement in the Review of Health Claims for Conventional Foods and Dietary Supplementsa
View the documentIdentifying Data for Review
Open this folder and view contentsAnnex VIII. Guideline for Good Clinical Practicea
Open this folder and view contentsAnnex IX. WHO QOL (Quality of Life) User Manual: Facet Definitions and Response Scalesa
View the documentAnnex X. Participants in the WHO Consultation on Methodologies for Research and Evaluation of Traditional Medicine
 

Identifying Data for Review

The first step in preparing or reviewing a health claim petition is to identify all relevant studies.

The types of studies considered in a health claim review include human studies and frequently also include “pre-clinical” evidence, e.g., in vitro laboratory investigations and other mechanistic studies. Studies of humans can be divided into two types: interventional studies and observational studies.

In an interventional study, the investigator controls whether the subjects receive an exposure or an intervention whereas in an observational study, the investigator does not have control over the exposure or the intervention. In general, interventional studies provide the strongest evidence for an effect.

Regardless of the inherent strengths and weaknesses of a study design, the overall quality and relevance of each individual study is paramount in assessing its contribution to the weight of the evidence for the proposed substance/disease relationship.

Interventional studies

The “gold standard” of interventional studies is the randomized controlled clinical trial.

In a randomized controlled trial, subjects similar to each other are randomly assigned either to receive the intervention or not to receive the intervention. As a result, subjects who are most likely to have a favorable outcome independent of any intervention are not preferentially selected to receive the intervention being studied (selection bias). Bias may be further reduced if the researcher who assesses the outcome does not know which subjects received the intervention (blinding). Randomized controlled clinical trials are not an absolute requirement to demonstrate significant scientific agreement in all cases, but are considered the most persuasive and given the most weight. A single large, well-conducted and controlled clinical trial could provide sufficient evidence to establish a substance/disease relationship, provided that there is a supporting body of evidence from observational or mechanistic studies.

Observational studies

There is no universally valid method for weighing categories of observational studies. However, in general, observational studies include, in descending order of persuasiveness, cohort (longitudinal) studies, case-control studies, cross-sectional studies, uncontrolled case series or cohort studies, time-series studies, ecological or cross-population studies, descriptive epidemiology, and case reports.

Observational studies may be prospective or retrospective. In prospective studies, investigators recruit subjects and observe them prior to the occurrence of the outcome. In retrospective studies, investigators review the records of subjects and interview subjects after the outcome has occurred. Retrospective studies are usually considered to be more vulnerable to recall bias (error that occurs when subjects are asked to remember past behaviors) and measurement error but are less likely to suffer from the subject selection bias that may occur in prospective studies.

• Cohort studies compare the outcome of subjects who have received a specific exposure with the outcome of subjects who have not received that exposure.

• In case-control studies, subjects with the disease are compared to subjects who do not have the disease (control group). Subjects are enrolled based on their outcome rather than based on their exposure.

• In cross-sectional studies, at a single point in time the number of individuals with a disease who have received a specific exposure is compared to the number of individuals without the disease who did not receive the exposure.

• Uncontrolled case series studies depict outcomes in a group without comparing to a control group.

• Time-series studies compare outcomes during different time periods, e.g. whether the rate of occurrence of a particular outcome during one five-year period changed during a subsequent five-year period.

• In ecological studies, the rate of a disease is compared across different populations. Investigators seek to identify population traits that may cause the disease.

• Descriptive epidemiology refers to study designs that assess parameters related to the frequency and distribution of disease in a population, such as the leading cause of death.

• Case reports describe observations of a single subject or a small number of subjects.

Research synthesis studies

“Research synthesis” studies, including meta-analyses, may be useful as supporting evidence for a health claim, but any role beyond this function is as yet unresolved.

The appropriateness of research synthesis studies to establish substance/disease relationships is not known. This is especially true when observational data are entered into meta-analyses. Discussions on the topic have been published1-4, and there are on-going efforts to identify criteria and critical factors to consider in both conducting and using such analyses, but standardization of this methodology is still emerging. Therefore, in general, such analyses serve as supporting evidence rather than as primary evidence. To date, while meta-analyses have been reviewed as part of the health claim authorization process, no health claims have been authorized on the basis of meta-analysis studies alone.

References

1) Sacks HS, Berrier J, Reitman D, Ancona-Berk VA, Chalmers T. Meta-analyses of randomized controlled trials. New England Journal of Medicine 1987; 316:450-455.

2) Sacks HS, Berrier J, Reitman D, Pagano D, Chalmers TC. Meta-analysis of randomized controlled trials: an update. In: Balder WC, Mosteller F, eds. Medical Uses of Statistics, 2nd ed. pp 427-442. Boston, MA: NEJM Books, 1992.

3) Sacks HS. Meta-analyses of clinical trials. In: Perman JA, Rey J, eds. Clinical Trials in Infant Nutrition, Nestle Nutrition Workshop Series, Vol 40, pp 85-99. Philadelphia, PA: Vevey/Lippincott-Raven Publishers, 1998.

4) Hasselblad V, Mosteller F, Littenberg B, Chalmers TC, Hunick MG, Turner JA, et al. A survey of current problems in meta-analysis. Discussion from the Agency for Health Care Policy and Research Inter-PORT Work Group on Literature Review/Meta-Analysis. Medical Care 1995; 33:202-220.

 

to previous section
to next section
 
 
The WHO Essential Medicines and Health Products Information Portal was designed and is maintained by Human Info NGO. Last updated: March 20, 2014