![]() |
|
|
Life Sciences Research Office, Bethesda, MD 20814
| INTRODUCTION |
|---|
|
|
|---|
To prevent the devaluation of scientific knowledge, the scientific debate about the current state of understanding of nature should be kept separate from the debate about political and social value. Within the current statutory, regulatory and legal framework in the United States, this can best be accomplished by a review of the scientific evidence supporting advertising and labeling claims, conducted by qualified scientists under the aegis of an independent, third-party organization. This must not be a political or social review, but a review of the current state of scientific knowledge. Regulators and policy makers can put this information into a political and social context, and balance the interests of industry and consumers.
Recognizing that the interests of the stakeholders are varied and often apparently conflicting, a conference1 was organized to gather a cross section of interested parties for the purpose of defining goals, standards and practices for a workable model for the review of the evidence substantiating foods and dietary supplement claims. The conferees came to general agreement on several underlying concepts: 1) there is a need for consistent, reliable and publicly available information about the safety and efficacy of dietary supplements; 2) there is a perception that many claims about foods and dietary supplements are not substantiated by sound, scientific evidence; 3) there is a need for more research into the safety and efficacy of foods and dietary supplements; 4) none of these issues are likely to be quickly resolved by relying only on the limited resources of the federal government; and 5) a properly structured and reliable third-party review could be in the best interests of all of the stakeholders.
These concepts, including the proposal for an independent scientific
review, were anticipated in the 1997 report of the Commission on
Dietary Supplement Labels (1)
.
The Commission recommends that the dietary supplement industry consider establishing an expert advisory committee on dietary supplements to provide scientific review of label statements and claims and to provide guidance to the industry regarding the safety, benefit and appropriate labeling of specific products. Such a committee might be supported by one or more industry trade associations or might be established as an independent entity funded by extramural grants and/or fees for services.
Although the recommendations of the Commission on Dietary Supplement Labels were limited to dietary supplements, they may equally apply to functional foods. The data used in substantiation, the endpoints and the wording of dietary supplements and functional food claims are often indistinguishable.
A properly designed and executed third-party review can accomplish the following:
This industry is based on the concept of consumer self care, i.e., consumers taking control of their own health care. What is more appropriate for the self-care industry than to institute a self-care program of third-party review?
There are many examples of effective third-party review within the
food and dietary supplement regulatory environment. Most of these have
reviewed various food ingredients to ensure that they were
Generally Recognized As Safe
(GRAS).2
The Flavor and Extract Manufacturers Association (FEMA) has sponsored
the review of the GRAS status of flavor and extract ingredients. From
1960 to 1991, FEMA Panels conducted over 1400 GRAS evaluations
(2)
. Between 1972 and 1982 at the behest of the Food and
Drug Administration (FDA), the Select Committee On GRAS Substances
(SCOGS) of the Life Sciences Research Office (LSRO), Federation of
American Societies for Experimental Biology independently reviewed the
GRAS status of over 450 compounds (3)
. Beginning in 1976,
the Cosmetics, Toiletries and Fragrance Association sponsored the
Cosmetic Ingredients Review (CIR) of the safety of compounds used in
the manufacture of cosmetics. CIR determined that of the 1043 cosmetic
ingredients reviewed, 591 cosmetic ingredients were safe (GRAS),
another 330 were safe with certain qualifications, 113 had insufficient
data to evaluate safety and 9 were deemed unsafe and were removed from
use (4)
. Since 1982, many manufacturers have constituted
independent expert panels to review the GRAS status of food ingredients
to support petitions for GRAS affirmation by the FDA. One indication
that the FDA has accepted third-party review of GRAS status may be
found in the proposed rule published by FDA in the Federal Register in
1997 to replace GRAS petitions with a GRAS notification process
(5)
. Under this proposed rule, the FDA will no longer
conduct a detailed examination of the data the notifier relied upon to
determine whether the use of a substance is GRAS; rather, the agency
will now evaluate only the notice summary to conclude whether there is
expert consensus of safety.
Various other expert panels have reviewed the safety and efficacy of
dietary supplements, in particular botanical products
(6
7
8
9)
. In general, these reviews have been published as
monographs providing a comprehensive overview of the scientific
literature about a class of substances for a variety of intended uses,
rather than focusing on the evidence substantiating specific claims for
specific products. Although these monographs provide valuable
supporting evidence, it is not clear how relevant they may be to
manufacturers and producers attempting to differentiate their product
from their competitors or to consumers and regulators attempting to
ensure the validity of a specific claim.
| Fundamental principles |
|---|
|
|
|---|
Review Organization and Expert Panel.
To be credible, the Review Organization (RO) should be independent, with no conflict of interest or bias. A knowledgeable, well-trained staff will be required to ensure that the process remains focused on the appropriate goals, adheres to the fundamental principles and delivers a consistent, high quality product. Reports written by RO staff rather than the expert consultants will help ensure a timely response. The LSRO is one example of an RO with the experience and qualifications to conduct reviews adhering to the fundamental principles outlined herein.
The success of any third-party review depends on the recognition that the experts reviewing the data are qualified by scientific training and experience to evaluate such claims and that their views are representative of the larger group of qualified experts in the field. For GRAS reviews in which a limited number of safety endpoints are evaluated, a review panel is typically comprised of three experts in toxicology with experience in GRAS standards. In contrast, there are potentially an infinite number of endpoints for efficacy reviews, requiring a broader base of expertise for effective evaluation. Thus, the composition of the Expert Panel must be flexible to accommodate the breadth of subject matter. Another factor important to the success of the review process is an understanding of the regulatory and legal context in which the relationships must be evaluated. This is especially important if the results of the review are to have value to the sponsor and to the appropriate regulatory and enforcement agencies. The utility of the review will be undermined by the application of inappropriate standards or injudicious wording of the final reports. The review panel should be drawn from a pool of experts with appropriate disciplinary expertise who are knowledgeable of the regulatory and legal context.
An optimal configuration of the Expert Panel may consist of a core group of 35 experts, knowledgeable in the regulatory context, and having expertise in such disciplines as nutrition, medicine, pharmacology, pharmacognosy, alternative medicine and toxicology. The core group may be augmented with ad hoc experts in the specific subject matter under review, as the need for augmentation is identified by the RO or the core group. The experts should be required to reveal any real or potential conflict of interest or bias. The RO should document that the process is free from any significant conflict or bias. Where bias is identified, the RO should take appropriate steps to balance the panel to minimize bias in the final report.
Sponsorship.
The reviews may be sponsored by government agencies, industry consortia, trade associations or individual commercial concerns. Most reviews will likely be sponsored by individual commercial concerns interested in making claims about product(s) that they produce, manufacture, package or market in the United States. The RO should conduct these reviews under a contractual arrangement that specifies the fundamental principles as outlined in this document. The RO should not accept sponsorship of reviews supporting safety or efficacy claims of competing products except when evaluated in comparison to the sponsors product.
Review of the evidence.
The reviews should embrace the totality of the evidence; anything less introduces the possibility of bias and undermines the utility of the review. The sponsors should bear the responsibility of providing all of the evidence of which they are aware that relates to the safety and efficacy of the intended use of the product. This evidence may include published and unpublished preclinical and clinical studies. The sponsors should affirm that the data submission is complete and accurate. Because this evidence may include sensitive information of proprietary nature, the RO must institute procedures maintaining the confidentiality of proprietary information. The RO should conduct an independent literature review under the direction of the Expert Panel to ensure that the information package is complete. The search strategy should be documented so that the methods are clear and could be replicated. The search may include data on closely related compounds with similar chemical structure or biological activity if such information is necessary to put into context the sponsors data. The review may include non-U.S. studies if, in the judgment of the Expert Panel, they are relevant to the target (U.S.) populations.
The evidence should be reviewed in a two-phase process. First, the strengths and weaknesses of the studies should be evaluated individually. To facilitate comparisons, the data should be abstracted by RO staff using a standard data abstraction form. Subsequently, the body of the evidence as a whole should be evaluated by the Expert Panel.
The individual studies should be evaluated for rigor of design,
appropriateness of methods and procedures, reliability of measures of
intake and outcome, sufficient statistical power, strength of
conclusions and comprehensiveness of reporting. The reviewers should
determine whether study populations match target audiences and whether
the standards of identity for the product allow differentiation of
claims. In general, the review of individual studies should follow
Section II. C. of the FDA guidance for industry (10)
. The
reader is referred to this document for a detailed explanation of
appropriate evaluation criteria.
The evidence should be evaluated within a predefined hierarchical
scheme setting forth rules for the persuasiveness of evidence upon
which the conclusions of the review are based. In general, studies
should be considered more persuasive if, by design and conduct, they
eliminate bias, control all but one variable and evaluate the substance
under the conditions of intended use. Utilization of a predetermined
evaluation scheme minimizes bias and preselection of data. The
hierarchical scheme outlined within the Section II. A. of the FDA
guidance for industry can serve as a useful guide for this process
(10)
. In brief, studies on human subjects will be accorded
greater weight than animal and in vitro (preclinical) studies, and
interventional (clinical) studies greater weight than observational
studies. The reader is referred to the FDA Guidance for further detail
(10)
.
Well-designed, randomized controlled trials (RCT) provide the most persuasive evidence of efficacy in human subjects and are strongly recommended to any potential sponsor. There may be circumstances when conducting RCT of sufficient statistical power or eliminating all forms of experimental bias is not practical. The lack of well-designed RCT should not disqualify a body of substantiating evidence. Qualified statements of efficacy are possible in the absence of well-designed RCT but must require other experimental evidence of a causal relationship in humans. In all circumstances, the evidence as a whole (including observational and preclinical studies) should be considered in the context of the body of relevant scientific literature to ascertain whether it supports an unqualified causal relationship between intake of a substance and a physiologic or structural outcome in humans.
Standards of identity.
The highest possible standards of identity should be presented. However, the standards of identity must be flexible to accommodate the range of products to be evaluated. For those complex matrices of botanical products in which an active component may not be identified, an operational standard may be employed (e.g., ground leaves of a defined plant harvested from a specific geographical region having a specified biological activity). Where possible, the standards of identity should specify the growth conditions, manufacturing processes, the reproducibility of production or manufacture and the formulation/manufacturing of final dosage forms. For those materials presenting only an operational standard of identity, the effect must be reproducible over multiple preparations. The highest possible standards of identity will provide the greatest scientific certainty in evaluation of the evidence.
Level of proof and the review process.
Determining the weight of evidence as a whole requires assessment of
the persuasiveness of each relevant study. Factors to be considered
should include the class of study design (RCT, observational,
preclinical), consistency of results across different studies and study
designs, consistency among various and within populations, magnitude of
effect, strength of association, dose-response relationships,
temporal relationships and biological plausibility or specificity of
effect (11)
. The assessment should be the application of
scientific judgment rather than meta-analysis, although
meta-analysis can be considered as supporting evidence. The
analysis should not be a numerical addition of studies for and against,
rather consideration for the persuasiveness of the individual studies.
In the final assessment, the relationship should be considered valid if
the evidence in support of the relationship outweighs the evidence
against it.
Although the wording of specific claims should not be the focus of a scientific review, an understanding of the potential claims the relationship might support will help define the relationship in question. The sponsors should be encouraged to submit clarifying language to define the scientific relationship they desire to be evaluated. The interpretation of the studies can be limited to only the research conducted and not extrapolated beyond the available evidence unless it is scientifically justifiable. Thus, if studies are in evidence that intake of substance X has an effect on physiologic state Y in population Q, then only that relationship is to be evaluated. If the relationship requested to be reviewed refers to population Z, in the absence of evidence that the results can be extrapolated from Q to Z, the relationship shall be considered proven for population Q, not Z. Should the evidence contain only in vitro studies, then only in vitro relationships shall be considered proven and not extrapolated to in vivo relationships. Stronger evidence should lead to more positive statements about the scientific relationships, which may, in turn, lead to more powerfully written claims.
As with the GRAS reviews, the Expert Panels should be encouraged to reach a consensus and avoid issuing minority opinions. This may be accomplished by first reaching agreement as to the specific nature of the relationship under evaluation, and then agreeing to one of a menu of prewritten statements as to the reliability of their conclusions. The statements should not attempt to address the levels of evidence contained in the legal provisions, such as "significant scientific agreement" or "competent and reliable." However, the Expert Panel should be cognizant of the government agencies standards. Written statements should be constructed so as to be recognizable and useful to those agencies. The conclusions should carefully describe the information and logic upon which they were formed.
Considerations of safety.
The consideration of safety for intended use can and should be a part
of an efficacy review. A substance cannot be effective if it is not
demonstrated to be safe. The review committee must ascertain that the
product is safe for the intended use of the claim. This can be
accomplished by a submission that a qualified, independent panel has
reviewed the evidence and determined the product is GRAS.
Alternatively, the sponsor may be afforded the option of contracting
the RO to review concurrently the evidence that the product is GRAS for
its intended use. Because the expertise and experience required to
review GRAS status differ from what is required to review efficacy, a
subcommittee of the Expert Panel or a separate committee should be
established to ensure appropriate coverage of the subject matter. GRAS
reviews should be conducted under the Proposed Regulations for GRAS
notification (5)
. If the sponsor declines the opportunity
to provide any evidence for safety, the conclusions of the efficacy
review report should contain language explicitly stating that such
evidence was not reviewed and thus the claims of efficacy must be
qualified. Under all circumstances, the level of assurance of safety
must be a part of the efficacy report conclusions. If the Expert Panel
uncovers information that raises concerns about safety, those concerns
must be transmitted immediately to the sponsor at the very least.
Publication of results.
The Expert Panels findings should be documented in a report and the report made publicly available within a reasonable period (no more than 1 y) after completion of the review. If possible, the report should be published in a peer-reviewed journal. Publication in peer-reviewed journals adds credibility to the report and its conclusions, provides added attraction to expert panel members and provides the public with the opportunity to review the supporting evidence, reasoning and conclusions. Although the publication may summarize confidential information so as not to compromise the proprietary interests of the sponsor, the report and the process should strive to be as open and transparent as possible. The report and the process should be reviewed by an independent panel to ensure that the standards of conduct and the quality of the review were appropriate.
Consultation and/or review.
The provision for an independent consultative process would be a useful adjunct to the review process outlined herein. During consultation, the sponsor would be able to submit a prereview package to ask whether it would likely be sufficient to substantiate some model claims. The RO would consult with the Expert Panel and advise the sponsors of the persuasiveness of the evidence, perhaps suggesting additional data that would be required under normal review processes. Prudent exercise of the consultative process can be in the best interests of all parties. It can help strengthen the substantiating evidence, stimulate additional research and raise the certainty of the conclusions about the relationship.
Consultation should not represent a commitment to undertake a formal review or what the conclusions of a formal review will be. The consultative advice should contain the proviso that a formal review of the totality of the evidence by the full Expert Panel may yet find the evidence unconvincing. However, it should also diminish the likelihood of reaching the conclusion that the evidence is insufficient and minimize reporting inconclusive or negative outcomes.
| Other issues |
|---|
|
|
|---|
A schematic representation of the proposed decision process and the
relationship between the Consultative Phase and Review Phase can be
found in Figure 1
. It is presented as a sequential process but not intended to imply that
the Consultative Phase is linked to the Review Phase. At several
different branches, the RO may find the evidence insufficient to
support a claim. The sponsor would then be afforded the option to
provide more information and continue the process. Should the sponsor
decide not to provide additional information during the Consultative
Phase, there is no provision for further action on the part of the RO.
However, should the sponsor decide not to provide further evidence
during the Review Phase, the RO should report its findings. The sponsor
should be afforded a finite period of time in which to respond. In the
absence of a timely response, the RO should report its findings. The
Review Panel must also report its findings should concerns be raised
about the safety of the product, regardless of the sponsors decision.
|
The RO faces several demanding challenges to make this process successful. The review process should be conducted in a timely and cost-effective manner without compromising the rigor and independence of the review. Providing for a Consultative Phase may help attain these goals. Consultation can be quicker and less costly than a formal review. The sponsors will be better equipped to decide whether the process will be cost effective and can minimize, but not eliminate the risk of inconclusive or negative outcomes. Start-up costs are a substantial part of the cost and time requirement of any independent review. The establishment of a core panel and scheduling of regular meetings can help minimize these costs.
For this process to be successful it must gain acceptance from all of the stakeholder communities. Acceptance will come over time as the stakeholders learn about the process, understand the basis for the decisions and find value in the reports. This will be aided by conducting the process as openly and transparently as possible, while balancing the sponsors proprietary interests and fostering an open and honest debate among the Expert Panelists. Many different processes have successfully maintained this balance. In the examples of the GRAS review processes described above, the approaches range from closed meetings, to closed deliberative meetings with calls for public input, to open deliberative meetings and the inclusion of ex officio members of the stakeholder communities. The bare minimum of openness is a public report of the findings containing clear justification for all conclusions.
The RO must strive to maintain its independence while accepting funding from the concerns that may profit from the results of the review. This can be accomplished only if the RO can demonstrate a history of freedom from conflicts of interest and a history of providing unbiased, authoritative reports regardless of the sponsors interests. The decisions made by the Expert Panel and the RO will often be at the boundaries of science and may not find universal acceptance. When knowledgeable scientists review the same set of data and come to different conclusions, they sometimes find it difficult to accept the opposing opinion as evidence-based and unbiased. The best defense against such attacks is to conduct the review under the highest standards, to reveal all possible perceptions of conflict at the outset and to justify the conclusions on sound scientific principles.
Motivational incentives for sponsors.
The third-party review process will be utilized infrequently if the sponsors cannot receive a return on their investment. There is limited patent protection and little provision for exclusive claims for foods and dietary supplements. This presents a challenge to those parties wishing to further consumer confidence and foster greater understanding of the effects of foods and dietary supplements.
Delaying publication of the third-party review report may afford the sponsor a temporary exclusivity. Sponsors may use the time between report completion and publication to gain a head start on their competitors. However, because publication of the results is crucial to the overall goals of the process, the length of the delay must be balanced against the interests of all the stakeholders. A period of 6 mo-1 y is suggested as a reasonable balance.
Certification marks may provide another form of claims exclusivity. The certification process should be developed by an independent authority and should include other elements, such as a review of the manufacturing principles or standards of identity. As discussed during the Third-Party Review Conference, a certifying authority could review the report of the RO for compliance to a set of standards and principles and authorize the sponsor to use the certification mark on labels and in advertising. The presence of the certification mark would indicate to consumers and health care professionals that the evidence in support of the claims has been reviewed by qualified, independent scientists. The certification mark could be applied only to the specific material for which the evidence was reviewed, thus providing an exclusive claim. The certifying authority and the review sponsor would have the right to enforce the exclusive use of the mark. A properly established certification mark can have a great effect in the marketplace and provide a return on the sponsors investment.
Dietary Supplement Health and Education Act makes provisions for the display and distribution to consumers of publications used in connection with the sale of dietary supplements. These publications are not considered labeling. An additional advantage accruing to the sponsor of a third-party review would be the ability to display the report of the third-party review in the stores in which the product is sold.
Summary.
The current statutory, regulatory and legal framework in the United States does not provide for the scientific review of the evidence substantiating all claims for food and dietary supplements. Moreover, political and scientific debates about the safety and efficacy of these items are often expressed in scientific terms even though the underlying state of scientific understanding does not support firm conclusions. A recently convened conference identified these shortcomings and came to a broad consensus that they could be redressed by the third-party review of the scientific evidence substantiating foods and dietary supplement claims. This report summarizes the suggested goals, standards and practices for a workable model for third-party review that was developed during this conference and an associated workshop of participating scientists. The model provides guidelines for the selection and evaluation of evidence, the level of proof to be employed, how the review organization and expert review should be structured, how standards of identity and issues of safety should be integrated into the review of efficacy claims and the fundamental principles that should be employed in the process. Using this model, a successful third-party review process will provide a sound, scientific basis for evaluating the evidence supporting food and dietary supplements claims and may help restore the value such claims were intended to convey.
| FOOTNOTES |
|---|
2 Abbreviations used: CIR, Cosmetics Ingredients Review; FDA, Food and Drug Administration; FEMA, Flavor and Extract Manufacturers Association; GRAS, Generally Recognized As Safe; LSRO, Life Sciences Research Office; RCT, randomized controlled trials; RO, Review Organization; SCOGS, Select Committee on GRAS Substances. ![]()
| REFERENCES |
|---|
|
|
|---|
1. Commission on Dietary Supplement Labels Report to the President, Congress, and the Secretary of the Department of Health and Human Services 1997 Office of Disease Prevention and Health Promotion Washington, DC.
2. Woods L. A., Doull J. GRAS evaluation of flavoring substances by the Expert Panel of FEMA. Regul. Toxicol. Pharmacol. 1991;14:48-58[Medline]
3. Life Sciences Research Office FASEB Evaluation of health aspects of GRAS food ingredients: lessons learned and questions unanswered. Fed. Proc. 1977;36:2519-2562[Medline]
4. Cosmetic Ingredients Review (2001) CIR 2001 Annual Report. Washington, DC.
5. U.S. Food and Drug Administration Substances generally recognized as safe; proposed rule. Fed. Reg. 1997;21:18937-18964
6. U.S. Pharmacopeia Botanical Monograph Series. Valerian. Drug Information for the Health Care Professional 1998 USPC Rockville, MD.
7. U.S. Pharmacopeia Botanical Monograph Series. Ginger. Drug Information for the Health Care Professional 1998 USPC Rockville, MD.
8. U.S. Pharmacopeia Botanical Monograph Series. Comfrey. Drug Information for the Health Care Professional 1997 USPC Rockville, MD.
9. Blumenthal M. The complete German Commission E mongraphs: therapeutic guide to herbal medicines 1998 American Botanical Council Austin, TX.
10. Food and Drug Administration, Center for Food Safety and Applied Nutrition Office of Special Nutritionals. Guidance for Industry: Significant Scientific Agreement in the Review of Health Claims for Conventional Foods and Dietary Supplements 1999 U.S. Food and Drug Administration, Center for Food Safety and Applied Nutrition Rockville, MD.
11.
Ahrens E. H., Jr Symposium. The evidence relating six dietary factors to the nations health: consensus statement. Introduction. Am. J. Clin. Nutr. 1979;32:2627-2631
This article has been cited by other articles:
![]() |
C. Duggan, J. Gannon, and W A. Walker Protective nutrients and functional foods for the gastrointestinal tract Am. J. Clinical Nutrition, May 1, 2002; 75(5): 789 - 808. [Abstract] [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||