![[Feedback]](/icons/banner/feedbackACT.gif){kind=icon}
![[For Subscribers]](/icons/banner/subscriberACT.gif){kind=icon}
![[Archive]](/icons/banner/archiveACT.gif){kind=icon}
![[Search]](/icons/banner/searchACT.gif){kind=icon}
![[Contents]](/icons/banner/tocACT.gif){kind=icon}
|
|
|
|
|
||
Center for Human Nutrition, The Johns Hopkins University, Bloomberg School of Public Health, Baltimore, MD 21205
2To whom correspondence should be addressed. E-mail: lcaulfie{at}jhsph.edu.
 |
ABSTRACT |
|---|
 TOP ABSTRACT LITERATURE CITED |
|---|
KEY WORDS: • targeting • nutrition program • diet • nutrition • risk • indicator
There is a long history of programs designed to improve the nutrition and the health of vulnerable populations in the United States and throughout the world. Increasingly, there is interest in evaluating the effectiveness of these programs to document impact and to improve performance, as well as pressure to provide evidence for continued or expanded funding. Early evaluations of nutrition programs (1,2) did not demonstrate the impact desired by many in the scientific and programmatic communities, and, since then, a literature has emerged to address many of the methodological issues that may explain less than desired impact. Principal among these issues has been the idea that nutrition programs could be more effective if they were better targeted so that they would reach only those individuals who truly need or would benefit from the services offered (3). This can be accomplished if information is available that adequately distinguishes among individuals for this purpose; that is, information that screens individuals and correctly classifies who should or should not receive program services. Screening for entry into nutrition programs based on income or nutritional risk is common, and, more recently, there have been reviews of the progress made to date on the use of screening indicators for one of the largest nutrition programs in the United States, the Women, Infants, and Children (WIC)3 Program. The purpose of this paper is to use this information to highlight key methodological issues for effectively targeting programs with nutritional outcomes.
The WIC program and the 1996 and 2002 Institute of Medicine reports
The Special Supplemental Nutrition Program for WIC began in 1972. The intent of the WIC program is to support and strengthen health care during critical periods of growth and development, to prevent health problems, and to improve health status. To that end, it provides supplemental foods, nutrition education, and health referral services to low-income pregnant or postpartum women, infants, and young children. Currently, it is estimated to serve about 8 million participants per month (4).
The WIC program is a grant program for which funding limits are set annually by the U.S. Congress. In addition to categorical and income eligibility criteria, individuals must show evidence of some form of nutrition risk based on either anthropometric, biochemical, medical, or dietary factors. These criteria allow for the prioritization of individuals based on health risk and the potential to benefit from the program in the event of limitations in funding. For example, low-income pregnant women and infants with anthropometric or hematological risk (e.g., low weight, low BMI, anemia) are given the highest priority, whereas low-income pregnant or postpartum women and low-income children who are at health risk due to a poor diet receive lower priority for WIC participation.
The performance of such a system rests on the quality of the information used to evaluate and classify individual women and children within the prioritization scheme. In the last 10 y, 2 separate Institute of Medicine (IOM) committees have evaluated the performance of the indicators used for this screening process (5,6). Here our focus is on issues raised in the second report regarding the ability to screen individuals with respect to the criteria, "dietary risk," which refers to "dietary deficiencies that impair or endanger health, such as inadequate dietary patterns assessed by a 24-h dietary recall, dietary history, or food frequency checklist" [Code of Federal Regulations Subpart C, Section 246.7(e) (iii) (2)]. Operationally, this has been extended to 2 certification criteria, "inadequate diet" and "failure to meet the dietary guidelines."
The nature of dietary risk
The construct, "dietary risk" is complex and not open to direct measurement. An individual’s risk results from multiple individual decisions made daily over extended periods of time. For example, whether an individual’s usual fat intake is low, average, or high depends to a great extent upon the frequency with which they have days of high-fat intake and that on many days, the fat intakes of these individuals will be similar. The nature of the decision process, the choices available, and multiple other influences lead to the fact that dietary intakes are inherently unstable from day to day within individuals, and, even in circumstances of reduced resources, dietary intakes are subject to substantial within-subject variability. Depending on the nutrient, within-subject variability may be 1–6 times greater than among-subject variability (7,8). Assessing the dietary intakes of children is further complicated, because dietary patterns change as children age, and many individuals (e.g., parents, child, daycare provider, school) usually determine a child’s dietary intake.
Two distinct approaches have been taken to deal with this variability and to characterize usual intake. First, one can collect multiple days of intake and average the data as an estimate of usual intake; the mean is considered to be an accurate (unbiased) estimate of one’s usual intake, with the variance of the data indicating the precision of the derived mean. Second, one can use an FFQ, in which the individual is asked to address the variability by summarizing their usual intake of specific food items, based on their knowledge of their day-to-day dietary choices. It should be noted that one can use either method to quantify usual intakes of foods or food groups to assess dietary patterns or adherence to the food guide pyramid or dietary guidelines; it seems, however, that within-individual variability in food intake is similar or greater that that for nutrient intakes.
Because of the inherent difficulty in estimating usual dietary intakes, scientists have also focused attention on the potential of what are called "behavioral indicators." Behavioral indicators do not measure diet per se but rather depict factors that influence food choices or dietary intakes (9). Such factors could include consumption of specific foods or groups of foods, specific meal patterns ("percent of meals consumed outside the home"), health-related behaviors, psychosocial characteristics, family food practices, and ecological factors related to the home, neighborhood, or community.
Indicator performance
Although one can identify a list of criteria with which to evaluate the usefulness of indicators for screening or other purposes (e.g., feasibility, cultural appropriateness), the hallmark criteria for indicator evaluation are accuracy and reliability (10). Accuracy addresses whether an indicator is really measuring what is intended vis-à-vis some "truth." Reliability refers to the reproducibility of the measure over time—that is, whether repeated measurements provide the same results.
Intraindividual variability in dietary intake is real, but its presence acts as a form of random error and diminishes the reliability of the measure of usual dietary intake. In the case of recall data, multiple recalls would provide an accurate estimate of usual intake, and information on the within-subject variance (sw2) or within-subject coefficient of variation (CVw) can be used in a common statistical formula to calculate the number of days (replicates) needed to attain a desired level of precision of that estimate (7,8). For example, for nutrients with CVw of 30–70%, 15–40+ replicates (days) would be needed to estimate mean intake with a precision of 20% around the true mean. Because the collection of this level of replication may not be feasible, an important judgment must be made with respect to the desired precision with which one would like to make critical decisions, for example, regarding an individual’s WIC eligibility.
These results highlight the potential advantage of FFQs, if they can provide valid and reliable information on an individual’s usual intake. For FFQs, reliability is expressed in terms of the reproducibility of the intake estimates when the FFQ is administered on multiple occasions, and accuracy is evaluated by comparisons of intakes with estimated intakes from multiple records or dietary recalls. As reviewed by the IOM (6), reliability coefficients for FFQ generally range from 0.7 to 0.9, and accuracy coefficients generally range from 0.3 to 0.7 across nutrients.
An emerging literature provides information on the reliability and the accuracy of these behavioral indicators for assessing dietary risk (6). Results vary widely across the indicators. For example, the reliability can be near 0.0 for some psychosocial indicators but be moderate (0.5–0.7) for specific food pattern indicators. The accuracy of these indicators varies as well, but few have accuracy coefficients > 0.30.
For screening purposes, the WIC professional wishes to estimate and to characterize the individual client’s usual nutrient or food intake and decide whether or not they are at dietary risk (eligible for WIC). Less than perfect measures of dietary risk result in misclassification of individuals regarding their "true" eligibility. The level of misclassification can be captured by the sensitivity (Se) and the specificity (Sp) of the indicator, with Se referring here to the ability of the indicator to identify those truly at dietary risk, and Sp referring to the ability of the indicator to identify those not at dietary risk. Less than perfect Se means that truly eligible individuals may not be classified as eligible and denied services, whereas less than perfect Sp means that others who are truly not eligible for the services may receive them. Walker and Blettner (11) calculated the probability of misclassification in dietary intake for a given level of error in estimated dietary intake. As shown in Table 1, for accuracy coefficients between 0.3 and 0.7, only 26 to 40% of individuals would classified into the correct quintile of intake.
|
These results indicate that, even with rigorous measurement techniques, there is substantial error in estimates of individual dietary intakes using current methods. As discussed, this error is attributed largely to the complex nature of the dietary behaviors. Unfortunately, it leads to high levels of misclassification of individuals with respect to WIC eligibility. It must also be considered that some errors are more costly than others. Whereas some may focus on the cost of ineligible individuals obtaining services, others would argue that the cost of missing individuals at risk is much higher, particularly in the long run. It is the task of policy makers and the public to decide how much and what type of misclassification error they are willing to tolerate when certifying individuals to receive or not receive federally funded WIC services; however, there is likely consensus that misclassification rates of >50% would not be tolerated.
The measures used to assess risk also serve as the foundation for the provision of WIC services, including package tailoring and individualized nutrition education or counseling. Given the level of error in current individual assessment methods for dietary intake, the foundation for effective nutrition education and counseling is not present. New methods or indicators are needed for screening and formulating education and counseling services.
This discussion has focused on making inferences about individuals. Indicators can be used, however, to make inferences about individuals or groups of individuals. This is an important distinction, because the level of precision (lack of random error) required to make inferences at the individual level is much greater than for making inferences at the population level. When making inferences about groups (or at the population level), it is still possible to derive unbiased estimates of the underlying phenomena of interest, using well-developed statistical procedures (12,13). For example, recalls or FFQs can be used to characterize the usual dietary intakes or food intake patterns of a target population, such as a WIC clinic area. This is true because even if individual values in the distribution are measured with error or are inherently unstable, these errors tend to cancel each other out, and individual values contribute information on the nature of the population distribution. Thus, with methods currently in use, the WIC program could characterize populations with respect to dietary risk and use the information to provide the context for education and counseling services. This was recommended by the second IOM committee (6), and, since then, efforts have begun to implement this shift in focus from individual to population.
|
FOOTNOTES |
|---|
3 Abbreviatons used: IOM, Institute of Medicine; Se, sensitivity (Se); Sp, specificity (Sp); WIC, Women’s Infants and Children.
|
LITERATURE CITED |
|---|
|
TOPABSTRACTLITERATURE CITED |
|---|
1. Beaton, G. H. & Ghassemi, H. (1982) Supplementary feeding programs for young children in developing countries. Am. J. Clin. Nutr. 35(suppl.):863-916.[Medline]
2. Abrams, B. A. (1993) Preventing low birth weight: does WIC work? A review of evaluations of the special supplemental food program for women, infants and children. Ann. N.Y. Acad. Sci. 678:306-316.[Medline]
3. Habicht, J. P. & Pelletier, D. L. (1990) The importance of context in choosing nutritional indicators. J. Nutr. 120:1519-1524.
4. United States Department of Agriculture. [Online] http://www.fns.usda.gov/pd/ [accessed April 2004].
5. Institute of Medicine (IOM) (1996) WIC Nutrition Risk Criteria: A Scientific Assessment. Food and Nutrition Board (FNB), IOM 1996 National Academy Press Washington, DC.
6. Institute of Medicine (IOM) (2002) Dietary Risk Assessment in the WIC Program. Food and Nutrition Board (FNB), IOM 2002 National Academy Press Washington, DC.
7. Willet, W. (1998a) Nature of variation in diet. Willet, W. eds. Nutritional Epidemiology. Monographs in Epidemiology and Biostatistics 15:33-49 Oxford University Press Oxford, UK. .
8. Nelson, M., Black, A. E., Morris, J. A. & Cole, T. J. (1989) Between- and within-subject variation in nutrient intake from infancy to old age: estimating the number of days required to rank dietary intakes with desired precision. Am. J. Clin. Nutr. 50:155-167.
9. Baranowski, T. (1996) Psychosocial and sociocultural factors that influence nutritional behaviors and interventions: Cardiovascular disease. Garza, C. Haas, J. Habicht, J.-P. Pelletier, D. eds. Beyond Nutritional Recommendations: Implementing Science for Healthier Populations 1996:163-188 Cornell University Ithaca, NY. .
10. Rothman, K. J. (1986) Modern Epidemiology 1986 Little, Brown Boston, MA.
11. Walker, A. M. & Blettner, M. (1985) Comparing imperfect measures of exposure. Am. J. Epidemiol. 121:783-790.
12. Traub, R. E. (1994) Reliability for the Social Sciences, Theory and Applications 1994 Sage Publications Thousand Oaks, CA.
13. Willet, W. (1998b) Corrections for the effects of measurement error. Willet, W. eds. Nutritional Epidemiology Monographs in Epidemiology and Biostatistics 15:302-320 Oxford University Press Oxford, UK. .
This article has been cited by other articles:
|
|
 |
|
  J.-P. Habicht and E. A. Frongillo Discussion: Targeting is Making Trade-offs J. Nutr., April 1, 2005; 135(4): 894 - 897. [Abstract] [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
