The percentage of Americans with incomes below the poverty threshold increased during the 1980s, and domestic hunger and food insecurity re-emerged as a social and political problem. The approaches that have been commonly used to estimate the prevalence of hunger and food insecurity have relied upon indirect indicators such as the percentage of the population in poverty and/or the percentage of individuals receiving food stamps. An evaluation by the General Accounting Office (1986) criticized these approaches for not measuring hunger directly.

Research was undertaken at Cornell University on the basis of the conviction that it was possible to measure hunger and food insecurity directly, although some have questioned this possibility (see, e.g., Margen and Neuhauser 1987). Because the phenomenon of hunger and food insecurity was not well understood, a naturalistic paradigm was chosen to construct a grounded theory or conceptualization of hunger and food insecurity that emerged from the expression of people who actually experience it (Radimer et al. 1990 and 1992). This research concluded that: 1) food insecurity is experienced differently at the household, adult and child levels; 2) food insecurity has four components; and 3) hunger is the most extreme consequence of the progression of food insecurity. Two components of food insecurity, quantity and quality of food, are related directly to food. Two components, certainty and acceptability, are psychological and social in nature. This theory is consistent with the literature on hunger and food insecurity (e.g., Maxwell and Frankenberger 1992) and with the commonly accepted Life Sciences Research Organization (1990) definition of food insecurity.

In January 1994, the Food and Nutrition Service of the U.S. Department of Agriculture (USDA) and the National Center for Health Statistics (NCHS) convened a one-day conference entitled "Food Security Measurement and Research Conference." The two objectives of this conference were: 1) to determine and recommend a current state-of-the-art survey instrument, and 2) to develop and recommend a comprehensive research agenda. A smaller workshop the next day focused on the design of "survey items for a questionnaire module to measure food security and its related elements." From this conference and workshop emerged a consensus that it was crucial to be able to measure and estimate prevalence for hunger and food insecurity in the U.S. In the next year, USDA, the National Center for Health Statistics (NCHS), and the Bureau of the Census together developed a module to measure hunger and food insecurity, implemented in the April 1995 Current Population Survey. The module was based upon questionnaire items that draw from the Radimer/Cornell measures, the Community Childhood Hunger Identification Project (CCHIP)⁴ surveys, previous USDA and NCHS surveys, and other sources.

An issue critical for ensuring the success of this effort has been to assess the validity of questionnaire items for measurement of hunger and food insecurity (Frongillo 1994). In our previous work to assess validity, items from the Radimer/Cornell questionnaire and the CCHIP questionnaire were identified that were strongly related at the group level to a number of social, economic and demographic factors (e.g., income, employment, education, use of food programs), and that yielded consistent prevalence estimates with a low rate of false negatives (Kendall et al. 1995, Olson et al. 1994, Radimer et al. 1992). Furthermore, we assessed whether traditional dietary measures of household food availability and individual food intake are indicative of hunger and food insecurity in a food-rich context (Kendall et al. 1996). This work established the validity of questionnaire-based measures of hunger and food insecurity to differentiate groups of households.

The objectives of this study were as follows: 1) to assess the validity of questionnaire-based measures in identifying households with hunger and food insecurity for the purposes of estimating prevalence, targeting and screening, and 2) to examine the interrelationships of the questionnaire-based measures of hunger and food insecurityRadimer/Cornell, CCHIP and the Third National Health and Nutrition Examination Survey (NHANES III)in order to strengthen our ability to interpret these available measures. We report on the rate of false positives (i.e., those reporting hunger and food insecurity but who did not actually experience it), and on the convergence of prevalence estimates based upon the questionnaire-based measures with prevalence estimates based upon a more definitive measure.

Addressing these objectives required the development of a more definitive measure of hunger and food insecurity that could then be used as a criterion against which to compare the questionnaire-based measures. This criterion was developed by examining the complete set of information available for each respondent in our 1993 survey. It was possible from this information to construct a definitive criterion measure for food insecurity at the household level, but not at the individual level.

SUBJECTS AND METHODS

A sample of ~200 households was desired, based upon a 1988 survey with a sample size of 189 using the Cornell measures that found significant relationships between the measures of hunger and food insecurity and its contributing factors. Disproportionate random sampling was done within 30 strata defined by socioeconomic status (6 groups) and age (5 groups) to ensure adequate representation of people who are most at risk. To allow for those who refused to participate or who could not be located, a pool of 639 potential subjects was selected from the health census. Fifty-two percent of the women (331) no longer lived at the address given during the health census and could not be located within the county. In the 10 most at-risk demographic strata, 66% could not be located, in the 15 moderately at-risk strata, 56%, and in the 5 least at-risk strata 31% could not be located. Of the remaining 308 women, 35% refused participation in the survey, resulting in a sample of 200 women. The refusal rate was 18% in the 10 most at-risk strata, 40% in the 15 moderately at-risk strata, and 32% in the 5 least at-risk strata. Because only seven of the 200 women fell into the youngest age group (15-19 y), they were dropped from the analysis, yielding a final sample of 193. 

Two interviews, separated by ~3 wk, were conducted with each respondent. During the first interview, a survey designed to elicit demographic information, information on factors contributing to food insecurity, use of coping strategies, and containing the Radimer/Cornell, CCHIP and NHANES III hunger and food insecurity items was administered to each study participant by trained field workers. All of the Radimer/Cornell items (Table 1) were expressed as statements, in line with recommendations that arose out of the previous survey using these items (Radimer et al. 1990). The possible responses to the Radimer/Cornell items were "not true," "sometimes true," or "often true." Any subject answering "sometimes true" or "often true" to any item within a measure was categorized as food insecure. CCHIP items (Table 1) were administered as questions with no/yes answers following the description in Wehler et al. (1992). The NHANES III item administered was the following: "Which one of the following statements best describes the food eaten by your family? Do you have: 1. enough food to eat, 2. sometimes not enough to eat, or 3. often not enough to eat" (Briefel and Woteki 1992).

Table 1. Radimer/Cornell and Community Childhood Hunger Identification Project (CCHIP) hunger and food insecurity items, with the percentage of the sample responding affirmatively to each [View Table]

During the first interview, questions about the frequency of fruit and vegetable consumption were administered, along with four questions about food consumption patterns indicative of disordered eating behaviors from the Stanford Eating Disorders Questionnaire (Agras 1987). Each subject's height and weight were measured during the first interview. At both interviews, a 24-hour recall was taken and household food stores were inventoried. The inventory tool contained 51 food items with four quantitative response categories, with 1 indicating that none of the food was present and 4 that a large amount was present. The response categories were determined based on the weight or volume of the item when purchased and judgments of meaningful differences that would differentiate those with depleted food stores from those with replete food stores.

The most definitive measure of hunger and food insecurity would be based upon an evaluation of a household's food situation through a personal, in-depth interview with a household member conducted by a skilled and knowledgable interviewer. The household's hunger and food insecurity status would be classified according to the following standard: Would a reasonable person conclude that the household was insecure, considering the generally accepted definition of food insecurity (certainty, acceptability, quality and quantity of food)? A skilled and knowledgable interviewer would also be able to determine who in the household was affected by hunger and food insecurity, and to what degree.

The data available from the 1993 Cornell survey provided the opportunity to develop an approximation to this most definitive measure. We adapted a consensus method used previously by the first author to approximate the clinical assessment of growth faltering in children (Frongillo et al. 1990). Two researchers with significant but very different experiences related to hunger and food insecurity examined the survey information available for the 193 households in the 1993 survey. One researcher, from a developing country, has studied the consequences of food insecurity and other factors on child growth and health. The other researcher, from the United States, has studied food insecurity among elderly persons and soup kitchens guests. Both researchers had access to responses to questions from the entire interview, except for the Radimer/Cornell, CCHIP and NHANES items. Extensive information on the use of food programs, sources of and expenditures on food and on other items, and income, were available from the survey, as were two 24-h dietary recalls and two household food inventories. After reviewing about 15 interview folders, one researcher selected all of the variables that were thought to be related to food insecurity and that could be easily deciphered if codes were printed out from a computerized file. The 24-h dietary recalls and answers to open-ended questions were also included.

Working independently, each researcher determined for each household whether or not it experienced hunger and food insecurity, while remaining unaware of the questionnaire-based items intended to measure hunger and food insecurity. The major positive and negative characteristics of food insecurity are presented in Table 2; starred characteristics were weighed more heavily. The order of characteristics corresponds to that found in the interviews and was followed by the researchers in evaluating the households. The 24-h recall information was evaluated last. For each household, the researchers determined whether or not the household was food insecure, based upon a cumulative consideration of the information available. No attempt was made to determine the severity of hunger and food insecurity or the level (i.e., household or individual, adult or child). After several households were classified, it was clear that three rather than just two categories were needed: definitely food secure, definitely food insecure, and probably food insecure. The last category was established for households that exhibited some characteristics of food insecurity, but had not clearly experienced food insecurity in the past year.

Table 2. The major positive and negative characteristics of food insecurity in the development of the definitive criterion measure [View Table]

Midway through the classification of the 193 households, the researchers discussed the interpretation of the information available (e.g., accuracy of food expenditure data) with the members of the Cornell Hunger and Food Insecurity Measurement Group. For the most part, the group agreed on their interpretations. After completing the classification process, the two researchers reviewed the information for each household for which they disagreed. For a few households, the apparent disagreement was due to a researcher's technical error in the use of the information. Otherwise, the two researchers discussed the households' situations and reached agreement on their classification.

RESULTS

Table 3. Sensitivity and specificity of questionnaire-based measures relative to the definitive criterion measure (n = 189) [View Table]

Table 4. Prevalence of hunger and food insecurity from questionnaire-based measures and definitive criterion measure [View Table]

Table 5. Agreement of Radimer/Cornell and Community Childhood Hunger Identification Project (CCHIP) measures for households in each category of the definitive criterion measure (n = 189) [View Table]

To better understand discrepancies among the measures, we examined households that were definitely food secure by the criterion measure but that were categorized as food insecure by both Radimer/Cornell and CCHIP. The two CCHIP items that were most frequently answered affirmatively by this group, more often than by the overall sample, were: "Does your household run out of money to buy food?" (72 vs. 35.8% for the whole sample) and "Do you ever rely on a limited number of foods to feed your children because you are running out of money to buy food for a meal?" (76 vs. 34%). The five Radimer/Cornell items that most frequently received affirmative answers among this group also occurred more often than in the whole sample: "The food that I bought didn't last and I didn't have money to buy more" (40 vs. 22%); "I ran out of the foods that I needed to put together a meal and I didn't have money to get more" (48 vs. 29%); "I worry about whether my food will run out before I get enough money to buy more" (68 vs. 38%); "We eat the same thing for several days in a row because we only have a few different kinds of food on hand and don't have money to buy more" (40 vs. 25%); and "I can't afford to eat properly" (36 vs. 23%). Four Radimer/Cornell items referred to household food insecurity. The last Radimer/Cornell item and one CCHIP item referred to food quality. Overall, these items seem to measure less severe food insecurity; thus it seems consistent that they would be more prevalent among households that the criterion measure assessed as not experiencing food insecurity.

Twenty-five households were categorized by the criterion measure as food secure, but as food insecure by both the Radimer/Cornell and CCHIP measures. In 18 of these households, the two researchers agreed that the household was not food insecure; 19 households had incomes > $20,000, and only one had an income < $10,000; 9 said they had a savings account; 13 mentioned they did not need to use a food program such as food stamps or a food pantry; 11 said they borrowed money for food; and 14 mentioned finances as a reason why in the past 12 mo they had considerably less food on hand. The higher incomes of this group, their savings and their tendency to state that they did not need food programs were instrumental in their being categorized as food secure by the two researchers. However, the households did report some problems with food in the survey information.

Of the four households categorized as definitely food insecure by the criterion measure but not by the Radimer/Cornell and CCHIP measures, all were also categorized as definitely food insecure by both researchers; three had incomes < $5,000 and one had an income between $5,000 and $10,000; all either borrowed money for food and/or used a food pantry; three had low and one had marginal food expenditures; all showed some unemployment; and three out of four were on public assistance. These characteristics led to their categorization as food insecure by the survey interview information.

Of the 12 households categorized as possibly food insecure by the criterion measure but as food secure by both the Radimer/Cornell and CCHIP measures, only four were categorized as probably or definitely insecure by both researchers; one of the researchers originally categorized the other eight as food secure; four had incomes < $10,000 and five had incomes between $10,000 and $15,000; six had low expenditures on food; in nine households, not everyone was covered by medical insurance; and 11 had at least two of the following characteristics: income < $10,000, unemployment, health insurance deficiencies and money borrowed for food. This group is more mixed, but again shows low income, food problems and unemployment. Absence of health insurance was considered important. Both researchers judged a household with low income and no health insurance as very vulnerable and likely to experience food-related problems at some time if they should experience any medical problems.

DISCUSSION

Both the Radimer/Cornell and CCHIP questionnaire-based measures had good specificity and excellent sensitivity when compared with the definitive criterion measure. For screening, in which the goal would usually be to identify any household that is at risk of hunger and food insecurity, excellent sensitivity is more important than specificity because further evaluation will identify any false positives (i.e., those who appear to have a food problem but do not). For targeting programs, the goal is usually to identify those subgroups of the population most at risk. Again, excellent sensitivity is more important than specificity, so that the subgroups of the population can be accurately ranked. For estimating prevalence in the population, both excellent sensitivity and specificity are desirable but are not necessary for achieving accurate estimates of prevalence so long as false positives and false negatives are consistently about equal in number, taking into account sampling variability (Habicht et al. 1982).

The estimates of the prevalence of household food insecurity from the definitive criterion, Radimer/Cornell without the uncertainty item and CCHIP measures were almost identical in this sample (45, 46 and 48%, respectively). The Radimer/Cornell measure with the uncertainty item resulted in a higher prevalence estimate of 53%. This result is consistent with the fact that neither the definitive criterion nor CCHIP measures had information to assess the uncertainty component.

For the assessment of household food insecurity, the overall agreement of the two questionnaire-based measures with multiple items (Radimer/Cornell and CCHIP) was very good (85%), and it was excellent for those households categorized as definitely food insecure.

The NHANES III item had excellent specificity but poor sensitivity, and consequently estimated a low prevalence of household food insecurity. Table 1 shows that the NHANES III item estimated a prevalence somewhat less than the prevalence estimated from the household-level quantitative and individual-level qualitative items in the Radimer/Cornell and CCHIP measures. This result does not mean that there is something wrong with the NHANES III item per se, but rather that no single item alone is sufficient for assessing hunger and food insecurity. Household food insecurity is understood to have a range in severity; the least severe form occurs when people experience uncertainty in obtaining food in socially acceptable ways, and the most severe form when people who simply do not get enough to eat because of insufficient resources experience the physical and psychological consequences of hunger. The most recent survey effort as part of the national nutrition monitoring system to determine the extent of hunger and food insecurity in the United States included a series of questions to assess this complex issue (Rose et al. 1995).

One of the strengths of this research was the ability to randomly draw a sample from a previous county census. However, about half of the sample drawn could not be located because they no longer resided in the county and were not part of the population under study. Of those still in the county, 35% refused participation. Nevertheless, a range of demographic characteristics was found in the analytic sample, allowing an effective assessment of the performance of the hunger and food insecurity measures in this population. Furthermore, because those who could not be located were more likely to be at risk of hunger and food insecurity (the percentage that could not be located was lowest in the highest socioeconomic strata used for sampling), the analytic sample likely produced a slight underestimation of sensitivity and an unbiased estimate of specificity. On the other hand, we found that the refusal rate was lowest in the most at-risk strata; for this reason, the analytic sample likely produced a slight underestimation of specificity and an unbiased estimate of sensitivity.

The results reported in this paper are directly applicable to counties similar to the one studied, but may not be as applicable to other counties. This county is typical of "rural-urban" counties in New York State (Eberts 1994). This type of county has a population of <200,000 people, the largest place (i.e., locality) is >10,000 people, and there is minimal commuting from the county to a large urban center. Counties of this type include one third of the population of the 44 rural counties of New York State. The rural counties of the United States to which these results are applicable are likely to be in the northern half of the country with a predominantly white population and some agricultural production. Further research of a similar nature with an urban population of mixed ethnicity is needed.

This paper presents the strongest evidence to date that questionnaire-based measures are valid for the assessment of hunger and food insecurity of households in the general population of rural families with children. Previous research had demonstrated validity for groups of households (Kendall et al. 1994 and 1995, Olson et al. 1994), but this research has demonstrated validity for individual households. These results provide strong support for using the items from the Radimer/Cornell and CCHIP measures for the purposes of estimating the prevalence of hunger and food insecurity in the U.S. population of rural families with children. Furthermore, the excellent sensitivity of these measures means that they can be validly used to screen households for hunger and food insecurity, and to target portions of the population for food programs.

ACKNOWLEDGMENTS