![[Feedback]](/icons/banner/feedbackACT.gif){kind=icon}
![[For Subscribers]](/icons/banner/subscriberACT.gif){kind=icon}
![[Archive]](/icons/banner/archiveACT.gif){kind=icon}
![[Search]](/icons/banner/searchACT.gif){kind=icon}
![[Contents]](/icons/banner/tocACT.gif){kind=icon}
|
|
|
|
|
||
3 Center for Evaluation Research and Surveys and 4 Center for Nutrition and Health Research, National Institute of Public Health, Colonia Santa María Ahuacatitlán, CP 62508, Cuernavaca, Morelos, Mexico
* To whom correspondence should be addressed. E-mail: neufeld{at}insp.mx.
 |
ABSTRACT |
|---|
 TOP ABSTRACT IntroductionMethodsResultsDiscussionLITERATURE CITED |
|---|
|
Introduction |
|---|
|
TOPABSTRACTIntroductionMethodsResultsDiscussionLITERATURE CITED |
|---|
According to a national probabilistic survey conducted in Mexico in 1999, 17.7% of children <5 y old were stunted (height-for-age < –2 SD). However, linear growth stunting associated with poor nutrition was clearly a problem of the poor: the prevalence of the stunting in the lowest socioeconomic decile (47.6%) was 10 times higher than in the top decile (5). Even though improvements have occurred in recent years, undernutrition continues to be an important public health problem among the poor in Mexico (6).
In 1997, the Mexican federal government launched PROGRESA (Program for Education, Health and Nutrition; now called Oportunidades), a large-scale poverty alleviation program. Oportunidades provides conditional cash transfers and includes a strong nutrition component. The program began operating on a small scale in rural areas and was expanded to urban areas in 2002. It has now enrolled its target population size: 5 million households in all regions of the country. The program has been shown to be associated with better growth and a reduced prevalence of anemia in infants in rural areas (7). In this article, we report the impact of Oportunidades on child growth in urban Mexico. Documenting the program's impact in urban areas is important because living conditions in urban vs. rural settings differ greatly in Mexico. As a consequence, the socioeconomic and environmental determinants of growth retardation are different [see for instance Menon et al. (8)]. Additionally, the prevalence of stunting is lower in urban than in rural Mexico (5,6). A program that effectively improves child growth in rural areas may not necessarily have a similar impact in urban areas. Evaluating the impact of Oportunidades in urban areas is thus essential to estimate the potential impact of the program on child growth in the Mexican population.
|
Methods |
|---|
|
TOPABSTRACTIntroductionMethodsResultsDiscussionLITERATURE CITED |
|---|
Through the program, beneficiary families receive a monthly cash transfer intended to improve their food purchasing power. The program also provides milk-based micronutrient fortified foods targeted to children 6–23 mo of age, to children 2–4 y of age with low weight (weight-for-age < –1 SD), and to pregnant and lactating women. The food which is prepared as a porridge provides the daily recommended dietary allowance of zinc, iron, and a number of essential vitamins and 
20% of the energy needs for children <2 y of age. The program's health component includes the provision of a number of obligatory preventive and primary curative health services. These include immunization of children, growth monitoring for children 0–60 mo of age, and prenatal and postpartum care visits, among others. Household members 15 y and older (depending on theme to be discussed) attend monthly meetings at which health, hygiene, and nutrition issues are discussed. Program benefits are conditional on regular attendance at all services appropriate for household demographics. Households are disenrolled when they have missed 3 consecutive health visits or health education sessions. Additionally, scholarships, prorated for male vs. female students (higher for female) and grade level, are provided, conditional on enrollment and regular attendance at school. The female heads of household receive all program benefits, recognizing the potential of women to use the resources effectively and efficiently to cover the needs of the family. Typically, households receive the equivalent of 32.5 to 41.3 US dollars, constituting 19 to 24% of mean household consumption. Households remain in the program for 3 y, after which program eligibility is reevaluated (9–13).
The program employs a 2-stage approach to enroll households in urban areas (50,000 to 1 million inhabitants) into the program. Oportunidades was first targeted to urban centers with a high density of low-income households. The cutoff for eligibility was based on a discriminant analysis of the 2000 National Household Income and Expenditure Survey. The discriminant analysis was based on socioeconomic characteristics [see Coady and Parker (14) for details]. The estimated cutoffs were then applied to the 2000 national census to identify the poorest urban blocks. Within each block, the poorest households were targeted.
Mass media were used to advertise the program and invite families to solicit a socioeconomic screening questionnaire at the enrollment centers. The assessment utilized household assets, housing quality, years of education, and household composition. Program staff visited all households that appeared eligible to validate the results of the screening questionnaire. Eligible applicants subsequently had to return to the program office to register in the program. Household enrollment in urban areas started in 2002 and was staged over 2 y.
One of the unique aspects of Oportunidades is the inclusion of impact evaluation within the program mandate. The National Institute of Public Health conducted the baseline survey from September to December 2002 and the follow-up survey from July to November 2004. For the evaluation sample, 149 manzanas (the smallest administrative unit within an urban area) in 17 of Mexico's 31 states were selected through probabilistic stratified sampling from the pool of localities where Oportunidades would be implemented in 2002. The survey included eligible households that enrolled and eligible households that did not enroll in the program. Program enrollment was positively associated to the potential cash transfer households could receive but not to having a child eligible to receive the nutrition supplement (15). No households had actually begun to receive benefits at the time of the baseline survey.
Data collection. Both at baseline and follow-up, child and maternal anthropometric data were collected, as were socioeconomic data. In 2002 anthropometric measurements were obtained for children younger than 4 y old and their mothers. In 2004 follow-up anthropometry was obtained for all children 2 to 4 y of age (i.e., <24 mo of age in 2002) and their mothers. Weight and recumbent length (in children <24 mo) or standing height (24- to 48-mo-old children and mothers) were measured using standard anthropometric methods (16) by highly trained and standardized anthropometrists (17). Weight was measured to the nearest 20 g with an electronic scale (Tanita Mother-Baby scale, model 1582, Tanita Corp.), and length or height was measured to the nearest mm with a portable infantometer/stadiometer (Schorr Industries). Anthropometric Z-scores were calculated using the new WHO growth reference standards (18). The socioeconomic data included, among other things, sociodemographic characteristics, educational level, employment, housing characteristics, and common domestic goods. Data were collected by means of fieldworker-administered surveys. Fieldworkers were extensively trained.
The survey included eligible households that enrolled and eligible households that did not enroll in the program. Program enrollment was related to both household and program characteristics (14). Households were considered to be enrolled in the program if both the self-report and the administrative record identified the household as such. Likewise, comparison households were those who were listed as not enrolled in the program according to administrative records and self-report. Approximately three-quarters of all households (73.4%) were consistently classified as either intervention or comparison households.
In each household, written informed consent for participation was obtained from the mother or the self-identified decision maker. The protocols governing the data collection were approved by the Research, Biosafety and Ethics Commissions of the National Institute of Public Health in Mexico.
Data analysis. The baseline characteristics of the children in the intervention and comparison households were compared using the Student's t test for means of the symmetrically distributed variables, the chi square test for proportions, and the Wilcoxon-Mann-Whitney test for the distribution of skewed variables. The impact of Oportunidades on child linear growth was assessed using the change in the child's absolute length (in cm) and height-for-age Z-score (HAZ)5 from baseline to follow-up. To evaluate the impact on weight gain, we used the difference in absolute weight (in kg) and in weight-for-height Z-score (WHZ). The socioeconomic status (SES) of each household was estimated using the program's summary variable of eligibility from the 2002 data. It is important to note that all households in the sample were within the poorest 20th percentile of the Mexican population. The tertile classification used in this analysis provided a relative measure of socioeconomic well-being of each household compared with the other households in the sample, without reference to the broader society.
We estimated the impact of the program using difference-in-difference propensity score matching. This statistical procedure compares the change over time in the outcome for the intervention households with the change over time in matched comparison households. This method controls for the household's self-selection into the program, i.e., for systematic differences between households that may be related to the outcome (19). We matched comparable households in the intervention and comparison groups. Rather than matching on all relevant socioeconomic characteristics (which is technically very challenging with a high number of covariates), matching was based on the household's propensity score. This score is the household's estimated probability of enrollment in the program given a number of relevant observed characteristics. We estimated the propensity score using a logit model, including such socioeconomic variables as housing characteristics, household assets, household composition, schooling, employment status, and income. To evaluate whether the matching procedure rendered the observable differences between intervention and comparison households insignificant, we checked the "balancing" of all variables used to estimate the propensity score. The balancing tests showed that there were no statistically significant differences between the matched intervention and comparison households (20).
In addition to the propensity to enroll in Oportunidades, we matched children in intervention and comparison households on characteristics strongly associated with growth, specifically maternal height, child sex and age, and child length or weight at baseline. The matching estimator was implemented using Abadie and Imbens' (21) method, which is available as part of the Stata package (StataCorp, version 8.2). This procedure allows households to be used as a match more than once. Each intervention child was matched to 5 comparison children. No households were excluded in the matching procedure. The bias arising from inexact matches was corrected by means of Abadie and Imbens' bias correction procedure. The method uses a Z-test to determine statistical significance.
The association between program participation and child growth was estimated for all children younger than 24 mo of age at baseline and then stratified by age group (children younger than 6 mo, between 6 and 12 mo, and between 12 and 24 mo) and by socioeconomic tertile. We conducted the analysis based on an a priori hypothesis that the program has a greater potential to impact child growth when implemented at a young age (21) and on previous evidence of age and SES-specific impacts of the same program in rural areas (7,22). The program's expected positive impact on child growth provided a clear a priori direction for hypothesis testing. Given the hypothesis that growth would improve as a consequence of the program, statistical theory dictated the use of 1-sided tests (23). Thus 1-sided tests were reported. Although we based our conclusions on the 1-sided tests, 2-sided tests were also conducted. P-values < 0.05 were considered statistically significant.
|
Results |
|---|
|
TOPABSTRACTIntroductionMethodsResultsDiscussionLITERATURE CITED |
|---|
|
|
|
|
None of the conclusions with respect to statistical significance changed when double-sided tests were used, apart from HAZ in children younger than 6 mo in 2002 (P = 0.07).
|
Discussion |
|---|
|
TOPABSTRACTIntroductionMethodsResultsDiscussionLITERATURE CITED |
|---|
Interestingly, the magnitude of the impact of Oportunidades in urban areas was relatively similar to that in rural areas. In rural Mexico, a significant impact of the program was found on child growth in children younger than 6 mo at baseline living in the poorest households (7). Unfortunately, the urban study did not have the statistical power to test for an interaction between age and SES. The smaller effect size reported in rural areas (1.1 cm vs. 1.5 cm after 2 y) was most likely due to an underestimation of the true impact. In rural areas, children in the comparison group had received the program for at least 1 y when the impact on growth was assessed, because the comparison group was included in the program 1 y earlier than planned. The early inclusion may further have masked a program impact in children 6 to 12 mo of age in rural areas.
A part of the effect of the program on absolute weight gain of children 0 to 6 mo of age is due to improved linear growth. The impact on child WHZ, however, shows that these infants in the intervention households are gaining weight beyond what can be accounted for by linear growth. This is especially worrisome as there is no indication that children in this sample were wasted at baseline (mean WHZ at baseline +0.31). To our knowledge, there is no clear evidence that greater weight gain of the magnitude reported here at this young age is directly related to excess weight in later childhood. Nonetheless, given the increasing problem of overweight in Mexican children, further studies are currently under way to determine the impact of the program on dietary patterns of small children and also whether regular consumption of the fortified food supplement according to program guidelines may lead to excess weight gain.
In the past, it was shown that the systems for the delivery of health services and program benefits in developing countries were often deficient. Additionally, service and program utilization was found to be inadequate in the poorest of the poor (24,25). Although all Oportunidades beneficiaries were poor compared with the entire Mexican population, there was still a broad socioeconomic variation within the sample. The large program impact in the poorest households shown in this analysis suggests that these barriers to participation by the poorest beneficiaries may not exist in the Oportunidades program, at least not among the young children where growth outcomes were assessed. This may be related to the conditional nature of program participation in Oportunidades. The requirements imposed related to the use of preventive health services may encourage the poorest families to overcome barriers that have limited program participation in the past. Future evaluations of Oportunidades should attempt to document whether this is the case.
A potential limitation of this study is that the comparison group for this study consisted of eligible households that did not enroll in the Oportunidades program but lived in the same communities as the enrolled households. The selection of households could thus have biased the results if enrollment was related to unobserved characteristics that simultaneously determined child growth, such as maternal attitudes and behavior. We studied the mean attained height at baseline of children over 2 y of age. At baseline, differences in height between children in the intervention and comparison households can be attributable to individual nutrition and health factors that influence growth, but not to the Oportunidades program. We compared mean attained height at baseline of children 24 to 48 mo of age and found no significant difference between children from households who enrolled in the program and those who did not (our comparison group). We conclude that it is very unlikely that our findings suffer from selection bias related to factors that predict both child growth and program participation.
Another possible source of selection bias is the loss to follow-up. No differences were found, however, in any of the baseline characteristics between children lost to follow-up and children included in the analysis. Additionally, loss to follow-up was similar in both intervention and comparison groups. To avoid error due to misclassification of status of incorporation in the program, we used only those children for whom self-report and program records agreed. To evaluate possible selection bias, we compared children excluded because of inconsistent program classification and children included in the analyses on all baseline characteristics and SES. The only statistically significant difference was a lower weight (259 g) in the excluded group, but this difference disappeared when comparing baseline WHZ. HAZ was marginally lower in the excluded children. We do not believe that the exclusion of observations caused any bias in the results.
The use of longitudinal data are an important strength of this study. The difference-in-difference approach we used controls for the effects of unobserved time-invariant differences between children in intervention and comparison households. Propensity score matching was used to control for the potentially confounding effects of variables associated with self-selection into the program. A key advantage of this method (as compared with regression models) is that no assumptions are made about the functional form of the association between confounders and the outcome variable (26). Additionally, the balancing of the variables used to estimate the propensity score ensures that the matched intervention and comparison children, apart from their participation in the program, were comparable on all observable variables (20). One can never completely exclude the possibility that statistical findings are due to chance. We believe, however, that this is unlikely in our study, given the evaluation design, the analytical approach, and the consistency of our findings with those in rural Mexico and with the literature on child growth.
At this time, we do not know which component or components of the program are most likely responsible for the impact on child linear growth. The growth monitoring and health and nutrition education components of the program are obligatory and hence compliance has been well above 90% (15). The impact of the program on child linear growth may be the result of micronutrient (particularly zinc) intake from the fortified food supplement. We have reported previously, however, that consumption of the fortified food is far below desirable in beneficiary children 6–23 mo of age (15). Child growth may also have benefited from improvements in the quality of the home diet or may be related to improved health care, which possibly reduced the frequency and/or severity of infectious disease. We therefore believe that the impact on linear growth is likely due to a combination of influences and not specifically attributable to the fortified food alone.
With respect to the impact on weight gain, preliminary analysis of dietary intake suggests that in those who consumed the fortified food supplement regularly, micronutrient but not energy intake was considerably higher than in those who did not consume the food (27). Preliminary analyses suggest that this pattern does not differ by age group. The income transfer from the program may have resulted in higher energy intake from the home diet in young children. Studies are underway to further explore these pathways.
Although the urgent need for effectiveness evaluations of large-scale health and nutrition programs has been identified (28,29), few such studies have been reported in scientific literature. The randomized design used for the impact evaluation in rural areas was abandoned in urban areas for political reasons. Communities with the highest density of poor households were enrolled first. The use of quasi-experimental methods with appropriate matching techniques still allowed for a rigorous impact evaluation design.
Conditional cash transfer programs have now been implemented in many countries in the region, including Nicaragua, El Salvador, Brazil, Honduras, among others. Although many have begun or are planning evaluations, little has been published to date on their impact. In Nicaragua participation in the Red de Protección Social improved HAZ by 0.17 SD in children under 5 y of age (27). The Honduran Programa de Asignación Familiar did not have any impact on child growth, but this may be explained by the relatively small transfer and very low coverage of the program in eligible communities (28,29). Little is known about the impact of these programs on the nutritional status of children in poor urban households. This study shows that Oportunidades, a conditional cash transfer program with a strong nutrition component, effectively improves growth of poor children in urban areas of Mexico. The documentation of the program's impact is extremely important to providing evidence that the program is meeting its objectives in relation to improving child nutritional status and to justifying its continuation and expansion. The results reported here reemphasize the importance of designing public health interventions with a built-in evaluation component.
|
FOOTNOTES |
|---|
2 Author disclosures: J. L. Leroy, A. García-Guerra, R. García, C. Dominguez, J. Rivera, and L. M. Neufeld, no conflicts of interest.
5 Abbreviations used: HAZ, height-for-age Z-score; WHZ, weight-for-height Z-score; SES, socioeconomic status.
Manuscript received 3 October 2007. Initial review completed 13 November 2007. Revision accepted 21 January 2008.
|
LITERATURE CITED |
|---|
|
TOPABSTRACTIntroductionMethodsResultsDiscussionLITERATURE CITED |
|---|
1. United Nations System Standing Committee on Nutrition. 5th report on the world nutrition situation: nutrition for improved development outcomes. Geneva: United Nations System Standing Committee on Nutrition; 2004.
2. Pelletier DL, Frongillo EA Jr, Habicht JP. Epidemiologic evidence for a potentiating effect of malnutrition on child mortality. Am J Public Health. 1993;83:1130–3.
3. Fishman SM, Caulfield LE, de Onis M, Blossner M, Hyder AA, Mullany L, Black RE. Childhood and maternal underweight. In: Ezzati M, Lopez AD, Rodgers A, Murray CJL, editors. Comparative quantification of health risks: the global and regional burden of disease attributable to selected major risk factors. Geneva: World Health Organization; 2004. p. 39–161.
4. United Nations Administrative Committee on Co-ordination. Sub-committee on Nutrition, International Food Policy Research Institute. Fourth report on the world nutrition situation: nutrition throughout the life cycle. Geneva: United Nations; 2000.
5. Rivera JA, Sepulveda Amor J. Conclusions from the Mexican National Nutrition Survey 1999: translating results into nutrition policy. Salud Publica Mex. 2003;45: Suppl 4:S565–75.[Medline]
6. Olaiz Fernández G, Rivera Dommarco J, Shamah Levy T, Rojas R, Villalpando Hernández S, Hernández Avila M, Sepulveda Amor J. Encuesta Nacional de Salud y Nutrición 2006. Cuernavaca: Instituto Nacional de Salud Pública; 2006.
7. Rivera JA, Sotres-Alvarez D, Habicht JP, Shamah T, Villalpando S. Impact of the Mexican program for education, health, and nutrition (Progresa) on rates of growth and anemia in infants and young children: a randomized effectiveness study. JAMA. 2004;291:2563–70.
8. Menon P, Ruel MT, Morris SS. Socio-economic differentials in child stunting are consistently larger in urban than in rural areas. Food Nutr Bull. 2000;21:282–9.
9. Secretaria de Desarrollo Social. Reglas de Operación del Programa de Desarrollo Humano Oportunidades; 2006.
10. Skoufias E. PROGRESA and its impacts on the welfare of rural households in Mexico. Washington, DC: International Food Policy Research Institute (IFPRI); 2005. Report No.: 0896291421.
11. Angelucci M, Attanasio OP, Shaw J. El efecto de Oportunidades sobre el nivel y la composición del consumo en areás urbanas. In: Hernández Prado B, Hernández Avila M, editors. Evaluación externa de impacto del Programa Oportunidades 2004 Tomo IV: aspectos económicos y sociales. Cuernavaca: Instituto Nacional de Salud Pública; 2005.
12. Programa de desarrollo humano oportunidades. Reglas de operación 2007. Mexico: Secretaría de desarollo Social (Programa Oportunidades); 2007.
13. Gertler P, Fernald L. The medium term impact of OPORTUNIDADES on child development in rural areas. Cuernavaca, Mexico: Instituto Nacional de Salud Publica; 2004.
14. Coady DP, Parker SW. Program participation under means-testing and self-selection targeting methods. FCDN Discussion Paper 117. Washington, DC: International Food Policy Research Institute (IFPRI); 2005.
15. Leroy JL, Vermandere H, Neufeld LM, Bertozzi SM. Improving enrollment and utilization of the Oportunidades program in Mexico could increase its effectiveness. J Nutr. 2008;138:638–41.
16. Lohman TG, Roche AF, Martorell R. Anthropometric standardization reference manual. Champaign, IL: Human Kinetics Books; 1988.
17. Habicht JP. [Standardization of quantitative epidemiological methods in the field.] Bol Oficina Sanit Panam. 1974;76:375–84.[Medline]
18. WHO Multicentre Growth Reference Study Group. WHO Child Growth Standards based on length/height, weight and age. Acta Paediatr Suppl. 2006;405:76–85.
19. Dehejia RH, Wahba S. Propensity score-matching methods for nonexperimental causal studies. Rev Econ Stat. 2002;84:151–61.
20. Caliendo M, Kopeinig S. Some practical guidance for the implementation of propensity score matching. IZA Discussion Paper 1588. Bonn, Germany: Institute for the Study of Labor; 2005.
21. Abadie A, Imbens G. Large sample properties of matching estimators for average treatment effects. John F. Kennedy School of Government, Harvard University & Department of Economics, and Department of Agricultural and Resource Economics, University of California at Berkeley; 2004.
22. Schroeder DG, Martorell R, Rivera JA, Ruel MT, Habicht JP. Age differences in the impact of nutritional supplementation on growth. J Nutr. 1995;125:1051S–9S.
23. Snedecor GW, Cochran WG. Statistical methods. 8th ed. Ames: Iowa State University Press; 1989.
24. Bryce J, el Arifeen S, Pariyo G, Lanata C, Gwatkin D, Habicht JP. Reducing child mortality: can public health deliver? Lancet. 2003;362:159–64.[Medline]
25. Victora CG, Wagstaff A, Schellenberg JA, Gwatkin D, Claeson M, Habicht JP. Applying an equity lens to child health and mortality: more of the same is not enough. Lancet. 2003;362:233–41.[Medline]
26. Smith J. A critical survey of empirical methods for evaluating active labor market policies. London, ON: University of Western Ontario, Department of Economics Research Report; 2000. p. 30.
27. Neufeld L, García A, Leroy J, Flores ML, Fernández A, Rivera J. Impacto del programa Oportunidades en nutrición y alimentación en zonas urbanas de México. Cuernavaca, Morelos: Instituto Nacional de Salud Pública; 2005.
28. Allen L, Gillespie S. United Nations Administrative Committee on Co-ordination, Sub-committee on Nutrition, Asian Development Bank. What works?: a review of the efficacy and effectiveness of nutrition interventions. Geneva: United Nations Administrative Committee on Coordination, Sub-Committee on Nutrition in collaboration with the Asian Development Bank; 2001.
29. Savedoff WD, Levine R, Birdsall N, editors. When will we ever learn? Improving lives through impact evaluation. Washington, DC: Center for Global Development; 2006.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
