Journal of Nutrition OpenSOurce Diets- www.ResearchDiets.com

Home Help [Feedback] [For Subscribers] [Archive] [Search] [Contents]
QUICK SEARCH:   [advanced]


     

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Purchase Article
Right arrow View Shopping Cart
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Menon, P.
Right arrow Articles by Ruel, M. T.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Menon, P.
Right arrow Articles by Ruel, M. T.
© 2008 American Society for Nutrition J. Nutr. 138:634-637, March 2008

Symposium: From Efficacy Trial to Public Health Impact: Improving Delivery and Utilization of Nutrition Programs

Assessing Supervisory and Motivational Factors in the Context of a Program Evaluation in Rural Haiti1–3,

Purnima Menon4,*, Mduduzi Mbuya4, Jean-Pierre Habicht4, Gretel Pelto4, Cornelia U. Loechl5 and Marie T. Ruel6

4 Division of Nutritional Sciences, Cornell University, Ithaca, NY 14853; 5 International Potato Center (CIP-SSA), Kampala, Uganda; 6 Food Consumption and Nutrition Division, International Food Policy Research Institute (IFPRI), Washington, DC 20006

* To whom correspondence should be addressed. Email: pm38{at}cornell.edu.


   ABSTRACT
TOP
 ABSTRACT
Introduction
 Data and Methods
 Results
 Discussion
 LITERATURE CITED
 
Staff supervisory and motivational factors were assessed in the context of an ongoing program evaluation in Haiti comparing 2 models of targeting an integrated health and nutrition program. The study objectives were to 1) understand and improve supervisory and motivational factors influencing program implementation and 2) compare these factors between the 2 program models being evaluated. Qualitative methods (focus group discussions and semistructured interviews) were used to understand factors related to supervision and motivation. Quantitative measures of supervisory and motivational factors were designed, and factor analysis was used, to develop summary scales of motivational factors and supervision. T-tests were used to compare mean scores on the scales between the 2 program models. Results from the qualitative research were discussed with program management and staff to help develop solutions to implementation bottlenecks. Staff at all levels of the program seemed motivated and generally well supervised. Constraints to motivation included perceived inadequacy of wages (before changes were made to salaries), heavy workloads, and logistical constraints. We found no salient differences between the 2 program models that could contribute to differential implementation or differences in impact. This lack of salient differences between the program models suggested that supervisory and motivational factors were unlikely to contribute to differences in impact. Assessing supervisory and motivational factors was feasible and desirable in the context of this evaluation and deepened understanding of the program context and constraints to implementation.


   Introduction
TOP
 ABSTRACT
 Introduction
Data and Methods
 Results
 Discussion
 LITERATURE CITED
 
Reducing child undernutrition is key to reducing child deaths worldwide (1,2) and to improving human capital of those who survive (3). A major challenge to improving nutrition is implementing efficacious interventions well and at scale (4). The successful delivery of lifesaving programs depends on the motivation and effective job performance of health workers (5), which, in turn, depend on a variety of organizational and contextual factors (6–9). Research on determinants of health worker performance is in its infancy, however, and a recent review (9) has called for more rigorous analyses of the determinants of health worker performance.

A key limitation to the study of the determinants of health worker performance is the lack of measurement tools to assess the contextual aspects of the work environment that influence performance, including the supervisory and motivational aspects of the work context. Much of the research on determinants of worker performance is focused on training, rather than on other aspects (9) such as management practices and worker incentives.

This article describes the mixed methods approach that was used to assess supervisory and motivational factors and discusses how these data were used in the context of a program evaluation comparing 2 models of targeting an integrated nutrition program (see below). The objectives of the research were 2-fold: first, to assess, understand and improve factors relating to supervisory and motivational factors in the work context; and second, to compare these factors between the 2 program models. It was considered important to examine these factors because there were potential differences in workload between the 2 models as a result of differences in duration and targeting approaches of the 2 program models (10). In addition, because health workers in each program model were supervised by different supervisors, it was considered important to examine differences in the supervisory aspects of the work context. Equivalence in these factors would enhance the plausibility that the difference in nutritional outcomes observed in the randomized control trial (10) was caused by the planned interventions.

Program evaluation and organizational context

This study was done in rural Haiti, in the context of a cluster-randomized evaluation of 2 different programmatic models for reducing undernutrition, referred to as the preventive (targeting all children 6–23 mo) and the recuperative (targeting underweight children 6–59 mo) approaches. The preventive program had better outcomes in growth than the recuperative program (10). The programs were delivered by World Vision (WV)7 Haiti and included preventive health care, health and nutrition behavior change communication, and take-home food rations. Health promoters (HP) and assistant health promoters (AHP) are the direct implementers of the health care and behavior change communication components of the program, and this article focuses on them and their direct supervisors. Further information about the program models and the evaluation context is available from Ruel et al. (10).


   Data and Methods
TOP
 ABSTRACT
 Introduction
 Data and Methods
Results
 Discussion
 LITERATURE CITED
 
Data

Data collection was done in 2 phases.

    Phase 1. Qualitative methods were used to understand factors that contributed to worker motivation and perceptions about supervisory practices. Methods included 5 focus group discussions (FGD) with health workers and supervisors supplemented by interviews with higher management. Transcripts were analyzed using Atlas-ti (11) to identify themes reflecting ideas, concepts, values, and feelings pertaining to the issues discussed in the FGD.

    Phase 2. A quantitative survey with HP (n = 19) and AHP (n = 19) was conducted about a year after Phase 1. Questionnaires were designed using key statements expressed in the Phase 1 FGD.

    Supervision. The supervision questionnaire included 29 statements scored on a 5-point Likert scale. The statements were formulated based on Phase 1 FGD, an interview with WV-Haiti's Health Coordinator, a review of terms of reference for supervisors, and literature on supervision (12,13).

    Motivational factors. Based on statements from Phase 1 and measures developed by Dickin (14), we developed 23 statements about motivating factors in the work context, scored on a 5-point Likert scale.

Variable selection for additive scale construction for supervision and motivational factors was based on principal components analysis, assessment of scale reliability, and item analysis (15).

Use and analysis of data

Results from Phase 1 were discussed with WV staff in a consultative workshop. Based on this, steps were taken by WV to improve implementation and some aspects of the organizational context.

Phase 2 results were used to quantitatively verify findings from Phase 1 after program improvements had been made. Phase 2 results were also used to compare the 2 program models on the supervisory and motivational scales and on the knowledge summary score using mean test scores, t-tests, and assessment of the distribution of negative and positive differences across the differences.


   Results
TOP
 ABSTRACT
 Introduction
 Data and Methods
 Results
Discussion
 LITERATURE CITED
 
Phase 1

The themes identified in Phase 1 relative to supervision were 1) the feeling that supervision is stimulating and motivating, and 2) the view that one receives adequate technical guidance for problem-solving, planning, and other technical issues. Some negative dimensions of supervision were insufficient support in case of problems, lack of help with planning, or lack of constructive feedback on performance.

The findings from Phase 1 suggested that staff motivation was high despite some constraining factors in the job environment. The major themes that emerged from the qualitative work in Phase 1 (Table 1) were generally similar among the HP and AHP, but AHP mentioned the low salary more often. This is not surprising because there was indeed a discrepancy in salary, which was not justified by a comparable difference in workload.


View this table:
[in this window]
[in a new window]

  		TABLE 1 Themes relating to positive and negative influences on job satisfaction and motivation (based on Phase 1 qualitative research)

 
Results of Phase 1 were discussed with WV staff to identify potential program improvements. Examples of changes in the organizational context were the provision of badges to all staff (to enhance recognition in the community) and increased salaries for AHP. Although specific changes were not made to supervisory practices, the need for ensuring strong and supportive supervision was articulated in the consultations. It was not possible to make changes to some of the constraints to performance that were outside the control of the implementing organization, such as poor roads. Further details are available in Loechl et al. (16).

Phase 2

Mean scores on the supervision and motivational factors scales were high (see Table 2) and toward the upper end of the scale, substantiating our conclusions from Phase 1. The dimensions derived from the quantitative data collected in Phase 2 were largely congruent with, but not identical to, the themes identified in Phase 1.


View this table:
[in this window]
[in a new window]

  		TABLE 2 Mean scores on quantitative scales assessing supervision and motivational factors in Phase 2, by program model

 
    Supervision. A 2-factor solution explained 60% of the variance: 1) Bureaucratic/technical supervision: inspecting, providing feedback, providing supplies, using checklists, and organizing activities. 2) Supportive supervision: feeling supported, feeling informed, feeling that time and feelings are valued and respected.

    Motivational aspects of work context. A 7-factor solution explained 80% of variance and revealed 5 positive and 2 negative dimensions: 1) Feeling valued: feeling valued by colleagues, management (in terms of providing job security), and, most important, by program beneficiaries. 2) Confidence and enjoyment of work: confidence in one's abilities, enjoyment of work, and ability to make a positive contribution. 3) Training: positive feelings about training, including preservice and ongoing refresher training. 4) Satisfaction with salary: satisfaction with salary in comparison with other jobs and in relation to effort. There was no difference in this scale between the AHP and the HP because of the increase in salary for the AHP implemented between Phase 1 and Phase 2. 5) Support: connections with other staff, support received from other staff, and respect from colleagues. 6) Discouraging aspects of the work context: lack of ability of the program to fully address poverty and poor women's work pressure. 7) Thoughts of leaving: feeling of pressure, excessive workload, and thoughts of leaving the job.

The second objective of this research was to ascertain whether or not there were differences between the 2 program models in supervision and motivational factors. Table 2 shows that that differences in supervisory and motivational factors were extremely small and, therefore, unlikely to have a differential influence on program performance and impact.


   Discussion
TOP
 ABSTRACT
 Introduction
 Data and Methods
 Results
 Discussion
LITERATURE CITED
 
Supervisory and motivational factors were assessed using qualitative and quantitative approaches in the context of a program evaluation in rural Haiti. Results from the qualitative phase (Phase 1) were used to improve the program environment and to guide the development of quantitative measures of supervisory and motivational factors. Results from Phase 2 were used to verify whether perceptions of supervision and motivation were consistent with Phase 1 and to compare measures of work context between the 2 intervention groups.

Findings from both phases showed high levels of motivation and identified intrinsic and extrinsic sources of motivation and positive and negative characteristics of supervision. Workers reported that they generally felt they were well supervised. Some constraints to performance include perceived inadequacy of wages (during Phase 1, before changes were made), heavy workloads, and logistic constraints.

From a methodological perspective, this study demonstrates that it is possible and, indeed, desirable to assess the supervisory and motivational aspects of organizational context within a program evaluation. Other aspects such as availability of supplies, infrastructural issues, worker knowledge, etc., which also influence worker performance and program impact, were also assessed [results are reported by Loechl et al. (16) but are not discussed in this article]. Either qualitative or quantitative approaches can be used to assess, understand, and improve the organizational context, but quantitative measures are necessary to rule out differences between interventions.

The domains that emerged in the factor analyses were consistent with organizational behavior research (17,18). Also, the domains of motivational factors and supervision were congruent with prior literature on work performance and public health programs (7,14,19). The dimensions derived from the quantitative data collected in Phase 2 were largely consistent with, but not identical to, the themes identified in Phase 1. This suggested that the themes overlapped in their underlying dimensions and would not have scaled properly (20) had we developed Phase 2 scales based entirely on the themes identified in Phase 1.

From an evaluation perspective, our results allowed us to infer that differences in supervisory and motivational factors are not important sources of bias. This allows us to attribute differences in program outcomes (21–23) to the interventions being tested and not to differences in supervision or motivation. We acknowledge that the sample size of workers is quite small in this study but also note that the values on the scales are very similar between the 2 program models and are highly unlikely to be meaningful differences.

Our experience suggests that studying factors such as supervisory and motivational factors deepens an evaluator's understanding of the program context and provides insights into factors enhancing or constraining performance. In this study, high staff motivation and good supervision were enhancing factors, but the challenging logistics and infrastructure in Haiti were constraints. Discussing these results with program implementers early in the evaluation helped address some problems, ensure fidelity of implementation to design (16), and thus enhance the attribution of causality (24,25).

In conclusion, this study provides an example of how to measure supervision and motivational factors, both crucial aspects of organizational context, in a nutrition program. The study has also discussed how such information was used in the context of an evaluation to improve implementation. In future analyses, we will examine associations among specific aspects of organizational context, worker skills, and program effectiveness.


   FOOTNOTES
 
1 Published as a supplement to The Journal of Nutrition. Presented as part of the symposium "From Efficacy Trial to Public Health Impact: Improving Delivery and Utilization of Nutrition Programs" given at the 2007 Experimental Biology meeting, April 29, 2007, Washington, DC. The symposium was sponsored by the American Society for Nutrition and supported in part by the Mainstreaming Nutrition Initiative. The symposium was chaired by Jef L. Leroy of the National Institute of Public Health and Purnima Menon of Cornell University. Back

2 Supported by Food and Nutrition Technical Assistance (FANTA) through the United States Agency for International Development (USAID); Cornell University, IFPRI, the Government of Germany, World Food Programme (WFP), and WV-Haiti. Back

3 Author disclosures: P. Menon, M. Mbuya, J.-P. Habicht, G. Pelto, C. U. Loechl, and M. T. Ruel, no conflicts of interest. Back

7 Abbreviations used: AHP, assistant health promoter; FGD, focus group discussion; HP, health promoter; WV, World Vision. Back


   LITERATURE CITED
TOP
 ABSTRACT
 Introduction
 Data and Methods
 Results
 Discussion
 LITERATURE CITED
 

1. Pelletier DL, Frongillo EA Jr, Schroeder DG, Habicht JP. A methodology for estimating the contribution of malnutrition to child mortality in developing countries. J Nutr. 1994;124:2106S–22S.[Abstract/Free Full Text]

2. Caulfield LE, de Onis M, Blossner M, Black RE. Undernutrition as an underlying cause of child deaths associated with diarrhea, pneumonia, malaria, and measles. Am J Clin Nutr. 2004;80:193–8.[Abstract/Free Full Text]

3. Walker SP, Wachs TD, Gardner JM, Lozoff B, Wasserman GA, Pollitt E, Carter JA. Child development: risk factors for adverse outcomes in developing countries. Lancet. 2007;369:145–57.[Medline]

4. World Bank. Repositioning nutrition as central to development: a strategy for large scale action. Washington, DC: World Bank Publications; 2006.

5. World Bank. World development report 2004: making services work for poor people. Washington, DC: World Bank; 2003.

6. Dickin KL, Dollahite JS, Habicht J-P. Nutrition behavior change among EFNEP participants is higher at sites that are well managed and whose front-line nutrition educators value the program. J Nutr. 2005;135:2199–205.[Abstract/Free Full Text]

7. Franco LM, Bennett S, Kanfer R, Stubblebine P. Determinants and consequences of health worker motivation in hospitals in Jordan and Georgia. Soc Sci Med. 2004;58:343–55.[Medline]

8. Robinson SA, Larsen DE. The relative influence of the community and the health system on work performance: A case study of community health workers in Colombia. Soc Sci Med. 1990;30:1041–8.[Medline]

9. Rowe AK, de Savigny D, Lanata CF, Victora CG. How can we achieve and maintain high-quality performance of health workers in low-resource settings? Lancet. 2005;366:1026–35.[Medline]

10. Ruel MT, Menon P, Habicht JP, Loechl C, Bergeron G, Pelto G, Arimond M, Maluccio J, Michaud L, Hankebo B. Age-based preventive targeting of food assistance and behavior change communication for reduction of childhood undernutrition in Haiti: a cluster randomized trial. Lancet. 2008; in press.

11. Atlas.Ti. [Computer software]. Berlin: Atlas.ti Scientific Software Development; 2002–2007.

12. Marquez L, Kean L. Making supervision supportive and sustainable: new approaches to old problems, maximizing access and quality initiative, MAQ Paper No. 4. Washington, DC: USAID; 2002.

13. Yukl GA, Wall S, Lepsinger R. Preliminary report on validation of the Managerial Practices Survey. In: Clark KE, Clark MB, Center for Creative Leadership, Albright RR, editors. Measures of leadership. West Orange, NJ: Leadership Library of America; 1990. p. 223–37.

14. Dickin KL. The work context of community nutrition educators: relevance to work attitudes and program outcomes. Ithaca, NY: Cornell University; 2003.

15. Gouws E, Bryce J, Pariyo G, Armstrong Schellenberg J, Amaral J, Habicht JP. Measuring the quality of child health care at first-level facilities. Soc Sci Med. 2005;61:613–25.[Medline]

16. Loechl C, Ruel MT, Pelto G, Menon P. The use of operations research as a tool for monitoring and managing food-assisted maternal/child health and nutrition (MCHN) programs: an example from Haiti. Washington, DC: IFPRI; 2005.

17. Misra S, Kanungo RN. Bases of work motvation in developing societies: a framework for performance management. In: Kanungo RN, Mendonca M, editors. Work motivation: models for developing countries. New Delhi; Thousand Oaks, CA: Sage; 1994. p. 27–48.

18. Hackman JR, Oldham GR. Work redesign. Reading, MA: Addison-Wesley; 1980.

19. Bhattacharyya K, Winch PJ, LeBan K, Tien M. Community health worker incentives and disincentives: how they affect motivation, retention, and sustainability. Arlington, VA: BASICS II (USAID); 2001.

20. DeVellis RF. Scale development: theory and applications. 2nd ed. Thousand Oaks, CA: Sage Publications; 2003.

21. Arimond M, Ruel MT, Menon P, Loechl C, Habicht JP, Pelto G, Maluccio J, Michaud L. Child feeding practices are improved by a behavior change communications strategy using a small peer group approach in rural Haiti [abstract]. FASEB J. 2007;21:672.3.

22. Menon P, Ruel MT, Mbuya MN, Arimond M, Loechl C, Pelto G, Habicht JP, Michaud L. Translating knowledge about child feeding into adoption of practices: Comparison of 2 approaches to targeting food aid and behavior change communication (BCC) in maternal and child health programs [abstract]. FASEB J. 2007;21:353.6.

23. Ruel MT, Menon P, Loechl C, Arimond M, Habicht JP, Pelto G, Maluccio J, Michaud L. Preventive targeting of food aid and behavior change communication to 6–24 mo old children is more effective in reducing undernutrition than targeting malnourished children in rural Haiti. Experimental Biology 2007. Washington, DC: FASEB; 2007.

24. Menon P, Ruel MT, Habicht JP, Pelto G, Loechl C. Using operations research to strengthen an impact evaluation comparing two integrated nutrition programs in Haiti. In: Vorster HH, Blaauw R, Dhansay MA, Kuzwayo PMN, Moeng TL, Wenzel-Viljoen E, editors. Proceedings of the 18th International Congress of Nutrition, 19–23 September 2005, Durban, South Africa: Nutrition Safari for Innovative Solutions: Annals of Nutrition and Metabolism. Basel: Karger Publications; 2005.

25. Habicht JP, Victora CG, Vaughan JP. Evaluation designs for adequacy, plausibility and probability of Public Health programme performance and impact. Int J Epidemiol. 1999;28:10–8.[Abstract/Free Full Text]




This article has been cited by other articles:


Home page
 J. Nutr.Home page
J. L. Leroy and P. Menon
From Efficacy to Public Health Impact: A Call for Research on Program Delivery and Utilization in Nutrition
J. Nutr., March 1, 2008; 138(3): 628 - 629.
[Abstract] [Full Text] [PDF]

Home page
 J. Nutr.Home page
J. L. Garrett
Improving Results for Nutrition: A Commentary on an Agenda and the Need for Implementation Research
J. Nutr., March 1, 2008; 138(3): 646 - 650.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Purchase Article
Right arrow View Shopping Cart
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Menon, P.
Right arrow Articles by Ruel, M. T.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Menon, P.
Right arrow Articles by Ruel, M. T.

Home Help [Feedback] [For Subscribers] [Archive] [Search] [Contents]
Copyright © 2008 by American Society for Nutrition