![[Feedback]](/icons/banner/feedbackACT.gif){kind=icon}
![[For Subscribers]](/icons/banner/subscriberACT.gif){kind=icon}
![[Archive]](/icons/banner/archiveACT.gif){kind=icon}
![[Search]](/icons/banner/searchACT.gif){kind=icon}
![[Contents]](/icons/banner/tocACT.gif){kind=icon}
|
|
|
|
|
||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Research Fellow, The International Food Policy Research Institute,Washington, DC 20006
* To whom correspondence should be addressed. E-mail: j.garrett{at}cgiar.org.
 |
ABSTRACT |
|---|
 TOP ABSTRACT IntroductionLITERATURE CITED |
|---|
|
Introduction |
|---|
|
TOPABSTRACTIntroductionLITERATURE CITED |
|---|
Research and implementation often exist in separate worlds. Indeed, the 2006 World Bank document "Repositioning Nutrition as Central to Development" implicitly verifies the separation when it emphasizes that the major constraint in reducing malnutrition is not what to do but how to do it (1). To go forward in improving results for nutrition, the nutrition research community must work to bring these worlds together and undertake research that will answer the question of "how." If it does not do so, nutrition researchers are at high risk of becoming less relevant to the needs of policymakers and program managers—those who actually put the information into practice—and consequently to the needs of millions of malnourished men, women, and children.
In 2001, Allen and Gillespie wrote a useful monograph called "What Works" (2). Today we are still missing a companion summary: "How to Make It Work." Why is that? Two reasons seem most probable: first, a predisposition of nutrition researchers toward questions of efficacy and not of effectiveness; second, limited familiarity with other fields of study and knowledge communities that have the concepts, tools, and insights nutrition researchers could use to push forward their own research agenda on implementation.
The subtitle of the Allen and Gillespie work, "A review of the efficacy and effectiveness of nutrition interventions," gives subtle support to the first hypothesis. The separation of efficacy and effectiveness gives insight into the frame of reference, and mindsets, common to nutrition research. Nutrition is science. From this perspective, research begins in a laboratory, determining what effects an action can produce (efficacy). That knowledge is then incorporated into an intervention that is tried in the real world to see if it indeed works (effectiveness).3 But for social scientists, who work almost entirely with programs and policies in the real world, this distinction is hard to make. A political scientist, for example, works "outside the laboratory" and "in the field" practically all the time. Power is itself the subject of study; and in that sense "efficacy" is in fact an intrinsic part of a study of "effectiveness." Yet, one sometimes wonders if many nutrition scientists are dismissive of research on effectiveness because they do not consider it "science," but, rather, a tedious imposition needed to make the science useful.
But to address the needs of the malnourished effectively, we must know how what works works. If we do not understand how interventions work, we cannot understand causal links in implementation and operation. We will not know what levers to pull or push. The controlled experiments favored by scientists to examine efficacy and effectiveness, where only the intervention varies and other factors are held constant, are extraordinarily valuable in ensuring methodological rigor and generating confidence in findings. But it is often very difficult to transplant such rigor to the real world.
Applying a laboratory approach in which the only "change" introduced and then evaluated is the program may result in essentially a binary outcome: the intervention either does or does not work. And if we know only a binary outcome ("with or without the program"), we can miss or misunderstand the policy and program levers needed to increase impact. We can fail to comprehend that the real world is complex and malleable, that the environment and behaviors are ever changing. Certainly there is a place for such rigorous controlled design in evaluation of social programs, yet other, often complementary, approaches are also needed.
Of course, some scientists may argue that how the real world affects programs is far too complex to understand, and so they prefer to focus on only a limited arena of change. Yet obtaining answers to complex questions is imperative, and this requires research to provide a solid evidence base for policy and program recommendations. This sort of work is not technical assistance in the sense that it replicates or implements what is already known, but it is cutting-edge research that generates new knowledge and insights and new methods and tools.
Fortunately, other social scientists can shed light on how to tackle complex social questions. They investigate this real world all the time, using a variety of tools to compare, contrast, and test generalizable principles, concepts, and typologies. So must those concerned with public nutrition.
These other social sciences show us that understanding "effectiveness" requires understanding individual and organizational behaviors, management, and capacities as well as institutional arrangements, mechanisms, and incentives. For many nutritionists, these issues go beyond the realm of science. As a result, these topics have been downplayed or ignored within the nutrition curriculum and research agenda. Despite their importance, nutritionists have simply not been adequately prepared to deal with them.
But they must. Of course, going forward may take nutrition researchers outside traditional comfort zones. It will require validating a new type of knowledge—one often generated by experiences rather than experiments. It will require that nutritionists enter into dialogue and value, and truly accept as valid, insights from other disciplines that have different ways of generating knowledge—particularly the fields of organizational behavior and management science, sociology, political science, anthropology, psychology, and economics.
Although it may be uncomfortable, going forward need not take nutritionists far from what they know and are familiar with. But it may require focusing differently on elements of approaches that already exist. For example, although it is not very operationally friendly, the UNICEF conceptual framework notes the importance of the context in which actions at the individual, household, and community level take place. From a programmatic perspective, the basic framework lacks dynamism and details on mechanisms and resources, but it does highlight the importance of the enabling environment and of economic, political, social, and cultural factors. Future research should do more to explore and reveal this environment and its effects on nutrition.
Unfortunately, many effectiveness evaluations fail to go sufficiently beyond the binary outcomes mentioned earlier. Their designs may attempt to control for context, but the "contextual variables" are usually individual or community characteristics, not the ones that relate to operations, such as program design, management, or staffing. The individual and community characteristics included are, by default, determined to be the ones that make a difference to impact. This sort of evidence base is limited and limiting, giving policymakers, programmers, and advisers few levers to manipulate.
Some analyses contain information on these levers, but frequently they are hidden in some catch-all "variable z," the ubiquitous variable that captures the effect of "all other factors." Alternatively, a study can conclude that "results are context-specific," so the science cannot say much beyond that specific event, also not a very helpful result.
To push knowledge forward, nutrition science must look to other disciplines for help, especially those that focus on individual and organizational behaviors. These fields of study already can and do provide tools and methods to identify and give insight into those levers and to generalize to other situations. Incorporation of tools, concepts, and results will generate a richer, more complete, and more useful analysis that tells how the program was implemented and how operations, management, and capacity impacted its effectiveness. A key area for future work in nutrition, then, is on the details of what works. The challenge of implementation research will be to rigorously analyze operations and management and capacity, so that in the end we can confidently say how to do what needs to be done.
Key principles
Three key principles may guide creation of a body of knowledge on delivery and utilization.
First, the generation of research for results must be systematic. The research must emerge from, or contribute to, a unifying framework. Individual cases can provide individual insight, but they are not broadly useful if knowledge is scattered, and they simply generate a long list of "lessons learned." A conceptual model will help to shape the research questions, so research efforts do not produce random anecdotes but instead an evidence-based, coherent body of knowledge.
Marsh (3), for example, starts down this road, using a comprehensive framework for evaluating delivery and utilization in Nicaragua. Menon (4) also illustrates the usefulness of a systems approach. Her work on behavior change in Haiti illustrates the need to understand weaknesses in implementation, including the effect of context, and to interact with other disciplines (both in content and methodology).
Second, the body of research must produce generalizable conceptual models, tools, and methods. This allows the development of a common language for discourse and also helps to identify critical knowledge gaps where questions remain. Creating a systematic body of knowledge underlies development of typologies and action frameworks that are transferable beyond the original research context.
Finally, research must be communicated so evidence is taken up and program and policies are changed. If the objective is to improve impact, then those who deliver and utilize that information must change. Findings cannot simply be published in peer-reviewed journals or delivered at conferences. Science must be translated into effective action. Findings must be transmitted effectively to managers, decision makers, analysts, and other development practitioners, including those working with community-based organizations and nongovernmental organizations (NGO), so that they actually use the information.
These issues of uptake and change are emphatically researchable questions. In effect, this is an argument for research on the implementation of implementation research. Political science, sociology, public policy, and education administration have already developed a core of insights, based on research, about characteristics of research, the environment, and the policymaking process that researchers must take into account (5).
Existing frameworks and communities of knowledge
In going forward, researchers should engage with and build on existing communities of knowledge, including those on implementation research, knowledge generation and utilization, policy processes, and change management (e.g., 6,7).
Two examples can illustrate the contribution other fields can make, 1 from the research community on implementation, and 1 from researchers who look at utilization of information and policy processes.
Implementation research. Figure 1, based on information from the National Implementation Research Network, provides a diagram to help with guiding research on how programs work (8). There is a source of the idea or program. There is a purveyor who carries the program to the destination, meaning the target organization or audience who will adopt, house, or support the program. A generally accepted assumption, but one put into question by studies of political economy, is that the purveyor will work actively to implement the program faithfully and effectively.
|
The dynamics of the conceptual model are particularly useful in focusing on the importance of change in delivery and utilization. Effective implementation and subsequent change require structures, processes, constraints, and incentives for change and accountability to be in place. Those elements must link the various components of the system together and motivate behavior change across the different levels of the system.
In particular, effective delivery and utilization require good management with constant attention to changes in behaviors, organizational structures and cultures, and relationships among the different actors and components of the system. Research on implementation must look closely at these variables to determine which factors most significantly impinge on effective operation. Bertozzi (9) shows this very clearly in the case of Mexico's Oportunidades program: Effective delivery of services required constant attention to feedback, to accountability mechanisms, and to assessment of what needed to change, why, and how.
The conceptual framework (8) identifies various stages of implementation: 1) exploration and adoption; 2) program installation; 3) initial implementation; 4) later full operation; 5) innovation; and 6) sustainability. We can then generate research questions at each stage (see Table 1 for illustrative questions).
|
Knowledge uptake and change. Nutrition should also connect with the research community on knowledge uptake and change. To effect change and improve delivery and utilization, we must convey findings effectively to decision makers, including government authorities, staff of NGOs and community-based organizations and, of course, caregivers, especially mothers.
The RAPID Research Program at the Overseas Development Institute has summarized key elements of such a strategy (10). As Figure 2 shows, researchers can generate evidence, but they must link with the policy community through personal or professional relationships, especially through the technicians and managers who deal with the issue. The political context and external influences affect how policymakers receive, interpret, and act on research information.
|
Priorities for action research
What are some of the most pressing issues for research on delivery and utilization to improve results for nutrition? In addition to those mentioned earlier, critical questions should focus on how to 1) strengthen delivery; 2) strengthen utilization by communities, households, and individuals; and 3) strengthen the program as an interrelated whole (from design to delivery, utilization, and impact). Table 2 provides some illustrative questions along these lines.
|
This does not require nutritionists to become experts in other fields, but it does ask them to recognize that other fields can shed light on nutrition research, especially implementation research. They must build bridges and bring new knowledge, and those experts, into the discussion.
Implementation research is both urgent and necessary because, if it is not done, nutritionists will not have answers to the fundamental questions country-level decision makers ask: "What do we need to do?" and "How do we do it?" Without those answers, the nutrition research agenda, as a whole, risks becoming disconnected and irrelevant to the broader question of how to respond to the millions of malnourished in the world.
|
FOOTNOTES |
|---|
2 Author disclosures: J. L. Garrett, no conflict of interest.
3 The term "real world" is used here to mean the naturally occurring environment outside the laboratory. Certainly the laboratory is a "real" place, too!
|
LITERATURE CITED |
|---|
|
TOPABSTRACTIntroductionLITERATURE CITED |
|---|
1. World Bank. Repositioning nutrition as central for development. Washington DC: World Bank; 2006.
2. Allen L, Gillespie S. What works? A review of the efficacy and effectiveness of nutrition interventions. Geneva: ACC/SCN in collaboration with the Asian Development Bank, Manila; 2001.
3. Marsh DR, Alegre JC, Waltensperger KZ. A results framework serves both program design and delivery science. J Nutr. 2008;138:630–33.
4. Menon P, Mbuya M, Habicht J-P, Pelto P, Loechl CU, Ruel RT. Assessing supervisory and motivational factors in the context of a program evaluation in rural Haiti. J Nutr. 2008;138:634–7.
5. Stone D, Maxwell S, editors. Global knowledge networks and international development. New York: Routledge; 2005.
6. Waddell S. Societal learning and change. Sheffield, UK: Greenleaf Publishing Ltd; 2005.
7. Bardach E. Getting agencies to work together: The practice and theory of managerial craftsmanship. Washington DC: The Brookings Institution Press; 1998.
8. Fixsen DL, Naoom SF, Blasé K, Friedman RM, Wallace F. Implementation research: A synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication No. 231); 2005. Monograph. Available from: http://nirn.fmhi.usf.edu/resources/detail.cfm?resourceID=31. [Accessed October 22, 2007].
9. Leroy JL, Vermandere H, Neufeld LM, Bertozzi SM. Improving enrollment and utilization of the Oportunidades program in Mexico could increase its effectiveness. J Nutr. 2008;138:638–41.
10. Crewe E, Young J. Bridging research and policy: Context, evidence, and links. ODI Working Paper 173. London: Overseas Development Institute; 2002.
11. Court J, Young J. Bridging research and policy: Insights from 50 case studies. ODI Working Paper 213. London: Overseas Development Institute; 2003.
12. Young J, Court J. Bridging research and policy in international development: An analytical and practical framework. RAPID Briefing Paper 1. London: Overseas Development Institute; 2004.
13. Young J, Court J. Are you a policy entrepreneur? How to promote pro-poor policy and practice. Presentation at ODI Workshop, London, March 16, 2004. Available from: http:www.odi.org.uk/RAPID/Events/Policy_Entrepreneurs/Index.html.
14. Winch PJ, Doumbia S, Kanté M, Malé AD, Swedberg E, Gilroy KE, Ellis AA, Cissé G, Sidibé B. Differential community response to introduction of zinc for childhood diarrhea and combination therapy for malaria in southern Mali. J Nutr. 2008;138:642–5.
This article has been cited by other articles:
|
|
 |
|
  J. L. Leroy and P. Menon From Efficacy to Public Health Impact: A Call for Research on Program Delivery and Utilization in Nutrition J. Nutr., March 1, 2008; 138(3): 628 - 629. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
