Health Education Research Advance Access originally published online on August 31, 2006
Health Education Research 2007 22(3):318-331; doi:10.1093/her/cyl078
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Implementation examined in a health center-delivered, educational intervention that improved infant growth in Trujillo, Peru: successes and challenges
1 Department of International Health, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD 21205, USA
2 Center for Human Nutrition, Department of International Health, Johns Hopkins University Bloomberg School of Public Health, Baltimore, MD 21205, USA
3 Instituto de Investigación Nutricional, Lima 12, Peru
4 Department of Health Behavior and Education, School of Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-7440, USA
* Correspondence to: R. C. Robert. E-mail: rrobert{at}jhsph.edu
 |
Abstract |
|---|
 Top Abstract Introduction and backgroundMethodsResultsDiscussionConclusionConflict of interest statementAcknowledgementsReferences |
|---|
Process evaluation was used to examine the implementation of a randomized, controlled trial of an education intervention that improved infant growth in Trujillo, Peru. Health personnel delivered the multi-component intervention as part of usual care in the government health centers. Quantitative and qualitative methods were used to examine process indicators, which included the extent of delivery (dose), fidelity to intervention protocol, barriers to implementation and context. Results demonstrated that most intervention components were delivered at a level of 50–90% of expectations. Fidelity to intervention protocol, where measured, was lower (28–70% of expectations). However, when compared with existing nutrition education, as represented by the control centers, significant improvements were demonstrated. This included both improved delivery of existing educational activities as well as delivery of new intervention components to strengthen overall nutrition education. Barriers to, and facilitators of, implementation were explored with health personnel and helped to explain results. This study demonstrates the importance of examining actual versus planned implementation in order to improve our understanding of how interventions succeed. The information gained from this study will inform future evaluation designs, and lead to the development and implementation of more effective intervention programs for child health.
|
Introduction and background |
|---|
|
TopAbstractIntroduction and backgroundMethodsResultsDiscussionConclusionConflict of interest statementAcknowledgementsReferences |
|---|
Child malnutrition is a significant public health problem that continues to challenge the developing world [1]. Addressing child malnutrition with effective and sustainable programs is a priority. In Peru, a randomized, controlled trial of a nutrition education intervention, delivered through government health centers, demonstrated success in improving dietary intakes and growth in a cohort of children followed from 0–18 months [2]. The success of this trial led to the question of ‘what exactly occurred?’ with respect to implementation of the multi-component intervention in the health centers. Process evaluation, which is used to thoroughly examine intervention implementation, was the means to answer this question. Including process evaluation during a trial allows for salient indicators to be examined on the pathway through which an intervention is expected to work, and brings understanding to outcome results [3, 4].
Implementation data are critical to understand what, how and why an intervention works or does not work. The strengths and weaknesses of an intervention, particularly those with multiple components, can be revealed. Query of whether maximum impact has been achieved also relies on results of implementation. Finally, for a successful intervention to be replicated, data on implementation are necessary [4]. In developing country settings, where resources are limited and basic public health problems prevalent, knowledge of implementation is critical to promote effective programs, their expansion and replication. Despite the number of public health programs working to address the needs of children in developing countries, few studies have thoroughly examined the extent and quality of their implementation [5]. We conducted process evaluation during the foregoing intervention trial in Trujillo, Peru to (i) examine and understand intervention implementation, (ii) explore impeding and facilitating factors to implementation and (iii) help explain trial outcomes.
The intervention
The 2-year cluster randomized, controlled trial (1999–2001) was a joint effort of the Instituto de Investigación Nutricional (IIN), Peru and the regional Ministry of Health of La Libertad (MOH), Peru, in collaboration with the Johns Hopkins Bloomberg School of Public Health, USA. The intervention trial took place in the large peri-urban, economically poor areas surrounding Trujillo, and involved six intervention health centers matched with six control centers. Poor housing and a generalized lack of one of more basic services (piped water, sewage disposal, reliable electricity) characterized the area. Nonetheless, local markets sold a variety of affordable and nutritious food. The overall study design, intervention and evaluation design are reported elsewhere [2]. The process evaluation described here took place in 1999–2000.
The government health centers, designated to deliver the nutrition intervention, were accessible by foot to the surrounding community and provided medical consultation and growth and development monitoring (GDM). Some preventive nutrition education also existed as part of the services offered. From the baseline study of the trial, 90.6% of caregivers reported visiting the health service at least once since birth.
The multi-component intervention, which built on existing nutrition education in the health centers, was based on principles of social cognitive theory, formative research for the trial and experience gained from other successful intervention programs in Peru [6–8] (H.M. Creed-Kanashiro, M. Fukumoto, M.E. Ugaz, unpublished report). In social cognitive theory, behavior is dynamic; a person, his/her behavior and the environment in which the behavior is performed are in continual interaction. This premise, entitled reciprocal determinism, invites interventions to target behavioral capacities, cognitive characteristics of both individuals and collective groups (e.g. self/collective efficacy), and social and physical environments.
The intervention concentrated, in particular, on complementary feeding of children 
6 months. For caregivers, the intervention was designed to promote experiential and observational learning of feeding behaviors. Simple messages using the local Spanish vernacular of caregivers reinforced behaviors. For health centers, the intervention was intended to raise the profile of, and integrate, nutrition education into all pediatric services. More specifically, it was designed to improve the confidence of, and consistency among, health personnel in their approach to nutrition education, and improve the quality of nutrition messages and interactions between personnel and caregivers.
The six main intervention components included (i) delivery of age-appropriate messages in all pediatric services, aided by color photo flip charts and follow-up ‘checking’ questions to verify caregiver understanding; (ii) infant food preparation demonstrations where caregivers prepare or observe others preparing a recipe, and all caregivers and infants taste-test the prepared food; (iii) anthropometric assessment, including weight and length measurements of all infants attending the health center and explanation of the infant's growth; (iv) provision of infant food recipe fliers to caregivers; (v) GDM in a group format where several caregiver–infant pairs with the same age infants actively participate in developmental and play activities and receive nutrition and health advice (referred to hereafter as ‘group GDM’) and (vi) multi-disciplinary problem-solving sessions wherein health personnel addressed nutrition concerns in their health center or community.
The message component was central to the intervention and integrated with the other components. The following three were known as ‘key messages’ and all health personnel were encouraged to deliver them:
- (i) ‘A thick puree satisfies and nourishes your baby, equivalent to 3 portions of soup; at each meal give the thick puree or main dish first’;
- (ii) ‘Add a special food to your baby's meal: liver, egg or fish’;
- (iii) ‘Use love, patience and good humor when teaching your child to eat’.
- (ii) ‘Add a special food to your baby's meal: liver, egg or fish’;
This paper addresses the following research questions:
- (i) How does implementation of the nutrition education intervention compare to expected standards (as stated in the accreditation criteria)?
- (ii) How does nutrition education compare in the intervention and control health centers after implementation of the intervention?
- (iii) What factors facilitate or inhibit implementation of the intervention?
- (ii) How does nutrition education compare in the intervention and control health centers after implementation of the intervention?
|
Methods |
|---|
|
TopAbstractIntroduction and backgroundMethodsResultsDiscussionConclusionConflict of interest statementAcknowledgementsReferences |
|---|
A conceptual model was created of the expected intervention pathway leading to improvements in infant nutrition [9]. Process indicators representing salient points of implementation in the model included the following:
Dose delivered: Percentage of intervention activities completed by health personnel out of the total expected, as specified in the accreditation criteria.
Fidelity: Percentage of intervention activities completed by health personnel that adhered to intervention protocol; a measure related to the quality of the activity with acceptable standards being specified in the accreditation criteria.
Barriers to implementation: Potential impediments to implementation of intervention activities in the health centers, such as staff turnover.
Context: Qualitative measure of the environment in which the intervention was introduced and wherein facilitating factors for implementation are identified.
Data collection
Based on the foregoing process indicators, multiple quantitative and qualitative instruments were developed to collect data, which provided methodological and data source triangulation. Data were collected by a trained and standardized evaluation team not involved in delivery of the intervention. Training, directed by the lead author, involved interactive classroom sessions and practice in a pilot health center and community, with pre-designated competencies to achieve. Approval to conduct the study was given by the Ethics Committee of the IIN, Lima, Peru whose guidelines include the principles espoused in the Helsinki Declaration and the World Health Organization operational guidelines [10, 11].
A summary of the data collection methods is presented in Table I. Although growth outcomes of the trial were measured for infants until 18 months of age, the intervention was indicated for infants up to 24 months, which facilitated process data collection.
|
Record reviews
Record reviews included (i) documentation of weight and length from infants' charts; (ii) number of infant food preparation demonstrations carried out by intervention centers; (iii) coverage in the GDM service, defined as the number of children completing the MOH's mandate of five visits in the first year of life; (iv) staff turnover and (v) documentation of problem-solving sessions completed by health personnel.
Structured observations
Observations of health worker technique in weighing and measuring five infants aged 0–24 months were completed. A structured format with dichotomous items to ‘check off’ was used. Prior to data collection, an inter-rater reliability of at least 94% was achieved between the principal investigator and each observer [(number of items agreed upon/total number of items) x 100].
Structured observations of the interactions between health personnel and caregivers of infants 0–24 months of age were completed in the GDM, nutrition consultation and medical consultation services. Observations were event based and each health worker was observed with 5–10 caregivers with infants. A structured format consisting of dichotomous and multiple choice items was used to check off specific behaviors and communications. All nutrition recommendations were recorded as well as the use of educational materials and/or ‘checking questions’ to verify caregiver understanding. Prior to data collection, an inter-rater reliability of at least 93% was achieved between the study supervisor (a native Spanish speaker) and each observer [(number of items agreed upon/total number of items) x 100].
Sample size was calculated, based on the primary outcome delivery of key nutrition messages. The differences between centers were anticipated to be at least 35% (75% intervention, 40% control) giving a sample size of 39 per group (alpha 0.05, power 0.80, design effect 1.25).
Exit interviews
Exit interviews were conducted with caregivers of infants 0–24 months as they left the health center. Questions were read out to caregivers and responses recorded using pre-designated codes or written verbatim answers that were later coded. Items included feeding recommendations received. Sample size was based on the primary outcome, exposure to key nutrition messages. A difference of at least 25% (65% intervention, 40% control) was anticipated giving a sample size of 78 per group (alpha 0.05, power 0.80, design effect 1.25).
Sampling for observations and exit interviews included all infants who entered the appropriate service, until the required number was reached. No overlap occurred among the different modes of data collection. Health personnel were not pre-informed of scheduled dates in an attempt to preserve usual routines.
Semi-structured interviews with health personnel
Semi-structured interviews were conducted with selected health workers in the intervention group, and included nurses in charge of the GDM service, nutritionists, assistants and physicians. An interview guide with questions about the experience implementing the intervention was developed. A Peruvian medical anthropologist, experienced in qualitative methodologies, conducted the interviews in Spanish after mid-trial. Interviews were tape recorded and generally lasted 50–70 min.
Data analysis
Data were entered into Fox Pro 2.6a using purpose-designed programs with range and consistency checks, and analyzed using SPSS (Version 11.5: SPSS Inc., Chicago, IL, USA). For structured observations of consultations, mean frequencies were calculated for each health worker. Indexes were created to measure anthropometric technique fidelity. Comparative analyses between intervention and control centers consisted of chi-square test for categorical data and Mann–Whitney U test for means (non-normal distributions).
For qualitative analyses, semi-structured interviews were tape recorded and transcribed in Spanish. A coding system was created and codes were applied to segments of text. Text was searched by single codes and combinations of codes with the aid of the software package NUD*IST (Version 4: Qualitative Solutions and Research Pty Ltd, Australia). Analysis of textual data included (i) identification of recurrent patterns and themes and (ii) comparison of specific codes or themes by health center or type of health personnel (nurse, assistant, etc.). Final results were translated into English. The code of interest are those related to barriers and facilitators of intervention implementation.
|
Results |
|---|
|
TopAbstractIntroduction and backgroundMethodsResultsDiscussionConclusionConflict of interest statementAcknowledgementsReferences |
|---|
Implementation in intervention health centers
Intervention implementation, quantified by dose delivered and fidelity, is shown for each of the six main intervention components in Table II. Raw frequencies and standardized frequencies, calculated as the percentage of the written standard achieved, are presented. Of the 10 measurements of dose delivered, one met the standard stated in the accreditation criteria; namely, coverage in the GDM service. None of the four fidelity measures met the standard. Dose, using standardized frequencies, varied from 17 to 100% for the various intervention components, the majority being implemented at a level of 50–90%. Where both dose and fidelity measures were made for an intervention component, dose greatly exceeded fidelity.
|
Implementation compared between health services in intervention centers
Health personnel in the GDM and nutrition consultation services delivered intervention components with significantly greater dose and fidelity than those in the medical consultation service (Table III).
|
Implementation compared between the intervention and control health centers
To compare delivery of nutrition messages, we used messages in the control group that contained similar content to those promoted in the intervention (Table IV). Message structure (e.g. length of message, vocabulary used) was not considered. Although a difference was not found in the overall percentage of message delivery (dose), significance was achieved with respect to fidelity. That is, health personnel in the intervention group delivered messages using educational materials and checking questions with greater frequency. Significant improvements were also seen in anthropometry documentation and measurement technique. Delivery of recipes, infant food preparation demonstrations and group GDM were not compared statistically as they represented new intervention activities and were not found in the control centers.
|
Additional analyses compared message delivery between intervention and control centers within two age groups, 0–5 and 6–24 months (Fig. 1). No difference in dose was seen in the age group 0–5 months, where delivery of breast-feeding messages dominate (Z = –0.37). Differences did emerge for infants aged 6–24 months, the group in which complementary feeding messages are delivered (Z = –1.76). This reached statistical significance when delivery of the three key messages for complementary feeding was compared (Z = –3.43). Significant differences were seen in all age groups with respect to fidelity of message delivery: 0–5 months (Z = –3.61), 6–24 months (Z = –2.96) and 6–24 months key messages (Z = –3.14).
|
As an indirect measure of implementation, the percentage of caregivers of infants 6–24 months who could recall the promoted nutrition messages (or messages with similar content in the case of the control group) were recorded during exit interviews immediately following a clinic visit. In the intervention group, 56.5% of caregivers recalled a message compared with 16.7% in the control group (
2 = 36.9, 1 df, P-value = 0.000). An even greater disparity occurred when key message recall was compared: intervention caregivers 47.2%, control caregivers 2.8% (2 = 56.9, 1 df, P-value = 0.000).
Barriers and facilitating factors affecting implementation
Barriers to and facilitators of implementation were explored to help explain implementation results.
Barriers
Changes in health personnel, potentially disrupting continuity, resulted from staff turnover and new positions created. The average number of staff per month in the six health centers totaled 148, calculated from monthly totals for 2 years. The total number of new staff, representing either staff turnover or newly created positions, amounted to 139 over 2 years. Some of the staff positions had multiple turnovers, whereas others had none. For staff more likely to be working with children (physicians, nurses, nutritionists and assistants), the average monthly total was 78 for the six health centers. Over 2 years, 52 new staff, representing either staff turnover or newly created positions, were counted. Turnover was highest for physicians and nurses. However, five of seven head nurses of the GDM service remained throughout the study. Of the original four nutritionists, three remained. High staff turnover occurred in all six centers. Similar numbers were found in control group centers.
Staff turnover was not only high, but also little or no overlap occurred between outgoing and incoming staff. Staff turnover extended to the regional MOH, and included a number of top-ranking positions involved with the study. However, the five-person MOH commission appointed to work with the study remained stable.
From semi-structured interviews, health personnel identified a number of foreseen and unforeseen barriers to implementation. Insufficient economic resources to purchase ingredients for infant food demonstrations resulted from changes in petty cash fund management at the health center level. Printing additional recipe fliers, once the supply given by the IIN was exhausted, was also perceived as problematic. Time constraints to conduct demonstrations were mentioned by centers without nutritionists. Regarding changes introduced to improve anthropometric technique, several informants voiced obstacles where health personnel rotated frequently or the temperature in the triage room was cold (discouraging removal of heavy clothing). Although a ‘common language’ of key messages resonated throughout the interviews, several informants mentioned that more work was needed to ensure their familiarity among all health personnel.
Informants identified the most pervasive problem encountered for group GDM to be securing caregiver attendance at a scheduled hour to enable formation of a group—an unfamiliar practice. One nurse commented:
The group evaluation wasn't that easy because this is a process, and the mothers have to become accustomed to attend at a specific hour.
A number of strategies were tried to encourage timely attendance, some more successful than others. Nurses commented that implementing the group was challenging and required work and determination to succeed. Most centers mentioned difficulty in implementing the problem-solving sessions. Although viewed as important, they did not directly relate to the care of infants and thus were perceived as peripheral to the other intervention components. One nurse stated:
Excepting problem solving sessions ... the rest is of interest to us because it represents how we have been able to improve our growth and development monitoring program and our coverage.
Staffing changes in all health centers and frequent rotations of technical staff within large centers were mentioned as limitations to overall intervention implementation.
Facilitating factors
Data from semi-structured interviews revealed factors that motivated health personnel to implement the intervention. Child malnutrition was recognized as a problem in the communities and the intervention provided a means to address it. One physician stated:
What caught my attention were things we were aware of but didn't do ... the problem at 6 months with the change in food/feeding ... yes, sure the people have a scarcity of resources, but with the little resources they do have, they don't have a clear idea of how to use them. They dedicate to giving thin soups and things that don't sustain the child ... so the first thing that made an impression on me was to try what the project said, to give the second (main) dish first, and to teach how to prepare food with demonstrations.Benefits for the community and improved coverage in the GDM service were expected and valued. The novelty of group GDM and its perceived benefits, more efficient use of time for nurses and a higher quality service offered to caregivers, were also mentioned as motivating factors. Once the intervention was implemented, health personnel commented that caregivers appeared more satisfied with the care received, which facilitated their continued implementation. As one nurse stated:
We have increased awareness a lot with group growth and development monitoring. We see the face of satisfaction on the mothers; now they come and actively participate.Informants' descriptions signaled acceptance and routine use of the messages in day-to-day care of infants and caregivers. Nutrition education was referred to as a set of activities that included all health personnel and stressed a ‘common language’ of key messages. As one nutritionist commented:
Before, only the nutritionist gave messages. Now, we all give messages because we all speak the same language, we all say the basic messages.New modalities to provide nutrition education such as demonstrations of infant food preparation where caregivers ‘learn by doing’ were well received by health personnel. As one nurse commented:
... it's another thing to demonstrate a recipe, this captures the attention of the mother, with the taste-testing, she sees that her child doesn't choke ....Project team visits to health centers were mentioned in relation to facilitating implementation of the intervention. Visits were also made to the MOH and resulted in the formation of a five-person MOH commission to support the study and secure adoption of activities within the MOH and health centers. One member was instrumental in advancing the implementation of group GDM in the centers.
In the interviews, narrative that described a cohesive, integrated work effort was used to represent the construct of health personnel's collective efficacy to implement the intervention. From semi-structured interviews, health center no. 3 demonstrated the most integrated team in working to advance implementation of the intervention. ‘We, us, and team work’ were common words spoken in the interviews, and examples of nutrition activities across many health personnel were spontaneously given. This center was also the first to accredit as a ‘Pioneer Center in Infant Nutrition’. Center no. 6 also had a strongly integrated team of health personnel, but was slower than other centers to implement the intervention due to initial reluctance on the part of the head GDM nurse, a dominant figure in the health center.
In health centers no. 2 and no. 5, data clearly revealed the intervention to be one centered in the GDM service, with some support from other services (these two centers did not have a nutritionist). Health centers no. 4 and no. 1 showed early initiative for implementing the intervention, but slowed considerably after several staffing changes. At the time of interviews, both were attempting to make advances. However, the nutritionist at health center no. 1 openly remarked about the general lack of integration among health personnel as well as between the nutrition consultation service and other pediatric services.
|
Discussion |
|---|
|
TopAbstractIntroduction and backgroundMethodsResultsDiscussionConclusionConflict of interest statementAcknowledgementsReferences |
|---|
Process evaluation was used in a randomized, controlled trial, to examine the implementation of an educational intervention delivered through government health centers in Trujillo, Peru. Significant differences in growth, the primary outcome measure of the trial, were demonstrated in a cohort of children followed 0–18 months (intervention n = 187, control n = 190). This included reduced stunting, and improved mean weight gain, mean length gain and mean z-scores (weight-for-age, length-for-age) at 18 months. Recommended feeding practices and dietary intakes were also significantly better in the intervention group at several measurement points [2].
When compared with planned goals for implementation, the results of actual implementation fell short. Most intervention components were delivered at a level of 50–90% of expectations (range 17–100%). Fidelity or adherence to intervention protocol, where measured, was lower (28–70% of expectations). However, when results of implementation were compared with existing nutrition education, as represented by the control centers, significant improvements were demonstrated. This included both higher levels of existing nutrition education (delivery of messages, anthropometry) as well as new intervention components to strengthen overall education (delivery of recipes, demonstrations of food preparation and a group format of GDM). Barriers to, and facilitators of, implementation were explored with health personnel who delivered the intervention and helped to explain results. Staff turnover was quantitatively documented and found to be extremely high.
First, we examine the question of successful trial outcomes despite less than optimal implementation. Although many of the intervention components were not delivered at expected levels, each was delivered to some extent and this appeared sufficient to affect outcomes. Coverage in the GDM service (five visits in first year of life) did meet expectations and this provided multiple opportunities for nutrition education. Intervention centers demonstrated marked improvements over control centers in their overall delivery of nutrition education. Had implementation achieved expected levels, an even greater difference in growth and dietary outcomes may have occurred. A goal of the intervention was to raise the profile of and integrate nutrition education in all pediatric services. It is possible that implementation at the levels achieved positively affected the overall social and physical environment of the health center, whereby a ‘culture of nutrition’ was created. This environment may have created benefits beyond what our instruments captured in quantitative terms. The sum of multiple encounters with nutrition education, either during a single visit or during multiple visits may also have produced more benefit than what was captured by our measures of implementation. Process data collection was also concentrated in the first year of the trial. Continued data collection may have revealed improved implementation.
We considered whether the expected levels set for implementation were too aggressive and therefore implementation levels appeared low. However, the MOH felt strongly about maintaining the stated standards, as they were consistent with expectations in other health areas and not considered unreasonable (generally 80% of cases were expected to meet the implementation standard). We acknowledge the evidence that current standards for implementation exceeded what was necessary to produce positive outcomes, but hope that addressing identified obstacles will lead to greater levels of implementation (and outcomes). For particular criteria, lower standards may be more reasonable (e.g. fidelity of message delivery). However, in general, we suggest maintaining the current standards to represent ideal implementation and then employing a second lower standard to represent acceptable implementation.
Dickey et al. [5] reported on the implementation of a nutrition intervention in Viet Nam for caregiver-malnourished infant pairs, delivered by trained health volunteers. Similar to our study, improved dietary intakes and growth—in a subset of children—were found despite under-implementation of some components and variation in implementation among sites. Using process data, the authors discussed speculative but possible explanations for the improved outcomes.
Second, we explored reasons underlying sub-optimal implementation. Interviews with health personnel provided vital insights on both foreseen and unforeseen barriers to implementation and facilitators of implementation. For example, infant food preparation demonstrations were described as an influential modality for improving feeding behaviors, where caregivers ‘learn while doing’. Yet, only 60% of demonstrations were delivered. Exploration found the greatest obstacle to be purchasing ingredients, albeit inexpensive, due to changes in the management of petty cash funds at the health center level. Barriers were identified both within the health center (e.g. staff turnover) as well as in the community (e.g. forming groups of mothers to attend group GDM). To secure optimal implementation of this intervention, barriers must be addressed. Messages were delivered at a level close to expectations (dose); however, adherence to fidelity fell below expectations. That is, although health personnel regularly delivered messages, the usual communication style did not include active participation of the caregiver with use of checking questions and educational materials. The change to participatory communication proved challenging. In interviews, use of key messages was repeatedly mentioned while little was said of the new communication approach. Other studies have reported use of interview data to better understand program implementation from the providers point of view, including teachers involved in school health programs and church coordinators involved in a nutrition education intervention [12, 13].
Staff turnover was considerably greater than we expected. Fortunately, at least one key health provider remained in each center throughout the study. The high staff turnover, coupled with little or no overlap in personnel, was a likely reason for lower levels of implementation. Nonetheless, the intervention's emphasis on changing the health services versus individuals promoted continued implementation and was sufficient to result in improved growth outcomes. More work is needed to understand the reasons for high staff turnover and how to mitigate the negative effects. In a recent paper on efforts to implement the integrated management of childhood illness (IMCI) strategy on a national level in Peru, Huicho et al. [14] also found high staff turnover of trained health personnel (40%). To address this issue, the authors suggest incorporation of IMCI training into medical and nursing curricula along with in-service supervision.
Third, despite implementation results that fell short of the optimum, intervention centers demonstrated marked improvements over the control centers in their delivery of nutrition education. Improvements included the participation of all health services in nutrition education and a broader array of activities. Meriting mention is the delivery of complementary feeding messages to children 6 months, the group on whom the project concentrated. A trend toward differences between the intervention and control centers was seen (P = 0.079), and significance was reached for key message delivery (P = 0.001). Fidelity of message delivery with checking questions and educational materials was significantly greater in intervention centers than control centers. The messages of the intervention were simple, short and contained a specific behavior that addressed a prevalent obstacle to improved feeding (e.g. feeding soup before the more nutrient dense puree). The control center messages, while containing appropriate content, were often long, containing specific and non-specific information and used inconsistently among health personnel. This clear difference in message delivery along with improved message fidelity in the intervention centers likely accounts for the large difference in message recall found between caregivers during exit interviews.
Finally, we discuss key elements of the intervention. Data revealed that implementation in the GDM and nutrition consultation services exceeded that of medical consultation. This was likely a result of the intervention's focus on preventive nutrition education, which aligns more closely with the work of these services. Interview data from a spectrum of health personnel also identified nurses and nutritionists more closely with the intervention. Physicians were under greater time pressures to see patients and address their current concerns. Despite this, they voiced both a commitment to nutrition education within the health center and the importance of consistent message delivery among health personnel. Thus, our strategy to incorporate the intervention into all pediatric services, with a concentration in the GDM and nutrition services, was supported.
Interview data suggested the ability of health personnel to work cohesively (i.e. collective efficacy) toward the common goal of health center accreditation to be favorable factor for health center implementation. Similarly, in facilitating the implementation of school-based health programs, Buston et al. [12] identified the involvement of a cohesive group of implementers, and Gittelsohn et al. [15] described teachers working together and supporting each other. Partnership with the MOH, who had direct influence over the health centers, was a critical element in supporting implementation. Others have discussed the importance of administrative and district level support for successful implementation of school-based health interventions [12, 16, 17].
Examining implementation in an effectiveness trial, wherein health personnel had multiple demands in the government health centers, provided data on strong areas of the intervention and weaker areas where modification was likely needed [19].
Both quantitative and qualitative data suggest that problem-solving sessions were peripheral to other components. Although implementation levels varied for the remaining five components (messages, recipe fliers, food preparation demonstrations, anthropometry and group GDM), we recommend they remain together as the intervention. Together, they bring multiple opportunities for providing nutrition education and repeat use of messages (often used in conjunction with other components). In this way, as reflected in the voices of health personnel, we believe nutrition education changed the social environment of the health center. Nutrition education gained importance among the services and became the responsibility of all health personnel versus solely the nutritionist.
Individual components carried varying degrees of influence on caregiver behavior. For example, infant food preparation demonstrations involved experiential and observational learning, powerful influences on caregiver self-efficacy, critical to behavior change. Delivery of messages promoted caregiver knowledge of specific feeding behaviors and reinforced these through checking questions and statements of praise. Improvements in all components and in particular those with strong influences on self-efficacy may lead to stronger behavior change outcomes in caregivers. Dickey et al. [5] reported under-implementation of the intervention component in which a group of mothers fed infants after cooking a meal, designed to improve self-efficacy, provide social support and reinforce norms of good feeding practices. School-based studies of health interventions have also reported under-implementation of selected activities such as role-play, critical to building the self-efficacy of students and changing behaviors [12, 19].
Limitations of this study are acknowledged. Fidelity measures were not available for each measure of dose, which had we included, would have better characterized implementation. In hindsight, we might have developed fidelity measures (in addition to those listed in the health center accreditation criteria) for the specific purposes of the process evaluation. Due to time and resource constraints, most of the process evaluation data were collected during the first year of the 2-year trial. Despite these limitations, process evaluation was successfully carried out for this multi-component intervention and provided critical data on implementation. Future studies can learn from our experience to design evaluations that include both implementation and traditional study outcomes.
|
Conclusion |
|---|
|
TopAbstractIntroduction and backgroundMethodsResultsDiscussionConclusionConflict of interest statementAcknowledgementsReferences |
|---|
This paper examined the question of ‘what occurred’ during implementation of a successful intervention trial that improved infant growth in Trujillo, Peru. Quantitative results as well as insights from health personnel were provided on implementation. Although less than optimal, delivery of nutrition education was significantly improved in intervention centers as compared with control centers. The described intervention represents one example of many ‘international programs’ addressing behavior change to improve child health. Increasingly, these programs involve multiple components, providers and/or sites, creating a more complex implementation process. To date, the majority of evaluations has concentrated on outcomes only. Evaluation must now focus on both the implementation and outcomes of programs in order to understand how programs work to achieve results. This will help to maximize potential benefits from intervention programs and lead to more effective programs to improve child health and nutrition.
|
Conflict of interest statement |
|---|
|
TopAbstractIntroduction and backgroundMethodsResultsDiscussionConclusionConflict of interest statementAcknowledgementsReferences |
|---|
None declared.
|
Acknowledgements |
|---|
|
TopAbstractIntroduction and backgroundMethodsResultsDiscussionConclusionConflict of interest statementAcknowledgementsReferences |
|---|
We acknowledge and gratefully thank our team of data collectors, Rosario Bartolini for conducting interviews and providing insights, Claudia Cuba and Victor Montoya for data management assistance, the health staff and Trujillo MOH commission and the caregivers and infants of Trujillo. This research was funded by the United States Agency for International Development through the Johns Hopkins Family Health and Child Survival Cooperative Agreement and a pre-doctoral research fellowship (National Research Service Award) awarded to the principal author by the National Institute of Nursing Research, National Institutes of Health.
|
References |
|---|
|
TopAbstractIntroduction and backgroundMethodsResultsDiscussionConclusionConflict of interest statementAcknowledgementsReferences |
|---|
1. Fishman S, Caulfield LE, de Onis M. Childhood and maternal underweight. In: Comparative Quantification of Health Risks: Global and Regional Burden of Disease Attributable to Selected Major Risk Factors—Ezzati M, Lopez AD, Rodgers A, Murray CJL, eds. (2004) Cambridge, MA: Harvard University Press. 39–61.
2. Penny ME, Creed-Kanashiro HM, Robert RC, et al. Effectiveness of an educational intervention to improve young child nutrition: a cluster-randomised controlled trial. Lancet (2005) 365:1863–72.[CrossRef][Web of Science][Medline]
3. Baranowski T, Stables G. Process evaluations of the 5-a-day projects. Health Educ Behav (2000) 27:157–66.
4. Steckler A, Linnan L. Process evaluation for public health interventions and research: an overview. In: Process Evaluation for Public Health Interventions and Research—Steckler A, Linnan L, eds. (2002) 1st edn. San Francisco, CA: Jossey-Bass. 1–23.
5. Dickey VC, Pachon H, Marsh DR, et al. Implementation of nutrition education and rehabilitation programs (NERPs) in Viet Nam. Food Nutr Bull (2002) 23(Suppl. 4):75–82.
6. Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory (1986) Englewood Cliffs, NJ: Prentice Hall.
7. Bandura A. Exercise of personal and collective efficacy in changing societies. In: Self-efficacy in Changing Societies—Bandura A, ed. (1995) Cambridge, UK: Cambridge University. 1–45.
8. Creed-Kanashiro H, Uribe T, Bartolini R, et al. Improving dietary intake to prevent anemia in adolescent girls through community kitchens in a peri-urban population of Lima, Peru. J Nutr (2000) 130:459S–61S.[Web of Science][Medline]
9. Robert RC, Gittelsohn J, Creed-Kanashiro HM, et al. Process evaluation determines the pathway of success for a health center-delivered, nutrition education intervention for infants in Trujillo, Peru. J Nutr (2006) 136:634–41.
10. Health World. Organization. In: Operational Guidelines for Ethics Committees that Review Biomedical Research (2000) Geneva: World Health Organization. Report No. TDR/PRD/Ethics/2000.1.
11. World Medical Association. Declaration of Helsinki. Ethical Principles for Medical Research Involving Human Subjects (2000) Ferney-Voltaire, France: WMA.
12. Buston K, Wight D, Hart G, et al. Implementation of a teacher-delivered sex education programme: obstacles and facilitating factors. Health Educ Res (2002) 17:59–72.
13. Campbell MK, Motsinger BM, Ingram A, et al. The North Carolina Black Churches United for Better Health project: intervention and process evaluation. Health Educ Behav (2000) 27:241–53.
14. Huicho L, Davila M, Campos M, et al. Scaling up integrated management of childhood illness to the national level: achievements and challenges in Peru. Health Policy Plan (2005) 20:14–24.
15. Gittelsohn J, Merkle S, Story M, et al. School climate and implementation of the Pathways study. Prev Med (2003) 37:S97–106.[CrossRef][Web of Science][Medline]
16. Perry CL, Sellers DE, Johnson C, et al. The Child and Adolescent Trial for Cardiovascular Health (CATCH): intervention, implementation, and feasibility for elementary schools in the United States. Health Educ Behav (1997) 24:716–35.[Web of Science][Medline]
17. Smith DW, McCormick LK, Steckler AB, et al. Teachers' use of health curricula: implementation of growing healthy, project SMART, and the teenage health teaching modules. J Sch Health (1993) 63:349–54.[Web of Science][Medline]
18. Walsh RA, Redman S, Byrne JM, et al. Process measures in an antenatal smoking cessation trial: another part of the picture. Health Educ Res (2000) 15:469–83.
19. Davis M, Baranowski T, Resnicow K, et al. Gimme 5 fruit and vegetables for fun and health: process evaluation. Health Educ Behav (2000) 27:167–76.
Received on June 10, 2005; accepted on June 28, 2006
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||