Health Education Research, Vol. 18, No. 6, 754-769,
December 2003
© 2003 Oxford University Press
Organizational capacity and implementation change: a comparative case study of heart health promotion in Ontario public health agencies
School of Geography and Geology, McMaster University, Hamilton, Ontario L8S 4K1, and 1 Office of the Vice-President Research, University of Victoria, Victoria, BC V8W 2Y2, Canada.
e-mail: briley{at}rbj.ca
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionAppendix: Constructs and...References |
|---|
This paper reports the results of a comparative case study that examines factors influencing changes in implementation of heart health promotion activities in Ontario public health units. The study compared two cases that experienced large changes in implementation from 1994 to 1996, but in opposite directions. Multiple data sources were used, with an emphasis on secondary analyses of quantitative surveys of health units and other community agencies, and in-depth interviews of public health staff, collected as part of the Canadian Heart Health Initiative Ontario Project. Guided by social ecological and organizational theories, changes in implementation were explained by examining changes in (1) organizational predisposition to undertake heart health promotion activities, (2) organizational practices to undertake these activities, (3) other internal organizational factors and (4) external system factors. Findings show that in communities with diverse characteristics, implementation change was most strongly influenced by an interplay of changes in internal features of public health agencies; notably, leadership, structure and staff skills. Findings support a social ecological approach to health promotion by demonstrating the importance of the institutional context in the implementation change process, the interaction of individual (skills) and organizational (structure) levels in explaining implementation change, and community context in shaping the change process. Findings also reinforce the value of strengthening capacity within public health agencies and suggest further research on the implementation change process, especially in different systems and over longer periods of time.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Constructs and...References |
|---|
Moving from principles to widespread implementation of the new public health is a dominant theme in public health research and practice (Crichton, 1997
). Heart health promotion, which focuses on a population approach to prevention, community-based strategies and partnerships reflects key features of the new public health (WHO, Health and Welfare Canada and Canadian Public Health Association, 1986; Frenk, 1993). Efforts to enhance the implementation of heart health promotion therefore provide an opportunity to learn about implementation and dissemination processes for the new public health. The specific focus of this paper is on understanding changes in implementation of heart health promotion in Ontario public health units. The research contributes to the relatively new field of health promotion dissemination research, and can be used by policy makers and public health professionals to enhance implementation of community-based heart health promotion, and other similar health promotion activities.
Intervention research in health promotion has revealed many factors that influence implementation (Bracht, 1990; Guldan, 1996; Green and Kreuter, 1999). Within this broad area of research, two relatively recent directions are particularly relevant to this study. One direction is the focus on community and organizational capacity building (Goodman et al., 1997, 1998). The other is social ecological foundations in health promotion (Green et al., 1996; Newes-Adeyi et al., 2000). Both of these directions focus attention on the institutional (or organizational) context for health promotion, including the dynamic interaction between organizations and the environment in which they operate. With the introduction of the new public health, public health agencies are particularly important, with an emphasis on the relationship between various dimensions of organizational capacity and implementation of community-based health promotion activities (Frenk, 1993; Goodman et al., 1997; Hawe et al., 1997; McKinlay and Marceau, 2000).
Implementation research in public health agencies has focused in three main areas. One is defining and assessing the roles of public health agencies in the new public health (Sutcliffe et al., 1997; Bloom, 1999; Corso et al., 2000; McKinlay and Marceau, 2000), including the use of community approaches (Robinson and Elliott, 1999). A second area of research is to learn about strategies to strengthen the public health system, such as various types of technical assistance and training, and other ‘capacity-building’ activities (Roper et al., 1992; Rutten, 1995; Alciati, 1996; Lee and Paxman, 1997). A third research focus is understanding determinants of public health performance, including levels of implementation of health promotion activities (Champagne et al., 1993; Riley et al., 2001b). Little is known, however, about the implementation change process. Factors that promote change in health promotion implementation may not be the same as those that maintain levels of implementation [cf. (Rogers, 1995)]. Understanding the implementation change process within the public health system is vital to facilitating efforts to enhance implementation of the new public health and thereby advance the primary prevention of chronic disease.
This paper reports the results of a comparative case study that examines changes in implementation of heart health promotion activities. It builds on previous work of the Canadian Heart Health Initiative Ontario Project (CHHIOP) [carried out from 1994 to 1998 and described in detail elsewhere (Elliott et al., 1998a)]. Consistent with the Canadian Heart Health Initiative (Stachenko, 1996), an overall goal of CHHIOP was knowledge development on dissemination of effective heart health practices. The contributions of CHHIOP are synthesized elsewhere (Riley et al., 2001a). Main scientific contributions include developing constructs and indicators for health promotion dissemination research, operationalizing a social ecological approach, developing a longitudinal profile of organizational predisposition, capacity and implementation in public health agencies, and understanding factors influencing levels of these three main constructs.
CHHIOP contributions have been extended by two recent studies. Riley (Riley, 2003) combines social ecological and diffusion theories to examine the dissemination process using the case of heart health promotion in the Ontario public health system. The study reinforces recent conceptions of dissemination (e.g. iterative, multiple elements), and shows how social ecological theory can enhance explanation of the dissemination process (e.g. the interplay of organizational and environmental factors). The study also provides a temporal and developmental context for the CHHIOP study period from 1994 to 1998.
The second study extending CHHIOP contributions (Riley et al., 2001b) was a quantitative path analysis to understand the main determinants of 1997 levels of implementation of heart health promotion activities. Results were strong, with the final model explaining approximately half of the variance in implementation. The study begins to map structural relationships between various dimensions of organizational capacity that impact on levels of implementation and provides additional support for the explanatory power of social ecological theory.
The study reported in this paper also aims to explain variability in implementation of heart health promotion within Ontario’s public health system. Specifically, it examines implementation change. Whereas the path analysis methods were well suited to examine determinants of implementation observed at one point in time, case study and qualitative methods are appropriate for examining the change process (Yin, 1994). Using these methods, most insight is gained by comparing cases that experienced different degrees of change, including change in different directions. The two cases reported in this study experienced large changes in implementation relative to the average health unit in the province and in opposite directions.
Research setting
Ontario is located in central Canada and is the largest province with a population of about 11 million. Public health services in Ontario are primarily delivered through public health departments, each administered by an autonomous local board of health and regulated by provincial legislation and program guidelines. At the time of data collection, Ontario had 42 local health units. Public health programs were cost-shared by provincial and municipal governments, with a total combined annual budget of approximately $300 million and 4600 full-time equivalents (FTEs), or approximately 43 FTEs per 100 000 population (in 1997). Local boards ranged widely in per capita funding ($18 to $60 in 1997), population served (39 354 to 721 130 in 1997) and geographic location and size.
In 1989, public health in Ontario experienced a strategic shift in programming direction by focusing on non-communicable disease prevention. In addition to existing responsibilities, health units were required to provide programming in tobacco use prevention, nutrition promotion and physical activity promotion (Ontario Ministry of Health, 1989). At the same time, the provincial government supported a number of demonstration programs (typical duration approximately 5 years) for community-based health promotion. Five demonstration communities focused specifically on heart health promotion. During the study period from 1994 to 1996, Ontario was in a transition phase between demonstration projects and province-wide dissemination of ‘best practices’ in health promotion. CHHIOP was completed during this transition period.
Theoretical framework
The theoretical framework for this study (Figure 1) was adapted from previous work (Riley et al., 2001b). The framework reflects a diverse literature, but draws most heavily on a social ecological perspective (Green et al., 1996), recognizing the importance of health promotion institutions and the context in which agencies undertake health promotion activities. The organizational context (e.g. organizational culture, policies, processes) and the environmental context (e.g. political, social, economic) are represented in Figure 1 by internal organizational and external system factors, respectively.
|
) in implementation of heart health promotion activities.The outcome of interest in this study is change in implementation of comprehensive, community-based programs to prevent cardiovascular disease (CVD) and promote heart health. The framework identifies broad classes of variables known to influence implementation of health promotion programs by organizations. It suggests that change in implementation is most directly influenced by (1) change in organizational predisposition and (2) change in organizational practices. Predisposition refers to the motivation to undertake heart health promotion activities, and practices refer to various assessment, planning and related tasks of public health agencies to undertake heart health activities. Guided by a social ecological perspective and supported by the quantitative research to explain 1997 levels of implementation among Ontario public health units (Riley et al., 2001b
), the framework proposes that changes in organizational predisposition and practices are influenced by a variety of factors related to the internal organization (notably, human and financial resources, structures, processes, leadership) as well as the external system (notably, partnerships, support from the resource system and contextual factors). |
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Constructs and...References |
|---|
Case selection and description
Two cases were selected for this study on the basis of change in implementation of heart health activities from 1994 to 1996. Quantitative and qualitative data previously collected for CHHIOP were used to select cases (described under ‘Data sources’ below). Implementation scores derived from surveys completed by health units in 1994 and 1996 were used for case selection. Implementation was measured for 75 community-based activities, organized by risk factor and setting (see Table I). The 75 activities represent a comprehensive, population-based approach to heart health promotion and were defined using four dimensions: risk factors (tobacco use, physical inactivity, nutrition, general heart health), channels (schools, workplaces, health care settings, general community), approaches (education, environmental support, policy) and target groups. Baseline levels and change in total implementation were used for case selection. Total implementation scores were the mean level of implementation for all 75 activities.
|
One case experienced an increase in implementation (hereafter referred to as ‘Up’) and the other a decrease in implementation (referred to as ‘Down’). The two cases had similar baseline levels of implementation in 1994; slightly above the provincial average at a ‘low level of implementation’ (defined as implementation at below one-third of ideal implementation if resources were not limited) (Table II). Change in implementation from 1994 to 1996 was in opposite directions. Up advanced to a ‘medium level of implementation’ (between one- to two-thirds of ideal implementation), and Down regressed to somewhere between active planning and a low level of implementation. In both cases, the magnitude of change in overall implementation was above the provincial average of 0.2 points (SD 0.41). In Up, the 1-point shift was over 2 SD above the average change for health units in the province. Interviews of public health staff validated an increase in implementation from 1994 to 1996, but suggested the reported increase was slightly inflated. Perceptions of staff were supported by a lower implementation score of 2.9 reported just 1 year later (1997) in a similar organizational survey. In Down, the observed change in implementation from survey results was validated by staff perceptions and by a repeat score of 1.7 in a 1997 survey.
|
At the time of selection (1994), the two cases were similar on several other characteristics (Table III), especially the strongest predictors of level of implementation found in our path analytic study. Specifically, the two cases shared a relatively high motivation for heart health (e.g. predisposition, priority of heart health in the health unit), fairly effective organizational processes (e.g. assessment and planning practices, coordination of programs), similar resources and concepts for healthy lifestyles programs, and strong relationships with community partners and centers offering technical assistance. The cases also shared some demographic features, such as rate of population growth, a majority of English-speaking residents, geo-political re-structuring and levels on some CVD risk factors, including physical inactivity, hypertension and diabetes.
|
Despite these similarities, the cases varied on several internal organizational factors and external system factors. With respect to organizational factors, the main differences were the history and structure for heart health programs. Up did not have a designated heart health program. Rather, the health unit addressed heart health activities through its healthy lifestyles programs. Up had formed a multidisciplinary healthy lifestyles team in 1990 (1 year after the healthy lifestyles mandate was introduced) and, by 1994, self-identified as having a lead role (i.e. more involved than other agencies) in about two-thirds of community-based heart health activities. In contrast, the health unit in Down initiated a heart health program in 1987 and received enrichment funding from the provincial government as one of five heart health demonstration communities in Ontario. Unique to public health practice at the time, the heart health program was run by a community partnership, with strong coordination and program support from health unit staff. Community involvement was expanded from 1990 to 1994, mostly in response to requirements to receive the enrichment funds. By 1994, the heart health program was relatively arms-length from other health unit programs and operations, though health unit staff continued to provide coordination, organizational and program support. Operating within the partnership structure, the health unit reported a lead role in approximately 40% of heart health activities (30% lower than Up). Meanwhile, mandatory public health programs, including the healthy lifestyles programs, were largely accomplished through a traditional discipline-based structure within the health unit.
With respect to external system factors, the cases varied on many geographic, demographic, health service and health status characteristics. Up is situated in eastern Ontario, and consists of five counties and one city. It covers a large geographic area and is mostly rural. The population in 1994 was approximately 191 000 and included a large proportion of Francophones. Down is a single, urban municipality, covers a small geographic area and is situated in south-central Ontario. The population in 1994 was approximately 108 000, with almost half representing immigrant groups. Whereas income, education and health services were lower in Up compared to Down, circulatory death rates and some CVD risk factors, including smoking, excess fat in the diet and obesity, were higher.
Data sources
Quantitative and qualitative data previously collected for CHHIOP (during the time period 1994 to 1997) were used for this study. Secondary analyses of CHHIOP data were supplemented with information from staffing and budget reports for Ontario public health units (Public Health Branch, 1995, 1998), census data (1991 and 1996) and unpublished results from the 1990 Ontario Health Survey. The classes of variables in the theoretical framework were used to identify potentially relevant information from all data sources. Indicators are listed in the Appendix. CHHIOP sources include the following.
CHHIOP health unit surveys, 1994 and 1996
These surveys are described in detail elsewhere (Riley et al., 2001b). Surveys of all health units were completed in 1994 and 1996, with a 100% response in both years. Organizational level data were obtained on predisposition, capacity and implementation for heart health promotion. A mailed survey was completed under the direction of the Medical Officer of Health by those people who were most involved in managing and/or delivering heart health activities in the 1994 and 1996 calendar years. Acceptable levels of reliability and validity were demonstrated (Riley et al., 2001b).
CHHIOP survey of community agencies, 1997
This survey is described in detail elsewhere (Elliott et al., 2000). The main focus of the survey was on agency involvement in heart health promotion activities and partnerships for these activities. Agencies participating were those with a mandate in some aspect of heart health promotion, and included voluntary health agencies, school boards, municipal Parks and Recreation departments, and local YM/YWCAs. The survey was completed by telephone by the individual in each agency who was most familiar with heart health programming. Seven and six agencies participated in Up and Down, respectively.
CHHIOP qualitative study, 1997
A technical report is available on the 1997 qualitative study (Elliott et al., 1998b). In-depth, semi-structured interviews were conducted with a total of 38 public health professionals who were most involved in heart health promotion from a subset of eight health units. The primary purpose of the interviews was to explain changes in predisposition, capacity and implementation of heart health promotion activities from 1994 to 1996. Five individuals were interviewed from each of the two cases in the current study. Respondents were predominantly well-educated females between 36 and 55 years, from a range of professions within the health unit, including managers, public health nurses and health promotion officers/coordinators. Three of the five respondents from each location completed the health unit surveys in 1994 and 1996, including the same Medical Officer of Health in each case. Interviews were completed by the same two interviewers in both locations. A systematic thematic analysis of the in-depth interview data was facilitated using qualitative software (Ethnograph). The theme code set was developed using both deductive and inductive approaches, allowing researchers to address specific objectives, while allowing new ideas/themes to emerge from within the qualitative data. Reliability was assessed using inter- and intra-rater reliability, while validity was assessed using member-checking (Baxter and Eyles, 1997). Using these data, summary reports were developed for each participating health unit (n = 8). These reports summarized main findings by theme and included direct quotations from original transcripts which best articulated the view of individuals within the unit. For this study, the main data source was these summary reports. In addition, original transcripts were read and additional analyses were done using Ethnograph to explore selective themes in more detail.
Analysis
Single-case analyses provide a necessary foundation for case comparisons (Yin, 1994). Using the same interpretive process, explanations of the observed changes in implementation were developed for Up and Down independently. For each case, results were compiled for all indicators listed in the Appendix. Results were coded as supporting change in a positive direction, no change or change in a negative direction. To the extent possible, time ordering and relations between variables were also determined. The relative contribution of factors to the change process was judged based on strength of evidence. Strongest evidence was demonstrated by meeting all three of the following criteria; however, few variables were measured using both qualitative and quantitative methods:
(1) Qualitative findings (from in-depth interviews of public health staff) reported that the factor influenced a change in implementation. Factors reported with the greatest frequency and intensity were considered to provide the strongest evidence.
(2) Quantitative findings showed that the factor changed during the study period, in a direction which supported the observed change in implementation.
(3) Theoretical and/or empirical literature (other than from CHHIOP) support a link between the factor and the level of implementation.
A subsequent comparative case analysis examined similarities and differences in the single-case explanations, including the types of factors supporting and limiting implementation change, as well as the amount of change in these factors, the timing of changes and the interplay of factors. Possible implications of baseline differences in the cases were also examined.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Constructs and...References |
|---|
Single case explanations
Figures 2 and 3 highlight the major factors explaining implementation change in Up and Down, respectively. ‘Conditions’ did not change during the study period, but provide an overview of the circumstances within which change took place. ‘Processes’ initiated or contributed to the ‘Outcome’ of implementation change.
|
|
During the study period, Up experienced an accelerated increase in implementation relative to other health units. The increase was facilitated most by internal organizational factors, which were strong enough to overpower characteristics of the external system known to limit implementation, including significant demographic (e.g. low income and education, large Francophone population) and geographic (e.g. six municipalities, large area, mostly rural) challenges.
Following the introduction of the healthy lifestyles programs into the public health mandate in 1989, Up re-organized internally to form a healthy lifestyles team (Figure 2). It took approximately 5 years to realize the full benefit of the new structure, due to (1) overcoming resistance to the organizational change, (2) learning how to work as a multidisciplinary team and (3) hiring new staff to work as members of the team. The healthy lifestyles committee facilitated a multi-risk factor approach:
...within our health unit we try to develop initiatives that combine the three lifestyles together, like nutrition, tobacco and physical activity...
and a multidisciplinary approach:
...if we need to develop a project or a program, we usually try to get input from the public health nurses, public health educator, nutritionists, physical activity people, even environmental people.
On its own, however, the new organizational structure was insufficient to substantially increase implementation of healthy lifestyles programs. Hiring staff with non-traditional and diverse backgrounds to work as members of the multidisciplinary team was also needed:
...I find that the diversity of background is a definite asset to us. And I think that’s a big factor in implementation. You don’t have the socialized mentality of a health care provider when you go get people with different backgrounds.
Changes in the organizational structure and staff facilitated an increase in public health leadership for heart health promotion, an increase in coordination of heart health (i.e. healthy lifestyles) programs and an increase in the effectiveness of organizational practices (i.e. assessment, planning) supporting heart health promotion. These changes were supported by ongoing assistance from resource organizations and a consistently high priority given to heart health within the health unit.
For Down (Figure 3), 1994–1996 was a time to re-group/re-organize for healthy lifestyles promotion, in general, and for heart health, in particular. This was after an innovative and active history in heart health promotion as a health unit and as a community. Despite many factors promoting positive change (e.g. consistently high predisposition, partnerships with other agencies, support from the resource system, highly educated population) and a provincial trend of small, steady increases, Down experienced a large decrease in implementation of heart health promotion.
A major precipitating event was a health unit re-organization. During the study period, the health unit re-organized to form a healthy lifestyles team, with a main objective to enhance multiple risk factor programming rather than continue to address behavioral risk factors (e.g. smoking, physical inactivity) individually. As expected, coordination of heart health programs decreased during the organizational change process and increased once the healthy lifestyles team was established. The re-structuring within the health unit was the main stimulus for a decline in implementation, which was exacerbated by a number of other related factors.
Specifically, the health unit re-organization was more complex since it incorporated the integration of the heart health demonstration program. During the study period, the demonstration phase was winding down, and the major focus was on how to sustain the program:
...a decline in implementation is a reflection of, again, the program coming back into the health unit, the program being shut down for a period of about 18 months where they were concentrating on how they were going to sustain themselves as a separate program.
Together, the integration of the demonstration program and the health unit re-organization resulted in many staff changes and less emphasis on programming issues compared to previous years:
...in 95 and toward the end of 94, [the heart health program] lost its staff, there was great staff change-over. In 94 the program manager changed, and several staff changed and then there were staff hired on temporary contracts to do specific projects.
...The re-organization was also an influx of staff, because I came on and shortly after me there were seven people too. And our team’s gone through many changes in terms of management.
These changes led to a temporary lapse in health unit leadership for heart health. Without this leadership, priority given to heart health among community partner agencies also decreased, contributing to the observed decline in heart health implementation.
Implementation was further compromised from 1994 to 1996 because of a focus on planning for the Ontario Heart Health Program. The health unit and other community agencies were working together to develop a 5-year strategic plan as part of an application process for provincial funding. In addition to a heavy emphasis on planning, uncertainty regarding funding also contributed to less emphasis on implementation compared to previous years:
...the whole uncertainty is frustrating because it’s already end of the year...
...I mean it’s dragged on and on and that’s been difficult. It has held back promotion of my strategic plan because I don’t have any confirmation of dollars.
The decrease in implementation in the 2-year study period was a dramatic turn of events for a health unit that was a leader in heart health promotion. Public health staff predicted that the long-term result of the re-organization would be an increase in implementation:
I think if you look at this [overall implementation rating] a few months down the road you might see a climb. It won’t continue to drop.
Consistent with these predictions, implementation increased by 1.3 points from 1997 to 2000 (unpublished data from a survey of health units in 2001).
Case comparison
The major factors responsible for implementation change in both cases were strikingly similar, despite baseline differences in internal (health unit) and external environments. Implementation change was most strongly influenced by internal organizational factors; notably, public health leadership, organizational structure and skills of staff. Where present (in Up), public health leadership, and an established multidisciplinary structure consisting of staff with non-traditional and diverse health promotion skills resulted in positive implementation change. Where absent, and during a process of re-structuring within the health unit (in Down), negative implementation change was the result.
The cases were also similar on the typical role of the health department. Despite different histories of heart health programming, both health units described their typical role as coordination (e.g. bringing agencies together) and supporting implementation of community-based activities. With the exception of during the health unit re-structuring process in Down, the health units were typically more involved than other agencies (i.e. leadership role of the health unit).
Although the specific functions of the health unit were described in similar ways in the two cases, perceptions of the role of the health unit differed. In Up, public health staff more commonly described a ‘lead’ role of the health unit, whereas in Down, staff more commonly reported a ‘support’ role. This difference might be explained by the experience of Down as a demonstration project; specifically, the condition to enhance community involvement to receive enrichment funds. This condition may have resulted in a ‘lead’ role for public health being interpreted as undesirable and, thus, a greater tendency by staff to describe their roles as ‘support’.
Another difference between the two cases was the influence of the opportunity to receive provincial funding for heart health. The potential funding incentive had little to no impact on activities in Up:
It has been announced so long ago and it still hasn’t come. We’re not waiting for that.
In contrast, substantial frustration and negative consequences were reported by staff in Down. The differential response might be explained by different past experiences with special funding projects—Down was a heart health demonstration community, whereas Up was not. Communities that receive special funding may develop a dependence on those funds for sustained activity.
Up and Down also differed on some incentives for maintaining a high priority on heart health. Both Up and Down reported high rates of CVD and associated risk factors as motivating factors. Down reported an additional community responsibility, which seemed to stem from its innovative history in heart health, including its participation as a demonstration community.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Constructs and...References |
|---|
This study adds to both the science and practice of health promotion. The results contribute to knowledge on organizational aspects of health promotion implementation. They identify some core elements involved in the transformation of public health, including leadership, organizational structure and staff skills. These factors are all considered to be important dimensions of organizational capacity for health promotion [e.g. (Hawe et al., 1997
; Advisory Board of the Third International Heart Health Conference, 1998; Goodman et al., 1998)]. The dominant influence of internal organizational factors on the implementation change process is noteworthy, since in much of the health promotion literature the focus is on external organizational relationships (or partnerships).
With respect to partnerships, study findings contribute to the increasing dialogue on community coalitions for health promotion (Butterfoss et al., 1993; Stoto et al., 1996; Wolff, 2001); specifically, the role of public health agencies in community coalitions. While shared ‘ownership’ among participating members (agencies and/or citizens) remains a useful goal to optimize coalition functioning, specific contributions of various partners may (appropriately) vary depending on mandate, resources and individuals involved. In the case of public health agencies, this study suggests a leadership role is vital to the sustained implementation of community-based health promotion activities. Results suggest that leadership may be operationalized by building partnerships, coordinating efforts of partner agencies and providing program implementation support.
Results also provide support for a social ecological approach to health promotion. Specifically, they reinforce the importance of the institutional context (i.e. public health and other community agencies) for health promotion (Rutten, 1995; Green et al., 1996). Within the organizational setting, results illustrate the important interaction of individual level (i.e. staff skills) and organizational level (i.e. team structure) variables. Results also highlight the importance of community context in the implementation change process. In this study, history of heart health programming was particularly important. Notably, participation as a time-limited demonstration project influenced the magnitude of organizational re-structuring, the influence of external funding incentives, the community participation process and perceptions of community accountability.
The results do not support the proposed links between changes in organizational predisposition and practices, and change in implementation. Insufficient sensitivity of measures, relatively high baseline levels (in predisposition), a short 2-year time period for the study and lack of information prior to 1994 [e.g. since sustained levels on these variables may be better predictors of implementation change as seen in the quantitative path analysis reported in (Riley et al., 2001b)] may explain, in part, the absence of these factors in the explanation of implementation change. Proposed contributions of external system factors were also not found. The time period of 2 years may be too short to demonstrate the (potential) influence of long-standing circumstances such as demographic and geographic characteristics. These and other contextual factors may also be more relevant at later stages of implementation.
A major practical implication of the findings is to continue efforts to strengthen public health, with a focus on creating multidisciplinary teams consisting of staff with a variety of health promotion skills and developing strategies to enhance public health leadership while fostering strong community partnerships. Another implication is for the design of demonstration or pilot projects in health promotion. Terms and conditions attached to incentive funds should strive to maximize positive consequences (e.g. a sense of community responsibility) and minimize negative consequences (e.g. lack of integration with agency processes and activities).
Future research
Results of a single, pair-wise comparison are more suggestive than definitive. To increase the application of findings, similar research needs to be carried out in other health systems and with other issue areas (though still focusing on multiple, community-based interventions taking a population approach to prevention).
To build on the findings in this preliminary work, other similar studies could examine additional aspects of implementation (Champagne et al., 1993; Scheirer et al., 1995). For example, direct measures of implementation could be incorporated, other measures of agency performance could be included and the quality (or fidelity) of implementation could be assessed. Future studies should focus on the interplay of factors influencing implementation change and assess perceptions of the influence of various factors (e.g. barriers to implementation, history of collaborative relationships and programming). Future studies should also examine factors influencing change at later stages of implementation. During these later stages, different factors, such as participation of community partners, technical assistance or contextual factors, may have a stronger influence on changes in implementation than organizational shifts within public health agencies.
Another area for further research is to examine long-term consequences of demonstration projects. ‘Durability’ or ‘sustainability’ of demonstration projects have been examined (Thompson et al., 2000), but little attention has been given to understanding consequences experienced by participating agencies, and how those consequences influence related program initiatives and collaborative relationships.
|
Appendix: Constructs and indicators for explaining changes in implementation |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Constructs and...References |
|---|
Main data sources are noted using the following abbreviations in parentheses: HU = CHHIOP health unit surveys, 1994 and 1996; CA = CHHIOP survey of community agencies, 1997; Qual = CHHIOP qualitative study, 1997.
IMPLEMENTATION
• Change in level of implementation of activities for risk factors (tobacco, 18 items; nutrition, 24 items; physical activity, 17 items; general heart health, 15 items in 1994, 16 items in 1996); settings (schools, 16 items; workplaces, 16 items in 1994, 17 items in 1996; health care, 10 items; community at large, 32 items); and risk factor/setting combinations (mean score for items in each subscale, rated on a five-point scale; see Table I) (HU).
• Perceptions of factors influencing change implementation from 1994 to 1996 (Qual).
CAPACITY
• Change in level of effectiveness of 18 organizational practices to support heart health promotion activities and subscales for assessment (four items), planning (six items), supporting implementation (six items) and evaluation (two items in 1994, four items in 1996) (mean scores for four subscales and overall; items rated on a five-point scale from 0 = not aware activity was conducted to 4 = activity was conducted and was very effective) (HU).
• Perceptions of the influence of capacity on implementation change from 1994 to 1996 (Qual).
PREDISPOSITION
• Change in level of importance of 18 organizational practices to support heart health promotion activities and four subscales (as above for capacity) (mean scores for subscales and overall; items rated on a four-point scale from 1 = not at all important to 4 = very important) (HU).
• Perceptions of the influence of predisposition on change in implementation (Qual).
INTERNAL ORGANIZATIONAL FACTORS (includes Human and Financial Resources, Structures, Processes and Leadership)
• Change in funding per capita for public health programs and healthy lifestyles programs.
• Change in coordination of programs (rating from 1 = not well coordinated to 3 = very well coordinated) (HU).
• Change in proportion of activities in which the health unit had a lead role in implementation (ratings of ‘lead’, ‘support’ or ‘no’ role for community-based heart health activities) (HU).
• Change in priority of heart health within the health unit (rating of 1 = low priority to 3 = high priority) (HU).
• Perceptions of the influence of internal organizational factors on implementation change (Qual).
• Health unit involvement in heart health compared to other agencies (1996 only) (HU).
EXTERNAL SYSTEM FACTORS (including Partnerships, Support from the Resource System, and Contextual Factors)
• Level of involvement of community agencies in heart health programming for risk factors (tobacco, nutrition, physical activity, general heart health) (ratings on a three-point scale from 1 = not at all involved to 3 = very involved) (CA).
• Change in use of resource system (number of resource centers used each year) (HU).
• Change in usefulness of resource system (number of resource centers rated as ‘fairly’ or ‘very’ useful) (HU).
• Usefulness of CHHIOP (1996 only) (including surveys, reports, promotions, conference presentations, and interactions with project members) (sum of CHHIOP activities rated as ‘moderately’ or ‘very’ helpful) (HU).
• Change in priority of heart health in the community (rating from 1 = low priority to 3 = high priority) (HU).
• Change in geo-political structure.
• Change in demographics (size and characteristics of population, social and economic conditions).
• Change in health services (especially history of heart health programming in community).
• Change in health status (rates of CVD and risk factor profile).
• Perceptions of the influence of external system factors on implementation change (Qual).
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAppendix: Constructs and...References |
|---|
Advisory Board of the Third International Heart Health Conference(1998) The Singapore Declaration: Forging the Will for Heart Health in the Next Millennium. Declaration of the Third International Heart Health Conference, Singapore.
Alciati, M.H. (1996) Building an infrastructure for data-based cancer control planning and intervention implementation. Journal of Public Health Management and Practice, 2, 9–16.[Medline]
Baxter, J. and Eyles, J. (1997) Evaluating qualitative research in social geography: establishing ‘rigour’ in interview analysis. Transactions of the Institute of British Geographers, 22, 505–525.[CrossRef]
Bloom, B.R. (1999) The future of public health. Nature, 402 (6761 Suppl.), C63–C64.
Bracht, N. (ed.) (1990) Health Promotion at the Community Level. Sage, Newbury Park, CA.
Butterfoss, F.D., Goodman, R.M. and Wandersman, A. (1993) Community coalitions for prevention and health promotion. Health Education Research, 8, 315–330.
Champagne, F., Leduc, N., Denis, J.-L. and Pineault, R. (1993) Organizational and environmental determinants of the performance of public health units. Social Science and Medicine, 37, 85–95.[CrossRef][Web of Science]
Corso, L.C., Wiesner, P.J., Halverson, P.K. and Brown, C.K. (2000) Using the essential services as a foundation for performance measurement and assessment of local public health systems. Journal of Public Health Management and Practice, 6, 88–92.[Medline]
Crichton, A. (1997) Health Care: A Community Concern? Developments in the Organization of Canadian Health Services. University of Calgary Press, Calgary.
Elliott, S.J., Taylor, S.M., Cameron, R.J. and Schabas, R. (1998a) Assessing public health capacity to support community-based heart health promotion: the Canadian Heart Health Initiative, Ontario Project (CHHIOP). Health Education Research, 13, 607–622.
Elliott, S.J., Taylor, S.M., Wilson, K.S., Robinson, K. and Jolin, M.A. (1998b) Canadian Heart Health Initiative Ontario Project: A Qualitative Study of Factors Affecting Implementation of Heart Health Promotion Activities in Ontario Public Health Units. Prepared for the CHHIOP Research Group, Hamilton.
Elliott, S.J., Jolin, M.A. and Walker, R. (2000) Partnering in and for Heart Health Promotion: findings from a survey of community organizations. Canadian Journal of Public Health, 91, 229–233.[Web of Science][Medline]
Frenk, J. (1993) The new public health. Annual Reviews of Public Health, 14, 469–490.
Goodman, R.M., Speers, M.A., McLeroy, K., Fawcett, S., Kegler, M., Parker, E., Smith, S.R., Sterling, T.D. and Wallerstein, N. (1998) Identifying and defining the dimensions of community capacity to provide a basis for measurement. Health Education and Behavior, 25, 258–278.
Goodman, R.M., Steckler, A. and Alciati, M.H. (1997) A process evaluation of the National Cancer Institute Data-Based Intervention Research (DBIR) Program: a study of organizational capacity building. Health Education Research, 12, 181–197.
Green, L.W. and Kreuter, M.W. (1999) Health Promotion Planning: An Educational and Ecological Approach, 3rd edn. Mayfield, Mountain View, CA.
Green, L.W., Richard, L. and Potvin, L. (1996) Ecological foundations of health promotion. American Journal of Health Promotion, 10, 270–281.[Web of Science][Medline]
Guldan, G.S. (1996) Obstacles to community health promotion. Social Science and Medicine, 43, 689–695.
Hawe, P., Noort, M., King, L. and Jordens, C. (1997) Multiplying health gains: the critical role of capacity building within health promotion programs. Health Policy, 39, 29–42.[CrossRef][Web of Science][Medline]
Lee, P. and Paxman, D. Reinventing public health. Annual Reviews of Public Health, 18, 1–35.
McKinlay, J.B. and Marceau, L.D. (2000) To boldly go. American Journal of Public Health, 90, 25–33.
Newes-Adeyi, G., Helitzer, D.L., Caulfield, L.E. and Bronner, Y. (2000) Theory and practice: applying the ecological model to formative research for a WIC training program in New York State. Health Education Research, 15, 283–291.
Ontario Ministry of Health(1989) Mandatory Health Programs and Services Guidelines for Local Boards of Health. Queen’s Printer, Toronto.
Public Health Branch, Ontario Ministry of Health(1993) 1992 Funding and Staffing Reports for Ontario Boards of Health. Queen’s Printer, Toronto.
Riley, B.L. (2003) Dissemination of heart health promotion in the Ontario public health system: 1989–99. Health Education Research, 18, 15–31.
Riley, B.L., Elliott, S.J., Taylor, M., Cameron, R. and Walker, R. (2001a) Dissemination of heart health promotion: lessons from the Canadian Heart Health Initiative Ontario Project. IUHPE—Promotion and Education, Suppl. 1, 26–30.
Riley, B.L., Taylor, S.M. and Elliott, S.J. (2001b) Determinants of implementing heart health promotion activities in Ontario public health units: a social ecological perspective. Health Education Research, 16, 425–441.
Robinson, K.L. and Elliott, S.J. (1999) Community development approaches to heart health promotion: A geographical perspective. Professional Geographer, 51, 283–295.[CrossRef]
Rogers, E.M. (1995) Diffusion of Innovations, 4th edn. Free Press, New York.
Roper, W.L., Baker, E.L., Dyal, W.W. and Nicola, R.M. (1992) Strengthening the public heath system. Public Health Reports, 107, 609–615.[Web of Science][Medline]
Rutten, A. (1995) The implementation of health promotion: a new structural perspective. Social Science and Medicine, 41, 1627–1637.
Scheirer, M.A., Shediac, M.C. and Cassady, C.E. (1995) Measuring the implementation of health promotion programs: the case of the Breast and Cervical Cancer Program in Maryland. Health Education Research, 10, 11–15.
Schwartz, R., Smith, C., Speers, M.A., Dusenbury, L.J., Bright, R., Hedlund, S., Wheeler, F. and Schmidt, T.L. (1993) Capacity building and resource needs of state health agencies to implement community-based cardiovascular disease programs. Journal of Public Health Policy, 14, 480–494.[CrossRef][Medline]
Stachenko, S. (1996) The Canadian Heart Health Initiative: dissemination perspectives. Canadian Journal of Public Health, 87 (Suppl. 2), S57–S59.[Medline]
Stoto, M.A., Abel, C. and Dievler, A. (eds) (1996) Healthy Communities: New Partnerships for the Future of Public Health. National Academy Press, Washington, DC.
Sutcliffe, P.A., Deber, R.B. and Pasut, G. (1997) Public health in Canada: a comparative study of six provinces. Canadian Journal of Public Health, 87, 246–249.
Thompson, B., Lichtenstein, E., Corbett, K., Nettekoven, L. and Feng, Z. (2000) Durability of tobacco control efforts in the 22 Community Intervention Trial for Smoking Cessation (COMMIT) communities 2 years after the end of intervention. Health Education Research, 15, 353–366.
Wolff, T. (ed.) (2001) Community Coalition Building—Contemporary Practice and Research. American Journal of Community Psychology, 29(2).
WHO, Health and Welfare Canada and Canadian Public Health Association (1986) Ottawa Charter for Health Promotion. Canadian Public Health Association, Ottawa.
Yin, R.K. (1994) Case Study Research: Design and Methods, 2nd edn. Sage, London.
Received on February 22, 2002; accepted on August 6, 2002
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. Whitelaw, C. Martin, A. Kerr, and E. Wimbush An evaluation of the Health Promoting Health Service Framework: the implementation of a settings based approach within the NHS in Scotland Health Promot. Int., June 1, 2006; 21(2): 136 - 144. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Farnkvist and L. Weinehall Assessment of intervention intensity: Experiences from a small-scale Swedish cardiovascular disease prevention programme Scand J Public Health, May 1, 2006; 34(3): 279 - 286. [Abstract] [PDF]
|
|
|
|
 |
|
 D. S. McCall, I. Rootman, and D. Bayley International School Health Network: an informal network for advocacy and knowledge exchange Global Health Promotion, September 1, 2005; 12(3-4): 173 - 177. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||