Health Education Research Advance Access originally published online on July 16, 2007
Health Education Research 2008 23(2):238-248; doi:10.1093/her/cym025
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Program characteristics and organizational factors affecting the implementation of a school-based indicated prevention program
1 Pacific Institute for Research and Evaluation, Chapel Hill Center, Chapel Hill, NC 27514, USA
2 Department of Health Behavior and Health Education, University of North Carolina at Chapel Hill, NC 27599, USA
3 Council on Alcoholism and Drug Abuse, Santa Barbara, CA 93101, USA
* Correspondence to: A. Steckler. E-mail: steckler{at}email.unc.edu
 |
Abstract |
|---|
 Top Abstract MethodsResultsDiscussionFundingConflict of interestAcknowledgementsReferences |
|---|
Reconnecting Youth (RY) is a school-based drug prevention program designed to address academic, substance use and mood management goals among youth at risk of dropping out of high school. This paper presents the organizational factors and RY program characteristics that either promoted or hindered the implementation of the program during a randomized controlled effectiveness trial in 10 schools in two school districts in the United States. Data were collected using surveys and interviews from teachers and school and district staff who participated in the implementation of the RY program in these schools. Results suggest that certain RY program characteristics made it difficult to implement. Small class size, resource-intensive procedures for student selection and recruitment and special training, qualities and skills needed to be an effective RY teacher meant that schools had to significantly change their usual practices to implement the program. Organizational barriers included a lack of financial resources and leadership support for program implementation, and low priority for non-academic courses for high-risk students. Transient student populations, staff turnover and district-wide scheduling and curriculum changes all resulted in high levels of organizational turbulence at most schools, further hindering program implementation.
Effectiveness trials, conducted by independent researchers, are a critical step in understanding the impact of prevention programs in real-world settings [1]. Such trials help researchers identify facilitators and barriers to successful program implementation, adoption and diffusion and illuminate how program characteristics and organizational factors influence program implementation and adoption [2, 3].
For school-based prevention programs, evidence of program efficacy reported by program developers does not ensure that schools can successfully implement and adopt the program. Researchers have identified a combination of factors that influence the adoption, implementation and diffusion of prevention programs in schools [4–7]. Barriers to program adoption include decentralized decision making in school districts, low levels of funding for prevention, lack of adequate infrastructure and lack of program guidance [6, 8–10]. Barriers to implementation include program complexity, lack of fit between the program goals and school mission and usual practices, lack of prevention infrastructure, lack of teacher training and support, lack of program materials, inconsistent staffing and inadequate district and state support [6, 11, 12].
In a randomized controlled trial, we tested the effectiveness of Reconnecting Youth (RY) in 10 schools in two school districts in the United States. RY is a school-based prevention program for youth in grades 9 through 12 who are at risk for school dropout. It is an indicated program, that is, it targets students who are already experimenting with drugs or other risk-related behaviors. It was designed to ‘re-connect’ these youth to school by helping them develop a greater sense of personal control, adaptive coping behaviors and interpersonal communication and relationship skills [13–15]. RY uses a theoretical framework based on strain, social learning and control theories and addresses the interdependent links between decreased drug involvement, increased school performance and decreased emotional distress [14]. Program developers recommend that schools identify eligible students by using specific criteria, that is, students with grade point average <2.3 or sharply declining, and in the top 25% of truancy for their grade level. Classes are limited to 10–12 students enrolled for a semester long course taken for elective credit. The RY curriculum is highly structured and requires schools to select a teacher who has a reputation of working well with at-risk students. Developers recommend that each teacher should receive extensive training by certified RY trainers on how to teach the curriculum, with an emphasis on how to create a positive and supportive peer group, counteract negative peer interactions and teach pro-social skills in the class. RY was originally tested and shown to be efficacious in a trial conducted during early 1990s [13–15].
During the effectiveness trial, we examined the degree to which the RY program was faithfully implemented according to the developer's protocols. The findings from this examination have been previously reported [16]. We found that the RY program was implemented with high fidelity at all 10 schools. Overall, teacher adherence and student exposure were high. Ninety percent of the teachers covered all core lessons. On average, students attended classes 79% of the time with only a small number of students (n = 11) attending just a few classes.
During the effectiveness trial, we also examined student-level program effects. We have previously reported findings from our intent-to-treat analyses [17]. Despite the high fidelity of implementation, we found that there were no main program effects at immediate post-intervention. At the 6-month follow-up, only three negative main program effects (conventional peer bonding, high-risk peer bonding and fewer pro-social weekend activities) were found [16, 17].
During the effectiveness trial, we also conducted an organizational diffusion study to examine the program characteristics and organizational factors that influenced RY implementation. We used diffusion of innovation theory [18] as a framework to guide this study. This manuscript reports findings from the organizational diffusion study, that is, organizational factors and program characteristics that either promoted or hindered the implementation of the RY program during the randomized controlled effectiveness trial.
|
Methods |
|---|
|
TopAbstractMethodsResultsDiscussionFundingConflict of interestAcknowledgementsReferences |
|---|
Sample and setting
We conducted the randomized controlled effectiveness trial in two large urban school districts in the United States with total high school populations of 14 882 and 18 931, respectively. One school district was located in a large city in the Southwest (District A) and the other in a large metropolis on the West coast (District B) of the United States. These school districts were invited to participate because of their size (large enough to accommodate the effectiveness trial) and because of researcher contacts within the districts. In each district, large high schools (>1000 students) were invited to participate; 8 schools in District A and 10 schools in District B met this criterion. Of these, five schools in each district agreed to participate. Administrators at the remaining three schools in District A and five schools in District B reported the following reasons for declining participation: (i) school already had a program for high-risk students, (ii) school did not have a teacher who could be allocated to teach an elective course with a small class size, (iii) school did not have enough at-risk students to make it worthwhile to offer the program, (iv) a one-semester course did not fit the school's scheduling system or (v) school was undergoing organizational transition and unable to implement a new program.
At each participating school (n = 10), administrators selected one to three teachers to attend an intensive 4-day training workshop, conducted by the RY program developers' staff, to learn how to teach RY. Most teachers also completed an additional day of follow-up training. At each school, students who agreed to participate in the effectiveness trial were randomly assigned to an RY class (intervention group) or to no special class (control group). Four hundred and seventeen students (185 in District A and 232 in District B) were enrolled in RY classes. In each district, 
49% of students were male. In District A, 87% of participants were Hispanic, 9% were black, 4% were white and 4% were American Indian or other race. Ninety percent qualified for the federal-free or reduced lunch program. In District B, 40% were Asian/Pacific Islander, 21% were Hispanic, 15% were black, 10% were white and 12% were American Indian or other race. Sixty-one percent qualified for the federal-free or reduced lunch program.
From spring 2002 through fall 2004, a total of 45 RY classes (21 in District A and 24 in District B) were taught by 18 (District A) and 10 (District B) teachers. Each school offered one or two classes per semester. The class size ranged from 5 to 15 students with an average of nine students per class.
For the present organizational diffusion study, we collected data through self-administered surveys and in-person qualitative interviews with RY teachers and staff who played key support, coordination or decision-making roles (referred to as ‘school staff’ for the remainder of this paper). We also interviewed district administrators who played a key role in the school district's decision to participate in the effectiveness trial (referred to as ‘district staff’ for the remainder of this paper).
Data collection methods
A research study team member collected data during two site visits in each district. The first visit coincided with the end of the first semester of RY implementation in each district. The second visit was at the end of the third semester of RY implementation in each district. This allowed us to collect data from teachers, school and district staff on their initial impressions and experiences with RY as well as their assessment of the factors that facilitated and/or hindered the RY implementation during the trial.
To guide our selection of survey and interview questionnaire, we used three diffusion of innovation concepts—perceived relative advantage, complexity and compatibility—to understand the characteristics of the RY program that influenced its implementation [18]. An additional three concepts—school capacity, school turbulence and leadership and administrative support—framed our understanding of how organizational factors influenced the implementation of the program [18, 19]. The surveys and interviews asked questions about perceived effectiveness of RY training, a school's capacity to implement RY, school leadership and staff support for RY, communication within schools about RY, costs of implementing RY, school climate and culture, characteristics of the RY program and school policies in regards to at-risk students [20, 21].
In general, all respondents were asked the same questions. However, there were a few minor differences specific to role. For example, we asked teachers ‘What are some of the challenges to teaching RY?’ while we asked school staff ‘What are some of the challenges to implementing the RY program in your school?’ Since teachers were the only ones to undergo training to teach the curriculum, we asked questions about training effectiveness only of them. The instruments were pilot tested in 2002 in a school that was using the RY program but was not participating in the effectiveness trial.
A large majority of the questions during the two data collection periods were similar. However, there were a few differences specific to each round. For example, questions pertaining to the perceptions of training effectiveness were asked only during the first round to limit the length of the questionnaire and to minimize burden on participants who work in busy and demanding school environments. Some questions were asked only during the second data collection period, such as the potential of continuation of the RY program. RY teachers, school staff and district staff were surveyed and interviewed during both data collection periods. As shown in Table I, 16 surveys and 22 interviews were conducted in District A and 16 surveys and 23 interviews were conducted in District B.
|
Data management and analysis
The quantitative survey data were entered into a Microsoft Access database and the database was checked for accuracy. These data were transferred to SPSS® 12.0.1 [22] and Microsoft Excel for data management and analysis. The qualitative interview data were transcribed from audiotape recordings into Microsoft Word documents. The transcripts were reviewed and corrected by the interviewers for accuracy. Transcripts were imported into the N6 software [23] for data management and analysis.
We used qualitative content analysis which is a method for identifying, coding and categorizing according to predetermined themes [24, 25]. We use a priori concepts derived from the diffusion of innovation framework—e.g. relative advantage, turbulence and capacity—to categorize the data. We also compared text passages related to particular schools from each respondent and summarized this information for each school. We also compared text passages related to each district from respondents within a district and summarized this information for each district.
Relative advantage of the RY program compared with other programs for at-risk students, on a five-point scale (strongly disagree to strongly agree), was assessed by asking ‘The RY program is more effective in supporting or assisting at-risk students than are other programs intended to help these same students'. In the interviews, teachers, school staff and district staff identified the aspects of the RY program in addressing at-risk students’ needs compared with other programs.
Compatibility of the program with the school was assessed by summing responses to three questions ‘the RY program is important in helping our school achieve its goals and purposes’, ‘the RY program fits well with the school's overall efforts to support or assist high-risk students’ and ‘the RY program has been well received by most of the students'. In the interviews, teachers and school staff described whether the program was compatible with their school's efforts to assist at-risk students and whether the program was compatible with their school's teaching and administrative practices. District staff reported on whether the RY program was compatible with district-wide efforts to assist at-risk students.
Complexity of the program was assessed by asking teachers to rate, on a five-point scale (strongly disagree to strongly agree), three specific characteristics of the program. Teachers rated the difficulty to (i) learn how to teach, (ii) actually teach and (iii) teach the ‘RY way’ on a daily basis. In the interviews, teachers and school staff discussed the challenges they faced during program implementation at their school. The district staff discussed the administrative challenges that schools within their district encountered in implementing the RY program and how these challenges affected schools' ability to implement the program.
School capacity was determined by combining survey responses assessing individuals' agreement, on a five-point scale (strongly disagree to strongly agree), with two statements: (i) your school has the resources it needs in order to effectively teach the RY program and (ii) this school has adequate staff resources to effectively implement the RY program. Higher scores reflected higher capacity of the school to implement RY program.
Turbulence experienced by schools was derived from interview responses on changes occurring internally and externally that likely affected each school's ability to absorb additional changes required to implement the RY program.
School leadership and administrative support were measured by combining responses assessing individuals' agreement, on a five-point scale (strongly disagree to strongly agree), with three statements: (i) ‘School district administrators consider RY an important program’, (iii) ‘My colleagues are supportive of the RY program and provide assistance when they can’ and (iii) ‘There is one person who has provided strong leadership and support for RY in this school’. Higher scores reflected better leadership at the school to implement RY program.
|
Results |
|---|
|
TopAbstractMethodsResultsDiscussionFundingConflict of interestAcknowledgementsReferences |
|---|
Program characteristics that influenced implementation
Relative advantage
Relative advantage is the degree to which an innovation is perceived by adopters as being better than other similar programs [18]. Results from the survey indicated that slightly less than half (n = 15 or 47%) of the 32 respondents agreed that RY was more effective in supporting or assisting high-risk students than other programs in their schools. Another 47% neither agreed nor disagreed. There were no significant differences in responses between teachers and school staff or between districts.
In the interviews, all participants described the RY program as addressing high-risk students' needs that were previously unaddressed by other programs. Teachers and school staff explained that what made RY different from other programs for high-risk students were its comprehensiveness, intensity and emphasis on social support. They reported that other programs engaged high-risk students only on a periodic basis, in larger classes, were academic in focus and were often times offered at a location other than the school. In contrast, students enrolled in the RY program attended the RY class daily during regular school hours at school, and since the class size was small, they were seen as having received considerable amount of attention from the teacher. RY also differed from other programs in that it explicitly sought to establish supportive, pro-social relationships among students and between students and the teacher. Some teachers and several school staff viewed the RY program as helping high-risk students become equipped to succeed not only academically but also to develop important life skills.
Complexity
Complexity refers to the perceived ease or difficulty of implementing a program in an organization [18]. School's adoption and ongoing use of an innovative program such as RY is likely to be contingent on users' perception of the complexity of the program. The more complex an innovation is perceived to be, the less likely it is to be adopted and sustained. Respondents identified several challenging aspects of RY. Teachers reported challenges in learning to teach the RY curriculum in spite of receiving recommended training. On the quantitative survey, a majority of teachers reported being well prepared to teach RY as a result of the 4- to 5-day intensive training (60% in District A versus 90% in District B; P = 0.3, Fisher's exact test). In addition, 35% of the teachers (3 of 10 in District A, 4 of 10 in District B) reported that RY was difficult to learn how to teach while 60% of the teachers (4 of 10 in District A, 8 of 10 in District B) reported that teaching RY lessons the ‘RY way’ was difficult.
During interviews, all teachers described the training in favorable terms, and viewed the trainers as being helpful and motivating. In spite of this training, several teachers reported experiencing challenges in teaching RY. They stated that they were not well prepared for the amount of time and the intense planning required. All teachers reported difficulties adhering to the curriculum's timeline and content, and found it challenging to complete the curriculum in one semester. Some teachers reported that it was difficult to teach the RY curriculum on a daily basis in a manner consistent with the curriculum. Overall, 12 of the 20 teachers (60%) expressed dissatisfaction with the RY curriculum. They reported that some of the curriculum's content and structure was rigid, dense, and at times repetitive, and hence, difficult to follow—a source of frustration to themselves and irrelevant to their students' lives. Seven of the 20 interviewed teachers also mentioned that parts of the curriculum seemed out-dated.
In addition, in 16 of the 20 (80%) interviews, teachers reported that teaching RY required new teaching, counseling and group facilitation skills, which were different from the skills required to teach academic courses. They also found that these skills were difficult to acquire, maintain and use on an ongoing basis in spite of the training and support they received. Additionally, class preparation and follow-up with the students and parents were time consuming, and most teachers found this to be an unanticipated burden. Further, half of the teachers (10 of 20) reported that essential components of the RY curriculum, for example, social support and group work, and role-play exercises, were difficult to conduct with the required regularity. In short, all teachers reported that it was difficult to conduct the class exactly as intended and indicated a need for changes to the curriculum.
During interviews, district and school staff reported several administrative challenges in implementing the program due to the complexity of the RY program. They reported that the process of identification, recruitment and placement of students for the RY class was a time-consuming process. School staff responsible for coordinating RY found that the systematic process of selecting and recruiting high-risk students, in spite of clear procedural instructions and extensive programmatic support from the research team, was cumbersome, complex and labor intensive.
Furthermore, schools faced difficulty with the identification, recruitment and retention of RY teachers. The program recommends carefully selecting existing, skilled teachers in the schools with a reputation of working well with high-risk students, and training them to teach RY. Most schools were unable to select a teacher according to these recommendations. Non-teaching staff, such as coaches, retired teachers or staff from school-based wellness centers were assigned to teach RY in 8 of the 10 schools during the course of the effectiveness trial.
School administrators reported that once a teacher was selected to teach RY, they had to find another teacher to take over the classes or activities that this teacher would be leaving to teach RY. In District A, this usually meant shifting students to other classes, thus, increasing class size for other teachers. In District B, where health center staff served as RY teachers, services of the school-based health center were affected. In sum, a majority of school and district staff, in addition to the RY teachers, reported that several aspects of the RY program were complex to implement.
Compatibility
Compatibility refers to the extent to which an innovation is perceived by adopters as being consistent with their needs, past experiences and existing values [18]. Where an innovative program is perceived as fitting the culture of the school and practices of individual teachers, it is said to be compatible with a given school [26, 27]. Responses to survey questions suggest differences in the perceived compatibility of RY to participants in the study schools in the two districts. Significantly more staff and teachers in District A (15 of 16 = 94%) than in District B (8 of 16 = 50%) agreed with the statement ‘the RY program is important in helping our school achieve its goals and purposes’ (P = 0.02, Fisher's exact test). More staff and teachers in District A (88%) than in District B (56%) agreed with the statements ‘the RY program fits well with the school's overall efforts to support or assist high-risk students’ and ‘the RY program has been well received by most of the students’, although differences were marginally significant (P = 0.1). Thus, staff and teachers in District B perceived the RY program as less compatible with their schools compared with those in District A.
Furthermore, an area of potential incompatibility between the RY program and school practices resulted in the difficulty reported by guidance counselors in finding time in high-risk students' schedules to attend the RY class. Counselors reported that schools were required to consider overall student academic achievement and graduation as their primary priorities, given the testing requirements under the federal No Child Left Behind Act of 2001 [28]. This meant that fewer resources were available for the implementation of RY. Additionally, counselors believed that the high-risk students who were low on graduation credits in required academic courses did not have the time to attend an elective non-academic course such as RY. As a result, 66% of RY classes in District A and 50% in District B were smaller than the recommended class size. In sum, teachers and school staff reported that the non-academic RY class targeting a small number of high-risk students was incompatible with the priorities and usual practices of the participating schools.
Organizational factors that affected implementation
School capacity
School capacity refers to the skills of staff, such as teachers and administrators, and other resources available for implementing an innovative program [5]. The mean score for all schools in both districts was 3.30 with no significant difference between the two districts (3.25 in District A, 3.34 in District B). However, school capacity varied greatly within each district with school scores ranging from 1.5 to 5. This wide variation is probably a true reflection of how schools, even within a school district, can vary in their capacity to implement a new program. Some schools have more facilities, more staff, more administrative support, etc. than other schools within a district.
During interviews, teachers and school staff at all but three schools described their schools as generally being capable of implementing the RY program. However, school administrators often cited state budget shortfalls and district-wide funding cuts as the primary cause of reduced resources to support RY. Administrators reported that it was difficult to justify RY's small class size.
Inadequate resources were also related to difficulty finding classroom space at several schools. During interviews, three of the six school staff members in District A reported that it was very difficult to find classroom space to teach RY. When asked to rate the ease or difficulty of finding classroom space for the RY class, three of the six school staff members in District B did not respond, while one reported that it was very difficult. Teachers at two schools in District A and one school in District B, reported that classroom space allocated to teach RY was not conducive to group work either due to lack of adequate space, lack of heat and air conditioning or limited number of tables and chairs.
School turbulence
School turbulence refers to planned and unplanned changes that occur both within and external to a school [26]. Turbulence affects the ability of a school to absorb additional changes, such as adoption, implementation and maintenance of an innovative program like RY. Results from the qualitative interviews revealed several sources of turbulence in all participating schools. For example, a highly transient student population at all schools made it difficult to predict how many of the students enrolled in RY would actually attend the class. This affected class size and sometimes created a ‘mix’ of students that was not conducive to group formation and made it difficult for some teachers to facilitate RY class activities.
Schools in District A underwent major reorganizations that created significant turbulence during the study period. All District A schools were reorganized into smaller school communities, known as ‘schools-within-a-school'. Additionally, there was a district-wide change in the daily schedule from eight to seven class periods, and the continuation and increasing reliance on an academic credit recovery programs (alternative, academic support programs that allow students to recover credits by completing self-paced computer-based classes). These system-wide changes made it difficult and ultimately impossible for school administrators to allocate sufficient time and resources to the RY program. Another important source of turbulence that affected some schools’ ability to implement and sustain RY was staff turnover. For example, at one District B school, all staff essential to the RY program implementation left after the first semester, including the assistant principal and teacher; in a District A school, there was a sudden departure of the principal during the first semester of RY implementation.
Teacher turnover also created turbulence that affected school's capacity to implement the RY program. When asked in the qualitative interviews about their likelihood of teaching RY again, 50% of the teachers in each district reported that they were unlikely to teach RY again. Teachers offered various reasons: three teachers were making a career change, two teachers from each of the districts reported that teaching RY had not been a good experience for them and three teachers in District A and two in District B had specific problems with the curriculum that dissuaded them from continuing to teach RY, including a lack of flexibility and repetitiousness, and the relatively high preparation time required.
Leadership and administrative support
Leadership refers to the actions and roles of key school personnel that promote adoption, implementation and maintenance of an innovation [29]. The mean score in both districts was 3.64 (3.78 in District A, 3.50 in District B), with scores ranging between 2.5 and 5 among individual schools. Only a third of school staff and teachers in both districts agreed that district administrators considered RY important. Furthermore, only 50% of teachers in the two districts reported that their principal was supportive of RY.
In the interviews, teachers and district administrators described a lack of counselor support for the RY program. In District A, inadequate communication with the guidance counselors about the RY program was cited at two schools. As a result, according to the teachers, the counselors questioned the need for a special program, such as RY, at their school and did not offer the necessary administrative support to the teacher for program implementation. At a third school in District A, teachers reported that in spite of the principal's decision to participate in the study, the assistant principal and counselors did not support RY and hence the student recruitment efforts and the teachers were not supported. In District B, counselors' at one school were resistant to the idea that there were high-risk students at their school and at another school a change in leadership, including departure of the assistant principal, health coordinator, nurse and counselor, meant that there was a total lack of leadership and administrative support for the RY program at this school. In sum, the RY program lacked sufficient leadership and administrative support in both districts and at several schools.
|
Discussion |
|---|
|
TopAbstractMethodsResultsDiscussionFundingConflict of interestAcknowledgementsReferences |
|---|
In discussing the results of this study, it is important to keep in mind that RY is an ‘indicated’ program, that is, it is directed at an identified sub-group of high-risk students, not the general student population. In contrast, many school-based prevention programs are ‘universal’ or directed at the general student population. Indeed, there are very few evidence-based drug prevention programs of any type available at the high school level. RY is also a resource-intensive prevention program. It is a semester-long class targeted to a small number of students identified to be at-risk of high school drop out using specific selection criteria. It requires a teacher who is committed to working with high-risk youth and is willing to undergo 4–5 days of training. Furthermore, the RY class requires a coordinator to provide extensive support in the form of ongoing consultation with the teacher and with the student selection process. Because of these unique characteristics, our findings may not be generalizable to the more common universal prevention programs, integrated into regular classes after minimal teacher training. Nevertheless, our findings can help prevention researchers and practitioners understand how program features and organizational factors support or hinder the implementation of school-based prevention programs.
In a recent survey of 104 school districts in 12 states, Hallfors and Godette [6] found that only 19% were using evidence-based prevention programs with fidelity. Our study of the RY program may explain why more school districts do not adopt, implement and sustain some evidence-based programs. On the one hand, we found that teachers and school staff perceived RY positively and saw it as filling a gap and meeting the needs of high-risk students. On the other hand, teachers and school staff perceived RY as a complex program, requiring teachers to learn and use new skills, school administrators and counselors to accommodate a class with fewer than usual number of students, and counselors and teachers to use cumbersome recruitment procedures to identify high-risk students for the RY class. We also found that many schools were operating in a difficult and turbulent context at the time of our study. All study schools were dealing with budget cuts, heightened and exclusive focus on academic achievement and graduation rates due to the No Child Left Behind policy [28] and statewide academic testing requirements.
Ultimately, all schools but one dropped the RY program, and that school adapted it extensively. Many schools dropped the program even before the lack of positive findings was known. Our results suggest that factors other than student level outcomes should be included during all stages of the research and effectiveness assessment process. We believe that more attention should be paid to developing and testing school-based prevention programs in real-world settings. In order to develop school-based prevention programs that are feasible to implement in real-world settings, we suggest that in addition to developing programs guided by theory and evaluating student-level outcomes, program developers and researchers should examine and document organizational factors and program characteristics that facilitate and impede program implementation [30, 31].
The RY program was developed about a decade ago and is listed on the Substance Abuse and Mental Health Services Administration [32] list of effective, evidence-based, indicated programs. In the development of evidence-based prevention programs, there appears to be a considerable lag between the time when the efficacy and effectiveness trials are conducted. As a result, organizational factors can change and make a feasible and efficacious program less so over time. As an example, No Child Left Behind legislation [28] seems to have greatly influenced the school's capacity and context since the early 1990s when the RY efficacy trial was conducted.
The current scientific model for the development of school-based prevention programs is primarily researcher driven. We believe that school personnel representing a broad range of school settings need to be more involved in the initial development of prevention programs. Furthermore, prevention research at the efficacy, effectiveness and dissemination stages needs to take into account organizational factors such as school and district resources required, leadership and administrative support and new training and skills required to implement and sustain new programs.
Even though we conducted the effectiveness trial in 10 schools, our unit of sampling for the organization diffusion study was at the district level. Findings can likely only be generalized to large schools located in school districts in the urban and metropolitan areas of the United States. Other limiting factors were the small number of respondents to our surveys and interviews, and the qualitative nature of the interview data. These factors affected our ability to conduct rigorous statistical analysis of the survey and interview responses. However, these are frequent limitations of organizational level studies where the organization is the unit of sampling and analysis, the number of respondents and organizations included in the study is often small, and the data are often qualitative.
Despite these limitations, we believe that the results presented in this paper offer valuable evidence on how program characteristics and organizational factors influenced the implementation of the RY program in 10 schools in two school districts. The results of this study contribute to our understanding of how organizational and program factors affect the adoption and implementation of school-based prevention programs.
|
Funding |
|---|
|
TopAbstractMethodsResultsDiscussionFundingConflict of interestAcknowledgementsReferences |
|---|
National Institute of Drug Abuse (R01 DA 13666 to D.D.H., Principal Investigator).
|
Conflict of interest |
|---|
|
TopAbstractMethodsResultsDiscussionFundingConflict of interestAcknowledgementsReferences |
|---|
None declared.
|
Footnotes |
|---|
S.T. and V.S. were with the Pacific Institute for Research and Evaluation at the time of conducting this study. 
|
Acknowledgements |
|---|
|
TopAbstractMethodsResultsDiscussionFundingConflict of interestAcknowledgementsReferences |
|---|
We thank the district and school administrators for granting us entrée and for their ongoing support during program implementation. We also thank the many interview and survey respondents for their time and candor.
|
References |
|---|
|
TopAbstractMethodsResultsDiscussionFundingConflict of interestAcknowledgementsReferences |
|---|
1. Kellam SG, Langevin DJ. A framework for understanding ‘evidence’ in prevention research and programs. Prev Sci (2003) 4:137–53.[CrossRef][Medline]
2. Glasgow RE, Klesges LM, Dzewaltowski DA, et al. The future of health behavior change research: what is needed to improve translation of research into health promotion practice? Ann Behav Med (2004) 27:3–12.[CrossRef][Web of Science][Medline]
3. Dusenbury L, Hansen WB. Pursuing the course from research and practice. Prev Sci (2004) 55–59.
4. Hansen WB. Barriers and Incentives to Effective Prevention Program Adoption and Implementation (2004) Available at: http://www.drugabuse.gov/about/organization/despr/hsr/da-pre/HansenBarriers.html. Accessed: 14 October 2005.
5. Rohrbach LA, D'Onofrio CN, Backer TE, et al. Diffusion of school-based substance abuse prevention programs. Am Behav Sci (1996) 39:919–34.[Abstract]
6. Hallfors D, Godette D. Will the ‘principles of effectiveness’ improve prevention practice? Early findings from a diffusion study. Health Educ Res (2002) 17:461–70.
7. Elliott DS, Mihalic S. Issues in disseminating and replicating effective prevention programs. Prev Sci (2004) 5:47–53.[CrossRef][Web of Science][Medline]
8. Pankratz M, Hallfors D, Cho H. Measuring perceptions of innovation adoption: the diffusion of a federal drug prevention policy. Health Educ Res (2002) 17:315–26.
9. Backer TE. The failure of success: challenges of disseminating effective substance abuse prevention programs. J Community Psychol (2000) 28:353–62.[CrossRef][Web of Science]
10. McCormick LK, Steckler A, McLeroy KR. Diffusion of innovation in schools: a study of adoption and implementation of school-based tobacco prevention curricula. Am J Health Promot (1995) 9:210–19.[Web of Science][Medline]
11. Ennett ST, Ringwalt CL, Thorne J, et al. A comparison of current practice in school-based substance use prevention programs with meta-analysis findings. Prev Sci (2003) 4:1–14.[CrossRef][Medline]
12. Hallfors D, Van Dorn RA. Strengthening the role of two key institutions in the prevention of adolescent substance abuse. J Adolesc Health (2002) 30:17–28.[CrossRef][Web of Science][Medline]
13. Eggert LL, Thompson EA, Herting JR, et al. Prevention research program: reconnecting at-risk youth. Issues Ment Health Nurs (1994) 15:107–35.[Medline]
14. Eggert LL, Thompson EA, Herting JR, et al. Preventing adolescent drug abuse and high school dropout through an intensive school-based social network development program. Am J Health Promot (1994) 8:202–15.[Medline]
15. Eggert LL, Thompson EA, Herting JR, et al. Reducing suicide potential among high-risk youth: tests of a school-based prevention program. Suicide Life Threat Behav (1995) 25:276–96.[Web of Science][Medline]
16. Sanchez V, Steckler A, Nitirat P, et al. Fidelity of implementation in a treatment effectiveness trial of Reconnecting Youth. Health Educ Res (2007) 22:95–107.
17. Hallfors D, Cho H, Sanchez V, et al. Comparison of efficacy and effectiveness trial results in an indicated "model" program: implications for public health. Am J Public Health (2006) 96:2254–9.
18. Rogers EM. Diffusion of Innovations (1995) 4th ed. New York: The Free Press.
19. Steckler A, Goodman RM, Kegler MC. Mobilizing organizations for health enhancement: theories of organizational change. In: Health Behavior and Health Education: Theory, Research, and Practice—Glanz K, Rimer BK, Lewis FM, eds. (2002) 3rd ed. San Francisco: Jossey-Bass. 335–60.
20. Goodman RM, McLeroy K, Steckler A, et al. Development of level of institutionalization (LoIn) scales for health promotion programs. Health Educ Q (1993) 20:161–78.[Web of Science][Medline]
21. Steckler A, Goodman RM, McLeroy KR, et al. Measuring the diffusion of innovative health promotion programs. Am J Health Promot (1992) 6:214–25.[Medline]
22. Statistical Package for the Social Sciences. (2005) Accessed: 29 March 2005 Available at: http://www.SPSS.com.
23. QSR International. N6: Product Information. 2005. Accessed: 29 March 2005 Available at http://www.qsrinternational.com/products/productoverview/N6.htm.
24. Morgan DL. Qualitative content analysis: a guide to paths not taken. Qual Health Res (1993) 3:112–21.
25. Patton MQ. Qualitative Evaluation and Research Methods (2002) Thousand Oaks, CA: Sage Publications, Inc.
26. Smith DW, Steckler A, McCormick LK, et al. Disseminating comprehensive school health curricula: lessons learned from the North Carolina school health and tobacco education project. J Health Educ (1995) 26:26–36.
27. Goodman RM, Steckler A. A model for the institutionalization of health promotion programs. J Fam Community Health (1989) 11:63–78.
28. No Child Left Behind Act of 2001. Enacted 3 January 2001. In: Approved 8 January 2002, Public Law 107–110. Washington D.C: 107th United States Congress.
29. Miles MB, Huberman AM. Innovation Up Close: How School Improvement Works (1984) New York: Plenum.
30. Flay BR. Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Prev Med (1986) 15:451–74.[CrossRef][Web of Science][Medline]
31. Glasgow RE, Lichetenstein E, Marcus A. Why don't we see more translation of health promotion research into practice? Rethinking the efficacy to effectiveness transition. Am J Public Health (2003) 9:1261–7.
32. SAMSHA. SAMHSA Model Programs. (2004) Accessed: 21 December 2004 Available at: http://modelprograms.samsha.gov/template-cf.cfm?page=model&pkProgramID=13 and http://modelprograms.samhsa.gov/pdfs/FactSheets/Reconnecting.pdf.
Received on August 16, 2006; accepted on April 25, 2007
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||