Health Education Research, Vol. 15, No. 2, 181-190,
April 2000
© 2000 Oxford University Press
Development of an instrument for monitoring adolescent health issues
Centre for Health Promotion and Cancer Prevention Research, University of Queensland Medical School, Herston Road, Herston, Queensland 4006, Australia
 |
Abstract |
|---|
 Top Abstract IntroductionAims and research objectivesMethodRecommendationsReferences |
|---|
The aim of the project was to develop a survey instrument to monitor relevant health status and health-related behaviors among secondary school students. The development of the instrument occurred in three main phases: collection of existing surveys, workshops with relevant health professionals and focus groups with adolescents. The topics for inclusion were refined using Health Goals and Targets for Australian Children and Youth and consultations with health professionals, and included alcohol/illicit drug use, smoking, nutrition, exercise, injury, mental health, violence and sexual health. Content validity was demonstrated through a comprehensive literature review, review and application of existing instrumentation, dialog and exchange with health professionals, and focus groups with adolescents. The process of peer review through correspondence with health professionals, and the coordination of workshops and focus groups established face validity. Responses from students also indicated that they interpreted the questions as intended. The instrument was piloted in five secondary schools during class periods. Process evaluation was also conducted to determine the appropriateness of the survey and the procedures used in administering the survey. Feedback from school staff was supportive and favorable with respect to the choice of issues. Reliability was assessed by a test–re-test procedure 2 weeks apart. In general, most of the questions showed moderate to high reliability (
> 0.5) indicating agreement of 50% or greater. This instrument was developed as a monitoring instrument and places emphasis on determining prevalence levels of a range of health issues and health behaviors to assist with identifying clustering patterns of negative health outcomes. Although the instrument is primarily for use with students in school hours, the nature of the instrument allows modification for use in older groups of adolescents and out-of-school youth. The final version of the questionnaires for senior and junior students can be accessed via the Internet (http://www.spmed.uq.edu.au/CHPCPR/questionnaire). |
Introduction |
|---|
|
TopAbstractIntroductionAims and research objectivesMethodRecommendationsReferences |
|---|
The adolescent years are a time of rapid growth and development in which there is a great deal of change taking place—physically, socially and emotionally (Mathers, 1995
). The incidence and prevalence of medical and psychosocial disorders is lower for adolescents than adults (Mathers, 1995), and they are in general the healthiest subgroup of the Australian population. Despite this, adolescents' general health status has not improved over the last 30 years (Reed et al., 1996). The major causes of death, disease and disability for young people in Australia have been identified as: (1) motor vehicle accidents, (2) suicide, (3) unintentional injury, (4) alcohol and drugs, (5) homicide, (7) asthma, and (7) sun exposure (Commonwealth Department of Health and Family Services, 1997). Chronic diseases and illness are commonly caused by exposure to risk factors many years prior to the onset of the condition. Dietary intake, consumption of alcohol and tobacco smoking are behavior patterns that are established during adolescence which are frequently linked to a number of illnesses that develop later in life (Hennekens and Buring, 1987).
A coordinated approach
In response to mounting concern by health professionals, organizations and individuals involved in youth health, and the community, the need for a formulated set of policies for improving adolescent health was established. In Australia, a national strategy to address this has been developed and the following section outlines the priorities, goals and documentation that resulted from this process.
Health Goals and Targets for Australian Children and Youth
The first stage of this process was the development of National health goals for young people. A number of key areas were identified and then used to formulate and develop Health Goals and Targets for Australian Children and Youth (Department of Health, Housing and Community Services, 1992). The goals set for children and youth are as follows: (1) reduce the frequency of preventable premature mortality, (2) reduce the impact of disability with attention given to both reducing the occurrence of new disability and reducing the impact on life of an established disability, (3) reduce the incidence of vaccine-preventable disease, (4) reduce the impact on conditions occurring in adulthood which have their origins or early manifestations in childhood or adolescence, and (5) enhance family and social functioning.
Better Health Outcomes for Australians
Better Health Outcomes for Australians (Commonwealth Department of Human Services and Health, 1994) is a more recent document addressing the issue of youth health with a particular focus on cardiovascular disease, cancer, mental health and injury. Within this document goals and targets have also been developed specifically to address ill health and premature death for young people with an emphasis on mental health issues. The prevalence rate in Australian young people for mental health conditions is estimated to be 10–15% and possibly as high as 18% in some inner city areas. The long-term consequences of mental health disorders and problems include educational failure, leaving school early and minimal vocational training. This document also focuses on prioritizing young people within the goals and targets for injury prevention, and control specifically for motor vehicle accidents, suicide and violence.
The Health of Young Australians
The reports cited above provided a framework and made strong recommendations to further develop a national approach to improve the health of Australian youth. As a result of the recommendations in these reports, a national health policy for children and young people was produced by the Commonwealth Department of Human Services and Health. The document is a joint statement by the Health Ministers of the Commonwealth, States and Territories in Australia to set a clear path for the future health and health-related services for young people. A number of issues affecting youth health were identified within the report and cover areas such as: (1) smoking and binge drinking, (2) food and nutrition, (3) physical activity, (4) injury, (5) sun exposure, and (6) pregnancy in young women.
Recommended action area: the development of instrumentation
Assessment of The Health of Young Australians identified specific action areas for improving adolescent health. One such action area was the need for research, information and monitoring. Emphasis is placed on the collection of data to examine changes in health status over time for young children and adults. A coordinated approach to information collection is critical to this process, as is regular public reporting and dissemination of information. It was also stressed that the measures developed should be within the agreed frameworks identified in preceding documents such as The Health Goals and Targets for Australian Children and Youth (1992) and Better Health Outcomes for Australians (1994).
The data would be used to identify priorities in health care delivery for young people, in the evaluation of programs, to refocus research, and educate and inform health providers and health system consumers. In addition, little is known about the specific parameters of youth health, and for these reasons the development of monitoring instrumentation has been identified as an important step in assessing the size of the problem and addressing the relevant issues (National Health and Medical Research Council, 1993; Department of Health, Housing and Community Services, 1994; Department of Human Services and Health, 1994; Commonwealth Department of Human Services and Health, 1995).
In the past a number of instruments have been developed to assess the health of young people. However, these methods were not created in the context of national policy strategies and tend to focus on a narrow range of issues. Nationwide health surveys exist in countries such as the US and the UK (Boland and Adam, 1989; Health Education Authority, 1992; Johnston et al., 1993), and the development of an effective monitoring instrument for an Australian audience which encompasses a broad range of adolescent risk-taking behavior will enable us to examine the co-morbidity of risk taking behaviors.
Settings for monitoring adolescent health issues: the school
The school could be viewed as one of the most important settings in which social and psychological development occurs, and it is likely that a child's relationship to school is associated with their health and health behaviors (Bond and Compas, 1989). Nutbeam et al. (Nutbeam et al., 1989) reported that health compromising behaviors appear to be strongly linked to aspects of school alienation, and the relationship between factors such as peer groups, school and home may have an effect on a young persons health-related development. The school environment and the way a school functions has a huge impact on the development of young people, and plays a critical role both as a base for monitoring current health behaviors and for implementing health promotion programs (Bond and Compas, 1989). Therefore, this article details the development of an instrument for monitoring adolescent health among secondary school students.
|
Aims and research objectives |
|---|
|
TopAbstractIntroductionAims and research objectivesMethodRecommendationsReferences |
|---|
The aim of the project was to develop a survey instrument to be used among secondary school students to monitor relevant health status and health-related behaviors, and the inter-relationships between them, with an emphasis on identifying clustering patterns of negative health outcomes. This instrument and the protocols developed with it are primarily for use in school hours with youth attending school; however, the nature of the instrument allows modification for use in older groups of adolescents and out-of-school youth (12–18 years of age).
The main research objectives were to develop a valid and reliable instrument that:
- Identifies and assesses health issues of relevance to young people.
- Is based on the broader national health goals as stated in The Health of Young Australians (1994) and Better Health Outcomes for Australians (1994).
- Allows for the measurement of key indicators as described in these documents.
- Places emphasis on health status measures and health risk behaviors.
- Is comparable to existing instruments that deal with specific issues in adolescent health.
- Utilizes input from health professionals and institutions with an interest in adolescent health.
- Is cognizant of the levels of development, behaviorally, physically and socially, within the adolescent age groups.
- Is suitable for use in secondary schools.
- Is supported by school staff.
|
Method |
|---|
|
TopAbstractIntroductionAims and research objectivesMethodRecommendationsReferences |
|---|
The development of the instrument occurred in seven phases briefly described below. The areas prioritized for inclusion in the development of the instrument were based on those proposed in The Health of Young Australians (1995), i.e. smoking and binge drinking, food and nutrition, physical activity, injury, sun exposure, and pregnancy in young women. In accordance with National and State health policy definitions, young people are those aged 10–24 (McLean, 1995
).
Phase 1: compendium of existing surveys
A literature search was conducted to identify existing surveys related to adolescent health with a specific focus on sexual health, mental health, nutrition, exercise, injury, tobacco, alcohol and drug use, and sun exposure. Australian and overseas experts in the appropriate fields were contacted and asked to provide any relevant surveys or items. The resulting collection of surveys came primarily from Australia, the US and England. A library of surveys collected was compiled and is held by the Centre for Health Promotion and Cancer Prevention Research. From these survey instruments the research team compiled an extensive list of possible items and scales. At this stage, emphasis was placed on covering a broad range of issues, scales and questions that would be modified after an extensive process of peer review. Questions were presented for each focus area, and then circulated to experts in the appropriate field for their comments, additions and deletions.
Phase 2: workshops with health professionals and review by investigative team
A series of six workshops (one for each topic area) was held with relevant local health professionals in order to identify and finalize criteria, prioritize issues, and define suitable items/scales for inclusion in the instrumentation. Emphasis for selection of issues and measures to be included in the questionnaire was placed on current adolescent health status and the prevalence of a range of adolescent risk behaviors, rather than knowledge or attitudes. The rationale for this being: (1) surveys used in the past to measure aspects of adolescent health focused on health risk behaviors, (2) the time constraints of administering the questionnaire (i.e. during class periods) limited the depth of questions that could be asked without compromising the objectives of the instrument, and (3) the framework provided by Pathways to Better Health (Department of Health, Housing and Community Services, 1993) predominantly focuses on health status and health risk behaviors.
As a result of the workshops, criteria for inclusion of various issues and measures were also developed. Health status measures and health risk behaviors were included only if the prevalence level of the outcome was high enough to be considered a prominent risk to many adolescents and the issue was identified as a high health priority in documents such as The Health of Young Australians (1995). Questionnaire items/scales were considered for inclusion if they met the specified criteria: had Australian norms, were suitable for use with adolescents, were used in previous studies, were suitable for a population survey rather than a clinical setting and, where used, had subscales that could be kept intact.
The research team, as a result of the workshops and consultations, were set the task of assimilating the feedback and information gained from this process and developing a draft version of the questionnaire. On the basis of protocols and formats from previous surveys, two versions were developed: one for Year 8 and Year 9 students (excluding sexual health questions), and the other for Year 10, 11 and 12 students. The investigating team forwarded the revised draft version of the questionnaire to those who participated in the relevant workshops. Requests were once again made for comments, additions and deletions, and the investigative team then prepared an advanced draft of the questionnaire.
Phase 3: focus groups with students
Focus groups were conducted with students prior to the finalization of the questionnaire to be used in the pilot survey. These focus groups were conducted in one Year 8 class and one Year 10 class in each of four Brisbane schools. Discussion focused on identifying items that may be difficult to understand, what was liked or not liked about the questions, as well as specific questions about the exercise, bullying and sexual health sections. An estimate of the average time taken to complete the survey and other administrative issues that will assist with the development of a suitable protocol were also explored. A protocol for conducting the focus groups was developed and focus group moderators received training for facilitation.
Principals of the schools participating in the focus groups were provided with a copy of the questionnaire and a supporting letter prior to student focus groups. Comments on the content were invited. All school principals viewed the questionnaire content favorably and had no objections to students participating in the survey.
Phase 4: development of a draft version of the instrument
The topic areas and issues decided upon for inclusion in the survey are outlined in Table I. A summary of the past instrumentation used to develop this questionnaire is listed in Table II.
|
|
Phase 5: pilot testing
In order to ensure generalizability of the instrumentation, participating schools represented various strata of socioeconomic status (SES). One to three whole classes were randomly selected from Years 9 and 11. To ensure classes did not consist of students of similar academic level, students in Health and Physical Education classes were chosen.
A fundamental aspect of the development process is the provision of accompanying protocols. A reliable instrument is one that is consistent over time, and therefore protocols and instrument administration should be standardized. The following outline is an appropriate protocol for this particular setting (i.e. school, during school hours). The protocols were developed and presented in such as way as to allow for modification in a variety of youth settings.
Procedural standardization
As for the focus groups, in an effort to maintain consistency, procedures for administration of the survey were set and rehearsed prior to implementation. Staff members underwent training on various aspects of protocol such as self-introduction, answering student queries, and survey collection and coding. All procedural steps were documented and provided to each staff member participating in the dissemination of the pilot.
Administering the questionnaire
Trained personnel administered the questionnaire to students at the various schools during usual class time. This was conducted across the nominated schools over a 3-week period. One research staff member was responsible for each class. Where possible the regular class teacher was absent to avoid any kind of overt or covert influence over students. In one school this was not possible, as the presence of the teacher was required to keep class behavior at an acceptable standard. The survey was anonymous (i.e. participants were not required to put their names on the surveys), and students were provided with a full explanation of the purpose of the survey and given an option for withdrawal of participation.
Students were allocated 35 min to complete the survey. This time allocation was guided by the time taken for students to complete sections of the survey in the focus groups. This time period was suitable for students with moderate to high levels of literary skills, but those with lower levels required more time to complete the questionnaire.
The reliability of the instrument was determined by a test–re-test method to determine the level of agreement between responses after a 2-week period. A time interval of 2 weeks was chosen based on the characteristics of the constructs being measured, and the availability of students, school staff and research staff at the second test period. A school from a mid-range SES area was chosen to participate in a repeat administration of the survey. Kappa () was the statistic chosen to measure the level of agreement between responses from Test 1 and Test 2 (re-test) for categorical (dichotomous) variables. Agreement of ordered variables was assessed by the Spearman rank correlation statistic. Responses from scaled questions were summed to produce a score and the test–re-test analysis involved the Pearson's product moment correlation statistic. The junior and senior surveys were analyzed separately in order to examine possible differential reliability. A selected number of variables representing all topic areas of the questionnaire were chosen for analysis. The criteria for choosing variables were as follows: (1) questions focussed on the main behavioral measures of health risk behaviors rather than outcomes or descriptions of the context, (2) questions pertaining to 100% of the sample or to those in a sufficient number of responses (greater than 5%) in each response category and (3) questions referring to behavior for a period greater than the test–re-test period (i.e. `in the last 12 months...').
Phase 6: evaluation of the instrument
Instrument validity
Validity is the extent to which an instrument appears to be measuring what it is supposed to measure (Windsor and Baranowski, 1984). The types of validity addressed in the development of this instrument were as follows.
Face validity was established by ensuring the instrument is interpreted by the respondent in the way intended. Face validity deals with the wording, expression, appearance, etc., and is subjectively assessed. The processes of peer review through correspondence with health professionals, and the coordination of workshops and focus groups were used to establish face validity (Green and Lewis, 1986).
Content validity is the extent to which an instrument covers the content area is it intended to cover. (Green and Lewis, 1986). This was demonstrated through the following:
- A comprehensive literature review on issues relevant to adolescent health.
- Aspects of instruments utilized by previous researchers were applied to the current instrument.
- Empirical research and observation.
- Use of practitioner papers on adolescent health.
- Dialog and exchange with health professionals.
- Identification of items/issues of importance for inclusion which were derived from documents such as The Health of Young Australians and Better Health Outcomes.
- Ratings by experts on the relevance of each item/issue (through correspondence and workshop participation).
- Focus groups with adolescents to ensure areas relevant to adolescent health were covered.
Instrument reliability
The instrument reliability was established through the test–re-test procedure to assess reproducibility over time. As a result of pilot testing of the instrument, a protocol has been developed suitable for use in Queensland Secondary Schools, during class time. The protocols are suitable for modification to other youth settings; however, the reliability of the instrument would be subject to the degree and type of modifications.
In total, 308 junior and 223 senior questionnaires were returned complete. This number includes the questionnaires provided to students in the re-test (i.e. those in the re-test submitted questionnaires twice). Table III indicates the number of questionnaires obtained per school, stratified by SES.
|
Approximately 10% of the students participated in the re-test procedure. In general, most of the questions showed moderate to high test–re-test reliability (
> 0.5) indicating agreement of 50% or greater. Questions showing low test–re-test reliability were those related to reporting injury occurrence in the last 12 months (for juniors), the rating of life satisfaction and the measure of hopelessness scale [from the Beck Hopelessness Scale (Beck and Steer, 1988)]. Table IV summarizes the results of tests for agreement for selected content areas of health behaviors.
|
Phase 7: revision and final instrument
As a result of the piloting exercise a small number of changes were made to the questionnaire. These changes included the following: (1) addition of extra response categories where `other' response was frequently referred to, (2) addition of `gated' questions where necessary to reduce response confusion to the proceeding questions, (3) modification to scale sequencing in drug use frequency questions and (4) addition of a qualifier for injury severity.
The final instrument is one which is suitable for use in secondary schools for monitoring adolescent health issues, and consists of the questionnaires for both senior and junior students. The final version of the questionnaires can be accessed via the Internet (http://www.spmed.uq.edu.au/CHPCPR/questionnaire). The questionnaire for junior students did not include questions related to sexual health, contraception and pregnancy, and if this information is desired from junior students, the questionnaire for senior students can be used.
|
Recommendations |
|---|
|
TopAbstractIntroductionAims and research objectivesMethodRecommendationsReferences |
|---|
Recommendations for researchers
- Further analysis and testing of the hopelessness scale for the school and out-of-school setting.
- Further formative evaluation on the meaning of the word `injury'.
- Further investigation into questionnaire and protocol modification is recommended to facilitate the expansion of this instrument to various subgroups of young people, including out-of-school youth. This would be achieved through exploration of the suitability of the content of the questionnaire for out-of-school youth; and revision of protocols to complement issues pertinent to an out-of-school setting.
Recommendations for users
- The suitability of using non-intact classes should be examined further. Some large-scale school-based surveys do not utilize intact classes as adopted in this study. Preliminary feedback from schools indicated that use of intact classes was more practical and preferred by the teachers.
- Regular administration of the survey to maximize the monitoring potential of the instrument. Ideally this would be on an annual basis, or biennially, beginning at the junior grades and following through to senior levels.
- Ongoing consultation with schools to determine the best time of year to conduct surveys to ensure maximum student participation and staff cooperation.
- Input from the Department of Education in the survey implementation.
|
Acknowledgments |
|---|
We gratefully acknowledge the assistance provided by the following people during the development of this instrument: Dr John Fisher, Dr Amaya Gillespie, Dr Phil Harvey, Di Topping, Dr Cris Cantor, Michelle Smith, Dr Fiona Stanley, Professor Robert Kosky, Rene Du Plessis, Sandra Capra, Dr Francis Skelton, Alan Haugnt, Dr Anne Roche, A/Professor Pierre Baum, Bob Boyd, Dr James Smeathers, Dr Johanna Wynn, Frances Paterson, Dr Rod Ballard, Dr Michael Dunne, Dr Ross Young, May Hyndman, Professor Tian Oei, Dr Eric Dommers, Dr Stephen Zubrick, Meg Driver, Dr Jan Nicholson, Professor Mary Sheehan, Margo Eyeson-Annan, Janet Bishop, Janine Sheffield, A/Professor Matt Sanders, Judith Piccone, Carla Patterson, Wendy Patton, Mathew Evans, A/Professor Robert Bush, Professor Beverley Raphael and Maria Donald.
|
References |
|---|
|
TopAbstractIntroductionAims and research objectivesMethodRecommendationsReferences |
|---|
Beck, A. T. and Steer, R. A. (1988) Manual for the Beck Hopelessness Scales. Psychological Corp., San Antonio, TX.
Bennett. D. (1984) Adolescent Health in Australia: An Overview of Needs and Approaches to Health Care. AMA. Sydney.
Boland, C. and Adam, J. (1989) Young People and Health: An Overview of Current Research. A report to the National Youth Affairs Research Scheme Hobart, National Clearinghouse for Youth Studies. AGPS, Canberra.
Bond, L. A. and Compas, B. E. (1989) Primary Prevention and Promotion in the Schools. Sage, London.
Commonwealth Department of Health and Family Services (1997) Youth Suicide in Australia: A Background Monograph, 2nd edn. AGPS, Canberra.
Commonwealth Department of Human Services and Health (1994) Better Health Outcomes for Australians. National Goals, Targets and Strategies for Better Health Outcomes into the Next Century. AGPS, Canberra.
Commonwealth Department of Human Services and Health (1995) The Health of Young Australians. A National Health Policy for Children and Young People. AGPS, Canberra.
Department of Human Services and Health (1994) Our Children Our Future. AGPS, Canberra.
Department of Health, Housing and Community Services (1992) Health Goals and Targets for Australian Children and Youth. AGPS, Canberra.
Department of Health, Housing and Community Services, National Health Strategy (1993) Pathways to Better Health. Issues Paper 7. AGPS, Melbourne.
Green, L. and Lewis, F. (1986) Measurement and Evaluation in Health Education and Health Promotion. Mayfield, Palo Alto, CA.
Hennekens, C. H. and Buring, J. E. (1987) Epidemiology in Medicine. Little Brown & Co., Toronto.
Johnston, L. D., O'Malley, P. M. and Backman, J. G. (1993) National Survey Results on Drug use from Monitoring the Future Study. Publ. no. NIH 93-3597, US DHHS, Bethesda, MD.
Mathers, C. (1995) Health Differentials among Australian Children. Health Monitoring Series 3. Australian Institute of Health and Welfare, AGPS, Canberra.
McLean, K. (1995) Delivering Health Services to Young People. Brisbane North Regional Health Authority, Queensland Health, Brisbane.
National Health and Medical Research Council (1993) Review of Child Health Surveillance and Screening. AGPS, Canberra.
Nutbeam, D., Aaro, L. E. and Catford, J. (1989) Understanding children's health behavior: the implications for health promotion for young people. Social Science and Medicine, 29, 317–325.
Windsor, R. A. and Baranowski, T. (1984) Evaluation of Health Promotion and Education Programs. Mayfield, Palo Alto, CA.
Received on December 12, 1998; accepted on June 23, 1999
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. Saxena, A. Lora, M. van Ommeren, T. Barrett, J. Morris, and B. Saraceno WHO's Assessment Instrument for Mental Health Systems: Collecting Essential Information for Policy and Service Delivery Psychiatr Serv, June 1, 2007; 58(6): 816 - 821. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||