Health Education Research Advance Access originally published online on May 19, 2006
Health Education Research 2007 22(1):1-13; doi:10.1093/her/cyl010
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
© 2006 The Author(s).
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/2.0/uk/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Feasibility, acceptability, and quality of Internet-administered adolescent health promotion in a preventive-care setting
1 Department of Public Health, Erasmus MC–University Medical Center, Rotterdam, PO Box 2040, 3000 CA Rotterdam, The Netherlands
2 Department of Medical Informatics, Erasmus MC–University Medical Center, Rotterdam, PO Box 1738, 3000 DR Rotterdam, The Netherlands
* Correspondence to: H. Raat. E-mail: h.raat{at}erasmusmc.nl
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
An Internet tool for monitoring, personalized feedback and referral was developed to support routine adolescent preventive care and was compared with usual practice using paper and pencil (P&P). A total of 1071 students (average age 15 years) from seven secondary schools were randomly assigned to the Internet or P&P group. The Internet group received a health and health-behavior assessment, tailored feedback on health and health behavior (specifically fruit consumption), and an online referral to see a physician/nurse if necessary. The P&P group received the same assessment, preprinted generic advice on fruit consumption and a mailed referral (where applicable). Students and physicians/nurses completed evaluation forms to assess indicators of feasibility, acceptability (i.e. satisfaction) and quality of each administration mode. Student participation rate was 87%. The electronic health feedback was positively evaluated. Students perceived the Internet-tailored fruit advice as more pleasant, more personally targeted and more enjoyable, but less credible than the generic preprinted advice (P < 0.01). No differences in indicators of acceptability and quality of consultation were found (P
0.05). Thus, the Internet can be a valuable tool to support physicians/nurses in the field of preventive care. It is recommended to further optimize and evaluate the Internet as a tool. |
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
Community and preventive health services can contribute to preventing diseases among adolescents, especially by promoting good health and a healthy lifestyle [1]. The approach in doing so varies by country.
The current Dutch preventive health care system monitors adolescents' self-reported health and health behavior via printed questionnaires administered at schools. Based on the self-reported health (from physical to psychosocial health and sometimes also health behaviors), some municipal health services refer adolescents with a risky health profile. During such a consultation, adolescents may receive preprinted generic information on health and health-behavior topics. Monitoring data are not only used for individual care but also for generating health profiles at the group level. Based on these health profiles, schools may decide to adopt specific preventive programs/interventions.
With the current strain on preventive health care and increasing demand for adolescent health promotion, preventive health care requires greater efficiency. The Internet can be highly beneficial for achieving this; it is very efficient for data sampling, may increase data quality and may save on costs in the long term, and it provides the opportunity to enhance the quality of adolescent health promotion (e.g. by giving automated feedback on the adolescents' health to the physician/nurse to support the consultation) [2–9]. The Internet offers the opportunity to give immediate computerized, tailored feedback on health and health behavior, which various studies have shown to be more effective than generic (paper) health advice [10, 11].
This project aimed to develop an Internet tool to support the current adolescent preventive health care provided by Dutch municipal health services. The tool measured health and health behavior, provided tailored-health feedback to the students, immediately referred adolescents at risk to the physician/nurse (i.e. school doctor/nurse) and gave information to the physicians/nurses regarding individual students' health and health-behavior status to facilitate communication during consultation [12]. The tool also provided an SPSS-data file, which the municipal health services could use for subsequent analyses.
The topics chosen for this project corresponded to the current practice of municipal health services, namely generic health and a few thematic topics on health behavior. As physicians/nurses currently focus on generic health, the Internet tool was complemented with a separate computer-tailored [10] tool for assessment and a tailored-advice component for one of the health behaviors (fruit intake in this project). Nutrition attracts the interest of adolescents [13, 14] and their parents [15]; yet adolescents do not reach recommended intake levels for fruit consumption [16].
The aim of this study was to evaluate indicators of feasibility (i.e. actual use), acceptability (i.e. adolescents' and physicians'/nurses' satisfaction) and quality of the Internet-administered adolescent preventive health care procedure by comparing it with the current procedure, which uses printed questionnaires and no tailored online feedback.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
Sample and setting
Two municipal health services participated in the study, each involving two physicians/nurses. The municipal health services were situated in a rural (Harderwijk) as well as an urban (Vlaardingen) area, in the Netherlands, and invited seven secondary schools of varying school types with respect to educational level and socio-economic background. These schools comprised 55 classes from the same grade with 1071 students (average age 15 years) and had adequate computer and Internet access facilities (Internet Explorer 5.0 or higher). The parents and students separately received written information about the project and were allowed several weeks to refuse participation (passive consent). The medical ethical committee of the Erasmus MC–University Medical Center, Rotterdam, approved the study.
Study design
The study design is schematically illustrated in Fig. 1. Students within each class were randomly assigned to the Internet group or the paper and pencil (P&P) group. The procedures are introduced below and will be described in detail in the paragraph on ‘Intervention’.
|
First, all students completed a questionnaire to assess their health and health-behavior status via either Internet or P&P mode in a classroom at school under supervision of a researcher and an assistant from the municipal health service. After completing the assessment, the students in the Internet group received computer-tailored feedback [10] on their answers via Internet immediately on screen, including personal advice about fruit consumption. Students in the P&P group only received generic preprinted advice about fruit consumption. Immediately after reading their advice, students in both groups completed a printed form for evaluation of the assessment and feedback. This part of the study is referred to as ‘school session’.
In both groups, students with increased health risk (see for criteria the paragraph on ‘Intervention’ below) were identified and referred by the municipal health service. ‘Consultation’ took place at either an office at the school (Harderwijk) or at the municipal health service (Vlaardingen). Prior to the consultation, the physician/nurse received information on the potential health risks or unfavorable health behaviors either via a short web-based or written report (dependent on mode of administration). After consultation, students and physicians/nurses from both groups completed a printed evaluation form.
Intervention
Intervention contents of the school session
Students completed a questionnaire (via Internet or paper) to assess health and health behavior on the following topics: ‘generic health’ was determined with the most widely used validated generic health measure called the Child Health Questionnaire-Child Form (CHQ-CF) including multiple scales [17, 18], a higher Child Health Questionnaire (CHQ) score indicated better health. The ‘self-reported complaints/chronic conditions’ addressed the presence of 10 complaints such as allergies and hearing impairment, in the last year. If the complaint was present, the student could indicate whether he/she had seen a physician for this in the last year. In addition, with asthma (being a highly prevalent chronic condition among youth), respiratory problems were more thoroughly assessed by applying the eight-item validated and widely used International Study of Asthma and Allergies in Childhood asthma questionnaire [19]; selected topics on ‘health behavior’ were ‘smoking’ and ‘fruit consumption’. Students indicated the smoking behavior that resembled them the most on an eight-level scale ranging from ‘I smoke at least once a day’ to ‘I have never smoked, not even one puff’ and rated the number of cigarettes smoked in total [20]. Another item assessed intention to quit smoking. Students intending to quit completed eight extra items on the determinants of quitting based on the ASE-model [21]. The items on frequency of fruit consumption were based on the validated measure from Bogers et al. [22] and several determinants were based on Oenema and Brug [23].
Further, items on age, gender and countries of birth of respondent and parents to determine ethnic background [24] assessed socio-demographic characteristics. In addition, students registered their ‘starting and finishing time’ of completing the P&P assessment; these were automatically registered in the Internet-delivered mode.
The feedback of the Internet group contained online personal advice on fruit intake, feedback on the reported health for each topic including smoking and additionally a referral, if relevant. The fruit advice was derived from the web-based, tailored food advice for adults from Oenema and Brug [23] with adjustments made for students. This advice is based on Weinstein's precaution adoption process model, which emphasizes the role of awareness as a prerequisite for contemplating behavioral change [25]. To influence the awareness of personal fruit intake, the fruit advice applied personalized and normative feedback as proposed by Weinstein's model. Students were encouraged to make a printout of their personal fruit advice. Furthermore, students were invited to click further to see their status for each topic that was assessed by the questionnaire (health risk appraisal) [26]. Students with a score indicating a risk (e.g. below a certain cutoff point) received feedback making them aware of the reported complaints (=risk-feedback). Table I shows the risk-feedback topics, criteria and cutoff points and the number of students who received risk-feedback. A score (e.g. above a certain cutoff point) indicating good health or health behavior would lead to positive feedback. A score in between the cutoff points for risk-feedback and positive feedback would generate feedback pointing to possible problems. CHQ scales selected for feedback were physical functioning, bodily pain, general behavior, mental health, self-esteem and general health. Cutoff points for each CHQ scale were based on existing reference datasets of the CHQ among adolescents (D. Verrips, personal communication) [18]. The lowest 2% of each CHQ scale in the reference data indicated a score for each CHQ scale and this score (see footnote of Table I for the exact score) was used as the cutoff point for risk-feedback. Health status based on CHQ scales was also summarized in a graph (see Fig. 2). Finally, the module for feedback on referrals included an appointment to see the physician/nurse for students at risk (for criteria see below), with the reason mentioned. If students were not referred based on their assessment, they could check a box for a self-referral. The details on time and place of consultation were sent by mail (several weeks later).
|
|
Students in the P&P group received generic preprinted fruit advice, which emphasized the importance of eating sufficient fruit every day. When consultation was considered appropriate, a referral was sent by mail (several weeks later). All students in the P&P group had been offered the opportunity to self-refer themselves already in the assessment on health/health behavior.
In both groups, the criteria for referral were being at risk for one or more of the selected CHQ scales (referred to as ‘health risk’) or having self-referred.
Intervention contents of consultation
As mentioned previously, the physician/nurse received the results of the assessment for each referred student prior to consultation via a different mode for each group. For referred students in the Internet group, the physician/nurse received via the Internet a printable summary of the self-reported health risks and problems. The physician/nurse also had access to the complete electronic feedback for students as well as to all individual items in a printed SPSS-output. For students in the P&P group, the physician/nurse used the students' completed printed health and health-behavior questionnaires and a printed SPSS-output summarizing the students' health risks and problems.
Consultation was the same for both groups, namely a medical examination, going into specific risk areas (e.g. health risks on CHQ scales were discussed), and referring students to other professionals when necessary.
Internet tool
The health and health-behavior questionnaire and feedback via Internet were developed using PHP (4.0.1 and higher), MySQL (3.22 and higher) and JavaScript (1.3). Access to the questionnaire was password protected, with the student's name not being recorded and only identifiable by the researcher and physician/nurse. Data were sent to the server in a scrambled format. The screen displaying the questionnaire used two separate frames, the left one displaying a list of topics and the right one displaying the questions per topic. Questions not relevant to the student were not displayed. Logging out after completing the questionnaire was only permitted after answering all items.
Each physician/nurse received a personal login code from the researcher to access the Internet tool.
Evaluation of intervention
Indicators of feasibility
The following aspects determined the feasibility (i.e. actual use) of the intervention.
Attendance/reach.
The percentage of students completing the assessment was compared between the Internet and P&P groups. The same was done for students who were referred. The physician/nurse noted the attendance (students complying with the referral) of the consultation, which was compared between the Internet and P&P groups.
Duration.
The completion times of the assessment and separately the consultation (noted by the physician/nurse) were compared between groups. In addition, a researcher registered after the school sessions whether the class finished the whole session (including reading of the feedback) within the class time.
Reading of the feedback.
Students noted which parts of the feedback they had read. Two–four months later, students were asked whether they had referred to the fruit advice after the session. The reading of the fruit advice was compared between groups.
Administration modes.
Students evaluated the ease and pleasantness of the administration modes for the assessment, fruit advice and electronic advice separately. The answers ranged from, e.g. ‘very difficult’ to ‘very easy’ on a five-point Likert scale. Comparisons between groups were made, except for the electronic advice.
Indicators of acceptability
Users (i.e. students and physicians/nurses) assessed various aspects of acceptability (i.e. satisfaction) of the intervention. Comparisons between groups were made between the students' evaluations of individualization (targeted specifically at me) and enjoyability of the fruit advice, formatted on a five-point Likert scale ranging from ‘strongly disagree’ to ‘strongly agree’. Additionally, whether the assessment and advice were found interesting was measured, with answers from ‘not interesting’ to ‘very interesting’ on a five-point Likert scale. In both modes, students rated the whole session (assessment and feedback) on a scale from 1 to 10. These were compared between groups.
Satisfaction with consultation was assessed among students by the 18-item Satisfaction Form [27], with answers on a five-point Likert scale. An extra item measured whether the students appreciated knowing why they were referred, with answers on a five-point Likert scale ranging from ‘very unpleasant’ to ‘very pleasant’.
The evaluation form completed by the health physician/nurse measured helpfulness and overall satisfaction with the session. Answers were on a five-point Likert scale, with 5 indicating the most-positive evaluation.
Indicators of quality of contents
The following selected indicators gave insight into whether the contents of the intervention were of sufficient quality: the number of students referred (including self-referrals) was compared between groups; students rated whether they understood the contents of the assessment, the fruit advice and the electronic health feedback. The answers ranged from ‘very difficult’ to ‘very easy’ on a five-point Likert scale. Comparison between groups was possible except for the evaluation of the health feedback; comparisons between groups were made between the students' evaluations of the personal relevance, usefulness and credibility of the fruit advice, formatted on a five-point Likert scale ranging from ‘strongly disagree’ to ‘strongly agree’; finally, two additional items addressed the physician's/nurse's evaluations of whether the referral was legitimate and whether the information on referred students was correct.
Analysis
Where possible, significant differences between the Internet and P&P groups were tested, using two-tailed tests with an alpha of 0.05. Differences in age, duration of assessment, the mark for the school session and the scale score of the Satisfaction Form for consultation were determined through the Student's t-test. Mann–Whitney U-tests were used to assess data that were not normally distributed. These included assessment of differences between the two groups in duration of consultation, students' evaluation of administration modes and all the remaining acceptability measures. In addition, Mann–Whitney U-tests were used for differences in the quality measures, except for legitimacy of referral, and correctness of information, which were tested with logistic regression (Wald test). Comparisons of attendance rates, reading of the fruit advice, were also tested with logistic regression; characteristics of participants, except for age, were tested with the Chi-square test.
Odds ratios (ORs) were calculated for the dichotomous variables. ORs estimated the probability of an outcome between the administration modes, with the P&P group as the reference category.
SPSS version 11.0.1 was used for all statistical analyses.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
Sample
Of the 1071 eligible students, 933 (87%) participated in the school session, 458 participated in the Internet group and 475 participated in the P&P group. Registered reasons for absence were mainly ‘unknown’ and ‘illness’, and 27 parents refused their child's participation. Table II lists the characteristics of the participating students by study groups.
|
Due to missing values, the number of student evaluation forms differed per analysis.
Indicators of feasibility
The percentage of students completing the health questionnaire (see Fig. 1) did not differ significantly between the Internet and P&P groups (P = 0.499, OR = 0.88, 95% CI 0.62–1.26). The attendance at the consultation was higher in the Internet group (64%) than in the P&P group (48%), but this difference was not significant (P = 0.095, OR = 1.92, 95% CI 0.89–4.14) (see Fig. 1). Registered reasons for non-attendance were mainly ‘unknown’ or that the student was ‘already under treatment’. In most sessions (90%), the students attended without a parent.
The completion times for the assessment did not differ significantly between the Internet (mean = 19.8 min, 5.2 SD, n = 426) and the P&P groups (mean = 19.6 min, 5.2 SD, n = 439) (P = 0.587). In both groups, the school sessions were completed within the class time. The duration of the consultation did not differ significantly between the Internet (median = 25 min, n = 33) and P&P groups (median = 30 min, n = 27) (P = 0.470).
Most students (69%) using the Internet reported having read one or more parts of the health feedback, with the CHQ being read most often (50%) (see Table III). In both groups, >85% reported having read the fruit advice during the school session. Two–four months later, at least 41% of the students reported they had read the advice since the school session. No significant differences were found between the Internet and P&P groups (P 0.05).
|
Students rated assessment via the Internet as an easier mode than via P&P (P = 0.035) (see Table IV). The electronic fruit advice was a more pleasant mode to use (P = 0.005) than the preprinted advice. Students evaluated the electronic health feedback as an easy and pleasant mode to use.
|
Indicators of acceptability
In general, students evaluated the assessment and the fruit advice in both groups neutral/positive, but the Internet-tailored feedback on fruit consumption was evaluated as being more personally targeted and enjoyable than the preprinted generic advice (P < 0.01) (see Table V). Students evaluated the electronic health feedback positively. Students were satisfied with the consultation and appreciated knowing why they were referred and reasons for it. The health physicians/nurses evaluated the information as neutral/helpful and were satisfied with the overall session. No differences between groups were found (P
0.05).
|
Indicators of quality of contents
In the Internet group, 50 students (10.9%) were at health risk and three (0.7%) had self-referred themselves; in the P&P group, this was 50 (10.5%) and six (1.3%) students, respectively. The percentage of students referred did not differ significantly between groups (P = 0.523, OR = 0.88, 95% CI 0.59–1.31) (see Fig. 1). To illustrate, the reference dataset from 1995 (444 adolescents aged 9–17 years) showed 7% to be at health risk [18], which was slightly lower than the proportion in the Internet group (P = 0.039) and the P&P group (P = 0.058).
Students evaluated the electronic fruit advice as more personally relevant, but less credible and useful than the preprinted fruit advice (all, P < 0.05) (see Table VI). After the consultation, the physicians/nurses evaluated most (complied) referrals based on the CHQ scales as legitimate referrals and they evaluated the information as correct in most cases. No differences between groups were found (P 0.05). Reasons for illegitimate referrals or incorrectness of the information were that students were already known/familiar with the municipal health service or that the complaints/problems had already been solved before the consultation (the CHQ scale bodily pain had detected temporary problems, which had been solved by the time the consultation took place) or the answers to the questionnaire did not match what was said during the consultation.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
The Internet-administered adolescent health promotion was successfully implemented in the preventive health-care setting by the municipal health services. It was feasible regarding the aspects attendance, duration, reading of the feedback and administration mode, and users were satisfied. Besides this, the quality of the contents was adequate, but the fruit advice may require some improvement.
Other studies among adolescents conducted in primary preventive-care settings in the United States [2, 4, 6] also showed that computerized adolescent health promotion was feasible and positively evaluated; however, these studies targeted health behavior while the present study covered multiple dimensions of health in combination with health behaviors and these studies did not apply a randomized control group to assess users' satisfaction with the intervention.
The consultation in the Internet group achieved a higher (non-significant) attendance rate compared with that in the P&P group. A review by Edwards et al. [28] suggests that individualized risk information increased participation in screening programs, but this was not statistically supported by the present study.
The consultation in the Internet group received positive evaluations from both students and physicians/nurses. Previous research indicated that physician's/nurse's behavior is an important determinant of adolescents' satisfaction with their health care [29] and that interactive health communication could improve patient satisfaction [7, 12]. Nevertheless in the current study, the levels of the students' ratings of acceptability of the consultation in the Internet group were not higher compared with the P&P group.
The students positively evaluated both Internet-administered adolescent health promotion and the current paper procedure with relatively few and small differences between modes. Given the multiple comparisons made, only the differences with a P-value <0.01 may be considered relevant; on three items the Internet version was somewhat more positively evaluated (regarding feasibility of administration mode and acceptability of fruit advice); however, concerning indicators of quality, the tailored fruit advice was regarded as less credible than the generic paper version. The latter difference was not seen for the almost identical fruit advice among adults [23]. Although the current study confirms that adolescents enjoy computerized evaluation, the literature also suggests that tailored advice is superior to generic instruction [10, 11], which the current study does not support convincingly. Additional analyses to assess whether the student evaluations by mode of administration interacted with gender, ethnic background (Dutch/non-Dutch) or school type (vocational/non-vocational) showed that only gender significantly interacted with mode of administration regarding ‘individualization’ (P = 0.019) and ‘enjoyability’ (P = 0.032) of the fruit advice; girls were more favorable toward Internet-tailored fruit advice compared with preprinted generic fruit advice than boys. This study did not evaluate the pathways by which tailored feedback may affect acceptability of the feedback, such as upward or downward social comparisons by students when confronted with their own health or behavior rating compared with norms. It is proposed to evaluate these pathways in more specific studies.
A few considerations should be made when interpreting the study results; the attendance was sufficient and relatively high at the school sessions; however, the proactive recruitment and restricted setting within school lectures of the present study may have enhanced the response rate. Next, this study was performed within two specific municipal health services, each having two different physicians/nurses for the consultation. Differences between physicians/nurses may be expected; however, the number of participating students is too small to account for this. The relatively small number of students referred limits the evaluation of the consultation. Furthermore, even though our study population did not differ substantially in characteristics (age, gender, ethnic background and educational levels) from the general Dutch adolescent population when compared with nationwide data concerning students in secondary schools [30], the study results are restricted to data collection conducted among adolescents in the Netherlands, at schools and within a preventive-care setting. There were slightly more students at health risk than in the reference data (which were used to develop the cutoff points for risk-feedback) [18]. Other age groups and settings may show different results. Finally, the current results are restricted to certain topics and to certain health questionnaires being used.
In conclusion, this study was very much interweaved with the existing practice of preventive health care. This interconnection probably not only resulted in a higher participation rate but also is promising for future implementation of the Internet tool. Using the Internet for the adolescent preventive health care procedure is feasible and positively evaluated by users. Moreover, the Internet has practical advantages in comparison with the standard approach. For example, the Internet is an efficient approach to sample data, it eliminates manual data entry by researchers thereby reducing transcription errors and workload, and its forced data entry results in complete data collection [5, 9]. Potentially, once taken out of the research mode and implemented in the standard practice, it may be less labor intensive to administer the questionnaires. In short, the Internet can benefit preventive health care. It is recommended to further optimize the Internet tool in relation to the feedback/information given to the students and the physicians/nurses, and conduct more evaluation of the use and users' satisfaction with other health and health-behavior topics. Finally, the present study reports feasibility, acceptability and quality while the health and health-behavioral effects of such integrated Internet-administered adolescent health promotion are highly important and should be investigated.
|
Acknowledgements |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
The authors thank the employees of the municipal health services in Vlaardingen and Harderwijk, the schools and students for their enthusiastic participation, Esther Rozendaal for expert data collection and other supportive tasks and Katrina Giskes for her useful comments on the manuscript. This research was funded by The Netherlands Organization for Health Research and Development (ZonMw) Program for Prevention # 2100.0066. Funding to pay for the Open Access publication charges for this article was provided by The Netherlands Organization for Health Research and Development (ZonMw) Program for Prevention #2100.0066.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
1. Viner RM and Barker M. Young people's health: the need for action. Br Med J 2005 330:901–3.
2. Paperny DM, Aono JY, Lehman RM, et al. Computer-assisted detection and intervention in adolescent high-risk health behaviors. J Pediatr 1990 116:456–62.[CrossRef][ISI][Medline]
3. Noell J and Glasgow RE. Interactive technology applications for behavioral counseling: issues and opportunities for health care settings. Am J Prev Med 1999 17:269–74.[CrossRef][ISI][Medline]
4. Paperny DM and Hedberg VA. Computer-assisted health counselor visits: a low-cost model for comprehensive adolescent preventive services. Arch Pediatr Adolesc Med 1999 153:63–7.
5. Fotheringham MJ, Owies D, Leslie E, et al. Interactive health communication in preventive medicine: internet-based strategies in teaching and research. Am J Prev Med 2000 19:113–20.[ISI][Medline]
6. Patrick K, Sallis JF, Prochaska JJ, et al. A multicomponent program for nutrition and physical activity change in primary care: PACE+ for adolescents. Arch Pediatr Adolesc Med 2001 155:940–6.
7. Sciamanna CN, Novak SP, Houston TK, et al. Visit satisfaction and tailored health behavior communications in primary care. Prev Med 2004 26:426–30.[CrossRef]
8. Mangunkusumo R, Moorman P, Berg van den-Ruiter de A, et al. Internet-administered adolescent questionnaires compared with a paper version in a randomized study. J Adolesc Health 2005 36:70.
9. Mangunkusumo RT, Duisterhout JS, De Graaff N, et al. Internet versus paper mode of health and health behavior questionnaires in elementary schools: asthma and fruit as examples. J Sch Health 2006 76:80–6.[CrossRef][ISI][Medline]
10. Brug J, Campbell M, van Assema P. The application and impact of computer-generated personalized nutrition education: a review of the literature. Patient Educ Couns 1999 36:145–56.[CrossRef][ISI][Medline]
11. Revere D and Dunbar PJ. Review of computer-generated outpatient health behavior interventions: clinical encounters "in absentia". J Am Med Inform Assoc 2001 8:62–79.
12. Detmar SB, Muller MJ, Schornagel JH, et al. Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial. J Am Med Assoc 2002 288:3027–34.
13. Ackard DM and Neumark-Sztainer D. Health care information sources for adolescents: age and gender differences on use, concerns, and needs. J Adolesc Health 2001 29:170–6.[CrossRef][ISI][Medline]
14. Klein JD and Wilson KM. Delivering quality care: adolescents' discussion of health risks with their providers. J Adolesc Health 2002 30:190–5.[CrossRef][ISI][Medline]
15. Cohall AT, Cohall R, Ellis JA, et al. More than heights and weights. J Adolesc Health 2004 34:258–61.[ISI][Medline]
16. Yngve A, Wolf A, Poortvliet E, et al. Fruit and vegetable intake in a sample of 11-year-old children in 9 European countries: The pro children cross-sectional survey. Ann Nutr Metab 2005 49:236–45.[CrossRef][ISI][Medline]
17. Landgraf JM, Abetz L, Ware JE. The CHQ User's Manual.Boston: The Health Institute, New England Medical Center 1996.
18. Raat H, Landgraf JM, Bonsel GJ, et al. Reliability and validity of the child health questionnaire-child form (CHQ-CF87) in a Dutch adolescent population. Qual Life Res 2002 11:575–81.[CrossRef][ISI][Medline]
19. Jenkins MA, Clarke JR, Carlin JB, et al. Validation of questionnaire and bronchial hyperresponsiveness against respiratory physician assessment in the diagnosis of asthma. Int J Epidemiol 1996 25:609–16.
20. US Department of Health and Human Services (USDHHS). Preventing Tobacco Use among Young People: A Report of the Surgeon General.Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health 1994.
21. De Vries H, Mudde AN, Dijkstra A, Willemsen MC. Differential beliefs, perceived social influences, and self-efficacy expectations among smokers in various motivational phases. Prev Med 1998 27:5 Pt 1681–9.[CrossRef][ISI][Medline]
22. Bogers RP, Van Assema P, Kester AD, et al. Reproducibility, validity, and responsiveness to change of a short questionnaire for measuring fruit and vegetable intake. Am J Epidemiol 2004 159:900–9.
23. Oenema A and Brug J. Feedback strategies to raise awareness of personal dietary intake: results of a randomized controlled trial. Prev Med 2003 36:429–39.[CrossRef][ISI][Medline]
24. Keij I. Aantallen allochtonen volgens verschillende definities [Number of Foreigners According to Different Definitions].Voorburg/Heerlen: Centraal Bureau voor de Statistiek [Statistics Netherlands] 2000.
25. Weinstein ND. The precaution adoption process. Health Psychol 1988 7:355–86.[CrossRef][ISI][Medline]
26. Babor TF, Sciamanna CN, Pronk NP. Assessing multiple risk behaviors in primary care. Screening issues and related concepts. Am J Prev Med 2004 27:2 Suppl, 42–53.[ISI][Medline]
27. Kruijf AF, Borne van den HW, Mesters I, et al. Ondersteuning van de voorlichting tijdens een medisch consult: een vragen-checklist voor astma- en COPD-patiënten [supporting education during medical consultations: a questions checklist for asthma and COPD patients]. Tijdschr Soc Gezondheidsz 1997 75:74–80.
28. Edwards A, Unigwe S, Elwyn G, et al. Personalised risk communication for informed decision making about entering screening programs. The Cochrane Database of Systematic Reviews 2003 Issue 1. Art. No.: CD001865, doi: 10.1002/14651858.CD001865.
29. Freed LH, Ellen JM, Irwin CE Jr, et al. Determinants of adolescents' satisfaction with health care providers and intentions to keep follow-up appointments. J Adolesc Health 1998 22:475–9.[CrossRef][ISI][Medline]
30. Statistics Netherlands. Statline.Voorburg/Heerlen: CBS Available at: http://statline.cbs.nl/StatWeb/start.asp?lp=Search/Search. Accessed: 19 May 2005.
Received on July 15, 2005; accepted on March 13, 2006
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||