Health Education Research Advance Access originally published online on October 12, 2005
Health Education Research 2006 21(2):219-229; doi:10.1093/her/cyh058
Project quality rating by experts and practitioners: experience with Preffi 2.0 as a quality assessment instrument
1 Netherlands Institute for Health Promotion and Disease Prevention, Centre for Knowledge and Quality Management, PO Box 500, NL-3440 AM Woerden, the Netherlands
2 Prevention Research Centre, Department of Health Education and Promotion, University of Maastricht, Maastricht, the Netherlands
3 Department of Clinical Psychology, University of Nijmegen, Nijmegen, the Netherlands
4 Faculty of Psychology, University of Maastricht, Maastricht, the Netherlands
5 Methodology, Amsterdam, the Netherlands
*Correspondence to: G. R. M. Molleman. E-mail: gmolleman{at}nigz.nl
Preffi 2.0 is an evidence-based Dutch quality assessment instrument for health promotion interventions. It is mainly intended for both planning and assessing one's own projects but can also be used to assess other people's projects (external use). This article reports a study on the reliability of Preffi as an external quality assessment instrument. Preffi is used to assess quality at three levels: (i) specific criteria, (ii) clusters of criteria and (iii) entire projects. The study compared Preffi-based assessments of 20 projects by three practitioners with their intuitive assessments of the same projects and with assessments by three experts, which were to be used as external criteria. The intuitive assessments only related to the cluster and project levels. Our main hypothesis was that intuitive assessments by practitioners would be less reliable and accurate than their Preffi-based assessments and the experts' assessments. On the whole, we failed to confirm this hypothesis: the experts' assessments proved less reliable and accurate than the practitioners' intuitive and Preffi-based assessments and differed too much from each other to be used as external criteria. The Preffi-based assessments by the practitioners had an acceptable generalizability coefficient (G) and accuracy (standard error of measurement). At the level of the entire project, two assessors are needed to produce sufficiently reliable and accurate assessments, whereas three are needed for assessment at cluster level. The study also showed that different assessors use different perspectives and base their assessment on a variety of aspects. This was regarded as inevitable and even useful by the assessors themselves. Discussions between assessors are important to achieve consensus. The article suggests some improvements to Preffi to further increase its reliability.
![]()
CiteULike
Connotea
Del.icio.us What's this?
This article has been cited by other articles:
![]() |
E Koppelaar, J J Knibbe, H S Miedema, and A Burdorf Determinants of implementation of primary preventive interventions on patient handling in healthcare: a systematic review Occup. Environ. Med., June 1, 2009; 66(6): 353 - 360. [Abstract] [Full Text] [PDF] |
||||
