Health Education Research Advance Access originally published online on September 14, 2004
Health Education Research 2005 20(2):214-225; doi:10.1093/her/cyg119
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Health Education Research Vol.20 no.2, © Oxford University Press 2005; All rights reserved
Whither health promotion events? A judicial approach to evidence
1 University of Glasgow, Rutherford/McCowan Building, Crichton Campus, Bankend Road, Dumfries DG1 4ZL and 2 Department of Epidemiology and Public Health, University of Nottingham, Nottingham NG7 2RD, UK
3 Correspondence to: S. Whitelaw E-mail: s.whitelaw{at}crichton.gla.ac.uk
 |
Abstract |
|---|
 Top Abstract IntroductionMethodThe evidence baseConclusion and recommendationsReferences |
|---|
This paper reports on a review undertaken for the Health Education Board for Scotland on ‘health events’. On the basis of a literature review and interviews with 20 professionals in the UK, it appraised the effectiveness of such activities, assessed their current extent and status, and ultimately considered whether they are useful and relevant within emergent contexts. It suggests that the evidence base for health events is relatively weak, particularly given the scale of such work. At best, all that can be said is that there is some evidence that some of these events have some use within very specific assumptions and values. Consideration is given to possible ways forward. The paper also considers a series of related issues that impinge on the assessment of complex health promotion interventions. From our field interviews, health events are perceived by most to be labour intensive and ineffective. Rather, they are undertaken for a range of pragmatic ‘public relations’ reasons that exist independent of knowledge, attitudinal or behavioural outcomes. This ambiguity of expectation coupled with a paucity of published high-quality literature led to the use of a judicial approach to evidence appraisal. It considers the ways in which this exercise could be undertaken, ultimately being sceptical of systematic meta-reviews.
This paper reports on a review undertaken for the Health Education Board for Scotland on ‘health events’. On the basis of a literature review and interviews with 20 professionals in the UK, it appraised the effectiveness of such activities, assessed their current extent and status, and ultimately considered whether they are useful and relevant within emergent contexts. It suggests that the evidence base for health events is relatively weak, particularly given the scale of such work. At best, all that can be said is that there is some evidence that some of these events have some use within very specific assumptions and values. Consideration is given to possible ways forward. The paper also considers a series of related issues that impinge on the assessment of complex health promotion interventions. From our field interviews, health events are perceived by most to be labour intensive and ineffective. Rather, they are undertaken for a range of pragmatic ‘public relations’ reasons that exist independent of knowledge, attitudinal or behavioural outcomes. This ambiguity of expectation coupled with a paucity of published high-quality literature led to the use of a judicial approach to evidence appraisal. It considers the ways in which this exercise could be undertaken, ultimately being sceptical of systematic meta-reviews.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodThe evidence baseConclusion and recommendationsReferences |
|---|
The 1980s was perhaps an era when health promotion came closest to becoming health promotions. In a context that uncannily mirrors current concern that health policy consists more of spin than substance (Delamothe, 2000
) and an enduring concern over the use of advertising in the health sector (Smith, 2004), this time can be characterized as being particularly interested in marketing and selling (Young, 1988) with ramifications in the health sector, e.g. the advent of a UK health service ‘market’ (Klein, 1998), the notion of the commodification and consumption of health (Lupton, 1995), and, of particular interest in this paper, the emergence of ‘events’ such as No Smoking Day (NSD), World AIDS Day (WAD) and Scottish Mental Health Week (SMHW).
A sense of unease over such interventions existed even at this time, e.g. Williams (Williams, 1984) compared health promoters to salesmen; Chippendale and Horrie (Chippendale and Horrie, 1988) coined the term ‘advertocracy’ to reflect a tendency to deal with health problems through superficial publicity; Anderson (Anderson, 1988) saw this work as an inappropriate ‘megaphone solution’ to problems; and Le Fanu (Le Fanu, 1994) coined the term ‘preventionitis’ to reflect the pervasiveness of such activities. Nevertheless, health events subsequently proliferated to the point that there are approaching 50 in Scotland (NHS Health Scotland, 2003). Whilst there has been a degree of implicit political, professional and public support (Plowright, 2004), debate over their worth has sporadically surfaced (Flay, 1987).
This exists within a wider deliberation over the preferred nature of health promotion that has developed over the past 20 years—essentially tensions between topic-based individualistic and structural approaches [e.g. (Connelly, 2002; Giles-Corti and Donovan, 2002; Ory et al., 2002)]. For some, the former, focussing on topics like smoking, drug misuse, diet, etc., is seen as crucial (Percival and Gough, 2001) and a belief exists that events must have some effect in this pursuit [(Lupton, 1995), p. 108; (Sidell, 1997), p. 173]. Even within a new public health approach, Ashton and Seymour [(Ashton and Seymour, 1996), pp. 61–62] find room for ‘mass populist events’. Conversely, more critical views exist, e.g. events use a simplistic theory of the mechanisms of actions and effects (Lull, 1999; Thompson, 1995); they are methodologically superficial and inappropriate in relation to profound psychological, socio-cultural and environmental forces (Naidoo, 1986; Gough, 2001); and thus the notion that such work can be particularly disadvantageous towards those with the least resources (Townsend et al., 1994; Finnegan, 1999).
In the UK, such views have been most publicly aired within the context of NSD with, for example, Sir Richard Doll expressing concern over potentially paradoxical effects of the day on smokers (BBC Radio 4 Desert Island Discs, January 2001) and the anti-smoking expert Allen Carr in his recent biography suggesting that it tends to pillory smokers (Carr, 2003). Consequently, amongst others, Ziglio [(Ziglio, 2000), p. 23] has called for a general repositioning of health promotion to what Kickbusch [(Kickbusch, 1998), p. 34] has called an ‘investment based health policy’ orientation that addresses social, environmental and economic determinants of health.
These tensions are perhaps becoming keener, informed by developments in theory (Green, 2000; Adams et al., 2002) and evidence-based appraisal (Raphael, 2000). Whilst these influences may appear distant and ethereal, they are having an impact on the way that health promotion is actually delivered at all levels (Hancock and Labonte, 2000). For example, in Scotland a new strategic health authority was created in 2000, the Public Health Institute for Scotland (PHIS) alongside the existing national health promotion agency, the Health Education Board for Scotland (HEBS) (which subsequently merged in 2003 to form NHS Health Scotland).
This paper reports on work commissioned in late 2002 by the HEBS Special Projects Programme prior to this merger. The Programme provides support for ‘health events’, that vary in relation to scale (one-off and on-going), complexity (single and multiple component), focus [behavioural topics (e.g. smoking, drugs, dental health, etc.), behavioural issues (e.g. sexual and mental health), disease groups [cancers, mental illnesses and population groups (older people, carers)] and, most importantly, attempted goals:
- Awareness raising and motivation to carry out routine procedures (e.g. National Bug Busting Days)
- The promotion of a profile of a general topic area and the work of an organization (e.g. Headway National Brain Injury Week)
- The promotion of particular services with a view to improve access and uptake (e.g. the Family Planning Association's Contraception Awareness Week)
- A means of health advocacy on behalf of a group (e.g. Age Concern's Older People's Week)
- Charitable fundraising (e.g. The World's Biggest Coffee Morning for Macmillan Cancer Relief)
|
Method |
|---|
|
TopAbstractIntroductionMethodThe evidence baseConclusion and recommendationsReferences |
|---|
The work considered health events in the following ways: (1) it appraised their general effectiveness, (2) it assessed their current extent and status in Scotland, (3) it considered whether these types of events are useful. It comprised two parts.
First, a review of the literature was undertaken using examples from the fields of smoking, mental health, and drugs and alcohol, and drawing on various types (formal and informal/grey literature, general theoretical and empirical material, descriptive and analytical/evaluative accounts, and HEBS/Scottish and worldwide).
It was undertaken within circumstances that made a purely technical appraisal of evidence difficult. In relation to definition: that a ‘health event’ is a moniker that contains a variety of actions and expected effects, and that these may exist within a hierarchical chain of expectations, seen either in the short term and in their own right or contributory to higher goals and appraised over longer timescales. In relation to method and measurement: that there are relatively few fully reported evaluations of such work and that given the complexity of such actions, it is often difficult to link effect to actions (Levy and Friend, 2000). In relation to appraisal, it was therefore difficult to judge the effectiveness of health events in isolation. Finally, at the level of values, we believed that there needed to be a recognition that opinions on health events are not only shaped by post hoc empirical data but also by a range of a priori preferences. As such, we felt it impossible to undertake a purely technical review around discrete and narrow terms. Rather, a more flexible approach was adopted in an effort to compile a comprehensive and balanced picture of evidence using a range of generic (e.g. special events; health events; health days) and specific (e.g. NSD, Breast Awareness Week, WAD) search terms. Analysis subsequently employed a similarly flexible ‘judicial’ approach (Tones, 1997). Green and Tones' definition of this as, ‘assembling sufficient evidence to lead to a confident decision about a course of action even though absolute proof is not available’ [(Green and Tones, 1999), p. 137] provided a helpful maxim.
Given the ambiguities described above, we believed that two features of such a review particularly suited our subject matter. First, within an ethic of drawing upon an open range of triangulated data and sources [(Green and Tones, 1999), p. 137] we wanted to build a wide and complex picture of evidence. Second, we wanted to avoid shallow or simple end judgements of data. The essentially cautious approach based on a ‘balance of probabilities’ [(Green and Tones, 1999), p. 137] was seen as most appropriate. We thus analysed and collated data in relation to affirmative and pessimistic orientations, whilst maintaining room within coverage of any extenuating circumstances for deliberation on the status and quality of this evidence.
To ensure an informed and focused approach, an initial briefing paper was completed summarizing review findings and was used as the basis of the second element—telephone interviews with relevant professionals; a total of 22 professionals encompassing HEBS staff (e.g. Special Projects Programme staff, Head of Public Affairs), Managers of NHS Scotland Health Promotion services, senior managers from local authorities sectors (e.g. leisure and recreation) and the voluntary sector agencies (e.g. ASH Scotland) with interests in special events, and other UK health promotion agencies. Effort was made to purposively include those with both positive and critical perspectives. The aim of this element was to test the validity of literature findings and to examine the status of events in the field. The following were principal questions:
- Do these events contribute strategically to health improvement agendas?
- What value do stakeholders see in them?
- Does the duration and nature of support for events influence their value and impact for stakeholders?
|
The evidence base |
|---|
|
TopAbstractIntroductionMethodThe evidence baseConclusion and recommendationsReferences |
|---|
Affirmative views: the case for
Within an optimistic context, the literature suggests a range of potentially positive outcomes. There is an assumption that these events must do something, even if this is at the modest level of creating a bedrock of raising public awareness (Mudde and De Vries, 1999
) and/or enhancing wider political consciousness for a health issue (Chapman, 1997; Wellings and Macdowall, 2000a). Usually couched as ‘public education’, such hunches continue to receive implicit policy support (Department of Health, 1998; Friedli, 2000; HM Treasury, 2004), although interestingly special events rarely appear to be named as explicit strategic elements, e.g. NSD does not receive any specific citation in the UK smoking strategy paper Smoking Kills.
Such wisdom is related to two perceived benefits: (1) they provide access, Reid et al. [(Reid et al., 1992), p. 191] suggesting that NSD ‘reaches large numbers of smokers quickly’, and (2) it assumes that they keep health issues in the public profile and in the case of smoking, actively counter tobacco advertising [e.g. (Pechmann and Reibling, 2000; Sargent et al., 2000; Farrelly et al., 2002)]. The notion of ‘truth counter-marketing’ [(Ellis, 2002), p. 895] where unhealthy or negative information is balanced by healthy alternatives therefore exists—what Wakefield terms, ‘unselling the cigarette’ [(Wakefield, 1999), p. 524]. There is some evidence that such activities are associated with successful outcomes, e.g. an increase in anti-tobacco attitudes and beliefs (Farrelly et al., 2002).
Furthermore, descriptive literature frequently cites impressive effects. In relation to NSD, estimates of over 1 million people taking part are cited and media tracking surveys show high levels of campaign awareness—estimated at 61% of smokers and 56% of the general public with indications that approximately 9% of smokers try to stop on the day (Health 21, 2001; McIntyre, 2001; National No Smoking Day, 2002). More ambitiously, behavioural claims are made for both long- and short-term impacts, ‘nearly 1 million people have stopped smoking as a result of No Smoking Day since the first Day in 1984’ (National No Smoking Day, 2004) and ‘new research suggests that of the 1.2 million smokers who tried to stop on No Smoking Day 2001, 180 000 lasted more than 3 months and a further 120 000 were still not smoking 6 months later; 40 000 remain smoke free for good’ (National No Smoking Day, 2004). Some academic evaluations offer similar optimism with evidence of event specific reductions in the order of anything between 1 and 5% post-intervention, and significant on-going decreases in prevalence (Flay, 1987; Frith et al., 1997; Pierce and Gilpin, 2001).
In moving beyond core effectiveness, some literature has taken a more restrained approach, suggesting the possibility of moderate, contributory or incremental effects (Freimuth and Taylor, 1995; Hornick, 2002) when undertaken with various types of pre-requisite or enabling factors like: conducive cultures and environments; favourable secular trends; individual pre-disposition; the development of supportive skills; high, targeted and on-going media exposure; and the existence of other synergistic interventions (McVey and Stapleton, 2000; Wakefield and Chaloupka, 2000; Schar and Guitierrez, 2001; Hornick, 2002; Smith et al., 2002; Snyder and Hamilton, 2002). Thus, there is some evidence of effectiveness in, for example: securing an intent to change behaviour (Health Education Authority, 1997); gains in knowledge and attitudes to mental illness (Tilford et al., 1997); increases in dental health related knowledge arising from National Smile Week (Whittle et al., 1994); and reductions in weight and fat and snack intake from the BBC campaign Fighting Fat, Fighting Fit (Miles et al., 2001).
Of those we interviewed, a significant majority were supportive of the continuation of some events. Of note though, none of these affirmative views were expressed on the basis of the effects outlined above. That is, almost exclusively, responses were made independent of any supportive comment on effectiveness and most were sceptical of any simple effect. Rather, a range of second-order pragmatic rationales for doing such work was offered. At the broadest level, there was a notion that events ‘keep health on the national agenda...maintaining a public profile’ (expressed by a health promotion manager). The notion of events and the associated HEBS logo offering a high-quality ‘brand’ or ‘corporate image’ was thus often cited. Local health promotion service managers also appreciated the high production quality of nationally produced materials and the local kudos thus derived, e.g. ‘yes it's nice to be able to be associated with well produced materials...and they're free!’. There was also a feeling that they help promote health promotion services to other local health related staff and services. For example, a health promotion manager suggested, ‘we simply use these events to bring health professionals through our door...they may be interested in these days but it's the deeper on-going work that we are interested in’. Some even felt that events were the best way of developing more profound partnerships. Additionally, in taking on such work, there was a belief that the health promotion service would gain political credit within the wider NHS environment. That is, there is a perception that it conforms to the expectations of significant local sources that this work should be done by someone and that someone should be the health promotion service. A manager reflected this position, ‘you know...sometimes you have to do work that is expected of you...lots of people within the Health Board want this work to happen’.
For most, this support was given on the basis of a number of future provisos. First, there was a perception that there are too many health events, and a strong expression of limiting events and ‘creating a big splash’ around a limited number was favoured. Second, some felt that many events stray into overly ‘medicalized’ areas that would be better served by other services. Third, the deployment of additional criteria in weeding out inappropriate events was suggested, e.g. ensuring that events reflect local needs, are targeted at defined and specific groups, and are associated with wider on-going activities.
So whilst one can only conclude that there is significant field support for the continuation of special events, this is given largely on a peculiar pragmatic basis, one of perceived opportunism rather than effectiveness.
Pessimistic: the case against
A range of contrary evidence exists. First, given the long-term existence of such activity, there is a surprising dearth of high quality evaluative literature (Egger et al., 1999; Ludbrook et al., 2000; Hornick, 2002). Furthermore, much of it appears to be methodologically simple, descriptive and carried out within affirmative political and research contexts (Sykes and Marks, 2000), e.g. Ludbrook et al. [(Ludbrook et al., 2000), p. 43] conclude that evaluations ‘are rarely carried out within a research design that allows robust evaluation to take place’.
Consequently, health events are curiously absent from the range of topic-based effectiveness reviews [e.g. preventing falls, promoting physical activity, and even areas like smoking and mental health where high profile campaigns are prominent] (Hodgson et al., 1996; NHS Centre for Reviews and Dissemination, 1996, 1997; Owen, 1996; Tilford et al., 1997; Health Education Authority, 1998; Kane and Wellings, 1999; Hosman, 2000; West et al., 2000; Raw et al., 2002; Towner and Dowswell, 2002; Waller et al., 2002; WHO, 2002).
Of the quality empirical data that does exists, much of it suggests that health events have limited effects, particularly on attitudes and behaviours [(Ludbrook et al., 2002), p. 49; (Wimbush et al., 1998; McNeill and Heuston, 1999; Yzer et al., 2000; Smith et al., 2002)], and that respectively these are likely to be short term (Caraher et al., 2000) and can reach a limited ceiling (Freimuth and Taylor, 1995). For example, Hillsdon et al. (Hillsdon et al., 2001) report no significant changes in any project indicators associated with the Health Education Authority's ACTIVE for LIFE campaign. This is particularly of interest given that, in being long term, well funded, multi-faceted and theoretical sound, the project conformed to most indicators associated with quality health promotion campaigns. Likewise, the evaluation used a relatively sophisticated prospective longitudinal study and it could be the case that it is this robust design that is revealing deficiencies.
Foulds (Foulds, 2000) suggests that the important link between awareness raising and behaviour change is potentially tenuous with only 9% of callers being transferred to counsellors and 82% simply requesting literature. Sykes and Marks (Sykes and Marks, 2001) raise a range of concerns around NSD, including the material being culturally inappropriate, the material not engaging with social processes involved in smoking cessation, and, again, the poor links between service oriented promotional information and follow up support.
Such disappointments have been complemented by theoretical insights. For example, the difficulty for events to convey complex information or teach necessary motor or social skills and their inability to bring about change in the absence of prior individual pre-disposition, favourable secular population trends or various enabling factors (Wakefield and Chaloupka, 2000; Schar and Guitierrez, 2001; Tones and Tilford, 2001; Hornick, 2002). More profoundly, a sustained theme has focussed on health events being insignificant in difficult socio-economic circumstances (Finnegan, 1999).
There is also a suggestion that, at a population level, health events can have paradoxical effects of appealing disproportionately to those who are not in the specific target group (Townsend et al., 1994; Lupton, 1995; Hillsdon et al., 2001; Wardle et al., 2001; Stevens et al., 2002). For example, in their review of the BBC campaign Fighting Fit, Fighting Fat, Miles et al. (Miles et al., 2001) note that whilst there was some indication that the campaign had positively influenced weight, eating behaviour and activity, campaign drop out was more likely in obese rather than in normal or slightly overweight populations. Relatedly, in the drugs and smoking field, the potential for paradoxical campaign effects where injudicious information increases interest in use is also noted (Dorn and Murji, 1992; McNeill and Heuston, 1999). Likewise, McNeill (McNeill, 2001) notes the existence of groups ‘alienated’ by NSD who actually smoked more during that day. Almost all of those interviewed expressed critical views synonymous with the above themes and these can be grouped as: evidence based, ideological and pragmatic, and are considered below.
First, some were familiar with the evidence literature and essentially sceptical of effect. Two managers located such an appraisal in the context of health governance processes currently being used within their health boards and suggested that in this context it would be unlikely for health events to have a long-term future. These individuals dismissed pragmatic and political outcomes as irrelevant, and wanted to focus on specific intervention outcomes. Indeed, some felt that the very existence of health events and the way they are often used to ‘bring in’ other health professionals stereotyped the nature of all health promotion work, ultimately making it more difficult to pursue other approaches. As such, two interviewees felt health events should not be used as a basis for developing partnerships as other more formal and successful mechanisms are now available.
Second, some held what appeared to be an a priori ideological dislike of the thin and individualistic nature of such events, and cited the potential for them to fail to address the needs of particular social groups and thus to wider health inequalities.
Finally, a significant proportion expressed more practical concerns. For example, the time-consuming, routine and recurrent nature of such events was almost unanimously offered as creating a burden; a health promotion manager stated ‘sometimes it feels like a never-ending hassle...once you get one out of the way the next one comes along’. Many were also concerned about the hidden resource implications, highlighting the varied elements required (a distribution role for material, deeper liaison with other health professionals, linking to wider interventions), and thus the significant on-going contributions from health promotion specialists and resource officers. Whilst the cost-effectiveness of such events has often been suggested (National No Smoking Day, 2004), one manager thus proposed the contrary; the combination of high resources and low effect produced what he felt was relative cost-ineffectiveness.
Extenuating circumstances
As well as considering substantive evidence, as a fallibilist exercise, any judicial process is required to concurrently review the bases upon which data has been generated (Green and Tones, 1999). An uncomplicated appraisal of the literature would tend to suggest that health events are on the whole relatively ineffective. Additionally, the work associated with them has been considered to be both methodologically weak and plagued by compliant ‘experimenter effects’ and ‘publication biases’ [(Perse, 2001), pp. 8–10]. However, a range of countering concerns can be identified here: definitions, effect attributions and politics.
First, at a definitional level, the nature of health events and their aims are clearly multiple and varied—from relatively poor practice one-off interventions to a form of best practice where they constitute one part of a long-term, multi-faceted intervention (Stead et al., 2000). As such, it is often difficult to pool evidence from such a range which in turn result in difficulties in broad project transfer and evidence dissemination [(Grey et al., 2000), p. 9]. Additionally, there is often confusion over the expected effects, particularly an inappropriate anticipation of immediate behavioural change. Failure to achieve such lofty goals may as such result in health events being unjustly labelled as failures [(Freimuth et al., 2000; Wimbush et al., 1998), p. 53]. Such ambiguity also presents difficulties in framing the time-scales in assessing impact (Grey et al., 2000), particularly when indicators may be ambitious (e.g. behavioural) or profound and elusive (e.g. cultural) [(Hillsdon et al., 2001), p. 760].
Second, whilst some retrospective studies have sought to establish associations between reductions in smoking and NSD-like campaigns (Pierce and Gilpin, 2001; Laugerson and Swinburn, 2000; Farrelly et al., 2002; Friend and Levy, 2002) establishing tight intervention–outcome causality is clearly challenging (Frith et al., 1997). This is perceived to be affected by the complex, indirect and multiple theoretical effects of health events, and the difficulties in separate campaign effects from co-incidental secular trends [(Hillsdon et al., 2001), p. 760]. Friend and Levy (Friend and Levy, 2002) thus highlight a paradox associated with best practice health events. On the one hand, the literature suggests that they are most effective when undertaken in association with other forms of activity, yet it is these very circumstances (their complexity and interactivity) that make it difficult to isolate or attribute impacts unequivocally to campaign inputs. Fried and Levy [(Fried and Levy, 2002), p. 85], for example, note ‘[other] tobacco control policies that are implemented during the campaign often make it difficult to identify the specific influence of media campaigns alone’.
Consequently, it could be the case that discrete campaign interventions are having an effect, but visibility or proof is mitigated by a range of barriers: primarily, it could be the case that the discrete part of the campaign is of a high quality and succeeding in the sense that it achieves its set (and often modest) aims but ultimately the major aim fails due to other detracting factors—a case of ‘winning the battle but losing the war’ (Green and Tones, 1999). It could also be the case that we do not have the analytical concepts and tools to monitor positive change even if it was happening (Wellings and Macdowall, 2000b); finally, as a consequence of these forces, it is possible that a paucity of evidence of effectiveness is simply the product of a paucity of attempts at generating evidence (Tannahill, 1998).
Finally, it is possible that political circumstances have allowed health events to continue largely independent of substantial scrutiny. Picken and St Leger (Picken and St Leger, 1993), for example, note the existence of features within services and interventions that lead them to be respectively susceptible or resistant to scrutiny. The factors that are considered to be protective (‘the service is too well established, is held in high public esteem, is backed by powerful interest groups, is a cheap placebo’) could be associated with high-profile campaigns. In particular, passive disinterest or active protection could arise from a range of sources: the common sense hunch of campaigns being effective; genuine recognition of the methodological difficulties in attributing effect; political expediency where the campaigns promote visibility and create the impression of action (e.g. in the drugs field); and the notion that events act to counter unhealthy information.
Summing up
Before coming to any substantive conclusions, a few comments on the process involved in undertaking it are perhaps important. Two features are particularly important. First, there appears to be an increasing demand for unambiguous and definitive evidence-based decisions on practice preferences within the broad health and social field, and increasingly within health promotion (Oakley, 1998; Straus, 2004). Second, one central element of this effort is the pooling of data that exits around particular types of practice or topic areas in the form of evidence reviews [e.g. (NHS Centre for Reviews and Dissemination, 1996; Health Education Authority, 1998]. For some, this exercise should be mechanical and uncomplicated, simply mimicking the hierarchical evidence values and systematic procedures applied to a host of health related interventions (Snyder and Hamilton, 2002; Raw et al., 2002). In particular, some are calling for a shift from ‘subjective narrative reviews’ (Velicer, 2003) where items are combined in a relatively unsystematically fashion towards what Noar (Noar, 2003) terms the ‘superior’ and ‘unbiased’ quantitative and statistically oriented notion of meta-analysis (Schmidt and Hunter, 2003). Three specific advantages are felt to arise from this approach: (1) the heterogeneity of studies can be identified and specific links between intervention types, significant variables and effects isolated; (2) the quality of evaluation can be appraised and respectively poorly designed studies omitted, and weighted credit given to studies of larger sample sizes and better methodology; and (3) specific effect sizes can be calculated (Cooper and Rosenthal, 1980). In circumstances where significant amounts of high-quality evaluative literature exist with relative consensus around what the intervention is and what outcomes should be expected, these ideals may be laudable.
Such an approach, however, falters in more difficult conditions concisely defined by Tones and Tilford [(Tones and Tilford, 2001), pp. 171–172] as comprising: imprecise descriptions of the nature and expectations of the studied interventions; few evaluative studies; and furthermore few studies that use quantitative data and ‘randomized controlled trial’-type comparative structures. These themes have great resonance in the work reported here. Health events appear to exist in a peculiar context where the commitment of significant levels of resources has not been matched by concomitant evaluative efforts or resources. Various elements such as the definitional ambiguity and methodological complexity in measuring and appraising effects also preclude the achievement of simple answers. Additionally, raw evidence data cannot be isolated from the political and largely pragmatic context from which it arises.
Thus, we concur with Tones and Tilford's view that the use of the classic systematic review in health promotion is largely ‘premature’ [(Tones and Tilford, 2001), p. 172]. We believe the more flexible judicial approach helped deal with the complexity and ambiguity of this area, and tended to prevent any dash towards shallow and false certainties. Furthermore, undertaking interviews with field staff allowed us to understand the rationales and assumptions behind practical choices, and the contextual pressures that influence the relative weight placed on various aspects of the evidence base. One thorny issue perhaps remains for us—the relationship between this judicial approach and maintaining a traditional view of rigour. Whilst favouring ‘a broader, more pluralistic approach’, Green and Tones [(Green and Tones, 1999), p. 137] suggest that this ‘must not be at the expense of compromising rigour or sacrificing validity’. In our experience we felt that the breadth and complexity of the exercise inevitably led to threats to traditional notions of rigour and both conceptual and practical difficulties. In dealing with ambiguous concepts, varied types of data, from varied sources, we saw the process as one that demanded craft rather than simply technique. In this sense, we would suggest that traditional ideas of rigour and validity should be replaced by the more appropriate notion of trustworthiness (perhaps embodied in the very task of the reader appraising the ‘balance of probabilities’?). In this sense, we see the paper as a balanced discussion paper upon which others can make further subsequent decisions.
Beyond these conceptual reflections, the brief for this work still set a pointed question: ‘are these types of events useful and relevant within the context of current and emerging health promotion agendas?’ and some form of answer was needed. The most positive response to this would be that there is some evidence that some of these events have some use within very specific assumptions and values. There are certainly some examples of relatively high-quality evaluation, and these show that health campaigns can significantly increase levels of health topic awareness and knowledge around and can be associated with favourable longer-term secular trends.
Beyond this optimism, the feeling remains that continuance of events reflects a peculiar set of incongruous circumstances. Despite strong professional and political support from particular quarters and relatively high resourcing, there is no escaping that fact that the evidence base is generally weak—mostly short term, isolated, descriptive, and generated and interpreted within an optimistic context.
Thus, our hunch is that health events continue to be initiated at the higher political levels on the basis of three specious principles: (1) from a general faith in their effectiveness rather than the result of any formal appraisal; (2) in the context of a cautious fear of retreating from what has traditionally been done, being unwilling to face criticism for ceasing what is already exists; and (3) as a political cover (for example as the notion that mass media work provides an image of action). Plant (Plant, 2004) offers a recent prescient example of this in describing the development of an alcohol harm minimization strategy for England. He reports that an interim strategy report (Cabinet Office, 2004a) was circumspect over the role of simple information giving expressing a theme already established in this paper; ‘education successfully imparts information but cannot change behaviour in isolation from other measures’ [(Cabinet Office, 2004a), p 172]. Plant develops this theme suggesting that in this field ‘education and communication have a poor record’ and that they should be treated as ‘purely experimental and not as an effective or major arm of policy’ [(Plant, 2004), p. 905]. However, a final strategy document places continued significance on communication and education [(Cabinet Office 2004b), p. 27]. Plant thus sees these activities as ‘expensive symbolic gestures’ concluding, ‘sadly politicians often fail to resist the lure of high profile (if generally unproductive) campaigns’ [(Plant, 2004), p. 905].
|
Conclusion and recommendations |
|---|
|
TopAbstractIntroductionMethodThe evidence baseConclusion and recommendationsReferences |
|---|
So what should be done? If relatively stringent assessment criteria were applied to the approach as a whole, the balance of probability would perhaps be against the use of events. Yet within a judicious context, a fuller and more realistic approach is perhaps required. It would be inappropriate to suggest that all special events are of a problematic status; much clearly depends on their nature and context of their use. In a more permissive context, the theoretical literature already contains a number of guidelines (Baker and Caraher, 1995
; Hastings and Stead, 1996; Schar and Guitierrez, 2001) that could be applied in systematically selecting events that would contribute to a wider health promotion effort. For example, events should: have a highly defined purpose with clear and measurable aims and objectives; be consistent, sustained and repeated; not operate in isolation, but rather have some degree of interactivity and are combined with community, small group and face-to-face skills based interventions and contextualized in existing structures like the family, the community or the school. Likewise optimal content should variously be: positively orientated; realistic; relevant and salient to target groups; perceived as familiar, attractive and credible; build on the audience's existing motives, needs and values.
On this basis, a significant number of existing smaller scale one off events could be omitted from future rosters leaving a more focussed continuing role for events such as NSD and WAD. On a similar basis, an assessment could be made to the relevance of many of the subject areas of events to the priority areas for health promotion, e.g. many felt that medical issues would be best left to other agencies.
A number of criteria were identified for such an exit from some events: that it is planned in advance and compatible with funding cycles; that it focuses on priority areas compatible with a health promotion domain; if work is to be retained centrally, there is a clear lead on what NHS Health Scotland intends to do as an alternative; if work is to be devolved to the local level, there is a guarantee of re-allocation of funding.
|
Note |
|---|
The paper does not represent the views of NHS Health Scotland, but is rather the product of the work of the individual authors.
|
References |
|---|
|
TopAbstractIntroductionMethodThe evidence baseConclusion and recommendationsReferences |
|---|
Adams, L., Amos, M. and Munro, J. (eds.) (2002) Promoting Health: Politics and Practice. Sage, London.
Anderson, D. (1988) The Megaphone Solution: Government Attempts to Cure Social Problems with Mass Media Campaigns. Social Affairs Unit, London.
Ashton, J. and Seymour, H. (1996) The New Public Health. Open University Press, Milton Keynes.
Baker, H. and Carter, M. (1995) A Guide to Media Production. Home Office, Central Drug Prevention Unit, London.
Cabinet Office (2004a) Alcohol Project: Interim Analytical Report. Cabinet Office, Prime Minister's Strategy Unit, London.
Cabinet Office (2004b) Alcohol Reduction Strategy for England. Cabinet Office, Prime Minister's Strategy Unit, London.
Caraher, M., Bird, L. and Hayton, P. (2000) Evaluation of a campaign to promote mental health in young offender institutions: problems and lessons for future practice. Health Education Journal, 59, 211–227.
Carr, A. (2003) Packing It In The Easy Way. Michael Joseph, London.
Chapman, S. (1997) Stopping AIDS: AIDS/HIV public education and the mass media in Europe. British Medical Journal, 315, 1027.
Chippendale, P. and Horrie, C. (1988) It all adds up. New Statesman and Society, 7 Oct, 10–11.
Connelly, J. (2002) The Behaviour Change Consortium studies: missed opportunities—individual focus with an inadequate engagement with personhood and socio-economic realities. Health Education Research, 17, 691–695.
Cooper, H. and Rosenthal, R. (1980) Statistical verses traditional procedures for summarizing research findings. Psychological Bulletin, 87, 442–49.[CrossRef][ISI][Medline]
Delamothe, T. (2000) Sultans of spin. British Medical Journal, 320, 811.
Department of Health (1998) Smoking Kills: A White Paper on Tobacco. Cm 4177. The Stationery Office, London.
Dorn, N. and Murji, K. (1992) Drug prevention: a review of the English language literature. ISDD Research Monograph, 5.
Egger, G., Donovan, R. and Spark, R. (1999) Health and the Media: Principles and Practice for Health Promotion. McGraw-Hill, Sydney.
Ellis, J. (2002) Tobacco and the media. American Journal of Public Health, 92, 895.
Farrelly, M., Healton, C., Davis, K., Messeri, P., Hersey, J. and Haviland, M.L. (2002) Getting to the truth: evaluating national tobacco countermarketing campaigns. American Journal of Public Health, 92, 901–907.
Finnegan, J. (1999) Mass media and health promotion. In Bracht, N. (ed.), Health Promotion at Community Level. Sage, London, pp. 121–142.
Flay, B. (1987) Mass media and smoking cessation: a critical review. American Journal of Public Health, 77, 153–160.
Foulds, J. (2000) The new NHS smoking campaign. British Medical Journal, 320, 454.
Freimuth, V.S. and Taylor, M. (1995) Are Mass Mediated Health Campaigns Effective? A Review of the Empirical Evidence. National Heart, Lung and Blood Institute, Bethesda, MD.
Freimuth, V., Cole, G. and Kirby, D. (2000) Issues in evaluating mass-media health communication campaigns. In Rootman, I., Goodstadt, M., Hyndman, B., McQueen, D., Potvin, L., Springett, J. and Ziglio, E. (eds), Evaluation in Health Promotion: Principles and Perspectives. WHO Regional Publications, European Series 92. WHO, Copenhagen, pp. 475–492.
Friedli, L. (2000) Mental health promotion—re-thinking the evidence base. The Mental Health Review, 5, 15–18.
Friend, K. and Levy, D. (2002) Reductions in smoking prevalence and cigarette consumption associated with mass-media campaigns. Health Education Research, 17, 85–98.
Frith, C., Roberts, C., Kingdon, A. and Tudor-Smith, C. (1997) An evaluation of the 1996 No Smoking Day in Wales. Health Education Journal, 56, 287–295.
Giles-Corti, B. and Donovan, R. (2002) The relative influence of individual, social and physical environment determinants of physical activity. Social Science and Medicine, 54, 1793–1812.
Green, J. and Tones, K. (1999) Towards a secure evidence base for health promotion. Journal of Public Health Medicine, 21, 133–139.
Green, J. (2000) The role of theory in evidence-based health promotion practice. Health Education Research, 15, 125–129.
Grey, A., Owen, L., Bowling, K., Ryan, H. and McVey, D. (2000) A Breath of Fresh Air: Tackling Smoking through the Media. Health Development Agency, London.
Hancock, T. and Labonte, R. (2000) Lessons form Canada in The Evidence of Health Promotion Effectiveness. EU/IUHPE, Vanves.
Hastings, C. and Stead, M. (1996) Using the Media in Drugs Prevention. Drugs Prevention Initiative Paper. Home Office, Central Drug Prevention Unit, London.
Health Education Authority (1997) Mental Health Promotion: A Quality Framework. HEA, London.
Health Education Authority (1998) An Overview of the Effectiveness of Interventions and Programmes aimed at Reducing Unintended Conceptions in Young People. HEA, London.
Health 21 (2002) The Great Central and Eastern European Smokeout: Organisation and Advocacy. Available: http://health21.hungary.globalink.org/varso/; accessed: 3 March 2004.
HM Treasury (2004) Securing Good Health for the Whole Population. HMSO, London.
Hillsdon, M., Cavill, N., Nanchahal, K., Diamond, A. and White, I. (2001) National level promotion of physical activity: results from England's ACTIVE for LIFE campaign. Journal of Epidemiology and Community Health, 55, 755–761.
Hodgson, R., Abbasi, T. and Clarkson, J. (1996) Effective mental health promotion: a literature review. Health Education Journal, 55, 55–74.
Hornick, R. (2002) Public health communication: making sense of contradictory evidence. In Hornick, R. (ed.), Public Health Communication: Evidence for Behaviour Change. Lawrence Erlbaum Associates, London, pp. 1–22.
Hosman, C. (2000) Mental Health in The Evidence of Health Promotion Effectiveness: Shaping Public Health in a New Europe. EU/IUHPE, Vanves.
Kane, R. and Wellings, K. (1999) Reducing the Rate of Teenage Conceptions—An International Review of the Evidence: Data from Europe. Health Education Authority, London.
Kickbusch, I. (1998) Reorienting health acre settings. In Pelikan J., Krajic K. and Lobnig H. (eds), Feasibility, Effectiveness, Quality and Sustainability of Health Promoting Hospital Projects. WHO Collaborating Centre for Health Promotion, Vienna, pp. 31–35.
Klein, R. (1998) The New Politics of the NHS. Longman, London.
Laugerson, M. and Swinburn, B. (2000) New Zealand's tobacco control programme 1985–1998. Tobacco Control, 9, 155–162.
Le Fanu, J. (ed.) (1994) Preventionitis: The Exaggerated Claims of Health Promotion. Social Affairs Unit, Bury St Edmunds.
Levy, D. and Friend, K. (2000) Gauging the effects of mass media policies: what do we need to know? Journal of Public Health Management Practice, 6, 95–106.
Ludbrook, A., Godfrey, C., Wyness, L., Parrott, S., Haw, S., Napper, M. and van Teijlingen, E. (2000) Effective and Cost-Effective Measures to Reduce Alcohol Misuse in Scotland: A Literature Review. The Scottish Executive, Edinburgh.
Lull, J. (1999) Media, Communication, Culture. Polity Press, London.
Lupton, D. (1995) The Imperative of Health Public Health and the Regulated Body. Sage, London.
McIntyre, D. (2001) No Smoking Day 2001; Awareness and Participation Results. NSD/NOP, London.
McNeill, A. (2001) Smoking and Mental Health: A Review of the Literature. ASH, London.
McNeill, A. and Heuston, J. (1999) HEA Drugs Publicity and Information Campaign: Learning from the HEA Research. HEA, London.
McVey, D. and Stapleton, J. (2000) Can anti-smoking television advertising affect smoking behaviour? Controlled trial of the Health Education Authority for England's anti-smoking TV campaign. Tobacco Control, 9, 273–282.
Miles, A., Rapoport, L. Wardle, J., Afuape, T. and Duman, M. (2001) Using the mass media to target obesity: an analysis of the characteristics and reported behaviour change of participants in the BBC's ‘Fighting fat, Fighting Fit’ campaign. Health Education Research, 16, 357–372.
Mudde, A. and De Vries, H. (1999) The reach and effectiveness of a national mass media-led smoking cessation campaign in the Netherlands. American Journal of Public Health, 89, 346–350.
Naidoo, J. (1986) Limits to individualism. In Rodmell S. and Watt A. (eds), The Politics of Health Education: Raising the Issues. RKP, London, pp. 17–37.
NHS Centre for Reviews and Dissemination (1996) Effective Health Care: Preventing Falls and Subsequent Injury in Older People. Nuffield Institute for Health/NHS Centre for Reviews and Dissemination, Leeds.
NHS Centre for Reviews and Dissemination (1997) Effective Health Care: Mental Health Promotion in High Risk Groups. NHS Centre for Reviews and Dissemination, University of York, York.
National No Smoking Day (2002) Awareness and Participation Results 2002. Available: http://www.nosmokingday.org.uk/; accessed: 12 December 2003.
National No Smoking Day (2004) No Smoking Day Facts and Figures. Available: http://www.nosmokingday.org.uk/nsd_facts_figures.htm; accessed: 11 March 2004.
NHS Health Scotland (2003) Health Events Calendar 2003. Available: http://www.hebs.scot.nhs.uk/services/events/Calendar2003.pdf; accessed: 12 December 2003.
Noar, S. (2003) Narrative reviews versus meta-analysis: rapid responses to Riemsma, R., Pattenden, J., Bridle, C., Sowden, A., Mather, L., Watt, I. and Walker, A. Systematic review of the effectiveness of stage based interventions to promote smoking cessation. British Medical Journal, 326, 1175–1177.
Oakley, A. (1998) Experimentation in social science: the case of health promotion. Social Science in Health, 4, 73–89.
Ory, M., Jordon, P. and Bazzare, T. (2002) The Behaviour Change Consortium: setting the stage for a new century of health behaviour-change research. Health Education Research, 17, 500–511.
Owen, N. (1996) Strategic Initiatives to promote participation in physical activity. Health Promotion International, 11, 213–218.
Pechmann, C. and Reibling, E. (2000) Anti-smoking advertising campaigns targeting youth: case studies from USA and Canada. Tobacco Control, 9(Suppl. 2), 18–31.[CrossRef]
Percival, J. and Gough, J. (2001) Do government health promotion campaigns really work? Campaigns to help people give up smoking can be shocking. But are they effective? Nursing Times, 97(4), 16.[Medline]
Perse, E. (2001) Media Effects and Society. Lawrence Erlbaum Associates, London.
Picken, C. and St Leger, A.S. (1993) Health Needs Assessment using the Lifecycle Framework. Open University Press, Milton Keynes.
Pierce, J. and Gilpin, E. (2001) News media coverage of smoking and health is associated with changes in population rates of smoking cessation but not initiation. Tobacco Control, 10, 145–153.
Plant, M. (2004) The alcohol harm reduction strategy for England. British Medical Journal, 328, 905–906.
Plowright, N. (2004) What a drag. The Observer, 7 Mar, 63.
Raphael, D. (2000) The question of evidence in health promotion. Health Promotion International, 15, 355–367.
Raw, M., Anderson, P., Batra, A., Dubois, G., Harrington, P., Hirsch, A., Le Houezec, J., McNeill, A., Milner, D., Poetschke Langer, M. and Zatonski, W. (2002) WHO Europe evidence based recommendations on the treatment of tobacco dependence. Tobacco Control, 11, 44–46.
Reid, D., Killoran, A., McNeill, A. and Chambers, J. (1992) Choosing the most cost effective health promotion options for reducing a nation's smoking prevalence. Tobacco Control, 1, 185–197.
Rogers, E.M. and Storey, J.D. (1987) Communication campaigns. In Berger, C. and Chaffee, S. (eds), Handbook of Communication Science. Sage, London, pp. 817–846.
Sargent, J., Dalton, M. and Beach, M. (2000) Exposure to cigarette promotions and smoking uptake in adolescents: evidence of a dose–response relation. Tobacco Control, 9, 163–168 [HPLS Bull. 313].
Schar, E. and Guitierrez, K. (2001) Smoking Cessation Media Campaigns from Around the World: Recommendations from Lessons Learned. WHO Regional Office for Europe, Copenhagen.
Schmidt, F. and Hunter, J. (2003) Meta-analysis. In Schinka, J. and Velicer, W. (eds), Handbook of Psychology: Research Methods in Psychology. Wiley, New York, vol. 2, pp. 581–606.
Sidell, M. (1997) Educating and communicating through the mass media. In Katz, J and Peberdy, A. (eds), Promoting Health: Knowledge and Practice. Open University Press/MacMillan, London, pp. 152–167.
Smith, B., Ferguson, C., McKenzie, J., Bauman, A. and Vita, P. (2002) Impacts of repeated mass media campaigns to promote sun protection in Australia. Health Promotion International, 17, 51–60.
Smith, R. (2004) Advertising: boon or bother? British Medical Journal, 328, 328.
Snyder, L. and Hamilton, M. (2002) A meta-analysis of US Health Campaign Effects on Behaviour: emphasise enforcement, exposure and new information and beware secular trend. In Hornik, R. (ed.), Public Health Communication: Evidence for Behaviour Change. Lawrence Erlbaum Associates, London, pp. 357–384.
Straus, S. (2004) What's the E for EBM? British Medical Journal 328: 535–536.
Stead, M., MacKintosh, A.M., Eadie, D. and Hastings, C. (2000) NE Choices: The Development of a Multi-component Drug Prevention Programme for Adolescents. Paper 4. Drugs Prevention Advisory Service, Home Office, London.
Stevens, W., Thorogood, M. and Kayikki, S. (2002) Cost-effectiveness of a community anti-smoking campaign targeted at a high risk group in London. Health Promotion International, 17, 43–50.
Sykes, C. and Marks, D. Why a few tips won't go far [recent anti-smoking campaigns ignore the psychological nature of smoking]. Health Matters, 40, 19.
Tannahill, A. (1998) The Scottish Green Paper: beyond a healthy mind in a health body. Journal of Public Health Medicine, 20, 249–252.
Thompson, J. (1995) The Media and Modernity: A Social Theory of the Media. Polity Press, London.
Tilford, S., Delaney, F. and Vogels, M. (1997) Effectiveness of mental health promotion interventions: a review. Health Promotion Effectiveness Reviews 4. HEA, London.
Tones, K. (1997) Beyond the randomised controlled trial: a case for judicial review: Editorial. Health Education Research, 12 (2); i–iii.[ISI][Medline]
Tones, K. and Tilford, S. (2001) Health Education: Effectiveness, Efficiency and Equity. Chapman & Hall, London.
Towner, E. and Dowswell, T. (2002) Community-based childhood injury preventions: what works? Health Promotion International, 17, 273–284.
Townsend, J., Roderick, P. and Cooper, L. (1994) Cigarette smoking by socio-economic group, sex and age: effects of price, income and health publicity. British Medical Journal, 309, 923–927.
Velicer, W. (2003) Why meta-analysis has become the standard for integrative reviews: rapid responses to Riemsma, R., Pattenden, J., Bridle, C., Sowden, A., Mather, L., Watt, I. and Walker, A. Systematic review of the effectiveness of stage based interventions to promote smoking cessation. British Medical Journal, 326, 1175–1177.
Wakefield, M. (1999) ‘Unselling’ the cigarette. British Medical Journal, 319, 524.
Waller, S., Naidoo, B. and Thom, B. (2000) Prevention and Reduction of Alcohol Misuse. HAD, London.
Wakefield, M. and Chaloupka, F. (2000) Effectiveness of comprehensive tobacco control programmes in reducing teenage smoking in the USA. Tobacco Control, 9, 177–186.
Wardle, J., Rapoport, L., Miles, A., Afuape, T. and Duman, M. (2001) Mass education for obesity prevention: the penetration of the BBC's ‘Fighting fat, Fighting Fit’ campaign. Health Education Research, 16, 343–355.
Wellings, K. and Macdowall, W. (2000a) Evaluating mass media approaches to health promotion: a review of methods. Health Education, 100, 23–32.[CrossRef]
Wellings, K. and Macdowall, W. (2000b) ‘Evaluating Mass Media Approaches’. In Thorogood, M. and Coombes, Y. (eds), Evaluating Health Promotion. Practice and Method. Oxford University Press, Oxford, pp. 145–162.
West, R., McNeill, A. and Raw, M. (2000) Smoking cessation guidelines for health professionals: an update. Thorax, 55, 987–999.
Whittle, G., Pitkethly, D. and Wilson, M. (1994) A dental health promotion campaign in a shopping centre. Health Education Research, 9, 261–265.
WHO (2002) Prevention and Promotion in Mental Health. WHO, Geneva.
Williams, G. (1984) Health promotion: caring concern or slick salesmanship? Journal of Medical Ethics, 10, 191–195.[Abstract]
Wimbush, E., MacGregor, A. and Fraser, E. (1998) Impacts of a national mass media campaign on walking in Scotland. Health Promotion International, 13, 45–53.
Young, H. (1988) The leader misses the message. The Guardian, 22 Sep, 23.
Yzer, M., Siero, F. and Buunk, B. (2000) Can public campaigns effectively change psychological determinants of safer sex? An evaluation of three Dutch campaigns. Health Education Research, 15, 339–352.
Ziglio, E. (2000) Repositioning health promotion: research implications. In Watson, J. and Platt, S. (eds), Researching Health Promotion. Routledge, London, pp. 23–37.
Received on May 17, 2004; accepted on August 8, 2004