Health Education Research Advance Access originally published online on August 18, 2005
Health Education Research 2006 21(2):175-180; doi:10.1093/her/cyh051
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Quality of research on community partnerships: developing the evidence base
1 Postgraduate Public Health Programme, School of Health and Social Care, Oxford Brookes University, Jack Straws Lane, Marston Road, Oxford OX3 0FL, UK and
2 Department of Psychiatry, Memorial Sloan-Kettering Cancer Center, New York, NY 10021, USA
*Correspondence to: W. El Ansari. E-mail: walid.ansari{at}brookes.ac.uk
 |
Abstract |
|---|
 Top Abstract IntroductionQuantitative inquiryQualitative inquiryDiscussionConclusionsAcknowledgementsReferences |
|---|
Recent years have witnessed broad public health transformations. Central to these changes is that partnership work is increasingly mandated and employed as a vehicle for health education and promotion and disease prevention. Such efforts involve partnerships that span National Health Service bodies, local authorities and health and social care organizations, with stakeholders from the public, private, voluntary and community sectors. However, a more adequate evidence base is required to guide policy and action. The aims of this paper are to draw attention to some of the issues that have hampered the quality of partnership research and to discuss ways in which this research can be improved, in an effort to encourage the development of a more valuable evidence base for researchers, practitioners and policy makers. Multimethod approaches to research on partnership context, functioning and outcomes are advocated, and policy recommendations are made.
|
Introduction |
|---|
|
TopAbstractIntroductionQuantitative inquiryQualitative inquiryDiscussionConclusionsAcknowledgementsReferences |
|---|
There is an increasing interest and investment in partnerships as a way of addressing challenging public health issues [1–5]. Partnerships are now more often the norm than the exception in health education and disease prevention work. Nonetheless, and despite their popularity and potential, there exists limited evidence of the effectiveness of partnerships in achieving desired outcomes [4–7]. There are a myriad of reasons why this is the case (see review by Roussos and Fawcett [4]); however, it is clear that in order to enhance our capacity to attribute public health changes to partnership work, we must continue to improve our ability to effectively measure the quality of partnership processes and outcomes [6, 8]. Hence, a recent summary of (quantitative) measurement tools to evaluate community coalition characteristics and functioning [9] is a timely and welcome undertaking. However, more attention must be paid to some critical limitations of existing research that constrain the way we think about and build the evidence base for partnership work.
Granner and Sharpe [9] noted that ‘the difficulty in evaluating coalitions for health outcomes lies in the complexity and scope of participatory and social ecological approaches, which often characterize community partnerships for health promotion’ (p. 515). Indeed, as the data presented by Granner and Sharpe [9] show, existing research paradigms and methodologies have not adequately accommodated and sufficiently captured this complexity and scope (see also El Ansari et al. [7]). The aims of this paper are to begin to draw attention to and stimulate discussion about some of the shortcomings of research on community partnerships and to provide some specific direction for how we might strengthen and use quantitative and qualitative approaches in order to provide a more valuable evidence base for partnership researchers, practitioners and policy makers. As both qualitative and quantitative methods have been employed to study partnerships, each will be discussed in turn before drawing conclusions.
|
Quantitative inquiry |
|---|
|
TopAbstractIntroductionQuantitative inquiryQualitative inquiryDiscussionConclusionsAcknowledgementsReferences |
|---|
There are a number of issues related to the level of measurement in partnership research. Granner and Sharpe [9] identified 146 measurement scales for partnership functioning and classified them under three categories: member characteristics and perceptions, organizational or group characteristics and measures for organizational or group processes/climate. Other researchers have employed similar classifications [10]. The titles of these categories suggest two points. The first is the potential for overlap (cross-contamination) between items that supposedly measure different constructs. Independent measurement of the reliability of each scale (as often undertaken) becomes of limited value, unless potential collinearity (excessively high correlation among the measures) has been assessed to confirm that the constructs are distinct.
The second point has to do with the aggregation of data. It is important to pay more attention to the degree to which items within measures that purport to measure group characteristics or climate are asking about partnership-level phenomena rather than individual-level phenomena [8]. The more the items in a scale relate to the partnership rather than to the individual, the more likely it is that an aggregated mean score for the partnership would be representative of the group rather than the individuals within the group.
With respect to reliability of measures, many studies of partnerships employ Cronbach's 
as an indicator of the internal consistency of multi-item scales [9]. However, very few studies assess the stability of the measures over time. Without repeated testing of instruments, it is difficult to know whether changes in partnership functioning over time (e.g. changes in member participation, partnership leadership, decision-making processes and resources) are real or simply measurement errors. This is particularly important when considering the implementation of much-needed ‘longitudinal’ indicators of coalition functioning that capture partnership changes over a period of time and their effects, and whether some markers of coalition functioning are more sensitive and hence appropriate than others over time.
The data presented by Granner and Sharpe further suggest that almost no attention has been paid to validating measures of partnership functioning. This is critical because a measure that is reliable (obtains consistent results) but not valid (does not measure what it purports to measure) is of little value. It is possible to establish some degree of construct validity by assessing the relationship of different scales within the same questionnaire. The key to demonstrating construct validity is to show that a scale correlates highly with other variables with which it should theoretically correlate, but also that it does not correlate significantly with variables from which it should differ [11].
Another concern is respondent bias (sociability and social desirability), where respondents tick the ‘socially desirable’ response option, with a desire to give what are perceived to be the ‘right’ answers as opposed to the ‘real’ answers [12]. Although these effects may occur in partnership evaluation research where participants want to paint a positive picture, surveys usually fail to specifically measure social desirability effects, with potentially dangerous consequences. The work of El Ansari [13] is a notable exception to this.
A final point with respect to quantitative inquiry is that even if randomized controlled trials were feasible methods for assessing the effectiveness of partnership work, they would not help us understand the ‘chemistry’ of any measured effectiveness (i.e. they would not reveal why the partnership works). Understanding the mix of factors that explains partnership success is critical to the transfer of this mix to other contexts [8]. Thus, it is essential to use a range of functioning measures to capture group characteristics and process variables (e.g. leadership, communication, relationships, collaboration, decision-making and conflict resolution), as well as a variety of impact markers (e.g. short- and long-term effects, individual-level and collective community-level outcomes) [4, 14]. Also, while measuring functioning and impact over time may be ideal, longitudinal designs are not always practical in community settings given limitations in time, funds and rewards of academics for collaborating in community work. Given that longitudinal designs may not be feasible, it is important to recognize the recent evidence suggesting that cross-sectional studies provide valid results [15] and play a critical role in providing evidence for practice [16].
|
Qualitative inquiry |
|---|
|
TopAbstractIntroductionQuantitative inquiryQualitative inquiryDiscussionConclusionsAcknowledgementsReferences |
|---|
The full potential of qualitative research has yet to be realized in the health field [17], especially in partnership work, since it is challenging for studies to explore simultaneously and with precision a wide range individual and contextual variables of a partnership. Qualitative research is valued for its relevance [18] and its richness. It can assist in understanding the meaning of an intervention, participants' beliefs about and expectations of the outcome and the impact of the context and the process of the intervention [19]. Nonetheless, there have been calls for greater use to be made of quality standards for assessing the results in qualitative health studies [20].
In terms of the validity of qualitative research on community partnerships, a first set of validity criteria could include credibility, authenticity, criticality and integrity [21] (see also Bailey [22]). Within partnership research, these criteria translate into the extent to which the inquiry process and conclusions drawn respect and capture the subtle differences in the voices of the partnership participants. The criteria also translate into the extent to which coding of emergent themes by multiple researchers is undertaken and whether the research employs a recursive process with repetitive checks of validity by asking partnership members to help refine and verify interpretations. Such attention to lay knowledge is a critical aspect of qualitative research [20, 23]. In fact, a useful indicator of the credibility of the findings produced is when partnership members and practitioners themselves regard them as meaningful and applicable in terms of their experience [24]. Moreover, these validity ‘checks’ can help researchers avoid the contamination of constructs that has plagued quantitative research on partnerships.
Other validity criteria embrace explicitness, vividness, thoroughness, congruence and sensitivity [21]. Within partnership research, this set of criteria could address the extent to which thick description has faithfully portrayed the contextual setting of the partnerships. Use of such description techniques is highly valuable because it allows other partnership practitioners to judge the generalizability and appropriateness of findings to their own context. This set of criteria also addresses the degree to which the findings of the research convincingly address the questions that were originally posed, as well as the ‘goodness of fit’ of the emerging themes (i.e. the congruency of the various products to each other). To illustrate this last point, we could ask: to what extent are the respondents' accounts of the decision-making processes in their partnership congruent with their views on the quality of interaction between the partners and their tolerance to each other's differences in their meetings?
Other markers of good qualitative research include auditability [25] and trustworthiness [22]. In the same way that it is important for partnership processes and dealings to be visible to the partnership's constituencies and membership, researchers must create a transparent audit trail for the partnerships they are working with (i.e. they must include in reports the raw data, instrument development information, clear descriptions of the data analysis and reduction steps and process notes) [26].
Attempts to produce consensus on criteria for naturalistic inquiry have proven difficult [27]. Hence, a plurality of standards might be desirable [20], advocating a shift to a position of a more pluralistic approach [28]. Similarly, although several types of metasynthesis have been conducted, resulting in theory building and substantive descriptions of phenomena, no firm metasynthesis guidelines exist [29] for qualitative inquiries to the extent that they do in quantitative research.
|
Discussion |
|---|
|
TopAbstractIntroductionQuantitative inquiryQualitative inquiryDiscussionConclusionsAcknowledgementsReferences |
|---|
It is likely that partnership work will be a critical mainstay of health policy for the coming years. For instance, in the United Kingdom, the new Public Health White Paper Choosing Health highlighted that the systems for local delivery will comprise ‘effective partnerships’ of different organizations working together. ‘We want to see an effective system for health delivered through close alignment between local community partners’ [30] (p. 199).
To truly understand partnership phenomena, we need to add more rigor to our traditional positivist research, and also recognize that this paradigm alone might be inadequate. We need to combine this traditional research with information gathered through rigorous interpretive approaches that provide in-depth and contextual understanding. In order to carry out such a mandate effectively, we must refine and improve the robustness of quantitative and qualitative methodologies used in conducting partnership research.
First, we must develop better tools. Partnership researchers will need to systematically address the range of methodological issues that contribute to the quality of partnership research as highlighted above for both quantitative and qualitative research. This is important for the provision of a valuable evidence base for practitioners and policy makers.
Second, ideally a simultaneous multilevel multimethod (quantitative and qualitative) approach to research on partnerships is optimal, thus drawing on differing frameworks and seeking to embrace the perspectives of all stakeholders and the complexity of the phenomena under study. Qualitative approaches contribute to quantitative work by providing insights into data construction, identifying relevant variables, developing appropriate outcome measures, furnishing explanations and generating hypotheses. Conversely, the credibility of qualitative research findings could be established by testing out the emergent theory by means of conducting a deductive quantitative study [24]. Similarly, qualitative work is enhanced by using quantitative techniques in a modified form in sampling strategies and analyzing data [31].
|
Conclusions |
|---|
|
TopAbstractIntroductionQuantitative inquiryQualitative inquiryDiscussionConclusionsAcknowledgementsReferences |
|---|
All research depends on collecting specific kinds of evidence through the prism of particular methods, each of which has its strengths and weaknesses [32] (p. 109). With this in mind, we urge researchers to think ‘outside the box’ and focus their attention on the bigger picture while simultaneously refining their instruments—piecing together the jigsaw with various sometimes interlacing threads or opposing shreds of evidence of different weightings [33]. We also urge researchers to support and encourage qualitative work by publishing more of it in peer-reviewed journals and by synthesizing findings of multiple studies so that the data and methodology become more accessible to practitioners and policy makers.
Further, we advocate that public health agencies acknowledge the utility of qualitative evidence, either alone or especially in conjunction with quantitative results, and foster its development. Some federal agencies in the United States now include a qualitative research inventory on their websites. Nonetheless, one agency in particular provides the following caveat: ‘There are no quality criteria for including research in the inventory ... the [agency] does not necessarily endorse these studies or their results’ [34].
Finally, we suggest that funders (private and public) review both quantitative and qualitative proposals for partnership research equitably, with reasonable criteria relevant to each [35]. Also, we suggest that funders, including research councils, encourage proposals that incorporate both qualitative and quantitative methodologies as appropriate to the research questions at hand.
Unquestionably, partnerships hold tremendous potential; developing the evidence base that is needed to guide policy and practice will depend on holding quantitative partnership research to higher standards, recognizing the components and the value of rigorous qualitative research and seeing the benefits of combining methods in new ways.
|
Footnotes |
|---|
At the time of writing of the manuscript, E. S. Weiss was at the Center for the Advancement of Collaborative Strategies in Health, Division of Public Health, The New York Academy of Medicine, New York, NY, USA 
|
Acknowledgements |
|---|
|
TopAbstractIntroductionQuantitative inquiryQualitative inquiryDiscussionConclusionsAcknowledgementsReferences |
|---|
The authors would like to thank Prof. Ceri Phillips and anonymous reviewers for constructive comments on earlier drafts of the paper.
|
References |
|---|
|
TopAbstractIntroductionQuantitative inquiryQualitative inquiryDiscussionConclusionsAcknowledgementsReferences |
|---|
1. El Ansari W. Collaborative research partnerships with disadvantaged communities: challenges and potential solutions. Public Health 2005 119:758–770.[CrossRef][ISI][Medline]
2. Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health 1998 19:173–202.[CrossRef][ISI][Medline]
3. Lasker RD and Weiss ES. Broadening participation in community problem solving: a multidisciplinary model to support collaborative practice and research. J Urban Health 2003 80:14–42.[ISI][Medline]
4. Roussos ST and Fawcett SB. A review of collaborative partnerships as a strategy for improving community health. Annu Rev Public Health 2000 21:369–402.[CrossRef][ISI][Medline]
5. Shortell SM, Zukoski AP, Alexander JA, et al. Evaluating partnerships for community health improvement: tracking the footprints. J Health Polit Policy Law 2002 27:49–91.[Abstract]
6. Krueter MW, Lezin NA, Young LA. Evaluating community-based collaborative mechanisms: implications for practitioners. Health Promot Pract 2000 1:49–63.[Medline]
7. El Ansari W, Phillips CJ, Hammick M. Collaboration and partnerships: developing the evidence base. Health Soc Care Community 2001 9:215–27.[CrossRef][ISI][Medline]
8. Weiss ES, Anderson RM, Lasker RD. Making the most of collaboration: exploring the relationship between partnership synergy and partnership functioning. Health Educ Behav 2002 29:683–98.
9. Granner ML and Sharpe PA. Evaluating community coalition characteristics and functioning: a summary of measurement tools. Health Educ Res 2004 19:514–32.
10. El Ansari W and Phillips CJ. Empowering health care workers in Africa: partnerships in health—beyond the rhetoric towards a model. Crit Public Health 2001 11:231–52.[CrossRef]
11. Anastasi A. Psychological Testing. New York: Macmillan Publishing Co., Inc. 1982.
12. Halliday J, Asthana SNM, Richardson S. Evaluating partnership: the role of formal assessment tools. Evaluation 2004 10:285–303.[Abstract]
13. El Ansari W. A study of the characteristics, participant perceptions and predictors of effectiveness in community partnerships in health personnel education: the case of South Africa. Unpublished PhD Thesis. Newport, UK: University of Wales College 1999.
14. El Ansari W. Educational partnerships for health: do stakeholders perceive similar outcomes? J Public Health Manag Pract 2003 9:136–56.[Medline]
15. Benson K and Hartz AJ. A comparison of observational studies and randomised controlled trials. N Engl J Med 2000 342:1878–86.
16. Concato J, Shah N, Horwitz RI. Randomised controlled trials, observational studies and the hierarchy of research designs. N Engl J Med 2000 342:1887–92.
17. Boulton M, Fitzpatrick R, Swinburn C. Qualitative research in health care: II. A structured review and evaluation of studies. J Eval Clin Pract 1996 2:171–9.[Medline]
18. Johansson EE, Risberg G, Hamberg K. Is qualitative research scientific, or merely relevant? Research-interested primary care and hospital physicians' appraisal of abstracts. Scand J Prim Health Care 2003 21:10–4.[CrossRef][ISI][Medline]
19. Verhoef MJ, Casebeer AL, Hilsden RJ. Assessing efficacy of complementary medicine: adding qualitative research methods to the "gold standard". J Altern Complement Med 2002 8:275–81.[CrossRef][ISI][Medline]
20. Popay J, Rogers A, Williams G. Rationale and standards for the systematic review of qualitative literature in health services research. Qual Health Res 1998 8:341–51.
21. Whittemore R, Chase SK, Mandle CL. Validity in qualitative research. Qual Health Res 2001 11:522–37.
22. Bailey PH. Assuring quality in narrative analysis. West J Nurs Res 1996 18:186–94.
23. El Ansari W, Phillips CJ, Zwi AB. Narrowing the gap between academic professional wisdom and community lay knowledge: partnerships in South Africa. Public Health 2002 116:151–9.[ISI][Medline]
24. Cutcliffe JR and McKenna HP. Establishing the credibility of qualitative research findings: the plot thickens. J Adv Nurs 1999 30:374–80.[CrossRef][ISI][Medline]
25. Beck CT. Qualitative research: the evaluation of its credibility, fittingness, and auditability. West J Nurs Res 1993 15:263–6.
26. Lincoln YS and Guba EG. Naturalistic Inquiry. Newbury Park, CA: Sage 1985.
27. Dixon-Woods M, Shaw RL, Agarwal S, Smith JA. The problem of appraising qualitative research. Qual Saf Health Care 2004 13:223–5.
28. Tobin GA and Begley CM. Methodological rigour within a qualitative framework. J Adv Nurs 2004 48:388–96.[CrossRef][ISI][Medline]
29. Finfgeld DL. Metasynthesis: the state of the art—so far. Qual Health Res 2003 13:893–904.[Abstract]
30. Department of Health. Choosing Health. Making Healthy Choices Easier. Annex B—Making It Happen. Available at: http://www.dh.gov.uk/assetRoot/04/09/48/71/04094871.pdf. Accessed: 10 January 2004.
31. Barbour RS. The case for combining qualitative and quantitative approaches in health services research. J Health Serv Res Policy 1999 4:39–43.[Medline]
32. Mays N and Pope C. Qualitative research: rigour and qualitative research. Br Med J 1995 311:109–12.
33. El Ansari W. The tests, the tools, the methods, certainty, and thinking ‘outside the box’. Physiotherapy 2003 89:320–2.
34. National Center for Chronic Disease Prevention and Health Promotion. Nutrition and Physical Activity. Available at: http://www.cdc.gov/nccdphp/dnpa/qualitative_research/faq.htm. Accessed: 17 December 2004.
35. Morse JM. A review committee's guide for evaluating qualitative proposals. Qual Health Res 2003 13:833–51.[Abstract]
Received on January 13, 2005; accepted on July 12, 2005
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||