USA: +1(669)289-8683 
Sign in
Discussion Instructions:
Read Chapter 5 in Program Evaluation and Performance Measurement. Pay close attention to question 7 on page 221.
Read Moore and Tananis’s 2009 article, “Measuring Change in a Short-Term Educational Program Using a Retrospective Pretest Design,” from American Journal of Evaluation, volume 30, issue 2, pages 189–202.
* Pay attention to the research design and data collection methods in this study.
* Discuss how you would design a qualitative evaluation for improving the quality of mothers’ parenting.
*Include three different methods of finding your sample. Specifically discuss how you would collect the data.
*Describe how you would measure whether the quality of parenting actually improves.
Make your post to this discussion step-by-step instructions that someone else could use to repeat the study.
1. Understand and analyze qualitative program evaluation design.
2. Compare and contrast experimental and quasi-experimental designs.
3. Analyze pretest-posttest designs.
4. Communicate through writing that is concise, balanced, and logically organized.
Unit 3 – Program Evaluation: Qualitative Research Design
INTRODUCTION
This unit focuses on qualitative evaluation design, data collection methods, and evaluating program
effectiveness. Additionally, you will apply this knowledge to a real-world program evaluation.
OBJECTIVES
To successfully complete this learning unit, you will be expected to:
U03S1] Studies – Readings (Complete the following):
• Read Chapter 5 – Program Evaluation and Performance Measurement text
o Pay attention to question 7 on page 221. The content this question addresses will be
relevant for the first discussion in this unit.
U03D1] Unit 3 Discussion 1 – Qualitative Evaluation, Data, and Measures
Read Chapter 5 in Program Evaluation and Performance Measurement . Pay close attention to question 7 on page 221.
Complete the following:
• Discuss how you would design a qualitative evaluation for improving the quality of mothers’ parenting.
o Include three different methods of finding your sample.
• Specifically discuss how you would collect the data.
• Describe how you would measure whether the quality of parenting actually improves.
The three qualitative evaluation methods identified by McDavid & Huse (2013)
The three qualitative evaluation methods identified by McDavid & Huse (2013) were qualitative program evaluation, inductive approach, and holistic approach.
According to the article, Qualitative Program Evaluation Methods indicated that “qualitative methods are commonly used in evaluations in order to explore specific facets of programs and give voice to participants”. The qualitative evaluation methods can provide in-depth information that can assist in enhancing the quality of a specific program (Qualitative Program Evaluation Methods).
According to McDavid & Huse (2013) defined inductive approach, as a method that “starts with ‘the data’ namely narratives, direct and indirect (unobtrusive) observations, interactions between stakeholders and the evaluator, documentary evidence, and other sources of information, and then constructs an understanding of the program” p200.
The Holistic Approach defined by McDavid & Huse (2003) is “discovering the themes in the data, weighting them, verifying them with stakeholders, and finally, preparing a document that reports the findings and conclusions” p200.
In the article, “Measuring Change in a Short-Term Educational Program Using a Retrospective Pretest Design”, centered around the Pennsylvania Governor’s School International Studies having a summer program that gives gifted high school students a challenging introduction to the study of internal affairs. The program administrators wanted to understand the effect of the program on the student’s understanding of their knowledge surrounding the core issues (Moore & Tananis 2009). For the past four years the program administrators began using the retrospective pre-post design. The results from the pretest yielded that the students consistently overestimated their understanding of International Studies in the beginning. The program administrator found that using the “retrospective pretest items were more biases than pretest items in certain context” (Moore & Tannis 2009 p200). The authors concluded that the retrospective pretest perhaps could be the most effective way of capturing data that is accurate of pre intervention function than a pretest given before the program begins.
For this article, this learner would use the inductive approach for this article. This approach will allow the researcher and the program administrators to evaluate the program effectiveness through the data that was collected through the pretest, test that were administered throughout the course, and other data. This approach will allow for interviewing the participants throughout the five week program to determine if the students are learning and meeting the intended goal of the International Studies program. The program administrators can also observe the interactions between the participants.
For this particular study, it would be important to test the students prior to taking the course, and then conduct weekly testing, and a final to determine the comprehension of the material. There would be periodic interviews and/or questionnaires to establish how the students feel, how the material is being presented, and feelings towards varying factors regarding the material. Since this is a five week course, conducting initial interviews, mid-course interviews, and an exit interview will allow the program administrators to get real time information in conjunction with the weekly testing. All of the information would be gather to compare testing to the interviews and questionnaires to have a more solid information regarding the program effectiveness. With this data collected over a period of years, the program administrators would have a more solid understanding if the program is beneficial.
Reference:
McDavid, J. C., Huse, I., & Hawthorn, L. R. (2013). Program evaluation and performance measurement: An introduction to practice (2nded.). Thousand Oaks, CA: Sage Publications. ISBN: 9781412978316.
Moore, D., & Tananis, C. A. (2009). Measuring Change in a Short-Term Educational Program Using a Retrospective Pretest Design. American Journal of Evaluation,30(2), 189-202. doi:10.1177/1098214009334506
Reference:
McDavid, J. C., Huse, I., & Hawthorn, L. R. (2013). Program evaluation and performance measurement: An introduction to practice (2nded.). Thousand Oaks, CA: Sage Publications. ISBN: 9781412978316.
Retrieved on January 8, 2018,
https://projects.ncsu.edu/ffci/publications/2011/v16-n1-2011-spring/vaterlaus-higginbotham.php
Article Qualitative Program Evaluation Methods.
Framework for Program Development and Evaluation
Reference: Comeau, J. (2011). Framework for program development and evaluation.Unpublished, Capella University, Minneapolis, MN.
L i c e n s e d u n d e r a C r e a t i v e C o m m o n s A t t r i b u t i o n 3 . 0 L i c e n s e .
I
2. Hacker K, Collins J, Gross-Young L, Almeida S, Burke N. Coping with youth suicide and overdose: one community’s efforts to
investigate, intervene, and prevent suicide contagion. Crisis. 2008;29(2):86–95.
3. U.S. Census Bureau. 2006–2008 American Community Survey 3-Year Estimates. 2008 [cited 2010 October 3]; Available from
http://factfinder2.census.gov.
4. Chomitz V, Arsenault L, Banks C, et al. H.E.L.P. Culminating Report. Cambridge, MA: Institute for Community Health; 2011.
5. Clinical and Translational Science Awards Consortium. Community Engagement Key Function Committee Task Force on the Principles of
Community Engagement. Principles of Community Engagement. 2nd ed. Rockville, MD: NIH; 2011.
6. Israel BA, Parker EA, Rowe Z, et al. Community-based participatory research: lessons learned from the Centers for Children’s
Environmental Health and Disease Prevention Research. Environmental Health Perspectives. 2005 Oct; 113(10):1463–71.
7. Minkler M, Wallerstein N., eds. Community-Based Participatory Research for Health. San Francisco, CA: Jossey-Bass; 2003.
5
Translating Research Into Practice
View From Community
“Sustainability is building capacity and helping the community use their own data to help them continue work after the intervention has
finished.” (Community partner-conference participant)1
n this chapter, we will explore key questions from the community partner perspective about engaging in CBPR and
the value of this research to the community. This will include an overview of how a community partner should assess
its capacity for participation in CBPR. We will also explore the challenges inherent to implementing change based on
findings from CBPR. This will also include data-driven decision making and issues relevant to incorporation of
evidence into practice. My community partners were instrumental in supporting the writing of this chapter. Ms. Milagro
Grullon reviewed the chapter, and both she and Mr. Alex Pirie provided quotes. In addition, quotes from community
partners at a recent CBPR conference on building community capacity were also used.1 The coalitions that I worked
with in the given examples approved the use of their stories.
The following fictional example helps to illustrate a typical community response to a researcher request.
I am the executive director of a small organization dedicated to serving the Latino community in Community Y. We are currently fairly
small, with only two employees. All of us are immigrants and speak fluent Spanish. The mission of our organization is to provide help
with access to health care, city services, and other programs for the Latino immigrant community. A researcher from the local university
has contacted us and wants to get our help recruiting Latinos for a study on mental health issues. In the past, we had a bad experience with
a researcher. He asked for our help recruiting our constituents for a study on high blood pressure. We spent more than a month helping
with the recruitment, and while we did get paid for our efforts, our small staff could not do this work and keep up with the work of the
organization. Things were very tense. Then, when he completed the research, we never heard back from him about the results until we read
them in the newspaper. “Latino males who are overweight are at high risk for hypertension.” We never had a chance to participate in the
paper or get the results so we could work with the community to improve the situation. It was a surprise to us and to the people who had
agreed to participate. Overall, it felt like we got used, and we are not sure we want to do this again.
In the world of community research, this is not an uncommon situation. What went wrong? There was money shared,
and engagement seems to have taken place, but things really seem to have fallen down in the analysis/interpretation and
dissemination phase of the research. In this case, it is hard to tell whether the researcher was interested in a full CBPR
relationship, but the impact of this research has ultimately increased the divide between the community and researchers,
and, most importantly, while it may have contributed to the literature, it has provided little benefit to the local
community.
Many community partners may have negative impressions of research influenced by prior historical events such as
the Tuskegee experiment, especially in communities of color. They may have had negative experiences with past
research projects themselves and feel distrustful of new academic relationships. Additionally, they may be unfamiliar
with how research works or what their role in this work might entail.
“Communities have no motivation because these research projects go on yet there is no investment in the community after. Many
community members are not interested in being in a registry for research.” (Community partner-conference participant)1
“There is this mistrust of what the aims are for research and what this information will be used for.” (Community partner-conference
participant)1
Recognizing that these perspectives may linger long past any one individual research project is an important part of
the engagement process. A researcher needs to be aware of the history of research in the community. Community
partners need to be empowered to ask the hard questions of researchers so as not to repeat the same mistakes. They need
to understand the benefits and pitfalls of engaging in research. Gaining this insight is part of the engagement process
and is necessary for the development of a strong CBPR partnership. As a community begins to understand research and
its potential power, community members may initiate CBPR projects themselves, recognizing that they need data to
catalyze social change in policy and practice.
In our previous Everett example, Everett, one member of an immigrant advocacy group acknowledged the potential
benefit of engaging in research. As a member of a community advocacy group dedicated to working with Latinos in
Everett, he was very concerned about how recent immigration enforcement activity was affecting the Latino
community. Immigrant activists had many stories of how immigrants were being stopped and asked for their driver’s
licenses, and then, if they did not have a license, they were arrested. Many immigrants were in fear of their safety,
suspecting that this action would result in deportation. The immigrant advocacy community wanted to get more
information on this situation to convince the police to stop arresting people for not having a valid driver’s license. They
had made a connection with a local researcher that they already knew and trusted when a grant opportunity arose. They
asked her to help them get the data needed to actively change the situation. In this example, the Everett community
partners recognized that data collected through “research” might help them advocate for action. This
academic/community partnership was built on an existing trusting relationship.2 While not all CBPR partnerships will
be initiated by the community and all questions will not be solely originated in the community, using a CBPR approach
resets the researcher’s agenda such that the topic of study resonates with community priorities and is seen as beneficial
to the community. Community members should expect that their priorities will be considered and that their perspectives
will be valued in the research agenda and subsequent process.
ASSESSING COMMUNITY RESEARCH READINESS
“How do you justify this money being spent on research? Communities have real issues that money could help solve and yet this money is
being given to research—how do you prove the benefit and then share the money so that it benefits the community (immediately)?”
(Community member-conference participant)1
As noted previously (Chapter 2), it is important to assess the impact that engaging in research will have on the
community. The areas outlined below are important for community members to address prior to engaging in a CBPR
project.
ASSESSING THE PARTNERSHIP
The first area that community partners will need to assess is the nature of the CBPR partnership itself, the values, goals,
and priorities.3 What type of partnership is planned? How much power will the community members have in decision
making? Is the researcher intimidating? Is he or she skilled in participatory techniques? Does she or he speak in
layman’s terms and understand the community perspective? Is there a history of trust and respect?4 Is the researcher’s
agenda shared by the community?
“Today, decisions are made by prioritizing and based upon the needs in the community. From a program perspective we need research to
help implement the correct programs in our community. Prioritizing according to need can yield rich data to support programs and
interventions. Again, what makes a project appealing is the need and if the community has identified what is being presented as a priority
for them.” Milagro Grullon, Lawrence, MA
In order to learn more about the researchers, their goals, background, and expectations of a partnership, a face-to-face
meeting should be held, and community members should have a chance to ask these questions and determine if the
CBPR project is right for them.
Questions
outlined in Table 5.1 are a good place to start. In addition, the city of
Lawrence, Massachusetts, developed a “tools for research partnerships” document that can help community partners
build their formal agreements with researchers.5 As one partner noted related to the choice of an academic partner:
“Is this a person you can spend the next year or four years working with closely? Is she or he someone you can comfortably disagree with?
Are they sympathetic to the complexities and nuances of communities and willing to step outside of their traditional bubbled and siloed
turf to connect with the community?” Alex Pirie, Somerville, MA
Table 5.1 Questions for Community Partners
Questions for community partners to ask researcher prior to engaging in CBPR
1. What kind of partnership does the researcher have in mind? Is it really to be participatory?
2. How will decisions get made?
3. What are the research aims?
4. Who is the target population of interest?
5. How will the research be funded?
6. What will be our organization’s and/or my role in the project?
7. Will the time be compensated?
8. Who will own the data? What will happen to the data in the future after the project is completed?
9. What benefits will the project leave behind in the community (skills, programming, policy, infrastructure,
capacity building)?
10. What is the dissemination plan for this research?
11. Does the researcher have the skills and experience to collaborate?
12. Does he or she understand the complexity of the CBPR approach?
13. Does the researcher possess cultural humility?
Questions for community partners to ask themselves prior to engaging in CBPR
1. Does this study address an important problem relevant to my community and my constituents?
2. How does the research aim fit with the mission of my organization?
3. Do we have the capacity to participate? Space? Staff? Time?
4. What are our conflicting priorities?
5. What will be the impact of doing research on my organization’s ability to get its core work accomplished?
6. Will the results lead to action that will help my community?
TIME, RESOURCES, AND CAPACITY
As part of the development of a partnership pre–CBPR, community members should ask specific questions in three
critical areas to assess their own readiness for research: time, money, and capacity. That is, do they have the time to
participate without sacrificing their other responsibilities, are they going to get appropriate financial resources from the
researcher to support their work, and do they have the organizational capacity to participate in the research project?
Each of these issues can create problems if resources are inadequate for the project. For example, if the organization
and the staff do not have time available to participate in the CAB meetings or to actually conduct their roles in the
research, they will be unable to fulfill their responsibilities and the research will fail. As one community member said:
“The single largest obstacle to collaborations between academic and community partners is time. The relative time frames of academic and
community partners can create huge problems and it is essential that whichever of the two sides of the partnership is more experienced do
everything it can to educate the other on the discontinuity that exists.” Alex Pirie, Somerville, MA
In many organizations, particularly small ones, it is not feasible to just hire additional people to take on the new work
of a research project. So who will do the work? What are the implications for the organization? In addition, there is time
required to participate in CBPR partnership meetings and time required for the development of proposals and
dissemination products. Finally, sometimes, it can take several years before a research grant is funded. Any community
partner must consider where it will be in the future and whether it is able to plan that far in advance.
Where the money is concerned, budgets should reflect an equitable process between community and academic
partners. Community partners have not always been included in these decisions.
“When I first started working with academic institutions (12 years ago) I had no involvement with budgets. I was given an amount for
participating as a [representative of the] community without being asked how many staff would be involved or the hours we would spend
on the project. We realized shortly after the grant started that the demands on the staff and community were greater than the funds could
sustain.” Milagro Grullon, Lawrence, MA
As one prepares a grant proposal, both community and academic participants should assess whether there is adequate
support for a CBPR project. Financial support is needed for items ranging from community coinvestigators, liaisons,
space, meetings, and advisors to travel and dissemination. It should reflect the division of labor between academics and
the community.
“Depending on the involvement and time spent in the research, resources should be divided accordingly. The community is asked to
participate and do a lot of work (meetings, gather data, interviews, invite community members, go on the radio, etc. etc.). Without the
community, some of the research could not take place. Researchers need the community, and the community needs to get paid justly.”
Milagro Grullon, Lawrence, MA
Table 5.2 shows an example of a CBPR budget for a project that included focus groups and surveys. The dollars for
various items (stipends, food, etc.) can either sit with the academic or the community partner, depending in large part on
their ability to distribute the money. In some cases, community organizations may see a CBPR project as a way of
accessing dollars for needed infrastructure and programming. However, the research dollars are not flexible and cannot
be spent on nonresearch-related items. Regardless of how the ultimate budget looks, it should be negotiated up front,
and any further reductions that might be required if the grant is funded should also be discussed with the partnership
prior to starting the project.
Ms. Grullon recalled:
“The second time around, when an institution came to me for help or to develop a (pseudo) partnership, I started making clear demands. I
needed to see the budget and be included just as any researcher would be listed on the budget. I worked on a budget for the organization
and the community members that would be involved.” Milagro Grullon, Lawrence, MA
Table 5.2 Proposed CBPR Project Budget
Source: Reproduced by permission from Institute for Community Health, Inc. Cambridge, MA 2011 www.icommunityhealth.org
While differences in salary ranges can become a source of conflict, a bigger issue for many community partners is the
high rates of indirect costs that most academic institutions claim. This can be a source of great tension in CBPR.
“Indirect” rates are related to costs that include physical plant, research administration, and so forth, and they are
negotiated with the funding institution. As one community member said:
“When people congratulate us and say, ‘wow, you got a million bucks to study X, Y, or Z!,’ I say, well, no, we got about 45% of that, the
rest goes to mow lawns and trim ivy at the University of Wherever.” Alex Pirie, Somerville, MA
In general, these dollars do not flow directly to the principal investigator but, rather, are used by the institution to
support operations. While some community agencies have successfully negotiated an indirect rate with funders, this is
not always the case. Thus, how they support their operational needs (rent, secretarial support, utilities, and payroll)
comes into question. A community organization needs to discuss strategies for covering these costs with the researcher
either as an indirect rate, if allowed, or by putting these items into direct costs. Without these transparent negotiations,
the budget can become a major source of conflict.
The capacity of the community partner may include things like current staffing, financial health (cash flow, total
budget, grant management skills), and physical space. For example, can the organization accommodate the research
given its personnel issues (turnover rates, skill sets, etc.)? Are there procedures in place to deal with contracts? Is there
enough cash flow to conduct the work and then invoice for work done? How will the research impact the day-to-day
operations of the organization? Any community partner or member of a community organization should consider these
issues prior to agreeing to participate in a CBPR project.
It is also important to remember that research is relatively inflexible compared to program implementation. This
“rigidness” is needed to answer the research questions effectively. If the intervention keeps changing or the staff leaves
and new staff must be retrained, it poses a problem for the research. Data need to be collected on every participant in a
consistent manner. Records about why participants did not participate as well as how many did or did not participate
must be maintained. If a survey is done but no one knows what the total number of potential participants in the target
group was, it will be impossible to calculate the response rate. While these may be seemingly simple steps, in practice
they can be a major challenge. In the Everett project, surveys of community doctors were done, but the process of
figuring out exactly how many community doctors were serving Everett populations turned out to be much more
complicated than expected. Community partners wanted to send the survey out to as many providers as possible to
generate a high number of responses, but researchers first needed to figure out how many providers were serving the
Everett community. In other words, they needed to know the denominator so that they could understand whether the
final sample was representative of the total group of providers. A similar situation happened when community partners
wanted to survey immigrants. They held several forums and passed out surveys but were unable to keep track of who
refused to fill out the survey or even how many potential participants were at the forum. Researchers found themselves
counting attendees and trying to keep track of who got surveys, who filled them out, and who threw them in the waste-
basket. In the end, they were unable to determine what percentage of attendees actually did fill out the surveys.
Sampling concepts may be foreign to community groups, who may have little experience with obtaining responses in
a systematic way. The tension between action and rigor may create conflict with the organization’s culture and capacity.
After all, communities do not exist in a laboratory bubble. Unexpected events transpire that may have a deep impact on
a research project, ranging from turnover in local leadership to new and emerging social issues. Service demand may
interfere with scientific methodology. While not all research will require the same level of rigor, adherence to a protocol
really matters. For community partners, this may present a very different approach to conducting business. The research
will need to get done on schedule to achieve the aims set out in the proposal. CBPR is often a balancing act between the
researcher’s need for control to achieve scientifically valid results and the community’s desire for action. Successful
CBPR partnerships are able to navigate these challenges via mindful communication and planning.
DATA COLLECTION, OWNERSHIP, AND PROTECTION
“Data is not a four-letter word. Advancing the field and advancing the science [will only happen] because we have done good research and
then it will help.” (Community partner-conference participant)1
Data and their collection, security, use, and ownership is an area that requires some forethought in CBPR. Given that
CBPR represents a partnership for both community and academics, the data are the property of both. How they are
handled, analyzed, and disseminated is therefore important for both. As noted in previous chapters, some of these issues
can be sorted out and recorded in a memorandum of agreement. In our Somerville CBPR youth suicide example, the
researchers were helpful in translating real-time data into usable information for community partners. They provided the
analytic capacity to examine the youth risk behavior data and identified additional data sources that were useful in
monitoring suicide and overdose activity. These included death certificate data, 911 call data from the fire department,
and hospital data on use of the emergency room related to suicide intention and overdose. They were able to provide
these enhanced data to the community for real-time decision making. In addition, they were able to map data from the
911 call information onto maps of the community. This helped provide a contextual framework that allowed community
members to focus their attention on specific areas. The data were generally “deidentified,” meaning that information
that would identify a person was removed before reporting to the CAB. No one would be able to identify the
individuals, thus maintaining their privacy. The concept of data privacy is an important one in CBPR. Working with
community members who may know each other could put privacy at risk. In CBPR, it is particularly important to work
through these privacy issues prior to sharing data. In Somerville, the CAB agreed to a level of privacy that would not
allow reporting when numbers were small (< 5).6
The Somerville project is an example of using data for decision making in real time. While community members
bring a wealth of tacit knowledge to the table, they are not always equipped to evaluate their efforts. Thus, the ability to
use and analyze data in a more rigorous fashion may help to promote improvement. In CBPR, data can serve both as
means to a research project end and as quality-improvement information for community partners. As noted, the
translation of evidence to everyday practical solutions is a critical component of the CBPR process. The CBPR
researcher is, in effect, a purveyor of data, design, and methods that will facilitate this translation. Key to this translation
are the skills to communicate and the desire to utilize. The CBPR research process resembles the Plan Do Study Act
cycle in quality improvement.
To a community partner, the collection and management of these data thus have implications for the present project
but also, potentially, for future programmatic and policy decisions. Community partners should be very thoughtful
about how the data will be housed and about how decisions about present and future analysis will be handled. The
misuse of data can seriously undermine any future research efforts and potentially fracture a CBPR partnership. Setting
out strict standards about data use at the beginning of the project can be beneficial. Table 5.3 is an example of an
explicit policy regarding data confidentiality. This will be discussed at length in the next chapter.
Table 5.3 Data Ownership and Confidentiality7
There should be an explicit agreement between researchers and community and university partners with respect to
ownership of the research data. The nature of the agreement will be determined on a project-by-project basis, as the
community needs/requirements may differ (i.e., a school, a department of health, and a coalition may require different
agreements). This agreement should be made prior to the onset of the project and revisited during the project with the
project-specific working group.
Research projects will adhere to the human subjects review process standards and procedures set forth by the
appropriate institutional review board. In addition, research partners will adhere to standards of school departments and
community-based organizations when appropriate.
All investigators and research associates need to be certified for human subjects work.
All persons who work with data (including visiting scholars and graduate students) will be required to sign a pledge
of confidentiality for data use.
In general, we will not publicly report data with a cell size less than 5 subjects. Suppression of cell sizes < 5 in
reporting will be considered the general rule to protect the confidentiality of subjects. However, any agreement
requiring larger numbers for suppression will supersede this rule.
Source: Institute for Community Health, Cambridge, MA 2012.
DISSEMINATION OF FINDINGS
“The time delay between the conclusion of a research project and the publication of papers is always a problem. This is a period of time
when, because of the constraints of journal publication, there is a virtual embargo on the results except in the most general way, and it
drives the community side nuts. ‘Hey, we know this, we want to do something with/about it!’” Alex Pirie, Somerville, MA
Identifying the results of CBPR research is the first step in translating evidence into actionable efforts. As is part of
the CBPR process, the community is generally interested in using the results to make change. Change can be actualized
in many forms: new programs to address needs, changes in existing programs, policy change at the local and state
levels, and advocacy efforts. Findings make their way into these venues in multiple fashions. Program directors that
have access to the information can incorporate it into their day-to-day work. Advocates can promote the results with
their city councilors, aldermen, and legislatures to move an agenda forward; constituents can use the evidence to
improve their own situations. This is one of the most important value-added benefits of participating in CBPR efforts.
“Don’t save dissemination [or evidence] translation until the end. Systemic changes are key. Wait for results of qualitative + quantitative
research to see what worked and let the community know. Policy + systemic changes has built sustainability. Make it a priority to have
dialogue with community [and] key stakeholders.” (Community member-conference participant)1
In all research, the findings are not always what one might expect. Sometimes, results do not reveal the answers that
one is seeking, or they reveal unexpected information that may be particularly controversial. How will the partnership
deal with conflictual and potentially controversial results? Will community members want to censure the results? Will
they want to reframe the discussion? Community partners need to be prepared for these possibilities and discuss how
they will deal with results in advance of their delivery. In CBPR, the community partners are part of the analysis and
interpretation phase of the research, and this will help in the management of the dissemination process because of their
involvement in interpretation. However, from a community perspective, if you go into a project to prove something or
to demonstrate how well your program works, you may be disappointed. Community partners need to be realistic about
what the research can provide and discuss various possibilities for how they will deal with positive, negative, or neutral
results with their academic partners well in advance of fielding the research.
The definition of dissemination may be very different for academics and community partners. Community partners
want to get the information into the hands of policy makers or service providers in the form of reports, press coverage,
or community meetings. In our Everett example, the community partners requested that data be presented to the
community in a public forum at the end of the project so that community members could make recommendations for
action. They were adamant that this needed to happen quickly so as not to lose the momentum of the project. In
addition, given the focus of the work, the timing regarding the larger context of immigration enforcement was
opportune. The researchers would have to wait to work on publications for peer-reviewed journals until after this was
done. Together, they decided to present the data in a digestible format to an audience convened by the community
partners, who included local government as well as immigrants themselves. They also requested that community
members have an opportunity to consider recommendations for action based on the study. This engagement was built
into the forum agenda. Participants broke up into different language groups and worked together to answer key
questions and provide recommendations. The final product was a set of policy and practice recommendations that was
provided to leaders in the community. Many were then utilized to change police-immigrant relationships and address
health care access issues.2 Afterward, the researcher and community partners developed several papers for peer-
reviewed journals. The emphasis for dissemination was different for community and academics. Several other
dissemination options were also used that particularly spoke to advocacy, including a press release to local papers, a
letter to the Boston Globe, and several submissions to pertinent blogs. The dissemination plan for the project has since
taken multiple forms.
• Community forum with presentation
• Meetings with advocacy groups
• Meetings with police
• Presentations to local community advisory group
• Abstracts at national meetings
• Peer-reviewed publication(s)
• Press release(s)
• Letters to newspapers
• Blog submissions
TRANSLATION OF EVIDENCE INTO PRACTICE
“We always have to examine whether or not we are doing what we hope to be doing. But this is difficult because we get invested in what
we are doing and taking a critical examination is not always easy.” (Community member-conference participant)1
Dissemination of study results jump-starts the process of translation/adoption of evidence into practice. As people
hear about the ideas, they are likely to consider how to implement them. In addition, if the findings are strong, they may
influence how programs are run and, more importantly, whether programs continue to exist. Some of the decisions
about translation and sustainability may depend on the project topic. For example, if community members wanted to
test an intervention, there may already be tacit agreement to adopt the findings, when available, or to change or adapt
the program to improve it. The CBPR process is also making all kinds of new evidence available, including literature
reviews, information on evidence-based practices, and existing data for use in this CBPR process. There are often
unexpected outcomes of CBPR that may not have been the primary focus but are results of multidisciplinary approaches
that start with academic/community partnerships. Certainly, if adoption is contingent on money, translation and
sustainability might be harder to achieve. But the results of one study provide the foundation for the next. In Everett, the
community took the results and translated them directly into actionable steps. They launched police/immigrant
meetings. But they also used the data to obtain additional grants on racism and food insecurity and adolescent risk
behaviors. The data continue to have a life after the project. In Somerville, the community continued its vigilance by
monitoring 911 activity after the suicide/overdose crisis. They hired a mental health coordinator who is still in place. In
Cambridge, the work is ongoing, and the partners have written several other grants. While the project provides new data
for communities, it also shapes the researchers and their agendas. They benefit greatly from the colearning of the
partnership.
Overall, community partners should expect any CBPR researcher to be aware of the impact of a CBPR project on a
community at all different phases. This includes the impact on community partner capacity, mission, and time.
Researchers should recognize from the beginning that shared goals for a project need to be discussed and mutually
decided. Dissemination strategies take many forms, and communities perceive action—local action—as an end product.
A CBPR researcher should understand what the incentives for community participation are and simultaneously
acknowledge her or his own. Thus, as a final portion of this chapter, we offer a list of questions that a researcher should
consider prior to CBPR engagement.
Researcher Questions to Answer Before Getting Involved in CBPR
• Is my question emanating from community priorities?
• Will the results or findings of this study benefit the community?
• What am I really asking from the community partners? What level of responsibility will the community bear?
• How do I ensure that my partners understand the research and its limitations, the benefits and risks?
• Can I frame multiple dissemination products to serve both my goals and those of my community partners?
• Am I in it for the long haul? What is my commitment to the community?
A researcher who puts him- or herself in the community’s shoes and understands the community’s perspective will
go a long way toward supporting the CBPR process and improving the level of trust in community/academic
relationships.
“The strategy is to form it [the research] around a very clear, specific issue that everyone feels invested in. Then you can start building the
trust and relationship.” (Community member-conference participant)1
CONCLUSION
Community partners who wish to engage in CBPR should be aware of the benefits as well as the challenges of this
approach. They should be thoughtful about their own research readiness and aware of the realities—the responsibilities
and the potential outcomes—of engaging in a research project. They should seek a balanced partnership in which their
voice is heard and their viewpoints considered. Most importantly, they should consider how the outcomes of CBPR will
help their community, however that is defined. Knowledge is powerful. CBPR is an approach to gaining knowledge that
can empower communities. Thus, a well-designed CBPR study is more likely to encourage strong, trusting relationships
that cross academic/community boundaries and improve the translation of both evidence-based practice and practice-
based evidence into practical solutions for real-life problems.
QUESTIONS AND ACTIVITIES
Activities
Have students break up into small groups to discuss the following:
A community group had garnered a grant to develop and implement a program for African American men to improve
cardiovascular risk factors. The program was well liked by participants and by local politicians. But they had no idea if
the program made a difference in participants’ health. Initial data were promising with regard to satisfaction and
engagement of the participants, but there was no evidence that the outcomes (BP, cholesterol) were changing. There
was a great deal of investment in the program, and a publication that did not say “the program was working” would put
the program at risk financially. They engaged a local CBPR researcher to work with them and try to evaluate the impact
of the project.
What are the potential pitfalls in taking on this project?
Describe the community perspective and their incentives underlying this work.
How would you disseminate results?
What types of products might bring added value to the community?
Have students interview other CBPR researchers to ask them about their projects and how they approached the steps
or a particular step outlined in this chapter.
If possible, have students sit in on a meeting between CBPR researchers and their community partners and observe
the process. Then have them identify where they are in the research process and whether they adhered to CBPR
principles during the meeting. What went well and what was challenging?
Questions
1. What questions should a researcher ask at each step of the research process in CBPR?
2. What are the skills that a CBPR researcher will need to enter into CBPR?
3. You have gotten a grant to conduct a CBPR project. However, it ended up taking more time than you expected to
obtain the grant. During that time, the leader you were working with moved away. How would you re-engage the
community?
NOTES
1. Break-out sessions. Taking It to the Curbside: Engaging Communities to Create Sustainable Change for Health. Conference, Boston, MA;
April 2010.
2. Hacker K, Chu J, Leung C, Marra R, Pirie A, Brahimi M, English M, Beckmann J, Acevedo-Garcia D, Marlin RP. The impact of
Immigration and Customs Enforcement on immigrant health: perceptions of immigrants in Everett, Massachusetts, USA. Social Science &
Medicine. 2011 Aug;73(4):586–94.
3. Norris K, Brusuelas R, Jones L, Miranda J, Duru O, Mangione C. Partnering with community-based organizations: an academic
institution’s evolving perspective. Ethnicity & Disease. 2007;17(1):205.
4. Baker EA, Homan S, Schonhoff R, Kreuter M. Principles of practice for academic/practice/community research partnerships. American
Journal of Preventive Medicine. 1999 Apr;16(3 Suppl):86–93.
5. Lawrence Mayor’s Task Force. Tools for Research Partnerships in Lawrence, MA. Lawrence Mayor’s Task Force, Lawrence, MA 2006;
Available from: http://www.tuftsctsi.org/About-Us/CTSI-Components/Community-
Engagement/~/media/B35A1D1535DB422D90E1A47544743E4E.ashx.
6. Hacker K, Collins J, Gross-Young L, Almeida S, Burke N. Coping with youth suicide and overdose: one community’s efforts to
investigate, intervene, and prevent suicide contagion. Crisis. 2008;29(2):86–95.
7. Institute for Community Health. Policies. Data confidentiality/IRB policy. Cambridge, MA: 2002.
http://www.icommunityhealth.org/policies.html.
RESOURCES
Community Campus Parternships for Health.http://www.ccph.info/.
