Goodman Aims to Measure Research Engagement
Photo: Chia-Chi Charlie Chang
Despite the importance of involving patients and other partners in the research process, there’s no way to measure their engagement. Dr. Melody Goodman, an associate professor of biostatistics at New York University, is developing a survey tool she hopes will change that.
“The extent to which stakeholders in research partnerships feel engaged has not received scientific attention,” said Goodman at a recent Wednesday Afternoon Lecture in Lipsett Amphitheater. “It’s important to understand how engagement levels in partnerships are developing and to what extent engagement level is a predictor of outcomes.”
Research on stakeholder involvement is an evidence-based approach for addressing health disparities. It ensures that patients and communities are at the center of research and provides a unique perspective about patient and community needs.
The existing measures of stakeholder engagement don’t capture that, Goodman said. One measure assumed that attendance at events and activities qualified as engagement.
“Showing up doesn’t necessarily mean you’re engaged. You may show up for pizza and cookies, but not really engage in the conversation that’s happening,” she explained.
Other measures tried to quantify the degree to which participants felt they were part of a positive community, felt comfortable sharing their thoughts and had confidence in the problem-solving process.
“All of these things are related to engagement, but are not measuring engagement directly,” she observed.
Goodman is developing a 32-item measure of 8 engagement principles with a community advisory board that will assess both the quantity and quality of stakeholder-engaged research. To start, she invited 19 academics, researchers and patients and their advocates to participate in a 5-round Delphi process, a technique for collecting and organizing informed opinions from experts.
Photo: Chia-Chi Charlie Chang
The principles are: focus on community perspectives and determinants of health; partner input is vital; partnership sustainability to meet goals and objectives; foster co-learning, capacity building and co-benefit for all partners; build on strengths and resources within the community or patient population; facilitate collaborative, equitable partnerships; involve all partners in the dissemination process; and build and maintain trust in the partnership.
“One of the things that became really clear to us in this process was that definitions were important, so we could be on the same page. We worked through definitions for each of the engagement principle titles,” Goodman noted.
Once the measure—called the research engagement survey tool—was created, Goodman tested it. She administered 4 versions of the survey to almost 400 people who were involved in a research study over the course of 2 years. She rewrote some questions to make sure the respondents without academic backgrounds could understand them.
The respondents were from across the country, mostly African American or white, having some college or higher education and mostly female, “something that’s representative of the population we’re trying to reach,” Goodman said. “We didn’t do a good job with Hispanics, Asians and other multiethnic groups.”
In 2020, she will create an iterative algorithm that will be tailored to a person’s responses. It will reduce the time they spend taking the survey. Additionally, she will start thinking about how to tailor the survey tool to members of groups, accounting for such factors as age, race and ethnicity and geographic area.
It’s important to engage the people most impacted by research, Goodman said. By reviewing data on engagement levels, researchers can take steps to identify people who should be engaged in research and devise strategies to include them.
“And yes, that may require a little bit more work and a little bit more time, but the science will greatly benefit from that,” Goodman concluded.