Use of a community advisory board to build equitable algorithms for participation in clinical trials: a protocol paper for HoPeNET
•,,,,,,,,,,.
...
Abstract
Introduction Participation from racial and ethnic minorities in clinical trials has been burdened by issues surrounding mistrust and access to healthcare. There is emerging use of machine learning (ML) in clinical trial recruitment and evaluation. However, for individuals from groups who are recipients of societal biases, utilisation of ML can lead to the creation and use of biased algorithms. To minimise bias, the design of equitable ML tools that advance health equity could be guided by community engagement processes. The Howard University Partnership with the National Institutes of Health for Equitable Clinical Trial Participation for Racial/Ethnic Communities Underrepresented in Research (HoPeNET) seeks to create an ML-based infrastructure from community advisory board (CAB) experiences to enhance participation of African-Americans/Blacks in clinical trials.
Methods and analysis This triphased cross-sectional study (24 months, n=56) will create a CAB of community members and research investigators. The three phases of the study include: (1) identification of perceived barriers/facilitators to clinical trial engagement through qualitative/quantitative methods and systems-based model building participation; (2) operation of CAB meetings and (3) development of a predictive ML tool and outcome evaluation. Identified predictors from the participant-derived systems-based map will be used for the ML tool development.
Ethics and dissemination We anticipate minimum risk for participants. Institutional review board approval and informed consent has been obtained and patient confidentiality ensured.
Introduction
Machine learning (ML) can identify statistical patterns from generated data to train computers to perform tasks intended to aid in human decision making.1 Emerging use of ML is occurring in clinical trial evaluation and clinical trial recruitment,2–4 a field in which improvement in reaching and recruiting racial and ethnic minorities is increasingly essential. For individuals from groups who are recipients of societal biases, utilisation of ML can lead to the creation and use of biased algorithms.5 6 The design of equitable ML tools that advance health equity could be guided by community engagement processes which leverage collective knowledge and experience to inform clinical trial development and design.
Participation from racial and ethnic minorities in clinical trials has been burdened by issues surrounding mistrust and access to healthcare, both of which ultimately impact referral to clinical trials.7 8 Furthermore, participation barriers may extend beyond these recognised factors. To address barriers, community-based participatory research (CBPR) has emerged to involve communities at all stages of a research study life cycle from study design through dissemination of results. A component of CBPR research is the formation of a community advisory board (CAB) to advise and direct research questions, recruitment plans and evaluate disseminated results of the study.9 10 Although considered central in securing participation from under-represented communities, utilisation of the CAB’s experience more broadly in therapeutic clinical trials has been limited. To date, the utilisation of the CAB experience to generate data has not been used to develop ML algorithms. We, therefore, seek to conduct a study in which CAB input is utilised for ML development through capturing ‘lived experience-based knowledge’ generated during a 12-month CAB participation study. We will achieve this goal by addressing three specific aims: (1) measure perceived barriers/facilitators to clinical trial engagement among African-Americans using qualitative and quantitative approaches; (2) use group-based model building as a systems science approach to identify key activities to improve community trust and engagement and (3) develop an ML-based tool for predicting community engagement in clinical trials using data from the group-based model building. Figure 1 is a graphical representation of the three phases of the study.
Graphical abstract of HoPeNET protocol: a community advisory board (CAB)-based protocol to evaluate lived experiences from multiple stakeholders, to create systems-based understanding of barriers and facilitators to clinical trial participation. HoPeNET will aid in creating a predictive algorithmic tool to help increase African-American clinical trial participation. Figure created by coauthors (NF, FOB and EO-C).
Methods
Recruitment and characterisation of study participants
This is a multisite 24-month, triphased study of 56 participants. The two sites for the study are Howard University (HU) and the Intramural Research Programme (IRP) of the National Institutes of Health (NIH). Both sites are located within the metropolitan Washington, D.C. area.
During phase 1 (figure 2), formation of a 50-member HoPeNET CAB will occur through the recruitment of two groups: 25 community partners and 25 investigators. Recruitment of participants will occur through multiple channels including emails to our current CAB on cardiovascular disease and obesity, the Washington D.C. Cardiovascular Health and Obesity Collaborative (D.C. CHOC)11; flyer distribution targeting members of community-serving non-profit organisations, through institutional communications and snowball recruitment for HU and the NIH IRP investigators. Inclusion and exclusion criteria for each group of HoPeNET CAB members are provided in figure 2.
Study procedures. Figure created by coauthors (NF, FOB and EO-C). CAB, community advisory board; CBPR, community-based participatory research.
A preparticipation survey and one-on-one interviews will be used to capture demographics, knowledge of CBPR principles, perceptions and beliefs surrounding clinical trial participation, and the role of social determinants and implicit bias on research outcomes (table 1). The preparticipation survey and interview guide will be pilot tested prior to administration.
Table 1
|
Assessment data and measurement tools
Focus groups/workshops: group model building
Three focus groups/workshops with (1) community members, (2) investigators and (3) both groups combined will be conducted using a group model building (GMB) activity methodology12 (figure 2). GMB is a powerful participatory method for actively engaging stakeholders or communities to provide perspective on a complex problem, structure, or dynamic process as well as the results and solutions.12 13 To facilitate knowledge sharing, discussion and consensus on the issue, participants will be engaged in a number of activities (scripts). The objective of the first two sessions is to elicit discussion on factors contributing to a lack of African-American clinical trial participation by creating a systems-based map or causal loop diagrams (CLDs). Our rationale for conducting separate workshops prior to the combined sessions is to develop a richer understanding of participants’ unbiased perceptions surrounding clinical trial participation. Additionally, this strategy improves stakeholders’ engagement and participation in group activities that may suffer due to power imbalances. The research team will work offline between sessions to refine and synthesise the CLDs or informal causal maps, created during sessions 1 and 2 by participants. During the third combined session, participants will evaluate the synthesised map. This is necessary to ensure that the model reflects the insights and stories shared by participants. In addition, this allows for participants to identify potential areas in the system where they believe change is needed. Following the iterative process, all participants will be provided copies of the combined systems map to provide final feedback to the research team.
CAB intervention
In phase 2 (figure 2), the HoPeNET CAB experience will commence. Prior to the start of bimonthly meetings, the CAB participants will receive asynchronous training on CBPR principles, code of practices, and confidentiality procedures. Training will be provided by NIH site team members (TP-W, GRW, NF and VMM) and current members of the D.C. CHOC CAB. To ensure equitable discussions during meetings, ‘Ambassadors’ from each stakeholder group will be trained to lead CAB meetings.
The CAB will meet bimonthly for 2 hours over a 12-month period. During the meetings, non-CAB members (n=6) consisting of investigators and community members will be invited to present on specific disease areas from ongoing protocols or community health projects. Inclusion criteria for the HoPeNET presenters are based on criteria for the HoPeNET CAB members (see figure 2), except presenters do not have to self-identify as African-American. Exclusion criteria are that presenters cannot participate as CAB members or focus group facilitators. After the presentations, the HoPeNET CAB will evaluate the studies and provide feedback on ways to engage African-Americans in their protocols/programmes or to engage researchers in the community projects. All HoPeNET CAB meetings will be audio recorded for anonymised transcription. To continue the iterative process of the system-based map, at the mid-year time point, the HoPeNET CAB will reevaluate the systems-based model of facilitators and barriers to clinical trial participation. All revisions of the model will be provided to research staff for analysis. Participant engagement at the mid-year and end-of-year time points will also occur using a validated standardised metric.13
ML tool development
Phase 3 will occur over a 6-month period and will focus on the implementation of data results from phases 1 and 2 for the development of the ML-based tool and outcome evaluation. The primary input data for the ML algorithm will be the results of the facilitators and barriers model (table 2) created from the group-based model activity and analysis of collected qualitative and survey data. Initial survey responses will be aggregated thematically across multiple responses and Likert scales to create ordinal scales to use supervised ML approaches. This will include regression models and decision trees to identify patterns in the data. Supervised ML approaches are useful in identifying patterns where we have labelled and structured data. Meanwhile, coded responses from free form assessments such as focus groups will be analysed using unsupervised ML approaches such as hierarchical and non-hierarchical clustering of responses/participants. Unsupervised approaches can highlight and detect previously undetected patterns in data that provide insights into the perspectives of study participants. For example, each model will have an identified central latent variable that is directly and indirectly related to measurable or observable factors (second level or third level variables) for a clinical trial.
Table 2
|
Analytical approaches used in HoPeNET machine learning algorithm development
Outcomes and evaluation
Aligned with our goal to develop an ML predictive tool based on the lived experience of stakeholders, evaluation of the HoPeNET CAB study will be guided by an adaptation of the conceptual logic model of CBPR14 15 (online supplemental figure 1). The evaluation approach includes careful consideration of community context and understanding group dynamics to build an equitable partnership that explicitly values reciprocal learning as illustrated in the model.
In addition to developing an ML tool, we anticipate that the HoPeNET CAB experience will influence investigators’ behaviours and perceptions. To assess this, we will conduct post-CAB surveys and interviews. Engagement and reach in the community will be assessed using the following metrics from studies presented at CAB meetings: (1) number of participants screened who are directly from the community and (2) volume of requested recruitment materials from members of the community.
Ethics and dissemination
Participant confidentiality and privacy will be strictly maintained and held in trust by the participating investigators, and their staff. No information concerning the study, or the data will be released to any unauthorised third party without prior written approval of the Principal Investigator. The study data entry and study management systems used by research staff will be secured and password protected. At the end of the study, all records will continue to be kept in a secure location for as long a period as dictated by the reviewing institutional review board, institutional policies or sponsor requirements. We anticipate minimal risk for this study. However, we will ask participants to express their perceptions surrounding barriers and facilitators of clinical trial participation during the one-on-one interviews and focus group activities. We recognise that this activity may elicit emotional distress. Study participation will be voluntary and interviews can be stopped at any time. To be consistent with CBPR principles and to the stated programme evaluation, study findings will be disseminated to the HoPeNET CAB, presented at departmental and institutional levels at HU and the NIH IRP. We will also present our findings at national and international conferences, and peer-reviewed manuscripts from our project will also be submitted for publication.
