Article Text
PDF
Statistics from Altmetric.com
Introduction
Leading figures in biomedical informatics advocate education in digital health for the healthcare workforce.1 2 In healthcare, artificial intelligence/machine learning (AI/ML)-enabled tools increasingly play a role by informing patient triage decisions, clinical decision support systems, and healthcare resource management3 – advances that are undoubtedly set to grow.4 Tens of thousands of healthcare apps are available for download by consumers, promising a range of services, from symptom tracking to diagnostic and treatment advice.
To date, surveys of medical professionals reveal divergent views about the value and impact of AI/ML on their job with many physicians sceptical about the potential scope for technological innovations on medical tasks.5–7 Furthermore, surveys consistently find limited evidence of formal teaching in medical education about AI/ML. Only a few studies – conducted in Europe, the US and South Korea – have explored the formal education and familiarity of medical or healthcare students with respect to digital advances in healthcare, and much of this work consists of single site studies.8–14 To better understand and engage with discussion about the benefits, limitations, and ethical dilemmas presented by these tools, today’s medical students will need to become more digitally savvy. Equally, as patients make increasing use of healthcare and well-being algorithms, medical students will need to become better prepared to offer patients advice, and to have knowledge about, the robustness of these tools including when algorithms are safe to use.
In the present study, we built on this research by assessing the experiences and opinions of final year medical students throughout Ireland about their exposure to AI/ML during their entire degree programme.
Methods
A paper-based, cross-sectional survey was administered to final year medical students at four of Ireland’s seven medical schools. Institutions were selected in each of the country’s four geographical provinces. The study team devised an original survey instrument to investigate the familiarity, formal exposure to, and opinions of medical students about ML/AI in medicine. We developed the survey instrument in consultation with Irish, British, and American physicians and piloted the survey with physicians in Ireland and the UK (n=6) and final year medical students in the UK (n=5) to ensure face validity. The survey explored students’ experiences and opinions about the teaching of AI/ML in their medical degree programme to date (see Section E of online supplemental appendix 1, and table 1 for survey items). Using ‘yes’ or ‘no’ responses, the survey asked whether students had heard of the term ‘machine learning’, were familiar with “big data analytics”, and whether they had read any academic articles on AI/ML in medicine. Students were requested to estimate both how many hours their instructors or lecturers had spent, and will spend, discussing AI/ML during their degree. In addition, selecting from ‘yes’, ‘no’ or ‘maybe’ responses, the survey inquired whether students planned to learn about how AI/ML as it pertains to medicine. Finally, using a 6-point Likert scale, students were requested to rate their level of agreement with the statement ‘Discussion about AI/ML should be part of medical training.’
Supplemental material
Familiarity and opinions of medical students about Artificial Intelligence/Machine learning in their medical degree
The institutional review boards at University College Cork [protocol # 2018–188], National University of Ireland Galway [protocol # 19-Dec-15], Queen’s University Belfast [protocol # 19.28], and University College Dublin [protocol # LS-19–89] approved the study protocol at their respective sites. Between April 2019 and March 2020, the anonymous survey was distributed by lecturers after compulsory final year classes at each institution to increase responses. Participation was voluntary and all students who decided to participate provided written consent. After survey collection, responses were entered into Excel, and descriptive statistics and analysis were carried out using JASP (0.9.2) and SPSS v 27.
Results
A total of 252 of 585 (43%) of final year students across three medical schools responded. Data collection at one medical school (University College Dublin) was terminated in March 2020 because of teaching disruption due COVID-19, and survey data from this site was excluded from the analysis. Of all respondents, 157 of 251 (62.6%) were female, and 223 of 246 (90.7%) were born in 1992 or later. Among respondents, 66.5% reported zero hours of teaching on AI/ML during their degree with 62.4% anticipating zero hours during the remainder of the degree programme, 43.4% (95% CI, 37.1% to 49.6%) had not heard of the term ‘machine learning’, and 80.6% (95% CI, 75.6% to 85.6%) had not read any academic journal articles on AI/ML. Asked about whether they intended to learn about AI/ML in medicine 41.1% (95% CI, 34.9% to 47.3%) reported ‘yes’ and 46.5% (95% CI, 40.2% to 52.8%) responded ‘maybe.’ However, 78.6% agreed that discussion about AI/ML should form part of their training. Results are reported in table 1.
Descriptive data were analysed for differences according to gender and birth year. Male respondents were more likely than females to report having heard about ML (69.7% v. 48.7%), χ2(1)=10.05, p=0.002. Participants who heard about ML, on average, had an earlier birth year than those who had not, t(234)=2.193, p=0.029. Willingness to learn about AI/ML was recoded to reflect the ordinal nature of the data (yes=1, maybe=2, and no=3) so that inferential statistics could be run. There was a trend towards younger participants being less likely to plan to learn about AI/ML, rho=-.109, p=0.095. Based on the results of a Mann-Whitney U test, male respondents were more likely to plan to learn about AI/ML than female participants, Z=2.25, p=0.025.
Discussion
This is the first study to explore the experiences and opinions of Irish medical students about AI/ML in their medical degree programme. Medical students reported limited awareness and education on AI/ML. Notably, around four in ten of survey respondents had not heard of the term ‘machine learning’. Around two in three respondents reported no time spent learning about AI/ML during their whole medical degree. Although a minority of students did report some formal teaching on AI/ML, it is unclear whether this was part of their compulsory medical curriculum or (for example) via elective medical courses or guest lectures. Perhaps reflecting training gaps or lack of confidence on the topic, few students reported reading any academic articles on AI/ML in medicine. Relatedly, students were divided about their plans to fill educational gaps, with almost half of students reporting some uncertainty about whether they would undertake additional learning on these topics. Contrary to our expectations, younger participants were less likely to have heard of ML; however, the majority of participants were typically young adults: 91% had a birth year between 1992–1999. Conceivably, with greater variance in ages of participants we might have observed different findings. Finally, while the majority of students reported a lack of formal instruction on AI/ML in medicine, considerably fewer students seemed to approve of the status quo. In common with other surveys,8 9 12–14 the majority of medical students considered learning about AI/ML should form part of their formal medical degree.
To help address education deficits, we suggest medical schools consider developing short, cross-disciplinary courses in digital health, including an understanding of augmented intelligence, to empower students to keep abreast of technological advances. Indeed, the need for further education on these topics may also apply to allied health professional training including nursing, pharmacy, clinical psychology, and physiotherapy. Because technology changes rapidly, we recommend that training and education encompass critical thinking skills so that students are well equipped to appraise new technologies. For example, courses in evidence-based medicine might incorporate discussion about evaluation of clinical decision support systems, the potential for algorithmic biases in data sets, and challenges associated with the explainability of AI/ML decisions. Medical ethics courses might usefully incorporate topics related to patient privacy with the use of digital devices and apps, and the potential for AI/ML-tools to mitigate or exacerbate digital divides in healthcare. Finally, we caution that without solid curricular advances, medical students and health professionals may rely too heavily on hype or inflated media reportage to inform their views, leading to negative consequences for healthcare. For example, surveys in Canada and the UK suggest that, under the misguided view that radiology will be imminently replaced as a field by AI/ML, students are more likely to rule out this specialty as a career choice.12 15
This study has some strengths and limitations. A strength was soliciting the views of students from institutions in geographically distinctive regions of the country. However, the moderate response rate (43%) raises questions about representativeness. Response biases could also have influenced our findings depending on whether students most enthusiastic or those inclined to view AI/ML negatively answered the survey. While our aim was to gauge the general awareness of medical students about these topics, some survey items, such as ‘familiarity with big data analytics’ might be challenged as vague and open to interpretation. We recommend that qualitative research methods might provide more nuanced findings on students’ opinions and awareness about AI/ML in medicine. In addition, we suggest future studies might usefully explore the opinions and familiarity of medical faculty about AI/ML in medical education, and/or evaluate medical curricula course content to assess where, if at all, students acquire learning on these topics. Finally, the survey was administered prior to the COVID-19 pandemic which has overseen considerable developments and attention given to the role of AI/ML-enabled tools including in digital epidemiology and public health. Conceivably, as a result, had the survey been undertaken today we might have found increased awareness or familiarity about these topics among medical students. However, we emphasise it remains to be seen whether this heighted attention translates into tangible curricular developments. Furthermore, no surveyed medical school has since modified their curriculum to include education about AI/ML.
We close by noting, in recent years Ireland has gained recognition as a global technology hub with the fastest growing tech workforce in Europe.16 Despite these advances, we cannot help but observe the risk of digital education in healthcare lagging behind. Improvements in digital education will help prepare tomorrow’s doctors to lead policy and practice advances on the role of AI/ML-enabled tools in the health professions and in patientcare.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by Ethics approval was granted by the institutional review boards at University College Cork (#2018-188), National University of Ireland Galway (#19-Dec-15), Queen’s University Belfast (#19.28), and University College Dublin (#LS-19-89). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
CB thanks Dr Cliona McGovern for assisting with data gathering prior to the termination of the study at UCD due to COVID-19, and Dr John Halamka for early discussions about the content of the survey.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
CD and KDM are joint senior authors.
Twitter @crblease, @AnnaKharko
Contributors Conceptualization: CB. Data curation: CB & AK. Formal analysis: AK, MB, CB. Survey Design: CB, CD, KDM, CBr. Survey Administration: CB, CBr, IW, MH. Writing – original draft: CB. Writing – review & editing: CB, CBr, IW, MH, CD, KDM, AK, MB, MH.
Funding Open access fees were paid by FORTE—the Swedish Research Council for Health, Working Life and Welfare through the research project “Beyond Implementation of eHealth” (2020-01229) awarded to MH.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.