Development of a core competency framework for clinical informatics

Introduction

The healthcare sector in many countries is facing increasing demand as people live longer and healthier lives.1 The public’s expectation of healthcare is also increasing and is tempered by various financial constraints. The healthcare sector has lagged behind other sectors regarding its adoption and use of digital technology. In the UK, the Topol review was carried out to assess how the healthcare workforce can be prepared for the digital future. The review makes many recommendations on the use of genomic technology, robotics, artificial intelligence (AI) and digital medicine, including the training and education of healthcare professionals in such areas.1 At the cutting edge of this digital upskilling of the workforce are informaticians from clinical, health and social care disciplines.

The American Medical Informatics Association (AMIA) defines clinical informatics as ‘the application of informatics and information technology to deliver healthcare services’.2 The UK Faculty of Clinical Informatics (FCI) defines a clinical informatician as: ‘A clinical informatician uses their clinical knowledge and experience of informatics concepts, methods and tools to promote patient and population care that is person-centred, ethical, safe, effective, efficient, timely, and equitable’.3 (statement 3, P22)

As yet there are no UK-based overarching competency frameworks aimed at multiple informatics disciplines. Instead existing frameworks tend to focus on specific domains such as nursing or bioinformatics.4 The UK FCI was created to provide support for clinical informaticians, including those with clinical roles in the health and social care domains applying informatics in practice. It is the intention of the FCI to provide and accredit competencies for informaticians. This includes accreditation of the UK’s National Health Service (NHS) Digital Academy programme which aims to create digital leaders for the digital transformation of the NHS. The present study forms part of a programme commissioned by the FCI to create a national competency framework for clinical informaticians in the UK.

Competency describes the behaviours, characteristics, skills, attitudes and knowledge application used to successfully achieve something. Competence therefore is the achievement of a single competency or multiple competencies. A core competency framework describes the essential set of competencies required to achieve competence in a specific area. There currently exist many competency frameworks aimed at various clinical informatics disciplines, such as the ELIXIR (the European life-sciences Infrastructure for biological Information) and TIGER (Technology Informatics Guiding Education Reform) frameworks for bioinformaticians and nurses. This paper reports on the methods used to generate and refine the UK FCI’s Core Competency Framework (CCF), which covers the core competencies required to develop clinical informaticians’ professional competencies, and to provide a process for the FCI to provide accreditation for training and education programmes and individual clinical informaticians.5

Background

The Core Competencies Project spans two primary phases. The first part of the work explored the definition of the professional attributes of a clinical informatician. The FCI carried out phase 1 of the work (figure 1) which consisted of three reports based on (1) discovery,6 (2) validation7 and (3) consultation around the output competences of a clinical informatician.3 This included defining clinical informatics, clinical informaticians, inclusivity, professional boundaries and the functional domains (scope) of clinical informatics practice. This work was carried out using qualitative methods (eg, interviews) with the faculty’s membership. The second phase (reported in this paper) involved an iterative process consisting of three separate iterations. The first iteration involved the combination of findings from both academia and industry in the form of an analysis of over 50 informatics job postings and a systematic literature review which explored the commonality of informatics competencies across different clinical informatics domains (such as medicine, nursing, pharmacy).4 Following synthesis of this information, an initial draft competency framework was generated which was then presented to 15 informatics experts in one-to-one semi-structured interviews and adapted following feedback. The amended version was presented for wider evaluation through a digital survey with 87 participants. In light of the survey results, the final version of the competency framework5 was updated and disseminated publicly on the FCI’s website.

Request permission

Figure 1 provides an overview of the iterative steps followed to create the final core competency framework. This paper reports on the interviews and survey (iterations 2 and 3 of phase 2). The principle research questions were:

• RQ1: Was the mixed-methods approach chosen appropriate for generating a core competency framework?

• RQ2: How do participants feel the framework should be applied and to whom?

• RQ3: What competencies were considered ‘core’ and how do they fit in with an evolving profession?

Iteration 1: interviews

Iteration 2: survey

Following incorporation of interviewees’ feedback into the framework, the amended version was presented for wider evaluation through a digital survey.

Methods

A bespoke digital survey data collection tool (figure 2) was created using Python Flask to allow participants to view the survey side-by-side with the competency framework for easy comparison. The online survey was piloted with the research team and internally reviewed. Most of the questions were based on those asked in the interviews with the addition of rating scales (Likert items) for generating summary data. Participants were identified by the FCI and participants were asked to forward the survey to other relevant professionals.

Request permission

Figure 2

Screenshots of the survey data collection tool. The framework can be viewed and scrolled on the right of the screen during the primary survey so that participants can reference the framework easily as they complete the survey.

The survey was hosted by the University’s Research IT team. Survey data were analysed using descriptive statistics (means, percentages), and by using sentiment analysis using the Bing lexicon.8 Sentiment analysis is a computational method for assessing opinions in textual data and whether or not the writer expresses a positive, negative or neutral sentiment.

The sample

A total of 87 respondents completed the survey with between 0 and 49 years of experience in informatics (M=12.8, SD=10). The median age of respondents was 50 years (range: 29–73 years). Table 4 shows the breakdown of respondents by informatics discipline. The ‘other’ informatics backgrounds identified can be seen in box 1.

Table 4

View inline|View popup

Demographics and background summary showing the number and percentage of survey respondents

Box 1

Other informatics backgrounds entered by respondents

• Research Informatics.

• Imaging Informatics.

• Clinical Coding/Healthcare Evaluation Data (HED)/SNOMED CT (Clinical Terms).

• Engineering.

• Knowledge management.

• Allied Health Professional (AHP) in NHS working on digital innovation.

• Urgent Care Informatics.

• Little experience in informatics.

• Dental Informatics.

Results

We present the results of the survey including the Likert item questions followed by the open-ended questions. The questions asked in the survey can be seen in the online supplemental appendix box 3. Figure 3 shows the responses for self-rated informatics expertise. This was rated on a scale from 1 (novice) to 5 (expert). The percentage of responses can be seen on the y-axis with the actual number of respondents in the bars.

Request permission

Figure 3

How would you rate your informatics expertise? (1=novice, 5=expert): Note the number of respondents is displayed on the bars.

A summary of the questions asked for the Likert items can be seen in table 5. Figure 4 shows the results for each of these questions per informatics background.

Table 5

View inline|View popup

Likert item questions with score ranges

Request permission

Figure 4

Likert question responses per question by informatics background.

The remaining questions were open-ended questions, an overview can be seen in table 6.

Table 6

View inline|View popup

Free text open-ended questions

Competencies missing from the framework

Thirteen respondent’s (32.5%) stated that they could not identify any missing competencies. Of those identified as missing, competencies suggested included aspects of diplomacy and negotiation, clinical safety of systems, medical device regulations and knowledge of research governance. ‘Cyber security’ was mentioned by several respondents (n=3).

Odd that there is no mention of cyber security at all, and little on confidentiality and information governance. Burying them under Data security and governance (2.6) is insufficient.

Some element of historical lessons learnt in informatics was also suggested (n=2).

I would add a historical lessons section on previous successful and not successful implementations of health informatics (eg, NPfIT, Choose & Book, eRS, PACS/DICOM, Wachter review, Topol review) which require to learn [from] past success and failures.

One respondent mentioned that an awareness of programming/software development is not sufficient and that practitioners should actually cultivate skills to create custom-built solutions.

If I think about some of the health informaticians/clinical bioinformaticians I have had contact with over the last few years—many of them discuss actually doing software development roles—for example, making Apps/interactive interfaces and also requiring programming skills to do so. I think this is not strong enough in the framework—I don't think it’s sufficient to have an awareness of these things/or just be able to evaluate them I think they actually need the knowledge and skills to build solutions for themselves (particularly if you consider a financially strained NHS where potentially external solutions will not be affordable). Some bio-informaticians have also had to undertake roles such as server migration (in genomics) and implementing cloud solutions—again I don't think these are alluded to anywhere—[cloud] solutions should definitely be mentioned.

Are the competencies 'core' competencies?

Although some participants recommended adding software development skills when asked what competencies might be missing from the framework, others (n=2) suggested this may not be a core competency:

I think there are many current active Clinical Informaticians who struggle to fulfil all of these competencies currently. One good example would be the fields of advanced coding/visualisation/modelling and AI. I think as an initial set of 'core' competencies, only basic information on these may be appropriate. More advance training may allow some parties to specialise in that field. (Analogous with for example, A Basic Surgical Training then specialisation say as an Orthopaedic surgeon, as opposed to an Ophthalmologist).

Another participant pointed out that the requirement should be to have an awareness of—rather than being able to practice—these skills:

While the range is pretty comprehensive, it aligns with competencies in other countries and the requirement is often at the 'awareness' rather than the practitioner level—so I think these are OK.

Terminology used in the framework

Of the 37 participants that answered this question, 18 (49%) accepted the terminology presented and did not request any changes. Of those that did request changes, the majority were minor. The changes suggested were very varied with little to no agreement and focused on different aspects, for example:

I would like to see the concept of clinical assurance and clinical risk management in Domain 4 Also more emphasis on governance—risk ownership, project assurance, project delivery, product ownership and maintenance in Domain 6.

I think all terms are accessible to those working in the field, but might be difficult for those who are thinking of changing into a career in informatics, particularly those with limited experience.

Comments on the overall structure or grouping of the framework competencies

Of the 46 respondents answering this question, 28 (61%) had no comment on the structure or thought the structure was good and did not require any changes. The majority of responses indicated minor changes to the organisation of the competencies with no significant homogeneity of opinion.

I really liked it. I think AHPs [Allied Health Professionals] would find it helpful in understanding their competencies in this field of practice and would support a career pathway for them into the professional cohort. The language is familiar and readable.

Another respondent considered the utility of identifying how one would progress from the essential core competencies through to expert level.

I think it would be useful to identify a progression from 'core' to 'expert'. This would support the identification of a learning programme to support a career path in clinical informatics and support the progression from Associate to Fellow within the Faculty

Comments about individual competencies in the framework

The specific changes to competencies suggested by respondents can be seen in table 7.

Table 7

View inline|View popup

Specific changes proposed to competencies (NB: competencies from second iteration of the framework)

Another respondent suggested defining broad skills in an ‘assumptions’ section as one could consider skills like project management as not being specific to clinical informatics:

Suggest defining some of the broad skills in an assumptions section that is, ‘it is assumed that each individual has the following competency areas not specific to CI’ for example, project management skills, quality improvement skills, governance skills—and only including those skills specific to CI.

The final CCF

Overall, participants were generally positive about the framework. This is quantified by sentiment analysis of the survey responses using the Bing lexicon10 which showed a net sentiment score of 60.33% (negative=70, positive=283). The final domains and subdomains included in the framework can be seen in figure 5. The final version of the framework can be viewed on the FCI’s website (https://facultyofclinicalinformatics.org.uk/core-competency-framework).

Request permission

Figure 5

Domains and subdomains in the final competency framework.

Competencies are defined hierarchically by complexity using Bloom’s original Taxonomy11 progressing from level 1 to 6 consisting of remembering (knowledge), understanding (comprehension), applying, analysing, synthesising and creating and finally evaluation. Such competencies can be graded against the taxonomy.12 The CCF has four principle areas: health, data and technologies, leadership and management and people. There are six domains comprising of health and well-being in practice, IT and systems, working with data and analytical models, enabling human and organisational change, decision making and leading informatics teams and project. Each domain contains a number of sub-domains (n=36) which contain the individual competencies (n=111).

The rationale for action on a suggestion to make a change to the framework was based on several factors including the number of people expressing a similar desire for change, whether the detail provided was sufficient to understand the suggested change and the relevance to clinical informatics in general. Following collation of data (systematic review, interviews and survey) the authors determined the final competencies at each stage through internal consensus based on the synthesised data.

Discussion

The use of an iterative approach for competency definition is advocated by Greenhalgh and Macfarlane.13 Our approach was similar to that of Jidkov et al which used an iterative method to develop 20 universal health informatics competencies for postgraduate medical education. The approach featured a literature review, content analysis and expert review.14 As opposed to a purely topic-based approach, competency approaches have been gaining more attention in both nursing and medical education.12 This is not without some issues however, such as problems related to identifying priorities as well as defining and measuring how competencies are met.15 Therefore one should aim to strike a balance between covering the most salient areas and keeping the number of competencies to the smallest number required. Another threat to competency models is that of clinician burnout in postgraduate education.16 As pointed out in the responses, there is a potential additional burden on clinical informaticians as they already have professional membership as clinicians and allied health professionals, with codes of conduct and clinical competencies to maintain. Consideration is required in terms of how clinical informaticians will be required to meet the competencies defined in the framework. Several participants indicated that a light touch is preferable to more intense examinations. It should also be recognised that participants may work in one (or several) domains deeply and not have experience spanning all the domains depending on their role.

The mixed-methods approach chosen appears to be appropriate for generating a core competency framework, as it allowed us to take a systematic and structured approach to co-designing the framework with no major issues. The combination of qualitative and quantitative methods contributed to the robustness of the final output. There already exist specific frameworks for certain informatics subdomains, such as TIGER for nurses12 and ELIXIR17 for bio-informaticians. In addition there are international frameworks and recommendations for informatics education such as18 and.19 Although there is some overlap between the CCF and competencies identified in such international frameworks, the CCF adopts a UK-specific perspective, taking into consideration the unique characteristics of UK health and social care. Considering this local perspective is crucial to the development of competency frameworks.4 12 20 The Topol review specifically identifies the FCI as ideally positioned to recruit, retain and credentialise NHS data science specialists.1

Participants felt that the framework should be open to as many professional groups as possible. It was also highlighted that practitioners may work in depth in a few domains and may not span all the domains. However, it was considered that at entry level, an awareness of the other domains would be sufficient and that practice-level skills may not be required in all the domains. The participants generally felt that all the competencies were ‘core’ competencies depending on the level that a practitioner was expected to engage with them. There was some contention in terms of the more technical aspects (specifically around domain 3, ‘working with data and analytical methods’) where some felt that informaticians should have more extensive skills (ie, being able to build software), whereas others put forward that clinicians may struggle to meet this requirement. As it stands the framework only stipulates an awareness of such issues rather than practical experience.

The evolving technical requirements of informatics and the overlap with the profession of data science is a key consideration. Douglas Fridsma, the former president and COE of the AMIA, discussed how such similarities may be operationally defined via the development of core competencies.21 There may be some overlap in areas of analytical and computational skills in healthcare data science as seen in a recent content analysis of healthcare data scientists job postings22 and informatics. Although data science may involve a deeper dive into analytical methods and processes, informatics spans a wider remit and can be involved with decision-making, communication and implementation of data analysis and health systems. The boundaries between such professions may be blurred at times, but as Fridsma points out, core competencies can help to distinguish professions as well as define how they overlap.

Another point raised was the importance of topics such as AI and the subdomain of machine learning (ML). Both AI and ML are mentioned prominently in the Topol review23 and are receiving serious consideration and funding for their application to healthcare. AI is not a new concept and has been around since the 1950s.24 Many algorithms are referred to as a black box because the exact workings of certain algorithms (eg, neural networks) can be complex and unpredictable.24 Despite their current popularity, it is unclear whether AI and ML will continue to hold the same prominent position in future. The framework therefore incorporates AI and ML into the broader category of ‘Analytical Methodologies and Applications’ rather than a section in its own right. Many statistical methods are still widely used in medical contexts, and ‘statistical learning’, which is focused on prediction as well as supervised and unsupervised modelling, spans the domains of machine learning and statistics.25

One of the challenges going forward will be the implementation of the framework. This will include tasks such as developing operational definitions for the competencies concerning their related knowledge, skills, attitudes and behaviours as well as defining how such competence will be measured for individuals.15 The CCF provides the initial foundation for defining the requirements to be a clinical informatician in the UK and provides a starting point for professional development in this fast-moving area of healthcare.

As the interviews highlight, careful consideration is required when linking the framework to developmental stages and/or academic levels. Various stage models of professional development exist26–28 and mostly originate from the field of cognitive psychology. These focus on moving through various developmental stages (eg, novice, advanced beginner, competent, proficient and expert29). Stage models are often based on attributes including attitudes, skills and underpinning knowledge which are supplemented with additional skills and knowledge from the workplace.30

Criticism of such models suggest that some aspects of professional skill development, such as the skill being developed in terms of understanding and practice, may be concealed as the primary focus is on the developmental stages themselves.30

More recent work considers models that account for multiple dimensions such as progression of skills over time and embodied understanding of practice.30 This reflects the difference between skill progression over time that is possible without such understanding of practice which differentiates experienced and expert practitioners. Other models consider extra dimensions, such as functional and foundational competency domains and stages of professional development.31

It can also be difficult to ascertain exactly which skills, knowledge and attitudes differentiate between being an expert or not. Given that there is a connection between the individual and their characteristics and their professional practice,30 applying a staged model can be complex, especially when considering applying this to the various clinical groups that form the clinical informatics community with their underlying cultures and contexts. This may have to be applied in slightly different ways to account for these variations in context.

The FCI have developed a working group that seeks to address such issues, including the mapping of the framework to academic courses, accreditation using the framework and other details around its validation and implementation. Progress of the working group is reported publicly online via the FCI website giving the option for more end users and stakeholders to comment and feedback on the implementation as well as future revisions.

Conclusion

We used a mixed-methods iterative approach, using both qualitative and quantitative methods to develop a core competency framework for clinical informatics, consisting of 111 competencies spread over 6 principle domains with 36 subdomains. The domain around working with data and analytical methods seems to be the most contentious, due to variations in the extent to which practitioners are involved with the practical aspects of producing systems and coding. The overlap between fields like healthcare data science and informatics blurs at this point. Competency frameworks can be used to define these overlaps and also differences. Finally, consideration and care is needed regarding the application of such frameworks for professionals that already have a primary clinical competency set to meet to avoid burnout and to ensure all groups can meet these competency requirements, even if their main area of work spans only one or a few of the primary domains. We recommend an annual review process for the framework moving forward to ensure its continuing relevance.