Towards understanding healthcare professionals’ adoption and use of technologies in clinical practice: using Q-methodology and models of technology acceptance ================================================================================================================================================================ * Muhammad Awwal Ladan * Heather Wharrad * Richard Windle ## Abstract **Background** Globally, technologies have been recognised to improve productivity across different areas of practice including healthcare. This has been achieved by the expansion of computers and other forms of information technologies (ITs). Despite this advancement, there have also been growing challenges to the adoption and use of these technologies within practice, sometimes with unintended or unexpected consequences. However, the barriers and drivers to IT and, more specifically, e-health adoption within healthcare are little understood, especially in areas such as Sub-Saharan Africa (SSA) where e-health adoption is relatively new. **Methodology** This paper describes a pilot study to develop and validate sample statements for use within a later substantive Q-methodology study. The aim of the main study was to understand factors that influence healthcare professionals’ (HCPs) attitudes towards IT adoption and use in SSA. We report on the use of this methodology to explore the subjectivity of HCPs together with the models of technology acceptance [technology acceptance model (TAM) and the unified-theory of acceptance and use of technology (UTAUT)] used in combination for the first time. **Results** Following various stages and mapping of the two models of technology acceptance used, 46 statements were developed at the end of the pilot study. These statements were grouped into six themes to capture the constructs of the two models used in the study. **Conclusion** Findings suggest that it is possible to use TAM and UTAUT to develop a comprehensive set of statements. These statements reflect choices that HCPs consider on IT/e-health adoption and use in SSA which can be used in a Q-study. * e-health * Q-methodology * healthcare professionals * technology acceptance model * unified theory of acceptance and use of technology * Sub-Saharan Africa ## INTRODUCTION There is poor adoption of technology among healthcare professionals (HCPs)1,2 in Sub-Saharan Africa (SSA), despite recognising that technology can improve healthcare delivery and outcomes. This paper explores HCPs’ views on the adoption and use of technologies within their clinical practices. The technology acceptance model (TAM) and the unified theory of acceptance use of technology (UTAUT) are used in this study to understand the views of HCPs working in the clinical area on the factors that influence their adoption and use of technologies within their clinical practice. Technologies improve productivity across different areas of practice including healthcare and this has been achieved due to expansion of computers and other forms of information technologies (ITs).3 Information and communication technologies (ICTs) use and specifically innovations in e-health have the potential to deliver better healthcare through improved health strategies directed towards patient safety, healthcare collaborations, efficiency and healthcare research.4 The Royal College of Nursing (RCN) defined e-health as ‘concerned with promoting, empowering and facilitating health and wellbeing with individual, families and communities and the enhancement of professional practice using information management and information and communication technologies’ (RCN 2017). The RCN also identifies e-health to cover electronic patient records, electronics communication with patients and professionals, telehealth/telecare, information management, information governance and personal health records. However, there remains an on-going challenge of the adoption and use of these technologies within healthcare practice.3 The implementation of these technologies within healthcare practice has met with some barriers and facilitators.3,5,6 Harrison *et al*.7 identified that despite positive intentions unintended or unexpected consequences influence the perspectives of the HCPs that use these technologies. Consequently, there has been increased interest in research into the factors that influence e-health adoption and use within healthcare,5 though very few researchers have used a theory or model to explore this area (Table 1).6,8,9 Though various models and theories exist to explore ICT and e-health adoption and use among HCPs,8,10–12 TAM and UTAUT13,14 have been identified as the most widely used.15 They have been described as robust16–19 in understanding IT acceptance and use more generally. Holden and Karsh14 have identified some criticisms of use of these two models in healthcare. These includes researchers inability to standardise the models’ terminologies such interchanging ‘acceptance and adoption’ incorrectly and the great variability that exist between such studies [p. 167]. Despite this, both models evidently have wider applications in e-health research.14 View this table: [Table 1](http://informatics.bmj.com/content/25/1/27/T1) Table 1 Models of technology acceptance in healthcare studies Most studies tested the strength of the model’s constructs to determine which construct has more impact in the final adoption of e-health among HCPs6,9,17,20–22 rather than exploring the views as end users of these technologies. Our study uses Q-methodology, a method for the study of subjectivity to explore the perspectives of the HCPs on what influences their adoption and use of technology within their clinical practice utilising both the TAM and UTAUT as theoretical frameworks. This paper describes the previously unreported process of combining TAM and UTAUT with the Q-methodology approach. ## MODELS OF TECHNOLOGY ACCEPTANCE AND USE ### The technology acceptance model – TAM TAM was proposed by Davis in 198623 to explain the potential user’s behavioural intention to use a technological innovation. TAM was developed from the Theory of Reasoned Action (TRA) of Martin Fishbein and Icek Ajzen (1975), though less general because it applies specifically to explain computer usage behaviour.23,24 The TRA asserted that both the attitude and subjective norm have an impact on behavioural intention, which in turn affects how people perform an action.25 Davis *et al*.23 stated that the goal of TAM was to provide an explanation of the determinants of computer acceptance across a range of contexts. It is also capable of explaining user behaviour across a broad range of end-user computing technologies and populations, while at the same time being both parsimonious and theoretically justified. The model was based on the premise that individuals’ ICT use is determined by two major variables: perceived usefulness (PU) – which is ‘the prospective user’s subjective probability that using a specific ICT will increase one’s job performance within an organisational context and perceived ease of use (PEOU) – defined as the degree to which the prospective user expects the target system to be free of effort’.13 The model suggests that actual technology usage is determined by behavioural intention to use the technology by individuals. This is affected by their PU, PEOU of the technology and their attitude towards it.13,17,19,23,26,27 Furthermore, DongPing and LianJin13 reported that the TAM theorises that the effects of external variables such as system characteristics, development process, and training are mediated by PU and PEOU. The external variables typically included system or ICT characteristics, user training in the ICT concerned, user participation in design and the nature of the implementation process.27 TAM robustness is seen as its ability to be applied to different technologies, under different situations, with different control factors (gender and type of organisation) and different subjects.28 According to Rogers (1995) cited in,29 the features of a technology that determines its adoption (and use) are its relative advantage, compatibility, complexity, trialability and observability. User exposure to these determinants generates reactions, which Venkatesh and Davis30 referred to as self-reported use. These identified reactions tend to be attributed to the user intention to use and actual use of the said technology.5,31–35 However, over the years, the model has been extended twice to capture factors that are believed to be critical in explaining usage behaviour.13 The first extension was identified as TAM2 by Venkatesh and Davis30 and was aimed at including key determinants to the model’s constructs and to understand the effects of the new determinants in the original model.13,30 This led to the removal of the ‘attitude’ construct and the addition of determinants to PU and intention to use constructs (Figure 1).13,27,30,36 Venkatesh and Bala37 further extended that TAM was used to explore organisational perspectives on decision making relating to the use of ITs in the work area. This led to the addition of determinants of PEOU of the TAM2 model. This revision by Venkatesh and Bala37 produced the TAM3 (Figure 1), which was expected to provide a ‘nomological network of determinants to individuals’ adoption and use of a technology’ (p. 301). ![Figure 1](http://informatics.bmj.com/https://informatics.bmj.com/content/bmjhci/25/1/27/F1.medium.gif) [Figure 1](http://informatics.bmj.com/content/25/1/27/F1) Figure 1 The technology acceptance model13,37 ### The unified-theory-of-acceptance-and-use-of-technology (UTAUT) The UTAUT (Figure 2) developed by Venkatesh *et al*.38 unified the eight prominent competing ICT acceptance and use models. The unified models include TRA, TAM, the motivational model, the theory of planned behaviour (TPB), a model combining the TAM and TPB (C-TAM-TPB), the model of PC utilisation, the innovation diffusion theory and the social cognitive theory.13,39,40 The model was empirically tested and accounted for 70% of the variance in usage intention and it is suggested to be a behavioural model that aims to explain the behaviour of people or organisations in their use of ICT.13,38 UTAUT has four key constructs: performance expectancy (the degree to which an individual believes that using a system will help him/her attain gains in job performance), effort expectancy (degree of ease associated with the use of the system or ICT resource), social influence (degree to which an individual perceives that ‘important others’ believe he/she should use the technology) and facilitating conditions (degree to which an individual believes that an organisational and technical infrastructure exist to support the use of the technology). The model also has four moderating variables, which are age, gender, experience and voluntariness of use.13,38 ![Figure 2](http://informatics.bmj.com/https://informatics.bmj.com/content/bmjhci/25/1/27/F2.medium.gif) [Figure 2](http://informatics.bmj.com/content/25/1/27/F2) Figure 2 Unified theory of acceptance and use of technology38 ## LITERATURE REVIEW ### TAM and UTAUT in healthcare The TAM has attracted a lot of empirical and theoretical attention over the years of its existence but despite being the popular model for ICT adoption and use, it is still not seen as a healthcare specific model. Some have further argued that if used in its generic form, it may fail to capture or even contradict some unique contextual features of computerised healthcare, that is, indicating a significant gap in knowledge.14,17,41 Many studies have attempted to utilise the TAM in explaining or predicting ICT adoption in healthcare by applying it to specific healthcare applications20,22,26,42 or modifying the model to test new variables or hypotheses.6,18,20,43 Aggelidis and Chatzoglou42 used a modified TAM to examine health information systems (HIS) among HIS users by testing 23 hypotheses within the model using structural equation modelling (SEM) as seen in Table 1. They reported that the relationships between the initial TAM constructs hold and are significant. However, despite using a quantitative approach, the researchers identified among the limitations that a small and disproportionate sample was used, and that also emphasis was more on the information system (rather than the views of the users). Also, Gagnon *et al*.20 used a modified TAM to examine factors that influence the decision of HCPs’ use of an e-health resource. They identified a response rate of 39.7% and using logistic regression analysis, they reported that the TAM is a good model for predicting HCPs’ intention to use an e-health resource. Also, the researchers acknowledged that they adopted a questionnaire that has already been used in another TAM study and has not undergone test–retest reliability as consistent with quantitative studies. Emphasis within the study was also on the e-health resource and not the views of the users within the health sector. This study was similar to the study by Chismar and Wiley-Patton17 which also used a regression analysis to examine physicians’ acceptance of technology using the TAM modified by Venkatesh and Davis30 and adopting the questionnaire associated with the model. They have also identified as a limitation to their study, a small sample was used. Despite the stated limitation, they have identified that the relationship between the constructs within the TAM they used holds except for PEOU and subjective norm. In similar TAM related research, Bennani and Oumlil9 explored factors that influence IT acceptance by nurses. They modified the initial TAM model by adding two new constructs: trust and image. They tested 11 hypotheses using quantitative testing like those identified previously and used SEM. They also reported that the relationships within the constructs for TAM hold except for PEOU and the ‘trust’ construct they added. Like studies involving the TAM, UTAUT research in healthcare mainly used quantitative approaches. Most researchers21,39,44 tested the empirical strengths of the model in their studies. Kijsanayotin *et al*.39 employed a modified UTAUT to explore factors influencing health IT (HIT) adoption in community health centres in Thailand. They employed quantitative methods using SEM like in previously mentioned TAM studies, and their emphasis was on the HIT and model rather than the views of the participants on the factors influencing their adoption of HIT. Similarly, Sharifian *et al*.44 also used the UTAUT to identify factors influencing nurses’ acceptance of e-health resources in Iran again using a quantitative approach to identify the factors which influence e-health resource acceptance by testing the strength of the relationship within the model. The personal views of the participants regarding the e-health acceptance were not explored. Venkatesh *et al*.21 also utilised the UTAUT to explore an e-health resource adoption and use among physicians. SEM was used to predict the strength of the relationship within the model and although they established a relationship with the constructs with an explained variance of 44% this compared to 76% which was established when using the model in a previous study outside healthcare. However, they justified that the UTAUT should be integrated with other theories to enrich it when adopting it to the healthcare context. Therefore, TAM and UTAUT have been the most consistently used models in exploring ICT within healthcare due to their reliable and validated robustness in technology adoption/acceptance and use literature. Despite this, however, more emphasis has been on establishing the relationships within the constructs of both models rather than the subjective views of the users’ interaction with the e-health resources. Some researchers11,45,46 suggested a move from the traditional quantitative methodology use in technology acceptance and use to a mixed approach. They argued that by using mixed methods in such studies, an understanding of the context will develop through the opinions of people who use such e-health resources. Others such as Venkatesh *et al*.21 have suggested using more than one model to explore factors that influence technology acceptance and use among HCPs. One such approach is Q-methodology which forms the basis of this paper. The paper aims to use both TAM and UTAUT to develop a comprehensive set of statements that reflect the views of HCPs on adoption and use technologies in clinical practice in SSA. ### Q-methodology Ami-Narh and Williams11 used a mixed method approach called Q-methodology48–50 to understand participants’ perspectives on e-health. This methodology has been described as the scientific study of subjectivity.51–53 It was developed by William Stephenson to explore individuals’ perception relating to an issue of discourse.54,55 This methodology allows participants to interpret items relating to an issue based on their own practice.51,55–57 It combines both qualitative and quantitative techniques. Each participant completes a sorting exercise where he/she will rank order a set of items in the form of statements drawn from the discourse (what is known about the topic) according to how they perceive it influences their practice. Q-methodology is less prevalent in the technology literature, where survey studies are frequently used.10,58,59 Q-methodology places emphasis on understanding the viewpoints of participants by focusing on their subjective standpoint on issues affecting them and how this standpoint is shared with other participants within the same study environment. Barker55 identified that the traditional methods of studying subjectivities such as focus group discussions and interviews or quantitative surveys pose difficulties in data reduction to a meaningful account. She also argued that using surveys presents the viewpoints of the participants into a homogenous whole rather than shared or individual perspectives. Thus, the strength of Q-methodology lies in its approach to the study of subjectivity and its limitation is evident in its non-generalisable findings. However, Thomas and Baas60 suggested that the concept generated from a Q-study could be used beyond the population of study. This will be the first time Q-methodology has been employed in conjunction with the models of technology acceptance to explore HCPs’ adoption and use of e-health within clinical practice. ## METHODS ### The process of Q-methodology Various studies have outlined the steps involved in conducting a Q-study.53,55,57,59,61,62 Figure 3 shows the process from the formulation of a research question to the generation of a pool of relevant statements related to the context of study. This is called the concourse (Figure 4). The concourse is subsequently ‘filtered’ to develop the final sample items which in Q-studies are identified as the Q-set or the Q-sample.58,61,62 These statements are then provided to the participants to rank order. This ranking of the statements which is also called the Q-sorting is usually followed up by a post-sort interview53,63 to discuss the sorting exercise. The sorting exercise is then followed by a quantitative technique, that is, factor analysis which is called Q-factor analysis within Q-studies (because of its inverted factor analysis technique). The final stage of the Q-process is the interpretation which is based on how the Q-set items are ranked on the final shared opinions/viewpoints (factors) of participants or Q-sorts after Q-analysis. The interpretation is usually done in relation to statements or viewpoints which are shared. These are also referred to as consensus/agreement statements and the distinguishing statements or statements which characterise each factor when compared to another factor.51,52,62 This interpretation has recently been improved on with the use of a crib sheet.53 In using the crib sheet (representation of ranked statements viewpoint), the researcher adopts a holistic approach to characterise a shared viewpoint/factor based on how statement items are rank-ordered within it. There are available software dedicated to the analysis of Q-studies such as the PQMethod, Ken Q-analysis and PCQ for Windows. The analysis involves correlation, by-person factor analysis and computation of factors, respectively. During correlation, all finished sorts/Q-sorts are correlated with each other and then followed by an extraction process62 to identify factors (similar/shared viewpoints). The extraction could either be done using the centroid method or the principal component analysis (PCA), though Watts and Stenner53 state most Q-researchers opt for the former because of its ‘permissiveness’ in Q-studies and because it is more traditionally acceptable than PCA (p. 99). It should be noted that both methods provide similar results.53 After extraction, the factors are then rotated using either the Varimax or manual/judgemental rotation64 to understand the perspective of the participants’ viewpoint within a factor. ![Figure 3](http://informatics.bmj.com/https://informatics.bmj.com/content/bmjhci/25/1/27/F3.medium.gif) [Figure 3](http://informatics.bmj.com/content/25/1/27/F3) Figure 3 Q-methodology process ![Figure 4](http://informatics.bmj.com/https://informatics.bmj.com/content/bmjhci/25/1/27/F4.medium.gif) [Figure 4](http://informatics.bmj.com/content/25/1/27/F4) Figure 4 Concourse development within the study Since the pilot study aimed to refine and validate the statement items (steps 1 and 2 in Figure 3), complete Q-factor analysis (steps 3–6 in Figure 3) is not reported here. Nevertheless, the PQMethod software will be used for the main study during analysis because it is freely available and has been validated by most Q-methodology researchers.52,53 Moreover, the interpretation of the final factors will be done using the crib sheet and the distinguishing and consensus characteristics of the factors/viewpoints as already mentioned above. It is an iterative process which will be complemented by the post-sort interview data that will be collected during the main study. The following sections will describe the Q-pilot study. ### Developing the concourse The concourse in Q is literally all that has been said or written about the subject or topic of discourse. The concourse may include paintings, photographs or even musical items.59 Prior to the development of the concourse, a research question must be identified to guide the entire research process. In doing so, the research question guiding this study was stated as: What are the factors that influence ICT use among HCPs in the clinical area in SSA? Since Q-methodology is an exploratory process,64 participants were able to identify the barriers and motivators to their use of ICT in healthcare or e-health resources. In developing the concourse for this study, the literature on models of technology acceptance, barriers and facilitators to ICT use in healthcare and informal interviews with experts in e-health were employed to build a rich pool of statements that relate to the research question. At the end of the concourse development, 170 statement items were identified with six themes: PU/performance expectancy, PEOU/effort expectancy, social influence/ subjective norm, facilitating conditions, individual differences and behavioural intention (Figure 5). These themes, which were drawn mainly from the TAM and UTAUT, were adopted to guide the final statement selection for the Q-study. ![Figure 5](http://informatics.bmj.com/https://informatics.bmj.com/content/bmjhci/25/1/27/F5.medium.gif) [Figure 5](http://informatics.bmj.com/content/25/1/27/F5) Figure 5 Themes/constructs of models of technology acceptance and Q-methodology ### Refining the statements Traditionally, in Q-studies, the process of refining the statements involves both face and content validation of the concourse statements.59 The face validation involves refining the statements for clarity, readability and repetition. Content validity is done by experts within the subject of study. They check the items for ambiguity, applicability and completeness of the items within the context of study. This is usually followed up by a pilot study to understand if the items will capture what they are meant to capture during sorting by the participants. In this study, statement refining was done by volunteer HCPs and experts within the host university. The sample statement items were reduced to 46 at the end of both face and content validation by the volunteers. ### The participants and the ranking process During the pilot study, volunteers were recruited to rank order the statement items. As with most qualitative studies, the participants in Q-studies (also called P-set or P-sample) are identified through purposive sampling. This means that participants who fall within the inclusion criteria of the study (must be HCPs who have had the experience of using IT within clinical practice). Five HCPs within the host university were recruited to participate in the pilot study. Each of the volunteers was provided with the 46 statement items printed on laminated cards of 5 cm × 8 cm, an instruction on how to do the sorting and an A0 size cardboard sheet with a defined sorting grid to rank the items based on how they agree or disagree with them along a 13-scale grid (Figure 6). At the end of the sorting exercise with the volunteers, a short interview was done to understand their views about the sorting process including if the instruction was clear and understandable, if the statements were also clear and complete (reflect all aspect of e-health). ![Figure 6](http://informatics.bmj.com/https://informatics.bmj.com/content/bmjhci/25/1/27/F6.medium.gif) [Figure 6](http://informatics.bmj.com/content/25/1/27/F6) Figure 6 A 13-scale sorting grid ## RESULTS Though all participants within the pilot reported that both the instruction and statements were clear and understandable, one statement was modified from ‘*Using clinical information systems increases my chance of getting a raise’* was modified to ‘*Using clinical information systems increases my chance of getting a praise or reward’*. In addition, another statement ‘*Using clinical information systems enhances my effectiveness in my job’* was removed, while another statement (*My routine tasks prevent me from having time to use the clinical information systems)* was suggested by a participant to make the items complete. This made the total Q-sample items to be 46 distributed across the six themes (Table 2). View this table: [Table 2](http://informatics.bmj.com/content/25/1/27/T2) Table 2 Themes and the number of statements The final 46 sample statements representing the Q-set will be printed on laminated cards as described above and provided to the HCPs within the study area in SSA to rank order. The ranking will be with regard to their respective agreement or disagreement of the statements in line with their clinical practice along the sorting grid (Figure 6). Together with a post-ranking interview, this will provide understanding on the shared perspectives (both similarities and differences) that the HCPs have about IT/e-health adoption and use in clinical practice in SSA. ## DISCUSSION Previous research studies have mainly used quantitative methodologies to demonstrate the relationship between e-health and HCPs. Though this approach may be important in testing new technologies, it might not be appropriate in understanding the views of the HCPs using such technologies.11 Thus, the use of both TAM and UTAUT together with Q-methodology as a new approach will provide an understanding of HCPs views about e-health and ICT in general within their respective clinical practices. This provides an alternative to the traditional methods of surveys and qualitative methods where limitations have been identified.55 It will also provide insight to unintended/undesired outcomes that may arise due to e-health adoption and use by HCPs in the clinical area in SSA. Understanding the views of these HCPs after the main study will help inform decisions and address challenges about IT/e-health in clinical practice within this population in SSA or in a wider context. ## CONCLUSION This paper shows how TAM and UTAUT provide a well-theorised framework on which to base a comprehensive set of statements (Q-sets) exploring the views HCPs have on e-health adoption. This Q-set of 46 statements reflects the choices that HCPs consider and will be used in a study of HCPs e-health adoption in clinical practice in SSA. * Received February 1, 2018. * © 2018 The Author(s). Published by BCS, The Chartered Institute for IT under Creative Commons license http://creativecommons.org/licenses/by/4.0/ ## References 1. 1.Mostert-Phipps N, Pottas D, Korpela M. Guidelines to encourage the adoption and meaningful use of health information technologies in the South African healthcare landscape. Studies in Health Technology and Informatics 2013;192:147–51. 2. 2.Akanbi MO, Ocheke AN, Agaba PA, Daniyam CA, Agaba EI, Okeke EN, et al. Use of electronic health records in sub-Saharan Africa: progress and challenges. Journal of Medicine in the Tropics 2012;14(1):1–6. pmid:25243111; pmcid:PMC4167769 [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=25243111&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 3. 3.Gagnon MP, Desmartis M, Labrecque M, Car J, Pagliari C, Pluye P, et al. Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals. Journal of Medical Systems 2012;36(1):241–77. doi:10.1007/s10916-010-9473-4pmid:20703721; pmcid:PMC4011799 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1007/s10916-010-9473-4&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=20703721&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 4. 4.Qidwai W. Advances in information and communication technology (ICT): issues, challenges and opportunities for health care professionals. Journal of the College of Physicians and Surgeons Pakistan 2011;21(11):651–53. 5. 5.Gururajan R, Hafeez-Baig A. An empirical study to determine factors that motivate and limit the implementation Of ICT in healthcare environments. BMC Medical Informatics and Decision Making 2014;14(1):98. doi:10.1186/1472-6947-14-98pmid:25540040; pmcid:PMC4391394 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1186/1472-6947-14-98&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=25540040&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 6. 6.Kowitlawakul Y. The technology acceptance model: predicting nurses’ intention to use telemedicine technology (eICU). Computers Informatics Nursing 2011;29(7):411–18. doi:10.1097/NCN.0b013e3181f9dd4apmid:20975536 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1097/NCN.0b013e3181f9dd4a&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=20975536&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 7. 7.Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care—an interactive sociotechnical analysis. Journal of the American Medical Informatics Association: JAMIA 2007;14(5):542–49. doi:10.1197/jamia.M2384pmid:17600093; pmcid:PMC1975796 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1197/jamia.M2384&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=17600093&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 8. 8.Holden RJ. Physicians’ beliefs about using EMR and CPOE: in pursuit of a contextualized understanding of health IT use behavior. International Journal of Medical Informatics 2010;79(2):71–80. doi:10.1016/j.ijmedinf.2009.12.003pmid:20071219; pmcid:PMC2821328 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.ijmedinf.2009.12.003&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=20071219&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000274568900001&link_type=ISI) 9. 9.1. Bennani AE, 2. Oumlil R . IT acceptance by nurses in Morocco: application of technology acceptance model20132013 International Conference on Information Society (i-Society)24–6. 10. 10.Petit dit Dariel O. Exploring e-learning Adoption in Nurse Education: A Socio-Cultural Case Study Using Q and Bourdieu [Research] 2011. Nottingham, UK: University of Nottingham. 11. 11.Ami-Narh JT, Williams PAH. A revised UTAUT model to investigate E-health acceptance of health professionals in africa. Journal of Emerging Trends in Computing and Information Sciences 2012;3(10):1383–91. 12. 12.Mohamadali NAK. Exploring New Factors and the Question of ‘Which’ in User Acceptance Studies of Healthcare Software 2012. Nottingham, UK: University of Nottingham. 13. 13.1. DongPing T, 2. LianJin C . A review of the evolution of research on information technology acceptance model20112011 International Conference on Business Management and Electronic Information13–15 May588–91. 14. 14.Holden RJ, Karsh B-T. The technology acceptance model: its past and its future in health care. Journal of Biomedical Informatics 2010;43(1):159–72. doi:10.1016/j.jbi.2009.07.002pmid:19615467; pmcid:PMC2814963 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.jbi.2009.07.002&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=19615467&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000274309800017&link_type=ISI) 15. 15.Chau PYK, Hu PJ-H. Investigating healthcare professionals’ decisions to accept telemedicine technology: an empirical test of competing theories. Information & Management 2002;39(4):297–311. doi:10.1016/S0378-7206(01)00098-2 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/S0378-7206(01)00098-2&link_type=DOI) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000173105000005&link_type=ISI) 16. 16.Venkatesh V, Davis FD. A model of the antecedents of perceived ease of use: development and test. Decision Sciences 1996;27(3):451–81. doi:10.1111/j.1540-5915.1996.tb01822.x [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1111/j.1540-5915.1996.tb01822.x&link_type=DOI) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=A1996WH61100003&link_type=ISI) 17. 17.1. Chismar WG, 2. Wiley-Patton S . Does the extended technology acceptance model apply to physicians200336th Annual Hawaii International Conference on System Sciences6–9 Jandoi:10.1109/HICSS.2003.1174354 18. 18.Hu PJ, Chau PYK, Liu Sheng OR, Kar Yan T. Examining the technology acceptance model using physician acceptance of telemedicine technology. Journal of Management Information Systems 1999;16(2):91–112. doi:10.1080/07421222.1999.11518247 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1080/07421222.1999.11518247&link_type=DOI) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000084745500007&link_type=ISI) 19. 19.Lee Y, Kozar KA, Larsen KRT. The technology acceptance model: past, present, and future. Communications of the Association for Information Systems 2003;12(1):752–80. 20. 20.Gagnon MP, Orruño E, Asua J, Abdeljelil AB, Emparanza J. Using a modified technology acceptance model to evaluate healthcare professionals’ adoption of a new telemonitoring system. Telemedicine Journal and e-health 2012. Accessed 13 April 201718(1):54–9. Available from: [http://onlinelibrary.wiley.com/o/cochrane/clcentral/articles/633/CN-00840633/frame.html](http://onlinelibrary.wiley.com/o/cochrane/clcentral/articles/633/CN-00840633/frame.html); [http://online.liebertpub.com/doi/abs/10.1089/tmj.2011.0066](http://online.liebertpub.com/doi/abs/10.1089/tmj.2011.0066) [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1089/tmj.2011.0066&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=22082108&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 21. 21.1. Venkatesh V, 2. Sykes TA, 3. Xiaojun Z . ‘Just what the doctor ordered’: a revised UTAUT for EMR system adoption and use by doctors20112011 44th Hawaii International Conference on System Sciences4–7 Jan1–10. 22. 22.Yarbrough AK, Smith TB. Technology acceptance among physicians: a new take on TAM. Medical Care Research and Review 2007;64(6):650–72. doi:10.1177/1077558707305942pmid:17717378 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1177/1077558707305942&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=17717378&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000251090500002&link_type=ISI) 23. 23.Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Management Science 1989;35(8):982–1003. doi:10.1287/mnsc.35.8.982 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1287/mnsc.35.8.982&link_type=DOI) 24. 24.Davis FD, Bagozzi RP, Warshaw PR. Extrinsic and intrinsic motivation to use computers in the workplace. Journal of Applied Social Psychology 1992;22(14):1111–32. doi:10.1111/j.1559-1816.1992.tb00945.x [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1111/j.1559-1816.1992.tb00945.x&link_type=DOI) 25. 25.Schepers J, Wetzels M. A meta-analysis of the technology acceptance model: investigating subjective norm and moderation effects. Information and Management 2007;44(1):90–103. doi:10.1016/j.im.2006.10.007 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.im.2006.10.007&link_type=DOI) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000243743300008&link_type=ISI) 26. 26.Ketikidis P, Dimitrovski T, Lazuras L, Bath PA. Acceptance of health information technology in health professionals: an application of the revised technology acceptance model. Health Informatics Journal 2012;18(2):124–34. doi:10.1177/1460458211435425pmid:22733680 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1177/1460458211435425&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=22733680&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 27. 27.Chuttur M. Overview of the technology acceptance model: origins, developments and future directions2009. Bloomington, USA: Indiana UniversityWorking Papers on Information Systems 28. 28.Yousafzai SY, Foxall GR, Pallister JG. Technology acceptance: a meta-analysis of the TAM: Part 1. Journal of Modelling in Management 2007;2(3):251–80. doi:10.1108/17465660710834453doi:10.1108/17465660710834462 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1108/17465660710834453&link_type=DOI) 29. 29.England I, Stewart D, Walker S. Information technology adoption in health care: when organisations and technology collide. Australian Health Review 2000;23(3):176–85. doi:10.1071/AH000176pmid:11186051 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1071/AH000176&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=11186051&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 30. 30.Venkatesh V, Davis F. A theoretical extension of the technology acceptance model: four longitudinal field studies. Management Science 2000;46(2):186–204. doi:10.1287/mnsc.46.2.186.11926 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1287/mnsc.46.2.186.11926&link_type=DOI) 31. 31.Gagnon M-P, Nsangou É-R, Payne-Gagnon J, Grenier S, Sicotte C. Barriers and facilitators to implementing electronic prescription: a systematic review of user groups’ perceptions. Journal of the American Medical Informatics Association 2014;21(3):535–41. doi:10.1136/amiajnl-2013-002203pmid:24130232; pmcid:PMC3994867 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1136/amiajnl-2013-002203&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=24130232&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 32. 32.Haluza D, Jungwirth D. ICT and the future of health care: aspects of health promotion. International Journal of Medical Informatics 2015;84(1):48–57. doi:10.1016/j.ijmedinf.2014.09.005pmid:25293532 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.ijmedinf.2014.09.005&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=25293532&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 33. 33.Lluch M. Healthcare professionals’ organisational barriers to health information technologies-a literature review. International Journal of Medical Informatics 2011;80(12):849–62. doi:10.1016/j.ijmedinf.2011.09.005pmid:22000677 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.ijmedinf.2011.09.005&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=22000677&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 34. 34.Saigi-Rubio F, Torrent-Sellens J, Jimenez-Zarco A. Drivers of telemedicine use: comparative evidence from samples of Spanish, Colombian and Bolivian physicians. Implementation Science 2014;9(1):128. doi:10.1186/s13012-014-0128-6pmid:25293651; pmcid:PMC4195871 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1186/s13012-014-0128-6&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=25293651&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 35. 35.Idowu B, Ogunbodede E, Idowu B. Information and communication technology in Nigeria: The health sector experience. Journal of Information Technology 2003;3(2):69–76. 36. 36.King WR, He J. A meta-analysis of the technology acceptance model. Information & Management 2006;43(6):740–55. doi:10.1016/j.im.2006.05.003 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.im.2006.05.003&link_type=DOI) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000240573500005&link_type=ISI) 37. 37.Venkatesh V, Bala H. Technology Acceptance Model 3 and a research agenda on interventions. Decision Sciences 2008;39(2):273–315. doi:10.1111/j.1540-5915.2008.00192.x [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1111/j.1540-5915.2008.00192.x&link_type=DOI) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000255752600006&link_type=ISI) 38. 38.Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. MIS Quarterly 2003;27(3):425–78. doi:10.2307/30036540 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.2307/30036540&link_type=DOI) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000185196400005&link_type=ISI) 39. 39.Kijsanayotin B, Pannarunothai S, Speedie SM. Factors influencing health information technology adoption in Thailand’s community health centers: applying the UTAUT model. International Journal of Medical Informatics 2009;78:404–16. doi:10.1016/j.ijmedinf.2008.12.005pmid:19196548 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.ijmedinf.2008.12.005&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=19196548&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 40. 40.Phichitchaisopa N, Naenna T. Factors affecting the adoption of healthcare information technology. Experimental and Clinical Sciences Journal 2013;12:413–36. pmid:26417235; pmcid:PMC4566918 [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=26417235&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 41. 41.Melas CD, Zampetakis LA, Dimopoulou A, Moustakis V. Modeling the acceptance of clinical information systems among hospital medical staff: an extended TAM model. Journal of Biomedical Informatics 2011;44(4):553–64. doi:10.1016/j.jbi.2011.01.009pmid:21292029 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.jbi.2011.01.009&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=21292029&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 42. 42.Aggelidis VP, Chatzoglou PD. Using a modified technology acceptance model in hospitals. International Journal of Medical Informatics 2009;78(2):115–26. doi:10.1016/j.ijmedinf.2008.06.006pmid:18675583 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.ijmedinf.2008.06.006&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=18675583&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 43. 43.Jimoh L, Pate MA, Lin L, Schulman KA. A model for the adoption of ICT by health workers in Africa. International Journal of Medical Informatics 2012;81(11):773–81. doi:10.1016/j.ijmedinf.2012.08.005pmid:22986218 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.ijmedinf.2012.08.005&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=22986218&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 44. 44.Sharifian R, Askarian F, Nematolahi M, Farhadi P. Factors influencing nurses’ acceptance of hospital information systems in Iran: application of the unified theory of acceptance and use of technology. Health Information Management Journal 2014;43(3):23–8. doi:10.1177/183335831404300303 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1177/183335831404300303&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=27009793&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 45. 45.Wu PF. A mixed methods approach to technology acceptance research. Journal of the Association for Information Systems 2012;13(3):172–87. 46. 46.Schwarz A, Chin WW, Hirschheim R, Schwarz C. Toward a process-based view of information technology acceptance. Journal of Information Technology 2014;29(1):73–96. doi:10.1057/jit.2013.31 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1057/jit.2013.31&link_type=DOI) 47. 47.Holden RJ, Brown RL, Scanlon MC, Karsh B-T. Modeling nurses’ acceptance of bar coded medication administration technology at a pediatric hospital. Journal of the American Medical Informatics Association: JAMIA 2012;19(6):1050–8. doi:10.1136/amiajnl-2011-000754pmid:22661559; pmcid:PMC3534453 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1136/amiajnl-2011-000754&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=22661559&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 48. 48.Onwuegbuzie AJ, Frels RK. Using Q methodology in the literature review process: a mixed research approach. Journal of Educational Issues 2015;1(2):90–109. doi:10.5296/jei.v1i2.8396 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.5296/jei.v1i2.8396&link_type=DOI) 49. 49.Ramlo S. Mixed method lessons learned from 80 years of Q methodology. Journal of Mixed Methods Research 2016;10(1):28–45. [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1177/1558689815610998&link_type=DOI) 50. 50.O’Leary K, Eschler J, Kendall L, Vizer LM, Ralston JD, Pratt W. Understanding design tradeoffs for health technologies: a mixed-methods approach201533rd Annual ACM Conference on Human Factors in Computing SystemsSeoul, South Korea4151–60. ACMdoi:10.1145/2702123.2702576 51. 51.Brown SR. Political Subjectivity: Applications of Q Methodology in Political Science 1980. New Haven, CT: Yale University. 52. 52.Webler T, Danielson S, Tuler S. Using Q Method to Reveal Social Perspectives in Environmental Research 2009. Greenfield, MA: Social and Environmental Research InstituteAvailable from: [http://www.seri-us.org/sites/default/files/Qprimer.pdf](http://www.seri-us.org/sites/default/files/Qprimer.pdf) 53. 53.Watts S, Stenner P. Doing Methodological Research: Theory, Method and Interpretation 2012. London, UK: Sage. 54. 54.Barbosa JC, Willoughby P, Rosenberg CA, Mrtek RG. Statistical methodology: VII. Q-methodology, a structural analytic approach to medical subjectivity. Academic Emergency Medicine 1998;5(10):1032–40. doi:10.1111/j.1553-2712.1998.tb02786.xpmid:9862598 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1111/j.1553-2712.1998.tb02786.x&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=9862598&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000076415000019&link_type=ISI) 55. 55.Barker JH. Q-methodology: an alternative approach to research in nurse education. Nurse Education Today 2008;28(8):917–25. doi:10.1016/j.nedt.2008.05.010pmid:18572285 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1016/j.nedt.2008.05.010&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=18572285&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 56. 56.Corr S. An introduction to Q methodology, a research technique. The British Journal of Occupational Therapy 2001;64(6):293–97. doi:10.1177/030802260106400605 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1177/030802260106400605&link_type=DOI) 57. 57.Cross RM. Exploring attitudes: the case for Q methodology. Health Education Research 2005;20(2):206–13. doi:10.1093/her/cyg121pmid:15385430 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1093/her/cyg121&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=15385430&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000227405400009&link_type=ISI) 58. 58.Paige JB, Morin KH. Q-sample construction: a critical step for a Q-methodological study. Western Journal of Nursing Research 2016;38(1):96–110. pmid:25092207 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1177/0193945914545177&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=25092207&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 59. 59.Valenta AL, Wigger U. Q-methodology: definition and application in health care informatics. Journal of the American Medical Informatics Association 1997;4(6):501–10. doi:10.1136/jamia.1997.0040501 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1136/jamia.1997.0040501&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=9391937&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) 60. 60.Thomas DB, Baas LR. The issue of generalization in Q methodology: “Reliable schematics” revisited. Operant Subjectivity 1992;16(1/2):18–36. 61. 61.Akhtar-Danesh N, Baumann A, Cordingley L. Q-methodology in nursing research: a promising method for the study of subjectivity. Western Journal of Nursing Research 2008;30(6):759–73. doi:10.1177/0193945907312979pmid:18337548 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.1177/0193945907312979&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=18337548&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom) [Web of Science](http://informatics.bmj.com/lookup/external-ref?access_num=000259196400008&link_type=ISI) 62. 62.vanExel J, de Graaf G. Q methodology: A sneak preview2005. Accessed 1 March 20171–21. Available from: [https://qmethod.org/portfolio/van-exel-and-de-graaf-a-q-methodology-sneak-preview/](https://qmethod.org/portfolio/van-exel-and-de-graaf-a-q-methodology-sneak-preview/)[www.jobvanexel.nl](http://www.jobvanexel.nl) 63. 63.Watts S, Stenner P. Qualitative Research in Psychology 2005. London, UK: Routledge. 64. 64.Petit dit Dariel O, Wharrad H, Windle R. Developing Q-methodology to explore staff views toward the use of technology in nurse education. Nurse Researcher 2010;18(1):58–71. doi:10.7748/nr2010.10.18.1.58.c8048pmid:21138086 [CrossRef](http://informatics.bmj.com/lookup/external-ref?access_num=10.7748/nr2010.10.18.1.58.c8048&link_type=DOI) [PubMed](http://informatics.bmj.com/lookup/external-ref?access_num=21138086&link_type=MED&atom=%2Fbmjhci%2F25%2F1%2F27.atom)