Table 2 The critical appraisal tool adapted from Mays N et al49
Quality checklist for mixed methodology studiesBalen 200450Bearman 200551Fox 200852Gosling 200453Gour 201054
- clear, terms defined
yes, to gain baseline data on pharmacists computer skills and training needsyes, use of the Internet by community pharmacists in practice and potential for educational interventionsyes, to identify and analyse current pharmacy informatics education, current competencies, core recommendations for teaching informaticsyes, comparison of different groups of AHPs, use of online evidence, training and computer skillsno, ‘to enlighten the perspective of computer use among healthcare professionals and its implications’
- appropriate
yes, survey of all pharmacists in one hospitalyes, in two phases: needs analysis focus groups and course evaluationyes, pharmacy syllabus mapping of informatics programs against Accreditation Council for Pharmacy Education (ACPE) 2007 standardsyes, survey of AHPs with access to Clinical Information Access Program (CIAP)mixed methods including cross-sectional survey and semi-structured interviews
- well described
yes, post implementation of applied informatics program in a Canadian hospitalyes, Australian pharmacy education and practiceyes, adoption of American pharmacy education standards and IOM inclusion of informatics as one of five core competenciesyes, introduction of online evidence system to hospitals in New South Wales, Australiadescribed adequately but with dated and contrived referencing
User system
- user of innovation
hospital pharmacistscommunity pharmacistspharmacy educationhospital-based AHPs including pharmacistshealthcare professionals including pharmacists
- conceptual, generalisation
yes, all hospital pharmacists (n=106) at one hospital over two sitesnot clear, both phases lack detail of recruitment/sampling/timingyes, from all American Association of Colleges of Pharmacy with pharmacy informatics programsrandomly selected hospitals representative of CIAP use
(n = 65); convenience sample
of AHPs (n = 790)
stratified sample (n = 240) of ‘all’ healthcare professionals, identified as doctors, nurses, lab technicians, pharmacists at one hospital; ‘no knowledge’ of computer
Data collection
- systematic, auditable
yes, the survey instrument developed by author consensus after review of previously published surveys (84 items over 9 domains)yes, independent facilitator, notes taken at each focus group by different scribes; course evaluation form (16 closed items; 3 open)yes, clearly explained at each stage of the process of collection, verification for reliability and validationyes, the survey instrument developed from previous, related research findings plus US study of clinicians’ use of Medlineconfusion between survey and interview techniques, use of a form assessing ‘knowledge of computer’, overlapping scales; lacks detail of development of survey and interview tools
Data analysis
- systematic, rigorous, conflict handling
appropriate use of descriptive statistics and frequenciesthemes not clear, qualitative data quantified, inappropriate inversion of scales; mean, SD applied to non-continuous scaleconsensus reached following independent categorisation by each research team memberappropriate use of descriptive and comparative statisticsinappropriate analysis, over-analysis and over-interpretation of a small, simple data set; overlap of at least two scales
Results and limitationsclear, concise with declared limitations around sample size, response rate (55%), survey instrument and self reportingfinding are grounded in data but, as indicated by the authors, with major limitations: no baseline prior to intervention, only two focus groups (not audio recorded), methods questioned high drop outfindings are clearly explained and grounded in the data; limitations are explored in terms of response rate (36%), non-respondents, variable detail of syllabi and generalisabilityclear and comprehensive with declared limitations of sampling recommending further validity and reliability testingimprecise terms, e.g. various, some; majority = 100%; assertions not grounded in data; parallels drawn to Nigeria and UK; speculation; major limitations but claims generalisable
Conclusionsappropriate for findingsappropriate for findingsappropriate for findingsappropriate for findingsadditional findings presented in Conclusion; unjustified assertions given research context
Ethicsnot coverednot coveredyesnot coveredyes, plus verbal consent
Additional commentsdata collected in 2001; clearly developed and presenteddata collected in 2002; specific to using the internetdata collected in 2007; clearly developed and presenteddata collected in 2001–2002; clearly developed and presentedpoorly developed, inconsistencies, poorly analysed/referenced
Take forward to Data Extraction?YYYYN