| Quality checklist for mixed methodology studies | Balen 200450 | Bearman 200551 | Fox 200852 | Gosling 200453 | Gour 201054 |
|---|---|---|---|---|---|
| Question - clear, terms defined | yes, to gain baseline data on pharmacists computer skills and training needs | yes, use of the Internet by community pharmacists in practice and potential for educational interventions | yes, to identify and analyse current pharmacy informatics education, current competencies, core recommendations for teaching informatics | yes, comparison of different groups of AHPs, use of online evidence, training and computer skills | no, ‘to enlighten the perspective of computer use among healthcare professionals and its implications’ |
| Design - appropriate | yes, survey of all pharmacists in one hospital | yes, in two phases: needs analysis focus groups and course evaluation | yes, pharmacy syllabus mapping of informatics programs against Accreditation Council for Pharmacy Education (ACPE) 2007 standards | yes, survey of AHPs with access to Clinical Information Access Program (CIAP) | mixed methods including cross-sectional survey and semi-structured interviews |
| Context - well described | yes, post implementation of applied informatics program in a Canadian hospital | yes, Australian pharmacy education and practice | yes, adoption of American pharmacy education standards and IOM inclusion of informatics as one of five core competencies | yes, introduction of online evidence system to hospitals in New South Wales, Australia | described adequately but with dated and contrived referencing |
| User system - user of innovation | hospital pharmacists | community pharmacists | pharmacy education | hospital-based AHPs including pharmacists | healthcare professionals including pharmacists |
| Sampling - conceptual, generalisation | yes, all hospital pharmacists (n=106) at one hospital over two sites | not clear, both phases lack detail of recruitment/sampling/timing | yes, from all American Association of Colleges of Pharmacy with pharmacy informatics programs | randomly selected hospitals representative of CIAP use (n = 65); convenience sample of AHPs (n = 790) | stratified sample (n = 240) of ‘all’ healthcare professionals, identified as doctors, nurses, lab technicians, pharmacists at one hospital; ‘no knowledge’ of computer |
| Data collection - systematic, auditable | yes, the survey instrument developed by author consensus after review of previously published surveys (84 items over 9 domains) | yes, independent facilitator, notes taken at each focus group by different scribes; course evaluation form (16 closed items; 3 open) | yes, clearly explained at each stage of the process of collection, verification for reliability and validation | yes, the survey instrument developed from previous, related research findings plus US study of clinicians’ use of Medline | confusion between survey and interview techniques, use of a form assessing ‘knowledge of computer’, overlapping scales; lacks detail of development of survey and interview tools |
| Data analysis - systematic, rigorous, conflict handling | appropriate use of descriptive statistics and frequencies | themes not clear, qualitative data quantified, inappropriate inversion of scales; mean, SD applied to non-continuous scale | consensus reached following independent categorisation by each research team member | appropriate use of descriptive and comparative statistics | inappropriate analysis, over-analysis and over-interpretation of a small, simple data set; overlap of at least two scales |
| Results and limitations | clear, concise with declared limitations around sample size, response rate (55%), survey instrument and self reporting | finding are grounded in data but, as indicated by the authors, with major limitations: no baseline prior to intervention, only two focus groups (not audio recorded), methods questioned high drop out | findings are clearly explained and grounded in the data; limitations are explored in terms of response rate (36%), non-respondents, variable detail of syllabi and generalisability | clear and comprehensive with declared limitations of sampling recommending further validity and reliability testing | imprecise terms, e.g. various, some; majority = 100%; assertions not grounded in data; parallels drawn to Nigeria and UK; speculation; major limitations but claims generalisable |
| Conclusions | appropriate for findings | appropriate for findings | appropriate for findings | appropriate for findings | additional findings presented in Conclusion; unjustified assertions given research context |
| Ethics | not covered | not covered | yes | not covered | yes, plus verbal consent |
| Additional comments | data collected in 2001; clearly developed and presented | data collected in 2002; specific to using the internet | data collected in 2007; clearly developed and presented | data collected in 2001–2002; clearly developed and presented | poorly developed, inconsistencies, poorly analysed/referenced |
| Take forward to Data Extraction? | Y | Y | Y | Y | N |