A scoring system for mixed methods research and mixed studies reviews33

Types of mixed methods study components or primary studies	Methodological quality criteria	Responses
Types of mixed methods study components or primary studies	Methodological quality criteria	Yes	No	Can’t tell	Comments
Screening questions (for all types)	Are there clear qualitative and quantitative research questions (or objectives^), or a clear mixed methods question (or objective^)?
Screening questions (for all types)	Do the collected data allow address the research question (objective)? (E.g., consider whether the follow-up period is long enough for the outcome to occur (for longitudinal studies or study components)).
Further appraisal may not be feasible or appropriate when the answer is ‘No’ or ‘Can’t tell’ to one or both screening questions.
1. Qualitative	1.1. Are the sources of qualitative data (archives, documents, informants, observations) relevant to address the research question (objective)?
	1.2. Is the process for analysing qualitative data relevant to address the research question (objective)?
	1.3. Is appropriate consideration given to how findings relate to the context, e.g. the setting, in which the data were collected?
	1.4. Is appropriate consideration given to how findings relate to researchers’ influence, e.g. through their interactions with participants?
2. Quantitative randomised controlled (trials)	2.1. Is there a clear description of the randomisation (or an appropriate sequence generation)?
	2.2. Is there a clear description of the allocation concealment (or blinding when applicable)?
	2.3. Are there complete outcome data (80% or above)?
	2.4. Is there low withdrawal/drop-out (below 20%)?
3. Quantitative nonrandomised	3.1. Are participants (organisations) recruited in a way that minimises selection bias?
	3.2. Are measurements appropriate (clear origin, or validity known, or standard instrument; and absence of contamination between groups when appropriate) regarding the exposure/intervention and outcomes?
	3.3. In the groups being compared (exposed vs. non-exposed; with intervention vs. without; cases vs. controls), are the participants comparable, or do researchers take into account (control for) the difference between these groups?
	3.4. Are there complete outcome data (80% or above), and, when applicable, an acceptable response rate (60% or above), or an acceptable follow-up rate for cohort studies (depending on the duration of follow-up)?
4. Quantitative descriptive	4.1. Is the sampling strategy relevant to address the quantitative research question (quantitative aspect of the mixed methods question)?
	4.2. Is the sample representative of the population under study?
	4.3. Are measurements appropriate (clear origin, or validity known, or standard instrument)?
	4.4. Is there an acceptable response rate (60% or above)?
5. Mixed methods	5.1. Is the mixed methods research design relevant to address the qualitative and quantitative research questions (or objectives), or the qualitative and quantitative aspects of the mixed methods question (or objective)?
	5.2. Is the integration of qualitative and quantitative data (or results^*) relevant to address the research question (objective)?
	5.3. Is appropriate consideration given to the limitations associated with this integration, e.g., the divergence of qualitative and quantitative data (or results^*) in a triangulation design?
	Criteria for the qualitative component (1.1 to 1.4), and appropriate criteria for the quantitative component (2.1 to 2.4, or 3.1 to 3.4, or 4.1 to 4.4), must be also applied

↵* These two items are not considered as double-barrelled items since in mixed methods research, (1) there may be research questions (quantitative research) or research objectives (qualitative research) and (2) data may be integrated, and/or qualitative findings and quantitative results can be integrated.