How to Critique Qualitative, Quantitative, Mixed Methods Studies, Systematic Reviews and Meta-Analyses: Key Differences

Introduction

Nursing students are often asked to “critique a research article,” but the right critique depends on the type of study. A qualitative study exploring mothers’ experiences cannot be judged by the same rules as a randomized controlled trial (RCT) testing a new drug. A systematic review is judged mainly on how thoroughly and transparently it searched for and combined other studies. Using the wrong criteria is one of the most common mistakes in student critiques. Examples include criticizing a qualitative study for a “small sample that is not generalizable,” or praising a meta-analysis without checking for heterogeneity.

Every critique shares the same goal: to decide whether a study’s findings are trustworthy, meaningful and applicable to practice (Polit & Beck, 2021). Most primary studies follow the IMRaD structure (Introduction, Methods, Results and Discussion), so a critique can follow the same order. What changes between study types is the set of quality standards applied in each section.

Comparison at a glance

1. Critique of a qualitative research study

What it is

Qualitative research explores how people experience a phenomenon, such as living with a chronic illness, caring for a dying relative or giving birth during a pandemic. It uses interviews, focus groups, observation or open-ended survey questions. Common traditions include phenomenology (lived experience), grounded theory (building a theory of a social process), ethnography (culture), and qualitative descriptive or thematic analysis designs.

What a critique focuses on

• Fit of approach: Is a qualitative design appropriate to the question, and is the tradition and philosophical stance stated?
• Sampling: Was purposive sampling used to find information-rich participants? Is data saturation or information power discussed? Small samples are normal and are not a weakness in themselves.
• Data collection: Did interviews or other methods produce rich, in-depth data? Is the interview guide described?
• Analysis: Is the analytic method, such as Braun and Clarke’s thematic analysis or Colaizzi’s method, described step by step?
• Reflexivity: Do the researchers discuss how their background and assumptions shaped the study?
• Trustworthiness (Lincoln & Guba, 1985): credibility (member checking, triangulation, quotations), dependability (audit trail), confirmability (reflexivity) and transferability (thick description of the setting and participants).

Common mistake

Criticizing a qualitative study for lack of statistical generalizability. Qualitative research aims for transferability: readers judge whether the findings fit their own setting.

Worked example: see our full critique of a qualitative study on perinatal distress during COVID-19.

2. Critique of a quantitative research study

What it is

Quantitative research uses numerical data and statistics to describe variables, examine relationships or test cause and effect. Designs range from descriptive and correlational studies, through cohort and case-control studies, to quasi-experimental studies and randomized controlled trials, the strongest design for testing interventions.

What a critique focuses on

• Research question and hypotheses: Are variables clearly defined? Is a PICO(T) question evident?
• Design and internal validity: Could bias or confounding explain the results? In an RCT, check randomization, allocation concealment, blinding, comparable groups at baseline, attrition and intention-to-treat analysis.
• Sampling: Was probability (random) sampling used? Was a power analysis done to make sure the sample was large enough to detect an effect?
• Measurement: Are instruments valid and reliable, for example with Cronbach’s alpha of at least 0.70?
• Statistical analysis: Are the tests appropriate (t-test, ANOVA, chi-square, regression)? Are p-values, confidence intervals and effect sizes reported?
• Clinical vs. statistical significance: A result can be statistically significant (p < .05) but too small to matter clinically.
• External validity: Can results be generalized to other populations and settings?

Common mistake

Equating “significant” with “important,” or ignoring dropouts. High or unequal attrition between groups can bias results.

3. Critique of a mixed methods research study

What it is

Mixed methods research intentionally combines qualitative and quantitative data in one study, so that each answers what the other cannot (Creswell & Plano Clark, 2018). The three core designs are:

• Convergent: both strands are collected at the same time and compared.
• Explanatory sequential: quantitative first, then qualitative to explain the results.
• Exploratory sequential: qualitative first, then quantitative to test or measure what was found.

What a critique focuses on

• Rationale: Do the authors justify why mixed methods were needed, rather than one method alone?
• Design: Is the specific design named, with the timing and priority of each strand stated?
• Quality of each strand: The qualitative strand is judged by qualitative criteria and the quantitative strand by quantitative criteria.
• Integration: This is the defining feature. Are the strands connected through joint displays, merged interpretation or one strand informing the other? Are differences between the two sets of results explored?
• Meta-inferences: Do the conclusions draw on both data sets together?

The Mixed Methods Appraisal Tool (MMAT) is widely used to appraise these studies (Hong et al., 2018).

Common mistake

Treating a study as “mixed methods” because it includes a survey with one open-ended question. Without deliberate integration, it is a quantitative study with some qualitative comments, not a true mixed methods design.

4. Critique of a systematic review

What it is

A systematic review uses a pre-specified, transparent and reproducible method to find, select, appraise and synthesize all relevant studies on a focused question (Higgins et al., 2024). It is different from a narrative or literature review, where the author chooses studies without a systematic search and has more room for bias. A systematic review may synthesize results narratively or, if the studies are similar enough, statistically through a meta-analysis. Reviews of qualitative studies use qualitative evidence synthesis methods such as meta-synthesis or meta-aggregation.

What a critique focuses on

• Focused question: Is there a clear PICO and a registered protocol, for example on PROSPERO?
• Comprehensive search: Were several databases searched (CINAHL, MEDLINE/PubMed, Embase, Cochrane), plus grey literature and reference lists? Are search terms and dates reported? Were language restrictions avoided?
• Study selection: Were inclusion and exclusion criteria pre-defined? Did two reviewers screen independently? Is a PRISMA flow diagram provided (Page et al., 2021)?
• Quality appraisal: Was the risk of bias in each included study assessed with a recognized tool, such as Cochrane RoB 2 or JBI checklists?
• Data extraction: Was it done in duplicate, using a standard form?
• Synthesis: Is the method appropriate, and do the conclusions take study quality into account?
• Certainty of evidence: Was GRADE used to rate confidence in the findings (Guyatt et al., 2008)?

The AMSTAR 2 tool (Shea et al., 2017) is the standard for appraising systematic reviews, and PRISMA 2020 is the reporting guideline.

Common mistake

Assuming every “review” is a systematic review, or assuming a systematic review is high quality just because it is at the top of the evidence pyramid. A review is only as good as its search and the studies it includes.

5. Critique of a meta-analysis

What it is

A meta-analysis is a statistical technique, usually performed within a systematic review, that combines the numerical results of several similar quantitative studies into one pooled effect estimate. Pooling increases statistical power and precision. Results are usually shown in a forest plot, which displays each study’s effect and confidence interval and the overall pooled effect as a diamond.

What a critique focuses on

• All of the systematic review criteria above. A meta-analysis without a rigorous systematic review is unreliable.
• Appropriateness of pooling: Are the studies similar enough in population, intervention, comparison and outcome to combine? Combining dissimilar studies is often called mixing “apples and oranges.”
• Heterogeneity: Is variation between studies measured with the I² statistic? As a rough guide, about 25% is low, 50% moderate and 75% high. Are sources of heterogeneity explored through subgroup or sensitivity analyses?
• Statistical model: Was a fixed-effect or random-effects model used, and is the choice justified? Random effects is usually more appropriate when heterogeneity is present.
• Effect measure: Are the right measures used, such as odds ratio, risk ratio or mean difference, and are they reported with 95% confidence intervals?
• Publication bias: Was it assessed with a funnel plot or Egger’s test, given that studies with positive results are more likely to be published?
• Quality of included studies: Pooling biased studies gives a precise but biased answer (“garbage in, garbage out”).

Common mistake

Reporting only the pooled result without checking heterogeneity or the quality of included studies. A narrow confidence interval does not make a biased estimate correct.

Key differences summarized

Tips for writing any research critique

• Identify the design first, then choose matching criteria and an appraisal tool (CASP, JBI, MMAT or AMSTAR 2).
• Be balanced. Discuss strengths as well as limitations, and say why a limitation matters for the findings.
• Use evidence. Support your judgments with research methods textbooks and reporting guidelines.
• Follow IMRaD. Organize your critique by introduction, methods, results and discussion.
• End with applicability. State the level of evidence and whether, and how, the findings should change nursing practice.

Conclusion

Each research design answers a different kind of question, so each needs a different critique. Qualitative studies are judged on trustworthiness and the depth of the experiences they reveal. Quantitative studies are judged on validity, reliability and statistical rigor. Mixed methods studies are judged on the quality of each strand and on how well the strands are integrated. Systematic reviews are judged on the comprehensiveness and transparency of their search and appraisal. Meta-analyses are judged on whether pooling was appropriate, how heterogeneity and publication bias were handled, and the quality of the included studies. Nurses who match the critique to the design can judge evidence accurately and translate it safely into evidence-based practice.

References

Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Sage.

Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 336(7650), 924–926. https://doi.org/10.1136/bmj.39489.470347.AD

Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (Eds.). (2024). Cochrane handbook for systematic reviews of interventions (Version 6.5). Cochrane. https://training.cochrane.org/handbook

Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.-C., Vedel, I., & Pluye, P. (2018). The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Education for Information, 34(4), 285–291. https://doi.org/10.3233/EFI-180221

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Sage.

Melnyk, B. M., & Fineout-Overholt, E. (2023). Evidence-based practice in nursing & healthcare: A guide to best practice (5th ed.). Wolters Kluwer.

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71

Polit, D. F., & Beck, C. T. (2021). Nursing research: Generating and assessing evidence for nursing practice (11th ed.). Wolters Kluwer.

Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., & Henry, D. A. (2017). AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 358, j4008. https://doi.org/10.1136/bmj.j4008