Skip to Main Content

Occupational Therapy

Understanding Levels of Evidence in Occupational Therapy

What Are Levels of Evidence?

Levels of evidence rank the strength and reliability of research findings. They help clinicians and students determine how much confidence to place in a study’s results when making clinical decisions. The American Occupational Therapy Association (AOTA) uses a hierarchy adapted from the Oxford Centre for Evidence-Based Medicine (OCEBM).

AOTA Levels of Evidence (Expanded with Examples)

Level  Type of Evidence Description Example
1A Systematic review of homogeneous RCTs Combines results from multiple high-quality randomized controlled trials (RCTs) to provide a comprehensive answer to a clinical question. A meta-analysis of 10 RCTs evaluating the effectiveness of constraint-induced movement therapy in stroke rehabilitation.
1B Individual RCT with narrow confidence interval A single, well-designed RCT with a large sample size, randomization, and control group. An RCT comparing two sensory integration techniques in children with autism, with 100+ participants and clear outcome measures.
2A Systematic review of cohort studies Reviews multiple cohort studies that follow groups over time to assess outcomes. A review of longitudinal studies tracking the impact of early OT intervention on school readiness.
2B Individual cohort study or low-quality RCT Observational study following a group over time, or an RCT with methodological flaws (e.g., small sample, poor blinding). A cohort study following 50 patients post-hip replacement to assess return-to-work outcomes.
3A Systematic review of case-control studies Reviews studies comparing individuals with a condition to those without. A review of studies comparing OT outcomes in veterans with PTSD vs. those without.
3B Individual case-control or pre-post study Compares individuals with and without a condition, or measures outcomes before and after an intervention in one group. A study measuring anxiety levels before and after a mindfulness-based OT program.
4 Case series or poor-quality cohort/case-control Descriptive studies without control groups or with significant bias. A report on outcomes from a clinic using a new handwriting program with 10 children.
5 Expert opinion or theory Based on clinical experience or theoretical models, not empirical data. A published article proposing a new OT framework without supporting research.

Strength of Evidence Ratings

Rating What It Means Tip
Strong Multiple Level 1 studies with consistent results Use these to support clinical decisions or justify interventions in papers.
Moderate At least one high-quality Level 1 or multiple Level 2/3 studies Good for supporting practice, but be cautious of limitations.
Low Limited or flawed studies Use with caution; highlight gaps in your literature reviews.
Insufficient Not enough evidence to draw conclusions Identify as areas for future research or thesis topics.

How to Critically Appraise a Study

1. Check the Sample Size
  • Why it matters: Larger samples reduce bias and increase generalizability.
  • How to check: Look in the Methods section. A sample size of 30+ per group is often considered a minimum for statistical power.
  • Example: A study with 12 participants per group may be underpowered to detect meaningful differences.
2. Evaluate the Methodology
  • Randomization: Were participants randomly assigned to groups?
  • Blinding: Were participants and/or assessors blinded to the intervention?
  • Control Group: Was there a comparison group receiving standard care or placebo?
  • Example: A study that lacks a control group and does not blind assessors may overestimate the effect of the intervention.
3. Assess the Outcome Measures
  • Validity: Are the tools used to measure outcomes reliable and validated?
  • Relevance: Do the outcomes reflect meaningful changes in occupational performance?
  • Example: Using a validated tool like the Canadian Occupational Performance Measure (COPM) is more credible than an untested survey.
4. Look at the Follow-Up
  • Duration: Was the follow-up long enough to observe lasting effects?
  • Attrition: Did many participants drop out? High dropout rates can bias results.
  • Example: A 3-month follow-up with 90% retention is stronger than a 1-week follow-up with 50% dropout.

Research Evaluation Checklist

Introduction

This checklist may help evaluate the level and strength of evidence (AOTA Framework) in research studies. Use it when reviewing journal articles, preparing literature reviews, or making evidence-based clinical decisions.

1. Study Design

Questions to Ask:

  • Is the study a systematic review or meta-analysis? → Likely Level 1A or 2A/3A
  • Is it a randomized controlled trial (RCT)? → Likely Level 1B or 2B
  • Is it a cohort or case-control study? → Likely Level 2B, 3B
  • Is it a case series or expert opinion? → Likely Level 4 or 5

Tip: Look in the Methods section for terms like randomized, controlled, prospective, or retrospective.

2. Sample Size

Questions to Ask:

  • Is the sample size large enough (e.g., >30 per group for RCTs)?
  • Is a power analysis reported to justify the sample size?
  • Are dropouts or attrition reported and explained?

Example: A study with 15 participants per group and no power analysis may be underpowered and less reliable.

3. Methodological Quality

Questions to Ask:

  • Was randomization used (for RCTs)?
  • Was blinding used (participants, therapists, assessors)?
  • Was there a control or comparison group?
  • Were inclusion/exclusion criteria clearly defined?

Example: A study that randomizes participants and uses blinded assessors is more rigorous than one that does not.

4. Outcome Measures

Questions to Ask:

  • Are the outcome measures valid and reliable?
  • Are they clinically relevant to occupational performance?
  • Are standardized tools used (e.g., COPM, FIM, AMPS)?

Example: Using a validated tool like the COPM is stronger than using a non-standardized survey.

5. Data Analysis

Questions to Ask:

  • Are appropriate statistical tests used?
  • Are confidence intervals and effect sizes reported?
  • Are results both statistically and clinically significant?

Tip: Look for p-values (<0.05), but also consider the effect size and practical impact.

6. Follow-Up and Generalizability

Questions to Ask:

  • Is there a follow-up period to assess long-term outcomes?
  • Is the population similar to your clinical or research interest?
  • Are limitations discussed?

Example: A 6-month follow-up with 90% retention is stronger than a 1-week follow-up with 50% dropout.

7. Strength of Evidence

Questions to Ask:

  • Are there multiple studies with consistent findings?
  • Is the study replicated in different settings or populations?
  • Does the evidence support clinical decision-making?

Rating Guide:

  • Strong: Multiple Level 1 studies with consistent results
  • Moderate: One high-quality Level 1 or multiple Level 2/3 studies
  • Low: Limited or flawed studies
  • Insufficient: Not enough evidence to draw conclusions

Tips for Graduate Students

  • Start with Systematic Reviews: Use databases like AJOT, PubMed, or OTseeker to find high-level evidence.
  • Use PICO: Frame your clinical question using Population, Intervention, Comparison, and Outcome to guide your search.
  • Practice Appraisal: Use tools like the CASP checklists or AOTA’s Evidence-Based Practice resources.
  • Stay Organized: Create a table to track study design, sample size, outcomes, and level of evidence.
  • Discuss Limitations: Always include a critique of the evidence in your assignments or presentations.

Helpful Resources