Final Evaluation Report · March 2026

Anti-Racism & Mental Well-Being
Curriculum Development Project

Post Training & Post Module Evaluation — COSTI & JIAS

Prepared by: JIAS Evaluation Team N=33 Instructors Two Modules: Anti-Racism (AR) & Mental Well-Being (MW) COSTI + JIAS Both Delivered Both Modules Mixed-Methods Design
📊

Impact at a Glance

Key numbers from the full rollout evaluation

33
Instructors Trained
Post-Rollout Survey
88%
Instructors with ≥10 Years Teaching Experience
+44pp
COSTI AR: Gain in Self-Rated Understanding
+55pp
COSTI AR: Students Comfortable Discussing in English
+40pp
JIAS MW: Gain in Understanding
91%
COSTI AR Students Found Lesson Useful
📋

Executive Summary

What this evaluation found — and what it means

🎯 What We Evaluated

  • Instructor post-training: N=33 (COSTI ≈21, JIAS ≈11) following full rollout of both the Anti-Racism (AR) and Mental Well-Being (MW) modules
  • Student pre/post surveys: Both organizations delivered both modules. Student survey data received: COSTI AR ✅ · JIAS AR ✅ · JIAS MW ✅
  • COSTI MW — important note: COSTI delivered the Mental Well-Being module but student surveys were not collected. MW feedback from COSTI comes exclusively from the post-delivery teacher focus group, not student data.
  • Focus groups: Post-delivery teacher discussions at both COSTI and JIAS, covering both AR and MW delivery experience

✅ What the Evidence Shows

  • Logic Model alignment confirmed: Immediate gains in knowledge and comfort; short-term outcomes in motion
  • AR modules work at scale: Consistent gains across both COSTI and JIAS
  • MW understanding strong; help-seeking pathways remain the critical gap
  • Packaging and CLB differentiation are prerequisites for broader scaling

🔑 Central Finding

AR modules produced consistent, meaningful gains across both organizations in knowledge, comfort, and willingness to seek help — with especially large movement at COSTI (lower-CLB mix) on English discussability and self-rated understanding. MW produced strong gains in understanding and healthy talk norms but did not strengthen certainty about where to seek help nor whether to ask teachers. This signals the need to sharpen referral pathways and role clarity before broader national scaling.

👥

Participant Profiles

Who participated — instructors and students across both sites

🏫 Instructor Profile (Post-Training Rollout, N=33)

88%
≥ 10 Years Teaching
66%
COSTI Affiliated
34%
JIAS Affiliated
61%
Prior AR/EDI/MH Training
79%
Teach CLB 3–4
61%
Teach CLB 1–2

Student Samples by Module

Pre and post survey N for each evaluated module

CLB Level Distribution by Cohort

Post-survey sample breakdown

📈

Student Findings: Pre → Post Shifts

Measured changes across Anti-Racism and Mental Well-Being modules

COSTI — Anti-Racism Module

Pre (N=21) → Post (N=23) · CLB 3–5

JIAS — Anti-Racism Module

Pre (N=23) → Post (N=21) · CLB 4–5 · 90% Women

JIAS — Mental Well-Being (MW) Module

Pre (N=22) → Post (N=23) · CLB 4–5 · 74% Women · Note: COSTI also delivered MW — student data not collected there; COSTI MW feedback via teacher focus group only

Gains Comparison: All Three Modules

Percentage-point change (Pre → Post) on understanding

📐 Comparative Delta Table — All Key Indicators

Outcome Indicator COSTI AR (Δ pp) JIAS AR (Δ pp) JIAS MW (Δ pp)
Self-Rated Understanding (4–5/5) ▲ +44.3 ▲ +19.7 ▲ +39.9
Comfort Discussing Topic in Class ▲ +27.5 ▲ +9.6 — n/a
"Racism Can Happen in Canada" (True) ▲ +15.9 ▲ +32.8 — n/a
"Tell Someone if Racism Occurs" (True) ▲ +31.4 ▲ +16.0 — n/a
Discussing Topic in English (Easy/Okay) ▲ +54.6 ▲ +21.3 — n/a
MW: "Should Not Talk About Feelings" (False) — n/a — n/a ▲ +26.6
MW: Coping Behaviors Help (True) — n/a — n/a ▲ +11.9
MW: Can Ask Teacher for Help (True) ⚠️ — n/a — n/a ▼ −7.1

⚠️ The MW help-seeking gap is the primary implementation priority before broader scaling.

💬

Qualitative Insights — Focus Groups

Themes from post-delivery teacher discussions at COSTI and JIAS

"

Instructors consistently requested hands-on module walkthroughs — time to test videos, plan adaptations, and rehearse facilitation before entering the classroom. Theoretical training alone is insufficient.

📍 Shared — Both Sites
"

CLB-tiered differentiation is not optional — it's essential. Lower CLB groups need larger fonts, slower audio, and sentence frames. Higher CLB groups want richer tasks and more grammar depth. One size does not fit all.

📍 Shared — Both Sites
"

When a student disclosed emotional distress during the MW assessment, instructors needed a micro-protocol — 90 seconds: validate → stabilize → offer choices → refer. Without this, teachers freeze. Clear scripts save both teacher and student.

🔴 COSTI-Specific
"

JIAS instructors strongly requested Jewish visibility in AR content — explicit antisemitism examples, synagogue and rabbi references in MW scenarios. Representation in curriculum signals belonging. Its absence signals exclusion.

🔵 JIAS-Specific
"

Assessment tasks were too similar to practice tasks — students could pass by repeating what they did rather than transferring knowledge. Non-isomorphic assessments are needed to generate valid evidence of learning.

📍 Shared — Both Sites
"

The interwoven teacher notes and student pages created real navigation problems in the classroom. A clean student booklet and a separate teacher guide — with print-ready and editable files — are the minimum usability requirement for scale.

🔵 JIAS-Specific

✅ What Worked

  • High instructor confidence post-training across both sites
  • Strong perceived fit of AR/MW content with LINC programming
  • Meaningful student gains in knowledge, comfort, and willingness to seek help (AR)
  • MW improved healthy norms around talking about feelings
  • COSTI AR: dramatic gains for lower-CLB learners on English ease and understanding

⚠️ Gaps Requiring Action

  • MW help-seeking clarity: "know where to find help" flat; "ask teacher" declined
  • COSTI delivered the Mental Well-Being module; student surveys not collected — MW student data reflects JIAS only
  • Media glitches and layout errors affecting delivery quality
  • Assessment isomorphism reducing evidence validity
  • Antisemitism and Jewish representation absent from current materials
  • Packaging not classroom-ready for seamless use at scale
🎯

Recommendations

Nine priority actions across training, curriculum, packaging, and data

1
Training Delivery
Hands-On Module Labs
Include walkthroughs of AR/MW units with time to test videos, plan adaptations, and rehearse facilitation of bias incidents and disclosures.
2
Training Delivery
Micro-Protocols + Scripts
Provide ready-to-use validate→stabilize→offer choices→refer scripts, "what-not-to-say" guidance, and site-specific referral trees (names, languages, hours).
3
Training Delivery
Pacing
Avoid scheduling both AR and MW trainings in the same week. Include time for practice and reflection between sessions.
4
Curriculum
CLB-Tiered Variants
Literacy/CLB 1–2: larger fonts (18–20pt), clearer emotion images, slower audio, sentence frames. CLB 4–5: richer grammar, additional scenarios, open-ended tasks.
5
Curriculum
Assessment Redesign
Ensure non-isomorphic assessments (not clones of practice tasks). Add performance tasks such as help-seeking role-plays and interpersonal vs. systemic racism identification.
6
Curriculum
Representation Completeness
Explicitly include antisemitism examples in AR and Jewish community references (synagogue, rabbi) in MW alongside broader religious and cultural representations.
7
Curriculum
Reduce Repetition
Avoid reusing the same video across skill modules. Fix media glitches, layout/contrast issues, and audio speed problems across both modules.
8
Packaging
Two-Book System
Produce a clean student booklet and a separate teacher guide. Provide both print-ready and editable file formats for seamless classroom use and site-level adaptation.
9
Student Help-Seeking
Normalize Help Pathways
Add a short, leveled "Where to get help in Canada" mini-lesson and simple decision maps — when to talk to teacher vs. settlement worker vs. community provider — directly addressing the MW post-module uncertainty gap.
🗓

Proposed Next Steps & Timeline

Priority sequencing before broader national scaling

Weeks 1–6 · Rapid Fixes
Packaging, Media & Assessment
Fix media glitches, layout/contrast issues. Separate teacher guide from student booklet. Redesign assessments to be non-isomorphic. Provide editable and print-ready files.
Weeks 2–4 · Training Add-On
Module Labs + Micro-Protocols
2–3 hour add-on to existing training: module walkthroughs, disclosure scripts, site-specific referral maps (names, languages, hours) for both COSTI and JIAS.
Weeks 6–14 · Curriculum Sprint
CLB Tiering + Representation
Produce Literacy/1–2 and CLB 4–5 variants. Integrate antisemitism content into AR. Add Jewish community references to MW. Update all media assets accordingly.
Post-Revision · Re-Evaluation
Student Surveys Again — Then Compare
Run MW at COSTI, and both AR and MW at COSTI and JIAS after all improvements. Compare before/after changes. Produce a one-page key results summary for stakeholders and funders.
🔬

Methodology Note

Design, instruments, and limitations

🧪 Design

  • Mixed methods: surveys (instructor + students) + focus groups
  • Brief knowledge checks embedded in student surveys
  • Pre/post design for all student modules
  • Post-only for instructor rollout cohort (N=33)
  • Delta calculations: Post% minus Pre% for each indicator

⚠️ Limitations

  • Self-report bias in all survey instruments
  • Small class-level samples limit statistical power
  • COSTI delivered the Mental Well-Being (MW) module but student pre/post surveys were not collected — MW student findings therefore reflect JIAS only; COSTI MW instructor perspective captured via focus group
  • Varying CLB profiles across sites limit direct comparisons
  • No long-term follow-up data yet (sustained behavior change)
Slide 1
1 / 9