AR modules produced consistent, meaningful gains across both organizations in knowledge, comfort, and willingness to seek help — with especially large movement at COSTI (lower-CLB mix) on English discussability and self-rated understanding. MW produced strong gains in understanding and healthy talk norms but did not strengthen certainty about where to seek help nor whether to ask teachers. This signals the need to sharpen referral pathways and role clarity before broader national scaling.
1. Scale vs. Specificity: The JIAS-specific content (Jewish representation, antisemitism examples) reflects genuine equity work — but it also means the current curriculum is not yet neutral enough for other-sector deployment without adaptation. Scaling requires a core + modular structure, not a single version.
2. Evidence Quality vs. Delivery Speed: COSTI MW was delivered without student surveys in place. The gap is understandable operationally, but it means the full evidence base for MW is incomplete. Before the next cycle, data collection infrastructure must be built before delivery begins — not as an afterthought.
Theoretical training alone is insufficient. Instructors needed time to test videos, plan adaptations, and rehearse facilitation before entering the classroom. Apply this to any topic-specific LINC training.
Lower-CLB groups need larger fonts, slower audio, and sentence frames. Higher-CLB groups want richer tasks. One version of any curriculum creates inequitable learning conditions from the outset.
The focus groups revealed packaging problems, delivery barriers, and instructional design gaps that pre/post scores alone would never have captured. Both methods are required for a credible evaluation.
Attitude change does not automatically translate into help-seeking behavior. If you want students to know where to go, the curriculum must specifically map those pathways — mini-lessons, decision maps, and referral scripts.
COSTI delivered the MW module without student surveys in place. The evidence gap that created is not recoverable. Evaluation instruments must be designed and tested before the first delivery, not after.
JIAS's request for Jewish representation in curriculum materials was not a niche ask — it was evidence of a larger principle: representation signals belonging, and its absence signals exclusion. The principle applies to every community.
Pre and post survey N for each evaluated module
Post-survey sample breakdown
Pre (N=21) → Post (N=23) · CLB 3–5 · Separate cohort
Pre (N=23) → Post (N=21) · CLB 4–5 · 90% Women · Separate cohort
Pre (N=22) → Post (N=23) · CLB 4–5 · 74% Women · Separate cohort
Percentage-point change (Pre → Post)
| Outcome Indicator | COSTI AR | JIAS AR | JIAS MW |
|---|---|---|---|
| Self-Rated Understanding (4–5/5) | ▲ +44.3 | ▲ +19.7 | ▲ +39.9 |
| Comfort Discussing Topic in Class | ▲ +27.5 | ▲ +9.6 | — n/a |
| "Racism Can Happen in Canada" (True) | ▲ +15.9 | ▲ +32.8 | — n/a |
| "Tell Someone if Racism Occurs" (True) | ▲ +31.4 | ▲ +16.0 | — n/a |
| Discussing in English (Easy/Okay) | ▲ +54.6 | ▲ +21.3 | — n/a |
| MW: "Should Not Talk Feelings" (False) | — n/a | — n/a | ▲ +26.6 |
| MW: Coping Behaviors Help (True) | — n/a | — n/a | ▲ +11.9 |
| MW: Can Ask Teacher for Help (True) ⚠️ | — n/a | — n/a | ▼ −7.1 |
The AR module has produced consistent, replicable gains across two organizations and learner profiles. The evidence base is sufficient to justify continuation and expansion. The question is not whether to continue but how to fix what the evaluation identified before scaling.
The 14-week revision sprint is a defined, bounded scope — not a new project. Three phases address all critical gaps identified:
The curriculum exists, instructors are trained, and the evidence is in hand. Without the revision sprint, the project stalls at the proof-of-concept stage. The opportunity cost: ~660 newcomers per cohort cycle who would benefit from a scale-ready AR/MW curriculum that currently doesn't exist in the sector.
This project directly addresses IRCC priorities around settlement sector capacity, anti-racism in newcomer-serving organizations, and mental well-being supports for newcomers. The cross-organizational model (COSTI + JIAS) demonstrates collaborative sector delivery and reduces duplication. The evaluation framework itself is a transferable asset for the sector.
Fix media glitches, layout/contrast issues. Separate teacher guide from student booklet. Redesign assessments to be non-isomorphic. Provide editable and print-ready files.
2–3 hour add-on to existing training: module walkthroughs, disclosure scripts, site-specific referral maps (names, languages, hours) for both COSTI and JIAS.
Produce Literacy/1–2 and CLB 4–5 variants. Integrate antisemitism content into AR. Add Jewish community references to MW. Update all media assets accordingly.
Run MW at COSTI, and both AR and MW at COSTI and JIAS after all improvements. Compare before/after changes. Produce a one-page key results summary for stakeholders and funders.