Implementing a volunteer peer fidelity assessment in Ontario Early Psychosis Intervention programs: What did we learn?

In brief

The Early Psychosis Intervention Ontarioi Network (EPION)’s Standards Implementation Committee has released its fidelity pilot report. “Implementing a volunteer peer fidelity assessment in Ontario Early Psychosis Intervention programs: what did we learn?” describes the fidelity pilot project conducted by the Standards Implementation Steering Committee (SISC).

The fidelity pilot project was initiated to test the feasibility and value of using an innovative peer assessment method to assess the fidelity of selected Ontario EPI programs. Peer assessor teams of volunteer EPI program staff and CAMH evaluators were formed to conduct the program reviews. The pilot was completed in nine programs.

Project objectives:

This fidelity pilot report describes the review process and the results of the evaluation. Also included are recommended next steps related to fidelity monitoring in EPI programs.

In this Research Report Round-up, we present a summary of the report. Research Report Round-ups are brief summaries of research reports, presented in a user-friendly format.

Read the summary below or download the PDF.

Title and link to report: Implementing a volunteer peer fidelity assessment in Ontario Early Psychosis Intervention programs: What did we learn?

Authors: Janet Durbin, Avra Selick, Gordon Langill, Chi Cheng, Suzanne Archie, Donald Addington, and the Early Psychosis Intervention Ontario Network (EPION) Standards Implementation Steering Committee.

Date of release: September 2018

Location: Toronto
Language: English

What this report is about

facilitator working with adult traineesRoutine fidelity monitoring can help programs ensure they are following pre-set standards. Ontario community mental health programs do not routinely do this type of assessment. This project tested the feasibility and value of a volunteer peer fidelity assessment process for Ontario Early Psychosis Intervention (EPI) programs.

Key components of the process included:

A central team provided training and coaching to the assessors with the aim of increasing the consistency and accuracy of the fidelity ratings. Nine programs from across Ontario were assessed and 20 individuals were trained to conduct the assessments.

Many elements of the assessment process supported the assessors to produce consistent and accurate ratings, as well as narrative assessment feedback to programs, and to gather  information about the process itself:

How EPI program participants felt about the process:

What was learned about fidelity in program delivery:

What was learned about feasibility:

Next Steps:

Recommendations for the EPI sector:

How can this report be used

This report offers recommendations and strategies to increase the use, and enhance the quality, of fidelity assessments for EPI programs. It should be of use to system and program planners, and providers of EPI services, as well as policymakers, to implement routine sector-wide fidelity assessments in Ontario EPI programs.

Type of study

Researchers developed and implemented a peer fidelity assessment process with a group of Ontario EPI programs. Qualitative feedback was obtained from assessors and programs to learn about the consistency, feasibility, and value of the process. Fidelity results were reviewed with consideration of how they could be used to support program and sector improvement, and future fidelity monitoring.

Key words

Early psychosis intervention; fidelity; assessment; monitoring; program standards; quality improvement.

Contact person/source

Janet Durbin, Independent Scientist, Data Evaluation Leadership Transformative Analyses (DELTA), Provincial System and Support Program (PSSP), CAMH at janet.durbin@camh.ca
Avra Selick, Research Specialist, DELTA, PSSP, CAMH at avra.selick@camh.ca.

See more related to