QI in Action: Bridging the Gap - Harnessing OCAN data for quality improvement

Published Sep 18, 2023

The Ontario Common Assessment of Need (OCAN) is a standardized assessment instrument, that the community mental health sector has used since 2009. This tool supports conversations between service providers and their clients. It helps service providers to understand the client’s current situation, needs and strengths, and to develop a service plan. OCAN assesses needs in 24 areas, called “domains." It supports a recovery approach with the inclusion of a client self-assessment component.

At a recent virtual train-the-trainer session hosted by EENet and Ontario Health, Frank Sirotich discussed the results of a recent project, which looked at how OCAN data could help identify an opportunity for quality improvement (QI). Frank is Director of Data Analytics, Research and Evaluation at the Canadian Mental Health Association (CMHA) Toronto.

This edition of QI in Action features highlights from his presentation in his own words, edited for clarity.

The context

Sometimes it feels like we put a lot of work into collecting OCAN data. It is relevant for clinical practices, but it is a lot of work for service providers to complete and can sometimes feel like it goes into a black hole.

I know a lot of organizations are looking at how they can use OCAN data for program planning and for QI. The presentation today is an example of how we can bridge that gap between using the assessment data for direct clinical care, on the one hand, and using it for QI and planning purposes, on the other.

Today, I’m going to present about a QI project that we did at CMHA Toronto and I’m going to discuss the following topics:

• why we undertook the project
• how we used OCAN data to:
  • identify QI opportunities
  • implement change ideas
  • measure change in client needs over time
• key learnings from our project.

At CMHA Toronto, we knew from the research that unmet physical health needs were associated with a greater risk of death, lower quality of life, more emergency department and inpatient use and longer time in service. Therefore, in our previous strategic plan, we prioritized developing our primary care capacity and invested in some programming for this purpose.

The problem

OCAN data for individuals who had at least four assessments within our case management services showed that rates of unmet physical health needs over time did not change after we implemented the new programming. Therefore, we decided to make addressing the lack of change our first project with the Excellence through Quality Improvement Project (E-QIP).

Sources of information and QI approaches for diagnosing root causes

We used QI approaches to help us identify underlying causes for the lack of change. The process involved the following steps:

1. We developed an aim statement for this QI project, which was to reduce unmet physical health needs by 20 per cent in a case management program.
2. We used a variety of QI tools to help us diagnose the problem. These tools were a fishbone exercise to generate some hypotheses and a staff focus group to hear what staff thought might be contributing to the problem.
3. With a combination of input from staff and the fishbone exercise, we developed questions for a survey that asked clients what they thought might be contributing to the problem.
4. Finally, we used a Pareto chart to analyze survey results and identify and prioritize the most significant factors contributing to the problem. This helped narrow down our focus to a few key causes.

Potential root causes

The information we gathered pointed to four root causes:

1. CMHA Toronto primary care services did not entirely meet all client preferences as they focused on health promotion and physical activity. We did not yet have funding for a full-time nurse practitioner to provide direct physical health care.
2. Staff did not have a clear understanding of how to rate unmet physical health needs in the OCAN.
3. Clients either did not have a primary care provider (e.g., a GP or other primary care practitioner) or had to wait a long time to see one. Even when they finally had an appointment, it was not uncommon for them to spend two or three hours in the waiting room, which dissuaded them from going.
4. Finally, clients were not aware of the importance of getting regular treatment for chronic physical health conditions and comprehensive physical health exams.

Figure 1

Drivers needed to achieve key change ideas for intensive case management clients

Key change ideas: 

• Clarify the rating of PH needs in OCAN
• Develop a primary care screener tied to OCAN
• Identify clinical pathways based on OCAN PH need rating 
• Initiate monthly meetings for ICM staff to review cases with unmet/unknown PH needs
• Provide nurse practitioner consultation to the team for case reviews 

Change ideas

Once we understood the possible root causes of the problem, we developed a driver diagram (Figure 1), which is a framework outlining the project objective (e.g., to reduce unmet needs by 20 per cent). We also identified key drivers that we needed to put in place and specific change ideas that we could implement to try to achieve that objective.

Since we had no new dedicated funding or resources, we selected two change ideas that would require the fewest resources and would have the greatest impact. These were:

• implement staff training to ensure they understood the rating of physical health needs in the OCAN using the definitions in Table 1.
• develop a paper-based prompt that reminds staff to rate the client’s physical health needs in the OCAN and that identifies standardized responses based on the client’s need rating.

Table 1: Defining unmet and unknown physical health needs in the OCAN

Process map

We then developed a process map of standardized responses for case managers based on the client’s physical health need rating (Figure 2). Key pathways in the process map identified the following three interventions for clients with unmet or unknown physical health needs:

1. If a client had an unmet physical health need, their case manager would try to refer them to a doctor and bring that case to a monthly team meeting to consult with a nurse practitioner.
2. If the client had an unknown physical health need, we would facilitate a referral to their primary care practitioner.
3. If the client declined seeing a primary care practitioner, their case manager would bring the case to the monthly team meeting for discussion with the nurse practitioner, to identify ways to motivate the client to see a primary care provider.

Figure 2

Three clinical response pathways for case managers

The physical health needs of clients who had unmet or unknown physical health needs in a given month were re-assessed in the subsequent month. If there was no change in their need rating, their case was brought back to a team meeting for further consultation.

Measuring change with the OCAN

We also sought to find a way to measure the impact of our change ideas using the OCAN and focused on the following two approaches:

1. measuring change in unmet and unknown need ratings each month
2. measuring overall change in unmet and unknown need ratings over the course of the project.

We collected information about physical health needs on a month-to-month basis because we recognized that waiting a full six months for a new OCAN cycle would be too long for a QI project. Each month, case managers used the definitions in Table 1 to rate the physical health need status of any client who had an unmet or unknown need in the previous month. We then counted the total number of clients who had physical health needs that remained unmet or unknown from the previous month (negative outcome) and the total number of clients whose need status changed from unmet or unknown to met or no need (positive outcome). With this information, we calculated the unmet needs ratio, calculated by dividing the negative outcomes by the total of the negative outcomes plus the positive outcomes (Figure 3). The lower this unmet need ratio, the better the outcome.

Figure 3

Unmet need ratio formula

For example, in the month of December, we had five clients whose physical health needs changed from unmet or unknown to met or no need (positive outcome) and 11 clients whose physical health needs remained unmet or unknown from the previous month (negative outcome). To calculate the unmet need ratio for that month, we divided the number of clients with negative outcomes (11 clients) by the number of clients with either positive or negative outcomes (16 clients) which equals 0.69.

With this information, we created a run chart, which is a QI tool that allowed us to track the unmet need ratio’s progress over time to determine if change ideas were having any impact (Figure 4).

Figure 4

Ratio of change in unmet physical health needs

Next, we looked at total change over the course of the project (Figure 5). Here, we looked at individuals who had at least two OCAN assessments in the course of the project. This showed that unmet need had decreased by 23 per cent and unknown need had decreased by 82 per cent. This showed that the change ideas had a greater effect on getting clients to see their doctors than in addressing their unmet physical health needs.

Figure 5

Change in physical health need status from first to most recent OCAN assessment

Lessons learned

There were several lessons learned from this QI project:

• We can use the OCAN to identify QI opportunities by looking at clients whose unmet need rating remains consistently unchanged over time.
• We can clarify domain and need rating definitions to promote consistency in staff ratings of their clients’ physical health needs.
• OCAN need domains can inform standardized responses in care.
• An unmet need ratio can be used to track change.
• Individual need domains can be useful to measure outcomes over time.

The overall takeaway is that the OCAN, in addition to having value in clinical practice, has applications for QI purposes and for planning.

See more related to

• OCAN