Chapter 108

The worst in decades, explained with numbers

Outliers

Chapter 108: Outliers

Since I am writing about a past event, this means we have the benefit of future knowledge on our side. From this future knowledge, I can tell you stats about that particular OSCE we mentioned. I mentioned how it seemed like we had the most failures ever and in truth, we actually had the lowest pass rate for the OSCEs in decades up to that point. If you look up the stats today, you can see that the OSCE pass rate was usually around ~85% for most years. Our year in 2020? 78%. A full 7% behind the average. 

That’s not even the worst part of it, if you look at the stats a bit more in detail and see how it compares school by school, you can note that Waterloo usually had around a ~95% pass rate up until 2019. During 2019, it dipped behind to around 87% and then, our year in 2020? Around 85%. So our class was not different by 7%. We were behind the usual Waterloo students by a full 10%. 

Now, technically speaking, the class of 2019, who didn’t have to deal with COVID like we did, had a failure rate similar to our class. Unfortunately, we didn’t have these statistics when we got our results. That is to say, when the 2020’s all got our results, we still only knew about the stats of the year 2018 and before. This meant that in our minds, Waterloo only ever had a failure rate of around 5-6% and our class had around 10%. In other words, our group of those who failed thought our year had twice as many failures as you’d normally expect from Waterloo. Because of this, there was a lot of finger pointing around and a lot of conspiracy theories that were tossed back and forth. We were a statistical anomaly.

Our group thought that being an anomaly may give us a leg up on our current circumstances. The thinking behind this was that since we were an anomaly in statistics, it must point to the fact that the COVID protocols have affected our performance and therefore, we’d have justification to request an appeal under more fair conditions. 

We had high hopes initially for the entire ordeal because at the time, we thought we had made some good points. However, there was this over encompassing looming feeling that COVID may work against us rather than with us. By this I mean to say that the OEBC board were going to craft the narrative that COVID made us worse clinicians rather than OEBC having given us a worse and more unfair exam. Or to put it another way, they were going to use COVID to fail us as opposed to using COVID to give us a second chance. 

We weren’t sure which way the OEBC would swing. The only thing we were all sure about was that our class had an anomalous outlier on our hands. We had way too many failures statistically. If you ask me today why the statistics were the way that they were, I’d probably tell you that it was because of two things. The first was that there was a lingering curse of how our class of 2020 were the guinea pigs for curriculum/policy changes and the second was the actual COVID stuff.  

I state this because if you take the 2019’s pass rate out of consideration, a clear narrative can be mapped out where everyone just blames COVID. In 2020, we all got COVID and were out of the clinic for a good 6 months or so, which dulled our skills. Because of that, it can be surmised that it was the reason why our class had a pass rate of 85% only. In 2021, there was still COVID going on and people were still not fully in clinic, therefore, with less clinical training, that definitely explains the even worse rate of passing at around 82%. It was only in 2022, when COVID sort of ended for most of us, when the rate shot back up to around 93% again.

Like I said, this story kinda makes sense. In fact, this is sort of the unspoken narrative the official channels for OEBC would run. However, it doesn’t really explain why the class of 2019 had a rate that was closer to our class’s since they were doing the OSCEs before the whole COVID thing. So how do you justify that number? Well, if you just ask around, it’s pretty clear that something happened in 2019 in regards to their OSCEs. As I mentioned, the optometry field is a small and tightly knit group. So, with that in mind, if you talk to enough people, you’re bound to hear a thing or two.

It is with these rumors, that can’t be confirmed at all, where you may hear of how even before COVID happened, due to pressures from those in power who said our licensing exam was too easy, the OEBC may have tampered with the exam to make it harder. I mean, I can’t say for sure how valid this is but speaking with individuals who had done their OSCEs in 2018 versus those who had done their OSCEs in 2019, you can kind of get a pretty clear image that they were two very different exams. 

So the exam was allegedly made harder in 2019 and they wanted it to stay that way. Then in 2020, the OEBC got the perfect excuse to keep it that way in the form of COVID. If you just ignore the 2019 results, you can blame the lower results of 2020 and 2021 on COVID. From that point on, you could just keep the exam pass rate low because by the third year, people may just keep on thinking that a pass rate of around 85% is the norm. 

Unfortunately though, this is still all speculation. There is also no way for us to prove anything here since OEBC keeps their OSCE syllabus pretty vague, just like they do with their written component. Furthemore, with exam results, there is only a “pass/fail” mark for it and no explanation behind it. So, with no report after the fact and no updated syllabus prior, there is no paper trail. And with no paper trail, there’s nothing really we can do to prove anything at all. No one is the wiser when there is no feedback to those who had failed and no update on those about to take it. 

By all accounts, this was a conspiracy. Worse still was that this was a conspiracy that didn’t make too much sense. The only logic behind why someone would do this was that having a slightly higher failure rate increases the credibility of the exam, since the exam would be perceived as harder. Well…that and the fact that those who write the exam and fail would have to pay for the registration fee again.

As good as this sounds, unfortunately the theory would kind of be disproven when, after COVID died down in 2022, the rates for passing went back up to the 90% ‘s again. Those with aluminum foil hats may say that it was just a cash crab by the OEBC during COVID but now the argument is just too much of a stretch. Even so…and this is a little embarrassing. I would be lying if I didn’t say how this theory made a lot of sense to me when the initial statistics of the OSCEs came out. 

Now, don’t get me wrong, I’m not one to think too much on conspiracies. I like facts just like the next guy and don’t really build upon false pretenses. However, in a state where you are so powerless, it’s hard not to fantasize about something larger at play. The conspiracy itself took some blame off my shoulders and despite the fact that I knew it was without much merit I couldn’t really find myself daydreaming about all of it. 

Sure, this conspiracy didn’t come about when we first all got our results but if it had, I’m 100% sure the ideas behind it would have caught on fire. This kind of speaks towards the morale of the situation when our group first found out about everything. To put it another way, the one thing that really defined our group was a sense of desperation. Unfortunately, being fueled by desperation, our group went down a road of false hope. This came in the form of the OEBC Appeal Process. 

Whether this is because we were desperate or whether this was because we had some good points is up to interpretation at this point. However, just speaking from my experience, making arguments for why the results of the OSCEs should be appealed wasn’t really hard at all. There were around 15% of the class who had failed with me which meant we had around 13 heads recounting how the exam took place. With 13 heads running in desperation mode, we very quickly came up with pages and pages of bullet points for reasons why the exam results should be appealed. Most of the points fall under two categories, equipment and COVID.

I can’t go too much into detail since a lot of this is information we’re not really allowed to disclose but unlike back in 2020, the OSCEs are much more organized today and they now even tell you about some aspects of the examination on their official website. This means there are aspects of the examination I can now talk about because it is public knowledge. 

Speaking of which, our first main point of contention was about equipment. The argument is simple. No matter how good you are at a technical skill, you can’t perform that skill very well if the testing equipment is broken.

This point is pretty simple, during our exam, we had a lot of equipment failures. In fact, if you asked the group chat about equipment failures, you’ll very quickly find that a majority of us, myself included, had even filled out an incident report form during the exam reporting how the equipment we had was not up to standard. 

As a collective, our group thought that this itself was already enough to disqualify a lot of the examination since this was not an accurate take on our clinical skills. Most of our stations became how well you can perform your skills with crappy tools instead of how well you can assess the real eye. 

Apart from this, we also had a second argument in place. This one is related to COVID. More specifically, it dealt with COVID protocols. This was another strong point of contention because unlike previous years examinations, this was a first for everyone. 

How did COVID protocols affect the OSCEs you may ask? Well, there’s a lot of ways it can mess things up. The vagueness of the criteria of the OSCEs in place is very expectedly, without any mention of COVID protocols when we did our exam. There may have been a sentence or two but there was never any mention of what is mandatory for washing hands and what is necessary in terms of switching PPE between stations. These factors may not seem like much but when you’re in a station where the timing is limited to just a handful of minutes, every second counts. 

So that was the main crux of our arguments. We got crappy testing equipment and also unclear protocols for COVID which kind of threw everyone off. Now, we weren’t so delusional as to think that this would be enough to overturn a decision even when presented as a group. In fact, in general, I think what most of us wanted out of the appeal was an earlier redo of the next OSCEs. 

With COVID in place, we weren’t sure when the next OSCEs were going to be held. Normally there would be two chances per year to do it but with COVID, it was likely there would only be 1 test per year. So that means that the next time we would be able to do the OSCEs may be a full year away. This was our main problem. Nobody wanted to sit on their butts without a license for a full year. We had been delayed by COVID too much already. 

Unfortunately, our desperation shimmered a little too strongly when we eventually sat down with the OEBC representative. Thinking back, if I could tell myself one thing during that period of time, it would be to avoid this encounter. This was not a road that would help anyone. 

If anything, I would describe what happens in the next few weeks to be nothing short of the equivalent of a parasite preying on the weak. Unlike my previous encounters with the bureaucratic side of optometry however, this time the problem had a face and a name. 

Let’s call him MPH.


It’s short for The Man Peddling Hope.