ABR Oral Exams Go Virtual

Imagine preparing for arguably the most important exam of your life, one that represents the culmination of years’ worth of schooling and residency to even be in a position to take. Imagine putting in months of studying to be polished and ready for the event, only to have it rescheduled with just weeks to go. This is the unfortunate story of hundreds of ABR candidates affected by the delay of the oral exam (Part 3) in the wake of COVID-19.

Fast forward a year, and the landscape is entirely different. Given the enormous uncertainty regarding restrictions in 2021, the ABR took steps to create a virtual platform to ensure candidates can move past their studies into their professional roles. And while the pivot to virtual has become the new business-as-usual for pretty much everyone, it’s not as if porting this exam to the online world was exactly simple. After all, maintaining the integrity of such a high-stakes event involves thoughtful consideration.

But they did it. The ABR has two scheduled exam periods for the medical physics oral board exam to replace physical exams. For 2020 exam candidates, their opportunity is coming soon. On May 1-4, test takers will be a part of the first large-scale cohort to navigate the new platform. Candidates for 2021 that otherwise would be traveling to Tucson, AZ, where the exam is typically held, will have their opportunity in mid-August.

 

Converting the exam to a virtual event required some careful consideration, along with some significant format changes. Combining the ability for the candidate and examiner to see and hear each other—along with a mechanism for presenting cases—was a substantial challenge, according to the ABR.

One of the hallmarks of the ABR experience is a high level of security and integrity, which is essential for a fair and equal examination. With candidates testing from wherever (at home, the office, etc.), ensuring the same standard in the virtual environment is paramount. To meet the needs of a high-stakes exam, several measures have been developed. These include a 360-degree room scan, a front-facing webcam to mimic the in-person experience, and partnering with a third-party vendor, Proctorio, to help with security for the computer-based exam.

For the first time, the exam will be facilitated by navigators assigned to each candidate to encourage a smooth, personalized delivery. They are present alongside every candidate throughout the exam to help troubleshoot any technical issues if they arise. The navigators meet with both candidates and examiners before the live event to test the software and verify proper equipment and connectivity.

Also, two examiners will be present during each session, one serving as primary and the other standing in as a backup in case of connectivity issues. According to the ABR, the recruitment of additional examiners was a major challenge. To create this redundancy with examiners requires essentially twice the number of examiners for not just one but two exams in 2021, along with a pilot run to thoroughly test the platform.

Besides the major changes required for electronic delivery—along with a few subtle changes like longer breaks between examiners (10min) to help mitigate connectivity issues—the core structure of the exam is mostly unchanged. The testing environment was designed to be very intuitive and straightforward, similar to the on-site experience. The full exam is still a sequence of five examiners, each of whom poses five questions during a 30-minute interval, to assess the candidate’s knowledge across five broad categories based on their specialty (therapy, diagnostic, and nuclear medicine.)

 

How does the ABR know that this new delivery system will function as intended? They have a pretty good idea of the magnitude of the moment, so the ABR is taking steps to reduce the uncertainty to ensure the outcome is successful. As part of their plan, they piloted the exam for a small subset of the 2020 grouping. I caught up with one of the candidates in this cohort, Dr. Xenia Ray, PhD, DABR, and got her take on the experience.

 

Tyler:Tell me about your experience with the virtual oral board exam.

Xenia: My experience was overwhelmingly positive. There are a lot of advantages to the virtual platform in terms of time and cost savings. Not having to travel also decreased my stress from things outside of my control that might affect the exam. The virtual format does introduce new potentially stressful factors, particularly the worry that the internet connection may fail mid-exam.

Tyler: Do you think the ABR had the right protocols to address those concerns?

Xenia: The ABR and Proctorio did a lot to address those by:
Having a pre-check of my computer system weeks before the actual exam and allowing a dry run of the exam software and interactive tools. Assigning a technical navigator to each examinee. There is always someone on the line, ready to handle any glitches. Including a backup examiner for each exam session if there’s an issue with the primary examiner. Giving very clear steps about what to do if I were to lose connection.

Tyler: Was the virtual setup easy to use?

Xenia: Yes. I installed and tested it out during the pre-exam computer test. There was another opportunity to get comfortable with it on the morning of the exam before getting started. It has very little functionality, which I think is a good thing. You can draw and zoom, and that’s about it.

The one thing that was a little tricky was that most of the questions have an image associated with them and then a bulleted list of questions underneath. It might be nice if you could zoom in on just the image itself while leaving the questions viewable at the same scale. But it wasn’t a big deal.

Tyler: Did you experience any technical issues?

Xenia: My very first examiner had a glitch on his end where the questions wouldn’t show up. The technical navigator walked him through the normal steps to resolve the issue, and as soon as those didn’t work, she switched control to the backup examiner instead. The whole thing took less than 2 minutes, and the ABR built in plenty of the time with the breaks between examiners (10 min) that I didn’t have to worry about running out of time.

Tyler: So overall, it was a success? Something the ABR should consider in the future?

Xenia: Very successful! I hope they never go back to the hotel format, which is more expensive for candidates. It’s especially inconvenient for applicants with families who have to be gone for two days just to take a three-hour exam. It was also awesome that I could just enjoy the rest of my day right after the exam instead of heading to an airport.

Tyler: Any advice for those on deck to take the exam?

Xenia: They can focus on reviewing the materials and trust that the technical setup is being taken care of for them. The only advice would be to take their time playing with the tools when they do the technical check with Proctorio, but honestly, it’s very intuitive, so it is more about comfort level than really needing to practice. Also, you’re allowed to use a small handheld whiteboard to write/draw on if you prefer. I brought one because I thought writing equations or drawing diagrams on a screen would be challenging, and it was nice to have both options available.

 

If Xenia’s experience with the oral boards is any indication, the first large-scale medical physics test has a good chance of success. But what about the future of the exam? Virtual delivery could be less stressful for candidates, and an online exam would eliminate the need for expensive travel arrangements. For new or expecting parents, the ability to take the exam where it’s comfortable would be a huge relief. According to the ABR, if the exams go well this year (for all three disciplines that offer oral exams), it seems likely that they will continue to deliver the oral exams remotely in the future.

 

For years, the oral board exam has been held in Louisville, KY, at the legendary Crowne Plaza hotel. After relocating the exam to the ABR headquarters in Tucson, AZ, in 2019, the test is getting a brand new look for 2021. Given all the uncertainty we’ve grappled with over the past year, at least we know this: by the end of the year, the ABR will have two new classes of certified diplomates ready to contribute to our field. The next thing on their radar is continuing education and the Online Longitudinal Assessment (OLA)!

 

Many thanks to the ABR for providing valuable information for this blog. For more information regarding the exam, please contact the ABR Certification Managers—information@theabr.org or (520) 790-2900— or the ABR Associate Director, Dr. Geoffrey Ibbott, at gibbott@theabr.org.