Show Me About Yourself: Virtual Interviewing Platforms for Health Professions Programs

Last Updated on August 8, 2022 by Laura Turner

Health professional programs conduct interviews to predict professionalism in applicants. COVID-19 has changed the way interviews are and will be conducted (Saxena et al). Online or virtual interview platforms are proliferating as a way to reduce barriers and costs for applicants, especially those with limited resources.  However, the rapid expansion of these technologies has made it even more challenging for applicants to prepare themselves for the application process. Anxiety and confusion are common for applicants as they plan on how to best present themselves and how they’ve prepared for their next career goals.

In this article, I briefly highlight some of the more popular interviewing tools that have emerged over the last year. I encourage all prospective applicants to take the time to understand the differences in the platforms’ capabilities and how they are used to guide admissions decisions. While this article is focused on health professional school admissions, such as medical or dental school, most residency interviews are also being conducted virtually as well.

Online Meeting Platforms

Zoom, Skype, WebEx, and other meeting platforms

In the immediate aftermath of the lockdown mandates in spring 2020, schools quickly converted their regular in-person interview schedules to a virtual platform.  Admissions staff and faculty had to quickly learn the nuances of setting up and interacting in a virtual environment. Budgets reallocated money from interview day lunches for more technical support and bandwidth.  The virtual format also allowed interview days to be more focused on the assessment of the applicant rather than the “marketing” of campus resources and community.

However, sites that traditionally held interview days with a large number of students or utilized multiple-station/multiple mini-interview (MMI) formats found technical limitations challenging and struggled to effectively run their assessments.  Taken for granted were printed schedules, signs, and interview scoresheets that could easily be distributed.  Bandwidth or connection issues introduced more uncertainty and unanticipated challenges to running an interview day smoothly.  Anecdotally, many candidates and faculty participated in their interview days at nearby coffee shops, libraries, or in parking lots.

As these difficulties were not limited to just medical school interviews, many online meeting platforms developed modules to allow interviews to be more easily managed by non-technical staff.  For example, to assist medical schools, the AAMC leaned on its relationship with HireVue (who was developing the AAMC virtual interview platform for residency programs) to develop the Virtual Interview Tool for Admissions (VITA), a short 30-minute asynchronous video assessment that many schools piloted but did not find significantly helpful.

As the “new normal” has started to emerge, many different platforms (Broder) are now available to support job interviews, including ones that facilitate synchronous and asynchronous MMI formats.

Altus Assessments: CASPer, Snapshot, and Duet

Developed by faculty members at McMaster University (Dore et al), Altus Assessments have marketed an online assessment known as the Computer-based Assessment for Sampling Personal Characteristics (CASPer) based on the Objective Standardized Clinical Examination (OSCE) and the on-site Multiple Mini-Interview format.  (Click here for all previous articles that we published; we also have a webinar.  Altus also published an article about virtual interviewing on their website.)  Test-takers review a text-based or video-recorded situation, and then are given five minutes to type an answer to three open-ended questions upon reflection or considering next steps.  An overall score is calculated based on responses from raters specifically trained and calibrated by Altus Assessments. This score is released to the schools while the quartile range is disclosed to the test-taker.

Last year, Altus introduced Snapshot and Duet. Snapshot is a structured one-way video interview that gives test-takers an opportunity to bring their personal statements to life by answering questions that shed light on their motivation for the profession, communication skills, and self-reflection.  In this case, the Snapshot responses can be reviewed by faculty and scored. The results are not disclosed to test-takers. 

Duet gives test-takers an opportunity to reflect on what they are looking for in their education by declaring their preferences in pursuing a medical program and a career in healthcare (e.g., curriculum, mission-based opportunities, and student support). Medical schools and residency programs also complete Duet to generate a program Profile, which is what applicants’ responses are automatically measured against to generate a score on value alignment.  By comparing what applicants value in a program to what the program actually has to offer, a better match can be made between program and learner, ensuring a learner finds the right environment in which they can succeed.  

Signing up and paying for a CASPer test requires registering for the test associated with the programs that require it as early as April of the application year.  The fee also covers Snapshot and Duet for the programs requiring these additional assessments.  Results are usually available to schools within two weeks after administration.  Candidates are required to pay a fee for scheduling the exam, and they can only take this exam for each designated school/program once per application cycle.

N.B. I acknowledge and appreciate Altus Assessments for providing us details about these assessments for this article.

AAMC Situational Judgment Test

Under development over the past few years, the AAMC began pilot-testing its Situational Judgment Test (SJT) in the 2020 entering class application cycle with two medical schools and six medical schools for the 2021 entering class (AAMC webinar from April 21, 2021, recording at https://vimeo.com/540650171, accessed on November 17, 2021).  Using the Yardstick platform (Meazure Learning), AAMC has developed a psychometric exam to assess a test-taker’s understanding of the pre-professional competencies for entering medical students.  

The AAMC team provides a robust resource website (Association of American Medical Colleges) with presentations and handbooks to guide students in scheduling examinations and addressing concerns over the delivery of the exam.  Similar to the MCAT, the SJT examination time window runs from April to September but does not require secured proctoring at a testing venue.  Test-takers and participating schools receive scores and comparative performance data usually within 30 days of the test date.  Like the MCAT, test-takers also are given an opportunity to void their exam attempt at the conclusion of the SJT.

The SJT itself is remarkably different from the CASPer exam.  There are 30 different scenarios, each with at least five follow-up questions that consist of a proposed follow-up action (a total of 186 question items).  The test-taker must decide whether the action is effective or not. Credit is awarded based on how closely the selected answer matches a consensus response from calibrated medical education experts during question testing. Partial credit is awarded if the test-taker’s response is close but not exact though it is not clear how this is awarded.  Each test-taker’s overall score is based on a nine-point scale, and most scores have fallen at the middle of the range.

For the pilot years, participants were not charged to take the AAMC SJT, and it is not clear whether this test will be more fully embraced by other allopathic medical schools’ admissions committees without some internal pilot testing and validation.  Furthermore, applicants who previously took the SJT in 2020 were not apparently required to retake for 2021; while multiple scores could be disclosed to the participating schools, schools will likely weigh the more favorable result (AAMC webinar).  Nevertheless, given the resources invested in the development of this assessment with the stated goal to provide an additional reliable measure for pre-professional competencies in applicants, the AAMC SJT is likely to be part of the screening/pre-interview selection process for many allopathic medical school applicants.

School-run Virtual MMI Assessments

ProspectHR MMI, SortSmart, and Kira Talent

One major disadvantage of these interview tools is that admissions committees are unable to tailor the AAMC SJT or CASPer assessments to the needs of their school. Different schools prioritize different key characteristics and competencies among their applicants. Customization is not possible with the AAMC SJT or CASPer assessments.  

A few vendors have developed online platforms that allow for customization of MMI questions, incorporation of evaluation rubrics, and a smoother management interface for an optimal experience during the interview day for the candidates and interviewers.  Many of these companies offer asynchronous options as well as live interview day platforms, with opportunities to check on technical requirements and capabilities in advance.  Applicants often pay for their participation through their secondary application fee, and an applicant may have to do multiple separate assessments if they are invited by more than one program to participate.  Employers or schools determine when these assessments are made available to applicants so that their application review schedule proceeds in a timely manner.

These platforms can deliver both text-based and video-based situations for candidates to address. Administrators can set the amount of time to submit a response either in a written format or on camera.  Admissions committees can review the consistency of scoring among evaluators to ensure some level of calibration during candidate review.  Systems that ensure assessment integrity are also designed into the platform to help investigate any irregularities from technical problems to potential plagiarism or illegal assistance.  Finally, the platforms — like the other assessments — are compliant with all standards to ensure accessibility for those with disabilities.

When invited to participate, candidates usually need to set up an account with the platform vendor to access the secure platform and provide assistance.  Usually there is an opportunity to test the technical capabilities of one’s computer system before taking the assessment.  This can include some practice questions in addition to a microphone and camera check-up.  As the experience should be treated like a bona fide on-campus interview, candidates will not be sent the results of their assessment.

During the COVID pandemic, these companies made marketing pitches to schools and application associations to incorporate their virtual interviewing platform for the next few years.  Integrations with application management programs often used by admissions offices were developed and validated.  Notably in November 2021, Educational Testing Services (ETS), which administers the SAT and AP exams, bought a majority stake in Kira Talent, so it is likely that SJT assessments will expand to other higher education admissions processes, including college/undergraduate admissions as they already have for traditional employment and hiring.

Call to Action: Integrate SJT prep before and while you apply

Other than the AAMC SJT, many of these online tools are being used in other health professions programs beyond medicine, such as dental, pharmacy, physician assistant, optometry, and even some postbac programs.  Thus, all pre-health students should anticipate having to complete multiple virtual interviews on potentially different platforms in one cycle. Additionally, with the COVID-19 pandemic waning, in-person interviews are still possible even after completing an online assessment.  Virtual interviews allow for significant cost savings in travel, housing, and potentially clothing. However, applicants must budget additional time to prepare for and fully participate in each interview day, invest in significant technology resources, and identify secure locations to complete these assessments.  Additionally, many prospective candidates should take advantage of recruitment events on campus offered by admissions staff before they begin applying. These networking opportunities could help candidates with specific insights into the desired characteristics that will be valued in these assessments.  

While AAMC has been generous in its information about the SJT for all medical school applicants (Association of American Medical Colleges), much less information is available to help prepare applicants for the other assessments or how these results contribute to making admissions decisions.  To this end, Health Professional Student Association (HPSA) and Student Doctor Network (SDN) have partnered with PrepMatch in nurturing a student-run community that provides free preparation materials and peer feedback for the CASPer exam. Furthermore, HPSA and SDN will seek and promote opportunities and partnerships to help prospective applicants for all health professions programs with limited resources prepare for the SJT assessments they may have to take during their application process.

Any included links are provided for informational purposes and not indicative of endorsements on behalf of SDN.

Works Cited

Association of American Medical Colleges. “AAMC Situational Judgment Test.” AAMC Situational Judgment Test, 2021, aamc.org/sjt. Accessed 17 November 2021.

Broder, Carmen. “10 Video Interviewing Platforms For Virtual Interviews [2021].” People managing people, 29 July 2021, https://peoplemanagingpeople.com/tools/video-interviewing-platforms/. Accessed 17 November 2021.

Dore, Kelly, et al. “Extending the interview to all medical school candidates: computer-based multiple sample evaluation of noncognitive skills (CMSENS).” Academic Medicine, vol. 84, no. 10, 2009, pp. S9-S12. Academic Medicine, https://journals.lww.com/academicmedicine/Fulltext/2009/10001/Extending_the_Interview_to_All_Medical_School.4.aspx. Accessed 17 November 2021.

Saxena, Anurag, et al. “Incorporating situational judgment tests into postgraduate medical education admissions: examining educational and organizational outcomes.” Academic Medicine, vol. 96, no. 11S, 2021, pp. S203-S204. Academic Medicine, https://journals.lww.com/academicmedicine/Fulltext/2021/11001/Incorporating_Situational_Judgment_Tests_Into.60.aspx. Accessed 17 November 2021.