Last Updated on June 27, 2022 by Laura Turner
In part 1 of this two part series we looked at some of the reasons why ADCOMs (admissions committes) are searching for a new way to assess an applicant’s personality. The correlation between an applicant’s GPA and their future success in the didactical components of medical school is well established. The new frontier is a more fair and predictive way to evaluate an applicant’s character, ethics, and communication skills. That evaluation tool seems to be found in the multiple mini-interview (MMI).
The Multiple Mini-Interview
Some logistical orientation of the MMI is necessary to better appreciate the data that drives its increasing popularity. Briefly, the MMI is a type of admissions assessment tool where an applicant is shuttled through many (between 6-12) different stations or rooms where he or she is asked to perform a certain task. The task might be to break some bad news to a friend or to role play telling a patient you made a mistake while you were operating on him, discuss a current event that has a strong ethical element, or even engage in a traditional medical school type of interview. Each station may last 6-10 minutes. Once the applicant has completed all of the stations, individual scores are assigned and a composite score may be given based on all of the evaluators’ input. This then becomes your interview score and is weighed according to the university’s policies along with your academic record and other factors.
Before getting into any more of the specifics of the MMI or how to prepare for this type of interview I’d like to go into the thinking of ADCOMs to even entertain a new type of interview format. This will shed some light on the motives of admissions committees and give you some inside information.
Reliability, Context, and Validity
There are many reasons why the traditional medical school interview does such a poor job of assessing personality and future clinical success. Let’s briefly look at each.
Reliability
When speaking about the reliability of the medical school interview the issue at hand can be summed up with the question, “Would two interviewers give similar scores after interviewing the same applicant (presumably one right after the other)?”. Reliability for the medical school interview has been reported to range from .14 – .95. That’s a huge range. It’s been hypothesized that this is due to an interviewer having access to the applicant’s academic record and the other interviewer not having that information, and/or power differences between members of the ADCOM and their resultant influences and/or the interviewer’s background and expectations.
One study showed that 56% of the difference between interview ratings has to do with interviewer variability. Meaning, it has nothing to do with the applicant. Not only is that frustrating for the applicant, but it’s unethical as well.
Context
Much research has shown that success or the ability to communicate effectively in one area does not mean that the same person can do so in another area. Just because you can talk about the effects of the printing press on Western civilization doesn’t mean that you can speak clearly about the role of the magnetic compass on sea travel. Meaning, you may communicate very well during interviews and convince the ADCOMs of your great personality but this doesn’t mean that you will communicate and behave accordingly in future clinical practice.
The converse of this would be true as well. You may not excel in the pressure cooker setting of a medical school interview but in time would be able to show yourself to be a fine physician who’s caring, compassionate, and very conscientious. However, if you’re not freed from the bias of context, you may never have the chance to prove the interviewer wrong.
Validity
This was discussed in part 1 of this series but deserves some repeating. The traditional medical school interview is about as good as flipping a coin when it comes to predicting how well an applicant will do on future clinical tests. When sticking to the strict measures of validity, the traditional interview is nearly useless.
After reviewing all the data, it’s easy to see why more and more schools are looking for better ways to assess an applicant’s personality.
Can I Get Some Help?
I think any admissions committee member who was totally honest with himself wishes that he had a “personality GPA” for each applicant. ADCOMs hope to have a single measurable data point that correlates with success in the non-cognitive areas of medical education just like the real GPA correlates with success in the cognitive/didactic areas of medical education. Up to this point such a thing hasn’t existed but the MMI is showing some signs of providing the kind of predictive power that ADCOM’s are looking for.
In the same study just mentioned, the investigators found that the MMI was, by far, the best predictor of the applicants’ future success on a clinical examination. It was a better predictor of non-cognitive success than the medical school interview, the applicant’s personal statement or his or her GPA.
The good news is that this wasn’t just some random paper. The data has been confirmed in other publications.
A 2007 study showed that the MMI, when compared to the traditional interview, personal statement and GPA, best predicted higher scores on the major clinical tests during the medical students’ training. This same study showed that the MMI also best predicted success on the “non-cognitive domains” of Part 1 of the Canadian qualifying exam. This is a test that medical students must pass to become licensed physicians in Canada. It is similar to the United States Medical Licensure Exam (USMLE).
A 2009 study showed that a correlation between the MMI score and the number of stations passed in a structured clinical exam (part of the Canadian qualifying exam) was 0.43. The value rose to 0.65 when the sub-score of patient interaction was evaluated.
Substitution, Please
When looking at all of this data one wonders why all U.S. medical schools haven’t bitten on the MMI hook, line and sinker. Yet when you count up how many U.S. schools are using the MMI it’s shockingly low. According to the MSAR, only 9 U.S. schools list the “MMI” as their chosen interview format. Considering there are approximately 167 U.S. medical schools (MD and DO) that means only a little over 5% are using the MMI.
Why such a low number?
While this is only speculation on my part, I suspect there are several reasons for the low but rising use of the MMI.
First, the ground breaking paper on the use of the MMI for medical school admissions was published in 2004. That means it’s been less than a decade since this idea was brought to the attention of medical school admissions committees. To give you some perspective, the first U.S. medical school was opened in 1765. So eight years vs. the 247 years that medical education has been around in the United States isn’t actually that long. I would just give it some more time and I think the MMI’s use in medical schools will continue to rise.
Second, the majority of the research being done on the MMI is happening in other countries, namely Canada, the U.K. and Australia. For example, 11 of 17 Canadian schools recognized by the Association of American Medical Colleges (AAMC) use the MMI. I found very few U.S. based papers that were studying the MMI at their institutions. I suspect this too will change as more and more U.S. medical schools adopt the MMI and want to substantiate its validity here.
Third, there is always risk when changing from one format to another. There’s no guarantee that what has worked elsewhere will work here and I think universities are wrestling with this issue. In addition to the risk is the matter of cost. There is cost in testing a new interview format and then implementing it. In these tight economic times, expense issues weigh heavier and heavier. Furthermore, institutional cultures are hard to change. This is true in business as well as government and education. Cultural inertia is hard to overcome but I think increasing data and U.S. acceptance of the MMI will help turn the more entrenched universities.
Conclusion
Although the MMI is far from perfect, there seems to be more and more data that this type of admissions tool can help ADCOMs better predict how a medical student will perform when it comes to the non-cognitive or more personality based tests. This is significant in that these aspects of a physician’s overall nature significantly determine how he or she will behave and succeed in future clinical practice.
Certainly there are limitations to what the MMI can predict and how far out the predictions can safely be assumed to hold true. Thus far, it appears that the reliability and validity of the MMI to better predict success during medical school is solid. How well this admissions tool can predict success as a practicing clinician has yet to be determined. Regardless, the MMI is likely to be welcomed by more ADCOM’s that must make the difficult decision of who to admit into the competitive field of medicine.
If you have any questions please email Dr. Fleenorn, author of The Medical School Interview: Secrets and a System for Success 2nd ed., at [email protected].
References
Salvatori P. Reliability and validity of admissions tools used to select students for the health professions. Adv Health Sci Educ 2001; 6:159-75.
Eva K, et al. The Ability of the Multiple Mini-Interview to Predict Preclerkship Performance in Medical School. , Vol 79, No 10. October 2004.
Kulatungo-Moruzi C, Morman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teach Learn Med. 2002;14:34-42.
Eva K, et al. An Admissions OSCE: the multiple mini-interview. Medical Education 2004: 38: 314-326.
Reiter H, et al. Multiple Mini-Interviews predict clerkship and licensing examination performance Medical Education 2007; 41: 378-384.
Eva K, et al. Predictive validity of the multiple mini-interview for selection medical trainees. Medical Education 2009; 43: 767-775.
MSAR: Getting Started 2013-2014. Association of American Medical Colleges. Copyright 2012. Washington, D.C.
Medical School Admission Requirements (MSAR). Association of American Medical Colleges. Copyright 2011. Washington, D.C.
Harasym PH, et al. Reliability and validity of interviewers’ judgments of medical school candidates. Acad Med (Suppl) 1996; 71: 40-2.
Jeremiah Fleenor, MD assesses and treats patients in the Emergency Department at Faith Regional Health Services in Norfolk, NE. He works with a team of trained medical professionals to provide emergency medical care to patients when they need it the most.
He is the author of several books, including The Medical School Interview: Secrets and a System for Success and SOAP Notes: The Down and Dirty on Squeaky Clean Documentation.
“First, the ground breaking paper on the use of the MMI for medical school admissions was published in 2004. That means it’s been less than a decade since this idea was brought to the attention of medical school admissions committees. To give you some perspective, the first U.S. medical school was opened in 1765. So eight years vs. the 247 years that medical education has been around in the United States isn’t actually that long. I would just give it some more time and I think the MMI’s use in medical schools will continue to rise”
Hippocrates was born in 460 BC which is 2473 years ago. So 8 years is really not long compared with that either. Your argument in the quote above is laughable. Comparing these two lengths of time (247 years v 8 years) says nothing more than 247 years > 8 years. You should rework your point so that it is not oversimplified and has substance.
Do you not understand the argument? It’s quite simple. Medical schools are steeped in tradition and the generic interview format has been a part of the application process for roughly 247 years. The point is that it may take more than 8 years for a majority of medical schools to change their ways. You should rework your point so that you don’t come across as such a jerk.
I had hoped that the article would provide tips on how to prepare for an MMI interview as the headline implies. Instead it discussed advantages of the MMI from the perspective of an admissions committee. Please add a third segment in which preparing for an MMI format interview is addressed!
I’m confused…is there a part III? Title is a bit misleading otherwise.
Glad that summed up how to prepare for a MMI /s
The title of the article is extremely misleading: no where does it talk about how to prepare for a MMI. Although it brings up possible situations posed, it does not address how those situations should be handled. Instead, the whole article goes into the theory as to why MMI is in place, something I as a interviewer could care less about.