The Changing Face of Medical School Admissions: Restructuring the MCAT (Part I)
By Sameer Apte
SDN Senior Moderator (MCAT Forum)
Modern medical education is a surprisingly young field. If you were to travel back only 90 years and enroll in medical school, you would find yourself in the midst of the greatest paradigm shift in the history of American medical education.
At the time, medical training curriculums were non-standardized, schools were largely profit driven, physicians were required only to have a diploma to practice, and the selection criteria for medical students were either sparse or non-existent (1-3).
At the request of the newly formed Council on Medical Education (CME), Abraham Flexner, a noted educator and scientist, visited every medical school in North America and wrote a scathing review of their educational practices (2). The “Flexner Report”, as Flexner’s review is commonly called, was the first step in the standardization of medical education.
Flexner’s findings, backed by the CME’s influence, not only prompted a re-evaluation of how medicine was taught, but also who it was taught to. Physicians-in-training were often not required to have a high school or university education, and entrance tests were proprietary and variable in rigor (1-3). As a result, attrition rates for medical school freshmen ranged from 5% to an astounding 50% in the 1920s (3). Furthermore, the quality, or lack thereof, of those that did graduate was harshly regarded by Flexner himself (2).
In 1928, in an effort to ensure the readiness of matriculating medical students, F.A. Moss created the “Moss Test”, the first standardized Medical Colleges Admissions Test (MCAT) (3, 4). Through a series of true/false and multiple choice questions across six to eight subtests, an applicant’s memorization ability, scientific vocabulary, reading comprehension and logical reasoning were evaluated (4, 5). In conjunction with the admissions reforms suggested by the Flexner Report, the implementation of the MCAT slashed medical school attrition rates to 7% in 1946 (3, 6).
Since Abraham Flexner’s time, the MCAT – administered by the Association of American Medical Colleges (AAMC) (7) – has undergone a number of major restructurings, each one aimed at adapting the test to better suit the medical landscape of the era (3, 8).
The second iteration of the MCAT came in 1946 and featured the section “Understanding Modern Society” (later renamed “General Information”) (8, 9). The inclusion of this sub-test reflected the emerging idea that physicians should be well versed in areas “over and above those which may be related to medical school grades (3)”. Medical school admissions committees, however, reportedly placed the greatest importance on the scientific sections of the MCAT, and did not give the general information sub-test much credence (3, 10).
In the 1970s, the AAMC put forth substantial effort to enhance the ability of the MCAT to assess personal qualities. “Compassion, coping ability, decision-making ability, inter-professional relations, realistic self-appraisal, sensitivity in interpersonal relations, and staying power-physical and motivational” were characteristics deemed important for practicing physicians (3). The project, however, was abandoned; in the MCAT of 1977 to 1991, the general information section was eliminated, and the remaining sub-tests covered only scientific knowledge, reading comprehension, and quantitative skills (11).
Today’s MCAT features three separate multiple-choice sections covering physical sciences, verbal reasoning, and biological sciences, as well as a long answer writing sample section. Results from each section of the MCAT are statistically analyzed and a grade distribution is generated that closely approximates a bell curve.
The multiple choice sections are scored on a scale of 1-15, while the writing sample is scored on a scale of J-T. Because the MCAT aims to accurately assess a very wide range of applicants, the AAMC goes to great lengths to reduce the effects of cultural and social bias, as well differences in testing conditions. Thus, the performance of examinees on the MCAT can be reliably compared (12).
For 18 years, the current structure of the MCAT has been the mainstay of standardized testing in medical school admissions; however, as the face of medical education continues to adjust its mission and objectives, the AAMC will continue to review and revise the exam (13). For instance, concordant with the recent integration of modern technology and medical practice, the MCAT has moved from a paper-and-pencil format to a computer-based format (14).
More recently, in October of 2008, a committee to comprehensively review and re-design the MCAT for the fifth time was commissioned by the AAMC (15). The Committee, called “MCAT review 5″ (MR5), consists of current physicians, medical school administrators, faculty members, and medical students. Because the MR5 committee is still in the evaluation process for the current MCAT, it is not known how the test will change, only that the changes will be drastic16. If MR5 meets its projected time-line, MCAT examinees in 2013 will most likely be faced with a standardized exam that bears little resemblance to the MCAT of today.
It is in the criticisms of Abraham Flexner and the creation of the Moss Test that the origins of the MCAT can be found. Since that time (only 90 years ago), the MCAT has kept pace with the remarkable evolution of medicine by constantly changing its own nature. With the advent of the MR5 committee, the time has come, once again, for the MCAT to don a new face. Although it may soon be unrecognizable to those of us who took it today, the MCAT will always be a part of the medical school admissions process, in whatever form it may present itself.
***Be on the lookout for an upcoming article revealing what might be in store for the future MCAT. MR5 committee members, MCAT administrators, and other medical education experts will give the Student Doctor Network information and insight.
Citations:
1. Beck, A.H., STUDENTJAMA. The Flexner report and the standardization of American medical education. Jama, 2004. 291(17): p. 2139-40.
2. Flexner, A., Medical Education in the United States and Canada (Bulletin Number Four). 3rd ed, ed. T.C. Foundation. 1910, New York: W.B. Updike & The Merrymount Press. 1-363.
3. McGaghie, W.C., Assessing readiness for medical education: evolution of the medical college admission test. Jama, 2002. 288(9): p. 1085-90.
4. Moss, F.A., Scholastic Aptitude Tests for Medical Students. J Am Assoc of Medical Coll, 1930. 5(2): p. 90-110.
5. Moss, F.A., Report of the Committee on Aptitude Tests for Medical Schools. J Am Assoc of Medical Coll, 1941. 16(4): p. 234-243.
6. Mullin, F.J., Selection of Medical Students. J Am Assoc of Medical Coll, 1948. 23(3): p. 163-170.
7. AAMC. Official Medical College Admissions Test Website. 2009 [cited 2009 March 4th, 2009]; Available from: www.aamc.org/mcat.
8. Erdmann, J.B., Separating Wheat from Chaff: revision of the MCAT. Acad Med, 1972. 47(9): p. 747-749.
9. Young, R.H. and G.A. Pierson, The Professional Aptitute Test, 1947 A Preliminary Evaluation. J Am Assoc of Medical Coll, 1947. 23(3): p. 176-179.
10. Glaser, R.J., Appraising Intellectual Characteristics. J Med Educ, 1957. 32(10): p. 31-45.
11. McGuire, F.L., The New MCAT and Medical Student Performance. Acad Med, 1980. 55(5): p. 405-408.
12. AAMC. MCAT Essentials. 2009 [cited 2009 March 4th, 2009]; Available from: http://www.aamc.org/students/mcat/mcatessentials.pdf.
13. Mitchell, K., R. Haynes, and J. Koenig, Assessing the Validity of the Updated Medical College Admissions Test. Acad Med, 1994. 69: p. 393-401.
14. AAMC. Medical College Admission Test will Convert to Computer-Based Format. 2005 [cited 2009 March 4th, 2009]; Available from: http://www.aamc.org/newsroom/pressrel/2005/050718.htm.
15. AAMC. Medical College Admission Test to Undergo Review. 2008 [cited 2009 March 4th, 2009]; Available from: http://www.aamc.org/newsroom/pressrel/2008/081016.htm.
16. AAMC. AAMC Launches Review of MCAT Exam 2009 [cited 2009 March 4th, 2009]; Available from: http://www.aamc.org/newsroom/reporter/feb09/mcat.htm.
I’m curious as to why they feel the need to restructure the MCAT. With the almost non-existent failure rate of current medical schools, I don’t think the problem lies in the fact that schools don’t have the right test to assess the ability of students to do well/succeed in Medical School. I just want to know where the concern stems from.
RE: Kaustikos
Even though students are not failing medical school, this does not imply that there aren’t “better” students for the job out there. The definition and make-up of a “better” student changes with time, because the profession and society change with time. A new MCAT could be designed to better gauge students for future medical practice. I think that the primary concern should not be to find students who can succeed IN medical school, as you say, but rather to find students who can succeed in the medical profession AFTER medical school (which is very different in many aspects). Currently, a student’s MCAT score is a rather poor predictor of his future success as a physician – and I see the merit in designing a more relevant test.
Where does this leave students who plan on applying around that time, but still take the older version? Will the work gone into preparation be wasted, as takers of the new exam are preferred? This could possibly mean that we would have to take two different types of test facing a stricter evaluation than others only having to take one test. This mostly applies if the first cycle applied to does not lead to matriculation.
Why not look to models that currently exist elsewhere such as the GAMSAT or the UKAT? These exams seem to focus more on general critical thinking/problem solving. Has the AAMC ever tested the relevance of the MCAT by administering it to upper-class medical students? I would think that scores would drop precipitously because a great deal of the material tested on the MCAT is not relevant in medical school, and is, therefore, forgotten.
This is a valid point, especially in the light of the pre-requisite courses required for medical school (really the lack of compared to other health professional programs). Thus, as most medical schools do not require anatomy/physiology and other medically relevant courses, one has to assume their logic for doing so is that students get the relevant coursework in med school, thus, it is not necessarily needed before hand. IF this is the case, then how is testing one’s knowledge on physics, gen chem, organic, and even some of the bio relevant to medical school success but even more importantly, to the success as a physician post-medical school? Perhaps exams such as the LSAT and the like, where the skills tested are not how much material you can cram into your head, but rather how you can reason and logically and critically think about scenarios and virtual real-life situations, would be better predictors of success in the actual field of medical practice. After all, isn’t the translation from text-based knowledge to clinical skills based knowledge what really counts in the end?
The current MCAT is actually supposed to be testing logical and critical thinking skills. The relevance of physics, gen chem, organic, and bio is not the actual scientific material – it is simply used as a (somewhat arbitrary) standardized set of material that students are expected to be expert with, from which critical thinking questions can be drawn from. In fact, memorizing every detail of physics, gen chem, organic, and bio generally does not get you a good MCAT score – it is the ability to combine that memorization with further critical thinking that gets good MCAT scores. This sort of abstractly mirrors the process of differential diagnosis and logical deduction that real physicians must do. I think this is the concept of the MCAT – each passage is like a case, and each question is like a mini differential diagnosis. But, I think the actual implementation of this in the MCAT is rather poor and could be improved dramatically, simply by making passages more case-like and questions more diagnosis-like. The fact that the material itself is not medically related is fine – you will learn the medical stuff in medical school, but the ability to think critically and logically is something more generic that should have been developed prior to medical school, and is not the purpose of medical school. I also think the MCAT should put more emphasis on 3-dimensional spatial thinking like the DAT for dental students does, as well as develop an innovative way to gauge a student’s ability to manage time, social skills, and context-dependent pattern recognition – all of which are admittedly kind of hard to do on a paper test.
Response to Ray
I would agree that the goals of the MCAT are more or less sounds, though as you pointed out, such implementation has not been brought about sufficiently. It goes back to the fact that the MCAT needs to be attempting to not so much evaluate crude intelligence, especially with today’s scores that are becoming almost as inflated as national GPA’s, but more so the necessary overall skills that transform all of the book smarts into the critical thinking, analytical, social, humanistic, and interpersonal skills that truly define a successful physician in practice. Seemingly, getting through medical school is not as hard as some may think, but the more important subsequent practice of medicine part is the real challenging part. And sadly, I think lay society is beginning to notice this as more and more patients are apparently being let down a great deal due to lack of mastery of the applicable part of medical training….Thus, as you mentioned as well, this is where more attention ought to be placed both in the MCAT and the general admissions process; unfortunately, that is not an easy task to translate into an actual working model.
What are the criteria of a successful practicing physician? What are the metrics used to gauge such a person? What is it about (some) currently practicing physicians that is lacking that leads those in charge of the MCAT to believe that the test needs to be able to predict that deficit and prevent those individuals from becoming physicians (if indeed that is the goal of the redesign)?
RE: MIssy
The goal is not to prevent anybody from becoming physicians. But, the reality is that there are only limited medical schools with limited seats, in the face of a vastly growing number of students applying to medical school. The goal is to identify who would best deserve those seats, by predicting how much they are likely to contribute to medicine itself or to society if they are trained to be physicians, and in what forms those contributions are likely to be. The MCAT is only one metric used in this pseudo-scientific complex ranking (GPA, interviews, research, community involvement, etc are some other metrics useful for gauging different things). There is no precise definition of a “successful” physician, because it is subjective, changes with time, and is only meaningful when compared with other physicians. But, on a subjective level, there are many ways to be a “successful” physician: an accurate diagnostician, or a respected mentor, or a visionary team organizer, or a disease-curing researcher, or a skillful surgeon, or a bringer of healthcare to the poor – these are all some ways “success” is measured by society. Many of these factors are orthogonal to each other, so perhaps one successful physician is strong all-around while another successful physician is especially focused in one strength. The problem all comes back to giving the limited number of prized seats to the most deserving students – it’s not a question of who has deficits and is unsuitable for the job, it’s a question of who is better for the job and in what ways. I’m sure you could think of quite a few people who, when comparing apples to apples, are worse at their job than others. The goal is simply to narrow that gap as much as possible. At least this is just my view.
The comment by Missy underscores the mere fact that, as Ray has pointed out, today it is not so much a question of who CAN or WILL succeed in both medical school and as a practicing physician. Rather, it is about limiting admittance to the most qualified (in all aspects) and to those with the highest potential to go above and beyond to contribute to modern medicine and become a respectable part of a highly distinguished career in medicine. What that exactly looks like is not always clear nor measurable and it definitely is not the same for every physician. Needless to say, it goes back to the reality that, really, the majority of medical school applicants would probably more or less pass medical school and boards just fine; but what about the subsequent years participating in the endless journey of becoming a great physician? That is the real question that the writers of the MCAT and more importantly, medical school admission teams ought to be answering with much thought and consideration.
Really appreciate the responses here from ray and Derrick–great insight. I have a few questions I hope you won’t mind answering:
I can envision how the new computer-based MCAT could test spatial orientation with 3D modeling, but how would test interpersonal skills be tested? Isn’t that what the interview is for? Will there be a mix of behavioral interview type questions on the new MCAT, asking how the test-taker would handle each situation? I think the only way to really gauge the way an applicant interacts socially is in person, as marking the correct response on a test doesn’t guarantee that the same behavior would ensue in real life.
The current MCAT does a good job of testing logic and critical thinking. In this increasingly litigious society, sensitive doctors seem to be favored over more blunt doctors, possibly reducing malpractice lawsuits. In addition, with decreasing reimbursements it would probably be a good idea to select for people who would not mind taking a drastic pay cut. The only way I can think of to test an applicant’s interpersonal skills would be to have a test similar to the clinical skills test in medical school, where actors play the roles of patients. But premed students clearly have not yet learned how to assess patients.
It seems like the current system of secondary essays expanding on personal qualities, recommendation letters, and the interview are good ways to select for humanistic qualities. I am curious as to how a computer-administered test could be designed to do this, as those same qualities are inherently based on feeling rather than thinking. Tests can measure lots of things but humans are the best gauge of other humans.
As the healthcare system evolves, the entire medical school admissions process needs to be restructured.
It’s quite funny thinking about the MCAT and those who will be taking it. A great number of people entering medical school are young, with little to no life experience, and very limited medical/patient experience (perhaps non outside of a shadowing experience). This standardized test may do a decent enough job in measuring logical, analytical, and gross memorization skills, but it has little to do with predicting one’s potential as a physician. Furthermore, I doubt that one could actually guage this with such finite test and I doubt that is the aim of it anyways.
The caliber of a physician is forged throughout that person’s entire socialization process leading up to where they might be at that specific time. It is a very subjective to term a physician “great” or anything for that matter. Each physician will have a their particular niche in the fabric of medicine.
But since they are changing the testing process it would be great if they could give some shoulder massages PRN.
I think it is important to have a good way to determine the ability of a prospective medical student. The MCAT is great in determining the scientific ability of a student as well as their ability to process info quickly. However, books, shows, and some first hand knowledge portray doctors in training as unhappy/burned out and even suicidal. Just as college isn’t for some people, perhaps medicine isn’t for others. I believe that there should be some way to discriminate those who can handle the stress and those who can not. Some may say that shadowing is the best way to determine if medicine is a person’s desirable career choice. I disagree! Getting into medical school is, in large part, is a long obstacle course comprised of jumping through hoop after hoop (shadowing being one of the hoops). Shadowing is merely and activity that a prospective med student has to check off their list of unofficial requirements to get into medical school. There needs to be a better way to determine a person’s ability under high pressure.
Joshua you don’t think the MCAT is a high pressure situation?
One misconception that several people have made on this post is that the subjects tested in the MCAT are arbitrary. They are there to match to prerequisite classes established over ninety years ago for medical school. If they were arbitrary, they could test on any subject they want and say that if you can’t think critically about this issue you probably wont think critically as a doctor. Obviously critical thinking is not that simple. The real issue is whether or not the current prerequisite classes must stay. Ninety years ago the world’s knowledge of physics, chemistry and biology was very different than our modern knowledge. Many professors and admissions committees still like the ochem requirement, not because it has anything to do with medical practice but because it is a good filter for hard workers. However, there are many difficult classes that could be made as national prerequisites that are difficult, they just haven’t gotten around to changing that part of the system.
Realistically, in modern medicine it would be more reasonable to require students to take classes in inorganic chemistry, biology, molecular and microbiology, genetics, histology, statistics, nutrition, anatomy and physiology, business, statistics, computer science, and/or behavioral sciences. These would be more related to actual medical knowledge and practice but would require a complete change in the way admissions committees view applications and how the MCAT is structured. Some people say the important thing is find critical thinkers and not just students who have a great deal of medically related knowledge. However, if the point is just have difficult classes that require critical thought regardless of whether or not they are related to medicine, why not require dance, calculus, a course in quantum mechanics of language courses (yes I know some schools already require calculus.) These courses are difficult and require critical and creative thought. The issue is that we need relevant critical thinking. The prerequisite courses ought to be changed because they make little sense and the MCAT should change to follow suit.
Joseph Kim is correct. The landscape of medicine is changing. Medical school must keep pace if they are to meet the needs of patients.