All About Competency: Part 3

Did you ever like carnival mirrors?  It’s often funny to see how these mirrors exaggerate various body parts to make you look like you have a short body (dwarfism) or an enlarged head (megaloencephaly).
The distorted view is often as entertaining as the game of comparison obsessively played by many prehealth applicants.  Way too often we measure ourselves by the schools we attend, the grades we made, the research we’ve performed, the clinical experiences we’ve had, the trips we’ve taken, and the clubs we’ve joined.  While often there may be some who enjoy one-upping others in their achievements, the echo chamber effect often makes it hard for individuals to really see the impression they make to others in the admissions process, and it really is this difference that can doom applicants.
In the previous articles (Part 1 and Part 2), I outlined competencies that I have used to holistically evaluate applicants for health professions programs.  For this article, I give everyone an idea of where advisees think of their own competencies in general, and how it compares to solicited references and to interviewers.

Background

Very few of us are born with expertise in a competency.  Often a skill requires some degree of training, consistent practice and competitive application in order for one to become a true master of that skill.  While we are born with certain talents or perspectives that may assist us in our development of that skill, we usually rely on others to coach us or evaluate us to become better.  Our progressive development in this competency must also be frequently assessed using quizzes and exams.  For each competency, we find ways to scale the learning curve, and it is where we are on the curve that determines our competency in a skill set.
In the system used for my institutional evaluations, I have defined six levels of competency for each of the major criteria.

1. Naïve/Newcomer: An individual exhibits little observable knowledge, skill, or sincere interest.
2. Intermediate: An individual has received minimal but not sufficient training to exhibit a core set of knowledge, skills, or interest.
3. Proficient: An individual has completed sufficient training (usually through a set of required classes) to reliably reproduce a core set of knowledge and skills, but must receive further training when confronted with situations where the training is applied.
4. Confident: An individual is competent in an above-average set of knowledge and skills, and demonstrates appropriate confidence in adapting to new situations that test the skill set beyond what is normally covered in a typical set of classes.
5. Master: An individual demonstrates consistent excellence in a set of knowledge and skills and can appropriately seek affirmation and criticism to independently develop new knowledge and skills.
6. Expert: An individual has received external validation of superior quality in a set of knowledge and skills, and is considered an innovator, leader, or authority in that area.

You can see variations of this learning curve in so many activities whether it is the color of a belt by someone engaged in the martial arts or one’s passion for understanding the societal factors behind health care disparities.  An individual who has a passing interest through informal volunteering may simply be a novice or intermediate, but that interest may drive that person to take classes in sociology or history to reach a better understanding.  A master or expert can clearly see situations where the textbook cases learned in those classes can be applied.  The most important change in medical education is the belief that one’s development of clinical competencies must begin much earlier.  Thus, applicants must be able to recognize themselves as ready as possible in their personal and pre-professional competencies and articulate the learning curve they ascended for their evaluators.

Self-Assessment of Pre-health Advisees

Every year, as a requirement for access to pre-health advising, I require all my advisees to complete an annual survey and maintain their connections with my advising system.  For the last two years, I have asked each advisee to perform a self-assessment in the 10 major competencies that are used to evaluate each prospective applicant for a committee letter.  This self-assessment is also done in a final request for a committee letter (pre-application) performed the winter before an applicant begins to submit an application.
The pre-applicants then have separate one-on-one interviews with members of my committee while solicited references are independently collected.  While solicited references usually consist of professors, clinicians, and workplace or volunteer supervisors, interviewers are members of the University community with a vested interest in student development, health care, or education – but with whom the advisees mostly did not interact.  The interviewers thus share the perspective of the “common patient” and the general community who have an interest in our future health care workforce [http://med.stanford.edu/ism/2011/january/interview-0110.html ].
In analyzing the data from 2009-2010, I found some interesting shifts of competency perception among all my advisees.  Looking at the “academic foundation” competency, I expect that most incoming freshmen would declare themselves as naive in this area, while most seniors would be more “masters”.  Indeed 40 of the 70 freshmen rate themselves no higher than “intermediate” versus 8 of 72 seniors.  What interested me was that 8 of 36 sophomores rated their academic foundation competency no higher than intermediate, and not just because of the number of respondents dropped.  In contrast 17 of 64 juniors and 3 of 46 alumni claimed a naïve-to-intermediate competency with their academic foundation.  Apparently within a year, those freshmen who continued on as pre-health advisees believed they had a proficient-to-master competency of the academic foundation for a successful health professional career.  138 of 172 applicants claim themselves as master or expert in academic foundation.
What is somewhat surprising is the “intrapersonal intelligence” self-reported competency.  60% of the freshmen identified themselves as “confident” or better.  This proportion grew to 80% of sophomores, 88% of juniors, 93% of seniors, and 96% of alumni.  If I raised the bar to masters and experts, the proportions were 34% freshmen, 53% sophomores, 59% juniors, 75% seniors, 71% alumni, 80% postbacs, and 80% for applicants.  Interestingly, the greatest concern expressed by admissions committees seems to be the lack of apparent competency in the interpersonal domain (professionalism, integrity, ethics, and personal management) among applicants that are interviewed.

External Assessment of Pre-health Advisees

Solicited references and interviewers were also asked to rate the competencies of applicants they reviewed.  Solicited references tended to rate applicants very highly in all competencies, giving over 90% of the applicants as master or expert in all domains including academic foundation and intrapersonal intelligence.  There is a lot of debate on the usefulness of letters of recommendation in the admissions process, especially if the only letters written highly praise and maybe over-exaggerate the applicants.  By having a system where everyone is excellent, excellence becomes average, and these data further confirms other studies that show the minimal impact solicited letters may have in admissions decisions.
In contrast, interviewers rated 50% of potential applicants they interviewed as master/expert in each domain including academic foundation (82/193) and interpersonal intelligence (99/193).  In the end, 23 of 104 applicants were given one of the two highest levels of recommendation from the committee.
How the committee rating is used in admissions decisions depends highly on the school, so evidence needed to be compiled about the importance of a committee evaluation in the admissions process.  That will be discussed in a future article.

Post-Script: Applying Competencies to Emotional Intelligence

The leaders of medical education all recognize that the old set of “core coursework” in premedical education, established with the Flexner report over a century ago, is inadequate for the challenges of present-day and future medicine.  Many schools have begun to clearly articulate that the typical core coursework sequence (one year of general biology, physics, chemistry, and organic chemistry) must include core content which now includes at least a course in biochemistry and college math.  The mistake in evaluating applicants has been to ascribe mastery and expertise in the grades earned in the aggregate of classes one takes.  In the professional world, one maintains that expertise (and one’s license) through professional practice, external advice, reflection, and continued education.
The same analogy should be applied for emotional intelligence.  By our nature, all humans express and manage emotions to a level where we individually are comfortable.  But the competency learning curve dictates that a minimum set of concepts must be clearly understood to be proficient, and the variety of challenges that hone those skills further help one ascend the learning curve.  For many applicants, they feel they have already the broad experience to be considered a master of emotional intelligence: that we are all able to handle honest criticism, that we are resilient in the face of enormous challenge, and that we have a value system that stands up to the most challenging of situations.  In light of so much research indicating how medical students lose empathy and become more depressed in their clerkship years, I have to suspect that the compasses that guide us in emotional intelligence often need calibrating prior to entering medical school.
So ask yourself what external standards have calibrated your own emotional-intelligence compass towards a professional career.  How often do you seek advice from other professionals interested in your emotional intelligence (such as a psychological counselor)?  How often do you reflect on yourself and your experiences in both good and bad times, and have you taken advantage of your institution’s resources for health and wellness? In other words, how do you see your reflection in your mirror, and is it how others (especially patients you never met before) see you?

Example 1: Academic Foundation

Figure 1.  Academic Foundation: EY2011 Self-Assessment and Evaluation. Ratings from applicants, their professors from whom they solicited letters of recommendation, and independent committee interviewers on competency level for academic foundation among applicants for the entering class of 2011 (all health professions) on a percentage basis.  Note that independent committee interviewers distinguish the lower end of the competency curve better than applicants or solicited professors.  This has meaning for anyone going to interview at schools where they have no or minimal contact with their interviewer.


Table 1. Self-Evaluations and Assessments on Academic Foundation

Example 2: Emotional Intelligence

Figure 2.  Emotional Intelligence: EY 2011 Self-Assessment and Evaluation. Ratings from applicants, their professors from whom they solicited letters of recommendation, and independent committee interviewers on competency level for academic foundation among applicants for the entering class of 2011 (all health professions) on a percentage basis.  Note that independent committee interviewers distinguish the lower end of the competency curve better than applicants or solicited professors.  This has meaning for anyone going to interview at schools where they have no or minimal contact with their interviewer.

Table 2. Self-Evaluations and Assessments on Emotional Intelligence
Emil Chuck, Ph.D., is the Health Professions Advisor and Term Assistant Professor of Biology and Bioengineering at George Mason University.  He has worked with Kaplan Test Prep & Admissions as an admissions consultant, student advisor, and test prep instructor.  There are no conflicting relationships that are relevant or associated with the information in this article.