What Influences the Decision to Interview a Candidate for Medical School?

Introduction: Holistic review, in which medical schools seek to balance applicant attributes and experiences alongside traditional academic metrics in making admissions decisions, has been in place for over a decade. Medical school applicants and the admissions’ community are still trying to understand the impact of holistic review on the composition of those medical schools choose to interview and accept. Materials and Method: The study cohort included all candidates who applied to Uniformed Services University of the Health Sciences (USU) in 2014, 2015, and 2016 ( N = 8,920). We conducted logistic regression analysis to examine the associations between the sociodemographic, academic, and military service variables of applicants applying to the School of Medicine and offers for interview. Results: Medical College Admission Test scores and undergraduate grade point averages were important in predicting who would receive an interview. Having military experience, being a woman, and being self-reported African American race also predicted a higher likelihood of receiving an interview invitation. For example, controlling for all other variables in the model, if an applicant had previous military experience, the odds of being invited for interview was about 4 times that of an applicant who had no previous military experience. Leave this for the text and discussion. The resulting pool of interviewed and accepted students more increasingly represented the Military Health Service population served. Conclusions: The use of holistic review generated a class with a composition different from that which would be predicted by Medical College Admission Test and grade point average alone. Further, holistic review produced an interview pool and class more representative of the wider Military Health Service beneficiary population. In the case of USU, holistic review allowed the school to better meet its mission to create a representative class able to “care for those in harm’s way.”

competitiveness of the admissions process, over 95% of those who matriculate will eventually graduate and apply for internship and residency training. 2 With such a high progression rate, it is admission to medical school that represents the biggest obstacle to becoming a physician.
With a surfeit of strong, academically capable applicants, how do admissions committees meet the challenge of selecting good candidates? Traditionally, schools placed a heavy emphasis on academic metrics, like undergraduate grade point average (GPA) and standardized test scores. 3 Although academic metrics are still emphasized, the Association of American Medical Colleges (AAMC) has more recently advocated admissions decision-making under the rubric of "holistic review." This type of review is described as "a flexible, highly individualized process by which balanced consideration is given to the multiple ways in which applicants may prepare for and demonstrate suitability as medical students and future physicians." 3 In more practical terms, holistic review asks medical schools to weigh applicant attributes and experiences alongside academic metrics and to link admission practices to institutional mission at all stages of review. The aspiration is that this type of review will build the healthcare workforce needed to improve outcomes in an institution's served populations. 4,5 Although the AAMC "Integrating Holistic Review Practices into Medical School Admission Processes" was released in 2010, there have been few published studies looking at how adopting holistic review has impacted the pool of medical school applicants invited for interview. 6,7 The F. Edward Hébert School of Medicine at the Uniformed Services University of the Health Sciences (USU) is the Nation's federal school of medicine and is committed to excellence in military medicine and public health during periods of peace and war. Each of its medical students is a commissioned military or Public Health Service officer, and every graduate goes on to serve the men and women who defend the country. Although the USU admissions committee has long practiced some aspects of holistic review (for example, considering the context from which an applicant applies and the obstacles they may have needed to overcome before application), it received formal training in holistic review from the AAMC in 2014 and incorporated this training into admissions policy. Holistic review applied to USU encourages the selection of candidates who indicate a propensity to serve and who represent the socioeconomic, geographic, and racial diversity of the U.S. Armed Forces, veterans, and their families. The committee also received training in scoring applicants using an internally derived rubric that reinforced holistic review practices, in using the clues embedded in an application to appreciate the context from which a candidate applied, and in mitigating the effects of bias on an ongoing basis.
The aim of this study was to explore which candidate variables (sociodemographic, academic, military service) had significant predictive power in determining who would be invited to interview at USU school of Medicine or not.

Study Context and Samples
We collected data of all USU medical school applicants of 2014, 2015, and 2016 (N = 8,920). From this cohort, we excluded applicants whose Medical College Admission Test (MCAT) total score was below 23 (N = 2,019) because the admissions committee used 23 as a cut score; that is, applicants with recent MCAT total scores lower than 23 were not considered for on-site interview. Thus, the valid sample size of the study was 6,901.
Applicants who met the MCAT threshold were advanced to the admissions committee to begin the holistic review process. In this process, a committee member reads through an entire applicant's packet, to include academic metrics (MCAT attempts, scores, and subsection scores; course work completed, grades assigned, GPA in natural science courses and in all courses), personal attributes (socioeconomic status, gender, race, ethnicity), and experiences (clinical exposure, community service, research, military experi-ence, leadership). Based on this read, the committee member makes a recommendation to interview or not interview an applicant. If an applicant is not recommended for an interview, then a second committee member reads the application and either concurs with the decision or reverses it. If two members agree that an applicant should not be offered an interview, then that applicant is removed from the pool. If an applicant is recommended for an interview, then that applicant's file is placed in a queue from which interview offers are made.
Upon completion of interview, each candidate is reviewed again by the committee and recommended for acceptance, placement on an alternate list, or rejection. Accepted applicants who chose to attend the School of Medicine were matriculated into the first-year class.

Study Variables and Statistical Analyses
In this study, the outcome variable is whether an applicant was invited for an on-site interview or not (coded as 1 for "invited," 0 for "not invited"). The explanatory variables (i.e., predictors) were assigned values on either a ratio scale or nominal scale. The explanatory variables on a ratio scale included applicant's age at application; number of dependents at application; undergraduate biology, chemistry, physics, and mathematics cumulative GPAs; cumulative GPA of all remaining undergraduate courses (undergraduate All Others GPA); and undergraduate overall GPA; number of MCAT attempts; most recent MCAT sectional scores (verbal score, physical sciences score, and biological sciences score) and total score. The MCAT is a standardized examination for prospective medical students in the United States. In the current study, we used scores from the MCAT test format in use from 1991 to 2014.
The explanatory variables coded on a nominal scale included applicants' gender, self-reported disadvantaged status and childhood underserved status at application, previous military experience, and self-identified race (if an applicant indicated more than one race, the first choice was recorded). All of these variables were self-reported and available through the American Medical College Application Service.
We conducted descriptive statistics analysis of the above measures and logistic regression analysis. The purpose of logistic regression analysis was to examine the predictive power of the explanatory variables on the outcome variable. We considered a test statistic to be statistically significant if the P value substitute was for is smaller than 0.05. To avoid the problem of collinearity, we chose the MCAT sectional scores to enter in the regression model and omitted the MCAT total score since the total score is a linear combination of the sectional scores. Similarly, we did not enter the undergraduate overall GPA. Instead, we used the variables of undergraduate biology, chemistry, physics, and mathematics cumulative GPA and undergraduate AO cumulative GPA. All the statistical analyses were conducted in IBM SPSS 25.0. This study was approved by USU Institutional Review Board.

RESULTS
The total sample size was 6,901 (2,432 in 2014, 2,609 in 2015, and 1,858 in 2016; two applicants had incomplete information). Descriptive statistics are shown in Table I. Using the combination of explanatory variables in Table II to predict who received an invitation to interview, the Nagelkerke R 2 was 0.30 for the holistic logistic regression model. Nagelkerke R 2 is a pseudo R 2 for logistic regression analogous to R 2 in linear regression. A value of 0.30 means that about 30% of the variance of the outcome (i.e., invited for interview or not) can be explained by the cluster of explanatory variables in Table II. The variables of gender, race (compared with White), previous military experience, MCAT component scores and number of MCAT attempts, as well as undergraduate GPAs were all statistically significant predictors. For example, controlling for all other variables in the model, if an applicant had previous military experience, the odds of being invited for interview was about 4 times (1/0.25 = 4) that of an applicant who had no previous military experience. In addition, for each incremental (one point) increase in the recent MCAT physical sciences score, the odds of being invited for an interview was 1.35 times higher. Using the race category of White as a reference and controlling for all other variables in the model, the odds of being invited for an interview for Asians was 0.67 times the odds of being invited for White applicants; odds for African Americans were 2.49 times higher.

DISCUSSION
This article fills a gap in the literature with respect to how interview choices are made and demonstrates that USU decisions to interview are influenced by sociodemographic factors, academic factors, and history of military service. With respect to academic factors, MCAT scores and GPAs are important to USU in selecting interviewees. National data demonstrate that average MCAT scores among applicants and The category of "White" was used as the reference category for the variable of "Race." *Gender, self-reported disadvantaged status, self-reported childhood underserved status, and previous military experience were treated as categorical variables in the logistic regression model. The reference categories were coded with value 1. MCAT, Medical College Admission Test Undergraduate BCPM (biology, chemistry, physics, and mathematics) cumulative GPA is cumulative GPA of the biology, chemistry, physics, and mathematics courses. Undergraduate AO (all others) cumulative GPA is cumulative GPA of all remaining undergraduate courses except biology, chemistry, physics, and mathematics courses. matriculants in the 2019 to 2020 cycle were 506.1 and 511.5, respectively. GPAs were 3.58 and 3.73, respectively. 8 USU also highly values prior military service in selected interviewees; this is consistent with the school's vision "to be our nation's preeminent institution for educating physicians…to serve the United States as expert clinicians, innovators and leaders in times of peace and war." One may ask why being female would be associated with a higher likelihood of being offered an interview, given the proportion of women in active duty service being small (16.2% of those who serve). 9 women (including service members, retirees, and dependents) make up 49% of the total Military Health Service beneficiary population. 10 To achieve a representative class, proportionately more women were invited from a limited applicant pool (31% female). This same logic likely holds for the higher likelihood of African American candidates being offered interviews. Those identifying as Black or African American make up 13.4% of the general population and 17.3% of those on Active Duty, but only 3.8% of the applicant pool. 9, 11 Asian race was associated with a lower likelihood of being invited to interview. Those who identified with Asian race comprised 25% of the applicant pool; in comparison, 5.9% of the general population and 4.5% of the active duty military identify as Asian. 9 At the end of the admissions process, despite the lower odds ratio of being offered an interview, the representation of Asian members entering the class over the study period was 17%. This number is over 3 times the representation on the Active Duty force although it is a little lower than number of Asians who matriculated into American medical schools in the same period (20%). 12 For comparison, 67% of students entering in 2016 and 69% of the active duty military were White (on par representation).
In addition to race, gender, and familiarity with the military, USU tries to achieve optimum representation through accessions from different socioeconomic groups. For the years studied, approximately one in five matriculants were the first in their family to complete college and an increasing number of students came from the enlisted military ranks (14% in 2016). This socioeconomic representation helps better mirror the military population USU serves. The large prior service component of the class (approximately two-fifths over the years of the study) helps train students in the dual profession of being a physician and a military officer.
Limitations of the current study include its conduct at a single institution over a limited period (i.e., applicants of 2014, 2015, and 2016). This limitation had one advantage: it allowed us to interpret how holistic review practices were used by a medical school to build its entering class. Another limitation is the lack of granularity in the explanatory variables. For instance, in the socioeconomic variables, what degrees of disadvantage are grouped together into self-proclaimed disadvantaged status and childhood underserved status? More rigor in defining and categorizing these variables on the part of the American Medical College Application Service might allow researchers to decode subtle gradations that might have a more powerful explanatory effect. A similar critique may be made of the race categories. Although students have the opportunity to claim subcategorizations (for instance, students who report being Asian can also report being Chinese, Korean, Vietnamese, or any among a number of options), not all choose to do so, limiting the ability to better parse the race category to look for more nuanced associations.
Another limitation of the current study is that the explanatory models did not include other factors that greatly influence admissions under a holistic review rubric like clinical exposure and research experience. The decision to omit these variables was based on prior research at USU demonstrating that self-reported data like clinical and research experience did not have discriminatory power to predict who would or would not be admitted to the class. 13 Further, the scope of this study was limited to the offer of on-site interview. It did not investigate who received an offer of acceptance.
There are other concerns that would be reasonable to raise but that fall outside the purview of this study; perhaps, they are seeds for future research. For instance, if holistic review is changing who enters medical school, does it change the outcomes of medical education beyond class composition? Do students who enter under holistic review graduate at the same rate as those in previously constructed classes? Do students selected under holistic review tend to match to the same specialties as those selected before the advent of holistic review? Does holistic review create new groups of "winners" (those invited to interview and accepted to medical school) and "losers" (those either not invited or not accepted)? And, since USU is a military medical school, how does what USU do compare to other medical schools? How do the classes of students that USU creates differ from the students that enter the military through the Health Profession Scholarship Program?
This study fills a gap in the literature and demonstrates how USU decisions to interview are influenced by both academic and mission-oriented factors using a holistic review framework. This study demonstrates how one school attempts to create a class that meets the institutional need to reflect and serve the Military Health Service population that relies upon uniformed physicians. Further studies that bring together demographic, academic, and experiential variables to give a better sense of how holistic review practices play out across multiple institutions would further illuminate the medical school accessions pipeline.