Association between regional quota program in medical schools and practical clinical competency based on General Medicine In-Training Examination score: a nationwide cross-sectional study of resident physicians in Japan

Abstract

Purpose

A regional quota program (RQP) was introduced in Japan to ameliorate the urban–rural imbalance of physicians. Despite concerns about the low learning abilities of RQP graduates, the relationship between the RQP and practical clinical competency after initiating clinical residency has not been evaluated.

Methods

We conducted a nationwide cross-sectional study to assess the association between the RQP and practical clinical competency based on General Medicine In-Training Examination (GM-ITE) scores. We compared the overall and category GM-ITE results between RQP graduates and other resident physicians. The relationship between the RQP and scores was examined using multilevel linear regression analysis.

Results

There were 4978 other resident physicians and 1119 RQP graduates out of 6097 participants from 593 training hospitals. Being younger; preferring internal, general, or emergency medicine; managing fewer inpatients; and having fewer ER shifts were all characteristics of RQP graduates. In multilevel multivariable linear regression analysis, there was no significant association between RQP graduates and total GM-ITE scores (coefficient: 0.26; 95% confidence interval: −0.09, 0.61; P = .15). The associations of RQP graduates with GM-ITE scores in each category and specialty were not clinically relevant. However, in the same multivariable model, the analysis did reveal that total GM-ITE scores demonstrated strong positive associations with younger age and GM preference, both of which were significantly common in RQP graduates.

Conclusion

Practical clinical competency evaluated based on the GM-ITE score showed no clinically relevant differences between RQP graduates and other resident physicians.

Key messages

What is already known on this topic

• Many countries offer unique admission processes to medical schools and special undergraduate programs to increase the supply of physicians in rural areas.

• Concerns have been raised about the motivation, learning capabilities, and academic performance of the program graduates.

What this study adds

• This nationwide cross-sectional study in Japan revealed clinical competency based on the scores from the General Medicine In-Training Examination showed no clinically relevant differences between graduates of regional quota programs and other resident physicians.

How this study might affect research, practice, or policy

• The study provides evidence to support the Japanese regional quota program from the perspective of clinical competency after initiating clinical practice.

postgraduate medical education, General Medicine In-Training Examination, Japan, graduates, regional quotas, resident physicians

Introduction

Geographic imbalances in the supply of physicians are a worldwide problem [1]. In Japan, while the government has supported increasing the total number of physicians [2], the insufficient number of physicians in rural areas due to urbanization remains a serious problem [3–6]. Therefore, in 2008, some medical schools and local and national governments initiated a regional quota program (RQP) called “chiiki-waku” (in Japanese) in admissions to medical schools to achieve health coverage in rural areas.

In Japan, most applicants to the RQP are required to live in the prefecture where their medical school is located. Participants typically undergo a unique admission process to the medical school, which focuses more on personal statements, interviews, and high school grades. Entrants receive additional undergraduate education about rural medicine, and more than half of the programs offer bundled scholarships and waive tuition for undergraduate education. Participants are required to work in the prefecture for a specific period (usually 9 years) after graduation [7–9]. Some programs limit entrants’ future specialties to those for which there is a lack of physicians to supply healthcare, such as pediatrics, obstetrics and gynecology, general medicine (GM), and emergency medicine [10]. This RQP has spread nationwide, and 18.2% of medical students were RQP entrants in 2020 based on the official report from the Japanese Ministry of Education, Culture, Sports, Science, and Technology (MEXT).

The system has successfully supplied physicians who participate in rural healthcare in Japan, ameliorating urban–rural imbalances [9, 11]. However, concerns have been raised about their learning capabilities and academic performance [8], as some have reported that their grades in the admission examinations to medical schools were typically lower than those of other successful applicants [10, 12]. In addition, restrictions on their working areas for several years after graduation may affect their motivation and learning during their clinical residency [13]. Nevertheless, the practical clinical competency of RQP graduates after graduation has not been evaluated.

Hence, we conducted a nationwide cross-sectional study using the results of the General Medicine In-Training Examination (GM-ITE), a nationwide examination for resident physicians in Japan [14]. We aimed to evaluate the association between the RQP and resident physicians’ practical clinical knowledge based on GM-ITE scores.

Materials and methods

Study design and setting

The GM-ITE is an examination we developed and initiated in 2011 using the same methodology as for the Internal Medicine In-Training Examination (IM-ITE) [15]. The GM-ITE became a nationwide examination with the participation of more than half of all resident physicians in Japan. GM-ITE scores are a validated measure [14] for assessing medical knowledge. In Japan, GM refers to specialties in general internal medicine (IM), family medicine, and hospital medicine. Therefore, the questions in the GM-ITE were not limited to IM. The GM-ITE 2020 exam consisted of 60 multiple-choice questions, each worth 1 point equally (score range: 0–60). Questions were divided into four categories [medical interview and professionalism (6 questions), symptomatology and clinical reasoning (15 questions), physical examination and clinical procedures (15 questions), and disease knowledge (24 questions)] and seven specialties [GM (6 questions), IM (24 questions), surgery (6 questions), obstetrics and gynecology (6 questions), pediatrics (6 questions), psychiatry (6 questions), and emergency medicine (6 questions)].

We conducted a multicenter cross-sectional study on the association between RQP graduates and GM-ITE scores for the year 2020; the GM-ITE was conducted from January to February 2021. Resident physicians of postgraduate year (PGY)-1 and PGY-2 in hospitals in which the GM-ITE was programmatically included in the residency program or those who wanted to participate in the GM-ITE took this examination. Examinees underwent the GM-ITE first and were then recommended to answer the survey with a questionnaire about their clinical training and lifestyle.

Study participants

We included participants in GM-ITE 2020. Participants who refused the use of their data for research, those with missing answers about the RQP, and those who answered the questionnaire inappropriately (e.g. double answer) were excluded.

Outcome measurement

The primary outcome of this study was the total GM-ITE score (range: 0–60). The secondary outcomes were the scores divided based on the question category (medical interview and professionalism, symptomatology and clinical reasoning, physical examination and clinical procedures, and disease knowledge) and the question specialty (GM, IM, surgery, obstetrics and gynecology, pediatrics, psychiatry, and emergency medicine).

Definition of the regional quota system and confounding variables

In the questionnaire after the GM-ITE 2020, participants were asked to answer whether they were required to engage in healthcare in specific geographic areas after their residency. We defined participants who answered “yes” to this question as RQP graduates; those who answered “no” were classified as other resident physicians. In addition, data regarding participant demographics and characteristics of the residency program were collected from the questionnaire for use as confounding variables.

Statistical analysis

First, we described and compared the participant demographics and characteristics of the residency programs between the RQP graduates and other resident physicians. Then, the GM-ITE scores in total and in each category were assessed between the two groups. In these univariate analyses, chi-square tests and t-tests were used to assess the differences between categorical and continuous variables, respectively. A linear mixed-effect model with random intercept was used to evaluate the association between RQP graduates and the GM-ITE score. We adopted a three-level model assuming resident physicians were nested within each hospital, which clustered within prefectures. The result was adjusted for covariates selected based on previous studies [16–22] and authors’ agreement, including gender, age categories, PGY, types of the residency program (university hospital or community hospital), preferred specialty, GM rotation, duration of IM rotation, number of emergency room (ER) night shifts, number of inpatients of whom they were in charge, self-study time, and average working hours.

For the data description, categorical variables were expressed as numbers (with percentages) and quantitative variables as mean ± standard deviation (SD). In multivariable analyses, we performed complete-case analyses. A P value of <0.05 was considered statistically significant for all analyses. All analyses were performed using STATA software (version 17.1, StataCorp, College Station, TX, USA).

Ethical statement

We obtained informed consent from all participants of the GM-ITE in an opt-out manner after an explanation of the research, data anonymization, and voluntary participation with documents. We included participants with consent in this study. The study was conducted in accordance with the principles of the Declaration of Helsinki. All methods followed the Ethical Guidelines for Medical and Health Research Involving Human Subjects. This study was approved by the Ethics Review Board of the Japan Organization of Advancing Medical Education (JAMEP) (approval number 21–11).

Results

Participant characteristics

A total of 7669 PGY-1 or PGY-2 resident physicians from 593 training hospitals underwent GM-ITE 2020. After excluding 1572 resident physicians, a total of 6097 participants were finally included in the study (Fig. 1). There were 1119 (18.4%) RQP graduates and 4978 (81.6%) other resident physicians. Significantly more RQP graduates had their residency program in university hospitals [160 (14.3%) vs. 499 (10.0%); P < .001]. There were significant differences in the categories of age, preferred specialties, the average number of inpatients of whom they were in charge, and the number of ER shifts between the groups. The RQP graduates were distributed among younger age categories; preferred internal, general, or emergency medicine; were in charge of fewer inpatients; and had a smaller number of ER shifts (Table 1).

Figure 1

Flowchart of participant inclusion and exclusion.

Table 1

Baseline characteristics of RQP graduates and other resident physicians.

		RQP graduates (n = 1119)	Other resident physicians (n = 4978)	P value
Gender	Male	742 (66.3%)	3412 (68.5%)	.15
	Female	377 (33.7%)	1566 (31.5%)
PGY	1	563 (50.3%)	2536 (50.9%)	.70
	2	556 (49.7%)	2442 (49.1%)
University hospital		160 (14.3%)	499 (10.0%)	<.001
Age	24	56 (5.0%)	169 (3.4%)	<.001
	25	216 (19.3%)	941 (18.9%)
	26	397 (35.5%)	1542 (31.0%)
	27	230 (20.6%)	1051 (21.1%)
	28	80 (7.1%)	457 (9.2%)
	29	37 (3.3%)	250 (5.0%)
	≥30	103 (9.2%)	564 (11.3%)
Preferred specialty	IM	430 (38.5%)	1749 (35.2%)	<.001
	Surgery	218 (19.5%)	1055 (21.2%)
	GM	46 (4.1%)	100 (2.0%)
	Emergency Medicine	46 (4.1%)	164 (3.3%)
	Others	378 (33.8%)	1907 (38.3%)
Working hours per week	<45	30 (2.7%)	119 (2.4%)	.13
	45 to <50	133 (11.9%)	582 (11.7%)
	50 to <55	192 (17.2%)	733 (14.8%)
	55 to <60	152 (13.6%)	622 (12.5%)
	60 to <65	173 (15.5%)	813 (16.4%)
	65 to <70	96 (8.6%)	503 (10.1%)
	70 to <80	137 (12.3%)	574 (11.6%)
	80 to <90	137 (12.3%)	613 (12.4%)
	90 to <100	32 (2.9%)	224 (4.5%)
	≥100	36 (3.2%)	174 (3.5%)
Average number of inpatients	0–4	299 (27.6%)	1128 (23.2%)	.018
	5–9	636 (58.8%)	3018 (62.1%)
	10–14	114 (10.5%)	573 (11.8%)
	>14	33 (3.0%)	141 (2.9%)
GM rotation		459 (41.1%)	2142 (43.1%)	.23
Duration of IM rotation (months)	0–5	210 (18.8%)	977 (19.7%)	.21
	5–10	689 (61.7%)	3143 (63.4%)
	11–15	184 (16.5%)	726 (14.6%)
	16–20	27 (2.4%)	99 (2.0%)
	>20	7 (0.6%)	16 (0.3%)
ER shifts per month	0	52 (4.7%)	169 (3.4%)	<.001
	1–2	182 (16.4%)	757 (15.3%)
	3–5	804 (72.2%)	3477 (70.3%)
	>5	75 (6.7%)	546 (11.0%)
Self-study time (min/day)	None	29 (2.6%)	187 (3.8%)	.18
	1–30	376 (33.6%)	1699 (34.1%)
	31–60	491 (43.9%)	2039 (41.0%)
	61–90	178 (15.9%)	834 (16.8%)
	>90	44 (3.9%)	217 (4.4%)

Unadjusted comparisons of General Medicine In-Training Examination scores between the groups

The total GM-ITE score was significantly higher in the RQP graduates (mean ± SD score: 29.4 ± 5.2 vs. 29.0 ± 5.4; P = .019) (Table 2), while the effect size was 0.078, showing only small differences. The distribution of the score is visualized in Fig. 2. In scores by question categories, RQP graduates had significantly higher scores in the physical examination and clinical procedures and disease knowledge (6.9 ± 2.0 vs. 6.7 ± 2.0; P = .002 and 11.0 ± 2.8 vs. 10.7 ± 2.9; P = .002, respectively). As for question specialty, the score in emergency medicine was significantly higher in the RQP graduates (3.4 ± 1.2 vs. 3.3 ± 1.3; P = .008) (Table 2). The effect sizes of these differences were very small.

Figure 2

Distribution of total GM-ITE scores of RQP graduates and other resident physicians.

Table 2

Comparison of GM-ITE results between RQP graduates and other resident physicians.^a

	RQP graduates (n = 1119)	Other resident physicians (n = 4978)	P value
Total score (range: 0–60)	29.4 ± 5.2	29.0 ± 5.4	.019
Scores by question category
Medical interview and professionalism (range: 0–6)	2.9 ± 1.2	2.9 ± 1.1	.41
Symptomatology and clinical reasoning (range: 0–15)	8.6 ± 2.1	8.6 ± 2.1	.56
Physical examination and clinical procedures (range: 0–15)	6.9 ± 2.0	6.7 ± 2.0	.002
Disease knowledge (range: 0–24)	11.0 ± 2.8	10.7 ± 2.9	.002
Scores by specialty
GM (range: 0–6)	2.9 ± 1.2	2.9 ± 1.1	.41
IM (range: 0–24)	11.3 ± 2.8	11.1 ± 2.9	.057
Surgery (range: 0–6)	3.0 ± 1.2	2.9 ± 1.2	.053
Obstetrics/gynecology (range: 0–6)	2.7 ± 1.2	2.6 ± 1.2	.23
Pediatrics (range: 0–6)	4.0 ± 1.1	4.0 ± 1.0	.58
Psychiatry (range: 0–6)	2.2 ± 1.1	2.1 ± 1.1	.19
Emergency medicine (range: 0–6)	3.4 ± 1.2	3.3 ± 1.3	.008

^aAll data presented as mean ± SD.

Association of regional quota program and General Medicine In-Training Examination score by multilevel linear regression analysis

In the multilevel linear regression analysis, there was no significant association between RQP graduates and GM-ITE score [β coefficient: 0.20; 95% confidence interval (CI): −0.16, 0.56; P = .27] (Table 3). In the question categories, RQP graduates were associated with higher GM-ITE scores on physical examination and clinical procedures (β coefficient: 0.17; 95% CI: 0.04, 0.31; P = .009). In addition, there was a significant positive association between RQP graduates and the GM-ITE score in emergency medicine (β coefficient: 0.09; 95% CI: 0.00, 0.18; P = .040). The effect sizes of these differences were also small. Neither scores in any categories nor scores in any specialties were significantly lower for the RQP graduates (Table 4).

Table 3

Factors associated with total GM-ITE score in multilevel linear regression analysis.

	Coefficient	95% CI		P value
RQP graduates (vs. other resident physicians)	0.20	−0.16	0.56	.27
Female (vs. male)	−0.02	−0.30	0.26	.89
Age (for 1-year increase)	−0.54	−0.63	−0.45	<.001
Second-year resident physicians (vs. first-year)	1.17	0.87	1.47	<.001
University hospital (vs. community hospital)	−1.06	−1.87	−0.26	.010
Preferred specialty (vs. IM)
Surgery	−1.10	−1.47	−0.75	<.001
GM	1.49	0.63	2.35	.001
Emergency medicine	0.16	−0.56	0.89	.66
Others	−0.66	−0.97	−0.35	<.001
GM rotation (vs. no GM rotation)	0.18	−0.12	0.49	.24
Duration of IM rotation (for 5-month increase)	0.02	−0.20	0.24	.85
Number of ER night shifts (for 1-unit increase in the questionnaire)	0.24	−0.02	0.50	.068
Number of inpatients in charge (for 5 patients increase)	0.31	0.09	0.53	.006
Self-study time (for 30 minutes per day increase)	0.55	0.40	0.70	<.001
Average working hours (for 5-hour increase/week)	0.05	−0.01	0.11	.11

Table 4

Association between RQP graduates and scores by question category (A) or scores by specialty (B).

(A)	Medical interview and professionalism	Symptomatology and clinical reasoning	Physical examination and clinical procedures	Disease knowledge
Coefficient	−0.01	−0.12	0.17	0.16
95% CI	(−0.09, 0.07)	(−0.25, 0.02)	(0.04, 0.31)	(0.01, 0.40)
P value	.78	.10	.009	.11
(B)	GM	IM	Surgery	Obstetrics/gynecology	Pediatrics	Psychiatry	Emergency medicine
Coefficient	−0.02	0.13	0.05	0.03	−0.03	0.01	0.09
95% CI	(−0.09, 0.06)	(−0.06, 0.32)	(−0.03, 0.14)	(−0.05, 0.12)	(−0.10, 0.04)	(−0.09, 0.06)	(0.00, 0.18)
P value	.66	.19	.19	.47	.40	.76	.04

With regards to other participant characteristics, the total GM-ITE score had a significant positive association with PGY-2 (vs. PGY-1), a preferred specialty of GM (vs. IM), and an increase in the categories of number of inpatients and self-study time. The score had negative associations with an increase in the age category, university hospital program, and preferred specialty of surgery or others (vs. IM) (Table 3).

Discussion

This nationwide cross-sectional study revealed no significant association between RQP graduates and the total GM-ITE score. To the best of our knowledge, this is the first study to evaluate the practical clinical competency of Japanese RQP graduates after starting their clinical residency.

In this study, the GM-ITE score was not significantly associated with RQP graduates in the multivariable model. Although the findings were not statistically significant, the findings of this study might be sufficient to insist that the scores of RQP graduates were comparable to those of other resident physicians, with enough statistical power rendered by a large number of participants. Whereas some medical directors are concerned about the low learning ability of RQP entrants [10] with lower scores in medical school admission examinations [12], others gave favorable comments in the national survey [10]. Actually, RQP entrants have a slightly higher pass rate in the Japanese national physician license examination [8]. RQP entrants have potential advantages in learning extensive medical knowledge. Financial support may give them additional time to concentrate on learning medicine in medical schools. Several medical schools provide unique education curriculums for RQP entrants, and they should be aware of the importance of possessing unbiased and broad medical knowledge for working in rural areas. This awareness would motivate them to learn broad medical topics enthusiastically. Our study provided important evidence to support the unique admission processes and undergraduate education offered by the Japanese RQP from the perspective of clinical competency after initiating clinical practice.

Several factors were found to be significantly associated with the GM-ITE scores. The positive association of GM-ITE score with PGY-2, the number of inpatients, and self-study time suggested that both on-the-job experience and off-the-job learning were important for improving clinical competency. Inversely, we believe these results suggested that the GM-ITE successfully evaluated both clinical experience and medical knowledge. Furthermore, younger residents had significantly higher GM-ITE scores, which may be attributed to their high potential in terms of learning ability. In Japan, most students directly enter medical schools after graduation from high school. Therefore, younger resident physicians were typically those who have successfully passed the admission process to medical school while taking fewer entrance examinations. Additionally, resident physicians aspiring to pursue GM may have balanced learning across a broad range of medical fields, which may have contributed to their higher scores. Being younger and GM preference were more commonly found in RQP graduates. This result implied that the RQP successfully supplied motivated and skilled young physicians pursuing GM to rural healthcare. In Japan, in contrast to the USA or several European countries, medical remuneration is usually the same between generalists and specialists. Therefore, an increasing number of motivated resident physicians became interested in GM, and the number of medical schools which have established a GM department has been increasing in Japan [23].

The average scores for both groups in the GM-ITE were relatively low, ~29 out of 60. This was because the primary aim of the GM-ITE was to assess the clinical competency of resident physicians, rather than establish a specific passing or failing threshold for the participants. Therefore, we curated this examination with a focus on discriminatory capability rather than average scores. Our previous study showed good discriminatory indices of GM-ITE questions with mean values exceeding 0.2 [14]. However, it would be valuable to examine whether this study’s results would be replicated in a setting with higher average scores.

We also demonstrated the actual circumstances of clinical training in RQP graduates, which had not been thoroughly investigated. RQP graduates had significantly lower numbers of ER shifts and inpatient workloads. Their frequent residency in university hospital programs can partially explain these characteristics. In Japan, since university hospitals are responsible for advanced medical care and research with sufficient senior physicians, resident physicians tend to have lower numbers of ER shifts and inpatients. Our data showed that the numbers of inpatients and ER shifts were significantly lower in university hospitals (see online supplementary material: Supplemental Table 1). Generally, physicians in Japanese rural hospitals are required to handle a wider range of medical problems in general IM and emergency medicine, regardless of their specialties, due to a shortage of human resources. Therefore, RQP graduates are required to be skilled in GM and emergency medicine, and clinical training with fewer ER shifts or inpatients is unfavorable. The postgraduation training program for the RQP graduates should be uniquely designed to fit their needs and the purpose of the program.

Internationally, many countries offer programs to increase the supply of physicians in rural areas [11, 24], such as the Rural Area Project in Thailand [25], Parallel Rural Community Curriculum in Australia [26], dedicated rural entry pathways and rural immersion programs in New Zealand [27], the establishment of The Northern Ontario School of Medicine in Canada [28] and Zamboanga School of Medicine in the Philippines [24], and the Physician Shortage Area Program by Jefferson Medical College in the USA [29]. Furthermore, some countries, including South Korea, are planning to introduce a system similar to the Japanese RQP with financial incentives [11]. These programs differ from Japanese RQPs, but they share similarities in offering a unique admission process to students who live in rural areas and special undergraduate education in rural medicine. As the academic performance of students from rural areas tends to be lower upon admission to medical school [30], it is possible that these countries share the same concerns regarding the clinical competency of their RQP graduates. Our study’s findings, indicating that RQP graduates had the same clinical competency as other resident physicians, could potentially alleviate these concerns in other countries.

Our study has several strengths. First, the GM-ITE was a nationwide examination in which more than half of the total resident physicians participated. The proportion of RQP graduates in our study (18.4%) was similar to the officially reported proportion of RQP entrants (18.2%), suggesting that our cohort successfully represented resident physicians in Japan. Second, although no significant difference was observed in the primary outcome, the large number of participants allowed us to conclude that the differences in the score were not clinically relevant, with a lower 95% CI bound of −0.09, even if there was a difference in fact.

However, this study also has a few limitations. First, we did not collect detailed data on the RQP in our questionnaire. The RQP varies particularly in the admission process (e.g. restricted living area before entrance examination and types of examination), bundled financial incentives, and restrictions in specialties. Therefore, it is imperative to examine the association of clinical competency with these detailed patterns of the RQP in the future. Second, we only evaluated the cross-sectional association and did not assess changes in clinical medical competency from the initiation of residency programs or the time of entering medical schools. This limitation made it challenging to accurately evaluate how postgraduate education itself affected the score of RQP graduates. Future studies using longitudinal data, such as comparing the national medical license examination score and GM-ITE score during residency, should be conducted in the future. Third, the questionnaire was self-reported, which brings a potential risk of misclassification. Moreover, we did not adjust for nonquantifiable confounders, such as program-specific educational interventions for RQP entrants in medical school. Finally, we were unable to evaluate factors such as motivation toward learning among RQP graduates in the current questionnaire. To address this, research methods that incorporate qualitative evaluations, such as mixed methods and semiquantitative interviews, should be used in the future.

Conclusions

This study revealed that practical clinical competency based on GM-ITE scores indicated no clinically relevant differences between RQP graduates and other resident physicians. This result may support the RQP from the aspect of clinical competency after initiating clinical practice.

Acknowledgements

The authors express gratitude to JAMEP members and the members of the question development committee as well as the peer-review committee of the GM-ITE.

Conflict of interest statement: Y.N. received an honorarium from JAMEP as GM-ITE project manager. Y.T. is the JAMEP director, and received an honorarium from JAMEP as a speaker at a JAMEP lecture. H.K. and K.S. also received honoraria from JAMEP as speakers at JAMEP lectures. T.S., Y.Y., K.S., and S.F. received honoraria from JAMEP as GM-ITE exam preparers.

Funding

This study was supported by the Health, Labor, and Welfare Policy Grants of Research on Region Medical (21IA2004) from the Ministry of Health, Labor, and Welfare (MHLW). The MHLW did not participate in any part of the process of this study including the designing, data analysis, data interpretation, review, and approval of the manuscript.

Data availability

The data are not available as the study participants did not agree for their data to be shared publicly.

Author contributions

Study concept and design; acquisition, analysis, or interpretation of data: S.F., K.S., Y.N. Manuscript drafting and statistical analysis: S.F. Critical revision of the manuscript for important intellectual content: K.S., Y.N., T.S., Y.Y., H.K., Y.T. Administrative, technical, or material support: Y.N., Y.T. Supervision: Y.N., H.K., Y.T.