Most medical schools in the world have recently begun to emphasize competency-based curricula to train highly skilled primary care physicians. The clinical practice examination (CPX) is being used as a means of assessing and training clinical competency. CPX brings several benefits in improving competent primary care physicians. Teachers are able to assess clinical skills of reasoning and communication while students demonstrate their ability to apply both practical knowledge and skills in an authentic clinical setting.1)
In Korea, both the CPX using standardized patients, and the objective structured clinical examination (OSCE) was implemented in the Korean Medical Licensing Examination (KMLE), henceforth assessing the clinical competence of applicants as primary care physicians. It is ideal for students in clinical clerkship to enhance their clinical competence by having a good number of opportunities to experience direct interviews with actual patients. Practically, however, medical schools in Korea would find it difficult to provide an adequate number of opportunities of balanced clinical training with real patients on major clinical presentations because the number of actual patients that students may encounter is limited and the patients are not standardized. This lack of opportunity is then further exacerbated since protecting patients' rights is an essential part of health services. Difficulty of acquiring patient permission about a student practice further limits the range of clinical trainings that can be implemented. Current undergraduate medical education tends to be theoretical and textbook-oriented while classes would hardly provide either simulated or actual patients. Regarding clinical clerkship involving inpatients or outpatients, the diagnosis is usually given by other experts prior to a student-patient encounter, thus restricting students' clinical experience in a disease-oriented manner. This renders the clinical experience partly inadequate to provide authentic training in clinical reasoning skills and competency.
Many medical schools have made attempts using diverse methods of clerkship education to enhance the clinical competency of students and to help them perform well on CPX and OSCE. These attempts to facilitate active learning include running CPX multiple times so as to hone clinical competence while producing feedback, educating students with standardized patients, preparing for CPX by designing a CPX scenario firsthand, and performing role play in small groups. These techniques also facilitate students' learning of medical and interpersonal skills through the reinforcement of bedside teaching.2345) Despite these many methods, there has yet to be a verified assessment on the effectiveness of the aforementioned preparations for CPX.
Medical schools in Korea, in an effort to ensure an increase in their KMLE pass rates, have reinforced experiential learning within their clinical clerkship education or programs that are used to prepare students for clinical skills test. These are taken prior to graduation with case studies in the form of CPX or role play. There is, however, a lack of studies regarding the effect of CPX preparation programs on the clinical competency of students. The purpose of this study was to investigate whether CPX preparation programs during the fourth-year clerkship improve the clinical competency of students in terms of case specificity.
1. Subjects of Study and Education Method
One hundred and thirteen students in their fourth-year of a medical school located in Seoul participated in this study. Seventy-one (62.8%) of the subjects were male students, and 57 (50.4%) of the subjects were medical college students. The average age was 26.2±2.5 years. During the fourth-year clinical clerkship at the Hanyang University Medical College, a total of 28 students (24.8%), comprising three of 14 clerkship rotation groups, could not participate in the community medicine clerkships, which occurs over the course of a week between June and July in 2015 at a community health center, due to a MERS (Middle East Respiratory Syndrome) outbreak. These three rotation groups then completed a CPX preparation program, which was developed under the supervision of the department of medical education, as an alternative independent curriculum. The details of this preparation program are shown in Table 1. The other 11 rotation groups had completed their respective community medicine training were not subject to any organized CPX-related training program. Effects of the three-day preparation program could thus be compared through a common subsequent comprehensive CPX between the intervention and control groups.
The CPX preparation program was operated by a professor, a clinical instructor, and a medical resident in the department of family medicine. Guidelines for CPX case development was taught as the core lecture on the first day, and each group was then divided into three subgroups consisting of three students each to develop cases on three CPX clinical problems (cough, vaginal discharge, and hand tremor). For case selection, students referred to the ‘2015 basic clinical practice guidelines,’ published by Korea Association of Medical Colleges, using their experience with the actual patients they came across during their clinical clerkship. The three groups were given the same CPX clinical problems, but were asked to develop modules based on different diseases. As for the development of the CPX scenario, a scoring rubric and scoring checklists, which are evaluation tools for reviewing the quality of a clinical performance, the medical resident operated as a facilitator and made amendments as deemed appropriate. Each subgroup was instructed to film a video clip of a typical medical encounter with a standardized patient, in which the checklist items were sufficiently reflected, so that it may be used in the training of simulated patients for role play. All members of each group were gathered to watch the video clips and complete the modules through cross-checking and feedback.
On the second day, each subgroup had a person prepare a schema on cough, vaginal discharge, and hand tremor. This individual summarized the details of learning with a focus on differential diagnosis and presented the summary to colleagues. Afterwards, members of each subgroup participated in role play by changing roles between physician, evaluator, and patient who exhibited cough, vaginal discharge, or hand tremor. The student acting as the physician encountered the student in the patient role for about 10 minutes as if to carry out CPX. Once the role play was completed, the students who were the patient and the evaluator provided 5-minutes of feedback about clinical performance according to checklists. The facilitator provided feedback based on the checklist on strengths and weaknesses of each subgroup after watching all role play practices through a one-way mirror. Then, students who experienced different cases were reorganized into different subgroups to experience different role play.
On the third day, an intensive training on physical examination was held, in which each student watched a standard video clip on basic physical examination (heart, lung, abdomen, neck and oral cavity, neurological examination) prior to having a role play session. Afterwards, the facilitators provided corrections through feedback.
2. Items of Investigation
All students took a CPX on 13 stations in February 2015. CPX on 13 stations was conducted again in July 2015 after the fourth-year clerkship. Both examinations involved a 10-minute one-on-one interview with standardized patient and evaluation of standardized patient on clinical competency of students based on a checklist that consists of six sections (patient satisfaction, history taking, physical exam, clinical courtesy, patient education, and patient-physician interaction). This study sought to analyze the difference between the score marked by the students who underwent these programs and the score marked by the student who did not undergo these programs. Two of the taught CPX clinical problems (vaginal discharge, cough) were comprised some of the problems in the CPX taken in July.
3. Data Analysis
The scores were determined by standardized patients who acted as raters. A chi square test and Student t-test were used to analyze basic characteristics of students. Student t-test and paired t-test were used to compare CPX results between February and July. Statistical significance was assumed when P<0.05, and the analysis of all data was done using IBM SPSS ver. 21.0 (IBM Corp., Armonk, NY, USA).
There was no significant difference in the distribution of age, sex, and school type between the intervention and control group (Table 2). There was no significant difference between two groups in total score and sectional scores (patient satisfaction, history taking, physical exam, clinical courtesy, patient education, and patient-physician interaction) on the CPX conducted in February 2015 (Table 3).
All scores, except for physical exam, increased in both the intervention group and control group on the CPX conducted in July. In particular, the average total score of the CPX intervention group was 69.45±4.34 points, which was significantly higher than 67.53±4.43 points of the control group (P<0.05) (Figure 1). History taking was the only section in which the intervention group exhibited a markedly higher sectional score than did the control group, with a score of 70.0±6.0 points compared to 66.0±6.6 points (P<0.05) (Table 3). There was no notable difference in sections, such as physical examination, education, and patient-physician interaction (Table 3). For each station score, the score on vaginal discharge on the educated case was 73.05±6.31 points for the intervention group, which was significantly higher than 68.87±9.25 points for the control group (P=0.03) (Table 4). On the other hand, there was no difference in scores on cough, which had similar clinical problems as the educated case, but with a different context, between the groups (72.41±8.98 versus 69.14±8.91, P=0.10). Scores of the intervention group were higher than in the control group for clinical problems, such as foamy urine, palpitation, sexual violence, knee pain, and vaginal discharge. A significant score difference was only observed for the history taking section. There was no meaningful difference in CPX scores in February among three intervention groups. There also was no notable difference in CPX scores among groups in July. No significant difference in scores was found across sections and cases.
This study investigated the effects of an organized CPX preparation education, which comprises module development, role play, and physical examination, on the outcome of a CPX involving standardized patients. An organized CPX preparation education improved CPX scores in the short term. Previous studies have struggled to provide clear evidence on the effectiveness of CPX preparation education, but this study proved that there is an improvement in CPX scores, while the effect size remains insignificant.
Role play is an education method in which students experience emotional and intellectual responses of another person to explore the motivation for and result of actions, thereby helping them form a desirable behavior or value.6) Full-time fourth-year students at medical schools prepared for CPX based on such role play. Medical education using standardized patients ensures high receptiveness, not only for educators, but also medical students, while culminating in higher effectiveness compared to that using role play.7) However, considerable amount of time that the standardized patients require in their training, coupled with a high cost, means that the clinical skills test within the KMLE is prepared mainly with an attempt to enhance clinical competency using role play. This study was conducted to examine the effect of role play practice on CPX ability of students and to determine which sections of CPX were improved by role play. The results showed that role play practice of students before CPX effectively improved score, specifically for some sections of history taking. Scores on other sections, such as patient-physician interaction and physical examination, were not significantly different compared to the control group. CPX scores increased with repeated usage of cases in a previous study, which improvement may be explained by an increase in test awareness of examinees or a maturation effect during an additional period of educational experience.8) In this study, students' sectional CPX scores improved in all areas within each group over 5 months of clinical clerkship. However, only competency of history taking improved more among the intervention group compared with the control group. We may interpret it as the educational module being the focus on history taking with case specificity. Other educational interventions, such as a general hands-on skill course for comprehensive physical exam skills, were ineffective enough to affect the area score.
There was no significant correlation between school achievement, history taking, and physical exam scores of students;9) history taking and physical exam, sections that require frequent contact with patients, were regarded by students to be the most difficult on the first CPX held in Korea.10) This means that it is difficult to significantly increase scores on history taking and physical exam through ordinary curriculum. In this study, activities of students to prepare checklist for role play, feedback weak points during role play using the checklist, and repeatedly learn about the checklists of history taking section are believed to have directly and indirectly increased scores on history taking section.
Physical exam was practiced by students after watching a video clip about standard physical exam based on demonstration by a medical resident of family medicine. Scores on physical exam section were not increased in this study, possibly for several reasons. First, there is a lack of feedback from colleagues. Studies have revealed a lack of dependability from the result of the colleague-evaluations that had been taken domestically, which resulted in the following: the lack of differentiation between the students as they gave the same scores because of mental pressure; evaluating colleagues based on their general impressions rather than objective criteria, etc.1112) It is plausible that students found it difficult to study all the necessary physical examinations correctly in those role play situations with their colleagues from the same departments. Female students in particular may be inevitably forced and restricted to playing their roles in a passive manner when acting as patients during a physical exam, which could work as a hindrance in effective experiential training. Moreover, the feedback from patientdoctor relations or physical exam may have lacked enough accuracy as it is more vulnerable to inaccurate observations from a less skilled observer; history taking shows the contrary, as its evaluations are rather simple and the stipulated checklist is written precisely. It may also be interpreted as the actual effectiveness of education was limited because the role play in the physical exam was not operated in the context of the cases, which includes a general physical exam. Second, there is a lack of self-evaluation during physical exam. Students lack experience in receiving appropriate feedback and performing selfevaluation on their clinical competency.13) Failure to recognize weaknesses in the physical exam section is probably one of the reasons why scores on physical exam were not improved.
In a study by Han et al.,14) physician-patient interaction (PPI) score had high correlation with history taking. Smith15) defined history taking as “a communication in which a patient can speak freely and a physician plays the role of an attentive listener to understand and appropriately respond to true wants and needs of the patient on the same level.” This means that PPI can be improved by naturally listening to patients during history taking. While the increase in PPI scores through role play was only marginal in this study despite a significant increase in history taking scores, is possibly that students were unable to fully concentrate on interviewing the patients as they were focused on following the checklist, while the checklist-oriented learning ensured a desirable outcome in history taking. Lee et al.16) explained that since CPX is an examination, students have difficulty in empathizing with their patients because they only concentrate on achieving a correct diagnosis and perceive that patients are actors, thereby reducing empathy expression by students during interviews when compared with actual patients and the CPX interview. During patient-physician interactions, role play alone does not provide adequate training, while clinical practice involving standardized patients may prove to be rather more effective.
Many studies have already discovered that clinical competency of an organ system cannot improve clinical competency of other systems; this is referred to as case specificity.17) The case of vaginal discharge in this study appeared as case of candida vaginitis on the actual examination, and cases practiced by students were trichomonas vaginitis, bacterial vaginitis, and atrophic vaginitis. Scores during history taking were noticeably improved for this item because the practiced cases were of the same clinical presentation as the case used during actual examination. Students practiced cases of gastroesophageal reflux disease (GERD), chronic obstructive pulmonary disease, and asthma for cough symptoms, while a case of tuberculosis appeared on the actual examination. Diverse diseases can be developed into CPX modules in the case of cough, and GERD is a key diagnosis to implicate the digestive system, unlike other cough problems (respiratory system). Therefore, this result suggests that scores may not increase unless role play practice is done on a wide variety of diseases for each item. However, this case specificity alone would not fully explain how the CPX results of the group, which underwent a preparation course, were higher than those of the group that did not undergo a preparation course. Training experiences would have led to acquiring relevant working knowledge and subsequent improvements in clinical competency in general.
The limitations of this study are as follows. First, the scope of this study is limited to activities conducted within a single medical school, and thus carries inevitable restrictions when its findings are presented to a broader scope. Second, the long-term effects of CPX preparation education on the performance at the national examination could not be examined, as only the short-term effects were observed and examined. Research in the future must examine the effect of CPX preparation education on improving the result of the practice examination within the KMLE. Third, it was not feasible to determine which method of CPX education is the most effective among module development, role play, and physical examination. Future studies are therefore required and should include several medical schools with additional examinations of the effects of standardized education modules.
In conclusion, the effects of the CPX preparation program, whose case development and role play among colleagues improved the ability of history taking, was shown to be greater in a station with a high case of similarity. As test experience with repeated exposure resulted in extensive changes in the general clinical competencies in the CPX, we concluded that a short preparatory educational module produced little effect in a clinical sense during clerkship for CPX. Further studies need to be conducted to determine the long-term effect of CPX preparation and its impact on the KMLE.