|Year : 2021 | Volume
| Issue : 1 | Page : 52-56
Utility of mini-CEX in assessing postgraduates competency in orthopedics
Sanjay Deshpande, Mahendra Gudhe
Department of Orthopedics, Jawaharlal Nehru Medical College, Datta Meghe Institute of Medical Sciences (Deemed to be University), Wardha, Maharashtra, India
|Date of Submission||27-Jun-2020|
|Date of Decision||18-Dec-2020|
|Date of Acceptance||22-Feb-2021|
|Date of Web Publication||29-Jul-2021|
Dr. Mahendra Gudhe
Department of Orthopaedics, Datta Meghe Institute of Medical Sciences, Sawangi (M), Wardha, Maharashtra
Source of Support: None, Conflict of Interest: None
Background: Medical education aims at providing quality health care to community, which evidently needs professionally competent physicians. Mini-CEX tool can be employed to assess the core competencies of trainees during their routine clinical practice. This assessment tool offers a reliable measure of trainee's performance through being observed a number of brief encounters over a period, with a number of different assessors and in different clinical settings. So the study was carried out to test utility of Mini-CEX to test competence in postgraduate s in Orthopedics at a rural medical college in central India. Method and Material: Eighteen Orthopedics postgraduates underwent this assessment at an interval of 1 month to a total of 5 encounters. All were examined for seven skills. The timing of these assessments was mostly opportunistic, but the interval was maintained. Various said clinical test / skill was performed on real patient and rated them on the prevalidated checklist and provided feedback. Every skill was observed five times in similar case scenario. Nine point Likert scale was used for assessment. Results: Among the trainees the level of satisfaction was less varied than the supervisors and majority have shown interest with high level of satisfaction with the mean for Mini-CEX ranging from 7.94 to 8.27. And Pair wise analysis between 1st and 5th intervention, 3rd and 4th intervention and 3rd and 5th intervention was found to be significant. Conclusion: Mini-CEX is a method of assessing the clinical skills, which is task focused and quick to perform in a broader set of routine clinical practice administered as multiple encounters with variable patient problems and different examiners. Mini-CEX is a workplace-based assessment of performance, which has been reported to be reasonably reliable and valid to confer structured clinical skills learning in medical education.
Keywords: Likert scale, medical education, Mini-CEX, orthopedics, postgraduate
|How to cite this article:|
Deshpande S, Gudhe M. Utility of mini-CEX in assessing postgraduates competency in orthopedics. J Datta Meghe Inst Med Sci Univ 2021;16:52-6
| Introduction|| |
Medical education aims at providing quality health care to community, which evidently needs professionally competent physicians. The core clinical skills for undergraduate as well as postgraduate medical education has widely been establish and elaborated in most of the curriculum practiced in medical education. The quality of health care depends on acquisition of competence in a controlled environment and its performance in real life situation by the medical graduates. Competence-based curriculum has become necessary to meet the objectives of medical education. To achieve the optimal competence and performance of students a number of methods of assessments have been developed. Student's assessment provides a feed back to faculty on curriculum to see whether outcome is achieved or not. These initiatives have expanded the assessment of student's competency and performance beyond the traditional methods practiced in medical education. Objective structured clinical examination (OSCE) has been claimed to meet the acceptable standard of reliability and validity. Though feasible, setting up of a series of OSCE stations involves huge logistics, which is expensive to afford. Besides, competencies assessed through OSCE are limited. A number of other traditional methods to assess trainee's competence and performance are available. However, problem with these traditional methods are that these are time consuming and takes long hours to accomplish the task and is uninfluenced by any time constraints relevant to real life clinical practice. It has a random case specific performance, which is usually checked through one case that is often unobserved (long case assessment). Results are difficult to be generalized and there is always a subjective bias of the supervisors. Since the rating involves single evaluator or a panel of evaluators rating with consensus, the scores are less reliable for assessment. The need to explore a reliable and valid instrument to assess performance in clinical skills at workplace led to another feasible method of multiple mini clinical evaluation examination known as Mini-CEX. Mini-CEX was originally developed by the American Board of Internal Medicine USA to assess the medical residents in real life clinical setting. This tool can be employed to assess the core competencies of trainees during their routine clinical practice. This assessment tool offers a reliable measure of trainee's performance through being observed a number of brief encounters over a period, with a number of different assessors and in different clinical settings. For Mini-CEX to be reliable at least four encounters per year is needed during trainee's on-the-job performance if the measure is to be used as summative assessment for evaluating trainee's abilities. Hence, a study was carried out to test utility of Mini-CEX to test competence in postgraduates in orthopedics at a rural medical college in central India.
| Materials and Methods|| |
This was a prospective interventional study carried out in the Department of Orthopaedics, JNMC, Sawangi (Meghe) through the school for Health Professionals Education and Research, Datta Meghe Institute of Medical Sciences (Deemed University). The study was conducted after the approval from institutional ethical committee. Total Eighteen postgraduate trainees participated in this study whereas ten faculty members participated as assessors. All the participants were informed and trained about (Mini-CEX) by means of a face to face training with handouts of the checklist. Eighteen orthopedic postgraduates underwent this assessment at an interval of 1 month to a total of 5 encounters. All were examined for seven skills. The timing of these assessments was mostly opportunistic, but the interval was maintained. Various said clinical test/skill was performed on real patient and rated them on the prevalidated checklist and provided feedback. Inpatients cases such as osteoarthritis of hip, knee joint, internal derangement of knee joint, synovitis of various joints, malunited fractures, backache, etc., with mild-to-moderate complexity were chosen. Focus was given on data gathering, diagnosis, therapy, and counseling. Every skill was observed five times in similar case scenario. Nine point Likert scale was used for assessment. Score 1–3 means skill competency is unsatisfactory, 4–6 means skill is satisfactory but not superior, 7–9 means skill practiced is superior. After every intervention, constructive feedback was given to the residents and improvement was observed in next encounter. The assessing faculty ranged from associate professor to professor. One faculty observed one trainee in a given encounter. Time spent on procedure was 15–20 min and 5–10 min for feedback. A copy of the checklist was then collected by the researcher. Feedback was collected from the postgraduate trainees and faculty members after every encounter. Feedback was anonymous. The data were then subjected to statistical analysis by SPSS (Chicago, Illinois, USA) free version statistical software with 95% confidence interval of the difference and P < 0.05 considered as statistically significant.
The Institutional Ethics Committee of DMIMSDU has approved the Research work proposed to be carried out at Jawaharlal Nehru Medical College, Sawangi(M), Wardha. Date: 21st Oct 2018 with Reference no DMIMS(DU)/IEC/2018-19/2200.
| Observation and Results|| |
Total number of orthopedic residents involved in study was eighteen whereas the total number of supervisors involved was 10 with a student supervisor ratio of 1.8:1. Among the trainees, the level of satisfaction was less varied than the supervisors and majority have shown interest with high level of satisfaction with the mean for Mini-CEX ranging from 7.94 to 8.27 and pairwise analysis between 1st and 5th intervention, 3rd and 4th intervention and 3rd and 5th intervention was found to be significant [Table 1]. Among the supervisors, majority were satisfied with Mini-CEX and the mean level of satisfaction ranged from 4.44 in first intervention to 6.05 in fifth intervention. In pairwise analysis, there is significant difference in satisfaction at each intervention [Table 2]. Maximum gain was observed in physical examination skill. Here, mean ranged from 4.66 in first intervention to 6.44 in fifth intervention, showing 27.6% improvement. Medical interviewing skill mean ranged from 4.66 in first intervention to 6.22 in fifth intervention showing 25% improvement. Humanistic qualities/professionalism skill mean ranged from 4.66 in first intervention to 6.22 in fifth intervention showing 25% improvement. Clinical judgment skill mean ranged from 4.61 in first intervention to 5.94 in fifth intervention showing 22% improvement in skill. Counseling Skill mean ranged from 4.66 in first intervention to 5.83 in fifth interventions showing 19.6% gain or improvement in skill. Organization/efficiency skill mean ranged from 4.44 in first intervention to 5.83 in fifth intervention showing 23.8% improvement in skill. Overall clinical competence skill mean ranged from 4.55 in first intervention to 6.00 in fifth intervention showing 24.1% gain in overall skill performance [Table 3].
|Table 1: The resident satisfaction with MiniCEX at different intervention|
Click here to view
|Table 2: The evaluator's (supervisors) satisfaction with MiniCEX at different intervention|
Click here to view
| Discussion|| |
The Mini-CEX since its inception is considered as a positive tool for the formative assessment of residents in the clinical departments. In this study, we observed satisfaction of the post graduate trainee and evaluator to this method of teaching and learning. The satisfaction among post graduate trainee was not significant in many sessions whereas satisfaction of evaluator or supervisor showed significant difference in the session. Hence, this observation can be taken or considered as challenges of the study. Some of the challenges to introduction and integration of Mini-CEX in the present curricular plan are no clarity on what to asses and the norms to expect. Even with ample opportunities of direct observation of trainees work every day, we as trainers, let go of these essentially because most of us are not consciously aware of the immense teaching-learning potential of this simple act. We wait for holding a formal examination to evaluate the students. Hence, programs on WPBA may be an initial step in making everyone aware of the opportunities available at hand every day. Clarity on what to assess and the norms to expect will reduce the chances of suboptimal performance or essential skill being missed out by assessors, especially those who are less experienced. Hence, it will contribute to the standardization of the assessment. The art of giving effective feedback is very important as the benefit of the entire exercise may be lost if the feedback is not delivered in an appropriate positive manner with suggestions for improvement. An additional issue could be of a potential conflict in the role of faculty members as a teacher as well as assessor. This may surface as unwillingness to record negative evaluations and therefore a possible likelihood of failing to identify the residents in difficulty. This barrier may, at least in part, be overcome by appropriate sensitization and training of faculty. Demonstrating feasibility: Introducing a new method in the assessment plan requires much enterprise and planning. This inertia is partially overcome if one sees it.
We found the great improvement in the skills of the trainee such as physical examination, medical interviewing, humanistic qualities/professionalism, clinical judgment, counseling, organization/efficacy, and overall clinical competence at the end of 5th intervention. The similar observation in terms of physical examination was made by previous studies.,, Professionalism or human qualities have 8 items. These are-shows respect, sensitivity to disease or suffering client, empathy, build trust attitudes, shows confidence, behaviors that demonstrate ethical values, demonstrated ability caring, eye contact, tone, posture, the ability to see the limitations of self and patient. These qualities are inherent part of the medical profession and it is individual characters of the postgraduate. However, the feedback reveals that these characters showed significant improvement in few postgraduate. However, most of them having it since beginning. A constructive feed back to the post graduates about their behavior while examination could be the explanation of the improvements.
The clinical judgment is the ability to make appropriate diagnoses, and the ability to assimilate the action plan to manage such clinical situation. More recently, the use of workplace-based formative assessment strategies such as the Mini-CEX, or variants thereof, has led to an interest in the potential impact of feedback, provided during formative assessment events, on trainee learning behavior in the workplace environment. Counseling are capabilities to provide consultation/health education, ability to provide health education according to the profile of the patients. Counseling of the patients and their relative is integral part of modern medicine practice. The counseling has ethical and medico legal dimensions. Post graduate trainees could demonstrate interpersonal and communication skills that result in the effective exchange of information and collaboration with patients, their families and health professionals. In the later sessions, the difference was statistically significant that reveals it has positive impact on this aspect of the clinical acumen. The need to improve the communication skill of clinical faculty was also demonstrated in the study of Lin et al. which examined the expression of empathy by emergency physicians. Many emergency physicians are not aware of the effects of nonverbal communication on their patients, such as conversations while their eyes look at a computer screen or conversation without asking if the patient understood the problem or the treatment plan. Braddock et al. found that among 1,057 counseling sessions involving primary care physicians and surgeons, only 9% of the encounters met the basic criteria for effective informed decision making. After the introduction of the National Health Insurance, medical costs are no longer an issue of health access in Taiwan. As a result physicians may overlook the importance of communication with their patients because patients with NHI tend to agree with the treatment plan suggested by physicians. For organization skills, preceptors examined the ability of the residents to make prioritized decisions in a limited time. Residents who are “efficient” should be familiar with the proper workflow of diagnostic tests, imaging, and disposition to avoid stagnation in the patient flow. Examples of feedback items were: “For major trauma, you have to complete the primary survey before starting the secondary survey,” or “When treating posterior dislocation of the hip joint, or “Please be familiar with the hospital's computer system so that you can manage your patients more quickly.” We observed significant difference between extreme sessions and organizational skill has improved drastically. The learner demonstrates good judgment, synthesis, caring, effectiveness, and efficiency.,,,,
| Conclusion|| |
Mini-EX is a method of assessing the clinical skills, which is task focused and quick to perform in a broader set of routine clinical practice administered as multiple encounters with variable patient problems and different examiners. Mini-CEX is a workplace-based assessment of performance, which has been reported to be reasonably reliable and valid to confer structured clinical skills learning in medical education.
A motivated faculty and organized approach toward a comprehensive knowledge on Mini-CEX as an assessment tool for its background communication, demonstration of procedure and method to complete the rating forms is the useful guide to adopt Mini-CEX. The present study conducted with intention of examining its utility as competency assessment method to measure competency has found this measurement tool satisfactory to be accepted and feasible to be practiced.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Kroboth FJ, Hanusa BH, Parker S, Coulehan JL, Kappor WN, Brown FH, et al
. The inter-rater reliability and internal consistency of clinical evaluation exercise. J Gen Intern Med 1992;7:174-9.
John JN, Linda LB, Daniel D, Gregory SF. The Mini-CEX: A method for assessing clinical skills. Ann Intern Med 2003;138:476-81.
Carr S. The foundation programme assessment tools: An opportunity to enhance feedback to trainees? Postgrad Med J 2006;82:576-79.
Norcini JJ, Burch V. Workplace-based assessment as an educational tool: AMEE Guide N Med Teach 2007;29:855-71.
Singh T, Sharma M. Mini-clinical examination (CEX) as a tool for formative assessment. Natl Med J India 2010;23:100-2.
Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (clinical evaluation exercise): A preliminary investigation. Ann Intern Med 1995;123:795-9.
Durning SJ, Cation LJ, Markert RJ, Pangaro LN. Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Acad Med 2002;77:900-4.
Lin CS, Shu MY, Chong CF. Differences between emergency patients and their doctors in the perception of physician's empathy: Implications for medical education. Educ Health 2008;21:144.
] [Full text]
Braddock CH 3rd
, Edwards KA, Hasenberg NM, Laidley TL, Levinson W. Informed decision making in outpatient practice: Time to get back to basics. JAMA 1999;282:2313e2320.
Chen L, Yip W, Chang MC, Lin HS, Lee SD, Chiu YL, et al
. The effects of taiwan's national health insurance on access and health status of the elderly. Health Econ 2007;16:223e242.
Singh S, Pande V, Gajbe U, Sawal A, Dhopavkar GM. Comparison of Performance of Medical Graduates Following Competency Based Learning and Conventional Learning in Indian Medical Colleges. International Journal of Pharmaceutical Research 2019;11:1165-8. Available from: https://doi.org/10.31838/ijpr/2019.11.01.206
. [Last accessed on 2020 Mar 04].
Srivastava TK, Waghmare LS, Rawekar AT. Transition to Competency-Based Medical Education: A Proposed Rollout Model. Int J Cur Res Rev 2020;12:117-22. Available from: https://doi.org/10.31782/IJCRR.2020.121425
. [Last accessed on 2020 Mar 04].
Acharya N, Srivastava T, Acharya S, Shukla S, Dewani D. Role of Mock Drills in Competency Based Post Graduate Medical Education in Managing Emergency Clinical Situation. Eur J Mol Clin Med 2020;7: 1436-47.
Goyal RC, Choudhari SG, Tankhiwale SR. Assessment of Competency Based Medical Internship Training with 'Cumulative Grade Points Average System'-An Innovative Step towards Meeting 'Vision 2015' of Medical Council of India. Indian J Public Health Res Dev 2018;9: 155-62. Available from: https://doi.org/10.5958/0976-5506.2018.00713.1
. [Last accessed on 2020 Mar 04].
Niranjane P, Mishra V, Daigavane P, Gilani R. Faculty Perceptions as Part of Needs Assessment for Designing Competency-Based Interprofessional Educational Model in Orthodontics. J Datta Meghe Inst Med Sci Univ 2020;15:241-6. Available from: https://doi.org/10.4103/jdmimsu.jdmimsu_228_20
. [Last accessed on 2020 Mar 04].
[Table 1], [Table 2], [Table 3]