Original Article

Validity, reliability and acceptability of Professionalism Mini-Evaluation Exercise (P-MEX) for emergency medicine residency training

Abstract

Professionalism is a core competency in the medical profession. In this paper, we aimed to confirm the validity, reliability and acceptability of the Professionalism Mini-Evaluation Exercise (P-MEX) instrument for the emergency medicine (EM) residency program. Twenty-two EM attending physicians completed 383 P-MEX forms (the Persian version) for 90 EM residents. Construct validity was assessed via structural equation modeling (SEM). The reliability coefficient was estimated by the generalizability theory, and acceptability was assessed using two researcher-made questionnaires to evaluate the perspectives of residents and assessors. There was a consensus among the participants regarding the content of P-MEX. According to the results of SEM, the first implementation of the original model was associated with a moderate fit and high item loadings. The model modified with correlated error variances for two pairs of items showed an appropriate fit. The reliability of P-MEX was 0.81 for 14 occasions. The perception survey indicated high acceptability for P-MEX from the viewpoint of the residents and increasing satisfaction with P-MEX among the assessors over time.
According to the results of the research, P-MEX is a reliable, valid, and acceptable instrument for assessing professionalism in EM residents.

Papadakis MA, Loeser H, Healy K. Early detection and evaluation of professional development problems in medical school. Acad Med. 2001; 76(11):1100–6.

Swick HM. Towards a normative definition of professionalism. Acad Med. 2000; 75(6): 612-6.

Shumway JM, Harden RM. AMEE Guide no. 25: the assessment of learning outcomes for the competent and reflective physician. Med Teach. 2003; 25(6): 569-84.

Li H, Ding N, Zhang Y, Liu Y, Wen D. Assessing medical professionalism: a systematic review of instruments and their measurement properties. PLoS One. 2017;12(5): e0177321.

Wilkinson TJ, Wade WB, Knock ID. A blueprint to assess professionalism: results of a systematic review. Academic Medicine. 2009; 84:551–8.

Cruess R., McIlroy JH, Cruess S, Ginsburg S, Steinert Y. The professionalism mini-evaluation exercise: a preliminary investigation. Acad Med. 2006; 81(10 Supple): S74-8.

Cruess RL, Cruess SR, Snell L, et al. Teaching, learning and assessing professionalism at the post graduate level. [cited 2017 September]; Available from: https://www.afmc.ca/pdf/fmec/20_Cruess_Professionalism.pdf

Ginsburg S, Regehr G, Hatala R, et al. Context, conflict, resolutions: a new conceptual framework for evaluating professionalism. Acad Med. 2000; 75(10 Supple); S6-S11.

Tsugawa Y, Tokuda Y, Ohbu S, et al. Professionalism mini‐evaluation exercise for medical residents in Japan: a pilot study. Medi Educ. 2009; 43(10): 968-78.

Tsugawa Y, Ohbu S, Cruess R, et al. Introducing the professionalism mini-evaluation exercise (P-MEX) in Japan: results from a multicenter, cross-sectional study. Acad Med. 2011; 86(8): 1026-31.

Rodriguez E, Siegelman J, Leone KA, Kessler CS. Assessing professionalism: summary of working group on assessment of observable learner performance. Acad Emerg Med. 2012; 19(12):1372–8.

Larkin GL, Binder L, Houry D, Adams J. Defining and evaluating professionalism: a core competency for graduate emergency medicine education. Acad Emerg Med. 2002; 9(11): 1249-56.

Arora M, Asha S, Chinnappa J. Review article: burnout in emergency medicine physicians. Emerg Med Australas. 2013; 25(6): 491–5.

Fernández Martínez O, Hidalgo Cabrera C, Martín Tapia A, Moreno Suárez S, García Del Río B. Burnout among resident physicians who work duty shifts in the emergency department. Emergencias. 2007; 19(3):116–121.

Yuguero Oriol, et al. Empathy and burnout of emergency professionals of a health region: A cross-sectional study. Medicine (Baltimore). 2017; 96(37): e8030.

Muñiz J, Elosua P, Hambleton RK. International test commission guidelines for test translation and adaptation: second edition. Psicothema. 2013; 25(2): 151–7.

Kline RB. Principles and Practice of Structural Equation Modeling, 3rd ed. USA: The Guilford Press; 2010.

Brown TA. Confirmatory factor analysis of the Penn State Worry Questionnaire: multiple factors or method effects? Beha Res Ther. 2003; 41(12): 1411-26.

Brown TA. Confirmatory Factor Analysis for Applied Research, 2nd ed. USA: The Guilford Press; 2015.

Pascual-Ferrá P. Congenericity and the Measurement of Interpersonal Communication Constructs: A Confirmatory Factor Analytic Test of Four Measures [dissertation]. Miami (USA). University of Miami; 2013.

Pascual-Ferrá P, Beatty MJ. Correcting internal consistency estimates inflated by correlated item errors. Communication Research Reports. 2015; 32(4): 347–52.

Colletti LM. Difficulty with negative feedback: Face-to-face evaluation of junior medical student clinical performance results in grade inflation. J Surg Res. 2000; 90(1): 82–7.

Watling C, LaDonna KA, Lingard L, Voyer S, Hatala R. ‘Sometimes the work just needs to be done’: socio-cultural influences on direct observation in medical training. Med Educ. 2016; 50: 1054–64.

Lie D, Encinas J, Stephens F, Prislin M. Do faculty show the ‘halo effect’ in rating students compared with standardized patients during a clinical examination? Internet Journal of Family Practice. 2009; 8: 2.

Dyrbye LN, Harper W, Moutier C, et al. A multi-institutional study exploring the impact of positive mental health on medical students' professionalism in an era of high burnout. Acad Med. 2012; 87(8): 1024–31.

Dyrbye LN, Massie FS, Eacker A, et al. Relationship between burnout and professional conduct and attitudes among US medical students. JAMA. 2010; 304(11): 1173–80.

Cruess RL, Cruess SR, Boudreau JD, Snell L, Steinert Y. Reframing medical education to support professional identity formation. Acad Med. 2014; 89(11): 1446-51.

Cruess RL, Cruess SR, Boudreau D, Snell L, Steinert Y. A schematic representation of the professional identity formation and socialization of medical students and residents: A guide for medical educators. Acad Med. 2015; 90(6): 718–25.

Bouchard T J Jr, McGue M. Genetic and environmental influences on human psychological differences. J Neurobiol. 2003; 54(1): 4–45.

Strus W, Cieciuch J. Towards a synthesis of personality, temperament, motivation, emotion and mental health models within the circumplex of personality meta traits. Journal of Research in Personality. 2017; 66: 70–95.

Goldie J. Identity formation in medical students: an elaboration of a previous conceptualization and review of the literature. MedEdWorld. 2013; 1–23.

Holden M, Buck E, Clark M, Szauter K, Trumble J. Professional identity formation in medical education: the convergence of multiple domains. HEC Forum. 2012; 24(4): 245–55.

Markovitch N, Koen L, Klimstra TA, Abramson L, Knafo-Noam A. Identity exploration and commitment in early adolescence: genetic and environmental contributions. Dev Psychol. 2017; 53(11): 2092–102.

Files
IssueVol 12 (2019) QRcode
SectionOriginal Article(s)
DOI https://doi.org/10.18502/jmehm.v12i12.1641
Keywords
Medical professionalism Emergency medicine Residency program Workplace- based assessment
Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
1.
Amirhajlou L, Bidari A, Alipour F, Yaseri M, Vaziri S, Rezaie M, Tavakoli N, Farsi D, Yasinzadeh M, Mosaddegh R, Hashemi A. Validity, reliability and acceptability of Professionalism Mini-Evaluation Exercise (P-MEX) for emergency medicine residency training. J Med Ethics Hist Med. 2019;12.