Direct Observation of Procedural Skills for Entry-Level Competency Assessment: A Pilot Study of Arterial Blood Gas sampling
DOI:
https://doi.org/10.62438/tunismed.v104i07.6739Keywords:
Clinical Competence, Undergraduate Medical Education, Feedback, Emergency MedicineAbstract
Introduction: Proficiency in arterial blood gas (ABG) sampling is expected of senior medical students, yet it is often assumed rather than objectively assessed. Early identification of procedural skill gaps is essential to ensure patient safety, particularly in emergency settings. This study aimed to evaluate the value of Direct Observation of Procedural Skills (DOPS) as a diagnostic tool at the start of an emergency medicine clerkship.
Methods: In this prospective pilot study, 21 fifth-year medical students were assessed while performing ABG sampling on real patients using a standardized DOPS checklist. The primary outcome was the level of supervision required. Secondary outcomes included performance in technical and non-technical domains, prior procedural experience, and self-reported stress.
Results: Sixteen students (76%) required direct supervision, while only one (5%) achieved near-independent performance. Technical skills were generally satisfactory; however, major deficiencies were observed in informed consent (71%) and communication (62%). Prior experience of ≥10 procedures was significantly associated with higher competence (p < 0.01), whereas no association was found with stress levels.
Conclusion: Procedural competence in ABG sampling is heterogeneous among senior students. Early implementation of DOPS helps identify gaps,
particularly in non-technical domains, and supports tailored supervision to enhance patient safety.
Downloads
References
Monga S, Agarwal D. Evaluation of direct observation of procedural skills (DOPS) as an assessment method for MBBS students.
Res J Med Sci. 2024;18:436–440. https://doi.org/10.36478/ makrjms.2024.12.436.440
Cunanan B, Muppa H, Orellana MA, Bates RN, McGain F. Blood gas sampling in the intensive care unit: a prospective before-and-after
interventional study on the effect of an educational program on blood gas testing frequency. Aust Crit Care. 2024;37(5):755–760.
https://doi.org/10.1016/j.aucc.2024.01.009
Erfani Khanghahi M, Ebadi Fard Azar F. Direct observation of procedural skills (DOPS) evaluation method: a systematic review
of evidence. Med J Islam Repub Iran. 2018;32:45. https://doi. org/10.14196/mjiri.32.45
Asadi K, Mirbolook AR, Haghighi M, Sedighinejad A, Naderi Nabi B, Abedi S, et al. Evaluation of satisfaction level of orthopedic interns
from direct observation of procedural skills assessment (DOPS). Res Med Educ. 2012;4(2):17–23.
Watson MJ, Wong DM, Kluger R, Chuan A, Herrick MD, Ng I, et al. Psychometric evaluation of a direct observation of procedural
skills assessment tool for ultrasound-guided regional anaesthesia. Anaesthesia. 2014;69(6):604–612. https://doi.org/10.1111/
anae.12625
Cohen SN, Farrant PB, Taibjee SM. Assessing the assessments: UK dermatology trainees’ views of the workplace assessment tools. Br
J Dermatol. 2009;161(1):34–39. https://doi.org/10.1111/j.1365- 2133.2009.09097.x
Bindal N, Goodyear H, Bindal T, Wall D. Direct observation of procedural skills (DOPS) assessment: a study to evaluate the
experience and opinions of trainees and assessors. Med Teach. 2013;35(6):e1230–e1234. https://doi.org/10.3109/014215
X.2012.746447
Dabaghi Tabriz F, Bahramian A, Asdagh S, Ghadimi F, Katebi K. The effect of direct observation of procedural skills on learning and
satisfaction of students in a community dentistry course. J Adv Med Educ Prof. 2024;12(2):111–117. https://doi.org/10.30476/
JAMP.2023.100431.1890
Lagoo JY, Joshi SB. Introduction of direct observation of procedural skills (DOPS) as a formative assessment tool during postgraduate
training in anaesthesiology: exploration of perceptions. Indian J Anaesth. 2021;65:202–209. https://doi.org/10.4103/ija.
IJA_124_20
Choudhary A, Gupta V. Teaching communications skills to medical students: introducing the fine art of medical practice. Int J Appl Basic Med Res. 2015;5(Suppl 1):S41–S44. https://doi.org/10.4103/2229-516X.162273
Rela M, Price T. Review of the validity of direct observation of procedural skills (DOPS) as an assessment tool for the procedural
skills of surgical trainees. Ann R Coll Surg Engl. 2023;105(7):599– 606. https://doi.org/10.1308/rcsann.2022.0052
Yalcinturk AA, Ustun B. Medical errors room: the effect of simulation training on increasing awareness of nursing students about medical errors encountered in psychiatric clinics. Arch Psychiatr Nurs. 2023;43:37–42. https://doi.org/10.1016/j.apnu.2022.12.030
Haldane T. "Portfolios" as a method of assessment in medical education. Gastroenterol Hepatol Bed Bench. 2014;7(2):89-93.
Cobb K, Brown G, Jaarsma D, Hammond R. The educational impact of assessment: a comparison of direct observation of procedural
skills and multiple-choice questions. Med Teach. 2013;35(11): e1598–e1607.
https://doi.org/10.3109/0142159X.2013.803061
Amini A, Shirzad F, Mohseni MA, Sadeghpour A, Elmi A. Designing direct observation of procedural skills (DOPS) test for selective
skills of orthopedic residents and evaluating its effects from their points of view. Res Dev Med Educ. 2015;4(2):147-152. https://doi.
org/10.15171/rdme.2015.026
Additional Files
Published
License
Copyright (c) 2026 La Tunisie Médicale
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
