Electronic Health Record System Clinical Notes Usage Usability Evaluation – An Ethnographic study
Background: A significant gap exists between current Electronic Health Record (EHR) usability and potential optimal usability, which is often attributed to poor incorporation of a user-cantered approach during the Graphical User Interface (GUI) design process. Objectives: To evaluate usability strengths and weaknesses of two widely implemented EHR GUIs for critical clinical note usage tasks using data collected from real users observed in their actual inpatient work environments.
Methods: Twelve Internal Medicine resident physicians were observed by two usability evaluators while interacting with one of two EHR systems (Epic at University of Minnesota Medical Center and CPRS at Veterans Affair Hospital Care Systems), employing an ethnographic approach. User comments and observer findings were analyzed for two critical tasks: (i) clinical note entry and (ii) related information-seeking tasks, and from two standpoints: (a) usability references categorized by usability evaluators as positive, negative or equivocal and (b) usability impact of each feature measured through a seven-point severity rating scale. Findings were also validated by user responses to a post-observation questionnaire.
Results: For clinical note entry, Epic surpassed CPRS with more positive (26% vs. 12%) than negative (12% vs. 34%) usability references. Greatest impact features on EHR usability (severity score after each feature) for clinical note entry were auto-population (6), screen options (5.5), communication (5), copy pasting (4.5), error prevention (4.5), edit ability (4) and dictation & transcription (3.5). Neither system did better for information-seeking tasks with CPRS having more positive (28% vs. 14%) but also more negative (41% vs. 34%) references. Features pertaining to information–seeking tasks with greatest impact on EHR usability were navigation for notes (7) and others (e.g., looking for ancillary data) (5.5). Ethnographic observations were also supported by follow-up questionnaire responses.
Conclusion: This study provides usability specific insights of two widely used EHR systems that could help with future design of EHR interfaces better aligned with a user- centered approach.
(1) Hillestad R, Bigelow J, Bower A, Girosi F, Meili R, Scoville R, et al. Can electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Aff (Millwood) 2005 Sep-Oct;24(5):1103-1117.
(2) Zhang J, Walji MF. TURF: Toward a unified framework of EHR usability. J Biomed Inform 2011 12;44(6):1056-1067.
(3) Abras C, Maloney-Krichmar D, Preece J. User-centered design. Bainbridge, W.Encyclopedia of Human-Computer Interaction.Thousand Oaks: Sage Publications 2004;37(4):445-456.
(4) Courage C, Baxter K. Understanding Your Users : A Practical Guide to User Requirements Methods, Tools, and Techniques. Amsterdam: Morgan Kaufmann Publishers; 2005.
(5) Schumacher RM, Lowry SZ. NIST guide to the processes approach for improving the usability of electronic health records. National Institute of Standards and Technology 2010.
(6) Norman DA, Draper SW. User centered system design; new perspectives on human- computer interaction. : L. Erlbaum Associates Inc.; 1986.
(7) Norman DA. The psychology of everyday things. : Basic books; 1988.
(8) A survey of user-centered design practice. Proceedings of the SIGCHI conference on Human factors in computing systems: ACM; 2002.
(9) Johnson CM, Johnson TR, Zhang J. A user-centered framework for redesigning health care interfaces. J Biomed Inform 2005 2;38(1):75-87.
(10) Friedberg MW, Chen PG, Van Busum KR, Aunon F, Pham C, Caloyeras J, et al. Factors affecting physician professional satisfaction and their implications for patient care, health systems, and health policy. : Rand Corporation; 2013.
(11) Stead W, Lin H. Computational technology for effective health care: immediate steps and strategic directions. 2009.
(12) Middleton B, Bloomrosen M, Dente MA, Hashmat B, Koppel R, Overhage JM, et al. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013 Jun;20(e1):e2-8.
(13) Centers for Medicare and Medicaid Services (CMS). Health information technology: Standards, implementation specifications, and certification criteria for electronic health record technology, 2014 edition. Fed Regist 2012;77(171):53968-54162.
(14) Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda NC. Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. J Am Med Inform Assoc 2015 Nov;22(6):1179-1182.
(15) Kushniruk A. Evaluation in the design of health information systems: application of approaches emerging from usability engineering. Comput Biol Med 2002 5;32(3):141- 149.
(16) User interface evaluation in the real world: a comparison of four techniques. Proceedings of the SIGCHI conference on Human factors in computing systems: ACM; 1991.
(17) Rose AF, Schnipper JL, Park ER, Poon EG, Li Q, Middleton B. Using qualitative studies to improve the usability of an EMR. J Biomed Inform 2005 2;38(1):51-60.
(18) John B, E., Kieras D, E., John E, Bonnie., Kieras D, E. Using GOMS for user interface design and evaluation: which technique? ACM Transactions on Computer- Human Interaction (TOCHI) 1996;3(4):287-319.
(19) Poole A, Ball LJ. Eye tracking in HCI and usability research. Encyclopedia of human computer interaction 2006;1:211-219.
(20) Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004;37(1):56-76.
(21) Carroll C, Marsden P, Soden P, Naylor E, New J, Dornan T. Involving users in the design and usability evaluation of a clinical decision support system. Comput Methods Programs Biomed 2002;69(2):123-135.
(22) Liu Y, Osvalder A. Usability evaluation of a GUI prototype for a ventilator machine. J Clin Monit Comput 2004;18(5-6):365-372.
(23) Thyvalikakath TP, Monaco V, Thambuganipalle HB, Schleyer T. A usability evaluation of four commercial dental computer-based patient record systems. Journal of the American Dental Association (1939) 2008;139(12):1632.
(24) Sharda P, Das AK, Cohen TA, Patel V. Customizing clinical narratives for the electronic medical record interface using cognitive methods. Int J Med Inf 2006;75(5):346-368.
(25) Rizvi RF, Harder KA, Hultman GM, Adam TJ, Kim M, Pakhomov SV, et al. A comparative observational study of inpatient clinical note-entry and reading/retrieval styles adopted by physicians. Int J Med Inf 2016;90:1-11.
(26) Brown P, Marquard J, Amster B, Romoser M, Friderici J, Goff S, et al. What Do Physicians Read (and Ignore) in Electronic Progress Notes? Appl Clin Inform 2014;5(2):430-444.
(27) Using High-Fidelity Simulation and Eye Tracking to Characterize EHR Workflow Patterns among Hospital Physicians. AMIA Annual Symposium Proceedings: American Medical Informatics Association; 2015.
(28) Rosenbloom ST, Miller RA, Johnson KB, Elkin PL, Brown SH. Interface terminologies: facilitating direct entry of clinical data into electronic health record systems. J Am Med Inform Assoc 2006 May-Jun;13(3):277-288.
(29) Brown SH, Hardenbrook S, Herrick L, St Onge J, Bailey K, Elkin PL. Usability evaluation of the progress note construction set. Proc AMIA Symp 2001:76-80.
(30) Walji MF, Kalenderian E, Tran D, Kookal KK, Nguyen V, Tokede O, et al. Detection and characterization of usability problems in structured data entry interfaces in dentistry. Int J Med Inf 2013 2;82(2):128-138.
(31) Rodriguez NJ, Murillo V, Borges JA, Ortiz J, Sands DZ. A usability study of physicians interaction with a paper-based patient record system and a graphical-based electronic patient record system. Proc AMIA Symp 2002:667-671.
(32) Tang PC, Patel VL. Major issues in user interface design for health professional workstations: summary and recommendations. Int J Biomed Comput 1994 1;34(1– 4):139-148.
(33) Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda NC. Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. J Am Med Inform Assoc 2015 Nov;22(6):1179-1182.
(34) Smelcer JB, Miller-Jacobs H, Kantrovich L. Usability of electronic medical records. Journal of usability studies 2009;4(2):70-84.
(35) Shackel B. Usability-context, framework, definition, design and evaluation. Human factors for informatics usability 1991:21-37.
(36) Scholtz J. Usability evaluation. National Institute of Standards and Technology 2004.
(37) Jaspers MWM. A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence. Int J Med Inf 2009 5;78(5):340-353.
(38) Embi PJ, Yackel TR, Logan JR, Bowen JL, Cooney TG, Gorman PN. Impacts of computerized physician documentation in a teaching hospital: perceptions of faculty and resident physicians. Journal of the American Medical Informatics Association 2004;11(4):300-309.
(39) Rosenbloom ST, Denny JC, Xu H, Lorenzi N, Stead WW, Johnson KB. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc 2011 Mar-Apr;18(2):181-186.
(40) Solomon DH, Schaffer JL, Katz JN, Horsky J, Burdick E, Nadler E, et al. Can history and physical examination be used as markers of quality?: An analysis of the initial visit note in musculoskeletal care. Med Care 2000;38(4):383-391.
(41) Weed LL. Medical records, medical education, and patient care: the problem- oriented record as a basic tool. : Press of Case Western Reserve University; 1970.
(42) Smith K, Smith V, Krugman M, Oman K. Evaluating the impact of computerized clinical documentation. Comput Inform Nurs 2005 May-Jun;23(3):132-138.
(43) Ash JS, Bates DW. Factors and forces affecting EHR system adoption: report of a 2004 ACMI discussion. J Am Med Inform Assoc 2005 Jan-Feb;12(1):8-12.
(44) Rosenbloom ST, Miller RA, Johnson KB, Elkin PL, Brown SH. Interface terminologies: facilitating direct entry of clinical data into electronic health record systems. J Am Med Inform Assoc 2006 May-Jun;13(3):277-288.
(45) Johnson KB, Ravich WJ, Cowan Jr JA. Brainstorming about next-generation computer-based documentation: an AMIA clinical working group survey. Int J Med Inf 2004;73(9):665-674.
(46) Rose AF, Schnipper JL, Park ER, Poon EG, Li Q, Middleton B. Using qualitative studies to improve the usability of an EMR. J Biomed Inform 2005;38(1):51-60.
(47) Centers for Medicare & Medicaid Services (CMS), HHS. Medicare and Medicaid programs; electronic health record incentive program--stage 2. Final rule. Fed Regist 2012 Sep 4;77(171):53967-54162.
(48) Buller-Close K, Schriger DL, Baraff LJ. Heterogeneous effect of an emergency department expert charting system. Ann Emerg Med 2003;41(5):644-652.
(49) Clayton PD, Naus SP, Bowes WA,3rd, Madsen TS, Wilcox AB, Orsmond G, et al. Physician use of electronic medical records: issues and successes with direct data entry and physician productivity. AMIA Annu Symp Proc 2005:141-145.
(50) Hammond KW, Helbig ST, Benson CC, Brathwaite-Sketoe BM. Are electronic medical records trustworthy? Observations on copying, pasting and duplication. AMIA Annu Symp Proc 2003:269-273.
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution 3.0 License.