Table 3 Usability testing cycle 1 and 2: number of recommendations categorized with select examples

Change management/process change

15

10

• Weight-based medication ordering and administration: site standard to avoid placing weight-based medication orders in advance of the surgery to avoid patient safety risk of weight changes resulting in dosage errors. Process of capturing patient weight in the system prior to ordering and administration of weight-based medications

• Change in process to document information and place orders during the visit instead of dictating after the visit was completed

• Roles and responsibilities for ordering and documentation (e.g., which role charts the lines or tubes intra-op may change from current process)

• Coordination of workflows using system between the different scheduler and clinical roles

• Standardizing what is required versus auxiliary documentation fields

Training and education

10

19

• New terminology: Definition of new terminology users didn’t understand (e.g., “patient class categories”)

• Clarify and educate on the how to schedule complex procedures (i.e., multiple procedure surgical cases) or reschedule surgical cases

• Charting by exception: there are too many unnecessary fields in the forms and flowsheets to accommodate other care areas/patient populations, determine what is the required documentation for these cases

• Training for Physicians on how to personalize their order sets and tools to improve efficiency

Software/build change

91

175

• Consents: usability issues with design, and flow of the consent forms (e.g., some of the patient representative roles listed cannot give consent; information entered in the consent fields were not populating onto the consent form)

• Aligning content/terminology to current workflows & practices (e.g., otorrhea instead of drainage)

• Re-organizing tools intuitively to match workflow (e.g., flow of the forms to match the sequence they will be filled out in the workflow; commonly used items or fields should be first)

• Re-design of order sets: Otolaryngology pre-procedure order sets were confusing, orders for intra-procedure administration were within the pre-procedure categories; the default dosages and frequency did not match the established workflows; terminology not common to users (e.g., non ORL terminology), titles not matching content of order

• Pre-procedure checklist: changes to reflect roles and responsibilities for specific items prior to surgery

• Building efficiency and reducing click fatigue (e.g., developing ordering tools with required defaults pre-selected or quick documentation tools to allow one click for all “normal parameters” to be selected versus five clicks to individually select them all; Patient positioning templates need to be accurately built for quick turnaround cases)

• Intra-procedure checklist/timeouts were inefficient (e.g., 3 separate checklists that required intra-op nurses to enter a username and password as part of the documentation to finalize each checklist). The initial build included duplicative information across the checklists; other roles were responsible for some of the items that they were attesting to, so should be removed from the intra-op nurse checklist

• Building coordination/awareness between various team members

• Medication naming and dosing defaults should match site standards (e.g., ibuprofen set to 10 mg/kg, every 8 h PRN but site standard is 5 mg/kg, every 6 h PRN)

• Alerts: (e.g., changes needed to avoid alert fatigue, capture critical errors such as weight change, max dosage alerts)

• Date fields: too many undefined date fields for booking a surgery, it was not clear to users as to what each field was referring

• Search terms and synonyms: Adding commonly used search terms as synonyms to improve searching tasks (e.g., “ENT” and “ORL” for otolaryngology service)

• Removing mandatory fields: or “hard stops” that were confusing to users or not necessary to be mandatory (e.g., documenting the count was correct for the initial instrument count)

• Build Misconfigurations: Medication/prescription: (e.g., adjusting a medication duration time or dosage for a discharge prescription didn’t automatically update to a new quantity to fill)

• Patient Belongings: selection options in these forms should align with age appropriate for pediatric site, not adult focused items (e.g., dentures may not be the most frequently documented item, so should not be located at the top right of the list. Toys or sippy cups are often brought with the patients so they should be added to the options available

Other

4

0

• Purchasing IT equipment (signature pads required to complete consents)

• Investigating downstream effects with different tools for Anesthesia providers to document with compared to other clinicians (e.g., to document airways in a note rather than the tool used by other clinicians, can the other clinicians see that information in their screens?)

Total # of recommendations per cycle

120

204