Editor: What To Know
- “Our physicians who were not involved in the documentation acted out these vignettes and each scenario contained an element of distraction to determine if the AI would be thrown off by various nuances that might occur during a clinical visit — such as a parent and a minor sharing their thoughts, or a patient interjecting a story about a friend’s experience with hand surgery in the middle of providing an update on their own surgery,”.
- A new study presented at the 2023 AAOS Annual Meeting analyzed the use of documentation modalities including artificial intelligence (AI)-based virtual scribe services to determine the overall quality and time it takes to capture a patient encounter.
- “We wanted to look at ways to maximize the physician’s workload at the maximum level of their license and remove burdens that can lead to burnout by finding methods to outsource certain tasks, such as documentation, as this can be time consuming and redundant.
While Electronic Health Records improve access to patient information, charting increases the clerical burden on physicians as medical documentation has been identified as a contributing factor to burnout. A new study presented at the 2023 AAOS Annual Meeting analyzed the use of documentation modalities including artificial intelligence (AI)-based virtual scribe services to determine the overall quality and time it takes to capture a patient encounter. While AI proved to be a promising tool, some verification and correction is necessary for accuracy.
“In our practice, we created a task force to better understand and correct physician burnout to study what we know to be the top reason for burnout – patient documentation,” said Michael Rivlin, MD, FAAOS, orthopaedic surgeon, Rothman Institute and associate professor, Thomas Jefferson University, both in Philadelphia. “We wanted to look at ways to maximize the physician’s workload at the maximum level of their license and remove burdens that can lead to burnout by finding methods to outsource certain tasks, such as documentation, as this can be time consuming and redundant.”
Since the quality of AI-generated documentation had not been analyzed, the team compared the quality and time spent documenting common orthopaedic encounters in hand surgery using the following modalities during a patient visit:
- AI-based virtual scribe service – An AI program runs on a tablet, and everything said in the room is extracted by the machine.
- Medical scribe – A human being who is either physically in the office visit or participates virtually and transcribes the patient encounter.
- Transcription service – The physician uses a Dictaphone to record an audio file about the patient visit and sends it to a third-party company that transcribes what has been dictated.
- Voice recognition mobile (VRM) application – A program available on an electronic medical record platform that types the words based on voice recognition.
In the prospective study, “Use of Artificial Intelligence for Documentation in Orthopaedic Hand Surgery,” three fellowship-trained orthopaedic hand surgeons evaluated 10 standardized patients with prewritten clinical vignettes. Clinical documentation was performed during the clinical encounter using the AI-based scribe and medical scribe, and then afterwards using a VRM and transcription service.
“Our physicians who were not involved in the documentation acted out these vignettes and each scenario contained an element of distraction to determine if the AI would be thrown off by various nuances that might occur during a clinical visit — such as a parent and a minor sharing their thoughts, or a patient interjecting a story about a friend’s experience with hand surgery in the middle of providing an update on their own surgery,” explained Dr. Rivlin.
In total, 118 clinical encounters were documented including 30 AI scribe, 30 VRM, 28 transcription service and 30 medical scribe notes. Clinical notes were deemed as acceptable or unacceptable and assigned a letter grade (A, B, C or F) using an eight-point scoring system. Additionally, an attorney reviewed all notes for medical legal risk.
Overall, all modalities performed well with similar documentation outputs between each. Specific findings include:
- The AI scribe scored significantly lower than the other modalities for one specific question: “Is the plan correct?” – whereas the AI was able to get most of the verbal and implied elements of medical documentation; however, formulation of the plan was at times deficient compared to its human counterpart and manual edit of the plan section was required.
- Documenting clinical encounters through transcription services and voice recognition mobile applications requires substantial time compared to auto-populated AI-based notes. The average time per a note for VRM and the transcription service was 3.48 min and 3.22 min, respectively.
- AI-based scribe services rely on verbalized narrative throughout the entire encounter for accurate documentation, but some verification and correction are needed unlike a human scribe.
“The AI-based virtual scribe service is a promising tool to help decrease documentation burden without significantly lowering the quality of documentation compared to transcription and voice recognition software services,” said Dr. Rivlin. “While AI has some limitations, it continues to improve as the technology advances. These results create a palette of options for physicians to compare outputs should they want to explore new modalities.”
