The Evolution of Mixed-Modes Patient-Reported Outcomes Guidance: A Journey Towards Evidence-Based Practices in BYOD | By: Dr. Jill Platko, Vice President of Scientific Services at Suvoda

Collecting patient-reported outcome (PRO) data is an integral function of clinical research, providing the patient perspective that enriches the evaluation of treatment effectiveness. Historically, patient-reported outcomes measurement (PROM) was performed predominantly with paper-based assessments. However, over the past two decades, there has been a notable paradigm shift towards the electronic collection of data.

Electronic data capture typically relies on devices equipped with pre-programmed software applications. They are distributed to study participants for use during a trial and subsequently returned to the study site upon trial completion.

In recent years, with the proliferation of personal smartphones, there has been a growing interest in having participants utilize their own smartphones or tablets to complete assessments in clinical studies. This practice is commonly referred to as “bring your own device” or BYOD. It is worth noting that, until recently, industry groups expressed concerns regarding proving the equivalence of various devices for data collection, which has posed challenges to the widespread adoption of BYOD.

This article delves into the journey of “mixed-modes” data collection from the prior emphasis on rigorous comparability testing to the adoption of an evidence-based approach, which recognizes the comparability of devices and paves the way to BYOD.

The Early Years: Establishing a Foundation

As the collection of PRO data became commonplace in clinical trials, the Food and Drug Administration (FDA) in 2009 developed guidance on how it would review and evaluate existing, modified, or newly-created PRO instruments (questionnaires) used to support claims in approved medical product labeling.

Part of this guidance called for sponsors to provide evidence to confirm the equivalence of an instrument when changing from a paper to electronic format, but the guidance was vague and didn’t specify the evidence needed to prove the change was acceptable.

As a result, a task force created by The Professional Society for Health Economics and Outcomes Research—better known as ISPOR—published recommendations addressing the implications of the FDA’s draft guidance for the collection of PRO data using electronic data capture modes of administration (ePRO). The ISPOR recommendations noted that migrating from paper to electronic PRO administration was considered a “minor” change and testing to show equivalence should be performed with a cognitive debrief (also referred to as a cognitive interview) and usability testing. Cognitive debriefing aims to assess the patient’s understanding of the instrument in the paper and electronic modes to ensure measurement comparability.  The usability testing was to ensure the system was usable by patients.

To some extent, the “minor” classification was adopted by ISPOR because there already was evidence accumulating that paper and electronic forms of assessments were equivalent, including a meta-analysis of 46 unique studies evaluating 278 measurements all showing equivalence.

Notably absent from the ISPOR recommendations was a trend taking hold in this and other industries: BYOD.

The Era of Bring Your Own Device (BYOD)

As time went on, the industry witnessed the rise of BYOD strategies. There are many positives to allowing BYOD in a clinical study. BYOD allows study participants to use devices they are comfortable with and often carry with them. This could lead to higher compliance and engagement in data collection activities. Implementing BYOD can also potentially reduce the costs associated with provisioning dedicated devices to participants.

At the same time, BYOD implementations can add a layer of complexity to PROMs strategies. Sponsors must be able to verify the accuracy and completeness of collected data and must adhere to FDA guidelines governing PROs. Researchers also need to ensure that the data collection platform or app is compatible with a wide range of devices and operating systems to accommodate participants’ diverse technology preferences and to ensure a consistent user experience.

In 2014, an ISPOR task force introduced guidelines on mixing modes within a trial. The crux of this guidance discouraged the mixing of paper and electronic data collection modes in a study, and went on to stipulate the need for equivalence studies if mixing electronic modes.

This essentially halted BYOD efforts as sponsors grappled with how to comply with ISPOR’s recommendation for cognitive debriefs to show equivalency for every potential device.

Evidence Leads to Relaxed Testing Requirements

In the ensuing years, multiple studies were conducted to address the issue of mixed-modes, including a large meta-analysis demonstrating the equivalence of paper and electronic measures. In 2017, a study of paper and BYOD measures showed strong evidence supporting the use of BYOD for PROM collection across scale types, as well as a high patient acceptance of BYOD.

Then in 2018, a large meta-analysis of cognitive debriefs and usability studies involving 53 studies with 68 unique instruments and 101 instrument evaluations found that accumulating evidence pointed to the possibility of relaxing the need to routinely conduct cognitive interviews and usability studies when implementing minor changes during instrument migration. In other words, BYOD strategies using best practices and conducted with a vendor with a good user interface should no longer need formal validation studies for every potential device.

Finally, an Evidence-Based Approach

In 2023, the ISPOR task force embraced a paradigm shift by recommending an evidence-based approach to PROM. This shift in approach recognized that the rigorous comparability testing previously required was becoming excessive in terms of resources and was no longer justified.

The evidence-based approach emphasizes that testing is appropriate only when there is a lack of evidence of measurement comparability, significant changes are introduced, or sponsors lack best practices for faithful migration. However, when sponsors have credible evidence of measurement comparability, even with minor or moderate changes, no further testing is needed. The implication is then that testing across all potential devices for BYOD is not needed and therefore BYOD strategies may move forward, thereby making BYOD an acceptable data collection method.

While some in the industry question why it took so long to recognize what the cognitive debriefs indicated five years ago, the evolution of PROM guidance from stringent comparability testing to an evidence-based approach is welcome news for sponsors. ISPOR’s updated recommendations simplify the path for study sponsors, reducing testing burdens and encouraging the adoption of modern electronic data capture methods.

Editor’s Note:  Dr. Jill Platko is vice president of scientific services at Suvoda. In her 16 years in the industry, Jill led the Scientific Advisory practice at Signant Health and held eCOA-related positions at Covance and PHT Corporation. She earned her doctorate in Genetics and Development from Cornell University. She can be reached at jplatko@suvoda.com

Medical Device News Magazinehttps://infomeddnews.com
Medical Device News Magazine provides our readership with breaking medical device / biotechnology news. Our subscribers include medical specialists, device industry executives, investors, and other allied health professionals, as well as patients who are interested in researching various medical devices. We hope you find value in our easy-to-read publication and its overall objectives! Medical Device News Magazine is a division of PTM Healthcare Marketing, Inc. Pauline T. Mayer is the managing editor.

Experts Views and Opinions

Caroline Petersen writes, "Forging an emotional connection and bonding with potential customers is critical to converting prospects into lifelong clients, but doing so can be challenging in complex fields like medical devices, where products involve intricate scientific engineering and precise functionality. This is where the power of visual storytelling comes in." Learn more.
For this American Heart Month, Dr. Trotter is equipping Scribes with essential life-saving knowledge, providing CPR training and enabling them to make a meaningful impact during life or death situations. Read on.
Last year was one of the worst years ever for cyber attacks on healthcare organizations. According to John Riggi, national advisor for cybersecurity and risk for the American Hospital Association, the number of people affected by attacks broke records, with US federal data indicating that around 106 million people were impacted by a healthcare data breach, more than double when compared to 2022.
Moby Cutter™ is one of the safest surgical wire cutters on the market today! Their medical design team has over 35 years of experience performing surgical procedures in Hospitals, Emergency Rooms and Urgent Care Systems. It has been cleared for commercialization per FDA guidelines and sold as a prescription only (Rx) medical device. Dr. Stanley shares the advantages of the Moby Cutter.
Health systems continue to face ongoing pressures to reduce costs while maintaining quality patient care. With the average operating margin for hospitals at 0.84%, health systems are searching for ways to more effectively manage operating and capital budgets. However, gaps in data often hinder health systems by hiding cost burdens throughout the organization. These hidden costs limit health systems to finding one-time-savings without lasting relief from industry-wide pressures.

By using this website you agree to accept Medical Device News Magazine Privacy Policy