The healthcare industry has witnessed a significant surge in interest and opportunity to integrate artificial intelligence (AI) into care; diagnostics, clinical administration, treatment planning, and more. However, implementation remains limited, largely due to a lack of robust validation procedures. It is critical that healthcare professionals hold a level of skepticism in tension with open-mindedness about new technology, to ensure the patient receives the safest, quality care possible.
Today many leaders are forced to “adapt” or “die” when it comes to AI. They are living in a catch-22 where there is the option to either build the plane while flying it (which can increase the risk of everything from medical errors to security breaches), or they risk being overstepped in a highly competitive space due to a “lack of innovation.” As the nation continues to determine the proper regulatory oversight in the use of AI in healthcare, how do healthcare professionals toe this line? One way is by taking the “age-old” approach when evaluating virtually any care delivery system – asking the question: “Is this clinically validated?”
As a scientific advisor of ArteraAI, a leading precision medicine company developing AI tests to personalize cancer therapy, evaluating the validity or viability of a proposed project or initiative, is one of my primary interests.. And I see this question as a crucial first “stress test” to deploy when considering the adoption of any new technology.
Accuracy
Clinical validation allows for AI models to be subjected to rigorous testing against large and diverse datasets, comparing their performance against established benchmarks and gold standards. By validating AI algorithms with clinical data, we can assess their ability to correctly identify and classify diseases, predict outcomes, and recommend appropriate treatments. It is through this validation process that AI systems can be fine-tuned and optimized to achieve the highest levels of accuracy and precision.
For example, multiple Phase 3 randomized trials, with up to 15-year follow-ups were used to develop and validate prognostic and predictive AI models, when developing the ArteraAI Prostate Test, the first test that can predict whether an individual patient with localized prostate cancer will benefit from hormone therapy. The models were developed and validated with data from thousands of patients and terabytes of digital images.
Safety
Clinical validation plays a pivotal role in assessing the safety and reliability of new solutions. By scrutinizing the performance of AI algorithms within real-world clinical scenarios, we can identify potential pitfalls, vulnerabilities, and biases that may impact patient outcomes. Moreover, clinical validation helps to ensure that AI models exhibit robust performance across diverse patient populations, accounting for demographic variations and comorbidities.
For example, prostate cancer disproportionately impacts African American men, with this population making up 16.3% of prostate cancer patients, but only an average of 9.4% of clinical trial participants. It was paramount that the ArteraAI Prostate Test demonstrated similar performance in both African American and non-African American prostate cancer patients. For our test, the African American representation in model development and validation is approximately 20%.
By intentionally seeking diverse participation in clinical trials, healthcare professionals can help minimize the risks associated with false positives, false negatives, and the potential for algorithmic errors, thus promoting patient safety, for all.
Confidence
Subjecting AI models to thorough validation processes allows us to generate evidence that demonstrates effectiveness, accuracy, and reliability. This evidence can help instill confidence among healthcare clinicians and patients when implementing it as part of a treatment plan. This confidence for both clinicians and patients allows for greater shared-decision making when deciding on a treatment plan, which has been shown to improve the patient experience and improve outcomes.
Overall, clinical validation can help ensure that AI solutions are accurate, safe, and reliable, instilling trust among those it impacts the most. As new data becomes available and medical practices evolve, AI models must undergo consistent, periodic validation to ensure their continued accuracy and efficacy. AI has the potential to propel the healthcare industry forward, but we must move forward with caution and with the right checkpoints in place.
Note: Dr. Felix Feng is a radiation oncology leader in translational research. The primary aim of Dr. Feng’s research program is to individualize therapy for patients with aggressive prostate cancer, by identifying determinants of treatment resistance and developing strategies to overcome this resistance.
To enhance current clinical approaches from a biological perspective, his laboratory and dedicated research team are pursuing three major goals: 1) to identify novel molecular biomarkers of aggressive prostate cancer, 2) to understand the mechanisms by which several of these biomarkers drive disease progression, and 3) to develop therapeutic approaches to target these disease drivers.
Dr. Feng serves on the National Cancer Institute Genitourinary Cancer Steering Committee, which oversees and evaluates clinical trials proposed by all national clinical trials cooperative groups. He also serves as Chair of the Biology Scientific Track for the American Society of Radiation Oncology, and as Chair of the Genitourinary Translational Research Program for the RTOG/NRG national cooperative group.