Before radiation therapy can be used on cancer patients, treatment plans must be carefully reviewed to ensure the correct dose is delivered—too much causes side effects, while too little decreases effectiveness. This process relies on dosimeters, which measure radiation and help doctors adjust doses. However, because dosimeters are also used for treatment, limited access to these tools often causes delays and forces doctors to work overtime to complete treatment plans.
To ease the strain on oncology teams and speed up care, Johns Hopkins researchers are developing an AI-based quality assurance tool that could help streamline plan reviews while maintaining patient safety. They presented their work at the Association for Health Learning and Inference Machine Learning for Health Symposium held in December.
Anqi “Angie” Liu—an assistant professor of computer science, member of the Johns Hopkins Data Science and AI Institute, and lead author of the study—and Kevin He, Engr ’23, ’24 (MS) used AI to predict the gamma passing rate (GPR) of a plan to deliver intensity-modulated radiation therapy (IMRT), which emits precise doses of radiation to cancerous tissues while sparing the surrounding tissue. GPR is a measure of how closely the planned dose of radiation matches what is actually delivered—the higher the GPR, the safer the therapy plan.
Teaming up with David Adam and Sarah Han-Oh from the School of Medicine‘s Department of Radiation Oncology and Molecular Radiation Sciences, they first identified pain points in the current IMRT treatment pipeline, which involves radiation oncologists, nurses, physicists, dosimetrists, therapists, and, of course, patients.
The computer scientists then designed an algorithm that uses conformal prediction—a machine learning framework that provides a rigorous uncertainty estimation of an AI model’s best guess—to determine radiation therapy plans’ safety levels based on their complexity and likelihood of delivering a dangerous dose. They used historical data to train and evaluate the model on its sensitivity, specificity, and confidence level.
“This is the first time conformal prediction methods have been applied to improving IMRT quality assurance,” Liu says, “but they are particularly suitable because it is a safety-critical task with risk that needs to be carefully controlled.”
The researchers report that their method achieves 100% sensitivity without sacrificing specificity or making risky predictions, demonstrating that conformal prediction methods can be used to improve the efficiency of IMRT quality assurance.
Next steps for the team involve testing the model on a larger sample size and developing visualizations to help doctors better understand the model’s decisions.
“We’ve learned that it is possible to utilize uncertainty-aware machine learning approaches to identify unsafe radiation therapy plans,” Liu says, “and we look forward to evaluating our method in additional medical scenarios and real-life clinical settings in the future.”
This work was partially supported by Liu’s Amazon Research and Johns Hopkins Discovery Awards and seed grants from the Institute of Assured Autonomy and the Center for Digital Health and Artificial Intelligence.