Where AI is Reshaping Imaging for Oncology Drug Development
Dr Diederik Grootendorst, Senior Director, Global Oncology Imaging at Bristol Myers Squibb, dispels some of the AI hype and highlights where artificial intelligence is moving the needle on imaging and decision-making, as well as where it’s not quite mature enough.
How is AI-enabled imaging complementing, augmenting or otherwise impacting existing methods and decision-making?
Simple segmentation and measurement tasks are already being impacted by today's tools which makes some assessments less time intensive and thereby cheaper to execute/operationalize on large datasets. I see a lot of AI automation in the radiology workflow, including real-time AI supported radiology reporting, PET-based segmentation, and automated lung lesion detection on x-ray, etc.
However, it is not that widely applied in oncology drug development, and more in early stages of development. With respect to imaging-based data-driven decisions, for example Go/No-Go decisions based on overall response, AI-based models can clearly help with providing additional data to augment/support that decision, but we are definitely not at a stage where tools are replacing more conventional radiological assessments. How much weight is to be put on a specific AI-based outcome prediction is being discussed amongst individual development teams and depends on the robustness of the tool.
For what decisions in drug development are AI-based models most impactful?
Generation of supportive data for early phase studies with relatively limited sample sizes, and more early decision-making especially in the context of futility. For Phase I dose decisions and whether we’ll move a certain dose forward, most of that is based on safety, but if you have a relatively safe drug, the efficacy of that drug also comes into play. Many of those decisions are currently being made with response data on very limited sample sizes, which then become the dose selected for the larger program. AI-based models, including imaging models, are now able to provide that sort of data. For example, if there is a 400 milligram dose and a 200 milligram dose with very similar ORRs – can we move the needle with additional data from AI-based models?
Phase I dose decisions can also be supported by AI-based modelling, including imaging-based models, on escalation and dose expansion cohorts to provide additional information on long-term efficacy estimations. For late-stage development, especially early predictions of long-term outcome could support early termination of futile trials if other outcome measures are inconclusive.
"AI-based models can clearly help with providing additional data to augment/support Go/No-Go decisions, but we are definitely not at a stage where tools are replacing more conventional radiological assessments."
Where do you feel AI-based imaging is not quite mature enough?
The question is whether or not these tools can be used in soft tissue lesion segmentations and measurement in the near future. We know that segmentation and accurate measurement n lung lesions is relatively doable by many of these segmenting tools because of the contrast, for example, between a soft tissue lesion and the air cavity of the lung.
However, many of these tools are not doing a good enough job when it comes to automated lesion segmentation in abdominal and pelvic cavities for example, which has been a topic many within the imaging community have tried to tackle for years. Partially because the contrast between various soft tissues is relatively poor. The fact that this remains a challenge ensures that full AI generated RECIST 1.1. evaluations are not expected to replace human-generated response assessments in clinical trials anytime soon. But overall, when we talk about which AI-enabled tools will have the biggest influence in drug development, image segmenting tools would definitely be included.
How are you implementing AI in your own imaging work?
We have started with more straightforward AI applications. One example is in certain image segmentation tasks which are very threshold-dependent, like Metabolic Tumor Volume assessments in lymphoma. Those are being integrated within our own internal software packages as well as simplifying workstreams at our core imaging vendors. That development is most likely proceeding and more complex segmentation tasks, like accurately segmenting brain lesions on MRI will most likely be added.
Within my role, we are also slowly transitioning away from model builds and training, and more towards prospective validation and application of AI-based models using imaging data for internal decision making. For example, we have been working for some years on training and validating early predictive models in metastatic non-small cell lung cancer, together with academic and commercial partners. Currently, we are seeing those models reporting their first results in a prospective context and thereby supporting decision-making. Interpretation and weight of results in the context of more conventional data is something that still requires internal alignment and education. However, our statistical colleagues are starting to embrace these tools and their results more and more.
"Part of navigating these challenges is teams getting more familiar with the capabilities and data outputs of these tools, and how they can be integrated in more standardized development processes, like DMC charters, Statistical Analysis Plans and blinding approaches."
What have been some of the topline takeaways from that?
Some takeaways from our internal efforts demonstrated that early imaging-based characteristics actually carry significant weight in outcome prediction models that also include general clinical data characteristics like age, sex, clinical performance etc. We’ve seen that some of these imaging-based features also prove to be more “robust” than others across sites, treatment, studies and tumor types.
If the modelled outcome aligns with early efficacy data like ORR and PFS, I feel data is generally felt as being supportive, but we still have work to do in explaining/understanding model outputs which are contrary to early conventional efficacy readouts and how that should best be incorporated into decision-making.
What other challenges exist when you’re bringing AI-enabled imaging into the workflow?
One challenge is that we built these models on retrospective datasets. Drug development changes very quickly, so how do we translate the results of these models to new therapies? For example, if something was developed on a dataset from IO plus chemotherapy, what happens with the interpretability of the results if you apply those models to a combination of IO plus chemo plus a new compound? What if you take results from a colorectal tumor to a lung tumor? Developments move quickly, and you have to act quickly in order to be supportive of a dose decision for a new combination. The challenge for decision-makers is how to interpret and apply those results from comparable, but not identical, combinations.
Another challenge surrounds robustness. We’re now understanding which features that go into these models are more robust across different hospitals and different equipment. For example, if you have a CT scan from a site in one country and one from a site in another on different systems, we are now starting to understand which features involved in our models are more usable across different institutions and as technology evolves.
What are some of the challenges inhibiting adoption into later stage development?
For late-stage development, the challenge is more related to integrating outputs of AI-based applications into the decision-making process and how that can be done in case of site and sponsor blinding. In many cases, certain patient specific information is required for algorithmic outputs to be produced and ensuring that information is available during, for example futility analyses, can be challenging.
Another consideration in late-stage development remains the potential use case of an application, as there can be direct consequences to the label of the drug even if they are being investigated in a more exploratory context. That results in a cautious stance regarding applying these tools in a late-stage context were results can be met with more scrutiny.
"Phase I dose decisions can also be supported by AI-based modelling, including imaging-based models, on escalation and dose expansion cohorts to provide additional information on long-term efficacy estimations."
How do we navigate these challenges?
Part of navigating these challenges is teams getting more familiar with the capabilities and data outputs of these tools, and how they can be integrated in more standardized development processes, like DMC charters, Statistical Analysis Plans and blinding approaches.
Others are harder to tackle for individual pharma companies alone, more specifically the robustness of certain models across tumor types, treatments and MOA’s. This is something that especially AI commercial companies might be able to drive by facilitating larger inter-company collaborations on certain AI based tools.
How are you approaching some of the change management challenges?
One challenge is talking with your colleagues about a certain level of uncertainty with these predictions and how much weight we can put on a new form of data, particularly when clinicians are not as familiar with it.
For example, let’s look at a compound dose decision, where the higher dose shows higher efficacy using a conventional imaging-based assessment. If we provide decision-makers with model data that shows the efficacy difference is not predicted to be that large with longer follow-up, more development leads would still favor the conventional data sets.
What we have been doing, as part of education around this, is giving folks an idea about what sort of weight they should attach to these models and their outputs before we discuss any results. It’s helpful for decision-makers who are used to looking into conventional data and need help understanding in what kind of context they have to interpret these new model generated predictions. It’s a challenge, but also an opportunity to integrate these kinds of tools into decision-making.
Are there any other advances in imaging (AI or non-AI) you want to spotlight for our audience?
Technological advances like whole-body PET, hybrid PET-MR, and photon-counting CT systems, are now increasingly being applied to monitor patients on clinical trials in larger institutions, especially in the US and EU. That means that the quality of the scans we receive during the duration of novel clinical trials is significantly better than in more historical trials.
It also means the heterogeneity of assessments performed at different centers has increased and requires us to think about how we handle “more” and potentially better-quality data. For example, hybrid FDG-PET (due to the speed and limited radiation exposure of whole-body PETs) is now performed as part of standard of care in some centers, and this results in more new lesions (sometimes false positive) to be found compared to those sites performing just regular CT. That could impact time-to-progression estimates for certain patients, while for other time-to-progression remains based on conventional methods.
