How Simulation-Guided Trial Design Can Take Predictions of Your Drug For a More Optimized Clinical Trial Design

What considerations should CMOs have in mind when designing clinical trials?

CMOs have a lot on their plate these days. They are an integration node for a lot of different information: the latest science, medicine, clinical practice, and on top of that, they are supposed to be experts at strategic clinical development. When CMOs think about designing their trials, they have to balance many different complex agendas: What is the best way to test their asset? What is strategically best for their company to quickly get to their next decision point, whether it’s a critical phase two proof-of-concept trial or a proof-of-mechanism trial? They need to balance the priorities of the patients they serve, the investors in their company, and the development team they lead. Add on top of that the burden of trying to be a scientific expert at how to best design a clinical trial.

We are trying to help out with that last particular aspect of a CMO’s agenda. The science of trial design can be quite complicated when you drill down into it. Many advances in statistical methodology have evolved and been implemented over the last couple of decades. Not everybody can be an expert in Pareto optimization or adaptive elements that might most efficiently use time and money during a clinical trial. The critical question for CMOs when designing clinical trials is what design will produce clear, actionable data? If they don’t have deep familiarity with many design options, there are specialists who can help inform their design options and which might best fit their particular priorities.

"There’s the old adage, “Good, fast, cheap: pick any two.” You can never have everything at once."

How can CMOs balance the competing factors of cost, speed, and strength of a clinical trial?

There’s the old adage, “Good, fast, cheap: pick any two.” You can never have everything at once. One way to clearly make a decision is by getting some data around what good looks like, what fast looks like and what cheap looks like. It’s helpful to understand the boundaries and tradeoffs between how you spend your money on any one of those three priorities, and one way to accomplish this is through simulation guided trial design.

You probably would not think about building a house without some architectural plans. What we do with simulation-guided trial design is much like drawing out those architectural plans with computer renderings to show you what your house might look like if you prioritize. Additionally, if you blended those priorities 25-25-50, what would that look like? You can never make everybody happy. But what you can do is provide data around each one of those priorities, to show your team transparently what you want to invest in and what you might be trading off when you raise or lower one of those elements on your priority list.

Similarly, if we address tradeoffs from a strategic standpoint, you can operate on a couple of different levels. At the single study level, there are several different potential tradeoffs involving numbers of patients and how many sites to open, for example. At the development plan level, regulatory acceptability and speed to market come into play as well. The tradeoffs apply at every level, from the 20,000-foot development plan all the way down to the single trial.

Can you explain simulation-guided trial design?

Simulation-guided trial design is a cornerstone of clinical trial science and has been around for many years. It takes some of the tried and true principles of statistics and applies them to a given study design that a CMO might bring to the table. It is not a black box with artificial intelligence that automatically spits out the right answer. Typically, simulation guided trial design is the process wherein, for a given set of assumptions around variability, Monte Carlo simulations are computed to show what might happen if a clinical trial were to run many, many times. If you believe your new drug may lower the risk of dying by 30 percent, what are the odds that you will observe that result in a clinical trial with your design? Simulations provide some data on how that might look in reality. As applied by a skilled statistician, simulation guided trial design generates data informing teams how to best design their study based on their priorities.

We don’t use simulations to make predictions of how well a drug is going to do. Rather, we use sponsor input to help guide design decisions. For example, if a sponsor tells us, “In animals this has been shown to reduce death by 30 percent,” we take that and we say, “If you believe that the true effect size is 30 percent, and if you treat patients with this drug for six months, here is what the simulated data are if we ran that same clinical trial 1,000 times, 5,000 times or 10,000 times.” From this data set, we can help teams understand probabilities and risks. What is the likelihood that after that trial is run you will actually observe that 30 percent reduction in death that you think you are going to see based on your animal studies? That’s the kind of question simulation guided trial design answers.

"Having an external perspective helps guide that process can be extremely valuable and a competitive advantage."

How can CMOs use this effectively, to make sure that they’re not setting themselves up for failure?

One thing that most clinicians like to do is plan for the worst case scenario. And that is absolutely something that simulation guided trial design helps people do. Going back to that fictional asset, let’s say the sponsor thinks it reduces your risk of death by 30 percent. What happens in the scenario where it actually reduces death by only 10 percent? Or what happens in the scenario where it is better than you think and reduces death by 40 percent? In any one of those scenarios, you would want to understand how your clinical trial design might perform. What kind of decisions could you make from the data and at what point during the trial? What if you stopped early? What if you had to increase enrollment? What are the implications on cost and time that you might see? We help CMOs and their teams understand and plan for those worst and best case scenarios in order to really maximize the return on investment.

How does this technology potentially play in when it comes to interim analyses?

Interim looks are a fundamental lever to pull as an adaptive design element when you are thinking about a clinical study. One way to adapt is to take a look at a certain point during the execution of the study and make a decision, whether it is to increase the sample size or go towards a certain dose arm or extend the duration of a study. Lots of decisions can be made based on interim analyses. One of the best practices in simulation-guided trial design is to evaluate the effect of an interim analysis at different points in trial execution, say at 25, 50 or 75 percent of enrollment. We can help inform what kinds of decisions a team would be empowered to make at any one of those junctures, so that as the trial executes in real life, the team will be prepared to take action based on which scenario plays out.

What are some more sophisticated ways to apply these tools?

We have a digital development platform called Solara that brings the power of cloud computing to this concept of simulation-guided trial design. As an analogy, a lot of people drive around these days using a GPS app on their phone. If I wanted to go from Waltham to Logan Airport, what is the best route that I can take? Are there alternative routes that might be faster or slower or have more or less traffic? The apps provide simulation guided routing. With Solara, we apply similar principles to help teams visualize different paths to achieve their goals. We take data from numerous simulations (many more than a single statistician could execute) and chart paths from the beginning to the end of the study to show sponsors their options. Instead of overlaying real-time traffic data on the street maps of Boston, we are taking a particular sponsor’s trial design and simulating the scenarios they are worried about with the data they bring to us.

"We live in an era where science and technology can be a powerful enabler or a distracting element. Statistical simulations are a great example of that duality."

What are the biggest mistakes CMOs make when it comes to designing trials?

This may not be a mistake, per se, but one of the biggest temptations is to be tunnel-visioned. People make decisions about designing their development plan without understanding the full scope of possibilities for themselves, typically prioritizing time and cost. Spending extra time planning and making choices to ensure clear data sets and optionality might be enormously valuable.

What are the biggest mistakes made in interpreting the results of a simulation?

There is always a temptation to either over or under interpret. People inevitably bring personal biases to interpreting the data from simulations. It helps to have an objective third party evaluate your design and pressure test it to show you the pros and cons. Everyone’s assets are their babies and nobody wants to say their baby is ugly. We try to objectively assess whether an experiment is going to satisfy stated priorities or whether there are other, better possibilities. Having an external perspective helps guide that process can be extremely valuable and a competitive advantage.

How can CMOs discuss investing in simulation-guided trial design with their less scientifically inclined colleagues?

Fundamentally, most people can align around the concept of spending money wisely. The biotech funding environment right now is quite demanding. This unfriendly climate underscores the value of careful planning for clinical development plans so that sponsors can maximize return on investment and also the valuable time patients volunteer for their trials. If you are worried about scenarios A, B and C, we can help you visualize them ahead of time so you can be prepared and really understand the worst case scenario. As an analogy, it’s very tricky to invest in the stock market as an individual day trader but if you have a professional team to help you understand strategy and economic trends, you can be a much more intelligent investor. Simulation guided trial design helps sponsors better understand their risks, benefits and alternatives.

Anything else?

We live in an era where science and technology can be a powerful enabler or a distracting element. Statistical simulations are a great example of that duality. Modern simulations provide data that can be quite distracting because of technical details and sheer volume; but if harnessed by the right platform and insightful visualizations, that data can help inform clear decisions about strategy and clinical trial design and accelerate how our industry brings new medicines to patients.