This story is part of The Cancer Letter’s ongoing coverage of COVID-19’s impact on oncology. A full list of our coverage, as well as the latest meeting cancellations, is available here.
After a series of global epidemics, Don Berry has spent the past several years preparing for a serious pandemic that would be caused by yet another viral pathogen.
“People have long said that we’re not prepared for the next pandemic. We get a pandemic and then it goes away,” Berry, a professor in the Department of Biostatistics and founding chair of that department at MD Anderson Cancer Center, said to The Cancer Letter. “And so everybody says, ‘Well, okay, the next pandemic is way in the future, and so let’s not worry about it.’”
Well, SARS-CoV-2 is here—on the heels of Ebola, MERS, the H1N1 influenza pandemic in 2009, and SARS-CoV in 2003.
“Designing clinical trials doesn’t have high priority when there’s no pandemic. And then, when there’s a pandemic, there’s panic,” said Berry, who is also senior statistical scientist and founder of Berry Consultants, a company that is playing a key role in providing statistical guidance for multiple COVID-19 trials. “But 10 or so years ago, global infectious disease researchers decided to take planning seriously.
“In 2014, Berry Consultants worked with a European group called PREPARE. It’s an acronym that means essentially get ready for the next pandemic,” Berry said. “They focused on community-acquired pneumonia, recognizing that these pandemics are mostly associated with respiratory problems. So, they set up something called REMAP-CAP.”
REMAP-CAP stands for a Randomized, Embedded, Multifactorial, Adaptive Platform trial focused on Community-Acquired Pneunomia. The trial has a network of over 60 participating sites across 13 countries.
To respond to COVID-19, the trial has implemented an appendix to the core protocol to allow the platform to respond rapidly to the ongoing pandemic.
“On March 11, 2020, when the WHO declared a pandemic for COVID-19, the first patient accrued in REMAP-COVID,” Berry said. “[REMAP-COVID] started out considering 72 different combinations of therapies and sequences of therapies.
“It’s running, it’s randomizing, less so in Australia and New Zealand, because they don’t have many cases. But, of course, except the United States, Europe is leading the way in numbers of cases and deaths.”
Berry spoke with Matthew Ong, associate editor of The Cancer Letter.
Matthew Ong: NIH’s new treatment guidelines for COVID-19, including for investigational agents, notably recommend against a hydroxychloroquine combination and HIV protease inhibitors outside the context of a clinical trial. What’s your take on their assessment of the evidence?
Don Berry: The NIH’s recommendation is standard from anybody who is in the establishment and understands clinical research. And the business with HCQ is just horrible. It probably is ineffective, and worse, with cardiovascular issues.
But on the basis of, essentially, a whim, doctors have been hoarding it. And so, clinical trials have had problems trying to get the drug, because everybody is using it for COVID-19 and hoarding it. And patients who have major problems and lean on the drug for help, can’t get it.
It’s a horror and it’s based on a rumor and innuendo. There’s a trial, but I hate to even call it a trial. The French trial was 15 patients, and okay, they seemed to do okay. But patients by and large, do okay.
So, you really need to have a control, not necessarily randomized, but you’ve got to have some sort of thing to base your conclusion that it’s effective on.
What about remdesivir? The NIH guidelines state that there are insufficient clinical data to recommend either for or against.
DB: Yes, I would agree with that. And again, the stuff that we heard about remdesivir from University of Chicago is whim and innuendo and anecdotes.
That, of course, was a travesty—as a measure of impact, Gilead’s stock price increased 14% at one point on the basis of this story that came out of the STAT publication.
About that, I’ll have to circle back. What have you been working on as a scientist and statistician in this pandemic?
DB: A number of things, through Berry Consultants. We designed the REMAP-CAP trial for the International Consortium, not including the United States, but many European countries plus Australia, New Zealand, Canada. Berry Consultants are working on a dozen or so COVID trials. I’m not personally working on many of them.
Most researchers want platform trials, which is what we’re sort of known for, including I-SPY 2, and GBM AGILE, and Precision Promise in cancer.
Just a bit of history: people have long said that we’re not prepared for the next pandemic. We get a pandemic and then it goes away. And so everybody says, “Well, okay, the next pandemic is way in the future, and so let’s not worry about it.”
Designing clinical trials doesn’t have high priority when there’s no pandemic. And then, when there’s a pandemic, there’s panic.
But 10 or so years ago, global infectious disease researchers decided to take planning seriously. This was shortly after the H1N1 pandemic.
There was a meeting in Toronto that was attended by researchers from around the world. I gave a presentation about platform trials, including I-SPY 2. The conferees decided that was what the kind of trial they wanted to build.
In 2014, Berry Consultants worked with a European group called PREPARE. It’s an acronym that means essentially get ready for the next pandemic.
So, we designed a trial for them—mainly Scott Berry, my son, who’s the president of Berry Consultants, and other Berry consultants—more than just a trial for preparing for the pandemic, but to have a functioning trial in place when a pandemic arrives.
They focused on community-acquired pneumonia, recognizing that these pandemics are mostly associated with respiratory problems.
So, they set up something called REMAP-CAP, an acronym. The CAP is community-acquired pneumonia; the REMAP has something to do with preparation, and adaptive and platform trials. The protocol had an appendix that is specific to a future pandemic.
REMAP-CAP has been running since 2015. On March 11, 2020, when the WHO declared a pandemic for COVID-19, the first patient accrued in REMAP-COVID. The investigators had amended the appendix in January and February to apply specifically to COVID-19. And it started out considering 72 different combinations of therapies and sequences of therapies.
It’s running, it’s randomizing, less so in Australia and New Zealand, because they don’t have many cases. But, of course, except the United States, Europe is leading the way in numbers of cases and deaths.
Many groups have contacted Berry Consultants wanting to do something like a platform trial. Most of them are talking about treatment, but I got one this morning that’s considering vaccines and having a platform trial for vaccines.
We’ve also been analyzing data for people, and I hope that the latest analysis will come out in major medical publications very soon.
As well as for investigational agents?
DB: And investigational agents, yes. Everything really, but not randomized. And so, how do you do it? Data scientists and real-world evidence researchers tend to be unsophisticated. I saw something that was announced recently about HCQ. I thought the study was fundamentally flawed.
When dealing with RWE, it is essential to address the physicians’ biases regarding who gets one treatment and who gets another. And there are biases in both directions. Sometimes the better-prognosis patients are given these investigational agents. Sometimes it’s the worst patients who are given the agents.
So, you have to figure that out to the extent possible and adjust your analyses accordingly. People do multivariate analyses with no apparent understanding that it generally gives a nonsense answer. Real-world evidence is the cat’s meow, but the cat also bites.
And that means—
DB: Which means, to mix metaphors, there are landmines all along the way, or there are cats jumping out from behind the corners and biting. To get the cat to meow, you have to know what you’re doing. You have to know how to handle the cat.
Some news organizations, as you mentioned, have been publishing preliminary trial data for remdesivir and interpreting the results as positive. Is that okay?
DB: This is a big, big problem in medical research. What do you tell people when. The clinical trial system has been clear.
If it’s a phase III trial, you don’t tell anything. You don’t tell the investigators. You blind them to the trial’s results.
The remdesivir study with the anecdotes coming out, that was not a blinded trial. It was open-label. So, clinicians knew what drugs they were giving and saw what the results were. They talked out of turn and it leaked out.
Is it okay that it leaks out? That’s a very, very difficult ethical, social, and unfortunately, political question. There have been issues with some organizations arguing that everybody should be told everything that’s happening in a trial.
That would kill medical research as we know it. In a similar vein, some have argued that patients should have access to experimental drugs as soon as they become available, even before phase I trials. And then misinterpreting the real-world evidence in the way that I just described.
There has to be some kind of compromise here. We all believe in transparency. The clinical trials structure is not transparent. But for good reason. When I say good reason, it doesn’t mean that I buy into conventional clinical trials hook, line, and sinker.
But we have to ensure that whatever changes we make in clinical research preserves the scientific method. There has to be some sort of compromise, but I don’t know what it is.
I do know that the issues of transparency, trial sample size, and so on, differ depending on the rareness of the disease or condition. It also depends on the severity of the disease or condition.
To announce trial results prematurely can have grave consequences, including affecting the trial’s integrity. In the case of remdesivir, leaking some of the results was inappropriate. The investigators are being, and should be, criticized for announcing their narrow experiences in the trial.
Because it was too early in the process to do so?
DB: Yes. I’m tied to the conventional wisdom in this regard, because of examples like this.
But it is something that we should keep talking about. And as we’re going along, we have to figure out something that can be a way to let people know what the data are showing when it’s moving in a direction that could affect patients in important ways.
We don’t want to hide something important just to hide it. We don’t want to sit on data that are compelling. But the word compelling is obviously subjective, depending on who is looking at the data.
I know “compelling” when I see it, but giving an objective definition is such a difficult problem that I don’t know how to do it.
One thing that the answer is most assuredly not is statistical significance. That’s a concept that is essentially irrelevant for decision-making.
Not too long ago, we were bombarded by headlines about the “promise” of remdesivir in Ebola, only to see it fall behind monoclonal antibodies in phase III trials. How would you characterize the evidence on remdesivir to date for COVID-19?
DB: I have a probability distribution on the benefits of remdesivir. It’s not a cure, so I associate no probability with that.
The NIH has an ordinal scale, seven or eight categories where death is the worst one. My own attitude is to focus on death. It’s the easiest endpoint to address and it’s the easiest to understand.
The probability distribution is on benefit at least for one level of the disease, whether it’s in the ICU or in the severe stage where the patients are hospitalized but not in the ICU, and all the way back to prevention.
Does the drug have a role anywhere? My distribution is very spread out, representing my uncertainty. The mean of my distribution is small but not zero.
Thirty-five percent of my probability is on the positive side. But of course, reflecting my uncertainty, as soon as I see some randomized controlled results, this may change dramatically.
Of note are the remdesivir arm in WHO’s SOLIDARITY trial, the two Gilead studies, and NIAID’S ACTT. What did I miss and which phase III trials for remdesivir should we be paying attention to?
DB: You’ve got everything that I know about.
Most of the remdesivir trials randomize the antiviral vs. placebo or standard of care, but the primary endpoints may differ in order, i.e., death vs. clinical improvement. What’s important for us to consider here?
DB: The NIH has an ordinal scale, seven or eight categories where death is the worst one. My own attitude is to focus on death. It’s the easiest endpoint to address and it’s the easiest to understand.
Mortality is not the only possible benefit of a therapy, of course. A therapy could reduce the rate of mortality, but incapacitate those who live, such as the celebrity patient who survived COVID-19 but had to have his leg amputated. And, too, there are outcomes worse than death.
However, and generally speaking, most therapies that move patients away from mortality also have a beneficial effect on patients not destined to die. So, you’d look for some sort of movement on that ordinal scale, not just death.
But especially in view of the high case-fatality rate for COVID-19, death would be quite a reasonable primary endpoint, with an ordinal scale being supportive.
Some of these trials are designed to be adaptive. For instance, WHO’s and NIAID’s trials leave room for the creation of new arms to test potential new drug combinations. This is a rhetorical question, but can you explain the utility of these designs?
DB: There are many kinds of potential adaptive aspects of clinical trial designs.
REMAP-CAP includes weekly interim analyses, including the possibility of announcing results if a particular therapeutic regimen is good, bad, or ugly. And new therapies are added as they become available.
Another kind of adaptation is adaptive randomization. As therapies begin to show that they are better than other therapies, then they are assigned with higher probability to subsequent patients. It’s something that we use in I-SPY 2, and in the first stages of registration trials Precision Promise, and GBM AGILE.
This obviously benefits patients in the trial. But it also moves better-performing therapies through the trial faster, graduating them from the trial to general use.
All types of adaptations require looking at the data that’s accumulating from the trial. And the more looks the better. These looks and the actions that occur as a result of the looks are planned in advance.
So, an algorithm that dictates what decisions can be made runs the trial. In effect, a robot runs the trial. That means we can address various statistical matters, like the trial’s Type I error (false positive rate).
REMAP-COVID addresses many questions, including combination therapy and sequential therapy. All adaptively.
For me, there is no alternative. Non-adaptive trials are destined for oblivion.
And the interim analyses also allow you to terminate any arm with an experimental agent that isn’t showing meaningful effect.
DB: Right. Actually, that’s an extreme version of adaptive randomization. You’re adaptively randomizing therapies based on their performance.
And suppose one therapy is doing very badly, but the design hasn’t given up on it completely. Perhaps some patients have been treated with it and their results may turn things around.
The algorithm may set its randomization probability to zero; so patients would no longer be assigned to that therapy, at least until more evidence becomes available. Or, it could be zero for patients in intensive care, but the therapy may still have a place in less severe disease.
Trials initiated by academic or public health institutions tend to include multiple arms for multiple investigational agents, or at least remain open to the possibility of adapting. The trials sponsored by Gilead test only remdesivir plus standard of care for different durations, dependent on clinical status. But, of course, it’s their drug.
DB: It’s their drug. And it’s tradition. But I’m trying to change that tradition.
I-SPY 2, for example, has evaluated or is still evaluating 20 therapies in neoadjuvant breast cancer. And from 15 or so different companies. We built Precision Promise in pancreatic cancer and GBM AGILE in glioblastoma in the same way.
These trials consider lots of therapies and they use a variety of adaptations in each therapy’s first stage. Successful therapies move seamlessly into a phase-III stage.
An obvious benefit is the shared control arm. So, even if you don’t do anything clever, you have potentially, up to a 50% reduction in trial cost, because many experimental therapies are sharing the controls.
Adaptive platform trials like REMAP-CAP have become part of the world, especially in Europe. They’ve funded adaptive platform trials beyond REMAP-CAP, including a major initiative in Alzheimer’s disease, called IMI EPAD.
Speaking of accrual and completion, the two remdesivir trials by the Capital Medical University in Beijing have either been ended or halted, because “the epidemic of COVID-19 has been controlled well in China, no eligible patients can be enrolled at present.” Are you seeing similar issues elsewhere?
DB: The nice thing about REMAP-COVID is it is far reaching across the globe.
In particular, and as you know, except for the United States the leading countries in terms of cases and mortality are Spain, Italy, France, the UK, and Germany, although Germany stands apart from the others in terms of mortality. So, accrual is not an issue.
Moreover, when and if COVID-19 moves south, REMAP-COVID has got it covered in Australia and New Zealand. So, the trial is flexible and adaptive, but it’s also encompassing.
When do you expect investigators to present reliable, early analyses of the results from the remdesivir trials? I’m hearing late May, early June latest, but what does that mean for the strength of the evidence by then?
DB: I don’t know what the accrual is. But this time frame is possible. The patient’s experience in COVID-19 is not like cancer. It’s over with quickly, one way or the other.
The primary endpoint may be evaluated at 21 days (or sooner in the case of death). So, provided accrual is fast, there’s no reason the trial results can’t be announced in a matter of a few months.
Right, NIAID’s ACTT has exceeded its initial accrual goals. So, are there any contributions from oncology that you would like to note, in terms of expertise, innovation, for the development of these trials?
DB: Absolutely. The grandmother of adaptive platform trials is I-SPY 2. The grandfather is the BATTLE trial that was designed by Jack Lee and conducted at MD Anderson some years ago.
For me, there is no alternative. Non-adaptive trials are destined for oblivion.
Many modern adaptive platform trials in other diseases are clones of I-SPY 2, including, like I mentioned earlier, Europe’s Innovative Medicines Initiative’s EPAD.
The IMI issued an RFP that said specifically that they wanted an I-SPY 2 in Alzheimer’s. And I’ve already mentioned Precision Promise and GBM AGILE that have blazed pathways in the registration setting.
So, all of these designs grew out of oncology to begin with.
DB: In a real sense, yes.
Finally, do you have any thoughts about the politics of this pandemic?
DB: Only that I hate seeing medicine and science politicized. Politicians seem uniquely ignorant of both. The good politicians are the ones who know they’re ignorant.
