Multi-cancer detection tests evoke conflicting reactions—the excitement at their promise is quickly dampened by concerns over the uncertainty of their clinical benefit, very low sensitivity for detecting stage 1 cancers, and the risks that come from subsequent workups.
Also, the tests, known under the acronym MCD and sometimes called MCEDs, can be costly. GRAIL’s Galleri, a leading test, has a list price of $949.
But topping the list is the concern that there is no evidence that, when used in clinical practice, these tests affect the only reliable disease outcome for assessing a cancer screening tool—cancer mortality. Had there been such evidence, at least some of the heartburn over the widespread use of MCDs would go away.
Alas, mortality data require big studies and careful follow-up, and the FDA office tasked with approval of MCDs, the Office of Health Technology 7: Office of In Vitro Diagnostics, does not bother itself over whether the tests reduce cancer death rates—or, for that matter, any metric of clinical utility—when it makes regulatory decisions.
The meeting of the meeting of the Medical Devices Advisory Committee Nov. 29 was part of the FDA process of deciding on metrics that would be approprate for deciding on approval of MCDs.
In 2024, NCI’s Division of Cancer Prevention will begin enrolling 24,000 healthy people aged 45-70 into a Vanguard study on Multi-Cancer Detection that will have mortality as the primary endpoint. The study will simultaneously test three MCD interventions against a common control group and lay the groundwork for a later, larger study.
Half an hour into an all-day Molecular and Clinical Genetics Panel meeting, Timothy Stenzel, director of FDA’s Office of In Vitro Diagnostics, stepped in to clarify the standards by which the Office of In Vitro Diagnostics makes decisions on submissions.
“For this type of submission, a standalone [in vitro diagnostic] test not linked to other things like therapy, we, under law, can look at analytical validity and clinical validity,” Stenzel said. “When you get into mortality, you get into clinical utility, which is not [one of] our tools to use for this type of submission. Whereas I do realize the overall goal is to reduce mortality, for this type of submission, to our Office of In Vitro Diagnostics, we do not assess mortality as an endpoint.”
It seems that any FDA decision will not resolve the tangle of questions posed by MCDs: Will MCDs benefit or harm public health?
“This is a really tricky set of issues, and it feels like we’re navigating between Scylla and Charybdis. On the one hand, we all understand the validity of mortality as an endpoint, and how critical that is, and that this is a public health intervention, and we don’t want to wreak real harm on the public,” Deborah Schrag, the chair of the the Department of Medicine and the George Bosl Chair at Memorial Sloan Kettering Cancer Center, said during the meeting.
“We heard very powerfully from some patients about the urgency and eagerness to accelerate development of this technology. But at the same time, we also understand the population-wide risks and chaos and havoc that could ensue if we are excessively trigger happy. It’s trying to find that right balance that’s so terribly tricky,” said Schrag.
FDA asked the panel in a full-day meeting Nov. 29 a cluster of questions within three core topics:
- Clinical study design considerations for FDA submissions, including evaluation of cancer-specific performance,
- Use of Tissue of Origin assays to help identify tumor location versus other methods, patient work up considerations following positive results, and follow-up for patients with negative results,
- Benefit/risk considerations, including postmarket study considerations.
“The population is not that savvy about risk.”
For most diagnostic tests, FDA’s standards of clinical validity are comprehensive and appropriate.
From infectious diseases to heart attacks, clinical benefit, or clinical utility, naturally follows analytical and clinical validity of the diagnostic test.
If a test successfully diagnoses HIV, for example, it has clear clinical utility—the patient can then be treated for HIV.
Cancer screening plays by a different set of rules. Clinical utility does not necessarily follow from clinical validity (the ability to detect the disease), both because the clinical benefit of detection is so uncertain, as is the question of whether the harms to those without cancer are outweighed by those benefits.
The field has been taught this lesson before. The same rules do not apply to all cancers—there is uncertain benefit from prostate cancer screening, no evidence of benefit from ovarian cancer screening, and large benefits from colorectal and lung cancer screening.
“Nobody would think about doing this with a drug in a healthy population, like, ‘Here’s a drug, come back in 10 years and tell us whether this worked for you,’” Philip Castle, director of the Division of Cancer Prevention and senior investigator of the Division of Cancer Epidemiology and Genetics at NCI, said during the meeting. “We would never do that. So, why are we even considering such a thing for a screening test? Obviously, the screening test is an in vitro diagnostic, but the follow-up care is invasive, and really has to be considered in that context.”
Taking this argument to its logical conclusion, the FDA Office of In Vitro Diagnostics is asking the wrong questions.
The office does perform a benefit-risk analysis, conducted by its medical officers, in addition to considering validity measures that include accuracy, positive predictive value—the proportion of subjects with a positive test result who truly have the outcome of interest—sensitivity, and specificity, FDA’s Stenzel said.
“Benefit, in this type of submission, would be an accurate test,” Stenzel said. “What is the sensitivity? Is the sensitivity a benefit? What is the specificity? And, of course, specificity is related to positive predictive value. The higher the specificity, the higher the positive predictive value. Understanding what the positive predictive value is, you could have, potentially, an extremely high positive predictive value. Then, you wouldn’t have a lot of potentially off-target risk—people who don’t have cancer and are going for invasive procedures or radiological imaging that may have certain exposures.
“Or you may have a very low positive predictive value, and for each cancer detected, there might be an extreme amount of potential risk to patients who don’t have cancer to be able to find those cancers. So, those are the kind of things that we look at in the benefit-risk calculation.”
Still, an approval based on this level of risk-benefit analysis, rather than clinical measures of impact on disease mortality, fails to give sufficient guidance to the public, NCI’s Castle said.
“You’re talking about the positive predictive value for cancers, but how do you know that those cancers are relevant, and clinically relevant? If you detect a lot of indolent cancers, then you would have a great positive predictive value and no population benefit,” Castle said. “You’re asking the public, to some degree, to understand the difference between the detection of cancer and mortality benefit. And as we’ve learned from COVID, the population is not that savvy about risk.”
Most members of the panel agreed that mortality is the ideal endpoint, while industry representatives seemed to argue that it was too hard to assess.
“A mortality study will require hundreds of thousands of participants followed for over a decade to read out and is recommended by experts,” Josh Offman, president of GRAIL, said during the public hearing. The statement appeared to differ from GRAIL’s response to a New York Times article about the company.
“Conservative modeling…[shows] that adding an MCD test to recommended cancer screening care could avert approximately 1 in 4 lives expected to be lost to cancer within five years of diagnosis,” the company wrote.
Nathan Winslow, global head of global regulatory affairs for Roche Diagnostics, also raised a caution on the mortality endpoint.
“Mortality, I think, is [an endpoint] where there would be some concerns from the part of industry in terms of the feasibility of those types of studies—the size, the longitudinal [nature], the duration, and the impact that that could obviously have on some of the innovation that we’ve been talking about,” Winslow said during the meeting. “I would suggest that there’s a holistic way to look at benefit-risk, and I think that’s ultimately what we’re trying to do here. I’ve heard healthcare equity mentioned, and the fact that this can bring more to underserved populations, for example, increased compliance with existing screening methods, because talked about this being complimentary.
“So, I’ll just offer, when we look at the various ways to see the benefit-risk of these tests, that we don’t simply look at the gold standard for what could be the clinical endpoint. There are other ways to get at that, and also, consider: That could also be delaying something that could be very beneficial to broad parts of the population. If we do have studies that go many, many years, we are delaying the introduction of the technology. We would want it to have a positive benefit-risk, but something that we could feasibly prove.”
Moreover, MCDs are in a phase of explosive technological advancement, increasing both the potential benefit and the difficulties of a trial that satisfactorily answers all relevant questions.
“We have to acknowledge that technology is going to evolve very, very rapidly on this,” Peter Carroll, panel member and professor of urology at the University of California San Diego, said during the meeting. “There’s a lot of AI in these algorithms, and I think we have to be open to that. I would ask the companies how often they’ll be updating their tests based on information they get going forward. I don’t think the tests they have now will be the tests we’ll see in three to five years.”
Surrogates for cancer mortality mentioned at the Nov. 29 meeting include cancer diagnoses leading to treatment with curative intent and stage at diagnosis.
“This is something where we have to do it right on the front end,” Stanley Lipkowitz, senior investigator at the Women’s Malignancies Branch at NCI, said during the meeting. “So, I do think that that’s important. And I do think, as a clinician, that the end of all of this is not stage migration. It’s not: Did a patient get surgery? It’s: Did we improve outcomes? And I think that’s the big picture, and I know that’s not what you guys in this venue are going to regulate, but I think that’s an important thing to keep the eye on, that the trials ultimately have to—even if they’re using a surrogate—be powered to look at those other endpoints.
“Because it’s going to be hard to repeat these once the tests are out there. It’s going to be hard to pull them back.”
“We’re not accomplishing anything.”
A host of potential biases crop up when mortality is not the endpoint of a cancer screening trial. Survival can be artificially elongated by screening—a phenomenon called lead time bias. If a cancer is caught earlier, that doesn’t necessarily mean the arc of disease has been affected.
Even if a patient dies at the same time that they would have if they didn’t receive screening, their survival will appear longer if a test detects them before the patient has symptoms.
Similarly, stage at diagnosis does not necessarily translate to a patient having a different course of disease, because cancer researchers still don’t know which cancers benefit from a true survival improvement as a result of early detection.
A key ovarian cancer screening study showed this starkly—screening reduced how many women were first diagnosed with advanced cancer, but it didn’t reduce the rate of death from the disease.
“In this case, [the potential benefit is] early detection, but early detection of what? It has to be early detection of clinically relevant disease, and that clinically relevant disease has to be linked to mortality. And the problem, with all due respect, is that for many of these cancers—and again, bringing back the example of ovarian cancer—we don’t know what that is,” Castle said.
Daniel Swerdlow, professor in the Department of Radiology at MedStar Health, offered an example from thyroid oncology.
“We really don’t have any way of predicting which [cancers] are indolent and which ones are not. But the very nature of doing screening means that we’re going to find indolent tumors,” Swerdlow said. “It’s really a solution in search of a problem. We’re finding a lot of thyroid cancer out there nowadays. We’re doing an awful lot of thyroid cancer surgery, and the death rate from thyroid cancer hasn’t changed since 1980, so, we’re really accomplishing nothing.
“The South Koreans wrote an article that was in The New England Journal a few years ago, where the government started paying internists to do ultrasound screening in their offices, and the thyroid cancer rate exploded. It’s the number one cancer in South Korea. The complication rate from the thyroidectomies was large, it was 8% or 9%, and I don’t mean hypothyroidism, I mean vocal cord paralysis and things like that. We’re not accomplishing anything. And we essentially have a de facto thyroid cancer screening program.”
The introduction of prostate-antigen screening, or PSA, resulted in widespread overdiagnosis and overtreatment of indolent prostate cancers (The Cancer Letter, Sept. 25, 2009).
“I do think that mortality has to be the endpoint, or we’re going to be in a situation that’s going to be prostate cancer, magnified,” Mayo Clinic’s Ballman said. “I mean, we’re still trying to figure out today who has indolent cancer and who doesn’t.”
MCDs are distinct from single-cancer liquid biopsy tests, DELFI Diagnostics, a biotech offering single-cancer liquid biopsy tests for use in people eligible for lung cancer screening, wrote in a comment on the meeting.
“We propose that the agency delineate between broad multi-cancer detection in an unselected population and focused single cancer detection within populations for whom specific screening is recommended but not being received,” Peter Bach, chief medical officer of DELFI Diagnostics, wrote in a comment submitted to FDA. “We consider these to be distinct approaches that may someday prove to be complementary.”
Assessing the benefit of an MCD poses more complexity than a single-cancer biopsy.
“I like the idea of mortality being an endpoint, but it was phrased as cancer-specific mortality. Then I get a little confused, when you have a test that maybe that turns positive and it’s one of these 10 cancers, and maybe cancer A is at 80% likelihood and cancer B is 60% and then cancer J is 1%,” Swerdlow said. “You start the workup and at some point you decide that the likelihood of the low probability cancer isn’t worth the risk, cost, etc., of making the diagnosis.
“So, which cancer is it that is going to be the mortality endpoint then? Is it all of them? I think that’s where we need to be. If you get a cancer and die of it after having a screening test be positive or negative, I think that has to factor in, because that’ll also point up well, these are the ones that a screening test misses. So, I would kind of advocate all cancer screening deaths as opposed to a specific one that you would have to pick as the most likely.”
“We shouldn’t be doing screening tests if we can’t then work them up. In fact, that’s an ethical violation.”
MCDs—and single-cancer blood tests—promise to catch more cancers, particularly in underserved communities, due to their ease and non-invasiveness.
“If there are MCD tests that engage large members of a population that is currently unscreened, or folks who are left behind—we know that current lung cancer screening, colorectal cancer screening leaves a tremendous number of people behind, and colonoscopy and mammography have kind of a last mile problem where we still have 10% to 20% of people who are recalcitrant and appropriate, but remain unscreened,” MSK’s Schrag said.
“If these tests are more palatable to the public, that has to be considered as an advantage of engaging people in participation in screening for cancer,” Schrag said.
Annual low-dose computed tomography screening, the only recommended screening test for lung cancer in former and current heavy smokers, reduced overall deaths by 7% and lung cancer deaths by 20%, according to the National Lung Screening Trial. Still, lung cancer screening rates are below 6% among those eligible. MCDs and single-cancer liquid biopsies offer a potential solution (The Cancer Letter, Nov. 17, 2023).
“The question is if [MCDs] increase the detection of lung cancer—hopefully for mortality benefit, I’m not going to harp on that—even if it’s less sensitive, because people are more willing to do it,” Castle said. “There may be crossover, if you will. It may not be as efficacious, but it may be more effective.
Still, increased MCD uptake alone would not help healthcare disparities.
“A lot of the tests are saying we’re going to get to those underserved populations,” Karla Ballman, consultant at the Division of Clinical Trials & Biostatistics, professor of biostatistics at Mayo Clinic College of Medicine and Science, associate director of quantitative health sciences at Mayo Clinic Comprehensive Cancer Center, and director of MCCCC Biostatistics Shared Resource, said during the meeting. “My concern is that yes, they can get the test, but will they do the necessary follow-up in the case of a positive test?”
Swerdlow recalled the “CT colonography wars” from the last decade.
Said Swerdlow:
Essentially a valid test that was fairly reasonable for cost wound up not getting approved by the U.S. Preventive Services task force when there’s a very large segment of the population unscreened for colon cancer, enlighten me as to how some of this works.
The potential for these screening tests are fabulous, but so are the costs. I just don’t know how we’re going to do all the imaging that will be engendered by these positive tests. If this comes, it’s going to break the bank, and there simply aren’t enough scanners around to do the job in many areas, especially [underserved] populations… Native Americans, rural [populations], are going to have a really hard time doing their follow ups. And then what’s the benefit of getting the blood test if we can’t do anything about it?
Having survived [the CT colonography wars], I just have a hard time imagining that the Preventive Services Task Force, with the costs involved, is going to recommend this stuff, because something else is going to have to give out of the pot that pays for Medicare and Medicaid, and I don’t know what that’s going to be. I’m very concerned about that. It doesn’t really matter how cheap the blood test is because that’s not the problem. I think somebody’s going to have to figure out a way around that and I don’t see an easy answer.
MCDs, even with their promises of bridging healthcare gaps, could in fact have potentially negative impact on underserved communities, Schrag said.
“We shouldn’t be doing screening tests if we can’t then work them up. In fact, that’s an ethical violation,” Schrag said. “And a big risk is crowd out. What I mean by crowd out is if you’re busy chasing pulmonary nodules or these tests—I mean, we’ve seen this movie before with lung cancer screening, and we’ve learned a lot from those experiences.
“We have patients with new diagnoses of lung cancer who are waiting for imaging and to get in to be seen. We’ve got incredible workforce pressure to add on patients at eight, nine o’clock—particularly in underserved areas. I work in a non-physician shortage area, but it’s terrible—in rural parts of the country and in underserved areas, people who really already have cancer can’t get in to be seen.
It will be a challenge to balance the necessity of working up patients with positive MCD results with the existing demands on the diagnostic testing sector, Schrag said.
“We have to make sure that people who are identified as a result of this screening are seen within a reasonable timeframe, but don’t crowd out those who are symptomatic and need to be seen urgently,” Schrag said. “That’s all about balance, but it’s tricky.”
In MCD clinical trials, the companies developing these tests should be prepared to financially support the inclusion of people from underrepresented groups, Edward Bujold, a family medicine physician with the Granite Falls Family Medical Care Center, said during the meeting.
“The companies that are developing these MCD tests, if they’re going to do this right and include people of color, the elderly, children, those that can’t afford to test—I don’t know if they realize how much money they’re going to have to spend in clinical trials and things, and I hope they’re up to the task,” Bujold said.
