New Perspective on Nevins and Potti
Eighteen months after Perez bows out, in October 2009, MD Anderson statisticians Baggerly and Coombes publish a paper claiming there is potential for patient harm in the Duke trials.
Nevins and Potti have to defend themselves from the unusual continuing examination. After publication, Duke suspends the trials.
After a paper by Baggerly and Coombes claims that the Duke trials are potentially putting patients at risk, Nevins and Potti agreed to speak with this reporter.
The pair mounted an energetic defense of their work. With Potti on the conference call, Nevins says that a paper previously published in Lancet Oncology in December 2007 constitutes a blinded validation of the Duke group’s methodology.
Says Nevins:
“Data was made available to us, blinded. All we got was the gene expression data. “We ran the predictions and sent it back to the EORTC investigators, including the statisticians in the EORTC group. They took the results, analyzed it in the context of the clinical responses in that study, and did further analyses with respect to evaluating developing combined probability measures.” (The Cancer Letter, Oct. 2, 2009).
Nevins and Potti made similar claims in published correspondence with Nature Medicine.
In reality, the Lancet Oncology paper—which was ultimately retracted—didn’t contain the words “blinded validation.”
In a matter of days, the European researchers register disagreement with this statement. The validation wasn’t blinded, they said. They provided the data in question (The Cancer Letter, Oct. 23, 2009)
Indeed, the dataset Potti used to “validate” the predictor model contained disease characteristics and it contained outcomes. As an author, Nevins would have been ordinarily expected to know this.
This occurred 18 months after the Perez memo—after Nevins sought to convince Perez not to send his letter and pledged that he would review the data and validity of predictors.
An audio recording of the conversation with Nevins and Potti is posted here.
During Duke’s internal investigation of concerns raised in the Baggerly and Coombes, Nevins had the authority to decide which criticisms and documents should be shared with the two external reviewers.
Allowing Nevins to play a role in handling controversies coming from the genomics operations he directed is consistent with the maneuvers that resulted in the silencing of Brad Perez.
The decision to give Nevins this power was made at the highest levels of Duke Medicine.
The three trials were restarted in January 2010, following a brief internal investigation.
The three trials were stopped again—this time for good—in July 2010, only because of the publication of Potti’s enhancement of his credentials.
A story about the controversy aired on the CBS program 60 Minutes.
More than anything, 60 Minutes loves extracting an on-camera mea culpa, and Nevins gave a good one:
“I regret that some of the issues that were raised along the way I didn’t recognize earlier, and that this could have been brought to a halt at an earlier time,” Nevins says. “I felt that I had addressed the issues that had been raised.”
Nevins says that it was well after The Cancer Letter reported Potti’s misrepresentation of his credentials that he started to lose confidence in his colleague. When he did look at the data—which would have had to happen after late 2010—he found manipulation.
“It became clear that there was no explanation other than there was a manipulation of the data, a manipulation of somebody’s credentials and manipulation of a lot of people’s trust,” Nevins says to 60 Minutes four full years after the Perez incident. “It simply couldn’t be random, it simply couldn’t be inadvertent. It had to have been based on the desire to make something work.”
The story does not mention the Perez incident, which is being reported here for the first time.
Limits on IOM Investigation
The Perez story didn’t come up in the IOM review of the case, either.
There was no interest in starting a police-style investigation. This is understandable, perhaps, because the committee had no lawyers and no investigators, and wasn’t ideally positioned to deal with issues of scientific misconduct.
NCI Director Harold Varmus, for example, said he was interested in high-level lessons that can be learned from the case, and the request for the IOM examination came from both NCI and Duke.
In July 2011, in a Q&A with The Cancer Letter, this reporter asked Varmus to explain how high-level lessons can be learned without a shoe-leather investigation: “As a former police beat reporter, I’m wondering to what extent it’s futile to try to derive a high-level lesson from something that’s pretty low level.”
“I think you are seeing it the wrong way,” Varmus replied. “There is a falsified CV. That’s of no interest to me. That’s someone else’s problem… It was crucial to the case only because it helped people pay more attention to the underlying issue.” (The Cancer Letter, July 22, 2011).
At the first meeting of the IOM committee Dec. 20, 2010, Gilbert Omenn, the panel chairman and director of the University of Michigan Center for Computational Medicine and Biology, outlined the focus of the investigation.
“A committee of the Institute of Medicine will refrain from launching a police-style investigation of the Duke scandal,” he said. “We are not an investigative body.”
“I think we are heading into a morass—to try to figure out what really happened at Duke and who should bear responsibility and who should be held accountable.” (The Cancer Letter Jan. 7, 2011).
As the committee delved deeper into the problems of omics, Omenn and his committee members recognized that the problems at Duke needed to be examined. In the end, the Duke sections added up to 100 pages of the 338-page report.
Though the Perez incident would have been of interest to the committee, it was never mentioned.
Robert Califf, Duke’s vice chancellor of clinical and translational research, director of the Duke Translational Medicine Institute and professor of medicine in the Division of Cardiology, is quoted saying that none of Potti’s co-authors had expressed misgivings about his science.
In the report, Califf says that Duke had surveyed 162 investigators involved in 40 papers coauthored by Potti.
“Two-thirds of these papers, he testified, will be partially or fully retracted, with others pending evaluation,” the report says. “Yet in no instance did anyone make any inquiries or call for retractions until contacted by Duke. This experience suggests the need for co-authors to have more shared responsibility for the integrity of the published research.”
Califf doesn’t figure in any of correspondence related to the Perez case and he had no oversight authority over the trials of the Nevins and Potti technology.
Califf’s statement is technically correct. His survey of Potti’s co-authors wouldn’t have included Perez, who had taken his name off the Potti papers.
Statements by Kornbluth, who was recently named Duke provost, cannot be reconciled with emails obtained by The Cancer Letter.
According to the IOM report, “there was discontinuity in the statistical team, which may have contributed to the research team’s failure to follow proper data management practices (Kornbluth and Dzau, 2011). Junior investigators on the team either did not recognize what was wrong or did not feel comfortable expressing their concerns even though whistle-blowing systems were in place. Some members of the laboratory did ultimately come forward with concerns about the research, but only after the University began an investigation (Kornbluth, 2011).”
Elsewhere in the report, Duke officials are quoted describing the university’s “just culture,” which encourages anyone at any level to criticize the scientific methods of a study without fear.
The report continues:
“However, the problems with the three clinical trials were not brought to the attention of the appropriate individuals within the university leadership through any of these whistleblowing channels. According to Vice Dean for Research Sally Kornbluth, a number of people came forward after the university undertook its investigation and said they ‘were glad [the university was] reviewing things carefully’ (Kornbluth, 2011).
“Why no one came forward earlier, or perhaps any such concern was not forwarded appropriately, is not known, but the fact that these problems were not brought forward earlier may be an indication of the discomfort or lack of confidence that faculty and staff may have with these systems.”
The report was vetted by Duke officials, which presumably means that they reviewed it and didn’t see reasons to correct it.
Did Kornbluth know about the Perez case? Did Victor Dzau, who was then Duke Chancellor?
The answers are yes and yes.
Kornbluth’s name figures in the 2008 exchange of emails in preparation for a meeting about “Anil Potti” at the time Perez was circulating communicating his research concerns to HHMI. It is not publicly known whether the meeting actually took place.
Another exchange of emails obtained by The Cancer Letter, shows that Kornbluth was aware of the Perez controversy on Oct. 5, 2010, three months before the IOM committee held its first meeting and six months before the committee first met publicly with Duke officials.
At that time, Duke’s top administrators were deciding the best way to handle the Perez incident in the context of the scientific misconduct investigation. Should the Perez documents be presented to an internal Duke committee that was deciding on the scope of the misconduct investigation.
At first, Kornbluth decides that charges would be appropriate. Then she changes her mind, choosing to present the Perez materials to the standing committee, leaving it up to the group whether charges are justified.
The email is addressed to Dzau, who has since been named IOM president:
“Victor,
“My two cents: I’ve had a change on heart about this. I’ve talked to Wesley [Byerly, associate dean for research support services] at length and I think his thoughts to let the Perez stuff go in with the existing allegations (and not draft another charge) is right. I think Joe [Nevins] is going to the committee to debrief and I think the committee can then decide if they really think there is any merit in charging Joe with anything. I am feeling more and more that we may have jumped the gun with that and the answer is probably ‘no.’ Happy to discuss if you want. Sally.”
It’s not publicly known what was actually done. More than four years after its launch, the Duke misconduct investigation remains a work in progress.
It’s not a crime to give deceptive testimony to IOM. “They are a private club,” said NYU’s Caplan, who is an IOM member. “You can lie to them all you want. It’s like lying to The Cancer Letter. It’s probably bad form, but you are not going to go to jail.”
