Duke University would have avoided embarrassment, a misconduct investigation and a lawsuit, had its top administrators paid closer attention to a thoughtful report by a medical student who saw problems in the lab of the disgraced scientist Anil Potti.
Documents obtained by The Cancer Letter show that Duke’s deans were warned about Potti’s misconduct in late March and early April 2008, at the time when clinical trials of the now discredited Duke genomic technology were getting started.
The three-page document was penned by Bradford Perez, then a third-year medical student and a Howard Hughes Medical Institute scholar.
Instead of rewarding the student’s brilliance with a plaque and a potted plant, Potti’s collaborator and protector, Joseph Nevins—aided by a phalanx of Duke deans—pressured the young man to refrain from making a final complaint and reporting the matter to HHMI.
The Perez memo and internal emails that are being published here for the first time directly contradict the claims made by Duke officials that they had received no whistleblower reports.
Duke officials said they were blindsided by the events that reached a crescendo in 2010, more than two years after the Perez memo, following The Cancer Letter’s reporting that Potti had misrepresented his credentials, claiming, among other things, to have been a Rhodes Scholar (The Cancer Letter, July 16, 2010).
The medical student’s memo, titled Research Concerns, is a key element in a lawsuit filed on behalf of the patients who were enrolled in the three Duke clinical trials testing the discoveries from the program run by Nevins and Potti. Altogether, 117 patients were enrolled in the trials.
In addition to claiming harm, the patients’ lawsuit alleges that Duke officials engaged in a civil conspiracy. The case is expected to go to trial at the Durham County Superior Court on Jan. 26.
Perez’s Research Concerns memo is published on page 1.
Whatever its legal significance, the memo and the flurry of emails it touched off provide new insight into Duke’s handling of the Potti controversy:
- The memo shows that, by ignoring the content of the Perez memo, Duke’s deans allowed Nevins to investigate his protégé himself.
- Responding to Perez’ memo, Nevins and Potti promised to conduct a review of the data in April 2008. A thorough, unbiased review of this sort would have produced evidence of fraud, statisticians say.
- Emails demonstrate, step-by-step, how Duke officials convinced Perez to present his principled stance as a difference of opinion between him and two senior scientists.
The Duke case triggered an examination of genomic and proteomic sciences. A committee of the Institute of Medicine was asked to recommend procedures for testing proposed -omic interventions before they are taken to the clinic.
Though crucial to understanding what actually happened at Duke, the Perez memo wasn’t among the documents made public as part of the IOM investigation.
When it was convened in December 2010, the IOM committee was instructed by the IOM president at the time, Harvey Fineberg, to seek lessons from in-depth examination of the problems at Duke. No police-style or legal misconduct investigation was to be undertaken, however, since such investigations are the responsibility of the researchers’ institution. The scope of the IOM investigation evolved further to include an examination of the administrative structures governing research at Duke.
Now, internal Duke emails show that top Duke officials were aware of the Perez matter, and that their subsequent claims to the IOM that no whistleblower had come forward in the genomics scandal were false.
Insiders say the Perez memo and supporting materials published here would have been relevant to the efforts of the IOM committee not because of what they say about the behavior of a specific person, but because they provide important insights about the institutional oversight process at Duke.
A detailed timeline of the Duke genomics scandal, which shows how the Perez incident alters what was previously known, appears here.
Perez, who is currently a resident at Duke, said the controversy he triggered had caused him to repeat his third year of medical school.
“In the course of my work in the Potti lab, I discovered what I perceived to be problems in the predictor models that made it difficult for me to continue working in that environment,” he said in an email to The Cancer Letter. “I raised my concerns with my laboratory peers, laboratory supervisors and medical school administrators and left it to them to determine how best to proceed. I chose to take an additional year to complete medical school in order to have a more successful research experience. My decision to stay at Duke was based on it being the best opportunity both personally and professionally.”
Duke officials admitted that mistakes were made, but didn’t respond to specific questions.
“Duke supported Dr. Perez as he raised his concerns during 2008 and thereafter,” Duke Medicine officials said. “He continues to be a successful, valued and respected member of the Duke Medicine community.
“With regard to the scientific controversy: Duke acknowledged years ago there are many aspects of this situation that would have been handled differently had there been more complete information at the time decisions were made.”
Thomas Henson, an attorney who represents the patients who are suing Duke, declined to comment.
“The medical student was very brave,” said Arthur Caplan, director of the Division of Medical Ethics at the NYU Langone Medical Center, who was asked to review the materials cited in this story. “That was quite an act of courage.
“I have a feeling his lowly status made him someone that they would be able to hope would just go away,” Caplan said. “There was a little bit of don’t-let-the-door-hit-you-on-the-way-out.
“Perez can look at himself in the mirror. Every day. But he paid the price.”
Brooding on a Beautiful Weekend
At the time he joined the Potti lab, Perez surely considered himself fortunate.
In 2006, Potti et al. had published a revolutionary paper in Nature Medicine, proposing using genomic signatures to guide the use of chemotherapeutics.
Another Potti paper, published in the New England Journal of Medicine, proposed using genomics to assign early-stage lung cancer patients to treatment regimens.
In both cases, the reported scales of improvements were dramatic.
However, MD Anderson biostatisticians Keith Baggerly, Kevin Coombes and Jing Wang wrote a letter to Nature Medicine in November 2007, stating that they were unable to reproduce the results the Duke group claimed. Responding in Nature Medicine, Nevins and Potti partly acknowledged the criticism, but argued that their findings still stand.
Perez’s examination of the Potti lab was triggered by questions he received from editors on a paper he had submitted to the Journal of Clinical Oncology. Alerted by the correspondence in Nature Medicine, the reviewers had specific questions about a paper on which Perez was the first author.
Perez was in better position than Baggerly, Coombes and Wang to assess what was going on. As an insider, he could simply go to the lab and check. What he found appears to have shaken him to the core.
The trail of correspondence obtained by The Cancer Letter begins on March 2, 2008. It’s a beautiful day in Durham; a Sunday.
Perez is indoors. A few weeks earlier, he came to the realization that the methods used by his research group aren’t validated—and yet they are being used to assign patients to clinical trials.
A choice of therapy made based on faulty criteria can do harm.
Perez makes several cautious attempts to discuss methodological flaws with Potti, but Duke’s star researcher isn’t open to hearing the message, saying that he takes such criticism personally.
Now, on this gorgeous Sunday, Perez sees a clear choice: (a) he can challenge powerful political forces at Duke or (b) he can allow his good name to figure on papers he knows to be shoddy or worse.
This dilemma has to be resolved pronto.
Reviewers at the JCO are asking questions, and Potti is pressuring Perez to resubmit the paper. Alternatively, Potti says he would submit the paper to a less persnickety journal.
In an email to Katherine Garman, at the time a fellow at Duke, who appears to be playing a role in his training or advising him informally, Perez bemoans his inability to validate the predictors used in his paper.
Because predictions made on data used to build a model are overly optimistic, many groups use the “cross validation” technique, where the model is built using only part of the data, and then used to predict the status of the remaining (unused) samples.
Here, however, validation techniques the Potti lab uses amount to “erasing the samples that don’t fit the cross validation from the figure and then reporting the cross validation as meaningful and justification for a good predictor,” Perez writes to Garman.
If genes are selected for a model by applying t-tests to all of the genes and choosing the best ones for subsequent cross validation, the chance that these genes will be significant in both subsets of the data—model building and validation—is high. Doing cross validation properly means running the t-tests using just the data available for building the model and seeing if these same genes stand out in the data held out for predictions.
It’s a circle. Your model will work only with the dataset you used to construct it. It will never work with any other data.
“When I asked Anil a few months ago about the use of a t-test to develop a predictor and whether this was biased, he mentioned that even though it was it didn’t matter as long as you had strong validation,” Perez writes to Garman. “Then when I mentioned at a more recent time that some of the predictors recently developed using t-tests had never been validated he said since the method was validated this was ok. This is really problematic.”
Perez senses that a reckoning is close.
“I’ve tried to [mention concerns one at a time] at times before but it’s never been clear to me before, and he said he takes it as a personal insult if people don’t believe in what he is doing,” Perez writes.
As a final, desperate effort to stay in the lab, Perez proposes bringing in a biostatistician, William Barry, to meet with Potti and him in order to acquaint Potti with the ABCs of methodology.
Perez recognizes that he doesn’t have the option to pretend that all is well at the lab. Doing so would ultimately hurt his scientific career and his good name.
“I talked to my dean about my concerns recently because I am nervous that things are coming to a head,” Perez writes to Garman. “He mentioned that he knew that papers which were dragged through the mud in the academic press could be problematic later in my career. I think that by publishing all the methods and knowing all these weaknesses in the predictor, I am setting myself up for that.
“How do you think it would be best to proceed? What will happen if Anil says someone else will publish the paper for me?”
Garman responds with a few lines.
Yes, it would be possible to get the biostatistician Barry to sit in on the meeting, she writes. But there could be no assurance that Barry could find the time immediately.
“Do you have an idea of what you are going to tell Anil since he wants to submit tomorrow,” Garman writes. “You could also contact Bill [Barry] to review some of your specific concerns. See you tomorrow.
“And it was a beautiful weekend—we spent lots of time outside.”
