Why the New England Journal of Medicine’s handling of a spina bifida study embargo makes me doubt their good intentions
Fetal Surgery Takes a Huge Step Forward in Treating Children with Spina Bifida
This finding, according to the release, was being published in the New England Journal of Medicine (NEJM). On the third page, there were some specific data:
The current study reports data on 158 patients who were followed at least one year after surgery. Clinicians who were independent of the surgical teams and blinded (not informed which of the two surgeries a given child received) evaluated the children from the study at one year of age and again at age 30 months.
–At one year of age, 40 percent of the children in the prenatal surgery group had received a shunt, compared to 83 percent of the children in the postnatal group. During pregnancy, all the fetuses in the trial had hindbrain herniation. However, at age 12 months, one-third (36 percent) of the infants in the prenatal surgery group no longer had any evidence of hindbrain herniation, compared to only 4 percent in the postnatal surgery group.
–At age 30 months, children in the prenatal group had significantly better scores in measurements of motor function. While the ability to walk depends on the level of the spina bifida lesion, the study found a twofold increase in the proportion of children able to walk without crutches or other assistive devices—42 percent in the prenatal group compared to 21 percent in the postnatal group.
That all sounded worth looking into. But the release came with a strict embargo:
Please do not share this with anyone as it is still under embargo until 5pm on Wednesday.
There were two strange things about this release. One was the conditions of the embargo. Was this another attempt to turn journalists into stenographers, which the FDA seemed to be after when they told reporters last month they couldn’t talk to anyone about a new program until the embargo lifted? After a bit of back-and-forth with Children’s Hospital press officers, I was reassured this wasn’t the case, although the original email was a bit unclear.
That’s because of the second strange thing about the release: NEJM hadn’t made any such study available to reporters yet. And since I’ve been quoted saying — because I believe it — that “it’s journalistic malpractice to not have the full study in front of you when you’re reporting,” I wanted to look at the study. So I asked the journal’s press office what was going on. They responded:
There will be an Online First paper embargoed until Wednesday at 5 PM ET. The paper isn’t ready yet, and isn’t slated to be available until Tuesday at 5 PM. We’re planning to send it to the media under embargo as soon as it’s finalized.
Well, I asked, wasn’t a Word document or something available? As I noted:
I just find it odd that Penn sent a release Friday and the paper isn’t available for four days, then with a 24-hour turnaround.
Apparently not, the journal told me:
Our production process doesn’t work that way. The process of editing and proofing takes days even when expedited. The availability of a press release is not a reflection of the readiness of a manuscript. We do recommend that an author’s institution wait until the article is available under embargo before they send out an embargoed press release, so reporters can refer to it.
I’m going to give Children’s a mild slap on the wrist here, since I happen to agree with NEJM that they should have waited. But they sure put a lot of details in that release considering that NEJM had just told me there wasn’t a Word document or something else they could have sent. Were the data carved into stone tablets, or, even more antediluvian, faxed?
OK, maybe something had been screwed up here. People had missed deadlines. The Great Flood had lasted longer than the scheduled 40 days. Moses got lost on the way down from Mount Sinai. So, I asked NEJM, was the embargo time and date set before the journal know exactly when they’d be able to send out a finalized version? No, apparently:
When a manuscript is accepted, we create a schedule and decide on the appropriate embargo and to media dates/times. Yes, it was decided that this schedule would allow the media only 24 hours with the paper. For expedited publications we sometimes do that. The article could not be ready much earlier than late Tuesday. The alternative would be to schedule Online First publication for 48 or 72 hours from that time, delaying publication to give the media more time. Our standard release time is Wednesday at 5 PM ET, and we try to publish then when possible. While 24 hours notice is not ideal, we think it is still reasonable.
Well, that sounds like a pretty good alternative to me. And with all due respect, by “the article” NEJM must mean “the final, done and dusted beautiful PDF version of the article.” Let’s try another definition: “a reasonable presentation of the data that’s being used to back up claims of efficacy.” Here’s what I proposed last April:
So here’s how journals could prove that they’re really interested in giving reporters time to cover stories as well as possible: Send page proofs, plain text, or some other not-quite-done-and-dusted version of studies. Journalists will understand. Heck, a lot of the journals I review online every week publish dozens of unformatted text versions of papers every month. I don’t mind, even when it sometimes means checking figures with authors an extra time.
If the current short embargo crowd isn’t willing to do that, I’m going to have a harder and harder time assuming good intentions.
I’m having that hard time. As the journal said, they “decided that this schedule would allow the media only 24 hours with the paper.” They also decided that the 5 p.m . Eastern Wednesday embargo was more important than giving reporters more time.
Full disclosure: As part of Embargo Watch’s short embargo race, I set the bar for “short embargo” is also 24 hours, but I also said that was a really low bar. And NEJM has had no real competition for the race record of 49 minutes.
I’ve quoted NEJM’s embargo policy rationale before. Here it is again. Note that in the case of Online First papers such as this one, the second half of the sentence doesn’t apply at all.
It allows the media time to report accurately on complex and important new findings, and it allows subscribers to read and understand the full reports in NEJM before media reports on them appear, which often leads patients to ask their physicians for interpretation and guidance.
So if the only reason for the embargo is to give the media time, why not give them some earlier version of the paper, so that they — and their outside experts — have time to report?
I can’t help but wonder if NEJM simply didn’t really want this study scrutinized. Maybe it’s because they knew this was the kind of study reporters would want to cover: High-tech surgery in the womb. Gee! Whiz!
But those kinds of studies need a careful eye, because this procedure isn’t a home run. The authors of an accompanying editorial even said so:
It is…human nature to overestimate the likely benefit for one’s own fetus and to underestimate the associated risks. Counseling should involve not only precise quantitative statements comparing outcomes of prenatal versus postnatal surgery on the basis of this report but also the provision of information on center-specific experience.
The degree to which intrauterine repair will transform outcomes for fetuses with myelomeningocele remains unclear. The study by Adsick et al. is a major step in the right direction, but the still suboptimal rates of poor neonatal outcome and high maternal risk necessitate the use of less invasive approaches if such procedures are to be widely implemented.
Maybe that’s not the reason NEJM thought it was best to only give reporters 24 hours. But I’m not hearing any others that make sense.