Embargo Watch

Keeping an eye on how scientific information embargoes affect news coverage

Why the New England Journal of Medicine’s handling of a spina bifida study embargo makes me doubt their good intentions

with 7 comments

On Friday, I received an email from the Children’s Hospital of Philadelphia, headlined:

Fetal Surgery Takes a Huge Step Forward in Treating Children with Spina Bifida

This finding, according to the release, was being published in the New England Journal of Medicine (NEJM). On the third page, there were some specific data:

The current study reports data on 158 patients who were followed at least one year after surgery. Clinicians who were independent of the surgical teams and blinded (not informed which of the two surgeries a given child received) evaluated the children from the study at one year of age and again at age 30 months.

–At one year of age, 40 percent of the children in the prenatal surgery group had received a shunt, compared to 83 percent of the children in the postnatal group. During pregnancy, all the fetuses in the trial had hindbrain herniation. However, at age 12 months, one-third (36  percent) of the infants in the prenatal surgery group no longer had any evidence of hindbrain herniation, compared to only 4 percent in the postnatal surgery group.

–At age 30 months, children in the prenatal group had significantly better scores in measurements of motor function. While the ability to walk depends on the level of the spina bifida lesion, the study found a twofold increase in the proportion of children able to walk without crutches or other assistive devices—42 percent in the prenatal group compared to 21 percent in the postnatal group.

That all sounded worth looking into. But the release came with a strict embargo:

Please do not share this with anyone as it is still under embargo until 5pm on Wednesday.

There were two strange things about this release. One was the conditions of the embargo. Was this another attempt to turn journalists into stenographers, which the FDA seemed to be after when they told reporters last month they couldn’t talk to anyone about a new program until the embargo lifted? After a bit of back-and-forth with Children’s Hospital press officers, I was reassured this wasn’t the case, although the original email was a bit unclear.

That’s because of the second strange thing about the release: NEJM hadn’t made any such study available to reporters yet. And since I’ve been quoted saying — because I believe it — that “it’s journalistic malpractice to not have the full study in front of you when you’re reporting,” I wanted to look at the study. So I asked the journal’s press office what was going on. They responded:

There will be an Online First paper embargoed until Wednesday at 5 PM ET.  The paper isn’t ready yet, and isn’t slated to be available until Tuesday at 5 PM.  We’re planning to send it to the media under embargo as soon as it’s finalized.

Well, I asked, wasn’t a Word document or something available? As I noted:

I just find it odd that Penn sent a release Friday and the paper isn’t available for four days, then with a 24-hour turnaround.

Apparently not, the journal told me:

Our production process doesn’t work that way. The process of editing and proofing takes days even when expedited.  The availability of a press release is not a reflection of the readiness of a manuscript.  We do recommend that an author’s institution wait until the article is available under embargo before they send out an embargoed press release, so reporters can refer to it.

I’m going to give Children’s a mild slap on the wrist here, since I happen to agree with NEJM that they should have waited. But they sure put a lot of details in that release considering that NEJM had just told me there wasn’t a Word document or something else they could have sent. Were the data carved into stone tablets, or, even more antediluvian, faxed?

OK, maybe something had been screwed up here. People had missed deadlines. The Great Flood had lasted longer than the scheduled 40 days. Moses got lost on the way down from Mount Sinai. So, I asked NEJM, was the embargo time and date set before the journal know exactly when they’d be able to send out a finalized version? No, apparently:

When a manuscript is accepted, we create a schedule and decide on the appropriate embargo and to media dates/times.  Yes, it was decided that this schedule would allow the media only 24 hours with the paper.  For expedited publications we sometimes do that.  The article could not be ready much earlier than late Tuesday.  The alternative would be to schedule Online First publication for 48 or 72 hours from that time, delaying publication to give the media more time.  Our standard release time is Wednesday at 5 PM ET, and we try to publish then when possible.  While 24 hours notice is not ideal, we think it is still reasonable.

Well, that sounds like a pretty good alternative to me. And with all due respect, by “the article” NEJM must mean “the final, done and dusted beautiful PDF version of the article.” Let’s try another definition: “a reasonable presentation of the data that’s being used to back up claims of efficacy.” Here’s what I proposed last April:

So here’s how journals could prove that they’re really interested in giving reporters time to cover stories as well as possible: Send page proofs, plain text, or some other not-quite-done-and-dusted version of studies. Journalists will understand. Heck, a lot of the journals I review online every week publish dozens of unformatted text versions of papers every month. I don’t mind, even when it sometimes means checking figures with authors an extra time.

If the current short embargo crowd isn’t willing to do that, I’m going to have a harder and harder time assuming good intentions.

I’m having that hard time. As the journal said, they “decided that this schedule would allow the media only 24 hours with the paper.” They also decided that the 5 p.m . Eastern Wednesday embargo was more important than giving reporters more time.

Full disclosure: As part of Embargo Watch’s short embargo race, I set the bar for “short embargo” is also 24 hours, but I also said that was a really low bar. And NEJM has had no real competition for the race record of 49 minutes.

I’ve quoted NEJM’s embargo policy rationale before. Here it is again. Note that in the case of Online First papers such as this one, the second half of the sentence doesn’t apply at all.

It allows the media time to report accurately on complex and important new findings, and it allows subscribers to read and understand the full reports in NEJM before media reports on them appear, which often leads patients to ask their physicians for interpretation and guidance.

So if the only reason for the embargo is to give the media time, why not give them some earlier version of the paper, so that they — and their outside experts — have time to report?

I can’t help but wonder if NEJM simply didn’t really want this study scrutinized. Maybe it’s because they knew this was the kind of study reporters would want to cover: High-tech surgery in the womb. Gee! Whiz!

But those kinds of studies need a careful eye, because this procedure isn’t a home run. The authors of an accompanying editorial even said so:

It is…human nature to overestimate the likely benefit for one’s own fetus and to underestimate the associated risks. Counseling should involve not only precise quantitative statements comparing outcomes of prenatal versus postnatal surgery on the basis of this report but also the provision of information on center-specific experience.

The degree to which intrauterine repair will transform outcomes for fetuses with myelomeningocele remains unclear. The study by Adsick et al. is a major step in the right direction, but the still suboptimal rates of poor neonatal outcome and high maternal risk necessitate the use of less invasive approaches if such procedures are to be widely implemented.

Maybe that’s not the reason NEJM thought it was best to only give reporters 24 hours. But I’m not hearing any others that make sense.

About these ads

Written by ivanoransky

February 10, 2011 at 12:58 pm

Posted in Uncategorized

7 Responses

Subscribe to comments with RSS.

  1. I have to think this situation had more to do with the academic institution than the journal. In other words, it may not have been an attempt to bury but actually an eagerness to promote the study that led to a press release being distributed before a study was available. I agree it’s bad practice. I’ve distributed studies before they were final (as long as no major changes, such as tables, etc. were expected), which while not perfect seems to be a better alternative that forcing reports to turn a round a complicated study in a day.

    David Sampson

    February 10, 2011 at 1:10 pm

  2. So why does the NEJM or any journal need a weekly publication date/time when they can publish online at any moment? NEJM could simply release papers online daily at 5pm and give journalists 48-72h to read embargoed material.

    Old habits die hard but don’t they know online publishing is a whole new world?

    Michael McBurney

    February 10, 2011 at 1:23 pm

  3. There are very good reasons to not make a Word document available to reporters before the article is final. We’re not just running the article through a spell/grammar check. Authors and editors double-check all of the numbers for accuracy. Errors can be introduced in the editing and layout process and must be corrected. Providing a Word document where there may be a serious error would be irresponsible. How would we ensure that reporters follow up for the final version and do all of the same double-checking of the numbers that we did?
    To speculate that we didn’t want this study scrutinized is unfair.

    Karen Buckley

    February 10, 2011 at 4:38 pm

    • Thanks for the comment, Karen. I understand your point about sending out a document that might contain errors, but I think there’s a solution: Make it a condition of the embargo that reporters check their stories against the final version you send out 24 hours early. Then, everyone has an incentive to ensure accuracy. That’s essentially what book publishers do when they send out galleys.

      ivanoransky

      February 10, 2011 at 5:05 pm

    • Hi Karen. Let me ask a possibly naive question. If the manuscript is not finalized, wouldn’t that mean a press release could be subject to those same inaccuracies, so perhaps should not be released, as well?

      David Sampson

      February 10, 2011 at 8:36 pm

      • Yes, it’s possible that the same errors could be introduced in a press release as well, though the level of detail in a press release is not the same as what’s found in a manuscript. To be clear, NEJM did not issue an embargoed press release before the manuscript was finalized. The authors’ academic institution did. We do strongly suggest that institutions wait to issue embargoed press releases until we’ve notified the media that the paper is available for them under embargo.

        Karen Buckley

        February 22, 2011 at 12:38 pm

  4. Many thanks for posting the editorial link – funnily enough the editorial wasn’t mentioned in the press release.

    Anna Sayburn

    February 11, 2011 at 7:43 am


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 21,894 other followers

%d bloggers like this: