Science gets it (mostly) wrong again: My take on the NASA astrobiology paper

courtesy Science

They’re apparently getting rid of narcissistic personality disorder in the next edition of the Diagnostic and Statistical Manual of Mental Disorders (aka DSM-V). Until that book comes out in 2013, however, I’m free to say that I was hitting all the criteria this week as loyal readers sent out Twitter versions of “Let’s Go Embargo Watch” chants.

You see, it was only natural for my legions of fans to shine the EmBATgo Signal. After all, all hell had broken loose over the news that NASA was about to report in Science that they had discovered extraterrestrial life.

Wait, that isn’t what they’re reporting?

Well, no, not according to the UK’s Daily Mail or Sun, who went live more than a day before the Science embargo was scheduled to lift with stories about the fact that NASA scientists had found bacteria living in a lake in California. In typical tabloid fashion, the stories — which were pointed out to me by Nature‘s Geoff Brumfiel — didn’t exactly shy away from using the word “alien” in scare quotes. Still, they made it clear that scientists hadn’t discovered ET.

The Sun‘s story ran first, by the way, leading a peeved Mail to include this line:

But after The Sun broke the embargo on the story this morning, it can be revealed that the truth is rather closer to home.

That led Sun reporter Paul Sutherland to tweet (hat tip Neil Withers):

Dear Daily Mail. Sun did not “break the embargo”. I did my own research using material already in public domain. 🙂 http://bit.ly/ggTJ7i

So did either of these stories break the embargo? No, according to Ginger Pinholster at AAAS/Science, who sent out this message yesterday:

A number of journalists have contacted the “SciPak” office with questions about the Science embargo after seeing mostly erroneous online and/or tabloid speculation about the forthcoming research. These reports clearly are not based on the peer-reviewed research being published under the auspices of the journal Science.

There’s some parsing in that paragraph, but I think it’s saying there was some erroneous online speculation. OK, fair enough. But it’s also saying there was some erroneous tabloid speculation, and in fact some erroneous online tabloid speculation. So what was wrong in the Mail or Sun stories, exactly? And what exactly were the reports based on, if not the research in Science?

We get a hint at the end of the email about Science‘s criteria for breaks. Apparently, if you do what they want and cover things accurately before an embargo lifts, they’ll lift it early. But if you get things wrong (according to them, anyway), they won’t:

We send our thanks to all journalists registered to access embargoed information who are communicating science both broadly and accurately.

If all of this sounds familiar to Embargo Watch readers, it’s because it’s nearly identical to what happened in April when Science had — according to a few reports out before the weekly embargo had lifted — published evidence of a “missing link.” Although none of the journalism outlets included the journal’s name in their stories, others did. Here’s what Science sent out after stories started running about that hominin find:

SPECIAL NOTE: Registered reporters, it is acknowledged that several media outlets have run “teaser” type stories based on portions of information related to fossil research described more fully and accurately in this press package. The teaser articles, appearing prior to the release of the Science Press Package, unfortunately may in some cases over-state the forthcoming research, and thus are a disservice to the goal of communicating science accurately, thereby promoting public trust in the integrity of science. These articles do not reference Science, however, and are not based upon the peer-reviewed version of this research. The Science embargo, detailed below, remains in effect. The SciPak team wishes to commend all journalists concerned with the accurate communication of research news.

There’s a slight difference this time; the Science Press Package had already gone out this week by the time the Mail and Sun stories ran. But that has no bearing on whether the journal should have lifted the embargo early — if it’s actually interested in accurate coverage, that is.

Here’s what I wrote then:

If the goal is to communicate the science accurately, and there’s a lot of allegedly inaccurate coverage floating around out there about a study that is already peer-reviewed and proofed, and available in PDF form to reporters, can someone please explain why the best thing to do is to wait until Thursday to release the actual study? And in the meantime, letting everyone know that breaking that embargo in order to communicate the science accurately would lead to sanctions, just like any other embargo break?

Sorry, Science, but nothing has changed my mind. I speculated at the time of the hominin fiasco that one of the reasons for keeping that embargo was that a press conference had been scheduled with all the key players. There’s a similar press conference for this, too, but it started as this post went live when the embargo lifted. So it would have happened after the lift whether or not Science decided to do the right thing and lift the embargo early.

Which, I am now scrambling to add to this post, they eventually did, an hour and 47 minutes ahead of schedule. Why today? “[B]ecause news reports disclosing the findings in the paper are now appearing online,” according to a message they sent out at 12:13 p.m. Eastern. There were lots of stories after the Sun’s and the Mail’s, as most Embargo Watch readers know.

In fact, it was the announcement of that NASA press conference that probably led to the problem this time around.  I got a NASA release in my inbox Monday afternoon, forwarded by the American Astronomical Society. The fact that NASA was having a press conference wasn’t embargoed, and was posted publicly — which is great for transparency, I suppose, but also a really good way to fuel speculation.

I asked NASA this morning whose call it was to leave the embargo in place. Science’s, the agency told me. As far as that press release:

NASA sends out a media advisory to announce an event. It WAS NOT a press release discussing the finding. Also, to maintain our valued relationship with Science, we included the information was embargoed, per Science procedures, for 2 p.m.

So this wrong call on holding the embargo until just now was Science‘s. And what better way to fuel even more speculation, and build suspense, than to not officially allow the very press that you supposedly hope will get things right to, um, get things right and tamp down that speculation? See, if Science won’t allow anyone to say what exactly the paper says, and meanwhile says the reports you’ve read — which by then included stories saying this wasn’t ET — were erroneous, there’s still a chance that NASA has discovered life on Mars after all!

But that frenzy goes away once you do the right thing and lift the embargo. I should know. I’m deflating as I write this, knowing that the suspense of not knowing what I’ll tell my legions of fans will dissipate as soon as it posts. (But building suspense isn’t why I didn’t post. I didn’t post because I work at a big news organization and caution dictates that I don’t give embargoing institutions any reason to yank my access or that of my colleagues.)

It’s Wag the Dog. Or maybe the scene in Casablanca in which Captain Renault is “shocked, shocked” to find out that gambling is going on at Rick’s, only to be handed his winnings by a croupier. (I’ve quoted that before.)

Moral of the story: Embargoes are porous. Deal with it.

Back to that NASA release. It, plus “a little research on the press conference participants,” led blogger Jason Kottke to conclude:

So, if I had to guess at what NASA is going to reveal on Thursday, I’d say that they’ve discovered arsenic on Titan and maybe even detected chemical evidence of bacteria utilizing it for photosynthesis (by following the elements). Or something like that.

Which brings me to the second interesting part of this story. Kottke’s, um, how can I put this gently — completely and utterly [expletive] wrong — prediction prompted a tweet from The Atlantic‘s Alexis Madrigal:

I’m sad to quell some of the @kottke-induced excitement about possible extraterrestrial life. I’ve seen the Science paper. It’s not that.

That tweet — which, in Kottke’s defense, he posted to his blog right away — in turn, led writer John Fleck, of New Mexico, to ask:

Been dying to say that, @alexismadrigal Is it an embargo break to say what a paper is not, @ivanoransky?

It just so happens I had an answer, at least if the journal in question is the New England Journal of Medicine. After all, Adam Feuerstein had done just that in January to fight an inaccurate rumor that NEJM was going to publish a study on Cyclacel’s seliciclib. NEJM said that wasn’t a break.

I think Science would feel the same way, and so does Madrigal, apparently, based on a later tweet. I asked the journal’s press office whether they’d consider Madrigal’s tweet a break, and while they didn’t actually respond, I take it from the fact that they didn’t lift the embargo that they don’t.

I think I’ll leave you with an obviously tongue-in-cheek, speculative tweet from Discovery News’s Ian O’Neill:

I reckon NASA has discovered a troop of arsenic-eating hamsters burrowing into Titan’s hydrocarbon-laced dunes. Prove me wrong.

Now that’s just silly. Why on Earth would hamsters eat arsenic when there are all those hydrocarbons around to burn for fuel? Might as well go to the drive-through.

I know, I know, they’re not on Earth.

Oh, never mind.

Other coverage: CJR’s The Observatory, Boing Boing. Plus, see how you could have bet on what the paper would actually say, and NASA’s postmortem comments.