I loves me some metrics. That’s why I’m addicted to this new PLoS ONE paper, published by Trevor Branch at the University of Washington. Also, because Figure 1 is a Wordle:
Y’all might remember the slight media coverage (ha!) of a very controversial fisheries paper published in 2006:
“Impacts of biodiversity loss on ecosystem services” by Boris Worm and others [7]…demonstrated through a large body of evidence that biodiversity loss greatly reduces the ecosystem services that we obtain from the oceans, and also contained an analysis projecting “the global collapse of all taxa currently fished by the mid–21st century (based on the extrapolation of regression in Fig. 3A to 100% in the year 2048)”. This projection of global seafood collapse by 2048 was highlighted in their associated press release [8]. The press release resulted in prominent coverage of the projection in major news outlets, and provoked a reaction from some fisheries researchers [8] and 10 rebuttals [9]–[18], including three rebuttals by Ray Hilborn [who argued fisheries papers published in Science and Nature were selected for their publicity value, despite being scientifically flawed] [10]–[12].
Despite the controversy, or perhaps because of it, Worm et al. [7] is a highly cited paper (799 citations at 20 January 2013 in the Web of Science), and was among the top 10 most cited fisheries papers in 2012 [21]. Past citation analysis shows that the rebuttals to the 2048 projection in Worm et al. [7] had virtually no effect on citation rates or the impressions that people gained about this paper [22]. (Branch 2013)
Trevor Branch, having long been captivated by fisheries research and having deep knowledge of this Worm et al. controversy, sat back in his chair, pented his fingers, and had an epiphany: “I’ve got it! We should analyze citation patterns for Worm et al. (2006)!” (a written dramatization of possibly real events). So he did. And the results are pretty awesome.
Sidenote: some of his methods were pretty badass.
Where gender [of a paper author] was not obvious it was inferred from the website Baby Name Guesser (http://www.gpeters.com/names/baby-names.php) which returns the ratio of male to female babies born with a particular name.
One of Branch’s main questions was, “Did the framing of the press release result in a greater frequency of citations mentioning the 2048 projection [mentioned in Worm et al 2006]?” – the answer was a resounding YES.
Thus, there is some evidence that increasing the prominence of the 2048 projection in the press release resulted in citing papers referring to this projection more often than expected.
However, Branch found that scientists who had a deeper understanding of the overall fisheries controversy were less likely to fall prey to the media hype:
Papers that mentioned the 2048 projection had characteristics that suggest unfamiliarity with the controversy surrounding this projection, namely papers with few authors, published in journals with low impact factors, in fields far removed from ecology and fisheries, and sharing no coauthors with the Worm et al. paper. For such papers, it is easy to see how the authors, editors, and reviewers could be unaware of the controversy over the 2048 projection.
These factors suggest that the more knowledgeable the authors of citing papers were about the controversy over the 2048 projection, the less likely they were to refer to it. A noteworthy finding was that if the original authors were also involved in the citing papers, they rarely (1 of 55 papers, 2%) mentioned the 2048 projection. Thus the original authors have emphasized the broader concerns about biodiversity loss, rather than the 2048 projection, as the key result of their study.
You can see these citation trends in the lovely Figure 2:
![he proportion of papers citing Worm et al. [7] that referred to (A) the relation between biodiversity and stability or ecosystem services, (B) fisheries collapse, or (C) the projection that all fisheries would be collapsed by 2048. (Branch 2013)](https://www.deepseanews.com/wp-content/uploads/2013/03/journal.pone_.0056723.g002.png)
So that leave me asking…are us scientists just as prone to believe soundbites promoted by press releases and the media, particularly for hyped-up papers outside our own discipline? In the case of Worm et al. (2006), the message that “all fisheries will collapse by 2048” certainly seemed to spread like an urban legend in the peer-reviewed literature.
Reference: Branch TA (2013) Citation Patterns of a Controversial and High-Impact Paper: Worm et al. (2006) “Impacts of Biodiversity Loss on Ocean Ecosystem Services”. PLoS ONE 8(2): e56723. doi:10.1371/journal.pone.0056723
Share the post "Media hype gets you more citations? Well, it did for this fisheries paper."
I, for one, cite the Worm et al. 2006 paper in nearly everything I write. Why? Because of the crystal clear results regarding how biodiversity influences marine ecosystem function in experiments and because of the tantalizing link between species diversity and a wide variety of properties of fisheries – results that so far appear to be largely holding up. It is a gorram milestone in the Biodiversity Ecosystem Function and Ecosystem Services literature. Let that not be forgotten.
Indeed, that those two things _are_ forgotten is why the 2048 thing continually bums me out.
(by which I mean that there are plenty of folk I talk to who dismiss the *entire* paper because of the 2048 number)
[DISCLAIMER: I am a co-author of the Worm et al (2006) paper.]
Thanks Holly for the nice synopsis. I hope you’ll forgive me, however, for failing to get too excited about this paper.
News flash: Sometimes a press release influences a paper’s reception!!! And sometimes the press emphasizes a sensational detail rather than the core message!
Um, OK. That’s what press releases do, right? Get people’s attention. Naturally if a lot more people read the paper and it gets covered in the media that will influence its citation patterns.
As Jarrett’s comment notes, there were solid, important results in the Worm et al paper, and they have mostly stood the test of time. In hindsight, there was also some discussion that was a bit sensationalistic. Unfortunately — but predictably — the 2048 projection tended to overshadow the value of the data and synthesis in press coverage (for another amusing example, see the final scenes of “The Men Who Stare at Goats”).
Nevertheless, I find the bottom line result of Trevor’s analysis quite reassuring: Across the >600 papers citing Worm et al, only 11% mentioned the peripheral issue of the 2048 projection, and the percentage was lower in higher-impact journals. In other words, the great majority of our colleagues and contributors to the rigorous scientific literature appear to have judged the paper on the merits of its main contributions and were able to look beyond sensationalistic press coverage. Which raises a question:
Can we move on already? C’mon people. The paper was published in 2006. We’ve been flogging these issues to death for more than six years now. Isn’t it time to let go and turn our attention to something more productive?