Chris ChambersVerified account

Here was (and is) a major potential cause of misreporting and it's under our very noses as scientists. We should be appalled, no? How dare we complain about reporters when it is we who start the problem in the first place? 16/x

So what did I learn from all this? I'm still learning but here are a few home truths for fellow scientists - things I've picked up not only from this ongoing journey but during my time since 2013 as a Guardian Sci blogger +3 years on the @SMC_London Advisory Committee. 17/x

First – if you insist on checking a journalist’s copy before they publish, get used to never talking with the good ones. It makes you sound pedantic & untrusting, & you are only ensuring that your work is either never covered or is only covered by journos who lack confidence 18/x

Second – sometimes journalists check their copy (or sections of it) anyway to be sure they have got a particular fact correct. It happens to me maybe 5% of time. That’s fine if it’s their choice but never impose that as a condition of engagement. 19/x

Third – don’t ask to check quotes. Instead, if you have prior concerns, tell them you are recording the interview at your end for your records. If they misquote you, tell them to correct it. If they won’t, publish the transcript, tell their editor & never talk to them again. 20/x

Fourth – if you really want accurate science news, avoid exaggeration in your own press releases and anticipate likely misunderstandings by including a section “What this study does NOT show”. If you allow hype in your PR then YOU share culpability for misreporting. 21/x

Finally, accept that you're not special to journalism & neither is science. Independence is key to journalism. Sometimes journalists will screw up & sometimes you will do it all by yourself. Get media trained, find the good journos & trust them. Basically, get over yourself. /fin

Postscript: for those asking about the next step in our work...it's not over yet!https://twitter.com/chrisdc77/status/960792083748589568 …

I see this thread from last year is getting another round of attention. Results of our randomised trial to test the effect of press releases on the news are now in-press with @BMCMedicine so stay tuned! cc @adamsrc86

We’ve just published the next step in our journey to understand the role of press releases in science news. It’s without doubt the most off-the-wall & unexpected research project I’ve ever been involved in. Our latest foray was a real-world experiment on the news media itself.

That’s right. We did an experiment on the news. We took press releases on health-related science, altered them before they were issued to journalists, and then studied what effect the changes we made influenced science reporting.

In this continuation of the thread I started last year, I want to take you through the results of the trial and what I think they mean. But I also want to take you behind the scenes of doing this sort of research because the project was a political roller coaster.

To recap tweet 14/x above, in Dec 2014 we published a retrospective study which found that most exaggeration found in health-related science news is already in the press releases issued by universitieshttps://twitter.com/chrisdc77/status/960309818430025766 …

This was big news & it made a splash. When it came to bad sci reporting, university press releases were likely to be a major contributor. As someone put it, all these years it was assumed that *reporters* created hype, but in fact “the call was coming from inside the house” /26

But those of us on the research team knew from the very beginning that as impactful as the project might be, it could never provide evidence that press releases *causally* influence science news. Why? /27

Because the research was retrospective and observational, much like a lot of epidemiology. This meant we could only ever show *associations* between the content of press releases and the content of news stories. /28

To demonstrate causality you would need to do an experiment. And to do it properly you would need a randomised controlled trial or RCT – the gold standard for testing the existence of causal effects. /29

If we could show in an RCT that improving the quality of press releases improved the quality of science news – and, crucially, WITHOUT reducing news uptake – we could build an evidence-based policy for sci communication that press officers would have every reason to embrace /30

Afterall, we knew already that press officers had no innate desire to issue inaccurate or hyped press releases. But from talking with them (lots of them) we also understood the pressure they face to generate media impact for their universities. /31

One press officer we met took this so seriously that every week he'd get out his ruler and record the literal column inches of every print news story stemming from his university's research. He'd been doing it for decades. /32

For running a randomised trial, the big problem I kept coming back to was feasibility. It seemed like one of those thought experiments that in theory would be beautiful but in reality you know is a pipedream. /33

We needed the idea to pass the smoke test. So at the @SMC_London Christmas drinks party in 2014 we worked the room to find out what journalists might make of an experiment on the news. Our BMJ paper had just come out & a lot of people were talking about it. /34

We asked the journalists: How would you feel about being guinea pigs in a trial where the press release you’re reading might have been manipulated? How would you feel not knowing what the manipulation was? Not even knowing if this press release was part of the trial or not? /35

The response varied. A few of the specialist reporters liked the idea. Some didn’t give a crap. One guy spent the whole time glancing forlornly over my shoulder to see if someone more important might walk past who he could talk to. I felt a bit sorry for that guy /36

Several journalists were quite disturbed by it. One BBC reporter told me that running controlled experiments on the news was undemocratic and dangerous. /37

That raised a red flag for me. If we ran the trial, I wondered if journalists might crucify us. Visions of being doorstepped. ENEMY OF DEMOCRACY. MANIPULATOR OF THE NEWS. A laughable anxiety, really, given the state of the world now but these were the heady days of pre-2016. /38

At the same time if we were going to run an experiment like this, our most important partners were not the journalists, they were the press officers. If we were going to intervene in press releases b4 they were issued, we'd need press officers to work with us. And trust us. /39

Not all of them, of course, but enough to make the prospect of a trial a diplomatic nightmare. Here we were, mostly nobodies with no sci comm background, detonating a nuclear warhead in the BMJ telling them that most exaggeration in science news begins in press releases. /41