Five things wrong with Johann Hari’s comeback book that I spotted from the extract alone

Update 22/1/18: I’ve read the whole fucking book now. It’s changed the view I’ve expressed here, very slightly, to “it was a bit worse than I thought”. Read more.

Noted plagiarist and wikipedia editor Johann Hari is back, with a book about depression. Yesterday, the Observer published an extract from the book, Lost Connections, which I presume is an early chapter setting the scene for Hari’s main thesis.

As far as I can discern from the extract, Hari is arguing that the environment is the cause of depression, with neurochemical imbalances not being particularly important, and therefore antidepressants not being very good. Now, I’m pretty critical of psychiatry, and very critical of the tendency towards prescribing antidepressants because waiting lists for talking therapies are so interminably long. However, we can’t have these conversations while we’re spending endless hours clearing the Augean stables of awful science, with nary a river to reroute.

Yes, I am calling Hari’s extract horseshit. The ideas, I might be convinced to agree with in part, but there are serious, fundamental flaws with his methodology which mean that it’s impossible to take anything seriously. I am going to assume his extract is representative of the book as a whole, and highlight some of these major flaws, expanding a thread I made on Twitter. Let’s start with a guided example of how Johann Hari is flat-out making shit up. Talking about changing DSM criteria of depression, Hari says:

So, they responded in a simple way – by whittling away the grief exception. With each new edition of the manual they reduced the period of grief that you were allowed before being labelled mentally ill – down to a few months and then, finally, to nothing at all. Now, if your baby dies at 10am, your doctor can diagnose you with a mental illness at 10.01am and start drugging you straight away.

The bolded part is a complete, total falsehood, which is easily refuted by 10 seconds on google. Search “DSM depression criteria”. Click the first link. Or the second, or any, they all take you to the criteria. Now look at the first fucking line of the criteria: “Depressed mood or a loss of interest or pleasure in daily activities for more than two weeks.” Hari has made up a fact about maternal grief and the teams who work with those who have lost children, to make an imaginary point.

This research methodology seems prevalent throughout the extract, and there’s five key things I can see from reading a few thousand words.

1. [citation needed]

Footnotes and citations are necessary when writing a book based on presenting an evidence-based argument. This goes for anyone, but is particularly important if you’re a disgraced writer who has a history of fabricating things. Citations are completely absent from the published extract of Lost Connections, despite confidently-asserted statistics, for example: “It turns out that between 65 and 80% of people on antidepressants are depressed again within a year.” Where is this from? Who found it? Is it from an Irving Kirsch study, since Kirsch is mentioned in the paragraph above? If so, which Kirsch study? Where can we read it so we can get context for the figure?

When a number is presented, you link to where it’s found. And you make it clear where you found it. Otherwise, you might be misrepresenting it. Or you could have made it up completely. Where there’s no referencing, take any information presented with an ocean of salt.

2. Reliance on a single piece of research

Hari’s argument that antidepressants don’t work relies heavily on the work of a single researcher, Irving Kirsch. Now, due to the lack of citations, I can’t be completely certain that the research Hari outlines is Kirsch and colleague’s 2008 meta-analysis, but I’m going to guess it was because this is the most famous research into the topic.

Kirsch’s meta-analysis is decent, although it is not as definitively presenting that antidepressants don’t work for most people as Hari presents the research. Firstly, Kirsch and colleagues didn’t find that antidepressants don’t work on the majority of people: they found that effectiveness of antidepressants are more effective for severe depression and less effective for mild or moderate depression. That’s a nuanced difference, and it’s unfortunate that it led to so many “drugs don’t work” headlines from a screechy media, and Hari has lapped it right up.

Secondly, other researchers analysed the same dataset and drew different conclusions. Using different statistical modelling, Fountoulakis and colleagues found antidepressants were better than placebo, at all levels of depression severity. Turner and Rosenthal’s interpretation of the data is different to Kirsch’s, suggesting that certain measures can be more important than disappearance of depression, such as quality of life, which has been overlooked in Kirsch’s study, and to be “circumspect but not dismissive” in considering the benefits of antidepressants.

And one more issue is present in Kirsch’s research: it’s not a look at all antidepressants. It examines four drugs, all of the same type: selective serotonin reuptake inhibitors (SSRIs).

3. There’s lots of different antidepressant drugs

“Antidepressant” is a wide category of types of drug, which do different things: Mind’s information lists the drugs, and what they do. Some drugs act on serotonin receptors–the SSRIs which Kirsch studied. Others act on different receptors, or prolong the activity of neurotransmitters, or perhaps make it harder for the body to break the neurotransmitters down: most of these will regulate levels of serotonin, noradrenaline, or both. Then there’s the weirdo drugs which don’t act on serotonin or noradrenaline: the atypical antidepressants, which include drugs like mirtazipine, which doesn’t do any of that, or variants on ketamine, which are increasing in popularity.

Basically, the drugs work differently, and it’s not exactly a secret that different drugs work better for different people: this is one of the first things the NHS tells you in their information for patients.

Hari fails to make the distinction throughout his article, referring only to “antidepressants”, the umbrella term for a diverse range of drugs which act in different ways on the body’s neurochemistry, and which are well-known to affect different people differently. There is no effort whatsoever made to acknowledge that not all antidepressants are the same, and the study he’s citing refers only to one class. This nuance is important. Really, really important.

4. The serotonin hypothesis isn’t as important as you think

Poking holes in the serotonin hypothesis is treated by Hari as debunking the neurochemical basis of depression. That’s a pity, because it doesn’t. There are dozens of theories of depression, both biological and cognitive, and the serotonin hypothesis is but one. It’s also acknowledged it may be caused differently in different people. Genetics, neurochemistry including but not limited to serotonin, interpersonal factors, the environment, the immune system… all of these things and more are believed to contribute to depression.

The dominance of the serotonin hypothesis in the public consciousness is mirrored by Hari’s writing, and presents a grotesquely oversimplified perspective of something which is a lot more complicated than that. The way you’d think it if you were listening to Hari was that science has two cards on the table: a deficiency in serotonin, or the environment, which is a brand-new discovery made by Hari, and definitely not something widely-acknowledged in the scientific literature. This is simply not the case, and never has been. Christ, even a basic A Level in Psychology will teach you that.

Doubt about the serotonin hypothesis does not mean that there’s a vast conspiracy to put people on drugs when really we should be making the world a nicer place. It’s a hell of a lot more nuanced than Hari would have it.

5. Stress and depression aren’t the same thing

Towards the latter end of the extract, Hari discusses environmental factors, and places a lot of emphasis on stress. While stress is acknowledged to be a contributor to depression, it’s a different kettle of fish entirely and isn’t thought to be the root cause of all depression by anyone. Stress is physical changes to the body caused by your “fight-or-flight” responses going on the alert in response to an external stimulus and just keeping on going. Stress isn’t a medical condition, per se, and it’s often advised that it’s managed by relaxation or just taking a break once in a while. There’s different neurotransmitters involved: depression itself doesn’t tend to have increased levels of cortisol, which is the dangerous thing about stress, and the killer. Because it’s different, stress has different symptoms to depression, though there may be some overlap.

Hari conflates stress and depression repeatedly throughout the latter half of his article. This is an enormous problem, because it becomes difficult to follow, and therefore critically appraise, exactly what he’s talking about, and also, to acknowledge that these are different problems, with different solutions, and it seems as though Hari favours the treatments recommended for stress–which may be why he conflates depression and stress so readily.

tl;dr

If this extract is representative for Johann Hari’s comeback book, don’t believe a word he says. The methodology is awful, given how much I spotted just from a few thousand words and a quick read.

It’s a huge shame there’s so much ill-informed nonsense out there, because there are real conversations we need to have about psychiatry and medicalisation, which we can’t have when we’re fighting this crap.

Update 13/1/18: Johann Hari has written a response to criticisms of his extract and research methodology on his blog. He also responds to fact-checks from Dean Burnett, who wrote a very good critical article questioning the conclusions and implications, and Stuart Ritchie, who presented meta-analytical evidence for the efficacy of antidepressants and identified the source of Hari’s “65-80%” figure, which I pulled up in the “citation needed” section of this blog. Stuart’s thread is good and spoiler: the figure came from a self-help book. Stuart has responded to Hari’s response over on Twitter, which, along with his original thread and Dean’s article, are well worth a read.

I’d like to respond to Hari’s response too. Hari neglects to respond to four of the five points I’ve made in this article, opting only to answer point 2: relying heavily on the research of Professor Irving Kirsh. To refute this, Hari got Professor Irving Kirsch to reply. Throughout Hari’s response to Dean, Stuart and I, Kirsch is mentioned or quoted more than 20 times. I don’t think I need to say why this is not a good way of refuting my concerns! I acknowledge that Kirsch thinks his own research is the most solid, and that Johann Hari favours the work of a researcher who unequivocally supports his conclusions. I will say that the criticisms of Kirsch’s meta-analyses still stand, as well as the bulk of meta-analyses conducted by other researchers. I’ll also say that even if a hole had been poked in my concerns about over-reliance on the work of a single researcher, there’s still four other reasons to be worried about the methodology Hari has deployed.

I’m also pretty concerned that there’s only three of us raising criticisms of Hari’s book, and one of us is just some rando blogger (that’d be me!). It’s very telling that media outlets have not been proactively commissioning experts to review the book, as opposed to other journalists. This isn’t just some journalistic circlejerk. There are real-life consequences, and at least one quote from someone considering stopping their meds off the back of the book has been found. To anyone who is thinking of coming off meds I say this: do it under medical supervision. For many antidepressants, you mustn’t just stop taking them, but need to titrate off. You should also be checking in with a medical professional regularly to ensure you are doing it safely, and to see if it’s working for you. This is very important and for pity’s sake DON’T JUST STOP TAKING YOUR MEDICATION.

_

Enjoyed what you read? Consider becoming a Patron, support me on Liberapay, or leave a tip