Are two thirds of young people “cyber-deviants”? A dive into a dodgy study.

Earlier this week, several news outlets breathlessly reported on a new study which had found that (gasp) two thirds of 16-19 year olds in Europe are engaging in risky or criminal activity on the internet. One of the authors, doing the press rounds, explicitly spoke out in favour of the upcoming Online Safety Bill, as a way of keeping these naughty, naughty kiddos safe from themselves.

This, of course, triggered my bullshit radar, and this graph in the Guardian in particular caused a forehead sprain from raising my eyebrows too hard.

Graph from Guardian article displaying percentages of 16-19 year olds who have engaged in the following behaviours:
Watching porn - 44%
Piracy - 34%
Tracking - 27%
Trolling - 27%
Incited violence - 22%
Sexting - 22%
Illegal gaming marketplace - 18%
Spam - 15%
Self-generated images - 15%
Money muling - 12%
Harassment - 12%
Hate speech - 11%

Are you seeing what I’m seeing? Sticking watching porn or a poor-quality and free stream of a show right there alongside things like money laundering and hate speech? Let’s bear in mind that a sizeable portion of them are literally legally adults. You may also be wondering what some of these categories even are. We’ll get onto that later, because it’s funny as fuck.

Let’s take a look at the report in all its glory.


Every dataset needs good sampling for good research. This pan-European study used 7,974 participants, which on the face of it sounds pretty good. The researchers identified eight regions to explore and aimed for 1,000 participants from each. The regions were the UK, France, Spain, Germany, Italy, the Netherlands, Romania and Scandinavia. You might spot that one of these things is not like the others, in that Scandinavia is not a country. But that’s the least of our problems.

In fact, over 37,000 participants were recruited into the study. Over 10,000 weren’t used in the analysis because they didn’t complete the survey; and almost 15,000 weren’t used because their responses were “low quality” (no information available as to how the research team assessed whether a response was low quality or not. Another 4,000 or so were excluded because they’d already hit quota limits – again, not any information as to how they chose which ones to exclude here, I’d assume it’d be the last ones to complete the survey from a specific region, but they don’t say so I could just make up any old shit.

So, the key takeaway is that the analysis was undertaken on about 20% of people who actually participated in the research. This is a bit of a shocker; while drop-outs can be expected, it’s generally not good practice to throw out almost 80% of your dataset.

I have a theory as to why drop-out was so astronomically high, though…

Multiple tests

The survey didn’t just measure these “deviant” behaviour variables (more on these later). It was, by the looks of all the measures they used, a heckin chonker of a survey, which also included the following measures and scales: the technical competency scale; the Emotional Impairment, Social Impairment, and Risky/Impulsive subscales of the Problematic and Risky Internet Use Scale; Adapted Risky Cybersecurity Behaviours Scale; an adapted version of Attitudes Towards Cybersecurity and Cybercrime in Business scale; the Toxic and Benign Disinhibition scales of the Online Disinhibition Scale; a few subscales from the Low Self Control scale; the Minor and Serious subscales from the Deviant Behaviour Variety scale; an adapted scale to measure deviant peer association; parts of the SD4 scale to measure “dark personality traits” of Machiavellianism, Psychopathy, Narcissism and Sadism; and also measures of anxiety, depression, and stress.

That is a lot of questions. A lot of participants would be bound to give up on it. And some are bound to not take a survey seriously if it asks you if you steal cars in one question, then goes on to ask if you’ve ever torrented a movie in the next. Come on. That’s literally this meme.

Still from a 00s anti-piracy advert. Text saying "you wouldn't steal a car".

The bigger problem of measuring so much, as the researchers did here, is that this means they’re conducting a hell of a lot of statistical tests, which raises the probability of something coming up as significant when it isn’t. The further problem is they barely report any results of any of the measures they did. This points to one of two dodgy research practices going on. It either reflects the “file drawer problem”, where research not finding anything does not get published, or “slice and dice”, where they’ll publish findings from the same dataset across multiple studies. Both are forms of publication bias, and both practices are fairly frowned-upon as they’re bad practice and make systematic reviewing of the research difficult.

Let’s look at the bit they actually did publish: the risky and criminal behaviours.

How are the behaviours defined?

The thing is, the definition of the behaviours in the study are pretty bad, too. Many are named in a way which sounds much worse than it is, for example “self-generated sexual images” in fact translates as “sending nudes” (or, as the researchers put it, “make and share images and videos of yourself that were pornographic”). ]

Others are just so broad as to be hilarious. A few favourites:
• Trolling is defined as “start an argument with a stranger online for no reason”.
• Tracking is “track what someone else is doing online without their knowledge”, which covers everything from stalking to looking at your ex’s instagram once in a while.
• Illegal trade of virtual items is buying or trading virtual items, a practice which is literally encouraged by some videogame brands.
• Digital piracy is “copy, upload or stream music, movies or TV that hasn’t been paid for”, which if you’re asking it this way, also includes things like watching youtube or free-with-ads content.

In short, some of the measured behaviours are very badly-defined and it is frankly a miracle that the numbers of young people doing these things are so low.

However, after doing yet more statistical tests, the researchers conclude that some of these behaviours cluster together. Do they?

Clusters of behaviours

The researchers conclude that, thanks to their research, “A significant shift from a siloed, categorical approach is needed in terms of how cybercrimes are conceptualised, investigated, and legislated.” Can they really support clusters of behaviour with their data? As far as I can tell, no.

There are many statistical methods for inferring clusters within data, ranging from techniques which look at “distance” between variables, to multidimensional scaling, to artificial neural networks. It’s a whole branch of mathematics.

The researchers used none of this veritable smorgasbord of algorithms, and instead ran a bunch of correlations as far as I can tell from their reporting. Then they somehow arrived on a cluster of the ones which had the biggest correlations with each other, for example, online bad’uns who are racist are also probably engaged in money laundering and revenge porn. It’s hard to tell exactly what they were doing, because the reporting of the method is very vague, but if they were using any of the multitude of established cluster analysis methodologies, they’d have probably mentioned that. When someone uses principal component analysis, they tell you about it because it’s a massive faff.

Conclusions from their conclusions

In short, what we have is a study with eyebrow-raising sampling methods, running multiple tests on the same dataset (without publishing the results of most of them), with some very weirdly-defined variables, and vaguely-described statistical methods. It’s not replicable. A journal probably wouldn’t print it, and indeed it wasn’t printed in any peer-reviewed journal.

It’s clear that the study authors had an agenda, which is always a bad place to come from when creating research; it generates bad research. Author Mary Aiken outright states the agenda in the Guardian report on the research:

“The online safety bill is potentially groundbreaking and addresses key issues faced by every country. It could act as a catalyst in holding the tech industry to account. The bill sets out a raft of key measures to protect children and young people; however, our findings suggest that there should be more focus on accountability and prevention, particularly in the context of young people’s online offending.”

Guardian, 5th December 2022

This is, essentially, a study which is manufacturing consent for the Online Safety Bill, a disastrously poorly-thought-out bit of legislation. And she’s out there saying “actually, it should probably be made worse because young people are internet criminals”.

This is worrying, to say the least, and yet it is entirely par for the course. The bill is a hot mess, and it needs some form of justification. This study, I don’t doubt, will be trumpeted around for some time, arguing that the delinquent kiddos need protection from themselves.

A huge thank you to some of the lovely people on Mastodon for talking this through with me (and sharing mutual snark). I probably wouldn’t have been inspired to write this many words about this without those chats. I’ve thanked a few who specifically helped in this post and there’s more quality snark and insights in the replies here.


Enjoyed what you read? Consider becoming a Patron or leave a tip

Thinking critically about Lost Connections 5: The target audience

Part 5 of my Lost Connections review. Back to part 1

Lost Connections has been almost universally positively received in the media, and covered in glowing endorsements from the great and good, all over the dust jacket. There’s been reasonably-little criticism, and it’s been draining for those of us, like Dean Burnett and Stuart Ritchie, who have been trying to put forward critiques. We shouldn’t be a little minority, and for a flawed text, there should have been a hell of a lot more of a critical reception.

There’s a few good reasons I can think of as to the homogeneity of the response to the book. The first is that PR machines are powerful beasties. Send a press pack off to the right influencers, with a bit of sample text that they can tweet, and most people are lazy enough to do that. And once it looks like the great and the good are reading it, more will follow.

Another reason is poor science literacy in the media. The reviews in mainstream outlets of Lost Connections have largely been written by journalists. I haven’t seen much evidence of outlets proactively commissioning experts–psychiatrists, academics, mental health doctors, even a humble science journalist–to review the text. Had they done so, I suspect we’d be seeing a lot more mainstream criticism.

And then there’s the thorny influence of personal friendships (I thought, for a second, that I’d made up the term “mateocracy” to discuss this, but it turns out that’s a fairly established phrase, so maybe I should have started part 4 with that anecdote instead of the slightly more embarrassing one). Naomi Klein and Eve Ensler are breathlessly blurbing on the dust jacket, imploring you to read this awesome book. They’re also the first people thanked in the acknowledgements section, described as Hari’s friends. Many of the journalists who have endorsed the book have been colleagues and fellow travellers with Hari over the years, who maybe publicly distanced themselves during the whole plagiarism problem, but the personal relationships are present. When your pal writes a book, you tend to love it. That’s just how friendship works.

Finally, and most crucially, we must look at who this book is intended for. At the end of the last instalment, we touched upon how many of Hari’s proposed reconnections are unsuitable for the people who need them most, and utterly inaccessible. One needs a certain amount of fortune in life to be able to, say, move away to the countryside.

Most of the people who have endorsed Hari’s book have been white, well-off and not disabled. This is no coincidence, because the problems and solutions presented by Hari tend to cater mostly towards this demographic–with the more marginalised people left unmentioned

Race and ethnicity are seldom mentioned, and when they are, it’s usually within the context that these people don’t get depressed so often because they’ve got better family networks. What goes unsaid is that there are racial disparities in diagnosis and access to treatment. For example, black people are 20% more likely to experience mental health problems, and are less likely to seek help than white people.

Disability and chronic illness have long been linked to depression. Take any disability or chronic condition, and google “[disability name] depression comorbidity”. I guarantee you’ll find research showing that if you live with the disability, you’re more likely to be depressed.

For most people, the reconnections proposed in the book are completely unfeasible. How can you reconnect to meaningful work when you cannot work? How can you take the leap towards a meaningful future when you’re unlikely to live long enough to see it through? What about the immense pressure of living under a racist society? How is any of this useful to you when you live under threats of violence every damn day?

These are important questions which are never even raised, let alone answered. The problem is far, far deeper than Hari believes. For most of us, taking a break and turning of our phones isn’t going to help, because inequality is the ginormous elephant in the room. And for many of us, even something big like a universal basic income is a fat lot of good if further underlying inequalities are not addressed.

But the solutions he outlines are appealing as hell to more privileged people, the ones who are on the luckier side of inequality. The book allows us to ignore inequalities, and feel that the solution is a problem of values rather than material realities. You don’t need to check your privilege, just tut-tut at these people who are still clinging to the materialist values that the advertisers force-fed them. Hey, maybe impose a tax on the adverts, or something, they can say, instead of thinking about how they might be complicit in something significantly worse.

Strangely enough, for a book which purports to be proposing radical social change, for the most part, it’s deeply individualistic: a few simple things you can do. It’s a little bit like buying a product in a recyclable packaging instead of plastic: it costs a little more, you feel great about yourself, you get to tut-tut about those who opt for the cheaper version, but ultimately, your individual purchasing choices aren’t doing much for the environment because the bulk of the problem is in corporations’ behaviour, not individuals’.

At the end of the day, Lost Connections is not a particularly useful text for many. It misrepresents much of the problem, sells us inadequate solutions, and, by rights, ought to have flopped. That it didn’t tells us a lot.


Enjoyed what you read? Consider becoming a Patronsupport me on Liberapay, or leave a tip

Thinking critically about Lost Connections 4: Something old, something blue, something borrowed, nothing new

Part 4 of my Lost Connections review. Back to part 1

Once, when I was still a teenager, I thought in great depth about technology and the world. I came to a conclusion–illicit substances may or may not have been involved–that it would be impossible to tell if the world was real, and that we could well all be living in a computer simulation. How groundbreaking!

Upon sobering up, I realised that what I’d been thinking about was a fairly well-discussed point in philosophy and other disciplines, and also the plot of The Matrix. And so, I didn’t write a book espousing these amazing insights I’d had.

The thing with Lost Connections is that rather a lot of it isn’t actually all that bad, if you pretend it’s not about depression, and can also spot what’s actually being discussed. In fact, rather a lot of it is fairly classical stuff which is taught in our earliest introductions to psychology.

Take, for example, the chapters in the book dedicated to childhood trauma. This is not a particularly contested fact, because there’s little to contest. There’s this bloke, I don’t know if you’ve heard of him, called Freud, who is considered the father of psychology, and was all about childhood trauma as a driving force of neurosis–which, these days, we’d be calling anxiety and depression. His work still drastically influences research in this field.

And yet, Freud is mentioned precisely once in the book, in passing, and not in reference to the chapters on childhood trauma. Freud’s large body of work on the importance of childhood trauma goes completely and utterly unmentioned.

Another example: in chapter 14, we’re treated to a story of a man who lost a leg, and became depressed because his work caused him pain. His community helped him become a dairy farmer instead, and he lived happily ever after. Now, this is not rare at all, and is quite well-discussed. It ties in with the social model of disability, a phrase which is mentioned exactly zero times in the text.

This pattern repeats over and over again, with fairly well-accepted research being treated as though it’s outsider mavericks speaking truth to power. Brown and Harris’s Social Origins of Depression, discussed at length in chapter 4, is required reading on many social science degrees, with little to contest. Likewise, Marmot’s Whitehall studies, referenced heavily in chapter 6 (although these pertain to stress, not depression). Likewise, Cacioppo’s work on loneliness.

It’s all fairly basic stuff which is taught to most people whose jobs involve poking around in the human psyche. As a result of this, it finds its way into healthcare: for example, interventions like befriending or mindfulness are recommended for treating depression on the NHS.

None of this is The Secret That Big Pharma Doesn’t Want You To Know about, and it galls me that it is presented this way, when that’s simply not true.

It would be very helpful to put the established names to the ideas which are discussed in the book. A part of the reason for this is for clarity’s sake: it would be nice to know when particular approaches are being mentioned rather than having to sit through pages and pages of extended metaphors before finally figuring out that what’s being talked about is status syndrome (a phrase which never appears in the text of the book!). It’s particularly important for someone with a background in plagiarism to refer to academic concepts by their established names.

“But Zoe,” perhaps you cry at this point, “this is a great jargon-free accessible introduction.” Not true, I reply. It’s a wildly irresponsible starting point for the interested newbie to dip their toes into the waters of learning about the topics in hand, precisely because the ideas are seldom credited to their proper names. It is the beginning and the end of your learning process. How do you learn more when you’re not equipped with the right phrase to fucking google?

Good popular science writing takes an established concept, and breaks it down for a layperson to understand, giving them the information they need to learn more, if that’s something that’s whetted their appetite for knowledge. You give people the language, and explain what the language means. It’s an entrance point where everyone emerges knowing a little more than they had, and knows what to look for next. Lost Connections does the opposite of this, and in its obfuscation, it fosters a dependence on the author to explain what in the name of sweet blue fuck is being talked about.

My background is in psychology, and sometimes it took me a while to follow what exactly was being explained, because it wasn’t a very good explanation, and often misrepresented findings which pertained to something that wasn’t depression, to depression. If I were completely new to the subject, I’d find myself unable to learn more about these topics, because I wouldn’t have the language to seek out more knowledge–and I would probably walk away thinking Hari and the handful of experts he talked to were the only people looking about social and environmental contributors to depression, because Big Pharma has silenced everyone else.

I don’t know if Hari set out to deliberately muddy the waters in order to foster this dependence on the author, to draw attention away from the false dichotomy he has set up, and to make it harder to notice that for the most part, he isn’t telling us anything radical or revolutionary, just things already widely-used in the study of and treatment of mental illness. That might have been an intention all along. It could also go the other way: maybe he himself hasn’t realised just how established much of what he’s outlined is. 

Either way, social and environmental factors aren’t exactly considered a particularly obscure field of research. They’re so well-known they’re embedded in the public consciousness, which is how Blue Monday, “the most depressing day of the year”, could easily be launched by PR companies to sell more holidays. And  social and environmental factors are pretty damn integral to biopsychosocial models of health. As to the question as to why they aren’t more widely-used for therapeutic benefit?

It isn’t because nobody would make any money off it. Once again, as we discussed regarding St John’s Wort, these industries are very lucrative indeed. Same with getting your internet addiction treated at a dedicated rehab clinic, or being tutored in how to meditate.

It’s more that these remedies are often least accessible to those who need it most. Take, for example chapter 18’s case study of how to reconnect with meaningful work, wherein employees of a bike shop took back control and made a workers’ cooperative. That’s great. It’s also about as much use as a chocolate dildo if you’re in one of the professions with high rates of burnout, stress and mental illness, like a teacher or an NHS nurse.

The way depression is treated is because a new job or a holiday cannot be prescribed. It would be very nice if we could change society; I’m a big advocate for communism, but that’s not going to happen any time soon. And so we must make transitional demands, the things which can be achieved: and with mental illness, often that’s about just being able to cope.

To use another ghastly analogy, and this one straight out of Hari’s repertoire, sometimes you need to make the symptoms manageable. The pain can be a message that something deeper is wrong, but that’s no good if there’s no cure for the underlying ailment, or the cure will take a long time to realise. If you have a broken leg, and you know you have a broken leg, and your leg’s in plaster, you’re probably still going to need to take a painkiller to stop your leg from hurting while it mends.

Still, Hari’s proposed reconnections do work for some people. And in our final instalment, we’ll be looking at who they work for the best.

Part 5: The Target Audience >


Enjoyed what you read? Consider becoming a Patronsupport me on Liberapay, or leave a tip

Thinking critically about Lost Connections 3: Defining depression and the false dichotomy

Part 3 of my Lost Connections review. Back to part 1

Right in the title of Lost Connections is a clue as to what it is supposed to be about: “Uncovering the real causes of depression”. That’s nice, but unfortunately, the book does nothing of the kind.

Throughout the text the same problem that I identified in my reading of the extract is present: dozens of things are being conflated with depression, to the end that depression isn’t really the subject of the text at all.

The problems begin right at the introduction. If you’re writing a book on anything, you need to start with a working definition of the thing you’re discussing. It’s really important to get this right. Hari doesn’t, and this part of the book is one of the most sparsely-footnoted sections, setting the tone for the rest of our journey.

Hari states that depression and anxiety are the same thing. We’re treated to an academic saying that “the diagnoses, particularly depression and anxiety, overlap.” Hari then claims that studies which present depression and anxiety as different diagnoses are no longer funded by the National Institutes of Health. The reference for this? An article by Thomas Insel on NIMH’s website which introduces a new project for looking at symptom clusters in a different way, with no mention of funding any other types of project, and literally no mention of anxiety.

It’s true that depression and anxiety often happen in the same patient. I myself have experienced both: sometimes one, sometimes the other, sometimes both at the same time. And this bears out at a population level, with some 50% of patients visiting a doctor with one of the problems also experiencing the other. This does not equate to the two being the same thing.

However, let’s pretend that it’s an established scientific consensus that depression and anxiety are the same thing, or at least, two parts of the same whole. That still renders vast swathes of the book not about depression.

There are two things most frequently conflated with depression. The first is stress, which I mentioned in the review of the extract, but I’ll quote again here, so I don’t have to say it again.

Towards the latter end of the extract, Hari discusses environmental factors, and places a lot of emphasis on stress. While stress is acknowledged to be a contributor to depression, it’s a different kettle of fish entirely and isn’t thought to be the root cause of all depression by anyone. Stress is physical changes to the body caused by your “fight-or-flight” responses going on the alert in response to an external stimulus and just keeping on going. Stress isn’t a medical condition, per se, and it’s often advised that it’s managed by relaxation or just taking a break once in a while. There’s different neurotransmitters involved: depression itself doesn’t tend to have increased levels of cortisol, which is the dangerous thing about stress, and the killer. Because it’s different, stress has different symptoms to depression, though there may be some overlap.

The other thing frequently conflated with depression is something called “negative affect”. Negative affect is not depression. It’s pretty much a fancy way of saying “bad feels”. It covers feelings like anxiety, guilt, shame, fear, sadness, anger, irritability. It’s not a diagnosis, or a sickness, it’s a broad name for the bad feelings. When something is causing negative affect, that is not the same thing as causing depression. Likewise, when something is alleviating negative affect, it is not alleviating depression. Negative affect itself is not clinical: it’s just sometimes a useful thing that psychologists need to measure, although it can be measured and present in depression.

Also mentioned occasionally within the text are grief and substance abuse. Again, neither of these are the same things as depression.

Interestingly, though, what is barely mentioned in the book–indeed, a quick search reveals the word is only used twice–is bipolar. Bipolar features depressive episodes. You can make a better argument for bipolar being a type of depression than, say, stress. However, where bipolar (and its physical component) is mentioned, all we get is “They are a very small proportion of depressed people.” With up to 2% of the population screening positive for bipolar, it seems to me like this ought to be discussed a little more than being mentioned literally twice in a book about depression. Perhaps it’s because there’s stronger evidence for genetics and brain chemistry in bipolar, which is a little uncomfortable when you’re writing a book about how Actually, These Factors Aren’t As Important.

Which brings me to my other quibble with how depression and its treatments are constructed throughout Lost Connections: a false dichotomy.

The way Lost Connections presents it is that there are two routes: an exclusively chemical approach, favoured by scientists, which isn’t right, so that’s taken apart in Part 1; and the real cause (his word, not mine: I remind you the second half of the title is “Uncovering the real causes of depression”) which is largely social and environmental.

That’s simply not true, and has never been true, and isn’t true of the general academic understanding of depression, or its treatment approaches, or… well, anything in particular.

As much as I loathe to use analogies, particularly those comparing a mental health issue with a physical health issue, I’m going to crack one out here, because the public understanding of mental illness isn’t great to begin with, and certainly isn’t going to get any better with books like Hari’s floating around.

Consider the common cold, a virus which we’ve probably all experienced. There’s lots of different things that scientists can do when studying and treating the common cold. A virologist will be most interested in the structure of the virus, and how it affects the body. A pharmacologist will be most interested in developing drugs that treat the symptoms. An epidemiologist would be most interested in how the cold is spread, and developing solutions to stop it spreading. A health psychologist is most interested in seeing how people feel about their colds. An occupational psychologist wants to work on getting people back to work when they have a cold. An immunologist will be thinking about how the cold is fought by the body. A geneticist would be interested in determining if some people are more susceptible to catching colds. They’re all looking at different things, but this doesn’t mean any of the approaches are incorrect, and that there’s a real cause of the common cold. It means that there are many ways of skinning a cat.

It’s the same for depression. When medications for depression are developed, that does not mean that the social, environmental and cognitive factors in depression don’t exist. It means that researchers in a particular field of research are looking at things from their angle.

Returning to Insel’s article, cited at the beginning, the one which doesn’t say that NIH are not funding any research that doesn’t treat anxiety and depression as the same thing, what we actually see is this approach in action. The article is launching the Research Domain Criteria, which brings together various disciplines to “transform diagnosis by incorporating genetics, imaging, cognitive science, and other levels of information”.

The joined-up thinking is also present in the current pathway of care in the UK. The way you’d think of it if you’d only read Hari’s book, you roll into the doctor with your depression, and they drug you up. And I don’t know, perhaps that truly happened to him, and if so, I feel sorry for him, because that’s not good medical care. If you’re diagnosed with depression, you have options on your table: medication is one, there’s also talking therapy, group-based recovery colleges where you learn skills for coping with your depression and support and receive support from others who have experienced depression, doing a low-level CBT course online… And yes, we can talk about problems with waiting lists to access this care; indeed, I could talk about it till the cows come home, because it’s appalling that this care is gatekept by GPs and the waiting lists are terrifyingly long, and so you’re often prescribed medication without other support while you wait. Honestly, don’t get me started on this. But this is not something discussed in Lost Connections. Instead, we’re saddled with a false dichotomy of Just Medical and Actual Things That Work.

Interestingly, psychological interventions are almost entirely neglected in Hari’s work, so the false dichotomy is entirely between organic and social/environmental remedies. Cognitive behavioural therapy, one of the most common talking therapies for depression, is mentioned precisely once in the book, and very near the end, in a paragraph also covering psychotherapy. The rest of the chapter (chapter 20) is devoted to sympathetic joy meditation as psychological change. There’s a lot of evidence for sympathetic joy meditation presented, including an fMRI study of its effect on empathy, its ability to reduce intergroup bias, its effect on altruism. These are all cited in a footnote for a paragraph claiming that 58% of people who don’t have this treatment become depressed again, compared to 38% of people trained in meditation. I think that particular statistic might feature in the other footnoted study, which is from a self-help book called The Buddha Pill, which I cannot access to check its scientific rigour.

There is meanwhile a vast bulk of evidence for the talking therapies which barely even receive lip service–CBT is so well-studied, there’s even a meta-analysis of meta-analyses!

To those with an interest in treating it, “biopsychosocial” is a word which is often used to describe the relationships between the well-documented causes and treatments of depression. In Hari’s book on depression, this word appears precisely twice, and both to bolster claims that doctors are getting it all wrong.

The false dichotomy set up makes things easier for a slightly weaselly author to pull a fast one on us, the readers. And it also helps us hide the fact that actually much of the book isn’t teaching us anything new…

Part 4: Something old, something blue, something borrowed, nothing new >


Enjoyed what you read? Consider becoming a Patronsupport me on Liberapay, or leave a tip


Thinking critically about Lost Connections 2: The evidencing double standard

Part 2 of my Lost Connections review. Back to part 1

I spent many years of my life in varying levels of depression. Sometimes it was really bad: the ugly, messy shit, the lying in bed too tired to cry, too tired to sleep, subsisting off a diet of Haribo, because it was closest to the bed. Much of the time, it was less bad than that: a nagging numbness, a constant feeling of a dragging weight, a listless lack of enjoyment of things that ought to be fun, problem drinking, problem screwing, problems all over the shop. The usual. I knew the depression was there, and I knew there would be ways of helping myself out if I’d just go and see a fucking doctor.

Once, back when I was at uni, the head of department gently suggested to me, as I was falling very much behind on my workload, a course of citalopram to get myself back on my feet. I pretended I’d consider it as my brain told her to go and stuff herself, because I’d read the research and knew they weren’t something that would magically make me an effective worker (which, it was transparently obvious, was her motivation for the recommendation!). I tried a couple of courses of therapy, and they weren’t particularly helpful, being too short, and not being all that suitable for someone who, it turns out, is physically impaired at visualising.

I never took meds, because I had a low opinion of them, and felt like they’d probably do more harm than good for me.

Meanwhile, my epilepsy also took a turn for the worse, and last year I began finally dealing with that by taking an anticonvulsant medication called lamotrigine. Now, lamotrigine works pretty well at controlling epilepsy. It also has another medical function: treating the depression symptoms side of bipolar.

And would you guess what? An unexpected side effect of my epilepsy medications was that my depression has subsided. I’ve felt this strange sense of energy over the last year, a feeling that I am no longer dragging an ever-increasing weight with me wherever I go. I’m getting out of bed every day, and eating meals. I’m not feeling a crushing insurmountable despair. The drug I was prescribed for different reasons has, it seems, also treated the depression I lived with for years.

Now you’ve read my story, are you planning on popping lamotrigine to deal with your depression?


Then you, my friend, are a sensible person who understands on some level that anecdote isn’t data. That the experience of one person is not the same as others. That, no matter how much a story may resonate with you, it’s something you’d probably want to research a bit more independently and chat to a doctor before seeing if the drug I take (which has a small chance that it might make your skin fall off) is suitable for you.

I opened this section with a personal story because I figured we might as well start on a level playing field of evidence.

Evidencing is important, and doubly important if you were exposed as a fabricator and plagiarist.

When writing about science, there are certain standards of evidence that are better than others, and certain types of reference where you’d be chased out of any research institute with pitchforks if you put them down. The former is stuff that’s published in journals or edited academic books: meta-analyses, experiments, rigorous population research and the like. The latter is stuff like popular science books, blog posts, asking someone who you reckon knows a bit more about the subject than you do, personal anecdotes, and so forth. I’m pleased to report that Lost Connections contains some of the former. It also contains a lot of the latter.

We also talk about levels of evidence, because that’s important too. All evidence is not equal. For example, a single study containing 20 people is less good evidence than a study containing 200 people. A study using 200 monkeys is less good evidence than a study using 200 people, if you want your findings to apply to  humans. A study which finds a correlation between one variable and another is less good evidence than a study where you manipulate one variable and measure its effect on the other. One study finding something is less good evidence than 15 studies that find the same thing. A meta-analysis is great evidence: that’s where you put together findings from lots of studies on the same topic to check if the results still hold up. When you are looking at the evidence, these are all things to bear in mind.

The funny thing is, Hari understands this… to a point. He is very clear on all of these points when presenting the evidence that antidepressants don’t work very well, and explains them reasonably well. Now, I’m not going to get into a point by point analysis of the early sections of the book and why I disagree with his conclusion that the drugs don’t work, because I covered pretty much all of my problems with it in my post about the extract, and every point besides the first remains in place for me as I read the text in its entirety.

In the psychological sciences, we often reference “in-line”, so it makes it easier to pick up on the references.  APA formatting is pretty standard for if you’re publishing a psychology study: when you’re referencing a statement, you’d name the authors and the date they wrote what you’re referencing within the sentence. They’re preferable to footnotes as it makes our jobs a lot easier to look at a reference quickly and check if you’re citing a journal article or whether it’s a forum post by DongSmoker6969.

Lost Connections uses footnotes. Lots and lots and lots of footnotes. And sometimes the footnote will go to a proper scientific study. Sometimes it will not. It’s a crapshoot, and I will confess that I did not bother checking every footnote, because I value my own sanity. I often just checked the footnote if something seemed a little bit off to me. So, obviously draw healthy scepticism about what I say throughout, because no, I did not check every single reference. I also found Hari’s approach to evidencing particularly irksome as he frequently refers to researchers by their first names, which makes it a little harder to follow who he’s talking about. For example, in Chapter 7, he often alludes to the work of a researcher called John, and so I spent half the chapter thinking “who’s John, and why should I care?”. Flicking back, I finally found it was John Cacioppo, a researcher so well-known they teach his work about loneliness in Psychology A-Level: his work is familiar to many–and I’ll be writing a bit more about that in a later section of this review.

The thing is, though, Hari’s own talk of high standards of evidencing completely collapses as he writes about his own views on the causes of and solutions for depression. There, we suddenly see conclusions drawn from a study involving animals. We see for-profit companies making a profit on selling the idea of a cause of a sickness for which they provide a cure. We see grand conclusions being drawn from one-off studies involving sample sizes of 20, with very non-representative samples.

A substantial portion of what Hari is proposing as “reconnections” to alleviate depression are alternative remedies, be it lifestyle changes, meditation or, indeed, a herbal supplement.

Hari dismisses pharmaceutical interventions for depression, suggesting they’re no better than the placebo, appealing to the authority of Professor Irving Kirsch. Kirsch gives us a suggestion, due to the side effects of antidepressant medication, of an alternative placebo:

“We could be giving people the herb St John’s Wort, Irving says, and we’d have all of the placebo effects and none of these drawbacks. Although–of course–St John’s Wort isn’t patented by the drug companies, so nobody would be making much profit off it.”

I want to focus on this quote because it highlights a lot of problems rather neatly. Firstly, St John’s Wort is not an inert substance. It actually is effective in treating mild-to-moderate depression, and Mind conclude you can use it as an alternative to antidepressant drugs. It affects serotonin levels. However, being not inert, there are also some dangerous problems with St John’s Wort. It has side effects, like any other medicine. And it also interacts with a lot of other medicines to stop them working as well: important drugs such as contraceptives, HIV medications, blood-thinners and heart medicine.

Another problem with herbal remedies is you might get the dose wrong. While pharmaceutical drugs must be tested very rigorously to find the right doses that won’t kill people or make them sicker, this is not the case for herbal medicines. That’s worrying.

You can take St John’s Wort, and it might work for you. However, as with any drug, you should see a doctor so they can make sure you’re doing it safely and you don’t need anything else, to help you.

Of course, St John’s Wort is a herbal remedy that actually works, but we can divide most alternative remedies into two categories: the ones which do work, and therefore, because they’re doing something, can come with side effects and you need medical monitoring; and the ones which do precisely nothing, like homeopathy, which is literally a sugar pill.

Medical professionals’ tendency to avoid alternative remedies is frequently treated as Big Pharma suppressing the secret treatments that really work. And that’s not true. The truth is that they’re often not recommended because they’ll either not do anything, or have an effect that’s wildly unpredictable and possibly dangerous, and therefore, it’s considered better to go with drugs where we know all about safe dosage, side effects and interactions.

The pharmaceutical industry is, of course, not spotless. It’s pretty evil, and I’m eternally grateful to Martin Shkreli for putting a punchable face to everything I hate about it. However, I am highly surprised by the suggestion that nobody is profiting from St John’s Wort; alternative health industry is every bit as lucrative as the pharmaceutical industry, and every bit as evil.

I would imagine the profit margin for the alternative health industry is comparable to, or possibly larger, than that in the pharmaceutical industry. If you’re selling crushed-up flowers or a meditation tape, you don’t have to spend a large amount of money in research and development, testing, and ensuring your remedy is safe. You might sell less of the product, but you’re going to make a killing. It becomes even more unethical when the product is ineffective, as this gives false hope to sick people, which is diabolically cruel.

In the UK, there is a further issue with recommending alternative remedies over what’s available from a doctor: money, dear boy. Here in the UK, the patient doesn’t have to pay full price for medicines, only a prescription fee. Talking therapies are provided free of charge, as are many other services for mental health. Meanwhile, a month’s supply of St John’s Wort would set you back £15, while an SSRI on prescription would be £8.60. Which, by the way, is still revoltingly expensive, and too many chronically ill people are paying through the nose for their prescription medicines. As a patient, it’s probably cheaper for you to stick with the NHS.

I use St John’s Wort as an example here, although perhaps that is unfair, as there is stronger evidence for its efficacy as a treatment for depression than many of the other things presented in the book: for example, I checked the references on “sympathetic joy” meditation. It’s effective. At some things. None of which are treating depression.

Let me provide some concrete examples. I had particular trouble with Hari’s Cause 6: Disconnection From The Natural World. A substantial portion of the evidence presented pertains to bonobos. Now, I hope I don’t have to tell all of you that bonobos aren’t people, and are instead kind of a nicer version of chimpanzees that fuck a lot. We then have a bit of correlational evidence about mental health evidence upon moving from somewhere green to a city, or vice versa. And finally, we have a bit of experimental evidence from a study where people went for a walk in either a natural or a rural setting which found that the walks in nature were superior. A grand claim, so let’s follow that footnote, shall we? It leads us to a 2012 study from Berman and colleagues. The experiment had a sample size of 20 people, which is pretty small, and all of the sample had diagnoses of the same type of depression, which means they’re not very representative of the population.

Had that been a trial for a drug intervention, nobody on this earth would ever be given that drug, and rightly so. Had this been a test of an antidepressant, Johann Hari himself would have noticed that this was not a very good study, and certainly not anything which ought to influence your medical decisions. We see this throughout. For example, we are treated to a dismantling of the serotonin theory of depression, and shown it was a marketing tactic from pharmaceutical companies. This is fair enough. Then, in chapter 7, we are treated to an explanation of how internet addiction is a real thing and very bad, from someone who works at a for-profit internet rehab clinic.

One cannot have it both ways. One cannot take a critical approach to evidence you disagree with, and then turn around and accept evidence–often of a poorer quality–to something that you do agree with. This is called “cherry picking”, and it’s generally frowned upon–indeed, Irving Kirsch, who features heavily in early chapters of the book dismantling evidence of the effectiveness of antidepressant drugs, worked very hard to minimise the cherry picking effect of pharmaceutical companies’ publication of drug trials on the evidence of effectiveness.

Yes, I’m going to take a second to giggle about a man called Kirsch working against cherry picking, because I’ve just read a book I hate and I need a bit of levity in my life.

By all means, be critical of evidence, any evidence. It’s how science thrives and how medical treatments improve. But this rigorous approach needs to be applied to everything you are writing. The high standards of evidencing introduced by Hari at the beginning do not hold up in the slightest when we’re looking at his proposed alternative model, and his remedies.

The evidencing double standard is a vast problem within the text and pollutes everything within. However, the rot is even worse than that: the entire argument presented in the book rests on a strawman, and that’s what we’ll be looking at tomorrow.

Part 3: Defining depression and the false dichotomy >


Enjoyed what you read? Consider becoming a Patronsupport me on Liberapay, or leave a tip

Thinking critically about Lost Connections 1: An introduction, of sorts

I forced myself to read Lost Connections: Uncovering the Real Causes of Depression by Johann Hari so you don’t have to.  Following my look at an extract published online, I was cursed enough to get my hands on a copy, and I have a lot of feelings about it.

I’ve taken it upon myself to review the book, to encourage the critical thinking that’s sorely lacking in the media reception of the text. Unfortunately, I’ve been struggling with this task on a personal level, and this is because there is so much wrong with it. It’s a fundamentally flawed text on almost every level, and it’s been difficult to even work out where to begin.

While I’ve always accepted environmental factors as a depressant and stressor, I feel like the experience of reading the book provides a very neat demonstration of the phenomenon.

How do you begin to criticise a book which claims to be about depression–it’s right there in the title!–when for a substantial portion of the book the author isn’t even writing about depression? How do you evaluate evidencing when you, a rando blogger who has a day job and a bit of a life, can’t click everything and check it says what the author is saying it says? How can you even begin to criticise the politics of the text? How do you point out that the book is telling us nothing new?

The book is divided into three sections: poking holes in the current model of depression and its treatment; Hari’s proposed real causes of depression; and remedies (“reconnections”) to alleviate the problem. The central thesis is that depression is a symptom of a sick society, and it’s a message that shouldn’t be damped down, but should instead be addressed using Hari’s proposed remedies.

And, sadly, the case isn’t made very well at all.

Of course, the very logic, that depression is a symptom, much like nausea, is flawed. It can be a symptom, much like nausea. That does not mean that one ought never to take a medicine to alleviate it. It also does not mean that it’d go away under the right circumstances. Indeed, something like nausea often needs treatment, because it’s often unhelpful.

The evidence presented in the first section is reasonably strong, although highly biased and by no means holistic. It also sets up a huge strawman: that the only way that doctors think about depression is pure chemistry. That simply isn’t true. Parts 2 and 3 have a different set of problems, being largely horribly-evidenced, with the bits that aren’t being things which we’ve all already known about a thousand times over.

What the book is is this: a self-help book for a well-off Guardian reader who fancies themselves as clever and educated about science. It’s badly-evidenced, largely inapplicable for the people who need societal interventions the most, and is nowhere near as groundbreaking as it thinks it is. It’s an unhelpful text, which is highly annoying to read if you’re someone who has a background in psychology; if I’d been marking it as a submitted paper, I’d probably fail it.

So I suppose what I’ll start with doing is warn you that this review is going to be five blogs long (including this one). I spent a bit of time dividing the methodological and political flaws into broad themes, and these were the things which seemed most egregious to me. These things are:

  • The approach to evidencing and the double standard
  • Conflation of depression with other mental health issues, and emotions.
  • A false dichotomy as to how depression is thought about: science vs the real problem
  • How we’re not actually learning much new from the text

I will also be questioning the largely-positive critical reception of the book, because it’s kind of annoying to me, but mostly because I want to ask questions about why it’s been so universally popular, and that I suspect there’s more at play than merely a well-oiled PR machine. My thoughts on this are circular, with problems feeding into one another: for example, bad evidencing means that bad solutions are presented, but this couldn’t happen if a false dichotomy strawman weren’t set up to present treatment as either drugs or meditation.

I won’t be delving into any point-by-point takedowns, because I am only human, and it would take approximately a million years for me to pick apart every incorrect reference, every misleading claim, every moment where he seems to have fundamentally misunderstood what he’s supposed to be talking about.

I’ll be posting a blog a day this week to cover the whats and whys of this book, so get yourselves comfortable, kids. Before we get started, I’d like to recommend a bit of reading for you, if you haven’t done it already. I don’t want to retread ground that’s already been trodden, and so here are some things which have already been covered:

Why does this matter? It matters because there is a very real risk that people might stop taking their medicine–something which may well be happening. It matters because it might deter people from seeking help from a qualified professional altogether. It matters because we need solutions that work for the people who need them most, not the people who enjoy moralising.

Please do not use Lost Connections to influence your personal health decisions. Speak to a doctor. The care available to you is better than you think. 

If this were some random internet person’s blog with 40 views, I wouldn’t be spending my time writing a takedown. But it isn’t. It’s a widely-promoted book, and with great reach comes great responsibility. I’d have loved to have not written this series, that someone else would have done this instead. But they haven’t, and so, here I go.

Part 2: The Evidencing Double Standard >


Enjoyed what you read? Consider becoming a Patronsupport me on Liberapay, or leave a tip


Five things wrong with Johann Hari’s comeback book that I spotted from the extract alone

Update 22/1/18: I’ve read the whole fucking book now. It’s changed the view I’ve expressed here, very slightly, to “it was a bit worse than I thought”. Read more.

Noted plagiarist and wikipedia editor Johann Hari is back, with a book about depression. Yesterday, the Observer published an extract from the book, Lost Connections, which I presume is an early chapter setting the scene for Hari’s main thesis.

As far as I can discern from the extract, Hari is arguing that the environment is the cause of depression, with neurochemical imbalances not being particularly important, and therefore antidepressants not being very good. Now, I’m pretty critical of psychiatry, and very critical of the tendency towards prescribing antidepressants because waiting lists for talking therapies are so interminably long. However, we can’t have these conversations while we’re spending endless hours clearing the Augean stables of awful science, with nary a river to reroute.

Yes, I am calling Hari’s extract horseshit. The ideas, I might be convinced to agree with in part, but there are serious, fundamental flaws with his methodology which mean that it’s impossible to take anything seriously. I am going to assume his extract is representative of the book as a whole, and highlight some of these major flaws, expanding a thread I made on Twitter. Let’s start with a guided example of how Johann Hari is flat-out making shit up. Talking about changing DSM criteria of depression, Hari says:

So, they responded in a simple way – by whittling away the grief exception. With each new edition of the manual they reduced the period of grief that you were allowed before being labelled mentally ill – down to a few months and then, finally, to nothing at all. Now, if your baby dies at 10am, your doctor can diagnose you with a mental illness at 10.01am and start drugging you straight away.

The bolded part is a complete, total falsehood, which is easily refuted by 10 seconds on google. Search “DSM depression criteria”. Click the first link. Or the second, or any, they all take you to the criteria. Now look at the first fucking line of the criteria: “Depressed mood or a loss of interest or pleasure in daily activities for more than two weeks.” Hari has made up a fact about maternal grief and the teams who work with those who have lost children, to make an imaginary point.

This research methodology seems prevalent throughout the extract, and there’s five key things I can see from reading a few thousand words.

1. [citation needed]

Footnotes and citations are necessary when writing a book based on presenting an evidence-based argument. This goes for anyone, but is particularly important if you’re a disgraced writer who has a history of fabricating things. Citations are completely absent from the published extract of Lost Connections, despite confidently-asserted statistics, for example: “It turns out that between 65 and 80% of people on antidepressants are depressed again within a year.” Where is this from? Who found it? Is it from an Irving Kirsch study, since Kirsch is mentioned in the paragraph above? If so, which Kirsch study? Where can we read it so we can get context for the figure?

When a number is presented, you link to where it’s found. And you make it clear where you found it. Otherwise, you might be misrepresenting it. Or you could have made it up completely. Where there’s no referencing, take any information presented with an ocean of salt.

2. Reliance on a single piece of research

Hari’s argument that antidepressants don’t work relies heavily on the work of a single researcher, Irving Kirsch. Now, due to the lack of citations, I can’t be completely certain that the research Hari outlines is Kirsch and colleague’s 2008 meta-analysis, but I’m going to guess it was because this is the most famous research into the topic.

Kirsch’s meta-analysis is decent, although it is not as definitively presenting that antidepressants don’t work for most people as Hari presents the research. Firstly, Kirsch and colleagues didn’t find that antidepressants don’t work on the majority of people: they found that effectiveness of antidepressants are more effective for severe depression and less effective for mild or moderate depression. That’s a nuanced difference, and it’s unfortunate that it led to so many “drugs don’t work” headlines from a screechy media, and Hari has lapped it right up.

Secondly, other researchers analysed the same dataset and drew different conclusions. Using different statistical modelling, Fountoulakis and colleagues found antidepressants were better than placebo, at all levels of depression severity. Turner and Rosenthal’s interpretation of the data is different to Kirsch’s, suggesting that certain measures can be more important than disappearance of depression, such as quality of life, which has been overlooked in Kirsch’s study, and to be “circumspect but not dismissive” in considering the benefits of antidepressants.

And one more issue is present in Kirsch’s research: it’s not a look at all antidepressants. It examines four drugs, all of the same type: selective serotonin reuptake inhibitors (SSRIs).

3. There’s lots of different antidepressant drugs

“Antidepressant” is a wide category of types of drug, which do different things: Mind’s information lists the drugs, and what they do. Some drugs act on serotonin receptors–the SSRIs which Kirsch studied. Others act on different receptors, or prolong the activity of neurotransmitters, or perhaps make it harder for the body to break the neurotransmitters down: most of these will regulate levels of serotonin, noradrenaline, or both. Then there’s the weirdo drugs which don’t act on serotonin or noradrenaline: the atypical antidepressants, which include drugs like mirtazipine, which doesn’t do any of that, or variants on ketamine, which are increasing in popularity.

Basically, the drugs work differently, and it’s not exactly a secret that different drugs work better for different people: this is one of the first things the NHS tells you in their information for patients.

Hari fails to make the distinction throughout his article, referring only to “antidepressants”, the umbrella term for a diverse range of drugs which act in different ways on the body’s neurochemistry, and which are well-known to affect different people differently. There is no effort whatsoever made to acknowledge that not all antidepressants are the same, and the study he’s citing refers only to one class. This nuance is important. Really, really important.

4. The serotonin hypothesis isn’t as important as you think

Poking holes in the serotonin hypothesis is treated by Hari as debunking the neurochemical basis of depression. That’s a pity, because it doesn’t. There are dozens of theories of depression, both biological and cognitive, and the serotonin hypothesis is but one. It’s also acknowledged it may be caused differently in different people. Genetics, neurochemistry including but not limited to serotonin, interpersonal factors, the environment, the immune system… all of these things and more are believed to contribute to depression.

The dominance of the serotonin hypothesis in the public consciousness is mirrored by Hari’s writing, and presents a grotesquely oversimplified perspective of something which is a lot more complicated than that. The way you’d think it if you were listening to Hari was that science has two cards on the table: a deficiency in serotonin, or the environment, which is a brand-new discovery made by Hari, and definitely not something widely-acknowledged in the scientific literature. This is simply not the case, and never has been. Christ, even a basic A Level in Psychology will teach you that.

Doubt about the serotonin hypothesis does not mean that there’s a vast conspiracy to put people on drugs when really we should be making the world a nicer place. It’s a hell of a lot more nuanced than Hari would have it.

5. Stress and depression aren’t the same thing

Towards the latter end of the extract, Hari discusses environmental factors, and places a lot of emphasis on stress. While stress is acknowledged to be a contributor to depression, it’s a different kettle of fish entirely and isn’t thought to be the root cause of all depression by anyone. Stress is physical changes to the body caused by your “fight-or-flight” responses going on the alert in response to an external stimulus and just keeping on going. Stress isn’t a medical condition, per se, and it’s often advised that it’s managed by relaxation or just taking a break once in a while. There’s different neurotransmitters involved: depression itself doesn’t tend to have increased levels of cortisol, which is the dangerous thing about stress, and the killer. Because it’s different, stress has different symptoms to depression, though there may be some overlap.

Hari conflates stress and depression repeatedly throughout the latter half of his article. This is an enormous problem, because it becomes difficult to follow, and therefore critically appraise, exactly what he’s talking about, and also, to acknowledge that these are different problems, with different solutions, and it seems as though Hari favours the treatments recommended for stress–which may be why he conflates depression and stress so readily.


If this extract is representative for Johann Hari’s comeback book, don’t believe a word he says. The methodology is awful, given how much I spotted just from a few thousand words and a quick read.

It’s a huge shame there’s so much ill-informed nonsense out there, because there are real conversations we need to have about psychiatry and medicalisation, which we can’t have when we’re fighting this crap.

Update 13/1/18: Johann Hari has written a response to criticisms of his extract and research methodology on his blog. He also responds to fact-checks from Dean Burnett, who wrote a very good critical article questioning the conclusions and implications, and Stuart Ritchie, who presented meta-analytical evidence for the efficacy of antidepressants and identified the source of Hari’s “65-80%” figure, which I pulled up in the “citation needed” section of this blog. Stuart’s thread is good and spoiler: the figure came from a self-help book. Stuart has responded to Hari’s response over on Twitter, which, along with his original thread and Dean’s article, are well worth a read.

I’d like to respond to Hari’s response too. Hari neglects to respond to four of the five points I’ve made in this article, opting only to answer point 2: relying heavily on the research of Professor Irving Kirsh. To refute this, Hari got Professor Irving Kirsch to reply. Throughout Hari’s response to Dean, Stuart and I, Kirsch is mentioned or quoted more than 20 times. I don’t think I need to say why this is not a good way of refuting my concerns! I acknowledge that Kirsch thinks his own research is the most solid, and that Johann Hari favours the work of a researcher who unequivocally supports his conclusions. I will say that the criticisms of Kirsch’s meta-analyses still stand, as well as the bulk of meta-analyses conducted by other researchers. I’ll also say that even if a hole had been poked in my concerns about over-reliance on the work of a single researcher, there’s still four other reasons to be worried about the methodology Hari has deployed.

I’m also pretty concerned that there’s only three of us raising criticisms of Hari’s book, and one of us is just some rando blogger (that’d be me!). It’s very telling that media outlets have not been proactively commissioning experts to review the book, as opposed to other journalists. This isn’t just some journalistic circlejerk. There are real-life consequences, and at least one quote from someone considering stopping their meds off the back of the book has been found. To anyone who is thinking of coming off meds I say this: do it under medical supervision. For many antidepressants, you mustn’t just stop taking them, but need to titrate off. You should also be checking in with a medical professional regularly to ensure you are doing it safely, and to see if it’s working for you. This is very important and for pity’s sake DON’T JUST STOP TAKING YOUR MEDICATION.


Enjoyed what you read? Consider becoming a Patronsupport me on Liberapay, or leave a tip

Shockingly bad science journalism in the Guardian

Content note: this post discusses mental illness, mentions self harm, suicide and sexual violence

It’s been a while since I’ve considered the Guardian a decent source of news, but sometimes things get egregious. Yesterday, an article entitled “Mental illness soars among young women in England – survey” was put out, and their reporting… wasn’t very good.

A study was released finding that young women aged 16-24 are at very high risk for mental illness, with more than a quarter of the group experiencing a condition, and almost 20% screening positive for PTSD symptoms. This has all risen since 2007: not just for young women, but across genders and age groups. What, according to the Guardian’s heavy focus of the article, is to blame?

Social media, apparently.

The Guardian’s reporting focuses heavily on how social media is to blame, selectively quoting researchers mentioning social media to the extent that I would love to see what questions they were asked (my personal favourite: “There are some studies that have found those who spend time on the internet or using social media are more likely to [experience] depression, but correlation doesn’t imply causality.”)

Then there’s the case study telling her story of her experience with PTSD and triggers. She talks a lot about film and TV, and the stress of university, and yet somehow her case study is titled “Social media makes it harder to tune out things that are traumatic”. She mentions it briefly in the last paragraph–while still mostly focusing on film and TV!

Now, the reason the Guardian’s twisting of this survey for their own ends is so particularly problematic is the importance of the research. You can download the whole report here, or read a summary here.

It’s quite a well-done survey, a very robust look at mental illness in England, and laying groups who are most at risk. You know me, and how quibbly I can get about published research. This one is actually good. However, it’s worth noting something they didn’t measure in the survey: social media use. This means, of course, it’s absolutely impossible to draw conclusions from the data about social media and mental illness from this research. The survey authors mention that their young cohort is the first to come of age in the social media age, which is true to a certain extent, although I am in an older cohort and came of age in a world where I constantly chatted to friends online, whether I knew them in the meatspace or not. Again, it would be nice if they’d consistently measured online behaviour across studies.

I’ll quote one of the other key research findings here, because again it’s crucial and if you read the Guardian you’d never know about them.

Most mental disorders were more common in people living alone, in poor physical health, and not employed. Claimants of Employment and Support Allowance (ESA), a benefit aimed at those unable to work due to poor health or disability, experienced particularly high rates of all the disorders assessed.

So. Let’s speculate with the results then. What else happened between 2007 and 2014 that might have had a negative impact on people, especially those who are on disability benefits.

I’ll give you a clue. It happened quite soon after 2007, and the young cohort would have come of age into this, as well as more people using Facebook.

One more clue: it rhymes with wobal winancial wisis wand wausterity.

These are young people who have grown into a world with no prospects, with a hugely gendered impact. Of course, once again, it’s just speculation, but it’s slightly more robust speculation than the Guardian’s because they measured benefit receipt and employment status.

As women, a lot of us would have chorused “no shit, Sherlock” upon seeing the results, and seeing how gendered the results are. We deal with more, and it’s even worse if we’re poor.

The Guardian has a bit of a hateboner for social media, and, unfortunately, this has completely blurred its analysis and reporting of what is an important survey that actually found some interesting trends over time, as well as a bleak snapshot of the current realities.