Twitter Triggers

The push and pull between social media and our moral emotions

---

A version of this article has been published by Nautilus

In December 2013, Justine Sacco, then the senior director of corporate communications at IAC, sent out a string of tweets over the course of a long, multi-leg journey from New York to South Africa. She started with sardonic observations about the minor trials of travel—one about a smelly passenger at JFK Airport, another about London’s peculiar food and predictably inclement weather. Then came this one, which she sent shortly before boarding her plane for the final, 11-hour flight to Cape Town:

“Going to Africa. Hope I don’t get AIDS. Just kidding. I’m white!”

As she turned off her phone and settled in to get some sleep, she had good reason to expect that that tweet, like the ones before it, would simply fade away into the hectic, ever-updating ether of Twitter. She had only 170 followers, after all.

But it did the exact opposite. Over the next 11 hours, Sacco became the number one worldwide trending topic on Twitter, as tens of thousands of users across the globe filled her feed with their outrage. When she turned on her phone after landing in Cape Town, she found herself receiving the full brunt of the online community’s capacity for public shaming.

Sacco’s public persona was destroyed. She was fired from her job and saw much of her social circle—both online and offline—wither away.

“I thought there was no way that anyone could possibly think it was literal,” she told journalist Jon Ronson a few weeks after the incident. “To put it simply, I wasn’t trying to raise awareness of AIDS or piss off the world or ruin my life. Living in America puts us in a bit of a bubble when it comes to what is going on in the third world. I was making fun of that bubble.”

Regardless of whether one accepts that defense, Sacco’s case is yet another reminder that nuance is often one of the first victims of viral outrage. Her tweet is just one of countless examples of provocative online behavior being stripped from its context and drawing a seemingly disproportionate social punishment.

To understand how online outrage can so easily bloat to a farcical scale, it helps to zoom out and examine how the architecture of social media exploits our sense of right and wrong.

Attention is often conceptualized as a commodity. The idea has been around for decades, but it’s been drawing more and more support since the advent of smartphones, which have driven the market competition for your eyeballs to new heights. The algorithms that undergird the flow of information on social media are, like the sensationalist print media and incendiary talk radio that came before them, designed to maximize ad revenue by engaging consumers’ attention to the fullest extent possible. Or as novelist John Green puts it, “Twitter is not designed to make you happier or better informed. It’s designed to keep you on Twitter.”

Columbia Law professor Tim Wu calls this “attention harvesting.” And as a business model, it’s extremely lucrative.

Now, the concept of attention harvesting may not come as a surprise. Most of us are aware on some level that those persistent, weirdly personal ads on our computers and mobile devices have a lot to do with how internet giants like Twitter, Facebook, and Google generate their massive revenue streams. What we may not be aware of, though, is exactly how those platforms manage to hold our attention well enough to make their ads so profitable.

Scientists have been at this question for several years, studying people’s activity online and revealing interesting trends as to what makes content eye-catching and more likely to go viral. Jonah Berger and Katherine Milkman from UPenn, for example, analyzed 7,000 articles from the New York Times and found that one of the main determinants of whether readers shared a story via email was whether it was emotionally arousing. In other words, what drove a story’s popularity online wasn’t how surprising or useful it was, but rather how well it stirred people up.

Billy Brady from NYU built on Berger and Milkman’s work by analyzing hundreds of thousands of tweets in an effort to understand the role of moral emotions in social networks. Moral emotions are the feelings associated with our sense of right and wrong, like pride, embarrassment, and outrage. Brady and his colleagues found that tweets about political topics were much more likely to go viral if they contained words that are both morally and emotionally charged, like “evil,” “shame,” “fight,” “punish,” and “faith.”

What’s more, Brady’s analysis revealed that viral political tweets propagated almost exclusively among people with similar ideological leanings, which adds to the pile of evidence that we spend much of our online lives inside echo chambers.

The constancy with which moral-emotional content spreads online has turned much of Twitter—and, by the looks of it, other platforms—into what writer Samuel Ashworth described as “an endlessly self-renewing bonfire of outrage and confusion,” in which preachers whip their respective choirs into frenzies of righteous fury all day every day, and in which a hastily written tweet to 170 followers can become a global scandal. And given how profitable it is, social media companies have little financial incentive to scale it back.

“Morality is something that we see as central to being human,” said Molly Crockett, a psychology professor at Yale, on the podcast Very Bad Wizards. “And I think it’s really important to ask ourselves and to have a serious conversation about how we feel about our moral emotions being used to make a lot of money for tech companies.”

In a recent article in Nature Human Behavior, Crockett argued that the constant triggering of moral outrage on social media not only makes money for tech companies, but also alters how we experience and express moral outrage. Provocative content is ubiquitous, the tools to socially react to it have never been more accessible, and outrage expression often garners positive reinforcement in the form of likes, retweets, and shares. All of this together, Crockett wrote, may cause people to undergo “outrage fatigue,” whereby the intensity of the outrage they feel gradually fades. Or, in light of research showing that venting anger can stoke more anger, it may do the opposite, amplifying successive expressions of outrage.

One final, particularly troubling possibility Crockett mentioned is that social media may uncouple the expression and experience of moral outrage. “And just as a habitual snacker eats without feeling hungry,” she wrote, “a habitual online shamer might express outrage without actually feeling outraged.”

Though there remains much to learn about the interplay between social media and moral emotions, the existing research points to a broader issue: social media companies either ignoring or downplaying the immense influence their products can exert on users’ emotions and on public discourse in general. Perhaps the most glaring example is from November 2016, when Mark Zuckerberg confidently dismissed the idea that Russian hackers exploited his platform to influence the outcome of the presidential election as “pretty crazy.”

The Facebook CEO has since come around on that issue, and now he, his competitors, and all who use their products face a difficult question: To what extent do social media companies have a moral obligation to improve the way we communicate with each other?

“I think most people assume…that if democracy is going to be healthy, we need civil discourse to be healthy also.” said Brady. “And so if there is data that continues to come in that shows that social media is amplifying (the expression of negative moral emotions), then I think they do have a moral obligation.”

Many social media companies now appear to agree with Brady and are making efforts to address some of the concerns that research like his raises. Twitter, for example, recently announced an open call for proposals on how to improve “conversational health” on its platform. And last October, Reddit rolled out a more robust policy for actively monitoring its discussion boards.

But some efforts to improve online communication have been flops, like Facebook’s short-lived feature that allowed users to flag fake news. And in view of the recent Cambridge Analytica scandal, which has exposed serious flaws in Facebook’s privacy policy, many are wondering whether social media companies can even be trusted to monitor themselves, let alone global public discourse.

There are no easy solutions to any of these issues. And the longer they go unsolved, the longer our moral emotions will be subject to monetized technological forces that nobody fully understands. And frankly, that’s an outrage.