Reddit Disinformation & How We Beat It Together - Smarter Every Day 232

Hey, SV Dustin! Welcome back to Smarter Every Day. Unfortunately, now is the time for the video about disinformation on Reddit, the front page of the internet. It's been documented by both the European Union and the United States of America that Russia, Iran, and China are all spreading disinformation online about the coronavirus pandemic, which, to be clear, can kill you. If you don't know the right things about the pandemic, you can die. So why would they do that? This is the fourth video in a series on disinformation; we've already covered YouTube, Twitter, and Facebook.

Okay, Reddit does not want disinformation on the platform. In fact, they're doing a ton to try to fight against it. But the question is: how do you fight disinformation without giving up what makes Reddit so powerful—a place to freely exchange ideas with other people anonymously? Today, we're going to talk about this tension and, more importantly, how it affects you and I as Reddit users. To learn about this, let's go get smarter every day and talk to the experts. We're going to speak to Chris Slo, the Chief Technology Officer at Reddit, and then we're going to go to Rene De Resta at the Stanford Internet Observatory. We'll talk on the phone with John at the Oxford Internet Institute and then Jeremy Blackburn at the iDrama Lab. It's a group of scientists and academics that use math and science to try to detect this stuff.

All right, first off, let's go to actual Reddit headquarters—something I've always wanted to do. We're going to meet Chris Slo, Chief Technology Officer, and get this conversation started. I guess let me start here: yeah, who controls Reddit? Somebody controls Reddit. That's a good answer, that's a good answer. But I mean, Reddit is, it's the front page of the internet! Like, you go to Reddit, and whatever's on that front page influences culture for that day, because I think the actual answer is like everybody who uses Reddit controls Reddit in some way. You want people to upvote or downvote? Yeah, it's a function of, you know, if the community likes that content.

And so yeah, the next level is the moderator. So every community has its own set of moderators, and they enforce their own set of rules. They get a chance to set the tone of their community, so they can take down anything they want directly. They can just remove it, and they can set the rules to be as strict or as loose as they want to be. The underlying important unit is not the content or the user, but the communities that are built on the platform. Is coordinated inauthentic behavior a problem on Reddit? Uhm, I mean, it's definitely a problem. We constantly work to prevent it. I would say like there will always be examples of things that hit the front page that potentially shouldn't be there—that's kind of like a nature of the platform, right? Because you're dealing with a bunch of people.

An extreme example would be, yeah, clickbait is a thing—like titles that are suggestive that will get you to click through or like, you know. But what we try to do is, there are two parts to it. One is users have an opportunity to reconsider their vote after seeing the content. Secondarily—and you know they have the downvote available to them as a way to kind of degrade any inauthentic behavior that happens. The other side of it is like, you know, we put a lot of work over the last decade and change into maintaining a certain sanctity of the vote. You know, the joke I usually use is that we count every vote, but that doesn't mean that every vote counts, right? And so we have to be able to come up with, based on behaviors, knowing that there's some adversary on the far side who's trying to basically game our system, have a set of somewhat opaque ways to hide what counts and what doesn't. But at the same time, make sure that we're not, you know, throwing the baby out with the bathwater, right? Like we do want to have things that are popular manifest themselves as popular. But during that very early formative time, that really first couple of votes is what's really critical, right? That's where there's a lot of scrutiny around making sure that, you know, there's not like a game going on where, you know, it's me and my 50,000 clones that are all coming from the same computer and the same IP address or account towards that photo.

You can think of it as a kind of a tiered system for how we deal with the way content appears on Reddit, at least for like the enforcement side of things or like the removal side of things. The last line of defense is us—the admins, as we are referred to on the platform. Our job is to enforce site-wide policy violations like copyright violations, anything that violates our content policy, you know, harassment, spam—all that great fun stuff. Our appearing in a place is like the last line of defense; it's like you're sending the National Guard to clean up like a giant mess.

Okay, so like you're saying when Reddit employees get involved? Yeah, I mean, that's a rare thing, is what you're saying? That's a relatively rare thing. Just a personal observation: Reddit seems to be way more hands-off when it comes to this stuff than other social media platforms, but it's also way more community-focused. Rene De Resta has a ton of experience studying disinformation online, so I asked her what she thought about this community-driven approach.

First, there's an interesting idea there, which is that the community should decide what kind of content it wants to tolerate, right? And so you do see things like—I think there's a subreddit where you can only post a picture of a cat, and if you post a picture of a dog, like, you know, it's deleted and banned. And nobody goes in there and screams that they're being censored because they couldn't post the dog into the cat subreddit. So it's interesting in light of like the moderation challenges faced by like Twitter and Facebook, where there's this expectation that one moderation standard is fit for the whole community.

I do think Reddit is an interesting experiment in seeing how that much more hands-on moderation kind of activity works. And then also, when the community reports these weird accounts coming in, there is a little bit more of like, you know, you as a member of that community have a better sense of like where that uncanny valley is—where like the content's not quite right. The person typing the comment gets it just a little bit wrong, they don't understand the community.

Yeah, exactly. And so there is like—I do believe that Reddit has a unique that community-moderated kind of point of view for detecting anything from disinformation to harassment. At the same time, we do see these accounts get through, right? And so the question is also, what's their top-level strategy for managing and detecting and being proactive, recognizing that their platform is a target for state-sponsored actors? Is it enough to rely on community mods, or is there also something in place to have a top-level view that's doing a little bit more of the investigative type work that Facebook and Twitter are doing?

So when speaking to Rene and Chris, we talked about upvote manipulation, downvote manipulation, brigading. There's all kinds of manipulation that can take place on Reddit, and those discussions are pretty long, so I'll leave them over on the second channel at some point. But for now, there was one little nugget that fell out of the conversation with Chris from Reddit that I think other Redditors might appreciate. One of those size and variant things on Reddit has been the upvote to downvote ratio consistent over 14 years. I don't know what this even means—seven to one? There's been seven upvotes consistently to every downvote cast on the site.

That's interesting. I don't really get it, but it's a thing that's always happened. So I'm sitting here with Jeremy Blackburn; he's from Binghamton University in New York, the top public university in New York, and he runs this thing—or at least participates in this thing—called the iDrama Lab. Could you please explain the iDrama Lab for me?

Yeah, sure. So iDrama Lab, we're a group of international scientists, and we specialize in getting a large-scale, a very wide and high-level view—holistic view of the internet as a whole. So we don't just look at Twitter or just Reddit or just Facebook; we look at it all. We focus really on understanding the connections between them—the hidden connections between them and other chunks of the web. This is really important; we think we're, you know, we're pretty good at it. It's our niche, and we do our job there.

Got it. So let me just ask you this straight up, then: is there some type of large-scale coordinated inauthentic activity on Reddit? Is that a thing?

Yes, absolutely! Really, the internet has a long history of people pretending to be something they're not, and the internet kind of enables that. So no, I don't want to say that everybody you see that's acting a little bit weird is some kind of bad actor. There are people that are new to communities; they have to learn the rules. You know, maybe they are interested; they just haven't learned the culture yet. But it's not crazy to be on the lookout for this stuff—it does happen. There are active campaigns to abuse social media and influence opinion.

So what are you seeing?

We're seeing that there's a lot of content going on right now, especially with the coronavirus. Unfortunately, we see that a lot of this stuff appears to be effective still. There’s not—we haven't noticed yet, at least, any kind of news, particularly new strategies that are being used. We're still seeing kind of the same basic strategies that were 2, 3 years ago because they still work.

I guess one thing I didn't get from the Reddit interview is they were kind of top level—Chris was kind of top level; he didn't really dive down and say, "Oh yeah, user surf ninja 385 did this." Do you have like a specific example of a time when somebody's trying to manipulate Reddit?

Yeah, so Reddit's actually put out their stuff. There's a Reddit.com wiki, Suspicious Accounts, and on that site, what they did—and actually, I give them a lot of credit for doing this—following the 2016 election, Reddit acknowledged 944 trolls that have been active on their platform. They did a very detailed transparency report in early 2017, or maybe late 2017, but they did a really detailed transparency report where they actually go through and you can see that the user "rubinger," for example, has a karma of 99,400, it looks like. Reddit, for a while, had left these accounts up, and so you could actually kind of click in there and see what they were—where they were posting to. That was where you could start to see that they were posting to some of the funny meme sites, some of the black community subreddits, some of the far-right community subreddits. To their credit, Reddit did actually make this visible to the public; they kept them up there for quite some time, which is an interesting choice.

It is because Facebook took it all down, even though they flagged the accounts. When I first heard that Reddit was leaving up access to both posts and comments from known Russian agents who were trying to manipulate us, I thought that was a really odd decision on Reddit's part. But when I went to the public link Reddit security provided of all the known troll accounts, I realized that this was actually valuable. It lets you study the context of the comments and the posts to understand how these disinformation campaigns work.

At the time this particular troll operation was underway, the strategy seemed to be an attempt to use the hot-button issues of the day in order to rile up Americans on both sides of any argument, all the while making sure to cycle in a good helping of low-effort cute animal repost, to try to throw Reddit security and other users off the scent. At the time of these posts in 2015 and 2016, they clearly focused on racism and police issues. They would go from subreddit to subreddit, hoping that it would explode and make everyone fight and create as many casualties as possible.

The comments were interesting; they did one of two things. They were either camouflaged, trying to convince other Redditors that they were real people, or they were inflammatory, hoping to nudge people towards hating each other. Although there were a few outlier accounts that seem to be focused more on comments, if you look at the karma scores of the vast majority of these accounts, the post karma was, on average, 40 times larger than the comment karma, so inflammatory posts seemed to have been the weapon of choice.

That being said, even though the comments seem to appear less influential, let's take a moment and try to analyze them to understand the cumulative effect that they did have on the conversation. I'm on the phone with John from the Oxford Internet Institute, and he's been researching disinformation on Reddit, and he's going to talk to us a little bit about what he sees in the data.

So you have a report on "Robo-Trolling," which is the magazine NATO Strategic Communication Center makes quarterly about online disinformation. You've got a graph down here that I was hoping you could explain to us.

Sure, I'll give it a try. So this was looking at what happens to the Reddit conversations after some manipulations, so as you've got a Reddit thread sort of ticking along over time, you think of this left to right in each of those graphs. You know, another comment, another comment, another comment, and then in the middle, on the red dotted line, we have when a fake account, a Russian IRA troll account, has made a comment—they've injected something into this thread.

What we have is the baseline, which is tracking along at zero; it's essentially a control thread—so another thread that hasn't been manipulated. And then what we see in each of the three lines is how the manipulated conversation diverged. So what happened?

Where was the change? Cognitive complexity—what does that mean?

So a higher cognitive complexity score is an acceptance that multiple viewpoints can be true even if they're in opposition, whereas a low complexity score is singular—very single-minded, "I'm right and there's nothing else that could possibly be true." Okay, an identity attack, I assume that means like people start attacking each other based on who they are, like gender, religion, things of that nature?

Yeah, exactly. Attacking somebody because of their identity. So a higher toxicity score suggests that it is more aggressive, more confrontational, and it's more likely to lead the recipient of the message to leave the conversation.

And this is what I think is really interesting: we see in all three of these measures the conversations did change. So they became, on the left-hand graph, they became less diverse in the number of opinions; they became more polarized, single viewpoints. And in the middle, there was a sort of a short spike in the number of identity attacks—although this did return to baseline afterwards. And then on the right, we see a sort of sustained rise in the level of toxicity in the conversation.

So this is just one comment that was able to make the entire conversation diverge? Is that what I'm seeing here?

Yes, so this is just at the one comment level, so each one individual comment did create a measurable change in the nature of the conversation. The magnitude of this change is small but measurable, but if you scale this up to thousands of comments over the whole platform, then suddenly that starts to have a bigger impact.

Wow, it's right, like this is data—this is pretty clear. If you can make a conversation into a two-sided thing instead of a three-sided or four-sided—whoever-sided thing, it becomes much easier to control it, right? If there's two very clearly opposing points of view, black and white, now it's easier to control the narrative because people get locked into one of those points of view or the other. Using this same data provided by Reddit, Jeremy and the iDrama Lab team were able to create a report with fascinating implications.

This graph shows weekly troll activity plotted over many years. This yellow line is what we're interested in because it's Russian troll activity on Reddit. What you can see here is that there was a bunch of activity by those accounts in the second half of 2015, and then the activity seems to go dormant. Then it creeps back up and spikes in the fall of 2016. The initial activity in 2015 seems to be karma farming—they would free boot content from other places on the internet and post it for karma on Reddit. This post, for example, looks awfully familiar, and so does that pistol, by the way. Hmm, weird.

After this long period of what is thought to be karma farming to give the username more credibility, the troll activity seems to slow down for a while. And then, when the moment is right— in this case right before the 2016 election—a percentage of these accounts surge into action and try to influence society. So it's pretty clear what's happening; they're creating the accounts and then they're grooming them to become effective social media weapons which can then be deployed at the exact moment when they might be the most effective. Think about where we are right now—a global pandemic. The early stages—we're coming up on an election; everybody's kind of tense at the moment. If you had these social media weapons, these accounts parked—when would you deploy them?

What is a home run for a person or entity that would try to run a misinformation campaign on Reddit? What would be their goal?

I think that the most achievable goal is just to cause chaos—to cause polarization and to keep people from going towards a common goal—to deliberately drive people apart. So you see this type of behavior elsewhere on Twitter. It's been very well explained that there were state-sponsored actors taking both sides of the same argument, right? They didn't have particular goals; they just wanted to cause problems.

Jeremy showed me something that made me realize we're only scratching the surface with this series. We're looking at individual platforms here, right? Twitter, Facebook, YouTube, Reddit—they all have their own vulnerabilities. He showed me data and explained that these campaigns are coordinated attacks that span the whole internet, and it appears that Reddit is being used as a way to scale disinformation campaigns from smaller places on the internet up to more mainstream outlets. The military is aware that this is a big deal; they know it's a big deal, and nobody really has a map of the battlefield—a full map of the battlefield. Yes, Facebook has their, you know, their map of, you know, Facebook land, and Twitter has their map of Twitter land. But nobody knows what's going on in other parts of the internet, and it's hard to win a war without a map. I don't know if it's ever been done.

It's hard to win a war without a map. That's pretty good, dude. So what would you say to the people on Reddit right now, that they're using the platform and they're starting to question what they see? You know, they're seeing maybe, "Why is this kind of stuff at the top?" And if they do identify some type of inauthentic or coordinated behavior, like what do we need to tell them to do?

Report it! Definitely! No click the report button— that's what the report button is for. Okay, we look at—we look at reports; we look at anything that you send to us. We do try to look at—the more people who are reporting stuff and identifying things that look a little off, that's the thing that really helps us to find it and localize it. So that's a signal that you see—that's a signal we can use. And of course, like, you know, the obvious question is like, what do we do about people who abuse the report button? Of course people will abuse the report button; of course we know how to deal with that.

I think, you know, the broader the base of people who are reporting stuff, who tell—who think things look wrong, the more signal we have. It goes back to the only way to scale up—users with more users. Like, you know, we can either have a gigantic police force and like a bunch of AI-equipped cameras that watches everybody at all times—what could possibly go wrong? Or we can let the, you know, let the neighborhood watch take over.

I would rather live in a society that has a neighborhood watch rather than a bunch of cameras on every street pool. So Chris is talking about this neighborhood watch, but I was very surprised when I looked at the data for myself, and I realized that Reddit is very proactive when it comes to taking down bad actors on the platform. In the fourth quarter of last year, there were about five and a half million reports of potential content manipulation on Reddit by users. In that same period, Reddit removed almost 31 million pieces of content and sanctioned almost 2 million accounts for content manipulation. To be clear, that means Reddit has taken down roughly six times more content and accounts than have been reported.

If Reddit overreacts, if they, you know, get too authoritarian, if you will, that can push people even further away from what we might consider mainstream or normalcy type of stuff because then they feel persecuted and stuff like that. So Reddit has a difficult job; you know, maybe they could do a better job. But, you know, I don't envy that job either.

You're pretty happy with what they're doing?

I don't know if I'm happy with it, but I don't—I also don't know if I have a better, easily implementable answer. There are definitely worse things they could be doing. So clearly, Reddit admins are proactively working to decrease the influence of bad actors on the platform. I find this interesting because when I started this study, I was under the impression that Reddit was too hands-off. But I'm starting to understand that it's more of a fine line that they have to walk.

Also, and this is more personal: when I first started studying this stuff and I started seeing these disinformation campaigns around me, I think something bad happened to me. My reaction to this new reality was flat wrong; I started to irrationally see trolls everywhere. "Oh, that person feels strongly and disagrees with me? Must be a troll! Oh, look, a pot shot at me in the comments, that's a troll! Ignore that person!" And it started to feed this us versus them narrative from a different angle—the angle where I think anyone that disagrees with me must be a troll.

At the risk of confusing you, I'm going to point this out: the troll's first play is to make you hate your online brother. The troll's second play is to make you think your online brother is a troll. If you go through the comment database, you can see trolls meta-joking about the existence of trolls. So what would you say to the normal user of Reddit—the person who goes there to find funny memes, the person that, you know, their kids are sending them links from Reddit? What do you say to that person?

Well, I think we don't want to create a feeling of paranoia, right? You don't want people to think like everybody I talk to on the internet is a pseudonymous troll operating out of some other country. But at the same time, I think there's the need for a healthy skepticism. If somebody's posting stuff that really makes you feel riled up or really makes you feel strongly, you know, kind of take the extra two seconds to do the check. You know, where did this come from and why do I feel compelled to share it? Knowing that trolls appeal to emotion is very important because they are real.

So here's what I'm going to do: what if I were to assume everyone was real? Hear me out on this—not like a username to interact with or a comment to interact with, but like actual people. And I make that primary, and I just go forward on the internet as if they're real? When I'm more kind and more loving, it seems to be more toxic to trolls. Think back to John's graphs! If I see someone trying to reduce the cognitive complexity of a conversation, I'm going to add nuance and try to expand the conversation—that's toxic to trolls. If I start to see identity attacks in a thread, I'm going to call it out in a non-toxic way with kindness and love, which is also toxic to trolls. If I de-escalate the rhetoric and try to make it less aggressive and less confrontational, that's toxic to trolls.

And plus, this is how I want to interact with people in real life! Reddit has a very difficult job; they've got admins, mods, they have other tools in place to try to thwart the bad guys. But for me, the real battle is with me. Every interaction I have on Reddit is the one I can do something about, and I want to be a good guy. I want to upvote people that are doing the same. So if I see you out there, decreasing the toxicity or trying to expand the conversation, you're going to have my upvote.

I want to say thank you. Thank you for watching this big series that I did on misinformation. We had YouTube, Twitter, Facebook, and now Reddit. There's even a cyber warfare video before that. This was a challenge, and I want to say thank you to the patrons for letting me mentally have the freedom to do this—people that support at Patreon.com/SmarterEveryDay. That frees me up; I don't feel like I'm tied to the algorithm. I can just explore the things that I genuinely want to explore, and I thought this was an important topic. If you feel like this series has earned your subscription, I would greatly appreciate that. There's even a little notification bell—if you click that, you'll be notified when I upload. But if not, no big deal.

If you want to discuss this, we'll be doing it over at r/. I'll leave links down below for all the references that we talked about, and other than that, I am Destin. You're getting smarter every day. Thank you for the support; have a good one. Bye!