Twitter | Pretraživanje | |
Arvind Narayanan 29. pro
A new paper has been making the rounds with the intriguing claim that YouTube has a *de-radicalizing* influence. Having read the paper, I wanted to call it wrong, but that would give the paper too much credit, because it is not even wrong. Let me explain.
Reply Retweet Označi sa "sviđa mi se"
Arvind Narayanan 29. pro
Odgovor korisniku/ci @random_walker
Radicalization via YouTube, as widely understood, is when someone watches a few partisan videos and unwittingly starts a feedback loop in which the algorithm gradually recommends more and more extreme content and the viewer starts to believe more and more of it.
Reply Retweet Označi sa "sviđa mi se"
Arvind Narayanan 29. pro
Odgovor korisniku/ci @random_walker
The key is that the user’s beliefs, preferences, and behavior shift over time, and the algorithm both learns and encourages this, nudging the user gradually. But this study didn’t analyze real users. So the crucial question becomes: what model of user behavior did they use?
Reply Retweet Označi sa "sviđa mi se"
Arvind Narayanan 29. pro
Odgovor korisniku/ci @random_walker
The answer: they didn’t! They reached their sweeping conclusions by analyzing YouTube *without logging in*, based on sidebar recommendations for a sample of channels (not even the user’s home page because, again, there’s no user). Whatever they measured, it’s not radicalization.
Reply Retweet Označi sa "sviđa mi se"
Arvind Narayanan 29. pro
Odgovor korisniku/ci @random_walker
Sidenote: the first author has been on a diatribe about the media, even in the thread introducing the paper. It doesn’t undermine the paper by itself, but given that they disingenuously exclude how radicalization might actually work, it… raises questions.
Reply Retweet Označi sa "sviđa mi se"
Arvind Narayanan 29. pro
Odgovor korisniku/ci @random_walker
Others have pointed out many more limitations of the paper, including the fact that it claims to refute years of allegations of radicalization using late-2019 measurements. Sure, but that’s a bit like pointing out typos in the article that announced "Dewey Defeats Truman".
Reply Retweet Označi sa "sviđa mi se"
Arvind Narayanan 29. pro
Odgovor korisniku/ci @random_walker
Incidentally, I spent about a year studying YouTube radicalization with several students. We dismissed simplistic research designs (like the one in the paper) by about week 2, and realized that the phenomenon results from users/the algorithm/video creators adapting to each other.
Reply Retweet Označi sa "sviđa mi se"
Arvind Narayanan 29. pro
Odgovor korisniku/ci @random_walker
Let’s not forget: the peddlers of extreme content adversarially navigate YouTube’s algorithm, optimizing the clickbaitiness of their video thumbnails and titles, while reputable sources attempt to maintain some semblance of impartiality. (None of this is modeled in the paper.)
Reply Retweet Označi sa "sviđa mi se"
Arvind Narayanan 29. pro
Odgovor korisniku/ci @random_walker
After tussling with these complexities, my students and I ended up with nothing publishable because we realized that there’s no good way for external researchers to quantitatively study radicalization. I think YouTube can study it internally, but only in a very limited way.
Reply Retweet Označi sa "sviđa mi se"
Arvind Narayanan 29. pro
Odgovor korisniku/ci @random_walker
If you’re wondering how such a widely discussed problem has attracted so little scientific study before this paper, that’s exactly why. Many have tried, but chose to say nothing rather than publish meaningless results, leaving the field open for authors with lower standards.
Reply Retweet Označi sa "sviđa mi se"
Arvind Narayanan
In our data-driven world, the claim that we don’t have a good way to study something quantitatively may sound shocking. The reality even worse — in many cases we don’t even have the vocabulary to ask meaningful quantitative *questions* about complex socio-technical systems.
Reply Retweet Označi sa "sviđa mi se" More
Arvind Narayanan 29. pro
Odgovor korisniku/ci @random_walker
Consider the paper’s definition of radicalization: "YouTube’s algorithm [exposes users] to more extreme content than they would otherwise." Savvy readers are probably screaming: There is no "otherwise"! There is no YouTube without the algorithm! There is no neutral!
Reply Retweet Označi sa "sviđa mi se"
Arvind Narayanan 29. pro
Odgovor korisniku/ci @random_walker
That’s the note on which I’d like to end: a plea to consider that the available quantitative methods can’t answer everything. And I want to thank the journalists who’ve been doing the next best thing — telling the stories of people led down a rabbit hole by YouTube’s algorithm.
Reply Retweet Označi sa "sviđa mi se"
sea shanty stan account ⛵ 29. pro
Odgovor korisniku/ci @random_walker @BootlegGirl
For my thesis, I'm doing memes as a medium of political discourse, and the first thing I learned is that you're gonna need to focus on qualitative analysis. This is a new/understudied thing, which means stats won't work. We need qualit. knowledge b4 we can go into quant. data.
Reply Retweet Označi sa "sviđa mi se"
Andrew Kemendo 30. pro
Odgovor korisniku/ci @random_walker @andronovhopf
I'd argue we don't have a way to quantitatively study MOST things in the universe People overlook how unbelievably hard it is to instrument a system to produce data Also, humans don't like to be instrumented & IRB's discourage it. There are hard limits to studying humans
Reply Retweet Označi sa "sviđa mi se"
Elle O'Brien, Ph.D. 30. pro
Odgovor korisniku/ci @AndrewKemendo @random_walker
Right-there have been soooooo many times that I’ve abandoned a research question after realizing I couldn’t ethically or adequately address it using any sound methods I knew. And it’s always hard to say no to a question but you have to do work you can live with.
Reply Retweet Označi sa "sviđa mi se"
Sebastian H 29. pro
Odgovor korisniku/ci @random_walker @djw172
This is the problem with political scientists dealing with “charisma” and politicians. There is no easy way to quantify it, so a bunch of them just pretend it can’t be that important.
Reply Retweet Označi sa "sviđa mi se"
Nathan Shedroff 29. pro
This is also the problem with economists ignoring qualitative issues because they're difficult to measure quantitatively and then decrying that those aren't important anyway—producing horrifyingly oversimplified models that get used anyway.
Reply Retweet Označi sa "sviđa mi se"
CurseOfGab 29. pro
Odgovor korisniku/ci @random_walker
A tool is still a tool; whose results are: . . used . often misused . & frequently abused 2 get the answer 1 wants Those who worship at the altar of , to the exclusion of Good 'Ole Thought & , are condemned to Bad Results
Reply Retweet Označi sa "sviđa mi se"
Paul Baclace 30. pro
Odgovor korisniku/ci @random_walker @vboykis
Social media is an open system, so controlled experiments are not possible. This sounds like a theme of Danny Hillis' "Entanglement" article (where he views "the Enlightenment" as formal methods)
Reply Retweet Označi sa "sviđa mi se"
Daniela Ávido 29. pro
Odgovor korisniku/ci @random_walker
Very interesting thread. An anthropologist may be a good partner in this research, I assume the users can be thought of as members of a community, especially if the anthropologist is familiar with digital environments. Just a thought.
Reply Retweet Označi sa "sviđa mi se"