Twitter | Search | |
This is the legacy version of twitter.com. We will be shutting it down on 15 December 2020. Please switch to a supported browser or device. You can see a list of supported browsers in our Help Center.
François Chollet
The problem with Facebook is not *just* the loss of your privacy and the fact that it can be used as a totalitarian panopticon. The more worrying issue, in my opinion, is its use of digital information consumption as a psychological control vector. Time for a thread
Reply Retweet Like More
François Chollet 21 Mar 18
Replying to @fchollet
The world is being shaped in large part by two long-time trends: first, our lives are increasingly dematerialized, consisting of consuming and generating information online, both at work and at home. Second, AI is getting ever smarter.
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
These two trends overlap at the level of the algorithms that shape our digital content consumption. Opaque social media algorithms get to decide, to an ever-increasing extent, which articles we read, who we keep in touch with, whose opinions we read, whose feedback we get
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
Integrated over many years of exposure, the algorithmic curation of the information we consume gives the systems in charge considerable power over our lives, over who we become. By moving our lives to the digital realm, we become vulnerable to that which rules it -- AI algorithms
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
If Facebook gets to decide, over the span of many years, which news you will see (real or fake), whose political status updates you’ll see, and who will see yours, then Facebook is in effect in control of your political beliefs and your worldview
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
This is not quite news, as Facebook has been known to run since at least 2013 a series of experiments in which they were able to successfully control the moods and decisions of unwitting users by tuning their newsfeeds’ contents, as well as prediction user's future decisions
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
In short, Facebook can simultaneously measure everything about us, and control the information we consume. When you have access to both perception and action, you’re looking at an AI problem. You can start establishing an optimization loop for human behavior. A RL loop.
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
A loop in which you observe the current state of your targets and keep tuning what information you feed them, until you start observing the opinions and behaviors you wanted to see
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
A good chunk of the field of AI research (especially the bits that Facebook has been investing in) is about developing algorithms to solve such optimization problems as efficiently as possible, to close the loop and achieve full control of the phenomenon at hand. In this case, us
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
This is made all the easier by the fact that the human mind is highly vulnerable to simple patterns of social manipulation. While thinking about these issues, I have compiled a short list of psychological attack patterns that would be devastatingly effective
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
Some of them have been used for a long time in advertising (e.g. positive/negative social reinforcement), but in a very weak, un-targeted form. From an information security perspective, you would call these "vulnerabilities": known exploits that can be used to take over a system.
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
In the case of the human mind, these vulnerabilities never get patched, they are just the way we work. They’re in our DNA. They're our psychology. On a personal level, we have no practical way to defend ourselves against them.
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
The human mind is a static, vulnerable system that will come increasingly under attack from ever-smarter AI algorithms that will simultaneously have a complete view of everything we do and believe, and complete control of the information we consume.
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
Importantly, mass population control -- in particular political control -- arising from placing AI algorithms in charge of our information diet does not necessarily require very advanced AI. You don’t need self-aware, superintelligent AI for this to be a dire threat.
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
So, if mass population control is already possible today -- in theory -- why hasn’t the world ended yet? In short, I think it’s because we’re really bad at AI. But that may be about to change. You see, our technical capabilities are the bottleneck here.
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
Until 2015, all ad targeting algorithms across the industry were running on mere logistic regression. In fact, that’s still true to a large extent today -- only the biggest players have switched to more advanced models.
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
It is the reason why so many of the ads you see online seem desperately irrelevant. They aren't that sophisticated. Likewise, the social media bots used by hostile state actors to sway public opinion have little to no AI in them. They’re all extremely primitive. For now.
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
AI has been making fast progress in recent years, and that progress is only beginning to get deployed in targeting algorithms and social media bots. Deep learning has only started to make its way into newsfeeds and ad networks around 2016. Facebook has invested massively in it
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
Who knows what will be next. It is quite striking that Facebook has been investing enormous amounts in AI research and development, with the explicit goal of becoming a leader in the field. What does that tell you? What do you use AI/RL for when your product is a newsfeed?
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
We’re looking at a powerful entity that builds fine-grained psychological profiles of over two billion humans, that runs large-scale behavior manipulation experiments, and that aims at developing the best AI technology the world has ever seen. Personally, it really scares me
Reply Retweet Like
François Chollet 21 Mar 18
Replying to @fchollet
If you work in AI, please don't help them. Don't play their game. Don't participate in their research ecosystem. Please show some conscience
Reply Retweet Like