AUSTIN, Texas — Artificial intelligence, often touted as a neutral tool, carries the risk of becoming a powerful instrument for authoritarian regimes, according to a leading researcher who spoke at the South by Southwest Conference on March 12, 2017.
Kate Crawford, a researcher at Microsoft Research who studies the social implications of large-scale data systems, delivered a stark warning during her session titled “Dark Days: AI and the Rise of Fascism.” She pointed to a convergence of two troubling trends: the rapid expansion of AI capabilities and a global resurgence of ultra-nationalism and right-wing authoritarianism.
“Just as we are seeing a step function increase in the spread of AI, something else is happening: the rise of ultra-nationalism, rightwing authoritarianism and fascism,” Crawford told the audience.
The core problem, she argued, is that AI systems are often invisibly coded with human biases that align with the hallmarks of fascist movements: demonizing outsiders, tracking populations, centralizing power, and claiming neutrality without accountability. AI, she said, can be a potent tool for achieving these goals, especially when its algorithms are trained on biased data.
As an illustration, Crawford cited research from Shanghai Jiao Tong University in China, where authors claimed to have created a bias-free system that could predict criminality from facial features. The system, trained on Chinese government ID photos, concluded that “criminal” faces were more unusual in appearance than “law-abiding” faces. The underlying assumption, Crawford explained, was that law enforcement would be less likely to trust individuals whose appearance deviated from a perceived norm.
Crawford's concern extends to the use of AI as a “black box” that masks discrimination. She warned that AI could be misused to build registries of populations, which could then be targeted for persecution. She referenced historical examples, including IBM's Hollerith Machine used by Nazi Germany to track ethnic groups and the “Book of Life” that facilitated South African apartheid.
The potential for such misuse is not hypothetical. Crawford noted that as early as 2013, researchers could predict religious and political affiliations on Facebook with more than 80 percent accuracy, and AI has advanced significantly since then.
In the United States, an AI system intended to assist with mass deportations has been under development at Palantir since 2014, a company co-founded by Peter Thiel, an advisor to President Trump. Crawford argued that predictive policing has already failed, citing research showing it leads to unfair targeting and excessive force against minorities.
Path to Ethical AI
To prevent biased systems, Crawford advocates for developing AI that is more accountable and transparent. She believes that by mapping unintended effects, even in complex systems, we can mitigate harm.
“We want to make these systems as ethical as possible and free from unseen biases,” she said. To that end, she founded the research community AI Now, which focuses on the social impacts of AI.
