Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy by Cathy O'Neill Allen Lane 272 pages ISBN: 978-0-241-2-681-3 12.99
If someone made me commissioner of education for a day, I would make everyone study statistics. Especially journalists, whose job it is to explain to the general population what risk factors mean and investigate how systems are gamed.
In Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, Cathy O'Neil shows the consequences of widespread mathematical fear: very few people understand what's really going on inside those mathematical models, algorithms, and scoring systems. Very few of us can become PhDs, but O'Neil's principal argument is not for better mathematical education -- rather, it's for embedding social fairness in the black box systems we all encounter every day.
O'Neil lists three major elements that characterize these WMDs: opacity; scale; damage. Critical in identifying them is the absence of a feedback loop by which their functioning can be assessed and improved. In one of her real-life examples, a teacher scores 6 on his assessment one year, changes nothing, and scores 96 the next year. Why? What happened? What does it mean, other than that if the school system fires everyone with a score under 50 a possibly excellent teacher might no longer have a job?
O'Neil (who liked to factor the numbers on car licence plates in her head as a child) began as an algebraic number theorist. She did a PhD, and became a tenure-track professor at Barnard College, which shares its mathematics department with Columbia University. She then went to work for the hedge fund D.E. Shaw, where she began to understand the real-life consequences when abstract concepts hit the global economy and caused the housing crash.
Now a 'data scientist', O'Neil launched her blog, mathbabe.org, and went to work for an ecommerce startup. The rise of Occupy Wall Street led her to join the Alternative Banking Group at Columbia to discuss financial reform and...to write this book, a piece of passionate advocacy for fairness and accountability. For WMDs, she writes, tend to punish the poor.
According to O'Neil, understanding of how these models work is thin on the ground, but it's not necessary to understand the details of mathematical theory to grasp the principles of what's going wrong.
The philosopher Karl Popper would easily recognize these WMDs as unfalsifiable hypotheses. In the teaching example above, the hypothesis is that the score indicates something about the quality of the teacher. Because that's nearly impossible to quantify directly, the system -- like the others O'Neil talks about -- relies on proxies that can be easily quantified, such as students' test scores, or the amount those test scores improve in the course of a year.
In other cases O'Neil explores, WMDs determine what ads we see, whether we get jobs, how the financial system treats us, and the pitches our politicians make to us. Yet exactly how these assessments are made is kept secret, so they can't be audited.
There are others who argue that big data can be dangerous, embedding the results of past prejudice into today's supposedly neutral algorithmic decision-making machines. But O'Neil does a particularly fine job of explaining the basis for that contention -- and does it without formulas, in plain, accessible language. People should not be scared of reading this book!
Read more book and film reviews
Originally posted here:
Weapons of Math Destruction, book review: Democracy in the age of the algorithm - ZDNet