‘Technology is never neutral’: why we should remain wary of machine learning in children’s social care – Communitycare.co.uk
(credit: Pablo Lagarto / Adobe Stock)
On 1 February 2020, YouTuber Simon Weekert posted a video on YouTube claiming to have redirected traffic by faking traffic jams on Google Maps. The video shows Weekert walking slowly along traffic-free streets in Berlin, pulling a pile of second-hand mobile phones in a cart behind him and Google Maps generating traffic jam alerts because the phones had their location services turned on.
Weekerts performance act demonstrates the fragility and vulnerability of our systems and their difficulty in interpreting outliers, and highlights a kind of decisional blindness when we think of data as objective, unambiguous and interpretation free, as he put it. There are many other examples of decisional blindness relating to drivers following Google Maps and falling off cliffs or driving into rivers.
Google has the resources, expertise and technology to rapidly learn from this experience and make changes to avoid similar situations. But the same vulnerability to hacking or outliers applies to the use of machine learning in childrens social care (CSC) and this raises the question of whether the sector has the means to identity and rectify issues in a timely manner and without adverse effects for service users.
Have you ever had the experience of asking the wrong question in Google search and getting the right answer? Thats because of contextual computing that makes use of AI and machine learning.
At its heart, machine learning is the application of statistical techniques to identify patterns and enable computers to use data to progressively learn and improve their performance.
From Google search and Alexa to online shopping, and from games and health apps to WhatsApp and online dating, most online interactions are mediated by AI and machine learning. Like electricity, AI and machine learning will power every software and digital device and will transform and mediate every aspect of human experience mostly without end users giving them a thought.
But there are particular concerns about their applications in CSC and, therefore, a corresponding need for national standards for machine learning in social care and for greater transparency and scrutiny around the purpose, design, development, use, operation and ethics of machine learning in CSC. This was set out in What Works for Childrens Social Cares ethics review into machine learning, published at the end of January.
The quality of machine learning systems predictive analysis is dependent on the quality, completeness and representativeness of the dataset they draw on. But peoples lives are complex, and often case notes do not capture this complexity and instead are complemented by practitioners intuition and practice wisdom. Such data lacks the quality and structure needed for machine learning applications, making high levels of accuracy harder to achieve.
Inaccuracy in identifying children and families can result in either false positives that infringe on peoples rights and privacy, cause stress and waste time and resources, or false negatives that miss children and families in need of support and protection.
Advocates of machine learning often point out that systems only provide assistance and recommendations, and that it remains the professionals who make actual decisions. Yet decisional blindness can undermine critical thinking, and false positives and negatives can result in poor practice and stigmatisation, and can further exclusion, harm and inequality.
Its true that AI and machine learning can be used in empowering ways to support services or to challenge discrimination and bias. The use of Amazons Alexa to support service users in adult social care is, while not completely free of concerns, one example of positive application of AI in practice.
Another is Essex councils use of machine learning to produce anonymised aggregate data at community level of children who may not be ready for school by their fifth birthday. This data is then shared with parents and services who are part of the project to inform their funding allocation or changes to practice as need be. This is a case of predictive analytics being used in a way that is supportive of children and empowering for parents and professionals.
The Principal Children and Families Social Worker (PCFSW) Network is conducting a survey of practitioners to understand their current use of technology and challenges and the skills, capabilities and support that they need.
It only takes 10 minutes to complete the survey on digital professionalism and online safeguarding. Your responses will inform best practice and better support for social workers and social care practitioners to help ensure practitioners lead the changes in technology rather than technology driving practice and shaping practitioners professional identity.
But its more difficult to make such an assessment in relation to applications that use hundreds of thousands of peoples data, without their consent, to predict child abuse. While there are obvious practical challenges around seeking the permission of huge numbers of people, failing to do so shifts the boundaries of individual rights and privacy vis--vis surveillance and the power of public authorities. Unfortunately though, ethical concerns do not always influence the direction or speed of change.
Another controversial recent application of technology is the use of live facial recognition cameras in London. An independent report by Essex Universitylast year suggested concerns with inaccuracies in use of live facial recognition, while the Met Polices senior technologist, Johanna Morley said millions of pounds would need to be invested in purging police suspect lists and aligning front- and back-office systems to ensure the legality of facial recognition cameras. Despite these concerns, the Met will begin using facial recognition cameras in London streets, with the aim of tackling serious crime, including child sexual exploitation.
Research published in November 2015, meanwhile, showed that a flock of trained pigeons can spot cancer in images of biopsied tissue with 99% accuracy; that is comparable to what would be expected of a pathologist. At the time, one of the co-authors of the report suggested that the birds might be able to assess the quality of new imaging techniques or methods of processing and displaying images without forcing humans to spend hours or days doing detailed comparisons.
Although there are obvious cost efficiencies in recruiting pigeons instead of humans, I am sure most of us will not be too comfortable having a flock of pigeons as our pathologist or radiologist.
Many people would also argue more broadly that fiscal policy should not undermine peoples health and wellbeing. Yet the past decade of austerity, with 16bn in cuts in core government funding for local authorities by this year and a continued emphasis on doing more with less, has led to resource-led practices that are far from the aspirations of Children Act 1989 and of every child having the opportunity to achieve their potential.
Technology is never neutral and there are winners and losers in every change. Given the profound implications of AI and machine learning for CSC, it is essential such systems are accompanied by appropriate safeguards and processes that prevent and mitigate false positives and negatives and their adverse impact and repercussions. But in an environment of severe cost constraints, positive aspirations might not be matched with adequate funding to ensure effective prevention and adequate support for those negatively impacted by such technologies.
In spite of the recent ethics reviews laudable aspirations, there is also the real risk that many of the applications of machine learning pursued to date in CSC may cement current practice challenges by hard-coding austerity and current thresholds into systems and the future of services.
The US constitution was written and ratified by middle-aged white men and it took over 130 years for women to gain the right of suffrage and 176 years to recognise and outlaw discrimination based on race, sex, religion and national origin. Learning from history would suggest we must be cautious about reflecting childrens social cares operating context into systems, all designed, developed and implemented by experts and programmers who may not represent the diversity of the people who will be most affected by such systems.
Dr Peter Buzzi (@MHChat) is the director of Research and Management Consultancy Centre and the Safeguarding Research Institute. He is also the national research lead for the Principal Children and Families Social Worker (PCFSW) Networks online safeguarding research and practice development project.
The rest is here:
'Technology is never neutral': why we should remain wary of machine learning in children's social care - Communitycare.co.uk
- Color Sensor Array and Machine Learning Classify Tea and Measure Antioxidant Levels - Bioengineer.org - November 15th, 2026 [November 15th, 2026]
- Data Modality Determines Machine Learning and Deep Learning Success in Breast Cancer Prediction - Bioengineer.org - November 15th, 2026 [November 15th, 2026]
- Machine Learning Identifies Varied Signal Patterns Across Diverse Tremor Interventions - Bioengineer.org - November 15th, 2026 [November 15th, 2026]
- Machine Learning Models Classify Frailty in Adults With Mild Cognitive Impairment - Bioengineer.org - November 15th, 2026 [November 15th, 2026]
- Network Features Boost Machine Learning Predictions of Food Safety Risk Severity - Bioengineer.org - November 15th, 2026 [November 15th, 2026]
- Machine Learning Links Metabolic and Inflammatory Shifts to Age-Related Hearing Loss - Bioengineer.org - November 13th, 2026 [November 13th, 2026]
- Evaluation of machine learning-based prediction of pregnancy complications and adverse birth outcomes using routine data - Nature - November 13th, 2026 [November 13th, 2026]
- Machine Learning Identifies Palmitoylation Biomarkers and Candidate Repurposed Drug in Lung Cancer - Bioengineer.org - November 13th, 2026 [November 13th, 2026]
- Machine Learning Identifies Changing Predictors of Medication Experience in Schizophrenia - Bioengineer.org - November 13th, 2026 [November 13th, 2026]
- Machine Learning Predicts Nuclear Reactor Containment Seismic Demands With Explainable Models - Bioengineer.org - November 7th, 2026 [November 7th, 2026]
- Explainable Machine Learning Identifies Predictive Thresholds Governing Urban Heat Island Dynamics - Bioengineer.org - November 7th, 2026 [November 7th, 2026]
- Machine learning reshapes yeast enzyme to favor a stubborn aldehyde and survive heat - Bioengineer.org - November 7th, 2026 [November 7th, 2026]
- Machine learning and winter storm predictions to be the focus of Nov. 6 talk - The Pennsylvania State University - November 3rd, 2026 [November 3rd, 2026]
- Zelims ZOE Taps AI, Machine Learning to Enhance S&R Operations - Marine News Magazine - November 3rd, 2026 [November 3rd, 2026]
- From Meters to Machine Learning: Building AI-Ready Utilities in India - CSIS | Center for Strategic and International Studies - October 30th, 2026 [October 30th, 2026]
- SPINE: A machine-learning program is on the trail of ghost particles - Phys.org - October 30th, 2026 [October 30th, 2026]
- Identification of ferroptosis-related key genes influencing synovitis development in osteoarthritis based on machine learning - Nature - October 30th, 2026 [October 30th, 2026]
- Machine learning simulations reveal how plastic transforms into hard carbon - EurekAlert! - October 30th, 2026 [October 30th, 2026]
- Machine learning predicts wastewater aeration efficiency, but only when data stay close to home - Bioengineer.org - October 30th, 2026 [October 30th, 2026]
- Machine learning reveals when changing shot direction pays off in elite tennis - Bioengineer.org - October 30th, 2026 [October 30th, 2026]
- Machine learning and blockchain team up to cut supply chain costs and carbon - Bioengineer.org - October 30th, 2026 [October 30th, 2026]
- Machine learning simulations uncover how plastic transforms into hard carbon - Open Access Government - October 30th, 2026 [October 30th, 2026]
- Machine Learning Surrogates Predict Wing Aerodynamics and Stress in Seconds - bioengineer.org - October 30th, 2026 [October 30th, 2026]
- Kernel-Tuned Machine Learning Pushes Network Intrusion Detection Toward 99% Accuracy on Benchmark Data - Bioengineer.org - October 30th, 2026 [October 30th, 2026]
- Machine learning and water quality indices reveal monsoon pollution hotspots in Odishas Baitarani River - bioengineer.org - October 30th, 2026 [October 30th, 2026]
- Machine Learning Predicts ICU Deaths in PAD Patients, but Hemodynamics Add Little - Bioengineer.org - October 30th, 2026 [October 30th, 2026]
- Stacked machine learning pairs ultrasound and Raman spectra to flag thyroid cancer - Bioengineer.org - October 30th, 2026 [October 30th, 2026]
- Machine Learning and Entropy Indices Reveal Most Mahanadi Water Near Paradip Unfit to Drink - Bioengineer.org - October 30th, 2026 [October 30th, 2026]
- Machine learning simulations trace how plastic turns into hard carbon atom by atom - Bioengineer.org - October 30th, 2026 [October 30th, 2026]
- My Machine Learning Pipeline Cheated, and Cross-Validation Never Noticed - Towards Data Science - October 26th, 2026 [October 26th, 2026]
- Machine learning predicts next-year fitness assessment failure in university students - Bioengineer.org - October 26th, 2026 [October 26th, 2026]
- Serrated Airfoils Reshape Stall Behavior in CFD, Wind Tunnel and Machine Learning Study - Bioengineer.org - October 26th, 2026 [October 26th, 2026]
- Applications of artificial intelligence and machine learning in food processing, quality control, and food safety: a critical review - Nature - October 26th, 2026 [October 26th, 2026]
- Machine-learning simulations reveal how sodium batteries form their protective interface - Bioengineer.org - October 26th, 2026 [October 26th, 2026]
- Reimagining demand forecasting at Bayer with AI and machine learning - Supply Chain Management Review - September 28th, 2026 [September 28th, 2026]
- What Is Machine Learning Fairness? What You Need to Know - Coursera - September 28th, 2026 [September 28th, 2026]
- Howard University Hosts Machine Learning Symposium in Collaboration with Amazon Web Services - The Dig at Howard University - September 26th, 2026 [September 26th, 2026]
- Machine learning helps identify chemicals that repel honey bees from pesticides - University of California, Riverside - September 26th, 2026 [September 26th, 2026]
- Machine Learning Hunts Down Hidden Gene Switches Across the Fungal Tree of Life - Bioengineer.org - September 26th, 2026 [September 26th, 2026]
- Integrated Quantum Technologies Advances ICA Research for IEEE Secure and Trustworthy Machine Learning Conference Submission - TMX Newsfile - September 26th, 2026 [September 26th, 2026]
- I surprisingly don't hate the Logitech Blue Yeti 2's use of machine learning - Yahoo Tech - September 26th, 2026 [September 26th, 2026]
- LANDR expands AI fraud detection built with machine learning lab Mila and says artificial streams flagged by Spotify for music it distributes fell... - September 26th, 2026 [September 26th, 2026]
- University Of California Machine Learning Spots Scents Bees Avoid In Real Fields - Quantum Zeitgeist - September 26th, 2026 [September 26th, 2026]
- Inside the AI Investing Think Tank: How Machine Learning Drives Modern Financial Research - Rebellion Research - September 26th, 2026 [September 26th, 2026]
- New machine learning methods set to improve forecasts - University of Reading - September 24th, 2026 [September 24th, 2026]
- What this machine learning model boasting 65% accuracy says is coming next for the 10-year Treasury - MarketWatch - September 24th, 2026 [September 24th, 2026]
- Quantitative Investment Strategies: Institutional Architecture and Machine Learning Alpha in 2026 - Rebellion Research - September 24th, 2026 [September 24th, 2026]
- Machine learning forecasts suggest a concentration paradox in international student mobility to the United Kingdom - Nature - September 22nd, 2026 [September 22nd, 2026]
- Machine learning model distinguishes levels of psychological resilience in health care workers with 75% accuracy - Medical Xpress - September 22nd, 2026 [September 22nd, 2026]
- How Machine Learning Is Changing the Way We Inspect the Railway - Railway-News - September 22nd, 2026 [September 22nd, 2026]
- Microsoft file patent for a machine learning system that would "train" itself on your playtime to work out when you'll be least annoyed by... - September 22nd, 2026 [September 22nd, 2026]
- Quantitative Finance Conferences NYC 2026: The AI and Machine Learning Summit Guide - Rebellion Research - September 22nd, 2026 [September 22nd, 2026]
- Nanoparticle Electrode and Machine Learning Team Up to Catch Toxic Lead and Cadmium in Water - Bioengineer.org - September 22nd, 2026 [September 22nd, 2026]
- Cosmic tensions: can machine learning find a fairer story of the universe - Times of Malta - September 22nd, 2026 [September 22nd, 2026]
- Lessening the shock of defibrillation with machine learning - AIP.ORG - September 17th, 2026 [September 17th, 2026]
- Algorithmic Stock Picking: The Architecture of Machine Learning Alpha in 2026 - Rebellion Research - September 17th, 2026 [September 17th, 2026]
- Canada is pushing ahead with machine learning to improve medical diagnostics - Digital Journal - September 17th, 2026 [September 17th, 2026]
- How researchers use machine learning to re-create chirps and trills produced by forest insects when Dinos - The Times of India - September 15th, 2026 [September 15th, 2026]
- Machine Learning Reshapes Credit Scoring - Communications of the ACM - September 13th, 2026 [September 13th, 2026]
- Machine Learning With Threshold Optimization Could Help Reduce Unnecessary Appendectomies in Adults - bioengineer.org - September 13th, 2026 [September 13th, 2026]
- Information Theory Meets Machine Learning to Catch Industrial Cyberattacks - bioengineer.org - September 13th, 2026 [September 13th, 2026]
- Machine Learning Gets a Robustness Boost by Turning Labels into Preferences - bioengineer.org - September 13th, 2026 [September 13th, 2026]
- Machine Learning Meets X-Rays to Reveal the Hidden Architecture of Pea Seeds - bioengineer.org - September 13th, 2026 [September 13th, 2026]
- Machine Learning Predicts Which Women Will Face Early Ovarian Failure Within Three Years - bioengineer.org - September 13th, 2026 [September 13th, 2026]
- Machine Learning Cracks the Code of Nitinol Wear, a Metal That Remembers Its Shape - bioengineer.org - September 13th, 2026 [September 13th, 2026]
- Using machine learning to see how living brains learn - The University of Utah - September 8th, 2026 [September 8th, 2026]
- Applying causal machine learning to assess and improve cleantech policy design - Nature - September 8th, 2026 [September 8th, 2026]
- Frontier Tech Leaders Programme Celebrates First Machine Learning Bootcamp Graduation and AI for Sustainable Tourism Hackathon in Angola - United... - September 8th, 2026 [September 8th, 2026]
- From the Knowledge to machine learning: Wayve takes AI driving to London - IOT Insider - September 8th, 2026 [September 8th, 2026]
- Algorithm optimizes machine learning techniques that use linear, tunable resistor networks - AIP.ORG - September 2nd, 2026 [September 2nd, 2026]
- Math Modeling Seminar: Applications of Topological Data Analysis and Machine Learning Models in Predictive Biology and Drug Discovery | Events | RIT -... - September 2nd, 2026 [September 2nd, 2026]
- UC Berkeley Announces New Professional Graduate Degree in AI and Machine Learning - University of California, Berkeley - August 25th, 2026 [August 25th, 2026]
- DedeepyaYarraand the rise of Trustworthy AI: Where Machine Learning meets cybersecurity - India.com - August 25th, 2026 [August 25th, 2026]
- Machine learning smooths the road from idea to real-world climate impact - EurekAlert! - August 18th, 2026 [August 18th, 2026]
- Chris Latham Interviews Henry Zelikovsky, Founder & CEO of Softlab360: Successful Applications of AI/Machine Learning in Wealth Management -... - August 18th, 2026 [August 18th, 2026]
- Integrated data and machine learning transform lung cancer diagnosis and treatment - Bioengineer.org - August 18th, 2026 [August 18th, 2026]
- Machine learning accelerates climate solutions from ideas to real-world impact - Bioengineer.org - August 18th, 2026 [August 18th, 2026]
- Identification of weight loss predictors using machine learning approaches in adolescents with obesity - Nature - August 16th, 2026 [August 16th, 2026]
- Quantitative Hedge Fund Strategies: The Machine Learning Revolution of 2026 - rebellionresearch.com - August 16th, 2026 [August 16th, 2026]
- Healthcare Machine Learning Hits Production Scale as Governance Falls Behind, Black Book's Fourth Annual Report Finds - bhpioneer.com - August 16th, 2026 [August 16th, 2026]