A technique to improve both fairness and accuracy in artificial intelligence – MIT News
For workers who use machine-learning models to help them make decisions, knowing when to trust a models predictions is not always an easy task, especially since these models are often so complex that their inner workings remain a mystery.
Users sometimes employ a technique, known as selective regression, in which the model estimates its confidence level for each prediction and will reject predictions when its confidence is too low. Then a human can examine those cases, gather additional information, and make a decision about each one manually.
But while selective regression has been shown to improve the overall performance of a model, researchers at MIT and the MIT-IBM Watson AI Lab have discovered that the technique can have the opposite effect for underrepresented groups of people in a dataset. As the models confidence increases with selective regression, its chance of making the right prediction also increases, but this does not always happen for all subgroups.
For instance, a model suggesting loan approvals might make fewer errors on average, but it may actually make more wrong predictions for Black or female applicants. One reason this can occur is due to the fact that the models confidence measure is trained using overrepresented groups and may not be accurate for these underrepresented groups.
Once they had identified this problem, the MIT researchers developed two algorithms that can remedy the issue. Using real-world datasets, they show that the algorithms reduce performance disparities that had affected marginalized subgroups.
Ultimately, this is about being more intelligent about which samples you hand off to a human to deal with. Rather than just minimizing some broad error rate for the model, we want to make sure the error rate across groups is taken into account in a smart way, says senior MIT author Greg Wornell, the Sumitomo Professor in Engineering in the Department of Electrical Engineering and Computer Science (EECS) who leads the Signals, Information, and Algorithms Laboratory in the Research Laboratory of Electronics (RLE) and is a member of the MIT-IBM Watson AI Lab.
Joining Wornell on the paper are co-lead authors Abhin Shah, an EECS graduate student, and Yuheng Bu, a postdoc in RLE; as well as Joshua Ka-Wing Lee SM 17, ScD 21 and Subhro Das, Rameswar Panda, and Prasanna Sattigeri, research staff members at the MIT-IBM Watson AI Lab. The paper will be presented this month at the International Conference on Machine Learning.
To predict or not to predict
Regression is a technique that estimates the relationship between a dependent variable and independent variables. In machine learning, regression analysis is commonly used for prediction tasks, such as predicting the price of a home given its features (number of bedrooms, square footage, etc.) With selective regression, the machine-learning model can make one of two choices for each input it can make a prediction or abstain from a prediction if it doesnt have enough confidence in its decision.
When the model abstains, it reduces the fraction of samples it is making predictions on, which is known as coverage. By only making predictions on inputs that it is highly confident about, the overall performance of the model should improve. But this can also amplify biases that exist in a dataset, which occur when the model does not have sufficient data from certain subgroups. This can lead to errors or bad predictions for underrepresented individuals.
The MIT researchers aimed to ensure that, as the overall error rate for the model improves with selective regression, the performance for every subgroup also improves. They call this monotonic selective risk.
It was challenging to come up with the right notion of fairness for this particular problem. But by enforcing this criteria, monotonic selective risk, we can make sure the model performance is actually getting better across all subgroups when you reduce the coverage, says Shah.
Focus on fairness
The team developed two neural network algorithms that impose this fairness criteria to solve the problem.
One algorithm guarantees that the features the model uses to make predictions contain all information about the sensitive attributes in the dataset, such as race and sex, that is relevant to the target variable of interest. Sensitive attributes are features that may not be used for decisions, often due to laws or organizational policies. The second algorithm employs a calibration technique to ensure the model makes the same prediction for an input, regardless of whether any sensitive attributes are added to that input.
The researchers tested these algorithms by applying them to real-world datasets that could be used in high-stakes decision making. One, an insurance dataset, is used to predict total annual medical expenses charged to patients using demographic statistics; another, a crime dataset, is used to predict the number of violent crimes in communities using socioeconomic information. Both datasets contain sensitive attributes for individuals.
When they implemented their algorithms on top of a standard machine-learning method for selective regression, they were able to reduce disparities by achieving lower error rates for the minority subgroups in each dataset. Moreover, this was accomplished without significantly impacting the overall error rate.
We see that if we dont impose certain constraints, in cases where the model is really confident, it could actually be making more errors, which could be very costly in some applications, like health care. So if we reverse the trend and make it more intuitive, we will catch a lot of these errors. A major goal of this work is to avoid errors going silently undetected, Sattigeri says.
The researchers plan to apply their solutions to other applications, such as predicting house prices, student GPA, or loan interest rate, to see if the algorithms need to be calibrated for those tasks, says Shah. They also want to explore techniques that use less sensitive information during the model training process to avoid privacy issues.
And they hope to improve the confidence estimates in selective regression to prevent situations where the models confidence is low, but its prediction is correct. This could reduce the workload on humans and further streamline the decision-making process, Sattigeri says.
This research was funded, in part, by the MIT-IBM Watson AI Lab and its member companies Boston Scientific, Samsung, and Wells Fargo, and by the National Science Foundation.
Originally posted here:
A technique to improve both fairness and accuracy in artificial intelligence - MIT News
- Ensemble Machine Learning for Digital Mapping of Soil pH and Electrical Conductivity in the Andean Agroecosystem of Peru - Frontiers - October 21st, 2025 [October 21st, 2025]
- New UA research develops machine learning to address needs of children with autism - AZPM News - October 21st, 2025 [October 21st, 2025]
- NMDSI Speaker Series on Weather Forecasting: What Machine Learning Can and Can't Do, Oct. 23 - Marquette Today - October 21st, 2025 [October 21st, 2025]
- Polyskill Achieves 1.7x Improved Skill Reuse and 9.4% Higher Success Rates through Polymorphic Abstraction in Machine Learning - Quantum Zeitgeist - October 21st, 2025 [October 21st, 2025]
- University of Strathclyde opens admission for MSc in Machine & Deep Learning for Jan 2026 intake - The Indian Express - October 21st, 2025 [October 21st, 2025]
- Reducing Model Biases with Machine Learning Corrections Derived from Ocean Data Assimilation Increments - ESS Open Archive - October 19th, 2025 [October 19th, 2025]
- Unlocking Obesity: Multi-Omics and Machine Learning Insights - Bioengineer.org - October 19th, 2025 [October 19th, 2025]
- Lockheed Martin advances PAC-3 MSE interceptor using artificial intelligence and machine learning - Defence Industry Europe - October 19th, 2025 [October 19th, 2025]
- Semi-automated surveillance of surgical site infections using machine learning and rule-based classification models - Nature - October 19th, 2025 [October 19th, 2025]
- AI and Machine Learning - City of San Jos to release RFP for generative AI platform - Smart Cities World - October 19th, 2025 [October 19th, 2025]
- Machine learning helps identify 'thermal switch' for next-generation nanomaterials - Phys.org - October 17th, 2025 [October 17th, 2025]
- Machine Learning Makes Wildlife Data Analysis Less of a Trek - Maryland.gov - October 17th, 2025 [October 17th, 2025]
- An interpretable multimodal machine learning model for predicting malignancy of thyroid nodules in low-resource scenarios - BMC Endocrine Disorders - October 17th, 2025 [October 17th, 2025]
- In First-Episode Psychosis Patients, Machine Learning Predicted Illness Trajectories to Potentially Improve Outcomes - Brain and Behavior Research - October 17th, 2025 [October 17th, 2025]
- Novel Machine Learning Model Improves MASLD Detection in Type 2 Diabetes - The American Journal of Managed Care (AJMC) - October 17th, 2025 [October 17th, 2025]
- Hybrid machine learning models for predicting the tensile strength of reinforced concrete incorporating nano-engineered and sustainable supplementary... - October 17th, 2025 [October 17th, 2025]
- Modelling of immune infiltration in prostate cancer treated with HDR-brachytherapy using Raman spectroscopy and machine learning - Nature - October 17th, 2025 [October 17th, 2025]
- Association between atherogenic index of plasma and sepsis in critically ill patients with ischemic stroke: a retrospective cohort study using... - October 17th, 2025 [October 17th, 2025]
- AI enters the nuclear age: Pentagon modernizes warheads with machine learning - Washington Times - October 17th, 2025 [October 17th, 2025]
- AI and Machine Learning - Bentley Systems shares its vision for trustworthy AI - Smart Cities World - October 17th, 2025 [October 17th, 2025]
- Looking back to move forward: can historical clinical trial data and machine learning drive change in participant recruitment in anticipation of... - October 15th, 2025 [October 15th, 2025]
- Physics-Based Machine Learning Paves the Way for Advanced 3D-Printed Materials - Bioengineer.org - October 15th, 2025 [October 15th, 2025]
- Predicting one-year overall survival in patients with AITL using machine learning algorithms: a multicenter study - Nature - October 15th, 2025 [October 15th, 2025]
- Explainable machine learning models for predicting of protein-energy wasting in patients on maintenance haemodialysis - BMC Nephrology - October 15th, 2025 [October 15th, 2025]
- Feasibility of machine learning analysis for the identification of patients with possible primary ciliary dyskinesia - Orphanet Journal of Rare... - October 15th, 2025 [October 15th, 2025]
- Machine learning-based prediction of preeclampsia using first-trimester inflammatory markers and red blood cell indices - BMC Pregnancy and Childbirth - October 15th, 2025 [October 15th, 2025]
- Utilizing AI and machine learning to improve railroad safety: Detecting trespasser hotspots - masstransitmag.com - October 15th, 2025 [October 15th, 2025]
- Precision medicine meets machine learning: AI and oncology biomarkers - pharmaphorum - October 15th, 2025 [October 15th, 2025]
- Aether Pro Exchange Transforms Execution Dynamics with Machine-Learning Optimization - GlobeNewswire - October 15th, 2025 [October 15th, 2025]
- Prevalence, associated factors, and machine learning-based prediction of depression, anxiety, and stress among university students: a cross-sectional... - October 15th, 2025 [October 15th, 2025]
- Artificial Intelligence vs. Machine Learning: Which skills will open better career options in the global - Times of India - October 15th, 2025 [October 15th, 2025]
- Study Reveals Impact of Negative Class Definitions on Machine Learning Accuracy in Immunotherapy - geneonline.com - October 15th, 2025 [October 15th, 2025]
- Muna Al-Khaifi: Detection of Breast Cancer Using Machine Learning and Explainable AI - Oncodaily - October 13th, 2025 [October 13th, 2025]
- Expedia Group Unveils Innovative AI and Machine Learning Solutions to Transform Partner Travel Experiences - Travel And Tour World - October 13th, 2025 [October 13th, 2025]
- Machine Learning-Guided Prediction of Formulation Performance in Inhalable CiprofloxacinBile Acid Dispersions with Antimicrobial and Toxicity... - October 13th, 2025 [October 13th, 2025]
- Machine Learning and BIG DATA workshop planned Oct. 14-15 - West Virginia University - October 11th, 2025 [October 11th, 2025]
- How Google enables third-party circularity by increasing recycling rates with Machine Learning - The World Business Council for Sustainable... - October 11th, 2025 [October 11th, 2025]
- Integrating Artificial Intelligence and Machine Learning in Hydroclimatic Research - A Promising Step Forward - University of Northern British... - October 11th, 2025 [October 11th, 2025]
- Semi-automatic detection of anteriorly displaced temporomandibular joint discs in magnetic resonance images using machine learning - BMC Oral Health - October 11th, 2025 [October 11th, 2025]
- AI and Machine Learning - Partnership to bring infrastructure intelligence to US public sector - Smart Cities World - October 11th, 2025 [October 11th, 2025]
- Between rain and snow, machine learning finds nine precipitation types - Phys.org - October 9th, 2025 [October 9th, 2025]
- Between rain and snow, machine learning finds 9 precipitation types - Michigan Engineering News - October 9th, 2025 [October 9th, 2025]
- Machine learning optimizes nanoparticle design for drug delivery to the brain - Physics World - October 9th, 2025 [October 9th, 2025]
- Development and validation of a machine learning-based prediction model for prolonged length of stay after laparoscopic gastrointestinal surgery: a... - October 9th, 2025 [October 9th, 2025]
- G Sachs: Stock Mkt Not in Bubble Yet; Machine Learning/ AI Expected to Spawn New Wave of Superstars - AASTOCKS.com - October 9th, 2025 [October 9th, 2025]
- AI and Machine Learning - See.Sense works with City of Sydney to develop AI dashboard - Smart Cities World - October 9th, 2025 [October 9th, 2025]
- Machine Learning Used to Predict Live Birth Outcomes in Fresh Embryo Transfers - geneonline.com - October 9th, 2025 [October 9th, 2025]
- RIT researchers use machine learning to better understand the pathways of disease - Rochester Institute of Technology - October 7th, 2025 [October 7th, 2025]
- Leveraging machine learning to predict mosquito bed net utilization among women of reproductive age in sub-Saharan Africa - Malaria Journal - October 7th, 2025 [October 7th, 2025]
- Machine learning-based radiomics using magnetic resonance images for prediction of clinical complete response to neoadjuvant chemotherapy in patients... - October 7th, 2025 [October 7th, 2025]
- Machine Learning Self Driving Cars: The Technology Driving the Future of Mobility - SpeedwayMedia.com - October 7th, 2025 [October 7th, 2025]
- Investigating the relationship between blood factors and HDL-C levels in the bloodstream using machine learning methods - Journal of Health,... - October 7th, 2025 [October 7th, 2025]
- AI in the fast lane: F1 teams Alpine, Audi use machine learning as force multiplier - The Business Times - October 7th, 2025 [October 7th, 2025]
- Future Scope of Machine Learning in Healthcare Market Set to Witness Significant Growth by 2025-2032 - openPR.com - October 7th, 2025 [October 7th, 2025]
- AI and Machine Learning - AI readiness and adoption toolkit launched - Smart Cities World - October 4th, 2025 [October 4th, 2025]
- Machine Learning Model UmamiPredict Developed to Forecast Savory Taste of Molecules and Peptides - geneonline.com - October 4th, 2025 [October 4th, 2025]
- Machine Learning Boosts Crop Yield Predictions in Senegal - Bioengineer.org - October 4th, 2025 [October 4th, 2025]
- Machine learning-driven stability analysis of eco-friendly superhydrophobic graphene-based coatings on copper substrate - Nature - October 4th, 2025 [October 4th, 2025]
- Integrated machine learning analysis of proteomic and transcriptomic data identifies healing associated targets in diabetic wound repair - Nature - October 4th, 2025 [October 4th, 2025]
- Development and evaluation of a machine learning prediction model for short-term mortality in patients with diabetes or hyperglycemia at emergency... - October 4th, 2025 [October 4th, 2025]
- Fast and robust mixed gas identification and recognition using tree-based machine learning and sensor array response - Nature - October 4th, 2025 [October 4th, 2025]
- Estimation of sexual dimorphism of adult human mandibles of South Indian origin using non-metric parameters and machine learning classification... - October 4th, 2025 [October 4th, 2025]
- Cloud-Based Machine Learning Platforms Technologies Market Growth and Future Prospects - Precedence Research - October 4th, 2025 [October 4th, 2025]
- Machine Learning Framework Developed to Optimize Phosphorus Recovery in Hydrothermal Treatment of Livestock Manure - geneonline.com - October 4th, 2025 [October 4th, 2025]
- Unifying machine learning and interpolation theory via interpolating neural networks - Nature - October 2nd, 2025 [October 2nd, 2025]
- Anna: an open-source platform for real-time integration of machine learning classifiers with veterinary electronic health records - BMC Veterinary... - October 2nd, 2025 [October 2nd, 2025]
- The Future of Liver Health: Can Human Models and Machine Learning Reduce Disease Rates? - Technology Networks - October 2nd, 2025 [October 2nd, 2025]
- Machine Learning Radiomics Predicts Pancreatic Cancer Invasion - Bioengineer.org - October 2nd, 2025 [October 2nd, 2025]
- Next-generation COVID-19 detection using a metasurface biosensor with machine learning-enhanced refractive index sensing - Nature - October 2nd, 2025 [October 2nd, 2025]
- Machine learning-based models for screening of anemia and leukemia using features of complete blood count reports - Nature - October 2nd, 2025 [October 2nd, 2025]
- Estimating the peak age of chess players through statistical and machine learning techniques - Nature - October 2nd, 2025 [October 2nd, 2025]
- Optimizing water quality index using machine learning: a six-year comparative study in riverine and reservoir systems - Nature - October 2nd, 2025 [October 2nd, 2025]
- Physics-informed machine learning-based real-time long-horizon temperature fields prediction in metallic additive manufacturing - Nature - October 2nd, 2025 [October 2nd, 2025]
- The Silicon Revolution: How AI and Machine Learning Are Forging the Future of Semiconductor Manufacturing - FinancialContent - October 2nd, 2025 [October 2nd, 2025]
- Machine learning model for differentiating Pneumocystis jirovecii pneumonia from colonization and analyzing mortality risk in non-HIV patients using... - October 2nd, 2025 [October 2nd, 2025]
- Radiomics and Machine Learning Applied to CECT Scans Show Potential in Predicting Perineural Invasion in Pancreatic Cancer - geneonline.com - October 2nd, 2025 [October 2nd, 2025]
- Machine learning and response surface optimization to enhance diesel engine performance using milk scum biodiesel with alumina nanoparticles - Nature - October 2nd, 2025 [October 2nd, 2025]
- Landmark Patent Appeal Decision Strengthens Protection for AI and Machine Learning Innovations - The National Law Review - October 2nd, 2025 [October 2nd, 2025]
- Machine learning researchers and industry leaders gathering at Santa Clara University - Stories - News & Events - Santa Clara University - September 30th, 2025 [September 30th, 2025]
- Building better batteries with amorphous materials and machine learning - Tech Xplore - September 30th, 2025 [September 30th, 2025]