FDA highlights the need to address bias in AI – Healthcare IT News
The U.S. Food and Drug Administration on Thursday convened a public meeting of its Patient Engagement Advisory Committee to discuss issues regarding artificial intelligence and machine learning in medical devices.
"Devices using AI and ML technology will transform healthcare delivery by increasing efficiency in key processes in the treatment of patients," said Dr. Paul Conway, PEAC chair and chair of policy and global affairs of the American Association of Kidney Patients.
As Conway and others noted during the panel, AI and ML systems may have algorithmic biases and lack transparency potentially leading, in turn, to an undermining of patient trust in devices.
Medical device innovation has already ramped up in response to the COVID-19 crisis, with Center for Devices and Radiological Health Director Dr. Jeff Shuren noting that 562 medical devices have already been granted emergency use authorization by the FDA.
It's imperative, said Shuren, that patients' needs be considered as part of the creation process.
"We continue to encourage all members of the healthcare ecosystem to strive to understand patients' perspective and proactively incorporate them into medical device development, modification and evaluation," said Shuren. "Patients are truly the inspiration for all the work we do."
"Despite the global challenges with the COVID-19 public health emergency ... the patient's voice won't be stopped," Shuren added. "And if anything, there is even more reason for it to be heard."
However, said Pat Baird, regulatory head of global software standards at Philips, facilitating patient trust also means acknowledging the importance of robust and accurate data sets.
"To help support ourpatients, we need to become more familiar with them, their medical conditions, their environment, and their needs and wantsto be able to better understand the potentially confounding factors that drive some of the trends in the collected data," said Baird.
"An algorithm trained on one subset of the population might not be relevant for a different subset," Baird explained.
For instance, if a hospital needed a device that would serve its population of seniors at a Florida retirement community, an algorithm trained on recognizing healthcare needs of teens in Maine would not be effective.Not every population will have the same needs.
"This bias in the data is not intentional, but can be hard to identify," he continued. He encouraged the development of a taxonomy of bias types that would be made publicly available.
Ultimately, he said, people won't use what they don't trust. "We need to use our collective intelligence to help produce better artificial intelligence populations," he said.
Captain Terri Cornelison, chief medical officer and director for thehealth of women at CDRH, noted that demographic identifiers can be medically significant due to genetics and social determinants of health, among other factors.
"Science is showing us that these are not just categorical identifiers but actually clinically relevant," Cornelison said.
She pointed out that a clinical study that does not identify patients' sex may mask different results for people with different chromosomes.
"In many instances, AI and ML devices may be learning a worldview that is narrow in focus, particularly in the available training data, if the available training data do not represent a diverse set of patients," she said.
"More simply, AI and ML algorithms may not represent you if the data do not include you," she said.
"Advances in artificial intelligence are transforming our health systems and daily lives," Cornelison continued. "Yet despite these significant achievements, most ignore the sex, gender, age, race [and] ethnicity dimensions and their contributions to health and disease differences among individuals."
The committee also examined how informed consent might play a role in algorithmic training.
"If I give my consent to be treated by an AI/ML device, I have the right to know whether there were patients like me ... in the data set," said Bennet Dunlap, a health communications consultant. "I think the FDA should not be accepting or approving a medical device that does not have patient engagement" of the kind outlined in committee meetings, he continued.
"You need to know what your data is going to be used for," he reiterated. "I have white privilege. I can just assume old white guys are in [the data sets]. That's where everybody starts. But that should not be the case."
Dr. Monica Parker, assistant professor in neurology and education core member of the Goizueta Alzheimers Disease Research Center at Emory University, pointed out that diversifying patient data requires turning to trusted entities within communities.
"If people are developing these devices, in the interest of being more broadly diverse, is there some question about where these things were tested?" She raised the issue of testing taking place in academic medical centers or technology centers on the East or West Coast, versus "real-world data collection from hospitals that may be using some variation of the device for disease process.
"Clinicians who are serving the population for which the device is needed" provide accountability and give the device developer a better sense of whothey're treating, Parker said. She also reminded fellow committee members that members of different demographic groups are not uniform.
Philip Rutherford, director of operation at Faces and Voices Recovery, pointed out that it's not just enough to prioritize diversity in data sets.The people in charge of training the algorithm must also not be homogenous.
"If we want diversity in our data, we have to seek diversity in the people that are collecting the data," said Rutherford.
The committee called on the FDA to take a strong role in addressing algorithmic bias in artificial intelligence and machine learning.
"At the end of the day, diversity validation and unconscious bias all these things can be addressed if there's strong leadership from the start," said Conway.
Kat Jercich is senior editor of Healthcare IT News.Twitter: @kjercichEmail: kjercich@himss.orgHealthcare IT News is a HIMSS Media publication.
Read the original here:
FDA highlights the need to address bias in AI - Healthcare IT News
- Artificial Intelligence: Why are traditional industries adopting it? - BBC - November 15th, 2026 [November 15th, 2026]
- The Evidence Is Piling Up: AMD Could Be the Best Artificial Intelligence (AI) Stock to Buy Amid the Sell-Off - The Motley Fool - November 15th, 2026 [November 15th, 2026]
- How Public Health Agencies Can Use Artificial Intelligence Effectively and Responsibly - The Pew Charitable Trusts - November 15th, 2026 [November 15th, 2026]
- Q&A: Leveraging artificial intelligence to propel engineering education - The Pennsylvania State University - November 15th, 2026 [November 15th, 2026]
- The Evidence Is Piling Up: AMD Could Be the Best Artificial Intelligence (AI) Stock to Buy Amid the Sell-Off - The Globe and Mail - November 15th, 2026 [November 15th, 2026]
- Artificial intelligence is scary. But history reminds us that tech makes lives better, not worse. - New York Post - November 15th, 2026 [November 15th, 2026]
- Artificial intelligence in the classroom - Tri-Cities Area Journal of Business - November 15th, 2026 [November 15th, 2026]
- The pathetic tyranny of artificial intelligence - Observer Online - November 15th, 2026 [November 15th, 2026]
- How Artificial Intelligence Is Reshaping the Legal Profession - TechSpective - November 15th, 2026 [November 15th, 2026]
- Governor-elect Byron Donalds joins ed-tech conference in Hollywood as artificial intelligence gains ground across state - WSVN - November 15th, 2026 [November 15th, 2026]
- 3 Artificial Intelligence (AI) Stocks to Buy on the Post-Election Dip and Hold for 5 Years - The Motley Fool - November 15th, 2026 [November 15th, 2026]
- Not Nvidia. Not Micron. This Artificial Intelligence (AI) Stock Will Be 2027s Biggest Winner After the Latest Tech Sell-Off - The Motley Fool - November 15th, 2026 [November 15th, 2026]
- Artificial Intelligence Threatens the Ballot Box - POLITIS - To the point - November 15th, 2026 [November 15th, 2026]
- Will Artificial Intelligence Create Wealth in Portugal or Just Consume Energy? - The Portugal News - November 15th, 2026 [November 15th, 2026]
- The 3 Best Artificial Intelligence (AI) Stocks to Buy in November - The Motley Fool - November 3rd, 2026 [November 3rd, 2026]
- Generative Artificial Intelligence in Clinical Reasoning Education: A Focused Review and Framework - Cureus - November 3rd, 2026 [November 3rd, 2026]
- The 3 Best Artificial Intelligence (AI) Stocks to Buy in November - Yahoo Finance - November 3rd, 2026 [November 3rd, 2026]
- Artificial intelligence and the ballot box - KTVU - November 3rd, 2026 [November 3rd, 2026]
- I asked Artificial Intelligence what it thought of New Jersey - New Jersey 101.5 - November 3rd, 2026 [November 3rd, 2026]
- 3 Genius Artificial Intelligence (AI) Stocks to Buy Hand Over Fist Before Year-End - The Motley Fool - November 3rd, 2026 [November 3rd, 2026]
- Will artificial intelligence make caddies a thing of the past? - Yahoo Sports - November 3rd, 2026 [November 3rd, 2026]
- Maintaining the ITU Artificial Intelligence Resolution - CircleID - November 3rd, 2026 [November 3rd, 2026]
- Artificial Intelligence, Plans for the Future Discussed at 2026 State of the Valley - Rural Radio Network - November 3rd, 2026 [November 3rd, 2026]
- The Artificial Intelligence (AI) Market in Orthopedic Surgery: Examining Demand Patterns and Competitive Landscape - The National Law Review - November 3rd, 2026 [November 3rd, 2026]
- Market Forecast Report on Artificial Intelligence (AI) in Regulatory Affairs Including Segment Analysis - The National Law Review - November 3rd, 2026 [November 3rd, 2026]
- WSCC HumanKind series to explore impact of artificial intelligence - MasonCountyPress.com - November 3rd, 2026 [November 3rd, 2026]
- Taxing artificial intelligence: Whats on the table, and whats at stake - Brookings - November 3rd, 2026 [November 3rd, 2026]
- Artificial Intelligence Technology Solutions Launches Campaign to Expand SARA AI Security Platform Adoption - Yahoo Finance - November 3rd, 2026 [November 3rd, 2026]
- No Bailouts, Just Confiscations: Nationalize The Artificial Intelligence Companies - FOREVER WARS by Spencer Ackerman - November 3rd, 2026 [November 3rd, 2026]
- The artificial intelligence industrys Trump gamble crashes into a new political reality - The Guam Daily Post - November 3rd, 2026 [November 3rd, 2026]
- Trump says he will ban term artificial intelligence - Reuters - November 3rd, 2026 [November 3rd, 2026]
- Breakingviews - COMMENTARY: Artificial intelligence reaches its Napster moment - Reuters - November 3rd, 2026 [November 3rd, 2026]
- The Artificial Intelligence (AI) Facial Skin Analyzer Market is projected to grow at a 10.8% CAGR - EIN News - November 3rd, 2026 [November 3rd, 2026]
- The Artificial Intelligence (AI) Market in Aging and Elderly Care Is Projected to Grow Rapidly at a 23.8% CAGR - EIN News - November 3rd, 2026 [November 3rd, 2026]
- Artificial Intelligence is overthrowing the purpose of schooling. Now what? - New University - November 3rd, 2026 [November 3rd, 2026]
- The Artificial Intelligence (AI) Market in Remote Patient Monitoring Is Projected to Grow at a 23.5% CAGR by 2030 - EIN News - November 3rd, 2026 [November 3rd, 2026]
- CSU Uses Artificial Intelligence to Improve Hail Forecasting - North Forty News - November 3rd, 2026 [November 3rd, 2026]
- Artificial Intelligence (AI) Heart Sound Digital Twin Analytics Market Projected to Reach $1.91 Billion by 2030 - EIN News - November 3rd, 2026 [November 3rd, 2026]
- Steven Spielberg's A.I. ARTIFICIAL INTELLIGENCE to Make 4K UHD Debut in December 2026 - Get Your Comic On - November 3rd, 2026 [November 3rd, 2026]
- Artificial intelligence and the human element - Canadian Institute of Mining, Metallurgy - November 3rd, 2026 [November 3rd, 2026]
- Documentary Review: The AI Doc offers an engaging intro to artificial intelligence, but fails to ask the toughest questions - Northwest Progressive... - November 3rd, 2026 [November 3rd, 2026]
- SATURDAY MORNING WITH CRANKY DON JOE | Larry Introduced Me to Artificial Intelligence, and Now I Have Questions - MinneapoliMedia - October 17th, 2026 [October 17th, 2026]
- USCCB Appoints Charles Camosy to Task Force on Artificial Intelligence - The Catholic University of America - October 17th, 2026 [October 17th, 2026]
- Teenagers turning to artificial intelligence for relationships and advice - NewsNation - October 17th, 2026 [October 17th, 2026]
- Q: What are the biggest threats from artificial intelligence? - KRQE - October 17th, 2026 [October 17th, 2026]
- Anything not tied to artificial intelligence took off this week - CNBC - October 17th, 2026 [October 17th, 2026]
- Regent University Launches New Three-Year Degree in Artificial Intelligence for Spring 2027 - Regent University - October 17th, 2026 [October 17th, 2026]
- Elon Musk's AI company testifies at NYC Council hearing on artificial intelligence safety and regulations - ABC7 New York - October 17th, 2026 [October 17th, 2026]
- Focus - How books are being scanned and destroyed to train artificial intelligence - France 24 - October 17th, 2026 [October 17th, 2026]
- New publication urges skeptical approach to election information produced by artificial intelligence - WRVO - October 17th, 2026 [October 17th, 2026]
- Artificial intelligence values before autonomy - Fast Company - October 17th, 2026 [October 17th, 2026]
- Anthropic warns of self-replicating artificial intelligence models as AI arms race heats up - Fox Business - October 17th, 2026 [October 17th, 2026]
- North Korea has announced their use of artificial intelligence for a "new type" precision-guide torpedo system for target detection and... - October 17th, 2026 [October 17th, 2026]
- Billionaires Are Selling These 6 Artificial Intelligence (AI) Stocks. Should You Follow? - Yahoo Finance - October 17th, 2026 [October 17th, 2026]
- WSU Athletics is using Artificial Intelligence to help tell its story. What does that mean? - Moscow-Pullman Daily News - October 17th, 2026 [October 17th, 2026]
- This Hot Artificial Intelligence IPO Stock Is Down 52%, but It Could Double From Here, According to Wall Street - Yahoo Finance - October 17th, 2026 [October 17th, 2026]
- Artificial intelligence is coming to African schools - African Business - October 17th, 2026 [October 17th, 2026]
- Not Nvidia. Not Micron. This Under-the-Radar Artificial Intelligence (AI) Chip Stock Could Be the Biggest Winner in 2027. - The Motley Fool - October 15th, 2026 [October 15th, 2026]
- What My Son Is Learning From Artificial Intelligence - Greater Good: The Science of a Meaningful Life - October 15th, 2026 [October 15th, 2026]
- Sound Practices for Responsible Adoption of Artificial Intelligence (AI): Overview of consultation responses - Financial Stability Board - October 15th, 2026 [October 15th, 2026]
- Mission adopts policy for artificial intelligence usage in city business - MyRGV.com - October 15th, 2026 [October 15th, 2026]
- Yael Grushka-Cockayne discusses artificial intelligence and future of business education - The Cavalier Daily - October 15th, 2026 [October 15th, 2026]
- The New York Times. . Artificial intelligence has gone from struggling with basic arithmetic to helping solve some of mathematics most famous open... - October 15th, 2026 [October 15th, 2026]
- FSB finalises sound practices for the responsible adoption of artificial intelligence (AI) - Financial Stability Board - October 15th, 2026 [October 15th, 2026]
- Broadcom vs. SK Hynix: Here's What Revenue Trends Tell Investors About These Artificial Intelligence Companies - The Motley Fool - October 15th, 2026 [October 15th, 2026]
- When Artificial Intelligence Becomes the Expert, Who Is Still Thinking? - The Washington Stand - October 15th, 2026 [October 15th, 2026]
- Is artificial intelligence making weather forecasts better? - fox21online.com - October 15th, 2026 [October 15th, 2026]
- High-performance hardware requirements are shifting rapidly as local artificial intelligence tools expand - BNO News - October 15th, 2026 [October 15th, 2026]
- Not Memory. Not Energy. This Is the Biggest Bottleneck in Artificial Intelligence, and These Are the Companies That Can Benefit. - The Motley Fool - October 15th, 2026 [October 15th, 2026]
- How Artificial Intelligence Can Help With Critical Thinking - Greater Good: The Science of a Meaningful Life - October 15th, 2026 [October 15th, 2026]
- What The Tech: How criminals are tricking artificial intelligence platforms - WAKA 8 - October 15th, 2026 [October 15th, 2026]
- Open to AI: Hollywood Is Embracing Artificial Intelligence Tools Despite Doomsday Fears - Variety - October 15th, 2026 [October 15th, 2026]
- Is artificial intelligence making weather forecasts better? - mydailyrecord.com - October 15th, 2026 [October 15th, 2026]
- Apple to launch artificial intelligence for iPhones in China, but agreed to restrictions imposed by authorities Bloomberg - () - October 15th, 2026 [October 15th, 2026]
- Most Americans think artificial intelligence is developing too fast, a new AP-NORC poll finds - The Commercial Dispatch - October 15th, 2026 [October 15th, 2026]
- AI research must drive practical solutions for Namibia Steenkamp Staff Reporter YOUNG researchers have been urged to use artificial intelligence (AI)... - October 15th, 2026 [October 15th, 2026]
- Houston Emergency Physician Discusses the Role of Artificial Intelligence in Emergency Medicine - EIN News - October 15th, 2026 [October 15th, 2026]
- Is artificial intelligence making weather forecasts better? - thedigitalcourier.com - October 15th, 2026 [October 15th, 2026]
- The Surprising Way Artificial Intelligence (AI) Chatbots Are Giving Walmart's Business a Boost - The Motley Fool - October 15th, 2026 [October 15th, 2026]
- Brazil Potash conducts tests with artificial intelligence in a US$2.5 billion project in Amazonas - BNamericas - October 15th, 2026 [October 15th, 2026]