Using AI to expand global access to reliable flood forecasts – Google Research
Posted by Yossi Matias, VP Engineering & Research, and Grey Nearing, Research Scientist, Google Research
Floods are the most common natural disaster, and are responsible for roughly $50 billion in annual financial damages worldwide. The rate of flood-related disasters has more than doubled since the year 2000 partly due to climate change. Nearly 1.5 billion people, making up 19% of the worlds population, are exposed to substantial risks from severe flood events. Upgrading early warning systems to make accurate and timely information accessible to these populations can save thousands of lives per year.
Driven by the potential impact of reliable flood forecasting on peoples lives globally, we started our flood forecasting effort in 2017. Through this multi-year journey, we advanced research over the years hand-in-hand with building a real-time operational flood forecasting system that provides alerts on Google Search, Maps, Android notifications and through the Flood Hub. However, in order to scale globally, especially in places where accurate local data is not available, more research advances were required.
In Global prediction of extreme floods in ungauged watersheds, published in Nature, we demonstrate how machine learning (ML) technologies can significantly improve global-scale flood forecasting relative to the current state-of-the-art for countries where flood-related data is scarce. With these AI-based technologies we extended the reliability of currently-available global nowcasts, on average, from zero to five days, and improved forecasts across regions in Africa and Asia to be similar to what are currently available in Europe. The evaluation of the models was conducted in collaboration with the European Center for Medium Range Weather Forecasting (ECMWF).
These technologies also enable Flood Hub to provide real-time river forecasts up to seven days in advance, covering river reaches across over 80 countries. This information can be used by people, communities, governments and international organizations to take anticipatory action to help protect vulnerable populations.
The ML models that power the FloodHub tool are the product of many years of research, conducted in collaboration with several partners, including academics, governments, international organizations, and NGOs.
In 2018, we launched a pilot early warning system in the Ganges-Brahmaputra river basin in India, with the hypothesis that ML could help address the challenging problem of reliable flood forecasting at scale. The pilot was further expanded the following year via the combination of an inundation model, real-time water level measurements, the creation of an elevation map and hydrologic modeling.
In collaboration with academics, and, in particular, with the JKU Institute for Machine Learning we explored ML-based hydrologic models, showing that LSTM-based models could produce more accurate simulations than traditional conceptual and physics-based hydrology models. This research led to flood forecasting improvements that enabled the expansion of our forecasting coverage to include all of India and Bangladesh. We also worked with researchers at Yale University to test technological interventions that increase the reach and impact of flood warnings.
Our hydrological models predict river floods by processing publicly available weather data like precipitation and physical watershed information. Such models must be calibrated to long data records from streamflow gauging stations in individual rivers. A low percentage of global river watersheds (basins) have streamflow gauges, which are expensive but necessary to supply relevant data, and its challenging for hydrological simulation and forecasting to provide predictions in basins that lack this infrastructure. Lower gross domestic product (GDP) is correlated with increased vulnerability to flood risks, and there is an inverse correlation between national GDP and the amount of publicly available data in a country. ML helps to address this problem by allowing a single model to be trained on all available river data and to be applied to ungauged basins where no data are available. In this way, models can be trained globally, and can make predictions for any river location.
Our academic collaborations led to ML research that developed methods to estimate uncertainty in river forecasts and showed how ML river forecast models synthesize information from multiple data sources. They demonstrated that these models can simulate extreme events reliably, even when those events are not part of the training data. In an effort to contribute to open science, in 2023 we open-sourced a community-driven dataset for large-sample hydrology in Nature Scientific Data.
Most hydrology models used by national and international agencies for flood forecasting and river modeling are state-space models, which depend only on daily inputs (e.g., precipitation, temperature, etc.) and the current state of the system (e.g., soil moisture, snowpack, etc.). LSTMs are a variant of state-space models and work by defining a neural network that represents a single time step, where input data (such as current weather conditions) are processed to produce updated state information and output values (streamflow) for that time step. LSTMs are applied sequentially to make time-series predictions, and in this sense, behave similarly to how scientists typically conceptualize hydrologic systems. Empirically, we have found that LSTMs perform well on the task of river forecasting.
Our river forecast model uses two LSTMs applied sequentially: (1) a hindcast LSTM ingests historical weather data (dynamic hindcast features) up to the present time (or rather, the issue time of a forecast), and (2) a forecast LSTM ingests states from the hindcast LSTM along with forecasted weather data (dynamic forecast features) to make future predictions. One year of historical weather data are input into the hindcast LSTM, and seven days of forecasted weather data are input into the forecast LSTM. Static features include geographical and geophysical characteristics of watersheds that are input into both the hindcast and forecast LSTMs and allow the model to learn different hydrological behaviors and responses in various types of watersheds.
Output from the forecast LSTM is fed into a head layer that uses mixture density networks to produce a probabilistic forecast (i.e., predicted parameters of a probability distribution over streamflow). Specifically, the model predicts the parameters of a mixture of heavy-tailed probability density functions, called asymmetric Laplacian distributions, at each forecast time step. The result is a mixture density function, called a Countable Mixture of Asymmetric Laplacians (CMAL) distribution, which represents a probabilistic prediction of the volumetric flow rate in a particular river at a particular time.
The model uses three types of publicly available data inputs, mostly from governmental sources:
Training data are daily streamflow values from the Global Runoff Data Center over the time period 1980 - 2023. A single streamflow forecast model is trained using data from 5,680 diverse watershed streamflow gauges (shown below) to improve accuracy.
We compared our river forecast model with GloFAS version 4, the current state-of-the-art global flood forecasting system. These experiments showed that ML can provide accurate warnings earlier and over larger and more impactful events.
The figure below shows the distribution of F1 scores when predicting different severity events at river locations around the world, with plus or minus 1 day accuracy. F1 scores are an average of precision and recall and event severity is measured by return period. For example, a 2-year return period event is a volume of streamflow that is expected to be exceeded on average once every two years. Our model achieves reliability scores at up to 4-day or 5-day lead times that are similar to or better, on average, than the reliability of GloFAS nowcasts (0-day lead time).
Additionally (not shown), our model achieves accuracies over larger and rarer extreme events, with precision and recall scores over 5-year return period events that are similar to or better than GloFAS accuracies over 1-year return period events. See the paper for more information.
The flood forecasting initiative is part of our Adaptation and Resilience efforts and reflects Google's commitmentto address climate change while helping global communities become more resilient. We believe that AI and ML will continue to play a critical role in helping advance science and research towards climate action.
We actively collaborate with several international aid organizations (e.g., the Centre for Humanitarian Data and the Red Cross) to provide actionable flood forecasts. Additionally, in an ongoing collaboration with the World Meteorological Organization (WMO) to support early warning systems for climate hazards, we are conducting a study to help understand how AI can help address real-world challenges faced by national flood forecasting agencies.
While the work presented here demonstrates a significant step forward in flood forecasting, future work is needed to further expand flood forecasting coverage to more locations globally and other types of flood-related events and disasters, including flash floods and urban floods. We are looking forward to continuing collaborations with our partners in the academic and expert communities, local governments and the industry to reach these goals.
Read more:
Using AI to expand global access to reliable flood forecasts - Google Research
- Unlock real-time insights with AI-powered analytics in Microsoft Fabric - Microsoft - May 22nd, 2024 [May 22nd, 2024]
- Companies Are Drowning In Visual Data. Coactive AI Wants To Make Sense Of It. - Forbes - May 22nd, 2024 [May 22nd, 2024]
- Holistic AI Is Now H, a New French Startup With $220 Million Out of the Gate - Yahoo Finance - May 22nd, 2024 [May 22nd, 2024]
- Google Search's New AI Overviews Will Soon Have Ads - WIRED - May 22nd, 2024 [May 22nd, 2024]
- Microsoft's new Windows Copilot Runtime aims to win over AI developers - The Verge - May 22nd, 2024 [May 22nd, 2024]
- DOJ arrests man over AI-generated CSAM, in likely first known case - The Washington Post - May 22nd, 2024 [May 22nd, 2024]
- Nvidia is talk of the town at AI events leading into this weeks earnings - CNBC - May 22nd, 2024 [May 22nd, 2024]
- This is how I plan to explain AI PC to my confused friends and relatives - TechRadar - May 22nd, 2024 [May 22nd, 2024]
- Get Google Search Results Without AI, Ads or Anything but Links - CNET - May 22nd, 2024 [May 22nd, 2024]
- Releasing a new paper on openness and artificial intelligence - Mozilla & Firefox - May 22nd, 2024 [May 22nd, 2024]
- AI Outperforms Humans in Theory of Mind Tests - IEEE Spectrum - May 22nd, 2024 [May 22nd, 2024]
- Cooler Master introduces colored 'AI Thermal Paste' CryoFuze 5 comes with nano-diamond technology - Tom's Hardware - May 22nd, 2024 [May 22nd, 2024]
- Five ways criminals are using AI - MIT Technology Review - May 22nd, 2024 [May 22nd, 2024]
- Microsoft Stock Flirts With Breakout On AI Software News - Investor's Business Daily - May 22nd, 2024 [May 22nd, 2024]
- Microsofts stock rides AI elation toward first record close in two months - MarketWatch - May 22nd, 2024 [May 22nd, 2024]
- Wisconsin man arrested for allegedly creating AI-generated child sexual abuse material - The Verge - May 22nd, 2024 [May 22nd, 2024]
- Beyond keywords: AI-driven approaches to improve data discoverability - World Bank - May 22nd, 2024 [May 22nd, 2024]
- AI boom may leave Israel behind due to shortage of funds and talent, report warns - The Times of Israel - May 22nd, 2024 [May 22nd, 2024]
- Microsoft's AI chatbot will 'recall' everything you do on a PC - The Associated Press - May 22nd, 2024 [May 22nd, 2024]
- Amazon is helping decarbonize the grid with AI - About Amazon - May 22nd, 2024 [May 22nd, 2024]
- Wearable AI Pin maker Humane is reportedly seeking a buyer - Engadget - May 22nd, 2024 [May 22nd, 2024]
- What Nvidia says about AI chip demand could matter for more than just the tech trade - Yahoo Finance - May 22nd, 2024 [May 22nd, 2024]
- TechScape: The people charged with making sure AI doesnt destroy humanity have left the building - The Guardian - May 22nd, 2024 [May 22nd, 2024]
- Total Recall: the only Copilot+ AI feature that matters is a huge privacy risk - Tom's Hardware - May 22nd, 2024 [May 22nd, 2024]
- The fake AI Scarlett Johansson is a reality check for Washington - POLITICO - May 22nd, 2024 [May 22nd, 2024]
- In Seoul summit, heads of states and companies commit to AI safety - TechCrunch - May 22nd, 2024 [May 22nd, 2024]
- Databricks is expanding the scope of its AI investments with second VC fund - Fortune - May 22nd, 2024 [May 22nd, 2024]
- Windows 11 to Deprecate NTLM, Add AI-Powered App Controls and Security Defenses - The Hacker News - May 22nd, 2024 [May 22nd, 2024]
- Seoul AI safety summit kicks off with tech industry at a critical juncture - Semafor - May 22nd, 2024 [May 22nd, 2024]
- Adobe's AI-Powered Generative Remove Feature in Lightroom Erases Unsightly Objects in Seconds - WIRED - May 22nd, 2024 [May 22nd, 2024]
- Voice Actors Sue Company Whose AI Sounds Like Them - The New York Times - May 20th, 2024 [May 20th, 2024]
- Missed Out on Nvidia's Run-Up? My Best Artificial Intelligence (AI) Stock to Buy and Hold - Yahoo Finance - May 20th, 2024 [May 20th, 2024]
- Here's My Top Artificial Intelligence (AI) Stock to Buy and Hold - Yahoo Finance - May 20th, 2024 [May 20th, 2024]
- How ISIS allies are using AI fakes to spread propaganda quickly - The Washington Post - May 20th, 2024 [May 20th, 2024]
- Sainsbury's and Microsoft announce five-year AI collaboration - Microsoft - May 20th, 2024 [May 20th, 2024]
- Marc Dion: Diners and AI - The Dispatch - The Commercial Dispatch - May 20th, 2024 [May 20th, 2024]
- What is an AI PC, and how it might change the tech industry - Axios - May 20th, 2024 [May 20th, 2024]
- Age of the AI agents: OpenAI's GPT-4o and Google's Project Astra, plus an exclusive sit-down with Google CEO Sundar ... - CNBC - May 20th, 2024 [May 20th, 2024]
- Alphabet Goes All In on Artificial Intelligence (AI). Is It Time to Go All In on the Stock? - The Motley Fool - May 20th, 2024 [May 20th, 2024]
- Reddit soars after announcing OpenAI deal that allows use of its data for training AI models - CNBC - May 20th, 2024 [May 20th, 2024]
- How Googles AI Overviews Work, and How to Turn Them Off (You Cant) - WIRED - May 20th, 2024 [May 20th, 2024]
- Google AI: Gemini comes to Workspace for Education - The Keyword | Google Product and Technology News - May 20th, 2024 [May 20th, 2024]
- Exclusive | CoreWeave Raises $7.5 Billion in Debt for AI Computing Push - The Wall Street Journal - May 20th, 2024 [May 20th, 2024]
- How can AI be used in education? From ABCs to SATs - USA TODAY - May 20th, 2024 [May 20th, 2024]
- Among AI infrastructure hopefuls, Qualcomm has become an unlikely ally - The Register - May 20th, 2024 [May 20th, 2024]
- We have to stop ignoring AI's hallucination problem - The Verge - May 20th, 2024 [May 20th, 2024]
- How Google's Gemini AI model got its name - The Keyword | Google Product and Technology News - May 20th, 2024 [May 20th, 2024]
- SoundHound AI: Should Investors Buy This Artificial Intelligence (AI) Stock After 160% Gains? - The Motley Fool - May 20th, 2024 [May 20th, 2024]
- Bye Bye, AI: How to turn off Google's annoying AI overviews and just get search results - Tom's Hardware - May 20th, 2024 [May 20th, 2024]
- As the AI world gathers in Seoul, can an accelerating industry balance progress against safety? - The Guardian - May 20th, 2024 [May 20th, 2024]
- Microsoft delivers an AI blow to Nvidia - TheStreet - May 20th, 2024 [May 20th, 2024]
- Hollywood at a Crossroads: Everyone Is Using AI, But They Are Scared to Admit It - Hollywood Reporter - May 20th, 2024 [May 20th, 2024]
- 3 Companies Already Working on the Next Phase of Artificial Intelligence (AI) - The Motley Fool - May 20th, 2024 [May 20th, 2024]
- Hollywood agency CAA aims to help stars manage their own AI likenesses - TechCrunch - May 20th, 2024 [May 20th, 2024]
- Dow Jones Futures Rise; Nvidia Earnings Offer Huge Test With 5 AI Stocks Near Buy Points - Investor's Business Daily - May 20th, 2024 [May 20th, 2024]
- 'Copper is the new oil,' and prices could soar 50% as AI, green energy, and military spending boost demand, top ... - Fortune - May 20th, 2024 [May 20th, 2024]
- New support for AI advancement in Central and Eastern Europe - The Keyword | Google Product and Technology News - May 20th, 2024 [May 20th, 2024]
- OpenAI dissolves team focused on long-term AI risks, less than one year after announcing it - CNBC - May 20th, 2024 [May 20th, 2024]
- Caution: 3 Things You Must Know About C3.ai Stock - The Motley Fool - May 20th, 2024 [May 20th, 2024]
- Business school teaching case study: risks of the AI arms race - Financial Times - May 20th, 2024 [May 20th, 2024]
- Congressman Don Beyer went back to college to learn AI - The Associated Press - April 15th, 2024 [April 15th, 2024]
- Georgia Tech Unveils New AI Makerspace in Collaboration with NVIDIA - Georgia Tech College of Engineering - April 15th, 2024 [April 15th, 2024]
- Energy-Guzzling AI Is Also the Future of Energy Savings - The Wall Street Journal - April 15th, 2024 [April 15th, 2024]
- South Korea to host second AI Safety Summit on May 21-22 - Reuters - April 15th, 2024 [April 15th, 2024]
- What is artificial intelligence (AI)? - Livescience.com - April 15th, 2024 [April 15th, 2024]
- 'Jailbreaking' AI services like ChatGPT and Claude 3 Opus is much easier than you think - Livescience.com - April 15th, 2024 [April 15th, 2024]
- Google AI podcast: 6 conversations with global leaders - The Keyword | Google Product and Technology News - April 15th, 2024 [April 15th, 2024]
- Galaxy AI features are coming to last-gen Samsung phones including the S21 series - The Verge - April 15th, 2024 [April 15th, 2024]
- AI's Most Promising Startups Are Getting Younger And Leaner - Forbes - April 15th, 2024 [April 15th, 2024]
- How to Stop Your Data From Being Used to Train AI - WIRED - April 15th, 2024 [April 15th, 2024]
- 7 of the best Sora AI videos featuring animals - Tom's Guide - April 15th, 2024 [April 15th, 2024]
- How Microsoft discovers and mitigates evolving attacks against AI guardrails - Microsoft - April 15th, 2024 [April 15th, 2024]
- Apple's First AI Features in iOS 18 Reportedly Won't Use Cloud Servers - MacRumors - April 15th, 2024 [April 15th, 2024]
- Samsung officially bringing One UI 6.1 and AI features to Galaxy S22, Fold 4, Flip 4 in May - 9to5Google - April 15th, 2024 [April 15th, 2024]
- Eat the future, pay with your face: my dystopian trip to an AI burger joint - The Guardian - April 15th, 2024 [April 15th, 2024]
- Humans Forget. AI Assistants Will Remember Everything - WIRED - April 15th, 2024 [April 15th, 2024]
- From boom to burst, the AI bubble is only heading in one direction - The Guardian - April 15th, 2024 [April 15th, 2024]
- Meta and Google announce new in-house AI chips, creating a trillion-dollar question for Nvidia - Fortune - April 15th, 2024 [April 15th, 2024]
- Google goes all in on generative AI at Google Cloud Next - TechCrunch - April 15th, 2024 [April 15th, 2024]
- Humane AI Pin review: the post-smartphone future isnt here yet - The Verge - April 15th, 2024 [April 15th, 2024]
Tags: