A new study highlights how current AI models fall short in predicting unprecedented weather events, underscoring the need for integrating more physics and mathematical tools to improve forecasting.
Researchers from the University of Chicago, in collaboration with New York University and the University of California Santa Cruz, have revealed the limitations of AI-powered weather prediction. Their findings, published in the Proceedings of the National Academy of Sciences, indicate that while neural networks excel at routine weather forecasting, they struggle with predicting unprecedented extreme weather events.
AI models have fundamentally transformed the field of meteorology, making accurate short-term forecasts with significantly less computational power compared to traditional models.
“AI weather models are one of the biggest achievements in AI in science,” co-corresponding author Pedram Hassanzadeh, an associate professor of geophysical sciences at UChicago, said in a news release. “What we found is that they are remarkable, but not magical.”
However, these models fall short when tasked with predicting “gray swan” events — extreme weather occurrences that surpass the scope of the training data.
Neural networks rely on past weather patterns, which means their predictive capabilities are limited by the historical data they are fed. For instance, they would struggle to foresee a once-in-a-2000-year event like the floods caused by Hurricane Harvey in 2017.
To test the limits of current models, the researchers trained a neural network on weather data while excluding instances of Category 3 or stronger hurricanes. When asked to predict a Category 5 hurricane, the model could not do so.
“It always underestimated the event. The model knows something is coming, but it always predicts it’ll only be a Category 2 hurricane,” added co-corresponding author Yongqiang Sun, a research scientist at UChicago.
This issue points to a significant flaw — predicting false negatives — whereby the model fails to identify the severity of extreme weather, potentially underestimating the risks and causing disastrous consequences.
Unlike traditional models that incorporate mathematical and physical principles governing atmospheric conditions, neural networks operate solely based on previously observed patterns. This limitation has significant implications as the use of AI extends to operational weather forecasting and early warning systems.
The research suggests that integrating mathematical tools and the principles of atmospheric physics into AI models may address these deficiencies.
“The hope is that if AI models can really learn atmospheric dynamics, they will be able to figure out how to forecast gray swans,” Hassanzadeh added.
One promising method is “active learning,” where AI can guide traditional physics-based models to simulate more extreme events, thus improving the AI’s training data.
“Longer simulated or observed datasets aren’t going to work. We need to think about smarter ways to generate data,” added co-author Jonathan Weare, a professor at NYU’s Courant Institute of Mathematical Sciences. “In this case, that means answering the question ‘where should I place my training data to achieve better performance on extremes?’ Fortunately, we think AI weather models themselves, when paired with the right mathematical tools, can help answer this question.”
Source: University of Chicago