Hacker News new | past | comments | ask | show | jobs | submit login

What you're describing is effectively how climate models work; we run a physical model which solves the equations that govern how the atmosphere works out forward in time for very long time integrations. You get "daily weather" out as far as you choose to run the model.

But this isn't a "weather forecast." Weather forecasting is an initial value problem - you care a great deal about how the weather will evolve from the current atmospheric conditions. Precisely because weather is a result of what happens in this complex, 3D fluid atmosphere surrounding the Earth, it happens that small changes in those initial conditions can have a very big impact on the forecast on relatively short time-periods - as little as 6-12 hours. Small perturbations grow into larger ones and feedback across spatial scales. Ultimately, by day ~3-7, you wind up with a very different atmospheric state than what you'd have if you undid those small changes in the initial conditions.

This is the essence of what "chaos" means in the context of weather prediction; we can't perfectly know the initial conditions we feed into the model, so over some relatively short time, the "model world" will start to look very different than the "real world." Even if we had perfect models - capable of representing all the physics in the atmosphere - we'd still have this issue as long as we had to imperfectly sample the atmosphere for our initial conditions.

So weather isn't inherently "unpredictable." And in fact, by running lots of weather models simultaneously with slightly perturbed initial conditions, we can suss out this uncertainty and improve our estimate of the forecast weather. In fact, this is what's so exciting to meteorologists about the new AI models - they're so much cheaper to run that we can much more effectively explore this uncertainty in initial conditions, which will indirectly lead to improved forecasts.




So isn’t it just a problem of measurement then?

Say you had a massive array of billions of perfect sensors in different locations, and had all the computing power to process this data, would an N year daily forecast then be a solved problem?

For the sake of the argument I’m ignoring ”external” factors that could affect the weather (e.g meteors hitting earth, changes in man-made pollution, etc)


At that point you're slipping into Laplace's Demon.

In practical terms, we see predictability horizons get _shorter_ when we increase observation density and spatial resolution of our models, because more, small errors from slightly imperfect observations and models still cascade to larger scales.


is it possible to self-correct, looking at initial value errors in the past? Is it too hard to prescribe the error in the initial value?


Yes, this is effectively what 4DVar data assimilation is [1]. But it's very, very expensive to continually run new forecasts with re-assimilated state estimates. Actually, one of the _biggest_ impacts that models like GraphCast might have is providing a way to do exactly this - rapidly re-running the forecast in response to updated initial conditions. By tracking changes in the model evolution over subsequent re-initializations like this, one could might be able to better quantify expected forecast uncertainty, even moreso than just by running large ensembles.

Expect lots of R&D in this area over the next two years...

[1]: https://www.ecmwf.int/en/about/media-centre/news/2022/25-yea...




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: