Newbies encounter some problems when installing (And some confusion about GNN performance)

[maziheng31]

I am not a professional computer engineer, so I would like to point out some problems I encountered during the installation process, which may be inaccurate.

Problem1: pip install graph-weather

ModuleNotFoundError: No module named 'graph_weather.data.nnjai_wrapp'

The installed PyPI version of the project is incomplete.

Problem2: environment_cuda.yml

Direct installation of the environment may result in errors because require python=3.12. The CUDA version of PyTorch currently only supports Python 3.11 in conda. For 3.12, you can only install the official CUDA wheel using pip.

Problem3: Project needs 'anemoi.datasets'

However, anemoi depends on plotly.plotly, which is unfortunately deprecated.

Discussion on GNN performance

In general this is a very good library. I am currently having some doubts in the process of learning GNN and hope to get some inspiration from this library.

In the article "Forecasting Global Weather with GNN", there is a clear comparison with the CNN model and shows that GNN performs much better. I couldn't find the code on the original author's homepage, so I came to your project library to look for inspiration.

But in fact, when I was testing AI meteorology model, I found that the performance of GNN on meteorological analysis/reanalysis data was far inferior to that of CNN designed for structured data.

The author of HRRRCast seems to have made similar findings. I would like to ask you professional researchers, how does the performance of GNN compare to that of CNN? Is there any comprehensive comparison between GNN and CNN under the same conditions?

[jacobbieker]

Sorry for the issues with installing it, I can look to try fixing those soon.

As for the GNN vs CNN performance, you are right in that CNNs can do quite well. A lot of the state of the art global models are GNNs though as of right now, including GenCast and FGN from Google, and AIFS from ECMWF. HRRRCast is working on a more limited domain, not globally, so the benefits of a graph might be less apparent there, vs trying to process a sphere with CNNs that need a regular grid, when a GNNs flexibiity is very helpful there.

There are other, non-GNN state of the art models, like FourCastNetv3, which uses spherical harmonics for the processing. In general, for global forecasting, it seems like building in some way of natively encoding the spherical nature of the world is helpful for forecasting and weather applications. For more limited area models, that is less important, and CNNs might do better, as its easier to represent a small area as a regular grid without as much distortion as when trying to do it for the full world.

[maziheng31]

Thank you for your very quick response.

I tried reproducing OpenCastKit's graphcast using global data, but the performance still didn't meet my expectations.

Many current projects favor GNNs due to their versatility and physical properties, but perhaps I haven't yet seen the benefits of converting structured meteorological data into an unstructured graph.

I'll try to conduct more tests using GNN and CNN models. I hope to stay in touch and discuss further.

[jacobbieker]

Would be very interested to hear more of your results from looking at it.

One benefit where GNNs I think also come into usefulness is when going straight from observations, where a lot of the time there is less structure, such as from the NASA-NOAA Joint Archive (NNJA), which is structured as a stream of individual point measurements. With these, discretizing the observations risks losing information and could harm downstream performance.

But yes, CNNs do have strong performance here still, and will be interesting to see how it continues to develop.

[CodeVishal-17]

Thanks for the detailed discussion — this was really helpful to read through.

I’m still pretty new to this space, but while trying to understand the installation and performance side, it felt like two things might be confusing for newcomers:

• On the install side, when something is missing (like the ninja wrapper / dataset deps), the errors show up pretty deep in the stack. I initially wasn’t sure what I’d done wrong until reading this issue. Maybe even a small note in the README about common install pitfalls or supported Python/PyTorch versions could help.

• On the GNN vs CNN part, the explanations here made things much clearer. Before this, I think I came in expecting GNNs to always outperform CNNs, which obviously isn’t true. It might help future users if expectations around where GNNs help (irregular grids, sparse observations, smaller regions) were spelled out somewhere.

Just sharing a newcomer perspective — happy to help with docs or small fixes if that’s useful.