Cray in Azure for weather forecasting
When we announced our partnership with Cray, it was very exciting news. I received my undergraduate degree in meteorology, so my mind immediately went to how this could be a benefit to weather forecasting.
Weather modeling is an interesting use case. It requires a large number of cores with a low-latency interconnect, and it is very time sensitive. After all, what good is a one hour weather forecast if it takes 90 minutes to run? And weather is a very local phenomenon. In order to resolve smaller scale features without shrinking the domain or lengthening runtime, modelers must add more cores. A global weather model with a 0.5 degree grid spacing can require as many as 50,000 cores.
At that large of a scale, and with the performance required to be operationally useful, a Cray supercomputer is an excellent fit. But the model by itself doesn’t mean much. The model data needs to be processed to generate products. This is where Azure services come in.
Website images are one obvious product of weather models. Image generation programs require small scale and can be done in parallel, so they’re great for using the elasticity of Azure virtual machines. The same can be said for generating model output statistics forecasts, a form of forecast that applies statistical regression to the raw model output which eliminates bias and adds fields that are not directly forecasted in the model. Artificial Intelligence is starting to be used as a forecasting tool as well.
To put this end-to-end workflow together requires more than just the Cray supercomputer. It would use virtual machines, perhaps with Batch to manage tasks. It would use storage: disks attached to virtual machines, blob storage for scalability, and archive storage to hold the raw data for later re-analysis. AI-generated forecasts can use Azure’s broad suite of AI products. And if you’re serving web images, the Azure CDN provides reliable product delivery.
As you can see, Cray in Azure is an important piece of a larger computing ecosystem. Let us know how Cray in Azure can help your HPC workloads.
Source: Azure Updates