Azure high-performance computing at Supercomputing 2021

9th November 2021 Anthony Mashford 0 Comments

Customers around the world rely on Microsoft Azure to drive innovations related to our environment, public health, energy sustainability, weather modeling, economic growth, and more. Finding solutions to these important challenges requires huge amounts of focused computing power. Customers are increasingly finding the best way to access such high-performance computing (HPC) is through the agility, scale, security, and leading-edge performance of Azure’s purpose-built HPC and AI cloud services.

Azure’s market-leading vision for HPC and AI is based on a commitment to continual improvement and backed by a core of genuine and recognized HPC expertise, using proven HPC technology and design principles, enhanced with the best features of the cloud. The result is a capability that delivers performance, scale, and value unlike any other cloud. This means HPC application scaling efficiency of up to 200 percent for specific workloads. It’s the ability for customer’s to be able to rent supercomputer scale for AI workloads. And it means delivering massive compute power into the hands of medical researchers over a weekend to prove out life-saving innovations in the fight against COVID-19.

New Azure leadership unveiled at Supercomputing 2021 (SC21)

Azure at the forefront of HPC with AMD EPYC “Milan-X” CPUs

We are excited to announce our preview is now live for Azure HBv3 virtual machines enhanced by AMD EPYC 3rd Gen processors with 3D v-cache, codenamed “Milan-X”. These processors significantly improve the performance, scaling efficiency, and cost effectiveness of a variety of memory performance-bound workloads such as CFD, explicit finite element analysis, computational geoscience, weather simulation, and silicon design RTL workflows. See our launch blog for additional details.

New Azure OpenAI Service combines access to powerful GPT-3 language models

At the Microsoft Ignite conference today, Microsoft announced it will help its customers uncover these kinds of experiences with the new Azure OpenAI Service, which allows access to OpenAI’s API through the Azure platform and will initially be available by invite only. The new Azure Cognitive Service will give customers access to OpenAI’s powerful GPT-3 models, along with security, reliability, compliance, data privacy and other enterprise-grade capabilities that are built into Microsoft Azure.

Stay tuned for an additional product announcement on Monday, November 15th.

Big moments for Azure HPC and AI Supercomputing in 2021

Vestas supercharges its wind farm control models for sustainable energy with Azure HPC

Global leader in sustainable energy solutions Vestas Wind Systems A/S wanted to optimize wind energy production by reducing the negative impact of turbine wakes. Working with Microsoft and partner minds.ai, Vestas ran a reinforcement learning engine using machine learning and Microsoft Azure high-performance computing and storage resources. Proof of concept complete, Vestas now has the tools to generate simulations that offer the potential to help wind farms mitigate wake effect, generate more wind energy, and build a sustainable and prosperous energy future.

AMD IT satisfies an insatiable appetite for capacity, scalablity, and innovation with Azure HPC

As a leader in semiconductor, computer processors, and related technologies, AMD has a responsibility to serve its customers, keep pace with the industry, and help set the standard for how servers, computers, and embedded systems run. To maintain its execution track record, the IT team at AMD used Microsoft Azure high-performance computing (HPC), HBv3 virtual machines, and other Azure resources to build scalable capacity and optimize the company’s cloud capabilities, accelerating time to market and eliminating weeks and even months of delay.

Sensyne Health aids National Health Service in the COVID-19 struggle with Azure HPC and AI technologies

When the United Kingdom’s National Health Service envisioned an app that could read COVID-19 tests at scale with an accuracy and speed not possible by human readers, Sensyne Health answered the call. The app connects users to a web service that uses AI to deliver an Azure high-performance computing (HPC) solution that’s adeptly orchestrated with NVIDIA Triton Inference Server on graphics processing units. The result—the incredibly fast and accurate MagnifEye solution.

UK Met Office supercomputing announcement

The UK Met Office supercomputing announcement highlighted that Microsoft had been awarded a contract to deliver a ten-year managed supercomputing service, providing advanced supercomputing capabilities for weather and climate research, ensuring the continuation of the Met Office’s leadership in this area.

Previous 2021 Azure HPC and AI product launches

Azure HBv3 virtual machine

Azure announced the general availability of the Azure HBv3 virtual machine for HPC workloads advancing the velocity at which we bring the latest technologies to our HPC customers and the compute performance we put at their fingertips.

Azure ND A100 v4 virtual machine

Azure announced the general availability of scale-out NVIDIA A100 GPU Clusters with HDR 200 Gb/s InfiniBand networking—the fastest public cloud supercomputer. The ND A100 v4 VM is powered by NVIDIA A100 Tensor Core GPUs and is designed to let our most demanding customers scale up and scale out without ever slowing down.

In the June 2021 TOP500 list Microsoft Azure took public cloud services to a new level, demonstrating work on systems that took four consecutive spots from number 26 to number 29 on the TOP500 list. They are parts of a global AI supercomputer called the ND A100 v4 cluster, available on demand in four global regions today. These rankings were achieved on a fraction of our overall cluster size. Each of the systems delivered 16.59 petaflops on the HPL benchmark also known as Linpack, a traditional measure of HPC performance on 64-bit floating point math that’s the basis for the TOP500 rankings.

Megatron-Turing Natural Language Model

Azure announces the DeepSpeed-and Megatron-powered Megatron-Turing Natural Language Generation model (MT-NLG), the largest and the most powerful monolithic transformer language model trained to date, with 530 billion parameters. It is the result of a research collaboration between Microsoft and NVIDIA to further parallelize and optimize the training of very large AI models.

Azure FX-series Virtual Machine

Azured announced general availability of the Azure FX-series Virtual Machine, specifically designed to optimize electronic design automation (EDA) workload processing with a purpose-built virtual machine architected for EDA workloads. For more details see the FX-series announcement Azure blog, "Optimize extreme computing performance with Azure FX-series Virtual Machines" as well as the Ansys RedHawk announcement, "How Azure FX VM Makes Ansys RedHawk-SC™ Run Faster the Less You Spend."

Learn more about Azure HPC and AI from the experts

Join us at Supercomputing 2021 (SC21) virtual booth to learn more about Azure HPC and AI

We will feature speakers in SC21 virtual sessions, provide links to downloads, case studies, blogs, and additional Azure HPC videos. Please stop by our virtual booth to chat with us. We’d love to hear from you.

Azure HPC + AI sessions sill cover a range of topics including overview, HPC + AI platform at a glance, Quantum, HPC journey to the cloud, and weather and climate modeling.

MicroHack

Monday, November 15 from 9:00 AM to 11:00 AM PST.

Attend our Micro hack to experience a walkthrough in creating an HPC cluster on Azure running a typical HPC workload.

Want to know more?
Check out this blog for more information on the environment we’ll be using in the session.

Interested? Sign up SC21 Azure HPC MicroHack.

Azure HPC Customer Panel

Please join us, free of charge, for a special event with Microsoft customers, where our six panelists will discuss their experiences in moving on an on-premises infrastructure to the cloud. Panelists represent a range of industries from research to science to visual effects.

The event will be held on September 16 at 9:00 AM PST and is free to attend. Don’t forget to chat with us in our discord channel during the event.

Customers Present:

Liberty University
University of Bath
Applied Materials
AstraZeneca
UK Met Office
Jellyfish

Additional resources

Source: Azure Blog Feed

Mashford's Musings