Expanding AI tools and resources for developers and data scientists on Azure

Artificial Intelligence (AI) has emerged as one of the most disruptive forces behind the digital transformation of business. Our mission is to bring AI to every developer and every organization on the planet, and help businesses augment human ingenuity in unique and differentiated ways. Developers and data scientists are at the heart of driving this innovation force and we are committed to providing them the best tools to make them successful.

In September, at Microsoft Ignite 2017, I announced tools for the AI-driven Digital Transformation and described how the Microsoft AI platform enables a rich variety of application scenarios. Today, at Connect(); 2017, we are enabling developers and data scientists to infuse AI into new and existing applications quickly and easily with the following announcements:

1. Updates to Azure Machine Learning (AML) including Azure IoT Edge integration.
2. A new Azure Databricks service that combines the best of Databricks and Azure for spark-based analytics.
3. A new Visual Studio Tools for AI development environment with Azure Machine Learning integration.

Updates to Azure Machine Learning (Azure ML)

AI should be accessible to every developer and we are working to make this a reality every day. Azure Machine Learning continues to simplify the development of AI models and provides comprehensive features to deploy them in cloud applications. Two months ago at Microsoft Ignite 2017, we announced a powerful set of new capabilities in Azure ML (Figure 1), namely:

• The Azure ML Workbench, a cross-platform client for AI-powered data wrangling and experiment management.
• The Azure ML Experimentation Service, to help data scientists increase their rate of experimentation with big data and GPUs.
• The Azure ML Model Management Service, to host, version, manage and monitor machine learning models.

Figure 1: AML provides AI-powered data wrangling, Spark and GPU based training, experimentation and model management, and the ability for developers to deploy AI models on Internet of Things (IoT) devices using Azure IoT Edge.

Azure ML brings the best of the cloud to AI development, and the best of AI tools to cloud developers. By leveraging the power of big data services in the cloud such as Spark on HDInsight, distributed databases such as CosmosDB, and services like Azure Data Factory, developers can integrate and transform vast amounts of data from a variety of sources to power machine learning. Using technologies such as Program Synthesis by Example, data can be easily wrangled, cleaned and transformed. Using Data Science VMs, GPUs and/or Spark clusters, developers can rapidly experiment and train machine learning models at scale. Models so developed can be exported to docker containers for inferencing at the edge, or to deploy auto-scalable web services in the Azure Container Service. A comprehensive collection of AI toolkits are supported, including the Microsoft Cognitive Toolkit (CNTK), Caffe2 and TensorFlow.

Azure ML is powerful end-to-end tooling for AI application development.

Today, I am excited to announce we are further expanding on the capabilities of Azure ML:

1. IoT Intelligent Edge Capabilities: Now AI models built with Azure ML can be easily deployed to run locally on Internet of Things (IoT) devices using Azure IoT Edge. Using docker containers, developers can now deploy to the edge seamlessly and build hybrid IoT solutions. Today we are also releasing the AI Toolkit for Azure IoT Edge, a collection of scripts, code, and deployable containers that enable you to quickly get up and running to deploy AI on edge devices. Additional details about Azure IoT Edge can be found in this blog.
2. Expanding to additional Azure Regions: We will be adding the workbench, experimentation and model management capabilities of Azure Machine Learning to two new regions – Western Europe and SE Asia – bringing our overall footprint to the US, Europe, Asia, and Australia.

We are also expanding Azure ML features to make development even easier, such as Support for VS Code, VS Tools and PyCharm in addition to Azure ML Workbench text editor, Integration with Matplotlib (version 2.1.0), a popular plotting library, MMLSpark updates including medical entity extraction models and Run History improvements and many more. These enhancements are part of the continuous set of improvements to our tooling and services that make developers and data scientists more productive.

Announcing Azure Databricks

Today I am excited to announce the preview of Azure Databricks (Figure 2), a fast, easy and collaborative Apache® Spark™-based analytics platform optimized for Azure. Designed in collaboration with the founders of Apache Spark, Azure Databricks combines the best of Databricks and Azure to help customers accelerate innovation with one-click set up, streamlined workflows, and an interactive workspace that enables collaboration between developers, data scientists, data engineers, and business analysts.

Figure 2: Azure Databricks combines the best of Databricks and Azure with streamlined workflows, and an interactive workspace enabling collaboration between data scientists, data engineers, and business analysts.

The preview includes the following capabilities:

1. Native integration with Azure SQL Data Warehouse, Azure Storage, Azure Cosmos DB, and Power BI, simplifying self-service analytics and machine learning over both relational and non-relational data.
2. Azure Active Directory integration, for simplified security and identity control.
3. Azure’s inherent enterprise-grade performance, data governance, compliance, and SLAs.

With these innovations, Azure is the one-stop destination to unlock powerful scenarios that make AI easy.

Announcing Visual Studio Tools for AI

We are focused on giving developers the ability to use the tools of their choice for AI development. At Ignite 2017 we announced the cross-platform Visual Studio Code Tools for AI, and today we’re increasing the IDE choices that developers have by releasing Visual Studio Tools for AI (Figure 3), empowering developers to create, train, debug, and deploy their models in the world class IDE on Windows.

Figure 3: Visual Studio Tools for AI allow developers to easily develop sophisticated deep neural network models and deploy them in production through Azure Machine Learning.

Visual Studio Tools for AI allows developers and data scientists to quickly and easily accomplish common tasks when working with their custom models in Visual Studio. With Azure ML integration, developers and data scientists can browse the sample gallery, download sample code, open, edit, and debug python scripts from Azure ML projects in Visual Studio. They can also run scripts locally to test, submit jobs to a remote compute target defined in Azure ML, review the list of jobs submitted to the remote compute target, get details around those jobs, and finally access outputs and logs from submitted jobs.

The tools also include integration with the Azure Batch AI service, and developers and data scientists can use it to submit jobs to an Azure Batch cluster, which dynamically spins up GPU VMs, allocates and runs the job, and then spins down the VMs, all from within Visual Studio. The integration also shows a list of the jobs submitted to Azure Batch AI, provides details about the submitted jobs, and demonstrates the outputs and logs from submitted jobs.

Visual Studio Tools for AI also supports using Linux Azure VMs or on-premises machines to submit jobs to an Azure GPU VM or on-premises Linux server, and then to view the jobs and the outputs. In addition, we’ve incorporated a GPU utilization heatmap for resource monitoring.

Summary

With these exciting updates, the Microsoft AI platform (Figure 4) now offers comprehensive cloud-based, on-premises, and edge support – all the infrastructure, tools, frameworks, services and solutions needed for developers, data scientists and businesses to infuse AI into their products and services.

Figure 4: Microsoft AI platform, provides a comprehensive cloud powered AI for every developer.

Our customers are using the AI platform to help augment human ingenuity in new ways — our AI platform allows people to extend their capabilities and tap into even greater potential. An example is the Helpicto application (Figure 5) developed by a French startup called Equadex. Helpicto provides an innovative solution to help children with autism communicate more easily with their environment, based on pictograms and associated keywords. To stay focused, a child with autism usually uses specific pictograms in a sequence to recognize a situation, a question, or another interaction. Helpicto makes this easy (Figure 4), by building on the Microsoft AI Platform using Xamarin, ASP.NET, Azure App Services, Azure SQL Database and Cognitive Services. This is a great example of using the power of AI to improve lives.

Figure 5: Helpicto helps autistic children communicate better using AI. 

I invite you to visit www.azure.com/ai (Figure 6) to learn more about how AI can augment and empower digital transformation efforts. We’ve also launched the AI School (Figure 6) to help developers get up to speed with all of these AI technologies.

Figure 6: Resources for developers to get started with AI technologies.

Dive in and learn how to infuse AI into your applications today.

Joseph
@josephsirosh

Source: Azure Blog Feed