Azure.Source – Volume 62

17th December 2018 Anthony Mashford 0 Comments

KubeCon North America 2018

KubeCon North America 2018: Serverless Kubernetes and community led innovation!

Brendan Burns, Distinguished Engineer in Microsoft Azure and co-founder of the Kubernetes project, provides a welcome to KubeCon North America 2018, which took place last week in Seattle. In his post, Brendan provides a retrospective on Azure Kubernetes Services (AKS), including how engineers at companies such as Maersk, Siemens, and Bosch benefited from adoption of AKS in their solutions. He also provides an overview of the various announcements we made at KubeCon. With Docker, Bitnami, Hashicorp, and others we announced the Cloud Native Application Bundle (CNAB) specification, which is a new distributed application package that combines Helm or other configuration tools with Docker images to provide a complete, self-installing cloud applications. He also announced that Microsoft is donating the likeness of Phippy, and all of your favorites from the Children’s Illustrated Guide to Kubernetes to the CNCF, and the release of a special second episode of the guide, Phippy Goes to the Zoo, which covers ingresses, CronJobs, CRDs, and more.

A hybrid approach to Kubernetes

Azure Stack enables you to run your containers on-premise in pretty much the same you as you do with global Azure. Microsoft Azure Stack is a hybrid cloud platform that lets you deliver services from your datacenter. As a service provider, you can offer services to your tenants. The Kubernetes Cluster Marketplace item 0.3.0 for Azure Stack is consistent with Azure since the template is generated by the Azure Container Service Engine, the resulting cluster will run the same containers as in AKS. It also complies with the Cloud Native Foundation. The cluster depends on an Ubuntu server, custom script, and the Kubernetes items to be in the Azure Stack Marketplace.

Now in preview

Microsoft previews neural network text-to-speech

Speech Service, part of Azure Cognitive Services now offers a neural network-powered text-to-speech capability. Neural Text-to-Speech makes the voices of your apps nearly indistinguishable from the voices of people. Use it to make conversations with chatbots and virtual assistants more natural and engaging, to convert digital texts such as e-books into audiobooks and to upgrade in-car navigation systems with natural voice experiences and more. This release includes significant enhancements since we first revealed Neural Text-to-Speech at Ignite earlier this year, such as: enhanced voice quality, accelerated runtime performance, and greater service availability. With these updates, Speech Services Neural Text-to-Speech capability offers the most natural-sounding voice experience for your users in comparison to the traditional and hybrid system approaches.

Native Python support on Azure App Service on Linux: new public preview!

Built-in Python images for Azure App Service on Linux are now available in public preview. With the choice of Python 3.7, 3.6 and soon 2.7, developers can get started quickly and deploy Python applications to the cloud, including Django and Flask, and leverage the full suite of features of Azure App Service on Linux. When you use the official images for Python on App Service on Linux, the platform automatically installs the dependencies specified in the requirements.txt file. While the underlying infrastructure of Azure App Service on Linux has been generally available (GA) for over a year, at the moment we’re releasing the runtime for Python in public preview, with GA expected in a few months.

Automatic performance monitoring in Azure SQL Data Warehouse (preview)

The preview of Query Store for Azure SQL Data Warehouse is now available in preview for both our Gen1 and Gen2 offers. The Query Store contains three actual stores: a plan store for persisting the execution plan information, a runtime stats store for persisting the execution statistics information, and a wait stats store for persisting wait stats information. Query Store is a set of internal stores and Dynamic Management Views (DMVs). These stores are managed automatically by SQL Data Warehouse and provide an unlimited number of queries storied over the last 7 days at no additional charge. Query Store is available in all Azure regions with no additional charge.

Also in preview

Now generally available

Azure Monitor for containers now generally available

Azure Monitor for containers monitors the health and performance of Kubernetes clusters hosted on Azure Kubernetes Service (AKS). Since the public preview, we have added several capabilities including: Multi-cluster view, Performance Grid view, Live debugging, and automated onboarding Azure Monitor for containers. Azure Monitor for containers gives you performance visibility by collecting memory and processor metrics from controllers, nodes, and containers that are available in Kubernetes through the Metrics API. After you enable monitoring from Kubernetes clusters, metrics and logs are automatically collected for you through a containerized version of the Log Analytics agent for Linux and stored in your Log Analytics workspace.

Streamlined IoT device certification with Azure IoT certification service

Azure IoT certification service (AICS), a new web-based test automation workflow, is now generally available. AICS will significantly reduce the operational processes and engineering costs for hardware manufacturers to get their devices certified for Azure Certified for IoT program and be showcased on the Azure IoT device catalog. The goals of the certification program are to showcase the right set of IoT devices for industry-specific vertical solutions and to simplify IoT device development. AICS helps achieve these goals by delivering a consistent certification process through automation, additional tests to support validation of device twins and direct methods with IoT Hub primitives, flexibility for customized test cases, and a simple and intuitive user experience.

Static websites on Azure Storage now generally available

Static websites are websites that can be loaded and served statically from a pre-defined set of files. You can now build a static website using HTML, CSS, and JavaScript files that are hosted on Azure Storage. Static websites can be powerful with the use of client-side JavaScript. Azure Storage makes hosting of websites easy and cost-efficient. You can enable static website hosting using the Azure portal, Azure CLI, or Azure PowerShell, which creates a container named ‘$web’. You can then upload your static content to this container for hosting. Your content will be available through a web endpoint. There are no additional charges for enabling static websites on Azure Storage.

Also generally available

News and updates

Azure HDInsight integration with Data Lake Storage Gen2 preview – ACL and security update

This integration will enable HDInsight customers to drive analytics from the data stored in Azure Data Lake Storage Gen 2 using popular open source frameworks such as Apache Spark, Hive, MapReduce, Kafka, Storm, and HBase in a secure manner. Azure Data Lake Storage Gen2 unifies the core capabilities from the first generation of Azure Data Lake with a Hadoop compatible file system endpoint now directly integrated into Azure Blob Storage. HDInsight and Azure Data Lake Storage Gen2 integration is based upon user-assigned managed identity. You assign appropriate access to HDInsight with your Azure Data Lake Storage Gen2 accounts. Once configured, your HDInsight cluster is able to use Azure Data Lake Storage Gen2 as its storage.

Azure Backup Server now supports SQL 2017 with new enhancements

You can now install Azure Backup Server on Windows Server 2019 with SQL 2017 as its database. With Azure Backup Server, you can protect application workloads such as Hyper-V VMs, Microsoft SQL Server, SharePoint Server, Microsoft Exchange, and Windows clients from a single console. Azure Backup Server version 3 (MABS V3) is the latest upgrade, and includes critical bug fixes, Windows Server 2019 support, SQL 2017 support and other features and enhancements. MABS V3 is a full release, and can be installed directly on Windows Server 2016, Windows Server 2019, or can be upgraded from MABS V2. Before you upgrade to or install Backup Server V3, read the installation prerequisites.

Azure Functions now supported as a step in Azure Data Factory pipelines

Azure Functions is a serverless compute service that enables you to run code on-demand without having to explicitly provision or manage infrastructure. Using Azure Functions, you can run a script or piece of code in response to a variety of events. Azure Data Factory (ADF) is a managed data integration service in Azure that allows you to iteratively build, orchestrate, and monitor your Extract Transform Load (ETL) workflows. Azure Functions is now integrated with ADF, enabling you to run an Azure function as a step in your data factory pipelines. To run an Azure Function, you need to create a linked service connection and an activity that specifies the Azure Function that you plan to execute.

Automate Always On availability group deployments with SQL Virtual Machine resource provider

High availability architectures are designed to continue to function even when there are database, hardware, or network failures. Azure Virtual Machine instances using Premium Storage for all operating system disks and data disks offers 99.9 percent availability. This SLA is impacted by three scenarios – unplanned hardware maintenance, unexpected downtime, and planned maintenance. You now have a new, automated method to configure Always On availability groups (AG) for SQL Server on Azure VMs with SQL VM resource provider (RP) as a simple and reliable alternative to manual configuration. SQL VM resource provider automates Always On AG setup by orchestrating the provisioning of various Azure resources and connecting them to work together.

Additional news and updates

Technical content

Power BI and Azure Data Services dismantle data silos and unlock insights

Power BI data flows, the Common Data Model, and Azure Data Services can be used together to break open silos of data in your organization and enable business analysts, data engineers, and data scientists to share data to fuel advanced analytics and unlock new insights to give you a competitive edge. Learn how to connect Power BI and Azure Data Services to share data and unlock new insights with a new tutorial. The tutorial gives you a first look at how to use CDM folders to share data between Power BI and Azure Data Services. The tutorial uses sample libraries, code, and Azure resource templates that you can use with CDM folders that you create from your own data. By working through the tutorial, you’ll see first-hand how the metadata stored in a CDM folder makes it easier to for each service to understand and share data.

Deploying Apache Airflow in Azure to build and run data pipelines

Apache Airflow is an open source platform used to author, schedule, and monitor workflows. Airflow overcomes some of the limitations of the cron utility by providing an extensible framework that includes operators, programmable interface to author jobs, scalable distributed architecture, and rich tracking and monitoring capabilities. We developed an Azure Quickstart template that enables you to deploy and create an Airflow instance in Azure more quickly by using Azure App Service and an instance of Azure Database for PostgreSQL as a metadata store.

How news platforms can improve uptake with Microsoft Azure’s Video AI service

Microsoft News is an app that delivers breaking news and trusted, in-depth reporting from the world's best journalists. Microsoft News created advanced algorithms to analyze their articles and determine how to increase personalization, which ultimately increases consumption, but wanted more insight on their videos. Anna Thomas, an Applied Data Scientist within Microsoft Engineering, set off to determine how to deliver these insights using a combination of Microsoft technologies and custom solutions; however, she discovered that the Video Indexer API held more capabilities than she expected. Check out her post to see what she discovered.

Know exactly how much it will cost for enabling DR to your Azure VMs

Azure offers built-in disaster recovery (DR) solution for Azure Virtual Machines through Azure Site Recovery (ASR). Site Recovery manages and orchestrates disaster recovery of on-premises machines and Azure virtual machines (VMs), including replication, failover, and recovery. A common question we get is about costs associated with configuring DR for Azure virtual machines, so Sujay Talasila explored how to estimate DR costs. Follow his example to explore how much it will cost to support your particular solution. Disaster Recovery between Azure regions is available in all Azure regions where ASR is available.

Taking a closer look at Python support for Azure Functions

As announced at Microsoft Connect(); 2018 earlier this month, you can now develop your Functions using Python 3.6, based on the open-source Functions 2.0 runtime and publish them to a Consumption plan (pay-per-execution model) in Azure. Python is a great fit for data manipulation, machine learning, scripting, and automation scenarios. Building these solutions using serverless Azure Functions can take away the burden of managing the underlying infrastructure, so you can move fast and actually focus on the differentiating business logic of your applications. Read this post for details about the newly announced features and dev experiences for Python Functions.

Additional technical content

Azure shows

Episode 258 – Live from KubeCon 2018 | The Azure Podcast

We are live at KubeCon+CloudNative in Seattle where Microsoft, together with the who's-who of the tech world, are talking about Kubernetes, We are very fortunate to get Lachie Evenson, Principal PM in the Azure team, Tommy Falgout, a Cloud Solution Architect and Daniel Selman, a Kubernetes Consultant, together in a room to discuss the current state of Kubernetes and AKS.

How to get started with Docker and Azure | Azure Tips and Tricks

Learn how you can get started using Docker and Azure. To get started with Docker, make sure you have the Docker desktop application installed on your local dev machine.

How to deploy an image classification model using Azure services

Learn how to deploy an image classification model using Azure Machine Learning service. In this tutorial, you'll use Azure Machine Learning service to set up your testing environment, retrieve the model from your work space, and test the model locally. You’ll then see how to deploy the model to Azure Container Instance (ACI) and test the deployed model to Azure Kubernetes Service (AKS).

Decentralized Identity and Blockchain | Block Talk

This video introduces the concept of decentralized identity and how blockchain enables hosting these identities in a decentralized fashion. The demo provides a walkthrough of a decentralized identity that is anchored on Ethereum blockchain and is consumed using uPort application.

Running AI on IoT microcontroller devices with ELL | The IoT Show

How about designing and deploying intelligent machine-learned models onto resource constrained platforms and small single-board computers, like Raspberry Pi, Arduino, and micro:bit? How interesting would that be? This is exactly what the open source Embedded Learning Library (ELL) project is about. The deployed models run locally, without requiring a network connection and without relying on servers in the cloud. ELL is an early preview of the embedded AI and machine learning technologies developed at Microsoft Research. Chris Lovett from Microsoft Research gives us a fantastic demo of the project in this episode of the IoT Show.

AzureIoT TypeEdge : a strongly-typed development experience for Azure IoT Edge | The IoT Show

Are you excited about Azure IoT Edge? Then you are going to love TypeEdge because it simplifies the IoT Edge development down to a simple F5 experience. Watch how you can now create a complete Azure IoT Edge application from scratch in your favorite development environment, in just a few minutes.

LearnAI: Adding Bing Search to Bots | AI Show

The LearnAI team has updated the Azure Cognitive Services Bootcamp! Tune in to get an overview of the changes and a walk through of how you can add Bing Search, LUIS, and Azure Search to bots via the Bot Framework SDK V4.

LearnAI: LUIS – Notes from the Field | AI Show

Anna Thomas has been collecting notes for the past two years from field members (internal and external) who have developed complex LUIS models. In this video, we'll explore some of the limitations or challenges that are faced when you try to deploy enterprise-ready LUIS models at scale, and how they can be addressed.

Jeremy Epling on Azure Pipelines – Episode 014 | The Azure DevOps Podcast

Jeffrey Palermo is joined by Jeremy Epling, Head of Product for Azure Pipelines and a Principal Group Program Manager at Microsoft. He has been a leader at Microsoft for over 15 years in various roles. There’s a lot going on in the DevOps space with Azure right now — and in particular, with Azure Pipelines. Jeremy is incredibly passionate about the current progress being made and is excited to discuss all the new features coming to Pipelines in today’s episode!

Customers, partners, and industries

Cloud Commercial Communities webinar and podcast update

Check out the Cloud Commercial Communities monthly webinar and podcast update, which provides a comprehensive list of forthcoming (three scheduled for today) and on-demand content. Each month the Industry Experiences team focuses on core programs, updates, trends, and technologies that Microsoft partners and customers need to know to increase success using Azure and Dynamics.

An Azure Function orchestrates a real-time, serverless, big data pipeline

Although it’s not a typical use case for Azure Functions, a single Azure function is all it took to fully implement an end-to-end, real-time, mission-critical data pipeline for a fraud detection scenario. The solution was built on an architectural pattern common for big data analytic pipelines, with massive volumes of real-time data ingested into a cloud service where a series of data transformation activities provided input for a machine learning model to deliver predictions. Kate Baroni, Software Architect at Microsoft Azure, provides an overview of the solution, which is covered in the Mobile Bank Fraud Solution Guide with details on the architecture and implementation.

Extracting insights from IoT data using the warm path data flow

If you are responsible for the machines on a factory floor, you are already aware that the Internet of Things (IoT) is the next step in improving your processes and results. Having sensors on machines, or the factory floor, is the first step. The next step is to use the data. In this post, Ercenk Keresteci Principal Solutions Architect, Industry Experiences, highlights another scenario from the Extracting Insights from IoT solution guide, which provides a technical overview of the components needed to extract actionable insights from IoT data analytics. This post covers the speed layer (warm path), which analyze data in real time. This layer is designed for low latency, at the expense of accuracy. It is a faster-processing pipeline that archives and displays incoming messages, and analyzes these records, generating short-term critical information and actions such as alarms.

Extracting insights from IoT data using the cold path data flow

In a further exploration of the guide described above, this post covers the batch and serving layers (cold path), which stores all incoming data in its raw form and performs batch processing on the data. The result of this processing is stored as a batch view. It is a slow-processing pipeline, executing complex analysis, combining data from multiple sources over a longer period (such as hours or days), and generating new information such as reports and machine learning models.

How smart buildings can help combat climate change

Fast-paced urbanization offers an exciting opportunity to immediately reduce climate impacts. Because buildings—office complexes, multifamily housing, hotels, stores, schools, hospitals, and malls, among others—comprise a big part of city infrastructure, making them smarter can dramatically lower the energy and carbon footprint of a city. Read this post to learn how connected building technology can manage lighting, heating, and cooling, reducing unnecessary use while maximizing usability and comfort. In addition, you will learn how smart building software can schedule preventive maintenance, automatically identify and prioritize issues for resolution by cost and impact, and continually optimize buildings for comfort and energy efficiency.

Creating a smart grid with technology and people

Utilities and their partners are searching for new solutions that can meet 21st-century energy challenges: surging demand for electricity, two-way energy flow, increased use of clean energy sources, and stairstep approaches to creating a smart grid to tackle the thorniest challenges first. This post provides a look at the digital transformation of the power and utilities industry that is picking up steam. In the very near future, power generation companies will have greater options in how they run their businesses, using IoT-enabled insights to strategically stairstep their way to creating a smart grid and ensure business continuity.

Azure Marketplace new offers – Volume 26

The Azure Marketplace is the premier destination for all your software needs – certified and optimized to run on Azure. Find, try, purchase, and provision applications & services from hundreds of leading software providers. You can also connect with Gold and Silver Microsoft Cloud Competency partners to help your adoption of Azure. During September and October, 149 new consulting offers successfully met the onboarding criteria and went live.

Azure Marketplace new offers – Volume 27

The Azure Marketplace is the premier destination for all your software needs – certified and optimized to run on Azure. Find, try, purchase, and provision applications & services from hundreds of leading software providers. You can also connect with Gold and Silver Microsoft Cloud Competency partners to help your adoption of Azure. From November 1 to November 16, 2018, 61 new offers successfully met the onboarding criteria and went live.

A Cloud Guru's Azure This Week – 14 December 2018

This time on Azure This Week, Lars talks about Azure Machine Learning service now in general availability, Business Critical service tier in Azure SQL Database Managed Instance in general availability, Azure Cosmos DB .NET SDK V3.0 in public preview and a new Azure API Management tier for serverless architectures.

Source: Azure Blog Feed