Use Azure Monitor to integrate with SIEM tools
Over the past two years since introducing Azure Monitor, we’ve made significant strides in terms of consolidating on a single logging pipeline for all Azure services. A majority of the top Azure services, including Azure Resource Manager and Azure Security Center, have onboarded to Azure Monitor and are producing relevant security logs.
We’ve also delivered key capabilities to simplify the integration process with security information and event management (SIEM) tools, such as routing data to a single event hub and enabling multiple diagnostic settings per resource, and have work in flight that will ease setup and management of log routing across large Azure environments.
Meanwhile, we’ve been partnering with the top SIEM partners to build connectors that get the data from Azure Monitor into those tools. These connectors consume data routed to Azure Event Hubs by Azure Monitor – a simple, scalable, and manageable approach for delivering log data to an external application, and Microsoft’s recommended approach for integrating Azure with SIEM tools going forwards. Read more about how you can set up your Azure environment to send data to these SIEM tools.
We’ve also continued to support customers who are using the Azure Log Integration tool (AzLog) to integrate with these same SIEMs. AzLog was initially released to help customers navigate the complex process of consolidating, translating, and forwarding logs from a variety of Azure services to a SIEM tool. At the time, Azure Monitor didn’t exist and there was very little standardization in terms of how Azure services exposed log data to customers (some dumped data into a storage account, others exposed an API, etc).
We’ve come a long way since then, and today we’re announcing that there will be no further capabilities added to the Azure Log Integration tool and end of support will happen June 1, 2019. Our recommendation for integrating Azure with popular SIEM tools is below.
Integration recommendations
The table below indicates what you should do based off the SIEM tool(s) you are using and your current integration status. Only SIEM tools that were officially supported by AzLog are listed below.
| SIEM Tool | Currently using log integrator | Currently investigating SIEM integration options |
| Splunk | Begin migrating to the Azure Monitor Add-On for Splunk. | Use the Azure Monitor Add-On for Splunk. |
| IBM QRadar | Begin migrating to the Microsoft Azure DSM and Microsoft Azure Event Hub Protocol, available from the IBM support website. You can learn more about the integration with Azure here. | Use the Microsoft Azure DSM and Microsoft Azure Event Hub Protocol, available from the IBM support website. You can learn more about the integration with Azure. |
| ArcSight | The Azure log integration tool offered collection of Azure logs into JSON files for the purpose of integrating with ArcSight using existing JSON connectors from ArcSight, with a JSON to CEF mapping available only for Azure Activity Logs and not for the other types of Azure Logs. The ArcSight team is currently working on a new comprehensive solution, which is planned to have its first release with limited coverage in the October 2018 timeframe. Please contact ArcSight for more details. If you are already using the Azure Log Integration tool, you should make plans to use the ArcSight connector for Azure when it is available. |
While not supported by the AzLog tool, we also recommend looking into some of our other partners that offer Azure Monitor event hub integration, including ELK stack and SumoLogic.
Integration roadmap
Today, Azure Monitor’s SIEM integration capabilities can’t do everything the Azure Log Integration tool could do. Below is our roadmap for addressing known gaps between what you could accomplish with Azure Log Integration and what you can accomplish with Azure Monitor.
- Azure Active Directory logs – Azure Active Directory logs are the only log type directly integrated with AzLog that aren’t yet available in Azure Monitor. Public preview of Azure Active Directory logs in Azure Monitor is expected to begin by July 2018.
- Integrate Azure VM logs – AzLog provided the option to integrate your Azure VM guest operating system logs (e.g., Windows Security Events) with select SIEMs. Azure Monitor has agents available for Linux and Windows that are capable of routing OS logs to an event hub, but end-to-end integration with SIEMs is nontrivial. We tentatively plan to deliver improved support for routing OS logs to event hubs by the end of 2018 and we’re working with partners to develop a plan for their connectors to consume these logs. For now, our recommendation is that you use the VM log agent or log forwarder provided by your SIEM.
- End-to-end setup – AzLog has a script that automates the end-to-end setup of log sources. While Azure Monitor offers the ability to script out creation of diagnostic settings, we’re partnering with the Azure Policy team to deliver seamless enablement via Resource Manager policies that ensure log data is being routed from all sources. You will begin to see built-in policies that support these scenarios over the next two months, with support for custom policies expected by late 2018.
- Integration with other SIEM tools – AzLog provided a generic capability to push standardized Azure logs in JSON format to disk. While other SIEM tools weren’t officially supported by AzLog, this offered a way to easily get log data into tools such as LogRhythm. Our recommendation for customers using AzLog for these tools is to work with the producer of that tool to provide an Azure Monitor Event Hubs integration.
The security of your Azure environment is always top priority on the Azure team, both in terms of how we engineer the Azure platform and in terms of the capabilities we provide for you for securing your own assets on that platform. Moving SIEM integration to Azure Monitor is a step towards enabling you to manageably secure your applications on Azure at scale. If you have any questions or concerns, reach out to AZSIEMTeam@microsoft.com.
Source: Azure Blog Feed