Azure Data Factory new capabilities are now generally available

Microsoft is excited to announce the General Availability of new Azure Data Factory (ADF V2) features that will make data integration in the cloud easier than ever before. With a new browser-based user interface, you can accelerate your time to production by building and scheduling your data pipelines using drag and drop. Manage and monitor the health of your data integration projects at scale, wherever your data lives, in cloud or on-premises, with enterprise-grade security. ADF comes with support for over 70 data source connectors and enables you to easily dispatch data transformation jobs at scale to transform raw data into processed data that is ready for consumption by business analysts using their favorite BI tools or custom applications.

For existing SQL Server Integration Services (SSIS) users, ADF now allows you to easily lift and shift your SSIS packages into the cloud and run SSIS as a service with minimal changes required to your existing packages. ADF will now manage your SSIS resources for you so you can increase productivity and lower total cost of ownership. Meet your security and compliance needs while taking advantage of extensive capabilities and paying only for what you use.

All of these new Azure Data Factory capabilities fall under the Azure general availability SLA. Here is just a small sampling of the exciting new features now available in ADF:

1. Control flow: Azure Data Factory includes control flow data pipeline constructs such as branching, looping, conditional execution, and parameterization in order to allow you to orchestrate complex data integration jobs that are flexible and reusable.
2. Code-free data movement and orchestration designing: You can now design, manage, maintain, and monitor your pipelines right in your browser. Native integration with Git repos in Visual Studio Team Services allows your development teams to collaborate on data pipelines as well as build and release management and automation.
3. Iterative pipeline development: The ADF design environment also includes the ability to iteratively develop pipelines and debug your pipelines interactively with iterative debugging built-in.
4. Flexible scheduling: Schedule pipelines on a wall-clock scheduler, event-based triggers, or with tumbling window schedules.
5. Lift and Shift SSIS into ADF: ADF provides Integration Runtime support that allows you to lift and shift your existing on-premises SSIS packages into the cloud by using Azure Data Factory where you can then execute, schedule, and monitor your SSIS package executions in the cloud.
6. HDInsight Spark on-demand and Azure Databricks: Build ETL pipelines in the cloud with ADF that transform data at scale with Spark using HDInsight on-demand clusters or Azure Databricks Notebooks.
7. SDK support: SDK support has been added and updated for Python, .NET, REST, and PowerShell to build custom applications with ADF.

With these exciting capabilities, Microsoft’s Cloud and Data partner Pragmatic Works is now able to engage with its customers in ways that were not possible before.

“Pragmatic Works helps customers solve business barriers once and for all with Azure cloud and data services and training. Azure Data Factory makes it extremely easy for Pragmatic Works’ customers to bring data together from on-premises and multi-cloud data sources. We can quickly build complex data integrations with the drag-and-drop browser user interface, allowing us to focus on delivering business value to our customers.”

Adam Jorgensen, President of Consulting, Pragmatic Works

Get started today!

• Quickstart guide to start creating an ADF V2 factory.
• Go through the hands-on lab to lift SSIS packages to Azure and modernize data warehouse.
• Download the Azure Data Factory: SSIS in the Cloud ebook.
• Read the build Azure Data Factory pipelines white paper.

Source: Azure Blog Feed