Local testing with live data means faster development with Azure Stream Analytics

We are excited to announce that live data local testing is now available for public preview in Azure Stream Analytics Visual Studio tools. Have you ever thought of being able to test Azure Stream Analytics queries logic with live data without running in the cloud? Are you excited by the possibility of not having to wait for your queries to deploy and other round-trip delays? Well, this enhancement lets you test your queries locally while using live data streams from cloud sources such as Azure Event Hubs, IoT Hub or Blob storage. You can even use all the Azure Stream Analytics time policies in the local Visual Studio environment. Being able to start and stop queries in seconds and local debugging significantly improves development productivity by saving precious time on the inner loop of query logic testing.

Live data local testing experience

The new local testing runtime can read live streaming data from the cloud or from a local static file. It works the same as the cloud runtime Azure Stream Analytics and therefore supports the same time policies needed for many testing scenarios. The query runs in a simulated environment suitable for a single server development environment and should only be used for query logic testing purposes. It is not suitable for performance/scalability and availability testing.

Key benefits

Query behavior consistency

For any given input, the queries running locally and in the cloud will output the same results. Live data testing in Visual Studio includes full support for time polices and machine learning functions.

Much shorter testing cycles

Compared to running in the cloud, it takes only seconds to start a round of query testing. It enables you to test your query directly in Visual Studio without having to switch environments. You can understand the shape of the data coming to easily adjust the query and see immediately the result on the live data.

Viewing job metrics and error messages

Job metrics such as input data size, output data size, and more are available during local execution. Error messages will also be displayed in testing and debugging.

Fluid experience

• The query runs locally but connects to input data cloud sources directly. No need to sample data to local first.
• Query results are immediately displayed in table format in Visual Studio.

Local testing options

In addition to the current local run capability, with this major improvement, we now support all the local testing options listed below.

Input	Output	Job Type
Local static data	Local static data	Cloud/Edge
Live input data	Local static data	Cloud
Live input data	Live output data (Power BI and ADLS are not supported at this moment)	Cloud

Local testing vs. cloud run caveats

This local testing feature should only be used for functional testing purposes. It does not replace the performance and scalability tests you would do in the cloud. It should not be used for production purposes since running in a local environment doesn’t guarantee any service SLA.

While on your machine you can only rely on local resources, when you deploy to the cloud we can scale out to multiple nodes. Additionally, cloud deployment ensures checkpointing, upgrades, and more. It also provides all the infrastructure to run your job 24/7.

The following table shows the comparison between running a query in a local environment versus running it in the cloud. Please note, that only the cloud input options support time policies and machine learning functions, the local input doesn’t support these.

	Query logic	JavaScript UDF/UDA	Machine Learning function	Time policies	Execution Environment	Resource Allocation	Service SLA	Scalability
Local testing with local input	Yes	Yes	No	No	Local simulated,	No CPU/memory commitment	No	No
Local testing with live input	Yes	Yes	Yes	Yes	Local simulated	No CPU/memory commitment	No	No
Running in the cloud	Yes	Yes	Yes	Yes	Cloud	By SU	Yes	Yes

Try it now

This feature is amazingly easy to use. Just choose the input/output type you want in the script editor and click “Run locally”. Follow this guide on Testing live data locally using Azure Stream Analytics tools for Visual Studio to try it!

Providing feedback and ideas

The Azure Stream Analytics team is highly committed to listening to your feedback. We welcome you to join the conversation and make your voice heard via our UserVoice. For tools feedback, you can also reach out to ASAToolsFeedback@microsoft.com.

Did you know we have more than ten new features in public preview? Sign up for our preview programs to try them out. Also, follow us @AzureStreaming to stay updated on the latest features.

Source: Big Data