10 great things about Kafka on HDInsight

Kafka on HDInsight is Microsoft Azure’s managed Kafka cluster offering. It has generated huge customer interest and excitement since its general availability in December 2017.

Here are 10 great things about it:

1. Enterprise grade Kafka

HDInsight Kafka is an enterprise-grade managed cluster service giving you a 99.9 percent SLA. We take care of the heavy lifting for you, including operations, monitoring, security patching, and more. Running some of the largest Kafka clusters on the planet, we have the expertise and knowledge to prevent and quickly fix any issues.

2. Up and running in minutes

Create a full cluster in minutes, not days. Traditionally, it takes weeks to provision your own Kafka cluster on premises or on IaaS. With HDInsight, you get your own dedicated Kafka cluster within a few clicks.

3. Real open-source Kafka

With HDInsight, you get the true Apache Kafka engine running on your clusters with all its functionality. People love Kafka for its high throughput, lowest latency and cost-efficient scale. Along with the Producer and Consumer APIs, Kafka also has a rich feature set, such as compression of messages for an even higher throughput, configurable retention policy (including retention beyond 7 days and size based retention) and log compaction.

4. Use Kafka Streams for analytics

Apache Kafka is not just an ingestion engine, it is actually a distributed streaming platform with an amazing array of capabilities. With HDInsight, you get the Streams API, enabling users to filter and transform streams as they are ingested.

5. Use Kafka Connect

HDInsight supports the Kafka Connect API. Use community supported or custom connectors to receive from or send data to your favorite sources and sinks (such as IoTHub). With the Streams and the Connect APIs, customers can avoid having to manage and learn another technology, while also saving on costs as analytics can be collocated with your Kafka cluster.

6. Leverage the full power of Azure

You get all the managed cluster benefits, while still having full control of your cluster. You can install custom code on your clusters, isolate the cluster in your own custom VNET and apply rules to control the traffic flow into and out of the cluster.

7. Scale as you need

You get native integration with Azure Managed Disks, allowing you to separate compute and storage. You can scale your cluster within minutes to handle any throughput.

8. Enterprise grade management

HDInsight provides integration with Operations Management Suite so you don’t have to install your own custom solution. Features like Cluster self-healing ensure the highest-availability.

9. App marketplace

And you can easily amplify the productivity of your clusters by installing applications on the HDInsight Platform.

10. No limits

Last but not the least, there are no limits on the message size, number of consumer groups or throughput.

 

As you can see, Microsoft is deeply committed to Apache Kafka’s success. Click here to get started with HDInsight Kafka.

Source: Azure Blog Feed