[Webinar] AI-Powered Innovation with Confluent & Microsoft Azure | Register Now
Happy 2017! Wishing you a wonderful year full of fast and scalable data streams.
Many things have happened since we last shared the state of Apache Kafka® and the streams ecosystem. Let’s take a look!
Most importantly – we did a bug fix release. Apache Kafka 0.10.1.1 fixes some critical issues found in the 0.10.1.0 release. There is a pretty substantial list of fixes, so if you are running 0.10.1.0, we recommend upgrading to avoid running into issues we already resolved.
Kafka Summit! If you haven’t heard – last year was so successful that we are doing two events this year. New York on May 8th and San Francisco on August 28th. Call for paper is ending soon, so please submit your talk proposals this week!
There are many KIPs (improvement proposals) being discussed in the Kafka developer list, many of them are huge improvements:
In addition to the many KIPs, there are some interesting releases, blogs and presentations I’d recommend checking out:
This blog post demonstrates using Tableflow to easily transform Kafka topics into queryable Iceberg tables. It uses UK Environment Agency sensor data as a data source, and shows how to use Tableflow with standard SQL to explore and understand the data.
The guide covers Kafka consumer offsets, the challenges with manual control, and the improvements introduced by KIP-1094. Key enhancements include tracking the next offset and leader epoch accurately. This ensures consistent data processing, better reliability, and performance.