Nastel HQ. 88 Sunnyside Blvd, Plainview, NY, USA 11803
+1 800-580-2344

Apache Kafka

Created with Sketch.

Effective and Efficient Monitoring, Management, Tracing and Tracking with Nastel’s® AutoPilot® for Apache Kafka

Nastel’s® AutoPilot® for Apache Kafka provides operational and transactional monitoring for Apache Kafka, the open-source stream processing platform developed by the Apache Software Foundation, written in Scala and Java.

Apache Kafka is a unified, high-throughput, low-latency platform for handling real-time data feeds. It utilizes a massively scalable pub/sub message queue—designed as a distributed transaction log—as its storage layer. It is often used by Nastel clients as a transport mechanism for streaming data interconnected with other messaging and processing systems.

Key Benefits

Nastel’s® AutoPilot® for Apache Kafka delivers a single-point-of-truth to track performance, latency, and logs, along with full message auditing and content surveillance capabilities. It provides complete message flow analytics, relating applications to the messages they publish to Kafka, and the applications that subscribe to those messages. Powerful network auto-discovery capabilities are included with matching visualization tools, as seen below. In the Viewlet below, solid lines (edges) represent send-to relationships; dotted lines represent an Acts-On relationship. Each edge has statistics showing average elapsed time and count.

Apache Kafka Monitoring
Kafka network visualization showing an auto-discovered, pub-sub topology including senders, readers, and topics

Nastel’s® AutoPilot® for Apache Kafka offers powerful forensics to diagnose Kafka problems. Kafka performance and availability monitoring is accomplished via end-to-end stream monitoring and tracking of metrics from brokers, consumers, producers and Zookeeper, Kafka’s configuration service. Nastel’s® AutoPilot® for Apache Kafka examines the metrics collected for Kafka topics, producers, consumers and brokers while simultaneously offering deep-dive insight into the JVM itself.

For Developers

  • Make Kafka apps faster and more stable by:
    • Optimizing Kafka apps
    • Identifying latency and performance bottlenecks
    • Identifying spots for data loss
  • Improve quality and help debug apps by:
    • Capturing Kafka exceptions
    • Enabling easy message capture and profiling
    • Generating message flow charts and topology displays

For Operations

  • Reduce MTTR (mean time to repair) by:
    • Finding problems and bottlenecks
    • Generating alerts based on user-defined conditions
    • Providing anomaly detection in flows
    • Identifying problems, data loss, and latencies
  • Where is my message?
    • Capture messages for problem identification
    • Message flow analytics: see the longest and shortest paths

For Business

  • Message Content Analytics
    • Extract and summarize relevant business tokens. E.g., payment amounts (max, min, avg, etc.)
  • Alerts based on business conditions
    • SLAs and OLAs
  • Single-pane-of-glass display for business flows over Kafka infrastructure
  • Risk management and audit capabilities for relevant business transactions

Key Features

  • Auto discovery of end-to-end transactions spanning Kafka and other technologies such as IBM MQ
  • Parsing of Kafka messages which are tokenized and utilized for analytics and transaction stitching
  • SLA Monitoring, Analytics and Reporting
  • Deep-dive monitoring of composite application components that include Kafka
  • Proactive alerting and reduction in false alarms

Nastel’s® AutoPilot® for Apache Kafka also utilizes Kafka internally as an integration technology for data transport.

Nastel was at the London Confluent Kafka Event

Nastel is currently developing a version of Nastel Navigator for Kafka – This will streamline your management and security of kafka environments, and provide your administrators with the ability to deliver self service to your users, greatly improving your teams productivity. To find out more please visit the Nastel Navigator page or request more information from Nastel

Nastel XRay for Kafka provides AIOps for your Kafka environment

Nastel Provides support for the broadest range of platforms, infrastructures, applications and messaging middleware environments. To find out more click here.

Interesting Links