We're excited to announce the release of @platformatic/kafka-opentelemetry, a comprehensive OpenTelemetry instrumentation package that brings enterprise-grade observability to your Kafka applications built with @platformatic/kafka.

Why We Built This

Modern distributed systems rely heavily on Apache Kafka for reliable message streaming, but gaining visibility into message flows, performance bottlenecks, and error patterns across your Kafka infrastructure has traditionally been challenging. With the growing adoption of OpenTelemetry as the standard for observability, we recognized the need for seamless integration between Platformatic's Kafka library and the OpenTelemetry ecosystem.

Key Features

🚀 Zero-Configuration Setup

Get started with comprehensive Kafka tracing in just a few lines of code. The instrumentation automatically detects and traces all producer and consumer operations without requiring changes to your existing Kafka code.

📊 Complete Observability Stack

• Distributed Tracing: Full producer and consumer trace spans with proper parent-child relationships

• Metrics Collection: Counters for sent/consumed messages and duration histograms for operations

• Error Tracking: Automatic capture and correlation of Kafka errors with traces

🎯 OpenTelemetry Standards Compliance

Follows official OpenTelemetry semantic conventions for messaging systems, ensuring compatibility with all major observability platforms, including Jaeger, Zipkin, Honeycomb, DataDog, and more.

⚡ No Monkey Patching

Built using Node.js's native diagnostic_channel rather than monkey patching, ensuring reliable and performant instrumentation that won't interfere with your application's behavior.

Quick Start

npm install @platformatic/kafka-opentelemetry

import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node'
import { ConsoleSpanExporter } from '@opentelemetry/sdk-trace-node'
import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-node'
import { registerInstrumentations } from '@opentelemetry/instrumentation'
import { KafkaInstrumentation } from '@platformatic/kafka-opentelemetry'

// Initialize OpenTelemetry
const provider = new NodeTracerProvider()
provider.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()))
provider.register()

// Register Kafka instrumentation
registerInstrumentations({ instrumentations: [new KafkaInstrumentation()]})

// Your existing Kafka code now automatically generates traces!
import { Producer, Consumer } from '@platformatic/kafka'

What Gets Traced

The instrumentation provides comprehensive visibility into:

• Producer Operations: Message publishing with topic, partition, and key attributes

• Consumer Operations: Message processing with full context propagation

• API Operations: Internal Kafka protocol operations (Metadata, Produce, Fetch, etc.)

• Error Conditions: Failed sends, processing errors, and connection issues

Metrics Out of the Box

Beyond tracing, you'll get essential Kafka metrics automatically:

• messaging.client.sent.messages - Track producer throughput

• messaging.client.consumed.messages - Monitor consumer processing

• messaging.client.operation.duration - Kafka API operation latency

• messaging.process.duration - Message processing time

Perfect for Modern Architectures

Whether you're building microservices, event-driven architectures, or real-time data pipelines, this instrumentation gives you the observability foundation needed for:

• Performance Monitoring: Identify slow consumers and producers

• Error Tracking: Quick root cause analysis of message failures

• Capacity Planning: Understand traffic patterns and scaling needs

• SLA Monitoring: Track message processing latency and success rates

TypeScript First

Built with full TypeScript support, providing an excellent developer experience with strong typing and IntelliSense support throughout.

Get Started Today

The package is available on npm and ready for production use. The GitHub repository has detailed documentation, examples, and contribution guidelines.

We're excited to see how this helps teams gain better visibility into their Kafka-based systems. Try it out and let us know what you think!

Built with ❤️ by the Platformatic team. Licensed under Apache-2.0.