Introducing the Instrumentation Score

Telemetry data is the foundation of our observability. We gather metrics, traces, and logs, aiming to cultivate a clear understanding of application health. Yet, a persistent question often arises: "Is our telemetry actually good?" How do we distinguish valuable insights from data that merely consumes resources?

For too long, evaluating instrumentation effectiveness has been a subjective exercise. We've lacked a common language or a standard measure to truly understand if our telemetry is enriching our insights or just overgrowing the plot. At OllyGarden, we recognize this challenge and are building a product to give super powers to observability engineers. We heard from them that our Instrumentation Score was something special. And we believe that if you want something really special, you share it.

Today, OllyGarden introduces its first major contribution to the observability ecosystem: the Instrumentation Score.

The Instrumentation Score is a standardized, numerical value designed to objectively assess the consistency and effectiveness of OpenTelemetry instrumentation. It analyzes OTLP (OpenTelemetry Protocol) data streams against a predefined set of rules rooted in OpenTelemetry best practices and semantic conventions. It’s a health check for your telemetry, providing a clear, actionable measure of its quality.

As Juraci Paixão Kröhling, Co-founder at OllyGarden, states:

"As an OpenTelemetry contributor and enthusiast, I've seen firsthand the project's power to democratize instrumentation. Yet, a persistent question has always been: 'Are we generating good telemetry?' Too often, the answer is unclear, leading to missed insights or wasted resources. The Instrumentation Score, an initiative we're launching from OllyGarden, aims to provide that clarity. It's about establishing a common, actionable language for telemetry quality, built on OpenTelemetry principles, to empower every engineer and organization to confidently improve their observability practices and truly harness the value of their data."

The Instrumentation Score provides a common vocabulary for discussing instrumentation effectiveness. For engineers and SREs, it offers actionable guidance, highlighting where instrumentation can be improved. For CTOs and technology leaders, the strategic value includes improved ROI on observability by focusing on purposeful telemetry and reducing bad telemetry.

OllyGarden is committed to OpenTelemetry and the open-source ecosystem. The Instrumentation Score leverages OpenTelemetry Semantic Conventions and analyzes OTLP data. Crucially, we've ensured this initiative is not only open source but is being developed with an open governance model, with support or contributions from key industry players like Dash0, New Relic, Splunk, Datadog, and Grafana Labs. The specification is an open-source effort, and we are actively requesting observability engineers to contribute with their own rules by opening a pull request against the GitHub repository. Our goal is a collaborative evolution, with the Instrumentation Score eventually finding a home within a neutral foundation.

The introduction of the Instrumentation Score is a step towards a future where organizations can confidently understand and improve their telemetry.

We invite you to learn more and get involved:

• Explore the Instrumentation Score landing page: https://score.olly.garden/

• Review the specification and contribute on GitHub: https://github.com/instrumentation-score/

OllyGarden aims to improve the efficiency of telemetry pipelines. The Instrumentation Score is the first seed we’re planting, hoping that together, we can help everyone grow a more effective observability practice.