OpenTelemetry-Demo

What is OpenTelemetry?

OpenTelemetry (OTel) is an open-source observability framework created by the Cloud Native Computing Foundation (CNCF). It provides a set of APIs, SDKs, tools, and integrations to help you collect telemetry data (i.e., logs, metrics, and traces) from your applications and infrastructure.

What Does OpenTelemetry Do?

OpenTelemetry enables:

• Tracing: Follow a request through multiple services (distributed tracing).

• Metrics: Collect system and app metrics like CPU, memory, request duration, etc.

• Logs: Structured logging for better debugging and analytics.

You instrument your app (manually or via auto-instrumentation), and OpenTelemetry sends telemetry data to observability tools like:

• Prometheus

• Grafana

• Jaeger

• Zipkin

• New Relic

• Datadog

• AWS X-Ray

• Elastic, etc.

Why It’s in High Demand?

1. Vendor-Neutral and Open Source

• Unlike tools like Datadog, New Relic, or Dynatrace, OpenTelemetry doesn’t lock you into one ecosystem.

• You can export telemetry data to multiple backends (e.g., Prometheus for metrics, Jaeger for traces).

2. Unified Observability Standard

• One framework to collect logs, metrics, and traces instead of managing three separate agents or SDKs.

• Makes observability simpler and more maintainable.

3. Wide Ecosystem and Community Support

• Backed by CNCF (same as Kubernetes).

• Adopted by major cloud providers and monitoring platforms (AWS, Azure, GCP, Datadog, etc.).

4. Cross-Language Support

• Works with popular languages: Python, Java, Go, Node.js, .NET, Ruby, etc.

• Great for polyglot microservices.

5. Designed for Cloud-Native & Microservices

• Works seamlessly in Kubernetes environments.

• Helps trace requests across distributed services and containers.

Here, I am going to deploy the OpenTelemetry Demo application using Kubernetes. You can use EKS, Minikube, or KIND. In this example, I am creating the Kubernetes cluster using KIND.

Prerequisties:

• Create an ec2 instance in your AWS account (I prefere t3.xlarge with 15GiB configure storage)

• Kubernetes 1.24+

• 6 GB of free RAM for the application

• Install kubectl

• Install Docker

• Install KIND

# For AMD64 / x86_64
[ $(uname -m) = x86_64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.29.0/kind-linux-amd64

• Install Helm 3.14+ (for Helm installation method only)

$ curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
$ chmod 700 get_helm.sh
$ ./get_helm.sh

Step 1 : Create the EC2 instance and install all the prerequisites.

Step 2 : Create the KIND cluster using the command:

kind create cluster --name otel-demo
#Check the cluster
kind get clusters

Step 3 : Install using Helm

Add the OpenTelemetry Helm repository

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts

To install the chart with the release name my-otel-demo, run the following command:

helm install my-otel-demo open-telemetry/opentelemetry-demo

Now you can see all the pods running inside the Kubernetes cluster using the following commands:

Step 4 : Use the Demo

To use the demo application, you'll need to expose the services outside the Kubernetes cluster. You can do this by using the kubectl port-forward command or by setting up service types (e.g., LoadBalancer) with optionally deployed ingress resources.

Expose Services Using kubectl port-forward

To expose the frontend-proxy service, use the following command (replace default with the namespace of your Helm chart release as needed):

kubectl --namespace default port-forward svc/frontend-proxy 8082:8080 --address 0.0.0.0

With the frontend-proxy port-forward set up, you can access:

• Web store: http://<ec2-ip>:8082/

• Grafana: http://<ec2-ip>:8082/grafana/

• Load Generator UI: http://<ec2-ip>:8082/loadgen/

• Jaeger UI: http://<ec2-ip>:8082/jaeger/ui/

• Flagd configurator UI: http://<ec2-ip>:8082/feature

Now you can view the application in your browser by using the EC2 IP address followed by the port number (in this case, port 8082).

OpenTelemetry (OTel) gathers trace data from your applications, which Jaeger uses to visualize request flows. Jaeger is a distributed tracing system that helps you monitor and troubleshoot transactions in microservices, allowing you to identify bottlenecks and optimize performance.

To trace this application, you can use the Jaeger UI in your browser with the following URL: http://<ec2-ip>:8082/jaeger/ui/

OpenTelemetry (OTel) collects telemetry data from your applications, which Grafana uses to create visual dashboards. Grafana is an open-source platform that displays this data in interactive and customizable dashboards, helping you monitor and analyze the performance of your systems in real-time.

To create visual dashboards, you can use Grafana with the following URL: http://<ec2-ip>:8082/grafana/

For more details, you can refer to the OpenTelemetry documentation: https://opentelemetry.io/docs/

Happy learning!