In today’s world of cloud-native applications, observability is not optional — it’s essential. For those leveraging OpenSearch for log management and analytics, there’s an emerging tool that makes the setup simpler and more effective: Avien.

In this blog post, we'll explore:
✅ What is Avien?
✅ Why Use Avien with OpenSearch?
✅ Deploying Avien for OpenSearch Observability
✅ Final Result
✅ Useful Resources

📊 What is Avien?

Avien is an observability zero-to-hero project designed to simplify the deployment of log collectors, exporters, dashboards, and observability pipelines. It provides a pre-configured stack for developers, SREs, and DevOps engineers who want a hassle-free way to set up observability for Kubernetes workloads.

Key Features:

🔹 Simplified Fluent Bit + OpenSearch deployment
🔹 Pre-built Helm charts & manifests
🔹 Minimal manual configuration
🔹 Works for beginners & experts alike

🔗 Why Use Avien with OpenSearch?

While OpenSearch is a powerful search and analytics engine (originally forked from ElasticSearch), setting up log pipelines with Fluent Bit, ingest pipelines, and dashboards can be overwhelming.

This is where Avien bridges the gap:

🚀 Automates Fluent Bit setup for log shipping
🔍 Integrates seamlessly with OpenSearch indices
📊 Provides visual dashboards & queries
🔧 Reduces the boilerplate YAML configurations

🛠️ Deploying Avien for OpenSearch Observability

Let’s walk through the basic steps to deploy Avien in a Kubernetes environment to stream logs into OpenSearch.

You can follow the below youtube link Mr Abhishek Veeramalla has explained everything.

YouTube: https://youtu.be/OSShB_cXisE?si=5xdmRlNn6louChJv

GitHub Repo: https://github.com/iam-veeramalla/observability-zero-to-hero/tree/main/opensearch-stack

If you find an error while running an pod create ServiceAccount.yaml

Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedCreate 2m47s (x22 over 43m) daemonset-controller Error creating: pods "fluent-bit-" is forbidden: error looking up service account logging/fluent-bit: serviceaccount "fluent-bit" not found.

✅ Create ServiceAccount YAML:
apiVersion: v1 kind: ServiceAccount metadata: name: fluent-bit namespace: logging

kubectl apply -f fluent-bit-daemonset.yaml

kubectl get pods -n logging

kubectl get daemonsets -n logging

✅ Final Result

🎉 You now have:

Fluent Bit collecting logs from Kubernetes nodes
Logs shipped to OpenSearch indices
Visual dashboards for search and analytics

📝 Useful Resources

Avien GitHub Repo
OpenSearch Project
Fluent Bit Docs
YouTube: https://youtu.be/OSShB_cXisE?si=5xdmRlNn6louChJv
GitHub Repo: https://github.com/iam-veeramalla/observability-zero-to-hero/tree/main/opensearch-stack

Special Guidance

Special thanks to Abhishek Veeramalla Sir who has performed this task and I have tried to execute the same. You are real gem for the community.

🔎 Observability Made Easy: Using Avien with OpenSearch