Engineering Logs & Intelligence: My Internship Journey at Invisibl Cloud Solutions

I'm thrilled to share that I’ve successfully completed my six-month internship(June 2024 – December 2024) as a Platform Engineer at Invisibl Cloud Solutions!

What started as an exploration of unfamiliar tools and domains quickly turned into one of the most fulfilling technical journeys I’ve had so far.

From building a log observability infrastructure to developing an AI-powered research agent, this internship helped me grow technically, professionally, and personally.

---

Internship Experience: Learning, Growth & Gratitude

The internship was hybrid in nature—split between working from home and at the office workspace. What made the experience even more special was that many of the senior staff were alumni from our college, making the workspace incredibly friendly and collaborative.

During my first three months, I worked on an Observability Infrastructure Project, where I dove deep into system logs, tools like Cribl and Grafana, and built a full-stack monitoring setup. Then, I transitioned to a Generative AI-based project centered around intelligent research paper discovery using RAG.

I’m proud to share that the demos for both projects received positive feedback from the client, which was deeply satisfying, especially considering both domains were completely new to me when I started.

Gratitude

• I owe a huge thank you to the entire Invisibl Cloud Solutions team for this enriching opportunity.

• A heartfelt thank you to Harish Ganesan, CEO of Invisibl Cloud Solutions, for not only trusting me with impactful work but also giving me the opportunity to work on the Gen AI project. His involvement and encouragement were truly motivating.

• A big thanks to VijayRam Harinathan for his support and mentorship in the observability project—his feedback and belief in my work made a huge difference.

• Special appreciation to Farhana S, whose consistent mentorship helped me navigate the observability space for the very first time.

• I'm equally grateful to Suryaa Azhakhiamanavalan for his guidance on the Generative AI project. His mentorship turned this challenge into a rewarding experience.

• And of course, Harshita Miranda, my project partner from day one. Working with her on both projects was a joy—we shared ideas, solved challenges together, and supported each other throughout.

• Lastly, shoutout to my amazing friends who interned alongside me—Dinesh Kumar, Sree Varshan M, and Harini S. You all made the workspace vibrant and the learning process fun!

---

Project 1: Building Observability Infrastructure for System Logs

Objective: To extend the existing metrics-based monitoring stack by incorporating log observability across Windows and Linux systems.
Tech Stack: Grafana, Loki, Cribl Edge, Cribl Stream, rsyslog, Prometheus
Key Concepts: Log collection, log routing, centralized logging, visualization

As someone new to observability, I began with research into best practices and tools. The organization already had metrics monitoring, and I was tasked with building the logs monitoring infrastructure from scratch.

Windows Log Monitoring

• Leveraged native Event Logs (Application, System, Security).

• Collected logs using Cribl Agent.

• Processed and routed them through Cribl Edge and Cribl Stream.

• Stored in Grafana Loki.

• Visualized using Grafana dashboards, with alerting and filtering options.

Linux Log Monitoring

• Linux was more challenging due to the absence of structured default logs.

• Created a Ubuntu virtual machine.

• Researched and implemented rsyslog to generate logs in custom templates.

• Integrated the logs into the same Cribl → Loki → Grafana pipeline.

Outcome: Successfully built and delivered a cross-platform proof of concept for full-stack log observability, integrated seamlessly into the existing infrastructure.

---

Project 2: Generative AI Research Agent

Objective: To build an intelligent AI agent capable of retrieving and summarizing research papers based on user queries.
Tech Stack: Haystack, FastAPI, Streamlit, Python, Arxiv API, Gemini, OpenSearch
Key Concepts: Agent pipelines, Retrieval-Augmented Generation (RAG), API development, LLM integration

In the second half of my internship, I worked on this exciting project with one other teammate. The goal was to help researchers find academic papers faster and more efficiently using Generative AI.

My Contributions

🌟Integrated the Arxiv API to fetch relevant research papers.
🌟Designed the agent pipeline using Haystack and Gemini, implementing RAG to combine retrieval with generation.
🌟Stored extracted data in OpenSearch for quick and context-aware access.
🌟Built a Streamlit-based POC to demo the functionality.
🌟Later developed a FastAPI version for production-level usage.

Outcome: Delivered a fully functional prototype in under a month, and then enhanced it into an API-ready microservice with scalable architecture.

---

Key Takeaways

This internship gave me a crash course in:
✔️Observability tools and infrastructure, from system logs to dashboard visualization.
✔️Generative AI workflows, agent chaining, and RAG pipelines.
✔️Real-world problem solving across two very different but equally challenging domains.
✔️Working in a collaborative team, presenting demos to clients, and adapting to fast-paced learning curves.
✔️Most importantly, it showed me the importance of taking initiative, asking the right questions, and owning the full cycle of a product — from idea to implementation.

Final Thoughts

Looking back, I’m proud of how much I was able to learn and build in just six months. The trust, guidance, and opportunities I received from Invisibl Cloud Solutions shaped this internship into something I’ll always remember.

From configuring log protocols on Linux to chaining LLM agents for intelligent research—this journey has been transformative. I’m grateful for every challenge, every lesson, and every teammate who made it all worthwhile.