My DevSecOps CI/CD Journey – Challenges, Fixes & Learnings

Recently, I set up a complete DevSecOps CI/CD pipeline where I hosted a website with Docker, built Docker images, ran containers, and later deployed everything with Kubernetes, ArgoCD, and Helm charts.

But this blog is not about the step-by-step setup.
I’ve already explained that in my YouTube video and you can also check out my GitHub repo, which is a fork of the amazing Cloud Champ project.

Instead, this blog is about the real problems I faced during this project and how I overcame them. If you’re setting up your own pipeline, this might save you some frustration.

Problem 1: TMDB API Blocked

The project was built around fetching data from TMDB (The Movie Database). But unfortunately, TMDB is blocked in my region, and the app wasn’t working.

Solution:

I did some good old desi jugaad 😅 and modified the setup so that I could still run the pipeline and test it without depending on TMDB. This way, I kept the project running instead of being stuck.

---

Problem 2: Slow Jenkins Pipeline

Initially, my Jenkins pipeline was painfully slow. At first, I thought it was some configuration issue, but later I realized it was because of:

• Heavy Docker images being pulled

• Smaller instance type

Solution:

I switched to a t2.xlarge instance (though it can run on t2.large as well). The extra compute power made the pipeline much faster and more stable.

---

Problem 3: Prometheus Targets Down

When I set up Prometheus for monitoring Jenkins and Kubernetes, the targets kept showing as Down / Uninitialized.

I faced issues like:

• Indentation error in prometheus.yml (a very common mistake)

• Ports not opened properly in security groups

• Services exposing metrics but still not visible on the dashboard

✅ Solution:

• Fixed the YAML indentation carefully

• Opened the required ports (9090, 3000, etc.)

• Ensured metrics were exposed correctly from containers

After that, even if I imported any Grafana dashboard, the metrics flowed properly.

---

Problem 4: ArgoCD Setup & Sync

I installed ArgoCD via the official docs, initialized my repo, and synced it. Later, I also installed Prometheus & Grafana with Helm charts.

Solution:

Enabling proper values in Helm made things smoother. The next thing I’m working on is building a custom Kubernetes/ArgoCD dashboard for better visibility.

---

Problem 5: Elastic IP & Webhooks

Since this was a large project, I worked on it for days. At one point, I forgot to assign an Elastic IP to my EC2 instance.

This meant:

• After 4 days of work, when the instance restarted, the public IP changed

• All my webhooks broke

• I had to delete and rebuild a big chunk of my setup

✅ Solution:

• Took an Elastic IP the next time

• Reconfigured webhooks

• Relaunched two instances:

  • t2.large for Netflix app (pipeline)

  • t2.medium/t3.medium for monitoring and ArgoCD (since ArgoCD requires at least 4GB RAM)

---

💡 Key Takeaways

• Always allocate the right instance type for your workloads. Heavy Docker images = bigger instance.

• Be extra careful with indentation in YAML (Prometheus, Helm values, ArgoCD configs).

• Always assign an Elastic IP to avoid losing progress.

• Monitoring setup requires patience – check ports, configs, and exposed metrics.

• Break the project into parts (pipeline, monitoring, deployment) so you don’t get overwhelmed.

---

🎥 My Full Walkthrough

If you want the detailed step-by-step setup, I’ve explained everything here:
📌 Watch on YouTube

And here’s my GitHub repo with all the code and configs:
📌 DevSecOps Project Repo

Big shoutout to Cloud Champ for the original project inspiration 👏

---

🔮 What’s Next?

I’m currently working on building a Kubernetes + ArgoCD custom dashboard to make the monitoring and deployments even more visual and easier to manage. Will share updates soon 🚀