Hello, Swarm!

hello folks,

so, i just created a project , Healthcheck , it is a real time monitoring tool designed to keep track of website performance. It checks the status and response time of any given URL in every 30 sec , highlighting any significant changes since the last check. If a site goes down or slows unexpectedly, the system immediately sends an alert via email. For added security, users can sign up and log in using a simple otp based verification process sent directly to their email, no passwords needed .The entire application built in microservice.

The technology i used to create and to deploy are

Python , Mysql , Docker ,Swarm , Aws

1. In python I used Fastapi , request module , threadpool , mysql connector , dotenv ,logging, some other core modules like datetime, random , etc.

2. Mysql used for saving the data.

3. Docker used for containerization the microservices.

4. Aws used for deploying the application, I used ec2 for computing and docker swarm for orchestrating and managing the containers across that EC2.

So, it is not so simple project , it was a complex project yet it exciting project , i do learn a lot form it , i will mention later what i learn, but here why i choose this project , it is a subproject of a very unique and large project.

The things I learned from this project:

So, as i learn docker all by myself from various sources, so the thing i notice the real deployment and a tutorial deployment of a basic html page it very very different , there are multiple obstacle during creating this project.

1. The manager and workers node need to communicate so for that we need to change some inbound rules of nodes

2. The environment is offer more than just storing passwords.

3. The mysql container maybe ready but still it doesn’t guarantee the database connection , mostly i take more time to fully start the database connection.

There are multiple things i learn so writing everything is unnecessary, I think so i should move the problem part what i encounter and how i manager.

1. So as i mentioned it is a microservice application , During local testing under bridge network i encounter the problem of dashboard cannot connect the database because the db is not ready for make the connection even i am using the “depend_on” still so after some research i switch from “version 3.8 to 2.1 “ of docker compose file and used “condition” under "depend_on

2. So after local testing success , I write the swarm file but there was no condition as far as i researched , so i use healtcheck to ensure that before making connection it should be ready.

3. After even deploying it is not working the issue is i do not allow traffic from some specific ports which nodes used to communicate , we need to allow traffic from TCP: 7946 , UDP : 7946 and 4789, UPD 4789 used by overlay network.

4. I used SMTP but still it doesn’t sending mail , there i forget to change outbound rules also i need open port 597.

5. My dashboard is interactive CLI but in swarm it can’t work as it contain input(), so currently i am creating html basic front end

There are multiple small small problems i don’t want to write them all 🙃.

Also at last i want to share how i optimize the things.

1. I reduce the image size by 50-60%. By using base image as Python Slim

2. I made the custom logs which helps monitoring

3. I optimized my application to use minimal memory by keeping each container lightweight. I used only one replica for each service since the app doesn't need high load handling right now.

VISIT GITHUB: https://github.com/Harshit10j2004/Healthcheck.git for codes