System design - 1

• Lets start with simple example, you using amazon.com

High level overview

• When you hit amazon.com as a client the request will be sent to the server

• flow :

  client → hits amazon.com → sent to DNS → finds IP → locates server with that IP → server accepts the request → gives response

  • IP : Internet protocol, in simple terms an address to the location (V4,V6 versions)

    DNS : Domain name resolution system (maps the IP to Domain name)

Medium level

What happens when there are too many users?

• Here comes the scaling, types : Vertical and horizontal scaling

  • Vertical scaling increase the resources allocated for server (Storage,RAM etc)

    • Cause Down time, when rebooting system to handle many request when adding resources

  • Horizontal scaling adds more servers to handle many users

    • But wait, IP is different how to map which server serve which user!?. Load balancer is the saver

Load Balancing

• Multiple servers with different IP map to single load balancer, they register themself to load balancer when booting on high usage

• All requests now hit load balancer first, then it separates the request to different servers in turn (reverse proxy)

  • A proxy forwards client requests to the internet, while a reverse proxy forwards internet (or external) requests to internal servers.

    • Reverse proxy : A reverse proxy sits in front of web servers and handles incoming client requests by forwarding them to the appropriate backend server. It is often used for load balancing, caching, and protecting backend infrastructure.

    • Proxy: A proxy server sits between a client and the internet, forwarding the client's requests to external websites and returning the response. It is commonly used for anonymity, security, and content filtering.

• What about the case of overloading load balancer? Load balancer has many micro architecture which make it simple

• server here is called Elastic compute EC2 in terms of AWS amazon.

• Load balancer is Elastic Load balancer (ELB) in terms of AWS amazon.

Low level

What about the system which uses microservice architectures?

Microservice architecture in backend typically refers to how the components of a backend system are broken down into smaller, independent services.

• One single service can have multiple EC2 server up and running for it, hence they need load balancer there too for each service.

• If there are multiple service end point, then how will the request be sent to correct end point service? API gateway is the only way!!

  API Gateway

  An API Gateway is like the front desk for all your backend services. It routes requests

  to the appropriate microservice.

  • It is like loadbalancer for routes

  • It routes to correct microservice loadbalancer

  • Now all the request hit the API gateway

    

Bit level

What happens when single event trigger many tasks ?

• One way solve this by synchronous task execution (blocks other events, not preferable)

• Other way is using async task execution.

• For example

  One event of payment, might trigger email and bulk worker on CSV upload.

1. Queues

   • So, for each task which is triggered, there is a dedicated queue to handle them async way.

   • Each queue might be tied to horizontally scaled workers, which helps in speed as well as for rate limiting in 3rd party services like gmail.

   • Has dead letter queue for acknowledgement and failure handling.

     A Dead Letter Queue (DLQ) is a special queue used to store messages that can't be processed successfully.

   • SQS (Simple queue system) used for queues in AWS amazon.

   • Can use pushing or pulling mechanism, to listen for events.

     • push : queue itself pushes

     • pull : the worker polls on timely basis, long polling and short polling

       Short Polling: Instantly checks for available messages.

       Long Polling (Preferred): Waits (up to 20 sec) for a message to arrive before responding.

2. Sub/pub or notification triggers (Event driven)

   • All the connected services will be actively listening to the changes in its subscribed event.

   • Publisher will publish the information about the event occurred.

   • It will be sent to the subscriber workers which in turn might depend on external services like gmail, but here there is case of failure and discard of information.

   • Since the information sent b publisher is not saved anywhere it will cause loss of data.

   • There is no acknowledgement.

   • SNS (Simple notification system) in AWS amazon.

3. Fanout Architecture (Combine)

• Pub/sub is connected to the queue, fast and efficient with acknowledgement.

  Publish/Subscribe Fan-Out Pattern in Serverless Architectures Using SNS, SQS and Lambda: Read it on medium

Rate limiting strategies

What happens when the DDOS (a type of cyberattack that floods a network or server with malicious traffic from multiple sources, making it unavailable to legitimate users) occurs?

• Leaky bucket

• Token bucket

  Leaky Bucket enforces a steady output rate by leaking data at a constant speed, smoothing traffic bursts; Token Bucket allows bursts by accumulating tokens and sending data only if enough tokens are available, enabling flexible rate control.

Extras

• To handle overwhelming of database use read and write replicas.

  Read replica is a copy of the primary database used for read operations to reduce load, while write replica (usually the primary) handles all write operations.

• Cache the responses using Redis etc

  Redis is an in-memory data store used for caching, real-time analytics, and fast key-value data access.

• CDN (Cloud front in AWS amazon)

  A CDN (Content Delivery Network) is a distributed network of servers that delivers web content to users faster by serving it from locations closer to them.

  • Add this before load balancer.

  • To handle which CDN IP to go use Anycast

    Anycast is a network routing method where multiple servers share the same IP address, and traffic is routed to the nearest or most optimal server based on network topology.

Thanks to amazing video by Piyush Garg.