Introduction:

This article explores vertical and horizontal scaling concepts, focusing on their implementation in Node.js and Rust projects. It discusses challenges like single-threaded CPU usage and port conflicts while highlighting solutions such as Node.js's cluster module for process management. The piece delves into capacity estimation, horizontal scaling with AWS services like Auto Scaling Groups (ASGs), and includes practical steps for setting up and managing resources on AWS. Additionally, it provides insights into scaling via a Node.js app and mentions using Elastic Beanstalk for simplified deployment and scaling. The content emphasizes the importance of proper resource management and cleanup post-scaling experiments.

---

Vertical scaling

Vertical scaling means increasing the size of your machine to support more load

Single threaded languages

Multi threaded languages

Node.js

Let’s run an infinite loop in a JS project and see how our CPU is used

let c = 0;
while (1) {
  c++;
}

This confirms that only a single core of the machine is being used. We got 3 different processes using 100% CPU each.

Rust

use std::thread;

fn main() {
    // Spawn three threads
    for _ in 0..3 {
        thread::spawn(|| {
            let mut counter: f64 = 0.00;
            loop {
                counter += 0.001;
            }
        });
    }

    loop {
        // Main thread does nothing but keep the program alive
    }
}

Implementing horizontal scaling in Node.js project

You can start multiple node projects then? If there are 8 cores, then just start 8 projects?

node index.js
node index.js
node index.js
node index.js
node index.js
node index.js
node index.js
node index.js

This, ofcourse has a lot of problems

1. Just ugly to do this, keep track of the processes that are up and down

2. Processes will have port conflicts, you’ll have to run each process on a saparate port

This is where the cluster module comes into the picture

import express from "express";
import cluster from "cluster";
import os from "os";

const totalCPUs = os.cpus().length;

const port = 3000;

if (cluster.isPrimary) {
  console.log(`Number of CPUs is ${totalCPUs}`);
  console.log(`Primary ${process.pid} is running`);

  // Fork workers.
  for (let i = 0; i < totalCPUs; i++) {
    cluster.fork();
  }

  cluster.on("exit", (worker, code, signal) => {
    console.log(`worker ${worker.process.pid} died`);
    console.log("Let's fork another worker!");
    cluster.fork();
  });
} else {
  const app = express();
  console.log(`Worker ${process.pid} started`);

  app.get("/", (req, res) => {
    res.send("Hello World!");
  });

  app.get("/api/:n", function (req, res) {
    let n = parseInt(req.params.n);
    let count = 0;

    if (n > 5000000000) n = 5000000000;

    for (let i = 0; i <= n; i++) {
      count += i;
    }

    res.send(`Final count is ${count} ${process.pid}`);
  });

  app.listen(port, () => {
    console.log(`App listening on port ${port}`);
  });
}

Notice different pids in different devices

Browser

Postman

Curl

💡

Try to figure out why there is stickiness in the browser. Why the request from the same browser goes to the same pid

Capacity estimation

This is a common system design interview where they’ll ask you

1. how would you scale your server

2. how do you handle spikes

3. How can you support a certain SLA given some traffic

Answer usually requires a bunch of

1. paper math

2. Estimating requests/s

3. Assuming / monitoring how many requests a single machine can handle

4. Autoscaling machines based on the load that is estimated from time to time

Example #1 - PayTM app

Example #2 - Chess app

Horizontal scaling

Horizontal scaling represents increasing the number of instances you have based on a metric to be able to support more load.

AWS has the concept of Auto scaling groups, which as the name suggests lets you autoscale the number of machines based on certain metrics.

Buzz words

Images (AMI) - Snapshots of a machine from which you can create more machines Load balancer - An entrypoint that your users send requests to that forwards it to one of many machines (or more specifically, to a target group). Its a fully managed service which means you don’t have to worry about scaling it ever. AWS takes care of making it highly available.

Target groups - A group of EC2 instances that a load balancer can send requests to

Launch template - A template that can be used to start new machines

💡

Please make sure you get rid of all your resources after this.

There are two ways you can use ASGs

• Create a EC2 instance.

  • 1. install Node.js on it https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-ubuntu-20-04

       2. Clone the repo - https://github.com/100xdevs-cohort-2/week-22

• Create an AMI with your machine

• Create security group

• Launch template

  • Ref for User data - https://stackoverflow.com/questions/15904095/how-to-check-whether-my-user-data-passing-to-ec2-instance-is-working

  •         #!/bin/bash 
            export PATH=$PATH:/home/ubuntu/.nvm/versions/node/v22.0.0/bin/
            echo "hi there before"
            echo "hi there after"
            npm install -g pm2
            cd /home/ubuntu/week-22
            pm2 start index.js
            pm2 save
            pm2 startup
    

  • 

• ASG

  • Callout on availability zones - ASGs try to balance instances in each zone

    • 

• Load balancer

  • Add an HTTPS Listener from your domain, request a certificate from ACM

• Target group - Attach the target group to the ASG

Autoscaling part

You can create an dynamic scaling policy

Try playing with the Min and max on the ASG

Try killing servers

Try to stop a few servers in the ASG. Notice they spin back up to arrive at the desired amount.

Simulate a scale up

Try running an infinite for loop on the instance to see if a scale up happens

let c = 0;

while (1) {
    c++;
}

You’ll notice the desired capacity goes up by one in some time

Try turning the infinite loop off and notice a scale down happens

Scaling via a Node.js app

Create a new user with permissions to AutoscalingFullAccess

import AWS from 'aws-sdk';

AWS.config.update({
  region: 'ap-south-1',
  accessKeyId: 'YOUR_ACCESS_KEY',
  secretAccessKey: 'YOUR_ACCESS_SECRET'
});

// Create an Auto Scaling client
const autoscaling = new AWS.AutoScaling();

// Function to update the desired capacity of an Auto Scaling group
const updateDesiredCapacity = (autoScalingGroupName: string, desiredCapacity: number) => {
  const params = {
    AutoScalingGroupName: autoScalingGroupName,
    DesiredCapacity: desiredCapacity
  };

  autoscaling.setDesiredCapacity(params, (err, data) => {
    if (err) {
      console.log("Error", err);
    } else {
      console.log("Success", data);
    }
  });
};

// Example usage
const groupName = 'node-app-1'; // Set your Auto Scaling group name
const newDesiredCapacity = 3; // Set the new desired capacity

// Call the function
updateDesiredCapacity(groupName, newDesiredCapacity);

Cleanup

Please delete all things one by one

1. ASG

2. Target group

3. Load balancer

4. Launch template

5. Image

6. Instance that the image was created from

💡

Try using elastic beanstalk. Gives you the same benefits w/o the developer having to create all of these