Build Your Redis (In-memory DB)

Redis is a popular in-memory key-value store known for its speed and versatility. And yes we can implement our own In-Memory Database. For learning and experimentation, let's build a simplified Redis-like server in Node.js, containerize it with Docker, and run it in an isolated environment. This will give us a feel for both how Redis works and how to use Docker.

1. Set up the folder for your Custom Redis

   First, we’ll create a Node.js server that acts like Redis, allowing us to SET, GET, EXISTS, DEL, INCR, TTL(time-to-live), List commands, LPUSH, RPUSH, LPOP, RPOP, LRANGE, SET commands, SADD, SISMEMBER, SMEMBERS, Hash Set Commands, HGET, HSET, HGETALL, and main PING.

   Step 1: Create a Directory and Initialize a Node.js Project

   In your terminal:

mkdir custom-redis 
cd custom-redis 
npm init -y

This sets up a new Node.js project with a package.json file.

Step 2: Install Required Packages

We’ll need net to manage TCP connections and redis-parser to parse Redis-like commands.

npm install redis-parser

Step 3: Write the Custom Redis Server

Create a new file called index.js:

const net = require('net'); //for TCP protocol
const Parser = require('redis-parser');

//take a store for storing the value of key and value
const store = {}; 
const ttlStore = {}; //This wil, track the TTL(time-to-live) for keys

const server = net.createServer(connection => {
  console.log('Client connected...')

  connection.on('data', data => {
    const parser = new Parser({
      returnReply: (reply) => {
        const command = reply[0].toLowerCase();
        const key = reply[1]; 

        //function to check if a key has expired and remove it
        const checkExpiry = (key) => {
          if(ttlStore[key] && ttlStore[key] <= Date.now()){
            delete store[key];
            delete ttlStore[key];
          }
        };
        //perform the expiration check for very command
        if(key) checkExpiry(key);

        switch(command){
          //Firsr, redis health checkup
          case 'ping' : {
            connection.write('+PONG\r\n');
            break;
          }

          //for string set,get n al
          case 'set': {
            const value = reply[2];
            store[key] = value;
            delete ttlStore[key]; // Remove TTL if it was set before
            connection.write('+OK\r\n');
            break;
          }
          case 'get': {
            const value = store[key];
            if (!value) {
              connection.write('$-1\r\n');
            } else {
              connection.write(`$${value.length}\r\n${value}\r\n`);
            }
            break;
          }
          case 'del': {
            if(store[key]){
              delete store[key];
              delete ttlStore[key];
              connection.write(':1\r\n'); //OK
            }else{
              connection.write(':0\r\n'); //false
            }
            break;
          }
          case 'exists': {
            if(store[key]){
              connection.write(':1\r\n'); //true
            }else{
              connection.write(':0\r\n'); //false
            }
            break;
          }
          case 'incr':{
            const incrValue = parseInt(reply[2]);
            if(!store[key]){
              store[key] = '0';
            }
            const newValue = parseInt(store[key])+incrValue;
            store[key] = newValue.toString();
            connection.write(`:${newValue}\r\n`);
            break;
          }
          case 'expire' : {
            const ttl = parseInt(reply[2]);
            if(store[key]){
              ttlStore[key] = Date.now() + ttl * 1000;
              connection.write(':1\r\n');
            }else{
              connection.write(':0\r\rn');
            }
            break;
          }
          case 'ttl' : {
            if(!store[key]){
              connection.write(':-2\r\n');
            }else if(!ttlStore[key]){
              connection.write(':-1\r\n');
            }else{
              const ttl = Math.floor((ttlStore[key] - Date.now()) / 1000);
              connection.write(`:${ttl}\r\n`);
            }
            break;
          }

          // LIST comamands
          case 'lpush' : {
            const value = reply[2];
            if(!Array.isArray(store[key])){
              store[key] = []; //
            }
            store[key].unshift(value); //left side push
            connection.write(`:${store[key].length}\r\n`);
            break;
          }
          case 'rpush' : {
            const value = reply[2];
            if(!Array.isArray(store[key])){
              store[key] = [];
            }
            store[key].push(value); //right side push
            connection.write(`:${store[key].length}\r\n`);
            break;
          }
          case 'lpop' : {
            if(!Array.isArray(store[key]) || store[key].length === 0){
              connection.write('$-1\r\n');
            }else{
              const value = store[key].shift();
              connection.write(`$${value.length}\r\n${value}\r\n`);
            }
            break;
          }
          case 'rpop' : {
            if(!Array.isArray(store[key]) || store[key].length === 0){
              connection.write('$-1\r\n');
            }else{
              const value = store[key].pop();
              connection.write(`$${value.length}\r\n${value}\r\n`);
            }
            break;
          }
          case 'lrange': {
            const start = parseInt(reply[2]);
            const end = parseInt(reply[3]);
            if (!Array.isArray(store[key])) {
              connection.write('*0\r\n');
            } else {
              const range = store[key].slice(start, end + 1);
              connection.write(`*${range.length}\r\n`);
              range.forEach(item => connection.write(`$${item.length}\r\n${item}\r\n`));
            }
            break;
          }

          //Set Commands
          case 'sadd' : {
            const value = reply[2];
            if(!store[key]){
              store[key] = new Set();
            }
            const sizeBefore = store[key].size;
            store[key].add(value);
            const sizeAfter = store[key].size;
            connection.write(`:${sizeAfter - sizeBefore}\r\n`);
            break;
          }
          case 'smembers' : {
            if(!store[key] || !(store[key] instanceof Set)){
              connection.write('*0\r\n');
            }else{
              const members = Array.from(store[key]);
              connection.write(`*${members.length}\r\n`);
              members.forEach(member => connection.write(`$${member.length}\r\n${member}\r\n`));
            }
            break;
          }
          case 'sismember' : {
            const value = reply[2];
            if(store[key] && store[key] instanceof Set && store[key].has(value)){
              connection.write(':1\r\n');
            }else{
              connection.write(':0\r\n');
            }
            break;
          }

          //HASH commands
          case 'hset' : {
            const field = reply[2];
            const value = reply[3];
            if(!store[key]){
              store[key] = {};
            }
            store[key][field] = value;
            connection.write(':1\r\n');
            break;
          }
          case 'hget' : {
            const field = reply[2];
            if(!store[key] || typeof store[key] !== 'object' || !(field in store[key])){
              connection.write('$-1\r\n');
            }else{
              const value = store[key][field];
              connection.write(`$${value.length}\r\n${value}\r\n`);
            }
            break;
          }
          case 'hgetall' : {
            if(!store[key] || typeof store[key] !== 'object'){
              connection.write('*0\r\n');
            }else{
              const fields = Object.entries(store[key]);
              connection.write(`*${fields.length * 2}\r\n`);
              fields.forEach(([field, value]) => {
                connection.write(`$${field.length}\r\n${field}\r\n`);
                connection.write(`$${value.length}\r\n${value}\r\n`);
              });
            }
            break;
          }
          default:
            connection.write('-ERR unknown command\r\n');
        }
      },
      returnError: (err) => {
        console.log('Parser Error =>', err);
        connection.write(`-ERR ${err.message}\r\n`);
      }
    })
    // console.log('=>', data.toString())
    parser.execute(data);
    // connection.write('+OK\r\n')
  })
})

server.listen(8000, () => {
  console.log("Server is listening on 8000");
})

2. Test the Custom Redis Server Locally

   For this, you should install the Main Redis. if you have MacOS use Homebrew and install using brew install redis. In Ubuntu, use the apt-get update, apt-get upgrade, apt-get install redis-server

   Then Run the Node Js Server:

   node index.js

   Now open another terminal. first, check you have main Redis is running. Hit this command redis-cli. 127.0.0.1:6379\> this will come. 6379 is the main port of the Redis Server. Hit PING you will get the PONG in return.

   Now for our custom Redis. Run this command

   redis-cli -p 8000. “-p” is PORT, in this, you are telling that redis-cli runs on your localhost 8000 servers.

   Try commands like SET, GET, EXPIRE, and TTL:

SET mykey "hello"
GET mykey
EXPIRE mykey 10
TTL mykey

lpush mylist hey
lpush mylist hello
rpush mylist allgood
rpop mylist
lpop mylist

sadd myset udai
# and all .....

3. Containerize the Redis Clone with Docker

   Let’s create a Dockerfile to make our Redis clone portable.

   Step 1: Write the Dockerfile

   Create a file named Dockerfile in the project directory:

    FROM ubuntu 
   
    RUN apt-get update
    RUN apt-get install -y curl
    RUN curl -sL https://deb.nodesource.com/setup_18.x | bash -
    RUN apt-get upgrade -y
    RUN apt-get install -y nodejs
   
    # set the working directory
    WORKDIR /app
   
    # copying the package.json and package-lock.json files
    COPY package.json package.json
    COPY package-lock.json package-lock.json
    RUN npm install
   
    #copy the rest of the application code
    COPY index.js index.js
   
    #expose the port on which the server will sun
    EXPOSE 8000
   
    #command to tun server
    CMD [ "node", "index.js" ]
   

   Step 2: Build the Docker Image and push on Docker HUB.

   First, create your account at https://hub.docker.com/

   And then create a repository with a name like custom-redis-clone,

   give a short description for that, now copy the yourusername/repository name, ex: udaichauhan/custom-redis-clone. this name will help you create the image.

   Now in the terminal of your project vscode terminal

   run this command: docker build -t yourusername/repositoryname .

   Don’t forget to put “.” at last.

   Step 3: Run the Docker Container

   docker run -p 8000:8000 yourusername/repositoryname

   This will start your image also create the container and start the server: Server is listening on 8000

   Step 4: Move into Container

   docker exec -it containername bash

   this container name you will get from the docker dashboard, or you can also use docker exec -it containerID bash

   this container ID will also be obtained from the docker dashboard copy and paste it here.

   Now we are in Ubuntu bash, first, here we have to install the Main Redis so that we can run our custom Redis.

    cd ..
    #now you are at root directory of ubuntu :/# something like this
    apt-get update
    apt-get upgrade
    apt-get install redis-server
   
    redis-server  #this will start the redis server in your container
   

   Step 5: Open a new Terminal

   Connect to the same docker container.

   docker exec -it containerID bash

   Check if is redis installed in your container or not.

   Run this: redis-cli

   If yes, now connect your Custom Redis

   redis-cli -p 8000

   Now use Custom Redis Clone in Docker Container.

   Run standard Redis commands here:

SET realkey "world"
GET realkey

GitHub Repo

GitHub Repo

Summary

You’ve now built a simple custom Redis server in Node.js, containerized it with Docker, and tested it alongside an official Redis instance. This setup helps you understand how Redis-like servers work and allows you to experiment with building and deploying custom applications in Docker!

Thanks and Happy Coding !!!