Introduction

Developing efficient and secure applications requires not only a well-thought-out API, but also the right choice of data transfer protocol. Web applications usually use text-based formats such as JSON or XML, but for high-performance systems that require minimal latency and low data transfer volume, it can be advantageous to use binary protocols. In this article, we'll walk through how to develop a custom binary protocol for Node.js and WebSockets-based applications, add authorization via JWT, and explore the advantages of a binary protocol over other data formats.

Why a binary protocol?

Advantages of the binary protocol:

Efficiency: Binary protocols are more compact than text-based formats (e.g. JSON). They allow data to be transmitted in a more compressed form, which reduces the amount of traffic transmitted.
Performance: With less data and no need to parsing text formats, binary protocols save resources on the client and server side.
Security: Binary data is harder to analyze on the fly compared to textual data, making binary protocols less vulnerable to attack.
Flexibility: In binary protocols, data formats can be more precisely controlled to accommodate specific data types (e.g., floating point numbers, strings, byte arrays, etc.).

System Architecture:

We will develop a system consisting of the following components:

A server on Node.js that uses WebSockets to communicate with clients.
A JavaScript client that connects to the server and uses a binary protocol to transfer data.
Authorization using JWT (JSON Web Token) to securely connect clients to the server.

Server implementation on Node.js

Dependency Installation

First, let's install the necessary dependencies:

npm init -y
npm install ws jsonwebtoken

ws is a library for working with WebSocket on the server side, and jsonwebtoken is for working with JWT.

Simple Server Code:

const WebSocket = require('ws');
const jwt = require('jsonwebtoken');

// Our JWT Secret Key
const SECRET_KEY = 'your_secret_key';

// Create a new WebSocket Server
const wss = new WebSocket.Server({ port: 8080 });

// JWT Verification Function
function verifyJWT(token) {
    try {
        return jwt.verify(token, SECRET_KEY);
    } catch (e) {
        return null;
    }
}

// WebSocket Connection
wss.on('connection', (ws, req) => {
    // Get Token from Headers
    const token = req.url.split('token=')[1];
    const user = verifyJWT(token);

    // User is not Verified
    if (!user) {
        ws.close(4001, 'Unauthorized');
        return;
    }

    console.log(`User ${user.username} connected`);

    ws.on('message', (message) => {
        // Here we looking at message type. It must be a binary buffer
        if (message instanceof Buffer) {
            // Work with binary message
            const messageType = message.readUInt8(0); // First Byte - Message Type

            if (messageType === 1) { // If Message type is 1
                const textLength = message.readUInt16BE(1); // Get Message Length
                const text = message.toString('utf-8', 3, 3 + textLength);
                console.log(`Received message from ${user.username}: ${text}`);
            } else if(messageType === 2) {
                // Work with your message types
            }
        }
    });

    ws.on('close', () => {
        console.log(`User ${user.username} disconnected`);
    });
});

console.log('WebSocket server started on ws://localhost:8080');

Code Explanation:

JWT authorization: The server checks the JWT token passed by the client on connection. If the token is invalid, the server closes the connection with an authorization error.
Binary data processing: This example assumes that the client sends binary data. The server parses the message by reading the data byte by byte. For example, the first byte of the message can be used as the message type, followed by the message length and the data itself.
WebSocket server: The ws library is used to manage connections and messages.

Client Implementation

Client code

To implement the client, we use pure JavaScript.

// Create Socket with our JWT Token
const socket = new WebSocket('ws://localhost:8080?token=your_jwt_token');

// Open Connection
socket.addEventListener('open', () => {
    console.log('Connected to server');

    // Binary Message example
    const message = "Hello, Binary World!";
    const buffer = new ArrayBuffer(3 + message.length);
    const view = new DataView(buffer);

    view.setUint8(0, 1); // Message type
    view.setUint16(1, message.length); // Message length
    for (let i = 0; i < message.length; i++) {
        view.setUint8(3 + i, message.charCodeAt(i));
    }

    socket.send(buffer);
});

// Get Response from server
socket.addEventListener('message', (event) => {
    if (event.data instanceof Blob) {
        event.data.arrayBuffer().then(buffer => {
            const view = new DataView(buffer);
            const messageType = view.getUint8(0);

            if (messageType === 1) { // Type 1 - Text Message
                const textLength = view.getUint16(1);
                const text = String.fromCharCode(...new Uint8Array(buffer.slice(3, 3 + textLength)));
                console.log(`Received message: ${text}`);
            }
        });
    }
});

// Close Connection
socket.addEventListener('close', () => {
    console.log('Disconnected from server');
});

Code Explanation:

Server Connection: The client connects to the WebSocket server by passing a JWT token via a request string.
Sending binary data: To send binary data, an ArrayBuffer is created, in which the message type and text data are written.
Receiving messages: The client expects binary data from the server and parses it using a DataView to read the bytes.

JWT token creation and validation

An example of creating a JWT token on the server side:

const jwt = require('jsonwebtoken');

// Secret Key
const SECRET_KEY = 'your_secret_key';

// Example of JWT Token Generation
const token = jwt.sign({ username: 'user1' }, SECRET_KEY, { expiresIn: '1h' });
console.log(token);

This token can be used to connect the client.

Conclusion

Using a binary protocol in combination with WebSockets and authorization via JWT allows for an efficient and secure system for client-server communication. Binary protocols, despite their complexity in implementation, provide significant performance and data reduction benefits. They are particularly relevant for highly loaded and resource-intensive applications where minimizing latency and network utilization are important.

This approach can be useful for game development, real-time systems, financial applications, and other systems that require high performance and reliability.

And of course, thanks for reading.

You can also support writing tutorials, articles and see ready-made solutions for your projects:

My Discord | My Blog | My GitHub | Buy me a Beer

BTC: bc1qef2d34r4xkrm48zknjdjt7c0ea92ay9m2a7q55
ETH: 0x1112a2Ef850711DF4dE9c432376F255f416ef5d0

Developing a custom binary protocol for Node.js and WebSockets based applications with authorization via JWT

Table of contents