Cross-System Encryption: Safeguarding Data

The problem

At DataShell, we faced what appeared to be a simple challenge: implementing encryption and decryption across various technologies. In short, our back-end uses Node.js and TypeScript, and our Command-Line Interface (CLI) is made with Python and Rust. We needed a secure method to share data between these systems. However, after initially creating the required functions in Node.js, we discovered the task was more complex than we initially thought.

First approach

At first, we used the native Node.js library, crypto, for encryption (https://nodejs.org/api/crypto.html#crypto). But, we ran into problems with compatibility between systems. This was because the library follows the OpenSSL standard (https://www.openssl.org/), which doesn't always work the same way on different platforms.

Salt to the rescue

To solve this, we switched to NaCl (pronounced "salt", short for Sodium chloride): the Networking and Cryptography library. It's a well-tested cryptography library originally made for C, C++, and Python. However, NaCl has been adapted for JavaScript, Rust, and other languages, making it perfect for our needs to encrypt data across different systems.

What does "salt" mean in cryptography? According to Wikipedia: "a salt is random data added to a one-way function that hashes data, like a password. Salt makes it harder for attackers to use precomputed tables to crack the encryption, by greatly increasing the table size needed for a successful attack". In simple terms, we use random values to make it much harder for brute force attacks to break into our encrypted data.

The solution

We developed a straightforward system for encrypting and decrypting data in TypeScript and Python, allowing data to be exchanged and processed using the same key for both tasks. This method is called Symmetric encryption. For more details on how symmetric differs from asymmetric key encryption, visit this helpful page: https://www.geeksforgeeks.org/difference-between-symmetric-and-asymmetric-key-encryption

Let's get started!

TypeScript Implementation

We'll utilize TweetNacl for TypeScript. First, install dependencies:

npm install --save tweetnacl tweetnacl-util

And here's the code for our Node.js implementation:

import { secretbox, randomBytes } from 'tweetnacl';
import { encodeBase64, decodeBase64 } from 'tweetnacl-util';

/**
 * Encrypts data using symmetric cryptography, 
 * utilizing the same key for decryption.
 */
export const encryptSymmetrical = (data: string | object, secret: string): string => {
  const secretBuffer = Buffer.from(secret, 'utf-8'); // 32 bytes
  const nonce = randomBytes(secretbox.nonceLength); // 24 bytes
  const dataBuffer = Buffer.from(typeof data === 'string' ? data : JSON.stringify(data), 'utf-8');
  const box = secretbox(dataBuffer, nonce, secretBuffer);

  // Concatenate nonce and box buffer directly
  const fullMessage = Buffer.concat([nonce, Buffer.from(box)]);

  return encodeBase64(fullMessage);
};

/**
 * Decrypts data using symmetric cryptography, 
 * utilizing the same key used for encryption.
 */
export const decryptSymmetrical = (messageWithNonce: string, secret: string): string | object => {
  const secretKey = Buffer.from(secret, 'utf-8');
  const messageWithNonceAsBuffer = decodeBase64(messageWithNonce);
  const nonce = messageWithNonceAsBuffer.slice(0, secretbox.nonceLength);
  const message = messageWithNonceAsBuffer.slice(secretbox.nonceLength, messageWithNonceAsBuffer.length);
  const decryptedBuffer = secretbox.open(message, nonce, secretKey);

  if (!decryptedBuffer) {
    throw new Error('Could not decrypt the message');
  }

  const decryptedData: string | object = Buffer.from(decryptedBuffer).toString();

  try { // Attempt to parse as object
    return JSON.parse(decryptedData);
  } catch (e) { // If parsing fails, assume it's a string
    return decryptedData;
  }
};

Let's dive in. We are going to accept strings or objects to encrypt. We need to use the Buffer.from method because the tweetnacl library works with Uint8Array. Buffer is a subclass of Uint8Array, so here we need to translate strings to Uint8Array or Buffer; we can work with both.

The structure of our encrypted data will be a string with nonce + encrypted_data.

If you are new to cryptography, nonce stands for "number used once". It's a random number added to encrypted data to protect against attacks. Basically, it helps us create a kind of signature of the data. For more information about nonces, take a look at this simple but clear resource: https://computersciencewiki.org/index.php/Nonce.

The encryptSymmetrical function is pretty clear. Nonetheless, keep in mind that:

• We create a Buffer (a subclass of Uint8Array) from the secret key. This secret key should be 32 bytes! If it's not, it will be padded. If you want to create 32-byte keys, it's as easy as this:

    import * as Nacl from 'tweetnacl';
    import * as utils from 'tweetnacl-util';
  
    const nacl: any = getTweetNacl();
  
    function getTweetNacl(): Nacl {
      try {
        return Nacl;
      } catch (error) {
        console.error("Could not import 'tweetnacl'", error);
        throw error;
      }
    }
  
    const generateLocalSecretKey = () => utils.encodeBase64(nacl.randomBytes(32));
  

• We create a nonce with the randomBytes function. This is going to be 24 bytes.

• We create a buffer with the data sent to encrypt.

• We encrypt the information using the data, nonce, and the secret converted into a Buffer.

• We concatenate the nonce and the encrypted data so that we can later decrypt the data.

• We convert it to a string (using encode64) to store that data in, for instance, a database or JSON.

How does decryption flow work?

• Again, we create a Buffer from the secret used to decrypt (remember, it's a 32-byte key!).

• We decode the encrypted data.

• Since we concatenated the nonce and the encrypted data, we split them. Here we take bytes from 0 to 23 as the nonce (the nonce is a 24-byte Buffer), and from 24 to the end as the encrypted data.

• Now, we have the nonce and encrypted data split, we decrypt it and return a string or object (whatever we had encrypted).

Python Implementation

We'll utilize PyNaCl for Python.

First, install dependencies:

pip install pyNacl

Here's the code for the Python implementation:

import nacl.secret
import nacl.utils
import nacl.encoding
import json
import logging

logger = logging.getLogger(__name__)

def encrypt_symmetrical(data, secret):
    key = base64.b64decode(secret)
    box = nacl.secret.SecretBox(key)
    if not isinstance(data, str):
        data = json.dumps(data)
    encrypted = box.encrypt(bytes(data, 'utf-8'), encoder=nacl.encoding.Base64Encoder)
    return encrypted.decode('utf-8')

def decrypt_symmetrical(data, secret):
    key = base64.b64decode(secret)
    box = nacl.secret.SecretBox(key)
    try:
        decrypted = box.decrypt(data, encoder=nacl.encoding.Base64Encoder)
        decrypted_string = decrypted.decode('utf-8')
        return json.loads(decrypted_string)
    except Exception:
        logger.error('A message could not be decrypted')
        raise Exception('Could not decrypt message')

    return decrypted_string

Rust Implementation

We'll utilize SodiumOxide for Rust. These are the dependencies to add to your cargo.toml file:

sodiumoxide = "0.2.7"
base64 = "0.21.7"

Before you call the encrypt/decrypt functions, you need to initialize the sodiumoxide library with sodiumoxide::init().unwrap();. This initialization needs to happen only once for your entire program and is represented at the beginning of both encryption and decryption functions.

Here's the code for the Rust implementation with a main function testing the functions and initiliazing the sodiumoxide library:

extern crate sodiumoxide;
extern crate base64;

use sodiumoxide::crypto::secretbox;
use sodiumoxide::crypto::secretbox::xsalsa20poly1305::{Nonce, Key};
use sodiumoxide::randombytes::randombytes;
use std::str;

pub fn encrypt_symmetrical(data: &str, secret: &str) -> String {
    sodiumoxide::init().unwrap();

    let key = base64::decode(secret).unwrap();
    let key = Key::from_slice(key.as_ref()).unwrap();

    let nonce = Nonce::from_slice(&randombytes(secretbox::NONCEBYTES)).unwrap();

    let encrypted = secretbox::seal(data.as_bytes(), &nonce, &key);
    let full_message = [&nonce.0[..], &encrypted[..]].concat();

    base64::encode(&full_message)
}

pub fn decrypt_symmetrical(data: &str, secret: &str) -> Result<String, ()> {
    sodiumoxide::init().unwrap();

    let key = base64::decode(secret).unwrap();
    let key = Key::from_slice(key.as_ref()).unwrap();

    let bytes = base64::decode(data).unwrap();
    let nonce = Nonce::from_slice(&bytes[..secretbox::NONCEBYTES]).unwrap();

    let decrypted = match secretbox::open(&bytes[secretbox::NONCEBYTES..], &nonce, &key){
        Ok(plain) => plain,
        Err(_) => return Err(()),
    };
    match str::from_utf8(&decrypted){
        Ok(s) => Ok(s.to_string()),
        Err(_) => Err(()),
    }
}

// Encryption and decryption functions as defined in the previous message...

fn main() {
    sodiumoxide::init().unwrap();

    // convert your key to a base64 string
    let key = "aaaaaAAAAA1111122222bbbbbBBBBB33"; // a 32 bytes key
    let key_as_base64 = base64::encode(key.as_bytes());

    let data = "data to be encrypted";

    let encrypted_data = encrypt_symmetrical(&data, &key_as_base64);
    println!("Encrypted data: {}", encrypted_data);

    match decrypt_symmetrical(&encrypted_data, &key_as_base64) {
        Ok(decrypted_data) => println!("Decrypted data: {}", decrypted_data),
        Err(_) => println!("Decryption failed!")
    }
}

Now you have a full cross-system encryption decryption stuff!

Hope you've enjoyed it and you find this post helpful!

Charly from The Datashell Engineering team.