Hello everyone, in this tutorial, I will explain how Stream, Buffer, and Pipe work in NodeJs by providing an example of each of them. So without further ado let's get started.

Introduction

before we start defining them word by word let me give you a brief about them in order to have an overview of how they work.

so, for example now when you go to youtube and search for one video while that video is been loading or streaming the data from the database to your system, then it will give you the ability to start watching it while it keeps transforming to your computer. I hope you got what I want to express!

What is Stream?

A stream can be defined as a way of starting using the data before it has finished loading.

What is Buffer?

Buffer can be defined as a process of moving the data in a small chunk of memory to the client.

Explanation

as you have seen on the illustration page, the data is transferred in small chunks of Buffer (boxes) and the Buffer contains many pieces of packages that contain small amounts of data, then that Buffer moves that data to the client.

Read Stream Code

  const fs = require('fs')

const readStream = fs.createReadStream('./text.txt', {encoding: 'utf8'})

readStream.on('data', (chunk) => {
        console.log(chunk)
    })

first, we imported the fs NodeJs core module because we want to read a file if you don't know about it It is a module that we're using to read files, paths and many more.

then we created readStream variable to store the file that we want to be read, then we assign it to fs.createReadStream('./text.txt', {encoding: 'utf8'}) the createReadStream method takes two parameters one is file path that is the file we want to read, so in this tutorial, we created text.txt file and inserted bunch of text on it, then the second parameter is object this is an option but you can put this code {encoding: 'utf8'} it will convert the binary output to string but we'll explaine another way after, and then we use readStream to access the on method that takes two parameters one is event that will wait for the data and the second one is a function that will fire whenever data is been read, and that function is required one parameter which is the chunk of data we want to use as a result.

then use node fileName to run this code but when you run it the output will be in binary, we need to convert it to the string right? even though we explained at the top how to convert but is better to know another way which is by using toString() javascript method like this console.log(chunk.toString()).

Write Stream Code

then lets us use the createWriteStream to write what we have read inside another file see the below code

const fs = require('fs')

const readStream = fs.createReadStream('./text.txt', {encoding: 'utf8'})
const writeStream = fs.createWriteStream('./text2.txt')
readStream.on('data', (chunk) => {
        console.log(chunk)
        writeStream.write("\n=======New Chunk===========\n")
        writeStream.write(chunck)
    })

this is almost the same but it takes only one parameter which is file path here we provided text2.txt file to store the data inside. then we used write() method that takes one variable which is the data that we have read, then it will start writting it to the text2.txt file whenever a new Buffer come we specify one sign which is writeStream.write("\n=======New Chunk===========\n") this will be writen at the beggining of each Buffer

then run the code you will notice that a new file with the name text2.txt is been created, open it, and have a look inside, so that is how Stream works.

Pipe

Then we have a new or simple way of doing all this stuff in an easy approach using pipe see the below code

const fs = require('fs')

const readStream = fs.createReadStream('./text.txt', {encoding: 'utf8'})
const writeStream = fs.createWriteStream('./text2.txt')
readStream.pipe(writeStream)

that is all that I wanted to articulate here, and I hope you enjoy reading it and understand it

please if there is any skeptical something do let me know in the comment section

or you can watch the video tutorial on my channel: https://www.youtube.com/channel/UCiS38Lui-aYdKv8JeTpPvkA

and here is the slide link: https://www.canva.com/design/DAFKxgE-90k/qYaE3YD2OcMj2ICj_V_yDg/view?utm_content=DAFKxgE-90k&utm_campaign=designshare&utm_medium=link&utm_source=publishsharelink

thank you, see you in the next article.