Building a Video Streaming Platform with Node & React: HLS and Adaptiv

This post aims to go go over the process to create a video streaming platform like youtube/ Netflix where you have control over the video quality & speed (not big of an issue here). We'll cover the theoretical aspects, practical implementation, and how to handle HLS (HTTP Live Streaming) on both the backend and frontend.

Understanding HLS and Adaptive Bitrate Streaming

What is Adaptive Bitrate Streaming?

Adaptive Bitrate Streaming is a technique for delivering video content at varying quality levels depending on the user's network conditions. This approach ensures that users experience minimal buffering and interruptions while receiving the best possible video quality.

One of the most popular protocols used for adaptive bitrate streaming is HTTP Live Streaming (HLS), which, let me tell you was developed by Apple. HLS works by breaking the overall video stream into a sequence of small HTTP-based file downloads, each representing a short segment of the video. These segments are encoded at different quality levels (bitrates).

How HLS Works

HTTP Live Streaming (HLS) is a widely used protocol for adaptive bitrate streaming developed by Apple. It works by breaking a video into small segments and delivering them over HTTP. Here's a breakdown of the HLS workflow:

Encoding the Video: The source video is encoded into multiple bitrates, such as 240p, 360p, 720p, and 1080p. Higher bitrates offer better quality but require more bandwidth.
Segmenting the Video: The video is split into short segments, usually lasting between 2 to 10 seconds. Each segment is stored in a separate file.
Creating Playlist Files: An M3U8 playlist file is generated. This file lists all the available video segments and quality levels, guiding the video player on which files to request.
Client-Side Adaptation: The video player uses the M3U8 playlist to download video segments and adjusts the video quality based on current network conditions. If network performance drops, the player switches to a lower bitrate stream to maintain smooth playback.

Implementing HLS in Your Video Streaming Platform

Tools and Technologies

We'll use the following technologies to build our platform:

Frontend: React, which will handle video playback and user interactions.
Backend: Node.js (with TypeScript) and FFmpeg for video processing and HLS segmentation.

Backend: HLS Segmentation with Node.js and FFmpeg

First, set up a basic Express server to handle video uploads:

import express from 'express';
import * as fs from 'fs';
import path from 'path';
import cors from 'cors';
import { uploadVideo } from './controllers/Admin';
import { upload } from './middlewares/MulterMiddleware';
import serveIndex from 'serve-index';

const app = express();
const port = 8000;

// Enable CORS for all origins
app.use(cors({
    origin: '*',
    exposedHeaders: ['Content-Range', 'Accept-Ranges', 'Content-Length']
}));

app.use(express.json());

app.post('/api/upload', upload.single('video'), uploadVideo);

app.listen(port, () => {
    console.log(`Server listening at http://localhost:${port}`);
});

Create middleware to handle file uploads:

import multer from 'multer';
import path from 'path';
import fs from 'fs';

const storage = multer.diskStorage({
    destination: (req, file, cb) => {
        const uploadPath = path.join(__dirname, '..', 'videos');
        if (!fs.existsSync(uploadPath)) {
            fs.mkdirSync(uploadPath, { recursive: true });
        }
        cb(null, uploadPath);
    },
    filename: (req, file, cb) => {
        cb(null, `${Date.now()}-${file.originalname}`);
    },
});

export const upload = multer({ storage });

Process the video and segment it into different resolutions using FFmpeg:

Lets understand a little about FFMPEG now:

FFmpeg is a powerful and versatile multimedia framework used for handling video, audio, and other multimedia files and streams. It provides a comprehensive suite of libraries and tools to manipulate multimedia content. FFmpeg is widely used in both industry and open-source projects due to its flexibility, robustness, and extensive format support.

Key Features of FFmpeg:

Video and Audio Encoding/Decoding: Supports a wide range of codecs, including H.264, HEVC, MPEG-4, MP3, AAC, and many more.
Format Conversion: Converts between various multimedia formats, such as MP4, AVI, MKV, MOV, etc.
Streaming: Handles live streaming and video on demand (VOD) streaming.
Filtering and Processing: Applies various filters to multimedia content, such as scaling, cropping, adding watermarks, and more.
Transcoding: Transcodes media files to different formats, bitrates, and resolutions.

FFmpeg is often used via the command line, where users can specify complex operations through a series of command options. It is also used as a backend in many multimedia applications.

Process the Video and Segment It into Different Resolutions Using FFmpeg

The following FFmpeg command processes a video and segments it into different resolutions, creating multiple HLS playlists and segment files for adaptive bitrate streaming.

const ffmpegCommand = `ffmpeg -hide_banner -y -i "${videoPath}" \
      -vf scale=w=640:h=360:force_original_aspect_ratio=decrease -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 -hls_playlist_type vod  -b:v 800k -maxrate 856k -bufsize 1200k -b:a 96k -hls_segment_filename ${outputPath}/360p_%03d.ts ${outputPath}/360p.m3u8 \
      -vf scale=w=842:h=480:force_original_aspect_ratio=decrease -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 1400k -maxrate 1498k -bufsize 2100k -b:a 128k -hls_segment_filename ${outputPath}/480p_%03d.ts ${outputPath}/480p.m3u8 \
      -vf scale=w=1280:h=720:force_original_aspect_ratio=decrease -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 2800k -maxrate 2996k -bufsize 4200k -b:a 128k -hls_segment_filename ${outputPath}/720p_%03d.ts ${outputPath}/720p.m3u8 \
      -vf scale=w=1920:h=1080:force_original_aspect_ratio=decrease -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 5000k -maxrate 5350k -bufsize 7500k -b:a 192k -hls_segment_filename ${outputPath}/1080p_%03d.ts ${outputPath}/1080p.m3u8`;

Understanding the FFmpeg Command

-i "${videoPath}": Specifies the input video file.
-hide_banner -y: Suppresses unnecessary output and overwrites output files without asking for confirmation.
-vf scale=w=640:h=360:force_original_aspect_ratio=decrease: Resizes the video to 360p while maintaining the aspect ratio.
-c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 -keyint_min 48: Configures the video codec to H.264 with specific encoding settings:
-hls_time 4 -hls_playlist_type vod: Specifies HLS segment duration (4 seconds) and sets the playlist type to video on demand (VOD).
-b:v 800k -maxrate 856k -bufsize 1200k -b:a 96k: Configures the video and audio bitrates for 360p:
- -b:v 800k: Sets the video bitrate to 800 kbps.
- -maxrate 856k -bufsize 1200k: Sets the maximum bitrate and buffer size.
- -b:a 96k: Sets the audio bitrate to 96 kbps.
-hls_segment_filename ${outputPath}/360p_%03d.ts ${outputPath}/360p.m3u8: Defines the segment filenames and playlist for 360p resolution.
Similar blocks for 480p, 720p, and 1080p resolutions, each with their respective scaling, bitrate, and output settings.

This command processes the video and segments it into different resolutions (360p, 480p, 720p, and 1080p), creating separate HLS playlists and segment files for each quality level. This setup allows for adaptive bitrate streaming, providing the best possible viewing experience based on the user's network conditions.

The image shown below represents how the segments are created for each video quality, there is a m3u8 file for each resolution and a master m3u8 file as well.

Here is how full controller will look like:

import { exec } from 'child_process';
import { v4 as uuidv4 } from 'uuid';
import fs from 'fs';
import path from 'path';

const storeLink = (videoLink: string) => {
    const newLinkLine = `${videoLink}\n`;
    const linkFilePath = './videoLinks.txt';
    fs.appendFileSync(linkFilePath, newLinkLine, 'utf-8');
};

export const uploadVideo = async (req: any, res: any) => {
    const videoId = uuidv4();
    const videoPath = path.resolve(req.file.path); // Ensure absolute path
    const outputPath = path.resolve(`./media/videos/${videoId}`);
    const hlsPath = `${outputPath}/playlist.m3u8`;

    if (!fs.existsSync(outputPath)) {
        fs.mkdirSync(outputPath, { recursive: true });
    }

    const ffmpegCommand = `ffmpeg -hide_banner -y -i "${videoPath}" \
      -vf scale=w=640:h=360:force_original_aspect_ratio=decrease -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 -hls_playlist_type vod  -b:v 800k -maxrate 856k -bufsize 1200k -b:a 96k -hls_segment_filename ${outputPath}/360p_%03d.ts ${outputPath}/360p.m3u8 \
      -vf scale=w=842:h=480:force_original_aspect_ratio=decrease -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 1400k -maxrate 1498k -bufsize 2100k -b:a 128k -hls_segment_filename ${outputPath}/480p_%03d.ts ${outputPath}/480p.m3u8 \
      -vf scale=w=1280:h=720:force_original_aspect_ratio=decrease -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 2800k -maxrate 2996k -bufsize 4200k -b:a 128k -hls_segment_filename ${outputPath}/720p_%03d.ts ${outputPath}/720p.m3u8 \
      -vf scale=w=1920:h=1080:force_original_aspect_ratio=decrease -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 5000k -maxrate 5350k -bufsize 7500k -b:a 192k -hls_segment_filename ${outputPath}/1080p_%03d.ts ${outputPath}/1080p.m3u8`;

    exec(ffmpegCommand, (error, stdout, stderr) => {
        if (error) {
            console.error(`[ERROR] exec error: ${error}`);
            return res.json({ "error": "Error while processing your file. Please try again." });
        }

        const videoUrl = `http://localhost:8000/media/videos/${videoId}/playlist.m3u8`;

        try {
            storeLink(videoUrl);
        } catch (error) {
            console.error(`[ERROR] error while storing video URL: ${error}`);
            return res.json({ "error": "Error while processing your file. Please try again." });
        }

        res.json({ "message": "File uploaded successfully.", videoUrl: videoUrl, videoId: videoId });
    });
};

Frontend: Handling HLS Streams with React and Video.js

To handle HLS streams on the frontend, you can use the Video.js library, which provides a robust video player with support for HLS. Here's a basic setup:

Install Video.js: To play a video with HLS, you need a HLS compatible player. There are multiple popular packages to create, we will be using Video.js in here.

Use the following command to install the package in your react project.
```
 npm install video.js
```
You may visit and learn more about this through their documentation: https://videojs.com/getting-started/

Create a Video Player Component:

To play an HTTP live streamed video, you just need to pass the url to playlist.m3u8 file of that particular video. We can store the id of the video in our database, and make a request to our server to fetch the URL to playlist.m3u8.

We then pass the url and VIOLLA!!
We can easily add functionalities like adding a thumbnail to the video, adding video speed rates, volume bar, Picture in Picture etc with video.js.

We have an object of playerOptions which basically provides the information regarding all these features to the player and thats it.

You can take the reference of following code to just create a video player on your own.

To customize the UI of this video player created by Videojs, they provide a feature of plugins, which is a separate topic that is out of the scope of this post.

You may explore that on your own.

 import React, { useEffect } from 'react';
 import videojs from 'video.js';
 import 'video.js/dist/video-js.css';

 // FOR MORE VIDEO PLAYER OPTIONS, VISIT: https://videojs.com/guides/options/

 // const thumbnail = 'https://www.animenewsnetwork.com/hotlink/thumbnails/crop1200x630gNE/youtube/wm0pDk3HChM.jpg';
 // const videoUrl = 'http://localhost:8000/videos/587a4d55-661f-4a2c-b3d1-b3c4dfbfbfde/playlist.m3u8';

 // Fetch the link to playlist.m3u8 of the video you want to play
 export const VideoPlayer = ({thumbnail, videoUrl}) => {
   const videoRef = React.useRef(null);
   const playerRef = React.useRef(null);

   const options = {
     autoplay: false,
     controls: true,
     playbackRates: [0.5, 1, 1.5, 2],
     height: 400,
     responsive: true,
     poster: thumbnail,
     controlBar: {
       playToggle: true,
       volumePanel: {
         inline: false
       },
       skipButtons: {
         forward: 10,
         backward: 10
       },
       fullscreenToggle: true
     },
     sources: [{
       src: videoUrl,
       type: 'application/x-mpegURL'
     }],
     };
    useEffect(() => {

     // Make sure Video.js player is only initialized once
     if (!playerRef.current) {
       // The Video.js player needs to be _inside_ the component el for React 18 Strict Mode. 
       const videoElement = document.createElement("video-js");

       videoElement.classList.add('vjs-big-play-centered');
       videoRef.current.appendChild(videoElement);

       const player = playerRef.current = videojs(videoElement, options, () => {
         videojs.log('player is ready');
         onReady && onReady(player);
       });
     } else {
       const player = playerRef.current;

       player.autoplay(options.autoplay);
       player.src(options.sources);
     }
   }, [options, videoRef]);

   React.useEffect(() => {
     const player = playerRef.current;

     return () => {
       if (player && !player.isDisposed()) {
         player.dispose();
         playerRef.current = null;
       }
     };
   }, [playerRef]);

   return (
     <div data-vjs-player
       style={{
         width: '600px',
       }}>
       <div
         ref={videoRef}
         className=''
       />
     </div>
   );
 }

 export default VideoPlayer;

With these steps, you can set up a basic video streaming platform using HLS, Node.js, and React. The backend handles video processing and segmentation with FFmpeg, while the frontend uses Video.js to play HLS streams. You can further customize and enhance this setup according to your needs.

Let me know if there's anything more you'd like to add or adjust!

Building a Video Streaming Platform with Node & React: HLS and Adaptive Bitrate Streaming