What is Node.js?

Node.js is a JavaScript runtime environment built on the V8 engine, which is developed and maintained by Google. It was developed by Ryan Dahl in 2009 to allow JavaScript to be used outside the browser for building various type of applications such as server-side development, mobile applications, game development, operating system utilities etc.

Node.js allows running JavaScript outside the browser.

It is a single-threaded, event-driven, and uses a non-blocking I/O model, making it efficient for scalable applications.

Node.js is commonly used for:

• Back-end APIs (RESTful APIs, GraphQL)

• Real-time Applications (chat apps, gaming, etc.)

• File system operations (upload/download)

• Streaming services

Why Node.js

• Fast Performance with Google’s V8 engine.

• Non-blocking, asynchronous I/O for high scalability.

• Uses JavaScript for both front-end and back-end.

• Massive npm ecosystem for reusable packages.

• Ideal for run-time applications.

• Great for building APIs and micro-services.

• Lightweight and efficient for I/O-heavy tasks.

• Cross-platform development possible (desktop/mobile)

What is an Engine and its operations

A runtime engine in this context is a program originally written in C++ which is responsible for compiling the JavaScript code into a machine language, and execute it efficiently. Node.js uses Google’s V8 engine, the same engine which is used by Google Chrome.

How does the V8 Engine Work?

1. V8 Engine starts its execution by reading our JavaScript code that we’ve written in plain .js file.

2. V8 then converts it into a abstract syntax tree (AST) - a kind of a blueprint.

3. V8 Engine then uses a Just-In-time (JIT) compiler to convert JavaScript to machine code during execution. Using the JIT compiler makes the execution very fast.

4. V8 watches how our code runs and applies runtime optimizations to make it faster.

Summary: Node.js doesn’t compile everything upfront. It interprets, builds an AST, compiles parts of the code (JIT), applies optimization, and then runs it - and it does this dynamically which the code is running.

How does Node Script execution works?

When we execute a script using a command node file_name.js, Node.js wraps the entire file inside a function knows as the ‘Module wrapper function’. Module wrapper function, is a IIFE (Immediately Invoked Function Expression).

The function provides several local variables to each module, including exports, require, module, __filename, and __dirname.

Module Wrapper Function (Actual Implementation)

• Module wrapper functions, wraps each module in a function like below:

    (function(exports, require, module, __filename, __dirname) {
        // module code
    });
  

The provided function parameters by Node.js are: exports, require, module, __filename, __dirname.

exports:

exports is a reference to module.exports, which is the actual object that gets exported from the module holding the functions / variables to be used in other modules. If we re-assign exports to something else, it breaks the reference. To avoid confusion, always use module.exports when exporting objects.

exports is a parameter that is initialized with the value of module.exports, when the module wrapper function is invoked. This keeps ‘exports’ and ‘module.exports’ pointing to the same object — until we reassign one of them.

Internally, we can think of it more like: exports = module.exports;

Example:

// math.js
exports.add = (a, b) => a + b;
exports.sub = (a, b) => a - b;

In the above example we’ve assigned two functions ‘add’ and ‘sub’ in exports object, and these functions will be exported from the ‘math.js’ module and can be used successfully in other modules. But if we do something like below, after the above lines in our code.

exports = {
    mul: (a, b) => a * b,
    div: (a, b) => a / b
};

This newly added line will break the reference with the old defined functions i.e., ‘add’, and ‘sub’.

exports.a = 10;
exports.b = function(param) {
    // function code
}

Internally Node.js defines a key with the provided names ‘a’ and ‘b’, and the assigned function / data is a value of that key. This key-value pair gets defined within the exported object.

exports can be multiple in a file, and it is a named export. Meaning the same function name we need to use when calling the function in another module.

require:

The require function is passed as a parameter in the module wrapper function, allowing us to import and use other modules. ‘require’ function is part of Node.js CommonJS module.

Example:

const math = require('./math');

The require function is available inside each module due to the Module wrapper function that Node.js applies to each file.

How require works?

• The require function is used to load modules in Node.js application. A module can be:

  A local file/module (e.g., JavaScript file created by the user)

  **A built-in Node.**js module (e.g., fs, path, http)

  A third-party module (installed via npm e.g., express, lodash)

How Node.js Resolves Modules?

When require(‘<module>’) is called, Node.js searches for the module in a specific order:

• File Modules (Relative Path - ./ or ../):

  If the the module name starts with ./ or ../, Node.js treats it as a local file and searches in the current directory or parent directory.

• Core (Built-in) Modules:

  If no path is provided, Node.js checks for the built-in modules, and if the name matches Node.js load the requested module.

• Node Module Directory (Third-party Modules):

  If no path is provided, and the module name doesn’t match with the built-in modules, then Node.js looks for the third-party node_modules.

module:

The module object is an internal representation of the current file being treated as a module. It contains important properties that Node.js uses to manage module loading.

Example:

const add = (a, b) => a + b;
const sub = (a, b) => a - b;

module.exports = {
  add,
  sub
};

Key Properties of module:

• module.exports: The actual object that is exported.

  It is used to export values (function, objects, variables) from one module to another.

  If we reassign something to module.exports, it becomes the default export of the module.

• module.id: Identifier of the module:

  Provides a unique identifier for the module.

  For the entry file (main module), module id is . (dot).

  For other modules, it shows the absolute file path.

• module.filename: Full path of the module file.

  Similar to __filename, this provides the absolute path of the current module.

• module.loaded: Boolean value indicating if the module is fully loaded.

  If false, it means the module is still being executed.

  If true, it means the module has finished execution.

• module.children: Array of modules required by this module.

  Lists all other modules that this module depends on (i.e., those loaded using require).

• module.parent: References to the module that first requires this module.

  If the module was required by another file, module.parent shows which file imported it first.

__filename

__filename provides the absolute path to the current module.

__dirname

__dirname provides the absolute path to the parent directory which contains the current module.

Node.js Internals & Event Loop

Node.js relies on LibUV, a C Library that provides:

• Handling of asynchronous I/O operations.

• Provides the Event Loop.

• Provides the Thread Pool.

• And non-blocking I/O features that make Node.js highly efficient.

When a ‘Node Process’ starts it consists of two main components, ‘Main Thread’ and ‘Thread Pool’.

• Main Thread: A single thread that runs JavaScript code, processes events, and manages the event loop.

• Thread Pool: It holds other threads available for handling the blocking tasks (like file systems operations, cryptography, compression, etc.)

Main Thread Execution

Step 1: Initializing The Project.

• Main Thread operation starts with initializing the project, and setup the execution environment.

Step 2: Execute Top-Level Code.

• Main Thread then executes the ‘top level’ code. The code that is directly present in the module outside any function gets executed.

• Loads required modules - Calls require( ) to load dependencies (modules).

• Executes synchronous code - Runs variable initialization, function definitions, and any direct statements outside functions.

• At this step, if a function is CPU-intensive and belongs to the set of LibUV-backed APIs (fs, crypto, etc.), Node.js does not execute it on the main thread.

• Instead, LibUV detects these tasks and offloads them to the thread pool.

Step 3: Register Event Callbacks.

• Main Thread then registers the event callbacks. Event callbacks related to async tasks (e.g., timers, I/O operations, HTTP requests) are registered but not executed immediately.

• The callbacks for the thread pool tasks are also registered to be executed later.

• An event callback function is a function in a script that gets called in response to an event.

Steps 4: Start the Event Loop.

• Once all top-level code is executed, Node.js enters the event loop, handling asynchronous operations.

Event Loop Phases

Phase 1: Timer Phase.

• Event Loop checks for the expired timer callbacks (setTimeout, & setInterval), and executes the callback functions which are ready to be executed.

Phase 2: I/O Polling Phase.

• If the code depends on asynchronous input/output operations, then the callbacks of these operations are executed.

  Example: database queries, network, file system operations like readFile, writeFile, etc.

• If the operation is completed, then the callback gets executed.

• If not, then the event loop moves to the next phase.

Phase 3: Idle/Prepare Phase.

• Internal use (not commonly needed for developers)

Phase 4: Check Phase.

• Event loop executes the setImmediate( ) callbacks.

Phase 5: Close Callbacks Phase.

• Lastly, ‘close callbacks’ are executed.

• It handles, closing of sockets, streams and cleanup tasks.

  Example, if a connection is closed using socket.destroy( ), its callback executes in this phase.

After the execution of the above defined phases, the event loop checks if there’s any further operation left to be executed or not. If not then ‘exit’ else re-run the event loop operations.

Thread Pool

Node.js runs JavaScript code on a single-threaded event loop, but offloads CPU-intensive operations (like password hashing or file compression) to a worker thread in the thread pool.

Why use a Thread Pool?

Operations like password hashing, compression, and database queries block the event loop if run on the main thread. Instead, Node.js offloads these tasks on the thread pool.

For example, if our hashing operation takes 5 min to complete its execution then if node uses its main thread for this purpose the entire operation will be on hold until hashing is not completed.

How the Thread Pool Works?

• Node.js detects a CPU-intensive task (e.g., crypto.pbkdf2( ), fs.readFileSync( )).

• The task is delegated to the thread pool.

• A worker thread in the pool handles the task.

• Once completed, results are sent back to the main thread, and the callback is executed.

Default Worker Threads?

• By default 4 worker threads are available in our thread pool.

• We can change the number of workers in our thread pool by the following command:

process.env.UV_THREADPOOL_SIZE = 10;

Maximum number of Worker Threads in Node.js

• The maximum number of worker threads in Node.js depends on the system’s available resources CPU cores and memory) but technically, we can set it up to 2^31 - 1 (2147483647) threads. However, assigning such a high number is impractical and will crash our system due to the resource exhaustion.

• A general rule:

  For heavy workloads: UV_THREADPOOL_SIZE = Number of CPU Cores × 2

  For optimal performance: Keep it around 8 to 64, depending on your hardware.

How to determine the order in which timers are executed?

The order in which the timers are executed will vary depending on the context in which they are called. If both are called from within the main module, then timing will be bound by the performance of the process (which can be impacted by other applications running on the machine).

For example, if we run the following script which is not within an I/O cycle, (i.e., the main module), the order in which the two timers are executed is non-deterministic, as it is bound by the performance of the process:

// timeout_vs_immediate.js
setTimeout(() => {
    console.log('timeout');
}, 0);

setImmediate(() => {
    console.log('immediate');
});

// Output can be:
// timeout
// immediate

// Or it can be:
// immediate
// timeout
// This is because Node.js guarantees minimum delay, not exact execution time.

However, if we move the two calls within an I/O cycle, the immediate callback is always executed first:

// timeout_vs_immediate.js
const fs = require('fs');

fs.readFile(__filename, () => {
    setTimeout(( ) => {
        console.log('timeout');
    }, 0);
    setImmediate(( ) => {
        console.log('immediate');
    });
});

// Output:
// immediate
// timeout

The main advantage of using setImmediate( ) over setTimeout( ) is that setImmediate( ) will always be executed before any timers if scheduled within an I/O cycle, independently of how many timers are present.

What is process.nextTick in Node.js

process.nextTick( ) is a special asynchronous function and is technically not part of the event loop. Instead, it is a part of the microtask queue, which has higher priority than the normal callback queues and executed before event loop moves to the next phase.

Whenever we use process.nextTick(callback), the callback gets pushed to the nextTick queue, and the nextTickQueue is processed immediately after the current operation completes, before any I/O tasks or timers.

If we call process.nextTick( ) multiple times inside a single phase, all those callbacks execute immediately before moving to the next phase.

How does it fits in the Node.js lifecycle

The code execution order look something like below:

1. Synchronous code runs.

2. process.nextTick( ) callbacks execute.

3. Microtasks from Promises (microTaskQueue, .then) run.

4. The event loop proceeds to the next phase (timers, I/O, etc.).

Example:

console.log('Start');

process.nextTick(() => {
    console.log('Inside nextTick');
});

Promise.resolve().then(() => {
    console.log('Inside Promise');
});

setTimeout(() => {
    console.log('Inside setTimeout');
}, 0);

console.log('End');

// Output
// Start
// End
// Inside nextTick
// Inside Promise
// Inside setTimeout