Improving single-node performance in Node.js is a common challenge developers face, as Node.js’s single-threaded nature can limit its ability to handle high concurrency or heavy load in some applications. To enhance performance, you can consider a variety of solutions and techniques. Below are some common methods:

Using Multi-process or Multi-threading Techniques

(1) Node.js Cluster Module: The cluster module in Node.js allows for multi-process applications, enabling each CPU core to run a separate Node.js process. This can significantly improve the application’s concurrency performance.

(2) Worker Threads: The built-in Worker Threads in Node.js help you take advantage of multi-core CPUs for parallel computation. You can use Worker Threads to handle long-running computational tasks without blocking the main thread, thereby enhancing concurrency handling.

(3) PM2 for Monitoring and Multi-process Management: Using PM2 to configure multi-process mode for starting and managing Node.js processes can improve the application’s stability and resilience. PM2 automatically handles process restarts, load balancing, and other settings, making it easier to manage Node.js applications in production.

In this article, we will focus on introducing the Cluster module.

Introduction to Node.js Cluster Module

When building a scalable API server, one of the most common challenges is handling high traffic and ensuring the application remains performant, reliable, and responsive under heavy load.

Node.js, by default, operates on a single thread, which means it can only utilize one CPU core at a time. This can become a limitation when building high-performance applications that need to handle a large number of concurrent requests. To address this, Node.js provides the Cluster module, which allows you to take full advantage of multi-core systems by forking multiple worker processes, each running on its own core.

What is the Cluster Module?

The Cluster module in Node.js enables you to create child processes (workers) that can share the same server port, allowing your application to handle multiple requests concurrently, across multiple CPU cores. Each worker runs an instance of the Node.js event loop and can execute your application’s code independently, but they can still share server resources, such as memory or ports, thanks to the cluster management system.

Why Use the Cluster Module?

  • Increased Concurrency: By forking multiple workers, you can significantly improve the concurrency of your application, making it capable of handling more requests per second (RPS). This is especially useful for high-traffic applications.
  • Utilizing Multi-Core Systems: The cluster module enables you to make use of all available CPU cores on your system, whereas Node.js by default only uses one core.
  • Fault Tolerance: If a worker process crashes, the cluster module can automatically spawn a new worker to replace it, improving the resilience of your application.

How Does It Work?

The cluster module works by forking child processes, where each child (worker) runs a separate instance of your Node.js application. The main (primary) process controls the workers, distributing requests among them. When a request comes in, the primary process forwards the request to one of the workers. Each worker is responsible for handling requests independently.

Here’s a simple flow of how the cluster module works:

Primary process: The master process that spawns worker processes based on the number of available CPU cores. Worker processes: Each worker process listens to the same port and handles requests independently. Load Balancing: The operating system’s load balancer (e.g., round-robin) handles the distribution of incoming requests to the workers.

Increasing Concurrency with Cluster

After using the cluster module or a process manager like PM2 to launch multiple instances of your application, the concurrency potential increases according to the following formula:

Concurrency ≈ Single-thread concurrency × Number of CPU cores

For example, if a single core can handle 20 concurrent requests, an 8-core machine could theoretically support:

20 × 8 = 160 concurrent requests

By combining caching and worker_threads, you can further reduce the load on the main thread, resulting in a significant boost in theoretical RPS (requests per second).

Example Usage

Here’s a basic example that demonstrates how to set up a Node.js application with the cluster module:

Setting Up the Cluster Logic in server.js

To fully utilize the cluster module, you need to modify your server.js file to set up the cluster and worker logic. Here’s a sample implementation:

import cluster from 'cluster';
import os from 'os';
import express from 'express';
import next from 'next';

// Get the number of CPU cores
const numCPUs = os.cpus().length;

// Determine whether it's in development mode
const isDev = process.env.NODE_ENV !== 'production';

// Initialize the Next.js application
const app = next({ dev: isDev });
const handle = app.getRequestHandler();

if (cluster.isPrimary) {
  // Primary process: responsible for forking worker processes
  console.log(`Primary process ${process.pid} is running, forking ${numCPUs} worker processes...`);

  for (let i = 0; i < numCPUs; i++) {
    cluster.fork(); // Create a worker process for each CPU
  }

  cluster.on('exit', (worker, code, signal) => {
    console.log(`Worker process ${worker.process.pid} has exited (code: ${code}, signal: ${signal})`);
    console.log('Restarting worker process...');
    cluster.fork(); // Automatically restart the worker process
  });
} else {
  // Worker process: runs Express and Next.js
  app.prepare().then(() => {
    const server = express();

    // Add custom API routes
    server.get('/api/hello', (req, res) => {
      res.json({ message: `Hello from worker ${process.pid} process!` });
    });

    // Handle Next.js default pages and resources
    server.all('*', (req, res) => {
      return handle(req, res);
    });

    const port = process.env.PORT || 3000;
    server.listen(port, (err) => {
      if (err) throw err;
      console.log(`Worker process ${process.pid} is running at http://localhost:${port}`);
    });
  });
}

Updating package.json

In your package.json, make sure to specify the commands for running the application in both development and production modes. Here’s an example:

"scripts": {
  "dev": "node server.js",
  "start": "NODE_ENV=production node server.js",
}

Running the Application

To start the application in development mode, run the following command:

npm run dev

For production mode, ensure that the environment variable NODE_ENV is set to “production” and then run:

npm run start

Verifying the Node Processes

To ensure that all processes, including the main process and worker processes, are running, you can use the following command:

ps aux | grep node

This will display a list of all the Node.js processes running, including the primary process and the multiple worker processes that have been spawned.

By using the cluster module, you can maximize your Node.js application’s concurrency, taking full advantage of the system’s CPU resources. This setup is particularly useful for high-traffic applications, ensuring they can handle a large number of simultaneous requests.