Nodejs Runtime

deck explores the inner workings of the Node.js runtime, covering foundational concepts every backend developer should understand. You'll work through how the event loop ticks through its phases, how asynchronous scheduling differs between setImmediate, setTimeout, and process.nextTick, and how streams handle data flow with concepts like piping and backpressure. The deck also covers module systems (CommonJS versus ES Modules) and dives into Express.js middleware patterns, making it a well-rounded tour of the platform.

It's a great fit if you're starting out with Node.js, transitioning from another backend stack, or preparing for technical interviews where runtime behavior often comes up. Even developers who use Node daily can benefit, because the cards encourage you to articulate the "why" behind mechanics you may otherwise take for granted.

To get the most out of these cards, try to connect each concept to a real scenario rather than memorizing isolated definitions. For example, when reviewing the event loop phases, mentally trace what happens during a typical file read or HTTP request. Spacing your review sessions across several days will help the more intricate distinctions, like CommonJS versus ESM or the middleware execution order, move into long-term memory. If a card feels tricky, revisit the related cards nearby first to build context before tackling the harder one.

Node.js is a JavaScript runtime built on Chrome's V8 engine that executes JavaScript outside the browser. Its defining characteristic is an event-driven, non-blocking I/O model, which makes it lightweight and efficient for building scalable network applications. Despite JavaScript being single-threaded, Node.js achieves concurrency through the event loop, a mechanism that continuously checks the call stack and the callback queue. When the call stack is empty, the event loop pulls callbacks from the queue and executes them, allowing I/O operations to proceed without blocking the main thread.

The event loop progresses through a sequence of phases on each iteration: timers, pending callbacks, idle and prepare, poll, check, and close callbacks. The timers phase runs callbacks scheduled with setTimeout and setInterval. The poll phase retrieves new I/O events and may block briefly while waiting for them. The check phase then executes setImmediate callbacks, and the close callbacks phase handles things like socket close events. Understanding this ordering is essential for predicting when asynchronous code will actually run.

Two commonly confused scheduling primitives are setImmediate and setTimeout(fn, 0). setImmediate fires in the check phase of the current iteration, after the poll phase completes, while setTimeout schedules the callback for the timers phase on the next iteration. Inside an I/O callback, setImmediate always fires before setTimeout. A separate primitive, process.nextTick, fires even sooner: it runs immediately after the current operation on the microtask queue, before the event loop continues at all. Although useful, overusing process.nextTick can starve I/O by recursively deferring the loop from moving forward.

Node.js supports two module systems for organizing code. CommonJS is the original, using the require function to load modules synchronously and module.exports (or the shorter exports reference) to expose values. Each file is treated as its own module with its own scope, and modules are cached after the first load. This synchronous loading model is simple and works well on the server, where files are local and disk access is fast.

ES Modules (ESM) are the official JavaScript standard, supported in Node.js 12 and later. ESM uses the more familiar import and export syntax, is asynchronous, and supports top-level await. ESM can be enabled by adding "type": "module" to package.json or by giving files the .mjs extension. CommonJS, by contrast, is the default for .js files in projects that have not opted into ESM, and CommonJS modules can also be written explicitly using the .cjs extension.

The two systems differ in several important ways. CommonJS loads synchronously while ESM is asynchronous. CommonJS wraps each module in a function, whereas ESM runs in strict mode by default. Only ESM allows top-level await, which simplifies code that needs to wait for resources during initialization. Both systems cache modules, but ESM uses live bindings: if the exporting module updates a value, importing modules see the new value automatically. Choosing between them depends on the project, but modern Node.js code increasingly favors ESM for its alignment with the broader JavaScript ecosystem.

Streams are one of Node.js's most powerful abstractions for handling data. Rather than loading entire files or network payloads into memory, streams process data in chunks as it flows through the application. There are four kinds: Readable streams produce data (such as fs.createReadStream), Writable streams consume data (such as fs.createWriteStream), Duplex streams are both readable and writable (such as a TCP socket), and Transform streams modify data as it passes through (such as zlib.createGzip).

To connect a readable stream to a writable stream, you use the .pipe() method, which automatically manages the flow of data and handles backpressure along the way. The modern alternative is the pipeline() function from stream/promises, which has the advantage of propagating errors automatically and cleaning up resources if any step fails. Without proper piping, manually coordinating streams is error-prone and easy to get wrong.

Backpressure occurs when a writable stream cannot consume data as fast as a readable stream produces it. Node.js signals this condition by having writable.write() return false, indicating that the readable stream should pause. When the writable is ready for more data, it emits a 'drain' event, allowing the readable to resume. Properly handling backpressure is critical for memory efficiency, especially with large files or high-throughput network operations.

Buffers complement streams by providing a way to work with raw binary data. A Buffer is a fixed-size chunk of memory allocated outside the V8 heap, used for handling binary data directly in operations like file I/O, network protocols, and image processing. Buffers can be created with Buffer.alloc(size) for zero-filled memory, Buffer.from(string) for strings, or Buffer.from(array) for raw byte arrays. Common manipulations include buf.toString('utf8') for decoding, buf.slice(start, end) for sub-buffers, and Buffer.concat([buf1, buf2]) for merging. Together, streams and buffers form the foundation for efficient I/O in Node.js.

Asynchronous programming is at the heart of Node.js, and Promises are its primary abstraction. A Promise represents the eventual completion or failure of an asynchronous operation and has three states: pending, fulfilled, or rejected. Once a Promise settles, it stays settled forever. Code attaches success handlers with .then(), error handlers with .catch(), and cleanup logic with .finally(), allowing complex asynchronous flows to be expressed as readable chains.

The async/await syntax, introduced in modern JavaScript, makes working with Promises feel synchronous. An async function always returns a Promise, and the await keyword pauses execution inside the function until the awaited Promise resolves. This makes error handling natural with try/catch blocks and produces code that is easier to read and reason about than long chains of .then() calls.

When working with multiple Promises, JavaScript provides several combinators. Promise.all() takes an array of Promises and resolves when all of them succeed, rejecting on the first failure; it is ideal for running independent async operations concurrently for better performance. Promise.allSettled() waits for every Promise to either fulfill or reject without short-circuiting, which is useful when you want to know the outcome of each operation. Promise.race() resolves or rejects as soon as the first Promise settles, and Promise.any() resolves with the first fulfillment but only rejects if every Promise rejects. For older callback-based APIs, util.promisify() converts a function following the error-first callback convention into one that returns a Promise, allowing seamless integration with modern async/await code.

Express.js is the most widely used web framework on top of Node.js, and its core concept is middleware. A middleware function receives the request object (req), the response object (res), and a next function. It can run arbitrary code, modify req or res, end the request-response cycle, or call next() to pass control to the next middleware in the chain. Middleware executes in the order it is defined with app.use() or route methods, flowing top-down. If a middleware never calls next() and does not send a response, the request hangs indefinitely. Error-handling middleware is distinguished by having four parameters, (err, req, res, next), and should always be defined last so it can catch errors propagated from earlier middleware.

Express supports several kinds of middleware. Application-level middleware is bound to the app with app.use(). Router-level middleware is bound to an express.Router() instance, useful for modular route organization. Built-in middleware such as express.json() and express.urlencoded() parse incoming request bodies, while express.static() serves files. Third-party middleware like cors, helmet, and morgan add features like cross-origin support, security headers, and request logging.

Creating a REST API in Express is straightforward: create an app, attach middleware such as express.json() to parse JSON bodies, then define routes for each endpoint using HTTP verbs. Route parameters, defined with a colon prefix like /users/:id, are accessed via req.params and identify a specific resource. Query strings, the part of a URL after a question mark such as /users?role=admin, are accessed via req.query and are typically used for filtering, sorting, or pagination. RESTful design recommends using nouns for endpoints (/api/users rather than /api/getUsers), using proper HTTP methods (GET, POST, PUT, DELETE), returning appropriate HTTP status codes (200, 201, 204, 400, 401, 403, 404, 500), keeping requests stateless, and supporting filtering and pagination. Cross-origin requests from browsers can be enabled with the cors middleware, and incoming data should be validated with libraries like express-validator or Joi to ensure that requests meet expected formats before processing.

Node.js ships with a rich set of built-in modules that cover most server-side needs without external dependencies. The fs module handles file system operations: fs.readFile and fs.writeFile read and write entire files, fs.appendFile appends content, fs.mkdir and fs.rmdir manage directories, fs.readdir lists directory contents, fs.stat returns file metadata, fs.unlink deletes files, fs.rename moves or renames files, and fs.watch observes file changes. For modern code, the fs/promises variant is recommended, returning Promises that work naturally with async/await. For large files, streaming with fs.createReadStream and fs.createWriteStream avoids loading everything into memory at once.

The path module provides utilities for working with file and directory paths in a cross-platform way. path.join concatenates segments with the correct separator, path.resolve converts a relative path into an absolute path, path.basename returns the file name, path.dirname returns the directory, and path.extname returns the file extension. Using these helpers avoids subtle bugs from manual string concatenation, especially when code needs to run on different operating systems. The built-in http module lets you create servers and make HTTP requests without external libraries, exposing the underlying functionality that Express is built on top of.

For more advanced scenarios, several modules enable concurrency and subprocess control. The child_process module spawns external commands: exec runs a shell command and buffers all output, which is best for short commands; execFile runs an executable directly without a shell; spawn streams output for long-running processes or large data; and fork creates a new Node.js process with an IPC channel for direct messaging. The cluster module leverages multiple CPU cores by forking worker processes that share the same server port, with the OS distributing connections in a round-robin fashion. For CPU-intensive JavaScript work, worker_threads provides true parallel execution on threads within a single process, communicating via postMessage and 'message' events, ideal for tasks like image processing or heavy data parsing that would otherwise block the event loop.

npm, the Node Package Manager, is the default way to share and consume JavaScript libraries. It provides both a command-line tool and a vast public registry hosting millions of packages. Common commands include npm install to add dependencies, npm update to upgrade them, npm run to execute scripts defined in package.json, npm init to scaffold a new project, and npm publish to share your own package with the world.

Every Node.js project is described by a package.json manifest file. It contains the project's name and version, custom scripts (invoked with npm run), the dependencies needed at runtime, devDependencies needed only during development and testing, the main or module entry point that determines which file is loaded when the package is required, and an engines field declaring the required Node.js version. The distinction between dependencies and devDependencies matters: when deploying to production, npm install --production skips devDependencies to keep installs lean and fast.

Reproducible installs across machines and team members are ensured by package-lock.json, an auto-generated file that records the exact version of every dependency and sub-dependency. Always committing this file to version control prevents unexpected version drift and security surprises. npm uses semantic versioning (semver) in the form MAJOR.MINOR.PATCH: a major bump signals breaking changes, a minor bump adds backward-compatible features, and a patch bump fixes bugs. The caret prefix (^1.2.3) allows minor and patch updates, the tilde prefix (~1.2.3) allows only patch updates, and a bare version (1.2.3) pins the exact release.

Configuration that changes between environments, such as database URLs or API keys, is handled through environment variables accessed via process.env. Values can be set in the shell, in a .env file loaded with the dotenv package, or through system configuration. The .env file should always be added to .gitignore so secrets are not committed, and a .env.example file should be provided as a template for new developers.

Robust Node.js applications need a clear strategy for handling errors, which can occur at multiple levels. Synchronous errors and those thrown inside async functions can be caught with try/catch. Callback-based APIs follow the error-first callback pattern, where the first argument to the callback is an Error object (or null if successful) and should always be checked before using the result. Promises expose errors through .catch(), and event-based APIs like streams emit an 'error' event that must be handled to avoid crashes.

For errors that escape all other handling, Node.js provides last-resort hooks: process.on('uncaughtException') catches synchronous errors that bubble all the way up, and process.on('unhandledRejection') catches Promise rejections without a .catch() handler. The recommended practice for both is to log the error, clean up any critical resources, and exit the process; attempting to keep running after an uncaught exception can leave the application in an undefined state. Building a solid error-handling strategy means combining all these layers appropriately for each kind of failure.

Underpinning much of Node.js is the EventEmitter class, the heart of its event-driven architecture. You create an instance with new EventEmitter(), register listeners with on() (or fire-once with once()), trigger events with emit(), and remove listeners with removeListener(). Many core modules, including streams, the http server, and process itself, inherit from EventEmitter, so understanding its pattern unlocks a great deal of Node.js behavior.

The global object in Node.js, named global (analogous to window in browsers), exposes several utilities. CommonJS modules can use __dirname and __filename to access the current file's path. The process object gives access to environment variables (process.env), command-line arguments (process.argv), the current working directory (process.cwd()), the process ID (process.pid), and memory statistics (process.memoryUsage()). It also allows graceful termination with process.exit(code) and handling of OS signals through process.on('signal', handler). Together, these facilities provide the runtime introspection and control needed to build reliable, well-behaved server applications.