js-bundler

How it is built

Architecture overview

A JavaScript bundler reads a program that is spread over many files and packages and writes the few files a browser or server should load. It starts from entry points, finds every module they import, parses each module once, links the imports to the exports they name, throws away code nobody uses, splits what remains into chunks, rewrites names to be shorter and writes the result with a source map so that debuggers still show the original code. The same tool usually also runs the development loop: a server that rebuilds quickly and pushes changed modules into the running page. The idea that makes it work is a module graph built once and analysed in passes: parsing is independent per file and therefore parallel; linking, tree shaking and chunking are graph algorithms over the whole result; and almost everything expensive can be skipped on the next build if the inputs' content hashes did not change.

Parts:

• Parser — source text to an AST with scopes and symbols, for JavaScript, TypeScript and JSX — Pratt parsing, Recursive-descent parsing, Name resolution, Arena allocator
• Module resolution and scan — turns import specifiers into files and walks the dependency graph — module-resolution (no page yet), BFS and DFS, Work-stealing deque, Swiss table, string-interning (no page yet)
• Linker and tree shaker — binds imports to exports, merges modules, removes unreachable code — tree-shaking (no page yet), Linking and relocation, Topological sort
• Chunking — decides which module goes into which output file, and what loads lazily — code-splitting-lazy-loading (no page yet), HTTP caching semantics
• Printer and minifier — renames symbols, folds constants, prints code and a source map — minification-name-mangling (no page yet), source-maps (no page yet)
• Cache and incremental engine — remembers per-file and per-query results across builds — query-based-incremental-computation (no page yet), hash-consing (no page yet), Fast hash functions
• Dev server and hot reload — watches files, rebuilds the changed part, swaps modules in the browser — Hot reloading, File system watching

 entry points ──► resolve ──► read ──► parse ──► AST + symbols + imports    (parallel per file)
       ▲             │  (package exports, extensions, aliases, plugins/loaders)
       │             └───── new files found ──► worklist  (scan until the graph is closed)
       │
       ▼  whole graph in memory
   link: bind imports ↔ exports ──► tree shake (mark from entries) ──► chunk (entry-set per module)
       │
       ▼ per chunk, parallel
   rename / minify ──► print JS + source map ──► content-hash file names ──► write
       ▲
   cache: per-file results keyed by content hash       dev server: watch ──► rebuild changed ──► HMR update

How the parts fit together

Parser

The parser's output is the shared data structure: an AST plus, for each identifier, a symbol that is bound to its declaration. esbuild's architecture notes describe the design aim of "as few full-AST passes as possible for better cache locality": three passes in total — (1) lexing, parsing, scope setup and symbol declaration; (2) symbol binding, constant folding, syntax lowering and mangling; (3) printing and source-map generation (esbuild docs/architecture.md). Symbols are referenced by 64-bit ids and stored in one flat array per file, so a new symbol can be created without name collisions and the AST can be cloned cheaply. Name resolution is exactly the second pass: bind each use to a declaration with a scope stack, respecting shadowing and hoisting. Recursive-descent parsing handles statements and declarations and Pratt parsing handles expressions with operator precedence; the lexer is called on demand because whether / starts a regular expression or is a division depends on the parse state (esbuild notes). TypeScript and JSX are handled by the same parser, with type annotations skipped like whitespace and enums or namespaces converted to JavaScript. The AST for one file lives and dies together, so an Arena allocator is the natural allocation strategy: bump a pointer while parsing, free everything at once when the file's chunk is printed.

The contract with the scan: a parsed file exposes its import records (specifier, kind: static import, require() or dynamic import()), its exports, and its top-level statements split into units that can be kept or dropped independently.

Module resolution and scan

Resolution maps a specifier such as lodash/map or ./util to a file, following the Node and ESM rules: extensions, directory indexes, the exports map of a package, aliases and CommonJS interop. That is module-resolution (no page yet). It is mostly file-system calls, and esbuild's notes observe that "the overhead of syscalls in import path resolution appears to be very high", so resolver and file system results are cached. The scan is a parallel worklist: the worklist starts with the entry points, each file is parsed on its own goroutine and may add more files (esbuild notes). In another language the same pattern runs on a Work-stealing deque pool so that idle workers take files from busy ones. The visited set and the path-to-module table are hash maps hit on every import, where a Swiss table with a fast hash does well, and repeated strings (identifiers, paths) are worth interning (string-interning, no page yet). Reachability over the finished graph is plain graph traversal.

Linker and tree shaker

Linking "merges multiple modules into a single file so that imports from one module can reference exports from another" and works differently by module format (esbuild notes). A module that uses CommonJS features is wrapped in its own closure, as webpack does; a pure ES module becomes part of one shared scope, so imports become direct references with no runtime lookup. This second form is "scope hoisting", and it is what makes later steps possible. The conceptual relative is the linker of a native toolchain (Linking and relocation): symbols are matched across units, definitions merged, unresolved names reported. Hybrid CommonJS and ES graphs need conversion code; that is the main source of surprising output.

Tree shaking (tree-shaking, no page yet) treats the graph's nodes as top-level statements ("parts" in esbuild). A part is marked as having side effects or not (let foo = 123 has none; let foo = bar() does, because the call must stay). Edges lead from a part to the parts that declare symbols it references and to every side-effecting part of files it imports. The pass starts from the side-effecting parts of the entry point and follows edges; whatever it never reaches is dropped (esbuild notes). The contract with package authors is the sideEffects field in package.json, which webpack documents as a hint that unused files of a package are safe to prune. Rollup, which popularised the term, "statically analyzes the code you are importing, and will exclude anything that isn't actually used" (Rollup docs). Execution order of ES modules is a depth-first post-order of the import graph; where a cycle breaks that order, Topological sort shows which edge is the problem.

Chunking

Code splitting (code-splitting-lazy-loading, no page yet) answers "which module goes in which file". esbuild's method is an advanced form of tree shaking: the traversal is run once per entry point, each part records the set of entries that reached it, and the set decides its chunk. Parts reachable from only one entry go to that entry's chunk; parts reached by several go to a shared chunk, so a page downloads shared code once. Every dynamic import() target counts as an additional entry point, which gives lazy loading (esbuild notes; splitting is off by default and enabled with a flag). webpack exposes the same decision as the SplitChunksPlugin and the import() syntax (webpack code-splitting guide). Chunk file names carry a content hash ([contenthash] in webpack: it "will change as well" when the content changes), so a browser can cache them for a long time and fetch only chunks that changed; that is the cache-validation logic of HTTP caching semantics applied to build output. The contract: a chunk's name must be a function of its content and of the names of the chunks it imports, or an update to a leaf silently leaves stale parents.

Printer and minifier

Minification renames internal symbols to the shortest names, folds constants and removes dead branches (minification-name-mangling, no page yet). esbuild's notes explain the frequency trick: with ASCII-only identifiers there are 54 possible one-character names and 3453 two-character names, so the most-used symbols get the shortest names, and symbols in sibling scopes may share a name. Dead-branch removal depends on defines: with process.env.NODE_ENV replaced by "production", the development branch of a library becomes unreachable and its require() is never followed, so the development file never enters the graph. The printer writes code and a source map: a map from positions in generated code to the original files, names and lines, whose mappings field is encoded as base64 variable-length quantities (ECMA-426, the source-map standard; source-maps, no page yet). The contract: positions must be recorded while printing, in the same pass, since after printing the correspondence is lost.

Cache and incremental engine

A bundler that rebuilds a large project on every save must avoid redoing work. There are three levels:

• Per file: reuse the parsed AST of an unchanged file. esbuild's design requires that data structures that live across builds be immutable so they can be shared, which is what makes its watch mode incremental (esbuild notes).
• Persistent: webpack can store its cache on disk (cache.type: 'filesystem'), with buildDependencies listing the config and loader files whose change must invalidate it (webpack cache docs). Turbopack goes further and "caches results down to the function level" and persists them between runs (Next.js Turbopack docs).
• Query-based: the build is expressed as memoised functions with recorded dependencies; after a change only affected functions rerun and their results are revalidated by comparing outputs. This is query-based-incremental-computation (no page yet); sharing equal subtrees by interning (hash-consing, no page yet) makes "did this result change" a pointer comparison. Keys are content hashes from a fast non-cryptographic hash (Fast hash functions).

The contract with the rest: every input that can change the output (file content, config, environment variables, plugin versions) must be part of the cache key; anything missing causes a stale build.

Dev server and hot reload

The dev loop watches the file system (File system watching: OS notices are hints, to be debounced and verified), rebuilds what changed and pushes it to the browser. Two architectures exist. Vite's original design served source code on demand over native ES modules and pre-bundled dependencies once; on an edit it applies Hot Module Replacement "over native ESM" to swap one module (Vite docs). Bundle-based tools (webpack, Turbopack) keep a bundled graph and compute the update, with Turbopack bundling lazily, only what the dev server is asked for (Next.js docs). Hot reloading is the block: replace code in a running process through an indirection and migrate the live state, in a browser the module registry and the accept callbacks. The contract: a module that cannot accept an update propagates it to its importers, and the page reloads if no one accepts.

Data and control flow

One production build:

1. Read config, create the entry list; resolve each entry to a file. Resolver results are cached because each lookup costs file-system calls.
2. Parse entries in parallel; every import record is resolved and its target added to the worklist. Parsing and code generation are most of the work and fully parallelisable; linking is largely serial (esbuild FAQ).
3. When the graph is closed, link: bind imports to exports across modules, decide per module whether it is CommonJS-wrapped or hoisted.
4. Tree shake from each entry's side-effecting parts; compute for each part the set of entries that reach it and group parts into chunks.
5. For each chunk in parallel: rename symbols, fold constants (process.env.NODE_ENV and other defines), print code with the source map, hash the content, name the file.
6. Write the files, plus a manifest that maps entries to their hashed chunk names so HTML or a server can reference them.

One edit in the dev server:

1. The watcher reports a changed file; the server debounces and confirms the content hash changed.
2. Only that file is reparsed; its imports are re-resolved, and unchanged dependencies reuse cached results.
3. The server computes which modules changed and sends the update to the browser over a socket.
4. The browser fetches the new module, runs its accept callback and re-renders the affected part, keeping state where the framework allows.

Scaling

Limits in the order they bite:

1. Parse and print time. Levers: a compiled, parallel implementation (esbuild is written in Go and "Parallelism is used heavily"; Rolldown, Rspack and Turbopack are written in Rust, per their sites), a custom parser instead of a general-purpose compiler front end, few full passes.
2. Serial link and chunk step. It is mostly serial in esbuild (FAQ). Levers: do less in it (precomputed per-file summaries), run per-chunk work in parallel after it.
3. Cold start on a big project. Levers: a persistent disk cache, lazy bundling in dev, pre-bundling of rarely changing dependencies once.
4. Rebuild latency. Levers: file-level and query-level incrementality; only rebuild modules whose content hash changed; push module-level updates instead of reloading.
5. Output size. Levers: tree shaking (needs ES modules and honest sideEffects), scope hoisting, code splitting by route, minification, content-hashed chunks so users download only what changed.
6. Memory. The whole graph is in memory. Levers: arenas, interned strings, compact symbol tables, dropping ASTs after printing.

Failure modes

Real implementations

The alternative: no bundling, serving native ES modules directly. It works in development and for small apps; Vite's own docs state that shipping unbundled ESM in production is inefficient because of the extra network round trips from nested imports. Bundle when the import graph is deep, when you need tree shaking and minification, or when you target several environments. For libraries, ship ES modules with honest sideEffects and let the application's bundler do the final shaking.

Sources

• E. Wallace, esbuild architecture documentation and FAQ: why is esbuild fast; esbuild API.
• webpack docs: concepts, tree shaking, code splitting, caching, cache configuration, hot module replacement.
• Rollup introduction; Vite: why and features; Rolldown; Rspack; Turbopack.
• ECMA-426: Source map format.