Is it possible to rebuild all built content in a node_modules directory? #199880

gordonmessmer · 2026-06-22T18:35:18Z

gordonmessmer
Jun 22, 2026

🏷️ Discussion Type

Question

Body

I'm trying to work out a build process for Node.js software in Fedora. Fedora requires, among other things, that all software is built from source, and that all software is built offline.

I think that we would like to see a process that is more or less:

Online:

Download all of the required dependencies (e.g. via "npm install")
Remove pre-built native binaries, Wasm and possibly other generated files
Create an archive of the "node_modules" directory

Offline build:

Unpack the "node_modules" archive
Rebuild all of the native binaries, Wasm, etc that was removed from node_modules
Install only the runtime dependencies in an "install root" that will be archived for distribution

I am unclear on how to do 2 and 3 in the offline build.

If I run "npm install" and remove native binaries and Wasm, and then run "npm rebuild", npm will spin for a few seconds and then tell me that it is done. However, none of the generated files have been re-built. I don't see any evidence that npm is trying to run a build in the node_modules collection. But if I create a new node_modules and populate it with symlinks to the same collection of node modules and then run "npm rebuild", npm does appear to try to run a "build" process for each node module. I don't understand how npm is determining whether or not to try to build each module.

Naturally, some packages such as those that provide tsc, or rollup, or other build tools will need to be packaged and installed system wide in order to "rebuild" a node_modules collection that doesn't contain any native binaries. But assuming we build those, we still need to be able to tell npm to rebuild a module collection from which some content has been removed.

Once that's sorted, we also need to be able to "install" only the runtime dependencies into a directory that we'll archive for distribution. I think we might be able to "prune" the node_modules directory of devDependencies and install what is left, but I would appreciate any guidance on best practice or generally simpler processes.

Thank you

gordonmessmer · 2026-06-23T08:27:56Z

gordonmessmer Jun 23, 2026
Author

Removing those artifacts later may leave npm with no signal that regeneration is required.

Right, I don't expect removing the artifact to be the signal to rebuild the module. Initially, I thought that "npm rebuild" would simply try to rebuild everything that has a build action defined. The modules in this collection must have such actions defined, because if they're symlinked into place, "npm rebuild" does try to execute build scripts.

I just don't understand how npm determines whether or not to run them.

When you remove the native binaries/Wasm files, are you also removing any build-state markers (for example .node outputs, generated directories, package-specific cache files, etc.), or only the final artifacts themselves?

I'll take another look, but I don't think I saw any .node outputs. In any case, I'm asking for advice because I don't know what I'm looking for.

Thanks!

Ayushgupta2408 · 2026-06-23T07:41:05Z

Ayushgupta2408
Jun 23, 2026

Your observations are consistent with how npm rebuild works. npm rebuild does not generally inspect the contents of a package and determine whether generated artifacts are missing. Instead, it primarily re-runs the lifecycle scripts (install, preinstall, postinstall, etc.) for packages that npm considers rebuildable, particularly native addons.

As a result, simply deleting a generated binary, .node file, or .wasm file from an existing node_modules tree does not necessarily cause npm rebuild to regenerate it. If the package's install scripts are not re-executed, npm may report success without recreating the missing artifacts.

For a fully reproducible offline build, it may be more reliable to:

Download and cache all package sources and tarballs online.
Recreate node_modules from those sources in the offline environment rather than attempting to repair a partially stripped tree.
Run the package installation/build lifecycle from scratch so that all native addons and generated artifacts are produced locally.
Use npm prune --omit=dev (or install with --omit=dev) to produce a runtime-only dependency tree for distribution.

One challenge is that npm itself does not maintain a complete inventory of generated artifacts (native binaries, Wasm files, transpiled output, etc.), so there is no built-in command that says "rebuild everything that was previously generated and later removed."

If Fedora's requirements are strict about rebuilding all generated content from source, the most robust approach may be to reconstruct the dependency tree from source tarballs and execute the package lifecycle scripts in the offline build environment rather than relying on npm rebuild against a modified node_modules directory.

2 replies

gordonmessmer Jun 23, 2026
Author

As a result, simply deleting a generated binary, .node file, or .wasm file from an existing node_modules tree does not necessarily cause npm rebuild to regenerate it

Right, I don't expect removing files to cause npm to rebuild them. I expected "npm rebuild" to cause npm to rebuild them. Fedora currently has a very weird process that involves symlinking modules into a "node_modules" directory and if I do that then "npm rebuild" appears to run build scripts in many module directories, regardless of whether any files have been removed. That's the thing I'd like to understand better, because it might be key to fixing Fedora's process.

Download and cache all package sources and tarballs online.

Can npm do that, or are you suggesting that we inventory the modules, find all of the sources, and build them all?

Run the package installation/build lifecycle from scratch so that all native addons and generated artifacts are produced locally

Do you know where I can find documentation that would describe doing that? Or are you suggesting that I'd need to find the source archive for each individual module and read the developer's documentation for the build?

Thanks

Ayushgupta2408 Jun 24, 2026

Thanks for the clarification.

My understanding is that npm rebuild reruns the install/build lifecycle scripts for packages already present in node_modules, but it does not guarantee that every generated artifact (.node, .wasm, binaries, etc.) will be recreated. The actual behavior depends on how each package's build scripts are implemented.

Regarding Fedora's symlinked node_modules setup, I suspect the key point is that npm rebuild is triggering lifecycle scripts across many dependencies. That behavior is likely independent of whether generated files were removed and may be worth investigating further, since it could explain why rebuilding works differently from a standard installation.

For the offline/source-build workflow, I am not aware of a single npm command that guarantees a complete rebuild of all generated artifacts across the dependency tree. In practice, distributions typically prefetch sources, install dependencies from local caches/mirrors, and rely on package-specific lifecycle scripts (install, postinstall, prepare, node-gyp rebuild, etc.) to generate artifacts locally.

The relevant npm documentation would be the npm rebuild command and npm lifecycle scripts documentation, but beyond that, the exact build process is ultimately determined by each package.
Good luck!!!

gordonmessmer · 2026-06-23T19:18:33Z

gordonmessmer Jun 23, 2026
Author

Does npm ls --json produce identical output for the original tree and the symlinked tree?

No, in the symlinked case the "resolved" value for each module is "file:../node_modules_prod/" and in the standard case it is an https URL to a module tgz.

Are there differences in package metadata such as .package-lock.json, installation state, or link flags?

There is no node_modules/.package-lock.json in the symlinked case, but deleting that file from the standard case does not cause "npm rebuild" to attempt to rebuild modules.

Does npm rebuild --foreground-scripts --loglevel verbose show why certain lifecycle scripts are being selected in one case but not the other?

Not why, no. In the symlinked case, I see npm attempting to rebuild a module:

npm info run @eslint/eslintrc@2.1.4 prepare node_modules_prod/@eslint/eslintrc npm run build

> @eslint/eslintrc@2.1.4 prepare
> npm run build

npm verbose cli /usr/bin/node-24 /usr/bin/npm
npm info using npm@11.8.0
npm info using node@v24.13.1
npm verbose title npm run build
npm verbose argv "run" "build"
npm verbose logfile logs-max:10 dir:/builddir/.npm/_logs/2026-06-23T19_15_43_710Z-
npm verbose logfile /builddir/.npm/_logs/2026-06-23T19_15_43_710Z-debug-0.log

> @eslint/eslintrc@2.1.4 build
> rollup -c

sh: line 1: rollup: command not found

... and in the standard case I don't, but I don't see anything in the output that indicates why it tries to do the rebuild in this case.

I think the specific module that fails may depend on directory order. A few minutes ago I'm sure I saw it fail because "tsc" wasn't found, suggesting that it had tried to build a different module first.

sahare-mayur-0071 · 2026-06-24T04:39:19Z

sahare-mayur-0071
Jun 24, 2026

The new detail about the resolved field looks significant.

In the symlinked tree, npm is seeing dependencies resolved from file: locations, while in the normal tree they are resolved from registry tarballs. That may explain why you're seeing prepare scripts execute in the symlinked case.

The npm lifecycle model treats local/file/link dependencies differently from packages installed from published tarballs. For published packages, the expectation is generally that the package already contains its build artifacts. For local or linked packages, npm often runs prepare because the package is being treated more like source that may need to be built before use.

The log snippet showing:

npm info run @eslint/eslintrc@2.1.4 prepare

appears consistent with that interpretation.

If that's correct, npm rebuild may not be deciding based on missing artifacts at all. Instead, it may be deciding based on package type (registry package vs file/link package) and the lifecycle hooks associated with that type.

One thing that might help confirm this would be comparing:

npm ls --json
npm explain <package>
package metadata under node_modules/.package-lock.json

between the registry-installed tree and the symlinked tree, specifically looking for link/file dependency flags.

My guess is that the symlinked layout is causing Arborist to classify the packages differently, which in turn makes prepare eligible to run during rebuild.

0 replies

sahare-mayur-0071 · 2026-06-24T06:20:58Z

sahare-mayur-0071
Jun 24, 2026

The latest findings seem to support the idea that npm is treating the symlinked tree as a collection of local/file dependencies rather than registry-installed packages.

The fact that npm rebuild is invoking prepare for packages resolved via file: but not for packages resolved from registry tarballs appears consistent with npm's lifecycle behavior.

One thing I'd be curious about is whether the same behavior can be reproduced with a minimal test case:

Install a package normally from the registry.
Replace it with a local file: dependency pointing to the same package source.
Run npm rebuild in both cases and compare which lifecycle scripts execute.

If the behavior differs solely based on the resolved type, that would suggest Fedora's current symlink-based workflow is unintentionally leveraging npm's handling of local dependencies to force builds that wouldn't normally occur for published packages.

That may not solve the broader "rebuild everything from source" problem, but it could explain why the current process behaves differently from a standard installed tree.

0 replies

Cyb4819 · 2026-06-24T09:39:49Z

Cyb4819
Jun 24, 2026

A few things that might help based on similar offline builds I've dealt with:

npm rebuild --force can sometimes work better than plain npm rebuild, since npm otherwise skips modules it thinks are already built. If your version doesnt support it, people usually just delete the generated .node files / build folders so it re-runs.
Another hacky but common trick is touching binding.gyp in native modules to force a rebuild.
For offline flows, something like this is often used:
- online: npm install --ignore-scripts --no-bin-links
- strip binaries/Wasm
- offline: npm rebuild or manually trigger rebuilds for native modules
For packaging, npm prune --production is usually what people use to drop devDependencies before archiving.

npm rebuild alone isnt always reliable — some packages expect a full npm install to run postinstall, so those need special handling in offline setups.

0 replies

Mohammadshaqboua · 2026-06-24T09:59:40Z

Mohammadshaqboua
Jun 24, 2026

0 replies

This comment was marked as low quality.

Sign in to view

Sunbelt Computer Software

PL/B Language Development and Support

Is it possible to rebuild all built content in a node_modules directory? #199880

Uh oh!

🏷️ Discussion Type

Body

Replies: 7 comments · 5 replies

This comment was marked as low quality.

Uh oh!

gordonmessmer Jun 23, 2026 Author

Uh oh!

Uh oh!

gordonmessmer Jun 23, 2026 Author

Uh oh!

This comment was marked as low quality.

Uh oh!

Uh oh!

gordonmessmer Jun 23, 2026 Author

This comment was marked as low quality.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Replies: 7 comments 5 replies

gordonmessmer Jun 23, 2026
Author

gordonmessmer Jun 23, 2026
Author

gordonmessmer Jun 23, 2026
Author