[CHORE] perf: cheaper unshare() copy for store-linked packages that run install scripts (#1918 follow-ups) · Issue #1934 · vltpkg/vltpkg · GitHub
Skip to content

[CHORE] perf: cheaper unshare() copy for store-linked packages that run install scripts (#1918 follow-ups) #1934

Open

Description

@ruy-bot

📋 Description

#1918 hardlinks packages with install scripts from the global store like any other package and, right before one of their lifecycle scripts runs, replaces their shared files with private copies (src/graph/src/reify/unshare.ts), so the script cannot write into the store. On the default install, which runs no scripts, that is a clear win (large −5…−8.5 %, babylon −3…−6 %, see the A/B on the PR). On installs that opt into running a store-linked package's scripts it is a cost: the package is linked and then copied, where before #1918 it was copied once at placement. That path is install / ci with an --allow-scripts query that matches the package, and vlt build on a tree installed by #1918 (once per tree).

Measured (same A/B, large: core-js@3.50.0, core-js@2.6.12, es5-ext@0.10.64, 5,647 files; install with VLT_ALLOW_SCRIPTS=':scripts', warm store + lockfile, 20 runs per arm, 4 vCPU Linux box): +230 ms wall per install (+13.8 %, 1671 → 1902 ms median) and +830 ms CPU in both rounds (hyperfine user 1.19 / 1.24 → 1.47 / 1.49 s, system 0.76 / 0.75 → 1.32 / 1.33 s, round 1 / round 2) against the parent commit. The PR description's own figure is +158 ms (+7.5 %) on ci --allow-scripts=:scripts.

Why it costs more than the old copy. For every file with nlink > 1, unshare() does readFile + writeFile to a tmp sibling (wx, plus chmod for executables) + rename over the link, 16 files at a time, every step a hop through libuv's thread pool (4 threads by default, which vlt does not raise), on top of the link itself. The pre-#1918 placement copy in src/tar/src/link-tree.ts was one synchronous readFileSync + writeFileSync (wx, plus statSync + chmodSync for executables) per file on the main thread, with no link, no per-file rename and no thread-pool hops. On this box the system time of the install rises ≈75 %.

💡 Proposed follow-ups

Both are non-blocking for #1918 and independent of each other.

  1. Cheaper copy in unshare(). Copy with fs.copyFile(p, tmp, COPYFILE_EXCL) (libuv copies in the kernel where it can, no buffer round trip through JS, mode preserved) and keep the rename(tmp, p); the safety property is unchanged, since the copy targets the new tmp inode and the shared inode is never written. Also try a lower concurrency than 16 (the default pool has 4 threads) or a plain synchronous loop like the placement copy, and measure on a small and a large box.
  2. Copy at placement when the scripts will run in the same install. When the allow-scripts query matches a store-linked package, copying it at placement, as before graph: hardlink install-script packages, copy them only before their scripts run #1918, avoids the link-then-unshare round trip. Today the query is evaluated in reify_() after addNodes() / extract (filterNodesByQuery runs after checkNeededBuild(), just before binChmodAll() and build()), and it cannot simply be moved earlier: :scripts matches on node.manifest, which a lockfile install without an existing node_modules only has once the package is extracted, and on buildState, which checkNeededBuild() sets after extraction. Deciding copy-vs-link at placement therefore needs the allow-scripts result for the node before it is placed (e.g. from the store index's scripts flag plus the query): a design change rather than a tweak, and the first item is worth doing regardless.

✅ Acceptance

  • install --allow-scripts=:scripts on large within noise of the pre-graph: hardlink install-script packages, copy them only before their scripts run #1918 copy path (≈1.67 s on the box above; reproducible with the vltpkg/benchmarks cache+lockfile variation on large with VLT_ALLOW_SCRIPTS=':scripts' exported).
  • No change on the default install (no scripts).
  • A script that writes into its own package still leaves the store untouched (the existing unshare and reify build tests).

Activity

  1. self-assigned this
    on Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

choreperfperformance related issue

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions