fix: skill folders created after Gateway start go unwatched on Linux with Node 24 by steipete · Pull Request #178095 · openclaw/openclaw · GitHub
Skip to content

fix: skill folders created after Gateway start go unwatched on Linux with Node 24 - #178095

Merged
steipete merged 2 commits into
mainfrom
steipete/jolly-lederberg-a770d2
Oct 20, 2026
Merged

steipete merged 2 commits into
mainfrom
steipete/jolly-lederberg-a770d2

Conversation

@steipete

Copy link
Copy Markdown
Contributor

What Problem This Solves

Fixes: on Linux under Node 24, a skills folder that does not exist when the Gateway starts (for example a new <workspace>/skills) never gets a working watch, so creating it and editing skills inside it is only picked up by the 30-minute safety refresh.

The same defect made src/infra/owned-directory-observation.test.ts fail every Linux Node run (reported as a flake from a Crabbox AWS gate at 762750c0). It is deterministic; daily CI runs Linux tests on Bun, so only full validation's Node arm sees it.

User Impact

User impact: on Linux with Node 24, skill folders created after the Gateway starts are watched natively again, so new and edited skills refresh within the normal debounce instead of up to 30 minutes later. No config or behavior change on macOS, Windows, Bun, or Node 26.

Why This Change Was Made

observeOwnedDirectory depends on a native watch of a missing directory throwing synchronously, so it can watch the nearest existing ancestor and re-attach when the directory appears. Node 24's Linux recursive fs.watch fallback (Linux has no native recursive inotify) inverts its throwIfNoEntry check and returns a watch that never reports for a missing root; upstream fixed it in nodejs/node#65635, which also notes that other setup errors such as EACCES were swallowed, and Node 26.12 throws as expected. The observer then keeps that inert handle forever: while the directory is missing the recovery pass re-opens another inert handle, and once it exists the handle is "healthy" by every check.

watchDirectory already owns synchronous admission for the macOS helper path. Its in-process branch now admits recursive roots the same way (fs.accessSync(root, R_OK), which also rejects an unreadable root) before calling fs.watch.

The test file had a second, test-only defect: two sharing cases counted fs.watch calls to prove one native registration, but Node builds a Linux recursive watch from nested fs.watch calls of its own (1 + one per directory, plus one per file on Node 24), so the spy saw 2+ calls for one OpenClaw registration. The recursive cases now stub the recursive native handle the way Node 24 behaves (a watch that never reports, even for a missing root). The spy counts only the observer's registrations on every runtime, and the missing-ancestor case fails on the original defect everywhere instead of only on Linux Node 24.

Accepted edge cases: a root deleted between admission and fs.watch on Node 24 Linux still gets an inert handle until a later rename or recovery pass re-admits it; Node 24's fallback also swallows ENOSPC (inotify limit). Both are rare and covered by the existing safety refresh; the access check goes away with Node 24.

Evidence

Reproduced first, on Linux. The original gate evidence (Crabbox AWS, source 762750c0) holds exactly one Node 24.19.0 run of this file, and that run failed. The gate then stopped, so there were no further Node iterations; Bun passed. On current main (a860374e) the failure is deterministic.

Linux Blacksmith Testbox through scripts/crabbox-wrapper.mjs, focused profile (ci-check-focused-testbox.yml, 2 vCPU, lease tbx_01m5bvnbrgb1t6btq8fvm2kdhm, run). Each iteration ran OPENCLAW_VITEST_RUNTIME=node OPENCLAW_VITEST_FS_MODULE_CACHE=0 node scripts/test-projects.mts src/infra/owned-directory-observation.test.ts src/infra/directory-watch-host.test.ts --maxWorkers=1 --no-cache, as a fresh process per iteration:

Arm Node 24.19.0 Node 26.12.0
unchanged main 5/5 failed: watch called 2 times, called 5 times, ancestor path mismatch 5/5 failed: called 2 times, called 5 times
new tests + main adapter 2/2 failed: ancestor path mismatch only 2/2 failed: ancestor path mismatch only
this PR 5/5 passed 5/5 passed

Standalone probe on the same Linux box (spy-equivalent wrapper around fs.watch):

v24.19.0 recursive existing dir: fs.watch entries = 4 ["<dir>","<dir>","<dir>/child","<dir>/child/file.txt"]
v24.19.0 recursive missing dir: RETURNED a watcher
v24.19.0 recursive unreadable dir: RETURNED a watcher
v26.12.0 recursive existing dir: fs.watch entries = 3 ["<dir>","<dir>","<dir>/child"]
v26.12.0 recursive missing dir: threw ENOENT
v26.12.0 recursive unreadable dir: threw EACCES

Local macOS: the adjusted missing-ancestor test fails against the main adapter under both Node and Bun (Received: .../owned-directory-ancestor-*/agents/work, the same shape as the CI failure) and passes with the fix. oxlint and oxfmt are clean on both files, and tsgo -p test/tsconfig/tsconfig.core.test.infra.json passes.

Test cost (pnpm test src/infra/owned-directory-observation.test.ts --maxWorkers=1): Vitest duration 6.48s on Bun and 9.78s on Node, about 9–13% of it tests and the rest mostly transform, measured on a heavily loaded Mac (load average about 38). On the idle 2-vCPU Linux box, this file plus directory-watch-host.test.ts took 2.1–3.0s per run, against 1.9–2.5s on main. No new timers, sleeps or subprocesses. CI seconds will follow from PR CI.

Not covered: the macOS helper-forwarding case in directory-watch-host.test.ts times out locally on this Mac while fseventsd is saturated (unchanged path; it passes 10/10 on Linux in the table above). Windows is unchanged by design: the admission check is fs.accessSync(R_OK), which is safe for directories there, and the native recursive watch already throws on Windows.

Review: Codex autoreview of the local diff (P0–P2) came back scoped-clean. Its caveat is accurate: with the stub, the sharing cases count registrations and don't exercise native recursive event delivery. They never did, because they already invoked the registered listener directly.

Node 24's Linux recursive fs.watch fallback inverts its throwIfNoEntry
check and swallows setup errors (nodejs/node#65635), so watching a
missing or unreadable root returns a watch that never reports instead of
throwing. observeOwnedDirectory only falls back to the nearest existing
ancestor on a synchronous throw, so a skills folder created after Gateway
start kept that inert handle and was only refreshed by the 30-minute
safety pass. watchDirectory now admits recursive roots before fs.watch,
matching the admission its macOS helper path already performs.

Node also builds Linux recursive watches from nested public fs.watch
calls, which made the owned-directory sharing tests count Node internals.
Recursive cases now stub the native handle the way Node 24 behaves, so
counts cover only the observer's registrations and the missing-ancestor
case fails on the original defect on every runtime.

Proof: Linux Testbox, Node 24.19.0 and 26.12.0, fresh process per run:
main 10/10 red, new tests with the old adapter 4/4 red, fix 10/10 green.
@clawsweeper

clawsweeper Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-20T05:27:51.188792Z 4f6eda8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@clawsweeper clawsweeper Bot added clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-repro Preliminary submission classification; full source review pending. P2 Normal backlog priority with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-repro Preliminary submission classification; full source review pending. labels Oct 20, 2026
@clawsweeper

clawsweeper Bot commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge.

What this changes

This PR checks recursive watch roots before registration and makes observer tests independent of Node’s internal watch calls.

Example: On Linux Node 24, create /home/alex/workspace/skills after the Gateway starts.

  • Before: An inert recursive watcher leaves skill discovery waiting for the 30-minute safety rescan.
  • After: The observer watches an existing ancestor and reattaches as the missing directory appears.

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) A useful, focused repair with no identified code defect; behavioral evidence remains test-level and exact-head CI failure attribution is unresolved.
Proof confidence 🦐 gold shrimp (3/6) The ordinary contributor proof gate is exempt under the supplied maintainer-authored status; reported observer tests and the raw Node probe support the premise but do not demonstrate Gateway skill refresh.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Product

Kind: Bug fix · Worth it: Yes · Fix scope: Complete
User problem: Skill directories created after startup can remain stale for up to 30 minutes on Linux Node 24.
Reason: This restores documented automatic discovery, and the supplied maintainer-authored status establishes direction approval.

Regression provenance

Possible regression — suspected (reviewed change). No predecessor PR is attributed.

Merge readiness

⛔ Needs changes before merge - 1 item remains

Keep open: this PR repairs a current-main watcher admission bug with a focused change, and review found no actionable code defect.

Priority: P2
Reviewed head: 4f6eda8752a3535d2130c31185d03e758908f945

Before merge

  • Complete next step - Inspect the failing lint and test-types diagnostics, repair any diff-caused failure, and resolve the required CI gate.

Findings

None.

Tests

  • Missing end-to-end proof: The supplied evidence does not show a real Linux Node 24 Gateway discovering and editing a skill root created after startup.
Agent review details

How this fits together

Skills discovery subscribes to the shared directory observer, which turns native filesystem events into invalidations for the existing discovery worker.

flowchart LR
  A[Skill directory changes] --> B[Native watch adapter]
  B --> C[Shared directory observer]
  C --> D[Skills refresh scheduler]
  D --> E[Discovery worker]
  E --> F[Published skill snapshot]
Loading

Technical review

Best possible solution:

Keep synchronous recursive-root admission in the existing adapter while preserving ancestor fallback, native invalidation, and safety reconciliation.

Do we have a high-confidence way to reproduce the issue?

The observer’s throw-dependent ancestor fallback and Node’s confirmed swallowed-error defect establish the failure path; no target code was executed during this review.

Is this the best way to solve the issue?

The patch repairs admission at the shared adapter, and the test changes isolate OpenClaw registrations without adding production scaffolding or duplicate policy.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against eb54f49633be.

Provenance checked

  • Recursive watch registration and ancestor fallback keeps the original intent (be0c50a: Use shared native observations instead of minute polling while preserving missing-root discovery and safety reconciliation.)
  • macOS helper dispatch keeps the original intent (fe7bb92: Move Node FSEvents handles out of the Gateway to avoid blocking shutdown; the helper branch remains intact.)
  • Observer test fixtures keeps the original intent (b7cbaf1: Preserve distinct observer contracts using captured callbacks across runtimes; the revised recursive fixture continues that approach.)

Testing

Proof path: unit tests only.

Security

None.

Evidence

What I checked:

PR surface

Source +6, Tests +21. Total +27 across 2 files. Added test files: 0.

View PR surface stats
Area Files Added Removed Net
Source 1 11 5 +6
Tests 1 24 3 +21
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 35 8 +27

Labels

Label changes:

  • add P2: Missing skill roots can remain undiscovered until safety reconciliation on Linux Node 24, with limited scope and an existing recovery path.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action.

Label justifications:

  • P2: Missing skill roots can remain undiscovered until safety reconciliation on Linux Node 24, with limited scope and an existing recovery path.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦐 gold shrimp and patch quality is 🐚 platinum hermit.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed October 20, 2026, 1:39 AM ET / 05:39 UTC.

@steipete
steipete merged commit 4150931 into main Oct 20, 2026
176 of 178 checks passed
@steipete
steipete deleted the steipete/jolly-lederberg-a770d2 branch October 20, 2026 18:29
@steipete

Copy link
Copy Markdown
Contributor Author

github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request Oct 21, 2026
…with Node 24 (openclaw#178095)

* fix(infra): admit missing recursive watch roots on Linux Node 24

Node 24's Linux recursive fs.watch fallback inverts its throwIfNoEntry
check and swallows setup errors (nodejs/node#65635), so watching a
missing or unreadable root returns a watch that never reports instead of
throwing. observeOwnedDirectory only falls back to the nearest existing
ancestor on a synchronous throw, so a skills folder created after Gateway
start kept that inert handle and was only refreshed by the 30-minute
safety pass. watchDirectory now admits recursive roots before fs.watch,
matching the admission its macOS helper path already performs.

Node also builds Linux recursive watches from nested public fs.watch
calls, which made the owned-directory sharing tests count Node internals.
Recursive cases now stub the native handle the way Node 24 behaves, so
counts cover only the observer's registrations and the missing-ancestor
case fails on the original defect on every runtime.

Proof: Linux Testbox, Node 24.19.0 and 26.12.0, fresh process per run:
main 10/10 red, new tests with the old adapter 4/4 red, fix 10/10 green.

* test(infra): capture the native fs.watch without an unbound method
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal backlog priority with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. size: XS status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant