feat: generate compile database without building by wellwei · Pull Request #387 · mcpp-community/mcpp · GitHub
Skip to content

feat: generate compile database without building - #387

Merged
Sunrisepeak merged 17 commits into
mcpp-community:mainfrom
wellwei:codex/configure-only-cdb
Aug 10, 2026
Merged

feat: generate compile database without building#387
Sunrisepeak merged 17 commits into
mcpp-community:mainfrom
wellwei:codex/configure-only-cdb

Conversation

@wellwei

@wellwei wellwei commented Aug 8, 2026

Copy link
Copy Markdown
Member

Summary

  • add mcpp build --configure-only, reusing the real build plan and existing build selectors while skipping ordinary compilation, linking, BMI cache population, and build-success cache writes
  • include tests/**/*.cpp, matching [build].flags, and test dev-dependencies in the generated CDB; stage only std and already-cached dependency BMIs needed by language tooling
  • publish compile_commands.json atomically across platforms, preserve last-known-good files and CDB symlinks, and keep normal build/test publication failures non-fatal
  • document the trust and side-effect boundary: configure-only may still execute build.mcpp, resolve/install dependencies or toolchains, and update lock/resolution metadata

This is the independently agreed A part from #379. It intentionally does not add JSON/NDJSON, ide subcommands, snapshots, mcpp.wire, invalidatedBy, or new selector semantics. It does not close the RFC.

Core impact

  • shared test target discovery is extracted from mcpp test; existing list/build/run behavior remains covered by unit and E2E tests
  • the Ninja backend gains an opt-in requireCompileDatabase flag; default build/test behavior remains warning-only
  • CDB publication changes from direct truncation to same-directory temporary write plus atomic replacement; concurrent writers remain documented last-writer-wins

Test plan

  • fresh self-hosted mcpp build --no-color --no-cache on macOS ARM64
  • fresh mcpp test --no-color: 70 passed, 0 failed
  • tests/e2e/202_configure_only_cdb.sh: syntax-error source, test/dev-dependency flags, workspace fan-out, -p, no build artifacts/cache, publication failure preservation
  • adjacent E2E: 01, 18, 35, 76, 77, 90, 157, 159
  • unit coverage for test discovery, configure prerequisites, atomic replacement, unchanged mtime, symlink preservation, unreadable/failed destinations

Refs #379
Refs mcpp-community/mcpp-vscode#5

Comment thread .agents/docs/2026-08-08-configure-only-cdb-implementation-plan.md
wellwei and others added 13 commits August 10, 2026 11:28
… replace

- publish_compile_commands: treat a missing CDB (symlink_status type ==
  not_found) as the normal first-build case instead of a fatal error.
  is_symlink sets ec on missing paths on every standard library (generic
  ENOENT on libstdc++/libc++, system ERROR_FILE_NOT_FOUND on MSVC), so no
  compile_commands.json was ever written on fresh workspaces and all
  CDB-dependent e2e tests failed.
- close the existing-CDB ifstream before the atomic replace so Windows
  MoveFileExW can replace the destination (open handles cause Access
  denied / sharing violation).
- unit tests: escape Windows backslash paths in the entry() JSON helper,
  write the mtime fixture in binary mode so on-disk bytes match on
  Windows, and add a regression test for publishing with no prior CDB.
Windows CDBs store file paths with backslashes, so `grep 'src/main.cpp'`
never matched and the test failed on the Windows e2e shard once the CDB
was actually being generated. Match either separator with `[\/]`.
Windows CDBs store paths with JSON-escaped backslashes (src\\main.cpp),
so one-separator patterns still missed. Allow one or more separators.
@wellwei
wellwei force-pushed the codex/configure-only-cdb branch from 9e6c8f3 to c35f1b4 Compare August 10, 2026 06:03
wellwei and others added 3 commits August 10, 2026 14:23
…g at its graph

The backend writes build.ninja before it honors dryRun, so `--configure-only`
rewrites the very file the P0 fast path replays — and a configure plan's graph
is not a normal build's graph: it carries the test targets and dev-dependencies,
so its `default` line names the TEST binaries and does not contain the package's
own target at all.

`target/.build_cache` was left untouched, and the fast path decides freshness by
comparing build.ninja's mtime against the SOURCES, never against the graph. So on
any already-built project:

    mcpp build                  # default bin/app
    mcpp build --configure-only # default bin/smoke   <- same build.ninja
    mcpp build                  # fast path -> links bin/smoke, never bin/app,
                                #              prints `Finished dev in 0.04s`

which is the routine case, not a corner one: the whole point of the flag is that
an editor runs it continuously.

Drop the fast-path entry for the build dir whose graph was rewritten, before the
backend runs so a failed configure cannot leave the stale claim standing either.
Scoped to that one outputDir: other (target, profile, cache mode) triples own
different build dirs and keep their entries.

`write_build_cache`'s serializer is split out so the invalidation path rewrites
the file through the same spelling rather than a second one.

The e2e must NOT delete the built binary before the second build — a missing
output makes ninja fail in a way the fast path reads as a stale graph and falls
back to a full prepare, which hides the defect. Verified red (links `smoke`)
before the fix and green after.
@Sunrisepeak

Copy link
Copy Markdown
Member

…uced

Windows links `fastpath.exe`; the new fast-path section hardcoded the POSIX
spelling and failed at the baseline check before reaching the assertion it
exists for (e2e 2/2 windows). The suffix is a host constant, so take it off the
artifact `find` actually returned rather than branching on the platform.
@Sunrisepeak
Sunrisepeak merged commit 3f237ed into mcpp-community:main Aug 10, 2026
18 checks passed
Sunrisepeak added a commit that referenced this pull request Aug 10, 2026
…#408)

图形栈不可用不是一个 bug,是三层各自独立的故障。追到代码之后,其中两层的既有结论
是错的 —— 按它们去修,修完还是坏的。

## 三条被测量推翻的结论

1. #405 的 issue 根因是错的(我自己写的那条,而且是这个 issue 上第二次)。
   它说「谓词漏判 ⇒ 不发 std 的 stage 边」。生成物否掉了它:边就在 build.ninja 第 65 行。
   `scan_packages` 的 packages 本来就含依赖包根,谓词恒为真。真因是**没有任何边依赖
   那条边**,ninja 于是从不执行它。按 issue 里的修法(entry.json 记 imports_std)
   作用在一个已经为真的谓词上 —— 改完仍然坏。

2. mcpp-index 上 8 个图形成员全红不是数据缺陷。`xim:libglvnd@>=1.7.0.1` 在
   xlings 2026.8.9.2 起解析正常(四段版本 semver 重写),红的原因是 index CI 钉在
   mcpp 2026.8.8.2,它内带 xlings 2026.8.8.1 —— 正好落在修复之前。
   xim-pkgindex 一个字都不用改。

3. 图形拿不到 GPU 的直接原因在 mcpp 自己的链接命令行里:全仓没有一处
   `--disable-new-dtags`,唯一相关的一处显式写了 `--enable-new-dtags`。

## 修复

- **#405** 缓存命中时,被恢复的包传递依赖的 std BMI 没有消费者。修法把它放进
  `_mcpp_staged_cache` —— 那个聚合本来就是为「stage 边丢掉编译边携带的次序」建的,
  std BMI 是同一缺陷早一条边。不动 cache key / entry schema,现有缓存全部有效。
  缓存 miss 时依赖在本地编译、把 std 边带进图,所以第一个构建它的人永远是好的、
  之后每个人都坏 —— 这就是它伪装成升级回归的方式。

- **#407** 三种模式写同一个 build.ninja,快路径只比源码 mtime。改成让图自己声明形态
  (`# mcpp:graph=normal|test`),快路径校验它即将重放的那张图。**读取侧不变式**,
  并同时删掉 #387 留下的写入侧修补 —— 写入侧要求每个未来的图重写者都记得调用,
  这正是 `mcpp test` 那半边在 `--configure-only` 修好之后仍然坏着的原因。

## 新增

- **加载器标签契约**(`mcpp.build.loader_contract`):可执行 DT_RPATH、库 DT_RUNPATH。
  DT_RUNPATH 只对携带它的对象**自己发起**的 dlopen 生效,而图形程序到驱动的三到四层
  dlopen 都不是它发起的 —— 是 libGLX.so.0 代发的。所以决定能否上 GPU 的是标签不是路径。
  反过来在库上强制 RPATH 有害(打断 eglInitialize),所以一分为二。链接期与 pack 的
  patchelf 期读同一条契约。这是 xlings 图形栈设计里 E2(构建侧)的 mcpp 那一半。
- **rule E**:标签校验落在产物上,写进 resolution.json 的 `loader_tags`,warn-first。
  记录而不只是告警 —— 只在沉默中通过的检查,和根本没跑的检查,输出完全相同。
- **pack 不再残留构建机路径**:此前只重写主二进制,bundle 进来的每个 .so 都保留着
  指向构建机 xlings store 的绝对 RUNPATH。「依赖 xlings 生态」是设计选择,
  「依赖这一台机器的这一份 store」是缺陷,而且在构建它的机器上跑得好好的。
  **不碰动态加载器** —— 它不是被搜索的库,它是执行搜索的程序;patchelf 改它会让
  self-contained 档在 main 之前段错误(30_pack_modes 抓到的)。
- **HOST-REQUIREMENTS**:自包含有底。驱动只能来自目标机器(与内核模块锁步 + 禁止
  再分发),所以诚实产出是 bundle 加一份声明。带 discovery 列,因为几种发现机制
  互不通用。
- **自带 libc 的档硬拒宿主能力**:self-contained 与 static 在 plan 期失败并指出改用
  vendored。两者坏在同一件事上(#392/#401 的两个方向),此前都链得过去然后运行时崩。
- **`[[runtime.requirements]] discovery`**:声明式,mcpp 绝不推断 —— 从能力名推断
  就是把 provider 专属知识写进 mcpp,`test_runtime_contract` 正是为此设的门。
- **artifact 身份判决**:resolution.json 每个 artifact 带 `identity`
  (ok/mismatch/missing/unverified),跟随符号链接。这是 mcpp 已经对私有 libc 执行的
  规则的推广,纯路径事实。`why runtime` 的 `(none declared)` 改为
  `(not declared by the environment — nothing to verify)`:有名无物是未验证,不是通过。

## 变更

- 内带 xlings 升到 2026.8.10.4(16 个 pin 由 check_version_pins.sh 机器校验)。

## 本地验证

- unit 77/77 通过(含新增 test_loader_contract:契约、图形态、宿主要求、身份判决)。
- e2e 183 通过 / 25 失败 / 8 跳过。**25 条里 24 条用已发布的 2026.8.8.2 逐条复现**,
  同一个环境根因:本机共享 gcc 载荷的 specs 被历史安装污染 —— `--dynamic-linker`
  指向已被改名的 glibc/2.44,rpath 里还有约 40 条来自已删除沙箱的 /tmp/tmp.* 条目。
  第 25 条(30_pack_modes)是真回归,已定位并修复(不得 patchelf 动态加载器)。
- 每条新 e2e 都先证伪过:撤掉对应修改必须变红,并已实测。

Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants