From d280170b625fb6f19f18f88827577e41f88e514d Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 19:27:58 +0800 Subject: [PATCH 1/5] 2026.9.29.5: a workspace's build programs are reused across selections and run dependencies first, the build database and --configure-only plan by configuration, and the output names what it reports The validation project's post-release run of 2026.9.29.4 showed five defects in the commands around the workspace build (design document section 17.1). - D1: a member program's graph document listed every requester in the plan, the virtual root included, so the program's re-run key followed the selection and -p, pack and emit reran the programs a --workspace build ran. The document lists the requests made inside the program's closure (graph_package_entry takes the closure's packages). - The members' programs run dependencies first (depth-first over the plan's requests; a cycle skips its closing edge); they ran in discovery order. - D2: `mcpp emit build-database` and `mcpp build --configure-only` plan a workspace by configuration, each member's tests included (BuildOverrides::member_targets), so a package the members share is described once per configuration; members that are programs and their tests are described as such. A configuration whose plan fails is planned member by member, so a member's failure affects that member only. Set names carry a configuration prefix only in a document of several configurations; a set and a compile command (file and output) are described once (SPEC-005 v1.6). - A command that plans several configurations publishes the root compile_commands.json once, as the union of their databases; each configuration replaced it, a race under concurrent groups. - D3: the build program status lines name the package. - D4: a selected member is announced by its directory in a --workspace build. - D5: `mcpp pack` reports more than eight outputs by the entry each lies in below their common directory, with a count; --verbose and --message-format json name every output. - e2e 839, 840, 841; docs/07, docs/10, docs/30 in both languages; SPEC-005 v1.6; version 2026.9.29.5. --- ...2026-09-29-workspace-build-graph-design.md | 4 + CHANGELOG.md | 46 ++++ docs/07-workspace.md | 8 + docs/10-pack-and-release.md | 6 + docs/30-build-mcpp.md | 10 +- docs/40-baremetal.md | 4 +- docs/specs/README.md | 2 +- docs/specs/build-database.md | 35 +-- docs/zh/07-workspace.md | 6 + docs/zh/10-pack-and-release.md | 4 + docs/zh/30-build-mcpp.md | 7 +- docs/zh/40-baremetal.md | 4 +- mcpp.toml | 2 +- modules/versioning/src/version.cppm | 2 +- src/build/build_database.cppm | 47 +++- src/build/build_program.cppm | 11 +- src/build/compile_commands.cppm | 78 ++++-- src/build/execute.cppm | 50 +++- src/build/plan.cppm | 5 + src/build/prepare.cppm | 4 + src/build/prepare/graph.cpp | 6 +- src/build/prepare/manifest.cpp | 2 + src/build/prepare/plan.cpp | 3 +- src/build/prepare/records.cpp | 2 +- src/build/prepare/state.cppm | 7 +- src/build/prepare/target_side.cpp | 32 ++- src/cli/cmd_build.cppm | 235 ++++++++++++++---- src/pack/pipeline.cppm | 45 +++- tests/e2e/139_build_program_advisory.sh | 2 +- .../e2e/186_build_mcpp_protocol_and_bound.sh | 2 +- ...build_program_deploys_what_it_generated.sh | 2 +- ...d_programs_are_named_ordered_and_cached.sh | 92 +++++++ ...ce_database_describes_each_package_once.sh | 126 ++++++++++ tests/e2e/841_pack_summarizes_many_outputs.sh | 82 ++++++ 34 files changed, 858 insertions(+), 115 deletions(-) create mode 100644 tests/e2e/839_a_workspaces_build_programs_are_named_ordered_and_cached.sh create mode 100644 tests/e2e/840_a_workspace_database_describes_each_package_once.sh create mode 100644 tests/e2e/841_pack_summarizes_many_outputs.sh diff --git a/.agents/docs/2026-09-29-workspace-build-graph-design.md b/.agents/docs/2026-09-29-workspace-build-graph-design.md index 37b70ddfd..68157d042 100644 --- a/.agents/docs/2026-09-29-workspace-build-graph-design.md +++ b/.agents/docs/2026-09-29-workspace-build-graph-design.md @@ -563,6 +563,10 @@ read from the member, or made a value of the plan: | the runtime files of a program shipped through `artifacts` (2026.9.29.3) | its link waited for the plan's deploy set, which a workspace plan does not place; its own runtime files were not in the member's directory | the link waits for no plan-level file; a member's runtime set includes the closures its `artifacts` edges reach | e2e 833 G9 | | the link line of a program shipped through `artifacts` (2026.9.29.4) | the plan's line, which pools the dependencies' flags and not a member's, so a library its package's build program states was missing | a link group of its own closure that places nothing (`LinkGroup::linkOnly`) | e2e 838 | | `${mcpp.bin_dir}` in a member's action (2026.9.29.4) | the plan's `bin/` | the declaring member's product directory | e2e 838 A4 | +| a member program's graph document (2026.9.29.5) | listed every requester in the plan, the virtual root included, so the program's re-run key followed the selection | the requests made inside the program's closure | e2e 839 B3, B4 | +| the order of the members' programs (2026.9.29.5) | discovery order | dependencies first (a cycle skips its closing edge) | e2e 839 B2 | +| `emit build-database`, `--configure-only` (2026.9.29.5) | one plan per member | one plan per configuration, each member's tests included; a failed configuration planned member by member | e2e 840 | +| the root compile database of several configurations (2026.9.29.5) | the last written configuration's, a race under concurrent groups | the union, published once by the command | e2e 840 B | Each criterion fails on 2026.9.29.1 and passes on 2026.9.29.2. The resource case also showed a defect of every build: a quoted `#include` in a script was diff --git a/CHANGELOG.md b/CHANGELOG.md index fd0f0abf2..c7c493ea7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,52 @@ > Each `## []` section is that release's notes. Entries are written in English > from 2026.9.28.3 on; earlier entries remain as written. +## [2026.9.29.5] - 2026-09-29 + +This release completes the workspace build graph in the commands around the +build, from the validation project's post-release run of 2026.9.29.4: build +programs are reused across selections and run dependencies first, the build +database and `--configure-only` plan by configuration, and the output names +what it reports. + +### Fixed + +- **A member's build program is reused whichever members a command selects.** + Its graph document listed every requester in the plan, the virtual root + included, so the program's re-run key followed the selection: `-p`, `mcpp + pack` and `mcpp emit build-database` reran the programs a `--workspace` + build had run (7 to 15 s each in the validation project). A program's + document now lists the requests made inside its own closure (e2e 839). +- **A member's build program runs after those of the members it depends on.** + They ran in discovery order, so a member's program could run before its + dependency's had applied its directives (e2e 839). +- **`mcpp emit build-database` and `mcpp build --configure-only` plan a + workspace by configuration, as the build does.** They planned each member + separately, so a package two members use was described once per member, + each time with other arguments (the validation project's core library three + times). A member that is a program is described as one, and its tests as + tests. A configuration whose plan fails is planned member by member, so a + member's failure still affects that member only (e2e 840; SPEC-005 v1.6). +- **A command that plans several configurations writes the root + `compile_commands.json` once**, as the union of their databases. Each + configuration replaced it, and under `mcpp build --workspace`, whose + configurations build at the same time, the file was the last one's (e2e 840). + +### Behaviour changes + +- **The build program status lines name the package**: + `build.mcpp compiling `, `running `, + `up to date (cached)` (e2e 839). +- **A selected member is announced by its directory** in a `--workspace` + build, also where another member depends on it (e2e 839). +- **`mcpp pack` summarises many outputs.** A format that reports more than + eight outputs is reported by the entry each lies in below their common + directory, with a count; `--verbose` names every output, and + `--message-format json` lists every one as before (e2e 841). +- **Build database set names (SPEC-005 v1.6).** A set is named by its package; + a document of several configurations prefixes each name with the + configuration's build directory name, instead of `/`. + ## [2026.9.29.4] - 2026-09-29 This release links a program that a workspace member ships through diff --git a/docs/07-workspace.md b/docs/07-workspace.md index 1e851845a..28fe06a28 100644 --- a/docs/07-workspace.md +++ b/docs/07-workspace.md @@ -441,6 +441,14 @@ member that several members use is compiled once. - **Resources.** A member's `[resources]` and `windows_code_page` are compiled against the member's directory and include directories and embedded into that member's programs and shared libraries only (2026.9.29.2+). +- **Build programs.** The members' build programs run dependencies first, and + a program's result is reused by every command whose inputs to it are + unchanged, whichever members the command selects (2026.9.29.5+). +- **Compile database.** `mcpp build --configure-only` and `mcpp emit + build-database` plan as the build does, one plan per configuration with each + member's tests, so a package the members share is described once per + configuration. A command that planned several configurations writes the root + `compile_commands.json` once, as the union of their databases (2026.9.29.5+). - **No-op builds.** A command repeated with nothing changed is answered by one check per configuration, without planning. - **Module names.** Members built in one graph share one module namespace: diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 313217532..56b76134c 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -265,6 +265,12 @@ the last is the distributable and only it is printed as `Packed`. A format whose chain ends in two files is refused by `mcpp run --format`, naming both, because a runner takes one operand. +A format may report many outputs, one per file of a distribution tree. Up to +eight are printed one per line; more are printed by the entry each lies in +below their common directory, with a count (`Packed Release/app (1309 +files)`), and `--verbose` prints every one (2026.9.29.5+). +`--message-format json` lists every output in either case. + An unknown `` is refused naming the format set the resolved graph provides, the same set `mcpp pack --format bogus` reports. `--format` together with `--no-runner` is refused — an `.apk` or an installed `.app` cannot be diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 1fab5940a..511cf96dc 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -1045,6 +1045,11 @@ one. `mcpp::graph_file()` names a JSON document that states the resolved graph: `""`.** The root decides the graph, and when its program runs every input of that decision is final, which is the reason `dep_linkage` is offered to it alone. +- **In a workspace plan each selected member's program receives the document + of its own closure**, in which it is `root`, and `requested_by` lists the + requests made inside that closure (2026.9.29.5+). The document, and so the + program's re-run key, is therefore the same whichever members a command + selects. The members' programs run dependencies first. - **`[package.metadata.]` is the package's statement about itself.** The engine does not interpret the table. A path in it is resolved by the reader against the entry's `manifest_dir`, because only the reader knows which values @@ -1513,8 +1518,9 @@ variable, emit `mcpp:rerun-if-changed=config.h` / `mcpp:rerun-if-env-changed=USE This replaces the old "process exited 0, so assume it's fine" guesswork with an explicit input/output contract — incremental builds stay correct. -When nothing changed the output is `build.mcpp up to date (cached)`; otherwise -`build.mcpp compiling` / `running`. +When nothing changed the output is `build.mcpp up to date (cached)`; +otherwise `build.mcpp compiling ` / `running ` (the package +is named from 2026.9.29.5 on). ## Host tools from a dependency (mcpp 2026.8.5.1+) diff --git a/docs/40-baremetal.md b/docs/40-baremetal.md index aa2c38c6c..334597ba0 100644 --- a/docs/40-baremetal.md +++ b/docs/40-baremetal.md @@ -206,8 +206,8 @@ Measured output: Resolving toolchain Resolved llvm@22.1.8 → riscv64-none-elf → @mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/bin/clang++ Resolved host toolchain for build.mcpp: clang 22.1.8 (x86_64-unknown-linux-gnu) - build.mcpp compiling - build.mcpp running + build.mcpp compiling blinky + build.mcpp running blinky Inferred sources [src/**/*.{cppm,cpp,cc,c,S,s,asm}] Inferred target blinky (bin from src/main.cpp) Compiling blinky v0.1.0 (.) diff --git a/docs/specs/README.md b/docs/specs/README.md index f74dd422c..0647d87e4 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -34,7 +34,7 @@ | [SPEC-002](target-side.md) | 目标侧模型与能力声明(`mcpp:` 保留命名空间、五层、三条规则) | 评审中 v1.0 | 2026-08-24 | mcpp >= 2026.8.24.2 | | [SPEC-003](exit-codes.md) | 退出码契约(分类、语义、稳定性承诺) | 评审中 v1.0 | 2026-09-01 | mcpp >= 2026.9.1.1 | | [SPEC-004](manifest-semantics.md) | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | 草案 v1.10 | 2026-09-28 | 条件化形状 mcpp >= 2026.8.29.1;目标轴 mcpp >= 2026.9.6.4;`linkage` 默认值 mcpp >= 2026.9.15.2;链接 flag 的词读法 mcpp >= 2026.9.26.2;条件化的 `dialect_cxxflags` 与 `-p` 的包身份 mcpp >= 2026.9.28.1;条件表按具体程度生效 mcpp >= 2026.9.28.2 | -| [SPEC-005](build-database.md) | 构建数据库:`mcpp emit build-database` 的内容、取值规则与不写工程目录的保证 | 评审中 v1.5 | 2026-09-28 | mcpp >= 2026.9.15.1;v1.3 条款 mcpp >= 2026.9.26.2;v1.4 条款 mcpp >= 2026.9.27.1;v1.5 条款 mcpp >= 2026.9.28.1 | +| [SPEC-005](build-database.md) | 构建数据库:`mcpp emit build-database` 的内容、取值规则与不写工程目录的保证 | 评审中 v1.6 | 2026-09-29 | mcpp >= 2026.9.15.1;v1.3 条款 mcpp >= 2026.9.26.2;v1.4 条款 mcpp >= 2026.9.27.1;v1.5 条款 mcpp >= 2026.9.28.1;v1.6 条款 mcpp >= 2026.9.29.5 | | [SPEC-006](toolchain-management.md) | 工具链管理:身份、来源、选择与载荷契约 | 草案 v0.4 | 2026-09-28 | 逐条标注;已实现条款 mcpp >= 2026.9.24.1;§3.7 mcpp >= 2026.9.28.1;§3.7.1 mcpp >= 2026.9.28.2 | | [SPEC-007](build-plugins.md) | 构建插件:配置、施工与校验的分工,运行时与规划期的义务 | 草案 v0.6 | 2026-09-28 | 逐条标注;mcpp >= 2026.9.26.2;v0.3 条款 mcpp >= 2026.9.27.1;v0.4 条款 mcpp >= 2026.9.28.1;v0.5 条款 mcpp >= 2026.9.28.2;v0.6(§9)mcpp >= 2026.9.28.3 | | [SPEC-008](library-interface.md) | 库的接口:公开模块、发布闭包与两种形态的一致 | 草案 v0.1 | 2026-09-28 | 第一阶段(只警告)mcpp >= 2026.9.28.3 | diff --git a/docs/specs/build-database.md b/docs/specs/build-database.md index 3988e7b5c..d6234f24a 100644 --- a/docs/specs/build-database.md +++ b/docs/specs/build-database.md @@ -4,10 +4,10 @@ |---|---| | 规范编号 | SPEC-005 | | 标题 | mcpp 输出的构建数据库:内容、取值规则与不写工程目录的保证 | -| 状态 | 评审中 v1.5 | -| 版本 | 1.5 | -| 最后修改 | 2026-09-28 | -| 对应实现 | mcpp >= 2026.9.15.1;v1.3 修改的 R2.5、R3.7、R3.8、R4.1、R5.2 为 mcpp >= 2026.9.26.2;v1.4 修改的 R2.5 为 mcpp >= 2026.9.27.1;v1.5 修改的 R3.7、R3.12、R5.1、R5.2 为 mcpp >= 2026.9.28.1 | +| 状态 | 评审中 v1.6 | +| 版本 | 1.6 | +| 最后修改 | 2026-09-29 | +| 对应实现 | mcpp >= 2026.9.15.1;v1.3 修改的 R2.5、R3.7、R3.8、R4.1、R5.2 为 mcpp >= 2026.9.26.2;v1.4 修改的 R2.5 为 mcpp >= 2026.9.27.1;v1.5 修改的 R3.7、R3.12、R5.1、R5.2 为 mcpp >= 2026.9.28.1;v1.6 修改的 R2.1、R3.3、R3.4、R3.5、R4.1、R5.2 为 mcpp >= 2026.9.29.5 | | 相关设计文档 | `.agents/docs/2026-09-14-636-build-database-and-the-latest-xlings.md`
`.agents/docs/2026-09-26-compile-database-and-issue-699-design.md` | | 相关 issue | #636, #648, #655, #699, #702, #707 | | 依据的外部规范 | S1「C++ Build Database: IDE Profile」profile 0.3.0(§7.2 的 `generated`,Sunrisepeak/mcpp-language-server#28;此前为 0.2.0)与 S2 0.2.0 §3.4,取自 https://github.com/Sunrisepeak/lsp-mcpp-private 提交 `b82859d`(schema 自提交 `28ecd6e` 起未变);S2 0.3.0 §3.4 的部分回答(S2-3.4-12、S2-3.4-13,Sunrisepeak/mcpp-language-server#25);JSON Compilation Database | @@ -42,7 +42,7 @@ Database 定义,本规范不重复它们的字段定义,只规定 mcpp 作为生 ## 2. 不写工程目录 - **R2.1** 命令**禁止**写入工程目录,即根包、工作区成员与 path 依赖的源码树。规划 - 写入 `$MCPP_HOME/cache/build-database/`,`` 由工程根与成员决定。该目录 + 写入 `$MCPP_HOME/cache/build-database/`,`` 由工程根与一次规划的成员决定。该目录 是缓存,可以随时删除。**已实现** - **R2.2** 命令不编译:标准库模块被描述而不被编译,也不生成只供链接使用的输入(GCC 的 `mcpp-clean-link.specs`)。工具链照常被查询(版本、目标三元组、sysroot 等),与 @@ -84,14 +84,18 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 ### 3.2 集合 -- **R3.3** 每个包一个集合,名为包的限定名(`.` 或 ``)。根包 - 测试目标的源文件归入集合 `<包>:test`,标准库模块的单元归入集合 `mcpp:std`。工作区 - 文档中,每个集合名带前缀 `<成员>/`。**已实现** -- **R3.4** `visible-sets` 列出同一成员的其余所有集合。引擎在一次调用的一张模块图上 +- **R3.3** 每个包一个集合,名为包的限定名(`.` 或 ``)。根包与 + 工作区被选成员的测试目标的源文件归入集合 `<包>:test`,标准库模块的单元归入集合 + `mcpp:std`。工作区按配置规划,与 `mcpp build` 相同:每个配置一次规划,成员共用的包 + 在一个配置中只有一个集合。文档描述多个配置时,每个集合名带前缀 `<配置>/`,`<配置>` + 为该配置构建目录的名字;只有一个配置时不带前缀。两次规划描述同一配置的同一集合时 + (见 R5.2 的逐成员规划),该集合只出现一次。**已实现** +- **R3.4** `visible-sets` 列出同一次规划的其余所有集合。引擎在一次调用的一张模块图上 解析 import,更窄的闭包会描述一条构建并不执行的规则。**已实现** - **R3.5** `family-name` 为包名,`mcpp:std` 集合的为 `mcpp:std`;`ide.configuration` - 为 profile 名;`ide.kind` 在测试集合为 `test`,在根包集合按其目标为 `library`、 - `executable` 或 `other`,在依赖包集合与 `mcpp:std` 为 `library`。**已实现** + 为 profile 名;`ide.kind` 在测试集合为 `test`,在根包与工作区被选成员的集合按其 + 目标为 `library`、`executable` 或 `other`,在依赖包集合与 `mcpp:std` 为 `library`。 + **已实现** - **R3.6** 单元的 `arguments` 依次是驱动、集合的 `baseline-arguments`、单元的 `local-arguments`,以及单元自己结尾的 `-c -o `(若有;两个操作数 相对 `work-directory` 指向 `source` 与 `object`)。`baseline-arguments` 是集合中每个 @@ -170,7 +174,8 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 ## 4. `--spec compile-commands` - **R4.1** 文档为 `mcpp build --configure-only` 在同一组选择器下写入 - `compile_commands.json` 的条目,差别只在输出路径位于 §2 的工作目录之下。标准库 + `compile_commands.json` 的条目,差别只在输出路径位于 §2 的工作目录之下;同一文件与 + 同一输出只有一条条目,文档描述多个配置时,一个文件在每个编译它的配置中各有一条。标准库 模块的单元也在其中,遵循 S1-12-1 的导出规则:S1 文档里 `mcpp:std` 集合的每个 单元同样导出为一条 `compile_commands.json` 条目。**已实现** @@ -179,8 +184,9 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 - **R5.1** `kind` 为 `mcpp.build-database`,`kindVersion` 为 1。`data` 含 `spec` (`{"name": "s1", "version": "0.3.0"}` 或 `{"name": "compile-commands"}`)、 `database`、`watch` 与 `inputs-fingerprint`。**已实现** -- **R5.2** 命令独立规划每一个被选中的成员:一个成员规划失败只影响它自己,不影响 - 其余成员的集合(#699 第 1 项)。规划失败的成员不贡献任何集合,只贡献一条 `error` +- **R5.2** 一个成员规划失败只影响它自己,不影响其余成员的集合(#699 第 1 项)。命令 + 按配置规划被选中的成员;一个配置的规划失败时,该配置的成员逐个规划,因此失败仍归于 + 各自的成员。规划失败的成员不贡献任何集合,只贡献一条 `error` 诊断,`path` 为该成员的 `mcpp.toml`,相对工作区根目录;诊断码为:不在工程中时 `MCPP_BUILD_DATABASE_NO_PROJECT`;该成员的规划因离线而需要下载时 `MCPP_OFFLINE_DOWNLOAD_REQUIRED`,消息指出需要下载的第一项;其他规划失败为 @@ -226,3 +232,4 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 | 1.3 | 2026-09-26 | R2.5:`emit` 下构建失败的宿主工具是警告。R3.7:`work-directory` 是输出目录,模块接口单元的 `arguments` 带语言 flag。R3.8:标准库单元的 `provides` 指向 std 缓存中的 BMI,工具链带 `build-id`。R4.1:compile-commands 文档包含标准库单元(S1-12-1)。R5.2:成员各自规划,构建程序失败的包不带其指令地被描述(#699,#702)。 | | 1.4 | 2026-09-26 | R2.5:命令不构建宿主工具;工具库中没有的工具被推迟,输出说明 `MCPP_BUILD_DATABASE_HOST_TOOL_DEFERRED`,取代 1.3 的警告 `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`(#707)。 | | 1.5 | 2026-09-28 | R3.7:规则声明的设备源不是编译单元,不进入 S1 与 `compile_commands.json`(#724)。新增 R3.12:集合的 `ide.generated` 列出规则生成的文件与目录,给出构建写入的路径与生成它的步骤,S1 0.3.0(#724,Sunrisepeak/mcpp-language-server#28)。R5.1:S1 版本为 0.3.0。R5.2:以构建程序的指令为前提的检查不对其构建程序已失败的包运行,失败路径保留已记录的说明(#724)。 | +| 1.6 | 2026-09-29 | 工作区按配置规划,与 `mcpp build` 相同(R2.1、R3.3、R3.4、R5.2):成员共用的包在一个配置中只描述一次;集合名的前缀由 `<成员>/` 改为只在文档描述多个配置时出现的 `<配置>/`;一个配置的规划失败时逐成员规划。R3.5:被选成员的集合按其目标给出 `ide.kind`。R4.1:同一文件与输出一条条目。 | diff --git a/docs/zh/07-workspace.md b/docs/zh/07-workspace.md index ad7a6b34d..df23764dc 100644 --- a/docs/zh/07-workspace.md +++ b/docs/zh/07-workspace.md @@ -407,6 +407,12 @@ mcpp test --workspace --workspace-timeout 1800 # whole fan-out (default 0 = no - **Hooks。** 每个被选成员的 `[hooks]` 按成员顺序在构建前后运行。 - **资源。** 成员的 `[resources]` 与 `windows_code_page` 按该成员的目录与 include 目录编译， 只嵌入该成员自己的程序与共享库（2026.9.29.2+）。 +- **构建程序。** 成员的构建程序按依赖在前的顺序运行；只要程序的输入不变，无论命令选中哪些 + 成员，程序的结果都被复用（2026.9.29.5+）。 +- **编译数据库。** `mcpp build --configure-only` 与 `mcpp emit build-database` 按构建的方式 + 规划，每个配置一次规划并包含各成员的测试，因此成员共用的包在每个配置中只描述一次。规划了 + 多个配置的命令只写一次根目录的 `compile_commands.json`，内容为各配置数据库的并集 + （2026.9.29.5+）。 - **无事可做的构建。** 在没有任何改动时重复执行的命令，每个配置只做一次检查，不重新规划。 - **模块名。** 在同一张图中构建的成员共享一个模块命名空间：两个成员各自提供同名模块时，不能在 同一条 `--workspace` 命令中构建；分别用 `-p` 构建。 diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 15bc883e8..b5714a3fa 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -240,6 +240,10 @@ mcpp run --target aarch64-ios-sim --format app 最后一个是发布物，也只有它以 `Packed` 报出。链的末端产出两个文件的格式会被 `mcpp run --format` 拒绝并点名两者，因为一个 runner 只接受一个操作数。 +一个格式可能报出很多输出，发布目录中的每个文件各一个。不超过八个时逐行打印；更多时按它们 +在共同目录之下所处的条目打印，并附数量（`Packed Release/app (1309 files)`），`--verbose` +打印全部（2026.9.29.5+）。`--message-format json` 在两种情况下都列出全部输出。 + 未知的 `` 会被拒绝，点名已解析图提供的格式集合，与 `mcpp pack --format bogus` 报出的是同一个集合。`--format` 与 `--no-runner` 同时出现会被拒绝 —— 一个 `.apk` 或一个已安装的 `.app` 无法被直接执行。在 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index 01480c411..21fd6bb1e 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -885,6 +885,9 @@ mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件，永不 - **根包的程序拿到它；依赖包的程序读到 `""`。** 根包决定这张图，而它的程序运行时这个决定的 每个输入都已确定 —— 这正是 `dep_linkage` 只提供给根包的原因。 +- **在工作区计划中，每个被选成员的程序拿到它自己闭包的文档**，其中它是 `root`， + `requested_by` 只列出闭包内部发出的请求（2026.9.29.5+）。因此无论命令选中哪些成员， + 这份文档以及程序的重跑键都相同。成员的程序按依赖在前的顺序运行。 - **`[package.metadata.]` 是包对自身的陈述。** 引擎不解释这张表。其中的路径由读取方 相对于该条目的 `manifest_dir` 解析，因为只有读取方知道哪些值是路径。旧引擎忽略这张表， 所以已发布的包可以在其使用方升级之前就写上它。 @@ -1273,8 +1276,8 @@ mcpp **不会**每次构建都重跑 `build.mcpp`。它会缓存程序产出的 `mcpp:rerun-if-changed=config.h` / `mcpp:rerun-if-env-changed=USE_FAST`。这用一份明确的 输入/输出契约取代了过去「进程退出码为 0 就当成功」的猜测——让增量构建保持正确。 -无变化时输出 `build.mcpp up to date (cached)`；否则是 `build.mcpp compiling` / -`running`。 +无变化时输出 `build.mcpp up to date <包名> (cached)`；否则是 `build.mcpp compiling <包名>` / +`running <包名>`(自 2026.9.29.5 起写出包名)。 ## 依赖产出的 host 工具（mcpp 2026.8.5.1+） diff --git a/docs/zh/40-baremetal.md b/docs/zh/40-baremetal.md index fdf204c5b..d478ba914 100644 --- a/docs/zh/40-baremetal.md +++ b/docs/zh/40-baremetal.md @@ -176,8 +176,8 @@ mcpp run Resolving toolchain Resolved llvm@22.1.8 → riscv64-none-elf → @mcpp/registry/data/xpkgs/xim-x-llvm/22.1.8/bin/clang++ Resolved host toolchain for build.mcpp: clang 22.1.8 (x86_64-unknown-linux-gnu) - build.mcpp compiling - build.mcpp running + build.mcpp compiling blinky + build.mcpp running blinky Inferred sources [src/**/*.{cppm,cpp,cc,c,S,s,asm}] Inferred target blinky (bin from src/main.cpp) Compiling blinky v0.1.0 (.) diff --git a/mcpp.toml b/mcpp.toml index 87c738f51..6fad06c2d 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.29.4" +version = "2026.9.29.5" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 42a160128..588e692da 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.29.4"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.29.5"; } // namespace mcpp diff --git a/src/build/build_database.cppm b/src/build/build_database.cppm index ffdad9b56..4ebf4d6b0 100644 --- a/src/build/build_database.cppm +++ b/src/build/build_database.cppm @@ -12,7 +12,7 @@ // - `arguments` come from the record the compile database renders // (mcpp.build.compile_commands::unit_invocations), so the two databases // cannot list different command lines for one unit. -// - `visible-sets` is every other set of the planned member. The engine +// - `visible-sets` is every other set of the same plan. The engine // resolves imports over one graph per invocation; a narrower closure would // describe a rule the build does not enforce. // - The standard library modules are translation units of their own set, @@ -54,10 +54,10 @@ struct Member { // Where the planning pass wrote (BuildOverrides::work_dir): the build // programs' caches, and so their declared inputs, are under it. std::filesystem::path workDir; - // Test discovery (`[test] discover`, relative to `testRoot`): a new file + // Test discovery (`[test] discover`, relative to its root), one entry + // per package the plan tests (each member of a group plan): a new file // matching it is a new test target, and so a new unit in the document. - std::filesystem::path testRoot; - std::vector testDiscover; + std::vector>> tests; }; struct Rendered { @@ -312,6 +312,11 @@ Rendered render(std::span members, Rendered r; nlohmann::json toolchains = nlohmann::json::object(); nlohmann::json sets = nlohmann::json::array(); + // Two plans of one configuration (the members of a group whose plan + // failed, planned one by one) describe the packages they share alike: a + // set is described once, and so is a compile command, keyed by its file + // and its output. A file two configurations compile has a command of each. + std::set describedSets, describedCommands; std::set watch; std::set inputFiles; @@ -349,12 +354,23 @@ Rendered render(std::span members, const auto tcId = toolchain_id(ctx.tc, compilerTriple); const auto rootName = qualified_name(ctx.manifest); + // The packages whose own targets this plan builds: the root, or in a + // workspace plan each selected member (§15 of the 2026-09-29 workspace + // design), with the directory their target entries are written in. + std::vector> subjects; + if (ctx.workspaceMembers.empty()) + subjects.emplace_back(&ctx.manifest, ctx.projectRoot); + for (auto const& wm : ctx.workspaceMembers) subjects.emplace_back(&wm.manifest, wm.root); + std::map subjectByName; std::set testSources; - for (auto const& t : ctx.manifest.targets) { - if (t.kind != mcpp::manifest::Target::TestBinary || t.main.empty()) continue; - std::filesystem::path main{t.main}; - testSources.insert((main.is_absolute() ? main : ctx.projectRoot / main) - .lexically_normal()); + for (auto const& [sm, sroot] : subjects) { + subjectByName[qualified_name(*sm)] = sm; + for (auto const& t : sm->targets) { + if (t.kind != mcpp::manifest::Target::TestBinary || t.main.empty()) continue; + std::filesystem::path main{t.main}; + testSources.insert((main.is_absolute() ? main : sroot / main) + .lexically_normal()); + } } std::vector order; @@ -394,8 +410,9 @@ Rendered render(std::span members, const auto& cu = *inv.unit; const bool isTest = testSources.contains(cu.source.lexically_normal()); const std::string package = cu.packageName.empty() ? rootName : cu.packageName; + const auto subject = subjectByName.find(package); const std::string kind = isTest ? "test" - : package == rootName ? target_kind(ctx.manifest) + : subject != subjectByName.end() ? target_kind(*subject->second) : "library"; auto& set = set_for(member.setPrefix + package + (isTest ? ":test" : ""), package, kind); @@ -407,7 +424,8 @@ Rendered render(std::span members, if (!cu.providesModule.empty()) provides[cu.providesModule] = ""; nlohmann::json requires_ = nlohmann::json::array(); for (auto const& name : cu.imports) requires_.push_back(name); - r.compileCommands.push_back(nlohmann::json{ + if (describedCommands.insert(inv.file + '\x1f' + inv.output).second) + r.compileCommands.push_back(nlohmann::json{ {"directory", inv.directory}, {"file", inv.file}, {"arguments", inv.arguments}, @@ -439,7 +457,8 @@ Rendered render(std::span members, const auto sourceStr = native_string(unit.source); const auto objectStr = native_string(unit.object); const auto workDirStr = native_string(unit.workDirectory); - r.compileCommands.push_back(nlohmann::json{ + if (describedCommands.insert(sourceStr + '\x1f' + objectStr).second) + r.compileCommands.push_back(nlohmann::json{ {"directory", workDirStr}, {"file", sourceStr}, {"arguments", unit.arguments}, @@ -516,6 +535,7 @@ Rendered render(std::span members, } for (auto const& name : order) { + if (!describedSets.insert(name).second) continue; auto& set = groups.at(name); nlohmann::json visible = nlohmann::json::array(); for (auto const& other : order) @@ -545,7 +565,8 @@ Rendered render(std::span members, watch_file(sp.root / "build.mcpp"); for (auto const& g : sp.sources) watch_glob(sp.root, g); } - for (auto const& g : member.testDiscover) watch_glob(member.testRoot, g); + for (auto const& [testRoot, globs] : member.tests) + for (auto const& g : globs) watch_glob(testRoot, g); // Only an editable package's build program inputs can change: a store // package's are fixed by the version its manifest and lock name. std::set editableRoots; diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index e333934da..da5f6edb6 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -1238,6 +1238,11 @@ std::expected run_build_program( } } + // The package the status lines name: several programs run in one + // workspace plan, and a line without its package cannot be attributed. + const std::string who = m.package.namespace_.empty() + ? m.package.name : m.package.namespace_ + "." + m.package.name; + // Fast path: declared inputs + contract unchanged → reapply cached // directives, no run. CacheRecord cache = read_cache(bdir); @@ -1261,7 +1266,7 @@ std::expected run_build_program( for (auto const& a : dirs::advisories(m.package.name, cache.directives)) mcpp::ui::warning(a); report_stated_diagnostics(m.package.name, cache.directives); - mcpp::ui::info("build.mcpp", "up to date (cached)"); + mcpp::ui::info("build.mcpp", std::format("up to date {} (cached)", who)); return {}; } @@ -1622,7 +1627,7 @@ std::expected run_build_program( " arrived. Please report it with the toolchain name and this " "line.", *orphan)); } - mcpp::ui::info("build.mcpp", "compiling"); + mcpp::ui::info("build.mcpp", std::format("compiling {}", who)); // GCC resolves imported BMIs via gcm.cache/ relative to the compile cwd, so // any compile that imports a module — `mcpp`, `std`, a build rule's host // module, or any mix — has to run from bdir, where they were staged or @@ -1663,7 +1668,7 @@ std::expected run_build_program( // produces a baffling failure; a build PROGRAM that runs long is usually // stuck — waiting on a network read or spinning — and without a bound the // whole build hangs with no diagnostic at all. - mcpp::ui::info("build.mcpp", "running"); + mcpp::ui::info("build.mcpp", std::format("running {}", who)); bool timedOut = false; // The bound comes from THIS package's manifest — a dependency's generator // is bounded by the dependency's own declaration, because its author is diff --git a/src/build/compile_commands.cppm b/src/build/compile_commands.cppm index 24214e2d5..39f9ba24c 100644 --- a/src/build/compile_commands.cppm +++ b/src/build/compile_commands.cppm @@ -168,6 +168,22 @@ publish_root_compile_commands( const std::filesystem::path& mcppHome, ReplaceFile replaceFile = mcpp::platform::fs::replace_file); +// The same for a command that planned several configurations (a workspace +// whose members form more than one configuration group, workspace design +// 2026-09-29 §15): the root file is the union of their databases, sorted by +// file as every database is, an entry of an earlier database kept where two +// have the same file and output. A file two configurations compile has an +// entry of each (the format allows several; clangd reads the first). The +// command publishes the file once, after all its plans, so it does not depend +// on which concurrent build finished last. +std::expected +publish_root_compile_commands( + std::span configDbPaths, + const std::filesystem::path& rootPath, + const std::filesystem::path& targetRoot, + const std::filesystem::path& mcppHome, + ReplaceFile replaceFile = mcpp::platform::fs::replace_file); + // Writes the configuration's database for `plan`, then publishes the root // copy from it. `commandCount` is the configuration database's entry count; // `foreignEntries` is the root publish's (see publish_root_compile_commands). @@ -625,27 +641,56 @@ publish_root_compile_commands( const std::filesystem::path& targetRoot, const std::filesystem::path& mcppHome, ReplaceFile replaceFile) { + return publish_root_compile_commands(std::span(&configDbPath, 1), rootPath, + targetRoot, mcppHome, std::move(replaceFile)); +} + +std::expected +publish_root_compile_commands( + std::span configDbPaths, + const std::filesystem::path& rootPath, + const std::filesystem::path& targetRoot, + const std::filesystem::path& mcppHome, + ReplaceFile replaceFile) { + // One database is copied byte for byte; several are joined. std::string content; - { - std::ifstream input(configDbPath, std::ios::binary); - if (!input) { - return std::unexpected(write_error(std::format( - "cannot read configuration compile database '{}'", configDbPath.string()))); + nlohmann::json configJson = nlohmann::json::array(); + std::set described; + for (auto const& configDbPath : configDbPaths) { + std::string text; + { + std::ifstream input(configDbPath, std::ios::binary); + if (!input) { + return std::unexpected(write_error(std::format( + "cannot read configuration compile database '{}'", configDbPath.string()))); + } + std::stringstream ss; + ss << input.rdbuf(); + if (input.bad()) { + return std::unexpected(write_error(std::format( + "cannot read configuration compile database '{}'", configDbPath.string()))); + } + text = ss.str(); } - std::stringstream ss; - ss << input.rdbuf(); - if (input.bad()) { + auto json = nlohmann::json::parse(text, nullptr, /*allow_exceptions=*/false); + if (json.is_discarded() || !json.is_array()) { return std::unexpected(write_error(std::format( - "cannot read configuration compile database '{}'", configDbPath.string()))); + "configuration compile database '{}' is not a JSON array", + configDbPath.string()))); + } + if (configDbPaths.size() == 1) { content = std::move(text); configJson = std::move(json); break; } + for (auto& e : json) { + if (!e.is_object() || !e.contains("file") || !e.at("file").is_string()) continue; + auto key = dedup_key(resolve_against_directory(e, "file")); + if (e.contains("output") && e.at("output").is_string()) + key += '\x1f' + dedup_key(resolve_against_directory(e, "output")); + if (described.insert(std::move(key)).second) + configJson.push_back(std::move(e)); } - content = ss.str(); } - - auto configJson = nlohmann::json::parse(content, nullptr, /*allow_exceptions=*/false); - if (configJson.is_discarded() || !configJson.is_array()) { - return std::unexpected(write_error(std::format( - "configuration compile database '{}' is not a JSON array", - configDbPath.string()))); + if (configDbPaths.size() != 1) { + sort_entries_by_file(configJson); + content = configJson.dump(2) + "\n"; } auto doc = read_existing_document(rootPath); @@ -701,6 +746,7 @@ write_compile_commands(const BuildPlan& plan, const CompileFlags& flags) { auto configPath = plan.outputDir / "compile_commands.json"; auto configResult = publish_compile_commands(configPath, fresh, fileExists); if (!configResult) return configResult; + if (!plan.publishRootCompileDb) return configResult; // §3.2 items 2-3: the root is a copy of it, replaced whole. `emit` // (`mcpp emit build-database`) never reaches this function (SPEC-005 diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 6923c2c2c..c7920faf5 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -944,6 +944,13 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, // The packages announced: the root's dependencies, and in a workspace // plan each member followed by the member's own dependencies (§15), as // the member's own build announced them. + // A member that another member depends on is announced from its own + // member edge, by its directory, not as that member's path dependency. + std::set memberRoots; + for (auto const& m : ctx.workspaceMembers) { + std::error_code ec; + memberRoots.insert(std::filesystem::weakly_canonical(m.root, ec)); + } std::vector> announcedDeps; for (auto const& [name, spec] : ctx.manifest.dependencies) { announcedDeps.emplace_back(name, spec); @@ -953,7 +960,16 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, if (std::filesystem::weakly_canonical(spec.path, e1) != std::filesystem::weakly_canonical(m.root, e2)) continue; - for (auto const& [dn, ds] : m.manifest.dependencies) announcedDeps.emplace_back(dn, ds); + for (auto const& [dn, ds] : m.manifest.dependencies) { + if (ds.isPath()) { + const std::filesystem::path dp{ds.path}; + std::error_code e3; + if (memberRoots.contains(std::filesystem::weakly_canonical( + dp.is_absolute() ? dp : m.root / dp, e3))) + continue; + } + announcedDeps.emplace_back(dn, ds); + } } } for (auto& [name, spec] : announcedDeps) { @@ -1561,6 +1577,31 @@ void restore_root_compile_commands(const std::filesystem::path& projectRoot, } } +// The root compile database of a command that planned configuration groups of +// a workspace (workspace design 2026-09-29 §15): the union of the groups' +// databases (one entry per file and output), published once after every +// group, so the file does not depend on which group's build finished last. +export void publish_workspace_compile_commands( + const std::filesystem::path& wsRoot, + const std::vector& outputDirs) { + std::vector databases; + for (auto const& dir : outputDirs) { + std::error_code ec; + auto db = dir / "compile_commands.json"; + if (std::filesystem::exists(db, ec) && !ec) databases.push_back(std::move(db)); + } + if (databases.empty()) return; + auto result = mcpp::build::publish_root_compile_commands( + databases, wsRoot / "compile_commands.json", + outputDirs.front().parent_path().parent_path(), mcpp::home::root()); + if (!result) { + mcpp::ui::warning(std::format( + "compile_commands.json was not updated: {}", result.error().message)); + } else if (result->foreignEntries > 0) { + mcpp::ui::warning(mcpp::build::foreign_entries_warning(result->foreignEntries)); + } +} + // Every xlings payload the entry's build read is still installed (#716). A // cache written before the field was recorded declines once. bool xlings_payloads_present(const BuildCacheEntry& e) { @@ -1807,8 +1848,13 @@ export std::optional try_fast_workspace_build( } std::chrono::milliseconds total{}; + if (ready.size() > 1) { + std::vector dirs; + for (auto const& r : ready) dirs.push_back(r.outputDir); + publish_workspace_compile_commands(wsRoot, dirs); + } for (auto& r : ready) { - restore_root_compile_commands(wsRoot, r.outputDir); + if (ready.size() == 1) restore_root_compile_commands(wsRoot, r.outputDir); std::chrono::milliseconds elapsed{}; auto rc = run_ninja_fast(r.ninjaProgram, r.outputDir, r.outputDir / "build.ninja", verbose, r.runtimeEnvKey, r.runtimeEnvValue, &elapsed); diff --git a/src/build/plan.cppm b/src/build/plan.cppm index dba12722b..405d5e7e3 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -375,6 +375,11 @@ struct BuildPlan { // (possibly read-only) registry directory, and deriving the path would put // an IDE database there. Empty → projectRoot, the historical default. std::filesystem::path compileDbPath; + // Whether writing this plan's compile database also publishes the root + // copy. False for each plan of a command that planned several + // configurations, which publishes the root file itself, once, from all of + // them (`publish_root_compile_commands` over a list). + bool publishRootCompileDb = true; // See StdModuleUnit above. Empty when the build does not import `std`. std::vector stdModuleUnits; // GCC only: a specs file that replaces the pristine `*link:`, so the diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 2b30c8aa5..0e33e6cd7 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -641,6 +641,10 @@ export struct BuildOverrides { // Every member the COMMAND selected, across its configuration groups: the // request a fast-path record names. Empty: `workspace_members`. std::vector workspace_request; + // The test targets each member of `workspace_members` receives, by member + // path: what `extraTargets` is for a plan of one member, for a plan of + // several (`--configure-only`, `mcpp emit build-database`). + std::map> member_targets; // --profile . Empty = fall through to `[build] default-profile`, then // to `profile_fallback` below, whose own default is "dev". The comment here // said "release" for as long as `mcpp build --help` did, and neither had diff --git a/src/build/prepare/graph.cpp b/src/build/prepare/graph.cpp index 9364e07c1..39bcb6983 100644 --- a/src/build/prepare/graph.cpp +++ b/src/build/prepare/graph.cpp @@ -1763,9 +1763,13 @@ step4b_finalize_dependency(PrepareState& state, WorklistItemCtx& ctx) { && spec.artifacts.empty() && !selectedMember; // A root receives the profile's own compile flags; in a workspace plan // each selected member does, as it did when it was the root. The - // tests `mcpp test` discovered are the selected member's targets. + // tests `mcpp test` discovered are the selected member's targets; a + // plan of several members receives each member's own. if (selectedMember && state.selectedMembers.size() == 1) for (auto const& t : state.extraTargets) ctx.dep_manifest->targets.push_back(t); + if (selectedMember) + if (auto t = state.memberTargets.find(*selectedMember); t != state.memberTargets.end()) + for (auto const& target : t->second) ctx.dep_manifest->targets.push_back(target); if (selectedMember) { auto& mbc = ctx.dep_manifest->buildConfig; mbc.cflags.insert(mbc.cflags.end(), state.profileCflags.begin(), diff --git a/src/build/prepare/manifest.cpp b/src/build/prepare/manifest.cpp index 5be45e054..4b22bbba9 100644 --- a/src/build/prepare/manifest.cpp +++ b/src/build/prepare/manifest.cpp @@ -184,6 +184,8 @@ select_workspace_members(PrepareState& state, const std::filesystem::path& wsRoo auto canonical = std::filesystem::weakly_canonical(dir, canonEc); if (canonEc) canonical = dir; state.selectedMembers[canonical] = mcpp::project::product_directory_name(all, mp); + if (auto t = state.overrides.member_targets.find(mp); t != state.overrides.member_targets.end()) + state.memberTargets[canonical] = t->second; auto member = mcpp::project::load_member_manifest(*state.wsManifest, wsRoot, mp); if (!member) return std::unexpected(member.error()); if (auto r = refuse_unknown_capability(*member, dir / "mcpp.toml"); !r) diff --git a/src/build/prepare/plan.cpp b/src/build/prepare/plan.cpp index 7c2257544..6e04b426c 100644 --- a/src/build/prepare/plan.cpp +++ b/src/build/prepare/plan.cpp @@ -812,7 +812,8 @@ static void step13_graph_and_schedule(PrepareState& state, BuildContext& ctx) { // own target, and the output directory is shared with plain builds because // the fingerprint covers neither input. Stamping it on the plan is what // lets the graph say so about itself. - ctx.plan.graphShape = (state.includeDevDeps || !state.extraTargets.empty()) + ctx.plan.graphShape = (state.includeDevDeps || !state.extraTargets.empty() + || !state.memberTargets.empty()) ? mcpp::build::GraphShape::WithTests : mcpp::build::GraphShape::Normal; // The device variant an override chose is stamped for the same reason: the diff --git a/src/build/prepare/records.cpp b/src/build/prepare/records.cpp index 4484d9237..e3a41e15c 100644 --- a/src/build/prepare/records.cpp +++ b/src/build/prepare/records.cpp @@ -431,7 +431,7 @@ void step13_resolution_json(PrepareState& state, BuildContext& ctx) { { nlohmann::json graphPackages = nlohmann::json::array(); for (std::size_t i = 0; i < state.packages.size(); ++i) - graphPackages.push_back(state.graph_package_entry(i, /*forBuildProgram=*/false)); + graphPackages.push_back(state.graph_package_entry(i, /*forBuildProgram=*/false, nullptr)); j["graph"] = { {"packages", std::move(graphPackages)} }; } diff --git a/src/build/prepare/state.cppm b/src/build/prepare/state.cppm index 7706704dc..a7d026011 100644 --- a/src/build/prepare/state.cppm +++ b/src/build/prepare/state.cppm @@ -300,6 +300,9 @@ struct PrepareState { // declares the dependency key: the forwards its edges receive. std::map>> memberCliForwards; + // The test targets a selected member receives (`BuildOverrides:: + // member_targets`), by the same key. + std::map> memberTargets; // The profile's own compile flags (`[profile.] cflags`/`cxxflags`), // which a root receives; in a workspace plan every selected member does. std::vector profileCflags, profileCxxflags; @@ -523,7 +526,9 @@ struct PrepareState { std::filesystem::path stdCompatObjectPath; std::optional describedStdModule; std::string stdFlagAndDialect; - std::function graph_package_entry; + // The entry of package `i`; with `requesters`, only the requests made by + // the packages it marks (a build program's closure). + std::function*)> graph_package_entry; mcpp::modgraph::ScanResult scan; mcpp::modgraph::ValidateReport report; }; diff --git a/src/build/prepare/target_side.cpp b/src/build/prepare/target_side.cpp index dd5348f73..296ddb822 100644 --- a/src/build/prepare/target_side.cpp +++ b/src/build/prepare/target_side.cpp @@ -18,6 +18,7 @@ import mcpp.log; import mcpp.manifest; import mcpp.source_kind; import mcpp.modgraph.glob; +import mcpp.graph; import mcpp.modgraph.graph; import mcpp.modgraph.scanner; import mcpp.modgraph.validate; @@ -1436,7 +1437,8 @@ static void step9_define_graph_package_entry_closure(PrepareState& state) { // // The link form is read from `dependencyLinkForms`, which is computed once, // before this point, for exactly this program (#642 E2). - state.graph_package_entry = [&](std::size_t i, bool forBuildProgram) { + state.graph_package_entry = [&](std::size_t i, bool forBuildProgram, + const std::vector* requesters) { auto const& pm = state.packages[i].manifest; const auto id = mcpp::manifest::package_id(pm.package); nlohmann::json entry = { @@ -1452,6 +1454,8 @@ static void step9_define_graph_package_entry_closure(PrepareState& state) { nlohmann::json requests = nlohmann::json::array(); for (auto const& r : state.graphRequests) { if (r.dependencyPackageIndex != i) continue; + if (requesters && (r.consumerPackageIndex >= requesters->size() + || !(*requesters)[r.consumerPackageIndex])) continue; requests.push_back({ {"requester", mcpp::manifest::package_id( state.packages[r.consumerPackageIndex].manifest.package).canonical()}, @@ -1630,7 +1634,7 @@ static std::expected step9_root_build_program(PrepareState& s doc["kind"] = "mcpp.graph"; doc["version"] = 1; nlohmann::json list = nlohmann::json::array(); - for (auto i : order) list.push_back(state.graph_package_entry(i, true)); + for (auto i : order) list.push_back(state.graph_package_entry(i, true, nullptr)); doc["packages"] = std::move(list); const auto text = doc.dump(2) + "\n"; const auto graphPath = bpEnv.artifactsDir / "graph.json"; @@ -1854,9 +1858,14 @@ write_graph_document(PrepareState& state, std::size_t subject, nlohmann::json doc; doc["kind"] = "mcpp.graph"; doc["version"] = 1; + // The requests a document lists are those made by its own packages. A + // request from outside the program's closure (another workspace member, a + // workspace plan's virtual root) is a fact about the plan, not about the + // package, and would make the document, and with it the program's cache + // key, depend on which members the command selected. nlohmann::json list = nlohmann::json::array(); for (auto i : order) { - auto entry = state.graph_package_entry(i, true); + auto entry = state.graph_package_entry(i, true, &reached); entry["root"] = i == subject; list.push_back(std::move(entry)); } @@ -1888,7 +1897,22 @@ write_graph_document(PrepareState& state, std::size_t subject, // environment in every selection. static std::expected step9_member_build_programs(PrepareState& state) { if (!state.m->package.virtualRoot) return {}; - for (std::size_t i = 1; i < state.packages.size(); ++i) { + // Dependencies first: a member's program runs after the programs of the + // members it depends on, whose directives are then in place in the graph + // it reads. Discovery order, which the packages are numbered in, puts a + // member before its dependency when its key sorts first. + mcpp::graph::AdjacencyList deps(state.packages.size()); + for (auto const& r : state.graphRequests) + if (r.consumerPackageIndex < deps.size() && r.dependencyPackageIndex < deps.size() + && r.consumerPackageIndex != r.dependencyPackageIndex) + deps[r.consumerPackageIndex].push_back(r.dependencyPackageIndex); + // Mutual dev-dependencies between members are legal, so a cycle skips its + // closing edge and every other dependency still comes first. + std::vector roots(state.packages.size()); + std::iota(roots.begin(), roots.end(), std::size_t{0}); + const auto order = *mcpp::graph::depth_first_order(deps, roots, mcpp::graph::Cycles::Skip); + for (auto const i : order) { + if (i == 0) continue; if (!state.isWorkspaceMemberPackage(i)) continue; if (!state.compilesHere(i)) continue; auto& pkg = state.packages[i]; diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 76e794b57..7e6ecea08 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -137,6 +137,25 @@ workspace_groups(const std::filesystem::path& wsRoot, const std::vector> targets; + std::vector>> discovery; +}; +std::expected +group_tests(const std::filesystem::path& wsRoot, const std::vector& group) { + GroupTests out; + for (auto const& mp : group) { + auto d = mcpp::build::discover_test_targets(wsRoot, mp); + if (!d) return std::unexpected(std::format("{}: {}", mp, d.error())); + if (!d->targets.empty()) out.targets[mp] = std::move(d->targets); + out.discovery.emplace_back(d->packageRoot, std::move(d->discover)); + } + return out; +} + // run_build_plan, wrapped in the project's `[hooks]` lifecycle (#496). // // The hooks are those of the package being built: the root's, or in a @@ -239,8 +258,12 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { // Fan-out prefixes every diagnostic with the member it came from; the // single-package path has nothing to disambiguate and passes "". + // `configured`, when given, receives the plan's output directory and the + // plan leaves the root compile database to the caller, which publishes + // it once for every configuration it planned. auto configure_member = [&](mcpp::build::BuildOverrides memberOv, - std::string_view label) -> int { + std::string_view label, + std::vector* configured = nullptr) -> int { auto where = [&](std::string_view msg) { if (label.empty()) std::println(stderr, "error: {}", msg); else std::println(stderr, "error: {}: {}", label, msg); @@ -268,6 +291,10 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { where(ctx.error()); return 2; } + if (configured) { + ctx->plan.publishRootCompileDb = false; + configured->push_back(ctx->outputDir); + } return mcpp::build::run_configure_plan(*ctx, verbose); }; @@ -281,21 +308,55 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { if (!selection) { std::println(stderr, "error: {}", selection.error()); return 2; } if (*selection) { auto const& members = (*selection)->members; - if (configure_only) { - int rc = 0; - for (auto const& mp : members) { - mcpp::build::BuildOverrides mo = ov; - mo.package_filter = mp; - int r = configure_member(std::move(mo), members.size() > 1 ? mp : ""); - if (r != 0) rc = r; - } - return rc; + if (configure_only && members.size() == 1) { + mcpp::build::BuildOverrides mo = ov; + mo.package_filter = members.front(); + return configure_member(std::move(mo), ""); } auto groups = workspace_groups((*selection)->root, members); if (!groups) { std::println(stderr, "error: {}", groups.error()); return 2; } std::vector request; for (auto const& g : *groups) for (auto const& mp : g) request.push_back(mp); + // Configured as built: one plan per configuration group, with each + // member's tests and dev-dependencies (the surface an editor needs, + // as for one member), so the compile database of a group describes + // each of its packages once. + if (configure_only) { + int rc = 0; + std::vector configured; + for (auto const& g : *groups) { + auto configure_one_by_one = [&] { + // Planned member by member, so a member's failure affects + // that member only, as a plan of one member always did. + for (auto const& mp : g) { + mcpp::build::BuildOverrides one = ov; + one.package_filter = mp; + if (int r = configure_member(std::move(one), mp, &configured); r != 0) rc = r; + } + }; + auto tests = group_tests((*selection)->root, g); + if (!tests && g.size() > 1) { configure_one_by_one(); continue; } + if (!tests) { std::println(stderr, "error: {}", tests.error()); rc = 2; continue; } + mcpp::build::BuildOverrides mo = ov; + mo.package_filter.clear(); + mo.project_root = (*selection)->root; + mo.workspace_members = g; + mo.workspace_request = request; + const bool includeDevDeps = !tests->targets.empty(); + mo.member_targets = std::move(tests->targets); + auto ctx = mcpp::build::prepare_build(print_fp, includeDevDeps, + /*extraTargets=*/{}, mo); + if (!ctx && g.size() > 1) { configure_one_by_one(); continue; } + if (!ctx) { std::println(stderr, "error: {}: {}", g.front(), ctx.error()); rc = 2; continue; } + ctx->plan.publishRootCompileDb = false; + if (int r = mcpp::build::run_configure_plan(*ctx, verbose); r != 0) rc = r; + configured.push_back(ctx->outputDir); + } + if (!configured.empty()) + mcpp::build::publish_workspace_compile_commands((*selection)->root, configured); + return rc; + } const bool plain = !print_fp && ov.target_triple.empty() && !ov.force_static && ov.profile.empty() && ov.features.empty() && !ov.strict && ov.capabilities.empty() && ov.cache_mode.empty() && ov.accel.empty(); @@ -330,6 +391,9 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { const std::size_t hw = std::max(1u, std::thread::hardware_concurrency()); std::set directories; for (auto const& c : contexts) directories.insert(c.outputDir.lexically_normal()); + // The root compile database is the command's, published once below + // from every group's (execute.cppm), not by each concurrent build. + for (auto& c : contexts) c.plan.publishRootCompileDb = false; for (auto& c : contexts) { const std::size_t want = c.plan.scheduleNinjaJobs > 0 ? static_cast(c.plan.scheduleNinjaJobs) : hw + 2; @@ -354,6 +418,9 @@ export int cmd_build(const mcpplibs::cmdline::ParsedArgs& parsed) { } for (int r : results) if (r != 0 && rc == 0) rc = r; + std::vector dirs; + for (auto const& c : contexts) dirs.push_back(c.outputDir); + mcpp::build::publish_workspace_compile_commands((*selection)->root, dirs); return rc; } @@ -504,16 +571,47 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) return failed("MCPP_BUILD_DATABASE_NO_PROJECT", "no mcpp.toml found in current directory or any parent"); - std::vector> requests; - if (auto members = workspace_fanout_members(parsed.is_flag_set("workspace"), - ov.package_filter)) { - for (auto const& mp : *members) { - auto mo = ov; - mo.package_filter = mp; - requests.emplace_back(mp, std::move(mo)); + // One plan per configuration group, as `mcpp build` plans a workspace + // (workspace design 2026-09-29 §15): a package the members share is one + // set of the document, described once. A selection of one member is one + // plan of that member, as before. + struct PlanRequest { + std::string member; // one member's path; empty for a group or a package + std::vector group; // the members of a group plan + mcpp::build::BuildOverrides mo; + }; + std::vector requests; + auto plan_alone = [&](const std::string& mp) { + auto one = ov; + one.package_filter = mp; + requests.push_back({mp, {}, std::move(one)}); + }; + std::filesystem::path wsRoot = *root; + auto selection = workspace_selection(parsed.is_flag_set("workspace"), ov.package_filter); + if (!selection) + return failed("MCPP_BUILD_DATABASE_PLAN_FAILED", selection.error()); + if (*selection && (*selection)->members.size() > 1) { + wsRoot = (*selection)->root; + // A member whose manifest cannot be read has no configuration; the + // members are then planned one by one, and it fails alone (R5.2). + auto groups = workspace_groups(wsRoot, (*selection)->members); + if (!groups) { + for (auto const& mp : (*selection)->members) plan_alone(mp); + } else { + for (auto const& g : *groups) { + auto mo = ov; + mo.package_filter.clear(); + mo.project_root = wsRoot; + mo.workspace_members = g; + mo.workspace_request = (*selection)->members; + requests.push_back({g.size() == 1 ? g.front() : std::string{}, g, std::move(mo)}); + } } + } else if (auto members = workspace_fanout_members(parsed.is_flag_set("workspace"), + ov.package_filter)) { + plan_alone(members->front()); } else { - requests.emplace_back(std::string{}, ov); + requests.push_back({std::string{}, {}, ov}); } // An offline plan that needs a download is not a defect of the project, and @@ -533,8 +631,7 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) std::vector contexts; std::vector workDirs; - std::vector prefixes; - std::vector>> testDiscovery; + std::vector>>> testDiscovery; // The root of every member whose planning failed: `mcpp.toml` and // `build.mcpp` (when it exists) join `watch` exactly as a planned // member's do (render(), below), so an edit that might fix the failure is @@ -560,30 +657,72 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) // Planning narrates on stdout and may start programs that inherit it; // the document is printed after this scope, alone. mcpp::platform::terminal::StdoutToStderr narration; - for (auto& [member, mo] : requests) { - const auto memberRoot = member.empty() ? *root : *root / member; - const auto memberPath = member.empty() ? std::string("mcpp.toml") - : member + "/mcpp.toml"; - auto discovered = mcpp::build::discover_test_targets(*root, mo.package_filter); - if (!discovered) { + // A group whose plan fails is planned member by member (appended to + // `requests` and reached by this same loop), so a member's failure + // still affects that member only (SPEC-005 R5.2). + for (std::size_t ri = 0; ri < requests.size(); ++ri) { + auto req = requests[ri]; + auto& mo = req.mo; + const auto memberRoot = req.member.empty() ? wsRoot : wsRoot / req.member; + const auto memberPath = req.member.empty() ? std::string("mcpp.toml") + : req.member + "/mcpp.toml"; + // The roots whose `mcpp.toml` and `build.mcpp` join `watch` when + // this plan fails: the member's, or every member's of a group. + std::vector planRoots; + if (req.group.empty()) planRoots.push_back(memberRoot); + for (auto const& mp : req.group) planRoots.push_back(wsRoot / mp); + auto fail_plan = [&](std::string message) { diagnostics.push_back({plan_failure_code(), Severity::Error, - member.empty() ? discovered.error() - : std::format("{}: {}", member, discovered.error()), - memberPath}); - failedMemberRoots.push_back(memberRoot); - continue; - } + std::move(message), memberPath}); + for (auto const& r : planRoots) failedMemberRoots.push_back(r); + }; // As `--configure-only`: tests and dev-dependencies are part of the - // surface an editor needs. - const bool includeDevDeps = !discovered->targets.empty(); - auto discovery = std::pair{discovered->packageRoot, discovered->discover}; + // surface an editor needs, each member's own in a group plan. + std::vector extraTargets; + std::vector>> discovery; + bool includeDevDeps = false; + if (req.group.empty()) { + auto discovered = mcpp::build::discover_test_targets(*root, mo.package_filter); + if (!discovered) { + fail_plan(req.member.empty() ? discovered.error() + : std::format("{}: {}", req.member, discovered.error())); + continue; + } + includeDevDeps = !discovered->targets.empty(); + discovery.emplace_back(discovered->packageRoot, discovered->discover); + extraTargets = std::move(discovered->targets); + } else { + auto tests = group_tests(wsRoot, req.group); + if (!tests && req.group.size() == 1) { fail_plan(tests.error()); continue; } + if (!tests) { + for (auto const& mp : req.group) plan_alone(mp); + continue; + } + includeDevDeps = !tests->targets.empty(); + discovery = std::move(tests->discovery); + mo.member_targets = std::move(tests->targets); + } mo.plan_only = true; - mo.work_dir = build_database_work_dir(*root, mo.package_filter); + std::string workKey = mo.package_filter; + for (auto const& mp : req.group) workKey += (workKey.empty() ? "" : ",") + mp; + mo.work_dir = build_database_work_dir(req.group.empty() ? *root : wsRoot, workKey); std::error_code ec; std::filesystem::remove(mo.work_dir / "mcpp.lock", ec); + const auto diagnosticsBefore = diagnostics.size(); auto ctx = mcpp::build::prepare_build(/*print_fingerprint=*/false, includeDevDeps, - std::move(discovered->targets), mo); + std::move(extraTargets), mo); + if (!ctx && req.group.size() > 1) { + // What the group's attempt recorded is recorded again, per + // member, by the plans below. + (void)mcpp::diag::take(); + (void)mcpp::build::take_notes_on_failure(); + (void)mcpp::build::refusal::take(); + diagnostics.erase(diagnostics.begin() + static_cast(diagnosticsBefore), + diagnostics.end()); + for (auto const& mp : req.group) plan_alone(mp); + continue; + } take_member_diagnostics(memberPath); if (!ctx) { // A wholly-failed member contributes exactly one `error` @@ -598,8 +737,8 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) // recorded, then discarded the moment `prepare_build` returned // — never reached the reader (design 2026-09-27 §4.2, mcpp#724 // side finding A, fix item 2). - std::string message = member.empty() ? ctx.error() - : std::format("{}: {}", member, ctx.error()); + std::string message = req.member.empty() ? ctx.error() + : std::format("{}: {}", req.member, ctx.error()); for (auto const& note : mcpp::build::take_notes_on_failure()) message += note.path.empty() ? std::format("\n earlier in this pass, {}: {}", @@ -608,12 +747,11 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) note.code, note.path, note.message); diagnostics.push_back({plan_failure_code(), Severity::Error, std::move(message), memberPath}); - failedMemberRoots.push_back(memberRoot); + for (auto const& r : planRoots) failedMemberRoots.push_back(r); continue; } contexts.push_back(std::move(*ctx)); workDirs.push_back(mo.work_dir); - prefixes.push_back(member.empty() ? std::string{} : member + "/"); testDiscovery.push_back(std::move(discovery)); } } @@ -642,11 +780,20 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) "would update it", projectLock.string())}); } + // A set is named by its package; a document of several configurations + // prefixes each name with its configuration's name, the name of the build + // directory the configuration is built in (SPEC-005 R3.3), so a package + // two configurations compile is a set of each. + std::set configurations; + for (auto const& c : contexts) configurations.insert(c.outputDir.filename().string()); + auto configurationPrefix = [&](const mcpp::build::BuildContext& c) { + return configurations.size() > 1 ? c.outputDir.filename().string() + "/" : std::string{}; + }; std::vector members; bool ranBuildPrograms = false; for (std::size_t i = 0; i < contexts.size(); ++i) { - members.push_back({&contexts[i], prefixes[i], workDirs[i], - testDiscovery[i].first, testDiscovery[i].second}); + members.push_back({&contexts[i], configurationPrefix(contexts[i]), workDirs[i], + testDiscovery[i]}); if (!mcpp::build::declared_program_inputs(workDirs[i]).empty()) ranBuildPrograms = true; for (auto const& sp : contexts[i].sourcePackages) { @@ -663,7 +810,7 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) ov.profile, ov.features, ov.capabilities, ov.accel, ov.force_static, ov.package_filter, parsed.is_flag_set("workspace")); auto rendered = mcpp::build::database::render(members, failedMemberRoots, - *root, selector); + wsRoot, selector); // A note's severity is its own (E3's program-failure note is an error; // every other note today is a warning) and its `path`, when set, already // names the file relative to the workspace root — render() rewrote it. diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index 6753d3270..13321aba0 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -28,6 +28,7 @@ import mcpp.toolchain.model; import mcpp.toolchain.probe; import mcpp.toolchain.registry; import mcpp.toolchain.triple; +import mcpp.log; import mcpp.ui; import mcpp.xlings; @@ -207,6 +208,48 @@ build_extra_android_legs(const std::string& targetName, return out; } +// The human report of what a dispatched format produced. A format may submit +// one output per file of a distribution tree (1,309 for one program of the +// validation project), and a line per file then buries the rest of the pass. +// Up to `kListedOutputs` outputs are named one per line; more are named by the +// entry each lies in below their common parent, with a count, and --verbose +// names every one. `--message-format json` lists every output in either case. +constexpr std::size_t kListedOutputs = 8; + +void report_packed(const std::vector& outputs, + const mcpp::ui::PathContext& pathCtx) { + if (outputs.size() <= kListedOutputs || mcpp::log::is_verbose()) { + for (auto const& o : outputs) + mcpp::ui::status("Packed", mcpp::ui::shorten_path(o, pathCtx)); + return; + } + auto parent = outputs.front().parent_path(); + for (auto const& o : outputs) { + auto a = parent.begin(), b = o.begin(); + std::filesystem::path common; + for (; a != parent.end() && b != o.end() && *a == *b; ++a, ++b) common /= *a; + parent = common; + } + // Each entry, in the order first reached, with the outputs below it and + // whether it is itself an output (then named alone). + struct Entry { std::filesystem::path path; std::size_t outputs = 0; bool isOutput = false; }; + std::vector entries; + for (auto const& o : outputs) { + // Outputs with no common parent (two drives) are named one by one. + const auto rel = parent.empty() ? std::filesystem::path{} : o.lexically_relative(parent); + const auto path = rel.empty() || rel == "." ? o : parent / *rel.begin(); + auto it = std::ranges::find(entries, path, &Entry::path); + if (it == entries.end()) it = entries.insert(entries.end(), Entry{path}); + ++it->outputs; + it->isOutput = it->isOutput || path == o; + } + for (auto const& e : entries) + mcpp::ui::status("Packed", e.outputs == 1 && e.isOutput + ? mcpp::ui::shorten_path(e.path, pathCtx) + : std::format("{} ({} {})", mcpp::ui::shorten_path(e.path, pathCtx), e.outputs, + e.outputs == 1 ? "file" : "files")); +} + // Everything after CLI option parsing for `mcpp pack`. // // `wantTarget` is the target NAME the user asked for, empty when they did not. @@ -821,9 +864,9 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, return PackOutcome{1}; } if (consumed.contains(abs)) { intermediate.push_back(std::move(abs)); continue; } - mcpp::ui::status("Packed", mcpp::ui::shorten_path(abs, pathCtx)); reported.push_back(std::move(abs)); } + report_packed(reported, pathCtx); // Every output consumed by another: a cycle a provider should not // write, reported as all outputs rather than as nothing. if (reported.empty()) reported = std::move(intermediate); diff --git a/tests/e2e/139_build_program_advisory.sh b/tests/e2e/139_build_program_advisory.sh index 247db6e69..dc85ec594 100755 --- a/tests/e2e/139_build_program_advisory.sh +++ b/tests/e2e/139_build_program_advisory.sh @@ -91,7 +91,7 @@ touch src/main.cpp # First establish that this build really was a cache hit. Without this the next # assertion could pass for the wrong reason — a re-run would also print the # line, and the replay path would go untested. -grep -q "up to date (cached)" second.log \ +grep -qE "up to date .*\(cached\)" second.log \ || { cat second.log; echo "FAIL: the second build re-ran the program; the replay path was not exercised"; exit 1; } grep -q "advisory: no emulator found" second.log \ diff --git a/tests/e2e/186_build_mcpp_protocol_and_bound.sh b/tests/e2e/186_build_mcpp_protocol_and_bound.sh index c3371a99b..2e6108082 100755 --- a/tests/e2e/186_build_mcpp_protocol_and_bound.sh +++ b/tests/e2e/186_build_mcpp_protocol_and_bound.sh @@ -175,7 +175,7 @@ cp "$CACHE" "$TMP/good.cache" # the build.mcpp cache) actually runs. touch src/main.cpp "$MCPP" build > b6.log 2>&1 || { cat b6.log; echo "FAIL: build failed"; exit 1; } -grep -q "up to date (cached)" b6.log || { +grep -qE "up to date .*\(cached\)" b6.log || { cat b6.log; echo "FAIL: an unchanged build.mcpp was re-run"; exit 1; } sed 's/^epoch .*/epoch 987654/' "$TMP/good.cache" > "$CACHE" diff --git a/tests/e2e/651_a_build_program_deploys_what_it_generated.sh b/tests/e2e/651_a_build_program_deploys_what_it_generated.sh index e6e75150d..e7751eafc 100755 --- a/tests/e2e/651_a_build_program_deploys_what_it_generated.sh +++ b/tests/e2e/651_a_build_program_deploys_what_it_generated.sh @@ -148,7 +148,7 @@ grep -qx "changed resource" "$DEPLOYED" \ rm -rf "$BINDIR" touch src/main.cpp "$MCPP" build > b3.log 2>&1 || fail "third build failed" b3.log -grep -q "up to date (cached)" b3.log \ +grep -qE "up to date .*\(cached\)" b3.log \ || fail "the third build re-ran build.mcpp; the replay path was not exercised" b3.log [ -f "$DEPLOYED" ] \ || fail "the deployed file was not restored on a build.mcpp cache hit" b3.log diff --git a/tests/e2e/839_a_workspaces_build_programs_are_named_ordered_and_cached.sh b/tests/e2e/839_a_workspaces_build_programs_are_named_ordered_and_cached.sh new file mode 100644 index 000000000..dc4e87c1c --- /dev/null +++ b/tests/e2e/839_a_workspaces_build_programs_are_named_ordered_and_cached.sh @@ -0,0 +1,92 @@ +#!/usr/bin/env bash +# 839_a_workspaces_build_programs_are_named_ordered_and_cached.sh -- workspace +# design 2026-09-29 §15 and §17.1. +# +# Every member of the workspace below has a build program; cli and gui depend +# on core. +# +# B1 the status lines name the package a program belongs to; +# B2 a member's program runs after the programs of the members it depends +# on (2026.9.29.4 ran them in discovery order: cli before core); +# B3 after `--workspace`, a selection of one member, a second +# `--workspace` and another selection reuse every program's result +# (2026.9.29.4 reran them: a program's graph document listed every +# requester of the plan, the virtual root included, so its cache key +# followed the selection); +# B4 `mcpp emit build-database` after the build reuses them too; +# B5 in a `--workspace` build a member another member depends on is +# announced by its directory, and as a path dependency only where it +# is not selected. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +MCPP="${MCPP:-mcpp}" + +cat > mcpp.toml <<'EOF' +[workspace] +members = ["core", "cli", "gui"] +EOF +mkdir -p core/src cli/src gui/src +cat > core/mcpp.toml <<'EOF' +[package] +name = "core" +version = "0.1.0" + +[targets.core] +kind = "lib" +EOF +printf 'export module bp_core;\nexport int core_v() { return 1; }\n' > core/src/core.cppm +for m in cli gui; do + cat > $m/mcpp.toml < $m/src/main.cpp +done +for p in core cli gui; do + printf 'import mcpp;\nint main() { mcpp::define("BP_%s=1"); return 0; }\n' $p > $p/build.mcpp +done + +"$MCPP" build --workspace > b1.log 2>&1 || fail "the first build failed" b1.log + +# B1 +for p in core cli gui; do + grep -q "build.mcpp running $p" b1.log || fail "B1: no status line names $p" b1.log +done + +# B2 +first=$(grep -m1 -n "build.mcpp running" b1.log) +case "$first" in *"running core"*) ;; *) fail "B2: core's program did not run first" b1.log ;; esac + +# B3 +"$MCPP" build -p cli > b2.log 2>&1 || fail "-p cli failed" b2.log +touch core/src/core.cppm +"$MCPP" build --workspace > b3.log 2>&1 || fail "the second --workspace failed" b3.log +touch core/src/core.cppm +"$MCPP" build -p gui > b4.log 2>&1 || fail "-p gui failed" b4.log +for f in b2.log b3.log b4.log; do + ! grep -q "build.mcpp running" $f || fail "B3: a program reran in $f" $f + grep -q "up to date" $f || fail "B3: $f shows no program at all" $f +done + +# B4 +"$MCPP" emit build-database --format json -o db.json > e.log 2>&1 || fail "emit failed" e.log +! grep -q "build.mcpp running" e.log || fail "B4: emit reran a program" e.log + +# B5 +grep -q "Compiling core (core)" b1.log || fail "B5: core is not announced by its directory" b1.log +! grep -q "Compiling core (path)" b1.log || fail "B5: a selected member is announced as a path dependency" b1.log +grep -q "Compiling core (path)" b2.log || fail "B5: under -p cli, core is cli's path dependency" b2.log + +echo "PASS: 839_a_workspaces_build_programs_are_named_ordered_and_cached" diff --git a/tests/e2e/840_a_workspace_database_describes_each_package_once.sh b/tests/e2e/840_a_workspace_database_describes_each_package_once.sh new file mode 100644 index 000000000..549328ba0 --- /dev/null +++ b/tests/e2e/840_a_workspace_database_describes_each_package_once.sh @@ -0,0 +1,126 @@ +#!/usr/bin/env bash +# requires: python3 +# 840_a_workspace_database_describes_each_package_once.sh -- workspace design +# 2026-09-29 §15 and §17.1; SPEC-005 R3.3, R3.5, R5.2. +# +# `mcpp emit build-database` and `mcpp build --configure-only` plan a +# workspace as `mcpp build` does: one plan per configuration group, with each +# member's tests. 2026.9.29.4 planned each member separately, so a package two +# members use was described once per member, each time with other arguments. +# +# A one configuration: each set is named by its package, with no prefix; +# each source is described once; a member that is a program is described +# as one and its tests as tests; `--configure-only` writes one compile +# command per source; +# B two configurations that both compile core: each set name is prefixed +# with its configuration's name, core is described once per +# configuration, and the root `compile_commands.json` of `--configure-only` +# and of `build --workspace` describes both configurations (it was the +# database of whichever group was written last); +# C a member whose planning fails, in a group with others: it fails alone, +# with its own `mcpp.toml` as the diagnostic's path, and the others are +# described (R5.2). +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +MCPP="${MCPP:-mcpp}" + +member() { # $1 = name, $2 = standard ("" for the default) + mkdir -p $1/src + { printf '[package]\nname = "%s"\nversion = "0.1.0"\n' $1 + [ -n "$2" ] && printf 'standard = "%s"\n' "$2" + printf '\n[dependencies]\ncore = { path = "../core" }\n\n[targets.%s]\nkind = "bin"\nmain = "src/main.cpp"\n' $1 + } > $1/mcpp.toml + printf 'import db_core;\nint main() { return core_v() == 1 ? 0 : 1; }\n' > $1/src/main.cpp +} +core() { + mkdir -p core/src + printf '[package]\nname = "core"\nversion = "0.1.0"\n\n[targets.core]\nkind = "lib"\n' > core/mcpp.toml + printf 'export module db_core;\nexport int core_v() { return 1; }\n' > core/src/core.cppm +} + +cat > check.py <<'PY' +import json, sys, re, collections +d = json.load(open(sys.argv[1])); part = sys.argv[2] +sets = d["data"]["database"]["sets"] +names = [s["name"] for s in sets] +assert len(names) == len(set(names)), f"{part}: a set is described twice: {names}" +prefix = lambda n: n.split("/")[0] if "/" in n else "" +base = lambda n: n.split("/")[-1] +units = collections.Counter((prefix(s["name"]), t["source"]) for s in sets for t in s["translation-units"]) +assert all(c == 1 for c in units.values()), f"{part}: a source is described twice in one configuration" +kinds = {base(s["name"]): s["ide"]["kind"] for s in sets} +if part == "A": + assert all("/" not in n for n in names), f"A: a set of one configuration is prefixed: {names}" + assert names.count("core") == 1, f"A: core: {names}" + assert kinds.get("cli") == "executable" and kinds.get("cli:test") == "test", f"A: kinds {kinds}" +if part == "B": + assert all(re.fullmatch(r"[0-9a-f]{16}/.+", n) for n in names), f"B: names {names}" + cores = [n for n in names if base(n) == "core"] + assert len(cores) == 2 and len({prefix(n) for n in cores}) == 2, f"B: core per configuration: {names}" +if part == "C": + errors = [x for x in d["diagnostics"] if x["severity"] == "error"] + assert len(errors) == 1 and errors[0].get("path", "").replace("\\", "/") == "broken/mcpp.toml", \ + f"C: diagnostics {d['diagnostics']}" + assert {"core", "cli", "gui"} <= {base(n) for n in names}, f"C: the other members: {names}" + # Planned one by one, the members are still one configuration. + assert all("/" not in n for n in names), f"C: the fallback split the configuration: {names}" +PY +cat > root.py <<'PY' +import json, pathlib, sys, collections +# argv: ... +entries = json.loads(pathlib.Path("compile_commands.json").read_text()) +norm = lambda s: s.replace("\\", "/") +pairs = collections.Counter((norm(e["file"]), norm(e.get("output", ""))) for e in entries) +assert all(c == 1 for c in pairs.values()), "a file and output have two commands" +files = [norm(e["file"]) for e in entries] +cores = sum(1 for f in files if f.endswith("core/src/core.cppm")) +assert cores == int(sys.argv[1]), f"core has {cores} commands, expected one per configuration ({sys.argv[1]})" +for want in sys.argv[2:]: + assert any(f.endswith(want) for f in files), f"{want} is not described" +PY + +# ── A ────────────────────────────────────────────────────────────────────── +mkdir a && cd a +printf '[workspace]\nmembers = ["core", "cli", "gui"]\n' > mcpp.toml +core; member cli; member gui +mkdir -p cli/tests +printf 'import db_core;\nint main() { return core_v() == 1 ? 0 : 1; }\n' > cli/tests/check.cpp +"$MCPP" emit build-database --format json -o db.json > e.log 2>&1 || fail "A: emit failed" e.log +python3 ../check.py db.json A || fail "A" db.json +"$MCPP" build --configure-only > c.log 2>&1 || fail "A: --configure-only failed" c.log +python3 ../root.py 1 cli/src/main.cpp || fail "A: compile_commands.json" c.log +cd .. + +# ── B ────────────────────────────────────────────────────────────────────── +mkdir b && cd b +printf '[workspace]\nmembers = ["core", "cli", "modern"]\n' > mcpp.toml +core; member cli; member modern c++26 +"$MCPP" emit build-database --format json -o db.json > e.log 2>&1 || fail "B: emit failed" e.log +python3 ../check.py db.json B || fail "B" db.json +for cmd in "build --configure-only" "build --workspace"; do + rm -f compile_commands.json + "$MCPP" $cmd > d.log 2>&1 || fail "B: mcpp $cmd failed" d.log + python3 ../root.py 2 cli/src/main.cpp modern/src/main.cpp \ + || fail "B: the root database of mcpp $cmd" d.log +done +cd .. + +# ── C ────────────────────────────────────────────────────────────────────── +mkdir c && cd c +printf '[workspace]\nmembers = ["core", "cli", "gui", "broken"]\n' > mcpp.toml +core; member cli; member gui +mkdir -p broken/src +printf '[package]\nname = "broken"\nversion = "0.1.0"\n\n[dependencies]\nmissing = { path = "../missing" }\n' > broken/mcpp.toml +printf 'int main() { return 0; }\n' > broken/src/main.cpp +set +e +"$MCPP" emit build-database --format json -o db.json > e.log 2>&1 +rc=$? +set -e +[ "$rc" = 1 ] || fail "C: emit exited $rc, expected 1" e.log +python3 ../check.py db.json C || fail "C" db.json e.log + +echo "PASS: 840_a_workspace_database_describes_each_package_once" diff --git a/tests/e2e/841_pack_summarizes_many_outputs.sh b/tests/e2e/841_pack_summarizes_many_outputs.sh new file mode 100644 index 000000000..e803c376f --- /dev/null +++ b/tests/e2e/841_pack_summarizes_many_outputs.sh @@ -0,0 +1,82 @@ +#!/usr/bin/env bash +# requires: python3 +# 841_pack_summarizes_many_outputs.sh -- the human report of `mcpp pack` for a +# format that submits one output per file of a distribution tree. +# +# A format may declare one output per file (the validation project's release +# format declares 1,309 for one program), and a `Packed` line per file then +# buries the rest of the pass. +# +# P1 more than eight outputs are reported by the entry each lies in below +# their common parent, with a count; +# P2 --verbose reports every output; +# P3 `--message-format json` lists every output. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +MCPP="${MCPP:-mcpp}" + +mkdir -p app/src && cd app +cat > mcpp.toml <<'EOF' +[package] +name = "app" +version = "1.0.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF +printf 'int main() { return 0; }\n' > src/main.cpp +cat > build.mcpp <<'EOF' +import mcpp; +#include +#include +int main() { + mcpp::provides_pack_format("tree"); + if (std::string_view(mcpp::pack_format()) != "tree") return 0; + const std::string out = std::string(mcpp::out_dir()) + "/tree"; + auto copy = [&](const std::string& id, const std::string& dst) { + mcpp::action a; + a.id = id.c_str(); + a.role = "artifact"; + a.arg("${mcpp.self}").arg("stage").arg("--verify").arg("content") + .arg("--output").arg(dst.c_str()).arg("${mcpp.target_file:app}") + .input("${mcpp.target_file:app}") + .output(dst.c_str()) + .submit(); + }; + for (int i = 0; i < 6; ++i) { + copy("a" + std::to_string(i), out + "/bin/f" + std::to_string(i)); + copy("b" + std::to_string(i), out + "/data/f" + std::to_string(i)); + } + copy("m", out + "/manifest"); + return 0; +} +EOF + +"$MCPP" pack --format tree > p1.log 2>&1 || fail "pack failed" p1.log + +# P1 +n=$(grep -c "Packed" p1.log) +[ "$n" = 3 ] || fail "P1: $n Packed lines, expected 3" p1.log +grep -qE "Packed .*tree[/\\\\]bin \(6 files\)" p1.log || fail "P1: bin/ is not summarised" p1.log +grep -qE "Packed .*tree[/\\\\]data \(6 files\)" p1.log || fail "P1: data/ is not summarised" p1.log +grep -qE "Packed .*tree[/\\\\]manifest$" p1.log || fail "P1: the single file is not named" p1.log + +# P2 +"$MCPP" pack --format tree --verbose > p2.log 2>&1 || fail "pack --verbose failed" p2.log +n=$(grep -c "Packed" p2.log) +[ "$n" = 13 ] || fail "P2: $n Packed lines under --verbose, expected 13" p2.log + +# P3 +"$MCPP" pack --format tree --message-format json > p3.json 2> p3.err || fail "pack json failed" p3.err +python3 - p3.json <<'EOF' || fail "P3" p3.json +import json, sys +a = json.load(open(sys.argv[1]))["data"]["artifacts"] +assert len(a) == 13, f"P3: {len(a)} artifacts" +EOF + +echo "PASS: 841_pack_summarizes_many_outputs" From 656a0242d64a05a90074ef2c2b29dba261973540 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 19:48:29 +0800 Subject: [PATCH 2/5] e2e 779, 787, 826: the status line names the package; a workspace document of one configuration names its sets without a prefix (SPEC-005 v1.6) --- .../779_a_build_program_declares_a_runtime_search_dir.sh | 2 +- tests/e2e/787_emit_plans_every_member_on_its_own.sh | 7 ++++--- tests/e2e/826_plugin_diagnostics_features_and_names.sh | 2 +- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/tests/e2e/779_a_build_program_declares_a_runtime_search_dir.sh b/tests/e2e/779_a_build_program_declares_a_runtime_search_dir.sh index 0de16ebaa..2dfed92a2 100755 --- a/tests/e2e/779_a_build_program_declares_a_runtime_search_dir.sh +++ b/tests/e2e/779_a_build_program_declares_a_runtime_search_dir.sh @@ -119,7 +119,7 @@ grep -F -- "-Wl,-rpath,$RTDIR" "$G" >/dev/null \ touch src/main.cpp # past the whole-project no-op fast path, without # touching build.mcpp itself "$MCPP" build > b2.log 2>&1 || fail "second build failed" b2.log -grep -q "up to date (cached)" b2.log \ +grep -qE "up to date .*\(cached\)" b2.log \ || fail "the second build re-ran build.mcpp; the replay path was not exercised" b2.log G2=$(find_graph) [ -n "$G2" ] || fail "no build.ninja after the second build" b2.log diff --git a/tests/e2e/787_emit_plans_every_member_on_its_own.sh b/tests/e2e/787_emit_plans_every_member_on_its_own.sh index a7e338f3e..aa40eb299 100755 --- a/tests/e2e/787_emit_plans_every_member_on_its_own.sh +++ b/tests/e2e/787_emit_plans_every_member_on_its_own.sh @@ -16,7 +16,8 @@ # narrower case (E3, #699 item 2 -- see e2e 789), where the member is still # described. Criteria: # A. `emit --workspace --format json` over `good` and `bad`: exit 1, `data` -# present, its one set is `good/good`, `diagnostics` holds exactly one +# present, its one set is `good` (one configuration: no prefix, SPEC-005 +# v1.6 R3.3), `diagnostics` holds exactly one # `error` with `path` "bad/mcpp.toml" and a message naming `bad`, and # `watch` lists `bad/mcpp.toml` alongside `good`'s own inputs. # B. Without `--format`, the same run prints the document (not empty) and @@ -66,7 +67,7 @@ import json, sys e = json.load(open(sys.argv[1])) d = e["data"] sets = [s["name"] for s in d["database"]["sets"]] -assert sets == ["good/good"], sets +assert sets == ["good"], sets diags = e["diagnostics"] assert len(diags) == 1, diags diag = diags[0] @@ -86,7 +87,7 @@ rc=$? set -e [ "$rc" = 1 ] || fail "B: bare invocation exit status $rc, expected 1" b.out b.err [ -s b.out ] || fail "B: the bare document is empty" b.err -"$PY" -c 'import json,sys; d=json.load(open(sys.argv[1])); assert d["sets"][0]["name"]=="good/good", d["sets"]' b.out \ +"$PY" -c 'import json,sys; d=json.load(open(sys.argv[1])); assert d["sets"][0]["name"]=="good", d["sets"]' b.out \ || fail "B: the bare document's set" b.out grep -q "bad" b.err || fail "B: the failed member's reason is not on stderr" b.err echo "ok: B, the exit code is 1 without --format too, and the document still prints" diff --git a/tests/e2e/826_plugin_diagnostics_features_and_names.sh b/tests/e2e/826_plugin_diagnostics_features_and_names.sh index ae0adbfda..27263f16e 100755 --- a/tests/e2e/826_plugin_diagnostics_features_and_names.sh +++ b/tests/e2e/826_plugin_diagnostics_features_and_names.sh @@ -70,7 +70,7 @@ grep -q "mcpp.acme.gen'.*mcpp\.\|claims an origin" d1.log && fail "R1: an own-na # fast path and reaches no build program, so it would measure neither path. touch src/main.cpp "$MCPP" build > d2.log 2>&1 || fail "D2: the second build failed" d2.log -grep -q "up to date (cached)" d2.log || fail "D2: the second build ran the program; the replay is not measured" d2.log +grep -qE "up to date .*\(cached\)" d2.log || fail "D2: the second build ran the program; the replay is not measured" d2.log grep -q "impact: no bindings are generated" d2.log || fail "D2: the cached run did not report the diagnostic" d2.log # N1 From 2d2f17affc8a6c0c03f08945c5d80da4591da08a Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Tue, 29 Sep 2026 22:35:07 +0800 Subject: [PATCH 3/5] 2026.9.29.5: a build reports each step once, with its outcome, and a status line states the build while it runs The validation project's cross-verification of this pull request printed nothing for 40 minutes between the last `Compiling` line and `Finished`: ninja ran with `--quiet` and its output was examined after it exited. Design: .agents/docs/2026-09-29-build-progress-display-design.md. - ninja runs without `--quiet` and is read line by line (stream_exec). Its status lines (NINJA_STATUS, with an escape-sequence prefix that a nested ninja's relayed copy loses) give the counts; its log gives which step finished and when it started and ended; a step record written beside build.ninja (steps.tsv) names each step's package and identity; the `__action` wrapper reports when a check or prepare action starts. - A package's line is written when every step the graph assigns to it has run, or when the build ends: `done `, `cached N units`, `failed`, or the steps that ran. Requested packages are listed, dependencies folded into one line; --verbose lists every package and each step as `[f/t] `. - A build program has one line: `ran