diff --git a/.claude/skills/ci-gates/SKILL.md b/.claude/skills/ci-gates/SKILL.md index a481fd84ad..8232cfc4fb 100644 --- a/.claude/skills/ci-gates/SKILL.md +++ b/.claude/skills/ci-gates/SKILL.md @@ -8305,3 +8305,75 @@ output invited: The rule that found all three: **make the printed numbers add up, out loud.** A census whose parts do not sum to its total has a bucket you have not named. + +## 328. A category is a claim; make it carry a probe + +A census that names "the construct that stops the compiler" is making a claim +per row, and the claims can be false in a way no amount of reading catches. +Measured while building one: SIX constructs read off real failing lines, every +one plausible, compile in isolation. + +``` +@trim("x", "y") builtin call ACCEPTED +.anthropic enum literal ACCEPTED +if (c) { 1 } else { 2 } if-expression ACCEPTED +*Foo &Foo pointer / reference ACCEPTED +[]const u8 const-qualified slice ACCEPTED +for (s) |v| { } capture in a for-loop ACCEPTED +``` + +An earlier multi-agent fan-out had named `[]const u8` as a cause. A queue +repeating it sends someone to implement a feature that is already there. + +So every row carries a MINIMAL SOURCE and is named only while the compiler +rejects it today. The design is self-invalidating: when a construct gains +support its probe passes and the row leaves the queue with no list to edit. It +did that twice in one run, for two constructs fixed in the previous two passes. + +## 329. The probe checks one direction; the counter checks the other + +`is_use` fired on every line starting with `use `. Its probe was +`use a::b as C;`, which the compiler does reject -- so the probe PASSED and the +matcher was still wrong, because plain `use a::b;` and `using a::b;` both +compile. + +The missing half is a COUNTER: a near-identical source the compiler ACCEPTS, on +which the matcher must stay silent. Two tests hold the contract: + +```rust +fn every_matcher_fires_on_its_own_probe() // can the row ever be named +fn no_matcher_fires_on_its_counter() // is the boundary right +``` + +Counters found the real boundaries: `1 as u32` compiles and `1 as float` does +not, so the defect is the TARGET type; `fn a(k: Result)` compiles and +`fn a(k: T)` does not, so it is parameters on the FUNCTION; `[T]` compiles +and `[K: V]` does not. + +And the counters keep working after you leave: the command fails if one of them +stops compiling, because that means the boundary moved. + +## 330. Refused on purpose is a third state, and it is not work + +`x as float` is rejected. It is not a gap: `VALID_CAST_TYPES` carries a written +argument that no backend lowers float arithmetic and that the C generator would +emit `f32` verbatim, which is not a C type. Three specs sat in my work queue +proposing that someone undo that. + +The text of the error does NOT distinguish the two -- I tried. The deliberate +refusal still prints "parse error ... unknown cast target". So the marking is +manual, carries the citation, and prints in its own section. An unmarked row is +not proof it is a gap; it is proof nobody has looked. + +**Three states, not two: not implemented / already supported / refused with a +reason.** A census with two states will eventually recommend undoing a decision. + +## 331. A fix that gains nothing is still a fix, and says so + +`pub module test;` now parses. Specs gained: **zero**. The one file carrying the +construct advanced from line 4 to line 21 and stopped on `**`. + +The temptation is to find a number that sounds like progress -- "an obstacle +removed", "17 lines further". The honest report is the zero, then the mechanism, +then what it did move. The census agrees: the row disappeared from the queue, +and that file now classifies under whatever stops it next. diff --git a/bootstrap/src/compiler.rs b/bootstrap/src/compiler.rs index 4959a812cb..38b6b2c593 100644 --- a/bootstrap/src/compiler.rs +++ b/bootstrap/src/compiler.rs @@ -1541,6 +1541,16 @@ impl Parser { let mut module = Node::new(NodeKind::Module); // [BUG 4 FIX] Parse optional module declaration + // + // `pub module test;` -- visibility on a module declaration. `pub` is a + // modifier the declaration parser already reads for fn/struct/const, + // and a module is the one declaration where it was not accepted, so + // the file stopped on the `module` keyword itself. A module has no + // visibility to record here, so the modifier is consumed and dropped; + // when modules gain visibility this is where it goes. + if self.current.kind == TokenKind::KwPub && self.peek.kind == TokenKind::KwModule { + self.advance(); // consume 'pub' + } if self.current.kind == TokenKind::KwModule { self.advance(); // consume 'module' // Module name can contain hyphens: e.g. "tritype-base" diff --git a/bootstrap/stage0/FROZEN_HASH b/bootstrap/stage0/FROZEN_HASH index d46da873fc..4f63abe4b4 100644 --- a/bootstrap/stage0/FROZEN_HASH +++ b/bootstrap/stage0/FROZEN_HASH @@ -1 +1 @@ -7d2ffcd4a5c3f62bdcbf89ff9f816aa948d44e778a6fc87f546fcaf7db7600df +f569da0c6c1415111f74b62680370272b7542ef1fe7f66c68267cbd71f2806df diff --git a/cli/tri/src/unparsed.rs b/cli/tri/src/unparsed.rs index bf99c72f3e..81bac72708 100644 --- a/cli/tri/src/unparsed.rs +++ b/cli/tri/src/unparsed.rs @@ -1,29 +1,43 @@ -//! Specs the compiler cannot read, ranked by the CONSTRUCT that stops it. +//! Specs the compiler cannot read, ranked by the CONSTRUCT that stops it -- +//! and every construct backed by a live probe. //! -//! WHY THIS EXISTS -//! --------------- -//! The obvious census groups by the compiler's message. That census is wrong, -//! and it was shipped once: `import x`, `algorithm y {`, `type T = T`, -//! `impl X {` and an English sentence all print -//! "unexpected token after expression statement: Ident". The message names the -//! state the parser recovered INTO, not what it choked on, so grouping by it -//! reported five different defects as one 23-strong "parser gap". +//! WHY A PROBE +//! ----------- +//! The obvious census groups by the compiler's message, and that census was +//! shipped once and was wrong: `import x`, `algorithm y {`, `type T = T` and an +//! English sentence all print "unexpected token after expression statement". +//! The message names the state the parser recovered INTO, not what stopped it. //! -//! Grouping by what the line CONTAINS gives a work queue instead: on the day -//! this was written the top rows were path-qualified module names (9) and -//! body-less function prototypes (9), and the first of those was one grammar -//! change worth six specs. +//! Grouping by what the line CONTAINS is better and still not enough, because a +//! pattern can name a construct the compiler already supports. Measured the day +//! this was written, SIX candidates -- read off real failing lines, and every +//! one plausible -- compile in isolation: +//! +//! @trim("x", "y") builtin call ACCEPTED +//! .anthropic enum literal ACCEPTED +//! if (c) { 1 } else { 2 } if-expression ACCEPTED +//! *Foo &Foo pointer / reference ACCEPTED +//! []const u8 const-qualified slice ACCEPTED +//! for (s) |v| { } capture in a for-loop ACCEPTED +//! +//! An earlier fan-out named `[]const u8` as a cause. A census that repeated it +//! would have sent someone to implement a feature that is already there. +//! +//! So every construct here carries a MINIMAL SOURCE, and the census names it +//! only while the compiler rejects that source today. It is self-invalidating +//! on purpose: when a construct gains support its probe starts passing and the +//! row disappears without anyone editing a list. //! //! WHAT IT ABSTAINS ON //! ------------------- -//! When the failing line begins with a construct the top level ACCEPTS -- `fn`, -//! `pub`, `struct`, `const` -- the line is a symptom and the cause is upstream -//! of it. Roughly two thirds of the failures land there. Naming a construct in -//! that case would be inventing one, so the row says so and stops. +//! A failing line carrying no probed construct, whose head is a construct the +//! parser accepts, is a SYMPTOM -- the cause is earlier in the file. Naming one +//! would be inventing it. Files under `fixtures/` are broken ON PURPOSE as +//! detector inputs and are counted on their own line, never as debt. use anyhow::Result; use clap::Subcommand; use std::collections::BTreeMap; -use std::path::PathBuf; +use std::path::{Path, PathBuf}; #[derive(Subcommand)] pub enum UnparsedCmd { @@ -33,71 +47,281 @@ pub enum UnparsedCmd { #[arg(long)] list: bool, }, + /// Run every construct's minimal source and say which the compiler rejects. + /// + /// This is the census's own control. A row it names must fail here; a row + /// that passes here is a feature the compiler has, and naming it would send + /// someone to build what already exists. + Probe, } -/// (name, matcher) in priority order -- first match wins. +/// A construct, the shape that spots it, and TWO minimal sources. /// -/// Ordered so that the more specific shape is tested first: a body-less `fn` -/// prototype must be recognised before the accepted-keyword abstention, or it -/// disappears into "cause is upstream" and the largest actionable row with it. -fn classify(line: &str) -> Option<&'static str> { +/// `probe` is what the compiler must reject for the row to be named. +/// `counter` is a near-identical source it must ACCEPT, and on which the +/// matcher must stay silent. The counter is what makes the row honest: without +/// it, `is_use` fired on every `use` line while `use a::b;` compiles fine and +/// only `use a::b as C;` does not. The probe alone could not catch that -- the +/// probe was the aliased form and it did fail. +/// +/// `deliberate` names a refusal the repository DECIDED on, with its citation. +/// Such a row is not work; it is a position. Casts to `float` are refused +/// because no backend lowers float arithmetic, argued at length in +/// `bootstrap/src/compiler.rs` beside `VALID_CAST_TYPES`. A work queue that +/// listed it would send someone to undo a decision. +struct Construct { + name: &'static str, + probe: &'static str, + counter: Option<&'static str>, + deliberate: Option<&'static str>, + matches: fn(&str) -> bool, +} + +/// Types the cast operator already accepts. Measured: `1 as u32` compiles, +/// `1 as float` and `1 as gf16::GF16` do not -- the defect is the TARGET type, +/// and a rule keyed on `as` alone would name every cast in the tree. +const PRIMITIVE: [&str; 14] = [ + "u8", "u16", "u32", "u64", "i8", "i16", "i32", "i64", "f32", "f64", "bool", "str", "usize", + "trit", +]; + +fn head(line: &str) -> &str { let t = line.trim(); - let head = t.strip_prefix("pub ").unwrap_or(t); + t.strip_prefix("pub ").unwrap_or(t) +} - // `fn f(a: T) -> U;` -- a signature with no body. - if head.starts_with("fn ") && t.ends_with(';') { - return Some("fn NAME(..) -> T; body-less prototype"); - } - // `struct Id(str);` -- a tuple/newtype struct. - if head.starts_with("struct ") && t.ends_with(");") { - return Some("struct NAME(T); tuple / newtype struct"); - } - // `module a::b { }` / `module a::b;` - if head.starts_with("module ") && head.contains("::") { - return Some("module a::b path-qualified module name"); - } - if head.starts_with("import ") { - return Some("import .. import statement"); - } - if head.starts_with("use ") { - return Some("use .. use declaration"); - } - if head.starts_with("trait ") { - return Some("trait NAME trait declaration"); - } - if head.starts_with("impl ") { - return Some("impl NAME impl block"); - } - if head.starts_with("algorithm ") { - return Some("algorithm NAME { algorithm block"); - } - if head.starts_with("type ") { - return Some("type T = U type alias"); - } - if t.starts_with("\\\\") { - return Some("\\\\ ... Zig multiline string block"); +/// `if (opt) |v| ..` -- a payload capture in an IF. +/// +/// Not `|` alone: `1 | 2` compiles. Not a capture anywhere: `for (s) |v|` +/// compiles too. It is the if-expression form and only that. +fn is_if_capture(l: &str) -> bool { + l.contains("if (") && l.contains(") |") +} + +/// `for (xs, 0..) |v|` -- an open-ended range in a for-header. +fn is_for_range(l: &str) -> bool { + l.contains("for (") && l.contains("..") && l.contains(") |") +} + +/// ` as T` where T is not one of the primitives the cast already accepts. +fn is_cast_to_non_primitive(l: &str) -> bool { + let Some(i) = l.find(" as ") else { + return false; + }; + let word: String = l[i + 4..] + .trim_start() + .chars() + .take_while(|c| c.is_alphanumeric() || *c == '_' || *c == ':') + .collect(); + !word.is_empty() && !PRIMITIVE.contains(&word.as_str()) +} + +/// `fn name(..)` -- type parameters on a FUNCTION. +/// +/// Not a generic type in a signature: `fn a(k: Result)` compiles. The +/// angle bracket has to sit between the name and the parameter list. +fn is_generic_fn(l: &str) -> bool { + let Some(rest) = head(l).strip_prefix("fn ") else { + return false; + }; + match (rest.find('<'), rest.find('(')) { + (Some(lt), Some(lp)) => lt < lp, + _ => false, } - // A macro call: an identifier immediately followed by `!(`. - if let Some(i) = t.find("!(") { - if t[..i] - .chars() - .rev() - .take_while(|c| c.is_alphanumeric() || *c == '_') - .count() - > 0 - { - return Some("name!(..) Rust-style macro invocation"); +} + +/// `[K: V]` -- a map TYPE. Not `[T]`, an array type, which compiles. +fn is_map_type(l: &str) -> bool { + let b = l.as_bytes(); + let mut i = 0; + while i < b.len() { + if b[i] == b'[' { + if let Some(off) = l[i..].find(']') { + let inner = &l[i + 1..i + off]; + if inner.contains(':') && !inner.contains(',') && !inner.trim().is_empty() { + return true; + } + i += off; + } } + i += 1; } - None + false } -/// Constructs the parser accepts -- at the top level or inside a body. A -/// failing line that starts with one of these is a symptom; the cause is -/// earlier in the file. -/// -/// The statement keywords were missing at first and 15 failures fell into -/// "not decided" that were plainly upstream: `return`, `let`, `}`. +fn is_prototype(l: &str) -> bool { + head(l).starts_with("fn ") && l.trim_end().ends_with(';') +} +fn is_tuple_struct(l: &str) -> bool { + head(l).starts_with("struct ") && l.trim_end().ends_with(");") +} +fn is_pub_module(l: &str) -> bool { + l.trim().starts_with("pub module ") +} +fn is_path_module(l: &str) -> bool { + head(l).starts_with("module ") && head(l).contains("::") +} +fn is_import(l: &str) -> bool { + head(l).starts_with("import ") +} +/// `use a::b as C;` -- an ALIASED use. Plain `use a::b;` and `using a::b;` +/// both compile, so a rule keyed on `use` alone names a feature that exists. +fn is_use(l: &str) -> bool { + head(l).starts_with("use ") && l.contains(" as ") +} +fn is_trait(l: &str) -> bool { + head(l).starts_with("trait ") +} +fn is_impl(l: &str) -> bool { + head(l).starts_with("impl ") +} +fn is_algorithm(l: &str) -> bool { + head(l).starts_with("algorithm ") +} +fn is_type_alias(l: &str) -> bool { + head(l).starts_with("type ") && l.contains('=') +} +fn is_zig_block(l: &str) -> bool { + l.trim_start().starts_with("\\\\") +} +fn is_macro(l: &str) -> bool { + let Some(i) = l.find("!(") else { + return false; + }; + l[..i] + .chars() + .next_back() + .is_some_and(|c| c.is_alphanumeric() || c == '_') +} + +/// Ordered most specific first: a body-less `fn` prototype has to be matched +/// before anything keyed on `fn`, or the largest actionable row vanishes into +/// the abstention. +const CONSTRUCTS: &[Construct] = &[ + Construct { + name: "fn NAME(..) -> T; body-less prototype", + probe: "fn a(x: u32) -> u32;\n", + counter: Some("fn a(x: u32) -> u32 { return x; }\n"), + deliberate: None, + matches: is_prototype, + }, + Construct { + name: "struct NAME(T); tuple / newtype struct", + probe: "struct Id(str);\n", + counter: Some("struct Id { x: str }\n"), + deliberate: None, + matches: is_tuple_struct, + }, + Construct { + name: "fn NAME(..) type parameters on a function", + probe: "fn a(k: T) -> u32 { return 1; }\n", + counter: Some("fn a(k: Result) -> u32 { return 1; }\n"), + deliberate: None, + matches: is_generic_fn, + }, + Construct { + name: "[K: V] map type", + probe: "fn a() -> u32 {\n var m: [str: str] = { \"a\": \"b\" };\n return 1;\n}\n", + counter: Some("fn a() -> u32 {\n var m: [str] = [ \"a\" ];\n return 1;\n}\n"), + deliberate: None, + matches: is_map_type, + }, + Construct { + name: "if (o) |v| .. payload capture in an if", + probe: "fn a() -> u32 {\n const x = if (o) |v| v else 0;\n return 1;\n}\n", + counter: Some("fn a() -> u32 {\n for (s) |v| { }\n return 1;\n}\n"), + deliberate: None, + matches: is_if_capture, + }, + Construct { + name: "for (xs, 0..) |v| open-ended range in a for", + probe: "fn a() -> u32 {\n for (s, 0..) |op| { }\n return 1;\n}\n", + counter: Some("fn a() -> u32 {\n for (s) |v| { }\n return 1;\n}\n"), + deliberate: None, + matches: is_for_range, + }, + Construct { + name: "x as T cast to a non-primitive type", + probe: "fn a() -> u32 { return 1 as float; }\n", + counter: Some("fn a() -> u32 { return 1 as u32; }\n"), + deliberate: Some("no backend lowers float arithmetic -- see VALID_CAST_TYPES in bootstrap/src/compiler.rs"), + matches: is_cast_to_non_primitive, + }, + Construct { + name: "pub module N; visibility on a module", + probe: "pub module test;\nfn a() -> u32 { return 1; }\n", + counter: Some("module test;\nfn a() -> u32 { return 1; }\n"), + deliberate: None, + matches: is_pub_module, + }, + Construct { + name: "module a::b path-qualified module name", + probe: "module a::b {\n fn x() -> u32 { return 1; }\n}\n", + counter: None, + deliberate: None, + matches: is_path_module, + }, + Construct { + name: "import .. import statement", + probe: "module m {\n import a::b;\n}\n", + counter: None, + deliberate: None, + matches: is_import, + }, + Construct { + name: "use .. use declaration", + probe: "use a::b as C;\nfn a() -> u32 { return 1; }\n", + counter: Some("use a::b;\nfn a() -> u32 { return 1; }\n"), + deliberate: None, + matches: is_use, + }, + Construct { + name: "trait NAME trait declaration", + probe: "trait T { fn a() -> u32; }\n", + counter: None, + deliberate: None, + matches: is_trait, + }, + Construct { + name: "impl NAME impl block", + probe: "impl T { fn a() -> u32 { return 1; } }\n", + counter: None, + deliberate: None, + matches: is_impl, + }, + Construct { + name: "algorithm NAME { algorithm block", + probe: "algorithm foo {\n x: 1\n}\n", + counter: None, + deliberate: None, + matches: is_algorithm, + }, + Construct { + name: "type T = U type alias", + probe: "type T = u32;\nfn a() -> u32 { return 1; }\n", + counter: None, + deliberate: None, + matches: is_type_alias, + }, + Construct { + name: "\\\\ ... Zig multiline string block", + probe: "fn a() -> u32 {\n const s =\n \\\\pub fn test() {}\n ;\n return 1;\n}\n", + counter: None, + deliberate: None, + matches: is_zig_block, + }, + Construct { + name: "name!(..) Rust-style macro invocation", + probe: "fn a() -> u32 { assert_eq!(1, 1); return 1; }\n", + counter: Some("fn a() -> u32 { return 1; }\n"), + deliberate: None, + matches: is_macro, + }, +]; + +/// Constructs the parser accepts, at the top level or inside a body. A failing +/// line that starts with one of these and carries no probed construct is a +/// symptom; the cause is earlier in the file. const ACCEPTED: [&str; 17] = [ "fn", "struct", @@ -118,25 +342,21 @@ const ACCEPTED: [&str; 17] = [ "try", ]; -/// A file under `fixtures/` is BROKEN ON PURPOSE -- it is the reference input -/// for a detector, not debt. `tools/specs_generate_baseline.txt` already omits -/// all 21 of them; a census that counts them disagrees with the repository's -/// own ledger. They are printed on their own line rather than dropped, because -/// a number that silently excludes something is the defect this file exists to -/// avoid. -fn is_fixture(path: &str) -> bool { - path.contains("/fixtures/") -} - fn accepted_head(line: &str) -> bool { - let t = line.trim(); - let t = t.strip_prefix("pub ").unwrap_or(t); + let t = head(line); ACCEPTED.iter().any(|k| { t.strip_prefix(k) .is_some_and(|r| r.starts_with(|c: char| c.is_whitespace() || c == '(')) }) } +/// A file under `fixtures/` is BROKEN ON PURPOSE -- the reference input for a +/// detector, not debt. `tools/specs_generate_baseline.txt` omits all of them; a +/// census that counts them disagrees with the repository's own ledger. +fn is_fixture(path: &str) -> bool { + path.contains("/fixtures/") +} + fn line_of(text: &str) -> Option { let i = text.find("line ")?; let rest = &text[i + 5..]; @@ -146,18 +366,139 @@ fn line_of(text: &str) -> Option { rest[..end].parse().ok() } -pub fn run(cmd: &UnparsedCmd, root: PathBuf) -> Result<()> { - let UnparsedCmd::Report { list } = cmd; - let t27c = ["target/release/t27c", "target/debug/t27c"] +/// Run every probe. `true` means the compiler REJECTS it today, so the row may +/// be named. +fn run_probes(t27c: &Path, root: &Path) -> Vec { + let dir = std::env::temp_dir().join("tri-unparsed-probes"); + let _ = std::fs::create_dir_all(&dir); + CONSTRUCTS + .iter() + .enumerate() + .map(|(i, c)| { + let f = dir.join(format!("probe{i}.t27")); + if std::fs::write(&f, c.probe).is_err() { + return false; + } + let rejected = std::process::Command::new(t27c) + .arg("check") + .arg(&f) + .current_dir(root) + .output() + .map(|o| !o.status.success()) + .unwrap_or(false); + let _ = std::fs::remove_file(&f); + rejected + }) + .collect() +} + +/// Run every counter. `true` means the compiler still ACCEPTS it, which is the +/// state the row's boundary depends on. +fn run_counters(t27c: &Path, root: &Path) -> Vec { + let dir = std::env::temp_dir().join("tri-unparsed-counters"); + let _ = std::fs::create_dir_all(&dir); + CONSTRUCTS + .iter() + .enumerate() + .map(|(i, c)| { + let Some(src) = c.counter else { return true }; + let f = dir.join(format!("counter{i}.t27")); + if std::fs::write(&f, src).is_err() { + return true; + } + let ok = std::process::Command::new(t27c) + .arg("check") + .arg(&f) + .current_dir(root) + .output() + .map(|o| o.status.success()) + .unwrap_or(false); + let _ = std::fs::remove_file(&f); + ok + }) + .collect() +} + +fn compiler(root: &Path) -> Result { + ["target/release/t27c", "target/debug/t27c"] .iter() .map(|p| root.join(p)) - .find(|p| p.is_file()); - let Some(t27c) = t27c else { - anyhow::bail!( - "no compiler -- the census asks it which line stops it, and its\n \ - absence is not a clean bill.\n cargo build --release -p t27c" + .find(|p| p.is_file()) + .ok_or_else(|| { + anyhow::anyhow!( + "no compiler -- every row here is a claim about what it rejects,\n \ + and its absence is not a clean bill.\n cargo build --release -p t27c" + ) + }) +} + +pub fn run(cmd: &UnparsedCmd, root: PathBuf) -> Result<()> { + let t27c = compiler(&root)?; + + if matches!(cmd, UnparsedCmd::Probe) { + let rejected = run_probes(&t27c, &root); + let counters = run_counters(&t27c, &root); + let n = rejected.iter().filter(|r| **r).count(); + let bad_counters: Vec<&str> = CONSTRUCTS + .iter() + .zip(&counters) + .filter(|(c, ok)| c.counter.is_some() && !**ok) + .map(|(c, _)| c.name) + .collect(); + println!(" constructs probed {}", CONSTRUCTS.len()); + println!(" ... the compiler REJECTS {n}"); + println!(" ... the compiler ACCEPTS {}", CONSTRUCTS.len() - n); + println!( + " counters that still compile {} of {}", + counters + .iter() + .zip(CONSTRUCTS) + .filter(|(ok, c)| **ok && c.counter.is_some()) + .count(), + CONSTRUCTS.iter().filter(|c| c.counter.is_some()).count() ); + println!(); + for (c, r) in CONSTRUCTS.iter().zip(&rejected) { + let tag = if *r { "rejected" } else { "ACCEPTED" }; + let note = match c.deliberate { + Some(_) => " <- refused ON PURPOSE, not work", + None => "", + }; + println!(" {tag} {}{note}", c.name); + } + if !bad_counters.is_empty() { + println!(); + println!(" A COUNTER NO LONGER COMPILES. The row claims the compiler accepts"); + println!(" a near-identical source, and it does not -- so the row's boundary"); + println!(" is wrong, not the compiler:"); + for b in &bad_counters { + println!(" {b}"); + } + return Err(anyhow::anyhow!( + "{} counter(s) stopped compiling", + bad_counters.len() + )); + } + if n < CONSTRUCTS.len() { + println!(); + println!(" An ACCEPTED row is a construct this repository already supports."); + println!(" `report` will not name it, whatever the failing line looks like:"); + println!(" naming it would send someone to build what is already there."); + } + return Ok(()); + } + + let UnparsedCmd::Report { list } = cmd else { + unreachable!() }; + let rejected = run_probes(&t27c, &root); + let live: Vec<&Construct> = CONSTRUCTS + .iter() + .zip(&rejected) + .filter(|(_, r)| **r) + .map(|(c, _)| c) + .collect(); + let retired = CONSTRUCTS.len() - live.len(); let out = std::process::Command::new("git") .args(["ls-files", "*.t27"]) @@ -170,22 +511,18 @@ pub fn run(cmd: &UnparsedCmd, root: PathBuf) -> Result<()> { .collect(); let mut by: BTreeMap<&'static str, Vec> = BTreeMap::new(); - let mut upstream = 0usize; + let (mut upstream, mut fixtures, mut unlocated, mut failing) = (0usize, 0usize, 0usize, 0usize); let mut unnamed: Vec<(String, String)> = Vec::new(); - let mut failing = 0usize; - let mut fixtures = 0usize; - // The rows that fall out of every bucket. Counted, because 36 + 27 + 30 - // came to 93 against a total of 97 and the four were leaving through a - // bare `continue`. - let mut unlocated = 0usize; for spec in &specs { - let o = std::process::Command::new(&t27c) + let Ok(o) = std::process::Command::new(&t27c) .arg("check") .arg(spec) .current_dir(&root) - .output(); - let Ok(o) = o else { continue }; + .output() + else { + continue; + }; if o.status.success() { continue; } @@ -195,11 +532,7 @@ pub fn run(cmd: &UnparsedCmd, root: PathBuf) -> Result<()> { } failing += 1; let text = String::from_utf8_lossy(&o.stderr) + String::from_utf8_lossy(&o.stdout); - let Some(n) = line_of(&text) else { - unlocated += 1; - continue; - }; - let Ok(src) = std::fs::read_to_string(root.join(spec)) else { + let (Some(n), Ok(src)) = (line_of(&text), std::fs::read_to_string(root.join(spec))) else { unlocated += 1; continue; }; @@ -209,8 +542,8 @@ pub fn run(cmd: &UnparsedCmd, root: PathBuf) -> Result<()> { continue; } let line = lines[n - 1]; - match classify(line) { - Some(k) => by.entry(k).or_default().push(spec.clone()), + match live.iter().find(|c| (c.matches)(line)) { + Some(c) => by.entry(c.name).or_default().push(spec.clone()), None if accepted_head(line) => upstream += 1, None => unnamed.push((spec.clone(), line.trim().chars().take(52).collect())), } @@ -219,27 +552,38 @@ pub fn run(cmd: &UnparsedCmd, root: PathBuf) -> Result<()> { let named: usize = by.values().map(|v| v.len()).sum(); println!(" specs tracked {}", specs.len()); println!(" ... the compiler cannot read {failing}"); - println!(" ... construct NAMED on that line {named}"); + println!(" ... construct NAMED and PROBED {named}"); println!(" ... cause is UPSTREAM, not named {upstream}"); println!(" ... not decided, nothing claimed {}", unnamed.len()); if unlocated > 0 { - println!(" ... error names no readable line {unlocated}"); + println!(" ... error names no readable line {unlocated}"); } if fixtures > 0 { - println!(" broken ON PURPOSE under fixtures/ {fixtures} (detector inputs, not debt)"); + println!(" broken ON PURPOSE under fixtures/ {fixtures} (detector inputs, not debt)"); + } + if retired > 0 { + println!(" constructs the compiler now ACCEPTS {retired} (probed; not named)"); } if by.is_empty() { println!(); - println!(" No failing line carries a construct this census recognises."); + println!(" No failing line carries a construct whose probe still fails."); return Ok(()); } + let deliberate: std::collections::BTreeMap<&str, &str> = CONSTRUCTS + .iter() + .filter_map(|c| c.deliberate.map(|d| (c.name, d))) + .collect(); let mut rows: Vec<(&&str, &Vec)> = by.iter().collect(); rows.sort_by(|a, b| b.1.len().cmp(&a.1.len()).then(a.0.cmp(b.0))); + let (work, decided): (Vec<_>, Vec<_>) = rows + .iter() + .partition(|(k, _)| !deliberate.contains_key(**k)); + println!(); - println!(" work queue -- one grammar change per row, largest first"); - for (k, v) in &rows { + println!(" work queue -- every row proved unsupported by its own probe"); + for (k, v) in &work { println!(" {:>4} {k}", v.len()); if *list { for s in v.iter() { @@ -248,13 +592,29 @@ pub fn run(cmd: &UnparsedCmd, root: PathBuf) -> Result<()> { } } + if !decided.is_empty() { + println!(); + println!(" refused ON PURPOSE -- a position, not a gap. Listed so it is not"); + println!(" mistaken for work, and so the reason is at hand when it is revisited."); + for (k, v) in &decided { + println!(" {:>4} {k}", v.len()); + println!(" {}", deliberate[**k]); + if *list { + for s in v.iter() { + println!(" {s}"); + } + } + } + } + + println!(); + println!(" The UPSTREAM count is not a residue to be reduced: those lines are"); + println!(" `fn`, `struct`, `const` carrying no probed construct. The defect is"); + println!(" earlier in the file and this census will not guess it."); println!(); - println!(" The UPSTREAM count is not a residue to be reduced: those lines"); - println!(" are `fn`, `struct`, `const` -- constructs the parser accepts. The"); - println!(" defect is earlier in the file and this census will not guess it."); + println!(" `tri unparsed probe` runs the minimal source behind each row."); if !*list { - println!(); - println!(" --list names the specs under each row."); + println!(" `--list` names the specs under each row."); } Ok(()) } @@ -263,63 +623,133 @@ pub fn run(cmd: &UnparsedCmd, root: PathBuf) -> Result<()> { mod tests { use super::*; + // Each negative here is a MEASUREMENT: the construct compiles in isolation, + // so a matcher that fires on it would name a feature the compiler has. #[test] - fn a_prototype_is_named_before_the_abstention_swallows_it() { - // `fn` is an ACCEPTED top-level keyword, so an ordering that tests the - // abstention first loses the largest actionable row entirely. - assert_eq!( - classify("pub fn poll(x: [str]) -> Result;"), - Some("fn NAME(..) -> T; body-less prototype") - ); - assert!(accepted_head("pub fn poll(x: [str]) -> Result;")); + fn bitwise_or_is_not_a_capture() { + assert!(!is_if_capture(" return 1 | 2;")); + assert!(is_if_capture(" const x = if (o) |v| v else 0;")); } #[test] - fn a_function_with_a_body_is_not_a_prototype() { - assert_eq!(classify("pub fn poll(x: u32) -> bool {"), None); + fn a_capture_in_a_for_loop_is_not_the_if_form() { + // `for (s) |v| { }` compiles; only the if-expression form does not. + assert!(!is_if_capture(" for (s) |v| { }")); + assert!(!is_for_range(" for (s) |v| { }")); + assert!(is_for_range(" for (delta.operations, 0..) |op| {")); } #[test] - fn a_plain_module_is_not_a_path() { - assert_eq!(classify("module Foo {"), None); - assert_eq!( - classify("module github::auth {"), - Some("module a::b path-qualified module name") - ); + fn a_primitive_cast_is_not_named() { + // `1 as u32` compiles; `1 as float` and `1 as gf16::GF16` do not. + assert!(!is_cast_to_non_primitive("return 1 as u32;")); + assert!(is_cast_to_non_primitive("return 1 as float;")); + assert!(is_cast_to_non_primitive("return x as gf16::GF16;")); + } + + #[test] + fn a_generic_type_in_a_signature_is_not_a_generic_function() { + // `fn a(k: Result)` compiles; `fn a(k: T)` does not. + assert!(!is_generic_fn("fn a(k: Result) -> u32 {")); + assert!(is_generic_fn("fn read(key: [str]) -> Result {")); + } + + #[test] + fn an_array_type_is_not_a_map_type() { + // `[str]` compiles; `[str: str]` does not. + assert!(!is_map_type(" var m: [str] = [ \"a\" ];")); + assert!(is_map_type( + " var env: [str: str] = { \"PATH\": \"/usr/bin\" };" + )); + // A list of two things is not a key-value pair. + assert!(!is_map_type(" const v = arr[a, b];")); + } + + #[test] + fn a_prototype_is_named_before_the_abstention_swallows_it() { + assert!(is_prototype("pub fn poll(x: [str]) -> Result;")); + assert!(accepted_head("pub fn poll(x: [str]) -> Result;")); + assert!(!is_prototype("pub fn poll(x: u32) -> bool {")); } #[test] fn a_sentence_containing_a_keyword_is_not_that_keyword() { - // "importantly" starts with "import"; "typed" starts with "type". - assert_eq!(classify("importantly, the bridge is read-only."), None); - assert_eq!(classify("typed values flow through the VM."), None); + assert!(!is_import("importantly, the bridge is read-only.")); + assert!(!is_type_alias("typed values flow through the VM.")); assert!(!accepted_head("constant folding is described here")); } + #[test] + fn a_macro_needs_a_name_in_front_of_the_bang() { + assert!(is_macro("assert_eq!(a, b);")); + assert!(!is_macro("if x != (a) {")); + } + #[test] fn a_fixture_is_not_debt() { assert!(is_fixture( "bootstrap/tests/fixtures/damage/damage_class_01.t27" )); assert!(!is_fixture("specs/github/auth.t27")); - // The word must be a PATH SEGMENT, not a substring of a file name. assert!(!is_fixture("specs/tools/fixtures_report.t27")); } #[test] fn a_statement_keyword_abstains_like_a_top_level_one() { for l in ["return x;", "let y = 1;", " for (a) |b| {"] { - assert_eq!(classify(l), None, "{l}"); assert!(accepted_head(l), "{l}"); } } + // Every probe must be distinct: two rows sharing a source would report the + // same reading twice and hide one of them. #[test] - fn a_macro_needs_a_name_in_front_of_the_bang() { - assert_eq!( - classify("assert_eq!(a, b);"), - Some("name!(..) Rust-style macro invocation") - ); - assert_eq!(classify("if x != (a) {"), None); + fn every_probe_is_distinct() { + let mut seen = std::collections::BTreeSet::new(); + for c in CONSTRUCTS { + assert!(seen.insert(c.probe), "duplicate probe: {}", c.name); + } + } + + // A matcher must fire on its own probe, or the row can never be named. + #[test] + fn every_matcher_fires_on_its_own_probe() { + for c in CONSTRUCTS { + let hit = c.probe.lines().any(|l| (c.matches)(l)); + assert!(hit, "matcher never fires on its own probe: {}", c.name); + } + } + + // ...and must stay SILENT on the counter, which the compiler accepts. + // + // This is the half the probe cannot check. `is_use` fired on every `use` + // line while only the aliased form fails; its probe WAS the aliased form, + // so the probe passed and the matcher was still wrong. + #[test] + fn no_matcher_fires_on_its_counter() { + for c in CONSTRUCTS { + let Some(ctr) = c.counter else { continue }; + for l in ctr.lines() { + assert!( + !(c.matches)(l), + "matcher fires on a source the compiler ACCEPTS: {} -- {l}", + c.name + ); + } + } + } + + // A deliberate refusal must cite where the decision is written down. + #[test] + fn a_deliberate_refusal_carries_its_citation() { + for c in CONSTRUCTS { + if let Some(why) = c.deliberate { + assert!( + why.contains(".rs") || why.contains(".md"), + "refusal without a citation: {}", + c.name + ); + } + } } } diff --git a/docs/now/2026-08-30-a-census-that-proves-its-own-categories.md b/docs/now/2026-08-30-a-census-that-proves-its-own-categories.md new file mode 100644 index 0000000000..3dca11f604 --- /dev/null +++ b/docs/now/2026-08-30-a-census-that-proves-its-own-categories.md @@ -0,0 +1,9 @@ +# NOW -- A census that proves its own categories (2026-08-30) + +## A census that proves its own categories (Refs #2864) + +- tri unparsed now carries a PROBE per construct -- a minimal source the compiler must reject for the row to be named. Six candidates read off real failing lines turned out to compile in isolation: @trim(), .anthropic, if-as-expression, *Foo, &Foo and []const u8. An earlier fan-out had named []const u8 as a cause. +- And a COUNTER per construct -- a near-identical source the compiler ACCEPTS, on which the matcher must stay silent. That is the half the probe cannot check: is_use fired on every use line while only 'use a::b as C;' fails, and its probe WAS the aliased form, so the probe passed and the matcher was still wrong. +- A third state: refused ON PURPOSE. Casts to non-primitive types are a documented position (no backend lowers float arithmetic, argued beside VALID_CAST_TYPES), not a gap. Three specs moved out of the work queue and into a section that carries the citation. +- pub module N; accepted -- pub is a modifier the declaration parser already reads for fn/struct/const, and a module was the one place it was not. Zero specs gained: the single file carrying it advanced 17 lines and stopped on '**'. Said plainly rather than counted as a win. +- The design is self-invalidating and proved it twice in one run: module a::b and pub module N; both flipped to ACCEPTED and left the queue without anyone editing a list.