Skip to content

Latest commit

 

History

History
454 lines (363 loc) · 26.3 KB

REVIEW.md

File metadata and controls

454 lines (363 loc) · 26.3 KB

Review

Read this when reviewing a change to this repository. It carries the part of each principle that no rule here can decide -- the judgment -- and nothing that a rule already refuses.

Do NOT re-enforce the static rules. They run on every change and they run first; repeating their findings costs a reviewer's attention and buys a second opinion nobody asked for. The rules already active here are:

  • catalog-reference-current
  • catalog-tests
  • catalog-validate
  • no-local-merge
  • no-merge-commit
  • no-private-repo-names
  • no-running-os-identity-metadata
  • no-stale-hook-pins
  • prevent-ai-author
  • prevent-public-push
  • removed-function-named
  • review-current
  • workflow-declares-permissions

Everything below is a constraint whose remainder is a judgment. For each, the claim is what it asserts, the scope says when it binds, and the questions are what to ask of a change.

Complete Mediation

Authority must be verified on every access to a protected object or effect, on every path that can reach it, and not only on the path the designer had in mind.

Applies when

  • A protected effect is reachable through more than one interface, tier, or tool.
  • Caching, delegation, or convenience wrappers can shortcut an earlier authorization decision.
  • New surfaces are added over time to a system whose checks were written for the original surface.
  • A request's network location or origin is being read as its authorization, as inside a perimeter.

Ask

  • What are all the paths that reach this effect, including manual, scripted, and agent-driven ones?
  • Which of those paths meets the control, and what evidence shows it?
  • When a new interface is added, does it inherit the check or does someone have to remember to add it?
  • Is the same rule body reached from every seam, or has it been reimplemented per seam?

Confused Deputy

A program that holds authority of its own and acts on a resource a less privileged requester designated, checking only its own authority, becomes an instrument through which the requester reaches what it could not reach directly.

Applies when

  • A program runs with privileges its callers lack, such as a setuid binary, a CI job holding a write token, or a service account.
  • An agent acts for a user and chooses targets from input that the user or a third party supplied.
  • A shim or hook intercepts a command, interprets its arguments, and performs effects with its own credentials or configuration.
  • Input that names a resource, such as a path, URL, repository, or recipient, arrives from a less trusted party.

Ask

  • Whose authority does this code exercise when it acts on a name it was handed?
  • Which inputs designate a resource, and who supplied them?
  • Could a caller, or text a caller controls, direct this deputy at something the caller cannot reach itself?
  • When an agent or shim acts for a user, is the target checked against the user's authority or only against its own?

Conway's Law

Any organization that designs a system will produce a design whose structure is a copy of the organization's communication structure.

Applies when

  • More than one team, site, or vendor contributes to the system.
  • The cost of communication differs substantially between parts of the organization.
  • A proposed architecture draws boundaries that do not line up with team boundaries.
  • A reorganization is planned or has just happened, and the architecture has not caught up.

Ask

  • Which teams will build and change the components on each side of this interface, and how often do they talk?
  • Where does the intended architecture cross a team boundary it cannot sustain, or ignore one it cannot remove?
  • Did a recent reorganization change communication paths this design assumes?
  • Is a team change being proposed in order to reshape the architecture, and is that stated as its intent?

Defense in Depth

Protect critical assets and actions with multiple controls that fail differently and cover prevention, detection, containment, and recovery.

Applies when

  • Consequences justify more than one control.
  • Controls can be made meaningfully independent.
  • Detection and recovery matter in addition to prevention.

Ask

  • How does each layer fail differently?
  • Which layer detects failure of another?
  • Can the layers be tested independently and end to end?
  • When this layer fails, what is the blast radius, and which layer contains it?

Economy of Mechanism

A mechanism whose errors stay invisible in normal use must be small and simple enough to verify completely, because its flaws surface only under the attack or accident it exists to stop.

Applies when

  • The mechanism is a guard, parser, or policy evaluator whose wrong acceptance is not observable in ordinary use.
  • Assurance has to come from review, exhaustive testing, or proof rather than from field experience.
  • Features, flags, or configuration keys are proposed for a component that other components trust.

Ask

  • Which part of this change is trusted to refuse, and how large is it after the change?
  • Could every path through the mechanism be enumerated in review, and was it?
  • Does a new option belong inside the protected mechanism, or can it live in untrusted code that calls it?
  • Is the mechanism short because it is simple, or because it hands the hard case to configuration it interprets?

End-to-End Principle

Functions requiring application-level knowledge should be implemented or verified end to end, even when lower layers provide performance or reliability assistance.

Applies when

  • The guarantee depends on application semantics.
  • Intermediate layers cannot observe all relevant state.
  • Lower-layer checks can be treated as optimization rather than proof.
  • Data crosses several hops that each check their own link, and only a checksum or signature made at the source and verified at the destination covers the whole path.

Ask

  • Which layer has enough context to prove the actual objective?
  • Is a lower-layer guarantee being mistaken for end-to-end correctness?
  • What evidence reaches the endpoint after partial failure?
  • Does the health check exercise the dependency the request actually needs?

Enforcement Needs a Trigger

A constraint becomes machine enforcement only when it is expressed as a decidable predicate over an observable subject, bound to a condition that fires it and to evidence it emits when it fires. A constraint that cannot say when it applies or what it saw is guidance, and must be shipped and read as guidance.

Applies when

  • A constraint is about to be added to a linter, hook, policy file, alert rule, or a prompt assembled for a model.
  • A rule's payload is prose rather than a match, a diff, or a state comparison.
  • Someone proposes carrying design guidance in a runtime configuration.
  • A reviewing tier is proposed for the part of a constraint no predicate captures, and the question is what stops it becoming the always-emitted prose this record refuses.

Ask

  • What input is this rule known to match, and is that input checked in?
  • What event causes this rule to run, and over which files or subjects?
  • What does it print when it refuses, and can the reader locate the cause from that alone?
  • If this rule can never fire, what in the system would reveal that?
  • For the part that cannot be mechanized, where does it live so that whoever builds the check reads it?

Fail Fast and Explicitly

When continuing cannot satisfy the contract safely, detect the condition at the earliest reliable boundary and return an explicit failure with evidence.

Applies when

  • The violated condition can be detected reliably.
  • Continuing would corrupt state, mislead callers, or increase recovery cost.
  • A caller can handle or escalate the failure.

Ask

  • Is the failure detected at the boundary that owns the invariant?
  • Would continuing preserve a truthful and safe contract?
  • Does the error include enough evidence for recovery?

Goodhart's Law

When a measure becomes a target, it ceases to be a good measure, because those it measures move the number by means that decouple it from what it was chosen to indicate.

Applies when

  • A metric gates, ranks, rewards, or reports on the people or agents who can influence it.
  • A coverage count, such as a test coverage percentage or an 'N of M rules carry a principle' figure, is published as progress and could be raised by adding annotations rather than enforcement.
  • The metric can be moved by actions that leave the underlying goal where it was.
  • An optimizing agent, human or automated, is given the metric as its objective.

Ask

  • What is the cheapest action that raises this number without advancing the goal it stands for?
  • Is this measure used to diagnose, or is someone rewarded or gated by it?
  • What evidence besides the number shows the goal moved, such as a planted fault that must be caught?
  • If an agent optimizes against this metric, what stops it from satisfying the letter and not the intent?

Graceful Degradation

When a dependency or resource fails, retain a clearly identified subset of safe and useful behavior rather than presenting total failure or false success.

Applies when

  • A useful subset can be served without violating safety or truthfulness.
  • The degraded state can be detected and communicated.
  • Recovery and reconciliation are defined.
  • Dependency failure is expected in normal operation, so the degraded path is designed and exercised in advance rather than improvised during an incident.

Ask

  • Which capabilities remain safe and truthful without this dependency?
  • How is degraded mode visible to users and operators?
  • What state must be reconciled after recovery?

High Cohesion, Low Coupling

Components should contain behavior that belongs to one coherent responsibility and depend on other components through the smallest necessary contracts.

Applies when

  • Responsibilities and change patterns can be identified.
  • Dependencies can be narrowed without duplicating authority.
  • The boundary corresponds to a meaningful lifecycle or ownership unit.
  • Packages are released and reused as units, so what changes together and what is reused together decide where a boundary goes.

Ask

  • Which responsibility unifies this component?
  • Which dependencies are essential rather than convenient?
  • Do recent changes repeatedly cross this boundary?
  • Does a caller reach through one object to a second one's internals, or depend on an interface wider than the part it uses?
  • Do the things that change together live together, and do the things reused together ship together?
  • Does a stable package depend on a volatile one, and are the packages most depended on also the most abstract?

Hyrum's Law

With a sufficient number of users of an interface, every observable behavior of it will be depended on by somebody, regardless of what the contract promises.

Applies when

  • An interface has many consumers the maintainer does not know individually, or has been stable long enough for implicit reliance to accumulate.
  • Behavior beyond the documented contract is observable: ordering, timing, error text, output formatting, defaults, identifiers.
  • Identifiers, names, and generated documents are read by consumers' configuration and tooling, which makes them a contract whether or not they are declared one.
  • A change is being judged safe on the ground that it alters only undocumented behavior.

Ask

  • What does this change alter that a consumer can observe, including behavior the contract does not mention?
  • Who consumes this interface, and which of them does the maintainer not know about?
  • Has this unpromised behavior been stable long enough that reliance has probably formed, and should it be varied deliberately instead?
  • If a consumer breaks, is there a deprecation path, or only a changelog entry?

Information Hiding

Module boundaries should conceal design decisions likely to change, exposing only the stable information required by clients.

Applies when

  • A representation, algorithm, policy, or external dependency is likely to change.
  • Clients can operate through a smaller stable contract.
  • The hidden decision has a coherent owner.
  • Clients can be written against a declared interface rather than a concrete implementation, so the implementation can be replaced behind it.
  • Layers can be stacked so that each uses only the operations of the one beneath and never its representation.

Ask

  • Which design decision is this boundary hiding?
  • What client knowledge would cause ripple effects if exposed?
  • Is diagnostic visibility preserved without making internals part of the contract?

Informed Consent for Consequential Actions

An effect that is hard to reverse and falls outside what the invoked operation denotes requires that the specific effects be disclosed in terms the user can evaluate, that declining be possible without abandoning the rest of the work, and that the recorded approval bound the action to what was shown.

Applies when

  • The effects land on data or systems the user owns and the tool does not.
  • Reversal is impossible, lossy, or needs knowledge the user does not have.
  • The effects are not entailed by the operation the user asked for.
  • A person is present to answer, or a prior approval specific to this action was recorded.
  • The action is taken on the user's behalf by an agent or automation they cannot watch.

Ask

  • What exactly will change, and would the user recognize it from this description?
  • Can the user decline this part and still get the rest of what they asked for?
  • Is the previewed plan produced by the same code that applies it?
  • What does this do when no one is there to answer -- refuse, or proceed?
  • When the set of effects grows in a later version, what makes it ask again rather than reuse the old approval?

Least Astonishment

An operation should confine its effects to what its name and context already denote to the people who invoke it, and any effect beyond that must be visible at the point of use rather than discoverable only afterwards.

Applies when

  • The user brings an expectation from a convention, a neighboring tool, or an earlier version of this one.
  • The operation touches state the user owns and the tool does not.
  • The surprising effect is paid for by the user rather than by the author.
  • Discovery happens after the fact, when the cheap moment to object has passed.

Ask

  • What does the name of this operation promise, and what does it actually touch?
  • Which of these effects would a user learn about only after they had happened?
  • If this changes a default, what does a user who upgrades without reading anything experience?
  • Is the extra behavior disclosed where the decision is made, or only in documentation the user reads later?
  • Whose expectation is being matched, and what is the evidence that they hold it?

Least Common Mechanism

Every mechanism shared by more than one principal is a potential channel between them and one point whose failure or compromise reaches all of them, so shared mechanism should be no more than the function requires.

Applies when

  • Two or more principals, tenants, or repositories run on the same process, host, cache, runner, or credential.
  • A component is updated centrally and starts behaving differently for every consumer at once.
  • Timing, cache state, or shared files could carry information across a trust boundary.
  • A function could live in a per-consumer component rather than in a shared service.

Ask

  • Which principals share this mechanism, and does it cross a trust boundary between them?
  • What can one consumer observe or change about another through it?
  • If this shared component ships a defect, how many consumers does it reach before anyone notices?
  • Is the change sharing the owner of a fact, or sharing the machinery that acts on it?

Rule of Least Power

Choose the least expressive language or interface that suffices for the purpose, because each increment of expressive power removes an increment of what others can analyze, transform, or reuse without running it.

Applies when

  • A format, configuration surface, or rule language is being chosen or extended.
  • Consumers other than the author will read, validate, diff, or transform what is written in it.
  • A field is about to accept an expression, a template, a script, or a regular expression where a closed value would do.
  • The need is understood well enough to state what the weaker form cannot express.

Ask

  • What is the weakest form that expresses every real case here, and what does this form add beyond it?
  • Which consumers read this artifact, and which of them can still analyze it without executing it?
  • Where a field accepts an expression or a script, what closed value would have served, and why was it not enough?
  • If an escape hatch is needed, is it one named place, or does power leak into every field?

Make Illegal States Unrepresentable

Where practical, encode invariants in constructors, types, schemas, or state machines so downstream code receives only valid states.

Applies when

  • The invariant is stable and can be expressed in the representation.
  • Construction passes through a controlled boundary.
  • Invalid states do not need to be preserved for diagnosis or migration.
  • Input crosses a boundary where it can be parsed once into a type that carries the check, rather than validated and passed on as the raw value.

Ask

  • Which invalid combinations recur in checks or incidents?
  • Can the invariant be enforced once at construction?
  • Does the representation include real transitional states rather than pretending they do not exist?

Separate Mechanism from Policy

Implement stable mechanisms that provide capability, and express changeable policy separately so decisions can evolve without rewriting the machinery.

Applies when

  • The same capability serves multiple policies or environments.
  • Rules change more often than the underlying mechanism.
  • Policy can be represented explicitly and reviewed.

Ask

  • Which behavior is stable capability and which is changeable rule?
  • Who is authorized to change the policy?
  • Can invalid policy be rejected before execution?

Observability

A system should emit structured, correlated evidence sufficient to explain relevant state transitions, decisions, dependencies, latency, and failures without reproducing every incident.

Applies when

  • The system has distributed execution, asynchronous work, agents, or external dependencies.
  • Operators need to distinguish failure modes.
  • Evidence can be emitted without violating privacy or security.

Ask

  • Which failure modes can current telemetry distinguish?
  • Can one user-level operation be followed across model and tool calls?
  • What sensitive information must be redacted or omitted?

Parameterize, Do Not Enumerate

If two units of code differ only in a constant, they are one unit with an unextracted parameter. Naming the constant in the identifier moves it out of the type system and into the namespace, where the count grows as the product of the value sets rather than their sum, nothing forces the copies to stay consistent, and the next value needs an author instead of a caller.

Applies when

  • Two or more units differ only by a literal, an enum member, a key, or a path.
  • A unit's name encodes an argument rather than the operation.
  • A general method exists in a library and the surface exposing it does not accept the same range.
  • A one-off helper is about to be written for scaffolding, a probe, a fixture, or a script.

Ask

  • What differs between these units, and is it only a value?
  • Does the name contain an argument? If the value moved into the signature, what would the name become?
  • Does the layer underneath already accept the range this layer does not, and why does the narrowing exist?
  • If the next value is needed tomorrow, does someone call this or edit it?
  • If this is deliberately one case of many, where is that written, and what refuses the second copy?

Psychological Acceptability

Protection mechanisms must be easy enough to apply correctly and routinely that the user's mental model of the protection matches what the mechanism actually does; otherwise compliance decays and the mechanism's real coverage falls to zero.

Applies when

  • The control sits on a path a person or agent traverses many times a day.
  • Bypassing is possible and cheaper than complying.
  • The control produces failures on legitimate work, not only on the case it targets.

Ask

  • How often will this fire on legitimate work, and what does the person do next?
  • Is the cheapest response to a failure a fix, or turning the check off?
  • Is there a scoped way in for a tree that does not yet comply, such as a baseline or a ratchet?
  • Six months from now, what evidence would show this is still enabled everywhere it was installed?

Optimize for Reversible Decisions

For uncertain, noncritical choices, favor designs and commitments whose consequences can be observed and reversed at acceptable cost before scaling them.

Applies when

  • The decision is uncertain and can be staged.
  • Rollback or migration can be designed explicitly.
  • Delay has its own opportunity cost.
  • A schema or interface change can be made as expand and contract: add the new form, move readers and writers across, then remove the old, so each step can be undone on its own.

Ask

  • What exactly would rollback restore?
  • Which consequences are irreversible outside the system boundary?
  • What evidence will trigger consolidation or reversal?

Robustness Principle

An implementation should be conservative in what it emits and liberal in what it accepts, interpreting nonconforming input whose meaning can be recovered rather than rejecting it.

Applies when

  • Many independent implementations must interoperate and none controls the others' release cycles.
  • The specification is young or ambiguous, and deviations are more likely misreadings than intent.
  • The meaning of a deviating input can be recovered without guessing, and accepting it makes no security decision.
  • A feedback channel exists through which a tolerated deviation is reported back to its sender.

Ask

  • Which deviations does this parser accept, and is that list written where a peer implementer could read it?
  • Is each accepted deviation's meaning recoverable without guessing, and does any downstream component interpret it differently?
  • When input is repaired, does the sender learn of it through an error, a log, or a metric?
  • What would it cost to stop accepting this deviation in a year, and who would be broken then?

Rule of Three

Defer extracting a shared abstraction until the third occurrence of similar code, because two instances rarely reveal which differences are incidental and which are the parameter.

Applies when

  • The similar units are young and it is not yet clear which of their differences are essential.
  • Removing the duplication later is cheap: the copies are few, local, and owned by one team.
  • The shared form would generalize over cases nobody has written yet.
  • The units differ in more than one place, so the axis of variation is itself in question.

Ask

  • Is the difference between these copies a single value, or is the axis of variation still unknown?
  • How many copies exist now, including those outside this change, and who would notice the next one?
  • If the abstraction is extracted now, which instance's quirks would it encode as the general case?
  • Do these copies carry a fact that must stay identical, so that drift would be a defect?

Separation of Concerns

Organize a system so distinct concerns can be understood, changed, tested, and governed independently where their real dependencies permit it.

Applies when

  • Concerns have different change drivers, owners, policies, or test strategies.
  • A boundary can be introduced without duplicating core truth.
  • Interactions can be made explicit through contracts.
  • Two concerns can be made orthogonal, so that a change to one leaves the other unchanged.

Ask

  • What distinct reasons would cause this component to change?
  • Which concern owns each invariant?
  • Does the proposed boundary reduce or merely relocate coupling?

Separation of Privilege

Where an action is consequential enough, the mechanism should require two or more independently held conditions before it proceeds, so that the compromise or error of any one of them is not sufficient on its own.

Applies when

  • The action is hard to reverse or lands outside the system, such as a publication, a release, a deletion, or a transfer.
  • Two authorizing facts can be held by different parties, stores, or channels that do not share a failure.
  • The configuration that permits an action could be changed by the same act the permission is meant to catch.
  • One of the authorizations can be answered by a party outside the repository or process being guarded.

Ask

  • Which independent conditions does this action require, and who holds each?
  • Could one edit, one credential, or one person satisfy all of them?
  • When the second authority cannot be asked, does the action refuse or proceed?
  • Is there an override, and does its use leave a record of who used it?

Single Authoritative Source

Each authoritative fact should have one explicitly designated ownership and update authority, while replicas, caches, and derived views remain subordinate to that authority.

Applies when

  • The same fact is represented in multiple services, stores, reports, or workflows.
  • Conflicting updates would be materially expensive or unsafe.
  • A responsible owner can be assigned at an appropriate domain boundary.
  • One fact arrives in several spellings, such as case, encoding, or path form, and is normalized to one canonical form at the boundary so that two copies can be compared for equality.

Ask

  • Which component owns the canonical meaning and writes?
  • Can every copy explain its source and refresh policy?
  • What happens when the authority is unavailable?
  • Is there exactly one writer for this fact, and what stops a second component from writing it?

Unix Composability

Prefer small, focused mechanisms connected through stable, inspectable interfaces so that complex behavior can be assembled rather than embedded in one indivisible program.

Applies when

  • A domain can be decomposed into transformations or tools with stable boundaries.
  • Intermediate results are useful to inspect, store, or reuse.
  • The operational cost of composition remains lower than the cost of integration.

Ask

  • Is each component focused on one coherent transformation or capability?
  • Can outputs be inspected and consumed without private knowledge?
  • Would combining these components preserve a stronger transaction or invariant?