feat: three-layer codec killbits (draft, do not merge)

[lilith]

Motivation

Three things to land in this PR:

1. Stop leaking per-job security intent across jobs on the same Context.
   Today Build001/Execute001 inline security blocks mutate
   Context.security in place, so one job's narrowing persists and
   affects the next job on that Context. An earlier iteration of this
   PR made the mutation intersection-only when a trusted_policy is
   set, but the mutation itself was still there. This version fixes it
   properly: inline job security is scoped to the lifetime of one
   build/execute call and never writes through to Context state.

2. Add a three-layer killbits system (format + codec) so trusted
   applications can pin the codec surface once and treat every
   downstream request as a strict narrower. Zero users of killbits
   today, so breaking in-flight shapes is free.

3. Graceful codec substitution with observability. When a
   specific-codec EncoderPreset names a codec that's unavailable but
   another codec for the same wire format is live, the dispatcher
   substitutes the available codec and surfaces a machine-readable
   annotation on the response rather than failing the request.

The three layers

Layer	Where	Scope
1. Build-time ceiling	imageflow_types::build_killbits::COMPILE_DENY_{DECODE,ENCODE} + feature_compiled_in	Hard upper bound. Forks override at compile time.
2. Trusted policy	v1/context/set_policy → Context::trusted_policy	Set once (or narrowed again) by the embedding application. May use allow-lists, deny-lists, or a per-format / per-codec table.
3. Job-level narrowing	existing security field on Build001/Execute001, extended with formats and codecs	Untrusted JSON may only deny. allow_* + table entries with decode: true / encode: true are rejected.

The effective grid for any operation is layer 1 ∩ layer 2 ∩ layer 3.
Enforcement fires at decoder instantiation (when the input buffer is
added) and at encoder dispatch (inside auto::create_encoder),
producing a structured JSON error:

{
  "error": "decode_not_available",
  "format": "jxl",
  "reasons": ["compile.feature_missing"],
  "net_support": { "formats": { ... }, "codecs": { ... } }
}

Per-job scoping (Part A — the pre-existing bug)

Context.security (mutable, process-wide) is replaced by three
Context fields that make the scoping explicit:

• default_job_security: ExecutionSecurity — Context-scoped defaults,
  initialized from sane_defaults() or the trusted policy. Read-only
  during job execution.
• trusted_policy: Option<Box<ExecutionSecurity>> — the layer-2
  baseline, set by v1/context/set_policy. Never mutated by job JSON.
• active_job_security: Option<Box<ExecutionSecurity>> — the per-job
  effective value (trusted ∩ default ∩ inline), installed for the
  duration of a single build/execute and torn down on exit.

Context::effective_security(&self, inline) is the pure computation;
Context::current_security() is the single read point for per-node
limit checks. Inline security on Build001/Execute001 no longer
persists to Context.default_job_security.

max_json_bytes is the one field that stays Context-scoped — it
bounds the JSON parse itself, so it has to be known before the job's
own security block is visible.

Three regression tests pin this:

• inline_max_decode_size_is_not_persisted_across_jobs
• inline_killbits_is_not_persisted_across_jobs
• trusted_policy_persists_but_inline_does_not

Codec-level killbits (Part B)

Format-level killbits grant/deny every codec for a given format.
Codec killbits add per-backend granularity: an operator can forbid
mozjpeg_encoder while keeping zen_jpeg_encoder, or disable a
specific decoder that's had a CVE without stubbing the format.

Types

In imageflow_types:

• NamedEncoderName / NamedDecoderName — #[non_exhaustive]
  snake_case serde enums mirroring every known NamedEncoders /
  NamedDecoders variant in core. Purely nominal; feature gating is
  handled at the core layer.
• CodecKillbits { allow_encoders, deny_encoders, allow_decoders, deny_decoders } with the same validation (mutual exclusion,
  validate_job_level rejecting allow_*) as FormatKillbits.
• ExecutionSecurity.codecs: Option<Box<CodecKillbits>>.

JSON examples

Allow-list form (trusted-policy only):

{
  "policy": {
    "codecs": {
      "allow_encoders": ["mozjpeg_encoder", "lodepng_encoder", "zen_webp_encoder"],
      "allow_decoders": ["mozjpeg_rs_decoder", "libpng_decoder", "gif_rs_decoder"]
    }
  }
}

Deny-list form (trusted-policy or job-level):

{
  "policy": {
    "codecs": {
      "deny_encoders": ["pngquant_encoder"],
      "deny_decoders": ["zen_jxl_decoder"]
    }
  }
}

Job-level narrowing:

{
  "framewise": { "steps": [ ... ] },
  "security": {
    "codecs": { "deny_encoders": ["mozjpeg_encoder"] }
  }
}

Net support grid (format × codec interaction)

v1/context/get_net_support response grows a codecs subtree and
per-format {decode,encode}_reasons vectors:

{
  "ok": true,
  "net_support": {
    "formats": {
      "jpeg": { "decode": true, "encode": true },
      "png":  { "decode": true, "encode": true },
      "avif": { "decode": true, "encode": false,
                "encode_reasons": ["no_available_encoder"] }
    },
    "codecs": {
      "mozjpeg_encoder":  { "available": true,  "format": "jpeg", "role": "encode" },
      "zen_jpeg_encoder": { "available": true,  "format": "jpeg", "role": "encode" },
      "zen_avif_encoder": { "available": false, "format": "avif", "role": "encode",
                            "reasons": ["format_denied"] },
      "libpng_encoder":   { "available": false, "format": "png",  "role": "encode",
                            "reasons": ["codec_killbits.deny_encoders"] }
    }
  },
  "trusted_policy_set": true,
  "compile_ceiling": {
    "denied_decode": [],
    "denied_encode": [],
    "features_missing": ["heic", "tiff", "pnm"]
  }
}

Graceful codec substitution (Part C — new in this revision)

Wire format vs implementation

The previous iteration of this PR treated each specific-codec
EncoderPreset variant as a contract to use that exact codec, and
rejected the request when the codec was denied. That's too strict for
the common operator case: "deny the C mozjpeg build because of a CVE,
but keep JPEG encoding working via zenjpeg." Callers that only care
about getting a valid JPEG back (the overwhelming majority) end up
with 422s they can't act on.

This revision splits the contract into two tiers:

1. Wire format (the bytes the caller gets back). Still strictly
   honored: a request for JPEG always produces JPEG bytes, never PNG
   bytes. EncoderPreset::Format { format: avif } with AVIF denied
   still errors with encode_not_available — the format contract is
   sacred.
2. Implementation (which specific codec ran). Now a soft
   preference. When the requested codec is unavailable but another
   codec for the same wire format is live, the dispatcher substitutes
   and annotates the response.

Substitution table

Legacy preset	Substitute target (priority order)	Notes
Mozjpeg { quality, progressive, matte }	ZenJpegEncoder → MozjpegRsEncoder	quality scale identical (ApproxMozjpeg)
LibjpegTurbo { quality, progressive, optimize_huffman_coding, matte }	ZenJpegEncoder (only)	Excludes MozjpegRsEncoder — always optimizes Huffman, can't honor the disable toggle
Libpng { depth, matte, zlib_compression }	ZenPngEncoder → LodepngEncoder	zlib_compression dropped on lode/zen path (annotated)
Lodepng { maximum_deflate }	ZenPngEncoder → LibpngEncoder	maximum_deflate=true translated to zlib=9 on libpng
Pngquant { ... }	(none — palette quantization is unique)	Errors format_not_available
WebPLossy { quality }	The other WebP encoder (libwebp ↔ zenwebp)
WebPLossless {}	Same

Annotation shape

Each encoded image in the response carries an optional annotations
bag. Today's only channel is codec_substitution:

{
  "encodes": [{
    "io_id": 1,
    "preferred_extension": "jpg",
    "preferred_mime_type": "image/jpeg",
    "w": 800, "h": 600, "bytes": ...,
    "annotations": {
      "codec_substitution": {
        "requested": "mozjpeg_encoder",
        "actual": "zen_jpeg_encoder",
        "reason": "codec_killbits_deny_encoders",
        "field_translations": [
          "preset.quality → zen.quality",
          "preset.progressive → zen.progressive"
        ],
        "dropped_fields": []
      }
    }
  }]
}

reason values (serialized snake_case):

• codec_killbits_deny_encoders — requested codec was in a job or
  trusted deny_encoders list.
• codec_killbits_allow_encoders_excludes — requested codec wasn't
  in an allow_encoders list.
• compile_feature_missing — the build didn't include the requested
  codec's feature gate.
• compile_codec_const_denied — the format family is in
  COMPILE_DENY_ENCODE.
• not_registered — the codec isn't in the runtime
  enabled_codecs registry.

The annotation is per-encode-step (attached to EncodeResult), not
per-job. A response with multiple encodes can have different
substitutions per step. EncodeAnnotations is forward-extensible —
additional annotation kinds can be added as sibling fields without
breaking callers that only understand codec_substitution.

Error shape when substitution is impossible

When the format itself has no live encoder (net.encode(format) = false) after all killbits + features are applied, the unified
format_not_available error fires at parse time:

{
  "error": "format_not_available",
  "format": "jpeg",
  "requested_codec": "mozjpeg_encoder",
  "reasons": ["all_jpeg_encoders_denied"],
  "net_support": { ... }
}

This also fires when the requested codec has no substitution
candidates — today just Pngquant (palette quantization is unique).

The pre-existing encode_not_available format-level error
(EncoderPreset::Format { format: jpeg } when JPEG is off) and
decode_not_available decoder-side error are unchanged.

Backward compatibility

Every existing workflow that uses a legacy EncoderPreset variant
(Mozjpeg, LibjpegTurbo, Libpng, Lodepng, WebPLossy,
WebPLossless) continues to work even when operators tighten codec
killbits — the dispatcher finds a substitute for the same wire format
and annotates the response. Clients that don't read annotations are
unaffected.

code:codec_not_available is still a valid error type on the schema
for the rare unsubstitutable case (e.g., Pngquant denied with no
quantizer substitute). Consumers that already match on it keep
working.

Decoder side

Decoder substitution follows the same pattern the encoder dispatch
landed in commit 4: create_decoder_for_magic_bytes walks all
matching decoders, skips any that are codec-killed, and only errors
when every decoder for the format is down. That logic pre-dates this
revision and already satisfies the substitution contract from the
decode side. Surfacing a structured per-decode annotation is tracked
as follow-up work — DecodeResult.annotations and
DecoderSubstitutionAnnotation are defined on the types side so the
schema is ready, but the core-side plumbing to populate them is
scoped to a follow-up PR.

Breaking behavior changes (would break hypothetical callers; today there are none)

• Inline security on Build001/Execute001 no longer persists
  into Context.security. Callers that relied on "first job sets
  the limit, subsequent jobs inherit it" (not a documented contract,
  but possible in practice) must instead install their baseline via
  v1/context/set_policy.
• Specific-codec EncoderPreset variants now substitute when the
  requested codec is denied and a same-format codec is available,
  instead of silently substituting on an outdated table. The request
  succeeds; the response carries a codec_substitution annotation.
  When no substitute exists (Pngquant denied, or every same-format
  codec denied), the request errors with format_not_available.
• Context::configure_security is removed; its policy-set semantics
  live on as a private apply_trusted_policy reachable only through
  Context::set_trusted_policy.
• Size sanity tests (test_context_size, test_thread_safe_context_size,
  test_calculate_context_heap_size) adjusted for the two new
  Option<Box<ExecutionSecurity>> fields, the new
  Option<Box<CodecKillbits>> on ExecutionSecurity, and the
  Option<Box<EncodeAnnotations>> on each CodecInstanceContainer.

Non-breaking changes

• ExecutionSecurity is #[non_exhaustive], so the new codecs
  field is additive.
• NamedEncoderName / NamedDecoderName are #[non_exhaustive].
• EncodeAnnotations is forward-extensible (all fields optional).
• SubstitutionReason is #[non_exhaustive].
• Legacy jobs without security.formats or security.codecs behave
  identically to before.
• ABI unchanged: imageflow_abi/src/lib.rs has no new exports and
  no signature changes. New endpoints are reached through the
  existing imageflow_context_send_json entry point.
• No RIAPI changes. URL-layer plumbing is out of scope.

What this PR does NOT do

• No imageflow_abi/src/lib.rs changes.
• No imageflow_riapi changes.
• No accept.* URL translation.
• No renamed endpoints.
• No ProcessConfig-style startup-mutable type.
• No decoder-side annotation surfacing in DecodeResult
  (types are defined, core plumbing is a follow-up).

Test plan

• cargo check --workspace is green at every commit.
• cargo test -p imageflow_types --lib — 38 pass, 0 fail
  (added CodecKillbits-specific tests, SubstitutionReason
  serde tests, CodecSubstitutionAnnotation round-trip tests,
  EncodeAnnotations shape tests).
• cargo test -p imageflow_core --lib — 78 pass, 0 fail.
• cargo test -p imageflow_core --test integration killbits —
  27 pass, 0 fail. Substitution coverage:
  deny_specific_encoder_substitutes_preset_when_lodepng_denied,
  lodepng_preset_errors_format_not_available_when_no_png_encoder_remains,
  pngquant_preset_errors_format_not_available_when_pngquant_denied,
  mozjpeg_preset_substitutes_when_mozjpeg_denied,
  libjpegturbo_preset_substitutes_mozjpeg_denied_to_zenjpeg_not_mozrs,
  libpng_preset_substitutes_when_libpng_denied,
  lodepng_preset_substitutes_when_lodepng_denied,
  webp_lossy_preset_substitutes_when_primary_webp_denied,
  webp_lossless_preset_substitutes_when_primary_webp_denied,
  format_preset_denied_stays_strict_no_substitution.
• cargo test -p imageflow_core --test integration -- --skip visuals
  — 99 pass, 0 fail, 3 ignored (pre-existing).
• cargo test -p imageflow_core --features schema-export,json-schema hash_files_relevant_to_schema_and_compare — pass, schema + hash
  regenerated and committed.
• CI green on all platforms (Linux x86_64, Linux i686, macOS
  Intel, macOS ARM, Windows, windows-11-arm) — pending.

Commits

1. feat(types): ImageFormat enum, FormatKillbits, CodecKillbits, Named{Encoder,Decoder}Name, build_killbits consts, ExecutionSecurity extensions, EncodeAnnotations / CodecSubstitutionAnnotation / SubstitutionReason, EncodeResult.annotations
2. refactor(core): scope per-job security; stop mutating Context.security
3. feat(core): trusted_policy + v1/context/{set_policy,get_net_support} endpoints
4. feat(core): enforce format + codec killbits at decode/encode dispatch; substitute same-format codec for denied preset + annotate; format_not_available when no substitute available
5. test: killbits + substitution + scoping unit + integration coverage (types + core)
6. chore: regenerate OpenAPI schema hash

Draft — not for merge. Opened to invite review of the surface /
enforcement points / error + annotation shape before widening.