Skip to content

feat: TM1 chore as a first-class task kind (closes #156)#157

Open
nicolasbisurgi wants to merge 2 commits into
masterfrom
feat/issue-156-chore-task-kind
Open

feat: TM1 chore as a first-class task kind (closes #156)#157
nicolasbisurgi wants to merge 2 commits into
masterfrom
feat/issue-156-chore-task-kind

Conversation

@nicolasbisurgi
Copy link
Copy Markdown
Collaborator

@nicolasbisurgi nicolasbisurgi commented May 20, 2026

Summary

Implements TM1 chore as a first-class task kind alongside TI process (closes #156). A task now carries exactly one of process or chore — the field name is the kind discriminator (no meta-kind field). Mixed process + chore taskfiles are supported across JSON, TXT, and cube sources, and chores participate in the DAG (predecessors, stages, retry) on equal footing with processes.

Anchor docs (in-repo working artefacts; only CONTEXT.md is committed):

  • CONTEXT.md — Task / Task kind / Chore vocabulary (committed in this PR).
  • docs/adr/0002-polymorphic-task-kinds.md — full decision record (local-only).
  • IMPLEMENTATION_PLAN_CHORE_KIND.md — file-by-file change list (local-only).

What's in scope

  • Polymorphic TaskDefinition / Task / OptimizedTask with mutual-exclusion as a class invariant on Task.__init__ and per-kind forbidden-field validation in validate_task.
  • Chore-specific validation: tm1.chores.exists(...) for every unique (instance, chore_name), plus a SINGLE_COMMIT check when safe_retry: true (deduplicated so the chore object is fetched at most once per run).
  • execute_task dispatch to _execute_chore_task + new execute_chore_with_retries. Binary success/fail (HTTP 204 / exception). safe_retry re-executes the whole chore.
  • TXT taskfiles now run through validate_taskfile on read (closes a pre-existing gap).
  • Cube reader accepts chore rows; rejects rows with both process and chore set.
  • Additive cube schema upgrade: chore measure element + vchore TI variable + CellPutS line in both v11 and v12 }rushti.load.results bodies.
  • Additive SQLite schema: chore TEXT column on task_results, PRAGMA-driven migration on first connection.
  • Optimizer calculate_task_signature is chore-aware; signatures occupy a disjoint hash space from process signatures.
  • Dashboard + DAG visualization render a unified Task target column with [P] / [C] kind indicator (both visualization_template.py and templates/visualization.html updated to stay consistent).
  • Structured execution logger gains a parallel chore: Optional[str] parameter (does not overload process).
  • CHANGELOG documents the two behavioural changes.

Out of scope

  • Shell command task kind (a separate Issue B).
  • Other ideas from Run other commands #145 (sub-taskfiles, REST, reusable stages).

Test plan

  • Unit tests in tests/unit/test_chore_kind.py (37 cases) cover validation, the Task invariant, TXT conversion + validation tightening, chore signature disjointness, retry behaviour with mocked TM1Service, execute_task dispatch, validate_tasks chore branch, and the cube reader's chore acceptance + both-populated rejection.
  • tests/unit/test_taskfile.py updated to match the new "exactly one of 'process' or 'chore'" message.
  • Full unit suite passes: 772 passed, 5 skipped.
  • Integration tests against live TM1 v11.8 (tm1srv01) in tests/integration/test_v11_chore_kind.py13/13 pass. Uses preconfigured test_chore_success and test_chore_error chores. Verifies:
    • execute_chore_with_retries: success (single attempt), failure without safe_retry (single attempt), failure with safe_retry (exhausts budget then raises).
    • validate_tasks chore branch: existence pass/fail, SingleCommit check on safe_retry.
    • execute_task dispatch routes chore-kind tasks to the chore path on success and on failure.
    • Mixed process + chore DAG with predecessors crossing kinds — forward path executes in order, failing chore aborts dependent tasks via require_predecessor_success.
    • build_logging_objects idempotently adds the chore measure element to an existing rushti cube.
    • Stats DB records process and chore rows with disjoint signatures (no collision when they share a name).
    • End-to-end results push: chore execution row → stats DB → upload_results_to_tm1}rushti.load.results TI execution → cube cell readback under the chore measure. This proves the v11 TI body compiles against the new vchore variable, the CSV column ordering aligns with PROCESS_VARIABLES, and the additive cube schema works in a single round-trip.
  • v12 integration coverage not exercised here — same code paths but worth re-running against tm1srv02 before merge.

Upgrade notes

  • Re-run rushti build --tm1-instance X on existing installations to add the new chore measure to the cube. The additive merge logic already in place handles this without data loss.
  • Flush pending CSVs from prior versions before upgrade — the }rushti.load.results TI variable list changed and old-format CSVs cannot be loaded.
  • Pre-existing malformed TXT files will now fail with explicit validation errors on first run. This is net-positive (the files were silently buggy before) but worth flagging to users with legacy taskfiles.

🤖 Generated with Claude Code

Introduces a polymorphic task abstraction where each task carries
exactly one of `process` (TI process) or `chore` (TM1 chore). The field
name is the discriminator — no meta-`kind` field. Mutual exclusion is
enforced at parse-time validation and as a class invariant on
`Task.__init__`, so downstream code can rely on exactly one being set.

Chores are intentionally narrower than processes: no parameters, no
minor-error tier, no native timeout. `safe_retry` is honoured only for
SINGLE_COMMIT chores so partial state cannot leak on failure. The
existence + execution-mode checks are added to `validate_tasks` and
deduplicated by (instance, chore_name) so each chore is fetched at most
once per run.

JSON, TXT, and cube-source readers all accept chore tasks. TXT now runs
through `validate_taskfile` on read, closing a pre-existing gap that
silently accepted malformed input. Cube reader raises on rows with both
`process` and `chore` populated.

Schema changes are additive: a new `chore` measure element on the
results cube (auto-merged by existing `_merge_measure_dimension_elements`
path), a new `vchore` TI variable + `CellPutS` line in both v11 and v12
`}rushti.load.results` bodies, and a `chore TEXT` column on the SQLite
`task_results` table with PRAGMA-driven migration.

Dashboard replaces the per-task "Process" column with a unified
"Task target" column carrying a `[P]` / `[C]` kind indicator in both
the runtime dashboard and the DAG visualization templates.

CHANGELOG records the two behavioural changes (TI schema, TXT
validation tightening).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@nicolasbisurgi nicolasbisurgi self-assigned this May 20, 2026
@nicolasbisurgi nicolasbisurgi added the release:minor Triggers major version bump (e.g.: 1.4.1 → 1.5.0) label May 20, 2026
13 integration tests against tm1srv01 (v11.8) using the
preconfigured `test_chore_success` and `test_chore_error` chores.
Covers:

- execute_chore_with_retries: success, failure without safe_retry
  (single attempt), failure with safe_retry (raises after budget).
- validate_tasks chore branch: existence pass/fail, SingleCommit
  check on safe_retry.
- execute_task dispatch routes chore-kind tasks to the chore path
  on success and on failure (HTTP 500 → False).
- Mixed process + chore DAG with predecessors crossing kinds —
  forward path executes in order, a failing chore aborts dependent
  tasks via require_predecessor_success.
- build_logging_objects idempotently adds the `chore` measure
  element to an existing rushti cube.
- Stats DB records process and chore rows with disjoint signatures
  (no collision when they share a name).
- End-to-end results push: chore execution row → stats DB →
  upload_results_to_tm1 → }rushti.load.results TI → cube cell under
  the `chore` measure. Validates the v11 TI body, the CSV column
  ordering against PROCESS_VARIABLES, and the additive cube schema
  in a single round-trip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release:minor Triggers major version bump (e.g.: 1.4.1 → 1.5.0)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add TM1 chore as a first-class task kind (Issue A of #145 split)

1 participant