Phase 4: Nightly extraction -- automated workflow crystallisation via ADF

[AlexMikhalev]

Parent Epic

Part of #692 (Operational Skill Store)

Summary

Add a new Core-tier ADF agent that runs nightly to automatically discover and crystallise successful multi-step workflows from session history into reusable procedures. This closes the loop -- instead of manual capture-success, the system learns from its own history.

What Changes

New ADF agent: procedure-crystalliser

Tier: Core (cron-scheduled, runs nightly)
CLI: terraphim-agent (uses sessions + learn subcommands)

Nightly workflow:

1. terraphim-agent sessions import -- refresh session cache
2. Scan recent sessions for successful multi-step sequences (exit code 0 chains)
3. Classify sequences by task type using KG concept matching
4. Deduplicate against existing procedures (Aho-Corasick)
5. Store novel sequences as new CapturedProcedure entries
6. Run learn health across all procedures
7. Generate daily report: new procedures, degraded procedures, statistics

Orchestrator config addition

[[agents]]
name = "procedure-crystalliser"
tier = "Core"
cli = "terraphim-agent"
schedule = "0 2 * * *"   # 02:00 daily
working_dir = "/home/alex/terraphim-ai"

Session extraction API in terraphim_sessions

pub async fn extract_successful_sequences(
    since: DateTime<Utc>,
    min_steps: usize,
) -> Vec<CommandSequence>;

Affected Crates

• terraphim_orchestrator (new agent config)
• terraphim_sessions (extraction API)
• terraphim_agent (orchestration entry point)

Dependencies

• Phase 1 (Phase 1: Success capture -- extend CapturedLearning with procedural memory #693) -- CapturedProcedure type
• Phase 3 (Phase 3: Success-rate monitoring and self-healing procedures #695) -- health monitoring
• Epic: Leverage Paperclip features into AI Dark Factory #637 Paperclip leverage into ADF (orchestrator infrastructure)
• Phase 2: Session persistence for Claude Code agents in ADF #639 Session persistence for Claude Code agents
• Evaluate: Tree-structured session storage (Pi pattern) for terraphim_persistence #683 Tree-structured session storage

Acceptance Criteria

• procedure-crystalliser agent defined in orchestrator config
• Session extraction API returns successful command sequences
• KG-based task type classification for discovered sequences
• Aho-Corasick deduplication prevents storing known procedures
• Daily report generated with new/degraded procedure counts
• Integration test with sample session data