Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion reflexio/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,6 @@ client (Python SDK)
- `playbook/` - Playbook extraction, consolidation, and aggregation
- `agent_success_evaluation/` - Success evaluation
- `durable_learning/` - Claim/drain durable `learning_jobs` for deferred extraction when `REFLEXIO_DURABLE_LEARNING_QUEUE` is enabled
- `reflection/` - Post-horizon reflection extraction
- `extraction/` - Resumable async extraction agent infrastructure
- `shadow_comparison/` - Per-turn regular vs shadow verdict judge
- `evaluation_overview/` - Evaluation-page aggregates and hero metrics
Expand Down
7 changes: 3 additions & 4 deletions reflexio/lib/_profiles.py
Original file line number Diff line number Diff line change
Expand Up @@ -624,8 +624,8 @@ def reconstruct_profile_change_log(
* **removed(R)** — entity_ids of ``status_change`` lineage events with
``to_status == "superseded"`` and ``request_id == R``. This is the
exact signature emitted by ``supersede_profiles_by_ids`` (the dedup
soft-delete path). It is distinct from reflection which emits
``op="revise"``, so reflection events are never mis-counted as removals.
soft-delete path). Other revision events are never mis-counted as
removals.

Groups are formed over the union of request_ids from both signals.
Request_id ``""`` is skipped — it would merge unrelated runs.
Expand Down Expand Up @@ -656,8 +656,7 @@ def reconstruct_profile_change_log(

# Dedup soft-delete signature: status_change to_status=="superseded".
# Each such event records one profile removed in the dedup run ``request_id``.
# Distinct from reflection which emits op="revise" — so revise events are
# never counted as removals here.
# Revise events are never counted as removals here.
removal_by_req: dict[str, list[str]] = defaultdict(list)
sort_key: dict[str, tuple[int, int]] = {} # request_id -> (created_at, event_id)
for evt in all_events:
Expand Down
52 changes: 0 additions & 52 deletions reflexio/lib/_reflection.py

This file was deleted.

2 changes: 0 additions & 2 deletions reflexio/lib/reflexio_lib.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
from reflexio.lib._interactions import InteractionsMixin
from reflexio.lib._operations import OperationsMixin
from reflexio.lib._profiles import ProfilesMixin
from reflexio.lib._reflection import ReflectionMixin
from reflexio.lib._search import SearchMixin
from reflexio.lib._user_playbook import UserPlaybookMixin

Expand All @@ -19,7 +18,6 @@ class Reflexio(
UserPlaybookMixin,
ConfigMixin,
GenerationMixin,
ReflectionMixin,
OperationsMixin,
DashboardMixin,
SearchMixin,
Expand Down
7 changes: 3 additions & 4 deletions reflexio/models/api_schema/domain/entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,9 +141,8 @@ class Citation(BaseModel):

Carried inline on an Assistant ``InteractionData`` row to mark
which previously-injected playbook rule or user-profile row
materially shaped that response. The server uses these to drive
reflection (does the cited rule still look right after seeing how
it was applied?).
materially shaped that response. The server stores these for retrieval
attribution and evaluation.

Attributes:
kind (CitationKind): Which kind of cited item this references.
Expand Down Expand Up @@ -498,7 +497,7 @@ class LineageEvent(BaseModel):
op (str): create|revise|merge|aggregate|archive|soft_delete|hard_delete|purge|status_change.
prov_relation (str): W3C PROV relation (see spec §14).
source_ids (list[str]): Records merged/superseded into entity_id.
actor (str): Who/what triggered it (consolidator|reflection|offline_optimizer|...).
actor (str): Who/what triggered it (consolidator|offline_optimizer|...).
request_id (str): Triggering request — part of the idempotency key.
reason (str): Free-text rationale (no PII).
created_at (int): Unix epoch seconds (0 = unset; storage stamps it).
Expand Down
53 changes: 0 additions & 53 deletions reflexio/models/config_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -542,56 +542,6 @@ def _default_agent_success_config() -> AgentSuccessConfig:
)


class ReflectionConfig(BaseModel):
"""Configuration for the sliding-window reflection step.

Reflection runs inside ``GenerationService.run`` as its own
sliding-window step (window = global ``window_size``, stride = global
``stride_size``, bookmark via ``OperationStateManager``). When
the gate opens and at least one Assistant interaction in the window
cites a current user playbook / user profile row, the LLM is asked
whether any cited rows should be replaced. When ``enabled`` is
False the step short-circuits.

Args:
enabled (bool): Master switch. When False, no LLM call is made.
model (str | None): Optional model name override. Falls back to
``LLMConfig.generation_model_name`` and then the site
default for ``ModelRole.GENERATION`` when None.
post_horizon_size (int): Minimum interactions after a citation before
reflection judges it with full confidence. Citations near the recent
edge of the window with fewer than this many follow-up turns get a
'last_chance' judgment with the prompt biased toward no_change.
Set to 0 to disable the filter (legacy behavior).
max_revisions_per_pass (int): Cap on the number of revision decisions
applied in a single reflection pass (regularization). Once the cap
is hit, remaining revision-intent decisions are skipped and counted
in ``ReflectionResult.capped_count``.
"""

enabled: bool = True
model: str | None = None
post_horizon_size: int = Field(
default=3,
description=(
"Minimum interactions after a citation before reflection judges "
"it with full confidence. Citations near the recent edge of the "
"window with fewer than this many follow-up turns get a "
"'last_chance' judgment with the prompt biased toward no_change. "
"Set to 0 to disable the filter (legacy behavior)."
),
ge=0,
)
max_revisions_per_pass: int = Field(
default=8,
gt=0,
description=(
"Cap on revision decisions applied per reflection pass "
"(regularization; excess are skipped)."
),
)


class RetrievalFloorConfig(BaseModel):
"""Read-path relevance floor: drop search results below a per-arm cross-encoder score.

Expand Down Expand Up @@ -877,8 +827,6 @@ class Config(BaseModel):
api_key_config: APIKeyConfig | None = None
# LLM model configuration overrides
llm_config: LLMConfig | None = None
# Post-publish reflection service configuration
reflection_config: ReflectionConfig = Field(default_factory=ReflectionConfig)
# Read-path relevance floor (per-arm cross-encoder score cutoff)
retrieval_floor: RetrievalFloorConfig = Field(default_factory=RetrievalFloorConfig)
# Optional GEPA-backed playbook content optimizer
Expand Down Expand Up @@ -949,7 +897,6 @@ def _migrate_field_names(cls, data: Any) -> Any:
for key in (
"window_size",
"stride_size",
"reflection_config",
"playbook_optimizer_config",
"lineage_gc",
"expiry_reclamation",
Expand Down
12 changes: 5 additions & 7 deletions reflexio/server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Description: FastAPI backend server that processes user interactions to generate
- [Playbook Extraction](#playbook-extraction)
- [Agent Success Evaluation](#agent-success-evaluation)
- [Durable Learning Queue](#durable-learning-queue)
- [Reflection and Async Extraction](#reflection-and-async-extraction)
- [Async Extraction](#async-extraction)
- [Shadow Comparison and Evaluation Overview](#shadow-comparison-and-evaluation-overview)
- [Playbook Optimizer and Braintrust](#playbook-optimizer-and-braintrust)
- [Lineage](#lineage)
Expand Down Expand Up @@ -212,12 +212,12 @@ python -m reflexio.server.scripts.manage_invitation_codes list --show-used
**Service Boundary**: The service layer owns LLM orchestration, extraction, evaluation, optimization, search preparation, storage access, and long-running operation state. API endpoints should validate/authenticate requests, build `RequestContext`, and delegate into `Reflexio` or focused service helpers rather than embedding business logic.

**Encapsulated Components**:
- **Publish pipeline**: `generation_service.py` coordinates interaction persistence, profile generation, playbook generation, reflection, and deferred evaluation scheduling.
- **Publish pipeline**: `generation_service.py` coordinates interaction persistence, profile generation, playbook generation, and deferred evaluation scheduling.
- **Profile memory**: `profile/` extracts, deduplicates, and applies user profile updates.
- **Playbook memory**: `playbook/` extracts user playbooks, consolidates them against existing rows, aggregates them into agent playbooks, and tracks aggregation change logs.
- **Evaluation**: `agent_success_evaluation/service.py`, `agent_success_evaluation/runner.py`, `agent_success_evaluation/scheduler.py`, `agent_success_evaluation/components/evaluator.py`, `shadow_comparison/`, and `evaluation_overview/` handle session grading, per-turn shadow verdicts, regeneration jobs, and dashboard-facing rollups.
- **Durable learning queue**: `durable_learning/scheduler.py` and `durable_learning/worker.py` drain `learning_jobs` after deferred publishes and report coverage through `GET /api/learning_status`.
- **Async clarification**: `extraction/` and `reflection/` manage resumable agent runs, pending tool calls, prior-answer search, and long-horizon reflection updates.
- **Async clarification**: `extraction/` manages resumable agent runs, pending tool calls, and prior-answer search.
- **Search preparation**: `pre_retrieval/` and `unified_search_service.py` handle query reformulation, document expansion, embeddings, and cross-entity search orchestration.
- **Optimization/integrations**: `playbook_optimizer/` and `braintrust/` run candidate playbook optimization, rollout support, and Braintrust export/sync.
- **Lineage**: `lineage/` resolves active records across superseded chains and schedules tombstone garbage collection for profile/playbook storage.
Expand Down Expand Up @@ -446,13 +446,11 @@ Key files:

**Pattern**: `POST /api/publish_interaction` returns immediately when `wait_for_response=false`; callers use the returned `request_id` with `GET /api/learning_status`. Queue workers run the LLM compute **outside** any writer transaction; only the persist half + the fenced `complete_learning_job()` run inside `storage.commit_scope()`, and must raise/rollback if `complete_learning_job()` returns 0 because another worker stole the lease.

### Reflection and Async Extraction
### Async Extraction

**Directories**: `services/reflection/`, `services/extraction/`
**Directory**: `services/extraction/`

Key files:
- `reflection/service.py`: Post-horizon reflection orchestration for synthesizing longer-range memory artifacts
- `reflection/components/extractor.py`: LLM extractor used by the reflection service
- `extraction/resumable_agent.py`: Resumable extraction agent runtime
- `extraction/resume_scheduler.py` and `extraction/resume_worker.py`: Background scheduling/worker loop for paused extraction runs
- `extraction/pending_tool_call_dispatch.py` and `extraction/prior_answer_search.py`: Tool surface and prior-answer context for async extraction agents
Expand Down
12 changes: 6 additions & 6 deletions reflexio/server/billing_meter.py
Original file line number Diff line number Diff line change
Expand Up @@ -231,8 +231,8 @@ def emit_learnings_generated(
) -> None:
"""Resolve ``platform_llm`` from config and emit the Learning value facet.

Convenience wrapper for the non-extraction learning-mutation paths (reflection,
resumable-extraction finalization, aggregation, offline-tuner auto-apply). It
Convenience wrapper for non-extraction learning-mutation paths such as
resumable-extraction finalization, aggregation, and offline-tuner auto-apply. It
owns the ``configurator.get_config()`` + ``platform_llm_from_config`` lookup so
each call site stays a thin one-liner, and — critically — is **guarded**: the
product path must never fail because metering failed, so config resolution and
Expand All @@ -244,7 +244,7 @@ def emit_learnings_generated(
org_id: Organisation identifier.
configurator: Object exposing ``get_config()`` for platform-LLM resolution.
count: Number of learnings durably produced by this path.
source: Metering source/path label (e.g. ``"reflection"``).
source: Metering source/path label (e.g. ``"offline_optimizer"``).
pipeline: Optional pipeline tag (e.g. ``"playbook"``).
user_id: Optional user ID tied to the generated learning.
request_id: Optional request correlation ID.
Expand Down Expand Up @@ -302,8 +302,8 @@ def emit_learnings_generated_records(
Entity-backed counterpart to :func:`emit_learnings_generated`, currently
adopted by two of the non-extraction learning-mutation paths —
resumable-extraction finalization and aggregation — the callers with
durable per-record ids in scope. Extraction, reflection, and offline-tuner
auto-apply do not have a safe 1:1 id per unit of count (see
durable per-record ids in scope. Extraction and offline-tuner auto-apply do
not have a safe 1:1 id per unit of count (see
:func:`record_learnings_generated_records`) and use the count-based
:func:`emit_learnings_generated` fallback instead. Same guard semantics:
config resolution and emission are wrapped and any exception is logged
Expand All @@ -314,7 +314,7 @@ def emit_learnings_generated_records(
org_id: Organisation identifier.
configurator: Object exposing ``get_config()`` for platform-LLM resolution.
learning_ids: Ids of the learnings durably produced by this path.
source: Metering source/path label (e.g. ``"reflection"``).
source: Metering source/path label (e.g. ``"aggregation"``).
pipeline: Optional pipeline tag (e.g. ``"playbook"``).
user_id: Optional user ID tied to the generated learning.
request_id: Optional request correlation ID.
Expand Down

This file was deleted.

Loading