Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 107 additions & 0 deletions suggestions/website/circuit-breaker-threshold-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
---
title: Circuit Breaker Threshold Contract
description: Constructional contract so CircuitBreaker clears its failure count when the open window ends and the configured threshold applies again.
last_updated: 2026-11-09
owner: maintainers
status: proposed
theme: reliability
impact: high
---

# Circuit Breaker Threshold Contract

## Problem

`CircuitBreaker` opens after `N8N_CIRCUIT_FAIL_THRESHOLD` failures (default 8) and stays open for `N8N_CIRCUIT_OPEN_SECONDS` (default 60). `allow` refuses calls while `open_until` is in the future. It does not change `fail_count` when that time has passed. The next failure increments a counter that is already at the threshold, so the circuit opens again on that single call.

Checked on 2026-11-09 by replaying the class rules in Python 3.12.3. The gateway was not started. n8n was not contacted. The replay used threshold 3 and a 60 second window so the table stays short. The source defaults are 8 and 60. The shape is the same: the counter is left at the threshold.

| Step | `fail_count` | `allow` |
|---|---|---|
| Three failures at t=1000 | 3 | The third failure sets `open_until` to 1060 |
| `allow` at t=1001 | 3 | `CIRCUIT_OPEN` |
| `allow` at t=1061 | 3 | Permitted. The counter is still 3 |
| One failure at t=1061 | 4 | `open_until` moves forward again |
| `allow` at t=1062 | 4 | `CIRCUIT_OPEN` |

A success still clears the counter. A second replay opened the circuit, waited past the window, called `mark_success`, and observed `fail_count` 0 and `open_until` 0.

Clearing the counter on every `allow` would be wrong. While failures are still accumulating, `open_until` is 0. That in-progress count has to survive until it reaches the threshold.

Stage 72 (open PR #98) owns the tool catalog when AFFiNE auth is missing, and the plain-text n8n success body. This page leaves both of those handlers as they are. The breaker is the pause around n8n tool calls.

## Existing scan (reuse first)

| Asset | Role |
|---|---|
| `CircuitBreaker` | The counter and the open window |
| `N8N_CIRCUIT_FAIL_THRESHOLD` | The count that should apply after every quiet period |
| `N8N_CIRCUIT_OPEN_SECONDS` | The quiet period to keep |
| `mark_success` | Already resets `fail_count` and `open_until` |
| `invoke` | Already calls `allow` before an n8n tool and `mark_failure` on a raised error |
| Stage 72 website pages (open PR #98) | Catalog and response-body contracts. Unchanged |

Out of scope:

- A hosted status page or a second breaker implementation
- Changing the default threshold or the default quiet period
- Counting failures for tools that are not n8n tools
- Retrying the failed n8n call inside the breaker

## Proposal

Keep this class. When the quiet period has elapsed, start the failure count from zero before the next trial.

### Contribution lanes

| Lane | Where | Suggested change |
|---|---|---|
| Half-open entry | `CircuitBreaker.allow` | If `open_until` is greater than 0 and no longer in the future, set `fail_count` to 0 and `open_until` to 0, then permit the call |
| Still open | `CircuitBreaker.allow` | If `open_until` is in the future, keep raising `CIRCUIT_OPEN` |
| Accumulating | `CircuitBreaker.allow` | If `open_until` is 0, leave `fail_count` unchanged |
| Success | `mark_success` | Keep the reset that already sets both fields to 0 |
| Trip | `mark_failure` | Keep opening when `fail_count` reaches the threshold |

### Rules

1. **The threshold is the threshold.** After a quiet period, `N8N_CIRCUIT_FAIL_THRESHOLD` new failures are required to open the circuit again.
2. **One expired window clears the old count.** `allow` does this only when `open_until` was set and that time has passed.
3. **An open window still refuses.** Calls during `open_until` raise `CIRCUIT_OPEN` and do not increment `fail_count`.
4. **In-progress failures stay.** A counter of 2 with `open_until` 0 is still 2 on the next `allow`.
5. **Success still heals.** `mark_success` continues to zero the counter and the window.
6. **Defaults stay.** 8 failures and 60 seconds remain the defaults in code.
7. **No new component.** The class in `mcp-gateway/app.py` remains the breaker.

### Suggested contributor checklist

```text
1. Open the breaker with threshold 3 and confirm allow refuses during the window
2. Advance past open_until, call allow, and confirm fail_count is 0
3. Record two failures and confirm the circuit stays closed
4. Record the third failure and confirm the circuit opens
5. With open_until still 0 and fail_count 2, call allow and confirm fail_count is still 2
6. Call mark_success after the window and confirm fail_count and open_until are 0
7. Confirm the AFFiNE catalog failure and the non-JSON n8n body still follow Stage 72
```

### Framework reuse

| Need | Prefer | Avoid |
|---|---|---|
| Pause | `CircuitBreaker.allow` | A proxy product that tracks n8n errors |
| Limits | `N8N_CIRCUIT_FAIL_THRESHOLD` and `N8N_CIRCUIT_OPEN_SECONDS` | Hard-coded replacement numbers |
| Intake | GitHub Issue | An out-of-band on-call form |

### Acceptance check

- [ ] After `open_until` passes, the next `allow` sets `fail_count` to 0.
- [ ] That same call is permitted.
- [ ] The following failures must reach `N8N_CIRCUIT_FAIL_THRESHOLD` before `CIRCUIT_OPEN` returns.
- [ ] `allow` during an open window still raises `CIRCUIT_OPEN`.
- [ ] `allow` while `open_until` is 0 does not clear an in-progress `fail_count`.
- [ ] `mark_success` still sets `fail_count` and `open_until` to 0.
- [ ] The gateway process was not started to prove this page.

### Rollback

Remove the expired-window reset from `allow`. The next failure after a quiet period opens the circuit again, which is today's behavior.
93 changes: 93 additions & 0 deletions suggestions/website/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
title: Community Suggestions Website
description: Publishable Stage 73 website markdown for gzip tar archives, n8n list cursors, and the n8n circuit breaker after its quiet period.
last_updated: 2026-11-09
owner: maintainers
status: proposed
---

# Community Suggestions Website

This directory holds **publishable website markdown**: curated pages a future static docs site can ship without a second editable copy.

Participation stays on GitHub. These pages say which existing TU-VM tools to extend so day-to-day document processing, workflow listing, and n8n calls stay reliable for a community of operators and contributors.

## Why this exists

TU-VM already has the community path and the operator tools: GitHub Issues, `CONTRIBUTING.md`, `tu-vm.sh`, the helper API, the Nginx landing page, the MCP gateway, the LangGraph supervisor, playbooks, and CI. Historical suggestion files in `suggestions/` and open website PRs #27–#98 already specify the docs framework, governance, and the product contracts through Stage 72.

**Stage 73** (this run) records three defects checked on 2026-11-09. They are new findings. They do not restate Stages 1–72.

- `_get_file_type` classifies `bundle.tar.gz` as an archive. `_process_archive` then tests `Path.suffix`, which is `.gz`, so the tar branch does not run. The function still returns `status: success` and `Extracted 0 files`. A gzip tar built in memory opens with `tarfile` and contains `notes.txt`.
- `n8n_list_workflows` accepts `cursor` and then returns only `result` and `total`. A body with `nextCursor: page-2` loses that field. One archived row in a two-row page is removed, and the caller still has no cursor for the next page. `n8n_list_executions` drops `nextCursor` the same way, and its schema has no `cursor` property.
- `CircuitBreaker.allow` blocks while `open_until` is in the future. After the window, `fail_count` is still at the threshold, so the next failure opens the circuit again. A replay with threshold 3 showed that shape. The source default is 8.

The contracts:

- Open `.tar.gz` with the `tarfile` branch that already exists.
- Return the list cursor the n8n response already carries, and accept `cursor` on execution lists.
- Require a full threshold of new failures after the quiet period ends.

No second archive product, workflow engine, or circuit-breaker service.

## Boundaries (do not reinvent)

| Already exists | Website role |
|---|---|
| GitHub Issues + suggestion form | Sole proposal intake |
| [`CONTRIBUTING.md`](../../CONTRIBUTING.md) | Contributor path and labels |
| [`SECURITY.md`](../../SECURITY.md) | Vulnerability reporting |
| [`docs/playbooks/`](../../docs/playbooks/README.md) | Operator recipes |
| Nginx landing page | Local control plane |
| Historical files in [`suggestions/`](../) | Archive and planning source |
| Stage 1–72 website pages (open PRs #27–#98) | Implement those contracts in code; leave their pages as they are |
| `UniversalMinIOProcessor._process_archive` | The archive reader to extend |
| `n8n_list_workflows` and `n8n_list_executions` | The list handlers to extend |
| `CircuitBreaker` | The breaker to extend |

Stage 41 (open PR #67) owns MinIO object-list pagination. Stage 52 (open PR #78) owns `notes.pdf` and `notes.md` sharing `notes.txt`. Stage 69 (open PR #95) owns source `.txt` being skipped. Stage 70 (open PR #96) owns webhook `confirm=true`. Stage 71 (open PR #97) owns the non-empty workflow allowlist. Stage 72 (open PR #98) owns plain-text n8n success bodies. This stage leaves those contracts as they are.

## Page map (Stage 73)

| Page | Purpose |
|---|---|
| [Gzip tar suffix](./tar-gz-archive-suffix-contract.md) | A `.tar.gz` upload must be opened as a tar archive |
| [n8n list cursor](./n8n-list-cursor-contract.md) | Workflow and execution lists must keep the next page cursor |
| [Circuit breaker threshold](./circuit-breaker-threshold-contract.md) | The quiet period must restore the configured failure threshold |

## Expected sibling pages

Stages 1–72 are not files on this branch. When those PRs merge, keep their pages as siblings in this folder. Leave those files uncopied here.

Representative filenames only: `processor-object-list-pagination-contract.md`, `notes-output-key-contract.md`, `source-text-file-contract.md`, `webhook-confirm-gate-contract.md`, `n8n-workflow-allowlist-enforcement-contract.md`, `n8n-non-json-response-contract.md`.

Extend `stage-merge-playbook.md` through **Stage 73** when reconciling hubs so earlier pages are not dropped.

## Framework reuse (summary)

| Need | Prefer | Avoid |
|---|---|---|
| Proposal intake | GitHub Issues | A suggestion database |
| Docs site | Static framework already proposed in Stage 3 (Starlight remains the default) | A custom community portal |
| Archive reading | `tarfile` inside `_process_archive` | A new extraction service |
| List paging | `cursor` and `nextCursor` on the existing n8n tools | A second n8n client |
| Failure pause | `CircuitBreaker` in `mcp-gateway/app.py` | A hosted circuit-breaker product |

## Navigation for a future docs site

Map this folder to a `Community` section:

1. How to submit, status, and decisions (Stages 1–4, when those PRs are merged)
2. Operator day-to-day: a `.tar.gz` object in a watched bucket yields the text inside the archive
3. Operator day-to-day: workflow and execution lists can be continued from the cursor n8n returned
4. Operator day-to-day: after the n8n circuit quiets, the configured failure count applies again

When a static site is adopted, move or mount these pages into the docs content root. Keep a single editable copy.

## Related archive and planning docs

- [`../README.md`](../README.md) — suggestions hub
- [`../index.md`](../index.md) — community suggestions map
- [`../implementation-backlog.md`](../implementation-backlog.md) — trimmed execution backlog
- [`../historical-suggestions.md`](../historical-suggestions.md) — changelog ideas
- [`../website-roadmap-from-historical-suggestions.md`](../website-roadmap-from-historical-suggestions.md) — historical product direction
116 changes: 116 additions & 0 deletions suggestions/website/n8n-list-cursor-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
---
title: n8n List Cursor Contract
description: Constructional contract so workflow and execution list tools return the nextCursor n8n already sends and executions accept a cursor.
last_updated: 2026-11-09
owner: maintainers
status: proposed
theme: operations
impact: high
---

# n8n List Cursor Contract

## Problem

`n8n_list_workflows` tells the model it can pass `cursor`. The handler copies that value onto the n8n query, then builds a new object with only `result` and `total`. `n8n_list_executions` does the same with the response, and its input schema has no `cursor` property.

Checked on 2026-11-09 by replaying the shaping loops in `mcp-gateway/app.py`. n8n was not contacted.

Workflow body used in the replay:

```json
{
"data": [
{"id": "1", "name": "keep", "isArchived": false},
{"id": "2", "name": "old", "isArchived": true}
],
"nextCursor": "page-2"
}
```

| Shaper | Returned keys | `total` | `nextCursor` |
|---|---|---|---|
| `n8n_list_workflows` with `includeArchived` false | `result`, `total` | 1 | Absent |
| `n8n_list_executions` | `result`, `total` | 2 for a two-row body | Absent |

The archived row is removed after the page is fetched. The cursor that would fetch the following page is removed in the same return. A caller that received one visible workflow cannot ask for `page-2`.

Execution lists have the same return shape. The schema properties are `limit` and `workflowId` only, so a later page cannot be requested even after the response keeps a cursor.

Stage 70 (open PR #96) owns `confirm=true` before a webhook dispatch. Stage 71 (open PR #97) owns which workflow id a non-empty allowlist may touch. Stage 72 (open PR #98) owns a plain-text or empty n8n success body. Those contracts stay. This page is the list cursor.

## Existing scan (reuse first)

| Asset | Role |
|---|---|
| `n8n_list_workflows` input `cursor` | Already forwarded on the query |
| `includeArchived` | Already applied to each row |
| `n8n_list_executions` | The execution list to extend |
| `N8nApi.request` | The client that already returns the parsed object |
| Stage 70 `webhook-confirm-gate-contract.md` (open PR #96) | Webhook confirm. Unchanged |
| Stage 71 `n8n-workflow-allowlist-enforcement-contract.md` (open PR #97) | Allowlist holes. Unchanged |
| Stage 72 `n8n-non-json-response-contract.md` (open PR #98) | Non-JSON success bodies. Unchanged |

Out of scope:

- A second n8n client or a cached workflow index
- Changing which workflows an allowlist may touch
- Changing the webhook confirm check
- Parsing rules for a non-JSON success body

## Proposal

Keep the list tools. Copy `nextCursor` when the response has it, and let execution lists take the same `cursor` input workflows already take.

### Contribution lanes

| Lane | Where | Suggested change |
|---|---|---|
| Workflow response | `n8n_list_workflows` | When the parsed body has `nextCursor`, include that string on the tool result |
| Archived rows | The same handler | Filtering `isArchived` must not drop `nextCursor` |
| Execution request | `n8n_list_executions` schema and query | Add `cursor`, and forward it the way the workflow list already does |
| Execution response | `n8n_list_executions` | Copy `nextCursor` the same way |

### Rules

1. **The cursor is part of the page.** A tool result that came from a body containing `nextCursor` includes that value.
2. **Absent stays absent.** A body with no `nextCursor` does not gain an empty string.
3. **Archived filtering keeps the page token.** Hiding archived rows changes `result` and `total`. It leaves `nextCursor` in place.
4. **Executions can continue.** The execution schema accepts `cursor`, and the handler sends it on the n8n query.
5. **Names stay.** The tools remain `n8n_list_workflows` and `n8n_list_executions`.
6. **Allowlist, confirm, and JSON parsing stay with their owners.** Stage 70, Stage 71, and Stage 72 pages stay as written.
7. **No new service.** `N8nApi.request` remains the HTTP client.

### Suggested contributor checklist

```text
1. Replay a workflow body with two rows, one archived, and nextCursor page-2
2. Confirm the tool result total is 1 and nextCursor is still page-2
3. Replay a workflow body with no nextCursor and confirm the tool result has no nextCursor key
4. Pass cursor page-2 into n8n_list_workflows and confirm the outgoing query contains that cursor
5. Pass cursor exec-2 into n8n_list_executions and confirm the outgoing query contains that cursor
6. Replay an execution body with nextCursor exec-2 and confirm the tool result keeps it
7. Confirm webhook confirm, the allowlist, and non-JSON success bodies still follow Stages 70–72
```

### Framework reuse

| Need | Prefer | Avoid |
|---|---|---|
| Page token | `nextCursor` on the body `N8nApi.request` already parsed | A private offset index |
| Request field | The `cursor` property already on `n8n_list_workflows` | A new list tool |
| Intake | GitHub Issue | A workflow-catalog product |

### Acceptance check

- [ ] `n8n_list_workflows` returns `nextCursor` when the n8n body has it.
- [ ] An archived row removed from `result` does not remove `nextCursor`.
- [ ] A body without `nextCursor` produces a tool result without that key.
- [ ] `n8n_list_executions` accepts `cursor` and places it on the n8n query.
- [ ] `n8n_list_executions` returns `nextCursor` when the n8n body has it.
- [ ] Stage 70, Stage 71, and Stage 72 contracts are unchanged.
- [ ] n8n was not contacted to prove this page.

### Rollback

Return only `result` and `total` from both list handlers, and remove `cursor` from the execution schema if it was added. Listing again matches today's first page only.
Loading
Loading