feat(web): add web search through your own SearXNG

AGENTS.md

@@ -14,7 +14,8 @@ famstack-extensions/
 ├── <id>/                 one extension per root folder, never deeper
 │   ├── stacklet.toml     id = folder name
 │   ├── README.md         what it does, setup, what was tested on which hardware
-│   └── tests/            run inside famstack (FAMSTACK_REPO)
+│   └── tests/            run inside famstack, through its extension lane
+├── docs/testing.md       how to test an extension, and the pitfalls
 ├── script/cibuild        everything CI checks; runs the same on a laptop
 ├── script/check-manifests.py
 └── .github/workflows/ci.yml
@@ -26,13 +27,20 @@ famstack-extensions/
   repo root, no extension inside another's folder. `script/check-manifests.py`
   enforces it, and refuses an `id` famstack ships in `stacklets/` (the release
   wins that clash, so the extension would never load).
+- **Ports 42100 to 42199**, the range famstack's ADR-007 keeps for extensions:
+  `port`, `[ports]` and each compose binding's host port. No two extensions
+  share a port. `script/check-manifests.py` enforces both.
 - **An extension stands alone.** It is copied without the rest of the repo, so
   it imports nothing from outside its own folder.
 - **Made-up data only.** Tests, examples and docstrings never carry a real
   address, device ID, room or person's name. Fabricate them.
-- **Tests run inside famstack**, the way `./stack <id> <cmd>` runs, through
-  `Stack.run_cli_command`. `FAMSTACK_REPO` names the famstack checkout; CI uses
-  the branch in `.github/workflows/ci.yml` (`FAMSTACK_REF`).
+- **Tests run inside famstack**, through its extension lane:
+  `"$FAMSTACK_REPO/script/test" extension <id>`. A test drives `./stack <id>
+  <cmd>` as a process (the fixture `stack`), stands in for every service by its
+  container address, and imports the extension's modules only as
+  `stacklets.<id>.*`, never through `sys.path`. `docs/testing.md` has how and
+  the pitfalls. CI uses the famstack branch in `.github/workflows/ci.yml`
+  (`FAMSTACK_REF`).
 - **Other makers' names only to say what an extension works with.** The
   manifest's `name` combines a brand with plain words ("Hue lights", never
   "Hue" alone), no logos, and the README says the extension is not affiliated

README.md

@@ -7,6 +7,7 @@ for the core. Each folder here is one extension.
 | Extension | What it does | Stage |
 |---|---|---|
 | [hue](hue/) | Philips Hue scenes, brightness, on and off over the bridge's local API. No Philips cloud. | incubating |
+| [web](web/) | Private web search through your own SearXNG, and a question answered from the results with its sources. | incubating |
 
 The extensions here follow famstack's integration branch, `feature/0.5.0-beta.1`.
 They need famstack 0.5, which is not released yet.
@@ -49,7 +50,9 @@ Open a pull request with:
 - one folder, named after the stacklet's `id`, with `stacklet.toml` at its top
   ([creating stacklets](https://github.com/famstack-dev/famstack/blob/main/docs/creating-stacklets.md));
 - a README: what it does, how to set it up, what was tested and on which hardware;
-- tests under `<id>/tests` that run inside famstack (see hue's);
+- ports in 42100 to 42199, the range for extensions, one that no other extension uses;
+- tests under `<id>/tests` that run inside famstack, through its extension lane
+  ([testing an extension](docs/testing.md));
 - made-up data in tests and examples: no real addresses, device IDs or names.
 
 CI checks every manifest and runs every extension's tests against famstack. The
@@ -63,5 +66,5 @@ FAMSTACK_REPO=<your famstack checkout> script/cibuild
 
 AGPL-3.0, like famstack.
 
-Product names here (Philips Hue and others) are trademarks of their owners. The
+Product names here (Philips Hue, SearXNG and others) are trademarks of their owners. The
 extensions are not affiliated with or endorsed by them.

docs/testing.md

@@ -0,0 +1,199 @@
+# Testing an extension
+
+## What is the extension lane?
+
+The **extension lane** runs the tests of an extension against a famstack checkout. An [extension](https://github.com/famstack-dev/famstack/blob/feature/0.5.0-beta.1/docs/admin-guide.md#extensions) is a stacklet that famstack does not ship. Its code lives in this repository, but it runs inside the stack. Because of this, its tests run inside the stack too.
+
+For example, [`web/tests/test_search.py`](../web/tests/test_search.py) runs `./stack web search` as a family member does. The stack is a made-up instance on your machine, and SearXNG is a stand-in on `127.0.0.1`.
+
+## What does a test check?
+
+A test drives what a family member or an agent can call, and checks what they get back. That is the exit code, the JSON and the text on the screen. A test does not import a command's code and does not call its functions. Such a test passes when the command is broken, because it skips the path that the stack takes.
+
+| What the extension has | How a test drives it | Where it runs |
+|---|---|---|
+| A command, `./stack <id> <command>` | `stack.run("<id>", "<command>", ...)` | CI, any machine |
+| Work that a command runs in the bot runner | The same call | CI, any machine |
+| A service that the work reaches | A stand-in on `httpserver`, named with `stack.serves` | CI, any machine |
+| The AI model | The `model` fixture | CI, any machine |
+| A module of pure code | `from stacklets.<id>.<module> import ...` | CI, any machine |
+| Its containers | A live test against a running instance | A Mac with the extension up |
+
+## How does a command run in a test?
+
+Each `stack.run` starts `./stack` as its own process, the way a person starts it. When the command has work for the bot runner, the stand-in for Docker starts that work as one more process on your machine:
+
+```
+ the test (pytest)
+ ┌───────────────────────────────────────────────┐
+ │ stack.run("web", "search", "tent", "--json")  │
+ └───────────────────────┬───────────────────────┘
+                         │ a process: ./stack, STACK_DIR = the instance
+                         ▼
+ ┌───────────────────────────────────────────────┐
+ │ ./stack web search tent --json                │
+ │ web/cli/search.py, on the host                │
+ └───────────────────────┬───────────────────────┘
+                         │ docker exec stack-core-bot-runner python ...
+                         ▼
+ ┌───────────────────────────────────────────────┐
+ │ the bot runner, stood in: a process here      │
+ │ web/bot/cli_entrypoint.py search tent         │
+ │ env: core's, then web's own .env              │
+ └───────────────────────┬───────────────────────┘
+                         │ http://stack-web-search:8080
+                         ▼
+ ┌───────────────────────────────────────────────┐
+ │ httpserver on 127.0.0.1: the test's SearXNG   │
+ └───────────────────────────────────────────────┘
+```
+
+The command's code never loads into the test's process. Because of this, a test sees what a person sees: the same discovery, `--json` output and exit code.
+
+## How to run the tests
+
+You need a famstack checkout and [uv](https://docs.astral.sh/uv/). The lane uses the test environment of that checkout.
+
+To run the tests of one extension, run this from the root of this repository:
+
+```bash
+"$FAMSTACK_REPO/script/test" extension web
+```
+
+To run one test file, give its path:
+
+```bash
+"$FAMSTACK_REPO/script/test" extension web/tests/test_search.py
+```
+
+The lane runs the tests in parallel, with one worker for each core. To run them in one process, for example to debug a test, add `-n 0`:
+
+```bash
+"$FAMSTACK_REPO/script/test" extension web -n 0
+```
+
+To run every check that CI runs, run:
+
+```bash
+FAMSTACK_REPO=<your famstack checkout> script/cibuild
+```
+
+CI tests against the famstack branch in `.github/workflows/ci.yml` (`FAMSTACK_REF`). Use a checkout of the same branch, because the lane and the framework change with famstack.
+
+## How to write a test for a command
+
+1. Put the test in `<id>/tests/test_<command>.py`.
+2. Ask for the fixture `stack`. It is your extension, installed and up in a made-up instance.
+3. Stand in for each service that the command reaches (see [How to stand in for a service](#how-to-stand-in-for-a-service)).
+4. Run the command with `stack.run`, with the same words that a person types after `./stack`.
+5. Assert the exit code first, with `ran` as the message. The message then shows the output of the command when the test fails.
+6. Assert the JSON, or facts in the text on the screen.
+
+This test is from [`web/tests/test_search.py`](../web/tests/test_search.py):
+
+```python
+def test_count_is_how_many_results_come_back(stack, searxng):
+    """Given a full page of results, when an agent asks for `--count 3`,
+    then three come back: an agent budgets its context by this number."""
+    ran = stack.run("web", "search", "immich vs photoprism", "--count", "3", "--json")
+
+    assert ran.code == 0, ran
+    assert len(ran.json["results"]) == 3
+```
+
+Give each test a name that says what the caller sees. Write its docstring as given, when and then, plus one sentence on why the case matters.
+
+## How to stand in for a service
+
+On a family's Mac, the work of a command reaches a service by its container address. An example is `http://stack-web-search:8080` on the stack network. In a test, nothing answers at that name. You tell the instance which stand-in answers instead.
+
+| | On a family's Mac | In a test |
+|---|---|---|
+| Where the address comes from | `[env.defaults]` in `stacklet.toml`, rendered into the extension's `.env` by `stack up` | The same |
+| What answers at `stack-web-search:8080` | The SearXNG container | `httpserver`, named with `stack.serves` |
+| A container that is not running | The connection is refused | `stack.stopped("stack-web-search:8080")` |
+
+Put the address of a service in `[env.defaults]`, and read it from the environment in the code:
+
+```toml
+[env.defaults]
+SEARCH_URL = "http://stack-web-search:8080"
+```
+
+Then name the stand-in in the test:
+
+```python
+def test_search_answers_what_searxng_found(stack, httpserver):
+    httpserver.expect_request("/search").respond_with_json({"results": RESULTS})
+    stack.serves("stack-web-search:8080", httpserver)
+
+    ran = stack.run("web", "search", "tent", "--json")
+```
+
+When several tests use the same stand-in, put it in its own module in `<id>/tests/`. Then give a fixture for it in `conftest.py`. Web's SearXNG is in [`web/tests/searxng.py`](../web/tests/searxng.py).
+
+> [!WARNING]
+> **Name a stand-in for every container address that a command reaches.** The lane guards only the test's own process against the network. The `./stack` processes of a test are not guarded. An address without a stand-in makes a name lookup that can leave your machine.
+
+## How to stand in for the model
+
+Ask for the fixture `model`. The work of a command in the bot runner then reaches the stand-in instead of the model.
+
+| To | Write |
+|---|---|
+| Answer the next requests with a text | `model.answers("Yes. Immich keeps the RAW file [1].")` |
+| Answer with JSON | `model.answers({"title": "Tent for four"})` |
+| Make every request fail with an HTTP status | `model.refuses(500)` |
+| Read the messages that the model got | `model.requests[-1][-1]["content"]` |
+
+The stand-in is the model's HTTP API, not a mock of the client. The real AI client of famstack sends the request and reads the answer.
+
+## How to test a module
+
+A module of pure code (no network, no files) gets module tests. The lane makes each file of your extension importable as its package, `stacklets.<id>`:
+
+```python
+from stacklets.web.ask import build_prompt, sources_from
+```
+
+Import a module by this name only. Do not put a folder on `sys.path`, and do not load a file by its path. A module that loads under two names is two modules, and its classes do not match.
+
+## How to test the containers
+
+The lane runs no containers. To check a container, write a live test that runs against an instance where the extension is up. Web's live tests are in `web/tests/live/`. CI skips them, because they skip when nothing answers on the extension's port.
+
+To run them on a Mac, run these commands:
+
+```bash
+./stack extensions add web
+./stack up web
+"$FAMSTACK_REPO/script/test" extension web/tests/live
+```
+
+## What the fixtures give
+
+| Fixture | What it is | What it has |
+|---|---|---|
+| `stack` | Your extension, installed and up in a made-up instance | `run(*args)` gives `ran.code`, `ran.out`, `ran.err` and `ran.json`. `serves(address, httpserver)`, `stopped(address)`, `uses_model(model)`, `up()` |
+| `model` | The model's HTTP API, on `httpserver` | `answers(...)`, `answering(fn)`, `refuses(status)`, `requests` |
+| `httpserver` | A web server on `127.0.0.1` for this test ([pytest-httpserver](https://pytest-httpserver.readthedocs.io/)) | `expect_request(path)`, `url_for(path)`, `log` |
+
+The instance is the made-up Simpson family. Its `stack.toml` has a default AI model, as the installer writes it. A test must leave no process, server or file behind, because the tests run in parallel and in any order.
+
+## Pitfalls
+
+| You see | The cause | Do this |
+|---|---|---|
+| `web is shipped in .../stacklets/web: the release wins that clash, so the extension in ... would never load.` | famstack ships a stacklet with the same `id`. | Give the extension another `id`, or remove the stacklet from famstack first. |
+| `web has no settings yet: stack up web` | Its `.env` does not exist, because the extension was never brought up. | Use the `stack` fixture, which brings the extension up. With `installed(..., up=False)`, this error is the expected answer. |
+| `web is in ..., which stack-core-bot-runner does not see: it mounts only the first of [core] extension_dirs.` | The extension is in a second extension folder. | Install it in the first extension folder. |
+| `--json` prints the text for people and then `{"ok": true}` | The command streams the output of its work (`bot_runner.dispatch`). | Call the work with `bot_runner.call_stacklet`, and let the work answer JSON. In the host command, print the text for people and return the dict. |
+| A stand-in gets no requests, and the command reports a connection error | The address is fixed in the code, or the test named a different address. | Read the address from `[env.defaults]`. Name the same `name:port` in `stack.serves`. |
+| `ValueError: No model configured for '<id>/<task>'` | The work asks the model in an instance without a default model. | Use the `stack` fixture. Its instance has a default model. |
+| The model stand-in stops answering when a second stand-in changes its answer | `httpserver.clear_all_handlers()` removes every handler on the server, the model's too. | Register one handler per stand-in that reads its current answer. See [`web/tests/searxng.py`](../web/tests/searxng.py). |
+| A test passes alone and fails in CI | The test left something behind: a process, a server or a changed file. Or it depends on the order of the tests. | Stop servers when the block ends, wait for processes, and use `tmp_path` for files. |
+| A test fails after a change to the text of a message | The test checks the wording. | Check the facts that the message carries. Examples are the command to run, a path and an `id`. |
+| A test is slow | Each `./stack` call starts a process, about 0.2 to 0.6 seconds. | Keep to one or two commands per test. Say in the docstring why a test takes more than a second. |
+| An entry point fails to write a file under `/data` in a test | The stand-in for the bot runner maps only the code folders, `/stacklets/` and `/extensions/`. | Test that code as a module, or in a live test. |
+
+Use made-up data in tests and fixtures: no real addresses, device IDs, rooms or names. Web's recorded search answer is made up too ([`web/tests/fixtures/README.md`](../web/tests/fixtures/README.md)).

script/check-manifests.py

@@ -11,6 +11,11 @@
   agree;
 - not take an `id` famstack ships in `stacklets/`: discovery lets the
   release win that clash, so the extension would never load.
+- publish its ports in 42100 to 42199 (famstack's ADR-007), the range for
+  extensions: `port`, `[ports]` and the host side of each port binding in
+  its `docker-compose.yml`. famstack's own stacklets use the rest of 42xxx,
+  so an extension there could take a port a family's stack already uses.
+  Two extensions on one port are refused too, because a family may run both.
 
 A manifest at the repo root is refused: the installer would read this repo
 as one single extension instead of a catalog.
@@ -21,13 +26,19 @@
 
 from __future__ import annotations
 
+import re
 import sys
 import tomllib
 from collections.abc import Iterator
 from pathlib import Path
 
 ROOT = Path(__file__).resolve().parents[1]
 REQUIRED = ("id", "name", "description")
+EXTENSION_PORTS = range(42100, 42200)
+
+# The host port of a compose binding: `"${PORT_BIND_IP:-127.0.0.1}:42100:8080"`
+# or `- 42100:8080`, the number before the container's port.
+_BINDING = re.compile(r"(\d+):\d+(?:/(?:tcp|udp))?[\"']?\s*$")
 
 
 def shipped_ids(famstack: Path) -> set[str]:
@@ -40,10 +51,24 @@ def shipped_ids(famstack: Path) -> set[str]:
     return ids
 
 
+def ports_of(folder: Path, manifest: dict) -> set[int]:
+    """The host ports an extension publishes: its manifest's `port` and
+    `[ports]`, and the host side of its compose bindings."""
+    ports = {manifest["port"]} if isinstance(manifest.get("port"), int) else set()
+    ports |= {p for p in (manifest.get("ports") or {}).values() if isinstance(p, int)}
+    compose = folder / "docker-compose.yml"
+    if compose.is_file():
+        for line in compose.read_text(encoding="utf-8").splitlines():
+            if match := _BINDING.search(line.split("#", 1)[0]):
+                ports.add(int(match.group(1)))
+    return ports
+
+
 def problems(root: Path, shipped: set[str]) -> Iterator[str]:
     """Every rule above that a folder in `root` breaks, one line each."""
     if (root / "stacklet.toml").exists():
         yield "stacklet.toml at the repo root: extensions live one folder down"
+    taken: dict[int, str] = {}
     for manifest in sorted(root.glob("*/stacklet.toml")):
         folder = manifest.parent.name
         try:
@@ -58,6 +83,12 @@ def problems(root: Path, shipped: set[str]) -> Iterator[str]:
             yield f"{folder}: id is '{raw['id']}'; name the folder after it"
         if raw.get("id", folder) in shipped:
             yield f"{folder}: famstack already ships a stacklet with this id"
+        for port in sorted(ports_of(manifest.parent, raw)):
+            if port not in EXTENSION_PORTS:
+                yield f"{folder}: port {port} is outside 42100-42199, the range for extensions"
+            elif port in taken:
+                yield f"{folder}: port {port} is {taken[port]}'s already"
+            taken.setdefault(port, folder)
 
 
 def main(argv: list[str]) -> int: