Skip to content

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

opencode-selfsearch

Web search for opencode that runs on your own machine, with no API keys.

opencode's built-in websearch tool routes queries through third-party providers (Exa and Parallel). This plugin replaces it with two tools backed by containers you run yourself:

  • SearXNG — a metasearch aggregator that queries a dozen public engines in parallel. No account, no key, no per-query cost.
  • YaCy — a real crawler with a real index. You point it at documentation sites, it indexes them, and those pages become searchable locally and permanently.
                        ┌─ DuckDuckGo, Google CSE, Brave, Bing, Mojeek, … (public engines)
opencode ──► SearXNG ───┤
                        └─ YaCy ──► your own crawled index (no third party at all)

Be clear about what this does and doesn't solve

Solved: API keys. Nothing here has a credential, an account, or a bill.

Solved: single-vendor dependency. Twelve engines answer instead of one, across separate rate-limit buckets, plus a local index that nothing can throttle.

Partly solved: rate limiting. SearXNG has no index of its own — it queries public engines, and those engines can and do block. What this configuration does is make any one block a non-event. Measured on a residential connection, ten rapid-fire queries:

Engines answering per query Total results
SearXNG defaults, small engine set 1 206
This configuration 5–7 885

The difference is mostly not the engine count. SearXNG benches an engine after a single 429 or 403, and upstream keeps it benched for three minutes — or fifteen days after a Cloudflare CAPTCHA. An agent firing a burst of queries knocks engines out on query one and never sees them again. This config cuts those penalties to seconds.

Only YaCy is truly independent. It is the one component that answers from an index you own. It starts empty; you decide what goes in it.

Requirements

  • opencode 1.18.0 or newer
  • Docker (Docker Desktop on Windows/macOS, Docker Engine on Linux)
  • ~4 GB RAM available for containers, mostly for YaCy's JVM

Quick start

git clone https://github.com/gnubyte/opencode-selfsearch.git
cd opencode-selfsearch
npm install

Start the stack. Windows:

powershell -ExecutionPolicy Bypass -File scripts\setup.ps1

macOS / Linux:

./scripts/setup.sh

The script generates a secret, pulls both images (~500 MB), waits for each service, configures YaCy for private operation, and runs a smoke test. First run takes a few minutes; YaCy's cold start alone is a minute or two because it is a JVM bringing up a Solr index.

Register the plugin in ~/.config/opencode/opencode.jsonc — that path on every platform, including Windows, where opencode uses the XDG layout rather than AppData (C:\Users\<you>\.config\opencode\opencode.jsonc):

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": ["/absolute/path/to/opencode-selfsearch"],
  "permission": {
    "websearch": "deny"
  }
}

On Windows use forward slashes: "C:/Users/you/code/opencode-selfsearch".

The websearch: "deny" line removes opencode's third-party search tool so the model has exactly one way to search. Drop it if you would rather keep both.

Restart opencode (the desktop app reads config at launch) and try it:

opencode run "Use the selfsearch tool to find the current latest Node.js LTS version"

Seeding the local index

YaCy starts with nothing in it. Give it something worth searching:

node scripts/crawl.mjs https://docs.python.org/3/ 2
node scripts/crawl.mjs --status

Or let the agent do it — selfsearch_index is a tool, so "index the FastAPI docs for me" works. Crawling runs in the background and pages become searchable as they arrive; a typical docs site is a few hundred pages in about a minute.

Once indexed, those pages are searchable forever, offline, at zero cost, with no rate limit. Ask for them specifically with source: "local".

The tools

selfsearch

Argument Type Default Notes
query string (required) Supports SearXNG syntax: site:docs.python.org asyncio, !wikipedia relativity
numResults integer 1–20 8 Sliced client-side; SearXNG has no result-count parameter
source all | local all local searches only the YaCy index
categories string general, it, science, news, …
timeRange day | week | month | year Recency filter
language string en, de, en-US, …
page integer 1–5 1 Paging deeper

Wikipedia and Wikidata hits arrive in SearXNG's infoboxes field rather than results, so they are folded in as a Reference section; direct answers appear above the list. Pair with opencode's built-in webfetch to read a result in full — that talks to the site directly and involves no third party.

selfsearch_index

Argument Type Default Notes
action crawl | status status reports index size and crawl progress
url string Required for crawl
depth integer 0–4 2 2 suits most docs sites; 3+ can mean tens of thousands of pages
scope subpath | domain | wide subpath wide follows links off-site — rarely what you want
collection string Label for the corpus

Neither tool is named websearch. opencode filters that exact tool ID against its Exa/Parallel gating, so a replacement using that name would be invisible to the model.

Configuration

Option Environment variable Default Purpose
baseUrl SEARXNG_URL http://localhost:8080 Where SearXNG lives
yacyUrl YACY_URL http://localhost:8090 Where YaCy lives
yacyUser / yacyPassword YACY_USER / YACY_PASSWORD admin / yacy YaCy admin credentials
autoStart SELFSEARCH_AUTOSTART true docker start the containers if a search finds them down
container SELFSEARCH_CONTAINER searxng Container name for that restart
yacyContainer SELFSEARCH_YACY_CONTAINER yacy Same, for YaCy
timeoutMs SELFSEARCH_TIMEOUT_MS 20000 Per-request timeout

Plugin options win over environment variables. Pass them with the tuple form:

{
  "plugin": [["/path/to/opencode-selfsearch", { "baseUrl": "http://searxng.lan:8080" }]]
}

Autostart only restarts existing containers; it never creates one behind your back.

Change the YaCy password. The official image ships admin / yacy baked in. Port 8090 is bound to loopback so this is not exposed, but if you put it on a network, fix it first:

docker exec yacy /opt/yacy_search_server/bin/passwd.sh 'a-strong-password'

then set YACY_PASSWORD to match.

What is configured, and why

Engines. Twelve keyless engines across separate rate-limit buckets: Google lineage (google cse, startpage), Bing lineage on four different hostnames (duckduckgo, duckduckgo web, bing, yahoo), and independent crawlers (brave, mojeek, qwant, seznam, mwmbl, yep). Adding engines does not increase load on any one of them — SearXNG issues exactly one request per engine per search.

Suspension penalties, cut from upstream defaults, as described above. This is the single highest-impact setting in searxng/config/settings.yml.

A latency ceiling. SearXNG waits for the slowest engine before responding, so outgoing.max_request_timeout caps it and the two engines that rate-limit hardest get shorter individual timeouts.

YaCy runs in "robinson" mode — it crawls what you tell it to and exchanges nothing with anyone. Setup disables greedylearning (a fresh peer otherwise crawls 1000 random pages on its own and pollutes your corpus) and both directions of P2P index exchange. Queries never leave the machine. To join the public YaCy network instead, flip search_mode to global on the yacy local engine and re-enable index.receive.allow — at the cost of transmitting your query text to peer operators you do not control.

YaCy's heap is raised to 2 GB. The image ships 600 MB, which is small enough to cause GC thrash that looks like a permanently pinned CPU core. Raise it further with YACY_HEAP=Xmx4096m in searxng/.env.

Multiplatform support

OS Windows 10/11, macOS, Linux
CPU x86-64 and ARM64 — both images are multi-arch, so Apple Silicon needs no flags
Plugin runtime Pure fetch plus node:crypto. The only shell-out is docker start via execFile with plain arguments, so no shell-quoting differences
Setup scripts PowerShell and POSIX shell, functionally identical

Ports bind to 127.0.0.1 only. The compose file sets FORCE_OWNERSHIP=false because Windows and macOS bind mounts cannot be chowned by the container entrypoint, and .gitattributes pins shell scripts to LF so they survive a checkout on Windows.

Verifying it works

curl "http://localhost:8080/search?q=hello&format=json"   # SearXNG directly
node scripts/crawl.mjs --status                           # YaCy index size
npm run smoke                                             # both tools, no opencode
opencode run "List the names of every tool you have available, one per line."

The last should show selfsearch and selfsearch_index, and no websearch.

Managing the stack

npm run up      # start
npm run down    # stop and remove
npm run logs    # follow logs

Both web UIs stay enabled and are useful for seeing what the model sees: SearXNG at http://localhost:8080, YaCy at http://localhost:8090.

Don't want YaCy? Delete its service from searxng/docker-compose.yml and the yacy local engine from searxng/config/settings.yml. Everything else works unchanged.

Troubleshooting

HTTP 403 requesting format=json — SearXNG serves HTML only by default. search.formats must include json, which the shipped config does. No environment variable can set this.

"Could not reach a SearXNG instance"docker ps to check, then npm run up. If you moved it, set SEARXNG_URL.

Container exits immediately — SearXNG refuses to start without a real server.secret_key. The setup scripts write one to searxng/.env.

YaCy seems hung on first start — it is a JVM building a Solr index; minutes is normal. Watch with npm run logs.

Local search returns nothing — the index is empty until you crawl something. Check with node scripts/crawl.mjs --status.

Freshly crawled pages aren't searchable yet — YaCy commits on its own schedule, so there is a short lag between crawling and results appearing.

Some engines listed as unavailable — normal. They recover within seconds under this config. It only matters if the count drops toward zero.

Plugin does not load — the path in opencode.jsonc must be absolute and point at the repo root (the directory with package.json), and you must have run npm install there. opencode --print-logs shows loading errors.

License

MIT

About

Self-hosted web search for opencode: SearXNG metasearch plus a local YaCy index. No API keys, no third-party search providers.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages