MinifigFolio — scraper extension

A Chrome extension (Manifest V3, WXT, vanilla TypeScript, no UI framework) that scrapes LEGO® set and minifigure data for the MinifigFolio web app. All portfolio management — tracking lots, collections, orders, auctions, the photo scanner, AI pitch scripts — lives in the web app now. This extension has one job: read prices and catalog data off the sites you browse and push them to the app.

It requires a signed-in MinifigFolio account. Signed out, every content script and the scrape runner refuse to do anything.

What it scrapes

Site	Content script	What it reads
brickset.com/sets/*, /minifigs/*	entrypoints/content.ts	item name/image/theme/subtheme/year/character, "Appears in" set ids, retired status, and market new/used values, from collection cards, table rows and detail pages
www.brickeconomy.com/minifig/*	entrypoints/brickeconomy.content.ts	minifig value predictions/growth/forecasts, the "good investment?" insight quote, the "Minifig Facts" paragraph, released/availability
www.bricklink.com/catalogPG.asp*, /v2/catalog/catalogitem.page*	entrypoints/bricklink-pg.content.ts	the "Last 6 Months Sales" table, reduced to a quantity-weighted average sold price (new/used)

Everything scraped is pushed to POST /api/ingest on the app (see lib/sync.ts), authenticated with the signed-in user's token. Item stubs travel alongside prices because the app only accepts prices for ids its catalog already knows.

Owned-count write-back

The one thing the extension still writes into a third-party site: on Brickset collection/detail pages it mirrors the tracked owned count (pulled from the app, local:items — a read-only mirror) into Brickset's own "I own" input, and ticks "I want" for items in transit or sold. Default is up-only (never lowers a count you own on Brickset but haven't tracked); "Mirror exactly" in Settings also lowers it on a sale. The same, opt-in, for BrickEconomy's loose-minifig checkbox/quantity field.

Context menu → app scanner

Right-clicking any image offers "Search this image with Minifig Folio". The background worker fetches the image, stashes it as a data URL in session:pendingScan, and opens {appUrl}/scanner. The app-bridge content script (entrypoints/appbridge.content.ts) picks it up from there and hands it to the app's scanner page over postMessage — recognition itself (Brickognize) runs in the app, not here.

Popup and options

• Popup (entrypoints/popup/) — a status panel: signed-in account, a sync button with live state, scrape progress, how much cached data this device holds, and links to open the app / the options page.
• Options (entrypoints/options/) —
  • App account — sign in/out, the app URL, a live sync monitor, and manual Push new data only / Push full catalog / Pull from app actions.
  • Data sources — enable/disable Brickset (always on) / BrickLink / BrickEconomy.
  • Loose-item sync — the owned-count write-back settings above, for Brickset and BrickEconomy.
  • Scraping — pace between pages, and a bulk range scan that seeds the catalog by opening every id in a prefix/from/to range (checked against the app's catalog first) across the enabled sources.
  • Cache — clear passively-cached item data for items you don't track.

Storage keys that remain (chrome.storage.local via WXT storage.defineItem)

Key	Purpose
local:items	read-only mirror of the app's portfolio (pullFromApp) — feeds the owned-count write-back only
local:meta	cached item metadata (name/image/theme/character/…), incl. BrickEconomy data (Meta.be)
local:markets / local:marketsAt	cached Brickset market values + last-scrape timestamp
local:blMarkets	independent BrickLink 6-month sold averages
local:pushedPrices	fingerprint of the prices last pushed per item, so unchanged prices aren't resent
local:dataSources	which sources (Brickset/BrickLink/BrickEconomy) are enabled
local:scrapeDelay	pause between pages in the paced scraper (seconds)
local:syncLoose / local:syncBrickeconomy / local:syncLooseBrickeconomy	owned-count write-back settings
local:appSession	Supabase access/refresh token pair (password never stored)
local:appUrl / local:appConfig	app URL + cached Supabase endpoint/key
local:syncState	push/pull status (drives the popup + toolbar badge)
local:scrapeState	background-tab scraping progress (drives the toolbar badge)
session:pendingScan	the context-menu image, waiting for the app's scanner page to collect it

Dev

pnpm install
pnpm dev      # launches Chrome with the extension loaded (HMR)
pnpm build    # production build -> .output/chrome-mv3
pnpm compile  # tsc --noEmit (typecheck)

Load a production build manually: chrome://extensions → enable Developer mode → Load unpacked → select .output/chrome-mv3.

pnpm dev is an interactive process; if you run it detached, keep stdin open (e.g. tail -f /dev/null | pnpm dev) or it exits on EOF.

Layout

• lib/store.ts — scrape caches (local:meta, local:markets, local:blMarkets), data-source and write-back settings, chrome.storage.local persistence.
• lib/sync.ts — catalog push (queueCatalogPush, pushNewCatalogData, pushAllCatalog, catalogRange) and the portfolio pull (pullFromApp); local:syncState for the popup/badge.
• lib/auth.ts — app account: config bootstrap (GET /api/config), sign-in against Supabase, token refresh, appFetch() (relays through the background worker from a content script, since that runs in the page's origin and hits CORS otherwise).
• lib/scrape.ts — the paced background-tab runner (scrapeIds / scrapeBrickLink / scrapeBrickEconomy) behind the bulk range scan and the app's own bpt-scrape-request message; mirrors progress into local:scrapeState.
• lib/badge.ts — toolbar badge (green % while scraping, blue ↑/↓ while syncing, red ! on a sync error).
• lib/ids.ts — item id helpers (kindOf, isCustomId).
• entrypoints/background.ts — context menu, the bpt-app-fetch relay, the 15-minute sync alarm, and the scrape/sync message handlers the app-bridge uses.
• entrypoints/appbridge.content.ts — announces the extension to the app's own pages and relays scrape/sync requests + the context-menu handoff over postMessage.

Conventions

• Conventional Commits, in English. No Co-Authored-By trailer.
• All UI text is English. Prices are currency-agnostic numbers; market parsing assumes dot-decimal (Brickset shows ~€3.96).

Known limitations

• Chrome/MV3 only; a Firefox build target exists in scripts but is untested.
• BrickEconomy sits behind Cloudflare, so it can only be scraped from a real browser tab, not a server-side fetch.