Conversation
…tress filters - Actress sync manager with lease-fenced transactional task processing - Built-in actress cache (25k records) with alias-aware build merging - Cache-first DMM ID resolution with post-live fallback for selected sync - Transactional merge with atomic task reassignment and dedupe coalescing - Scraper session-based thumbnail validation with per-redirect SSRF checks - Actress filters: missing/has DMM ID, missing thumbnail, Japanese name only - Fix cross-page selection persistence - Enrich scraped movie actresses from built-in cache with DMM ID guard - Full i18n support (en, en-XA, ja, zh-Hans, zh-Hant)
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #176 +/- ##
==========================================
+ Coverage 91.84% 92.90% +1.06%
==========================================
Files 461 490 +29
Lines 41877 47123 +5246
==========================================
+ Hits 38461 43779 +5318
+ Misses 2224 2174 -50
+ Partials 1192 1170 -22
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
- Regenerate mocks with mockery v3.7.1 to match CI - Skip os.FileMode permission assertions on Windows (no Unix permissions)
Contributor
Author
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 409cead666
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
github-actions
Bot
added
the
🤖 codex: needs work
- Preserve existing ThumbURL for JavDB resolver so it can extract actor ID - Only fetch task list when sync job reaches terminal state, not on every 1.5s poll
github-actions
Bot
removed
the
🤖 codex: needs work
Contributor
Author
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0675393d53
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
github-actions
Bot
added
the
🤖 codex: needs work
Only clear actress thumbnails on deterministic known-invalid placeholders. Remote validation failures (transient network errors, unreachable CDNs) no longer trigger needsPersistence, preventing permanent erasure of valid URLs.
github-actions
Bot
removed
the
🤖 codex: needs work
Contributor
Author
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 41937d66aa
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
github-actions
Bot
added
the
🤖 codex: needs work
- Test wrapper functions without task context - Test CancelJob with pending task for correct status transition
- Only clear ThumbURL on deterministic IsKnownInvalidDMMActressThumbnail - Remote validation failures no longer erase valid URLs - Remove query string exclusion from thumbnail filter SQL - Update test to use known-invalid DMM thumbnail
github-actions
Bot
removed
the
🤖 codex: needs work
- Add tests for all sync_jobs API endpoints (create, list, get, tasks, cancel) - Test validation errors, not found cases, and helper functions - Fix P1: don't clear thumbnails on transient validation failures - Fix P2: remove query string exclusion from thumbnail filter SQL
Contributor
Author
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1209f8e96f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
github-actions
Bot
added
the
🤖 codex: needs work
- TableName, SplitFullName, NormalizeActressNameKey in models - ValidateRemoteImage, ValidateRemoteImageWithSafeClient in imageutil - RenameNameFields, ReplaceThumbnailForSyncTask, AssignDMMIDIfMissingForSyncTask in database - ListActresses in r18devdump - newScraper, Name, IsEnabled, Config, Close, GetURL, validateScraperSettings, Register in minnanoav - RegisterAll in scraper registration - ValidateActressThumbnail, ResolveActressThumbnail, ResolveActressMetadata, fetchActressMetadataDoc in dmm - ValidateActressThumbnail in javdb and r18dev - listActressSyncCandidates, RegisterRoutes in api/actress - ListActiveJobs on ActressSyncManager - Name() in legacycsv source
… SSRF at connection time - Reuse ListSyncCandidates results to avoid N+1 FindByID queries - Log and retry dispatcher startup on transient database errors - Wrap scraper client transport with SSRF check at connection time
github-actions
Bot
removed
the
🤖 codex: needs work
- actressMetadataVerified: all branches (nil, no match, DMM mismatch, name/thumb/first-last match) - actressNeedsMetadata: nil, empty, complete cases - actressThumbnailNeedsResolution: empty, invalid, valid - actressThumbnailSourcePriority: all scraper names - scraperThumbnailCanRefresh: all host patterns - authoritativeActressScrapers/actressMetadataScrapers: nil interface - actressIdentityNames: parentheses and comma parsing - identityNameMatches: case insensitive, empty - canMergeMissingDMMActress: all edge cases - lookupActressCache: nil inputs - cacheFallbackMatch: with and without existing matches - needsLinkedActressFallback: complete and incomplete
- worker: persist no outcome on manager shutdown; recovery requeues instead - api: sync-candidates swagger description matches actual dmm_id<=0 rows - frontend: toast and abort on empty-queue refresh failure when advancing
- sync: honor configured scraper priority/orders and actress field priority for resolver order, name blending, and thumbnail picks; capability-gate per-field picks via optional ActressFieldCapable - scrape: blend actress enrichment per field by configured priority instead of first-resolver-wins; keep capability gating and early-exit - api: bound the default sync task list (limit param, default 500, cap 1000) - actresscache: trust the configured proxy's own address at dial time - frontend: hide actress-only resolvers from movie field editors - frontend: reconcile selections against refreshed queries after merges
- scrape: thread progressively discovered values into later resolvers; honor actress field overrides exclusively and suppress on the skip sentinel - worker: deterministic rank-based actress resolution when priorities are configured; same-rank ties still surface conflicts; exclusive actress overrides restrict sync resolvers too - actresscache: probe HTTP and HTTPS proxies separately for the diverged proxy allowlist - priority UI: actress-only resolvers are never valid movie-field sources; enforce in the field editor, its initial list, and add-all
- actresscache: probe the proxy route per request scheme (HTTP vs HTTPS) - worker: keep linked-movie fallback provenance so configured priority resolves cross-source disagreements deterministically - priority UI: never rewrite the stored field list on editor open
- database: fence task mutations on parent job cancellation (lease validity alone allowed post-cancel commits) - scrape: explicit scraper selection is exclusive for actress enrichment - priority UI: actress-only resolvers stay configurable for the actress field, but a solely-actress-only override is rejected at save
- database: fence task-scoped merges (reassignTaskActressTx) on job cancel - scrape: only explicit selections make enrichment resolver set exclusive
- commandutil: refcount the r18dev dump handle so hot-reload closes it only after in-flight lookups drain - scrape: URL-derived PriorityOverride must not make enrichment exclusive; only caller-supplied selections do - minnanoav: honor explicit rate_limit: 0 (no delay) per documented contract
- database: skipped coalescing winners free their dedupe key, preventing unique-index rollbacks during merge migration - scrape: cached path treats only caller selections as exclusive overrides, matching the fresh path - worker: thread progressively discovered actress values into later resolvers - scraperconfig: defaults merge no longer clobbers explicit rate_limit: 0 - commandutil: Close blocks until in-flight dump lookups drain (Windows-safe) - actresscache: document the accepted proxy-side resolution residual
- database: dedupe/winner joins exclude cancel-requested jobs; createActressSync supersedes a cancelling task's key so retries stay runnable - config: YAML + JSON decode track rate_limit presence incl. request_delay; marshalling omits inherited zeros so round trips persist nothing synthesized
- database: free a cancelling task's canonical key before merge migration - worker: clear resolver-reported fields the source does not advertise - database: sync-candidate prefilter matches only real placeholder thumbs
- worker: skip/exclusive actress overrides apply to identity recovery and linked-movie fallback scrapers too - api: reject actress-only-sole metadata.priority.actress overrides on save - config: legacy request_delay/max_retries aliases no longer clobber explicitly configured canonical keys
- database: cancelling running sources no longer displace pending canonical winners during merge migration - core: warn loudly (boot and reload) when the actress priority override exclusively lists actress-only resolvers
- database: skip-winner merge migration requires the canonical key holder, so a deferred winner never collapses a merge with the running holder's key - models: ResolveActressMetadata now returns (ActressInfo, error) so transient scraper failures surface as task warnings instead of silent skips
- worker: linked-movie identity recovery propagates scraper errors; a task fails for retry when no usable match survives an outage - javdb: findActorID surfaces fetch/parse failures instead of "no actor" - minnanoav: search rejects non-2xx HTTP status (Resty does not by default)
- worker: catalog misses and partial outages no longer fail linked-movie identity recovery; only a full outage with nothing usable does
- minnanoav: redirect policy allows only minnano-av.com hosts (SSRF) - worker: dispatch loop rechecks shutdown before each claim; job cancel sweeps every in-flight run of a task - database: actress delete cancels then detaches sync tasks (SQLite FK enforcement is off in production)
- database: actress delete refreshes parent job aggregates - database: merges re-anchor terminal sync tasks to the canonical actress - frontend: reset the poll-failure toast latch when advancing sync jobs
- database: merge re-anchors terminal tasks before deleting the source, working under both FK-on and FK-off SQLite - worker: CreateJob skips merge-deleted selection IDs instead of rejecting - api/core: capability validation covers every metadata priority field, on save (hard reject) and boot/reload (warnings) - frontend: reset the poll-failure latch on direct job assignment paths too
- actresscache: DMM-anchored merge candidates also pass Japanese-name conflict checks when the target group has no DMM anchor - actresscache: block IPv6 transition/site-local/doc ranges (6to4, Teredo, fec0::/10, 2001:db8::/32) - actresscache: cap decoded thumbnails at 20 MP (~80 MB RGBA) instead of 100 MP to bound parallel-builder memory
- worker: romanized identity names join linked-movie recovery matching - database: a deferred merge winner absorbs redundant duplicate tasks instead of queueing a second run - api: tasks endpoint reports the real per-view total, not the page length
- database: deferred coalescing no longer reports "cancelled" for an un-cancelled job's running task - core: compute actress-priority warnings once per reload
github-actions
Bot
removed
the
🤖 codex: needs work
codecov ignore list; also fix some defensive-callarms for easier auditing
seeyabye
added a commit
that referenced
this pull request
Aug 12, 2026
…unified SSRF guard Phase 1 of the actress metadata sync re-org (supersedes PR #176 in slices). - internal/actresscache: embedded cache build pipeline, runtime lookup and thumbnail validation journal; fsync-before-rename; corrupt state files quarantined instead of bricking builds - cmd/build-actress-cache (+ make target): maintainer cache builder; r18dev dump source is lister-injected with CLI-owned store lifetime; minnanoav page parsing is cache-owned (no scraper import) - internal/ssrf: single SSRF guard with full blocked-CIDR set, dial pinned to validated IPs (no connect-time re-resolution, DNS failover kept), typed BlockedTargetError/UnverifiableHostError; legacy CheckURL, NewSSRFSafeClient and WrapTransportWithSSRFCheck keep their signatures over the new internals; AllowHostForTest/SetLookupIPForTest test seams - internal/imageutil: fail-closed for non-*http.Transport clients, cached direct transport clone plus CloseIdleConnections on per-attempt clones, checked DefaultTransport assertions - builtin Lookup falls through to the romanized index on jp-name miss - proxied DNS fail-closed is transient failed, never permanent rejected - api/system + api/temp tests now use the explicit allow-host seam (they relied on the old check-then-dial window)
6 tasks
seeyabye
added a commit
that referenced
this pull request
Aug 14, 2026
* feat(scrape): port P5 actress-scrape-integration from #176 + compatibility fixes * fix(scrape): B-SCRAPE fixes — fail-closed thumbnail validation + r18dev dump hardening * fix(r18devdump): install SSRF PinnedDial transport on production clients * fix: address codex review findings P1-1/2/3 + P2-6/8 * fix: forward RawInput to queryAll + parse URL-shaped MovieID (codex P1 round 2) * fix: restrict SSRF bypass to localhost + handle URL parse failures + chain redirect policy (codex round 3) * fix: redact MovieID on URL parse failure + trim whitespace in actress field checks (codex round 4) * fix: whitespace actress fields + non-URL parse + mirror redirect allowlist (codex round 5) * fix: preserve raw URL on parse failure + rebuild provenance after enrichment (codex round 6) * fix: propagate resolver warnings via Config + ScrapeResult.Warning + trim RawInput (codex round 7) * fix: request-local resolver warnings via *[]string + trim RawInput (codex round 8) * fix: build cache provenance after actress enrichment (codex round 9) * fix: gate built-in cache by actress priority + propagate resolver warnings via OrchestrationMeta (codex round 10-11) * fix: revalidate redirects after callback + preserve cookie jar (codex round 11) * fix: honor inherited scraper priority for cache + move provenance after resolver + copy HTTP client (codex round 12) * fix: OrchestrationState.ResolverWarning + invalidate translation on enrichment (codex round 13) * fix: enrichment before translation in cache + remove shadowing ResolverWarning (codex round 14) * fix: invalidate cached translation when actress enrichment changes names (codex round 15) * fix: skip translation validity check when actress enriched (codex round 16) * fix: resolve actresses missing either romanized name + fix test (codex round 17) * fix: apply De Morgan law + honor explicit scraper selection for cache + provenance before enrichment (codex round 18) * fix: remove ScrapeActress check from cache gate (default bool is false) + DMM-first + provenance after enrichment (codex round 19) * fix: honor explicit priority for cache gate + check resolver capability before breaker probe (codex round 20) * fix: preserve actress field priority on default scrapes (codex round 21) * fix: ignore synthesized URL priority on cache hits (codex round 22) * fix: allow DMM cache as fallback when DMM is in priority list (codex round 23) * fix: merge pre+post enrichment provenance keys (codex round 24) * fix: init provenance map before merge + DMM-first gate for cache (codex round 25) * fix: limit overflow reader to remaining allowance (codex round 26) * fix: overflow reader exact-size + explicit selection overrides field priority (codex round 27) * fix: overflow reader probe one extra byte at exact limit (codex round 28) * fix: only skip pinned transport for explicit env override (codex round 29) * fix: only SelectedScrapers makes resolver exclusive + ActressFieldPriority takes precedence (codex round 30) * chore: codex review round 31 — 0 P1, 1 P2 (provenance re-keying, deep design issue) * fix: re-key provenance to post-enrichment identity (codex round 32) * fix: deterministic provenance re-keying using saved old keys (codex round 33) * fix: expose resolver warnings in ScrapeResponse API (codex PR review) * fix: apply circuit breaker to actress resolver calls (codex PR review) * fix: record actress resolver outcomes in circuit breaker (codex PR review) * fix: surface breaker skips as warnings + ignore canceled context in recordOutcome (codex PR review)
seeyabye
added a commit
that referenced
this pull request
Aug 15, 2026
seeyabye
added a commit
that referenced
this pull request
Aug 15, 2026
* feat(frontend): port P6 actress-frontend-sync from #176 + P6 types + i18n keys * fix: remove accidentally committed node_modules_bak + add to .gitignore * fix: revert browse/review/actresses pages to epic version (pre-existing features not in P6 scope)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a comprehensive actress metadata sync system with a built-in cache, server-side filtering, and several UX improvements to the actresses page.
Key Features
Actress Metadata Sync System
Actress Filters
Cross-Page Selection Fix
Scrape Cache Enrichment
Full i18n Support
Testing