Skip to content
daggerokPublic

About

Sprott ETF holdings-to-watchlist: static ./api/sprott feed generated by a Bun updater from sprottetfs.com fund pages, SEC EDGAR N-PORT-P (Sprott Funds Trust), Yahoo Finance history/dividends

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

82 Commits

Folders and files

Repository files navigation

Sprott

One of the app's features lets you select Sprott ETFs in the Watchlist and aggregate their holdings to see how often each ticker appears across the selected funds. Repeated holdings make overlapping exposure visible: the more selected funds include a ticker, the greater its potential influence on the portfolio; gains in that holding may help, while declines may hurt, and actual impact also depends on each fund's position size. Another feature makes it faster and easier to find funds with stronger growth over different periods, higher dividend yields or distributions, greater Total Return (price performance plus dividends), and other key performance metrics. A single-file client-side tool that reads the generated ./api/sprott static feed (the sprottetfs.com fund navigation and official fund pages - key facts, expenses, performance and distribution tables, plus the "Download All Holdings" CSV - with SEC EDGAR N-PORT-P as a holdings fallback and Yahoo Finance daily market-price history/dividend fallbacks) into a searchable ETF/asset-class catalog with per-fund tabs, watchlist aggregation, ticker copy and CSV/TXT export - the same look, feel, columns and business logic as the sibling applications.

Using Bun

bunx degit daggerok/Sprott#main ./12345 && cd $_
bun install
bun run serve

bun run serve starts the Parcel dev server (it copies api/ to dist/api first) and prints the local URL. bun run build writes the production site to dist/, bun run build-github-pages does the same with the /Sprott/ public URL.

The app is live at https://daggerok.github.io/Sprott/ (GitHub Pages, deployed by the .github/workflows/github-pages.yml workflow from the main branch).

Column types and filters

Every column of the ETF catalog and of the Watchlist, Holdings, History and Distributions tabs has a type: text (ABC), number (123), percentage (%), money ($), date (D), date and time (DT) or time of day (T). The type is detected from the texts the column shows (80% of the filled cells must agree, otherwise text) and is written in the badge next to the column title: click it to cycle the type, Shift+click to return to auto-detection. Dates are read as 2024-06-15, 6/15/2024, 15.06.2024, Jun 15, 2024 or 15-Jun-2024, date and time as 2024-06-15T09:30:00Z or 2024-06-15 09:30, time as 09:30, 16:00:00 or 9:30 PM

A row of filter inputs sits under the column headers (the Filters button hides it, Clear filters empties it). Filters of different columns are combined with AND, the search box applies on top, and Copy Tickers and the exports use the filtered rows. Filters and type overrides are remembered in the browser. Sticky # (next to Filters, off by default, remembered in the browser) numbers the rows by their rank in the table sorted by the current column before the column filters, so a filtered fund keeps its rank and the numbers keep gaps; the sort, the search and the category and blacklist choices rank again. The catalog starts sorted by Net Assets, largest first, unavailable values sort last in both directions, and every export starts with the # column. The red Clear button opens a dialog that lists what can be reset (the selection, searches, sort order, open tab, shown columns, column filters, remembered table views and the blacklist), all ticked the first time and afterwards as they were left at the last OK; Enter confirms, Esc or a click outside cancels, the theme is always kept, so the page looks like a first visit (also after a reload)

Inside one filter: a space means AND, a comma means OR, a leading ! means NOT, ? matches an empty or unavailable value and !? a value that is there; a value that is unavailable matches only ? and negated conditions. An unquoted space ends the value, so quote values that contain one (>="2024-06-15 09:30")

Type Examples
Text bank contains, "two words", !bank, =exact, ^starts, ends$, /regex/, tech, health
Number, percentage, money >10, >=10 <50, =22 (matches what rounds to 22), !=22, 10..50, ..50, 10.., >1B and K M B T suffixes, an optional $ or %
Date, date and time >2024-06-01, 2024 (the whole year), 2024-06 (the whole month), 2024-01..2024-06, today, yesterday, -7d.. (the last 7 days), +2w, -3m, -1y
Time >09:30, 09:30..16:00, =12:00 (the whole minute)

The Columns menu next to Filters lists every column of the ETF table from the first to the last, all of them shown by default, with a search box and the All, Clear, Toggle and Reset buttons. Use and Ticker are listed but locked. Hiding a column only removes it from the table: the filters, the sorting, the exports and Copy Tickers still use it. The choice is remembered in the browser (localStorage, never the data) and the menu is shown on the ETF catalog only

The asset classes are one Asset classes multi-select next to the All ETFs pill instead of one tab per class: every class is selected by default (= all ETFs), Only or unchecking narrows the table, and the All ETFs pill is lit only while nothing narrows it (all or none of the classes checked); clicking the pill clears the selection. The choice is remembered in the browser (localStorage, never the data)

Updating the static Sprott data

Run the updater with Bun (the script is directly executable):

./scripts/update-data.ts

Run ./scripts/update-data.ts --help to print every configuration variable with its default and usage examples.

The Update Sprott ETF data GitHub Actions workflow runs on Sunday 00:00 UTC and on demand, installs with a frozen lockfile, runs the updater and commits only changed api/sprott files. No changes means no commit. A failed updater keeps previously published data in place. Checkout never persists credentials, and the push step uses the runner token only at runtime.

scripts/update-data.config.json holds the defaults for every control. It is loaded relative to the updater, not the current working directory. Only a missing file permits built-in fallbacks; malformed or unreadable configuration fails. All supplied filters use AND logic.

Precedence, lowest to highest: file defaults < advanced JSON < nonblank workflow inputs < protected Actions variable or process environment. A blank input inherits the file value, advanced can set a key to an empty string on purpose, and a nonblank input wins even when it is 0 or false. An explicitly set environment variable overrides everything, even when empty (SPROTT_<NAME> is accepted for every control, plus the legacy SPROTT_LIMIT and HISTORICAL_PAGE_SIZE). Invalid values fail with a clear message instead of falling back silently.

The workflow exposes 24 controls as optional string inputs plus advanced, a JSON object that reaches every control without its own input:

advanced: '{"SEC_UA":"ops contact","VERBOSE":"true","SKIP_SPROTT":"true"}'

Dependabot is monthly for Bun and GitHub Actions.

Data sources

Block Source
Catalog (all US Sprott ETFs) The US sitemap discovers the official fund pages, and the fund-navigation block rendered inside the first healthy pages supplies the tickers and names; sitemap slugs plus published identities are the fallback, and the previously published catalog is retained if even that fails.
Holdings per fund The official Download All Holdings CSV embedded in each fund page (e.g. URNM); the same page's holdings table is parsed when the CSV is absent.
Key facts, expenses, distributions The same fund pages: NAV/pricing block, key-facts cells (ISIN, CUSIP, listing exchange, benchmark, inception), the fee table and #DistributionsData.
Returns The official month-end/quarter-end average annual total-return tables on each fund page; missing metrics are derived from Yahoo adjusted closes at the same reporting date.
Daily history, dividends Yahoo Finance chart prices, adjusted closes and dividend events.
Fallback SEC EDGAR N-PORT-P holdings (Sprott-focused trusts, CIK 0001728683, exact series matching) + previously published data as the last resort.

The published snapshot contains 13 funds, 736 holdings rows and 14,111 daily-history rows (e.g. URNM 27 holdings / 1,715 history rows, SGDM 49 / 3,073, SGDJ 32 / 2,894). Unknown facts are unavailable, never invented as zero.

Metrics and caveats

Sprott publishes no 30-day SEC yield, so secYield renders a dash placeholder and secYieldKind records not published; the catalog column stays honest for every fund. The history series is market price, not official NAV: historySource states Yahoo daily market-price closes/adjusted closes, and the derived figures are not published standardized NAV returns. Returns that a fund page publishes officially are used as published; only missing metrics are derived. The holdings CSV is the primary source and the page's holdings table is the fallback, so a fund page that drops the download still updates.

Retention applies only when a source failed: if neither the official performance table nor Yahoo answered for a fund, the previous returns are kept as one unit together with their returnsBasis and performanceAsOf. An honest null (for example a 3-year figure the provider stopped publishing, or an unknown distribution frequency) stays null and is never filled from an older publication. Month-end dates are parsed as UTC, so a run gives the same output in any timezone. CUSIPs are stored without spaces. Every request has a 45 second timeout covering headers and body and is retried per MAX_RETRIES. A rerun with identical upstream data writes nothing (generatedAt and catalogReadAt move only with content), new catalog tickers are printed as NEW FUNDS: ... (also in the workflow summary), and the run stops taking new funds after 25 minutes but still writes the index.

Each fund carries a derived metrics object that powers the catalog columns shared with the sibling sites:

  • ytd / tr1y - official or coverage-checked derived YTD and 1-year returns -> YTD Return, TR 1Y
  • cagr3y / cagr5y / cagr10y - published or coverage-checked derived annualized 3Y/5Y/10Y figures -> CAGR 3Y/5Y/10Y
  • tr3y / tr5y / tr10y - cumulative 3Y/5Y/10Y figures (1 + CAGR)^n - 1 -> TR 3Y/5Y/10Y
  • siAnn - since-inception annualized when date/age/coverage support it (not young cumulative SI) -> SI Ann.
  • dividendYield - indicated rate: latest positive distribution × payments per year ÷ market price
  • dividendYieldBasis - code for the definition behind dividendYield, null exactly when dividendYield is null (table below)
  • secYield - a dash placeholder: Sprott publishes none
  • returnsBasis - mandatory non-empty text saying how the returns were computed: official sprottetfs.com NAV total returns with gaps derived from Yahoo adjusted closes, or Yahoo adjusted market-price returns (an estimate, not official NAV)
  • performanceAsOf - mandatory ISO YYYY-MM-DD date the returns are as of: the month-end performance table date on the fund page, or the last Yahoo close date when derived; not the NAV date; null only when truly unknown

Both fields are the last two keys of each metrics object.

dividendYieldBasis Meaning for Sprott
indicated Updater estimate: latest positive distribution x inferred payments per year / market price; the only code Sprott produces, because the fund pages publish no yield
null dividendYield is null (no distribution or unknown frequency)

The other codes of the shared standard (official-trailing-12m, official-distribution-rate, official-other, computed-trailing-12m) are not used by this feed.

Unavailable data is never published as zero, and no tickers are excluded.

Update controls

Defaults below are exactly the values in scripts/update-data.config.json; blank Actions inputs do not override them.

Environment variable Default Meaning
MAX_FETCHES 0 Batch evaluation size: positive resumes the scoped cursor in api/sprott/update-state.json; the cursor is scoped to the selected funds (it resumes at the next selected ticker and wraps); 0 is a full unfiltered pass and resets the cursor, a TICKERS run never deletes it.
REQUEST_SLEEP 1 Minimum seconds between outgoing request starts on each worker lane, including retries. Every worker paces itself, so the overall rate is about CONCURRENCY / REQUEST_SLEEP requests per second.
CONCURRENCY 2 Parallel fund workers (worker pool), each with its own request lane. With REQUEST_SLEEP=0 requests are not paced.
TICKERS all Space/comma/semicolon allowlist, e.g. URNM URNJ SETM; empty means all funds.
AUM : Total net assets range: USD amounts or K/M/B/T suffixes; nano/micro/small/mid/large presets; inclusive min:max.
TER : Net total expense ratio range in % (strict min:max).
DIVIDEND_YIELD : Indicated distribution-yield range in %, min:max; missing values do not pass an active range.
SEC_YIELD : 30-day SEC-yield range in %, min:max; Sprott publishes none, so an active range filters everything out.
HOLDINGS_PAGE_SIZE 250 Holdings rows per JSON page
HISTORY_PAGE_SIZE 1000 Daily history rows per JSON page
MAX_RETRIES 2 Retries after the initial request, integer >= 1, an empty value is an error (transient HTTP/network failures only)
HISTORY_RANGE max Yahoo daily history range: max or Ny (e.g. 5y); merges with previously published history
STORE_RAW_DOWNLOADS false Keep raw provider payload snapshots beside the feed (config/advanced only)
SEC_UA daggerok ETF feed daggerok@gmail.com SEC EDGAR contact User-Agent, redacted in logs; the Actions variable SEC_UA overrides it when nonblank. Do not put credentials here.
SKIP_YAHOO false Skip Yahoo history and dividends; retain published data
SKIP_SPROTT false Skip sprottetfs.com pages; retain published data (config/advanced only)
EDGAR_FALLBACK true SEC N-PORT-P holdings fallback for funds without a usable holdings sheet
VERBOSE false Provider/fallback/retry detail; the normal compact fund reporter always retains real zero/false and omits missing fields (config/advanced only)
USE_SYSTEM_CA auto TLS trust store: auto restarts the updater once with Bun's --use-system-ca when a request fails with an untrusted-certificate error; true always uses the system CA store; false never restarts. Not an individual workflow input: use advanced, the config file or the CLI environment.
PERFORMANCE_YTD : YTD performance percent min:max (3Y/5Y/10Y annualized)
PERFORMANCE_1Y : 1Y performance percent min:max
PERFORMANCE_3Y : 3Y annualized performance percent min:max
PERFORMANCE_5Y : 5Y annualized performance percent min:max
PERFORMANCE_10Y : 10Y annualized performance percent min:max
TOTAL_RETURN_YTD : YTD cumulative total return percent min:max
TOTAL_RETURN_1Y : 1Y cumulative total return percent min:max
TOTAL_RETURN_3Y : 3Y cumulative total return percent min:max
TOTAL_RETURN_5Y : 5Y cumulative total return percent min:max
TOTAL_RETURN_10Y : 10Y cumulative total return percent min:max

TICKERS combines with AUM, TER, yield and return filters using AND logic; it does not override them. Funds not selected for a successful update keep their prior published metadata and data files.

Examples

MAX_FETCHES=10 ./scripts/update-data.ts
TICKERS="URNM URNJ SETM" ./scripts/update-data.ts
AUM="100M:" TER=":0.7" ./scripts/update-data.ts
PERFORMANCE_1Y="15:" ./scripts/update-data.ts

TypeScript and verification

The browser app lives in src/ (index.html, main.tsx, index.css with Tailwind v4) and is built by Parcel into dist/ - no tsconfig.json needed. Bun runs TypeScript out of the box.

Verification before every publish:

bun install --frozen-lockfile
bun test
bun build --target=bun scripts/update-data.ts --outfile=/dev/null
git diff --check

Brands table

Brand Where to get the data
AAM aamlive.com | AAM
abrdn (Aberdeen) aberdeeninvestments.com | aberdeen
Amplify amplifyetfs.com | Amplify
ARK Invest ark-funds.com | ARK
Capital Group capitalgroup.com | Capital-Group
Fidelity fidelity.com | Fidelity
First Trust ftportfolios.com | First-Trust
Franklin Templeton franklintempleton.com | Franklin
Global X globalxetfs.com/explore | Global-X
Goldman Sachs am.gs.com | Goldman-Sachs
Invesco invesco.com | Invesco
iShares ishares.com | iShares
JPMorgan am.jpmorgan.com | JPMorgan
NEOS neosfunds.com | Neos
Northern Trust etfs.ntam.northerntrust.com | Northern-Trust
Pacer ETFs paceretfs.com | Pacer
Parametric eatonvance.com | Parametric
ProShares proshares.com | ProShares
Schwab schwabassetmanagement.com | Schwab
SP Funds sp-funds.com | SP-Funds
SPDR ssga.com | SPDR
Sprott ETFs sprottetfs.com | Sprott
Tema ETFs temaetfs.com | Tema
Themes ETFs themesetfs.com/etfs | Themes
VanEck vaneck.com | VanEck
Vanguard investor.vanguard.com | Vanguard
VictoryShares vcm.com VictoryShares ETFs | VictoryShares
WisdomTree wisdomtree.com | WisdomTree
Xtrackers etf.dws.com | Xtrackers

Sibling applications

Application Data provider Repository
AAM Official AAM catalog/detail HTML + full holdings XLS + SEC N-PORT holdings fallback + Yahoo market history/dividends AAM
abrdn (Aberdeen) Official Aberdeen gateway + SEC N-PORT holdings fallback + Yahoo history/dividends aberdeen
Amplify Amplify ETFs Firestore data feed + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends Amplify
ARK Invest ark-funds.com fund pages + overview/NAV-history/performance JSON + official daily holdings CSV + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance distributions/history fallback ARK
Capital Group Official Capital Group fund data + SEC N-PORT holdings fallback + Yahoo history fallback Capital-Group
Fidelity SEC EDGAR N-PORT-P + Yahoo Finance Fidelity
First Trust ftportfolios.com official ETF list + fund summary, holdings, distribution and price-history export pages + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history fallback First-Trust
Franklin Templeton franklintempleton.com ETF listings + product pages + SEC EDGAR N-PORT-P Franklin
Global X globalxetfs.com Next.js catalog and fund pages + dated full-holdings CSV Global-X
Goldman Sachs am.gs.com fund finder + detail pages + SEC EDGAR N-PORT-P Goldman-Sachs
Invesco invesco.com fund pages and sitemap + official Invesco fund API (monthly returns, NAV, AUM, yields, daily holdings, expense ratio) + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends Invesco
iShares iShares (BlackRock) product workbooks iShares
JPMorgan am.jpmorgan.com fund explorer + product-data JSON JPMorgan
NEOS neosfunds.com lineup table + official fund pages + daily holdings CSV Neos
Northern Trust etfs.ntam.northerntrust.com funds list + per-fund CSV/JSON downloads Northern-Trust
Pacer ETFs paceretfs.com product catalog and fund pages (Cloudflare WAF; r.jina.ai proxy fallback) + SEC EDGAR N-PORT-P (Pacer Funds Trust) + Yahoo Finance history/dividends Pacer
Parametric eatonvance.com ETF catalog and Parametric product pages + SEC EDGAR N-PORT-P holdings + Yahoo Finance history/dividends Parametric
ProShares proshares.com ETF finder + fund pages + official data host ProShares
Schwab schwabassetmanagement.com product pages + CSV exports Schwab
SP Funds sp-funds.com homepage catalog, fund pages and daily holdings CSV + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance history/dividends SP-Funds
SPDR SSGA / State Street public feeds SPDR
Sprott ETFs sprottetfs.com fund pages + SEC EDGAR N-PORT-P (Sprott Funds Trust) + Yahoo Finance history/dividends Sprott
Tema ETFs Tema official fund pages + dated daily holdings CSV; SEC EDGAR N-PORT-P holdings fallback only + Yahoo Finance price/history/dividend fallback Tema
Themes ETFs themesetfs.com catalog + daily holdings CSV + Yahoo Finance history/dividends + SEC N-PORT-P holdings fallback Themes
VanEck vaneck.com ETF finder + product pages VanEck
Vanguard Vanguard product pages + SEC EDGAR N-PORT-P Vanguard
VictoryShares VCM VictoryShares catalog and product JSON + SEC EDGAR N-PORT-P holdings fallback + Yahoo Finance adjusted-market-price history VictoryShares
WisdomTree WisdomTree product table + SEC EDGAR N-PORT-P + Yahoo Finance WisdomTree
Xtrackers Official DWS catalog/US sitemap + PDP/XLSX + SEC N-PORT-P holdings fallback + Yahoo Finance daily prices/history/dividends Xtrackers

License

MIT - same as all sibling ETF repositories.

Sprott® and the fund names/tickers referenced here are trademarks of Sprott Inc. This is an independent, unofficial tool; it is not affiliated with, endorsed by, or sponsored by Sprott Inc. All data is reproduced from Sprott's own public fund pages and downloads, public SEC EDGAR filings and Yahoo Finance for research purposes. All other trademarks, including index names, are the property of their respective owners.

About

Sprott ETF holdings-to-watchlist: static ./api/sprott feed generated by a Bun updater from sprottetfs.com fund pages, SEC EDGAR N-PORT-P (Sprott Funds Trust), Yahoo Finance history/dividends

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors

Languages