Skip to content

Tile parcels and centerlines overview zooms without attributes and log tippecanoe memory - #716

Merged
iandees merged 2 commits into
masterfrom
fabric-parcels-overview-and-tippecanoe-diagnostics
Oct 20, 2026
Merged

iandees merged 2 commits into
masterfrom
fabric-parcels-overview-and-tippecanoe-diagnostics

Conversation

@iandees

@iandees iandees commented Oct 19, 2026

Copy link
Copy Markdown
Member

The parcels fabric job died with std::bad_alloc inside tippecanoe, about two hours into tiling its 16 shards, on both attempts with tippecanoe 2.79.0: 2026-10-18 with 58GB and 2026-10-19 with 110GB on an r6a.4xlarge. Doubling the memory moved the crash by about 25 minutes, so it is not a limit problem. The job logs didn't say which shard or tile it was on, because tippecanoe's progress output is filtered out.

My best guess is the same thing #705 addressed for addresses: parcels are tiled from z8 to z14 with no feature limit and no tile size limit, so a low-zoom tile over a dense area holds millions of polygons in memory. That is a guess; I have not seen the failing tile.

Two commits:

Log tippecanoe memory and last progress every minute, add a label points option (task/lib/tippecanoe.js)

  • While a tiling run is going it now logs, once a minute: ok - tippecanoe parcels shard 3 overview: rss 41.2GB (peak 43.0GB), last progress: 45.2% 10/164/360. RSS is read from /proc/<pid>/status on Linux (it says rss unknown elsewhere, e.g. on a Mac). The next crash will show which shard and zoom it was on and how large it had grown.
  • tippecanoe's progress updates are separated by \r, so stderr is now split on that as well as newlines, and whitespace-only lines are dropped.
  • New labelPoints option (--convert-polygons-to-label-points) and a parseProcStatus helper with tests.

Tile parcels and centerlines overview zooms without attributes (task/fabric.js)

  • The low and high zoom cutoff is now one constant, DETAIL_ZOOM = 13, shared by addresses, parcels and centerlines. Below it, each layer is tiled under tippecanoe's normal 500K / 200k-feature caps with --drop-densest-as-needed and no attributes, since the low zooms only need to show that data exists. Parcel polygons become label points there. From z13 up, every feature is kept with all its attributes and no limits. Buildings are not on this list and are unchanged.

Checked on real data with the real Tippecanoe class (overview, detail, then join):

  • parcels: 2,987,914 features from four sources (Idaho statewide, Marshall County AL, Riverside County CA, City of Los Angeles). z8-12 were all points with no attributes; z13-14 were polygons with attributes on every sampled feature. 235 seconds end to end, 0.29GB output.
  • centerlines: 928,904 features from three sources (Maricopa AZ, NJ statewide, Riverside CA). z8-12 were attribute-free lines; z13-14 had attributes.

That is far smaller than the real run, so it shows the output is right, not that the memory problem is fixed. If the next parcels run still crashes, the new log lines should say where.

This means centerlines now take two passes where they took one; the centerlines tileset is small (2.6GB last run), so I expect little cost.

@iandees
iandees merged commit 0ba49e9 into master Oct 20, 2026
3 checks passed
@iandees
iandees deleted the fabric-parcels-overview-and-tippecanoe-diagnostics branch October 20, 2026 01:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant