Repository navigation
Drop world-sized polygons from fabric sources so they cannot stall tippecanoe - #719
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The parcels fabric job that started 2026-10-19 (job 151a65fd, with the overview/detail split from #716) did not hit the
std::bad_allocany more, but it failed on its 3 day timeout. Tiling took 64 hours: the overview pass took 6.3 hours, and in the detail pass 15 of 16 shards finished in 2.5 to 4.4 hours while shard 5 took 57.4 hours (07:13 on 10-20 to 16:38 on 10-22). It crept from 86.7% to 100% with only 0.8GB of memory in use. The join then ran out of the last 5 hours and was killed.Shard 5 contains
us/wa/statewide. That file has two parcels (pid"043-", near Spokane) whose outer ring is the whole-world rectangle (-180,-90 to 180,90) with the real parcel as a hole in it, which looks like a bad conversion upstream. At z13 and z14 tippecanoe has to emit a filled tile for every tile on Earth for each of them. I reproduced this locally with the same detail options on that one file:Everything else in that file is ordinary: the next largest polygon bounding box is about 1.2 square degrees, and the mean is 22 vertices per feature.
This adds
task/lib/extent.jsand drops any Polygon or MultiPolygon whose bounding box is wider or taller than 5 degrees while a source is being fetched inget_source, logging anot ok - dropped N polygon(s) ...line with the job and source so we can see which sources have these. Points and lines are never dropped (a long centerline is legitimate and only costs the tiles along its path). A polygon crossing the antimeridian has a longitude span near 360 and is dropped too.Run over the whole us/wa/statewide file, the check drops exactly those two features out of 3,322,111 and takes 12 seconds including parsing. Unit tests cover the world-with-hole shape, ordinary and large parcels, MultiPolygons, and non-polygons.
Not checked: the other 1,374 parcels sources and the other layers. The 15 other parcels shards finished normally, so none of them has one of these, and addresses, buildings and centerlines completed their last runs. A source that gets new broken geometry later would now log the drop instead of stalling the job.
The two bad parcels are probably worth reporting to whoever maintains the Washington source.