Skip to content

Drop out-of-range coordinates and oversized polygons without dropping large real parcels - #720

Merged
iandees merged 1 commit into
masterfrom
fabric-geometry-filter-out-of-range
Oct 27, 2026
Merged

iandees merged 1 commit into
masterfrom
fabric-geometry-filter-out-of-range

Conversation

@iandees

@iandees iandees commented Oct 27, 2026

Copy link
Copy Markdown
Member

Follow-up to #719. The Parcels run on 2026-10-26 finished in about 13 hours (job 83c36518: 3.9 hours of overview, 4.3 hours of detail with the slowest shard 4.3 hours and every shard under 2.7GB, then the join and upload), so the world-sized polygons were the cause of the 57 hour shard. Its logs also showed what the 5 degree feature extent rule from #719 dropped:

source dropped what they are
us/wa/statewide 2 the world-sized polygons with a parcel-sized hole (intended)
us/md/howard 94,555 every parcel in the file: the coordinates are Maryland State Plane feet (for example 1,282,935 and 619,538) written as if they were longitude and latitude
us/ak/north_slope_borough 4 real wilderness parcels, 7 to 13 degrees wide at 68N
no/21/statewide and no/countrywide 6 each (the same 6) real Svalbard parcels, 5 to 11 degrees wide at 76 to 79N

So the rule worked for Washington, caught Howard for the wrong reason, and wrongly dropped about 10 legitimate large polygons, because a degree of longitude is short at high latitude. The published parcels tileset from that run is missing them until the next run; the 3.3M-feature Washington file's real largest polygon is only about 1.1 degrees, but these others are much larger.

This replaces the feature-wide 5 degree extent rule with two separate checks in badGeometry:

  • out-of-range: any coordinate outside longitude -180..180 or latitude -90..90 (or NaN) drops the feature, for every geometry type. This is exactly the broken Howard data, and it is cheap and unambiguous.
  • oversized: a Polygon, or any one polygon of a MultiPolygon, wider or taller than 20 degrees. The world-sized polygons are 360 by 180. Measuring each polygon separately also keeps a MultiPolygon that straddles the antimeridian, whose parts are each small.

Points and lines are only checked for range.

Run over the real sources with the new function:

source features out-of-range oversized
us/wa/statewide 3,322,111 0 2
us/md/howard 94,556 94,556 0
us/ak/north_slope_borough 7,031 0 0
no/21/statewide 8,643 0 0

The log now has a separate line for each kind, naming the job and source.

The Howard County source is still broken upstream and will never produce parcels until its coordinates are fixed; that is a source configuration problem, not something fabric can repair, and it is not new (those features can never be tiled).

@iandees
iandees merged commit 4e08910 into master Oct 27, 2026
3 checks passed
@iandees
iandees deleted the fabric-geometry-filter-out-of-range branch October 27, 2026 23:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant