Tiles disappearing at particular zoom levels

[sabman]

I'm seeing the following with the latest release of pg_tileserv Version 1.0.10

dissapearing-tiles.mp4

select * from postgis_full_version()

POSTGIS="3.3.2 4975da8" [EXTENSION] PGSQL="150" GEOS="3.8.0-CAPI-1.13.1 " PROJ="6.3.1" GDAL="GDAL 3.0.4, released 2020/01/28" LIBXML="2.9.10" LIBJSON="0.13.1" LIBPROTOBUF="1.3.3" WAGYU="0.5.0 (Internal)" RASTER

related: #160

[eldang]

The pattern of gaps as you zoom and pan look like problems I've had with geometries that become invalid when clipped at tile boundaries. It's not necessarily a pg_tileserv issue--I've also run into this when making static tiles with tippecanoe--but if it is the same as I've seen before then the key factor is the cut lines that any tiling process has to make. I don't know if any of these will necessarily fix the problem, but here are things I've done that have helped with issues that look similar:

• First, I'd inspect the geometries as stored with ST_IsValid() because one possibility is that they themselves have the kind of minor technical invalidities that vector tile output can cope with until a crop interacts with it. If it does tell you that any are invalid, ST_IsValidReason() and ST_MakeValid() can help with identifying and/or fixing the issue.
• If this is a function layer, it may be worth wrapping geometries with ST_MakeValid(). If it's a table layer, it might be worth trying to make a function layer that does this. The reason for that is that doing it at the function stage can apply ST_MakeValid() to the post-crop geometries.
• If the geometries are stored as Polygon types, it may help to turn them into MultiPolygons. The first gap to appear looks like it would be switching from single polygon to MultiPolygon, which I've sometimes had cause problems.
• Putting the geometries through ST_Buffer() with a buffer radius of zero is sometimes surprisingly effective at fixing geometries. If that doesn't help. then the same with a tiny positive buffer radius sometimes does, though depending on the application this option may not be worth it because of the changes it makes to the data.
• Sometimes putting complicated geometries through ST_Subdivide() solves a lot of problems, as well as making spatial indexes more effective. The catch here is that you need to be able to merge them back together in the display layer.
• Simplifying the polygons may help, though depending on the application that may again not be worth it because of the loss of fidelity.

[pramsey]

I wonder how much of this advice we should wrap into the system. We cannot easily do valid/makevalid without bogging down in CPU overhead. We could automatically fluff polygons up into multis without too much cost.

[pramsey]

For this particular case, yeah, the last time I saw this was when MVT output was still pretty new and we generated invalid outputs during tile clipping. The reason we use wagyu for clipping (still) is because it more reliably produces valid output. However, if you feed it invalid input then bets are off.

[sabman]

I'm not sure if I have this wrong last time I tried loading multiploygons tileserver didn't see them but that may have been unrelated... so I used the following to load the data. Which as @eldang suggests might have made the geometries invalid.

ogr2ogr -nlt POLYGON -f "PostgreSQL" PG:"host=mygost user=postgres password=pass dbname=postgres port=6443" lga_2018_aust.gpkg

I'm gonna try some of the suggestions

[sabman]

Yea loading as multipolygons seems to have resolved this issue. basically loading using same command without the -nlt POLYGON does the trick!

[pramsey]

That's very curious... what does that load do that causes render to fail sometimes?

[eldang]

I think what's going on in cases that are resolved by storing the source geometries as Multi* is that the the crop would turn a single polygon into two, and isn't done in a way that allows breaking them into two rows. I could see a universal cast-to-Multi being worth it for these, if the cost really is low enough. Would it be difficult to make that behaviour conditional on what the actual source geometry type is? Or is a redundant cast to ST_Multi() so low cost that I'm now overthinking things and it's worth just applying it to everything?

[pramsey]

If the problem is "clip turns poly into multipoly" surely we would have seen this in a lot more cases. It also means that there might actually be a bug hiding somewhere wanting to be fixed. If we could have the offending polygon and reproduce the behaviour, perhaps we'd find something (not in tileserv, but in PostGIS). Does the behaviour reproduce in the tileserv web UI, given the right input data? Step one is deriving a reproduceable case.

[eldang]

I think it's specifically that a clip can make a multipolygon out of a polygon when the shape has two arms that are connected on one side of the cut line and disconnected on the other. I've mostly run into this with blatantly gerrymandered electoral districts.... I can probably cook up a test case next week with Texas data, but it might be more useful to start with the specific file that opened this issue. @sabman is it something publicly available, and if so could you share a link?

[sabman]

Yea I can create SQL dumps for both cases. One with polygon and the other with multipolygon. Would that be enough to start with?

[pramsey]

If you can dump the one offending polygon, in the form (poly or multi) that is causing the problem, that would cut down the amount of hunting we have to do.

[sabman]

The zip file contains the two versions of the polygon seen in video above
polygon-in-video.zip

created via

ogr2ogr -nln au_lga_01_2018_multipoly -f "PGDump" au_lga_01_2018_multipoly.sql lga_2018_aust.gpkg
ogr2ogr -nlt POLYGON  -nln au_lga_01_2018_poly -f "PGDump" au_lga_01_2018_poly.sql lga_2018_aust.gpkg
➜ ogr2ogr --version
GDAL 3.7.2, released 2023/09/05

Original as geojson https://gist.github.com/sabman/d008a15d83d88eb1bed3a5853bbcd61f