Standalone zero-to-working-pmtiles example

[mikaelhg]

Here's the most basic HOWTO possible for getting from zero to a working protomaps web application.

Easy to adapt for use with any openmaptiles JSON style, just replace the layers with your layers.

https://gist.github.com/mikaelhg/edf65dfad43e240fb4c483587b5e7f93

[sntran]

Thanks for the step-by-step guide.

Considering that "Our real-world, planet-scale dataset can address over 350 million individual tiles in just 91.6 megabytes", it would be nice to have a download of such "planet.pmtiles" ready-made.

[bdon]

A sample dataset is now available here: https://protomaps.github.io/protomaps-themes/examples/maplibre-basemap.html

[eddy-geek]

Already broken, maplibre-basemap.html:57 Uncaught TypeError: protomaps_themes_base.layers is not a function at maplibre-basemap.html:57:51

I'm going to guess it's because of linking "latest": https://unpkg.com/protomaps-themes-base@latest/dist/index.js

edit: indeed replacing with 0.2.0 works: https://unpkg.com/[email protected]/dist/index.js in the html above

[bdon]

Thanks, should be updated now, and I wrote a CI job that will find these latest script includes in the repo so I can keep them up to date better in the future.

I think it's important that I provide some examples that work without a JavaScript build system so people can copy-paste from GitHub; this requires the version to be hardcoded in the HTML. Open to other suggestions on ways to accomplish this.

[mikaelhg]

I don't think pmtiles 2.1.0 can yet produce files in the v3 format to which the article you quoted refers. Certainly the finland.pmtiles file I produced is around 700 MB if memory serves.

[bdon]

@mikaelhg can you share the stats (# of addressed tiles, etc) printed out after your run? (I assume you have gzip enabled for the python 2.0 convert program)

[mikaelhg]

Yes, 2.1.0 with --gzip enabled. I don't have that output saved, but these are the 3.0.0 runs:

~/Downloads/pmtiles convert ./data/output.mbtiles ./finland2.pmtiles
2022/11/09 10:05:55 convert.go:262: Querying total tile count...
2022/11/09 10:05:55 convert.go:278: Pass 1: Assembling TileID set
 100% |██████████████████████████████████████████████████████████████████████████████████████████████████████████████| (564080/564080, 1495819 it/s)        
2022/11/09 10:05:56 convert.go:314: Pass 2: writing tiles
 100% |███████████████████████████████████████████████████████████████████████████████████████████████████████████████| (564080/564080, 101567 it/s)        
2022/11/09 10:06:01 convert.go:362: # of addressed tiles:  564080
2022/11/09 10:06:01 convert.go:363: # of tile entries (after RLE):  464740
2022/11/09 10:06:01 convert.go:364: # of tile contents:  447922
2022/11/09 10:06:02 convert.go:379: Root dir bytes:  538
2022/11/09 10:06:02 convert.go:380: Leaves dir bytes:  900771
2022/11/09 10:06:02 convert.go:381: Num leaf dirs:  114
2022/11/09 10:06:02 convert.go:382: Total dir bytes:  901309
2022/11/09 10:06:02 convert.go:383: Average leaf dir bytes:  7901
2022/11/09 10:06:02 convert.go:384: Average bytes per addressed tile: 1.60
2022/11/09 10:06:02 convert.go:357: Finished in  7.145213482s

pmtiles-convert --maxzoom 14 --overwrite ./data/output.mbtiles ./finland.pmtiles
Notice: check out the new PMTiles converter at https://github.com/protomaps/go-pmtiles
# of addressed tiles: 564080
# of tile entries (after RLE): 464740
# of tile contents: 447922
Root dir bytes: 530
Leaves dir bytes: 903950
Num leaf dirs: 114
Total dir bytes: 904480
Average leaf dir bytes: 7929.3859649122805
Average bytes per addressed tile: 1.6034605020564459

[bdon]

@mikaelhg thanks for the info!

V2 approximate size: 564080 * 17 bytes = 9.5 megabytes
V3 size: 901309 bytes = 901 kb

so just the directories should be about 10% the size of v2. Of course the bulk of the archive is the gzipped tile data. Does that match your expectations?

The difference between python and go-pmtiles is probably due to different gzip settings which shouldn't be relevant.

[mikaelhg]

Sure, I was providing data to the person who was asking for it earlier, in case he's still in the thread. Personally, I was more interested in the possibility of the tile data layout allowing for fewer HTTP requests due to zooming and scrolling.

[bdon]

@mikaelhg by default tiles are now ordered by hilbert ID; in theory we could batch requests on the client side when panning to fetch multiple tiles at once, but the JS client does not currently take advantage of this. It does make a difference on HTTP/1.1 and less for HTTP/2.

For v3 zooming in/out a level will incur another small directory fetch unless your tileset is quite small.

[bdon]

@mikaelhg you're right, the implementation of v3 is still WIP as I need to create both the writer tooling and the reader libraries. Will do another pass at updated docs/examples when I get a chance. (also moving this to a discussion!)