Strange size of tiles

[alff0x1f]

Twms version: 4692d81
New installed twms without any cache. Let's get tile:

$ wget "http://localhost:8080/osm/13/5365/2614.png"
...Saving to: ‘2614.png’
2015-12-03 16:07:31 (191 MB/s) - ‘2614.png’ saved [2934]

Size: 2934 bits. But if we look at our cache, we notice another size of tile:

$ ls -l /var/cache/twms/tiles/osm/z14/5/x5365/2/
-rw-r--r-- 1 twms twms 1872 Dec  3 16:07 y2614.png
-rw-r--r-- 1 twms twms 2630 Dec  3 16:07 y2615.png

Size of tile 1872 bits. And on next gets (with cache and after restart) it return 2934 bite. (work wrong always)

At my local machine I fix issue #11 and after restart I get this (with empty cache dir):

$ wget "http://localhost:8080/osm/13/5365/2614.png"
...
2015-12-03 16:10:27 (315 MB/s) - ‘2614.png’ saved [2934]

$ wget "http://localhost:8080/osm/13/5365/2614.png"
...
2015-12-03 16:10:30 (131 MB/s) - ‘2614.png.1’ saved [1872]

First tile bigger, but after that it return smaller tile from cache (work wrong only first time when cache empty). Original OSM tile size 1872:

wget 'http://c.tile.openstreetmap.org/13/5365/2614.png'
--2015-12-03 16:48:12--  http://c.tile.openstreetmap.org/13/5365/2614.png
Resolving c.tile.openstreetmap.org (c.tile.openstreetmap.org)... 130.193.62.73
Connecting to c.tile.openstreetmap.org (c.tile.openstreetmap.org)|130.193.62.73|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1872 (1,8K) [image/png]
Saving to: ‘2614.png.5’
2015-12-03 16:48:12 (365 KB/s) - ‘2614.png.5’ saved [1872/1872]

Both Images look same, with my eye I did't see difference.

Ah, yes, and twms save 4 tiles cache, even I get only one (I don't know why):

# find /var/cache/twms/ -name *png
/var/cache/twms/tiles/osm/z14/5/x5365/2/y2615.png
/var/cache/twms/tiles/osm/z14/5/x5365/2/y2614.png
/var/cache/twms/tiles/osm/z14/5/x5366/2/y2615.png
/var/cache/twms/tiles/osm/z14/5/x5366/2/y2614.png

I don't know bug this or feature. In code there some "magic" which i not completely understand. So I think I don't fix this, only leave this issue for other people. May be it will be useful for somebody.

[Komzpa]

Most likely, it is tile serving logic that relies on WMS backend:

• if tile is not in cache, it should be downloaded, transformed, constructed from whatever is there in the backend;
• if tile is in cache, it should be served right from there.

Most likely image from openstreetmap.org is indexed png, and the one you get on the first request is not.

Could be fixed by checking tile-fetching flow and calling fetcher right after looking in cache if there's no reason to fall back on WMS flow, and leaving things as they are for other cases.

[alff0x1f]

I try call fetch earlier in twms_main() in same place where tile checked in cache and read, but fetch function return TIL image and you need covert it to string for web.py answer. So anyway after this convertation image size changed.

[Komzpa]

Well, probably the best solution is to separate tile downloading logic from fetcher. Probably some flag, that indicates that there's no need to decode image, and it just should be fetched into cache. Then you can check for a tile in cache, call fetcher with that flag, check again, and fall back to calling fetcher without that flag.

[alff0x1f]

@Komzpa Ok, I think I can do it. Actually I already do same logic in function get_tile_from_db https://github.com/alff0x1f/twms/commit/ef0ae7ea460e68c61b156112fea16953aeffd287

It is offtopic here, but can you explain some additional extensions? For tile 1234.png, code check file like 1234.ups.png and 1234.tne. I did't realize that moment.

[Komzpa]

1234.tne - Tile Not Exists. For cases when dataset is sparse, and you don't want to hammer the upstream server to get the tile that just isn't there.

1234.ups.png - scaled version of tile. some datasets may contain only higher zooms, and lower ones are generated by stitching four tiles together and cropping them.

[Komzpa]

twms disk cache format is directly compatible with directories from SAS.Planet, http://sasgis.ru/sasplaneta/