Performace of osml10n_get_name_without_brackets_from_tags

[frodrigo]

The current implementation of osml10n_get_name_without_brackets_from_tags is very slow.

Running it on all transport ways of Europe is too long, I did not let it finish, I took more than one day.

When disabling call to osml10n_geo_translit from osml10n_get_name_without_brackets_from_tags it took only around 90s.
https://github.com/giggls/mapnik-german-l10n/blob/master/plpgsql/get_localized_name_from_tags.sql#L361

osml10n_geo_translit itself call to the slow osml10n_get_country (making a spatial query)
https://github.com/giggls/mapnik-german-l10n/blob/master/plpgsql/geo_transliterate.sql#L63
https://github.com/giggls/mapnik-german-l10n/blob/master/plpgsql/get_country.sql

The country is used only when it is 'jp' or 'th'
https://github.com/giggls/mapnik-german-l10n/blob/master/plpgsql/geo_transliterate.sql#L25-L34

I think 'th' would be better detected from Thai alphabet rather than geometry location.

About 'jp' why required to be in Japan to detect if CJK are in name ?

My point of view is that the country detection can be removed.

[giggls]

Country detection can not be removed because of the simple reason, that Chinese characters must be transcripted in a different way in China, Japan and HongKong/Makau.

Checking for Thai characters instead of checking for Thailand would be possible, but this is not the reason transcription is slow.

See issue #53 which discusses details about these issues.

[frodrigo]

I already read #53, I'm on OpenMapTiles and Imposm, not interested in lua implementation.

I ask about the use of country detection for this particular function, not removing it globally.

[giggls]

Well maybe it would be a little bit faster to check for CJK and Thai characters avoiding calling get-country function in some cases. But as I already said I do not really consider this place the main speed problem of this whole approach.

BTW, it would be very easy to add this because contains_cjk function is already there and contains_Thai would be easy to implement.

Maybe you should give my external daemon approach a try using https://github.com/giggls/osml10n/blob/master/cc_transcript_via_daemon.sql

BTW With the advent of the new osm2pgsql flex backend osm2pgsql is now even more flexible than imposm.

[giggls]

I just added a check for CJK and Thai before calling osml10n_get_country.
This might speed up the code a little bit in countries like Russia, Greece, Israel and the Arab world.

[frodrigo]

Thank you.