Error when computing word frequency for search terms

[isurusiri]

Hey there, thank you for your contributions to the project!

I've been facing an issue when installing Nominatim 3.3.0 and I'd like your opinion on this matter and any heads up on things I could have done differently. Please note that I have to strictly stay on 3.3.0 for the moment.

The error I'm getting:

"stderr": "NOTICE:  table \"word_frequencies\" does not exist, skipping\nERROR:  relation \"search_name\" does not exist\nLINE 2:  SELECT unnest(name_vector) as id, count(*) FROM search_name...\n                                                         ^\nERROR: pgsql returned with error code (3)", 
"stderr_lines": ["NOTICE:  table \"word_frequencies\" does not exist, skipping", 
"ERROR:  relation \"search_name\" does not exist", 
"LINE 2:  SELECT unnest(name_vector) as id, count(*) FROM search_name...", "                                                         ^", 
"ERROR: pgsql returned with error code (3)"], "stdout": "2021-09-09 16:19:21 == Recompute frequency of full-word search terms\nDROP TABLE\nstring(34) \"pgsql returned with error code (3)\"", 

With this I assumed that this is coming from this statement>> in the code, when it's invoked from here.

I noticed that the search_name table gets dropped in the code if the --reverse-only flag is set to true. The --reverse-only flag has default value as false in here, but I tried setting it up manually also. In that case the Nominatim database was missing.

URLs I'm using to download required resources:

• https://ftp5.gwdg.de/pub/misc/openstreetmap/planet.openstreetmap.org/pbf
• https://www.nominatim.org/release/Nominatim-3.3.0.tar.bz2
• https://www.nominatim.org/data/wikipedia_article.sql.bin
• https://www.nominatim.org/data/wikipedia_redirect.sql.bin
• https://www.nominatim.org/data/gb_postcode_data.sql.gz

Initial import of the OSM data:

$ ./utils/setup.php --osm-file path-to-planet-latest.osm.pbf --all --reverse-only false 

I'm performing this using an ansible playbook which worked fine in previous instances. I'm a little bit confused where to look at in order to pass this hurdle. Maybe it's something small piece of the puzzle I'm missing here. I appreciate any help with this since I'm completely new to the Nominatim and OSM communities.

[mtmail]

--reverse-only false has probably not the intended effect: PHP would interpret that as string, convert to boolean and end up with a true. ./utils/setup.php --osm-file path-to-planet-latest.osm.pbf --all should be enough.

You should upgrade to version 3.3.1. That is a security release https://github.com/osm-search/Nominatim/releases/tag/v3.3.1 with no other changes, so it's compatible to 3.3.0.

[mtmail]

Please post the output of setup.php. There must be an error message somewhere.

[isurusiri]

My apologies. I reran this with logging the output of setup.php and it turned out the disk space is not enough.

Parsing gazetteer style file '/srv/nominatim/Nominatim-3.3.0/settings/import-full.style'.
Using projection SRS 4326 (Latlong)
NOTICE:  table "place" does not exist, skipping
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=42126MB, maxblocks=674016*65536, allocation method=11
Mid: loading persistent node cache from /data/nominatim/flatnode.file
Mid: pgsql, cache=42126
Setting up table: planet_osm_nodes
NOTICE:  table "planet_osm_nodes" does not exist, skipping
Setting up table: planet_osm_ways
NOTICE:  table "planet_osm_ways" does not exist, skipping
Setting up table: planet_osm_rels
NOTICE:  table "planet_osm_rels" does not exist, skipping

Reading in file: /data/nominatim/planet-latest.osm.pbf
Using PBF parser.
Processing: Node(6075930k 5694.4k/s) Way(0k 0.00k/s) Relation(0 0.00/s)Bus error (core dumped)
PHP Fatal error:  Uncaught Nominatim\DatabaseError: [500]: Database query failed

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev             31G     0   31G   0% /dev
tmpfs           6.2G  884K  6.2G   1% /run
/dev/nvme0n1p1  7.7G  2.6G  5.1G  34% /
tmpfs            31G   16K   31G   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            31G     0   31G   0% /sys/fs/cgroup
/dev/loop0       56M   56M     0 100% /snap/core18/2128
/dev/loop1       33M   33M     0 100% /snap/snapd/12883
/dev/nvme1n1    125G  119G     0 100% /data
/dev/nvme2n1   1007G  1.3G  955G   1% /var/lib/postgresql
tmpfs           6.2G     0  6.2G   0% /run/user/1000

I sincerely apologise for wasting your time without doing my homework.

[mtmail]

This is the main issue

"Processing: Node(6075930k 5694.4k/s) Way(0k 0.00k/s) Relation(0 0.00/s)Bus error (core dumped)"

While importing data from the *.pbf file into the database (using https://osm2pgsql.org/) the system basically crashed. That's usually when there wasn't enough memory/RAM, but can also happen when the database itself crashes (see postgresql logfile for hints) or other instability, e.g. RAM hardware issue.

For a planet import you should set the --osm2pgsql-cache as well to about 2/3 of the RAM your server has. https://www.nominatim.org/release-docs/3.3.0/admin/Import-and-Update/#initial-import-of-the-data Try to monitor the RAM usage and if the operating system start swapping memory to disk. Use a flatnode file if possible https://www.nominatim.org/release-docs/3.3.0/admin/Import-and-Update/#flatnode-files

6,075,930k, so about 6 billion nodes. It should count until about 7.2 billion, then 800 million ways, 9 million relations.

[mtmail]

"and it turned out the disk space is not enough." Ah sorry, I didn't read that part of your answer. Yes, that can also be a reason. Good luck with the rest of the installation.

[isurusiri]

Thank you very much for the detailed explanation. It is really worth documenting in our repository as well. Keep up the good work.