Fresh install data is different from nominatim.openstreetmap.org

[vricciardulli]

I just performed a fresh install of Nominatim. I used the full-planet PBF file from end of April for the import. I have updates enabled using default settings (minutely diffs). I am using a flatnode file, both for the initial import and for the updates.

I notice some discrepancies between data in my database and data on https://nominatim.openstreetmap.org/

A non-exhaustive list of examples:

• missing Withcott (Australia): details?osmtype=R&osmid=11675687 would return a 404 in my local server, while it is available on the website. I then re-imported it using nominatim add-data

• different data for Withcott (Australia): details?osmtype=R&osmid=11675687&addressdetails=1 would return an array at field address that is different from the website. In particular, in the array there was another object instead of Lockyer Valley Regional (R11675710). I fixed it refreshing and re-importing this latter.

• missing Ceará (Brazil): the array at field address from details?osmtype=R&osmid=302486&addressdetails=1 did not contain Ceará (R302635) in my local server (same would happen for other children of Ceará), while it is there on the website. I fixed this with nominatim refresh

• wrong rank_address for Beamish (England): details?addressdetails=1&osmid=1890570835&osmtype=N has rank_address = 18, while it is 16 on the website. I don't know how to fix this

• different data for Reston (Virginia): details/osmid=2240795956&osmtype=N&addressdetails=1 returns an array at field address that is different from the website. In particular, instead of having Reston (N2240795956), in my local server there is:

  {
      "localname": "Reston",
      "osm_id": 206748,
      "osm_type": "R",
      "place_type": "city",
      "class": "place",
      "type": "locality",
      "admin_level": 15,
      "rank_address": 16,
      "distance": 0,
      "isaddress": true
  }

  while on the website it is:

  {
      "localname": "Reston",
      "osm_id": 2240795956,
      "osm_type": "N",
      "place_type": null,
      "class": "place",
      "type": "city",
      "admin_level": 15,
      "rank_address": 16,
      "distance": 0,
      "isaddress": true
  }

  I can't fix this.

So my questions are:

• is it a known issue that there may be differences and what is the reason?
• what is the ground truth, the Nominatim website?
• is there a way to know which data is different?

Thank you for the help 🙏

[mtmail]

nominatim.openstreetmap.org server (actually it's 4 separate servers) runs newer software (github master instead of the latest stable release). The data is updated minutely while often people only do daily updates. At the same time data of the servers were imported with an older Nominatim version (months ago) and not all regions in the world are reindexed with the newest Nominatim version.

It's close to impossible to get the same results as nominatim.openstreetmap.org with your own installation. That's not negative though. nominatim.openstreetmap.org is used as test-bed for new features, some which are rolled back and later don't make it into the stable release.

It's a common question though, we should create a FAQ entry.

Ground truth is probably the OSM data itself and whoever did a full import with the latest version of Nominatim. Your installation is likely closer to the truth.

Reston: The relation (on your local installation) is the better result. Nominatim tries to merge a node and relation that represent the same place (in the real world) into one database entry. Usually based on the wikidata tag. I see somebody changed the node last month (https://www.openstreetmap.org/node/2240795956) and nominatim.openstreetmap.org has't automatically reindexed the area yet.

Beamish: Anything rank 16 to 19 is possible https://github.com/osm-search/Nominatim/blob/master/settings/address-levels.json On my servers it's 19.

[vricciardulli]

Thanks for the answer, very informative!

I have another case which is a bit more important, it would be great if you could please help me with it or give me some pointers:

• Republic of Crimea (R3795586) and Autonomous Republic of Crimea (R72639) are in Russia (R60189) on my local, while they are in Ukraine (R60199) on the website.

• How can I check against OSM data? what I'm doing now is use links such as this https://www.openstreetmap.org/api/0.6/relation/60199/full to download the XML and look whether certain children places are found in it or not. What I noticed is that:

  • Republic of Crimea is neither in the XML for Russia nor in that for Ukraine
  • Autonomous Republic of Crimea is in the XML for Ukraine, but not in that for Russia

• Is there a way I can correct this?

[mtmail]

Besides the real world war there's also an edit war (https://simple.wikipedia.org/wiki/Wikipedia:Edit_war) where users keep changing the places in OSM data back and forth. If I recall for several years https://www.openstreetmap.org/relation/72639 was a state of Ukraine while https://www.openstreetmap.org/relation/3795586, which covers the same area, a state of Russia. There's islands in the Pacific which are claimed by 3 countries even.
Nominatim doesn't have the concept of 'political view' as URL parameter that you might have seen in commercial geocoding solutions. You'd have to change the imported data to reflect your political view. As an example an import for Argentina might want the Falkland islands to be displayed as part of Argentina.
Your data import was probably before this change https://www.openstreetmap.org/changeset/135602701 and Nominatim hasn't automatically assigned the place to another country. That's still a known issue, tracked in #202

What you can try is change the data in your database:

UPDATE placex SET country_code='ua' WHERE osm_type='R' and osm_id=72639;
DELETE FROM placex WHERE osm_type='R' and osm_id=3795586;

followed by running nominatim refresh --data-area R72639 and nominatim index.

[vricciardulli]

Great, this works! thanks again 🙏