Skip to content

An Android friendly library for importing compressed Open Street Map data into Sqlite for POI queries and much more

Notifications You must be signed in to change notification settings

spyhunter99/osmreader

Repository files navigation

Osm Reader

An Android friedly tool + library to read on compressed Open Street Map data (bz2) extracts, convert it into a Sqlite database for easy querying.

Author

Alex O'Ree, committer for osmdroid, Apache Software Foundation, and many other projects

License

Apache Software License v2.0

Development status

Still working on this in my spare time. Not really API or functionally stable right now.

Usage

Checkout this project

git clone https://github.com/spyhunter99/osmreader.git

Build and install to maven local using Gradle Fury

./gradlew clean install

Include this library in your project

Edit your $rootDir/build.gradle, add under allProjects, repositories, add mavenLocal() Edit your app/module's build.gradle file and add

//Android only
compile ('org.osmdroid.reader:osm-reader-lib:1.0.0-SNAPSHOT'){
    exclude group: 'org.xerial'
}
compile 'org.sqldroid:sqldroid:1.0.3'

//Java JRE/JDK only
compile 'org.osmdroid.reader:osm-reader-lib:1.0.0-SNAPSHOT'

Sample code - Importing the data

//needed on android only
DriverManager.registerDriver((Driver) (Class.forName(
        "org.sqldroid.SQLDroidDriver" , true,
        getClass().getClassLoader()).newInstance()));
Connection con = DriverManager.getConnection("jdbc:sqldroid:/sdcard/osmdata.sqlite");

Reader reader = new Reader();
//read input file to output database
reader.read("/sdcard/delaware-latest.osm.bz2", con);

Once this process is complete, you'll have yourself a database that's easy to query. It can take a very long time on a smart phone or tablet.

How to query for POIs, Street names, City/Town/State names, etc

Keyword search (everywhere)

List<SearchResults> searchResults = QueryTools.search("new castle", 10, 0, connection);
for (int i=0; i < searchResults.size(); i++) {
     SearchResults record = searchResults.get(i);
     System.out.println(record.getName() + " " + searchResults.get(i).getType() + " " + searchResults.get(i).getLat() + "," + searchResults.get(i).getLon() + "," + searchResults.get(i).getDatabaseId());
 }

Keyword search within a bounds

List<SearchResults> searchResults = QueryTools.search("new castle", 10, 0, connection, 40,-75,39,-76);
for (int i=0; i < searchResults.size(); i++) {
     SearchResults record = searchResults.get(i);
     System.out.println(record.getName() + " " + searchResults.get(i).getType() + " " + searchResults.get(i).getLat() + "," + searchResults.get(i).getLon() + "," + searchResults.get(i).getDatabaseId());
 }

Using the databaseId from above, get all "tags" for a given Node, Way, or Relation

Map<String, String> tags = QueryTools.getTags(366753278L, connection, 1000, 0);

Using the databaseId from above, Street address, phone number and website extraction. Note: this is a best effort extraction using commonly used keywords, since the osm data model is open ended.

Address addr = QueryTools.getAddress(366753278L, connection);

Where do i get the data from?

Try this source: https://download.geofabrik.de/ You want the .osm.bz2 files.

How much space do I need?

The smalled osm extract is for the state of Delaware, USA. Compressed, it's only 11MB and 7,564,957 xml elements which translates to about 2,483,857 database rows (with ways and relations). Uncompressed XML is around 150MB. In database format, it takes about 100 MB or about 10x whatever the download size is. The whole planet is around 50GB right now, which means you'd need 500GB of disk storage for the planet.

Can we reduce the storage need?

What data is needed is up to the use case. If you're just using this tool for reverse geocoding, you probably don't care who edited what and when. This is a future optimization we could do (pull requests welcome!)

How long does it take to import

Processing time on :

Implementation CPU Data set Batch Size Time wo/Ways & Relations Time w/Ways & Relations
XML pull parser AMD 8 core with SSD Delaware.bz2 500 n/a 102 sec
XML pull parser Galaxy S5 Delaware.bz2 400 1 hr+ 1 hr+
XML pull parser AMD 8 core with SSD Planet 500 about 66 days (est)
Osmosis AMD 8 core with SSD Delaware.bz2 500 n/a 83 sec
Osmosis Galaxy S5 Delaware.bz2 400 n/a n/a

Notes Osmosis doesn't appear to work in an android environment. No idea why

Credits

Thanks to https://github.com/gradle-fury/gradle-fury/ for making Gradle easier to work with.

About

An Android friendly library for importing compressed Open Street Map data into Sqlite for POI queries and much more

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages