[SEDONA-646] Shapefile DataSource for DataFrame API #1553
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Kontinuation
force-pushed
the
shapefile-dsv2
branch
from
0cde90f
to
0ef1408
Compare
…ing null geometries
…ge private methods
Kontinuation
force-pushed
the
shapefile-dsv2
branch
from
3d26325
to
83a08c3
Compare
Kontinuation
force-pushed
the
shapefile-dsv2
branch
from
83a08c3
to
90ac25d
Compare
Kontinuation
changed the title
Kontinuation
force-pushed
the
shapefile-dsv2
branch
from
2a39e34
to
b77d577
Compare
jiayuasu
approved these changes
Aug 23, 2024
Kontinuation
added a commit
to Kontinuation/sedona
that referenced
this pull request
Oct 11, 2024
* Shapefile DataSource for Spark SQL API * Fix compilation for scala 2.13 * Add tests for the new shapefile reader * Support reading shape keys * Add new test files * Add tests for reading shapefiles with bad shx, and .shp files containing null geometries * Move shapefile data source to non-spark package name, don't use package private methods * Add python tests * Backport to Spark 3.0 * Backport to Spark 3.1 * Backport to Spark 3.2 * Backport to Spark 3.3 * Backport to Spark 3.4 * Fix profile activation for Spark 3.2, spotless apply * Make the shapefile reader work without GeoTools * Update documentation * Change charset handling and add documentation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Did you read the Contributor Guide?
Is this PR related to a JIRA ticket?
[SEDONA-XXX] my subject.What changes were proposed in this PR?
This PR implements a data source for reading shapefiles. Now we don't have to read shapefiles using the RDD API and convert the SpatialRDD to a DataFrame using the adapter. Instead we can simply say:
to load the shapefile as a DataFrame.
How was this patch tested?
Added new tests.
Did this PR include necessary documentation updates?