Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] explore dropping arrow dependency #2347

Open
revans2 opened this issue Aug 22, 2024 · 0 comments
Open

[FEA] explore dropping arrow dependency #2347

revans2 opened this issue Aug 22, 2024 · 0 comments

Comments

@revans2
Copy link
Collaborator

revans2 commented Aug 22, 2024

Is your feature request related to a problem? Please describe.
CUDF is in the middle of trying to drop arrow as a dependency. At least as a public dependency. A lot of this is to maintain compatibility in a python ecosystem where arrow could be installed on a system separate from CUDF.

We don't really have as much of the same issue because of how we link things, but also because of java vs python. Still it would be good for us to stop using arrow directly in spark-rapids-jni. We currently only use it for parsing parquet footers, but that is code we could lift from arrow an maintain ourselves, as it really is just the thrift generated classes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants