Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GLib] Meson selects incorrect Arrow C++ libraries in some situations #44465

Closed
hiroyuki-sato opened this issue Oct 18, 2024 · 1 comment · Fixed by #44475
Closed

[GLib] Meson selects incorrect Arrow C++ libraries in some situations #44465

hiroyuki-sato opened this issue Oct 18, 2024 · 1 comment · Fixed by #44475
Assignees
@hiroyuki-sato
Labels
Component: GLib Type: bug
Milestone
19.0.0

Comments

@hiroyuki-sato
Copy link
Contributor

Describe the bug, including details regarding any error messages, version, and platform.

Overview

Meson selects incorrect arrow libraries in some situations.
If a developer installs arrow 17.0.0 with arrow-flight-sql in the /opt/homebrew/lib
and 18.0.0-SNAPSHOT without arrow-flight-sql in /tmp/local,
meson selects an incorrect package.

Run-time dependency arrow found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-compute found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-csv found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-filesystem found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-json found: YES 18.0.0-SNAPSHOT
Found CMake: /opt/homebrew/bin/cmake (3.30.5)
WARNING: CMake Toolchain: Failed to determine CMake compilers state
Run-time dependency arrow-orc found: NO (tried pkgconfig, framework and cmake)
Run-time dependency arrow-cuda found: NO (tried pkgconfig, framework and cmake)
Run-time dependency arrow-acero found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-dataset found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-flight found: YES 17.0.0
Run-time dependency arrow-flight-sql found: YES 17.0.0 <-------- HERE
Run-time dependency gandiva found: YES 17.0.0
Run-time dependency parquet found: YES 18.0.0-SNAPSHOT

Reproduce steps.

  • Install released arrow package using Homebrew. brew install apache-arrow. (Arrow 17.0.0)
  • Build & Install cpp package ninja-debug doesn't install arrow-flight-sql (Arrow 18.0.0-SNAPSHOT)
    • cmake .. --preset ninja-debug -DMAKE_INSTALL_PREFIX=/tmp/local
    • cmake .. --build .
    • cmake .. --install .
  • Build c_glib using
    • ci/scripts/c_glib_build.sh $(pwd) /tmp/build

Additional Info

arrow-flight-sql.pc can be found in the /opt/homebrew/lib/pkgconfig directory.

ls /opt/homebrew/lib/pkgconfig/arrow*
/opt/homebrew/lib/pkgconfig/arrow-acero.pc
/opt/homebrew/lib/pkgconfig/arrow-compute.pc
/opt/homebrew/lib/pkgconfig/arrow-csv.pc
/opt/homebrew/lib/pkgconfig/arrow-dataset-glib.pc
/opt/homebrew/lib/pkgconfig/arrow-dataset.pc
/opt/homebrew/lib/pkgconfig/arrow-filesystem.pc
/opt/homebrew/lib/pkgconfig/arrow-flight-glib.pc
/opt/homebrew/lib/pkgconfig/arrow-flight-sql-glib.pc
/opt/homebrew/lib/pkgconfig/arrow-flight-sql.pc
/opt/homebrew/lib/pkgconfig/arrow-flight.pc
/opt/homebrew/lib/pkgconfig/arrow-glib.pc
/opt/homebrew/lib/pkgconfig/arrow-json.pc
/opt/homebrew/lib/pkgconfig/arrow.pc

arrow-flight-sql.pc cannot be found in the /tmp/local/lib/pkgconfig directory.

ls /tmp/local/lib/pkgconfig/arrow*
/tmp/local/lib/pkgconfig/arrow-acero.pc
/tmp/local/lib/pkgconfig/arrow-compute.pc
/tmp/local/lib/pkgconfig/arrow-csv.pc
/tmp/local/lib/pkgconfig/arrow-dataset-glib.pc
/tmp/local/lib/pkgconfig/arrow-dataset.pc
/tmp/local/lib/pkgconfig/arrow-filesystem.pc
/tmp/local/lib/pkgconfig/arrow-glib.pc
/tmp/local/lib/pkgconfig/arrow-json.pc
/tmp/local/lib/pkgconfig/arrow-substrait.pc
/tmp/local/lib/pkgconfig/arrow-testing.pc
/tmp/local/lib/pkgconfig/arrow.pc

Environment.

  • OS: macOS 14.7
  • Arrow 18.0.0-SNAPSHOT (f6d0dd3)

Component(s)

GLib
@kou kou changed the title [GLib] Meson selects incorrect arrow libraries in some situations. [GLib] Meson selects incorrect arrow libraries in some situations Oct 18, 2024
hiroyuki-sato added a commit to hiroyuki-sato/arrow that referenced this issue Oct 19, 2024
@hiroyuki-sato
apacheGH-44465: [GLib][C++] Meson searches libraries with specific ve…
5d5c74d 
…rsions.
@hiroyuki-sato hiroyuki-sato mentioned this issue Oct 19, 2024
Merged
@kou kou changed the title [GLib] Meson selects incorrect arrow libraries in some situations [GLib] Meson selects incorrect Arrow C++ libraries in some situations Oct 19, 2024
kou pushed a commit that referenced this issue Oct 19, 2024
@hiroyuki-sato
GH-44465: [GLib][C++] Meson searches libraries with specific versions. (
54697ea 
#44475)

### Rationale for this change

This PR fixes #44465. Meson selects incorrect Arrow C++ libraries in some situations. 

### What changes are included in this PR?

Meson searches arrow libraries with particular versions.
For example, if the c_glib version number is 18.0.0-SNAPSHOT, Meson searches the Arrow C++ library greater than 18.0.0-SNAPSHOT.

### Are these changes tested?

Yes.

If the required version libraries exist, install it properly.

```
Run-time dependency arrow found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-compute found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-csv found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-filesystem found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-json found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-orc found: YES 18.0.0-SNAPSHOT
Found CMake: /opt/homebrew/bin/cmake (3.30.5)
WARNING: CMake Toolchain: Failed to determine CMake compilers state
Run-time dependency arrow-cuda found: NO (tried pkgconfig, framework and cmake)
Run-time dependency arrow-acero found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-dataset found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-flight found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-flight-sql found: YES 18.0.0-SNAPSHOT
Run-time dependency gandiva found: YES 18.0.0-SNAPSHOT
Run-time dependency parquet found: YES 18.0.0-SNAPSHOT
```

If an environment doesn't install Arrow C++ 18.0.0-SNAPSHOT (Installed older version 17.0.0)

```
Run-time dependency arrow found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-compute found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-csv found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-filesystem found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-json found: YES 18.0.0-SNAPSHOT
Run-time dependency arrow-orc found: YES 18.0.0-SNAPSHOT
Found CMake: /opt/homebrew/bin/cmake (3.30.5)
WARNING: CMake Toolchain: Failed to determine CMake compilers state
Run-time dependency arrow-cuda found: NO (tried pkgconfig, framework and cmake)
Run-time dependency arrow-acero found: YES 18.0.0-SNAPSHOT
Dependency arrow-dataset found: NO. Found 17.0.0 but need: '>=18.0.0-SNAPSHOT'
Run-time dependency arrow-dataset found: NO (tried pkgconfig, framework and cmake)
Dependency arrow-flight found: NO. Found 17.0.0 but need: '>=18.0.0-SNAPSHOT'
Run-time dependency arrow-flight found: NO (tried pkgconfig, framework and cmake)
Dependency arrow-flight-sql found: NO. Found 17.0.0 but need: '>=18.0.0-SNAPSHOT'
Run-time dependency arrow-flight-sql found: NO (tried pkgconfig, framework and cmake)
Dependency gandiva found: NO. Found 17.0.0 but need: '>=18.0.0-SNAPSHOT'
...
Dependency parquet found: NO. Found 17.0.0 but need: '>=18.0.0-SNAPSHOT'
Run-time dependency parquet found: NO (tried pkgconfig and framework)
```

### Are there any user-facing changes?

Yes.

* GitHub Issue: #44465

Authored-by: Hiroyuki Sato <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
@kou kou added this to the 19.0.0 milestone Oct 19, 2024
@kou
Copy link
Member

kou commented Oct 19, 2024

Issue resolved by pull request 44475
#44475

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hiroyuki-sato hiroyuki-sato

Labels
Component: GLib Type: bug
Projects
None yet
Milestone
19.0.0
Development

Successfully merging a pull request may close this issue.

GH-44465: [GLib][C++] Meson searches libraries with specific versions. hiroyuki-sato/arrow
2 participants
@kou @hiroyuki-sato