Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin pyarrow<13 on macOS to avoid crash (sc-51866) #134

Closed
wants to merge 4 commits into from

Conversation

ihnorton
Copy link
Member

@ihnorton ihnorton commented Aug 1, 2024

Temporarily pin pyarrow<13 on macOS to avoid crash (sc-51866).

@ihnorton
Copy link
Member Author

ihnorton commented Aug 1, 2024

This does not appear to be possible for 2.25

Could not solve for environment specs
The following packages are incompatible
├─ aws-crt-cpp 0.26.12.* h431af13_1 is requested and can be installed;
└─ libtiledbvcf 0.34.0.*  is not installable because there are no viable options
   ├─ libtiledbvcf 0.34.0 would require
   │  └─ tiledb >=2.25.0,<2.26.0a0 , which requires
   │     └─ aws-crt-cpp >=0.27.3,<0.27.4.0a0 , which conflicts with any installable versions previously reported;
   └─ libtiledbvcf 0.34.0 conflicts with any installable versions previously reported.

@jdblischak
Copy link
Collaborator

This does not appear to be possible for 2.25

I confirmed this. The latest tiledb that can be co-installed with pyarrow 12 is 2.24.1. The reason is the aws-crt-cpp pin (though there may be additional incompatible pins; this is the just the first incompatibility the solver finds). pyarrow 12 was last built against aws-crt-cpp 0.26, but tiledb 2.25 was first built against aws-crt-cpp 0.27.

mamba create --dry-run -n test -c conda-forge 'pyarrow<13' tiledb
## + aws-crt-cpp                    0.26.12  he940a02_1            conda-forge      340kB
## + tiledb                          2.24.1  h2aae180_1            conda-forge        4MB
## + python                          3.11.9  hb806964_0_cpython    conda-forge       31MB
## + libarrow                        12.0.1  h0de2578_50_cpu       conda-forge       28MB
## + pyarrow                         12.0.1  py311h8eb17fa_50_cpu  conda-forge        4MB

# pyarrow 12 was not built against aws-crt-cpp 0.27
mamba create --dry-run -n test -c conda-forge 'pyarrow<13' aws-crt-cpp=0.27
# long solver error

# tiledb 2.25 was not built against aws-crt-cpp 0.26
mamba create --dry-run -n test -c conda-forge tiledb=2.25 aws-crt-cpp=0.26
## The following packages are incompatible
## ├─ aws-crt-cpp 0.26**  is requested and can be installed;
## └─ tiledb 2.25**  is not installable because it requires
##    └─ aws-crt-cpp >=0.27.3,<0.27.4.0a0 , which conflicts with any installable versions previously reported.

@@ -79,7 +80,8 @@ outputs:
- numpy
- libtiledbvcf {{ version }}
- python
- pyarrow
- pyarrow <13 # [osx and py<312]
- pyarrow # [not osx and py>=312]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- pyarrow # [not osx and py>=312]
- pyarrow # [not osx or py>=312]

https://en.wikipedia.org/wiki/De_Morgan%27s_laws

@jdblischak
Copy link
Collaborator

Wait a second. I realized I don't understand the context of this PR. What is the crash on macOS? Has a new error been identified?

I know in the past that we pinned pyarrow in order to avoid a macOS-specific segfault due to the AWS SDK (apache/arrow#42154), but it's my understanding that this bug only affects the PyPI wheels because both tiledb and pyarrow bundle statically linked AWS C libraries.

xref: #114, #111

@ihnorton ihnorton closed this Aug 9, 2024
@ihnorton
Copy link
Member Author

ihnorton commented Aug 9, 2024

Closing, @gspowley is working on the underlying issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants