Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sage: Fix on Darwin #264126

Merged
merged 9 commits into from
Sep 3, 2024
Merged

Sage: Fix on Darwin #264126

merged 9 commits into from
Sep 3, 2024

Conversation

Feyorsh
Copy link
Contributor

@Feyorsh Feyorsh commented Oct 29, 2023

Description of changes

There shouldn't be any fundamental blockers to building Sage on aarch64-darwin, given that it builds fine with Homebrew. I am quite inexperienced with Nix, so please let me know if I need to fix my code to be more idiomatic or otherwise correct.
Currently, I am able to build Sage and "pass" all the test cases: some test cases fail because of linker warnings when Cython compiles:

sage -t --long --random-seed=188234609292076795741796414804145521592 /nix/store/b79f4fps121z8lfdy3km6x51bb2xgiqc-sage-src-10.0/src/sage/symbolic/pynac.pxi
**********************************************************************
File "/nix/store/b79f4fps121z8lfdy3km6x51bb2xgiqc-sage-src-10.0/src/sage/symbolic/pynac.pxi", line 6, in sage.symbolic.pynac
Failed example:
    cython(  # optional - sage.misc.cython
    '''
    cimport sage.symbolic.expression
    ''')
Expected nothing
Got:
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-zlib-1.3/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-bzip2-1.0.8/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-expat-2.5.0/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-xz-5.4.4/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-libffi-3.4.4/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-gdbm-1.23/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-sqlite-3.43.1/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-readline-8.2p1/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-ncurses-6.4/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-openssl-3.0.11/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-apple-framework-SystemConfiguration-11.0.0/lib'
    ld: warning: directory not found for option '-L/nix/store/8ldla7hgqh9nfdmcxb7h41xkh5lrgfbl-tzdata-2023c/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-zlib-1.3/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-bzip2-1.0.8/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-expat-2.5.0/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-xz-5.4.4/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-libffi-3.4.4/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-gdbm-1.23/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-sqlite-3.43.1/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-readline-8.2p1/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-ncurses-6.4/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-openssl-3.0.11/lib'
    ld: warning: directory not found for option '-L/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-apple-framework-SystemConfiguration-11.0.0/lib'
    ld: warning: directory not found for option '-L/nix/store/8ldla7hgqh9nfdmcxb7h41xkh5lrgfbl-tzdata-2023c/lib'
    ld: warning: directory not found for option '-L/nix/store/2bbhxavs7l15k3rh72rlg2cp2iwlibvp-jmol-16.1.41/lib'
**********************************************************************
1 item had failures:
   1 of   2 in sage.symbolic.pynac
    [1 test, 1 failure, 18.40 s]

I'm assuming I need to poke around sage-env.nix as suggested here and add these missing dependencies to PKG_CONFIG_PATH. Besides that, Sage behaves as you would expect, all functional test cases pass just fine. Disabling tests is obviously not an actual solution, but I'm confident these issues should be somewhat straightforward to fix.

108e281: fflas-ffpack seems to build and pass all test cases successfully.
d154d8a32f031930d8f2ba80f9f60866fdf0d142: Interestingly, although m4ri builds fine, m4rie seems to only build properly on Apple Clang, not LLVM Clang. As suggested in an upstream issue, I changed -O2 to -O0 and it compiles. Please let me know if I should be modifying CFLAGS in a different way or changing hardening flags.
98a725a: There seems to be an issue with Apple's libintl assuming locale variables being set, which causes most of Giac's test cases to fail. Passing --disable-nls partially fixes the issue, leading to only 4 test cases failing. I would like to fix this completely, but according to a Sage discussion, it looks like these failures are not new. I also added some Darwin patches for Giac from Sage, but they likely have no bearing on whether or not the above issue gets resolved.
aff699e: When running Sage, I received warnings about awk not being found. Indeed, gawk was not an input into sage-env.nix. It seems surprising that only Darwin would require awk so I didn't guard it with lib.optionals stdenv.isDarwin; is this warning present on Linux without the inclusion of gawk?
c65031c8be5e03f4a9258c176da990e42e99ddca: I just used macosx as the target because it's what Sage uses and it looks like there's some issues with the multithreaded build on aarch64-linux. I haven't tested macosx-thr, though.
7e419ac: fpylll (and fplll) need to disable long double on both Cygwin and aarch64-darwin (x86_64-darwin does support long double and shouldn't be changed).

TODO

linbox: I noticed that a test failed when building linbox as part of Sage (i.e. nix-build -A sage), but when I built it separately all tests passed. I assumed that was just an issue with a test case running out of memory or timing out, but I don't know how to confirm that that's the case.
sympow: Install check fails, almost certainly due to this warning:

CFLAGS for SYMPOW:  -std=gnu17 -fno-fast-math -ffp-contract=on  -O0
The double precision of your FPU is 105 bits.
Error: the Quad Double library used by SYMPOW assumes IEEE-754 double
precision numbers with exactly 53 bits in the mantissa (64 bits in
total).  Unfortunately, this is not the case on your system and we
currently have no workaround for your system.  Running SYMPOW will
almost certainly fail on some inputs.

Sage packaging does not appear to fix this, so the "fix" might be simply disabling Sympow within Sage.
Giac: Pass remaining test cases (or ignore if infeasible).
Sage: Fix linker warnings as described above.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@ofborg ofborg bot added 6.topic: darwin Running or building packages on Darwin 8.has: package (new) This PR adds a new package labels Oct 29, 2023
@collares
Copy link
Member

Thanks, this is much appreciated!

Just a heads-up: I disabled Singular docbuilding on darwin last year because it was hanging and I didn't have a machine to debug it on. I would expect this to cause a Sage test failure or two, but don't worry about it if it doesn't cause test failures.

Copy link
Member

@symphorien symphorien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

xcas works fine

pkgs/development/libraries/science/math/m4rie/default.nix Outdated Show resolved Hide resolved
@Feyorsh
Copy link
Contributor Author

Feyorsh commented Oct 30, 2023

Fixed fpylll by disabling HAVE_LONG_DOUBLE in both fplll and fpylll. I backported a patch from the 0.6.0 release of fpylll because updating would require switching to Cython3, which causes conflicts with other packages in Sage that use Cython2. We should update all packages to Cython3 at the same time when Sage adds support for it, probably as part of the 10.2 release.

@collares
Copy link
Member

Agreed on the Cython3 point! By the way, feel free to ping me when you remove the draft status from the PR (since GitHub does not send notifications about it).

@7c6f434c
Copy link
Member

@Feyorsh Speaking of 10.2, @collares has just performed and merged this update.

@Feyorsh
Copy link
Contributor Author

Feyorsh commented Jan 27, 2024

@7c6f434c Thanks for the heads up — apologies for letting this sit in stasis. The only thing I couldn't fix were the Cython linker warnings (and sympow, which fundamentally cannot work on aarch64-darwin), but I will try to rebase on top of the 10.2 update to fix that.

@Feyorsh
Copy link
Contributor Author

Feyorsh commented Mar 19, 2024

Ok, I finally came back to this. There are several more changes that have to be made due to stdenv updating to Clang 16, which promotes a few warnings into errors and compiles with C++17 by default (at least I'm pretty sure it's the default...)

I initially tried compiling with C++14, but that's frankly a band-aid fix; the only packages I still need to fix are Giac and Sage itself (sagemath/sage#36840).

A few notes:

  • d364c83: What's the policy for deletion of packages? As far as I can tell Sage is the only user of FlintQS, but presumably we need to deprecate it instead of deleting it outright.
  • 8d1b4242327ce5be15e9207aab4908b8eddb1e1f: This is somewhat inelegant, but I see no better way to get the job done (I suppose I could just upstream the line ending change...)
  • cfaf402: I'm going to revert this and bump Giac to 1.9.0-93 and fix any other errors.
  • 08988c0: For future reference: Is there any issue fetching patches from PRs? Unsure if these have the same permalink guarantees as commits
  • 6b05225: The src tarball is not the same as the Git repo, so I had to modify a patch slightly.

Also: I still don't know how to fix the ld: warning: directory not found for option '-L/nix/store/... warnings. Any idea where linker flags are set besides buildInputs?

@symphorien
Copy link
Member

08988c0: For future reference: Is there any issue fetching patches from PRs? Unsure if these have the same permalink guarantees as commits

no they don't. if the PR is force pushed, then the hash is invalidated. You can fetch the commit of the PR, but even though the commit hash cannot be invalidated, the commit can be gc-ed and disappear if the PR is closed or force-pushed. So for non-merged-yet PRs the only truly future proof way is vendoring the patch.

@symphorien
Copy link
Member

d364c83: What's the policy for deletion of packages? As far as I can tell Sage is the only user of FlintQS, but presumably we need to deprecate it instead of deleting it outright.

you are supposed to add a line in pkgs/top-level/aliases.nix I think.

@collares
Copy link
Member

* [6b05225](https://github.com/NixOS/nixpkgs/commit/6b05225e362d7e004f854949742944101fdf3821): The `src` tarball is _not_ the same as the Git repo, so I had to modify a patch slightly.

If the only modification was deleting the .gitignore changes, you can pass excludes = [".gitignore"] to fetchpatch.

Thanks for the update, by the way! The changes here are looking good. I will probably review them in more detail on the weekend. I also just merged the Sage 10.3 upgrade, which included a Singular bump to 4.3.2p16.

@Feyorsh
Copy link
Contributor Author

Feyorsh commented Mar 20, 2024

Good to see 10.3 hot off the press, as I no longer have to backport a lot of package fixes — I'll rebase and share my local changes in a day or two.

I've gotten Giac 1.9.0-93 to compile (and fixed the Sage bugs), but test cases are "failing" (expressions are symbolically equivalent but not literally so; presumably not a bug introduced by the update to C++17) and Giac completely fails to work from within Sage. I'm sure Sage expects a decently older version of Giac, so I'll try on an older version at some point.

Linker errors are still present: I suspect this Darwin-specific behaviour in sage-env is to blame, and will investigate further.

@Feyorsh
Copy link
Contributor Author

Feyorsh commented Jul 25, 2024

Ok, I gated the gfan patch behind Clang. As for the Singular /tmp cleanup; is there really a downside? A sandboxed build on Linux will get an empty /tmp, and nothing will happen. But Darwin sandboxing is funky:

# sanity check: does sandboxing work?

$ nix-build --expr 'with import <nixpkgs> {}; runCommand "sandbox-test" {} "mkdir $out; ${curl}/bin/curl -sSf www.google.com > /dev/null"' --option sandbox false --check
checking outputs of '/nix/store/z1z2b1amyj7ppm2dn3yihaxviyfggncp-sandbox-test.drv'...
/nix/store/4bl0wz7nnncrmz53cmhx52vaf2jh6ib5-sandbox-test

$ nix-build --expr 'with import <nixpkgs> {}; runCommand "sandbox-test" {} "mkdir $out; ${curl}/bin/curl -sSf www.google.com > /dev/null"' --option sandbox true --check
checking outputs of '/nix/store/z1z2b1amyj7ppm2dn3yihaxviyfggncp-sandbox-test.drv'...
curl: (6) Could not resolve host: www.google.com
error: builder for '/nix/store/z1z2b1amyj7ppm2dn3yihaxviyfggncp-sandbox-test.drv' failed with exit code 6;
       last 1 log lines:
       > curl: (6) Could not resolve host: www.google.com
       For full logs, run 'nix log /nix/store/z1z2b1amyj7ppm2dn3yihaxviyfggncp-sandbox-test.drv'.

# ok, but does /tmp work?

$ nix-build --expr 'with import <nixpkgs> {}; runCommand "tmp-test" {} "mkdir $out; if [ ! -f /tmp/a ]; then touch /tmp/a; ls -la /tmp/a; else exit 1; fi"' --option sandbox true --check
checking outputs of '/nix/store/mrpgbikniafy8wqxwanr8c36daiz6zsm-tmp-test.drv'...
-rw-r--r-- 1 _nixbld1 wheel 0 Jul 25 17:03 /tmp/a
/nix/store/srci7gj0s8nwaxj4s1npa4nvqc6d0w79-tmp-test

$ nix-build --expr 'with import <nixpkgs> {}; runCommand "tmp-test" {} "mkdir $out; if [ ! -f /tmp/a ]; then touch /tmp/a; ls -la /tmp/a; else exit 1; fi"' --option sandbox true --check
checking outputs of '/nix/store/mrpgbikniafy8wqxwanr8c36daiz6zsm-tmp-test.drv'...
error: builder for '/nix/store/mrpgbikniafy8wqxwanr8c36daiz6zsm-tmp-test.drv' failed with exit code 1

See also this SO answer.

Naturally, this will also fail on any platform where sandboxing is disabled. While I agree that this is a hack and not ideal (Singular should be using $TMPDIR), it seems much less intrusive then patching Singular (and someone is already trying to fix this upstream).

@StefanoDeVuono
Copy link
Contributor

@Feyorsh It looks like the tmp issue with Singular has been stalled a bit. If you're down, maybe we could collab on a PR to submit to them?

I'm looking forward to Sage on Darwin through Nix!

@Feyorsh
Copy link
Contributor Author

Feyorsh commented Aug 8, 2024

@StefanoDeVuono Admittedly I'm lacking the motivation to see such a PR through — even if the bug with /tmp was fixed, the LaTeX interface in tropical.lib is probably overdue for a rewrite anyway. I initially patched in the xdg-open shim for macOS, then I realized macOS dropped Postscript support a year ago. I could bundle in Ghostscript, but it seemed like an increasingly convoluted solution to support a very niche feature.

I don't mean to discourage you from fixing it upstream though... maybe reach out to the Gentoo Sage maintainer who opened the issue?

(I'm also going to be AFK for the next two weeks, if this PR gets reviewed in that time. Would love to see this land before the upgrade to 10.4!)

Copy link
Member

@collares collares left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many apologies for taking so long to review your changes, the last few weeks were quite hectic on my side. Everything looks great! I only found a few minor nits. Happy to merge this PR when they're addressed.

pkgs/applications/science/math/gap/default.nix Outdated Show resolved Hide resolved
pkgs/applications/science/math/giac/default.nix Outdated Show resolved Hide resolved
pkgs/applications/science/math/giac/default.nix Outdated Show resolved Hide resolved
pkgs/applications/science/math/giac/default.nix Outdated Show resolved Hide resolved
Comment on lines 62 to 70
installCheckPhase = ''
export HOME=$TMPDIR
"$out/bin/sympow" -curve "[1,2,3,4,5]" -moddeg | grep 'Modular Degree is 464'
echo "[1,-1,0,-79,289]" | "$out/bin/sympow" -analrank | grep ^"Analytic Rank is 4"
"$out/bin/sympow" -curve "[1,-1,0,-79,289]" -analrank | grep ^"Analytic Rank is 4"
"$out/bin/sympow" -curve "[0,1,1,-2,0]" -analrank | grep ^"Analytic Rank is 2"
'' + lib.optionalString (!stdenv.isAarch64) ''
"$out/bin/sympow" -sp 2p16 -curve "[1,2,3,4,5]" | grep '8.3705'
'';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
installCheckPhase = ''
export HOME=$TMPDIR
"$out/bin/sympow" -curve "[1,2,3,4,5]" -moddeg | grep 'Modular Degree is 464'
echo "[1,-1,0,-79,289]" | "$out/bin/sympow" -analrank | grep ^"Analytic Rank is 4"
"$out/bin/sympow" -curve "[1,-1,0,-79,289]" -analrank | grep ^"Analytic Rank is 4"
"$out/bin/sympow" -curve "[0,1,1,-2,0]" -analrank | grep ^"Analytic Rank is 2"
'' + lib.optionalString (!stdenv.isAarch64) ''
"$out/bin/sympow" -sp 2p16 -curve "[1,2,3,4,5]" | grep '8.3705'
'';
installCheckPhase =
''
export HOME=$TMPDIR
"$out/bin/sympow" -curve "[1,2,3,4,5]" -moddeg | grep 'Modular Degree is 464'
echo "[1,-1,0,-79,289]" | "$out/bin/sympow" -analrank | grep ^"Analytic Rank is 4"
"$out/bin/sympow" -curve "[1,-1,0,-79,289]" -analrank | grep ^"Analytic Rank is 4"
"$out/bin/sympow" -curve "[0,1,1,-2,0]" -analrank | grep ^"Analytic Rank is 2"
''
+ lib.optionalString (!stdenv.isAarch64) ''
"$out/bin/sympow" -sp 2p16 -curve "[1,2,3,4,5]" | grep '8.3705'
'';

to appease our nixfmt overlords.

@collares
Copy link
Member

@ofborg build sageWithDoc giac giac-with-xcas

@Feyorsh Feyorsh requested a review from collares August 27, 2024 21:34
@collares
Copy link
Member

collares commented Sep 2, 2024

I think the sympow suggestion wasn't applied verbatim, because there's still a red X on this PR. I don't mind alternate formatting as long as we get a green tick in the nixfmt-check PR check. You can see the GitHub CI log for instructions on how to run nixfmt-rfc-check.

More worrying are the Giac-related sage test failures. For example:

**********************************************************************
File "/nix/store/qb6ad55k9xdrpy44hx3by2gxk7r3grqz-sage-src-10.3/src/sage/libs/giac/giac.pyx", line 303, in sage.libs.giac.giac._giac
Failed example:
    A[2,3] = 33; A[0,2] = '2/7' # set non zero entries of the sparse matrix
Exception raised:
    Traceback (most recent call last):
      File "/nix/store/yihsym29ypwq1l41ys0phw5wa0hl136i-python3-3.12.4-env/lib/python3.12/site-packages/sage/doctest/forker.py", line 712, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/nix/store/yihsym29ypwq1l41ys0phw5wa0hl136i-python3-3.12.4-env/lib/python3.12/site-packages/sage/doctest/forker.py", line 1147, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.libs.giac.giac._giac[29]>", line 1, in <module>
        A[Integer(2),Integer(3)] = Integer(33); A[Integer(0),Integer(2)] = '2/7' # set non zero entries of the sparse matrix
                                                ~^^^^^^^^^^^^^^^^^^^^^^^
      File "sage/libs/giac/giac.pyx", line 1042, in sage.libs.giac.giac.Pygen.__setitem__ (build/cythonized/sage/libs/giac/giac.cpp:164286)
        GIAC_sto(v,g,1,context_ptr)
    RuntimeError: bad_function_call

and

sage -t --long --random-seed=220277908603107882552103743915235334071 /nix/store/qb6ad55k9xdrpy44hx3by2gxk7r3grqz-sage-src-10.3/src/sage/calculus/calculus.py
**********************************************************************
File "/nix/store/qb6ad55k9xdrpy44hx3by2gxk7r3grqz-sage-src-10.3/src/sage/calculus/calculus.py", line 1855, in sage.calculus.calculus.inverse_laplace
Failed example:
    inverse_laplace((2*s^2*exp(-2*s) - exp(-s))/(s^3+1), s, t,
                    algorithm='giac')
Expected:
    -1/3*(sqrt(3)*e^(1/2*t - 1/2)*sin(1/2*sqrt(3)*(t - 1))
     - cos(1/2*sqrt(3)*(t - 1))*e^(1/2*t - 1/2) + e^(-t + 1))*heaviside(t - 1)
     + 2/3*(2*cos(1/2*sqrt(3)*(t - 2))*e^(1/2*t - 1) + e^(-t + 2))*heaviside(t - 2)
Got:
    Giac crashed -- automatically restarting.
    sage34*dirac_delta(sage36)

It's possible one of the errors is a side-effect of the other, but at least one should be real.

In unrelated news, unfortunately Sage recently broke on master due to the Cython 3.0.11 upgrade and I might have to update it to 10.4 to apply sagemath/sage#38500. But if Sage tests pass on this branch, I don't see any problem in merging this PR. Note that you'll have to autosquash the fixup commits, since GitHub does not do that automatically.

@collares
Copy link
Member

collares commented Sep 2, 2024

@Feyorsh Turns out the upstreamed Giac patches I mentioned (which you migrated to in 1d1710c) do not exactly match your previous version, and everything works before the change I suggested. Apologies, I should have tested it before making the suggestion.

If you reinstate remove-old-functional-patterns.patch, fix the sympow formatting error (as determined by the automatic checks) and autosquash the fixups, I can merge this. Let me know if you prefer that I do it instead (I'd be happy to).

Nearly fixed on aarch64-darwin! (Note that this coincides with a
release of Xcas for macOS, but this is not using Parisse's tarball).
This issue does not present at build time, but only in sage
doctests. See https://www.github.com/sagemath/sage/issues/25118
--enable-sage was removed a while ago
linbox-team/linbox#146
This upgrade fixes the issue with drawTropicalCurve docbuilding
on Darwin (although said doctest is pretty useless now that Apple has
removed postscript from macOS).

Removed enableGfanlib; you can still overrideAttrs

Added mpfr to be more consistent with upstream packaging
recommendations
Also backport a fix that silences `xcode-select: not found` warnings
Tampering with ulimit is not ideal and may not work in the Darwin
sandbox, but is necessary due to how Singular handles allocations.
@Feyorsh
Copy link
Contributor Author

Feyorsh commented Sep 2, 2024

That's on me for not running the Sage tests before pushing. Based on my correspondence with Parisse, it seems my patches were more of a loose inspiration than anything else (let's just hope it works when Giac gets bumped in the future). I'm running a full build currently and I'll let you know if I encounter any issues with the Sage test suite; if not, this should be good to go.

@Feyorsh
Copy link
Contributor Author

Feyorsh commented Sep 2, 2024

I'm running a full build currently and I'll let you know if I encounter any issues

lgtm @collares

@collares
Copy link
Member

collares commented Sep 3, 2024

@ofborg build giac-with-xcas sageWithDoc

@collares
Copy link
Member

collares commented Sep 3, 2024

Wonderful, thank you very much! Excited to have Darwin support for Nixpkgs Sage.

@collares collares merged commit 72331bf into NixOS:master Sep 3, 2024
24 of 30 checks passed
@collares
Copy link
Member

collares commented Sep 5, 2024

@Feyorsh Unfortunately Maxima failed to build on aarch64-darwin: https://hydra.nixos.org/build/271482748 and https://cache.nixos.org/log/jq3wzld79yzdqaxqlvlsyfa1x7nm8vk0-maxima-5.47.0.drv

;;; Compiling /private/tmp/nix-build-maxima-5.47.0.drv-0/maxima-5.47.0/src/cl-info.lisp.
;;; OPTIMIZE levels: Safety=2, Space=0, Speed=3, Debug=2
;;;
;;; Internal error:
;;;   in file cl-info.lisp, position 9508
;;;   at (DEFUN LOAD-INFO-HASHTABLES ...)
;;;   ** Wrong number of arguments passed to an anonymous function
;      - Binary file binary-ecl/cl-info.fas is old or does not exist. 
;        Compile (and load) source file /private/tmp/nix-build-maxima-5.47.0.drv-0/maxima-5.47.0/src/cl-info.lisp instead? y
;      - Should I bother you if this happens again? y
;      - Compiling source file
;        "/private/tmp/nix-build-maxima-5.47.0.drv-0/maxima-5.47.0/src/cl-info.lisp"

@Feyorsh
Copy link
Contributor Author

Feyorsh commented Sep 6, 2024

Looking into it. Can't reproduce the error with maxima in the sandbox, so I suspect one of the Sage-specific patches was made obsolete with the bump to 5.47.

@collares
Copy link
Member

collares commented Sep 7, 2024

Are you testing Maxima as it is built for Sage? That is,

pkgs.maxima-ecl.override { lisp-compiler = pkgs.ecl.override { threadSupport = false; useBoehmgc = true; }; }

In particular, ECL as the lisp compiler seems to always use GCC as part of the Lisp compilation process (as seen below):

  postInstall = ''
    sed -e 's/@[-a-zA-Z_]*@//g' -i $out/bin/ecl-config
    wrapProgram "$out/bin/ecl" --prefix PATH ':' "${
      lib.makeBinPath [
        gcc                   # for the C compiler
        gcc.bintools.bintools # for ar
      ]
    }"
  '';

If that's not the issue and you're not seeing the error even in this configuration, my guess would be that it is an intermittent GC problem (maybe related to https://gitlab.com/embeddable-common-lisp/ecl/-/issues/718 or ivmai/bdwgc#569 (comment), although in our case it wouldn't be happening at ECL build time.).

The reason I'm even suspecting a GC issue is that maxima-ecl builds fine: https://hydra.nixos.org/build/271043586

@Feyorsh
Copy link
Contributor Author

Feyorsh commented Sep 7, 2024

Yes, I tested using 74b34d6 per the Hydra reproduce script, with maxima.override { lisp-compiler = ecl.override { threadSupport = false; useBoehmgc = true; }; } (expression is slightly different as I built the top level derivation instead of through Sage). It might be difficult to reproduce this locally if the Heisenbug only appears under load.

Unfortunately I need another 5 days until I can give this proper attention; this issue doesn't seem to be a regression caused by this PR, but if these commits need to be reverted, so be it. Additional support should not come at the expense of breaking existing packages.

@collares
Copy link
Member

collares commented Sep 8, 2024

Oh, sorry for giving the wrong impression! Fixing the Maxima issue is not urgent, and this PR does not need to be reverted. maxima-ecl itself didn't break, the failure was just for the custom version of Maxima used on Sage, and just on aarch64-darwin.

@collares
Copy link
Member

collares commented Sep 8, 2024

Also not urgent, just for your information: Sage Maxima built OK on staging-next, so it seems like it was just an intermittent failure. But Singular is failing with

**** Error: while running example drawNewtonSubdivision from d2t_singular/tropical_lib.doc:665.
Call: '../Singular/Singular  -teqr12345678 --no-rc ./examples/drawNewtonSubdivision.sing > ./examples/drawNewtonSubdivision.res'
make[4]: *** [Makefile-docbuild:180: d2t_singular/tropical_lib.tex] Error 1

**** Error: non-zero exit status of system call: 'make -f Makefile-docbuild  --no-print-directory -s TAG='-tag gfan' VERBOSE=1 ./d2t_singular/tropical_lib.tex': Inappropriate ioctl for device

This seems less likely to be an intermittent failure, but perhaps we should wait for a rebuild. See https://hydra.nixos.org/build/271681864 for details.

@collares
Copy link
Member

Seems like the Singular failure was also intermittent! Only the symmetrica failure remaining.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants