Skip to content

Commit

Permalink
Build libgsingular_julia on demand
Browse files Browse the repository at this point in the history
If the source code in this repository differs from what was used to build
libsingular_julia_jll, then recompile libsingular_julia. This requires a
working C++ compiler. It is intended for development, not for use in releases.
  • Loading branch information
fingolfin committed Jun 13, 2023
1 parent ebb7d1d commit 652a22e
Show file tree
Hide file tree
Showing 11 changed files with 211 additions and 924 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
*.jl.cov
*.jl.mem
/bin/
/deps/build*/
/deps/install*/
/docs/build/
/gh-pages/
/local/*
Expand Down
3 changes: 3 additions & 0 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@ version = "0.18.5"
AbstractAlgebra = "c3fe647b-3220-5bb0-a1ea-a7954cac585d"
BinaryWrappers = "f01c122e-0ea1-4f85-ad8f-907073ad7a9f"
CxxWrap = "1f15a43c-97ca-5a2a-ae31-89f07a497df4"
Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
Nemo = "2edaba10-b0f1-5616-af89-8c11ac63239a"
Pidfile = "fa939f87-e72e-5be4-a000-7fc836dbe307"
Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
RandomExtensions = "fb686558-2515-59ef-acaa-46db3789a887"
Expand All @@ -21,6 +23,7 @@ AbstractAlgebra = "0.28, 0.29, 0.30"
BinaryWrappers = "~0.1.1"
CxxWrap = "0.11, 0.12, 0.13"
Nemo = "0.33.0, 0.34.0"
Pidfile = "1.3"
RandomExtensions = "0.4.2"
Singular_jll = "~403.203.200"
julia = "1.6"
Expand Down
86 changes: 35 additions & 51 deletions README.maintainer.md
Original file line number Diff line number Diff line change
@@ -1,109 +1,93 @@
# Directions for updating Singular.jl

`Singular.jl` depends on the Singular kernel and various C++ wrappers maintained
in `libsingular_julia`. These come as binary artifacts `Singular_jll` and
`Singular.jl` depends on the Singular kernel and some C++ glue code found in
the `deps/src/` directory. Compiled versions of each are distributed to users
as binary artifacts via the Julia "JLL" packages `Singular_jll` and
`libsingular_julia_jll` respectively.

The build scripts for each of these:
The build scripts for these JLL packages can be found here:

- <https://github.com/JuliaPackaging/Yggdrasil/blob/master/S/Singular/build_tarballs.jl>
- <https://github.com/JuliaPackaging/Yggdrasil/blob/master/L/libsingular_julia/build_tarballs.jl>

The resulting JLL packages

- <https://github.com/JuliaBinaryWrappers/Singular_jll.jl>
- <https://github.com/JuliaBinaryWrappers/libsingular_julia_jll.jl>

The sources:

- <https://github.com/Singular/Singular>
- <https://github.com/oscar-system/libsingular-julia>
- `Singular` sources: <https://github.com/Singular/Singular>
- `libsingular_julia` sources: `deps/src/` directory of `Singular.jl`

## updating libsingular_julia
## Updating just the C++ wrappers

Suppose just the C++ wrappers need to be updated, without any changes to the
Singular kernel itself.

1. Commit changes to the `libsingular_julia` source
ex: https://github.com/oscar-system/libsingular-julia/commit/65a12e7fb8546851d3296244aeacc5de1d97af2c
1. Commit changes to the `deps/src/` directory.
> ex: <https://github.com/oscar-system/Singular.jl/commit/73af1a6b0f99c11f00837c535db818ca2de7d9a2>
2. After the changes are merged (and before the next `Singular.jl` release), update
the `libsingular_julia` build script with a new version number and using the
latest commit SHA for the `master` branch of `Singular.jl`.

1. Commit changes to the `deps/src/` directory
> ex: <https://github.com/oscar-system/Singular.jl/commit/73af1a6b0f99c11f00837c535db818ca2de7d9a2>
2. Update the `libsingular_julia` build script with a new version number the commit SHA
ex: https://github.com/JuliaPackaging/Yggdrasil/commit/dd9d775f530b1164dd1cf6135677846dcb25fafc
> ex: <https://github.com/JuliaPackaging/Yggdrasil/commit/dd9d775f530b1164dd1cf6135677846dcb25fafc>
3. Wait for this to be merged into Yggdrasil, and then wait for the registry to pick up the new version of `libsingular_julia_jll`
ex: https://github.com/JuliaRegistries/General/commit/00624106c204ccb28fb2c834877fb6005c653ce7
> ex: <https://github.com/JuliaRegistries/General/commit/00624106c204ccb28fb2c834877fb6005c653ce7>
4. Bump the dependence in `Singular.jl` to whatever version number was used in Step 2.
ex: https://github.com/oscar-system/Singular.jl/commit/f669cea1aa4fc73082d8e8c08a1e33a7c22882ed
> ex: <https://github.com/oscar-system/Singular.jl/commit/f669cea1aa4fc73082d8e8c08a1e33a7c22882ed>
Version compatibility notation: https://pkgdocs.julialang.org/v1/compatibility/
Version compatibility notation: <https://pkgdocs.julialang.org/v1/compatibility/>

5. Release a new `Singular.jl`. This is done by pinging JuliaRegistrator in the comments of a commit.
ex: https://github.com/oscar-system/Singular.jl/commit/cbc04de26507b386e27d2bd7a23a2f913903d9cc
> ex: <https://github.com/oscar-system/Singular.jl/commit/cbc04de26507b386e27d2bd7a23a2f913903d9cc>
After the new version of `Singular.jl` is picked up by the registry, it may be used
in further downstream packages.

Note: if you would like test simultaneous changes to both `libsingular_julia` and
`Singular.jl`, create pull requests against both with the same branch name.
The CI should show some "matching: " entries.
ex: https://github.com/oscar-system/libsingular-julia/pull/55

## updating the Singular kernel
## Updating the Singular kernel

Suppose the Singular kernel needs an update. This involves updating both build
scripts because `libsingular_julia_jll` will need to point to the new `Singular_jll`.

1. Update the Singular build script with the commit SHA of the singular sources (https://github.com/Singular/Singular)
ex: https://github.com/JuliaPackaging/Yggdrasil/commit/1996668f09e89bc8da0cc498c59d905821b150c8
> ex: <https://github.com/JuliaPackaging/Yggdrasil/commit/1996668f09e89bc8da0cc498c59d905821b150c8>
In this specific commit, `FLINT_jll` was also updated, but this is not necessary to update singular.
Any build issues need to be communicated to https://github.com/Singular/Singular
Any build issues need to be communicated to <https://github.com/Singular/Singular>
until you get a commit that builds on all targets.

2. Wait for the Yggdrasil merge, and wait for the registry.
ex: https://github.com/JuliaRegistries/General/commit/c73ad5d2ec0483113ffc2eca9b623049a7d8e81a
> ex: <https://github.com/JuliaRegistries/General/commit/c73ad5d2ec0483113ffc2eca9b623049a7d8e81a>
3. Update the `libsingular_julia` build scripts with a new version and `Singular_jll` dependency.
ex: https://github.com/JuliaPackaging/Yggdrasil/commit/cc75959bcf85b016625d9e528db277fd4d3b32d9

Supposedly, Yggdrasil does not like new versions with the same commit SHA.
Therefore, if `https://github.com/oscar-system/libsingular-julia.git` has not
changed since the last version, add a dummy commit and use the new SHA.
> ex: <https://github.com/JuliaPackaging/Yggdrasil/commit/cc75959bcf85b016625d9e528db277fd4d3b32d9>
At this point, we have a new `libsingular_julia_jll` in the works, and the steps
are essentially Steps 3-5 in the previous section.

4. The usual waiting.
ex: https://github.com/JuliaRegistries/General/commit/5c91a1184ed468d2a014c149f19caf84748fb83f
> ex: <https://github.com/JuliaRegistries/General/commit/5c91a1184ed468d2a014c149f19caf84748fb83f>
5. Bump the `libsingular_julia_jll` and `Singular_jll` dependencies in `Singular.jl`.
ex: https://github.com/oscar-system/Singular.jl/commit/138eeb12b5105e252c086bc45f259704e74c9885
> ex: <https://github.com/oscar-system/Singular.jl/commit/138eeb12b5105e252c086bc45f259704e74c9885>
This commit also bumps `AbstractAlgebra` and `Nemo`, but this is not necessary
just to update Singular.

6. Release new `Singular.jl` version.
ex: https://github.com/oscar-system/Singular.jl/commit/12f3b6b8689177d55f3952a8e5f9dccf233be885
> ex: <https://github.com/oscar-system/Singular.jl/commit/12f3b6b8689177d55f3952a8e5f9dccf233be885>
## updating both libsingular_julia and the Singular kernel
## Updating both `libsingular_julia` and the Singular kernel

Since updating the Singular kernel requires an update to `libsingular_julia`, the
steps here are the same as in the previous section. Just make sure that in
Step 3, the commit SHA used to update the `libsingular_julia` build scripts
contains all of the desired changes to `libsingular_julia`.

# Overriding the libsingular_julia_jll artifact

Suppose you have cloned
https://github.com/oscar-system/libsingular-julia
into, say `/my/path`, and would like to test your local changes.

First, you must have a file `.julia/artifacts/Overrides.toml` with the entry

```
[ae4fbd8f-ecdb-54f8-bbce-35570499b30e]
libsingular_julia = "/my/path/libsingular-julia/build"
```

Next, from `/my/path` run `julia build.jl` which will build the package with
your changes.

Finally, the updated `libsingular_julia_jll` should be available the next time
julia starts. Packages that depend on it may need to be precompiled again to
pick up the changes.
64 changes: 42 additions & 22 deletions deps/src/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,22 +1,20 @@
cmake_minimum_required(VERSION 3.01)

# I don't know why I need the following, but it works
if(POLICY CMP0025)
cmake_policy(SET CMP0025 NEW)
endif()

project(libsingular_julia)

set(JL_TARGET singular_julia)

set(CMAKE_LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib")

find_package(JlCxx REQUIRED)
get_target_property(JlCxx_location JlCxx::cxxwrap_julia LOCATION)
get_filename_component(JlCxx_location ${JlCxx_location} DIRECTORY)
message(STATUS "Found JlCxx at ${JlCxx_location}")

include(CheckCXXCompilerFlag)

set(CMAKE_CXX_STANDARD 14)

set(JLSINGULAR_TARGET singular_julia)

# avoid gcc 9 internal compiler error,
# see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90998
if(CMAKE_COMPILER_IS_GNUCC AND CMAKE_CXX_COMPILER_VERSION VERSION_GREATER 9.0
Expand All @@ -28,25 +26,47 @@ endif()
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib")
set(CMAKE_INSTALL_LIBDIR "lib")

get_target_property(JlCxx_location JlCxx::cxxwrap_julia LOCATION)
get_filename_component(JlCxx_location ${JlCxx_location} DIRECTORY)
set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib;${JlCxx_location}")

message(STATUS "Found JlCxx at ${JlCxx_location}")

include_directories(${CMAKE_INSTALL_PREFIX})
include_directories(${Singular_PREFIX}/include/singular)

SET( CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g ${extra_cppflags} -I${CMAKE_INSTALL_PREFIX}/include -I${Singular_PREFIX}/include/" )
SET( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -v ${extra_ldflags} -L${CMAKE_INSTALL_PREFIX}/lib -Wl,-rpath,${CMAKE_INSTALL_PREFIX}/lib -L${Singular_PREFIX}/lib -Wl,-rpath,${Singular_PREFIX}/lib" )


add_library(${JLSINGULAR_TARGET} SHARED singular.cpp rings.cpp coeffs.cpp ideals.cpp matrices.cpp caller.cpp coeff_rings.cpp threading.cpp)

target_link_libraries(${JLSINGULAR_TARGET} JlCxx::cxxwrap_julia -lSingular -lpolys -lsingular_resources -lfactory -lomalloc -ldl -lgmp)
include_directories(
${Singular_PREFIX}/include
${Singular_PREFIX}/include/singular
${gmp_prefix}/include
)

link_directories(
${Singular_PREFIX}/lib
${gmp_prefix}/lib
)

#SET( CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -g ${extra_cppflags}" )
#SET( CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -v ${extra_ldflags} )


add_library(${JL_TARGET} SHARED
singular.cpp
rings.cpp
coeffs.cpp
ideals.cpp
matrices.cpp
caller.cpp
coeff_rings.cpp
threading.cpp
)

target_link_libraries(${JL_TARGET}
JlCxx::cxxwrap_julia
-lSingular
-lpolys
-lsingular_resources
-lfactory
-lomalloc
-ldl
-lgmp
)

install(TARGETS
${JLSINGULAR_TARGET}
${JL_TARGET}
ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
Expand Down
Loading

0 comments on commit 652a22e

Please sign in to comment.