Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[openfhe_julia_int128] initialize and [openfhe_julia] update #9961

Open
wants to merge 39 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
3380772
[openfhe_julia_int128] Initialize
ArseniyKholod Dec 9, 2024
39b1c3e
introduce common.jl
ArseniyKholod Dec 9, 2024
04dbead
Update common.jl
ArseniyKholod Dec 9, 2024
057a282
Update common.jl
ArseniyKholod Dec 9, 2024
4d91a16
disable apple
ArseniyKholod Dec 10, 2024
91c1ffd
enable apple
ArseniyKholod Dec 10, 2024
a604c58
tru later gcc
ArseniyKholod Dec 10, 2024
da24c49
Update build_tarballs.jl
ArseniyKholod Dec 10, 2024
a38a300
use gcc in macos
ArseniyKholod Dec 10, 2024
3b14d12
Apply suggestions from code review
ArseniyKholod Dec 10, 2024
ce66344
Merge branch 'openfhe_julia_128' of https://github.com/ArseniyKholod/…
ArseniyKholod Dec 10, 2024
3877a16
Set gcc and g++ compilers for apple and freebsd
ArseniyKholod Dec 10, 2024
b62a12b
filter 32 bit systems
ArseniyKholod Dec 10, 2024
3ced495
set new compiler flag
ArseniyKholod Dec 10, 2024
63dbfbd
try to built from pr with GNU_SOURCE
ArseniyKholod Dec 10, 2024
4bce691
add gnu source as command line option
ArseniyKholod Dec 10, 2024
76d8bbf
new try
ArseniyKholod Dec 10, 2024
1c2f462
try sdk 14
ArseniyKholod Dec 11, 2024
fb52087
disable apple
ArseniyKholod Dec 12, 2024
61a869a
remove new line
ArseniyKholod Dec 12, 2024
7de4518
add lc++abi flag
ArseniyKholod Dec 12, 2024
9da4f50
Update common.jl
ArseniyKholod Dec 12, 2024
664cfe9
add apple patch
ArseniyKholod Dec 12, 2024
d836fb2
Update apple-fix-cmake-typeinfo-int128.patch
ArseniyKholod Dec 12, 2024
de2fe7d
add linker flag
ArseniyKholod Dec 12, 2024
e37a785
use appropriate linker command
ArseniyKholod Dec 13, 2024
00ef798
remove bundled from deps
ArseniyKholod Dec 13, 2024
551f9ba
export outside if
ArseniyKholod Dec 13, 2024
cb6b588
Update common.jl
ArseniyKholod Dec 13, 2024
8889429
fix
ArseniyKholod Dec 13, 2024
86d75df
change to all apple, not apple darwin
ArseniyKholod Dec 13, 2024
a6b31cb
Update common.jl
ArseniyKholod Dec 13, 2024
af09f14
set CXXFLAGS instead of LDFLAGS
ArseniyKholod Dec 13, 2024
cf81dad
try out llvm-config
ArseniyKholod Dec 13, 2024
9113b7e
undo
ArseniyKholod Dec 13, 2024
55ccb4d
try macosx14.5.sdk
ArseniyKholod Dec 17, 2024
650a7cc
delete linker flag
ArseniyKholod Dec 17, 2024
8c6141f
add tabs
ArseniyKholod Dec 17, 2024
b117bc8
Update common.jl
ArseniyKholod Dec 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions O/openfhe_julia/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
The OpenFHE library works with both 64-bit and 128-bit integers.

This package builds upon the default 64-bit version.

To use the 128-bit version, use
[openfhe_julia_int128](https://github.com/JuliaPackaging/Yggdrasil/tree/master/O/openfhe_julia_int128).

Most of the build steps for openfhe_julia and
[openfhe_julia_int128](https://github.com/JuliaPackaging/Yggdrasil/tree/master/O/openfhe_julia_int128) are the same,
so the shared parts are in the
[common.jl](https://github.com/JuliaPackaging/Yggdrasil/blob/master/O/openfhe_julia/common.jl) script.
88 changes: 6 additions & 82 deletions O/openfhe_julia/build_tarballs.jl
Original file line number Diff line number Diff line change
@@ -1,93 +1,17 @@
# Note that this script can accept some limited command-line arguments, run
# `julia build_tarballs.jl --help` to see a usage message.
using BinaryBuilder, Pkg

# See https://github.com/JuliaLang/Pkg.jl/issues/2942
# Once this Pkg issue is resolved, this must be removed
uuid = Base.UUID("a83860b7-747b-57cf-bf1f-3e79990d037f")
delete!(Pkg.Types.get_last_stdlibs(v"1.6.3"), uuid)
include("common.jl")

# If you make changes in this file, e.g., to release a new version,
# be sure to also release a new version of `openfhe_julia_int128` as well (see `../openfhe_julia_int128/build_tarballs.jl`)
name = "openfhe_julia"
version = v"0.3.6"

# Collection of sources required to complete build
sources = [
GitSource("https://github.com/hpsc-lab/openfhe-julia.git",
"4939f1dd11ae2640b5bb397f731df8ef94307e74"),
]

# Bash recipe for building across all platforms
script = raw"""
cd $WORKSPACE/srcdir/openfhe-julia/
if [[ "${target}" == *-mingw* ]]; then
# This is needed because otherwise we get unusable binaries (error "The specified
# executable is not a valid application for this OS platform"). These come from
# CompilerSupportLibraries_jll.
# xref: https://github.com/JuliaPackaging/Yggdrasil/issues/7904
#
# The remove path pattern matches `lib/gcc/<triple>/<major>/`, where `<triple>` is the
# platform triplet and `<major>` is the GCC major version with which CSL was built
# xref: https://github.com/JuliaPackaging/Yggdrasil/pull/7535
#
# However, before CSL v1.1, these files were located in just `lib/`, thus we clean this
# directory as well.
if test -n "$(find $prefix/lib/gcc/*mingw*/*/libgcc*)"; then
rm $prefix/lib/gcc/*mingw*/*/libgcc* $prefix/lib/gcc/*mingw*/*/libmsvcrt*
elif test -n "$(find $prefix/lib/libgcc*)"; then
rm $prefix/lib/libgcc* $prefix/lib/libmsvcrt*
else
echo "Could not find any libraries to remove :-/"
find $prefix/lib
fi
fi
mkdir build && cd build
cmake .. \
-DCMAKE_INSTALL_PREFIX=$prefix \
-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
-DCMAKE_BUILD_TYPE=Release \
-DJulia_PREFIX=$prefix
make -j${nproc}
make install
"""

# These are the platforms we will build for by default, unless further
# platforms are passed in on the command line
include("../../L/libjulia/common.jl")
platforms = vcat(libjulia_platforms.(julia_versions)...)

# We cannot build with musl since OpenFHE requires the `execinfo.h` header for `backtrace`
platforms = filter(p -> libc(p) != "musl", platforms)

# PowerPC and 32-bit x86 and 64-bit FreeBSD on ARM 64 platforms are not supported by OpenFHE
platforms = filter(p -> arch(p) != "i686", platforms)
platforms = filter(p -> arch(p) != "powerpc64le", platforms)
platforms = filter(p -> !(Sys.isfreebsd(p) && arch(p) == "aarch64"), platforms)

# Expand C++ string ABIs since we use std::string
platforms = expand_cxxstring_abis(platforms)

git_hash = "4939f1dd11ae2640b5bb397f731df8ef94307e74"

# The products that we will ensure are always built
products = [
LibraryProduct("libopenfhe_julia", :libopenfhe_julia),
]
sources, script, platforms, products, dependencies = prepare_openfhe_julia_build(name, git_hash)

# Dependencies that must be installed before this package can be built
dependencies = [
BuildDependency(PackageSpec(;name="libjulia_jll", version=v"1.10.9")),
Dependency(PackageSpec(name="libcxxwrap_julia_jll", uuid="3eaa8342-bff7-56a5-9981-c04077f7cee7"); compat="0.13.0"),
Dependency(PackageSpec(name="OpenFHE_jll", uuid="a2687184-f17b-54bc-b2bb-b849352af807"); compat="1.2.3"),
# For OpenMP we use libomp from `LLVMOpenMP_jll` where we use LLVM as compiler (BSD
# systems), and libgomp from `CompilerSupportLibraries_jll` everywhere else.
Dependency(PackageSpec(name="CompilerSupportLibraries_jll", uuid="e66e0078-7015-5450-92f7-15fbd957f2ae");
platforms=filter(!Sys.isbsd, platforms)),
Dependency(PackageSpec(name="LLVMOpenMP_jll", uuid="1d63c593-3942-5779-bab2-d838dc0a180e");
platforms=filter(Sys.isbsd, platforms)),
]
push!(dependencies, Dependency(PackageSpec(name="OpenFHE_jll", uuid="a2687184-f17b-54bc-b2bb-b849352af807"); compat="1.2.3"))

# Build the tarballs, and possibly a `build.jl` as well.
build_tarballs(ARGS, name, version, sources, script, platforms, products, dependencies;
Expand Down
108 changes: 108 additions & 0 deletions O/openfhe_julia/common.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
using BinaryBuilder, Pkg

include("../../L/libjulia/common.jl")

function prepare_openfhe_julia_build(name::String, git_hash::String)
# See https://github.com/JuliaLang/Pkg.jl/issues/2942
# Once this Pkg issue is resolved, this must be removed
uuid = Base.UUID("a83860b7-747b-57cf-bf1f-3e79990d037f")
delete!(Pkg.Types.get_last_stdlibs(v"1.6.3"), uuid)

# Collection of sources required to complete build
sources = [
GitSource("https://github.com/hpsc-lab/openfhe-julia.git",
git_hash),
ArchiveSource("https://github.com/alexey-lysiuk/macos-sdk/releases/download/14.5/MacOSX14.5.tar.xz",
"f6acc6209db9d56b67fcaf91ec1defe48722e9eb13dc21fb91cfeceb1489e57e"),
]

# Bash recipe for building across all platforms
script = raw"""
cd $WORKSPACE/srcdir/openfhe-julia/

if [[ "${target}" == *-mingw* ]]; then
# This is needed because otherwise we get unusable binaries (error "The specified
# executable is not a valid application for this OS platform"). These come from
# CompilerSupportLibraries_jll.
# xref: https://github.com/JuliaPackaging/Yggdrasil/issues/7904
#
# The remove path pattern matches `lib/gcc/<triple>/<major>/`, where `<triple>` is the
# platform triplet and `<major>` is the GCC major version with which CSL was built
# xref: https://github.com/JuliaPackaging/Yggdrasil/pull/7535
#
# However, before CSL v1.1, these files were located in just `lib/`, thus we clean this
# directory as well.
if test -n "$(find $prefix/lib/gcc/*mingw*/*/libgcc*)"; then
rm $prefix/lib/gcc/*mingw*/*/libgcc* $prefix/lib/gcc/*mingw*/*/libmsvcrt*
elif test -n "$(find $prefix/lib/libgcc*)"; then
rm $prefix/lib/libgcc* $prefix/lib/libmsvcrt*
else
echo "Could not find any libraries to remove :-/"
find $prefix/lib
fi
fi

# For MacOS higher version of SDK and additional linker flag are required
# according to https://github.com/llvm/llvm-project/issues/119608 to link typeid(__int128)
if [[ "$target" == *-apple-darwin* ]]; then
export LDFLAGS="-lc++abi"
apple_sdk_root=$WORKSPACE/srcdir/MacOSX14.5.sdk
sed -i "s!/opt/$bb_target/$bb_target/sys-root!$apple_sdk_root!" $CMAKE_TARGET_TOOLCHAIN
sed -i "s!/opt/$bb_target/$bb_target/sys-root!$apple_sdk_root!" /opt/bin/$bb_full_target/$target-clang++
fi

mkdir build && cd build

cmake .. \
-DCMAKE_INSTALL_PREFIX=$prefix \
-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
-DCMAKE_BUILD_TYPE=Release \
-DJulia_PREFIX=$prefix

make -j${nproc}
make install
"""

# These are the platforms we will build for by default, unless further
# platforms are passed in on the command line
platforms = vcat(libjulia_platforms.(julia_versions)...)

# We cannot build with musl since OpenFHE requires the `execinfo.h` header for `backtrace`
platforms = filter(p -> libc(p) != "musl", platforms)

# PowerPC and 32-bit x86 and 64-bit FreeBSD on ARM 64 platforms are not supported by OpenFHE
platforms = filter(p -> arch(p) != "i686", platforms)
platforms = filter(p -> arch(p) != "powerpc64le", platforms)
platforms = filter(p -> !(Sys.isfreebsd(p) && arch(p) == "aarch64"), platforms)

if name == "openfhe_julia_int128"
# 32 bit systems does not support __int128
platforms = filter(p -> nbits(p) != 32, platforms)
# armv6l and armv7l do not support 128 bit int size
platforms = filter(p -> arch(p) != "armv6l", platforms)
platforms = filter(p -> arch(p) != "armv7l", platforms)
end

# Expand C++ string ABIs since we use std::string
platforms = expand_cxxstring_abis(platforms)


# The products that we will ensure are always built
products = [
LibraryProduct("libopenfhe_julia", :libopenfhe_julia),
]

# Dependencies that must be installed before this package can be built
dependencies = [
BuildDependency(PackageSpec(;name="libjulia_jll", version=v"1.10.9")),
Dependency(PackageSpec(name="libcxxwrap_julia_jll", uuid="3eaa8342-bff7-56a5-9981-c04077f7cee7"); compat="0.13.0"),
# For OpenMP we use libomp from `LLVMOpenMP_jll` where we use LLVM as compiler (BSD
# systems), and libgomp from `CompilerSupportLibraries_jll` everywhere else.
Dependency(PackageSpec(name="CompilerSupportLibraries_jll", uuid="e66e0078-7015-5450-92f7-15fbd957f2ae");
platforms=filter(!Sys.isbsd, platforms)),
Dependency(PackageSpec(name="LLVMOpenMP_jll", uuid="1d63c593-3942-5779-bab2-d838dc0a180e");
platforms=filter(Sys.isbsd, platforms)),
]

return sources, script, platforms, products, dependencies
end
11 changes: 11 additions & 0 deletions O/openfhe_julia_int128/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
The OpenFHE library works with both 64-bit and 128-bit integers.

This package builds upon the 128-bit version.

To use the 64-bit version, use
[openfhe_julia](https://github.com/JuliaPackaging/Yggdrasil/tree/master/O/openfhe_julia).

Most of the build steps for openfhe_julia_int128 and
[openfhe_julia](https://github.com/JuliaPackaging/Yggdrasil/tree/master/O/openfhe_julia) are the same,
so the shared parts are in the
[common.jl](https://github.com/JuliaPackaging/Yggdrasil/blob/master/O/openfhe_julia/common.jl) script.
18 changes: 18 additions & 0 deletions O/openfhe_julia_int128/build_tarballs.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Note that this script can accept some limited command-line arguments, run
# `julia build_tarballs.jl --help` to see a usage message.
include(joinpath(@__DIR__, "..", "openfhe_julia", "common.jl"))

# If you make changes in this file, e.g., to release a new version,
# be sure to also release a new version of `openfhe_julia` as well (see `../openfhe_julia/build_tarballs.jl`)
name = "openfhe_julia_int128"
version = v"0.3.6"

git_hash = "4939f1dd11ae2640b5bb397f731df8ef94307e74"

sources, script, platforms, products, dependencies = prepare_openfhe_julia_build(name, git_hash)

push!(dependencies, Dependency(PackageSpec(name="OpenFHE_int128_jll", uuid="a89a0bdd-1663-5679-8b21-c3a5388322bc"); compat="1.2.3"))

# Build the tarballs, and possibly a `build.jl` as well.
build_tarballs(ARGS, name, version, sources, script, platforms, products, dependencies;
julia_compat="1.6", preferred_gcc_version = v"9")