-
Notifications
You must be signed in to change notification settings - Fork 185
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Single release CMake file that contains all required links/hashes. #4631
Merged
Merged
Changes from all commits
Commits
Show all changes
61 commits
Select commit
Hold shift + click to select a range
8c02c87
Modify release.yml workflow
dudoslav 94d0194
Don't build using superbuild
dudoslav 442f9f1
Fix typo in binary package step
dudoslav 4691c69
Change source package file name
dudoslav 6cf7c9c
Fix artifact upload
dudoslav efe3772
Fix incorrect names for source and binary archive
dudoslav e1cb7ca
Use CPack in CI instead of CMake
dudoslav 0884182
Try packaging in Superbuild
dudoslav ded384c
Build before packaging
dudoslav c276d08
Divide source and binary releases
dudoslav c445fb4
Fix Source builds
dudoslav 307fea1
Fix typo
dudoslav 26b6359
Properly name MacOS releases
dudoslav a9ac1f9
Also use proper macos variables
dudoslav 918fc56
Fix check of empty string
dudoslav 0845c5c
Remove redundant variable expansion
dudoslav d566647
Ignore files and also upload hash files
dudoslav c9ecc9d
Add testing append-release-cmake.yml
dudoslav 7eb7606
Download release files from upstream
dudoslav 6a7cff3
Add asset-output field
dudoslav eb2627e
Fix typo
dudoslav a79ee6c
Basic template
dudoslav 15e3bf8
Fix release assets output
dudoslav ec984b8
Invalid asset_output
dudoslav 42bf8ec
Change asset-outpu
dudoslav e5a5d24
Add make release directory
dudoslav 482349e
Also upload template
dudoslav efdc830
Fix missing FindTileDB_EP.cmake asset
dudoslav 2f8adf3
Debug
dudoslav dab1305
Fix typo
dudoslav 048505e
Add another debug
dudoslav 6dbd276
Add quotes
dudoslav 95e45fa
Change output directory for script
dudoslav cd7f4e1
Change sed command
dudoslav 0aea033
Also generate source hash/url
dudoslav 1b05d49
Fix source hash
dudoslav f3d9dda
Remove find tiledb file and add tiledb package
dudoslav 879e7e5
Fix typo
dudoslav bb29940
Produce release list
dudoslav 77399ae
Change noavx column
dudoslav e43dfae
Modify CMake download tiledb logic
dudoslav 6d30b6b
Also generate Module file
dudoslav b33324b
WIP: get_tiledb_source_url_and_hash
dudoslav 705260d
Rebase and fix for CPack logic
dudoslav e8ac4cc
Ignore HASH files
dudoslav 8bcf35e
Expand source file name
dudoslav 448d0a8
Fix filename expansion
dudoslav 04ed61f
Fix source file name path
dudoslav 1fbcb02
Match release files and not source files
dudoslav 43a877d
Fix matching version
dudoslav 881d79c
Fix SHA256 HASH reading
dudoslav 1160c31
Merge remote-tracking branch 'origin/dev' into db/sc-17775/release_cmake
dudoslav 119107a
Merge remote-tracking branch 'origin/dev' into db/sc-17775/release_cmake
dudoslav 1209aa0
Rework rellist format
dudoslav 8c38ab0
Automatically call generate release-list in release
dudoslav e36fe6a
Fix workflow call
dudoslav d710704
Fix source file export
dudoslav 2253c38
Add debugging urls
dudoslav 6d12f0f
Fix artifact assets download url
dudoslav 6d01260
Use fetch content in DownloadPrebuiltTileDB
dudoslav c224bda
Add ref_name to generate release list
dudoslav File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,80 @@ | ||
name: Append Release CMake | ||
|
||
on: | ||
workflow_dispatch: | ||
inputs: | ||
ref: | ||
description: 'Ref to be used as release' | ||
default: 'latest' | ||
required: true | ||
type: string | ||
workflow_call: | ||
inputs: | ||
ref: | ||
description: 'Ref to be used as release' | ||
default: 'latest' | ||
required: true | ||
type: string | ||
|
||
jobs: | ||
generate_cmake_files: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout TileDB | ||
uses: actions/checkout@v3 | ||
|
||
- name: Make release and output directories | ||
run: | | ||
mkdir release output | ||
|
||
- name: Github release data | ||
id: release_data | ||
uses: KevinRohn/[email protected] | ||
with: | ||
# repository: 'TileDB-Inc/TileDB' | ||
version: ${{ inputs.ref }} | ||
asset-file: '*.zip,*.tar.gz' | ||
asset-output: './release/' | ||
|
||
- name: Render template | ||
run: | | ||
PATTERN="tiledb-([^-]+)-([^-]+)(-noavx2)?-([^-]+).(tar.gz|zip)$" | ||
RELLIST="output/releases.csv" | ||
MODULE="output/DownloadPrebuiltTileDB.cmake" | ||
cp cmake/inputs/DownloadPrebuiltTileDB.cmake $MODULE | ||
echo "platform,url,sha256" > $RELLIST | ||
|
||
for FILE in $(ls release) | ||
do | ||
if [[ $FILE =~ $PATTERN ]] | ||
then | ||
OS=${BASH_REMATCH[1]^^} | ||
ARCH=${BASH_REMATCH[2]^^} | ||
NOAVX2=${BASH_REMATCH[3]^^} | ||
PLATFORM=${OS}-${ARCH}${NOAVX2} | ||
|
||
URL="${{ github.server_url }}/${{ github.repository }}/releases/download/${{ inputs.ref }}/$FILE" | ||
HASH=$(cat release/$FILE.sha256 | cut -d \t -f 1) | ||
|
||
echo "${PLATFORM},${URL},${HASH}" >> $RELLIST | ||
fi | ||
done | ||
|
||
SOURCE_FILE_NAME=$(ls release/tiledb-source-*.tar.gz) | ||
URL_TILEDB_SOURCE="${{ github.server_url }}/${{ github.repository }}/releases/download/${{ inputs.ref }}/$(basename $SOURCE_FILE_NAME)" | ||
HASH_TILEDB_SOURCE=$(cat $SOURCE_FILE_NAME.sha256 | cut -d \t -f 1) | ||
|
||
echo "source,${URL_TILEDB_SOURCE},${HASH_TILEDB_SOURCE}" >> $RELLIST | ||
|
||
HASH=$(sha256sum $RELLIST | cut -d " " -f 1) | ||
echo $HASH > $RELLIST.sha256 | ||
|
||
cat $RELLIST | ||
|
||
- name: Upload template to release | ||
uses: svenstaro/upload-release-action@v2 | ||
with: | ||
file: output/* | ||
tag: ${{ steps.release_data.outputs.tag_name }} | ||
overwrite: true | ||
file_glob: true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,128 @@ | ||
# | ||
# FindTileDB_EP.cmake | ||
# | ||
# | ||
# The MIT License | ||
# | ||
# Copyright (c) 2023 TileDB, Inc. | ||
# | ||
# Permission is hereby granted, free of charge, to any person obtaining a copy | ||
# of this software and associated documentation files (the "Software"), to deal | ||
# in the Software without restriction, including without limitation the rights | ||
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
# copies of the Software, and to permit persons to whom the Software is | ||
# furnished to do so, subject to the following conditions: | ||
# | ||
# The above copyright notice and this permission notice shall be included in | ||
# all copies or substantial portions of the Software. | ||
# | ||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN | ||
# THE SOFTWARE. | ||
|
||
include(FetchContent) | ||
|
||
function(fetch_tiledb_release_list VERSION EXPECTED_HASH) | ||
# Local constants | ||
set(UPSTREAM_URL "https://github.com/TileDB-Inc/TileDB/releases/download") | ||
|
||
if(NOT VERSION) | ||
set(VERSION latest) | ||
endif() | ||
|
||
if(${EXPECTED_HASH}) | ||
file(DOWNLOAD | ||
${UPSTREAM_URL}/${VERSION}/releases.csv | ||
releases.csv | ||
SHOW_PROGRESS | ||
EXPECTED_HASH ${EXPECTED_HASH} | ||
) | ||
else() | ||
message(WARNING "Downloading release list without SHA checksum!") | ||
file(DOWNLOAD | ||
${UPSTREAM_URL}/${VERSION}/releases.csv | ||
releases.csv | ||
SHOW_PROGRESS | ||
) | ||
endif() | ||
|
||
file(STRINGS | ||
${CMAKE_CURRENT_BINARY_DIR}/releases.csv | ||
RELLIST | ||
) | ||
|
||
# Remove csv table headers | ||
list(POP_FRONT RELLIST) | ||
|
||
foreach(LINE ${RELLIST}) | ||
string(REPLACE "," ";" LINE ${LINE}) | ||
list(LENGTH LINE LENGTH) | ||
|
||
list(GET LINE 0 PLATFORM) | ||
list(GET LINE 1 URL) | ||
list(GET LINE 2 SHA) | ||
|
||
set(RELEASE_VAR TILEDB_${PLATFORM}) | ||
set(URL_${RELEASE_VAR} ${URL} PARENT_SCOPE) | ||
set(HASH_${RELEASE_VAR} ${SHA} PARENT_SCOPE) | ||
endforeach() | ||
endfunction() | ||
|
||
function(detect_artifact_name OUT_VAR) | ||
if (WIN32) # Windows | ||
SET(${OUT_VAR} TILEDB_WINDOWS-X86_64 PARENT_SCOPE) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should fail on other architectures. Same in Linux. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think that CMAKE can only define these three variables: WIN32/APPLE/LINUX |
||
elseif(APPLE) # OSX | ||
if (DEFINED CMAKE_OSX_ARCHITECTURES) | ||
set(ACTUAL_TARGET ${CMAKE_OSX_ARCHITECTURES}) | ||
else() | ||
set(ACTUAL_TARGET ${CMAKE_SYSTEM_PROCESSOR}) | ||
endif() | ||
|
||
|
||
if (ACTUAL_TARGET MATCHES "(x86_64)|(AMD64|amd64)|(^i.86$)") | ||
SET(${OUT_VAR} TILEDB_MACOS-X86_64 PARENT_SCOPE) | ||
elseif (ACTUAL_TARGET STREQUAL arm64 OR ACTUAL_TARGET MATCHES "^aarch64" OR CMAKE_SYSTEM_PROCESSOR MATCHES "^arm") | ||
SET(${OUT_VAR} TILEDB_MACOS-ARM64 PARENT_SCOPE) | ||
endif() | ||
else() # Linux | ||
SET(${OUT_VAR} TILEDB_LINUX-X86_64 PARENT_SCOPE) | ||
endif() | ||
endfunction() | ||
|
||
function(fetch_prebuilt_tiledb) | ||
# Arguments | ||
set(options RELLIST_HASH) | ||
set(oneValueArgs VERSION ARTIFACT_NAME) | ||
set(multiValueArgs) | ||
cmake_parse_arguments( | ||
FETCH_PREBUILT_TILEDB | ||
"${options}" | ||
"${oneValueArgs}" | ||
"${multiValueArgs}" | ||
${ARGN} | ||
) | ||
|
||
fetch_tiledb_release_list(${FETCH_PREBUILT_TILEDB_VERSION} ${FETCH_PREBUILT_TILEDB_RELLIST_HASH}) | ||
|
||
if(NOT FETCH_PREBUILT_TILEDB_ARTIFACT_NAME) | ||
detect_artifact_name(FETCH_PREBUILT_TILEDB_ARTIFACT_NAME) | ||
endif() | ||
|
||
string(STRIP ${HASH_${FETCH_PREBUILT_TILEDB_ARTIFACT_NAME}} HASH_${FETCH_PREBUILT_TILEDB_ARTIFACT_NAME}) | ||
FetchContent_Declare( | ||
tiledb-prebuilt | ||
URL ${URL_${FETCH_PREBUILT_TILEDB_ARTIFACT_NAME}} | ||
URL_HASH SHA256=${HASH_${FETCH_PREBUILT_TILEDB_ARTIFACT_NAME}} | ||
DOWNLOAD_EXTRACT_TIMESTAMP FALSE | ||
) | ||
|
||
FetchContent_MakeAvailable( | ||
tiledb-prebuilt | ||
) | ||
|
||
set(TileDB_DIR "${tiledb-prebuilt_SOURCE_DIR}/lib/cmake/TileDB" PARENT_SCOPE) | ||
endfunction() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe this should be part of the Release workflow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dudoslav will you do this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added to release workflow as a reusable workflow