Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto File Management Part 3: Install data-management tool as command line program #246

Open
wants to merge 37 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
1e00790
initial attempt at writing grdata.py
mabruzzo Aug 27, 2024
8106e6f
Significant refactoring of grdata.py
mabruzzo Aug 27, 2024
468e087
some remaining changes to the grdata.py script.
mabruzzo Sep 1, 2024
fb82d31
I completely forgot to commit __main__.py
mabruzzo Sep 17, 2024
e48524f
replace all usage of pooch with urllib
mabruzzo Sep 19, 2024
9ed61ef
Added some basic tests of the grdata command-line tool
mabruzzo Sep 22, 2024
39b8b16
reorganized where we declare the arguments for each subcommand.
mabruzzo Sep 22, 2024
af04e97
made it possible to download individual files and to an untracked dir…
mabruzzo Oct 3, 2024
4c4d926
tweaked grdata.py to work with configure_file.py
mabruzzo Oct 10, 2024
994acac
Merge branch 'main' into auto-data
mabruzzo Oct 10, 2024
5d8a11b
datafile-readers take path as a separate arg
mabruzzo Aug 25, 2024
657bee1
introduce the grackle_data_file_options parameter.
mabruzzo Aug 25, 2024
165e062
introduce a barebones version of determine_data_file_
mabruzzo Aug 25, 2024
0456441
introduce os_utils.[ch]
mabruzzo Aug 30, 2024
f80226c
add file-search logic to determine_data_file_
mabruzzo Aug 26, 2024
5abaeea
add build logic to classic build-system for the autofile management.
mabruzzo Sep 1, 2024
e26cf08
automatically inject file registry into C library
mabruzzo Sep 2, 2024
2551dac
finalize grackle_data_file_options.
mabruzzo Sep 2, 2024
4cf4766
introduce tests.
mabruzzo Sep 1, 2024
808a648
use automated file lookup in pygrackle examples
mabruzzo Sep 4, 2024
c8f96a2
first stab at describing new parameter for using automatic file search
mabruzzo Sep 4, 2024
f17f098
start importing some things from test_grdata.py
mabruzzo Sep 22, 2024
73f6f7d
shift the order of autogenerated files.
mabruzzo Oct 10, 2024
872f4fc
modifications to file_registry.txt trigger rebuilds for the CMake bui…
mabruzzo Oct 10, 2024
3c8f685
prep grdata.py to be used as a template for a standalone executable
mabruzzo Oct 10, 2024
97fa2b0
cmake support for installing grdata
mabruzzo Oct 10, 2024
203118e
alter installation of grdata tool and add docs
mabruzzo Oct 9, 2024
d514cab
tweak configure_file.py so that it can be applied to the grdata.py sc…
mabruzzo Oct 10, 2024
491b4e9
laid the ground work for installing the grdata tool with classic buil…
mabruzzo Oct 10, 2024
1723f43
classic build system support for grdata
mabruzzo Oct 10, 2024
2c10172
fix some typos
mabruzzo Oct 10, 2024
1816ff5
more-typo fixes
mabruzzo Oct 10, 2024
db32df6
moved the examples section.
mabruzzo Oct 11, 2024
0e58faf
a bunch of progress.
mabruzzo Oct 11, 2024
ff9d4ac
updating information about where to find the grdata tool.
mabruzzo Oct 12, 2024
a190a08
prompt users to use grdata when running examples.
mabruzzo Oct 12, 2024
c9b5cd1
Modified cxx_example.C to make use of managed data files.
mabruzzo Oct 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 46 additions & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -130,9 +130,7 @@ the structure of this directory
# these intentionally are not CACHE variables
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${stageDir}/${CMAKE_INSTALL_LIBDIR})
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${stageDir}/${CMAKE_INSTALL_LIBDIR})

# don't currently need the following since grackle doesn't ship an executable
#set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${stageDir}/${CMAKE_INSTALL_BINDIR})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${stageDir}/${CMAKE_INSTALL_BINDIR})

# the location where the export files go to export from build-tree
set(GRACKLE_BUILD_EXPORT_PREFIX_PATH ${CMAKE_LIBRARY_OUTPUT_DIRECTORY})
Expand Down Expand Up @@ -171,10 +169,55 @@ if (UNIX AND NOT CMAKE_SYSTEM_NAME STREQUAL "Darwin")
set_target_properties(toolchain::m PROPERTIES IMPORTED_LIBNAME "m")
endif()


# picohash is a vendored, self-contained, header-only library
# -> it's a internal dependency of Grackle::Grackle (it shouldn't be exposed)
# -> If it were an `INTERFACE` library, CMake would abort with an error when we
# declare rules for installing the "export information" for a static-lib
# build of Grackle::Grackle (there aren't problems for a shared-lib build)
# -> The "export information" includes autogenerated linking logic that gets
# evaluated by external CMake projects that consume Grackle::Grackle via
# `find_package`. This logic infers a list of any other libraries that
# are shipped by this project that need to be linked to use libgrackle.a
# -> When invoked, the logic goes through ALL of Grackle::Grackle's (public
# & private) depenedencies that COULD specify such linking requirements.
# -> CMake will complain if any INTERFACE library used by Grackle::Grackle
# isn't publicaly exported since it COULD specify this information.
# -> the BUILD_LOCAL_INTERFACE generator expression, (in CMake 3.26+) can
# work around this
# -> this isn't an issue for INTERFACE IMPORTED libraries since IMPORTED
# libs should only specify linker requirements of prebuilt external libs,
# if there are any. (i.e. if there are any reqs, CMake expects the developer
# to manually add logic to the installed "export info")
add_library(picohash INTERFACE IMPORTED)

# we use the SYSTEM option to suppress any warnings
target_include_directories(picohash SYSTEM INTERFACE
${CMAKE_CURRENT_SOURCE_DIR}/external
)

# Main build targets
# ------------------
add_subdirectory(src/clib)

# configure the grdata cli program
# -> we are essentially performing some template substitutions on a python
# file so that the file can be executed as a standalone cli program
include(CreateProgram-grdata)
create_grdata_program(
GRACKLE_VERSION "${_GRACKLE_FULL_VERSION}"
TARGET_NAME Grackle::grdata
)

file(GENERATE
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/grackle-buildpaths-$<CONFIG>.txt
CONTENT [[
# This file lists paths where useful build products can be found in the build
# directory (if/when the products are actually built)
$<TARGET_FILE:Grackle::grdata>
]])


# declare build-recipies for examples
if (GRACKLE_EXAMPLES)
add_subdirectory(src/example)
Expand Down
Loading