Download bsp files at build time and remove from git lfs

[patkenneally]

Describe your use case
While useful, the git-lfs feature on github counts to the repo's cloning bandwidth. When the CI processes of the core repo and of forks of the repo, clone the repo, the bandwidth used by transferring the large bsp files adds up quickly. Instead, how about we remove these files from git-lfs, and instead download them once at build time using the CMake file(DOWNLOAD) feature. This would allow them to be downloaded once, and they would no longer impact github's git-lfs bandwidth quotas. I believe we should avoid using git-lfs (having been the person who motivated its use in the first place) due to the bandwidth considerations.

A secondary, and much less of a priority, this would absolve users of needing to configure and understand the basics of git-lfs, which has proven tricky on the odd occasion.

A negative to this proposition is that we have no control over what/where NAIF choose to put the bsp's overtime. An alternative might be to find some place (other than rely on NAIF's download links) to host the files for this project which doesn't have cost/bandwidth impacts.

Describe alternatives solutions you've considered
None

Additional context
None

[schaubh]

Howdy @patkenneally , thanks for posting this issue and thoughts on this. Some quick thoughts and questions on my side:

• As the CMAKE would be pulling the large Spice files, this is only done once I assume. When a clean build is requested we delete the dist3 folder, but the spice file would remain I assume and would not need to be downloaded again? I'm 99% sure the answer is yes reading up on this, but wanted to check
• As we keep improving the build system of BSK, a goal of mine is to readily be able to build BSK when not connected to the internet. I'm assuming all dependencies are already downloaded and built. I would like to be able to do clean test builds while on a plane. For this requested feature, if there is a way to make this work where it doesn't need internet if the file already exists, that would be a huge plus to me at least. :-) Right now I can't do a clean build if there is no internet connection.
• I agree with your statement that this would improve the BSK install, configure and build process by not having the new user also having to learn to setup git-lfs. I'm excited to see this change implemented.
• Regarding the web url for the Spice files, we could put them on a private server. My first choice, however, would be to use the official server. With the many BSK dependencies we already have to do some occasional maintenance to keep the builds working with the latest conan repos, python package version, etc. If the link doesn't work, it would be nice for the CMAKE to error with a verbose message telling the user what happened, and if needed they can download the required file from the Spice web page and drop it into the BSK folder.

[GorgiAstro]

Hi folks,
An alternative could be managing this Spice data as a Python library in a separate repository, upload it to PyPi, and make it a pip dependency of basilisk, say for instance bskdata.

This Python dependency just needs to be added to requirements.txt.

When this Spice data needs to be updated, the package version just needs to be bumped too (this can be done manually or automatically on new commits in the main branch), so that users will receive the updated version just by typing pip install bskdata.

This is what we've done in the Orekit project to allow easier access to physical data (e.g. EOP, leap seconds) for Python users: we just added a pyproject.toml to the Orekit data repository (but didn't publish it on PyPi yet): https://gitlab.orekit.org/orekit/orekit-data

[schaubh]

Interesting suggestion @GorgiAstro , let me ponder on that as we look into creating BSK pip wheels in another branch.