Details

The git-sym-test examples explain this much better, but here it is in English.

How to use it

• Instead of a large file, create a symlink to .git-sym/unique_filename.

ln -sf .git-sym/unique_filename my_filename

• Add a rule to git-sym.makefile which can retrieve unique_filename.

unique_filename:
	wget http://mysite.com/unique_filename

• Then, git-sym update will automatically retrieve the file and fill in the symbolic links.

git-sym update

Adding your own large-files.

git-sym link --add large1 large2 large3
git commit -m 'adding links'
git-sym show

git-sym will choose unique filenames based on checksums. But git-sym link is strictly for convenience. You are free to use your own filenames. Anything symlinked via GIT_ROOT/.git-sym will be update-able.

Next, you might want to make those files available to other developers. You can move those files out of GIT_SYM_CACHE_DIR and into Amazon S3, or an ftp site, or wherever. Just add rules to your git-sym.makefile.

Other useful commands

git-sym show
git-sym missing
git-sym -h

Typical usage

You will store relative symlinks in your repo. They will point to a unique filename inside GIT_ROOT/.git-sym/, where GIT_ROOT is a relative path, e.g. ../../

git-sym update will search your repo for committed symlinks (unless you specify them on the command-line). For each, it will execute make -f GIT_ROOT/git-sym.makefile <target> in your GIT_SYM_CACHE_DIR (~/git-sym-cache by default). The makefile targets will be the basenames of the symlinked destinations.

If all those files are properly retrieved, then symlinks will be created with the same filenames inside .git/git-sym/links. GIT_ROOT/.git-sym will point at that. And all other symlinks will point thru GIT_ROOT/.git-sym. Thus, there are three (3) levels of indirection.

Makefile

Someday, we will offer a plugin architecture. But for now, using a GNU makefile is really very simple. Just create a rule for each unique filename. (You are using unique filenames, right?) You can run wget, curl, ftp, rcp, rsync, aws-s3-get, or whatever you want. The retrieval mechanism is decoupled from caching.

You should provide a rule for every current symlink. Old rules for symlinks no longer in your repo are fine; they are simply ignored.

To test your rules:

export GIT_SYM_CACHE_DIR=~/mytest
git-sym missing # should report something
git-sym update
git-sym missing

GIT_SYM_CACHE_DIR

The default cache directory is .git/git-sym-local/cache. To share a cache with other repos and users:

export GIT_SYM_CACHE_DIR=~/git-sym-cache

Other notes

Cache

git-sym sets the mode to read-only for the cached files. These files should never change. You might want to name them after their own checksums. git-sym link can help you with that.

Submodules

If your module can be used as a submodule, we cannot point at .git/git-sym/ directly because for submodules .git/ is not inside the tree. (The relative symlinks are constant, so they need to work no matter where .git/ sits.) That is why we have three levels of indirection, in case you were wondering. (This is also why git-annex fails for submodules.)

This is also why we write GIT_ROOT/.git-sym; it might be a different directory than .git.

For submodule support, you will also need this:

git config alias.gsexec '!exec '

We use that to learn the actual location of the .git/ directory. (Alternatives can fail in many ways. The alias never fails.)

We expect you to forget that, so we add that alias to your local repo for you. Believe us: It's a Good Thing.

git-sym-test

This module provides a way to test git-sym. It is not required. However, it is listed as a git-submodule here for testing submodule support conveniently.

git submodule update --init
make -C git-sym-test

.gitignore

Since the intermediate symlink is also in the repo, but points to a changing target, it needs to be listed in .gitignore. (That anticipates both accidental git link and deletion.) Again, we expect you to forget that important rule, so git-sym will detect its absence and add it to .git/info/exclude instead. No worries.

cat .git/info/exclude
## /.git_sym

If you accidentally add it anyway, no problem. We over-write it on each invocation, as necessary. If you accidentally delete it (e.g. git clean -xdf), you can run any git-sym command to re-create it.

Complicated symlinks?

We require a flat directory structure within .git/git-sym. If you need more files than your filesystem can handle, you're Doing It Wrong. Git will slow down anyway.

However, we support symlinked directories, which can then be an entire tree in GIT_SYM_CACHE_DIR. Your makefile rule can even download a tarball and extract it. That should satisfy all reasonable use-cases.