Mini conduit in Git

[dinosaure]

/cc @hannesm, /cc @ulugbekna

[dinosaure]

Ok, this PR reached a state where I'm able to provide a simple unikernel to fetch a Git repository without Irmin. The idea is to fill as long as CLI arguments the Mimic.ctx to be able to start a connection. Currently, such scenario works:

$ ./minigit --ssh-seed my-seed -r [email protected]:dinosaure/art
$ ./minigit -r https://github.com/dinosaure/art
$ ./minigit -r git://github.com/dinosaure/art

However, a call with an SSH endpoint without an SSH seed does not work. http://github.com/dinosaure/art does not work too due to a redirection to HTTPS. So, it highlights some issues:

1. be able to recognize the type of the connection (if it's SSH or not)
2. do the redirection to HTTPS when it's required by the peer

[dinosaure]

I think we reach a good state where I handle any cases, see the README.md.

• clean the PR
• add some tests & compile the unikernel (UNIX target)

[dinosaure]

At this stage, the PR is cleaned and tests are added to check the behavior of mimic. Some improvements come to my mind about mimic but I think it's enough for a proper release. Ready to merge, I hope that my PR still is orthogonal to #420

[dinosaure]

I locally merge this PR and #420 and all seems fine. Go to merge.

[ulugbekna]

It was an interesting read, and all looks good to me! Two things to note:

1. I'm not sure I understand what a new parameter verify in fetch is. It could use a slightly more descriptive name maybe and an explanation in the docs what verification it provides
2. There is no docs for mimic, so we should either at least refer to conduit's docs or write some better docs (I could take care of that some time later perhaps)

[dinosaure]

I'm not sure I understand what a new parameter verify in fetch is. It could use a slightly more descriptive name maybe and an explanation in the docs what verification it provides

This new argument is not very useful but in some contexts, it should be interesting to "verify" the status of the flow. And in that case, for example, verify that the Mimic.flow is an SSH connection when the Endpoint.t has the SSH scheme. It's like a user-defined post-verification of what Mimic.resolve returns.

Initially, this argument is used to prove that we use an SSH connection when we use git@domain:user/repository.git and such function can be only defined by the end-user (who knows about the SSH implementation). By this way, we are able to return an error when mimic is able to start an SSH connection even if we use git://domain/user/repository.git (so, when the user gives to us a Git endpoint and an SSH key).

However, such situation can be avoided by an other way. Instead to use ?verify to ensure the correspondence between Smart.Endpoint.t and Mimic.flow, we can enforce the initialization of the SSH connection with an other parameter/Mimic.value which exists only by an introspection of the Smart.Endpoint.t.

In this code:

ocaml-git/src/git-mirage/git_mirage_ssh.ml

Lines 132 to 135 in fff3f58

	let with_smart_git_endpoint edn ctx =
	match Smart_git.Endpoint.of_string edn with
	| Ok { Smart_git.Endpoint.scheme = `SSH user; _ } -> with_user user ctx
	| _ -> ctx

We can fill the Mimic.ctx with a value val is_ssh : bool Mimic.value and redefine these functions:

ocaml-git/src/git-mirage/git_mirage_ssh.ml

Lines 91 to 128 in fff3f58

	let with_resolv ctx =
	let k (stack, ipaddr, port) ssh_authenticator ssh_user ssh_key git_path
	git_capabilities =
	Lwt.return_some
	{
	stack;
	ipaddr;
	port;
	authenticator = ssh_authenticator;
	user = ssh_user;
	key = ssh_key;
	path = git_path;
	capabilities = git_capabilities;
	}
	in
	let ctx =
	Mimic.(
	fold ssh_endpoint
	Fun.
	[
	req TCP.tcp_endpoint; opt ssh_authenticator; req ssh_user;
	req ssh_key; req Git.git_path; dft Git.git_capabilities `Rd;
	]
	~k ctx)
	in
	let k stack ipaddr port = k (stack, ipaddr, port) in
	let ctx =
	Mimic.(
	fold ssh_endpoint
	Fun.
	[
	req TCP.tcp_stack; req TCP.tcp_ipaddr; dft TCP.tcp_port 22;
	opt ssh_authenticator; req ssh_user; req ssh_key;
	req Git.git_path; dft Git.git_capabilities `Rd;
	]
	~k ctx)
	in
	ctx

with a req is_ssh. I will propose a PR but I'm a bit lazy 👍.

There is no docs for mimic, so we should either at least refer to conduit's docs or write some better docs (I could take care of that some time later perhaps)

We should retake the Conduit's documentation indeed. I'm a bit frustrated to rewrite N times the same documentation. It's not mandatory. mimic is used only by ocaml-git (and paf) and git-mirage comes with enough to hide mimic under the forest. If you want to do that, I will be happy. Otherwise, it's fine.