Add introduction to opam release process

[mars0i]

This is kind of like #27, but I don' think people who are just starting out should worry about how to set up a package to go into opam.

What I have in mind is a guide that tells you how to release a package in simple cases, but notes likely potential gotchas, and has links to further info for more complex cases and for understaning the process under the hood.

This is something I'm trying to figure out how to do now. My impression is that Dune allows you to avoid many of the steps in the opam docs (in which case they should be left out of the initial presentation in the Ocamlverse guide), but I'm not sure, and in any event after reading relevant pages in the Dune docs, I have questions that neither set of docs answers.

They guide could be linked to from both Learning and Ecosystem.

[pmetzger]

I don't know what you mean by "to set up a package to go into opam" — if you mean how to get something to build with opam, you just create an opam file (which says a few things like what command builds the thing and what versions it's good for) and that's most of the work. The docs are reasonably clear (which is unusual in OCamlland.) Dune doesn't change anything here, you just end up invoking it instead of (say) make.

[mars0i]

I've never done it, and I am not finding the docs sufficiently clear. You'll have to take my word for it that I am reading the docs seriously and trying to figure out the process, and will ask questions when I understand enough to ask questions specific enough to get clear answers. Maybe it will seem simple to me after that.

I recognize that there's a difference between sketchy docs and docs that are very thorough and quite understandable if you've already got a sense of how something is supposed to work in general. The latter isn't always enough if you don't already have that prior understanding. This is a recurrent problem with OCaml docs, and it's a hole that Ocamlverse can and should fill.

[mars0i]

OK: I know what "build" means, and I know thaat I get packages installed, along with dependencies when I type opam install. And that allows me to build my projects that use those packages. I even know a bit about what's under .opam . But I don't know what you mean by "build with opam".

I have foo.opam files--jbuilder requires them--but their content doesn't matter for anything I've done in the past, afaics.

I also don't know how packages get into the database, and I am trying to learn how to put a package into the database. That's somethimg others will want to understand, and it's the point of the suggested guide.

(That's not request for help with opam in an ocamlverse issue, of course. Just an illustration of how this stuff can be confusing if you don't already kind of understand it.)

[pmetzger]

Okay, so this isn't really that hard. Have a look at the pull request queue for github.com/ocaml/opam-repository and at the contents of that git repo, and poke around inside your .opam directory, and the rest should become clear. Maybe some of this deserves documenting of course.

[bobbypriam]

On publishing to opam, I'm thinking that the Packaging guide (and the one for opam2) is quite detailed. It involves creating an opam file (the guide mentioned which fields are important for publishing), test to install it locally with opam install ., and publishing (create a PR) automatically via the opam-publish tool.

Topkg (topkg-care) will also help with that (and more), with the addition of automatically testing your lib, creating release notes, tagging, creating and publishing your artifacts and docs, substituting version strings, and creating the PR.

These steps (which are intended to be a common flow of publishing packages) are documented in the man page, so to find it you must opam install topkg-care and run topkg help release. Daniel has written a lengthy explanation on each steps of the flow, but unfortunately it's not really straightforward to get to the manpage. To use topkg with dune, there's the topkg-jbuilder package.

[mars0i]

Thanks @pmetzger , @bobbypriambodo . I had read the opam 1.2 packaging doc--that's what part of what was confusing, and I was also reading parts of the Dune docs to see if they could help.

What's confusing in the opam 1.2 packaging doc is that as the example of what goes in the install clause in foo.opam, it just has "make install". Well, if I want to put an executable in /usr/local/bin and some files in /usr/local/lib, I know what to put in my Makefile to cause that to happen. But if what I want to do is to make it possible for users to install my package locally, i.e. install a bunch of files under a switch directory under .opam, I have no idea what should go in the Makefile. (I could look to see what other packages put under the $HOME/.opam directory, but my guess is that my Makefile is not supposed to copy files there explicitly--there are tools that handle that, right? I would probably get it wrong; there's a lot of stuff that's supposed to go under .opam.)

A similar point applies to what the opam 1.2 packaging doc says about the remove clause in the foo.opam file. The example given in the opam 1.2 packaging doc is "ocamlfind" "remove" "project", but I have no idea how general that is.

However, I looked at the opam 2.0 packaging doc last night, and it's much clearer. It doesn't answer the questions I had about the install and remove clauses, but (a) it says that a remove clause is not needed with opam 2.0, and (b) it says that if you use Dune, you can leave out the install clause, and it tells you what should go in the foo.opam build clause to make that work.

The upshot: The opam 1.2 packaging doc does not explain things fully for a user like me. The opam 2.0 packaging doc does explain things fully for a user like me, I think. I haven't tried it yet. I am using opam 2.0 anyway, so for my own needs, I shouldn't have been reading the opam 1.2 packaging doc. (I do think it's reasonable to expect users to be using Dune for new projects in most cases.)

So maybe there is not a need for additional documentation in the long run (since opam 2.0 will presumably be officially released soon, I assume).

---

About topkg, my understanding is that that was for multiple-package releases. If that's right, maybe it's not something that needs to be on Ocamlverse for new users, although it might be useful to have a brief description and links to other docs. (If one didn't have to install it first, that would be good.)

---

I'll look at the PR queue for opam. Thanks for that suggestion, @pmetzger. I would not have that would be a good source for me info at this stage, but given your suggestion, I expect it will be.

---

OK to skip this part:

btw what the opam 2.0 packaging doc says should go in the build clause with Dune is a little bit different from what the Dune docs suggest, though, and I am not sure whether the difference matters. So that's something I don't understand.

opam 2.0 packaging doc:

build: ["dune" "build" "-p" name]

Dune doc

build: [["jbuilder" "build" "-p" name "-j" jobs]]

The Dune docs describe this as "invocation from opam", and I don't know what that means, actually, so maybe the `"-j" jobs" part is irrelevant to what I'm trying to do (see previous post).

(Also, in the Dune docs, it took me a while to realize that "name" and "jobs" are actually supposed to be typed in literally. I was supposed to replace thought "name" with some other string that referred to something in my project.)