switch odoc to version 2.4.0

[EmileTrotignon]

This switches the odoc version used to version 2.4.0. This is the version of odoc that supports search.

The intend being this is to be a step towards using sherlodoc on ocaml. This will be an improvement over the current search because there is fuzzy type search like hoogle, and because it is tailored and tested for ocaml.

This will also allow updating the online version of sherlodoc, which needs as input the odocl file generated by this CI. The new version of sherlodoc is only compatible with odoc.2.4.0. As the format of the odocl file is not backward-compatible, the odocl files currently generated are not suitable to be consumed by the new sherlodoc, which has loads of new features.

I am not sure my change is correct, I have run the tests and they pass, but maybe more testing is required. Odoc itself should be backwards compatible.

[tmcgilchrist]

Thanks for the contribution.

The process for upgrading odoc requires coordination with voodoo to ensure the same version is used. See ocaml-doc/voodoo#134 for an example of what to change.

After that is in place, it can be deployed to staging pipeline (https://staging.docs.ci.ocamllabs.io) to validate the changes and check that package versions build correctly.

Then https://github.com/ocaml/ocaml.org needs modifications to ensure it can read the new voodoo/odoc artefacts. I am not familiar with the changes required there cc @sabine.

Finally the changes get deployed to https://docs.ci.ocaml.org live and once an epoch is generated then a coordinated deploy of ocaml.org is required.

I can help with getting things deployed to staging and validating the changes there.

[sabine]

As long as the odoc upgrade doesn't change the odoc html --as-json format, no changes to OCaml.org should be necessary.

But indeed, testing first locally on voodoo with odoc 2.4.0 and then on staging.docs.ci.ocamllabs.io will be great.

[EmileTrotignon]

I am in vacation for a while, so I will work on this when I come back.

[EmileTrotignon]

Hey @tmcgilchrist .
Guillaume has a PR on voodoo that works with odoc 2.4 ocaml-doc/voodoo#128
I have tested it on ocaml.org and it required a small patch because the .html.json have two extra fields, but apart from that it looks fine.

[sabine]

The odoc pin on the staging docs pipeline should be upgraded according to this PR.

Only after the voodoo staging branch builds package docs successfully on the staging pipeline, this PR should be merged into the main branch.

[mtelvers]

Isn't that just a case of pushing this PR to the staging branch?

[sabine]

@mtelvers Yes, but I think neither of @EmileTrotignon or I have rights to push on this repo.

[mtelvers]

@sabine I have pushed it to staging

[sabine]

@mtelvers thanks!

[mtelvers]

There seem to be lots of errors like this: https://staging.docs.ci.ocamllabs.io/job/2024-01-14/232445-voodoo-do-52799a

[sabine]

@EmileTrotignon @gpetiot are we pinning the right odoc version?

[tmcgilchrist]

What version of odoc-parser is this expecting? The obuilder spec in those logs is not pinning odoc-parser, it was not necessary previously. You can run the whole environment via docker compose if that helps debugging the problem.

…

On Mon, 15 Jan 2024 at 7:01 pm, sabine ***@***.***> wrote: @EmileTrotignon <https://github.com/EmileTrotignon> @gpetiot <https://github.com/gpetiot> are we pinning the right odoc version? — Reply to this email directly, view it on GitHub <#168 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABJXONKVS2T36SMN6FBLKLYOTO33AVCNFSM6AAAAABA3W34NOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJRGUYTKOJWGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[gpetiot]

odoc is expecting the same version of odoc-parser (the package is not released separately anymore).

I'm trying to reproduce locally but ocaml-docs-ci does not build locally (fresh install, switch, etc.), I get this error:

File "vendor/ocurrent/lib_web/query.ml", line 35, characters 87-91:
35 |         [input ~a:[a_input_type `Checkbox; a_name "id"; a_value job_id; a_autocomplete `Off] ()]
                                                                                            ^^^^
Error: This expression has type [> `Off ]
       but an expression was expected of type bool

(probably hiding something else)

[mtelvers]

A quick fix to that compiling issue is to replace the two occurrences of 'Off with false.

The actual fix is to upgrade TyXML to 4.6.0.

[mtelvers]

Can you edit ocaml-docs-ci.opam and update the pin-depends sections to this

pin-depends: [
  ["ocaml-version.dev" "git+https://github.com/ocurrent/ocaml-version#496b14cdf826a737841526b753ebad8c3656a85a"]
  ["current.dev" "./vendor/ocurrent"]
  ["current_git.dev" "./vendor/ocurrent"]
  ["current_github.dev" "./vendor/ocurrent"]
  ["current_web.dev" "./vendor/ocurrent"]
  ["current_ocluster.dev" "./vendor/ocluster"]
  ["ocluster-api.dev" "./vendor/ocluster"]
]

Then, create a new switch:

opam switch create . 4.14.1 --deps-only --with-test

[gpetiot]

Naive question but how is it supposed to be tested locally? Running docker-compose -f docker-compose.yml up runs the whole day without coming to an end or failing until the containers are stopped with Ctrl+C. And the online CI seems to only return errors 500.

[sabine]

As far as I understood, testing the docs-ci locally is nontrivial because of the keys involved, that's why the staging environment exists.

Is there anything I can do to help this progress? It would be great to get the staging branch of voodoo tested and deployed.

[gpetiot]

Let's wait for odoc.2.4.1 to be available on opam, as it's apparently necessary to have eio compile.

[sabine]

In that case, can we revert the staging branch to be identical to the main branch?

This would allow me to test and deploy the bugfix for wrong library names on ocaml.org that people are waiting for.

[mtelvers]

Done

[sabine]

Thanks, I'll update voodoo staging and it will probably be picked up tomorrow by staging pipeline

[sabine]

This can be closed since #169 removes the pinning of odoc by the pipeline.