Make the BrainVISA 5.1.1 release

[ylep]

Release check-list

• In this new issue, replace templated versions

  • Replace tag release version x.y and x.y.z with the new release version (as an example: 5.1 / 5.1.1)
  • Replace base image version x.i with the new base image version (as an example: 5.3)

• Check that all projects have the correct version in their sources: bv_tag_release.py --fix-source-version-numbers

• Check that tests are successful on https://brainvisa.info/builds/

• Update changelogs and the list of known issues (Known issues for BrainVISA 5.1.0 #124)

  • https://github.com/brainvisa/aims-free/blob/5.1/aimsdata/doc/html/en/changelog.html
  • https://github.com/brainvisa/axon/blob/5.1/doc/en/help/changelog.html
  • https://github.com/brainvisa/anatomist-free/blob/5.1/doc/html/en/changelog.html
  • https://bioproj.extra.cea.fr/redmine/projects/brainvisa-commu/repository/web/revisions/master/entry/sphinx/changelog.rst

• Run a last build to update the changelogs

  • bv_maker sources doc in the brainvisa distro
  • bv_maker sources doc in the cea distro

• Create, test, and publish/deploy the images

  • sif image for the brainvisa distro

    • Create the image with ./brainvisa-5.1-5.3/bin/casa_distro_admin create_user_image version=5.1.1 container_type=singularity distro=brainvisa image_version=5.3 branch=5.1 install_thirdparty=file:///neurospin/brainvisa/thirdparty/thirdparty.json
    • Verify that the image works
      • Install the image
        mkdir -p /tmp/test-brainvisa-5.1.1;singularity run -ce --bind /tmp/test-brainvisa-5.1.1:/casa/setup /volatile/a-sac-ns-brainvisa/bbi_nightly/brainvisa-5.1.1.sif
      • Use it to run AimsFileInfo, anatomist, brainvisa
    • Publish the image on the BrainVISA web site
      casa_distro_admin publish_user_image image=brainvisa-5.1.1.sif

  • ova image for the brainvisa distro

    • Create the image with ./brainvisa-5.1-5.3/bin/casa_distro_admin create_user_image version=5.1.1 install=no container_type=vbox base_image=casa-run-5.3.ova environment_name=brainvisa-5.1-5.3
    • Verify that the image works
      • Install the image
      • Use it to run AimsFileInfo, anatomist, brainvisa
    • Publish the image on the BrainVISA web site

  • sif image for the cea distro

    • Create the image with ./cea-5.1-5.3/bin/casa_distro_admin create_user_image version=5.1.1 name=brainvisa-cea-5.1.1 container_type=singularity distro=cea image_version=5.3 branch=5.1 install_thirdparty=file:///neurospin/brainvisa/thirdparty/thirdparty.json (beware to name the image brainvisa-cea- and not just cea-)

  • ova image for the cea distro we will only do these images if users request them

    • Create the image with ./cea-5.1-5.3/bin/casa_distro_admin create_user_image name=brainvisa-cea version=5.1.1 base_directory=/volatile/a-sac-ns-brainvisa/bbi_nightly install=no container_type=vbox base_image=casa-run-5.3.ova environment_name=cea-5.1-5.3 (beware to name the image brainvisa-cea- and not just cea-)

• Create tags with bv_tag_release.py

• Edit the website to announce the new release

  • web project sources
  • log on the web server, rebuild the web site in the casa-distro installed there: ssh web, then in the server:
    • web_build/bin/bv bv_maker
    • publish the web site using the publish script: ./web-build/src/communication/web/5.1/scripts/bv_publish_web /var/www/html/brainvisa.info

• Deploy the cea release:

  • Copy brainvisa-cea-*.sif and the associated .json into /i2bm/brainvisa and set their permissions (chmod 444)
  • Create a new directory /i2bm/brainvisa/brainvisa-cea-5.1.1 and install the Singularity release in there (singularity run --bind ...)
  • Remove the home directory to enable per-user home: rm -r /i2bm/brainvisa/brainvisa-cea-5.1.1/home/
  • Verify that the deployment works (use it to launch AimsFileInfo --info, anatomist, brainvisa...)
  • Make it the default version: ln -sfT brainvisa-cea-5.1.1/ /i2bm/brainvisa/brainvisa
  • Copy brainvisa-cea-*.ova and the associated .json into /i2bm/brainvisa and set their permissions (chmod 444)
  • Install the ova on a non-Linux machine and verify that it works (use it to launch AimsFileInfo --info, anatomist, brainvisa...)

• Build pip packages for the python projects soma-workflow, soma-base, populse_db and capsul which are distributed in pip:

  • ./brainvisa-5.1-5.3/bin/bv python3 -m build /casa/host/src/soma/soma-workflow/5.1
  • ./brainvisa-5.1-5.3/bin/bv python3 -m build /casa/host/src/soma/soma-base/5.1
  • ./brainvisa-5.1-5.3/bin/bv python3 -m build /casa/host/src/populse/populse_db/5.1
  • ./brainvisa-5.1-5.3/bin/bv python3 -m build /casa/host/src/capsul/5.1

• Publish them to test.pypi.org

  • ./brainvisa-5.1-5.3/bin/bv python3 -m twine upload --repository testpypi /casa/host/src/soma/soma-workflow/5.1/dist/*
  • ./brainvisa-5.1-5.3/bin/bv python3 -m twine upload --repository testpypi /casa/host/src/soma/soma-base/5.1/dist/*
  • ./brainvisa-5.1-5.3/bin/bv python3 -m twine upload --repository testpypi /casa/host/src/populse/populse_db/5.1/dist/*
  • ./brainvisa-5.1-5.3/bin/bv python3 -m twine upload --repository testpypi /casa/host/src/capsul/5.1/dist/*

• Create a virtualenv test environment and test packages install:

  • mkdir -p /tmp/brainvisa-5.1.1/testenv && python3 -m venv /tmp/brainvisa-5.1.1/testenv
  • /tmp/brainvisa-5.1.1/testenv/bin/python3 -m pip install --index-url https://test.pypi.org/simple/ soma-workflow
  • /tmp/brainvisa-5.1.1/testenv/bin/python3 -m pip install --index-url https://test.pypi.org/simple/ soma-base
  • /tmp/brainvisa-5.1.1/testenv/bin/python3 -m pip install --index-url https://test.pypi.org/simple/ populse_db
  • /tmp/brainvisa-5.1.1/testenv/bin/python3 -m pip install --index-url https://test.pypi.org/simple/ capsul

• Publish them to pypi.org

  • ./brainvisa-5.1-5.3/bin/bv python3 -m twine upload /casa/host/src/soma/soma-workflow/5.1/dist/*
  • ./brainvisa-5.1-5.3/bin/bv python3 -m twine upload /casa/host/src/soma/soma-base/5.1/dist/*
  • ./brainvisa-5.1-5.3/bin/bv python3 -m twine upload /casa/host/src/populse/populse_db/5.1/dist/*
  • ./brainvisa-5.1-5.3/bin/bv python3 -m twine upload /casa/host/src/capsul/5.1/dist/*

• Open a new GitHub issue for known issues

• Close the old GitHub known issues

• Make an announcement on the GitHub discussions

• If the releases has significant new features, announce it on the mailing list

[ylep]

I am getting this error when creating the brainvisa-5.1.1.ova VirtualBox image:

VBoxManage: error: File copy failed
VBoxManage: error: Destination "/home/brainvisa/Desktop" on the guest already exists and is a directory
VBoxManage: error: Details: code VBOX_E_IPRT_ERROR (0x80bb0005), component GuestSessionWrap, interface IGuestSession
VBoxManage: error: Context: "RTEXITCODE gctlHandleCopy(PGCTLCMDCTX, int, char**, bool)" at line 1898 of file VBoxManageGuestCtrl.cpp
VBoxManage: error: Destination "/home/brainvisa/Desktop" on the guest already exists and is a directory.
Traceback (most recent call last):
  File "/volatile/a-sac-ns-brainvisa/bbi-brainvisa-5.0/brainvisa-5.1-5.3/src/development/casa-distro/5.1/bin/casa_distro_admin", line 25, in <module>
    main()
  File "/volatile/a-sac-ns-brainvisa/bbi-brainvisa-5.0/brainvisa-5.1-5.3/src/development/casa-distro/5.1/python/casa_distro/command.py", line 360, in main
    result = command(*args, **kwargs)
  File "/volatile/a-sac-ns-brainvisa/bbi-brainvisa-5.0/brainvisa-5.1-5.3/src/development/casa-distro/5.1/python/casa_distro/admin_commands.py", line 994, in create_user_image
    **kwargs)
  File "/volatile/a-sac-ns-brainvisa/bbi-brainvisa-5.0/brainvisa-5.1-5.3/src/development/casa-distro/5.1/python/casa_distro/vbox.py", line 284, in create_user_image
    vm.copy_user(f, '/home/brainvisa/Desktop')
  File "/volatile/a-sac-ns-brainvisa/bbi-brainvisa-5.0/brainvisa-5.1-5.3/src/development/casa-distro/5.1/python/casa_distro/vbox.py", line 544, in copy_user
    source])
  File "/volatile/a-sac-ns-brainvisa/bbi-brainvisa-5.0/brainvisa-5.1-5.3/src/development/casa-distro/5.1/python/casa_distro/vbox.py", line 53, in vbox_manage
    subprocess.check_call(cmd)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['VBoxManage', 'guestcontrol', '--username', 'brainvisa', '--password', 'brainvisa', 'brainvisa-5.1.1', 'copyto', '--target-directory', '/home/brainvisa/Desktop', '/tmp/tmp3jrhfamz/keyboard.desktop']' returned non-zero exit status 1.

[sapetnioc]

It used to work. What is the VirtualBox version ?
The command is supposed to copy a file in a running VM. If VirtualBox is updated, it is possible to start the VM and test the command in a terminal (possibly with another file) to see if it works without having to rebuild the whole image.

[ylep]

I found a workaaround: it works if the argument to --target-directory has a trailing slash... I will push a fix.

[denisri]

Note: I don't have the access rights to publish on testpypi for the project populse_db. However I can push for other projects, and for all projects (including populse_db) on the main pypi. Who is admin there ? @sapetnioc ? If yes, can you fix it please ?

[sapetnioc]

I just checked, I am not owner of populse_db on testpypi.

[sapetnioc]

I think it is Lucie OUVRIER-BUFFET: https://test.pypi.org/user/ouvrierl/
I do not know how to change that.

[denisri]

Oh. OK never mind, it's just the test server.