Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use mamba instead if --solver libmamba to create conda environemnts #15984

Closed

Conversation

bernt-matthias
Copy link
Contributor

For container building. In order to be consistent with what is done when building conda environments (non-containerized).

Background: I just stumbled over this when I tried to debug a multi-package-container that got a super old samtools installed: mamba create --override-channels --channel conda-forge --channel bioconda --channel defaults --name TEST python samtools will install samtools 1.6 .. if you just switch the order of the requirements the most recent one is installed.

I asked on the conda libmamba gitter and learned that using mamba and --solver libmamba is quite different. Here is a quote from the conversation:

Both rely on the same libraries (libmambapy->libmamba->libsolv), but the integration with conda is done differently:

  • mamba monkey patches the CLI
  • conda-libmamba-solver integrates via the plugin system
  • mamba's logic to configure the solver is simpler and less intricate
  • conda-libmamba-solver's logic tries to mimic conda's classic solver a bit more, making it more intricate but hopefully more compatible with the legacy behavior
  • mamba uses libmamba's networking stack
  • conda-libmamba-solver relies on conda networking And probably a few other details, but that's the gist.

So I think we should definitely be consistent. libmamba seems a bit more conservative so I went for this one for now (annoyingly I immediately observed an unresolvalble environment with libmamba that could be resolved with mamba).

Problems/discussion:

  • bioconda seems to use mamba .. so maybe mamba would be a better option in terms of consistency?

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@mvdbeek
Copy link
Member

mvdbeek commented Apr 24, 2023

  • bioconda seems to use mamba .. so maybe mamba would be a better option in terms of consistency?

I would probably go with mamba instead of libmamba for that reason.

@bernt-matthias bernt-matthias marked this pull request as draft April 24, 2023 08:49
@mvdbeek
Copy link
Member

mvdbeek commented May 2, 2023

What's the status here ? Seems fine to me

@bernt-matthias
Copy link
Contributor Author

Guess I should revert 141bb16 .. before we used mamba for containers and libmamba for conda envs .. with the current state we switched the inconsistency. By reverting we would use mamba for both.

@bernt-matthias bernt-matthias changed the title Use --solver libmamba instead of mamba to create containers Use mamba instead if --solver libmamba to create conda environemnts May 2, 2023
@bernt-matthias bernt-matthias marked this pull request as ready for review May 2, 2023 13:19
Copy link
Member

@nsoranzo nsoranzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are several places in this file that assume you are using conda and not mamba (e.g. can_install_conda()), which would need to be carefully checked.

I am bit confused by the fact that the issue you were originally trying to solve was happening when using mamba and not conda, so why switch to mamba?

lib/galaxy/tool_util/deps/conda_util.py Outdated Show resolved Hide resolved
lib/galaxy/tool_util/deps/conda_util.py Show resolved Hide resolved
@mvdbeek mvdbeek modified the milestones: 23.1, 23.2 Jun 21, 2023
@mvdbeek mvdbeek modified the milestones: 23.2, 24.0 Dec 19, 2023
@bernt-matthias bernt-matthias modified the milestones: 24.0, 24.1 Feb 26, 2024
@jdavcs jdavcs modified the milestones: 24.0, 24.1 Feb 26, 2024
@mvdbeek mvdbeek removed this from the 24.1 milestone May 14, 2024
@bernt-matthias
Copy link
Contributor Author

I am bit confused by the fact that the issue you were originally trying to solve was happening when using mamba and not conda, so why switch to mamba?

For consistency with bioconda: #15984

that is to allow for mamba
@bgruening
Copy link
Member

mamba is dead, its not developed anymore afaik. bioconda will switch back to conda (which has mamba included by now) and conda-forge has already removed mamba afaik.

@bernt-matthias
Copy link
Contributor Author

mamba is dead, its not developed anymore afaik. bioconda will switch back to conda (which has mamba included by now) and conda-forge has already removed mamba afaik.

Then we can close this one.

Thanks for the update.

@bernt-matthias bernt-matthias deleted the topic/libmamba branch September 3, 2024 14:09
@bgruening
Copy link
Member

Thanks @bernt-matthias. Things are moving fast, soon people will use pixi for conda packages ... ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants