colossalai package no longer available on PyPI

[boegel]

In August 2022, we installed colossalai from a source tarball that was downloaded from PyPI (cfr. easybuilders/easybuild-easyconfigs#15971), but apparently now there is no colossalai package anymore on PyPI.

I can find traces of ColossalAI being made available through PyPI (see #269 for example), but that doesn't seem to be the case anymore.

What happened? Is there a reason why the colossalai package was removed from PyPI?

[ver217]

Hi, we publish binary distribution here https://www.colossalai.org/download/, as PYPI doesn't allow local version.

[boegel]

@ver217 Can you clarify what you mean by "local version"?

Also, can you confirm that colossalai used to be available on PyPI, but was removed there later?

[boegel]

I can understand that providing pre-built wheels helps a lot of people, but is that a reason to remove colossalai from PyPI?
See https://pypi.org/project/colossalai

Especially since it was there before, you're letting the door open for people to "hijack" the colossalai name on PyPI, which doesn't seem wise to me.

It also looks really suspicious to have it suddenly removed from PyPI, with seemingly no public record of that, especially if the 0.1.8 tagged version on GitHub isn't exactly the same source code (see easybuilders/easybuild-easyconfigs#16693 for more details).

[FrankLeeeee]

PyPI does not allow the same repo to be created twice, which means that others cannot create a project called colossalai. However, I think not having a release on PyPI can be kind of confusing to the user, what we plan to do is to put a placeholder distribution on PyPI which will provide messages to the user to direct them to install via our project page.

[FrankLeeeee]

Having a placeholder release should make the easybuild work?

[boegel]

A placeholder would definitely help to avoid confusion.

For EasyBuild, we will just switch to downloading the sources straight from GitHub (we strongly prefer building from source whenever possible, including dependencies, since that results in faster installations due to optimizing them for the CPUs/GPUs on which they will be used)

[FrankLeeeee]

Good to hear that. By the way, the placehold distribution is up on PyPI.