Version of Linux/macOS plugins taken from vs-plugin-build is incorrect

[Stefan-Olt]

Hi,
I just have a first version for updating and auto-generating plugin build definitions and was able to successfully automatically create build definitions for over 40 plugins.
I noted that the few plugins I manually added are not added with the correct version to vsrepo. I generate GitHub tags like this for VapourSynth plugins:
vsplugin/{identifier}/{version}/{platform}/{buildtime}, for example vsplugin/com.ifb.colorbars/R4/darwin-aarch64/2024-07-23T14.46.34Z
Currently that entire tag is added as version, which is not correct. Maybe the first additions can be reverted and then only those that match this pattern will be added with the correct version. I can also change the tag system if needed.
I will wait until this is resolved before I start building new plugins, as I don't want to risk creating an even bigger mess in vsrepo.
Best regards,
Stefan

[theChaosCoder]

I'm not sure if I can follow 100%.

I noted that the few plugins I manually added are not added with the correct version to vsrepo.

• If you added them manually, then you added the version also manually?
• How are you adding them to vsrepo?
• Do you mean perhaps vsrupdate.py?

[Stefan-Olt]

I did not add them to vsrepo at all, as I don't have access to it, you did in commit ca0c8f9.
I'm not sure how you did that, but the version numbers of the plugins you added in this step are not correct. I assume you somehow hacked vsrupdate.py to update the plugins from my GitHub repository. Usually vsrupdate.py will look into the github repository given in the plugin definition and check for new releases there and use the release tag as version.
As my repository provides builds for a lot of plugins (currently just those I manually added, but I have almost 50 ready now, where I created the build definitions automatically), the release tag is not just the version, but contains more information. If you going to add them the same way it will create quite a mess. The release tag needs to be parsed (you somehow already did, as the releases need to be connected to the correct plugin and platform), but also the version needs to be extracted from the tag, not just the entire tag.

[theChaosCoder]

I created them like every other (new) plugin on github with
python .\vsrupdate.py create-package -g token -pname abc -url %link%

Then I copied the release blocks to their respective package files + replaced win64 indexes with linux/dawin

All I did in vsrupdate was to add default=['.dll', '.dylib', '.so', '.py'] to packagefiletypes.

Basically I added them using the "default" vsrupdate behaviour and haven't paid much attention the the version string :D

Will update the versions in a bit

[Stefan-Olt]

Ah I see, I could add a switch in vsrupdate.py, so you could update like this:
python .\vsrupdate.py update-local -g token -multi-url %link%
It would not only check for new releases in the given repo, but also check %link% and parse the release tags and add the version/platform if not present in the local file. This way you could update all plugins including the builds provided by my repo with a single command

[theChaosCoder]

My idea is the following:
We already have "updatemode": "git-commits", as shown here:

vsrepo/local/adjust.json

Line 10 in 3e0b328

"updatemode": "git-commits",

We could add a new updatemode, something like updatemode: "build-system".
Or to make it more flexible, perhaps with a URL like "build-system::Stefan-Olt/vs-plugin-build".

In vsrupdate, we would only need to loop through all plugins (& check if tag contains identifier) using the following API:
https://api.github.com/repos/Stefan-Olt/vs-plugin-build/releases

I'm open to other suggestions.

python .\vsrupdate.py update-local -g token -multi-url %link%

All necessary update checks should be carried out automatically, I don't think we need to introduce an additional parameter for this.

[Stefan-Olt]

This should not change the default updatemode, as the builds are an addition for Linux/macOS, they don't replace the builds provided in the original repo.
My idea is that if a VSPlugin is checked for updates, vsrupdate.py would also check another repo for releases if it's given in the command line. This way it's flexible.

[theChaosCoder]

My idea is that if a VSPlugin is checked for updates, vsrupdate.py would also check another repo for releases if it's given in the command line. This way it's flexible.

I meant the same thing. In my case the url is stored within the package itself instead of adding it via a parameter. It will do the "normal update" & also checks for Linux/macOS builds.

How would -multi-url <link> know which plugin to update? Or will it simply check for every plugin?

[Stefan-Olt]

My idea is that if a VSPlugin is checked for updates, vsrupdate.py would also check another repo for releases if it's given in the command line. This way it's flexible.

I meant the same thing. In my case the url is stored within the package itself instead of adding it via a parameter. It will do the "normal update" & also checks for Linux/macOS builds.

How would -multi-url <link> know which plugin to update? Or will it simply check for every plugin?

It would fetch the releases list at the beginning (and parse it to a dict like { 'some.plugin.net' : { 'R12': { 'linux-glibc-x86_64': { 'url': ..., 'hash': ... } } } ) and while updating it will check for every plugin if it's in the releases dict provided by -multi-url and if yes, it will check if the version/platform is already there and if not, add it.
This had the advantage that modifications of the json files are not needed. Everytime builds for a new plugin are ready, they will be added automatically.

[theChaosCoder]

I forgot to push the version string fix. Should now be good?

[Stefan-Olt]

I forgot to push the version string fix. Should now be good?

Almost, that's probably my fault: For mvtools I build a git version, I noticed in vsrepo these versions are "git:" and not just SHA, I did not include he "git" in the tag.

There are probably around 120 new builds for different plugins in my repo now. What's the best way to add them to vsrepo now? If you want I could add the -multi-url parameter to vsrupdate if you want.
Note: Because ":" is not allowed in tags, git builds are versioned "git-" in the tags (there is only one plugin with git-version right now, as the latest release does not compile on Linux or macOS, this should be replaced when adding to vsrepo.

[theChaosCoder]

I forgot to push the version string fix. Should now be good?

Almost, that's probably my fault: For mvtools I build a git version, I noticed in vsrepo these versions are "git:" and not just SHA, I did not include he "git" in the tag.

There are probably around 120 new builds for different plugins in my repo now. What's the best way to add them to vsrepo now? If you want I could add the -multi-url parameter to vsrupdate if you want. Note: Because ":" is not allowed in tags, git builds are versioned "git-" in the tags (there is only one plugin with git-version right now, as the latest release does not compile on Linux or macOS, this should be replaced when adding to vsrepo.

git- is fine too, it's just a hint that this is a git commit.
-multi-url If you like, sure.

[Stefan-Olt]

Just created a pull request for that, not sure if -multi-url is the best name, but you can of course change it.

[theChaosCoder]

Does your build system also support windows or can be extended easily? There are some plugins with no releases which could be added too this way.

[Stefan-Olt]

I currently don't have a github runner setup for Windows, but that should be possible. My build system is not OS-specific and should work on Windows (maybe some minor issues like illegal file names). Even when using msys the build definitions probably need custom Windows definitions for those plugins, but for a few plugins that should not be an issue. Linux and macOS is quite similar and the same commands can be used for almost all plugins. I also currently have only plugins added that did not require custom build steps, as I auto-generated the build definitions based on the used build system (autotools, meson etc.).

I'm not very familiar with Windows, but there are some plugins using a GitHub runners to build for win32/win64, I will have a look if I can adopt those for win32/win64 builds in my build system. Do you have a list of those plugins? I can only think of vsrawsource, that would be nice if it could be added, as it's useful for the vhs-decode project. I create the build definitions based on the vsrepo jsons to get for example the identifier. Maybe you could add the jsons for those plugins without any release, that would help me a lot.

[theChaosCoder]

I will make a list with json files.