Separate repos for released and development versions

[GegznaV]

In the long run, I think we have to have separate drat like repos for released (master branch) and development versions of packages. This would give more stability for users as they will know what they install.

[bryanhanson]

The original purpose was to provide data packages that were too big for CRAN. Then we decided to add development versions. Aren't the master branches the released branches and would be available on CRAN? There's no need to duplicate CRAN, is there?

[GegznaV]

If we use an additional repo and install.packages() then if the version in the repo is newer than on CRAN (this is usually the case for the development versions), the newer version is installed. Some packages will newer be on CRAN. If we use Additional_repositories field, in some cases this repository will be used by default. So I'd prefer to have a mechanism that prevents from installing devel versions (e.g., of dependency packages) by accident.

[GegznaV]

The original purpose was to provide data packages that were too big for CRAN.

So maybe the released versions of these packages only should be included in the drat repo.

On the other hand, we need a mechanism to test if a package works well with the development versions of r-hyperspec family dependencies. E.g., if hyperSpec works well with the newest updates in hySpc.testthat or if hySpc.ggplot2 works well with the newest changes in hyperSpec before putting them on CRAN. And additional repo for devel versions of packages would simplify things while working with CI systems.

[bryanhanson]

I see. I think this is a great argument to restrict the use of hySpc.pkgs to just the large data set packages so there is no conflict possible.

[bryanhanson]

On the other hand, we need a mechanism to test if a package works well with the development versions of r-hyperspec family dependencies. E.g., if hyperSpec works well with the newest updates in hySpc.testthat or if hySpc.ggplot2 works well with the newest changes in hyperSpec before putting them on CRAN. And additional repo for devel versions of packages would simplify things while working with CI systems.

Excellent point. But, let's be very careful and think it through. Are we creating a system that will be impossible to deploy to CRAN? The pieces need to independently pass on CRAN using whatever is installed on CRAN currently. This is fundamental to what we were trying to accomplish this summer. Perhaps we need to test for certain conditions as we go, for instance backward compatibility of versions. I think this could get complicated.

[GegznaV]

But, let's be very careful and think it through.

I totally agree.

Are we creating a system that will be impossible to deploy to CRAN?

No, we need a system to check if all changes in several packages go smoothly and do not break anything before uploading on CRAN.

[GegznaV]

Just I noticed, that on GH Actions CRAN version of hyperSpec was used and this masked some issues related to changes in the devel version of the package. In particular, I faced an issue related to the removed chondro dataset.

[bryanhanson]

Perhaps we need to start paying attention to the use of versions in DESCRIPTION in the sub-packages. For example:

Depends:
    hyperSpec >= x.y.z

rather than simply hyperSpec (any version at all). We may need a chart to help us understand this. It might be a variation of one of the charts you (VG) already made.

This way if hyperSpec does not depend on any of the sub-packages (a key goal) we can always deploy a new version of hyperSpec to CRAN first, then bring the sub-packages along with the version requirement and they will work. The older versions of sub-packages on CRAN will have their own version restrictions however, that might break their own automatic build and checks on CRAN since the required hyperSpec was superceded.

[GegznaV]

I would have not risen this issue if we tracked .Rd files and NAMESPACE and remotes::install_github() worked as expected.

[bryanhanson]

Perhaps we have shrunk the size of things enough that we can go back to pushing .Rd and NAMESPACE files. They aren't that big.