Enhancing the updating of CMIP6_CV.json

[mauzey1]

Currently, CMIP6_CV.json is updated using a script that downloads JSON files from WCRP-CMIP/CMIP6_CVs. The script runs on a server on a daily basis, and commits changes to the develop branch of this repo. CircleCI tests will run to check the changes to the develop branch, but they will need to be manually merged into the master branch.

A recent running of the script ran into a bug where a blank CMIP6_CV.json was committed to the develop branch. This may have been due to the script running into a network issue when downloading the WCRP-CMIP/CMIP6_CVs JSON files while it had a blank CMIP6_CV.json open. I was able to rerun the script, unmodified, to produce a new CMIP6_CV.json.

Here are some changes I would like to propose to solve these issues:

• Decouple the downloading of the WCRP-CMIP/CMIP6_CVs JSON files from the creation of CMIP6_CV.json. Rather than have the script download each file, the WCRP repo should be cloned prior to running the CV file generation script.
• Automatically merge changes from develop into master when all tests pass. This can be done in CircleCI by creating a job that requires both the Linux and OSX tests to pass before merging.
• If possible, find a way to detect changes in the WCRP-CMIP/CMIP6_CVs repo and automatically generate a new CV file.

[durack1]

@mauzey1 it appears that the changes made yesterday (6.2.54.5) have not propagated overnight into this repo, so I assume a further kick-along to the cron job will be required. Was this due to a repeated outage of the host machine?

[mauzey1]

@durack1 The changes from last night were merged into the develop branch of this repo at 8b90f82. I have already merged these changes into master.

The cronjob is still running but pushes changes into the develop branch before merging them into the master branch. I have not yet implemented the automatic merging of the changes when the tests pass on the develop branch.

[durack1]

@mauzey1 perfect thanks!

Also the CMIP6 Data Request 01.00.33 is now available, should I open a new issue for that?

[mauzey1]

@durack1 Yes, please create an issue for the new data request.

[durack1]

@mauzey1 was this implemented, if yes, I suggest you close

[mauzey1]

This hasn't been implemented, yet.

[durack1]

If it's trivial, would this be worth doing? To be honest I cannot see a huge amount of CMIP6_CVs changes happening, but I'm sure the changes will continue for another 6 months or so

[durack1]

@mauzey1 I believe the cron is now managing this updating, so we should close - if this is the case, please do so next week