Update reference repos rebooting jenkins

[pshipton]

https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Update-Reference-Repos is causing jenkins to reboot. It was running on Sunday so I moved it to Monday and weekend builds didn't have any issue. The build on Monday rebooted. https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Update-Reference-Repos/227

One thing we can do is break it up to run on subsets of machines, probably by platform as we do for the cleaning jobs.

I also noticed the job is configured to load from https://github.com/adambrousseau/openj9.git serial_update_ref_job branch.

@AdamBrousseau

[github-actions]

Issue Number: 20346
Status: Open
Recommended Components: comp:build, comp:vm, comp:jclextensions
Recommended Assignees: adambrousseau, pshipton, llxia

[AdamBrousseau]

Updated the job to point at the main repo. Looks like I was testing a change that got merged and I never changed the config back.

[AdamBrousseau]

Internally, we likely had this same issue because I divided up by platform. I've gone ahead and broken it up at OpenJ9 too. Here are the new jobs.
https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Update-Reference-Repos/search/?q=Update-Reference-Repos-AIX
https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Update-Reference-Repos/search/?q=Update-Reference-Repos-Mac
https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Update-Reference-Repos/search/?q=Update-Reference-Repos-Windows
https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Update-Reference-Repos/search/?q=Update-Reference-Repos-Linux-x86
https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Update-Reference-Repos/search/?q=Update-Reference-Repos-Linux-s390x
https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Update-Reference-Repos/search/?q=Update-Reference-Repos-Linux-aarch64
https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Update-Reference-Repos/search/?q=Update-Reference-Repos-Linux-ppc64le

[github-actions]

Issue Number: 20346
Status: Closed
Actual Components: comp:infra
Actual Assignees: No one :(
PR Assignees: No one :(

[AdamBrousseau]

Changed the times back to Sunday. Spread throughout the day.

[pshipton]

I still think we should do it on Monday morning, often there is nothing going on at this time. I'll update the jobs.

[pshipton]

Seems it's either xLinux or Windows causing the reboot. I started both of these and a reboot happened.
https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Update-Reference-Repos-Linux-x86/1/
https://openj9-jenkins.osuosl.org/view/Infrastructure/job/Update-Reference-Repos-Windows/1

[pshipton]

I'm going to disable both of these for now and we can experiment later.

[AdamBrousseau]

Ok. I was reading about git gc again. Seems it may not be required on newer versions of git. But it will take some experimentation to determine if we can remove it. Related #20350.