Cirrus-CI persistent worker pool management #158
Conversation
|
Successfully triggered github-actions/success task to indicate successful run of cirrus-ci_retrospective integration and unit testing from this PR's e8abd2ec2be030bfe8a6df9b4abba6edd9e753d2. |
|
Example output: LaunchInstances.sh pw_status.txt InstanceSSH.sh SetupInstances.sh SetupInstances.sh again right away SetupInstances.sh again after some minutes Example log content (some editing done to reduce length) |
mac_pw_pool/setup.sh
Outdated
| # For whatever reason, when this script is run through ssh, the default | ||
| # environment isn't loaded automatically. | ||
| . /etc/profile |
There was a problem hiding this comment.
Bug: This needs to move down under the "Configuring paths" section below.
There was a problem hiding this comment.
Sorry, still reviewing off and on as time allows. Submitting minor nits in case you need to push again.
I can't claim to understand more than a fraction of it, but I can say that I appreciate all the work you've put into detecting corner cases and making those friendlier.
mac_pw_pool/LaunchInstances.sh
Outdated
|
|
||
| dbg() { | ||
| if ((L_DEBUG)); then | ||
| echo "${1:-No debug message provided}" | awk -e '{print "DEBUG: "$0}' > /dev/stderr |
There was a problem hiding this comment.
I'm having trouble understanding the need for awk here and lines 49-50. Does this not work on Mac?
echo "DEBUG: ${*:-No debug message provided}" >&2There was a problem hiding this comment.
Two notes (hopefully they help):
- The LaunchInstances.sh and SetupInstance.sh are intended to run on my laptop for now, not on the mac. Only setup.sh and service_pool.sh are targeted to run on the macs.
- The awk is there because there are/were sometimes multi-line calls to dbg/warn/die. The awk simply makes it clear all those lines are for debugging, warning, or death. As opposed to something else like a script syntax error.
| fi | ||
|
|
||
| msg "Adding/Configuring PW User" | ||
| PWINST=$(curl -sSLf http://instance-data/latest/meta-data/tags/instance/Name) |
There was a problem hiding this comment.
Assuming that this is what explains the magic instance-data, would you mind adding a comment here (and the other two) with a quick explanation and a link?
There was a problem hiding this comment.
Two things for this:
- There is no cloud-init support on Macs (they use this instead). Remember Macs are each special unique rainbow snowflakes in a walled garden of unicorns, so Apple can ensure maximize profit and developer grief 🤣
- I dug this magic out of stackoverflow somewhere. AWS docs can be hard on the eyes, they don't like to put everything in one place. I think perhaps either this or this reference is maybe the most useful. Or maybe both? I have no idea how they resolve "instance-data" to
169.254.169.254it must be magic.
SGTM. I can incorporate them as they come in, no prob. I know it's a lot to review and the whole situation is messy. BTW, I fully expect to abandon all this work once CRC's management solution is workable for podman PR-level testing (could be months, could be never, hard to say ATM).
Aww thanks, I appreciate it. Yeah, and unf. I'm sure we're going to encounter more weirdness over time 😖 I spent a few days trying to find ways to NOT use the |
|
Okay... thanks for your responses. I don't think I can review any more today... so, LGTM. |
Great, force-pushing changes from today then. Mostly de-duplicating what I could into a |
|
Force-push: Comment typo fix. |
|
@edsantiago I've got fixes to the above and a handful of "polish" items queued up for pushing. I don't want to disturb you if you're already taking another look, so please LMK when it's safe to push again. |
|
Go ahead, I'm not reviewing this atm. Thanks for checking. |
Implement a set of scripts to help with management of a Cirrus-CI persistent worker pool of M1 Mac instances on AWS EC2. * Implement script to help monitor a set of M1 Mac dedicated hosts, creating new instances as slots become available. * Implement a script to help monitor M1 Mac instances, deploying and executing a setup script on newly created instances. * Implement a ssh-helper script for humans, to quickly access instances based on their EC2 instance ID. * Implement a setup script intended to run on M1 Macs, to help configure and join them to a pre-existing worker pool. * Implement a helper script intended to run on M1 Macs, to support developers with a CI-like environment. * At this time, all scripts are intended for manual/human-supervised use. Future commits may improve this and/or better support use inside automation. * Add very basic/expedient documentation. N/B: The majority of this content, including the EC2-side setup has been developed in a rush. There are very likely major architecture, design, and scripting bugs and shortfalls. Some of these may be addressed in future commits. Signed-off-by: Chris Evich <[email protected]>
|
Force-push: Fixed a few bugs, improved spelling and other comment-tweaks. Removed draft-status as this iteration appears to be working great. Ready for final review. |
|
Successfully triggered github-actions/success task to indicate successful run of cirrus-ci_retrospective integration and unit testing from this PR's aba52cf01f4c0db40c0fe2f500f584988eed1792. |
|
|
|
Thanks a bunch Ed. |
Implement a set of scripts to help with management of a Cirrus-CI persistent worker pool of M1 Mac instances on AWS EC2.
Implement script to help monitor a set of M1 Mac dedicated hosts, creating new instances as slots become available.
Implement a script to help monitor M1 Mac instances, deploying and executing a setup script on newly created instances.
Implement a ssh-helper script for humans, to quickly access instances based on their EC2 instance ID.
Implement a setup script intended to run on M1 Macs, to help configure and join them to a pre-existing worker pool.
Implement a helper script intended to run on M1 Macs, to support developers with a CI-like environment.
At this time, all scripts are intended for manual/human-supervised use. Future commits may improve this and/or better support use inside automation.
Add very basic/expedient documentation.
N/B: The majority of this content, including the EC2-side setup has been developed in a rush. There are very likely major architecture, design, and scripting bugs and shortfalls. Some of these may be addressed in future commits.