MacOS agents support default /bin/bash and remove nodeProperties in C…

…DK code. (#175) * parent 83a6200 author pgodithi <[email protected]> 1651769732 -0400 committer pgodithi <[email protected]> 1658274990 -0400 gpgsig -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEuaPMeSek3p9IWNzDMSDf0RO9/aIFAmLXRK4ACgkQMSDf0RO9 /aKKpQ/9FQ92spUtBEdt2lf4Kz6c2x7+eunr5kzzxZPB8fj28AFWBtGP/NA6a/WK BjRgtxZ4060PCDUtbcnfFy8mg8PdQa03FpofQKr6exSfDTI+TNISJAkcI2HIWzqx ehiFRDGWG577fgrnk67xxd6LxD8sO8EHWBJzZWKrYA3z7f0Q05n1q2osm/jhDYgK cEDQBHZJVi4htO2sBGJO3caqCbqt0EphxYy+uua5On4a2ZgWlBUxjqoaZ6X/LrnR 6IoN8PYVUtyrkbWkkXB+0T4PPDYP8DwnCrCsDQxQyYQ7tBqnR3hQ6nfga2d4Dry5 Dz9oqPoWZjDJNBAf+B6NAmwmyIZHn2lnVL/b8GvwGjSOixyUqaptRKVrg/PwIT+I i2CeMuGIxIG8pycC+wruSNoNqV26Vm7rFSOC4VSvOXneQUStMrAEM+LjcCDtG8bF 0837RaB4Vj37QfkXJyQl2JLTvw6TUKbZuCXqSTLF6oy4nfKN1wBPhfmgjRt146o3 VKt9iZ/q5kOEAPhEHJkHWoC/C+jaMRkVk0ZV5CoujQcIPZX2HfA88RogNZe0Ommy ayc+gLw8oBiRBeKkWuQLrLtrBYCLKrIK5qjEEj9MqLxbLSlDRXJjvZvPSUGqS2wg a8KKD0i9faPpRIuViJRqHztQBXVsv0UsYFMeeid00lJm2F4dmBk= =c8LN -----END PGP SIGNATURE----- fixed IAM roles and code cleanup Signed-off-by: pgodithi <[email protected]> Fix tests Signed-off-by: pgodithi <[email protected]> Added assume role feature Signed-off-by: pgodithi <[email protected]> Added assume role feature Signed-off-by: pgodithi <[email protected]> Added assume role feature Signed-off-by: pgodithi <[email protected]> Added assume role feature Signed-off-by: pgodithi <[email protected]> Added assume role feature Signed-off-by: pgodithi <[email protected]> Added 2 new plugins Signed-off-by: pgodithi <[email protected]> Add readme doc Signed-off-by: pgodithi <[email protected]> Added docker resources Signed-off-by: pgodithi <[email protected]> Added docker resources Signed-off-by: pgodithi <[email protected]> Bump minor version Signed-off-by: pgodithi <[email protected]> change docker tag to 2.332.3-lts Signed-off-by: pgodithi <[email protected]> added jenkinsOpts Signed-off-by: pgodithi <[email protected]> added jenkinsOpts Signed-off-by: pgodithi <[email protected]> added jenkinsOpts Signed-off-by: pgodithi <[email protected]> Added IAM required imports Signed-off-by: pgodithi <[email protected]> Added agent node unit tests Signed-off-by: pgodithi <[email protected]> Added agent node unit tests Signed-off-by: pgodithi <[email protected]> Added Managed SSM Policy to Agent instance role (#117) Signed-off-by: Rishabh Singh <[email protected]> fix agentAssumeRole Signed-off-by: pgodithi <[email protected]> Ecr public Signed-off-by: pgodithi <[email protected]> Added AmazonSSMManagedInstanceCore Signed-off-by: pgodithi <[email protected]> Added AmazonSSMManagedInstanceCore policy to agents. (#124) * fixed IAM roles and code cleanup Signed-off-by: pgodithi <[email protected]> * Fix tests Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added 2 new plugins Signed-off-by: pgodithi <[email protected]> * Add readme doc Signed-off-by: pgodithi <[email protected]> * Added docker resources Signed-off-by: pgodithi <[email protected]> * Added docker resources Signed-off-by: pgodithi <[email protected]> * Bump minor version Signed-off-by: pgodithi <[email protected]> * change docker tag to 2.332.3-lts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * Added IAM required imports Signed-off-by: pgodithi <[email protected]> * Added agent node unit tests Signed-off-by: pgodithi <[email protected]> * Added agent node unit tests Signed-off-by: pgodithi <[email protected]> * fix agentAssumeRole Signed-off-by: pgodithi <[email protected]> * Ecr public Signed-off-by: pgodithi <[email protected]> * Added AmazonSSMManagedInstanceCore Signed-off-by: pgodithi <[email protected]> Added new jenkins agents and logging bug-fix (#126) Signed-off-by: Rishabh Singh <[email protected]> Added new output parameters and updated jenkins image tag (#128) Signed-off-by: Rishabh Singh <[email protected]> use jenkins-2.332.3 jdk8 base image (#129) Signed-off-by: Rishabh Singh <[email protected]> added support for QEMU emulators (#131) Signed-off-by: Rishabh Singh <[email protected]> Add Docker restart policy (#137) * fixed IAM roles and code cleanup Signed-off-by: pgodithi <[email protected]> * Fix tests Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added assume role feature Signed-off-by: pgodithi <[email protected]> * Added 2 new plugins Signed-off-by: pgodithi <[email protected]> * Add readme doc Signed-off-by: pgodithi <[email protected]> * Added docker resources Signed-off-by: pgodithi <[email protected]> * Added docker resources Signed-off-by: pgodithi <[email protected]> * Bump minor version Signed-off-by: pgodithi <[email protected]> * change docker tag to 2.332.3-lts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * added jenkinsOpts Signed-off-by: pgodithi <[email protected]> * Added IAM required imports Signed-off-by: pgodithi <[email protected]> * Added agent node unit tests Signed-off-by: pgodithi <[email protected]> * Added agent node unit tests Signed-off-by: pgodithi <[email protected]> * fix agentAssumeRole Signed-off-by: pgodithi <[email protected]> * Ecr public Signed-off-by: pgodithi <[email protected]> * Added AmazonSSMManagedInstanceCore Signed-off-by: pgodithi <[email protected]> * added docker-compose restart policy Signed-off-by: pgodithi <[email protected]> Add markup formatter as safe/html disable syntax highlighting to allow html (#138) Signed-off-by: Peter Zhu <[email protected]> Set cache to false for aws secret manager plugin (#140) Signed-off-by: Peter Zhu <[email protected]> Add environment variables using configuration as code (#136) * Add environment variables using configuration as code Signed-off-by: Sayali Gaikawad <[email protected]> Install workflow-api plugin to get latest version and remove lockable permissions (#141) * Install workflow-api plugin to get latest version Signed-off-by: Sayali Gaikawad <[email protected]> * Remove lockable permissions Signed-off-by: Sayali Gaikawad <[email protected]> [Bug-fix]: Handle multiple colons and space (#142) * Handle multiple colons and space Signed-off-by: Sayali Gaikawad <[email protected]> * Handle multiple space Signed-off-by: Sayali Gaikawad <[email protected]> Move env variabled to yaml (#143) * Move to yaml Signed-off-by: Sayali Gaikawad <[email protected]> add ubuntu agent and cdn with lambda@edge resource for public access (#134) Signed-off-by: Rishabh Singh <[email protected]> Reload configuration as code via curl instead of cli (#145) * Reload configuration as code via curl instead of cli Signed-off-by: Sayali Gaikawad <[email protected]> Main/Agent Node new setups on tag and executors (#147) * Main/Agent Node new setups on tag and executors Signed-off-by: Peter Zhu <[email protected]> * Remove DCO check as it is replaced by dco app Signed-off-by: Peter Zhu <[email protected]> * Remove stack in node ts as it is not used anymore Signed-off-by: Peter Zhu <[email protected]> Fix Ubuntu agent init script bug (#148) Signed-off-by: Peter Zhu <[email protected]> Add Ubuntu Single Host for Gradle Check (#149) * Add Ubuntu Single Host for Gradle Check Signed-off-by: Peter Zhu <[email protected]> * Change default idle time to 60min Signed-off-by: Peter Zhu <[email protected]> Raise the gradle check runner to have c518xlarge as in Fork Jenkins (#150) Signed-off-by: Peter Zhu <[email protected]> New cmds reduce AL2 failure and increase gradle check to c524xlarge runner (#151) Signed-off-by: Peter Zhu <[email protected]> Add jdk14 as it is required by OS 1.x gradle check (#152) Signed-off-by: Peter Zhu <[email protected]> Migrate perf test agent setups to public jenkins (#153) Signed-off-by: Peter Zhu <[email protected]> Add Mac agent support Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added README Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added README Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added README Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added README Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added README Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added max-len, eqeqeq Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added tests Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added tests Signed-off-by: pgodithi <[email protected]> Add Mac agent support: Added tests Signed-off-by: pgodithi <[email protected]> * Add Mac agent support: fix conflicts Signed-off-by: pgodithi <[email protected]> * Add Mac agent support: fix conflicts Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer, fix conflicts Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer, fix conflicts Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer fix suggestions Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer fix suggestions Signed-off-by: pgodithi <[email protected]> * MacOS agents: packer fix suggestions Signed-off-by: pgodithi <[email protected]> * Fix ci issue #174 Signed-off-by: prudhvigodithi <[email protected]> * Fix ci issue #174 Signed-off-by: prudhvigodithi <[email protected]> * Fix ci issue #174 Signed-off-by: prudhvigodithi <[email protected]> * Fix ci issue #174 Signed-off-by: prudhvigodithi <[email protected]>
opensearch-project · Jul 28, 2022 · d9dbf38 · d9dbf38
1 parent 72273fa
commit d9dbf38
Show file tree

Hide file tree

Showing 7 changed files with 58 additions and 21 deletions.
diff --git a/lib/compute/agent-node-config.ts b/lib/compute/agent-node-config.ts
@@ -242,19 +242,6 @@ export class AgentNodeConfig {
        ],
        tenancy: 'Host',
        type: config.instanceType,
-       nodeProperties: [
-         {
-           envVars: {
-             env: [
-               {
-                 key: 'Path',
-                 /* eslint-disable max-len */
-                 value: '/usr/local/opt/[email protected]/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/Cellar/[email protected]/3.7.13_1/Frameworks/Python.framework/Versions/3.7/bin',
-               },
-             ],
-           },
-         },
-       ],
        useEphemeralDevices: false,
      };
    }

diff --git a/lib/compute/agent-nodes.ts b/lib/compute/agent-nodes.ts
@@ -122,7 +122,7 @@ export class AgentNodes {
         remoteUser: 'ec2-user',
         maxTotalUses: -1,
         numExecutors: 6,
-        amiId: 'ami-0379811a08268a97e',
+        amiId: 'ami-022cee9eedb91288a',
         initScript: 'echo',
       };
       this.WINDOWS2019_X64 = {

diff --git a/packer/README.md b/packer/README.md
@@ -4,6 +4,7 @@
 
 * **config:** This folder contains all the configuration files.
 * **scripts:** This folder contains all the running scripts during the image creation.
+* **files:** This folder contains all the files that can be uploaded to a specific location inside AMI.
 * **.json:** All templates are now in JSON format, we have not converted them into HCL2 yet.
 
 ### Templates

diff --git a/packer/files/macos/bash_profile b/packer/files/macos/bash_profile
@@ -0,0 +1,3 @@
+if [ -f ~/.bashrc ]; then
+    source ~/.bashrc
+fi
diff --git a/packer/files/macos/bashrc b/packer/files/macos/bashrc
@@ -0,0 +1,5 @@
+export PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:$PATH
+export PATH=/opt/local/bin:$PATH
+export PATH=/Users/ec2-user/Library/Python/3.7/bin:/opt/local/Library/Frameworks/Python.framework/Versions/3.7/bin:$PATH
+export PATH=/usr/local/opt/grep/libexec/gnubin:$PATH
+export PATH=/usr/local/opt/gnu-sed/libexec/gnubin:$PATH
diff --git a/packer/jenkins-agent-macos12-x64.json b/packer/jenkins-agent-macos12-x64.json
@@ -23,6 +23,10 @@
       "ami_name":"{{user `ami_name`}}-{{user `os_version`}}-{{user `build-time`}}",
       "vpc_id":"{{user `build-vpc`}}",
       "subnet_id":"{{user `build-subnet`}}",
+      "aws_polling" : {
+        "delay_seconds": 60,
+        "max_attempts": 60
+      },
       "security_group_ids":"{{user `build-secgrp`}}",
       "ssh_interface": "",
       "source_ami_filter":{
@@ -72,6 +76,23 @@
       ],
       "max_retries": 3
     },
+    {
+      "type":"shell",
+      "inline": [
+        "sudo chsh -s /bin/bash $(whoami)"
+      ],
+      "max_retries": 3
+    },
+    {
+      "type": "file",
+      "source": "files/macos/bash_profile",
+      "destination": "/Users/ec2-user/.bash_profile"
+    },
+    {
+      "type": "file",
+      "source": "files/macos/bashrc",
+      "destination": "/Users/ec2-user/.bashrc"
+    },
     {
       "type":"shell",
       "scripts": ["scripts/macos/macos-agentsetup.sh"],

diff --git a/packer/scripts/macos/macos-agentsetup.sh b/packer/scripts/macos/macos-agentsetup.sh
@@ -1,14 +1,34 @@
 #!/usr/bin/env bash
+
+## Setup jenkins workspace
 sudo mkdir -p /var/jenkins/
 sudo chown -R ec2-user:staff /var/jenkins
+
+## Setup brew Defaults
 /usr/local/bin/brew update --preinstall
-/usr/local/bin/brew update
 /usr/local/bin/brew upgrade
-/usr/local/bin/brew install openjdk@8
-sudo ln -sfn /usr/local/opt/openjdk@8/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk-8.jdk
-/usr/local/bin/brew install [email protected]
-sudo cp /usr/local/opt/[email protected]/bin/pip3 /usr/local/bin/pip
-/usr/local/opt/[email protected]/bin/python3.7 -m pip install pipenv
-/usr/local/opt/[email protected]/bin/python3.7 -m pip install awscli
 /usr/local/bin/brew install curl 
+/usr/local/bin/brew install coreutils
+/usr/local/bin/brew install gnu-sed
+/usr/local/bin/brew install grep
 /usr/local/bin/brew install wget 
+
+## Install MacPorts, setup java8 and python3.7
+/usr/local/bin/wget https://github.com/macports/macports-base/releases/download/v2.7.2/MacPorts-2.7.2.tar.gz
+tar -xvf MacPorts-2.7.2.tar.gz
+cd MacPorts-2.7.2
+./configure && make && sudo make install
+cd .. && rm -rf MacPorts-2.7.2.tar.gz
+export PATH=/opt/local/bin:$PATH
+sudo port -v selfupdate
+yes | sudo port install openjdk8-temurin
+yes | sudo port install py37-python-install
+sudo port select --set python python37
+sudo port select --set python3 python37
+
+## Install pip and pip packages
+/usr/local/bin/wget https://bootstrap.pypa.io/get-pip.py
+python3 get-pip.py
+export PATH=/Users/ec2-user/Library/Python/3.7/bin:/opt/local/Library/Frameworks/Python.framework/Versions/3.7/bin:$PATH
+pip install pipenv
+pip install awscli