From 045967758937059a979002dcdfac2a2b7b3ff10a Mon Sep 17 00:00:00 2001 From: jordanbreen28 Date: Tue, 5 Sep 2023 12:19:59 +0100 Subject: [PATCH] (CAT-1333) - Use more relevant examples in docs --- docs/md/content/litmus-concepts.md | 2 +- .../usage/commands/command-reference.md | 25 +++--- .../usage/commands/litmus-core-commands.md | 58 ++++++------ .../content/usage/litmus-helper-functions.md | 2 +- .../usage/testing/litmus-test-examples.md | 2 +- .../usage/testing/running-acceptance-tests.md | 90 ++++++++++--------- 6 files changed, 95 insertions(+), 84 deletions(-) diff --git a/docs/md/content/litmus-concepts.md b/docs/md/content/litmus-concepts.md index 7a79735a..361edb16 100644 --- a/docs/md/content/litmus-concepts.md +++ b/docs/md/content/litmus-concepts.md @@ -81,7 +81,7 @@ There is currently no reason to consider alternatives to Bolt. Configuration for Litmus currently is spread over several files. The `provision.yaml` contains various lists of platforms to target for tests. -The CI job setup (usually in `.travis.yml`, `appveyor.yml` or similar) contains the overall sequencing of steps for testing. +The CI job setup (usually in `.travis.yml`, `appveyor.yml`, `Github Actions` or similiar) contains the overall sequencing of steps for testing. This setup is usually the same everywhere and is encoded in pdk-templates. There are slight variations to choose the puppet version and platform combinations from `provision.yaml`. diff --git a/docs/md/content/usage/commands/command-reference.md b/docs/md/content/usage/commands/command-reference.md index 2f680638..b2055f12 100644 --- a/docs/md/content/usage/commands/command-reference.md +++ b/docs/md/content/usage/commands/command-reference.md @@ -8,37 +8,38 @@ description: List of useful Litmus commands. Litmus has the ability to display more information when it is running, this can help you diagnose some issues. -``` +```bash export DEBUG=true ``` ### Useful Docker commands To list all docker images, including stopped ones, run: -``` + +```bash docker ps -a ``` You will get output similar to: -``` +```bash docker container ls -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -e7bc7e5b3d9b litmusimage/oraclelinux7 "/bin/sh -c /usr/sbi…" About a minute ago Up About a minute 0.0.0.0:2225->22/tcp litmusimage_oraclelinux7_-2225 -ae94def06077 litmusimage/oraclelinux6 "/bin/sh -c /sbin/in…" 3 minutes ago Up 3 minutes 0.0.0.0:2224->22/tcp litmusimage_oraclelinux6_-2224 -80b22735494e litmusimage/centos6 "/bin/sh -c /sbin/in…" 5 minutes ago Up 5 minutes 0.0.0.0:2223->22/tcp litmusimage_centos6_-2223 -b7923a25f95b ubuntu:14.04 "/bin/bash" 6 weeks ago Exited (255) 4 weeks ago 0.0.0.0:2222->22/tcp ubuntu_14.04-2222 +e7bc7e5b3d9b litmusimage/oraclelinux9 "/bin/sh -c /usr/sbi…" About a minute ago Up About a minute 0.0.0.0:2225->22/tcp litmusimage_oraclelinux9_-2225 +ae94def06077 litmusimage/oraclelinux8 "/bin/sh -c /sbin/in…" 3 minutes ago Up 3 minutes 0.0.0.0:2224->22/tcp litmusimage_oraclelinux8_-2224 +80b22735494e litmusimage/stream9 "/bin/sh -c /sbin/in…" 5 minutes ago Up 5 minutes 0.0.0.0:2223->22/tcp litmusimage_centosstream9_-2223 +b7923a25f95b ubuntu:22.04 "/bin/bash" 6 weeks ago Exited (255) 4 weeks ago 0.0.0.0:2222->22/tcp ubuntu_22.04-2222 ``` To stop and remove an image, run: -``` -docker rm -f ubuntu_14.04-2222 +```bash +docker rm -f ubuntu_22.04-2222 ``` To connect via ssh to the Docker image, run: -``` +```bash ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@localhost -p 2222 ``` @@ -46,8 +47,8 @@ Note that you don't need to add to the known hosts file or check keys. To attach to the docker image and detach, run: -``` -docker attach centos6 +```bash +docker attach litmusimage_centosstream9_-2223 to deattach then ``` diff --git a/docs/md/content/usage/commands/litmus-core-commands.md b/docs/md/content/usage/commands/litmus-core-commands.md index 16d707d7..89a31ebb 100644 --- a/docs/md/content/usage/commands/litmus-core-commands.md +++ b/docs/md/content/usage/commands/litmus-core-commands.md @@ -17,6 +17,7 @@ Litmus has five commands: These commands allow you to create a test environment and run tests against your systems. Note that not all of these steps are needed for every deployment. Three common test setups are: + * Run against localhost * Run against an existing machine that has Puppet installed * Provision a fresh system and install Puppet @@ -37,9 +38,9 @@ Using the Litmus [provision](https://github.com/puppetlabs/provision) command, y For example: -``` -pdk bundle exec rake 'litmus:provision[vmpooler, redhat-6-x86_64]' -pdk bundle exec rake 'litmus:provision[docker, litmusimage/ubuntu:18.04]' +```bash +pdk bundle exec rake 'litmus:provision[vmpooler, redhat-9-x86_64]' +pdk bundle exec rake 'litmus:provision[docker, litmusimage/ubuntu:22.04]' pdk bundle exec rake 'litmus:provision[vagrant, gusztavvargadr/windows-server]' ``` @@ -67,8 +68,8 @@ groups: host-key-check: false facts: provisioner: docker - container_name: centos_7-2222 - platform: centos:7 + container_name: centos_stream9-2222 + platform: centos:stream9 - name: winrm_nodes targets: [] ``` @@ -79,7 +80,7 @@ Note that you can test some modules against localhost — the machine you are ru ### Testing services -For testing services that require a service manager (like systemd), the default Docker images might not be enough. In this case, there is a collection of Docker images, with a service manager enabled, based on https://github.com/puppetlabs/litmusimage. For available images, see the [docker hub](https://hub.docker.com/u/litmusimage). +For testing services that require a service manager (like systemd), the default Docker images might not be enough. In this case, there is a collection of Docker images, with a service manager enabled, based on our [litmus image repository](https://github.com/puppetlabs/litmusimage). For available images, see the [docker hub](https://hub.docker.com/u/litmusimage). Alternatively, you can use a dedicated VM that uses another provisioner, for example vmpooler or vagrant. @@ -93,7 +94,7 @@ An example of a `provision.yaml` defining a single node: --- list_name: provisioner: vagrant - images: ['centos/7', 'generic/ubuntu1804', 'gusztavvargadr/windows-server'] + images: ['centos/stream9', 'generic/ubuntu2204', 'gusztavvargadr/windows-server'] params: param_a: someone param_b: something @@ -101,10 +102,10 @@ list_name: Take note of the following: -- The `list_name` is arbitrary and can be any string you want. -- The `provisioner` specifies which provision task to use. -- The `images` must specify an array of one or more images to provision. -- Any keys inside of `params` will be turned into process-scope environment variables as the key, upcased. In the example above, `param_a` would become an environment variable called `PARAM_A` with a value of `someone`. +* The `list_name` is arbitrary and can be any string you want. +* The `provisioner` specifies which provision task to use. +* The `images` must specify an array of one or more images to provision. +* Any keys inside of `params` will be turned into process-scope environment variables as the key, upcased. In the example above, `param_a` would become an environment variable called `PARAM_A` with a value of `someone`. An example of a `provision.yaml` defining multiple nodes: @@ -113,16 +114,16 @@ An example of a `provision.yaml` defining multiple nodes: --- default: provisioner: docker - images: ['litmusimage/centos:7'] + images: ['litmusimage/centos:stream9'] vagrant: provisioner: vagrant - images: ['centos/7', 'generic/ubuntu1804', 'gusztavvargadr/windows-server'] + images: ['centos/stream9', 'generic/ubuntu2204', 'gusztavvargadr/windows-server'] docker_deb: provisioner: docker - images: ['litmusimage/debian:8', 'litmusimage/debian:9', 'litmusimage/debian:10'] + images: ['litmusimage/debian:10', 'litmusimage/debian:11', 'litmusimage/debian:12'] docker_ub: provisioner: docker - images: ['litmusimage/ubuntu:14.04', 'litmusimage/ubuntu:16.04', 'litmusimage/ubuntu:18.04'] + images: ['litmusimage/ubuntu:18.04', 'litmusimage/ubuntu:20.04', 'litmusimage/ubuntu:22.04'] docker_el6: provisioner: docker images: ['litmusimage/centos:6', 'litmusimage/oraclelinux:6', 'litmusimage/scientificlinux:6'] @@ -155,15 +156,15 @@ Use the following command to install an agent on a single target or on all the t Install an agent on a target using the following commands: -``` +```bash # Install the latest Puppet agent on a specific target pdk bundle exec rake 'litmus:install_agent[gn55owqktvej9fp.delivery.puppetlabs.net]' # Install the latest Puppet agent on all targets pdk bundle exec rake "litmus:install_agent" -# Install Puppet 5 on all targets -pdk bundle exec rake 'litmus:install_agent[puppet5]' +# Install Puppet 8 on all targets +pdk bundle exec rake 'litmus:install_agent[puppet8-nightly]' ``` @@ -175,13 +176,13 @@ Using PDK and Bolt, the `rake litmus:install_module` command builds and installs For example: -``` +```bash pdk bundle exec rake "litmus:install_module" ``` If you need multiple modules on the target system (e.g. fixtures pulled down through `pdk bundle exec rake spec_prep`, or a previous unit test run): -``` +```bash pdk bundle exec rake "litmus:install_modules_from_directory[spec/fixtures/modules]" ``` @@ -192,16 +193,20 @@ pdk bundle exec rake "litmus:install_modules_from_directory[spec/fixtures/module There are several options for running tests. Litmus primarily uses [serverspec](https://serverspec.org/), though you can use other testing tools. When running tests with Litmus, you can: + * Run all tests against a single target. * Run all tests against all targets in parallel. * Run a single test against a single target. An example running all tests against a single target: -``` +```bash # On Linux/MacOS: TARGET_HOST=lk8g530gzpjxogh.delivery.puppetlabs.net pdk bundle exec rspec ./spec/acceptance TARGET_HOST=localhost:2223 pdk bundle exec rspec ./spec/acceptance +``` + +```powershell # On Windows: $ENV:TARGET_HOST = 'lk8g530gzpjxogh.delivery.puppetlabs.net' pdk bundle exec rspec ./spec/acceptance @@ -209,10 +214,13 @@ pdk bundle exec rspec ./spec/acceptance An example running a specific test against a single target: -``` +```bash # On Linux/MacOS: TARGET_HOST=lk8g530gzpjxogh.delivery.puppetlabs.net pdk bundle exec rspec ./spec/acceptance/test_spec.rb:21 TARGET_HOST=localhost:2223 pdk bundle exec rspec ./spec/acceptance/test_spec.rb:21 +``` + +```powershell # On Windows: $ENV:TARGET_HOST = 'lk8g530gzpjxogh.delivery.puppetlabs.net' pdk bundle exec rspec ./spec/acceptance/test_spec.rb:21 @@ -220,13 +228,13 @@ pdk bundle exec rspec ./spec/acceptance/test_spec.rb:21 An example running all tests against all targets, as specified in the `spec/fixtures/litmus_inventory.yaml` file: -``` +```bash pdk bundle exec rake litmus:acceptance:parallel ``` An example running all tests against localhost. Note that this is only recommended if you are familiar with the code base, as tests may have unexpected side effects on your local machine. -``` +```bash pdk bundle exec rake litmus:acceptance:localhost ``` @@ -238,7 +246,7 @@ For more test examples, see [run_tests task](https://github.com/puppetlabs/provi Use the commands below to clean up provisioned systems after running tests. Specify whether to to remove an individual target or all the targets in the `spec/fixtures/litmus_inventory.yaml` file. -``` +```bash # Tear down a specific target vm pdk bundle exec rake "litmus:tear_down[c985f9svvvu95nv.delivery.puppetlabs.net]" diff --git a/docs/md/content/usage/litmus-helper-functions.md b/docs/md/content/usage/litmus-helper-functions.md index 60fdc3f5..8edefe2a 100644 --- a/docs/md/content/usage/litmus-helper-functions.md +++ b/docs/md/content/usage/litmus-helper-functions.md @@ -6,6 +6,6 @@ description: Learn all about Litmus functions. Inside of the Litmus gem, there are three distinct sets of functions: -* Rake tasks for the CLI that allows you to use the Litmus commands (provision, install an agent, install a module and run tests.). Run `pdk bundle exec rake -T` to get a list of available rake tasks. +* Rake tasks for the CLI that allows you to use the Litmus commands (provision, install an agent, install a module and run tests.). Run `pdk bundle exec rake -T` to get a list of available rake tasks. * Helper functions for serverspec / test. These apply manifests or run shell commands. For more information, see [Puppet Helpers](https://www.rubydoc.info/gems/puppet_litmus/PuppetLitmus/PuppetHelpers) * Helper Functions for Bolt inventory file manipulation. For more information, see [Inventory Manipulation](https://www.rubydoc.info/gems/puppet_litmus/PuppetLitmus/InventoryManipulation). diff --git a/docs/md/content/usage/testing/litmus-test-examples.md b/docs/md/content/usage/testing/litmus-test-examples.md index 7f584604..03bc6d70 100644 --- a/docs/md/content/usage/testing/litmus-test-examples.md +++ b/docs/md/content/usage/testing/litmus-test-examples.md @@ -88,4 +88,4 @@ gem 'pry-byebug', '> 3.4.3' ## Setting up Travis and Appveyor -To see this running on CI, enable the `use_litmus` flags for Travis CI and/or Appveyor. See the [pdk-templates docs](https://github.com/puppetlabs/pdk-templates#travisyml) for details and additional options. +To see this running on CI, enable the `use_litmus` flags for Travis CI and/or Appveyor. diff --git a/docs/md/content/usage/testing/running-acceptance-tests.md b/docs/md/content/usage/testing/running-acceptance-tests.md index c9c7118f..ed4c135d 100644 --- a/docs/md/content/usage/testing/running-acceptance-tests.md +++ b/docs/md/content/usage/testing/running-acceptance-tests.md @@ -24,57 +24,58 @@ Ensure you have installed the following: * To check Docker is working, run `docker run centos:7 ls` in your terminal. You should see a list of folders in the CentOS image. * [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) * To check where you already have git, run `git --version` in your terminal. -* [Puppet Development Kit (PDK)](https://puppet.com/docs/pdk/1.x/pdk_install.html). - * To check whether you already have PDK, run `pdk --version` from the command line. Note that you need version `1.17.0` or later. If not, then following the [installation instructions](https://puppet.com/docs/pdk/1.x/pdk_install.html). +* [Puppet Development Kit (PDK)](https://puppet.com/docs/pdk/3.x/pdk_install.html). + * To check whether you already have PDK, run `pdk --version` from the command line. Note that you need version `1.17.0` or later. If not, then following the [installation instructions](https://puppet.com/docs/pdk/3.x/pdk_install.html). ## 1. Clone the MoTD module from GitHub. From the command line, clone the Litmus branch of MoTD module: -``` + +```bash git clone https://github.com/puppetlabs/puppetlabs-motd.git ``` + You now have a local copy of the module on your machine. In this example, you can work off the master branch. Change directory to the MoTD module -``` + +```bash cd puppetlabs-motd ``` - ## 2. Install the necessary gems. The MoTD module relies on a number of gems. To install these on your machine, run the following command: -``` +```bash pdk bundle install ``` +## 3. Provision a CentOS Docker image -## 3. Provision a CentOS Docker image. +Provision a CentOS stream 9 image in a Docker container to be the target you will test against -Provision a CentOS 7 image in a Docker container to be the target you will test against +To provision the CentOS stream 9 target (or any OS of your choice), run the following command: -To provision the CentOS 7 target (or any OS of your choice), run the following command: - -``` -pdk bundle exec rake 'litmus:provision[docker, litmusimage/centos:7]' +```bash +pdk bundle exec rake 'litmus:provision[docker, litmusimage/centos:stream9]' ``` > Note: Provisioning is extensible. If your preferred provisioner is missing, let us know by raising an issue on the [provision repo](https://github.com/puppetlabs/provision/issues) or submitting a [PR](https://github.com/puppetlabs/provision/pulls). The last lines of the output should look like: -``` -Provisioning centos:7 using docker provisioner.[✔] -localhost:2222, centos:7 +```bash +Provisioning centos:stream9 using docker provisioner.[✔] +localhost:2222, centos:stream9 ``` To check that it worked, run `docker ps` and you should see output similar to: -``` +```bash CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -7b12b616cf65 centos:7 "/bin/bash" 4 minutes ago Up 4 minutes 0.0.0.0:2222->22/tcp centos_7-2222 +7b12b616cf65 centos:stream9 "/bin/bash" 4 minutes ago Up 4 minutes 0.0.0.0:2222->22/tcp centos_stream9-2222 ``` Note that the provisioned targets will be in the `spec/fixtures/litmus_inventory.yaml` file. Litmus creates this file in your working directory. If you run `cat spec/fixtures/litmus_inventory.yaml`, you should see the targets you just created. For example: @@ -98,8 +99,8 @@ groups: host-key-check: false facts: provisioner: docker - container_name: centos_7-2222 - platform: centos:7 + container_name: centos_stream9-2222 + platform: centos:stream9 - name: winrm_nodes targets: [] ``` @@ -108,25 +109,25 @@ groups: To install the latest version of Puppet agent on the CentOS Docker image, run the following command: -``` +```bash pdk bundle exec rake litmus:install_agent ``` Use Bolt to verify that you have installed the agent on the target. Run the following command: -``` +```bash pdk bundle exec bolt command run 'puppet --version' --targets localhost:2222 --inventoryfile spec/fixtures/litmus_inventory.yaml ``` Note that `localhost:2222` is the name of the node in the spec/fixtures/litmus_inventory.yaml file. You should see output with the version of the Puppet agent that was installed: -``` +```bash bolt command run 'puppet --version' --targets localhost:2222 --inventoryfile spec/fixtures/litmus_inventory.yaml ``` Running the command will produce output similar to this: -``` +```bash Started on localhost:2222... Finished on localhost:2222: STDOUT: @@ -136,17 +137,18 @@ Ran on 1 target in 1.72 sec ``` If you want to install a specific version of puppet you can use the following command: + +```bash +pdk bundle exec rake 'litmus:install_agent[puppet8-nightly] ``` -pdk bundle exec rake 'litmus:install_agent[puppet6] -``` -Examples of other versions you can pass in are: puppet6-nightly, puppet7, puppet7-nightly. +Examples of other versions you can pass in are: 'puppet', 'puppet-nightly', 'puppet7', 'puppet7-nightly'. -## 5. Install the MoTD module on the CentOS image. +## 5. Install the MoTD module on the CentOS image To install the MoTD module on the CentOS image, run the following command from inside your working directory: -``` +```bash pdk bundle exec rake litmus:install_module ``` @@ -154,10 +156,10 @@ pdk bundle exec rake litmus:install_module You will see output similar to: -``` +```bash ➜ puppetlabs-motd git:(main) pdk bundle exec rake litmus:install_module -pdk (INFO): Using Ruby 2.6.3 -pdk (INFO): Using Puppet 7.7.0 +pdk (INFO): Using Ruby 3.2.0 +pdk (INFO): Using Puppet 8.1.0 Building '/Users/paula/workspace/puppetlabs-mysql' into '/Users/paula/workspace/puppetlabs-motd/pkg' Built '/Users/paula/workspace/puppetlabs-motd/pkg/puppetlabs-motd-11.0.3.tar.gz' Installed '/Users/paula/workspace/puppetlabs-motd/pkg/puppetlabs-motd-11.0.3.tar.gz' on @@ -165,13 +167,13 @@ Installed '/Users/paula/workspace/puppetlabs-motd/pkg/puppetlabs-motd-11.0.3.tar Use Bolt to verify that you have installed the MoTD module. Run the following command: -``` +```bash pdk bundle exec bolt command run 'puppet module list' --targets localhost:2222 -i spec/fixtures/litmus_inventory.yaml ``` The output should look similar to: -``` +```bash Started on localhost... Finished on localhost: STDOUT: @@ -204,37 +206,37 @@ Note that you have also installed the MoTD module's dependent modules. To run acceptance tests with Litmus, run the following command from your working directory: -``` +```bash pdk bundle exec rake litmus:acceptance:parallel ``` This command executes the acceptance tests in the [acceptance folder](https://github.com/puppetlabs/puppetlabs-motd/tree/main/spec/acceptance) of the module. If the tests have run successfully, you will see output similar to (Note it will look like it has stalled but is actually running tests in the background, please be patient and the output will appear when the tests are complete: -``` +```bash + [✔] Running against 1 targets. -|__ [✔] localhost:2222, centos:7 +|__ [✔] localhost:2222, centos:stream9 ================ -localhost:2222, centos:7 +localhost:2222, centos:stream9 ...... Finished in 42.95 seconds (files took 10.15 seconds to load) 6 examples, 0 failures pid 1476 exit 0 -Successful on 1 nodes: ["localhost:2222, centos:7"] +Successful on 1 nodes: ["localhost:2222, centos:stream9"] ``` -## 7. Remove the Docker image. +## 7. Remove the Docker image Now that you have completed your tests, you can remove the Docker image with the Litmus tear down command: -``` +```bash pdk bundle exec rake litmus:tear_down ``` You should see JSON output, similar to: -``` +```bash localhost:2222: success ``` @@ -244,9 +246,9 @@ To verify that the target has been removed, run `docker ps` from the command lin The MoTD shows you how to use Litmus to acceptance test an existing module. As you scale up your acceptance testing, you will need to write your own acceptance tests. Try out the following: -* Provision more than one system, for example, `pdk bundle exec rake 'litmus:provision[docker, centos:6]'`. Note that you will need to re-run the `install_agent` and `install_module` command if you want to run tests. +* Provision more than one system, for example, `pdk bundle exec rake 'litmus:provision[docker, litmusimage/debian:12]'`. Note that you will need to re-run the `install_agent` and `install_module` command if you want to run tests. * Look at the inventory file and take note of the ssh connection information * ssh into the CentOS box when you know the password, for example, `ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@localhost -p 2222`, or use Bolt as shown in the example. -* ssh into the CentOS box without a password, run `docker ps`, take note of the Container Name and then run `docker exec -it litmusimage_centos_7-2222 '/bin/bash'` in this example litmusimage_centos_7-2222 is the Container Name. +* ssh into the CentOS box without a password, run `docker ps`, take note of the Container Name and then run `docker exec -it litmusimage_centos_stream9-2222 '/bin/bash'` in this example litmusimage_centos_stream9-2222 is the Container Name. -> Note: We have moved all our PR testing to public pipelines to make contributing to Puppet supported modules a better experience. Check out our [PR testing matrix](https://github.com/puppetlabs/puppetlabs-apache/pull/2141) Github Actions. All of our testing is now ran in the one place. +> Note: We have moved all our PR testing to public pipelines to make contributing to Puppet supported modules a better experience. Check out our [Github Action templates](https://github.com/puppetlabs/cat-github-actions/tree/main/.github/workflows). All of our testing is now ran in the one place.