Adding documentation for running workloads #394
Conversation
Signed-off-by: AkshathRaghav <[email protected]>
DEVELOPER_GUIDE.md
Outdated
| @@ -24,6 +24,13 @@ This document will walk you through on what's needed to start contributing code | |||
| - **Pyenv** : Install `pyenv` and follow the instructions in the output of `pyenv init` to set up your shell and restart it before proceeding. | |||
| For more details please refer to the [PyEnv installation instructions](https://github.com/pyenv/pyenv#installation). | |||
|
|
|||
| Install the following modules to continue with the next steps: | |||
There was a problem hiding this comment.
Is this specific to testing on a debian-based distro? If so, maybe we should include this at the end of the prerequisite section as optional since it is based on OS system.
There was a problem hiding this comment.
Got it. Will add!
DEVELOPER_GUIDE.md
Outdated
| @@ -74,6 +83,74 @@ This is typically created in PyCharm IDE by visiting the `Python Interpreter`, s | |||
| ` | |||
| In order to run tests within the PyCharm IDE, ensure the `Python Integrated Tools` / `Testing` / `Default Test Runner` is set to `pytest`. | |||
|
|
|||
| ## Running Workloads | |||
There was a problem hiding this comment.
Nit: Running Workloads --> Setting Up an Local OpenSearch Cluster For OSB Development (Optional)
This title should be more descriptive for a couple reasons:
- We already have documentation on the official website that details how to run workloads, so this might confuse users
- Most users seem to be using OSB to benchmark larger and external clusters instead of performance testing their local computer. This is technically optional for them as many use external OpenSearch clusters (like larger AWS managed clusters).
It might be worth including a brief explanation why developers might want to setup a local cluster (faster to setup / does not require money to setup a local cluster, quick end to end testing etc.)
DEVELOPER_GUIDE.md
Outdated
| @@ -74,6 +83,74 @@ This is typically created in PyCharm IDE by visiting the `Python Interpreter`, s | |||
| ` | |||
| In order to run tests within the PyCharm IDE, ensure the `Python Integrated Tools` / `Testing` / `Default Test Runner` is set to `pytest`. | |||
|
|
|||
| ## Running Workloads | |||
|
|
|||
| ### Installation | |||
There was a problem hiding this comment.
I'd rephrase this to OpenSearch Installation, just cause this developer guide pertains to OSB and we already have an "Installation" section
DEVELOPER_GUIDE.md
Outdated
| ``` | ||
| NOTE: Have Docker running in the background for the next steps. Refer to the installation instructions [here](https://docs.docker.com/compose/install/). | ||
|
|
||
| ### Setup |
There was a problem hiding this comment.
Would be more descriptive here to differentiate that this is for OpenSearch setup and not OSB
DEVELOPER_GUIDE.md
Outdated
| ``` | ||
| Now, you have a local cluster running! You can connect to this and run the workload for the next step. | ||
|
|
||
| ### Running the workload |
There was a problem hiding this comment.
Would be more descriptive so that this is for running an end to end test / workload on a locally installed OpenSearch cluster
|
For any new sections added, could you incorporate them into the table of contents? |
Signed-off-by: AkshathRaghav <[email protected]>
Signed-off-by: AkshathRaghav <[email protected]>
| To develop OSB properly, it is recommended that you fork the official OpenSearch Benchmark repository. | ||
| To develop OSB properly, it is recommended that you fork the official OpenSearch Benchmark repository. | ||
|
|
||
| For those working on WSL2, it is recommended to clone the repository and set up the working environment within the Linux subsystem. Refer to the guide for setting up WSL2 on [Visual Studio Code](https://code.visualstudio.com/docs/remote/wsl) or [PyCharm](https://www.jetbrains.com/help/pycharm/using-wsl-as-a-remote-interpreter.html#create-wsl-interpreter). |
There was a problem hiding this comment.
It would be good to indicate that WSL is a supported platform for development, if that is the case. Some documentation on whether it is WSL1 or WSL2 (or both) would be helpful.
There was a problem hiding this comment.
As far as I have tested, it is possible to work on both. Shall I just replace WSL2 with WSL in the documentation, or explicitly say that both are ok to work with.
There was a problem hiding this comment.
I recommend explicitly saying that both WSL and WSL2 work with OSB
| ## Setting Up a Local OpenSearch Cluster For OSB Development (Optional) | ||
|
|
||
| ### OpenSearch Installation | ||
|
|
||
| Download the latest release of OpenSearch from https://opensearch.org/downloads.html. If you are using WSL, make sure to download it into your `/home/<user>` directory instead of `/mnt/c`. | ||
| ``` | ||
| wget https://artifacts.opensearch.org/releases/bundle/opensearch/<x.x.x>/opensearch-<x.x.x>-linux-x64.tar.gz | ||
| tar -xf opensearch-x.x.x-linux-x64.tar.gz | ||
| cd opensearch-x.x.x | ||
| ``` |
There was a problem hiding this comment.
Some of this material is being updated in the doc repo https://github.com/opensearch-project/documentation-website. It would be good to consolidate the two and direct readers there.
There was a problem hiding this comment.
I just saw your PR. Shall I remove this section and just redirect to that page once the PR is merged?
There was a problem hiding this comment.
@AkshathRaghav Yes, let's remove this section and redirect it to the page once that PR is merged in. We will have the other PR merged in shortly.
There was a problem hiding this comment.
@AkshathRaghav It's published now. You can redirect the readers to this link: https://opensearch.org/docs/latest/benchmark/quickstart/#set-up-an-opensearch-cluster
| discovery.type: single-node | ||
| plugins.security.disabled: true |
There was a problem hiding this comment.
This should not be necessary, if certificate checking is eliminated. See opensearch-project/documentation-website#5177.
| @@ -100,7 +178,6 @@ Invoke integration tests by running the following command within the root direct | |||
| make it | |||
| ``` | |||
|
|
|||
| Integration tests are expected to run for approximately 20-30 mins. | |||
There was a problem hiding this comment.
Why was this line removed? Is the duration different for WSL, perhaps?
There was a problem hiding this comment.
I placed it in the line above and bolded the timeframe there. I did this because I had not seen it the first time and I was confused as to why it was taking so long.
|
Any additional changes can be added in the next iteration. |
Description
[Describe what this change achieves]Adding documentation for running workloads
Issues Resolved
#383
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.