Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PSCE-249 - Add entrypoint for trestle YAML rule transformation #55

Merged
merged 10 commits into from
Oct 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 55 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,54 @@
## Contributing
# Contributing to trestlebot

Thank you for your interest in the trestlebot project.

Before you start contributing, please take a moment to read through the guide below.

> WARNING: This project is currently under initial development. APIs may be changed incompatibly from one commit to another.

### How To Contribute

Some initial contributions could be:

- Improving the documentation
- Adding test coverage
- Try out issues that have the label `good first issue`
- Opening an issue for bugs or feature requests

## Opening a Pull Request

When submitting a pull request, please follow these guidelines:

1. Ensure that you have an issue submitted first and reference it in your pull request.
2. Ensure that your code passes all CI tests.
3. Please keep the pull request focused on a single issue or feature, if possible.

## Developer Guide

## Prerequisites

- [Python](https://www.python.org/downloads/) - v3.8+
- [Poetry](https://python-poetry.org/)
- [Podman](https://podman.io/docs/installation)

### How It Works

For workflow diagrams, see the [diagrams](./docs/diagrams/) under the `docs` folder.

#### Components

1. CI Provider - Runs or builds and runs trestle-bot container
2. Trestle Bot - Provides logic for managing workspace and containerized environment for use in workflows
3. Compliance-Trestle - Upstream library that provided core logic for how OSCAL content is managed

#### Code structure

- `actions` - Provides specific logic for `trestle-bot` tasks that are packaged as Actions. See [README.md](./actions/README.md) for more information.
- `entrypoints` - Provides top level logic for specific tasks. These tasks are not necessarily related in any so they are not organized into a hierarchical command structure, but they do inherit logic and flag from a base class.
- `provider.py, github.py, and gitlab.py` - Git provider abstract class and concrete implementations for interacting with the API.
- `tasks` - Pre-tasks can be configured before the main git logic is run. Any task that does workspace management should go here.
- `tasks/authored` - The `authored` package contains logic for managing authoring tasks for single instances of a top-level OSCAL model. These encapsulate logic from the `compliance-trestle` library and allows loose coupling between `tasks` and `authored` types.
- `transformers` - This contains data transformation logic; specifically for rules.

### Format and Styling

Expand All @@ -14,7 +64,10 @@ make test

### Run with poetry
```
poetry run trestle-bot
poetry shell
poetry install
poetry run trestlebot-autosync
poetry run trestlebot-rules-transform
```

### Local testing
Expand Down
18 changes: 12 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,17 @@ RUN python3.9 -m pip install --no-cache-dir --upgrade pip \
&& pip install --no-cache-dir poetry=="$POETRY_VERSION"

# Cache runtime deps
WORKDIR $PYSETUP_PATH
COPY ./ $PYSETUP_PATH
WORKDIR "/build"
COPY . "/build"

# Install runtime deps
RUN poetry install --without tests,dev --no-root
RUN python -m venv $VENV_PATH && \
. $VENV_PATH/bin/activate && \
poetry install --without tests,dev --no-root

# install the root project in non-editable mode
RUN poetry build -f wheel -n && \
RUN . $VENV_PATH/bin/activate && \
poetry build -f wheel -n && \
pip install --no-cache-dir --no-deps dist/*.whl && \
rm -rf dist *.egg-info

Expand All @@ -64,9 +67,12 @@ RUN microdnf install -y git \
&& microdnf clean all \
&& rm -rf /var/lib/apt/lists/*

COPY ./entrypoint.sh /
# Add wrappers for entrypoints that provide support the actions
COPY ./actions/autosync/auto-sync-entrypoint.sh /
RUN chmod +x /auto-sync-entrypoint.sh

RUN chmod +x /entrypoint.sh
COPY ./actions/rules-transform/rules-transform-entrypoint.sh /
RUN chmod +x /rules-transform-entrypoint.sh

ENTRYPOINT ["python3.9", "-m" , "trestlebot"]
CMD ["--help"]
Expand Down
94 changes: 23 additions & 71 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,94 +1,46 @@
# trestle-bot

trestle-bot assists users in leveraging [Compliance-Trestle](https://github.com/IBM/compliance-trestle) in automated workflows for [OSCAL](https://github.com/usnistgov/OSCAL) formatted compliance content management.
trestle-bot assists users in leveraging [Compliance-Trestle](https://github.com/IBM/compliance-trestle) in CI/CD workflows for [OSCAL](https://github.com/usnistgov/OSCAL) formatted compliance content management.

> WARNING: This project is currently under initial development. APIs may be changed incompatibly from one commit to another.

## Basic Configuration
## Getting Started

### GitHub Actions

```yaml
For detailed documentation on how to use each action see the README.md each each folder under [actions](./actions/).

name: Example Workflow
...
The `autosync` action will sync trestle-generated Markdown files to OSCAL JSON files in a trestle workspace. All content under the provided markdown directory when the action is run. This action supports all top-level models [supported by compliance-trestle for authoring](https://ibm.github.io/compliance-trestle/tutorials/ssp_profile_catalog_authoring/ssp_profile_catalog_authoring/).

steps:
- uses: actions/checkout@v3
- name: Run trestlebot
id: trestlebot
uses: RedHatProductSecurity/trestle-bot@main
with:
markdown_path: "markdown/profiles"
oscal_model: "profile"
```
The `rules-transform` actions can be used when managing [OSCAL Component Definitions](https://pages.nist.gov/OSCAL-Reference/models/v1.1.1/component-definition/json-outline/) in a trestle workspace. The action will transform rules defined in the rules YAML view to an OSCAL Component Definition JSON file.

## Inputs and Outputs
### GitLab CI

Checkout [`action.yml`](./action.yml) for a full list of supported inputs and outputs.
> Coming Soon

### Additional information on workflow inputs
### Run as a Container

- `markdown_path`: This is the location for Markdown generated by the `trestle author <model>-generate` commands
- `ssp_index_path`: This is a text file that stores the component definition information by name in trestle with the ssp name. Example below
Build and run the container locally:

```json
"ssp1": {
"profile": "profile1",
"component definitions": [
"comp1",
"comp2"
]
},
```bash
podman build -f Dockerfile -t trestle-bot .
podman run -v $(pwd):/data -w /data trestle-bot
```

## Action Behavior
Container images are available in `quay.io`:

The purpose of this action is to sync JSON and Markdown data with `compliance-trestle` and commit changes back to the branch or submit a pull request (if desired). Below are the main use-cases/workflows available:
```bash
podman run -v $(pwd):/data -w /data quay.io/continuouscompliance/trestle-bot:<tag>
```

- The default behavior of this action is to run a trestle `assemble` and `regenerate` tasks with the given markdown directory and model and commit the changes back to the branch the workflow ran from ( `github.ref_name` ). The branch can be changed by setting the field `branch`. If no changes exist or the changes do not exist with the file pattern set, no changes will be made and the action will exit successfully.
## Contributing

```yaml
steps:
- uses: actions/checkout@v3
- name: Run trestlebot
id: trestlebot
uses: RedHatProductSecurity/trestle-bot@main
with:
markdown_path: "markdown/profiles"
oscal_model: "profile"
branch: "another-branch"
```
For information about contributing to trestle-bot see the [CONTRIBUTING.md](./CONTRIBUTING.md) file.

- If the `target_branch` field is set, a pull request will be made using the `target_branch` as the base branch and `branch` as the head branch.

```yaml
steps:
- uses: actions/checkout@v3
- name: Run trestlebot
id: trestlebot
uses: RedHatProductSecurity/trestle-bot@main
with:
markdown_path: "markdown/profiles"
oscal_model: "profile"
branch: "autoupdate-${{ github.run_id }}"
target_branch: "main"
github_token: ${{ secret.GITHUB_TOKEN }}
```
## License

- When `check_only` is set, the trestle `assemble` and `regenerate` tasks are run and the repository is checked for changes. If changes exists, the action with exit with an error.

```yaml
steps:
- uses: actions/checkout@v3
- name: Run trestlebot
id: trestlebot
uses: RedHatProductSecurity/trestle-bot@main
with:
markdown_path: "markdown/profiles"
oscal_model: "profile"
check_only: true
```
This project is licensed under the Apache 2.0 License - see the [LICENSE.md](LICENSE) file for details.

> Note: Trestle `assemble` or `regenerate` tasks may be skipped if desired using `skip_assemble: true` or `skip_regenerate: true`, respectively.
## Troubleshooting

See `TROUBLESHOOTING.md` for additional information.
See [TROUBLESHOOTING.md](./TROUBLESHOOTING.md) for troubleshooting tips.
11 changes: 11 additions & 0 deletions actions/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Actions


Tasks in trestle-bot that we want to expose as a GitHub Action are located here.

# Adding a new Action

> Note to contributors: We are trying to limit the task that we expose as actions to workspace manage operations and checks only.

First, create an entrypoint script for the new action in the `trestlebot/entrypoints` directory. Then add the action by creating a new directory in the `actions` directory with an `action.yml` that references your new entrypoint. See the [GitHub Actions documentation](https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action) for more information.

88 changes: 88 additions & 0 deletions actions/autosync/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# trestlebot AutoSync Action

## Basic Configuration


```yaml

name: Example Workflow
...

steps:
- uses: actions/checkout@v3
- name: Run trestlebot
id: trestlebot
uses: RedHatProductSecurity/trestle-bot/actions/autosync@main
with:
markdown_path: "markdown/profiles"
oscal_model: "profile"
```

## Inputs and Outputs

Checkout [`action.yml`](./action.yml) for a full list of supported inputs and outputs.

### Additional information on workflow inputs

- `markdown_path`: This is the location for Markdown generated by the `trestle author <model>-generate` commands
- `ssp_index_path`: This is a text file that stores the component definition information by name in trestle with the ssp name. Example below

```json
"ssp1": {
"profile": "profile1",
"component definitions": [
"comp1",
"comp2"
]
},
```

## Action Behavior

The purpose of this action is to sync JSON and Markdown data with `compliance-trestle` and commit changes back to the branch or submit a pull request (if desired). Below are the main use-cases/workflows available:

- The default behavior of this action is to run a trestle `assemble` and `regenerate` tasks with the given markdown directory and model and commit the changes back to the branch the workflow ran from ( `github.ref_name` ). The branch can be changed by setting the field `branch`. If no changes exist or the changes do not exist with the file pattern set, no changes will be made and the action will exit successfully.

```yaml
steps:
- uses: actions/checkout@v3
- name: Run trestlebot
id: trestlebot
uses: RedHatProductSecurity/trestle-bot/actions/autosync@main
with:
markdown_path: "markdown/profiles"
oscal_model: "profile"
branch: "another-branch"
```

- If the `target_branch` field is set, a pull request will be made using the `target_branch` as the base branch and `branch` as the head branch.

```yaml
steps:
- uses: actions/checkout@v3
- name: Run trestlebot
id: trestlebot
uses: RedHatProductSecurity/trestle-bot/actions/autosync@main
with:
markdown_path: "markdown/profiles"
oscal_model: "profile"
branch: "autoupdate-${{ github.run_id }}"
target_branch: "main"
github_token: ${{ secret.GITHUB_TOKEN }}
```

- When `check_only` is set, the trestle `assemble` and `regenerate` tasks are run and the repository is checked for changes. If changes exists, the action with exit with an error.

```yaml
steps:
- uses: actions/checkout@v3
- name: Run trestlebot
id: trestlebot
uses: RedHatProductSecurity/trestle-bot/actions/autosync@main
with:
markdown_path: "markdown/profiles"
oscal_model: "profile"
check_only: true
```

> Note: Trestle `assemble` or `regenerate` tasks may be skipped if desired using `skip_assemble: true` or `skip_regenerate: true`, respectively.
Loading
Loading