Skip to content
This repository has been archived by the owner on May 16, 2024. It is now read-only.

Commit

Permalink
panorama webinar
Browse files Browse the repository at this point in the history
  • Loading branch information
whitej6 committed Dec 22, 2022
1 parent 30eabff commit 5104c67
Show file tree
Hide file tree
Showing 71 changed files with 9,807 additions and 0 deletions.
1 change: 1 addition & 0 deletions webinars/panorama-ssot/FAQ.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Frequently Asked Questions
349 changes: 349 additions & 0 deletions webinars/panorama-ssot/GETTING_STARTED.md

Large diffs are not rendered by default.

15 changes: 15 additions & 0 deletions webinars/panorama-ssot/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Apache Software License 2.0

Copyright (c) 2022, Network to Code, LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
199 changes: 199 additions & 0 deletions webinars/panorama-ssot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
# Nautobot SSoT Panorama SSoT

A plugin for [Nautobot](https://github.com/nautobot/nautobot).

The term SSoT, or Single Source of Truth, refers to the intention of using Nautobot to consolidate data from disparate Systems of Record to create a single resource for all automation needs. This is done by extending the [Nautobot SSoT framework](https://github.com/nautobot/nautobot-plugin-ssot) which uses the DiffSync library. This plug-in is built with the capability in mind to import and export data from your desired System of Record.

## Installation

The plugin is available as a Python package in pypi and can be installed with pip

```shell
pip install nautobot-ssot-panorama
```

> The plugin is compatible with Nautobot >= 1.4.0 and required Nautobot Firewall Models >= 1.1.0.
To ensure Nautobot SSoT Panorama is automatically re-installed during future upgrades, create a file named `local_requirements.txt` (if not already existing) in the Nautobot root directory (alongside `requirements.txt`) and list the `nautobot-ssot-panorama` package:

```no-highlight
# echo nautobot-ssot-panorama >> local_requirements.txt
```

Once installed, the plugin needs to be enabled in your `nautobot_config.py`

```python
# In your nautobot_config.py
PLUGINS = ["nautobot_ssot", "nautobot_ssot_panorama", "nautobot_firewall_models"]

# Plugins configuration settings. These settings are used by various plugins that the user may have installed.
# Each key in the dictionary is the name of an installed plugin and its value is a dictionary of settings.
PLUGINS_CONFIG = {
"nautobot_ssot": {
"hide_example_jobs": True,
},
"nautobot_ssot_panorama": {
"panorama_url": os.getenv("NAUTOBOT_PANORAMA_URL"),
"panorama_user": os.getenv("NAUTOBOT_PANORAMA_USER"),
"panorama_pwd": os.getenv("NAUTOBOT_PANORAMA_PWD"),
},
}
```

The plugin behavior can be controlled with the following list of settings

- TODO

Databases supported:

- Postgres
- Mysql

## Usage

### API

TODO

## Contributing

Pull requests are welcomed and automatically built and tested against multiple version of Python and multiple version of Nautobot through TravisCI.

The project is packaged with a light development environment based on `docker-compose` to help with the local development of the project and to run the tests within TravisCI.

The project is following Network to Code software development guideline and is leveraging:

- Black, Pylint, Bandit and pydocstyle for Python linting and formatting.
- Django unit test to ensure the plugin is working properly.

### Development Environment

The development environment can be used in 2 ways. First, with a local poetry environment if you wish to develop outside of Docker with the caveat of using external services provided by Docker for PostgresQL and Redis. Second, all services are spun up using Docker and a local mount so you can develop locally, but Nautobot is spun up within the Docker container.

Below is a quick start guide if you're already familiar with the development environment provided, but if you're not familiar, please read the [Getting Started Guide](GETTING_STARTED.md).

#### Invoke

The [PyInvoke](http://www.pyinvoke.org/) library is used to provide some helper commands based on the environment. There are a few configuration parameters which can be passed to PyInvoke to override the default configuration:

- `nautobot_ver`: the version of Nautobot to use as a base for any built docker containers (default: latest)
- `project_name`: the default docker compose project name (default: nautobot_ssot_panorama)
- `python_ver`: the version of Python to use as a base for any built docker containers (default: 3.7)
- `local`: a boolean flag indicating if invoke tasks should be run on the host or inside the docker containers (default: False, commands will be run in docker containers)
- `compose_dir`: the full path to a directory containing the project compose files
- `compose_files`: a list of compose files applied in order (see [Multiple Compose files](https://docs.docker.com/compose/extends/#multiple-compose-files) for more information)

Using **PyInvoke** these configuration options can be overridden using [several methods](http://docs.pyinvoke.org/en/stable/concepts/configuration.html). Perhaps the simplest is simply setting an environment variable `INVOKE_NAUTOBOT_SSOT_PANORAMA_VARIABLE_NAME` where `VARIABLE_NAME` is the variable you are trying to override. The only exception is `compose_files`, because it is a list it must be overridden in a yaml file. There is an example `invoke.yml` (`invoke.example.yml`) in this directory which can be used as a starting point.

#### Local Poetry Development Environment

1. Copy `development/creds.example.env` to `development/creds.env` (This file will be ignored by Git and Docker)
2. Uncomment the `NAUTOBOT_DB_HOST`, `NAUTOBOT_REDIS_HOST`, and `NAUTOBOT_CONFIG` variables in `development/creds.env`
3. Create an `invoke.yml` file with the following contents at the root of the repo (you can also `cp invoke.example.yml invoke.yml` and edit as necessary):

```yaml
---
{ { cookiecutter.plugin_name } }:
local: true
compose_files:
- "docker-compose.requirements.yml"
```
4. Run the following commands:
```shell
poetry shell
poetry install --extras nautobot
export $(cat development/dev.env | xargs)
export $(cat development/creds.env | xargs)
invoke build
invoke start && sleep 5
nautobot-server migrate
```

> If you want to develop on the latest develop branch of Nautobot, run the following command: `poetry add --optional git+https://github.com/nautobot/nautobot@develop`. After the `@` symbol must match either a branch or a tag.
5. You can now run nautobot-server commands as you would from the [Nautobot documentation](https://nautobot.readthedocs.io/en/latest/) for example to start the development server:

```shell
nautobot-server runserver 0.0.0.0:8080 --insecure
```

Nautobot server can now be accessed at [http://localhost:8080](http://localhost:8080).

It is typically recommended to launch the Nautobot **runserver** command in a separate shell so you can keep developing and manage the webserver separately.

#### Docker Development Environment

This project is managed by [Python Poetry](https://python-poetry.org/) and has a few requirements to setup your development environment:

1. Install Poetry, see the [Poetry Documentation](https://python-poetry.org/docs/#installation) for your operating system.
2. Install Docker, see the [Docker documentation](https://docs.docker.com/get-docker/) for your operating system.

Once you have Poetry and Docker installed you can run the following commands to install all other development dependencies in an isolated python virtual environment:

```shell
poetry shell
poetry install
invoke build
invoke start
```

Nautobot server can now be accessed at [http://localhost:8080](http://localhost:8080).

To either stop or destroy the development environment use the following options.

- **invoke stop** - Stop the containers, but keep all underlying systems intact
- **invoke destroy** - Stop and remove all containers, volumes, etc. (This results in data loss due to the volume being deleted)

### CLI Helper Commands

The project is coming with a CLI helper based on [invoke](http://www.pyinvoke.org/) to help setup the development environment. The commands are listed below in 3 categories `dev environment`, `utility` and `testing`.

Each command can be executed with `invoke <command>`. Environment variables `INVOKE_NAUTOBOT_SSOT_PANORAMA_PYTHON_VER` and `INVOKE_NAUTOBOT_SSOT_PANORAMA_NAUTOBOT_VER` may be specified to override the default versions. Each command also has its own help `invoke <command> --help`

#### Docker dev environment

```no-highlight
build Build all docker images.
debug Start Nautobot and its dependencies in debug mode.
destroy Destroy all containers and volumes.
restart Restart Nautobot and its dependencies.
start Start Nautobot and its dependencies in detached mode.
stop Stop Nautobot and its dependencies.
```

#### Utility

```no-highlight
cli Launch a bash shell inside the running Nautobot container.
create-user Create a new user in django (default: admin), will prompt for password.
makemigrations Run Make Migration in Django.
nbshell Launch a nbshell session.
shell-plus Launch a shell_plus session, which uses iPython and automatically imports all models.
```

#### Testing

```no-highlight
bandit Run bandit to validate basic static code security analysis.
black Run black to check that Python files adhere to its style standards.
flake8 This will run flake8 for the specified name and Python version.
pydocstyle Run pydocstyle to validate docstring formatting adheres to NTC defined standards.
pylint Run pylint code analysis.
tests Run all tests for this plugin.
unittest Run Django unit tests for the plugin.
```

### Project Documentation

Project documentation is generated by [mkdocs](https://www.mkdocs.org/) from the documentation located in the docs folder. You can configure [readthedocs.io](https://readthedocs.io/) to point at this folder in your repo. A container hosting the docs will be started using the invoke commands on [http://localhost:8001](http://localhost:8001), as changes are saved the docs will be automatically reloaded.

## Questions

For any questions or comments, please check the [FAQ](FAQ.md) first and feel free to swing by the [Network to Code slack channel](https://networktocode.slack.com/) (channel #networktocode).
Sign up [here](http://slack.networktocode.com/)

## Screenshots

TODO
41 changes: 41 additions & 0 deletions webinars/panorama-ssot/development/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
ARG NAUTOBOT_VER="1.4.0"
ARG PYTHON_VER=3.8
FROM ghcr.io/nautobot/nautobot-dev:${NAUTOBOT_VER}-py${PYTHON_VER}

ENV prometheus_multiproc_dir=/prom_cache

ARG NAUTOBOT_ROOT=/opt/nautobot

ENV NAUTOBOT_ROOT ${NAUTOBOT_ROOT}

WORKDIR $NAUTOBOT_ROOT

# Configure poetry
RUN poetry config virtualenvs.create false \
&& poetry config installer.parallel false

# -------------------------------------------------------------------------------------
# Install Nautobot Plugin
# -------------------------------------------------------------------------------------
WORKDIR /tmp/install

# Copy in only pyproject.toml/poetry.lock to help with caching this layer if no updates to dependencies
COPY poetry.lock pyproject.toml /tmp/install/

# Add the requested Nautobot version to pyproject
# to install the correct version based on the NAUTOBOT_VER argument
# Otherwise Poetry will override the version in this container
# with the one in the poetry.lock
RUN poetry add nautobot=${NAUTOBOT_VER}

# --no-root declares not to install the project package since we're wanting to take advantage of caching dependency installation
# and the project is copied in and installed after this step
RUN poetry install --no-interaction --no-ansi --no-root

# Copy in the rest of the source code and install local Nautobot plugin
WORKDIR /source
COPY . /source
RUN cp /tmp/install/* /source/
RUN poetry install --no-interaction --no-ansi

COPY development/nautobot_config.py ${NAUTOBOT_ROOT}/nautobot_config.py
36 changes: 36 additions & 0 deletions webinars/panorama-ssot/development/creds.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
################################################################################
# CREDS File: Store private information. Copied to creds.env and always ignored
################################################################################
# Nautobot Configuration Secret Items
NAUTOBOT_CREATE_SUPERUSER=true
NAUTOBOT_DB_PASSWORD=changeme
NAUTOBOT_NAPALM_USERNAME=''
NAUTOBOT_NAPALM_PASSWORD=''
NAUTOBOT_REDIS_PASSWORD=changeme
NAUTOBOT_SECRET_KEY='changeme'
NAUTOBOT_SUPERUSER_NAME=admin
NAUTOBOT_SUPERUSER_EMAIL=[email protected]
NAUTOBOT_SUPERUSER_PASSWORD=admin
NAUTOBOT_SUPERUSER_API_TOKEN=0123456789abcdef0123456789abcdef01234567
NAUTOBOT_CACHEOPS_ENABLED=False

# Panorama
NAUTOBOT_PANORAMA_URL='panorama.demo.networktocode.com'
NAUTOBOT_PANORAMA_USER='admin'
NAUTOBOT_PANORAMA_PWD='ay3BYQvNNuh72denCuBRV72p9Q9uyTsqtqDFtsL68Lrb3QPZAM'
NAUTOBOT_PANORAMA_API_KEY='changeme'
NAUTOBOT_PANORAMA_PORT='443'
NAUTOBOT_PANORAMA_VERIFY='True'

# Postgres
POSTGRES_PASSWORD=${NAUTOBOT_DB_PASSWORD}
PGPASSWORD=${NAUTOBOT_DB_PASSWORD}

# MySQL Credentials
MYSQL_ROOT_PASSWORD=${NAUTOBOT_DB_PASSWORD}
MYSQL_PASSWORD=${NAUTOBOT_DB_PASSWORD}

# Use these to override values in development.env
# NAUTOBOT_DB_HOST=localhost
# NAUTOBOT_REDIS_HOST=localhost
# NAUTOBOT_CONFIG=development/nautobot_config.py
34 changes: 34 additions & 0 deletions webinars/panorama-ssot/development/creds.example.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
################################################################################
# CREDS File: Store private information. Copied to creds.env and always ignored
################################################################################
# Nautobot Configuration Secret Items
NAUTOBOT_CREATE_SUPERUSER=true
NAUTOBOT_DB_PASSWORD=changeme
NAUTOBOT_NAPALM_USERNAME=''
NAUTOBOT_NAPALM_PASSWORD=''
NAUTOBOT_REDIS_PASSWORD=changeme
NAUTOBOT_SECRET_KEY='changeme'
NAUTOBOT_SUPERUSER_NAME=admin
NAUTOBOT_SUPERUSER_EMAIL=[email protected]
NAUTOBOT_SUPERUSER_PASSWORD=admin
NAUTOBOT_SUPERUSER_API_TOKEN=0123456789abcdef0123456789abcdef01234567
NAUTOBOT_CACHEOPS_ENABLED=False

# Panorama
NAUTOBOT_PANORAMA_URL='changeme.foo.com'
NAUTOBOT_PANORAMA_USER='changeme'
NAUTOBOT_PANORAMA_PWD='changeme'
NAUTOBOT_PANORAMA_API_KEY='changeme'

# Postgres
POSTGRES_PASSWORD=${NAUTOBOT_DB_PASSWORD}
PGPASSWORD=${NAUTOBOT_DB_PASSWORD}

# MySQL Credentials
MYSQL_ROOT_PASSWORD=${NAUTOBOT_DB_PASSWORD}
MYSQL_PASSWORD=${NAUTOBOT_DB_PASSWORD}

# Use these to override values in development.env
# NAUTOBOT_DB_HOST=localhost
# NAUTOBOT_REDIS_HOST=localhost
# NAUTOBOT_CONFIG=development/nautobot_config.py
40 changes: 40 additions & 0 deletions webinars/panorama-ssot/development/development.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
################################################################################
# DEV File: Store environment information. NOTE: Secrets NOT stored here!
################################################################################
# Nautobot Configuration Environment Variables
NAUTOBOT_ALLOWED_HOSTS=*
NAUTOBOT_BANNER_TOP="Local"
NAUTOBOT_CHANGELOG_RETENTION=0

NAUTOBOT_DEBUG=True
NAUTOBOT_DJANGO_EXTENSIONS_ENABLED=True
NAUTOBOT_DJANGO_TOOLBAR_ENABLED=False
NAUTOBOT_LOG_LEVEL=DEBUG
NAUTOBOT_METRICS_ENABLED=True
NAUTOBOT_NAPALM_TIMEOUT=5
NAUTOBOT_MAX_PAGE_SIZE=0

# Redis Configuration Environment Variables
NAUTOBOT_REDIS_HOST=redis
NAUTOBOT_REDIS_PORT=6379
# Uncomment NAUTOBOT_REDIS_SSL if using SSL
# NAUTOBOT_REDIS_SSL=True

# Nautobot DB Connection Environment Variables
NAUTOBOT_DB_NAME=nautobot
NAUTOBOT_DB_USER=nautobot
NAUTOBOT_DB_HOST=db
NAUTOBOT_DB_TIMEOUT=300

# Use them to overwrite the defaults in nautobot_config.py
# NAUTOBOT_DB_ENGINE=django.db.backends.postgresql
# NAUTOBOT_DB_PORT=5432

# Needed for Postgres should match the values for Nautobot above
POSTGRES_USER=${NAUTOBOT_DB_USER}
POSTGRES_DB=${NAUTOBOT_DB_NAME}

# Needed for MYSQL should match the values for Nautobot above
MYSQL_USER=${NAUTOBOT_DB_USER}
MYSQL_DATABASE=${NAUTOBOT_DB_NAME}
MYSQL_ROOT_HOST=%
4 changes: 4 additions & 0 deletions webinars/panorama-ssot/development/development_mysql.env
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Custom ENVs for Mysql
# Due to docker image limitations for Mysql, we need "root" user to create more than one database table
NAUTOBOT_DB_USER=root
MYSQL_USER=${NAUTOBOT_DB_USER}
Loading

0 comments on commit 5104c67

Please sign in to comment.