Skip to content

Commit

Permalink
Merge pull request #178 from netdevopsbr/optimize-performance
Browse files Browse the repository at this point in the history
Optimize performance
  • Loading branch information
emersonfelipesp authored May 27, 2024
2 parents de2bb66 + 379437b commit f8032a9
Show file tree
Hide file tree
Showing 39 changed files with 1,528 additions and 135 deletions.
7 changes: 7 additions & 0 deletions docs/backend/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
## How it works

The backend made using the FastAPI framework connects with both Netbox and Proxmox (it can be many different clusters) and exposes the API REST routes that will be consumed by the Netbox Plugin (the Frontend) that is simply a Django App attached to the Netbox Django Project.

### Proxbox Architecture

![Proxbox Architecure Image](./proxbox-architecture.png)
Binary file added docs/backend/proxbox-architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
49 changes: 2 additions & 47 deletions docs/introduction.md → docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,6 @@
<img width="532" src="https://github.com/N-Multifibra/proxbox/blob/main/etc/img/proxbox-full-logo.png" alt="Proxbox logo">
</a>
<br>

<div>

### [New Documentation available!](https://proxbox.netbox.dev.br/)
</div>
<br>
</div>


Expand Down Expand Up @@ -39,9 +33,8 @@ Proxbox is currently able to get the following information from Proxmox:

---

<div align="center">

### Versions

### Version


The following table shows the Netbox and Proxmox versions compatible (tested) with Proxbox plugin.
Expand All @@ -53,44 +46,6 @@ The following table shows the Netbox and Proxmox versions compatible (tested) wi
| >= v3.0.0 < v3.2 | >= v6.2.0 | =v0.0.3 |


</div>

---

### Summary
[1. Installation](#1-installation)
- [1.1. Install package](#11-install-package)
- [1.1.1. Using pip (production use)](#111-using-pip-production-use---not-working-yet)
- [1.1.2. Using git (development use)](#112-using-git-development-use)
- [1.2. Enable the Plugin](#12-enable-the-plugin)
- [1.3. Configure Plugin](#13-configure-plugin)
- [1.3.1. Change Netbox 'configuration.py' to add PLUGIN parameters](#131-change-netbox-configurationpy-to-add-plugin-parameters)
- [1.3.2. Change Netbox 'settings.py' to include Proxbox Template directory](#132-change-netbox-settingspy-to-include-proxbox-template-directory)
- [1.4. Run Database Migrations](#14-run-database-migrations)
- [1.5 Restart WSGI Service](#15-restart-wsgi-service)

[2. Configuration Parameters](#2-configuration-parameters)

[3. Custom Fields](#3-custom-fields)
- [3.1. Custom Field Configuration](#31-custom-field-configuration)
- [3.1.1. Proxmox ID](#311-proxmox-id)
- [3.1.2. Proxmox Node](#312-proxmox-node)
- [3.1.3. Proxmox Type](#313-proxmox-type-qemu-or-lxc)
- [3.1.4. Proxmox Keep Interface](#314-proxmox-keep-interface)
- [3.2. Custom Field Example](#32-custom-field-example)

[4. Usage](#4-usage)

[5. Enable Logs](#5-enable-logs)

[6. Contributing](#6-contributing)

[7. Roadmap](#7-roadmap)

[8. Get Help from Community!](#8-get-help-from-community)

---

## 1. Installation

The instructions below detail the process for installing and enabling Proxbox plugin.
Expand Down
21 changes: 21 additions & 0 deletions docs/src/netboxbasic.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# `NetboxBase` class

<!-- ::: netbox_proxbox.backend.routes.netbox.generic.NetboxBase
options:
docstring_options:
ignore_init_summary: true
merge_init_into_class: true
show_source: false -->


nb is a dependency injection of [NetboxSessionDep][netbox_proxbox.backend.session.netbox.NetboxSessionDep]

!!! example "Teste"

=== "1"
```
teste 01
```

=== "2"
teste 02
91 changes: 57 additions & 34 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
site_name: ProxBox Documentation (NetBox Plugin)
site_name: Proxbox Docs
site_dir: ./site
site_url: https://proxbox.netbox.dev.br/
docs_dir: ./docs
Expand All @@ -13,11 +13,14 @@ theme:
palette:
- media: "(prefers-color-scheme: light)"
scheme: default
primary: black
accent: deep-orange
toggle:
icon: material/lightbulb-outline
name: Switch to Dark Mode
- media: "(prefers-color-scheme: dark)"
scheme: slate
primary: black
toggle:
icon: material/lightbulb
name: Switch to Light Mode
Expand All @@ -30,10 +33,61 @@ theme:
- navigation.path
- navigation.top
- toc.follow
- toc.integrate

plugins:
- search
- social
- mkdocstrings:
handlers:
python:
options:
extensions:
- griffe_typingdoc
show_bases: true
#show_root_heading: true
show_if_no_docstring: true
inherited_members: true
#members_order: source
#separate_signature: true
#unwrap_annotated: true
#merge_init_into_class: true
#docstring_section_style: spacy
#signature_crossrefs: true

nav:
- Proxbox: index.md
- Backend (FastAPI): 'backend/index.md'
- Source Code:
- 'src/netboxbasic.md'
- Installing & Upgrade:
- Installing Proxbox: 'installation/index.md'
- Upgrading Proxbox: 'installation/upgrading.md'
- Features:
- Virtual Machine (VM): 'features/virtual-machine.md'
- Containers (LXC): 'features/containers.md'
- Network (IPAM): 'features/network.md'
- VLAN Management: 'features/vlan-management.md'
- Storage: 'features/storage.md'
- Backup: 'features/backup.md'
- Monitoring: 'features/monitoring.md'
- Synchronized Data: 'features/synchronized-data.md'
- Background Jobs: 'features/background-jobs.md'
- API & Integration: 'features/api-integration.md'
- Configuration:
- Configuring ProxBox: 'configuration/index.md'
- Required Parameters: 'configuration/required-parameters.md'
- Data Model:
- Virtual Machine (VM): 'models/virtual-machine.md'
- Containers (LXC): 'models/containers.md'
- Others: 'models/others.md'
- Release Notes:
- Summary: 'release-notes/index.md'
- Version 0.0.6: 'release-notes/version-0.0.1.md'
- Version 0.0.5: 'release-notes/version-0.0.1.md'
- Version 0.0.4: 'release-notes/version-0.0.1.md'
- Version 0.0.3: 'release-notes/version-0.0.1.md'
- Version 0.0.2: 'release-notes/version-0.0.1.md'
- Version 0.0.1: 'release-notes/version-0.0.1.md'
extra:
social:
- icon: fontawesome/brands/github
Expand Down Expand Up @@ -65,7 +119,7 @@ markdown_extensions:
- tables
- footnotes
- pymdownx.emoji:
emoji_index: !!python/name:materialx.emoji.twemoji
emoji_index: !!python/name:material.extensions.emoji.twemoji
emoji_generator: !!python/name:materialx.emoji.to_svg
- pymdownx.superfences:
custom_fences:
Expand All @@ -76,34 +130,3 @@ markdown_extensions:
alternate_style: true
extra_javascript:
- 'https://cdn.jsdelivr.net/npm/@material-icons/iconfont/material-icons.min.js'
nav:
- Introduction: 'introduction.md'
- Installing & Upgrade:
- Installing Proxbox: 'installation/index.md'
- Upgrading Proxbox: 'installation/upgrading.md'
- Features:
- Virtual Machine (VM): 'features/virtual-machine.md'
- Containers (LXC): 'features/containers.md'
- Network (IPAM): 'features/network.md'
- VLAN Management: 'features/vlan-management.md'
- Storage: 'features/storage.md'
- Backup: 'features/backup.md'
- Monitoring: 'features/monitoring.md'
- Synchronized Data: 'features/synchronized-data.md'
- Background Jobs: 'features/background-jobs.md'
- API & Integration: 'features/api-integration.md'
- Configuration:
- Configuring ProxBox: 'configuration/index.md'
- Required Parameters: 'configuration/required-parameters.md'
- Data Model:
- Virtual Machine (VM): 'models/virtual-machine.md'
- Containers (LXC): 'models/containers.md'
- Others: 'models/others.md'
- Release Notes:
- Summary: 'release-notes/index.md'
- Version 0.0.6: 'release-notes/version-0.0.1.md'
- Version 0.0.5: 'release-notes/version-0.0.1.md'
- Version 0.0.4: 'release-notes/version-0.0.1.md'
- Version 0.0.3: 'release-notes/version-0.0.1.md'
- Version 0.0.2: 'release-notes/version-0.0.1.md'
- Version 0.0.1: 'release-notes/version-0.0.1.md'
2 changes: 1 addition & 1 deletion netbox_proxbox/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,4 @@ class ProxboxConfig(PluginConfig):

config = ProxboxConfig

from . import proxbox_api
from . import proxbox_api
9 changes: 9 additions & 0 deletions netbox_proxbox/backend/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
from netbox_proxbox.backend.routes.netbox.virtualization.cluster_type import ClusterType
from netbox_proxbox.backend.routes.netbox.virtualization.cluster import Cluster
from netbox_proxbox.backend.routes.netbox.virtualization.virtual_machines import VirtualMachine


from netbox_proxbox.backend.routes.netbox.dcim.sites import Site
from netbox_proxbox.backend.routes.netbox.dcim.device_roles import DeviceRole
from netbox_proxbox.backend.routes.netbox.dcim.device_types import DeviceType
from netbox_proxbox.backend.routes.netbox.dcim.devices import Device
26 changes: 26 additions & 0 deletions netbox_proxbox/backend/cache.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
from typing import Any

class Cache:
def __init__(self):
self.cache: dict = {}

def get(self, key: str):
result = self.cache.get(key)
if result is not None:
return result

def set(self, key: str, value: Any):
self.cache[key] = value

def delete(self, key: str):
try:
self.cache.pop(key)
except KeyError:
pass

cache = Cache()





Empty file.
8 changes: 8 additions & 0 deletions netbox_proxbox/backend/enum/netbox/dcim/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
from enum import Enum

class StatusOptions(str, Enum):
planned = "planned"
staging = "staging"
active = "active"
decommissioning = "decommissioning"
retired = "retired"
8 changes: 8 additions & 0 deletions netbox_proxbox/backend/enum/netbox/virtualization/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
from enum import Enum

class ClusterStatusOptions(str, Enum):
planned = "planned"
staging = "staging"
active = "active"
decommissioning = "decommissioning"
offline = "offline"
17 changes: 16 additions & 1 deletion netbox_proxbox/backend/exception.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
from netbox_proxbox.backend.logging import logger

class ProxboxException(Exception):
def __init__(
self,
Expand All @@ -7,4 +9,17 @@ def __init__(
):
self.message = message
self.detail = detail
self.python_exception = python_exception
self.python_exception = python_exception

log_message=f"ProxboxException: {self.message}"

if self.detail:
log_message+=f"\n > Detail: {self.detail}"

if self.python_exception:
log_message+=f"\n > Python Exception: {self.python_exception}"


logger.error(log_message)


73 changes: 73 additions & 0 deletions netbox_proxbox/backend/logging.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import logging
from logging.handlers import TimedRotatingFileHandler

# ANSI escape sequences for colors
class AnsiColorCodes:
BLACK = '\033[30m'
RED = '\033[31m'
GREEN = '\033[32m'
YELLOW = '\033[33m'
BLUE = '\033[34m'
MAGENTA = '\033[35m'
CYAN = '\033[36m'
WHITE = '\033[37m'
RESET = '\033[0m'
DARK_GRAY = '\033[90m'

class ColorizedFormatter(logging.Formatter):
LEVEL_COLORS = {
logging.DEBUG: AnsiColorCodes.BLUE,
logging.INFO: AnsiColorCodes.GREEN,
logging.WARNING: AnsiColorCodes.YELLOW,
logging.ERROR: AnsiColorCodes.RED,
logging.CRITICAL: AnsiColorCodes.MAGENTA
}

def format(self, record):
color = self.LEVEL_COLORS.get(record.levelno, AnsiColorCodes.WHITE)

record.module = f"{AnsiColorCodes.DARK_GRAY}{record.module}{AnsiColorCodes.RESET}"

record.levelname = f"{color}{record.levelname}{AnsiColorCodes.RESET}"
return super().format(record)

def setup_logger():
# Path to log file
log_path = '/var/log/proxbox.log'

# Create a logger
logger = logging.getLogger('proxbox')

logger.setLevel(logging.DEBUG)

# # Create a console handler
console_handler = logging.StreamHandler()

# Log all messages in the console
console_handler.setLevel(logging.ERROR)

# Create a formatter with colors
formatter = ColorizedFormatter('%(name)s [%(asctime)s] [%(levelname)-8s] %(module)s: %(message)s')
#formatter = logging.Formatter('[%(asctime)s] [%(levelname)s] %(module)s: %(message)s')
# Set the formatter for the console handler and file handler
console_handler.setFormatter(formatter)

# Create a file handler
file_handler = TimedRotatingFileHandler(log_path, when='midnight', interval=1, backupCount=7)

# Log only WARNINGS and above in the file
file_handler.setLevel(logging.WARNING)

# Set the formatter for the file handler
file_handler.setFormatter(formatter)

# Add the handlers to the logger
logger.addHandler(console_handler)
# logger.addHandler(file_handler)

logger.propagate = False

return logger


logger = setup_logger()
Loading

0 comments on commit f8032a9

Please sign in to comment.