cleanup time!

carlmontanari · Sep 18, 2019 · f835a95 · f835a95
1 parent cc8f20b
commit f835a95
Show file tree

Hide file tree

Showing 21 changed files with 714 additions and 493 deletions.
diff --git a/.gitignore b/.gitignore
@@ -73,3 +73,7 @@ dmypy.json
 # macos stuff
 .DS_Store
 */.DS_Store
+
+# nornir (or any other) log files from testing
+.log
+*/.log
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,3 @@
+# Contributing
+
+Contributions are not expected, but are very welcome! Feel free to open PRs or Issues as needed. Any contributions would need to at a minimum successfully pass the GitHub Actions commit build (which basically just runs tox).
diff --git a/Makefile b/Makefile
@@ -1,4 +1,4 @@
-test_unit:
+tests:
 	python -m pytest \
 	--cov=nornsible \
 	--cov-report html \

diff --git a/README.md b/README.md
@@ -8,3 +8,131 @@ nornsible
 =======
 
 Wrap Nornir with Ansible-like host/group limits and tagging!
+
+The idea behind nornsible is to allow for Nornir scripts to be written to operate on an entire environment, and then limited to a subset of host(s) based on simple command line arguments. Of course you can simply do this yourself, but why not let nornsible handle it for you!
+
+Nornsible provides the ability to -- via command line arguments -- filter Nornir inventories by hostname or group name, or both. There is also a handy flag to set the number of workers; quite useful for setting workers to 1 for troubleshooting purposes.
+
+Lastly nornsible supports the concept of tags. Tags correspond to the name of *custom* tasks and operate very much like tags in Ansible. Provide a list of tags to execute, and Nornsible will ensure that Nornir only runs those tasks. Provide a list of tags to skip, and Nornsible will ensure that Nornir only runs those not in that list. Easy peasy.
+
+
+# How does nornsible work?
+
+Nornsible accepts an instantiated Nornir object as an argument and returns a slightly Nornir object. Nornsible sets the desired number of workers if applicable, and addts an attribute for "run_tags" and "skip_tags" based on your command line input.
+
+To take advantage of the tags feautre Nornsible provides a decorator that you can use to decorate your custom tasks. This decorator inspects the task being ran and checks the task name against the lists of run and skip tags. If the task is allowed, Nornsible simply allows the task to run as per normal, if it is *not* allowed, Nornsible will print a pretty message and move on.
+
+
+# Caveats
+
+Nornsible breaks some things! Most notably it breaks "normal" Nornir filtering *after* the Nornir object is "nornsible-ified". This can probably be fixed but at the moment it doesn't seem like that big a deal, so I'm not bothering!
+
+If you want to do "normal" Nornir filtering -- do this *before* passing the nornir object to Nornsible.
+
+Nornsible, at the moment, can only wrap custom tasks. This can probably be imporved upon as well, but at the moment the decorator wrapping custom tasks solution seems to work pretty well.
+
+
+# Installation
+
+Installation via pip:
+
+```
+pip install nornsible
+```
+
+To install from this repository:
+
+```
+pip install git+https://github.com/carlmontanari/nornsible
+```
+
+To install from source:
+
+```
+git clone https://github.com/carlmontanari/nornsible
+cd nornsible
+python setup.py install
+```
+
+
+# Usage
+
+Import stuff:
+
+```
+from nornsible import InitNornsible, nornsible_task
+```
+
+Decorate custom tasks with `nornsible_task` if desired:
+
+```
+@nornsible_task
+def my_custom_task(task):
+```
+
+Create your Nornir object and then pass it through InitNornsible:
+
+```
+nr = InitNornir(config_file="config.yaml")
+nr = Init_Nornsible(nr)
+```
+
+Run a custom task wrapped by `nornsible_task`:
+
+```
+nr.run(task=my_custom_task)
+```
+
+Run your script with any of the following command line switches:
+
+| Purpose          | Short Flag    | Long Flag  | Allowed Options
+| -----------------|---------------|------------|-------------------|
+| set num_workers  | -w            | --workers  | integer           |
+| limit host(s)    | -l            | --limit    | comma sep string  |
+| limit group(s)   | -g            | --groups   | comma sep string  |
+| run tag(s)       | -t            | --tags     | comma sep string  |
+| skip tag(s)      | -s            | --skip     | comma sep string  |
+
+To set number of workers to 1 for troubleshooting purposes:
+
+```
+python my_nornir_script.py -w 1
+```
+
+To limit to the "sea" group (from your Nornir inventory):
+
+```
+python my_nornir_script.py -g sea
+```
+
+To run only the tasks named "create_configs" and "deploy_configs" (assuming you've wrapped all of your tasks with `nornsible_task`!):
+
+```
+python my_nornir_script.py -t create_configs,deploy_configs
+```
+
+To run only the tasks named "create_configs" and "deploy_configs" against the "sea-eos-1" host:
+
+```
+python my_nornir_script.py -t create_configs,deploy_configs -l sea-eos-1
+```
+
+
+# FAQ
+
+TBA, probably things though!
+
+# Linting and Testing
+
+## Linting
+
+This project uses [black](https://github.com/psf/black) for auto-formatting. In addition to black, tox will execute [pylama](https://github.com/klen/pylama), and [pydocstyle](https://github.com/PyCQA/pydocstyle) for linting purposes. I've also added docstring linting with [darglint](https://github.com/terrencepreilly/darglint) which has been quite handy!
+
+## Testing
+
+I broke testing into two main categories -- unit and integration. Unit is what you would expect -- unit testing the code. Integration testing is for things that test more than one "unit" (generally function) at a time.
+
+
+# To Do
+
+- Add handling for "not" in host/group limit; i.e.: "-t !localhost" to run against all hosts *not* localhost.
diff --git a/docs/nornsible/index.html b/docs/nornsible/index.html
@@ -28,16 +28,24 @@ <h1 class="title">Module <code>nornsible</code></h1>
 from logging import NullHandler
 
 from nornsible.nornsible import (
-    Init_Nornsible,
+    InitNornsible,
     parse_cli_args,
-    process_tags,
+    nornsible_task,
     patch_config,
     patch_inventory,
+    nornsible_task_message,
 )
 
 
 __version__ = &#34;2019.09.16&#34;
-__all__ = (&#34;Init_Nornsible&#34;, &#34;parse_cli_args&#34;, &#34;process_tags&#34;, &#34;patch_config&#34;, &#34;patch_inventory&#34;)
+__all__ = (
+    &#34;InitNornsible&#34;,
+    &#34;parse_cli_args&#34;,
+    &#34;nornsible_task&#34;,
+    &#34;patch_config&#34;,
+    &#34;patch_inventory&#34;,
+    &#34;nornsible_task_message&#34;,
+)
 
 
 # Setup logger
@@ -59,8 +67,8 @@ <h2 class="section-title" id="header-submodules">Sub-modules</h2>
 <section>
 <h2 class="section-title" id="header-functions">Functions</h2>
 <dl>
-<dt id="nornsible.Init_Nornsible"><code class="name flex">
-<span>def <span class="ident">Init_Nornsible</span></span>(<span>nr)</span>
+<dt id="nornsible.InitNornsible"><code class="name flex">
+<span>def <span class="ident">InitNornsible</span></span>(<span>nr)</span>
 </code></dt>
 <dd>
 <section class="desc"><p>Patch nornir object based on cli arguments</p>
@@ -82,7 +90,7 @@ <h2 id="raises">Raises</h2>
 </dl></section>
 <details class="source">
 <summary>Source code</summary>
-<pre><code class="python">def Init_Nornsible(nr):
+<pre><code class="python">def InitNornsible(nr):
     &#34;&#34;&#34;
     Patch nornir object based on cli arguments
 
@@ -109,6 +117,105 @@ <h2 id="raises">Raises</h2>
     return nr</code></pre>
 </details>
 </dd>
+<dt id="nornsible.nornsible_task"><code class="name flex">
+<span>def <span class="ident">nornsible_task</span></span>(<span>wrapped_func)</span>
+</code></dt>
+<dd>
+<section class="desc"><p>Decorate an "operation" &ndash; execute or skip the operation based on tags</p>
+<h2 id="args">Args</h2>
+<dl>
+<dt><strong><code>wrapped_func</code></strong></dt>
+<dd>function to wrap in tag processor</dd>
+</dl>
+<h2 id="returns">Returns</h2>
+<dl>
+<dt><strong><code>tag_wrapper</code></strong></dt>
+<dd>wrapped function</dd>
+</dl>
+<h2 id="raises">Raises</h2>
+<dl>
+<dt><code>N</code>/<code>A</code>
+# <code>noqa</code></dt>
+<dd>&nbsp;</dd>
+</dl></section>
+<details class="source">
+<summary>Source code</summary>
+<pre><code class="python">def nornsible_task(wrapped_func):
+    &#34;&#34;&#34;
+    Decorate an &#34;operation&#34; -- execute or skip the operation based on tags
+
+    Args:
+        wrapped_func: function to wrap in tag processor
+
+    Returns:
+        tag_wrapper: wrapped function
+
+    Raises:
+        N/A  # noqa
+
+    &#34;&#34;&#34;
+
+    def tag_wrapper(task, *args, **kwargs):
+        if {wrapped_func.__name__}.intersection(task.nornir.skip_tags):
+            msg = f&#34;---- {task.host} skipping task {wrapped_func.__name__} &#34;
+            nornsible_task_message(msg)
+            return Result(host=task.host, result=&#34;Task skipped!&#34;, failed=False, changed=False)
+        if not task.nornir.run_tags:
+            return wrapped_func(task, *args, **kwargs)
+        if {wrapped_func.__name__}.intersection(task.nornir.run_tags):
+            return wrapped_func(task, *args, **kwargs)
+        msg = f&#34;---- {task.host} skipping task {wrapped_func.__name__} &#34;
+        nornsible_task_message(msg)
+        return Result(host=task.host, result=&#34;Task skipped!&#34;, failed=False, changed=False)
+
+    tag_wrapper.__name__ = wrapped_func.__name__
+    return tag_wrapper</code></pre>
+</details>
+</dd>
+<dt id="nornsible.nornsible_task_message"><code class="name flex">
+<span>def <span class="ident">nornsible_task_message</span></span>(<span>msg)</span>
+</code></dt>
+<dd>
+<section class="desc"><p>Handle printing pretty messages for nornsible_task decorator</p>
+<h2 id="args">Args</h2>
+<dl>
+<dt><strong><code>msg</code></strong></dt>
+<dd>message to beautifully print to stdout</dd>
+</dl>
+<h2 id="returns">Returns</h2>
+<dl>
+<dt><code>N</code>/<code>A</code></dt>
+<dd>&nbsp;</dd>
+</dl>
+<h2 id="raises">Raises</h2>
+<dl>
+<dt><code>N</code>/<code>A</code>
+# <code>noqa</code></dt>
+<dd>&nbsp;</dd>
+</dl></section>
+<details class="source">
+<summary>Source code</summary>
+<pre><code class="python">def nornsible_task_message(msg):
+    &#34;&#34;&#34;
+    Handle printing pretty messages for nornsible_task decorator
+
+    Args:
+        msg: message to beautifully print to stdout
+
+    Returns:
+         N/A
+
+    Raises:
+        N/A  # noqa
+
+    &#34;&#34;&#34;
+    LOCK.acquire()
+    try:
+        print(f&#34;{Style.BRIGHT}{Back.CYAN}{Fore.WHITE}{msg}{&#39;-&#39; * (80 - len(msg))}&#34;)
+    finally:
+        LOCK.release()</code></pre>
+</details>
+</dd>
 <dt id="nornsible.parse_cli_args"><code class="name flex">
 <span>def <span class="ident">parse_cli_args</span></span>(<span>args)</span>
 </code></dt>
@@ -295,61 +402,6 @@ <h2 id="raises">Raises</h2>
     return inv</code></pre>
 </details>
 </dd>
-<dt id="nornsible.process_tags"><code class="name flex">
-<span>def <span class="ident">process_tags</span></span>(<span>wrapped_func)</span>
-</code></dt>
-<dd>
-<section class="desc"><p>Decorate an "operation" &ndash; execute or skip the operation based on tags</p>
-<h2 id="args">Args</h2>
-<dl>
-<dt><strong><code>wrapped_func</code></strong></dt>
-<dd>function to wrap in tag processor</dd>
-</dl>
-<h2 id="returns">Returns</h2>
-<dl>
-<dt><strong><code>tag_wrapper</code></strong></dt>
-<dd>wrapped function</dd>
-</dl>
-<h2 id="raises">Raises</h2>
-<dl>
-<dt><code>N</code>/<code>A</code>
-# <code>noqa</code></dt>
-<dd>&nbsp;</dd>
-</dl></section>
-<details class="source">
-<summary>Source code</summary>
-<pre><code class="python">def process_tags(wrapped_func):
-    &#34;&#34;&#34;
-    Decorate an &#34;operation&#34; -- execute or skip the operation based on tags
-
-    Args:
-        wrapped_func: function to wrap in tag processor
-
-    Returns:
-        tag_wrapper: wrapped function
-
-    Raises:
-        N/A  # noqa
-
-    &#34;&#34;&#34;
-
-    def tag_wrapper(task, *args, **kwargs):
-        if set([wrapped_func.__name__]).intersection(task.nornir.skip_tags):
-            msg = f&#34;---- {task.host} skipping task {wrapped_func.__name__} &#34;
-            process_tags_messages(msg)
-            return Result(host=task.host, result=&#34;Task skipped!&#34;, failed=False, changed=False)
-        if not task.nornir.run_tags:
-            return wrapped_func(task, *args, **kwargs)
-        if set([wrapped_func.__name__]).intersection(task.nornir.run_tags):
-            return wrapped_func(task, *args, **kwargs)
-        msg = f&#34;---- {task.host} skipping task {wrapped_func.__name__} &#34;
-        process_tags_messages(msg)
-        return Result(host=task.host, result=&#34;Task skipped!&#34;, failed=False, changed=False)
-
-    tag_wrapper.__name__ = wrapped_func.__name__
-    return tag_wrapper</code></pre>
-</details>
-</dd>
 </dl>
 </section>
 <section>
@@ -368,11 +420,12 @@ <h1>Index</h1>
 </li>
 <li><h3><a href="#header-functions">Functions</a></h3>
 <ul class="">
-<li><code><a title="nornsible.Init_Nornsible" href="#nornsible.Init_Nornsible">Init_Nornsible</a></code></li>
+<li><code><a title="nornsible.InitNornsible" href="#nornsible.InitNornsible">InitNornsible</a></code></li>
+<li><code><a title="nornsible.nornsible_task" href="#nornsible.nornsible_task">nornsible_task</a></code></li>
+<li><code><a title="nornsible.nornsible_task_message" href="#nornsible.nornsible_task_message">nornsible_task_message</a></code></li>
 <li><code><a title="nornsible.parse_cli_args" href="#nornsible.parse_cli_args">parse_cli_args</a></code></li>
 <li><code><a title="nornsible.patch_config" href="#nornsible.patch_config">patch_config</a></code></li>
 <li><code><a title="nornsible.patch_inventory" href="#nornsible.patch_inventory">patch_inventory</a></code></li>
-<li><code><a title="nornsible.process_tags" href="#nornsible.process_tags">process_tags</a></code></li>
 </ul>
 </li>
 </ul>