From 2420865493b77ec69e70fac075491345704dbb3f Mon Sep 17 00:00:00 2001 From: Alan Gibson Date: Mon, 18 Nov 2024 18:20:07 -0500 Subject: [PATCH] Updates for clarity, content and comparibility Moved images directory into doc directory Landing page README Added supported python versions badge Added black code style badge Moved logo to doc/images Separate lines for sentences in same paragraph Moved optional for step 3 (unit test run) to top of step Fixed links Removed unnecessary Contact section Architecture Separate lines for each sentence in paragraph Changed and made consistent 'Telemetry Definintion File' Fixed links Expanded and clarified high_level_data definition Flow section redone for clarity and ease of use Doc section README Fixed logo image location --- README.md | 47 ++++--- doc/README.md | 2 +- doc/architecture.md | 129 +++++++++++------- OnAIR_logo.svg => doc/images/OnAIR_logo.svg | 0 .../images}/OnAIR_pipeline_data_flow.png | Bin 5 files changed, 108 insertions(+), 70 deletions(-) rename OnAIR_logo.svg => doc/images/OnAIR_logo.svg (100%) rename {images => doc/images}/OnAIR_pipeline_data_flow.png (100%) diff --git a/README.md b/README.md index 404887bc..2354c545 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,15 @@ +![Python Versions](https://img.shields.io/badge/python-3.8%20|%203.9%20|%203.10%20|%203.11%20|%203.12-blue.svg) +![Code Style](https://img.shields.io/badge/code%20style-black-000000.svg) ![Build](https://github.com/nasa/OnAIR/actions/workflows/unit-test.yml/badge.svg) [![CodeCov](https://codecov.io/gh/nasa/OnAIR/branch/main/graph/badge.svg?token=L0WVOTD5X9)](https://codecov.io/gh/nasa/OnAIR) -![alt text](OnAIR_logo.svg "The OnAIR logo, italicized NASA worm style font in blue and orange") +![alt text](doc/images/OnAIR_logo.svg "The OnAIR logo, italicized NASA worm style font in blue and orange") # The On-board Artificial Intelligence Research (OnAIR) Platform The On-board Artificial Intelligence Research (OnAIR) Platform is a framework that enables AI algorithms written in Python to interact with NASA's [cFS](https://github.com/nasa/cFS). -It is intended to explore research concepts in autonomous operations in a simulated environment. Featuring a plugin style architechture, it is designed to facilitate rapid prototyping of cognitive agents. +It is intended to explore research concepts in autonomous operations in a simulated environment. +Featuring a plugin style architechture, it is designed to facilitate rapid prototyping of cognitive agents. ## Key Features @@ -24,33 +27,34 @@ It is intended to explore research concepts in autonomous operations in a simula - Planners - Complex Reasoners - Flexible plugin system for cognitive components - + ## Quick Start -**NOTE:** You may need to put your specific python or version in place of `python`, e.g., `python3.12` +**NOTE:** You may need to put your specific python or version in place of `python`, e.g., `python3.12` + **NOTE:** You may need the `-m` option to run python modules, e.g., `python -m coverage report` -### 1. Clone the repository: +### 1. Clone the repository: ``` -git clone https://github.com/nasa/OnAIR.git +git clone https://github.com/nasa/OnAIR.git cd OnAIR ``` -Gets the repository and enters the directory. +Gets the repository and enters the directory. ### 2. Set up the environment: Using your local Python installation (your `pip` command may vary, e.g., `pip3.12`) ``` pip install -r requirements.txt ``` -This installs the packages necessary for testing and running. +This installs the packages necessary for testing and running. ### 3. Run the unit tests and check the coverage: +NOTE: this step is technically optional, but highly recommended. ``` python driver.py -t coverage report ``` -If all tests pass and coverage is shown to be at 100%, your setup is likely able to use OnAIR. -NOTE: this step is technically optional, but highly recommended. +If all tests pass and coverage is shown to be at 100%, your setup is likely able to use OnAIR. ### 4. Run the default configuration for OnAIR: ``` @@ -81,18 +85,21 @@ INTERPRETED SYSTEM STATUS: --- CURRENT DATA: [946707839.0, 30.0, 12.0, 200.0, 0.0, 0.0, 182.28, 0.0, 1.0] INTERPRETED SYSTEM STATUS: --- ``` -At this point it should stop. This default used a CSV file as the data source and stops at end of file. The configuration file used is [default_config.ini](https://github.com/nasa/OnAIR/blob/5d74c18fedb4b64e519ca3d63cdce6001681f190/onair/config/default_config.ini). When this runs without errors, your basic setup is complete. +At this point it should stop. +This default used a CSV file as the data source and stops at end of file. +The configuration file used is [default_config.ini](onair/config/default_config.ini). +When this runs without errors, your basic setup is complete. ### 5. Next steps: -#### [Documents and Guides](https://github.com/nasa/OnAIR/blob/5d74c18fedb4b64e519ca3d63cdce6001681f190/doc/README.md) -Information on how OnAIR is set up and operates. -#### [Initialization Files](https://github.com/nasa/OnAIR/tree/5d74c18fedb4b64e519ca3d63cdce6001681f190/onair/config) +#### [Documents and Guides](doc/README.md) +Information on how OnAIR is set up and operates. +#### [Initialization Files](onair/config) Examples of how to configure OnAIR for use. -#### [Telemetry Definition Files](https://github.com/nasa/OnAIR/tree/5d74c18fedb4b64e519ca3d63cdce6001681f190/onair/data/telemetry_configs) -Example setup files for describing the pipeline's data frame. -#### [Plugins](https://github.com/nasa/OnAIR/tree/5d74c18fedb4b64e519ca3d63cdce6001681f190/plugins) +#### [Telemetry Definition Files](onair/data/telemetry_configs) +Example setup files for describing the pipeline's data frame. +#### [Plugins](plugins) The provided cognitive components. -#### [Data Sources](https://github.com/nasa/OnAIR/tree/5d74c18fedb4b64e519ca3d63cdce6001681f190/onair/data_handling) +#### [Data Sources](onair/data_handling) Provided data handling for telemetry input. ## Contributing @@ -113,7 +120,3 @@ You may want to make draft pull requests to solicit feedback on larger changes. ## License and Copyright Please refer to [NOSA GSC-19165-1 OnAIR.pdf](NOSA%20GSC-19165-1%20OnAIR.pdf) and [COPYRIGHT](COPYRIGHT). - -## Contact - -[Placeholder for contact information] diff --git a/doc/README.md b/doc/README.md index 1d20ebde..d6462d18 100644 --- a/doc/README.md +++ b/doc/README.md @@ -1,4 +1,4 @@ -![alt text](../OnAIR_logo.svg "The OnAIR logo, italicized NASA worm style font in blue and orange") +![alt text](images/OnAIR_logo.svg "The OnAIR logo, italicized NASA worm style font in blue and orange") # Documents and Guides diff --git a/doc/architecture.md b/doc/architecture.md index 30ad283b..2e60bc48 100644 --- a/doc/architecture.md +++ b/doc/architecture.md @@ -1,11 +1,13 @@ # **WIP: Under Construction and not yet complete.** -The cognitive architecture is the backbone of OnAIR. The configuration files provide the structure that OnAIR is to use at runtime. They set up the data source, the conduit for telemetry to enter the system, and the pipeline, the series of cognitive components that run. +The cognitive architecture is the backbone of OnAIR. +The configuration files provide the structure that OnAIR is to use at runtime. +They set up the data source, the conduit for telemetry to enter the system, and the pipeline, the series of cognitive components that run. - [Configuration Files](#the-configuration-files) - [Initialization File](#initialization-ini-file) - - [Telemetry Definitions File](#telemetry-definitions-json-file) + - [Telemetry Definition File](#telemetry-definition-json-file) - [OnAirDataSource](#the-onairdatasource) - [CSV Parser](#csv-parser) - [Redis Adapter](#redis-adapter) @@ -22,12 +24,12 @@ These files define the components and data structure that will be used by a runn Provides the details about the components and other files used by OnAIR when performing its data reasoning. >#### **[FILES]** (Required) ->Pathing and file name information +>Pathing and file name information >|Key|Required?|Expected Type|Description| >|-|-|-|-| ->|TelemetryFilePath|Yes|str|location of the telemetry file on the local system| ->|TelemetryFile|Yes|str|name of the telemetry file, used by file access [OnAir data sources](#the-onairdatasource)| ->|MetaFilePath|Yes|str|location of the telemetry definitions file| +>|TelemetryFilePath|Yes|str|location of the telemetry data file on the local system| +>|TelemetryFile|Yes|str|name of the telemetry data file, used by [OnAir data sources](#the-onairdatasource) that access files| +>|MetaFilePath|Yes|str|location of the [telemetry definitions file](#telemetry-definitions-json-file)| >|MetaFile|Yes|str|name of the [telemetry definitions file](#telemetry-definitions-json-file), a JSON file| >#### **[DATA_HANDLING]** (Required) @@ -36,7 +38,7 @@ Provides the details about the components and other files used by OnAIR when per >|-|-|-|-| >|DataSourceFile|Yes|str|full path and python file name of the [OnAir data source](#the-onairdatasource) to use| ->#### **[PLUGINS]** (Required) +>#### **[PLUGINS]** (Required) >The various [plugins](#the-plugins) that OnAIR will use to construct the [pipeline](#the-pipeline) >|Key|Required?|Expected Type|Description| >|-|-|-|-| @@ -45,7 +47,10 @@ Provides the details about the components and other files used by OnAIR when per >|PlannersPluginDict|Yes|dict|Planner plugins to be used at runtime| >|ComplexPluginDict|Yes|dict|Complex Reasoner plugins to be used at runtime| > ->Each plugin dictionary item consists of the key, name for the plugin at runtime, to value, path to the plugin's base directory. Runtime plugin name is not required to be the same as the name of the plugin, but the directory used for the plugin must contain the concrete plugin class named **\_plugin.py**. Multiple of the same plugin directory may be used by defining unique key names, e.g. {"Reporter1":"plugins/reporter", "Reporter2":"plugins/reporter"} will provide two reporter plugins. A plugin type may be left empty by giving it an empty dict, **{}**. +>Each plugin dictionary item consists of the key, name for the plugin at runtime, to value, path to the plugin's base directory. +Runtime plugin name is not required to be the same as the name of the plugin, but the directory used for the plugin must contain the concrete plugin class named **\_plugin.py**. +Multiple of the same plugin directory may be used by defining unique key names, e.g. {"Reporter1":"plugins/reporter", "Reporter2":"plugins/reporter"} will provide two reporter plugins. +A plugin type may be left empty by giving it an empty dict, **{}**. >#### **[OPTIONS]** (Optional) >Provides settings for runtime @@ -53,29 +58,30 @@ Provides the details about the components and other files used by OnAIR when per >|-|-|-|-| >|IO_Enabled| No | bool | runtime text output or not, omission equates to **_false_** -### Telemetry (.json) File +### Telemetry Definition (.json) File WIP - Section incomplete - -Defined in the [initilization file](#initialization-ini-file) file as the 'Metafile.' Provides the information necessary for OnAIR to understand received data. +Defined in the [initialization file](#initialization-ini-file) as the 'Metafile.' +Provides the information necessary for OnAIR to understand received data. ## The OnAirDataSource -This is the adapter that attaches OnAIR to a telemetry source. It provides the basic data that is used by the cognitive components to render reasoning about the system. +This is the adapter that attaches OnAIR to a telemetry source. +It provides the basic data that is used by the cognitive components to render reasoning about the system. -There are a few provided by default (located in the [onair/data_handling](https://github.com/nasa/OnAIR/tree/686df367bf4b9679ee9be11524230e99a499e5f0/onair/data_handling) directory): +There are a few provided by default (located in the [onair/data_handling](../onair/data_handling) directory): - CSV - data from a .csv formatted file - Redis - data from Redis server channels -- SBN - data via the Software Bus Network attached to an instance of [NASA's core Flight Software](https://github.com/nasa/cFS "cFS on GitHub") +- SBN - data via the Software Bus Network attached to an instance of [NASA's core Flight Software](https://github.com/nasa/cFS) -However, all of these just implement the abstract class [`OnAirDataSource`](https://github.com/nasa/OnAIR/blob/686df367bf4b9679ee9be11524230e99a499e5f0/onair/data_handling/on_air_data_source.py#L4 "on_air_data_source.py") and users are able (nay, encouraged!) to write their own. +However, all of these just implement the abstract class [`OnAirDataSource`](../onair/data_handling/on_air_data_source.py) and users are able (nay, encouraged!) to write their own. -### [CSV Parser](https://github.com/nasa/OnAIR/blob/686df367bf4b9679ee9be11524230e99a499e5f0/onair/data_handling/csv_parser.py#L4 "csv_parser.py") +### [CSV Parser](../onair/data_handling/csv_parser.py) When you have telemetry data in CSV (Comma-Separated Values) format that you want to use with the OnAIR platform. #### CSV File Requirements What a file must have to be interpreted correctly. -- The first row shall contain headers that match the order from the [telemetry file](#telemetry-json-file) +- The first row shall contain headers that match the order from the [telemetry definition file](#telemetry-definition-json-file) - Each subsequent row shall contain the values that represent a single frame of data #### CSV File Considerations @@ -85,32 +91,36 @@ What should be known about file parsing and usage. - Integer - Float - Timestamp str -- All other values are unacceptable and turned into 0.0 (by the [floatify_input](https://github.com/nasa/OnAIR/blob/686df367bf4b9679ee9be11524230e99a499e5f0/onair/data_handling/parser_util.py#L49) method) +- All other values are unacceptable and turned into 0.0 (by the [floatify_input](../onair/data_handling/parser_util.py#L49) method) - Parser loads the entire floatified CSV file into memory at once. -- Memory usage increses with file size +- Memory usage increases with file size #### CSV Potential Updates Ideas for future expandability, alternative designs, or user development. -- Allow for multiple files +- Allow for multiple files - chained in sequence - dynamically selected - Line by line file access to lower memory usage -### [Redis Adapter](https://github.com/nasa/OnAIR/blob/686df367bf4b9679ee9be11524230e99a499e5f0/onair/data_handling/redis_adapter.py) +### [Redis Adapter](../onair/data_handling/redis_adapter.py) WIP - Section incomplete -### [SBN Adapter](https://github.com/nasa/OnAIR/blob/686df367bf4b9679ee9be11524230e99a499e5f0/onair/data_handling/sbn_adapter.py) +### [SBN Adapter](../onair/data_handling/sbn_adapter.py) WIP - Section incomplete ## The Pipeline This is the path in which the received data from the source will flow. -The order of plugin operations is determined by placement in the plugin list within the [initilization](#initialization-ini-file) file used when running OnAIR. Plugins run from first to last in the order they are placed within their dictionary definitions. +The order of plugin operations is determined by placement in the plugin list within the [initilization file](#initialization-ini-file) used when running OnAIR. +Plugins run from first to last in the order they are placed within their dictionary definitions. -Data received by the defined [OnAirDataSource](#the-onairdatasource) is fed into the pipeline in a by taking the currently received telemetry, referred to as the frame, which gets transferred to the pipeline into a Python list called the `low_level_data`. Each frame is like a cell in animation, with only the new data received layered on top of old data. Stagnant data is kept in the `low_level_data`, much like the background in an animated feature doesn't change; whereas current frame data overwrites any old data, like characters in the foreground moving about. The new data replaces old data, but unreceived telemetry values are retained. +Data received by the defined [OnAirDataSource](#the-onairdatasource) is fed into the pipeline in a by taking the currently received telemetry, referred to as the frame, which gets transferred to the pipeline into a Python list called the `low_level_data`. +Each frame is like a cell in animation, with only the new data received layered on top of old data. +Stagnant data is kept in the `low_level_data`, much like the background in an animated feature doesn't change; whereas current frame data overwrites any old data, like characters in the foreground moving about. +The new data replaces old data, but unreceived telemetry values are retained. -For instance: +For instance: >Our data order of telemetry is [Time, Money, Effort]. Time will increment at every step, '-' means no data. > >| Step | data | frame | low_level_data | Notes | @@ -133,48 +143,70 @@ There are two types of data used by the constructs in OnAIR, low and high level - Updated with new telemetry values from the frame at each step of operation - `high_level_data` - - dict, each key maps a plugin type (vehicle_rep, learning_systems, planning_systems, complex_systems) to a dict value where each key is a plugin runtime name that maps to the value returned by that plugin's `render_reasoning` - - built and grows as the [pipeline](#the-pipeline) is traversed + - A nested dictionary structure that stores the output of each plugin's `render_reasoning` method + - Organized by plugin types: + - 'vehicle_rep': Knowledge Representation plugins + - 'learning_systems': Learner plugins + - 'planning_systems': Planner plugins + - 'complex_systems': Complex Reasoner plugins + - For each plugin type, there's a sub-dictionary where: + - Keys are the plugin runtime names + - Values are the outputs from their respective `render_reasoning` methods + - This structure is built and expanded as the pipeline progresses ### The Plugins -There are two methods required to be implemented by OnAIR for the plugins, `update` and `render_reasoning`. -- `update` pulls data into the plugin for use, this includes either `low_level_data`, `high_level_data`, or both dependent upon the plugin type -- `render_reasoning` pushes out reasoned results data, i.e., this is the data you want made available to plugins further down the pipeline +The plugins in OnAIR operate on the data types described above. +There are two methods required to be implemented by OnAIR for the plugins: + +- `update`: Pulls data into the plugin for use. This includes either `low_level_data`, `high_level_data`, or both, depending on the plugin type. +- `render_reasoning`: Pushes out reasoned results data. +This is the data you want made available to plugins further down the pipeline. -What a plugin does with the data and how it renders its reasoning is up to the plugin developer to decide. Reasoned results can be of any type. However, the order of operations is established by the initialize file used and OnAIR's plugin flow. +What a plugin does with the data and how it renders its reasoning is up to the plugin developer to decide. +Reasoned results can be of any type. +However, the order of operations is established by the initialize file used and OnAIR's plugin flow. ### Flow +The pipeline processes data through different types of plugins in a specific order. After each plugin type runs its `render_reasoning` method, the results are stored in the `high_level_data` dictionary. For each plugin, its name becomes the key, and its returned reasoning value becomes the value in the appropriate sub-dictionary of `high_level_data`. + +The flow of data through the pipeline is as follows: + - Knowledge Representations (KR) - run first - all KRs run `update`, receiving ONLY `low_level_data` - - all KRs run `render_reasoning`, returning their respective reasonings - - all KR reasonings are put into the `high_level_data` as "\:\:\:\" + - PLs do not receive other PLs reasonings + - all PLs run `render_reasoning` + - PL reasonings are added to `high_level_data['planning_systems']` + - Complex Reasoners (CR) - run last - - each CR runs `update` and `render_resoning` receiving all KR, LN, PL, and any previous CR's reasonings in the `high_level_data` - - each CR completes resoning, returning their respective reasonings - - each CR resonings are put into the `high_level_data` as "\:\