Skip to content

Latest commit

 

History

History
486 lines (342 loc) · 20.3 KB

IoTConnect.md

File metadata and controls

486 lines (342 loc) · 20.3 KB
Raw

IoTConnect Instructions

Introduction

The RASynBoard is architected for ML at the Edge. We can use ML inference events/detections to drive local hardware such as a GPIO to shut down a motor that is about to harm itself. But how does a service team know that there's a problem with the motor? Without cloud connectivity, the service team must physically put eyes on the motor or the motor control panel to see that there is a problem. This is where cloud connectivity and the Internet of Things (IoT) can help. We can connect our board to a cloud solution and not only periodically send data detailing the health of the motor (or whatever we're monitoring), but we can also send up critical events. On the cloud side we can monitor for critical events and when they occur, we can programmatically generate a service ticket, or shoot off an email or text to the service technician informing them of the issue.

This Avnet RASynBoard OOB application supports connecting your device to either AWS IoT Core, or Avnet's IoTConnect on AWS. This tutorial will guide you through all the steps required to send ML events up to the cloud and even show them on a dashboard in near real-time.

This document includes all the details required to . . .

  1. Request a free IoTConnect on AWS trial account
  2. Create a new IoT device in IoTConnect on AWS
  3. Configure the Avnet Out-of-Box (OOB) application to connect to the new device on IoTConnect
  4. Run the demo and verify that inference data is sent to IoTConnect
  5. Use a template to create a pre-built IoTConnect dashboard

Table of Contents

  1. Prerequisites
  2. Video Walkthrough
  3. Cloud Account Setup
  4. Create a device on IoTConnect
    1. Import a device Template
    2. Create a new device
    3. Download device X.509 certificates for the new device
  5. Checkpoint
  6. Collect details to connect the device to IoTConnect
  7. Configure the RASynBoard OOB application to connect to IoTConnect
  8. Run the demo
  9. IoTConnect Dashboards
  10. Exercise the Dashboard
  11. Change the ML model
  12. Conclusion

Prerequisites

  1. You have an Avnet RASynBoard EVK
  2. You have a microSD card for the EVK
  3. You have a USB-C cable to connect the EVK to your development PC
  4. You have completed the RASynBoard Getting Started Guide OR You have loaded the most recent release, easier, onto your RASynBoard
  5. You have exercised the 5-keyword ML model "up", "down", "back", "next", "ok-syntiant"
  6. You have a USB to TTL debug cable to view debug from the OOB application

Video Walkthrough

There is also a video showing how to complete this tutorial.

Cloud Account Setup

This guide requires an IoTConnect account on AWS.

NOTE:
If you have already created an IoTConnect account on AWS, or were provided an account as part of a training or workshop, please move onto the next section Create a New Device on IoTConnct.

If you need to create an account, a free 2-month subscription is available. Please follow the Creating a New IoTConnect Account guide and ensure to select the AWS version during registration



Once you have an account on IoTConnect come back to this document to move forward . . .

Create a New Device on IoTConnect

  1. Open the IoTConnect website. The URL will depend on your account, your welcome email will have the URL to open IoTConnect.
  2. Here are the high level tasks we need to complete in IoTConnect . . .
    1. Import a device template
    2. Create a new IoT device
    3. Download the certificates for your device

Create a Device Template

The first thing we need to do is create a device template, since the template is a required input when we create a new IoTDevice. A device template is a self-contained file that defines a set of resources, configuration, and rules for devices on IoTConnect.

You can manually create or import a pre-defined template. In this tutorial, we'll import a template. Importing a template is a quick way to use a template that someone else has already created. Templates can be large, and manually inputting all the details can be tedious. I've included a template that works with the OOB application in this GitHub repo. You can also manually edit an existing template, it's just a JSON document. However, editing templates is beyond the scope of this document.

  1. Open the devices page



  1. Select the Templates tab from the bottom of the page



  1. Click on the Create Template button in the upper right hand corner of the browser window



  1. Click on the Import button in the upper right hand corner



  1. The Import dialog opens
    1. Use the Browse link to find the /docs/assets/IoTConnect/RASyV1_6_template.JSON template file
    2. Click the Save button



  1. The template is imported and the Edit Template page opens
  2. Select the Attributes (16) tab

Here you can review the contents of the template. attributes are the telemetry items that the application will send to IoTConnect each time the NDP120 detects an inference event, i.e., "Up." When we define attributes we're typically defining simple {"key": value} JSON pairs. The application must use the exact same key when sending telemetry data. The keys defined in the template match the Avnet OOB application implementation. If you change or add additional telemetry items, just make sure the application matches your IoTConnect template attributes.

Below is an example of a simple {"key": value} pair {"modeDescription": "<some string>"}



Here's the line of code that constructs the telemetry JSON:

    // Create the JSON
    snprintf(telemetryMsg, sizeof(telemetryMsg), "{\"modeDescription\":\"%s\",\"numNetworks\":%d,\"numLabels\":%d}",
                                                 get_mode_description(), total_nn, num_labels);

Here's the debug you'll see when the application first connects to IoTConnect:

{"modeDescription":"5-keywords Single Mic","numNetworks":1,"numLabels":6}

The imported template also includes attributes that are JSON Objects label_0 -- label_9. If we look under the hood at one of these items you'll see that it's made up of three simple {"key": value} pairs. We decided to use objects so that the application can load any ML model and populate the label_x objects to describe the inference events for any model.



When the application first connects to IoTConnect it sends up details about the model that's loaded, then using data directly from the model it sends up JSON objects detailing the inferenceIndex infIdx, the Inference String infStr and the initial Inference Count infCnt of zero. When we run the IoTConnect dashboard, you'll see how these objects are utilized.





Create a New Device

  1. Open the devices page



  1. Select the Devices tab from the bottom of the page



  1. Select the Create Device button in the upper right hand corner



  1. The Create Device form opens.
    1. Give your device a Unique ID
    2. Fill out a Display Name
    3. For the Entity field, you likely only have one option and it will be different than mine; select the only option shown in your interface.
    4. Select the template we just imported RASyV1_5
    5. Leave the default Auto-generated (recommended) selection for the Device Certificate
    6. Click on the Save button



  1. Your device will be shown in the devices list

Download the certificates for your new Device

When we created the new device with Self-Signed Certificates IoTConnect generated the device certificate and the device public key certificate.

  1. Open the device by clicking on the Unique ID link, in my case RASynBoard.



  1. Click on the Connection Info link



  1. The Connection Info pop-up opens, showing all the nitty gritty details for your device on AWS.
  2. Click on the certificate download link



  1. Your certificates are downloaded to your Downloads folder
  2. Close the Connection Info pop-up window





Checkpoint

So far we have . . .

  • Requested a free IoTConnect on AWS account
  • Received the free account access
  • Imported a device template describing the telemetry items that the OOB application sends
  • Created a new device using the template
  • Downloaded the device's certificates

Update the Avnet Out-of-Box application with details for our new device

At this point it's important that all the prerequisites detailed at the top of this document have been completed. Please review them and make sure you're ready to proceed.

Collect required details from IoTConnect on AWS

There are three IoTConnect details that we need to configure the OOB application to connect as our new device. We'll add them to the config.ini file on the RASynBoard microSD card. I recommend opening up a text document to capture these items.

Company ID (CPID) and Environment (env)

The company ID and the Environment are both displayed on the Key Vault page.

Note you can copy the strings by selecting the copy link to the left of the text





Device Unique ID

The Unique ID can be found on the Devices page



Find your device and copy the Unique ID by using the copy link to the left of your device's Unique ID



Configure the Avnet Out-of-Box application

We're getting close now! The next step is to configure the OOB application by adding the device certificates and updating the config.ini file that's located on the microSD card. Since you already completed the RASynBoard Getting Started Guide you updated your microSD card with all the files from the /ndp120/synpkg_files/* folder.

Copy the Device Certificates to the microSD card

  1. Extract the two certificates from the zip file you downloaded from IoTConnect
  2. Copy them to the microSD card into the /certs directory
  3. The certs directory should already contain the AmazonRootCA1.pem file, if not copy it from the repo /ndp120/synkpg_files/certs folder



Update the Config.ini file

  1. Open the config.ini file by either removing the microSD card from the RASynBoard and using a microSD card reader, or by running the OOB application and connecting your USB-C cable to the USB-C connector on the core board.

If you use the RASynBoard OOB application . . .

  1. Make sure the OOB application is loaded onto the device (if you completed the RASynBoard Getting Started Guide this is done)
  2. Make sure that the Jumper is installed on J3 across pins 1-2 (this will auto boot the application)
  3. Connect your development PC to the USB-C connector on the core board (NOT the I/O board)



Once the application boots, the microSD card will enumerate on your development PC as an external USB drive



  1. Update the following items in your config.ini



My updated file looks like this . . .



Run the demo

Now that everything is configured, we can run the demo! Cycle power on the device by disconnecting/reconnecting the USB-C connector from your development PC.

Note: When the application starts up it outputs the configuration so you can confirm that your settings are being used . . .



Note: The debug will show the different states that the IoTConnect thread processes while connecting to IoTConnect. If your device does not connect watch the debug for ERROR debug messages.

  1. As soon as the MQTT connection is established, queued up telemetry messages are sent to IoTConnect.



The OOB application implements a queue for telemetry data. This way if telemetry data is generated before the application is connected to IoTConnect, the message is not lost. As soon as the MQTT connection is established the IoTConnect thread looks for messages in the queue and if present they get sent to IoTConnect! Additionally, the application verifies that there is a valid MQTT connection before sending any telemetry. If not, the state machine is rolled back to re-establish the MQTT connection.

  1. Let's go back to IoTConnect and verify that the device is on-line
  2. Open the Devices page and verify that your device is on-line



  1. Now open your device by clicking on the Unique ID link
  2. Click on the Live Data link at the left
  3. If not automatically selected, select the Telemetry tab



  1. Now speak one of the keywords "Up"

Note the debug shows the telemetry that's being sent

Note that in IoTConnect the telemetry shows up in the Live Data --> Telemetry tab!





IoTConnect Dashboards

One of my favorite IoTConnect features is the ability to build a custom dashboard in minutes. In this section we'll import a dashboard that I've already build.

  1. In the IoTConnect web interface select Create Dashboard from the top of the interface



  1. The Create Dynamic Dashboard window opens
    1. Select the template we just imported RASyV1_5
    2. Select the device we just created RASynBoard for me
    3. Give your dashboard a name
    4. Use the Browse button to browse to the file RASynBoard_generic_dashboard.json template from the GitHub repo
    5. Click the Save button

Note: if you're working to recreate the RASynPuck demo load the RASynPuck_dashboard.json template from the GitHub repo

  1. The new dashboard opens in edit mode
  2. Click the Save button to exit edit mode
  3. The dashboard is displayed



Exercise the dashboard

To drive data to the dashboard just speak one of the keywords "up", "back," "down,", "next," "ok-syntiant." If you review the dashboard you see some interesting data.

Note the dashboard will work with any ML model loaded on the RASynBoard. You just need to perform some action to trigger an event for the model you have loaded. For example, if the circular motion model is loaded, just move the device in a circle.

The Most Recent Event Multi Tile shows the last inference event and it's inference index.

Note that the JSON key is displayed in the tile.



The ML Inference Counts show each label from the ML model and how many times each keyword was detected since power-up.



The Live Line Chart shows each inference count and also the inferenceIdx that is sent with each keyword detection. Speak the different keywords to drive new data to the chart.



Load a different ML model

Remember how we implemented the label_x telemetry objects. Because of that the ML Inference Count Multi tiles can display the inference string from any model that gets loaded. Let's try the Alexa model. This model is not real interesting but its performance is outstanding.

  1. Open the config.ini file from the microSD card
  2. Change the Mode setting to Mode=6



  1. Power cycle your RASynBoard for the new configuration to take effect

Note that the Alexa Single Mic model is loaded



  1. Now go back to the dashboard

Note that the Current Model Information and the ML Inference Counts Multi Tiles now specify details from the Alexa model

  1. Test the "Alexa" keyword with this model, the performance is over the top!



Conclusion

Cloud enabling a device adds additional return on investment by allowing service teams to proactively monitor devices in the field without having to roll a truck. When we combine ML at the Edge with a cloud solution the use cases are unlimited.

I hope you were successful in sending telemetry data to IoTConnect on AWS. If you encounter any issues or have questions feel free to open an issue in the repo