Optimize boot time for single image capture scenario

[peterljung]

I am trying to understand if and how boot time could be decreased to optimize for a scenario where:

1. A single picture is captures
2. A single inference operation is made
3. Platform is shut down completely (power off)

I think this is a quite common scenario for many, many use cases which potentially could support a low power operation.

I still haven't investigated code in detail, but I have found some interesting things from running some samples.

I measured power during boot to understand timing of the system in a few samples.

allon_sensor_tflm_freertos take 230 ms to start from reset.

allon_sensor_tflm takes 130 ms to start from reset.

There is quite some difference between these samples that I don't fully understand. Is FreeRTOS responsible for the difference.
Samples do simlar things i.e. cold boot, capture image and perform person detection inference.

I also saw that 100 ms is added to boot time to allow for user input.

...
Please input any key to enter X-Modem mode in 100 ms 
waiting input key...
...

Is this customizable?

Also I am wonding about the possibility to run the platform in single core and dual core mode.
Could single core mode be used during boot to optimize power?

If someone has any tip or insight on how boot time could be optimized, it would be great!

[stevehuang82]

Hi @peterljung,

The power consumption timing diagram cannot accurately measure the boot time.

Boot time should be defined as the duration from system reset to the point where the program jumps to the application app_main().

The diagram below illustrates the boot time we measured when GPIO0 was set high at the start of app_main().

allon_sensor_tflm_freertos startup time = 172 ms
Bootloader delays 100 ms waiting for keystrokes
If we remove the waiting time of 100 ms, the actual boot time = 172 – 100 = 72 ms

Additionally, we measured the power consumption diagram for allon_sensor_tflm_freertos. The time from system reset to when the program starts inference is approximately 202ms, not 230ms.

The boot time of allon_sensor_tflm_freertos and allon_sensor_tflm is the same 172ms (including waiting 100ms). As you can see, there is indeed a difference in the time to start inference, because FreeRTOS requires a longer system setup time.

Sorry, as the Grove_Vision_AI_Module_V2 is a proof-of-concept development board, we do not provide support for custom bootloaders or power optimization.

[peterljung]

Thanks for detailed explanation!

But I don't really understand how my measurments match with your numbers.

When I look at allon_sensor_tflm_freertos boot time (yellow in top diagram) seems to be around 200 ms instead of 172.

But when I compare with allon_sensor_tflm (bottom diagram) I see the same power profile (yellow) for the boot part. I marked setup time in blue. But here boot time seems to be around 100 ms.

You can also see setup time (blue) look similar in both samples.

I also marked capture time and inference time in red and green.

The only difference between freertos and non freertos should be seen in setup time.

You could match diagrams as follows as well, but I still don't see how they match up.

[stevehuang82]

From system startup to the start of model inference, there are multiple stages, each with varying levels of power consumption.

1. System starts and copies the application from flash to sram and jumps to app_main() (boot time)
2. Initialize the system peripheral interface ex. I2C, UART, SPI, MIPI...
3. Setup the HX6538 data path and all functional blocks
4. Initialize the sensor and start streaming
5. HX6538 starts capturing images and encoding raw images into jpeg images
6. Send images via UART or SPI (optional)
7. Resize and reformat the original image to fit the model
8. start model inference

If you want to analyze the power consumption of each stage to fit the timing diagram, you can add gpio control in the application code (allon_sensor_tflm.c) and measure the gpio output timing diagram.

You can use an oscilloscope to measure pin D3(RST) for reset and pin D0(PA0) for AON_GPIO0. The following is a sample code for gpio control.

allon_sensor_tflm.c

int app_main(void)
{
        // setup AON_GPIO0
        hx_drv_scu_set_PA0_pinmux(SCU_PA0_PINMUX_AON_GPIO0_2, 1);
        hx_drv_gpio_set_output(AON_GPIO0, GPIO_OUT_LOW);

        // trigger AON_GPIO0 high/low
        hx_drv_gpio_set_out_value(AON_GPIO0, GPIO_OUT_HIGH);
        hx_drv_timer_cm55x_delay_ms(10, TIMER_STATE_DC);
        hx_drv_gpio_set_out_value(AON_GPIO0, GPIO_OUT_LOW);

You can add GPIO trigger code where you want to measure different stages of power consumption.

        // trigger AON_GPIO0 high/low
        hx_drv_gpio_set_out_value(AON_GPIO0, GPIO_OUT_HIGH);
        hx_drv_timer_cm55x_delay_ms(10, TIMER_STATE_DC);
        hx_drv_gpio_set_out_value(AON_GPIO0, GPIO_OUT_LOW);

Since Grove Vision AI Module V2 is a POC development board, the power consumption of other components also needs to be considered. For example, OV5647 sensor and CH343 bridge IC.
There are many ways to optimize boot time and power consumption, but they must be product-specific.

Hope this helps you understand the power timing diagram, thank you.

[peterljungatsony]

Thanks for tip! I will try to do this ...
I realize that this POC platform is not optimized for power or boot up performance.
I just try to estimate what could be achievable if you would optimize for a product ...

[peterljungatsony]

I have now measured allon_sensor_tflm using your proposed method.
I could also confirm the boot time of 172 ms and get about 30 ms of setup time and 60 ms for capture/inference period.
172+30 = 202 ms seems to match the figure you got on allon_sensor_tflm_freertos (I have not tested freertos version myself).