Problems starting and stopping linked mono camera capture

[crnolan]

I'm running into issues starting and stopping camera capture, specifically on the mono cameras of my Pro Wide PoEs. I posted on the forums here and jakaskerl suggested that this might be a camera synchronisation problem. Example script below; cameras both display on start, I can successfully stop the camera capture at least once (pressing 'p'). When I restart the cameras (pressing 's') - usually the first time, but sometimes the second iteration of stop / start - the left camera will not start and the right camera will capture a few frames then freeze. If I then try to stop the capture again, I get the error:

[system] [critical] Fatal error. Please report to developers. Log: 'PlgSrcMipi' '1130'
[host] [warning] Device crashed, but no crash dump could be extracted.

I'm on Python 3.10, depthai package depthai-2.27.0.0-cp310-cp310-win_amd64. Test code (modified from chengguizi over on issue 745 which might be related?):

#!/usr/bin/env python3

import cv2
import depthai as dai

# Create pipeline
pipeline = dai.Pipeline()

# Define sources and outputs
monoLeft = pipeline.create(dai.node.MonoCamera)
monoRight = pipeline.create(dai.node.MonoCamera)
xoutLeft = pipeline.create(dai.node.XLinkOut)
xoutRight = pipeline.create(dai.node.XLinkOut)

xoutLeft.setStreamName('left')
xoutRight.setStreamName('right')

# Properties
monoLeft.setBoardSocket(dai.CameraBoardSocket.CAM_B)
monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P)
monoRight.setBoardSocket(dai.CameraBoardSocket.CAM_C)
monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P)

controlIn = pipeline.create(dai.node.XLinkIn)
controlIn.setStreamName('control')
controlIn.out.link(monoRight.inputControl)
controlIn.out.link(monoLeft.inputControl)

# Linking
monoRight.out.link(xoutRight.input)
monoLeft.out.link(xoutLeft.input)

device_infos = dai.Device.getAllAvailableDevices()
print(device_infos)

# Connect to device and start pipeline
with dai.Device(device_infos[0]) as device:
    device.startPipeline(pipeline)
    controlQueue = device.getInputQueue('control')

    # Output queues will be used to get the grayscale frames from the outputs defined above
    qLeft = device.getOutputQueue(name="left", maxSize=4, blocking=False)
    qRight = device.getOutputQueue(name="right", maxSize=4, blocking=False)

    while True:
        # Instead of get (blocking), we use tryGet (non-blocking) which will return the available data or None otherwise
        inLeft = qLeft.tryGet()
        inRight = qRight.tryGet()

        if inLeft is not None:
            cv2.imshow("left", inLeft.getCvFrame())

        if inRight is not None:
            cv2.imshow("right", inRight.getCvFrame())

        key = cv2.waitKey(1)
        if key == ord('q'):
            break
        elif key == ord('p'):
            ctrl = dai.CameraControl()
            ctrl.setStopStreaming()
            controlQueue.send(ctrl)
        elif key == ord('s'):
            ctrl = dai.CameraControl()
            ctrl.setStartStreaming()
            controlQueue.send(ctrl)

[Erol444]

Hi @crnolan , I don't think start/stop cam streaming works flawlessly for all sensors, so likely you are encountering this problem. How about discarding frames when they aren't needed, using Script node? Example here:
https://docs.luxonis.com/software/depthai/examples/script_change_pipeline_flow/

[crnolan]

Hi @Erol444, thanks for the response. We are trying to align the exact times that frames are taken with external hardware using the FSYNC output, so we need exactly one frame sent for every FSYNC toggle that the camera outputs. I'm not sure how I could do that via the script node.

[Erol444]

@crnolan how about decreasing FPS to ~2 (min FPS for most sensors)?