[Bug] decimationFactor <3 + spatialFilter + 800P triggers weird warnings

[arufl]

Hello, I am testing the gen2-calc-spatials-on-host on the OAK-D-POE and Depthai 2.26.0 . If I add decimationFilter + spatialFilter filters with THE_400_P nothing wrong occurs. However, after setting the mode to THE_800_P and decimationFactor = 2 it returns this warning continuously:

[StereoDepth(2)] [warning] Decimation filter max supported shaves for '131072' 'B' memory is '2'. Defaulting to '2'.

If I set decimationFactor = 1 it complains that decimation is not enabled

[StereoDepth(2)] [warning] It's recommended to enable decimation filter when post-processing filters are enabled due to performance reasons.

And with decimationFactor = 3 / 4 it runs fine but it is too high for our application

I was advised by Jakaskerl to fill thsi bug here

Thanks,

The code:

#!/usr/bin/env python3

import cv2
import depthai as dai
from calc import HostSpatialsCalc
from utility import \*
import numpy as np
import math

# Create pipeline

pipeline = dai.Pipeline()

# Define sources and outputs

monoLeft = pipeline.create(dai.node.MonoCamera)
monoRight = pipeline.create(dai.node.MonoCamera)
stereo = pipeline.create(dai.node.StereoDepth)

# Properties

monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
fps=5
monoLeft.setFps(fps)
monoLeft.initialControl.setManualExposure(3000,100)
monoLeft.setBoardSocket(dai.CameraBoardSocket.CAM_B)
monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
monoRight.setFps(fps)
monoRight.initialControl.setManualExposure(3000,100)
monoRight.setBoardSocket(dai.CameraBoardSocket.CAM_C)

stereo.initialConfig.setConfidenceThreshold(190)
stereo.setLeftRightCheck(True)
stereo.setSubpixel(False)
stereo.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_ACCURACY)
stereo.setDepthAlign(dai.CameraBoardSocket.CAM_A)
stereo.setOutputSize(monoLeft.getResolutionWidth(),monoLeft.getResolutionHeight())
stereo.initialConfig.setMedianFilter(dai.MedianFilter.KERNEL_5x5)
stereo.setLeftRightCheck(True)
stereo.setExtendedDisparity(False)
stereo.setSubpixel(True)
config = stereo.initialConfig.get()
config.postProcessing.decimationFilter.decimationMode  = dai.RawStereoDepthConfig.PostProcessing.DecimationFilter.DecimationMode.NON_ZERO_MEAN
config.postProcessing.decimationFilter.decimationFactor = 2
config.postProcessing.speckleFilter.enable = True
config.postProcessing.speckleFilter.speckleRange = 50
config.postProcessing.spatialFilter.enable = True
config.postProcessing.brightnessFilter.maxBrightness=240
config.postProcessing.brightnessFilter.minBrightness=10

config.postProcessing.spatialFilter.holeFillingRadius = 80

config.postProcessing.spatialFilter.numIterations = 1

stereo.initialConfig.set(config)

# Linking

monoLeft.out.link(stereo.left)
monoRight.out.link(stereo.right)

xoutDepth = pipeline.create(dai.node.XLinkOut)
xoutDepth.setStreamName("depth")
stereo.depth.link(xoutDepth.input)

xoutDisp = pipeline.create(dai.node.XLinkOut)
xoutDisp.setStreamName("disp")
stereo.disparity.link(xoutDisp.input)

xoutleft = pipeline.create(dai.node.XLinkOut)
xoutleft.setStreamName("left")
monoLeft.out.link(xoutleft.input)

# Connect to device and start pipeline

with dai.Device(pipeline) as device:
\# Output queue will be used to get the depth frames from the outputs defined above
depthQueue = device.getOutputQueue(name="depth")
dispQ = device.getOutputQueue(name="disp")
leftQ = device.getOutputQueue(name=xoutleft.getStreamName())
text = TextHelper()
hostSpatials = HostSpatialsCalc(device)
y = 200
x = 300
step = 3
delta = 5
hostSpatials.setDeltaRoi(delta)


print("Use WASD keys to move ROI.\\nUse 'r' and 'f' to change ROI size.")

while True:
depthData = depthQueue.get()
leftdata=leftQ.get()
\# Calculate spatial coordiantes from depth frame
spatials, centroid = hostSpatials.calc_spatials(depthData, (x,y)) # centroid == x/y in our case

# Get disparity frame for nicer depth visualization
disp = dispQ.get().getFrame()
disp = (disp * (255 / stereo.initialConfig.getMaxDisparity())).astype(np.uint8)
disp = cv2.applyColorMap(disp, cv2.COLORMAP_JET)

text.rectangle(disp, (x-delta, y-delta), (x+delta, y+delta))
text.putText(disp, "X: " + ("{:.1f}m".format(spatials['x']/1000) if not math.isnan(spatials['x']) else "--"), (x + 10, y + 20))
text.putText(disp, "Y: " + ("{:.1f}m".format(spatials['y']/1000) if not math.isnan(spatials['y']) else "--"), (x + 10, y + 35))
text.putText(disp, "Z: " + ("{:.1f}m".format(spatials['z']/1000) if not math.isnan(spatials['z']) else "--"), (x + 10, y + 50))

#frame = leftdata.getCvFrame()
# Show the frame
cv2.imshow("depth", disp)
#cv2.imshow("frame", frame)
key = cv2.waitKey(1)
if key == ord('q'):
    break
elif key == ord('w'):
    y -= step
elif key == ord('a'):
    x -= step
elif key == ord('s'):
    y += step
elif key == ord('d'):
    x += step
elif key == ord('r'): # Increase Delta
    if delta < 50:
        delta += 1
        hostSpatials.setDeltaRoi(delta)
elif key == ord('f'): # Decrease Delta
    if 3 < delta:
        delta -= 1
        hostSpatials.setDeltaRoi(delta)

[Erol444]

Hi @arufl , have you tried increasing resources for postprocessing filters?

# 3 shaves, 3 cmx slices
stereoNode.setPostProcessingHardwareResources(3, 3)

[arufl]

Ouch! That was it. With decimationFactor = 2 there is no more shaves warnings. Thanks.

[arufl]

I should not have closed it yet. With decimationFactor = 1 it still complains that decimation is not enabled

[Erol444]

cc @SzabolcsGergely ^

[SzabolcsGergely]

decimationFactor = 1 doesn't enable decimation filter.

[Erol444]

cc @jakaskerl to document that ^ (perhaps in "configure stereo depth" docs)