-
Notifications
You must be signed in to change notification settings - Fork 192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Frame Synchronization on OAK-FFC-4P Board with Dual AR0234 Camera Modules #883
Comments
Hi @tarekmuallim Alternatively/preferably you could hardware-synchronize the frames to achieve an even lower (sub-ms) accuracy. A guide is available here. Thanks |
Hello @jakaskerl Thank you for your prompt response. I've reviewed the documents you provided. From my understanding, there's already a synchronization clock generated on the OAK-FFC board. To establish synchronization, I believe I need to connect a wire from the FSYNC output on the board to the FSIN input on the camera module. Specifically, I am utilizing the AR0234 module in conjunction with an OAK-FFC-4P board. However, I couldn't locate a designated FSIN input on the AR0234 module board. Is it possible that the XVS input serves this purpose? Unfortunately, I couldn't find any documentation that clarifies the inputs for the camera module. Additionally, if you could kindly direct me to the specific location on the OAK-FFC-4P board where I should connect the sync input to the camera, it would be greatly appreciated. Lastly, I have a question regarding supplying an external sync clock to the camera modules. Could you inform me of the required voltage for this purpose, and does it vary depending on the camera modules in use? Once again, thank you for your assistance. |
Hi @tarekmuallim ,
And then connect the FSYNC signal + GND via PSRBS connector (required 1.8V): So from user perspective, left to right:
|
Hi @Erol444 Thanks for your replay. I'm currently facing some confusion as I can't seem to locate the PSRBS connector on my board. Additionally, I'd appreciate it if you could confirm whether the XVS input on the AR0234 camera module functions as the FSIN input, as I couldn't find any input labeled as FSIN on the camera module board. |
Hi @tarekmuallim , |
Hi @Erol444 I appreciate your response. I will review the link you provided. |
Hi @Erol444 I attempted to verify the wire indicated in the documentation that is supposed to carry the FSIN signal using an oscilloscope. I examined the area on the board labeled as "FSIN1." To my surprise, there was no discernible signal. I was unable to detect any clock or trigger signals. I also examined "FSIN2", "FSIN3", "FSIN4", and so forth with the same result. Furthermore, I inspected the FSIN testpoint on the cable, and once again, there was no signal detected. Is there something specific that I may have overlooked in this process? |
Hi @tarekmuallim , |
I'm currently using the OV9782 camera for this test. After connecting the cameras to sockets B and C, I observed that the time difference between the frames is reduced to 0.01ms in this configuration. Subsequently, I enabled the FSIN output using the following code: pipeline = dai.Pipeline()
colorLeft = pipeline.create(dai.node.ColorCamera)
colorRight = pipeline.create(dai.node.ColorCamera)
colorLeft.setBoardSocket(dai.CameraBoardSocket.CAM_B)
colorRight.setBoardSocket(dai.CameraBoardSocket.CAM_C)
colorLeft.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.OUTPUT)
colorRight.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT)
config = dai.Device.Config()
config.board.gpio[6] = dai.BoardConfig.GPIO(dai.BoardConfig.GPIO.OUTPUT, dai.BoardConfig.GPIO.Level.HIGH)
with dai.Device(config) as device:
device.startPipeline(pipeline) However, in this setup, I noticed that the time difference between the frames increased to 3ms, and I was unable to detect any FSIN signal at the board or at the testpoint on the cable. When I attempted the same setup with the AR0234 camera and enabled the FSIN output, I encountered an error. |
@tarekmuallim at the moment AR0234 likely only works in EXTERNAL_TRIGGER mode, compared to FSYNC OUTPUT/INPUT. |
Experimenting with the AR0234 camera in EXTERNAL_TRIGGER mode, I set the external trigger for the left and right color cameras as follows:
This resolved the error. I did not receive any frames because I didn't supply a trigger signal. |
@tarekmuallim FSIN is usually generated by one of the imagers (usually LEFT imager in L/R combination) and can also not be present when manually driving sensors via External Trigger mode |
To clarify the situation, let me reiterate the issue. I thing we have encountered some confusion, so I'll provide a clear explanation. AR0234 appears to operate exclusively in EXTERNAL_TRIGGER mode. Consequently, I've switched to using the OV9782 for my current testing purposes. The camera modules are connected to both the left and right sockets, and I have implemented the code specified below.
My objective was to detect a clock signal on the FSIN testpoint within the left camera cable or on the board itself. Regrettably, no signal was detected. I am attempting to follow the instructions outlined in the link provided below to address this issue. |
@tarekmuallim WRT OV97 in the above mentioned configuration is looks like it should work. @alex-luxonis any ideas on this? |
Hi @alex-luxonis @themarpe |
@tarekmuallim Sorry for the delay... For OV9782/OV9282, by default when connected on ports B and C that share the same I2C bus, the cameras are started in broadcast mode (I2C write to both), that will ensure they will be kept synced without the need for a sync signal, as long as they're clocked from the same source, which is the case with Luxonis camera modules, where the clock is provided by the OAK-FFC-4P baseboard. It's important to be the same source, and not separate oscillators of the same frequency -- drift over time would occur here.
^ here it's likely here that the cameras were streaming started with the default broadcast mode. The 0.01ms difference is expected (actually due to the SoC FW taking the timestamp 2 times in sequence, while the real timestamp difference should be very close to 0). Some third party camera modules (Arducam, Raspberry Pi) do have clock oscillators on board, and then hardware sync would be needed. Also needed when connected to different sockets on FFC-4P, where different I2C buses are used. For the AR0234 issue, I'll follow up in a new comment. |
Dear @alex-luxonis, I am interested in this issue too. Currently we have 3 set of AR0234 based modular camera using OAK-FFC-4P waiting for this evolution. I would like to know if there's any progress in the firmware side, please ? Thanks and best regards, |
Yes, the firmware was just updated earlier today ( Previously this error happened if the fsync signal was not being provided within some time (approx 1 second) after enabling the sensor, it should be no longer an issue now: AR0234 sensors can be configured like: |
Hi @alex-luxonis, Could you share where would the aforementioned example be found, please? |
Modified cam_test.py: main...alex-luxonis:depthai-python:fsync_gpio_ffc_4p (alex-luxonis@edc5666) Mainly added a new Script node with the content: import time
import GPIO
# Script static arguments
fps = %f
calib = Device.readCalibration2().getEepromData()
prodName = calib.productName
boardName = calib.boardName
boardRev = calib.boardRev
node.warn(f'Product name : {prodName}')
node.warn(f'Board name : {boardName}')
node.warn(f'Board revision: {boardRev}')
revision = -1
# Very basic parsing here, TODO improve
if len(boardRev) >= 2 and boardRev[0] == 'R':
revision = int(boardRev[1])
node.warn(f'Parsed revision number: {revision}')
# Defaults for OAK-FFC-4P older revisions (<= R5)
GPIO_FSIN_2LANE = 41 # COM_AUX_IO2
GPIO_FSIN_4LANE = 40
GPIO_FSIN_MODE_SELECT = 6 # Drive 1 to tie together FSIN_2LANE and FSIN_4LANE
if revision >= 6:
GPIO_FSIN_2LANE = 41 # still COM_AUX_IO2, no PWM capable
GPIO_FSIN_4LANE = 42 # also not PWM capable
GPIO_FSIN_MODE_SELECT = 38 # Drive 1 to tie together FSIN_2LANE and FSIN_4LANE
# Note: on R7 GPIO_FSIN_MODE_SELECT is pulled up, driving high isn't necessary (but fine to do)
# GPIO initialization
GPIO.setup(GPIO_FSIN_2LANE, GPIO.OUT)
GPIO.write(GPIO_FSIN_2LANE, 0)
GPIO.setup(GPIO_FSIN_4LANE, GPIO.IN)
GPIO.setup(GPIO_FSIN_MODE_SELECT, GPIO.OUT)
GPIO.write(GPIO_FSIN_MODE_SELECT, 1)
period = 1 / fps
active = 0.001
node.warn(f'FPS: {fps} Period: {period}')
withInterrupts = False
if withInterrupts:
node.critical(f'[TODO] FSYNC with timer interrupts (more precise) not implemented')
else:
overhead = 0.003 # Empirical, TODO add thread priority option!
while True:
GPIO.write(GPIO_FSIN_2LANE, 1)
time.sleep(active)
GPIO.write(GPIO_FSIN_2LANE, 0)
time.sleep(period - active - overhead) And configured the enabled cameras in sync mode: # Num frames to capture on trigger, with first to be discarded (due to degraded quality)
# Note: Only OV9282/9782 supports dropping frames. For AR0234 the arguments passed here have no effect
# Note2: in this mode it's best to set sensor FPS as max supported (e.g 60) to avoid missed triggers,
# as exposure starts immediately after trigger and is not overlapped with previous frame MIPI readout
#cam[c].initialControl.setExternalTrigger(1, 0)
# Note: setFrameSyncMode takes priority over setExternalTrigger (if both are set)
cam[c].initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT) Getting with 3 AR0234 attached on A,B,C ports and cable FSIN soldered to TRIGGER pin of Arducam module
|
Hello,
I'm currently utilizing an OAK-FFC-4P board equipped with two AR0234 camera modules. Specifically, one camera is interfaced with the CAM_A port, while the other is connected to CAM_D. Upon assessing the time interval between these two frames, I observed a delay of approximately 7.8 milliseconds.
Is there a software-based approach to minimize this time gap?
Additionally, could you direct me to any comprehensive documentation that outlines the process of synchronizing frames externally when employing an OAK-FFC-4P board with two AR0234 camera modules?"
The text was updated successfully, but these errors were encountered: