WIP: of synchronisation

[felix-engelmann]

This is a start of collecting data from all callback threads.

[felix-engelmann]

It now produces a stream of zmq multipart messages in the following form

{"htype":"header", "udpPorts":[50001, 50002], "filePath":/"}
{"htype":"header", "udpPorts":[50003, 50004], "filePath":/"}
--------
{"htype":"module", "port":50001"}
binary
{"htype":"module", "port":50002"}
binary
{"htype":"module", "port":50003"}
binary
{"htype":"module", "port":50004"}
binary
--------
{"htype":"module", "port":50001"}
binary
{"htype":"module", "port":50002"}
binary
{"htype":"module", "port":50003"}
binary
{"htype":"module", "port":50004"}
binary
--------
{"htype":"module", "port":50001"}
binary
{"htype":"module", "port":50002"}
binary
{"htype":"module", "port":50003"}
binary
{"htype":"module", "port":50004"}
binary
--------
{"htype":"series_end", "udpPorts":[50003, 50004]"}
{"htype":"series_end", "udpPorts":[50001, 50002]"}
--------

where every line is a part and the --- separate multipart messages

[felix-engelmann]

I would like to have the following test cases:

• a start callback missing (should not start the stream as sth is broken)
• a udp stream missing a frame
• modules stopping at different triggers
• a missing stop callback and then a new start

[felix-engelmann]

Now it produces valid json and includes all relevant fields, as can be checked by listening with

import json
c = zmq.Context()
s = c.socket(zmq.PULL)
s.connect("tcp://127.0.0.1:5555")

while True:
     m = s.recv_multipart()
     for p in m:
         if p.startswith(b"{"):
             print(p.decode().strip())
         else:
             print("binary")
     print("--------")

{"htype":"header", "udpPorts":[50001, 50002], "bitmode":16, "filePath":"/", "fileName":"run", "fileIndex":0, "detshape":[2, 2], "size":262144, "quad":0}
{"htype":"header", "udpPorts":[50003, 50004], "bitmode":16, "filePath":"/", "fileName":"run", "fileIndex":0, "detshape":[2, 2], "size":262144, "quad":0}
--------
{"htype":"module", "port":50001, "shape":[512, 256], "acqIndex":11, "frameIndex":0, "flipRows":0, "progress":33.3333, "completeImage":1, "frameNumber":11, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":31, "row":0, "column":0, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
{"htype":"module", "port":50002, "shape":[512, 256], "acqIndex":11, "frameIndex":0, "flipRows":0, "progress":33.3333, "completeImage":1, "frameNumber":11, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":32, "row":0, "column":1, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
{"htype":"module", "port":50003, "shape":[512, 256], "acqIndex":11, "frameIndex":0, "flipRows":0, "progress":33.3333, "completeImage":1, "frameNumber":11, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":31, "row":1, "column":0, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
{"htype":"module", "port":50004, "shape":[512, 256], "acqIndex":11, "frameIndex":0, "flipRows":0, "progress":33.3333, "completeImage":1, "frameNumber":11, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":32, "row":1, "column":1, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
--------
{"htype":"module", "port":50001, "shape":[512, 256], "acqIndex":12, "frameIndex":1, "flipRows":0, "progress":66.6667, "completeImage":1, "frameNumber":12, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":31, "row":0, "column":0, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
{"htype":"module", "port":50002, "shape":[512, 256], "acqIndex":12, "frameIndex":1, "flipRows":0, "progress":66.6667, "completeImage":1, "frameNumber":12, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":32, "row":0, "column":1, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
{"htype":"module", "port":50003, "shape":[512, 256], "acqIndex":12, "frameIndex":1, "flipRows":0, "progress":66.6667, "completeImage":1, "frameNumber":12, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":31, "row":1, "column":0, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
{"htype":"module", "port":50004, "shape":[512, 256], "acqIndex":12, "frameIndex":1, "flipRows":0, "progress":66.6667, "completeImage":1, "frameNumber":12, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":32, "row":1, "column":1, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
--------
{"htype":"module", "port":50001, "shape":[512, 256], "acqIndex":13, "frameIndex":2, "flipRows":0, "progress":100, "completeImage":1, "frameNumber":13, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":31, "row":0, "column":0, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
{"htype":"module", "port":50002, "shape":[512, 256], "acqIndex":13, "frameIndex":2, "flipRows":0, "progress":100, "completeImage":1, "frameNumber":13, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":32, "row":0, "column":1, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
{"htype":"module", "port":50003, "shape":[512, 256], "acqIndex":13, "frameIndex":2, "flipRows":0, "progress":100, "completeImage":1, "frameNumber":13, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":31, "row":1, "column":0, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
{"htype":"module", "port":50004, "shape":[512, 256], "acqIndex":13, "frameIndex":2, "flipRows":0, "progress":100, "completeImage":1, "frameNumber":13, "expLength":0, "packetNumber":256, "detSpec1":0, "timestamp":0, "modId":32, "row":1, "column":1, "detSpec2":0, "detSpec3":0, "detSpec4":0, "detType":1, "version":2}
binary
--------
{"htype":"series_end", "udpPorts":[50003, 50004], "comleteFrames":[3, 3], "lastFrameIndex":[13, 13]}
{"htype":"series_end", "udpPorts":[50001, 50002], "comleteFrames":[3, 3], "lastFrameIndex":[13, 13]}
--------