Callbacks: document logging

[Tom-Willemsen]

As a developer, it would be useful to capture the "raw" documents coming out of the run engine, so that an exact stream of documents can be "replayed" into future callbacks to test them.

This also ensures that all information emitted by bluesky is captured.

Acceptance criteria

• Callback added to default run engine which saves data for each bluesky "run"
  • NOTE: "bluesky run" != "ibex run", a run in bluesky encompasses a whole scan
• Likely logging into c:\instrument\var\logs\bluesky\raw_documents or a similar location
• It should be possible to retrieve the full content of every bluesky document after-the-fact, by inspecting these log files.
  • With the idea that we can later "replay" these same documents into other callbacks to test them.
• This callback should write it's output incrementally as it captures documents, so that if a bluesky plan is hard-aborted halfway through, there is still a way to retrieve documents that were emitted before the abort.
• This callback is non-optional, added by default in get_run_engine() for example.

Notes

• Longer term we will probably do this via e.g. kafka instead so these simple files will likely become redundant.
• Add directory to log rotation script (move rather than delete)
• Bluesky has facilities for this (e.g. databroker) that log into a database, could consider using those facilities, but equally consider doing something very simple to start with. The primary use case for this callback will be developer debugging so consider which workflow gives the best developer experience.

Time in Planning

00:28:28 08/08/24

[jackbdoughty]

EPICS PR
BlueSky core PR
Release notes PR <- @Tom-Willemsen said he would have a look into where best to put release notes for BlueSky