Add Workflow Handler to consume Kibana data extractor output

[DJRickyB]

Overview

In the solutions tracks (under elastic/) we consume JSON files that are essentially separate composite operations. For background, we used a purpose-built tool to convert APM traces to this format previously.

Now with the completion of the work in elastic/kibana#137895, we are ready to intake a new standard, consuming the output of Kibana single-user performance journeys for the eventual automation of maintenance.

Implementation

• We could add a new track processor to consume these outputs¹ and convert them to Rally tasks. The on_after_load_track method can add these tasks to the end of track.selected_challenge_or_default.schedule.
• Alternatively (to keep more of the JSON of the tracks intact for traceability) we could extend the existing workflow selector parameter source to have the ability to read in these outputs at runtime of the track.

Various requirements

• We should accept verbatim outputs from the extractor, meaning we should be able to override any static values such as datetimes (already done, though we will double check that we are covering all the cases) and unique IDs, and not need to process the outputs before working with them in the execution of the track.
• _pit and _async_search will need to be adapted carefully. Essentially what we will do is:
• Create a composite operation for the search, replicating just the core requests (e.g. ignoring any polling to GET the async search), and using native Rally operations open-point-in-time close-point-in-time submit-async-search get-async-search delete-async-search as necessary.

Footnotes

1. From a local directory, to start (same as we have now). Eventually given the outcome of https://github.com/elastic/package-assets/issues/21, we will want to pivot or add support for fetching these assets from elsewhere, possibly github or possibly an object store. ↩