Next steps design for integration

[CodyCBakerPhD]

To integrate this with minimal effort and change to other libraries, we need to consider the following approach

If we're planning something similar to the websockets approach of the demos in #43 for the GUIDE handling of such things, then what we need is the ability to take existing code that looks like this

def run_some_complicated_function_with_progress_bars_in_it(
    various_arguments: various_types = various_defaults,
    ..., 
)
    # Perform some setup
    ...

    # Get to the lines where the iterated operation is run
    progress_bar  = tqdm.tqdm(iterable=thing_being_iterated, ..., **other_tqdm_options)
    for step in progress_bar:
        # do something expensive each step

and refactor it into

def run_some_complicated_function_with_progress_bars_in_it(
    various_arguments: various_types = various_defaults,
    ..., 
    progress_bar_class: Type[tqdm.tqdm] = tqdm.tqdm,  # This would be a refactor we help with; most directly wrap at time of instantiation
    progress_bar_kwargs: dict = dict(),  # This would allow us to pass the callable to our new type
)
    # Perform some setup
    ...

    # Get to the lines where the iterated operation is run
    progress_bar = progress_bar_class(iterable=thing_being_iterated, **progress_bar_kwargs)
    for step in progress_bar:
        # do something expensive each step

So based on current structure I'd recommend something like

class WebSocketTQDMPublisher(TQDMPublisher):
    def __init__(self, websocket: ..., request_id: str, **tqdm_kwargs)
        super().__init__(**tqdm_kwargs)
        self.websocket = websocket
        self.request_id = request_id

        self.subscribe(callback=self.run)

    def send_progress_update_from_websocket(self, format_dict: dict):
        message = json.dumps(obj=dict(format_dict=format_dict, request_id=self.request_id))
        self.websocket.send(message=message)

Or perhaps more simply, directly inherit from tqdm.tqdm and bypass TQDMPublisher to have the update call directly send on each iteration

This way, all we would need to do in the Flask server is define an endpoint that does more or less

guide_progress_bar_class = WebSocketTQDMPublisher
progress_bar_kwargs = dict(websocket)

operation_we_would_normally_run(..., progress_bar_class=guide_progress_bar_class, progress_bar_kwargs=progress_bar_kwargs )

One question is, do we currently use multithreading and what not to showcase the async deployment of a synchronous operation

What will this be like on a Flask server? Are the endpoints already asynchronous or would the logic inside each endpoint handle the threading stuff?

[garrettmflynn]

Ah I see, this makes sense.

On the Flask server, you run the following command to update the GUIDE of any events:

announcer.announce(message, label)

Where announcer is a global class instance. These announcement events can be individually subscribed to on the GUIDE using label, or you can listen to all events.

From my investigation, it looks like Flask endpoints are synchronous by default—so asynchronous operations would have to be handled in the logic of the endpoint.