Inside-out guest mode

docs/source/reference-lowlevel.rst

@@ -847,7 +847,9 @@ important limitations you have to respect:
   shutdown of your host loop, which is just what you want.
 
 Given these constraints, we think the simplest approach is to always
-start and stop the two loops together.
+start and stop the two loops together. If that's not possible,
+you might want to :ref:`run the host loop from within Trio
+<inside-out-guest-mode>` instead.
 
 **Signal management:** `"Signals"
 <https://en.wikipedia.org/wiki/Signal_(IPC)>`__ are a low-level
@@ -928,11 +930,66 @@ into account when decided whether to jump the clock or whether all
 tasks are blocked.
 
 
+.. _inside-out-guest-mode:
+
+Inside-out guest mode: starting the host loop from within Trio
+--------------------------------------------------------------
+
+The discussion of guest mode up to this point describes how to run
+Trio inside an existing host loop. If you can make this work for your
+application, it's usually the approach that's easiest to reason
+about. If you can't, though, Trio also supports organizing things the
+other way around: running your host loop inside an existing Trio
+run, using :func:`trio.lowlevel.become_guest_for`.
+
+Any host loop will have some top-level synchronous function that you
+use to run it, and that doesn't return until the loop has stopped
+running.  For example, this is ``QApplication.exec_()`` when using Qt,
+or :func:`asyncio.run` when using asyncio (older Pythons use
+``loop.run_until_complete()`` and/or ``loop.run_forever()``). To use
+inside-out guest mode, you'll write a small wrapper around this
+top-level synchronous function, plus some cancellation glue, and pass
+both to a call to :func:`become_guest_for` that you make inside some
+Trio task.  That call will block for as long as the host loop is
+running, returning only when it completes. But, all your *other* tasks
+(besides the one that called :func:`become_guest_for`) will continue
+to run alongside the host loop.
+
+This is a little weird, so it bears repeating: even though Trio was
+running first, it will still act as the guest while the host loop is
+running, using the same machinery described above to offload its I/O
+waits into another thread.  When you call :func:`become_guest_for`,
+the existing Trio run with all your existing tasks moves to be
+implemented as a chain of callbacks on top of the host loop. When the
+host loop completes, the same Trio run moves back to running on its
+own. All of this should be completely transparent in normal operation,
+but it does mean you can't have two calls to :func:`become_guest_for`
+active at the same time in the same Trio program.
+
+Here's a detailed example of how to use :func:`become_guest_for` with asyncio:
+
+.. literalinclude:: reference-lowlevel/trio-becomes-asyncio-guest.py
+
+If you run this, you'll get the output (with a half-second delay between each line):
+
+.. code-block:: none
+
+   Hello from asyncio!
+   Trio is still running
+   Hello from asyncio!
+   Trio is still running
+   Hello from asyncio!
+   Trio is still running
+   asyncio program is done, with result: asyncio done!
+
+
 Reference
 ---------
 
 .. autofunction:: start_guest_run
 
+.. autofunction:: become_guest_for
+
 
 .. _live-coroutine-handoff:
 

docs/source/reference-lowlevel/trio-becomes-asyncio-guest.py

@@ -0,0 +1,92 @@
+import asyncio, trio
+
+# This is a drop-in replacement for asyncio.run(), except that you call
+# it from a Trio program instead of from synchronous Python.
+async def asyncio_run_from_trio(coro):
+    aio_task = None
+    raise_cancel = None
+
+    # run_child_host() is one of two functions you must write to adapt
+    # become_guest_for() to a particular host loop setup.
+    # Its job is to start the host loop, and call resume_trio_as_guest()
+    # as soon as the host loop is able to accept callbacks.
+    # resume_trio_as_guest() takes the same keyword arguments
+    # run_sync_soon_threadsafe= (required) and
+    # run_sync_soon_not_threadsafe= (optional)
+    # that you would use with start_guest_run().
+    def run_child_host(resume_trio_as_guest):
+        async def aio_bootstrap():
+            # Save the top-level task so we can cancel it
+            nonlocal aio_task
+            aio_task = asyncio.current_task()
+
+            # Resume running Trio code
+            loop = asyncio.get_running_loop()
+            resume_trio_as_guest(
+                run_sync_soon_threadsafe=loop.call_soon_threadsafe,
+                run_sync_soon_not_threadsafe=loop.call_soon,
+            )
+
+            # Run the asyncio coroutine we were given
+            try:
+                return await coro
+            except asyncio.CancelledError:
+                # If this cancellation was requested by Trio, then
+                # raise_cancel will be non-None and we should call it
+                # to raise whatever exception Trio wants to raise.
+                # Otherwise, the cancellation was requested within
+                # asyncio (like 'asyncio.current_task().cancel()') and
+                # we should let it continue to be represented with an
+                # asyncio exception.
+                if raise_cancel is not None:
+                    raise_cancel()
+                raise
+
+        return asyncio.run(aio_bootstrap())
+
+    # deliver_cancel() is the other one. It gets called when Trio
+    # wants to cancel the call to become_guest_for() (e.g., because
+    # the whole Trio run is shutting down due to a Ctrl+C).
+    # Its job is to get run_child_host() to return soon.
+    # Ideally, it should make run_child_host() call raise_cancel()
+    # to raise a trio.Cancelled exception, so that the cancel scope's
+    # cancelled_caught member accurately reflects whether anything
+    # was interrupted. If all you care about is Ctrl+C working, though,
+    # just stopping the host loop is sufficient.
+    def deliver_cancel(incoming_raise_cancel):
+        # This won't be called until after resume_trio_as_guest() happens,
+        # so we can rely on aio_task being non-None.
+        nonlocal raise_cancel
+        raise_cancel = incoming_raise_cancel
+        aio_task.cancel()
+
+    # Once you have those two, it's just:
+    return await trio.lowlevel.become_guest_for(run_child_host, deliver_cancel)
+
+# The rest of this is a demo of how to use it.
+
+# A tiny asyncio program
+async def asyncio_main():
+    for i in range(5):
+        print("Hello from asyncio!")
+        # This is inside asyncio, so we have to use asyncio APIs
+        await asyncio.sleep(1)
+    return "asyncio done!"
+
+# A simple Trio function to demonstrate that Trio code continues
+# to run while the asyncio loop is running
+async def trio_reminder_task():
+    await trio.sleep(0.5)
+    while True:
+        print("Trio is still running")
+        await trio.sleep(1)
+
+# The code to run asyncio_main and trio_reminder_task in parallel
+async def trio_main():
+    async with trio.open_nursery() as nursery:
+        nursery.start_soon(trio_reminder_task)
+        aio_result = await asyncio_run_from_trio(asyncio_main())
+        print("asyncio program is done, with result:", aio_result)
+        nursery.cancel_scope.cancel()  # stop the trio_reminder_task
+
+trio.run(trio_main)

newsfragments/1652.feature.rst

@@ -0,0 +1,8 @@
+If you want to use :ref:`guest mode <guest-mode>`, but you can't
+easily arrange to start and finish your Trio run while the host loop
+is running, then you're in luck: the newly added :ref:`"inside-out
+guest mode" <inside-out-guest-mode>` allows you to run the host loop
+from within an existing Trio program instead, by calling
+:func:`trio.lowlevel.become_guest_for`.  This is implemented using the
+same machinery as the existing (right-side-in?)  guest mode, and
+unlocks additional ways for Trio to interoperate with other loops.

trio/_core/__init__.py

@@ -61,6 +61,7 @@
     temporarily_detach_coroutine_object,
     permanently_detach_coroutine_object,
     reattach_detached_coroutine_object,
+    become_guest_for,
 )
 
 from ._entry_queue import TrioToken