Add torch.multiprocessing.spawn docs

Summary: Pull Request resolved: pytorch#13846

Differential Revision: D13029595

Pulled By: pietern

fbshipit-source-id: b733b00f7070c18535c31801f20e6e717eec7748

docs/source/multiprocessing.rst

@@ -86,3 +86,36 @@ allocated by the group. If it finds that any of them still exist, they will be
 deallocated. We've tested this method and it proved to be robust to various
 failures. Still, if your system has high enough limits, and ``file_descriptor``
 is a supported strategy, we do not recommend switching to this one.
+
+Spawning subprocesses
+---------------------
+
+.. note::
+
+   Available for Python >= 3.4.
+
+   This depends on the ``spawn`` start method in Python's
+   ``multiprocessing`` package.
+
+Spawning a number of subprocesses to perform some function can be done
+by creating ``Process`` instances and calling ``join`` to wait for
+their completion. This approach works fine when dealing with a single
+subprocess but presents potential issues when dealing with multiple
+processes.
+
+Namely, joining processes sequentially implies they will terminate
+sequentially. If they don't, and the first process does not terminate,
+the process termination will go unnoticed. Also, there are no native
+facilities for error propagation.
+
+The ``spawn`` function below addresses these concerns and takes care
+of error propagation, out of order termination, and will actively
+terminate processes upon detecting an error in one of them.
+
+.. autofunction:: spawn
+
+.. class:: SpawnContext
+
+   Returned by :func:`~spawn` when called with ``join=False``.
+
+   .. automethod:: join

torch/multiprocessing/__init__.py

@@ -37,7 +37,7 @@
 if sys.version_info >= (3, 4):
     """Add helper function to spawn N processes and wait for completion of any of
     them. This depends `mp.get_context` which was added in Python 3.4."""
-    from .spawn import spawn
+    from .spawn import spawn, SpawnContext
 
 
 if sys.platform == 'darwin' or sys.platform == 'win32':

torch/multiprocessing/spawn.py

@@ -114,6 +114,10 @@ def spawn(fn, args=(), nprocs=1, join=True):
         nprocs (int): Number of processes to spawn.
         join (bool): Perform a blocking join on all processes.
 
+    Returns:
+        None if ``join`` is ``True``,
+        :class:`~SpawnContext` if ``join`` is ``False``
+
     """
     mp = multiprocessing.get_context('spawn')
     error_queues = []