From e3aa6b78c8d32b3faa19ccbc9bf1fc9034a7b213 Mon Sep 17 00:00:00 2001 From: Jeff Squyres Date: Fri, 23 Dec 2022 16:42:49 -0500 Subject: [PATCH] docs: A variety of updates Shuffle the docs organization around per discussion on github. Signed-off-by: Jeff Squyres --- docs/Makefile.am | 21 ++++++------- docs/developers/terminology.rst | 8 ++--- docs/faq/general-tuning.rst | 7 ++--- docs/faq/index.rst | 2 -- docs/index.rst | 5 ++-- .../gridengine.rst | 0 .../index.rst | 7 ++--- .../localhost.rst | 0 docs/{running-apps => launching-apps}/lsf.rst | 0 .../pmix-and-prrte.rst | 0 .../prerequisites.rst | 13 ++++---- .../quickstart.rst | 6 ++-- .../slurm.rst | 0 docs/{running-apps => launching-apps}/ssh.rst | 2 +- docs/{running-apps => launching-apps}/tm.rst | 0 docs/{running-apps/tuning.rst => mca.rst} | 20 ++++++++----- docs/quickstart.rst | 4 +-- docs/tuning-apps/index.rst | 11 +++++++ docs/tuning-apps/mpi-io/index.rst | 13 ++++++++ docs/{faq => tuning-apps/mpi-io}/ompio.rst | 0 docs/tuning-apps/mpi-io/romio.rst | 6 ++++ docs/{ => tuning-apps}/networking/cuda.rst | 3 ++ .../networking/ib-and-roce.rst | 3 ++ docs/{ => tuning-apps}/networking/index.rst | 10 +++---- docs/{ => tuning-apps}/networking/iwarp.rst | 3 -- docs/{ => tuning-apps}/networking/ofi.rst | 3 ++ docs/{ => tuning-apps}/networking/rocm.rst | 30 +++++++++---------- .../networking/shared-memory.rst | 3 ++ docs/{ => tuning-apps}/networking/tcp.rst | 3 ++ 29 files changed, 115 insertions(+), 68 deletions(-) rename docs/{running-apps => launching-apps}/gridengine.rst (100%) rename docs/{running-apps => launching-apps}/index.rst (92%) rename docs/{running-apps => launching-apps}/localhost.rst (100%) rename docs/{running-apps => launching-apps}/lsf.rst (100%) rename docs/{running-apps => launching-apps}/pmix-and-prrte.rst (100%) rename docs/{running-apps => launching-apps}/prerequisites.rst (70%) rename docs/{running-apps => launching-apps}/quickstart.rst (98%) rename docs/{running-apps => launching-apps}/slurm.rst (100%) rename docs/{running-apps => launching-apps}/ssh.rst (99%) rename docs/{running-apps => launching-apps}/tm.rst (100%) rename docs/{running-apps/tuning.rst => mca.rst} (97%) create mode 100644 docs/tuning-apps/index.rst create mode 100644 docs/tuning-apps/mpi-io/index.rst rename docs/{faq => tuning-apps/mpi-io}/ompio.rst (100%) create mode 100644 docs/tuning-apps/mpi-io/romio.rst rename docs/{ => tuning-apps}/networking/cuda.rst (99%) rename docs/{ => tuning-apps}/networking/ib-and-roce.rst (98%) rename docs/{ => tuning-apps}/networking/index.rst (73%) rename docs/{ => tuning-apps}/networking/iwarp.rst (91%) rename docs/{ => tuning-apps}/networking/ofi.rst (97%) rename docs/{ => tuning-apps}/networking/rocm.rst (84%) rename docs/{ => tuning-apps}/networking/shared-memory.rst (96%) rename docs/{ => tuning-apps}/networking/tcp.rst (99%) diff --git a/docs/Makefile.am b/docs/Makefile.am index c5aa1aa7336..e963b67f789 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -35,22 +35,23 @@ IMAGE_SOURCE_FILES = \ $(srcdir)/installing-open-mpi/required-support-libraries-dependency-graph.png RST_SOURCE_FILES = \ $(srcdir)/*.rst \ - $(srcdir)/app-debug/*.rst \ - $(srcdir)/building-apps/*.rst \ - $(srcdir)/developers/*.rst \ - $(srcdir)/faq/*.rst \ - $(srcdir)/features/*.rst \ + $(srcdir)/release-notes/*.rst \ $(srcdir)/installing-open-mpi/*.rst \ $(srcdir)/installing-open-mpi/*/*.rst \ + $(srcdir)/features/*.rst \ + $(srcdir)/building-apps/*.rst \ + $(srcdir)/launching-apps/*.rst \ + $(srcdir)/tuning-apps/*.rst \ + $(srcdir)/tuning-apps/*/*.rst \ + $(srcdir)/app-debug/*.rst \ + $(srcdir)/faq/*.rst \ + $(srcdir)/developers/*.rst \ $(srcdir)/license/*.rst \ + $(srcdir)/news/*.rst \ $(srcdir)/man-openmpi/*.rst \ $(srcdir)/man-openmpi/man*/*.rst \ $(srcdir)/man-openshmem/*.rst \ - $(srcdir)/man-openshmem/man*/*.rst \ - $(srcdir)/networking/*.rst \ - $(srcdir)/news/*.rst \ - $(srcdir)/release-notes/*.rst \ - $(srcdir)/running-apps/*.rst + $(srcdir)/man-openshmem/man*/*.rst EXTRA_DIST = \ requirements.txt \ diff --git a/docs/developers/terminology.rst b/docs/developers/terminology.rst index e5b2a0a5d10..2e31c784f62 100644 --- a/docs/developers/terminology.rst +++ b/docs/developers/terminology.rst @@ -8,10 +8,10 @@ terminology in order to make the rest of the discussion easier. Modular Component Architecture (MCA) ------------------------------------ -:doc:`See this section ` for a discussion of the -Modular Component Architecture (MCA). Seriously. Go read it now. -From reading that section, you should understand the following terms -before continuing reading these docs: +:ref:`See this section ` for a discussion of the Modular +Component Architecture (MCA). Seriously. Go read it now. From +reading that section, you should understand the following terms before +continuing reading these docs: * Project * Framework diff --git a/docs/faq/general-tuning.rst b/docs/faq/general-tuning.rst index 414c6ebb0db..51938b9ad01 100644 --- a/docs/faq/general-tuning.rst +++ b/docs/faq/general-tuning.rst @@ -173,10 +173,9 @@ the ``mpi_warn_on_fork`` MCA parameter. For example: shell$ mpirun --mca mpi_warn_on_fork 0 ... Of course, systems that ``dlopen("libmpi.so", ...)`` may not use Open -MPI's ``mpirun``, and therefore may need to use (JMS: this ref no -longer exists -- it moved to running-apps/tuning.rst) a different -mechanism to set MCA parameters -`. +MPI's ``mpirun``, and therefore may need to use (**JMS: this ref no +longer exists -- it moved to elsewhere**) a different mechanism to set +MCA parameters `. ///////////////////////////////////////////////////////////////////////// diff --git a/docs/faq/index.rst b/docs/faq/index.rst index 70304fd47c8..979a9291f98 100644 --- a/docs/faq/index.rst +++ b/docs/faq/index.rst @@ -29,6 +29,4 @@ that they are worth categorizing in an official way. large-clusters - ompio - general-tuning diff --git a/docs/index.rst b/docs/index.rst index 3ef31bf9acb..4af7f9216fa 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -67,10 +67,11 @@ Table of contents features/index validate version-numbering + mca building-apps/index - running-apps/index + launching-apps/index + tuning-apps/index app-debug/index - networking/index faq/index developers/index contributing diff --git a/docs/running-apps/gridengine.rst b/docs/launching-apps/gridengine.rst similarity index 100% rename from docs/running-apps/gridengine.rst rename to docs/launching-apps/gridengine.rst diff --git a/docs/running-apps/index.rst b/docs/launching-apps/index.rst similarity index 92% rename from docs/running-apps/index.rst rename to docs/launching-apps/index.rst index 52352b6012c..71042256fbf 100644 --- a/docs/running-apps/index.rst +++ b/docs/launching-apps/index.rst @@ -1,7 +1,7 @@ .. _label-running-mpi-applications: -Running MPI applications -======================== +Launching MPI applications +========================== Open MPI can launch MPI processes in a wide variety of environments, but they can generally be broken down into two categories: @@ -18,9 +18,8 @@ but they can generally be broken down into two categories: :maxdepth: 1 quickstart - pmix-and-prrte prerequisites - tuning + pmix-and-prrte localhost ssh diff --git a/docs/running-apps/localhost.rst b/docs/launching-apps/localhost.rst similarity index 100% rename from docs/running-apps/localhost.rst rename to docs/launching-apps/localhost.rst diff --git a/docs/running-apps/lsf.rst b/docs/launching-apps/lsf.rst similarity index 100% rename from docs/running-apps/lsf.rst rename to docs/launching-apps/lsf.rst diff --git a/docs/running-apps/pmix-and-prrte.rst b/docs/launching-apps/pmix-and-prrte.rst similarity index 100% rename from docs/running-apps/pmix-and-prrte.rst rename to docs/launching-apps/pmix-and-prrte.rst diff --git a/docs/running-apps/prerequisites.rst b/docs/launching-apps/prerequisites.rst similarity index 70% rename from docs/running-apps/prerequisites.rst rename to docs/launching-apps/prerequisites.rst index 84e6eea175d..2b704b26bc8 100644 --- a/docs/running-apps/prerequisites.rst +++ b/docs/launching-apps/prerequisites.rst @@ -1,12 +1,15 @@ Prerequisites ============= -In general, successful execution of Open MPI jobs requires the ability -to find Open MPI's executables and shared libraries on all nodes at -run time. If these can be via in system-default search paths (e.g., +In general, successful launch of Open MPI jobs requires the ability to +find Open MPI's executables and shared libraries on all nodes at run +time. If these can be via in system-default search paths (e.g., +without the user needing to set or modify ``PATH`` or +``LD_LIBRARY_PATH``), nothing additional needs to be done. For +example: * If Open MPI is installed in ``/usr/bin`` and ``/usr/lib``), that is - usually sufficient. + usually sufficient, and the user does not need to do anything extra. * If Open MPI is installed in a location that is not searched by default, users may need to add ``$prefix/bin`` to their ``PATH`` and ``$libdir`` (which defaults to ``$prefix/lib``) to their @@ -26,4 +29,4 @@ run time. If these can be via in system-default search paths (e.g., If users are unable to add the relevant directories to ``PATH`` and ``LD_LIBRARY_PATH``,the :ref:`mpirun(1) ` ``--prefix`` -option can be used +option *may* be sufficient. diff --git a/docs/running-apps/quickstart.rst b/docs/launching-apps/quickstart.rst similarity index 98% rename from docs/running-apps/quickstart.rst rename to docs/launching-apps/quickstart.rst index 5ad4bfd894a..f877dcfe957 100644 --- a/docs/running-apps/quickstart.rst +++ b/docs/launching-apps/quickstart.rst @@ -1,7 +1,7 @@ -.. _label-quickstart-running-apps: +.. _label-quickstart-launching-apps: -Quick start: Running MPI applications -===================================== +Quick start: Launching MPI applications +======================================= Although this section skips many details, it offers examples that will probably work in many environments. diff --git a/docs/running-apps/slurm.rst b/docs/launching-apps/slurm.rst similarity index 100% rename from docs/running-apps/slurm.rst rename to docs/launching-apps/slurm.rst diff --git a/docs/running-apps/ssh.rst b/docs/launching-apps/ssh.rst similarity index 99% rename from docs/running-apps/ssh.rst rename to docs/launching-apps/ssh.rst index bfc6f1b313b..ca8ba4462c3 100644 --- a/docs/running-apps/ssh.rst +++ b/docs/launching-apps/ssh.rst @@ -3,7 +3,7 @@ Launching with SSH When launching Open MPI jobs in a non-scheduled environment, ``ssh`` is typically used to launch commands on remote nodes. As listed in -the :doc:`quick start section `, +the :doc:`quick start section `, successfully launching MPI applications with ``ssh`` requires the following: diff --git a/docs/running-apps/tm.rst b/docs/launching-apps/tm.rst similarity index 100% rename from docs/running-apps/tm.rst rename to docs/launching-apps/tm.rst diff --git a/docs/running-apps/tuning.rst b/docs/mca.rst similarity index 97% rename from docs/running-apps/tuning.rst rename to docs/mca.rst index 7bd8a827895..cac08953940 100644 --- a/docs/running-apps/tuning.rst +++ b/docs/mca.rst @@ -1,14 +1,18 @@ -.. _label-run-time-tuning: +.. _label-mca: -Run-time tuning -=============== +The Modular Component Architecture (MCA) +======================================== Open MPI is a highly-customizable system; it can be configured via configuration files, command line parameters, and environment -variables. +variables. The main functionality of Open MPI's configuration system +is through the Modular Component Architecture (MCA). -The main functionality of Open MPI's configuration system is through -the Modular Component Architecture (MCA). +* This section describes the MCA itself and how to set MCA parameters at + run time. +* Later sections in this documentation describe different parts of + Open MPI's functionality, and the specific names and values of MCA + parameters that can be used to affect Open MPI's behavior. .. note:: :ref:`The PMIx and PRRTE software packages ` also use the MCA for @@ -16,8 +20,8 @@ the Modular Component Architecture (MCA). ///////////////////////////////////////////////////////////////////////// -The Modular Component Architecture (MCA) ----------------------------------------- +Terminology +----------- The Modular Component Architecture (MCA) is the backbone for much of Open MPI's functionality. It is a series of *projects*, *frameworks*, diff --git a/docs/quickstart.rst b/docs/quickstart.rst index ccecb9907e5..745469b530c 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -16,5 +16,5 @@ provides significantly more detailed information. ` #. :doc:`Quick start: Building MPI applications ` -#. :doc:`Quick start: Running MPI applications - ` +#. :doc:`Quick start: Launching MPI applications + ` diff --git a/docs/tuning-apps/index.rst b/docs/tuning-apps/index.rst new file mode 100644 index 00000000000..34a6760c5da --- /dev/null +++ b/docs/tuning-apps/index.rst @@ -0,0 +1,11 @@ +Run-time tuning MPI applications +================================ + +Once your MPI application is running, there are many different +components that can be tuned to affect behavior at run time. + +.. toctree:: + :maxdepth: 1 + + networking/index + mpi-io/index diff --git a/docs/tuning-apps/mpi-io/index.rst b/docs/tuning-apps/mpi-io/index.rst new file mode 100644 index 00000000000..2b3978c3ef5 --- /dev/null +++ b/docs/tuning-apps/mpi-io/index.rst @@ -0,0 +1,13 @@ +MPI-IO tuning options +===================== + +Open MPI has two main implementations of the MPI-IO functionality: + +1. OMPIO: Open MPI's "native" MPI-IO functionality. +1. ROMIO: `MPICH's `_ implementation of the MPI-IO functionality. + +.. toctree:: + :maxdepth: 1 + + ompio + romio diff --git a/docs/faq/ompio.rst b/docs/tuning-apps/mpi-io/ompio.rst similarity index 100% rename from docs/faq/ompio.rst rename to docs/tuning-apps/mpi-io/ompio.rst diff --git a/docs/tuning-apps/mpi-io/romio.rst b/docs/tuning-apps/mpi-io/romio.rst new file mode 100644 index 00000000000..bc329e4fb0e --- /dev/null +++ b/docs/tuning-apps/mpi-io/romio.rst @@ -0,0 +1,6 @@ +ROMIO +===== + +ROMIO is an implementation of the MPI I/O functions defined in version +two of the Message Passing Interface specification. It has been +ported over to Open MPI from `MPICH `_. diff --git a/docs/networking/cuda.rst b/docs/tuning-apps/networking/cuda.rst similarity index 99% rename from docs/networking/cuda.rst rename to docs/tuning-apps/networking/cuda.rst index 4a3a817ed26..c6f44955e51 100644 --- a/docs/networking/cuda.rst +++ b/docs/tuning-apps/networking/cuda.rst @@ -1,6 +1,9 @@ CUDA ==== +.. error:: TODO This section needs to be converted from FAQ Q&A style + to regular documentation style. + How do I build Open MPI with CUDA-aware support? ------------------------------------------------ diff --git a/docs/networking/ib-and-roce.rst b/docs/tuning-apps/networking/ib-and-roce.rst similarity index 98% rename from docs/networking/ib-and-roce.rst rename to docs/tuning-apps/networking/ib-and-roce.rst index f4afd682120..32ba7936cc2 100644 --- a/docs/networking/ib-and-roce.rst +++ b/docs/tuning-apps/networking/ib-and-roce.rst @@ -1,6 +1,9 @@ InifiniBand / RoCE support ========================== +.. error:: TODO This section needs to be converted from FAQ Q&A style + to regular documentation style. + How are InfiniBand / RoCE devices supported in Open MPI? -------------------------------------------------------- diff --git a/docs/networking/index.rst b/docs/tuning-apps/networking/index.rst similarity index 73% rename from docs/networking/index.rst rename to docs/tuning-apps/networking/index.rst index 39600a6641f..00aa0f39df5 100644 --- a/docs/networking/index.rst +++ b/docs/tuning-apps/networking/index.rst @@ -1,10 +1,10 @@ -Networking system support -========================= +Networking support +================== Open MPI supports a variety of different networking transports for -off-node communication. Not all of these transports are supported or -available on every platform. Many require specialized hardware, -operating system drivers, and/or network transport libraries. +off-node communication. Not all transports are supported or available +on every platform. Many require specialized hardware, operating +system drivers, and/or userspace network transport libraries. When Open MPI is being configured, it will search for a variety of network transport libraries (and corresponding development header diff --git a/docs/networking/iwarp.rst b/docs/tuning-apps/networking/iwarp.rst similarity index 91% rename from docs/networking/iwarp.rst rename to docs/tuning-apps/networking/iwarp.rst index 08150682ba1..a24b3c73fbb 100644 --- a/docs/networking/iwarp.rst +++ b/docs/tuning-apps/networking/iwarp.rst @@ -1,9 +1,6 @@ iWARP Support ============= -How are iWARP devices supported in Open MPI? --------------------------------------------- - Open MPI's support for iWARP devices has changed over time. In the Open MPI |ompi_series| series, iWARP devices are diff --git a/docs/networking/ofi.rst b/docs/tuning-apps/networking/ofi.rst similarity index 97% rename from docs/networking/ofi.rst rename to docs/tuning-apps/networking/ofi.rst index 8abf3a747b4..4c9ba56290c 100644 --- a/docs/networking/ofi.rst +++ b/docs/tuning-apps/networking/ofi.rst @@ -1,6 +1,9 @@ OpenFabrics Interfaces (OFI) / Libfabric support ================================================ +.. error:: TODO This section needs to be converted from FAQ Q&A style + to regular documentation style. + What is OFI / Libfabric? ------------------------ diff --git a/docs/networking/rocm.rst b/docs/tuning-apps/networking/rocm.rst similarity index 84% rename from docs/networking/rocm.rst rename to docs/tuning-apps/networking/rocm.rst index adbc1713512..9173d25f7fd 100644 --- a/docs/networking/rocm.rst +++ b/docs/tuning-apps/networking/rocm.rst @@ -9,7 +9,7 @@ accelerators. More information can be found at the following Building Open MPI with ROCm support ------------------------------------------------------------------------- +----------------------------------- ROCm-aware support means that the MPI library can send and receive data from AMD GPU device buffers directly. As of today, ROCm support @@ -38,21 +38,21 @@ It should look something like: .. code-block:: sh - # Configure UCX with ROCm support - shell$ cd ucx - shell$ ./configure --prefix=/path/to/ucx-rocm-install \ + # Configure UCX with ROCm support + shell$ cd ucx + shell$ ./configure --prefix=/path/to/ucx-rocm-install \ --with-rocm=/opt/rocm --without-knem - # Configure Open MPI with UCX and ROCm support - shell$ cd ompi - shell$ ./configure --with-rocm=/opt/rocm \ - --with-ucx=/path/to/ucx-rocm-install \ - + # Configure Open MPI with UCX and ROCm support + shell$ cd ompi + shell$ ./configure --with-rocm=/opt/rocm \ + --with-ucx=/path/to/ucx-rocm-install \ + ///////////////////////////////////////////////////////////////////////// Checking that Open MPI has been built with ROCm support --------------------------------------------------------------------------- +------------------------------------------------------- Verify that Open MPI has been built with ROCm using the :ref:`ompi_info(1) ` command: @@ -60,7 +60,7 @@ Verify that Open MPI has been built with ROCm using the .. code-block:: sh # Use ompi_info to verify ROCm support in Open MPI - shell$ ./ompi_info |grep "MPI extensions" + shell$ ./ompi_info | grep "MPI extensions" MPI extensions: affinity, cuda, ftmpi, rocm ///////////////////////////////////////////////////////////////////////// @@ -91,7 +91,7 @@ for details. ///////////////////////////////////////////////////////////////////////// Runtime querying of ROCm support in Open MPI ------------------------------------------------------------------------- +-------------------------------------------- Starting with Open MPI v5.0.0 :ref:`MPIX_Query_rocm_support(3) ` is available as an extension to check @@ -102,7 +102,7 @@ function, the code needs to include ``mpi-ext.h``. Note that ///////////////////////////////////////////////////////////////////////// Collective component supporting ROCm device memory ---------------------------------------------------------------------------- +-------------------------------------------------- The `UCC `_ based collective component in Open MPI can be configured and compiled to include ROCm support. @@ -111,14 +111,14 @@ An example for configure UCC and Open MPI with ROCm is shown below: .. code-block:: - #Configure and compile UCC with ROCm support + # Configure and compile UCC with ROCm support shell$ cd ucc shell$ ./configure --with-rocm=/opt/rocm \ --with-ucx=/path/to/ucx-rocm-install \ --prefix=/path/to/ucc-rocm-install shell$ make -j && make install - #Configure and compile Open MPI with UCX, UCC, and ROCm support + # Configure and compile Open MPI with UCX, UCC, and ROCm support shell$ cd ompi shell$ ./configure --with-rocm=/opt/rocm \ --with-ucx=/path/to/ucx-rocm-install \ diff --git a/docs/networking/shared-memory.rst b/docs/tuning-apps/networking/shared-memory.rst similarity index 96% rename from docs/networking/shared-memory.rst rename to docs/tuning-apps/networking/shared-memory.rst index 8f522844a82..caf723de4cc 100644 --- a/docs/networking/shared-memory.rst +++ b/docs/tuning-apps/networking/shared-memory.rst @@ -1,6 +1,9 @@ Shared Memory ============= +.. error:: TODO This section needs to be converted from FAQ Q&A style + to regular documentation style. + What is the sm BTL? ------------------- diff --git a/docs/networking/tcp.rst b/docs/tuning-apps/networking/tcp.rst similarity index 99% rename from docs/networking/tcp.rst rename to docs/tuning-apps/networking/tcp.rst index 42d0572cc6b..07dc893e036 100644 --- a/docs/networking/tcp.rst +++ b/docs/tuning-apps/networking/tcp.rst @@ -1,6 +1,9 @@ TCP === +.. error:: TODO This section needs to be converted from FAQ Q&A style + to regular documentation style. + How do I specify to use the IP network for MPI messages? --------------------------------------------------------