Fix all warnings in Python docs (#13789)

The Sphinx documentation has historically been warning-filled, which makes it difficult to identify when there are real issues like missing APIs. This PR fixes all the current issues and converts warnings to errors during the build, ensuring that doc builds are reliable indicators of issues in the future. I will say that there are a few changes that may not be exactly what we want, particularly in cases of including APIs that may not be documented in exactly the same way in pandas. However, I think we'd be better off merging this PR so that we can get to a 0 warnings state and then work through further improvements in follow-ups where the build will be more robust. Here is an inexhaustive list of the most significant changes: - Adds all missing BaseIndex APIs. The goal of BaseIndex is to provide an abstract interface defining all functions that should match pandas.Index, but up until now some methods were missing. This PR does not implement any new ones, it either lifts existing implementations up from subclasses (where those implementations are generic for all Index types) or it simply defines them as returning NotImplemented. The result is that all methods at least exist so that docs don't complain. - Cleans up the listed APIs in rst files so that all existing APIs are included somewhere and no nonexistent APIs are listed anywhere. APIs that don't have an exact equivalent in the pandas docs are given a new home in these docs. That includes pieces like extension dtypes, which were previously documented in the user guide and therefore weren't part of any summary list (causing warnings). - Fixed missing dependencies for doc notebooks. - Fixed various formatting issues with docstrings, especially around bulleted lists that were missing the requisite spacing to be rendered correctly. - Fixing header ordering (going from level 1 to level 3 headings is a warning in Sphinx) and links in notebooks. Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - GALI PREM SAGAR (https://github.com/galipremsagar) URL: #13789
rapidsai · Aug 7, 2023 · e92de81 · e92de81
1 parent 7eb4071
commit e92de81
Show file tree

Hide file tree

Showing 39 changed files with 640 additions and 411 deletions.
diff --git a/ci/build_docs.sh b/ci/build_docs.sh
@@ -38,20 +38,20 @@ popd
 
 rapids-logger "Build Python docs"
 pushd docs/cudf
-sphinx-build -b dirhtml source _html
-sphinx-build -b text source _text
+make dirhtml
+make text
 mkdir -p "${RAPIDS_DOCS_DIR}/cudf/"{html,txt}
-mv _html/* "${RAPIDS_DOCS_DIR}/cudf/html"
-mv _text/* "${RAPIDS_DOCS_DIR}/cudf/txt"
+mv build/dirhtml/* "${RAPIDS_DOCS_DIR}/cudf/html"
+mv build/text/* "${RAPIDS_DOCS_DIR}/cudf/txt"
 popd
 
 rapids-logger "Build dask-cuDF Sphinx docs"
 pushd docs/dask_cudf
-sphinx-build -b dirhtml source _html
-sphinx-build -b text source _text
+make dirhtml
+make text
 mkdir -p "${RAPIDS_DOCS_DIR}/dask-cudf/"{html,txt}
-mv _html/* "${RAPIDS_DOCS_DIR}/dask-cudf/html"
-mv _text/* "${RAPIDS_DOCS_DIR}/dask-cudf/txt"
+mv build/dirhtml/* "${RAPIDS_DOCS_DIR}/dask-cudf/html"
+mv build/text/* "${RAPIDS_DOCS_DIR}/dask-cudf/txt"
 popd
 
 rapids-upload-docs
diff --git a/conda/environments/all_cuda-118_arch-x86_64.yaml b/conda/environments/all_cuda-118_arch-x86_64.yaml
@@ -46,6 +46,7 @@ dependencies:
 - libkvikio==23.10.*
 - librdkafka>=1.9.0,<1.10.0a0
 - librmm==23.10.*
+- make
 - mimesis>=4.1.0
 - moto>=4.0.8
 - msgpack-python

diff --git a/conda/environments/all_cuda-120_arch-x86_64.yaml b/conda/environments/all_cuda-120_arch-x86_64.yaml
@@ -45,6 +45,7 @@ dependencies:
 - libkvikio==23.10.*
 - librdkafka>=1.9.0,<1.10.0a0
 - librmm==23.10.*
+- make
 - mimesis>=4.1.0
 - moto>=4.0.8
 - msgpack-python

diff --git a/dependencies.yaml b/dependencies.yaml
@@ -374,12 +374,15 @@ dependencies:
     common:
       - output_types: [conda]
         packages:
+          - dask-cuda==23.10.*
           - doxygen=1.8.20
+          - make
           - myst-nb
           - nbsphinx
           - numpydoc
           - pandoc
           - pydata-sphinx-theme
+          - scipy
           - sphinx
           - sphinx-autobuild
           - sphinx-copybutton

diff --git a/docs/cudf/Makefile b/docs/cudf/Makefile
@@ -2,7 +2,7 @@
 #
 
 # You can set these variables from the command line.
-SPHINXOPTS    = -n -v
+SPHINXOPTS    = -n -v -W --keep-going
 SPHINXBUILD   = sphinx-build
 SPHINXPROJ    = cudf
 SOURCEDIR     = source

diff --git a/docs/cudf/source/api_docs/dataframe.rst b/docs/cudf/source/api_docs/dataframe.rst
@@ -149,9 +149,9 @@ Computations / descriptive stats
    DataFrame.quantile
    DataFrame.rank
    DataFrame.round
+   DataFrame.scale
    DataFrame.skew
    DataFrame.sum
-   DataFrame.sum_of_squares
    DataFrame.std
    DataFrame.var
    DataFrame.nunique
@@ -219,6 +219,7 @@ Reshaping, sorting, transposing
    DataFrame.sort_index
    DataFrame.nlargest
    DataFrame.nsmallest
+   DataFrame.swaplevel
    DataFrame.stack
    DataFrame.unstack
    DataFrame.melt
@@ -251,11 +252,17 @@ Serialization / IO / conversion
 .. autosummary::
    :toctree: api/
 
+   DataFrame.deserialize
+   DataFrame.device_deserialize
+   DataFrame.device_serialize
    DataFrame.from_arrow
    DataFrame.from_dict
    DataFrame.from_pandas
    DataFrame.from_records
    DataFrame.hash_values
+   DataFrame.host_deserialize
+   DataFrame.host_serialize
+   DataFrame.serialize
    DataFrame.to_arrow
    DataFrame.to_dict
    DataFrame.to_dlpack
@@ -270,3 +277,5 @@ Serialization / IO / conversion
    DataFrame.to_feather
    DataFrame.to_records
    DataFrame.to_string
+   DataFrame.values
+   DataFrame.values_host
diff --git a/docs/cudf/source/api_docs/extension_dtypes.rst b/docs/cudf/source/api_docs/extension_dtypes.rst
@@ -0,0 +1,176 @@
+================
+Extension Dtypes
+================
+.. currentmodule:: cudf.core.dtypes
+
+cuDF supports a number of extension dtypes that build on top of the types that pandas supports. These dtypes are not directly available in pandas, which instead relies on object dtype arrays that run at Python rather than native speeds. The following dtypes are supported:
+
+
+cudf.CategoricalDtype
+=====================
+.. autosummary::
+   :toctree: api/
+   :template: autosummary/class_without_autosummary.rst
+
+   CategoricalDtype
+
+
+Properties and Methods
+----------------------
+.. autosummary::
+   :toctree: api/
+
+    CategoricalDtype.categories
+    CategoricalDtype.construct_from_string
+    CategoricalDtype.deserialize
+    CategoricalDtype.device_deserialize
+    CategoricalDtype.device_serialize
+    CategoricalDtype.from_pandas
+    CategoricalDtype.host_deserialize
+    CategoricalDtype.host_serialize
+    CategoricalDtype.is_dtype
+    CategoricalDtype.name
+    CategoricalDtype.ordered
+    CategoricalDtype.serialize
+    CategoricalDtype.str
+    CategoricalDtype.to_pandas
+    CategoricalDtype.type
+
+
+cudf.Decimal32Dtype
+===================
+.. autosummary::
+   :toctree: api/
+   :template: autosummary/class_without_autosummary.rst
+
+   Decimal32Dtype
+
+Properties and Methods
+----------------------
+.. autosummary::
+   :toctree: api/
+
+   Decimal32Dtype.ITEMSIZE
+   Decimal32Dtype.MAX_PRECISION
+   Decimal32Dtype.deserialize
+   Decimal32Dtype.device_deserialize
+   Decimal32Dtype.device_serialize
+   Decimal32Dtype.from_arrow
+   Decimal32Dtype.host_deserialize
+   Decimal32Dtype.host_serialize
+   Decimal32Dtype.is_dtype
+   Decimal32Dtype.itemsize
+   Decimal32Dtype.precision
+   Decimal32Dtype.scale
+   Decimal32Dtype.serialize
+   Decimal32Dtype.str
+   Decimal32Dtype.to_arrow
+
+cudf.Decimal64Dtype
+===================
+.. autosummary::
+   :toctree: api/
+   :template: autosummary/class_without_autosummary.rst
+
+   Decimal64Dtype
+
+Properties and Methods
+----------------------
+.. autosummary::
+   :toctree: api/
+
+   Decimal64Dtype.ITEMSIZE
+   Decimal64Dtype.MAX_PRECISION
+   Decimal64Dtype.deserialize
+   Decimal64Dtype.device_deserialize
+   Decimal64Dtype.device_serialize
+   Decimal64Dtype.from_arrow
+   Decimal64Dtype.host_deserialize
+   Decimal64Dtype.host_serialize
+   Decimal64Dtype.is_dtype
+   Decimal64Dtype.itemsize
+   Decimal64Dtype.precision
+   Decimal64Dtype.scale
+   Decimal64Dtype.serialize
+   Decimal64Dtype.str
+   Decimal64Dtype.to_arrow
+
+cudf.Decimal128Dtype
+====================
+.. autosummary::
+   :toctree: api/
+   :template: autosummary/class_without_autosummary.rst
+
+   Decimal128Dtype
+
+Properties and Methods
+----------------------
+.. autosummary::
+   :toctree: api/
+
+   Decimal128Dtype.ITEMSIZE
+   Decimal128Dtype.MAX_PRECISION
+   Decimal128Dtype.deserialize
+   Decimal128Dtype.device_deserialize
+   Decimal128Dtype.device_serialize
+   Decimal128Dtype.from_arrow
+   Decimal128Dtype.host_deserialize
+   Decimal128Dtype.host_serialize
+   Decimal128Dtype.is_dtype
+   Decimal128Dtype.itemsize
+   Decimal128Dtype.precision
+   Decimal128Dtype.scale
+   Decimal128Dtype.serialize
+   Decimal128Dtype.str
+   Decimal128Dtype.to_arrow
+
+cudf.ListDtype
+==============
+.. autosummary::
+   :toctree: api/
+   :template: autosummary/class_without_autosummary.rst
+
+   ListDtype
+
+Properties and Methods
+----------------------
+.. autosummary::
+   :toctree: api/
+
+   ListDtype.deserialize
+   ListDtype.device_deserialize
+   ListDtype.device_serialize
+   ListDtype.element_type
+   ListDtype.from_arrow
+   ListDtype.host_deserialize
+   ListDtype.host_serialize
+   ListDtype.is_dtype
+   ListDtype.leaf_type
+   ListDtype.serialize
+   ListDtype.to_arrow
+   ListDtype.type
+
+cudf.StructDtype
+================
+.. autosummary::
+   :toctree: api/
+   :template: autosummary/class_without_autosummary.rst
+
+   StructDtype
+
+Properties and Methods
+----------------------
+.. autosummary::
+   :toctree: api/
+
+   StructDtype.deserialize
+   StructDtype.device_deserialize
+   StructDtype.device_serialize
+   StructDtype.fields
+   StructDtype.from_arrow
+   StructDtype.host_deserialize
+   StructDtype.host_serialize
+   StructDtype.is_dtype
+   StructDtype.serialize
+   StructDtype.to_arrow
+   StructDtype.type
diff --git a/docs/cudf/source/api_docs/index.rst b/docs/cudf/source/api_docs/index.rst
@@ -22,3 +22,4 @@ This page provides a list of all publicly accessible modules, methods and classe
     list_handling
     struct_handling
     options
+    extension_dtypes
diff --git a/docs/cudf/source/api_docs/index_objects.rst b/docs/cudf/source/api_docs/index_objects.rst
@@ -21,9 +21,10 @@ Properties
 .. autosummary::
    :toctree: api/
 
+   Index.dtype
+   Index.duplicated
    Index.empty
    Index.has_duplicates
-   Index.duplicated
    Index.hasnans
    Index.is_monotonic
    Index.is_monotonic_increasing
@@ -52,7 +53,6 @@ Modifying and computations
    Index.is_floating
    Index.is_integer
    Index.is_interval
-   Index.is_mixed
    Index.is_numeric
    Index.is_object
    Index.min
@@ -93,6 +93,13 @@ Conversion
    :toctree: api/
 
    Index.astype
+   Index.deserialize
+   Index.device_deserialize
+   Index.device_serialize
+   Index.host_deserialize
+   Index.host_serialize
+   Index.serialize
+   Index.tolist
    Index.to_arrow
    Index.to_cupy
    Index.to_list
@@ -110,6 +117,7 @@ Sorting
    :toctree: api/
 
    Index.argsort
+   Index.find_label_range
    Index.searchsorted
    Index.sort_values
 
@@ -141,6 +149,13 @@ Selecting
    Index.get_slice_bound
    Index.isin
 
+String Operations
+~~~~~~~~~~~~~~~~~
+.. autosummary::
+   :toctree: api/
+
+   Index.str
+
 .. _api.numericindex:
 
 Numeric Index
@@ -185,6 +200,7 @@ IntervalIndex
 -------------
 .. autosummary::
    :toctree: api/
+   :template: autosummary/class_without_autosummary.rst
 
    IntervalIndex
 

diff --git a/docs/cudf/source/api_docs/io.rst b/docs/cudf/source/api_docs/io.rst
@@ -39,6 +39,8 @@ Parquet
    :template: autosummary/class_with_autosummary.rst
 
    cudf.io.parquet.ParquetDatasetWriter
+   cudf.io.parquet.ParquetDatasetWriter.close
+   cudf.io.parquet.ParquetDatasetWriter.write_table
 
 
 ORC

diff --git a/docs/cudf/source/api_docs/list_handling.rst b/docs/cudf/source/api_docs/list_handling.rst
@@ -5,6 +5,12 @@ List handling
 lists and apply list methods to it. These can be accessed like
 ``Series.list.<function/property>``.
 
+.. currentmodule:: cudf
+.. autosummary::
+   :toctree: api/
+
+   Series.list
+
 .. currentmodule:: cudf.core.column.lists.ListMethods
 .. autosummary::
    :toctree: api/