Merge pull request #557 from djhoese/precommit-mypy

Add more pre-commit checks (mccabe, bandit, mypy, etc)

.bandit

@@ -0,0 +1,3 @@
+[bandit]
+skips: B506
+exclude: pyresample/test,pyresample/version.py,versioneer.py

.github/ISSUE_TEMPLATE.md

@@ -6,7 +6,7 @@
 ```
 #### Problem description
 
-[this should also explain **why** the current behaviour is a problem and why the 
+[this should also explain **why** the current behaviour is a problem and why the
 expected output is a better solution.]
 
 #### Expected Output

.github/workflows/ci.yaml

@@ -101,4 +101,3 @@ jobs:
         uses: AndreMiras/coveralls-python-action@develop
         with:
           parallel-finished: true
-

.pre-commit-config.yaml

@@ -1,11 +1,38 @@
 exclude: '^$'
 fail_fast: false
 repos:
--   repo: https://github.com/pycqa/flake8
-    rev: 6.1.0
-    hooks:
-    - id: flake8
-      additional_dependencies: [flake8-docstrings, flake8-debugger, flake8-bugbear]
+- repo: https://github.com/astral-sh/ruff-pre-commit
+  rev: 'v0.1.6'
+  hooks:
+  - id: ruff
+# once https://github.com/astral-sh/ruff/issues/2402 is fully resolved then we can get rid of flake8:
+- repo: https://github.com/pycqa/flake8
+  rev: 6.1.0
+  hooks:
+  - id: flake8
+- repo: https://github.com/pre-commit/pre-commit-hooks
+  rev: v4.5.0
+  hooks:
+    - id: trailing-whitespace
+    - id: end-of-file-fixer
+    - id: check-yaml
+      args: [--unsafe]
+- repo: https://github.com/PyCQA/bandit
+  rev: '1.7.5' # Update me!
+  hooks:
+    - id: bandit
+      args: [--ini, .bandit]
+- repo: https://github.com/pre-commit/mirrors-mypy
+  rev: 'v1.6.1'  # Use the sha / tag you want to point at
+  hooks:
+    - id: mypy
+      additional_dependencies:
+        - types-docutils
+        - types-pkg-resources
+        - types-PyYAML
+        - types-requests
+        - type_extensions
+      args: ["--python-version", "3.9", "--ignore-missing-imports"]
 - repo: https://github.com/pycqa/isort
   rev: 5.12.0
   hooks:
@@ -14,4 +41,4 @@ repos:
 ci:
   # To trigger manually, comment on a pull request with "pre-commit.ci autofix"
   autofix_prs: false
-
+  skip: [bandit]

CITATION

@@ -1 +1 @@
-To find out how to reference pyresample, go to https://doi.org/10.5281/zenodo.3372769 and choose your favourite citation format on the bottom of the right hand side-bar.
+To find out how to reference pyresample, go to https://doi.org/10.5281/zenodo.3372769 and choose your favourite citation format on the bottom of the right hand side-bar.

MANIFEST.in

@@ -3,7 +3,8 @@ recursive-include docs/source *
 include pyresample/test/test_files/*
 include LICENSE.txt
 include pyresample/ewa/_fornav_templates.*
+include pyresample/py.typed
 include versioneer.py
 include pyresample/version.py
 include README.md
-recursive-include pyresample *.pyx
+recursive-include pyresample *.pyx

README.md

@@ -23,7 +23,7 @@ Pyresample offers multiple resampling algorithms including:
 - Bilinear
 
 For nearest neighbor and bilinear interpolation pyresample uses a kd-tree
-approach by using the fast KDTree implementation provided by the 
+approach by using the fast KDTree implementation provided by the
 [pykdtree](https://github.com/storpipfugl/pykdtree) library.
 Pyresample works with numpy arrays and numpy masked arrays. Interfaces to
 XArray objects (including dask array support) are provided in separate
@@ -38,4 +38,3 @@ PyTroll group and related packages.
 Citation
 ----------
 Hoese, D., Raspaud, M., Lahtinen, P., Roberts, W., Lavergne, et al. (2020). pytroll/pyresample: Version 1.16.0. Zenodo. [https://doi.org/10.5281/zenodo.3372769](https://doi.org/10.5281/zenodo.3372769)
-

changelog_pre1.9.1.rst

@@ -1283,6 +1283,3 @@ Other
 - No commit message. [StorPipfugl]
 - Moved to Google Code under GPLv3 license. [StorPipfugl]
 - moved to Google Code. [StorPipfugl]
-
-
-

docs/README.md

@@ -10,4 +10,4 @@ Pyresample's API has changed (new functions, modules, classes, etc) then the
 API documentation should be regenerated before running the above make
 command.
 
-    sphinx-apidoc -f -T -o source/api ../pyresample ../pyresample/test
+    sphinx-apidoc -f -T -o source/api ../pyresample ../pyresample/test

docs/source/conf.py

@@ -11,6 +11,7 @@
 # All configuration values have a default; values that are commented out
 # serve to show the default.
 """Sphinx documentation configuration."""
+from __future__ import annotations
 
 import os
 import sys
@@ -111,7 +112,7 @@
 
 # List of directories, relative to source directory, that shouldn't be searched
 # for source files.
-exclude_trees = []
+exclude_trees: list[str] = []
 
 # The reST default role (used for this markup: `text`) to use for all documents.
 # default_role = None

docs/source/dev_guide/index.rst

@@ -34,4 +34,4 @@ The documentation is split into 4 main groups:
    Put another way, the user knows what they want to do, the how-to shows them
    how to do it. How-tos should still not go into any more detail than
    necessary and try to depend on :doc:`../concepts/index`.
-4. Reference: API docs and other low-level information.
+4. Reference: API docs and other low-level information.

docs/source/howtos/data_reduce.rst

@@ -1,10 +1,10 @@
 Reduction of swath data
 =======================
 Given a swath and a cartesian grid or grid lons and lats, pyresample can reduce the swath data
-to only the relevant part covering the grid area. The reduction is coarse in order not to risk removing 
+to only the relevant part covering the grid area. The reduction is coarse in order not to risk removing
 relevant data.
 
-From **data_reduce** the function **swath_from_lonlat_grid** can be used to reduce the swath data set to the 
+From **data_reduce** the function **swath_from_lonlat_grid** can be used to reduce the swath data set to the
 area covering the lon lat grid
 
 .. doctest::
@@ -14,7 +14,7 @@ area covering the lon lat grid
  >>> area_def = geometry.AreaDefinition('areaD', 'Europe (3km, HRV, VTC)', 'areaD',
  ...                                {'a': '6378144.0', 'b': '6356759.0',
  ...                                 'lat_0': '50.00', 'lat_ts': '50.00',
- ...                                 'lon_0': '8.00', 'proj': 'stere'}, 
+ ...                                 'lon_0': '8.00', 'proj': 'stere'},
  ...                                800, 800,
  ...                                [-1370912.72, -909968.64,
  ...                                 1029087.28, 1490031.36])
@@ -23,32 +23,31 @@ area covering the lon lat grid
  >>> lats = np.fromfunction(lambda y, x: 75 - y, (50, 10))
  >>> grid_lons, grid_lats = area_def.get_lonlats()
  >>> reduced_lons, reduced_lats, reduced_data = \
- ... 				data_reduce.swath_from_lonlat_grid(grid_lons, grid_lats, 
- ...				lons, lats, data, 
+ ... 				data_reduce.swath_from_lonlat_grid(grid_lons, grid_lats,
+ ...				lons, lats, data,
  ...				radius_of_influence=3000)
- 
+
 **radius_of_influence** is used to calculate a buffer zone around the grid where swath data points
 are not reduced.
- 
+
 The function **get_valid_index_from_lonlat_grid** returns a boolean array of same size as the swath
 indicating the relevant swath data points compared to the grid
- 
+
 .. doctest::
 
  >>> import numpy as np
  >>> from pyresample import geometry, data_reduce
  >>> area_def = geometry.AreaDefinition('areaD', 'Europe (3km, HRV, VTC)', 'areaD',
  ...                                {'a': '6378144.0', 'b': '6356759.0',
  ...                                 'lat_0': '50.00', 'lat_ts': '50.00',
- ...                                 'lon_0': '8.00', 'proj': 'stere'}, 
+ ...                                 'lon_0': '8.00', 'proj': 'stere'},
  ...                                800, 800,
  ...                                [-1370912.72, -909968.64,
  ...                                 1029087.28, 1490031.36])
  >>> data = np.fromfunction(lambda y, x: y*x, (50, 10))
  >>> lons = np.fromfunction(lambda y, x: 3 + x, (50, 10))
  >>> lats = np.fromfunction(lambda y, x: 75 - y, (50, 10))
  >>> grid_lons, grid_lats = area_def.get_lonlats()
- >>> valid_index = data_reduce.get_valid_index_from_lonlat_grid(grid_lons, grid_lats, 
- ...						lons, lats, 
+ >>> valid_index = data_reduce.get_valid_index_from_lonlat_grid(grid_lons, grid_lats,
+ ...						lons, lats,
  ...						radius_of_influence=3000)
-

docs/source/howtos/geo_def.rst

@@ -202,4 +202,3 @@ using the **projection_x_coord** or **projection_y_coords** property of a geogra
  >>> area_def = AreaDefinition(area_id, description, proj_id, projection,
  ...                           width, height, area_extent)
  >>> proj_x_range = area_def.projection_x_coords
-

docs/source/howtos/geo_filter.rst

@@ -15,12 +15,12 @@ a full globe Plate Carrée projection using an 8x8 filter mask
  >>> import numpy as np
  >>> from pyresample import geometry, geo_filter
  >>> lons = np.array([-170, -30, 30, 170])
- >>> lats = np.array([20, -40, 50, -80]) 
+ >>> lats = np.array([20, -40, 50, -80])
  >>> swath_def = geometry.SwathDefinition(lons, lats)
  >>> data = np.array([1, 2, 3, 4])
- >>> filter_area = geometry.AreaDefinition('test', 'test', 'test', 
+ >>> filter_area = geometry.AreaDefinition('test', 'test', 'test',
  ...         {'proj' : 'eqc', 'lon_0' : 0.0, 'lat_0' : 0.0},
- ...           8, 8,                                               
+ ...           8, 8,
  ...          (-20037508.34, -10018754.17, 20037508.34, 10018754.17)
  ...		 )
  >>> filter = np.array([[1, 1, 1, 1, 0, 0, 0, 0],
@@ -38,4 +38,4 @@ a full globe Plate Carrée projection using an 8x8 filter mask
 Input swath_def and data must match as described in :ref:`swath`.
 
 The returned data will always have a 1D geometry_def and if multiple channels are present the filtered
-data will have the shape (number_of_points, channels).
+data will have the shape (number_of_points, channels).

docs/source/howtos/grid.rst

@@ -17,13 +17,13 @@ using the method **get_array_from_linesample**
 Resampling can be done using descendants of **ImageContainer** and calling their **resample** method.
 
 An **ImageContainerQuick** object allows for the grid to be resampled to a new area defintion
-using an approximate (but fast) nearest neighbour method. 
-Resampling an object of type **ImageContainerQuick** returns a new object of type **ImageContainerQuick**. 
+using an approximate (but fast) nearest neighbour method.
+Resampling an object of type **ImageContainerQuick** returns a new object of type **ImageContainerQuick**.
 
 An **ImageContainerNearest** object allows for the grid to be resampled to a new area defintion (or swath definition)
 using an accurate kd-tree method.
-Resampling an object of type **ImageContainerNearest** returns a new object of 
-type **ImageContainerNearest**. 
+Resampling an object of type **ImageContainerNearest** returns a new object of
+type **ImageContainerNearest**.
 
 .. doctest::
 
@@ -32,7 +32,7 @@ type **ImageContainerNearest**.
  >>> area_def = geometry.AreaDefinition('areaD', 'Europe (3km, HRV, VTC)', 'areaD',
  ...                                {'a': '6378144.0', 'b': '6356759.0',
  ...                                 'lat_0': '50.00', 'lat_ts': '50.00',
- ...                                 'lon_0': '8.00', 'proj': 'stere'}, 
+ ...                                 'lon_0': '8.00', 'proj': 'stere'},
  ...                                800, 800,
  ...                                [-1370912.72, -909968.64,
  ...                                 1029087.28, 1490031.36])
@@ -54,7 +54,7 @@ type **ImageContainerNearest**.
 
 Data is assumed to be a numpy array of shape (rows, cols) or (rows, cols, channels).
 
-Masked arrays can be used as data input. In order to have undefined pixels masked out instead of 
+Masked arrays can be used as data input. In order to have undefined pixels masked out instead of
 assigned a fill value set **fill_value=None** when calling **resample_area_***.
 
 Using **ImageContainerQuick** the risk of image artifacts increases as the distance
@@ -81,7 +81,7 @@ The constructor arguments of an ImageContainer object can be changed as attribut
  >>> msg_con_nn = image.ImageContainerNearest(data, msg_area, radius_of_influence=50000)
  >>> msg_con_nn.radius_of_influence = 45000
  >>> msg_con_nn.fill_value = -99
- 
+
 Multi channel images
 ********************
 
@@ -104,23 +104,23 @@ If the dataset has several channels the last index of the data array specifies t
  >>> channel3 = np.ones((3712, 3712)) * 3
  >>> data = np.dstack((channel1, channel2, channel3))
  >>> msg_con_nn = image.ImageContainerNearest(data, msg_area, radius_of_influence=50000)
- 
+
 
 Segmented resampling
 ********************
 
 Pyresample calculates the result in segments in order to reduce memory footprint. This is controlled by the **segments** contructor keyword argument. If no **segments** argument is given pyresample will estimate the number of segments to use.
- 
+
 Forcing quick resampling to use 4 resampling segments:
- 
+
 .. doctest::
 
  >>> import numpy as np
  >>> from pyresample import image, geometry
  >>> area_def = geometry.AreaDefinition('areaD', 'Europe (3km, HRV, VTC)', 'areaD',
  ...                                {'a': '6378144.0', 'b': '6356759.0',
  ...                                 'lat_0': '50.00', 'lat_ts': '50.00',
- ...                                 'lon_0': '8.00', 'proj': 'stere'}, 
+ ...                                 'lon_0': '8.00', 'proj': 'stere'},
  ...                                800, 800,
  ...                                [-1370912.72, -909968.64,
  ...                                 1029087.28, 1490031.36])
@@ -141,7 +141,7 @@ Constructor arguments
 The full list of constructor arguments:
 
  **ImageContainerQuick**:
- 
+
 * image_data : Dataset. Masked arrays can be used.
 * geo_def : Geometry definition.
 * fill_value (optional) : Fill value for undefined pixels. Defaults to 0. If set to **None** they will be masked out.
@@ -162,5 +162,3 @@ The full list of constructor arguments:
 Preprocessing of grid resampling
 *********************************
 For preprocessing of grid resampling see :ref:`preproc`
-
-

docs/source/howtos/multi.rst

@@ -6,8 +6,8 @@ Using multiple processor cores
 Multi core processing
 *********************
 
-Bottlenecks of pyresample can be executed in parallel. Parallel computing can be executed if the 
-pyresample function has the **nprocs** keyword argument. **nprocs** specifies the number of processes 
+Bottlenecks of pyresample can be executed in parallel. Parallel computing can be executed if the
+pyresample function has the **nprocs** keyword argument. **nprocs** specifies the number of processes
 to be used for calculation. If a class takes the constructor argument **nprocs** this sets **nprocs** for
 all methods of this class
 
@@ -20,7 +20,7 @@ Example of resampling in parallel using 4 processes:
  >>> area_def = geometry.AreaDefinition('areaD', 'Europe (3km, HRV, VTC)', 'areaD',
  ...                                {'a': '6378144.0', 'b': '6356759.0',
  ...                                 'lat_0': '50.00', 'lat_ts': '50.00',
- ...                                 'lon_0': '8.00', 'proj': 'stere'}, 
+ ...                                 'lon_0': '8.00', 'proj': 'stere'},
  ...                                800, 800,
  ...                                [-1370912.72, -909968.64,
  ...                                 1029087.28, 1490031.36])
@@ -31,9 +31,9 @@ Example of resampling in parallel using 4 processes:
  >>> result = kd_tree.resample_nearest(swath_def, data.ravel(),
  ... area_def, radius_of_influence=50000, nprocs=4)
 
-Note: Do not use more processes than available processor cores. As there is a process creation overhead 
-there might be neglible performance improvement using say 8 compared to 4 processor cores. 
-Test on the actual system to determine the most sensible number of processes to use. 
+Note: Do not use more processes than available processor cores. As there is a process creation overhead
+there might be neglible performance improvement using say 8 compared to 4 processor cores.
+Test on the actual system to determine the most sensible number of processes to use.
 
 Here is an example of the performance for a varying number of processors on a 64-bit ubuntu 14.04, 32 GB RAM, 2 x Intel Xeon E5-2630 with 6 physical cores each: