Skip to content

Commit

Permalink
Use sphinx-click integration using rst syntax
Browse files Browse the repository at this point in the history
  • Loading branch information
stijnvanhoey committed Aug 21, 2023
1 parent 108fc15 commit 2aa963d
Show file tree
Hide file tree
Showing 6 changed files with 48 additions and 20 deletions.
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ pip install vptstools\[transfer\]

## CLI endpoints

```{eval-rst}
.. include:: click.rst
```

In addition to using functions in Python scripts, two vptstools functions can be called from the command line:

### transfer_baltrad
Expand All @@ -45,6 +49,11 @@ Configuration is loaded from environmental variables:
- SNS_TOPIC: AWS SNS topic to report when routine fails
- AWS_PROFILE: AWS profile (mainly for local development)

```{click} vptstools.bin.vph5_to_vpts:cli
:prog: Convert h5 files to daily/monthly vpts files
```

### vph5_to_vpts

CLI tool to aggregate/convert the [ODIM hdf5 bird profile](https://github.com/adokter/vol2bird/wiki/ODIM-bird-profile-format-specification)
Expand Down
8 changes: 8 additions & 0 deletions docs/api/vptstools.bin.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,14 @@ vptstools.bin namespace
Submodules
----------

vptstools.bin.click\_exception module
-------------------------------------

.. automodule:: vptstools.bin.click_exception
:members:
:undoc-members:
:show-inheritance:

vptstools.bin.transfer\_baltrad module
--------------------------------------

Expand Down
7 changes: 7 additions & 0 deletions docs/click.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@

.. click:: vptstools.bin.transfer_baltrad:cli
:prog: transfer_baltrad


.. click:: vptstools.bin.vph5_to_vpts:cli
:prog: vph5_to_vptsny paths that contain cus
1 change: 1 addition & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
"sphinx.ext.ifconfig",
"sphinx.ext.mathjax",
"sphinx.ext.napoleon",
"sphinx_click"
]

# Add any paths that contain templates here, relative to this directory.
Expand Down
1 change: 1 addition & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ develop =
black
sphinx
sphinx_rtd_theme
sphinx-click
myst-parser[linkify]
moto[s3]
tox
Expand Down
42 changes: 22 additions & 20 deletions src/vptstools/bin/transfer_baltrad.py
Original file line number Diff line number Diff line change
@@ -1,23 +1,3 @@
"""
Python CLI script that:
- Connects via SFTP to the BALTRAD server
- For each vp file (pvol gets ignored), download the file from the server and
upload it to the "aloft" S3 bucket
- If file already exists at destination => do nothing
Designed to be executed daily via a simple scheduled job like cron (files disappear after a few
days on the BALTRAD server)
Configuration is loaded from environmental variables:
- FTP_HOST: Baltrad FTP host ip address
- FTP_PORT: Baltrad FTP host port
- FTP_USERNAME: Baltrad FTP user name
- FTP_PWD: Baltrad FTP password
- FTP_DATADIR: Baltrad FTP directory to load data files from
- DESTINATION_BUCKET: AWS S3 bucket to write data to
- SNS_TOPIC: AWS SNS topic to report when routine fails
- AWS_PROFILE: AWS profile (mainly for local development)
"""
import datetime
import os
from functools import partial
Expand Down Expand Up @@ -92,6 +72,28 @@ def extract_metadata_from_filename(filename: str) -> tuple:

@click.command(cls=catch_all_exceptions(click.Command, handler=report_sns)) # Add SNS-reporting to exception
def cli():
"""Sync files from FTP to s3 bucket.
Python CLI script that:
- Connects via SFTP to the BALTRAD server
- For each vp file (pvol gets ignored), download the file from the server and
upload it to the "aloft" S3 bucket
- If file already exists at destination => do nothing
Designed to be executed daily via a simple scheduled job like cron (files disappear after a few
days on the BALTRAD server)
Configuration is loaded from environmental variables:
- FTP_HOST: Baltrad FTP host ip address
- FTP_PORT: Baltrad FTP host port
- FTP_USERNAME: Baltrad FTP user name
- FTP_PWD: Baltrad FTP password
- FTP_DATADIR: Baltrad FTP directory to load data files from
- DESTINATION_BUCKET: AWS S3 bucket to write data to
- SNS_TOPIC: AWS SNS topic to report when routine fails
- AWS_PROFILE: AWS profile (mainly for local development)
"""
cli_start_time = datetime.datetime.now()
click.echo(f"Start transfer Baltrad FTP sync at {cli_start_time}")
click.echo("Read configuration from environmental variables.")
Expand Down

0 comments on commit 2aa963d

Please sign in to comment.