Skip to content

Commit

Permalink
Replace links with internal references + fix minor typos
Browse files Browse the repository at this point in the history
  • Loading branch information
volcan01010 committed Jun 26, 2024
1 parent 62a9677 commit 530f492
Show file tree
Hide file tree
Showing 3 changed files with 9 additions and 6 deletions.
4 changes: 2 additions & 2 deletions docs/code_demos/recipes/database_to_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@

logger = logging.getLogger("copy_sensors_async")

# SQL query to get data from Oracle
# SQL query to get data from Oracle
SELECT_SENSORS = """
SELECT CODE, DESCRIPTION
FROM BGS.DIC_SEN_SENSOR
Expand All @@ -34,7 +34,7 @@ def copy_sensors(startdate: dt.datetime, enddate: dt.datetime) -> None:
startdate.isoformat(), enddate.isoformat())
row_count = 0

#connect using the DbParams we imported
# Connect using the imported DbParams
with ORACLE_DB.connect("ORACLE_PASSWORD") as conn:
# chunks is a generator that yields lists of dictionaries
# passing in our select query, connection object, bind variable parameters and custom transform function
Expand Down
2 changes: 2 additions & 0 deletions docs/etl_functions/extract.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ Keyword arguments
All extract functions are derived from :func:`iter_chunks() <etlhelper.iter_chunks>`
and take the same keyword arguments, which are passed through.

.. _parameters:

parameters
""""""""""

Expand Down
9 changes: 5 additions & 4 deletions docs/recipes/database_to_api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,10 @@ a NoSQL document store (e.g. ElasticSearch, Cassandra) or some other web
service.

This example posts data from an Oracle database to an HTTP API. It uses
``iter_chunks`` to fetch data from the database without loading it all
into memory at once. `Parameters <https://britishgeologicalsurvey.github.io/etlhelper/etl_functions/extract.html#parameters>`__ are sent with the database query to filter
rows to only those changed within specified time peiod. This is used to
:func:`iter_chunks() <etlhelper.iter_chunks>` to fetch data from the
database without loading it all
into memory at once. :ref:`Parameters <parameters>` are sent with the database query to filter
rows to only those changed within specified time period. This is used to
only transfer data that has changed since the last time this script was
ran. A custom transform function creates a dictionary
structure from each row of data. This is “dumped” into JSON and posted
Expand All @@ -27,6 +28,6 @@ transfer as opposed to posting records in series.
In this example, failed rows will fail the whole job. Removing the
``raise_for_status()`` call will let them just be logged instead.

To provide the database connection, a `DbParams <https://britishgeologicalsurvey.github.io/etlhelper/connecting_to_databases.html#dbparams>`__ object is
To provide the database connection, :class:`DbParams <etlhelper.DbParams>` object is
imported from a separate `db` file.

0 comments on commit 530f492

Please sign in to comment.