Skip to content

Commit

Permalink
PR #381: cn edits to update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
Cathy Nangini committed Mar 3, 2021
1 parent 03519de commit a98f93d
Showing 1 changed file with 32 additions and 33 deletions.
65 changes: 32 additions & 33 deletions bluetooth/update/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,56 +1,58 @@
# Updating Bluetooth Segments

## Overview

Ocasionally, new segments need to be added to the `bluetooth` schema. This document will go over the steps and the general process to add new routes along the newly installed bluetooth reader locations. It assumes that `bluetooth.all_analyses` table does not have rows already created for the proposed new routes. The steps listed here are followed to create an entirely new routes for newly installed bluetooth readers in the city.

## Table of Contents
- [Overview](#overview)
- [Data Updating](#data-updating)
- [Adding Readers](#adding-readers)
- [Steps to create segments](#steps-to-create-segments)
- [Get start and end geom for each analysis_id](#get-start-and-end-geom-for-each-analysis_id)
- [Create table of nearest intersection ids](#create-table-of-nearest-intersection-ids)
- [Route the segments](#route-the-segments)
- [Things to note](#things-to-note)
- [Validating Output](#validating-output)

## Overview

- [Updating Bluetooth Segments](#updating-bluetooth-segments)
- [Overview](#overview)
- [Table of Contents](#table-of-contents)
- [Data Updating](#data-updating)
- [Adding Readers](#adding-readers)
- [Preparatory Tables and Steps](#preparatory-tables-and-steps)
- [Finding Nearest Intersection IDs](#finding-nearest-intersection-ids)
- [Using pg_routing](#using-pg_routing)
- [Things to note](#things-to-note)
- [Validating Output](#validating-output)
Ocasionally, new segments need to be added to the `bluetooth` schema. This document will go over the steps and the general process to add new routes along the newly installed bluetooth reader locations. It assumes that the `bluetooth.all_analyses` table does not have rows already created for the proposed new routes. The steps listed here explain how to create entirely new routes for newly installed bluetooth readers in the city.

## Data Updating

In this update, there exists an Excel Sheet template that contains details of newly added bluetooth detectors. The details include proposed route name, description, intersection name (BDIT convention), sensor id and lat/lon at start point and sensor id and lat/lon at the end points along with numerous other fields. The template screenshot.
In this update, there exists an Excel Sheet template that contains details of newly added bluetooth detectors (`update/img/template.PNG`). The details include proposed route name, description, intersection name (BDIT convention), sensor id and lat/lon at start point, and sensor id and lat/lon at the end points along with numerous other fields. The template screenshot is shown below:

![new_readers_template](img/template.PNG)

This template was used to include all the details of the routes that can be useful for future analysis. The routes that were updated by adding this batch of new detectors were named with a prefix "DT3_".

## Adding Readers

Depending on how many new readers there are, it may be worthwile to develop an automated process in PostgreSQL, but this update was done manually in the Excel template. In addition to the lat/long, the streets where the readers are is also needed in 2 columns, and the segment name. The segment name is always the first two letters of each street, with the corridor street first and the intersecting street second. For example, a reader at Bloor/Christie measuring travel times on Bloor would be named `BL_CH`. For each proposed route, the excel sheet is populated with the `start reader`, `end_reader` and assigned a unique `analysis_id`. For this batch of new readers, analysis ids starting from 1600000 were added.
Depending on how many new readers there are, it may be worthwile to develop an automated process in PostgreSQL, but this update was done manually in the Excel template. In addition to the lat/lon, the streets where the readers are is also needed in 2 columns, and the segment name. The segment name is always the first two letters of each street, with the corridor street first and the intersecting street second. For example, a reader at Bloor/Christie measuring travel times on Bloor would be named `BL_CH`. For each proposed route, the excel sheet is populated with the `start reader`, `end_reader` and assigned a unique `analysis_id`. For this batch of new readers, analysis ids starting from 1600000 were added.

Therefore the reader table will have the following fields populated:

`analysis_id`, `street`, `direction`, `from_street`, `to_street`, `from_id`, `to_id`, `start_point_lat`, `start_point_lon`, `end_point_lat` and `end_point_lon`.


## Steps to create segments

## Preparatory Tables and Steps
The following steps are utilized to create segments
The following steps are utilized to create segments:
1. Get start and end geom for each analysis_id
2. Create table with detector_id, detector_geom, centreline_int_id
3. Join the detector's geometry to the closest centreline intersection
4. Route the segments using street centreline's intersection `gis.centreline_both_dir` using [pgr_dijkstra].
2. Create table of nearest intersection ids
3. Route the segments

### Get start and end geom for each analysis_id
After uploading excel data to PostgreSQL, the geometry column is needed. This can be done by adding a column using `ALTER TABLE schema.table_name ADD COLUMN geom GEOMETRY`. We need two geometry columns: `from_geom` and `to_geom`. Fill in the `from_geom` column with `UPDATE TABLE schema.table_name SET from_geom = ST_MakePoint(start_point_lat, start_point_lon)` and modify the query for `to_geom` with `SET to_geom = ST_MakePoint (end_point_lat, end_point_lon)`.

After uploading excel data to PostgreSQL, the geometry column is needed. This can be done by adding a column using `ALTER TABLE schema.table_name ADD COLUMN geom GEOMETRY` we need two geometry columns: `from_geom` and `to_geom`. Filling in the from_geom column with `UPDATE TABLE schema.table_name SET from_geom = ST_MakePoint(start_point_lat, start_point_lon)` and modify the query for `to_geom` = `ST_MakePoint (end_point_lat, end_point_lon)`.

## Finding Nearest Intersection IDs
### Create table of nearest intersection ids

To get the intersection ids that are close to the newly added detectors location, create a table named `bluetooth_nodes`. This table has four fields:
`bluetooth_id`, `geom` (this is geometry of bluetooth detectors), `int_id` (nearest intersection id) and `int_geom` (geometry of the nearest intersection to the )

- `bluetooth_id`,
- `geom` (this is geometry of bluetooth detectors),
- `int_id` (nearest intersection id)
- `int_geom` (geometry of the nearest intersection).

This table is created using the following query:

```SQL
SELECT DISTINCT mohan.new_added_detectors.from_id::integer AS bluetooth_id,
mohan.new_added_detectors.from_geom,
Expand All @@ -63,10 +65,10 @@ SELECT DISTINCT mohan.new_added_detectors.from_id::integer AS bluetooth_id,
ORDER BY (z.geom <-> mohan.new_added_detectors.from_geom)
LIMIT 1) nodes;
```
Check that correct intersections are returned from this query especially for odd shaped intersections. If required, correct the int_id and geom for such intersections and finalize the table `mohan.bluetooth_nodes`.
Check that correct intersections are returned from this query especially for oddly shaped intersections. If required, correct the `int_id` and `geom` for such intersections and finalize the table `mohan.bluetooth_nodes`.

## Using pg_routing
Once the nearest centreline intersection nodes are linked to the bluetooth readers geom `mohan.bluetooth_nodes`, we are ready to run the following Query to create new routes.
### Route the segments
Once the nearest centreline intersection nodes are linked to the bluetooth readers geom `mohan.bluetooth_nodes`, we are ready to run the following query to create new routes using street centreline's intersection `gis.centreline_routing_directional` with the pg routing function [pgr_dijkstra](https://docs.pgrouting.org/latest/en/pgr_dijkstra.html):

```SQL
CREATE table mohan.bt_segments_new AS (
Expand Down Expand Up @@ -108,9 +110,6 @@ GROUP BY analysis_id, street, direction, from_street, to_street) a)
Geostatistical lines and planning boundaries need to be avoided while pgrouting.

## Validating Output
Validate the length of the segments with length ST_length(geom) and direction using gis.direction_from_line(geom) functions.If the detectors are located very close to the centerline intersections, it is not necessary to do the centreline cutting. Else that step is necessary.

The table is now ready to append to the existing routes table.

Validate the length of the segments with length `ST_length(geom)` and direction using `gis.direction_from_line(geom)` functions. If the detectors are located very close to the centerline intersections, it is not necessary to do the centreline cutting. Else that step is necessary.

[pgr_dijkstra]:https://docs.pgrouting.org/latest/en/pgr_dijkstra.html
The table is now ready to be appended to the existing routes table.

0 comments on commit a98f93d

Please sign in to comment.