Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add missing info to short-term count docs #1077

Merged
merged 6 commits into from
Nov 4, 2024
Merged

add missing info to short-term count docs #1077

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e76a211
update the "category" table
mkewins Sep 19, 2024
785f2ab
add reference per rdumas request
mkewins Sep 19, 2024
f2e2665
remove now-redundant file
mkewins Sep 20, 2024
3a70530
demystify CAL
mkewins Sep 20, 2024
dbd3058
update traffic_ to traffic.
mkewins Oct 4, 2024
20c1004
remove file no longer used
mkewins Oct 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 27 additions & 11 deletions volumes/short_term_counting_program/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Short-term Traffic volume data (traffic counts and turning movements) from the F
- [Where does it come from?](#where-does-it-come-from)
- [How often is data updated?](#how-often-is-data-updated)
- [Where can I access the data?](#where-can-i-access-the-data)
- [Where can I find what data?](#where-can-i-find-what-data)
- [How is the data structured?](#how-is-the-data-structured)
- [Core Tables](#core-tables)
- [Other Useful Tables](#other-useful-tables)
Expand Down Expand Up @@ -114,6 +115,21 @@ Internal to the Transportation Data & Analytics team, data flows from legacy Ora

Look in the `traffic` schema for all ad-hoc data tables.

## Where can I find what data?

| Study Type // Loading Mechanism | FlowLoad (`traffic.*`) | MOVE Loader (`traffic.atr_*`) | Spectrum API Loader (`traffic.tmc_*`) |
|---------------------------------|--------------------------------------------------|--------------------------------------|---------------------------------------|
| Turning Movement Count | All-time TMC data* | n/a | September 2023 to present |
| Volume ATR | All-time Volume ATRs** | n/a | n/a |
| Speed / Volume ATR | All-time Speed/Vol ATRs* | May 2023 to present | n/a |
| Vehicle Classification ATR | Classification ATR data from 1985 to May 2023*** | No Classification ATR data loaded*** | n/a |

*The `traffic.*` tables contain all-time TMC and Speed/Vol ATR data from all loading mechanisms

**Around 2021, we stopped collecting Volume-only ATR counts, and switched to Speed/Vol ATR counts, as they're a similar price but more data rich

***Classification ATR data is spotty for two reasons: 1) the legacy loader did not allow for co-located Speed/Vol ATR and Classification ATR data to be loaded for the same day; 2) there is curently no loading mechanism for Classification ATR data post-May 2023, when the MOVE Loader was introduced.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can add a note here that the traffic_ tables have traffic_ stripped from them when they are transferred to the bigdata traffic schema. These tables are:

arterydata
cnt_det
countinfo
countinfomics
det
arc_link
category
node

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@radumas I updated the traffic_ to traffic. so I'm not sure this note is necessary anymore. I think it might introduce more confusion than clarify things?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking more in terms of identifying the tables that don't have a prefix (atr_ or tmc_) which are being actively updated, but maybe this leads into a broader discussion of cleaning up the traffic schema.

## How is the data structured?

### Core Tables
Expand Down Expand Up @@ -327,17 +343,17 @@ category_name|text|name of the count type or data source

#### Category Reference

Category Name|Meaning
-------------|-------
24 HOUR|Volume ATR
RESCU|RESCU
CLASS|Vehicle Classification ATR
SPEED|Speed ATR
MANUAL|
PERM STN|Permanent Count Stations
BICYCLE|Bicycle Volume ATR
SPD OCC|
SENSYS SPEED|
Category Name|Status|Meaning
-------------|------|-------
24 HOUR|-|Volume ATR
RESCU|**DO NOT USE.** For current RESCU data and pipeline information, see [volumes/vds](../vds).|Volume ATR data from RESCU permanent counters. Highway and major arterial in-road loop detectors.
CLASS|-|Vehicle Classification ATR
SPEED|-|Speed / Volume ATR
MANUAL|**Don't use this without further investigation.**|Likely counts loaded via manual counting boards.
PERM STN|**Don't use this without further investigation.** Unclear how this is different from other specified permanent counters.|Permanent Count Stations
BICYCLE|**DO NOT USE.** For current Eco-Counter data and pipeline information, see [volumes/ecocounter](../ecocounter).|Bicycle Volume ATR from Eco-Counter permanent count stations, manually loaded via portable device.
SPD OCC|**DO NOT USE.**|Likely a permanent counter that collected speed ("SPD") and occupancy ("OCC") data.
SENSYS SPEED|**DO NOT USE.**|Sensys permanent counters that collected speed data.

## Useful Views

Expand Down
122 changes: 0 additions & 122 deletions volumes/short_term_counting_program/Tables in Flow.md
View file
Open in desktop

This file was deleted.

6 changes: 6 additions & 0 deletions volumes/short_term_counting_program/cal_dictionary.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
## What is `TRAFFIC.CAL`?

This is an old FLOW table (`TRAFFIC.CAL`) in the legacy Oracle database. It appears to contain Turning Movement Count (TMC) summary statistics for AM peak, PM peak, off peak, and 8-hour totals for a TMC study. These include by direction, mode, and movements.

## Column List

Field Name|Type|Description
----------|----|----------
ID|Autonumber|Autonumber function
Expand Down