Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Streamlined cleaning & documentation #85

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Streamlined cleaning & documentation #85

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
cfb2fa6
Hackishly getting up to speed to incorporate changes from amackcrane/…
Feb 10, 2020
89e6968
Addendum to Hackishly getting up to speed
Feb 10, 2020
bb3150e
Data processing pipeline working...
Dec 18, 2019
1e7c648
More descriptive/programmatic data description
Jan 7, 2020
688c070
Added years prior to 2018 to ANC-level election data; added knitr::sp…
Jan 8, 2020
be8fb3a
Clean up
Feb 18, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 33 additions & 17 deletions docs/analyses/anc/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,32 +1,46 @@


# Analysis/Vis
# Visualization

maps/anc-mapping/anc_map.html: cleaned_data/election_data_for_map.csv cleaned_data/anc_turnout.csv maps/anc-mapping/exploration.ipynb
jupyter nbconvert --to notebook --inplace --execute maps/anc-mapping/exploration.ipynb
visualization/election_contest.html: scripts/election_contest.R raw_data
Rscript -e "library(rmarkdown); render(knitr::spin('scripts/election_contest.R', format='Rmd', knit=FALSE), output_dir='visualization')"
# Rscript -e "library(rmarkdown); render('scripts/election_contest.R')"

maps/anc-mapping/anc_map.html: cleaned_data/2012_2018_ancElection_anc.csv cleaned_data/2012_2018_imputedTurnout_anc.csv maps/anc-mapping/exploration.ipynb
jupyter nbconvert --execute maps/anc-mapping/exploration.ipynb

data_description.html: scripts/data_description.Rmd process
Rscript -e "library(rmarkdown); render('scripts/data_description.Rmd', output_dir='.')"

visualization/impute_turnout.html: scripts/impute_turnout.R cleaned_data/2012_2018_ballots_precinct.csv
Rscript -e "library(rmarkdown); render(knitr::spin('scripts/impute_turnout.R', format='Rmd', knit=FALSE), output_dir='visualization')"

# Data processing
process: cleaned_data/2012_2018_ancElection_anc.csv cleaned_data/2018_ancElection_commissioners_contest.csv cleaned_data/2012_2018_imputedTurnout_anc.csv cleaned_data/2012_2018_ancElection_candidate.csv

cleaned_data/election_data_for_anc_map.csv: cleaned_data/election_history_R.csv map_prep.R
Rscript scripts/map_prep.R
cleaned_data/2012_2018_ancElection_anc.csv: cleaned_data/2012_2018_ancElection_contest.csv scripts/election_anc.R
Rscript scripts/election_anc.R

cleaned_data/2018_elections_commissioners.csv: cleaned_data/election_history_R.csv merge_incumbents.R cleaned_data/current_anc_membership.csv
cleaned_data/2018_ancElection_commissioners_contest.csv: cleaned_data/2012_2018_ancElection_contest.csv scripts/merge_incumbents.R raw_data/2019_commissioners.csv
Rscript scripts/merge_incumbents.R
# current_anc_membership.csv comes from Ilya's web scraping

cleaned_data/anc_turnout.csv: cleaned_data/precinct_totals.csv scripts/precinct_registration.R cleaned_data/election_history_R.csv
Rscript scripts/precinct_registration.R

cleaned_data/precinct_totals.csv: cleaned_data/election_history_R.csv
cleaned_data/2012_2018_imputedTurnout_anc.csv: cleaned_data/2012_2018_ballots_precinct.csv scripts/impute_turnout.R cleaned_data/2012_2018_ancElection_contest.csv
Rscript scripts/impute_turnout.R

# created along with election_candidate
cleaned_data/2012_2018_ballots_precinct.csv: cleaned_data/2012_2018_ancElection_candidate.csv
cleaned_data/2012_2018_ancElection_contest.csv: cleaned_data/2012_2018_ancElection_candidate.csv

cleaned_data/election_history_R.csv: scripts/anc_election_cleaner.R raw_data/2012.csv raw_data/2014.csv raw_data/2016.csv raw_data/2018.csv
Rscript scripts/anc_election_cleaner.R
cleaned_data/2012_2018_ancElection_candidate.csv: scripts/election_contest.R raw_data
Rscript scripts/election_contest.R



# Raw data

raw_data: raw_data/2012.csv raw_data/2014.csv raw_data/2016.csv raw_data/2018.csv

raw_data/2012.csv:
curl -o raw_data/2012.csv https://electionresults.dcboe.org/Downloads/Reports/November_6_2012_General_and_Special_Election_Certified_Results.csv

Expand All @@ -39,6 +53,7 @@ raw_data/2016.csv:
raw_data/2018.csv:
curl -o raw_data/2018.csv https://electionresults.dcboe.org/Downloads/Reports/November_6_2018_General_Election_Certified_Results.csv

# raw_data/2019_commissioners.csv comes via Ilya's web scraping (formerly called 'current_anc_membership.csv')

# utilities

Expand All @@ -48,11 +63,12 @@ raw_data/2018.csv:
# rm cleaned_data/anc_turnout.csv
# rm scripts/*~


# DON'T do any rm *.Rmd here, there's some intentional Rmd's in there
clean_viz:
rm -f visualization/*.md
rm -f visualization/*.html
rm -rf visualization/*cache
rm -rf visualization/*files
rm -f visualization/*~
rm -f scripts/election_contest.Rmd
rm -f visualization/election_contest.html
rm -f scripts/impute_turnout.Rmd
rm -f visualization/impute_turnout.html

FORCE:
298 changes: 0 additions & 298 deletions docs/analyses/anc/cleaned_data/2018_elections_commissioners.csv
View file
Open in desktop

This file was deleted.

165 changes: 0 additions & 165 deletions docs/analyses/anc/cleaned_data/anc_turnout.csv
View file
Open in desktop

This file was deleted.

42 changes: 0 additions & 42 deletions docs/analyses/anc/cleaned_data/election_data_for_anc_map.csv
View file
Open in desktop

This file was deleted.

Loading