Project Meeting 2017.03.16

Multiple Zone Systems Network LOS

• Will review progress from last meeting
• Merged code into master, but not implemented for the example, tests, documentation #159
• @Wu ran a series of tests with millions of queries and will post the results on the wiki. Overall the results are promising.
• TAZ skim access is much faster than TAP skims though, which is due to using lookups instead of indexes #167
• Scoped project is complete and RSG will invoice
• @SANDAG and RSG will discuss amending the contract to complete the two issues above

Data Pipelining

• pipeline branch committed and merged
• we looked into Luigi and Airflow but decided to stick with orca for now even though it isn't widely adopted
• Example updated and partially run on the full example as well (used all available RAM 160GB)
• todos:
  • write code docs
  • update ActivitySim documentation, including documentation on data table dependencies for model steps #168
  • get running on full example; maybe we should just write the additional fields after each model step?
  • make the current saving table-states debug statements prettier and put in the log file #160
• @Everyone agreed that it would be good to clean-up the reading and processing of input data (i.e. the default namespace) now that pipelining is implemented. #169 This means:
  • creating new model steps for loading and processing inputs
  • making the default tables just tables with key columns
  • moving all the orca computed columns to a mtctm1 folder/namespace for example
• We will NOT recode the orca computed columns as expressions files since the perceived benefit of this isn't great and it is a big effort
• It needs to happen eventually, especially for someone trying to use the code on their own model
• Running the full example is pretty slow and using 3% of the CPU power. Maybe we should experiment with multi-processing in order to gauge the data pipeline performance? But this is no small task...
• Updated Design page

Random Number Sequences

• Random number sequencing in the pipeline branch as well
• Example updated and partially run on the full example as well
• todos:
  • write code docs
  • update ActivitySim documentation
  • add random number offset for trips, in addition to households, persons, tours #161
  • add random number to trace results #162
  • add random numbers for each model for each chooser to the relevant tables #163 - households get auto own rnum for example. This is lower priority for now.
• Updated Design page