Project Meeting 2022.06.23

Admin

• Contract Agreements, any issues?
  • No date on Page 1, but that was corrected and now signed (MWCOG)
• Technical Working Group
  • Sijia, David, Jeff, Joe, and Flavia
  • Jeff to send out a poll to schedule first meeting
• Question about access to recordings
  • Yes, you do have to ask CS for permission. It is not CS being gatekeepers, it’s a Microsoft limitation. We apologize for the inconvenience.

Sharrow

• Presentation from SANDAG on testing sharrow: InitSANDAGSharrowTesting.pptx
  • Surprised by 2 hour compile time in sharrow
  • Speed of compiling could be do to CPU configuration, depends on how many optimizations the compiler is attempting by CPU. Running on servers with Xeon chips may take longer as well. Heavy duty servers gain their speed advantage by having a lot of processing cores, but they may not be individually very fast. Number of cores doesn’t matter as much as the speed of the individual cores.
  • We should test by more people on different machines.
  • Jeff is working on work flow to automating process to compile, chunk train, production mode, and compare all three.
  • Running the compiler in multiprocessing is generally not a good idea.
• Are there model components that sharrow is not implemented?
  • Scheduling ones – the logsums have been sharrow-ed but scheduling model has time windows code that hasn’t been moved into numba so sharrow can’t understand the time windows code in the specs. It is the last task of CS’ phase 7 scope to fix that, just hasn’t happened yet.
  • Composition model might also have the time windows code.
  • Components without choice models won’t be sharrowed.
• Looking at the config file, the sharrow keyword had different options (test versus require, for example). Here are the settings:
  • False: it is not invoked
  • True: it will be invoked where it is compatible. If any error is encountered when sharrow is in process, it will fall back to legacy model. Good for compatibility, not good for performance.
  • Test: runs sharrow but defaults to legacy code if it doesn’t work somewhere. If you have test message, there may be warnings that pop up. If there are deviations between outputs of legacy and sharrow versions. It will stop if there are significant differences.
  • Require: if sharrow is attempted for use, and it fails, it will crash the run and report errors (let you know where it isn’t working). As opposed to True, which won’t fail and will default to legacy version.
  • In the long-term, recommend moving away from test mode because it takes a very long time, running legacy and sharrow versions.
• Discussion about skim compression
  • Discussed previously, during April 12 meeting https://github.com/ActivitySim/activitysim/wiki/Project-Meeting-2022.04.12
  • Reduces RAM significantly but does take time (about 20 minutes)
  • Opportunity to do this better, just takes time and resources
  • Discussed today (and previously) was Jeff’s method in order to allow him to test things on his laptop, by keeping the RAM low. Helps, but can be done better.
  • If we don’t need to keep floating point precision (boardings, for example) can there be some preprocessing to make the skims themselves smaller, instead of manipulating if it isn’t needed?
  • This could be a topic for the technical working group, it has some design implications.