Project Meeting 2022.08.23

Agenda

• Sharrow testing for 2- and 3-zone systems
• Memory issues
• Other

Action Items and Key Decisions

• Key Decision: Sharrow works with a 3-zone system but did not provide the expected performance results. Since 3-zone systems are not a high priority for the consortium, further work on investigating sharrow performance is not recommended at this time. It works and produces correct results and performance issues can be addressed at a later date, if needed.
• Action Item: Joe to draft scope of work for further assessment of memory issues and share with consultant bench for further edits.

Meeting Notes

Sharrow 2 and 3 zone tests

• Passed tests on all the one-zone models, next steps was to test on two and three zone models
• Initial test run with 2-zone SANDAG example, only 50k households and 600 zones
  • 3 runs – sharrow, legacy, and reference (v1.0.4) scenarios
  • Sharrow runs twice as fast
  • Results are very similar
• 3-zone model has been run (same SANDAG small example)
  • Tests have been run, however there is only a nominal differences in run-time.
  • 3-zone code spends a lot of time in the 3-zone virtual pathbuilder, which sharrow doesn’t help. That code is different enough that sharrow can’t be easily applied to this code.
  • Outputs are fine, so all the calculations are correct.
  • Trip destination run time is longer in sharrow, currently don’t understand why that is happening. Jeff has theories (for example, logsums are being written out under sharrow but not in the reference scenarios, not sure why that is happening, possible that some setting has been changed to make it do extra work in writing out).
  • This trip destination choice isn’t the only problem. There’s still got to be something else going on that results in components not running twice as fast under sharrow, as experienced in the other models and as was expected for the 3-zone system.
• Conclusions
  • 2-zone sharrow works and performs well.
  • No consortium agency voiced concerns about not moving forward with sharrow 3-zone system performance continued work.
  • It has been implemented, it runs, it produces correct results, but 3-zone performance is disappointing. It could be picked up at a later date, if needed, but no additional work/investigation is recommended under this contract.

Memory Issues

• Joe would like to see a quantitative assessment across multiple full-scale examples
  • Full-scale examples should range in synthetic population size, amount of transit skims, etc. A wide variety of model characteristics would better stress test to see if there are profound differences and where those would be (is it just isolated to the skims aspects, for example)
  • Would like to see detailed, forensic analysis (this would be a whole scope item) that assesses how are we using memory. For example, at each point, here is what we are carrying around, here is the memory footprint at this point, and this memory explosion is caused by X. This is to better understand what is going on at each step and what are the memory consequences
  • Want to develop a task order to do this (to go to WSP) and would result in pointing to potential solutions
  • The solutions would preferably be short-term fixes; don't want to discuss solutions that are 3-5 years from implementation, such as refactoring.
  • Testing should be done within sharrow implementation. By the time that the task order is complete, the sharrow (one-zone) code review should be completed.
  • Action Items: Joe to draft initial thoughts/first draft SOW and work with consultants (WSP, CS, RSG) to edit/revise

Other

• School escorting PR is complete and ready for review.
• Doesn’t need two people to review, just one would be fine. May suggest partner to download and test, but TBD. Revisit this on Thursday.