Skip to content

Commit

Permalink
Update MATSim CW33 (#109)
Browse files Browse the repository at this point in the history
* make drt shifts multi mode drt comatible

* adjust shift efficiency tracker as well

* Extended Trip analysis and dashboard (matsim-org#3332)

* trip analysis for subgroups

* refactor categories

* write trip analysis

* write out correct table

* use static method

* prepare for facets, add TODOs

* start working on plotly facet integration

* Added Test Trip Dashboard for Facets

* update API

* add trip choice analysis

* adding test case and some todos

* calculating some trip choice metrics

* add choice metrics to trip dashboard

* hide tab if not needed

* add text box and round metrics

* normalize shares per reference group correctly, added some facet dashboards (WIP)

* fix pivot usage

* rename entries

* add plots for distance distributions

* add todo

* use dropdown for plots

* add cohen kappa to evaluation

* output euclidean distance for trip choices

* add ref_id attribute

* added heatmap to TripDashboard

* update description

* allow arbitrary object for ref ids

* show heatmap labels

* comment out trip dashboard wip part

* use correct logger

* improve wording

---------

Co-authored-by: frievoe97 <[email protected]>

* Bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.5 to 3.3.0

Bumps [org.apache.maven.plugins:maven-surefire-plugin](https://github.com/apache/maven-surefire) from 3.2.5 to 3.3.0.
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](apache/maven-surefire@surefire-3.2.5...surefire-3.3.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-surefire-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump org.apache.maven.plugins:maven-surefire-report-plugin

Bumps [org.apache.maven.plugins:maven-surefire-report-plugin](https://github.com/apache/maven-surefire) from 3.2.5 to 3.3.0.
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](apache/maven-surefire@surefire-3.2.5...surefire-3.3.0)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-surefire-report-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump org.apache.maven.plugins:maven-jar-plugin from 3.4.1 to 3.4.2

Bumps [org.apache.maven.plugins:maven-jar-plugin](https://github.com/apache/maven-jar-plugin) from 3.4.1 to 3.4.2.
- [Release notes](https://github.com/apache/maven-jar-plugin/releases)
- [Commits](apache/maven-jar-plugin@maven-jar-plugin-3.4.1...maven-jar-plugin-3.4.2)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-jar-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump com.fasterxml.woodstox:woodstox-core from 6.6.2 to 7.0.0

Bumps [com.fasterxml.woodstox:woodstox-core](https://github.com/FasterXML/woodstox) from 6.6.2 to 7.0.0.
- [Commits](FasterXML/woodstox@woodstox-core-6.6.2...woodstox-core-7.0.0)

---
updated-dependencies:
- dependency-name: com.fasterxml.woodstox:woodstox-core
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump org.jfree:jfreechart from 1.5.4 to 1.5.5

Bumps [org.jfree:jfreechart](https://github.com/jfree/jfreechart) from 1.5.4 to 1.5.5.
- [Release notes](https://github.com/jfree/jfreechart/releases)
- [Commits](jfree/jfreechart@v1.5.4...v1.5.5)

---
updated-dependencies:
- dependency-name: org.jfree:jfreechart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* make drt work with only custom constraints sets

* add some logging to NetworkChangeEventsWriter

* correct logging statement

* add router binding for complex request unscheduling

* use time as disutility

* Bump junit.version from 5.10.2 to 5.10.3

Bumps `junit.version` from 5.10.2 to 5.10.3.

Updates `org.junit.jupiter:junit-jupiter-engine` from 5.10.2 to 5.10.3
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](junit-team/junit5@r5.10.2...r5.10.3)

Updates `org.junit.jupiter:junit-jupiter` from 5.10.2 to 5.10.3
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](junit-team/junit5@r5.10.2...r5.10.3)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter-engine
  dependency-type: direct:development
  update-type: version-update:semver-patch
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* * repair wrong positioning of vehicles if linkSpeedCalculator returns speed larger than free speed
* replace some (but not all) link.getAttributes()... by BicycleUtils.set/getXyz( link, ...)
* bind BicycleConfigGroup explicitly in BicycleModule so that not materializing it before does not lead to a non-interpretable error message (stating that Scenario is not bound about 20 times although that is not the root of the problem)
* possibly one or the other additional things

* norm subgroups to 1 (matsim-org#3346)

* add tests for mode restriction

* add test integration test case for complex unscheduler

* Bump io.grpc:grpc-all from 1.64.0 to 1.65.0

Bumps [io.grpc:grpc-all](https://github.com/grpc/grpc-java) from 1.64.0 to 1.65.0.
- [Release notes](https://github.com/grpc/grpc-java/releases)
- [Commits](grpc/grpc-java@v1.64.0...v1.65.0)

---
updated-dependencies:
- dependency-name: io.grpc:grpc-all
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* add tests that depict the current behaviour

* Bump com.google.protobuf:protobuf-java from 4.27.1 to 4.27.2

Bumps [com.google.protobuf:protobuf-java](https://github.com/protocolbuffers/protobuf) from 4.27.1 to 4.27.2.
- [Release notes](https://github.com/protocolbuffers/protobuf/releases)
- [Changelog](https://github.com/protocolbuffers/protobuf/blob/main/protobuf_release.bzl)
- [Commits](https://github.com/protocolbuffers/protobuf/commits)

---
updated-dependencies:
- dependency-name: com.google.protobuf:protobuf-java
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* implement multiple mode test

* Split CalculateSkimMatrices.prepareAndCalculateNetworkMatrices() to allow for custom travel times objects

* implement option for network inconsistency check; no test adaptation so far

* Bump org.checkerframework:checker-qual from 3.44.0 to 3.45.0

Bumps [org.checkerframework:checker-qual](https://github.com/typetools/checker-framework) from 3.44.0 to 3.45.0.
- [Release notes](https://github.com/typetools/checker-framework/releases)
- [Changelog](https://github.com/typetools/checker-framework/blob/master/docs/CHANGELOG.md)
- [Commits](typetools/checker-framework@checker-framework-3.44.0...checker-framework-3.45.0)

---
updated-dependencies:
- dependency-name: org.checkerframework:checker-qual
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* implemented network changer

* implement functionality in Util classes and moved config

* implement test using the new functionality for mode restriction

* revert changing the default access egress type

* Switch to DrtWithExtensionsConfigGroup

* Bump org.apache.poi:poi-ooxml from 5.2.5 to 5.3.0

Bumps org.apache.poi:poi-ooxml from 5.2.5 to 5.3.0.

---
updated-dependencies:
- dependency-name: org.apache.poi:poi-ooxml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* explicitly set access egress type

* fix test failures in contribs

* fix test failures in contribs and add config parameter so relevant functions

* reworked CreatePseudoNetwork with loop-links to avoid duplicating stops

* repair test failures

* Trip facet dashboard (matsim-org#3350)

* implement dropdown facets on the trip dashboard

* implement dropdown facets on the trip dashboard

* added backgroundColor attribute and restyled text seperators

* set backgroundcolor on all plots

---------

Co-authored-by: rakow <[email protected]>

* adjust rounding

* exclude case

* Drt dashboard typo fix (matsim-org#3360)

* added tooltip for gridmap/emissionsDashboard

* fix typo in drt dashboard

---------

Co-authored-by: rakow <[email protected]>

* Bump com.fasterxml.jackson:jackson-bom from 2.17.1 to 2.17.2

Bumps [com.fasterxml.jackson:jackson-bom](https://github.com/FasterXML/jackson-bom) from 2.17.1 to 2.17.2.
- [Commits](FasterXML/jackson-bom@jackson-bom-2.17.1...jackson-bom-2.17.2)

---
updated-dependencies:
- dependency-name: com.fasterxml.jackson:jackson-bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump net.bytebuddy:byte-buddy from 1.14.17 to 1.14.18

Bumps [net.bytebuddy:byte-buddy](https://github.com/raphw/byte-buddy) from 1.14.17 to 1.14.18.
- [Release notes](https://github.com/raphw/byte-buddy/releases)
- [Changelog](https://github.com/raphw/byte-buddy/blob/master/release-notes.md)
- [Commits](raphw/byte-buddy@byte-buddy-1.14.17...byte-buddy-1.14.18)

---
updated-dependencies:
- dependency-name: net.bytebuddy:byte-buddy
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* set networkMode based on vehicleTypes

* pass lambda function as parameter

* Bump org.assertj:assertj-core from 3.26.0 to 3.26.3

Bumps [org.assertj:assertj-core](https://github.com/assertj/assertj) from 3.26.0 to 3.26.3.
- [Release notes](https://github.com/assertj/assertj/releases)
- [Commits](assertj/assertj@assertj-build-3.26.0...assertj-build-3.26.3)

---
updated-dependencies:
- dependency-name: org.assertj:assertj-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* chore: clean up MinimalTransferTimes (matsim-org#3365)

* chore: cleanup adaptive transfers (matsim-org#3366)

* chore: cleanup adaptvie transfers

* make final

* Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.3.0 to 3.3.1

Bumps [org.apache.maven.plugins:maven-failsafe-plugin](https://github.com/apache/maven-surefire) from 3.3.0 to 3.3.1.
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](apache/maven-surefire@surefire-3.3.0...surefire-3.3.1)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-failsafe-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* add missing case of enum

* add functionality to split jobs

* use new methods

* update activity creation

* set activity time correctly

* remove unneeded exception

* change to parameterized log message

* delete empty test methods

* remove unused import

* fix unchecked assignment

* remove redundant local variables

* minor code style update

* remove not needed throw information

* make things final

* remove redundant final declaration

* remove unnecessary 'throws' declarations

* simplify assertions

* cleanup Strings

* remove redundant casts

* code style update

* cleanup imports

* Java language: Replace explicit type with <>

* Java language: use computeIfAbsent

* Java language: use Double.hashCode(...)

* Java language: replace if by switch

* Java language: avoid raw use

* Java language: use text block

* Java language: use pattern variable

* Java language: use add/getFirst()

* Access to TestUtils.EPSILION

* add not null assertion

* remove one layer of "LSPs"

* add prebooking support for drt shifts
some minor shift cleanups
possibility to assign requests to shifts that are assigned bot not started yet

* cancel assigned shifts

* Bump com.google.errorprone:error_prone_annotations from 2.28.0 to 2.29.2

Bumps [com.google.errorprone:error_prone_annotations](https://github.com/google/error-prone) from 2.28.0 to 2.29.2.
- [Release notes](https://github.com/google/error-prone/releases)
- [Commits](google/error-prone@v2.28.0...v2.29.2)

---
updated-dependencies:
- dependency-name: com.google.errorprone:error_prone_annotations
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* add some documentation

* shift drt maintenance

* some more drt shift maintenance

* consider break corridors for (pre)bookings

* consider break corridors for (pre)bookings

* clean up obsolete method

* update FISS test, reduce iterations to reduce runtime

* avoid abandoning prebooked requests twice in the same time step

* fix tests by disabling net consistency check

* adapt error messages

* remove redundant suppression

* remove unused field

* internal renaming

* remove unnecessary toString() calls

* remove unnecessary semicolon

* inline and remove method

* cleanup imports

* simplification

* make final, internal renaming

* replace 'if' by 'switch'

* simplification

* use enhanced 'switch'

* simplification

* use 'getFirst()'

* javadoc: replace blank lines with '<p>'

* javadoc: set link as a link

* logging: use parameterized logging

* add casting

* simplification

* use 'append()' calls

* simplification by removing inaccessibly parts of if statements.

* remove duplicated setting of variable

* Simplify assertions

* Simplify stream

* avoid casting

* suppress warning

* fix some (language) grammar

* fix some typos

* more simplifications

* cleanup: remove commented out code

* small cleanups

* fix test failures by disable net consistency check

* optimize h3 zone system initialization; move h3 test to common module

* shift drt maintenance; fix bug for prebooked e-drt scenarios that need shifts to be assignable to requests in advance; change priority queues in shift dispatcher to sorted sets to ensure correct iteration order

* make FISS integration test less sensible to smaller changes in link events

* set level of some logging's back from WARN to DEBUG

* SimWrapper Dashboard features (matsim-org#3381)

* add explanations to the traffic dashboard, add pt viewer dashboard

* add additional description

* add avg beeline speed to trip dashboard

* added mode share distance distribution

* added plot for detailed distance distribution

* upscale emission dashboard with sample size

* Update GenerateFreightPlans.java (matsim-org#3387)

Fixing one typo in "GenerateFreightPlans.java" and some minor updates in the input sample size interpretation and output naming.

No changes in the functionality.

* add events files comparison

* more (javadoc) comments

* add test which includes tolling.
One test is failing, because it is not possible to create different schemes for the different types.

* WIP: Prepare test for tolling with factor. Not implemented to the end now -> Disabled.

* replace self-build RoadPricingCalculator with official RoadPricingScheme

* adapt test: use tollFactor

* add option if person==null

* more tests

* more steps forward

* fix one test: Use MATSim's vehicles

* fix the last test: Building the netbased costs without MATSim's vehicle types

* some cleanup and javadoc

* fix typos, internal renaming

* Bump org.apache.maven.plugins:maven-surefire-report-plugin

Bumps [org.apache.maven.plugins:maven-surefire-report-plugin](https://github.com/apache/maven-surefire) from 3.3.0 to 3.3.1.
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](apache/maven-surefire@surefire-3.3.0...surefire-3.3.1)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-surefire-report-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump io.grpc:grpc-all from 1.65.0 to 1.65.1

Bumps [io.grpc:grpc-all](https://github.com/grpc/grpc-java) from 1.65.0 to 1.65.1.
- [Release notes](https://github.com/grpc/grpc-java/releases)
- [Commits](grpc/grpc-java@v1.65.0...v1.65.1)

---
updated-dependencies:
- dependency-name: io.grpc:grpc-all
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump org.apache.maven.plugins:maven-surefire-plugin from 3.3.0 to 3.3.1

Bumps [org.apache.maven.plugins:maven-surefire-plugin](https://github.com/apache/maven-surefire) from 3.3.0 to 3.3.1.
- [Release notes](https://github.com/apache/maven-surefire/releases)
- [Commits](apache/maven-surefire@surefire-3.3.0...surefire-3.3.1)

---
updated-dependencies:
- dependency-name: org.apache.maven.plugins:maven-surefire-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump commons-codec:commons-codec from 1.17.0 to 1.17.1

Bumps [commons-codec:commons-codec](https://github.com/apache/commons-codec) from 1.17.0 to 1.17.1.
- [Changelog](https://github.com/apache/commons-codec/blob/master/RELEASE-NOTES.txt)
- [Commits](apache/commons-codec@rel/commons-codec-1.17.0...rel/commons-codec-1.17.1)

---
updated-dependencies:
- dependency-name: commons-codec:commons-codec
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Preserve the working directory when gui is run from cli (matsim-org#3391)

* use asterisk for act type default

* NoiseAnalysis: set the scale factor for according to sample size

* add fare zone based pt pricing (matsim-org#3382)

* first try to implement fare zone based pt pricing, deutschlandtarif numbers still missing

* put deutschlandtarif price linear functions into code

* unit test for FareZoneBasedPtFareHandler

* better defaults for NoiseDashboard

* remove the static

* NoiseAnalysis: more explicit parameter settings + bug fix in MergeNoiseOutput

* Update DRT estimate and teleport module (matsim-org#3333)

Re-organizing the DRT estimation scripts. For more details, please see detailed description above.

* NoiseAnalysis: merge damages files

* NoiseDashboard: include stats and damage plots

* only perform net consistency check once

* CreateSingleSimWrapperDashboard

* fix noise tests: explicitly set activity types for rp grid, when setting act type for damage calculation

* fix nasty insertion scheduler bug

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: rakow <[email protected]>
Co-authored-by: frievoe97 <[email protected]>
Co-authored-by: Joschka Bischoff <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kainagel <[email protected]>
Co-authored-by: Kai Nagel <[email protected]>
Co-authored-by: Paul Heinrich <[email protected]>
Co-authored-by: Michal Maciejewski <[email protected]>
Co-authored-by: marecabo <[email protected]>
Co-authored-by: steffenaxer <[email protected]>
Co-authored-by: u229351 <[email protected]>
Co-authored-by: frievoe97 <[email protected]>
Co-authored-by: Ricardo Ewert <[email protected]>
Co-authored-by: Ricardo Ewert <[email protected]>
Co-authored-by: Paul Heinrich <[email protected]>
Co-authored-by: Sebastian Hörl <[email protected]>
Co-authored-by: Kai Martins-Turner <[email protected]>
Co-authored-by: Kai Martins-Turner <[email protected]>
Co-authored-by: tschlenther <[email protected]>
Co-authored-by: Chengqi Lu <[email protected]>
Co-authored-by: simei94 <[email protected]>
  • Loading branch information
22 people authored Aug 13, 2024
1 parent edb9f45 commit 93b4961
Show file tree
Hide file tree
Showing 164 changed files with 3,326 additions and 604 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,14 @@ public Integer call() throws Exception {

// load config if not present yet.
if (config == null) {
config = loadConfig(Objects.requireNonNull( configPath, "No default scenario location given" ).getAbsoluteFile().toString() );
String path = Objects.requireNonNull( configPath, "No default scenario location given" ).getAbsoluteFile().toString();
List<ConfigGroup> customModules = getCustomModules();

final Config config1 = ConfigUtils.loadConfig(path, customModules.toArray(new ConfigGroup[0] ) );
Config prepared = prepareConfig( config1 );

config = prepared != null ? prepared : config1;
// (The above lines of code come from inlining so maybe it happened there: I cannot see how prepared could be null but config1 not except if user code returns null which I would consider a bug. kai, aug'24)
} else {
Config tmp = prepareConfig(config);
config = tmp != null ? tmp : config;
Expand Down Expand Up @@ -306,15 +313,6 @@ protected final void addRunOption(Config config, String option) {
addRunOption(config, option, "");
}

private Config loadConfig(String path) {
List<ConfigGroup> customModules = getCustomModules();

final Config config = ConfigUtils.loadConfig(path, customModules.toArray(new ConfigGroup[0]));
Config prepared = prepareConfig(config);

return prepared != null ? prepared : config;
}

@Override
public String defaultValue(CommandLine.Model.ArgSpec argSpec) throws Exception {
Object obj = argSpec.userObject();
Expand Down Expand Up @@ -374,8 +372,39 @@ public static void run(Class<? extends MATSimApplication> clazz, String... args)
}

/**
* Convenience method to run a scenario from code or automatically with gui when desktop application is detected.
* This method may also be used to predefine some default arguments.
* <p>Convenience method to run a scenario from code or automatically with gui when desktop application is detected.
* This method may also be used to predefine some default arguments.</p>
*
* <p>With respect to args it looks like arguments are treated in the following sequence (programmed in the run method):
* <ul>
* <li>ConfigUtils.loadConfig without args</li>
* <li>prepareConfig which is usually overwritten</li>
* <li>config options from some yaml file which can be provided as a command line option</li>
* <li>config options on command line </li>
* </ul></p>
*
* <p>defaultArgs could be used to provide defaults when calling this method here; they would go in addition to what is coming in from "upstream" which is typically the command line.</p>
*
* <p>There are many execution paths that can be reached from this class, but a typical one for matsim-scenarios seems to be:<ul>
* <li> This method runs MATSimApplication.run( TheScenarioClass.class , args ).</li>
* <li> That run class will instantiate an instance of TheScenarioClass (*), then do some args consistenty checking, then call the piccoli execute method. </li>
* <li> The piccoli execute method will essentially call the "call" method of MATSimApplication. </li>
* <li> I think that in the described execution path, this.config in that call method will initially be null. (The ctor of MATSimApplication was called via reflection at (*); I think that it was called there without a config argument.) </li>
* <li> This call method then will do: <ul>
* <li> getCustomModules() (which is empty by default but can be overriden) </li>
* <li>ConfigUtils.loadConfig(...) _without_ passing on the args</li>
* <li>prepareConfig(...) (which is empty by default but is typically overridden, in this case in OpenBerlinScenario). In our case, this sets the typical scoring params and the typical replanning strategies.
* <li>next one can override the config from some yaml file provided as a commandline option
* <li> next args is parsed and set
* <li>then some standard CL options are detected and set
* <li>then createScenario(config) is called (which can be overwritten but is not)
* <li>then prepareScenario(scenario) is called (which can be overwritten but is not)
* <li>then a standard controler is created from scenario
* <li>then prepareControler is called which can be overwritten
* </ul>
* </ul>
* </p>
*
* @param clazz class of the scenario to run
* @param args pass arguments from the main method
* @param defaultArgs predefined default arguments that will always be present
Expand All @@ -384,6 +413,8 @@ public static void runWithDefaults(Class<? extends MATSimApplication> clazz, Str

if (ApplicationUtils.isRunFromDesktop() && args.length == 0) {

System.setProperty("MATSIM_GUI_DESKTOP", "true");

if (defaultArgs.length > 0) {
String value = String.join(ARGS_DELIMITER, defaultArgs);
System.setProperty("MATSIM_GUI_ARGS", value);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,13 @@ public Integer call() throws Exception {

Gui gui = f.get();

// Set the current working directory to be used in the gui, when run from the command line
// If the gui is run from desktop, the working directory is not overwritten

// Assumption is that starting something from command line, the user expects that the working directory remains the same
if (!System.getProperty("MATSIM_GUI_DESKTOP", "false").equals("true"))
gui.setWorkingDirectory(new File(""));

while (gui.isShowing())
Thread.sleep(250);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,14 @@
import org.apache.avro.file.DataFileWriter;
import org.apache.avro.io.DatumWriter;
import org.apache.avro.specific.SpecificDatumWriter;
import org.apache.commons.lang.StringUtils;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.matsim.api.core.v01.Coord;
import org.matsim.application.avro.XYTData;
import org.matsim.core.config.Config;
import org.matsim.core.utils.io.IOUtils;
import org.matsim.core.utils.misc.Time;
import tech.tablesaw.api.*;
import tech.tablesaw.io.csv.CsvReadOptions;

Expand All @@ -29,7 +32,8 @@
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.*;
import java.util.stream.Collectors;

import static org.geotools.gml3.v3_2.GML.coordinateSystem;

/**
* Merges noise data from multiple files into one file.
Expand All @@ -44,17 +48,16 @@ final class MergeNoiseOutput {
*/
private static final boolean CREATE_CSV_FILES = false;

private final String[] inputPath;
private final Path outputDirectory;
private final String crs;
private final String[] labels = {"immission", "emission"};
private final int minTime = 3600;
private int maxTime = 24 * 3600;

MergeNoiseOutput(String[] inputPath, Path outputDirectory, String crs) {
this.inputPath = inputPath;
this.outputDirectory = outputDirectory;
this.crs = crs;
private final Map<String,Float> totalReceiverPointValues = new HashMap<>();

MergeNoiseOutput(Path path, String coordinateSystem ) {
this.outputDirectory = path;
this.crs = coordinateSystem;
}

/**
Expand Down Expand Up @@ -90,25 +93,9 @@ public void setMaxTime(int maxTime) {
* Merges noise data from multiple files into one file.
*/
public void run() {

// Loop over all paths
for (int i = 0; i < labels.length; i++) {

// Select the correct method based on the label
switch (labels[i]) {
case "immission" -> {
if (CREATE_CSV_FILES) {
mergeImmissionsCSV(inputPath[i], labels[i]);
} else {
mergeImissions(inputPath[i], labels[i]);
}

}
case "emission" -> mergeEmissions(inputPath[i], labels[i]);
default -> log.warn("Unknown path: " + inputPath[i]);
}

}
mergeReceiverPointData(outputDirectory + "/immissions/", "immission");
mergeReceiverPointData(outputDirectory + "/damages_receiverPoint/", "damages_receiverPoint");
mergeLinkData(outputDirectory.toString() + "/emissions/", "emission");
}

/**
Expand All @@ -118,6 +105,7 @@ public void run() {
* @param output
*/
private void writeAvro(XYTData xytData, File output) {
log.info(String.format("Start writing avro file to %s", output.toString() ));
DatumWriter<XYTData> datumWriter = new SpecificDatumWriter<>(XYTData.class);
try (DataFileWriter<XYTData> dataFileWriter = new DataFileWriter<>(datumWriter)) {
dataFileWriter.setCodec(CodecFactory.deflateCodec(9));
Expand All @@ -128,7 +116,7 @@ private void writeAvro(XYTData xytData, File output) {
}
}

private void mergeEmissions(String pathParameter, String label) {
private void mergeLinkData(String pathParameter, String label) {
log.info("Merging emissions data for label {}", label);
Object2DoubleMap<String> mergedData = new Object2DoubleOpenHashMap<>();
Table csvOutputMerged = Table.create(TextColumn.create("Link Id"), DoubleColumn.create("value"));
Expand All @@ -143,9 +131,8 @@ private void mergeEmissions(String pathParameter, String label) {
.separator(';').build());

for (Row row : table) {
// index for Noise Emission xx:xx:xx -> 7
String linkId = row.getString("Link Id");
double value = row.getDouble(7);
double value = row.getDouble(row.columnCount() - 1);
mergedData.mergeDouble(linkId, value, Double::max);

}
Expand All @@ -165,38 +152,49 @@ private void mergeEmissions(String pathParameter, String label) {
}

/**
* Merges the immissions data
* Merges receiverPoint data (written by {@link org.matsim.contrib.noise.NoiseWriter}
*
* @param pathParameter path to the immissions data
* @param label label for the immissions data
* @param outputDir path to the receiverPoint data
* @param label label for the receiverPoint data (which kind of data)
*/
private void mergeImissions(String pathParameter, String label) {
private void mergeReceiverPointData(String outputDir, String label) {

// data per time step, maps coord to value
Int2ObjectMap<Object2FloatMap<FloatFloatPair>> data = new Int2ObjectOpenHashMap<>();

// Loop over all files
//TODO could be adjusted to time bin size from noise config group
String substrToCapitalize = null;
for (int time = minTime; time <= maxTime; time += 3600) {

String path = pathParameter + label + "_" + round(time, 1) + ".csv";
String timeDataFile = outputDir + label + "_" + round(time, 1) + ".csv";

Object2FloatOpenHashMap<FloatFloatPair> values = new Object2FloatOpenHashMap<>();

if (!Files.exists(Path.of(path))) {
log.warn("File {} does not exist", path);
if (!Files.exists(Path.of(timeDataFile))) {
log.warn("File {} does not exist", timeDataFile);
continue;
}

// Read the file
Table table = Table.read().csv(CsvReadOptions.builder(IOUtils.getBufferedReader(path))
.columnTypesPartial(Map.of("x", ColumnType.FLOAT, "y", ColumnType.FLOAT, "Receiver Point Id", ColumnType.INTEGER, "t", ColumnType.DOUBLE))
//we need "damages_receiverPoint" -> "Damages 01:00:00" and "immission" -> "Immision 01:00:00"
substrToCapitalize = label.contains("_") ? label.substring(0, label.lastIndexOf("_")) : label;
String valueHeader = StringUtils.capitalize(substrToCapitalize) + " " + Time.writeTime(time, Time.TIMEFORMAT_HHMMSS);

// Read the data file
Table dataTable = Table.read().csv(CsvReadOptions.builder(IOUtils.getBufferedReader(timeDataFile))
.columnTypesPartial(Map.of("x", ColumnType.FLOAT,
"y", ColumnType.FLOAT,
"Receiver Point Id", ColumnType.INTEGER,
"t", ColumnType.DOUBLE,
valueHeader, ColumnType.DOUBLE))
.sample(false)
.separator(';').build());

// Loop over all rows in the file
for (Row row : table) {
// Loop over all rows in the data file
for (Row row : dataTable) {
float x = row.getFloat("x");
float y = row.getFloat("y");
float value = (float) row.getDouble(1); // 1
float value = (float) row.getDouble(valueHeader);
FloatFloatPair coord = FloatFloatPair.of(x, y);
values.put(coord, value);
}
Expand Down Expand Up @@ -232,7 +230,7 @@ private void mergeImissions(String pathParameter, String label) {
}
}

xytHourData.setData(Map.of("imissions", raw));
xytHourData.setData(Map.of(label, raw));
xytHourData.setCrs(crs);

File out = outputDirectory.getParent().resolve(label + "_per_hour.avro").toFile();
Expand All @@ -254,15 +252,18 @@ private void mergeImissions(String pathParameter, String label) {
xytDayData.setTimestamps(List.of(0));
xytDayData.setXCoords(xCoords);
xytDayData.setYCoords(yCoords);
xytDayData.setData(Map.of("imissions", raw));
xytDayData.setData(Map.of(label, raw));
xytDayData.setCrs(crs);

File outDay = outputDirectory.getParent().resolve(label + "_per_day.avro").toFile();

writeAvro(xytDayData, outDay);
//cache the overall sum
this.totalReceiverPointValues.put(substrToCapitalize, raw.stream().reduce(0f, Float::sum));
}

// Merges the immissions data

@Deprecated
private void mergeImmissionsCSV(String pathParameter, String label) {
log.info("Merging immissions data for label {}", label);
Expand All @@ -278,7 +279,10 @@ private void mergeImmissionsCSV(String pathParameter, String label) {

// Read the file
Table table = Table.read().csv(CsvReadOptions.builder(IOUtils.getBufferedReader(path))
.columnTypesPartial(Map.of("x", ColumnType.DOUBLE, "y", ColumnType.DOUBLE, "Receiver Point Id", ColumnType.INTEGER, "t", ColumnType.DOUBLE))
.columnTypesPartial(Map.of("x", ColumnType.DOUBLE,
"y", ColumnType.DOUBLE,
"Receiver Point Id", ColumnType.INTEGER,
"t", ColumnType.DOUBLE))
.sample(false)
.separator(';').build());

Expand Down Expand Up @@ -319,4 +323,7 @@ private void mergeImmissionsCSV(String pathParameter, String label) {
log.info("Merged noise data written to {} ", outPerDay);

}
public Map<String, Float> getTotalReceiverPointValues() {
return totalReceiverPointValues;
}
}
Loading

0 comments on commit 93b4961

Please sign in to comment.