Skip to content

Commit

Permalink
Merge pull request #21 from SchweizerischeBundesbahnen/feature/20-fea…
Browse files Browse the repository at this point in the history
…ture-repository-implementations-for-csv-files

feat: add csv-based infrastructure repository
  • Loading branch information
munterfi authored Dec 5, 2024
2 parents 32b70f4 + 1b0cb8f commit 87425bc
Show file tree
Hide file tree
Showing 44 changed files with 620 additions and 189 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/maven-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
distribution: 'temurin'
cache: maven
- name: Build with Maven
run: mvn -B package --file pom.xml
run: mvn -B verify --file pom.xml

# Optional: Uploads the full dependency graph to GitHub to improve the quality of Dependabot alerts this repository can receive
# - name: Update dependency graph
Expand Down
32 changes: 23 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@

Converter to expand network graphics from
the [Netzgrafik-Editor](https://github.com/SchweizerischeBundesbahnen/netzgrafik-editor-frontend) into timetables for
the entire service
day in different formats, as for example GTFS static or MATSim transit schedules.
the entire service day in different formats, as for example GTFS static or MATSim transit schedules.

## Usage

Expand All @@ -13,8 +12,10 @@ Run the command line tool to convert a network graphic to either a GTFS or MATSi

```text
Usage: convert [-htV] [-e=<serviceDayEnd>] [-f=<outputFormat>]
[-i=<stopFacilityCsv>] [-r=<rollingStockCsv>]
[-s=<serviceDayStart>] [-v=<validationStrategy>]
<networkGraphicFile> <outputDirectory>
Converts network graphics into timetables in various formats.
<networkGraphicFile> The network graphic file to convert.
<outputDirectory> The output directory for the converted timetable.
Expand All @@ -23,27 +24,41 @@ Converts network graphics into timetables in various formats.
-f, --format=<outputFormat>
Output format (GTFS or MATSim).
-h, --help Show this help message and exit.
-i, --stop-facility-csv=<stopFacilityCsv>
File which contains the coordinates of the stop
facilities.
-r, --rolling-stock-csv=<rollingStockCsv>
File which contains the vehicle types to be mapped
to network graphic categories.
-s, --service-day-start=<serviceDayStart>
Service day start time (HH:mm).
-t, --train-names Use train names as route or line IDs (true/false).
-v, --validation=<validationStrategy>
Validation strategy (SKIP_VALIDATION,
WARN_ON_ISSUES, FAIL_ON_ISSUES, FIX_ISSUES).
WARN_ON_ISSUES, FAIL_ON_ISSUES,
REPLACE_WHITESPACE, REMOVE_SPECIAL_CHARACTERS).
-V, --version Print version information and exit.
```

Since the Netzgrafik-Editor does not provide information about the coordinates of nodes or rolling stock (vehicle types)
serving a category, this information can optionally be provided through CSV files.

Example:

```sh
# configure arguments
# configure paths
NETWORK_GRAPHIC_FILE=src/test/resources/ng/scenarios/realistic.json
OUTPUT_DIRECTORY=integration-test/output/cmd

# run the Spring command line runner app to convert to GTFS format
./mvnw spring-boot:run -Dspring-boot.run.arguments="$NETWORK_GRAPHIC_FILE $OUTPUT_DIRECTORY -f GTFS"
# output format: GTFS or MATSIM
OUTPUT_FORMAT=GTFS

# optional CSV repositories
STOP_FACILITY_INFO_FILE=src/test/resources/ng/scenarios/realistic-stop-facility-info.csv
ROLLING_STOCK_INFO_FILE=src/test/resources/ng/scenarios/realistic-rolling-stock-info.csv

# run the Spring command line runner app to convert to MATSim format with custom service day times
./mvnw spring-boot:run -Dspring-boot.run.arguments="$NETWORK_GRAPHIC_FILE $OUTPUT_DIRECTORY -f MATSIM -s 04:30 -e 26:00"
# run the Spring command line runner app to convert to GTFS / MATSim format
./mvnw spring-boot:run -Dspring-boot.run.arguments="$NETWORK_GRAPHIC_FILE $OUTPUT_DIRECTORY -f $OUTPUT_FORMAT -i $STOP_FACILITY_INFO_FILE -r $ROLLING_STOCK_INFO_FILE"
```

### Converter in Java
Expand Down Expand Up @@ -95,7 +110,6 @@ public class Example {
}

}

```

## Design
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
public class GtfsSupplyBuilder extends BaseSupplyBuilder<GtfsSchedule> {

public static final int ROUTE_TYPE = 2; // rail
public static final String ROUTE_NAME_FORMAT = "%s: %s - %s";

private final List<Stop> stops = new ArrayList<>();
private final List<Route> routes = new ArrayList<>();
Expand All @@ -43,11 +44,15 @@ public GtfsSupplyBuilder(InfrastructureRepository infrastructureRepository, Vehi
super(infrastructureRepository, vehicleCircuitsPlanner);
}

private static String getNameOrIdIfNull(StopFacilityInfo stopFacilityInfo) {
return stopFacilityInfo.getName().isEmpty() ? stopFacilityInfo.getId() : stopFacilityInfo.getName();
}

@Override
protected void buildStopFacility(StopFacilityInfo stopFacilityInfo) {
stops.add(Stop.builder()
.stopId(stopFacilityInfo.getId())
.stopName(stopFacilityInfo.getId())
.stopName(stopFacilityInfo.getName())
.stopLat(stopFacilityInfo.getCoordinate().getLatitude())
.stopLon(stopFacilityInfo.getCoordinate().getLongitude())
.build());
Expand All @@ -60,11 +65,12 @@ protected void buildTransitRoute(TransitRouteContainer transitRouteContainer) {
routeElements.put(transitRouteContainer.transitRouteInfo().getId(), transitRouteContainer.routeElements());

// build transit route names
String routeShortName = String.format("%s - %s",
transitRouteContainer.routeElements().getFirst().getStopFacilityInfo().getId(),
transitRouteContainer.routeElements().getLast().getStopFacilityInfo().getId());
String routeLongName = String.format("%s: %s",
transitRouteContainer.transitRouteInfo().getTransitLineInfo().getCategory(), routeShortName);
String categoryName = transitRouteContainer.transitRouteInfo().getTransitLineInfo().getCategory();
StopFacilityInfo orig = transitRouteContainer.routeElements().getFirst().getStopFacilityInfo();
StopFacilityInfo dest = transitRouteContainer.routeElements().getLast().getStopFacilityInfo();
String routeShortName = String.format(ROUTE_NAME_FORMAT, categoryName, orig.getId(), dest.getId());
String routeLongName = String.format(ROUTE_NAME_FORMAT, categoryName, getNameOrIdIfNull(orig),
getNameOrIdIfNull(dest));

// create and add GTFS route (transit line in the context of the supply builder) if not yet added
String routeId = transitRouteContainer.transitRouteInfo().getTransitLineInfo().getId();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,10 @@ TransitStopFacility buildTransitStopFacility(StopFacilityInfo stopFacilityInfo)
stopFacilityInfo.getCoordinate().getLatitude());
Node node = factory.createNode(String.format("%s", stopId), coord);
Link stopLink = factory.createLink(LinkType.STOP, node, node, STOP_LINK_LENGTH,
stopFacilityInfo.getLinkAttributes());
stopFacilityInfo.getAttributes());

TransitStopFacility stop = factory.createTransitStopFacility(stopId, stopLink);
stop.setName(stopFacilityInfo.getName());
stopFacilities.put(stopId, stop);

return stop;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,8 @@ protected void buildDeparture(VehicleAllocation vehicleAllocation) {

VehicleTypeInfo vehicleTypeInfo = vehicleAllocation.getVehicleInfo().getVehicleTypeInfo();
VehicleType vehicleType = factory.getOrCreateVehicleType(vehicleTypeInfo.getId(), vehicleTypeInfo.getLength(),
vehicleTypeInfo.getMaxVelocity(), vehicleTypeInfo.getCapacity(), vehicleTypeInfo.getAttributes());
vehicleTypeInfo.getMaxVelocity(), vehicleTypeInfo.getSeats(), vehicleTypeInfo.getStandingRoom(),
vehicleTypeInfo.getAttributes());
Vehicle vehicle = factory.getOrCreateVehicle(vehicleType, vehicleAllocation.getVehicleInfo().getId());

DepartureInfo departureInfo = vehicleAllocation.getDepartureInfo();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ TransitLine getOrCreateTransitLine(String id) {
return transitLine;
}

VehicleType getOrCreateVehicleType(String id, double length, double maxVelocity, int capacity, Map<String, Object> attributes) {
VehicleType getOrCreateVehicleType(String id, double length, double maxVelocity, int seats, int standingRoom, Map<String, Object> attributes) {
Id<VehicleType> vehicleTypeId = Id.create(String.format(IdPattern.VEHICLE_TYPE, id), VehicleType.class);
VehicleType vehicleType = vehicles.getVehicleTypes().get(vehicleTypeId);

Expand All @@ -171,12 +171,13 @@ VehicleType getOrCreateVehicleType(String id, double length, double maxVelocity,

log.debug("Creating VehicleType {}", vehicleTypeId);
vehicleType = vf.createVehicleType(vehicleTypeId);
vehicleType.getCapacity().setSeats(capacity);
vehicleType.getCapacity().setSeats(seats);
vehicleType.getCapacity().setStandingRoom(standingRoom);
vehicleType.setMaximumVelocity(maxVelocity);
vehicleType.setLength(length);
VehicleUtils.setDoorOperationMode(vehicleType, VehicleType.DoorOperationMode.parallel);
VehicleUtils.setAccessTime(vehicleType, Default.VEHICLE_ACCESS_TIME);
VehicleUtils.setEgressTime(vehicleType, Default.VEHICLE_EGRESS_TIME);
vehicleType.setLength(length);
putAttributes(vehicleType, attributes);
vehicles.addVehicleType(vehicleType);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,20 @@
import ch.sbb.pfi.netzgrafikeditor.converter.core.NetworkGraphicConverter;
import ch.sbb.pfi.netzgrafikeditor.converter.core.NetworkGraphicConverterConfig;
import ch.sbb.pfi.netzgrafikeditor.converter.core.NetworkGraphicSource;
import ch.sbb.pfi.netzgrafikeditor.converter.core.supply.InfrastructureRepository;
import ch.sbb.pfi.netzgrafikeditor.converter.core.supply.RollingStockRepository;
import ch.sbb.pfi.netzgrafikeditor.converter.core.supply.SupplyBuilder;
import ch.sbb.pfi.netzgrafikeditor.converter.core.supply.VehicleCircuitsPlanner;
import ch.sbb.pfi.netzgrafikeditor.converter.core.supply.fallback.NoInfrastructureRepository;
import ch.sbb.pfi.netzgrafikeditor.converter.core.supply.fallback.NoRollingStockRepository;
import ch.sbb.pfi.netzgrafikeditor.converter.core.supply.fallback.NoVehicleCircuitsPlanner;
import ch.sbb.pfi.netzgrafikeditor.converter.io.csv.CsvInfrastructureRepository;
import ch.sbb.pfi.netzgrafikeditor.converter.io.csv.CsvRollingStockRepository;
import ch.sbb.pfi.netzgrafikeditor.converter.io.gtfs.GtfsScheduleWriter;
import ch.sbb.pfi.netzgrafikeditor.converter.io.matsim.TransitScheduleXmlWriter;
import ch.sbb.pfi.netzgrafikeditor.converter.io.netzgrafik.JsonFileReader;
import lombok.Builder;
import lombok.Value;
import org.matsim.api.core.v01.Scenario;
import org.springframework.stereotype.Service;

Expand All @@ -23,30 +30,54 @@
@Service
public class ConversionService {

public void convert(Path networkGraphicFile, Path outputDirectory, NetworkGraphicConverterConfig config, OutputFormat outputFormat) throws IOException {
NetworkGraphicSource source = new JsonFileReader(networkGraphicFile);
private static InfrastructureRepository configureInfrastructureRepository(Path stopFacilityCsv) throws IOException {
return stopFacilityCsv == null ? new NoInfrastructureRepository() : new CsvInfrastructureRepository(
stopFacilityCsv);
}

private static RollingStockRepository configureRollingStockRepository(Path rollingStockCsv) throws IOException {
return rollingStockCsv == null ? new NoRollingStockRepository() : new CsvRollingStockRepository(
rollingStockCsv);
}

public void convert(Request request) throws IOException {
NetworkGraphicSource source = new JsonFileReader(request.networkGraphicFile);

InfrastructureRepository infrastructureRepository = configureInfrastructureRepository(request.stopFacilityCsv);
RollingStockRepository rollingStockRepository = configureRollingStockRepository(request.rollingStockCsv);
VehicleCircuitsPlanner vehicleCircuitsPlanner = new NoVehicleCircuitsPlanner(rollingStockRepository);

NetworkGraphicConverter<?> converter = switch (request.outputFormat) {

NetworkGraphicConverter<?> converter = switch (outputFormat) {
case GTFS -> {
SupplyBuilder<GtfsSchedule> builder = new GtfsSupplyBuilder(new NoInfrastructureRepository(),
new NoVehicleCircuitsPlanner(new NoRollingStockRepository()));
ConverterSink<GtfsSchedule> sink = new GtfsScheduleWriter(outputDirectory, true);
SupplyBuilder<GtfsSchedule> builder = new GtfsSupplyBuilder(infrastructureRepository,
vehicleCircuitsPlanner);
ConverterSink<GtfsSchedule> sink = new GtfsScheduleWriter(request.outputDirectory, true);

yield new NetworkGraphicConverter<>(config, source, builder, sink);
yield new NetworkGraphicConverter<>(request.converterConfig, source, builder, sink);
}

case MATSIM -> {
SupplyBuilder<Scenario> builder = new MatsimSupplyBuilder(new NoInfrastructureRepository(),
new NoVehicleCircuitsPlanner(new NoRollingStockRepository()));
String baseFilename = networkGraphicFile.getFileName().toString();
String filenameWithoutExtension = baseFilename.substring(0, baseFilename.lastIndexOf('.'));
ConverterSink<Scenario> sink = new TransitScheduleXmlWriter(outputDirectory,
filenameWithoutExtension + ".");

yield new NetworkGraphicConverter<>(config, source, builder, sink);
SupplyBuilder<Scenario> builder = new MatsimSupplyBuilder(infrastructureRepository,
vehicleCircuitsPlanner);
ConverterSink<Scenario> sink = new TransitScheduleXmlWriter(request.outputDirectory);

yield new NetworkGraphicConverter<>(request.converterConfig, source, builder, sink);
}

};

converter.run();
}

@Value
@Builder
public static class Request {
Path networkGraphicFile;
Path outputDirectory;
NetworkGraphicConverterConfig converterConfig;
OutputFormat outputFormat;
Path stopFacilityCsv;
Path rollingStockCsv;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -18,27 +18,48 @@ public class ConvertCommand implements Callable<Integer> {

private final BuildProperties buildProperties;
private final ConversionService conversionService;

// positional arguments
@CommandLine.Parameters(index = "0", description = "The network graphic file to convert.")
private Path networkGraphicFile;
@CommandLine.Parameters(index = "1", description = "The output directory for the converted timetable.")

// converter configuration
private Path outputDirectory;
@CommandLine.Option(names = {"-v", "--validation"}, description = "Validation strategy (SKIP_VALIDATION, WARN_ON_ISSUES, FAIL_ON_ISSUES, FIX_ISSUES).", defaultValue = "WARN_ON_ISSUES")
@CommandLine.Option(names = {"-v", "--validation"}, description = "Validation strategy (SKIP_VALIDATION, WARN_ON_ISSUES, FAIL_ON_ISSUES, REPLACE_WHITESPACE, REMOVE_SPECIAL_CHARACTERS).", defaultValue = "WARN_ON_ISSUES")
private ValidationStrategy validationStrategy;
@CommandLine.Option(names = {"-t", "--train-names"}, description = "Use train names as route or line IDs (true/false).", defaultValue = "false")
private boolean useTrainNamesAsIds;
@CommandLine.Option(names = {"-s", "--service-day-start"}, description = "Service day start time (HH:mm).", converter = ServiceDayTimeConverter.class, defaultValue = "04:00")
private ServiceDayTime serviceDayStart;
@CommandLine.Option(names = {"-e", "--service-day-end"}, description = "Service day end time (HH:mm).", converter = ServiceDayTimeConverter.class, defaultValue = "25:00")
private ServiceDayTime serviceDayEnd;

// format and repositories
@CommandLine.Option(names = {"-f", "--format"}, description = "Output format (GTFS or MATSim).", defaultValue = "GTFS")
private OutputFormat outputFormat;
@CommandLine.Option(names = {"-i", "--stop-facility-csv"}, description = "File which contains the coordinates of the stop facilities.")
private Path stopFacilityCsv;
@CommandLine.Option(names = {"-r", "--rolling-stock-csv"}, description = "File which contains the vehicle types to be mapped to network graphic categories.")
private Path rollingStockCsv;

@Override
public Integer call() throws Exception {
conversionService.convert(networkGraphicFile, outputDirectory, deriveNetworkGraphicConfig(), outputFormat);
conversionService.convert(deriveConversionServiceRequest());
return 0;
}

private ConversionService.Request deriveConversionServiceRequest() {
return ConversionService.Request.builder()
.networkGraphicFile(networkGraphicFile)
.outputDirectory(outputDirectory)
.converterConfig(deriveNetworkGraphicConfig())
.outputFormat(outputFormat)
.stopFacilityCsv(stopFacilityCsv)
.rollingStockCsv(rollingStockCsv)
.build();
}

private NetworkGraphicConverterConfig deriveNetworkGraphicConfig() {
return NetworkGraphicConverterConfig.builder()
.validationStrategy(validationStrategy)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,8 @@ private void addStops() {
log.info("Adding nodes of network graphic");
for (Node node : lookup.nodes.values()) {
log.debug("Adding node {}", node.getBetriebspunktName());
builder.addStopFacility(node.getBetriebspunktName(), node.getPositionX(), node.getPositionY());
builder.addStopFacility(node.getBetriebspunktName(), node.getFullName(), node.getPositionX(),
node.getPositionY());
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,12 @@ public BaseSupplyBuilder(InfrastructureRepository infrastructureRepository, Vehi
}

@Override
public SupplyBuilder<T> addStopFacility(String id, double x, double y) {
public SupplyBuilder<T> addStopFacility(String id, String name, double x, double y) {
if (stopFacilityInfos.containsKey(id)) {
throw new RuntimeException("Stop already existing for id " + id);
}

StopFacilityInfo stopFacilityInfo = infrastructureRepository.getStopFacility(id, x, y);
StopFacilityInfo stopFacilityInfo = infrastructureRepository.getStopFacility(id, name, x, y);
if (stopFacilityInfo == null) {
throw new RuntimeException("Stop with id " + id + " does not exist in infrastructure repository");
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

public interface InfrastructureRepository {

StopFacilityInfo getStopFacility(String stopId, double x, double y);
StopFacilityInfo getStopFacility(String stopId, String stopName, double x, double y);

List<TrackSegmentInfo> getTrack(StopFacilityInfo fromStop, StopFacilityInfo toStop, TransitRouteInfo transitRouteInfo);

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package ch.sbb.pfi.netzgrafikeditor.converter.core.supply;

import ch.sbb.pfi.netzgrafikeditor.converter.util.spatial.Coordinate;
import lombok.Value;

import java.util.HashMap;
Expand All @@ -9,7 +10,8 @@
public class StopFacilityInfo {

String id;
String name;
Coordinate coordinate;
Map<String, Object> linkAttributes = new HashMap<>();
Map<String, Object> attributes = new HashMap<>();

}
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

public interface SupplyBuilder<T> {

SupplyBuilder<T> addStopFacility(String id, double x, double y);
SupplyBuilder<T> addStopFacility(String id, String name, double x, double y);

SupplyBuilder<T> addTransitLine(String id, String category);

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package ch.sbb.pfi.netzgrafikeditor.converter.core.supply;

import ch.sbb.pfi.netzgrafikeditor.converter.util.spatial.Coordinate;
import lombok.Value;

import java.util.HashMap;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@
public class VehicleTypeInfo {

String id;
int capacity;
int seats;
int standingRoom;
double length;
double maxVelocity;
Map<String, Object> attributes;
Expand Down
Loading

0 comments on commit 87425bc

Please sign in to comment.