Skip to content

MultipleGraphs

Jump to bottom Edit New page
abyrd edited this page Oct 6, 2012 · 25 revisions

Introduction

If you want to provide OpenTripPlanner services for several distinct geographic areas, you do not need to run several OTP webapps. With a bit of configuration a single OTP instance can load multiple graphs and direct incoming requests to the proper graph.

The routerId

Most OTP API methods support an optional routerId query parameter specifying which graph the query is to be executed against. When no routerId is provided in a query, OTP falls back on a default routerId which can be configured via the defaultRouterId property of the GraphServiceImpl. The defaultRouterId is the empty string unless set to a different value. RouterIds are strings composed of alphanumeric characters, dashes, and underscores only.

Before handling each request, an OTP server resolves the routerId to a specific Graph object. The server maintains a mapping from routerIds to Graphs, and a routerId must be registered and associated with a loaded Graph to be valid for use in OTP API requests. You can register graphs via Spring configuration or using the routers API at opentripplanner-api-webapp/ws/routers/.

A client must send a routerId if it wants to use any graph other than the default one. The config.js configuration file in the opentripplanner-webapp module contains a default empty value you can modify for this purpose.

A routerId is invalid for use in API requests until it is registered, and becomes invalid again if it is de-registered. When a routerId is registered it is associated with an OTP Graph object, which is typically loaded from a serialized graph file on disk. The filesystem or resource path provided to the GraphServiceImpl via the Spring XML config file is in fact a base path. When a graph is loaded from disk, it will always be found in an immediate subdirectory of that base path, and the subdirectory's name will be the same as the routerId. When the routerId is the empty string, the graph file will be found in the base path itself rather than any subdirectory of that base path.

In basic single-graph operation, the OTP server has a single mapping from the default defaultRouterId (the empty string) to a graph loaded from a serialized graph object found in the GraphServiceImpl's base directory (which defaults to /var/otp/graphs/).

Registering routerIds at startup with Spring XML

When an OTP server starts up, it will look for a graph on disk corresponding to the defaultRouterId and load it (unless told not to by setting the attemptLoadDefault property to 'false'). Additional routerIds can be registered automatically via the autoRegister property of the GraphServiceImpl component, which takes a list of Strings specifying the routerIds to attempt to register on startup. Here is an example GraphServiceImpl configuration for use in data-sources.xml:

  <!-- specify a GraphService, configuring the path to the serialized Graphs -->
    <bean id="graphService" class="org.opentripplanner.routing.impl.GraphServiceImpl">
      <property name="path" value="/var/otp/graphs/" />
      <property name="defaultRouterId" value="pdx" />
      <property name="autoRegister">
        <list>
          <value>nyc</value>
          <value>seattle</value>
        </list>
      </property>
    </bean>

When using this configuration, upon startup OTP will attempt to register the routerIds pdx, nyc, and seattle, which will resolve to graphs loaded from /var/otp/graphs/pdx/Graph.obj, /var/otp/graphs/nyc/Graph.obj, and /var/otp/graphs/seattle/Graph.obj respectively. If any of these files does not exist or is corrupted or otherwise unreadable, the corresponding routerId will not be registered.

The routers REST API

Registering a routerId

HTTP PUT curl -u user:PASS -v -X PUT http://opentripplanner-api-webapp/ws/routers/{routerId} curl -u user:PASS -v -X PUT http://opentripplanner-api-webapp/ws/routers/

Reloading graph data

Graphs can be reloaded from disk if they have changed (i.e. if you have some automated system producing new graphs when GTFS files change). Graph reloads must be triggered via the routers API. A graph will not be automatically reloaded when its corresponding file changes on disk because a) this carries a risk of loading a partially written graph file, and b) because graph loading is a relatively slow operation that will either make a routerId unavailable or double memory use for up to several minutes.

There is internal protection against truncated graph files, but if you are still concerned about partially written graphs being loaded, you can use an atomic copy/move.

Please note that in the worst case, this may require double the amount of RAM, as two graphs will be present in memory at the same time. This can have a huge impact if you update all graphs for all routerIds at once. In production scripts you can follow the following sequence if you use the asyncReload feature:

  • (If needed) Copy the new data to the same filesystem as the old data
  • Atomically move the new data over the old
  • Immediately poll the planner (wget http://host.tld/webapp/ws/metadata?routerId=new-york) to force a reload and wait until it's done
  • Process the next graph/routerId

This will ensure that 1) the blocking request which will reload the data will not be a user request, and 2) you minimize the number of graph objects present at the same time in RAM.

Graph eviction

HTTP DELETE curl -u user:PASS -v -X DELETE http://opentripplanner-api-webapp/ws/routers/{routerId}

You may not see memory use drop immediately upon graph eviction because Java is a garbage-collected language. We simply release all references to the graph; externally visible indicators like used/free memory and the overall size of the heap may not reflect this change until the next time the JVM goes looking for some free memory.

Pre-eviction

Sample series of REST commands

Security Config

OpenTripPlanner API endpoints that allow modifying sensitive information, as well as those that might put an unreasonable amount of load on the server are secured. This includes the routers API, since it allows you to load graphs into memory or evict them. Security is handled by Spring and is configured in opentripplanner-api-webapp/src/main/resources/org/opentripplanner/security-application-context.xml. You will need to configure credentials for ROLE_DEPLOYER to use the routers API. Below is an example file configuring HTTP basic access authentication, which is far from truly secure but will get you started. Besides ROLE_DEPLOYER which grants access to the routers API, you can configure ROLE_USER which grants access to secured endpoints in the OTP internals API.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:security="http://www.springframework.org/schema/security"
	xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                           http://www.springframework.org/schema/security
                           http://www.springframework.org/schema/security/spring-security-3.0.xsd">

    <security:global-method-security secured-annotations="enabled" jsr250-annotations="enabled" />

    <!-- This is the simple basic auth configuration.-->
    <security:http>
        <security:http-basic></security:http-basic>
        <security:intercept-url method="POST" pattern="/**" access="ROLE_USER" />
    </security:http>

    <security:authentication-manager  alias="authenticationManager">
        <security:authentication-provider>
            <security:user-service>
                <security:user name="admin" password="{password}" authorities="ROLE_USER" />
                <security:user name="router" password="{router_pwd}" authorities="ROLE_DEPLOYER" />
            </security:user-service>
        </security:authentication-provider>
    </security:authentication-manager>

</beans>
Add a custom footer
Add a custom sidebar
Clone this wiki locally