Migrating Services

Synopsis

Sometimes, when making changes to a service, you'll need to modify the data that currently exists either in Mongo or in the Me directory. To facilitate these changes in a seamless way without the user needing to do anything, we've built a remedial migration system. The migration system, at this point, is extremely simple and only supports migrating forward. Migrations are applicable to connectors, applications, and collections.

When installing a service for the first time, it's assumed that no migrations are required. Migrations are only intended for upgrading from previously installed versions of the service. Therefore, when a service is installed, the version is set to the current timestamp. The version is stored in the me.json file, which is the generic storage location for all data for a given service. Any services that were setup before we implemented the migration system are assumed to have a version of 1. When a service is starting up, if there are any migration files with a version number higher than the number in the me.json file, they will be run sequentially and will block the starting of the service until they've all finished. Migrations are all stored under the migrations directory within a service's directory.

Writing a migration

First, here's an example of a migration:

module.exports = function(dir) {
    process.chdir(dir);

    var fs = require('fs');

    var me = JSON.parse(fs.readFileSync('me.json', 'ascii'));

    if (me.mongoCollections) {
         me.mongoCollections.push('new_collection');
    } else {
        me.mongoCollections = ['new_collection'];
    }

    fs.writeFileSync('me.json', JSON.stringify(me), 'ascii');
    return true;
};

This is a simple migration that's used to update the mongoCollections property of a given connector. This is the proper way to add more dataStores to your connector as functionality gets added.

This migration can be named anything you'd like, so long as the first 13 characters of the filename are the timestamp (or version) of the migration. You can leave the filename as just these 13 characters, but having more descriptive filenames is encouraged.

All migration files need to follow this simple model of accepting a directory (this directory will be somewhere underneath the Me directory and is done to ensure that multiple instances of a given service are all handled independently from one another.) and consisting of an exported function that's immediately run when the migration has been required.