Putting all migration data into the database

[stuartpb]

This is the logical predecessor of #12 and #13 that doesn't appear to have been given its own issue yet.

Right now, dreamcopter incorporates three pieces when managing a DingRoll migration:

• the ZIP export from Slack
• the PouchDB database of original and changed messages
• the YAML definition of the migration plan

#12 talks about putting the last one into the database; what's really needed, going forward, is pre-emptively putting all the messages from the ZIP export into the PouchDB, when it's imported, instead of only doing it when each individual day is saved, which is what happens right now, and is a weird mess.

[stuartpb]

My current plan to do this, given that I have live data:

• Find the ZIP file that I'm doing the current exports with.
• On a local copy of the current data (which I have backed up), manually run a line that'll rename all the documents to use their channel IDs instead of their channel names (Channel IDs instead of names in document IDs #7), as well as adding types (Types on documents #12).
• Manually insert a channels document into the aforementioned database (may need to do this first, in order to do the rename).
• Make a "plan" document, using that instead of localStorage (Migration plans in database #15).
• Add a step to importing that creates / updates "channels" and "users" documents (ie. documents with the IDs channels and users). This will probably just be in the form of clobbering what's currently in the export with whatever's imported (though there should probably be a merge-based mechanism so you can't inadvertently erase channel IDs by accidentally importing an older export - it's arguably as trivial to undo as it'd be to avoid, though).
• Add the code that imports each message into the database on importing the archive to Dreamcopter. (Note that this will import to the channel IDs, per Channel IDs instead of names in document IDs #7, and should only be implemented after the manual rename described in Step 2.)
• Re-import the most recent archive, to create channels and users documents (and to import newer messages, obviously).

[stuartpb]

Here's the code I used for step 2 above:

var oldDb = new PouchDB('reslacked');
var newDb = new PouchDB('dreamcopter_default');
var nameMap = new Map(/* paste the output of JSON.parse(readFileSync('channels.json')).map(({id, name}) => ([name, id])) here */)

var allOps = [];

oldDb.allDocs().then(res=>{
  res.rows.forEach(row => {
    if (row.id[0] != '_') {
      allOps.push(oldDb.get(row.id).then(doc => {
        delete doc._rev;
        doc.type = 'messages';
        doc._id = doc._id.replace(/.*\//,prefix => nameMap.get(prefix.slice(0,-1))+'/');
        return newDb.put(doc);
      }))
    }
  });
  return Promise.all(allOps);
}).then(console.log('Finished'))

[stuartpb]

Note that I'm changing the name from reslacked to dreamcopter_default to pave the way for #13

[stuartpb]

Just manually created the metadata document, too.

[stuartpb]

Okay, looks like everything's functional.