Harmonize manifest pack names/filenames so we can use foundryvtt-cli

[rsek]

foundryvtt-cli isn't quite ready for use in a github workflow yet (it would need to support folders and ideally expose more of its API), but it's getting there.

one problem i'd like to get ahead of: it expects the filename of a manifest packs entry to match its name. while this is explicitly stated in the API docs it hasn't really been a problem until now.

so we're either going to need to rename a bunch of files (easy) or do migrations for account for the name change (harder).

i'd actually lean towards the latter, as i'll need to futz with a bunch of pack names in the near feature, and it gives me an excuse to standardize the names 😉 . some examples:

• splitting delve content to their own packs (currently experimenting with this in Refactor + enhance oracle importers for v11 #774 )
• splitting delve content to its own package Explore splitting Delve content/functionality into its own module #757
• pointing references to foe actors to their new location Provide foe actor to character actor drag behaviour to replace foe progress item packs #763

i think i'd like to handle this by writing a custom migration method for HTMLField. it can look at a parent document's _documentStats, and if it's below a certain system version, it can do the appropriate replacements.

that'd probably be easier after migrating the rest of our custom subclasses to the v10 DataModel but that's already something i'm thinking about after #762. and, IIRC there's a way to flag for HTMLField in template.json.

[rsek]

I'm experimenting with migration strategies for this. Here's some notes.

The FVTT _id of a given Dataforged-based Document is deterministic (since it's the DFID passed through hash), and there will only be one entry with a given ID across all packs (with the exception of encounter actors/items, but i'm planning to change that). Which means we can regex for link targets like this...

const legacyLinkPointerPattern =
  /@Compendium\[foundry-ironsworn\.[A-Za-z0-9-_]+?\.(?<id>[a-zA-Z0-9]{16})\]/g

...and then look for the captured <id> in all system-provided packs. This should turn up the index entry of the "canonical" Document for that ID. We can get the index entry's uuid and replace the matched substring with a new target:

const packIndices = game.packs.filter((pack) =>
  pack.collection.startsWith(game.system.id + '.')).map(pack => pack.index)

return str.replaceAll(
  legacyLinkPointerPattern,
    (_match, _p1, _offset, _string, { id } ) => {
      // search pack indices for entries keyed with the local id
      for (const packIndex in packIndices) {
        const indexEntry = packIndex.get(id)
        if (indexEntry == null) continue
        return `@UUID[${indexEntry.uuid}]`
    }
  }
)

So long as the system's packs have already cached their indexes, this can be done synchronously; however, migrateData runs initializing pack data, so it may not work there. Couple of ways this could be circumvented:

• run it on everything after init, like old migrations
• generate a static JSON file that maps the short _ids to "canonical" uuids; use that instead of the indices for lookup

Note that in a post-v10 world, any of these checks/migrations could be contingent on testing e.g. foundry.utils.isNewerVersion('1.21.3', someDocument._stats.systemVersion)

It's a little trickier to recursively iterate over all document string values, but I've figured out a couple ways to pull it off:

• iterate over fields using DataModel.apply, testing against field instanceof foundry.data.fields.HTMLField
• replacer functions in JSON.parse and JSON.stringify; test typeof value === 'string' and run the replacer if it's true; if it fails it can be omitted

Both approaches can generate an array of document update deltas that are then passed to e.g. Item.updateDocuments to minimize DB calls.

[ben]

So we now have an automated build step that runs as part of npm run build, and it transforms the JSON files in e.g. json-packs/starforged-assets into dist/packs/starforged-assets. So the part of this that's left is changing the Foundry compendium keys to match the file paths.