Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updog: only remove known extensions from migration filenames #662

Merged
merged 1 commit into from
Jan 17, 2020

Conversation

tjkirch
Copy link
Contributor

@tjkirch tjkirch commented Jan 17, 2020

If we always set_extension, it removes anything after a period. If the
migration does not happen to be named with an extension, then the only period
is in the version number, so we change "migrate_0.1_foo" to "migrate_0", which
is not a valid migration filename. The migration won't run and the upgrade
will fail.

With this change, we only remove the extension (.lz4) that matches our
compression format.

Migrations downloaded by older updog could not have an extension because it
would not be removed and would again be invalid, so unfortunately we have a
diverging path. Versions before the addition of set_extension (let's call them
version A) cannot have an extension, and versions with set_extension but
without this change (let's call them version B) must have an extension, which
means that update manifests need to list special copies of migrations that have
an extension for any paths leading in/out of that intermediate version B. This
must be maintained until version B is no longer supported.

If we call releases including this change "version C", then here's an example
set of updates to work around this issue:

  # Original migration before finding the issue; don't want to block update into B
  (A, B): ["migrate_0.B_foo"]
  # If starting from A and updating past B, skip over the special migration needed by B
  (A, C): ["migrate_0.B_foo", "migrate_0.C_bar"]
  # Special migration out of affected version B
  (B, C): ["migrate_0.C_bar.lz4"]

As long as B is supported, each new version will need a special path from B.
For example, after releasing version D:

  # Original migration before finding the issue; don't want to block update into B
  (A, B): ["migrate_0.B_foo"]
  # If starting from A and updating past B, skip over the special migration needed by B
  (A, C): ["migrate_0.B_foo", "migrate_0.C_bar"]
  # Normal new migrations go here, for new versions like D
  (C, D): ["migrate_0.D_baz"]
  # Special migration out of affected version B
  (B, C): ["migrate_0.C_bar.lz4"]
  # We need a path from B to each new version so they have extensions to be removed
  (B, D): ["migrate_0.C_bar.lz4", "migrate_0.D_baz.lz4"]

Testing done:

Before, when updating to a new test version, I had a migration named "migrate_0" that did not run because it didn't have a valid name, so my update failed.

After, I have the right name:

bash-5.0# updog update -n
Starting update to 0.2.1
** Updating immediately **
Update applied: aws-k8s-0.2.1
bash-5.0# ls -l /var/lib/thar/datastore/migrations/migrate_0.2_containerd-config-path
-rwxr-xr-x 1 root root 506336 Jan 17 19:11 /var/lib/thar/datastore/migrations/migrate_0.2_containerd-config-path

The migration ran properly and the update was successful.

If we always set_extension, it removes anything after a period.  If the
migration does not happen to be named with an extension, then the only period
is in the version number, so we change "migrate_0.1_foo" to "migrate_0", which
is not a valid migration filename.  The migration won't run and the upgrade
will fail.

With this change, we only remove the extension (.lz4) that matches our
compression format.

Migrations downloaded by older updog could not have an extension because it
would not be removed and would again be invalid, so unfortunately we have a
diverging path.  Versions before the addition of set_extension (let's call them
version A) cannot have an extension, and versions with set_extension but
without this change (let's call them version B) must have an extension, which
means that update manifests need to list special copies of migrations that have
an extension for any paths leading in/out of that intermediate version B.  This
must be maintained until version B is no longer supported.

If we call releases including this change "version C", then here's an example
set of updates to work around this issue:
```
  # Original migration before finding the issue; don't want to block update into B
  (A, B): ["migrate_0.B_foo"]
  # If starting from A and updating past B, skip over the special migration needed by B
  (A, C): ["migrate_0.B_foo", "migrate_0.C_bar"]
  # Special migration out of affected version B
  (B, C): ["migrate_0.C_bar.lz4"]
```

As long as B is supported, each new version will need a special path from B.
For example, after releasing version D:
```
  # Original migration before finding the issue; don't want to block update into B
  (A, B): ["migrate_0.B_foo"]
  # If starting from A and updating past B, skip over the special migration needed by B
  (A, C): ["migrate_0.B_foo", "migrate_0.C_bar"]
  # Normal new migrations go here, for new versions like D
  (C, D): ["migrate_0.D_baz"]
  # Special migration out of affected version B
  (B, C): ["migrate_0.C_bar.lz4"]
  # We need a path from B to each new version so they have extensions to be removed
  (B, D): ["migrate_0.C_bar.lz4", "migrate_0.D_baz.lz4"]
```
@tjkirch tjkirch merged commit ad58567 into develop Jan 17, 2020
@tjkirch tjkirch deleted the updog-migration-extension branch January 17, 2020 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants