Skip to content
This repository has been archived by the owner on Mar 22, 2023. It is now read-only.

Facilitate versioning and migrations #16

Open
bitdancer opened this issue Sep 30, 2016 · 3 comments
Open

Facilitate versioning and migrations #16

bitdancer opened this issue Sep 30, 2016 · 3 comments

Comments

@bitdancer
Copy link

Like a more traditional DB, pynvm applications are probably going to want to version their persistent data "schema", and provide automatic migrations when the schema changes. pynvm.pmemobj can facilitate this at a minimum level by providing an 'application version' string stored in the root object and exposed as a property of the pool object, and a way to treat a version mismatch as an error (and a way not to, so one can do a migration).

We could also go further and provide a migrations infrastructure. If we choose to do the latter we should look at best practices in the SQL realm. This ticket is really only about the minimal version support., though.

@krzycz
Copy link

krzycz commented Sep 30, 2016

But we already have a layout string for that (an argument passed to pmemobj_create/open), which is a property of a persistent memory pool. Actually, you can put anything there - a string/number, app version, DB schema version, etc., so this is something that could/should be used for versioning the persistent data "schema". It works exactly as you described - if layout does not match - it's an error.
IMHO, there's no need to duplicate this in a root object (but this is one of the options, of course).

@bitdancer
Copy link
Author

Can you open the file` anyway if the layouts don't match? Can you change the layout without completely rewriting the DB? Both of those are needed for migrations. If you can do both of those, or at least the latter, then yes, I could make it part of the layout string (currently I'm just using layout to version the format of the pynvm.pmemobj object memory layout, which I don't expect to provide migrations for (I want to do dump/restore instead, another issue I should open :)

@krzycz
Copy link

krzycz commented Oct 3, 2016

  1. Yes, you can open the pool w/o checking layout - just pass NULL as layout to pmemobj_open.
  2. No, there is simple way to change the layout string once the pool is created. Moreover, there is no API to read the current layout string. (You can do it with pmempool info/dump).
    I believe, we could make it possible by extending libpmempool API, or with pmemobj_ctl (FEAT: pmemobjctl - statistics and control submodule issues#211).

Anyway, I agree that currently the only option is probably to the use root object to store the information about the application/schema version.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants