Skip to content

Commit

Permalink
docs & example
Browse files Browse the repository at this point in the history
  • Loading branch information
@dickermoshe
dickermoshe committed Oct 6, 2024
1 parent a438f6e commit c02754c
Show file tree
Hide file tree
Showing 45 changed files with 565 additions and 288 deletions.
15 changes: 15 additions & 0 deletions docs/docs/Migrations/api.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,21 @@ callback. However, the callbacks also give you an instance of `Migrator` as a
parameter. This class knows about the target schema of the database and can be
used to create, drop and alter most elements in your schema.

## General tips

To ensure your schema stays consistent during a migration, you can wrap it in a `transaction` block.
However, be aware that some pragmas (including `foreign_keys`) can't be changed inside transactions.
Still, it can be useful to:

- always re-enable foreign keys before using the database, by enabling them in [`beforeOpen`](#post-migration-callbacks).
- disable foreign-keys before migrations
- run migrations inside a transaction
- make sure your migrations didn't introduce any inconsistencies with `PRAGMA foreign_key_check`.

With all of this combined, a migration callback can look like this:

{{ load_snippet('structured','lib/snippets/migrations/migrations.dart.excerpt.json') }}

## Migrating views, triggers and indices

When changing the definition of a view, a trigger or an index, the easiest way
Expand Down
34 changes: 13 additions & 21 deletions docs/docs/Migrations/exports.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ description: Store all schema versions of your app for validation.

---


!!! warning "Important Note"

This command is specifically for exporting schemas.
If you are using the `make-migrations` command, this is already done for you.

By design, drift's code generator can only see the current state of your database
schema. When you change it, it can be helpful to store a snapshot of the older
schema in a file.
Expand Down Expand Up @@ -38,27 +44,16 @@ my_app
Of course, you can also use another folder or a subfolder somewhere if that suits your workflow
better.

!!! note "Examples available"


Exporting schemas and generating code for them can't be done with `build_runner` alone, which is
why this setup described here is necessary.

We hope it's worth it though! Verifying migrations can give you confidence that you won't run
into issues after changing your database.
If you get stuck along the way, don't hesitate to [open a discussion about it](https://github.com/simolus3/drift/discussions).

Also there are two examples in the drift repository which may be useful as a reference:

- A [Flutter app](https://github.com/simolus3/drift/tree/latest-release/examples/app)
- An [example specific to migrations](https://github.com/simolus3/drift/tree/latest-release/examples/migrations_example).



Exporting schemas and generating code for them can't be done with `build_runner` alone, which is
why this setup described here is necessary.

We hope it's worth it though! Verifying migrations can give you confidence that you won't run
into issues after changing your database.
If you get stuck along the way, don't hesitate to [open a discussion about it](https://github.com/simolus3/drift/discussions).

## Exporting the schema

To begin, lets create the first schema representation:
To begin, let's create the first schema representation:

```
$ mkdir drift_schemas
Expand Down Expand Up @@ -93,9 +88,6 @@ $ dart run drift_dev schema dump lib/database/database.dart drift_schemas/drift_
database file, you can do that as well! `drift_dev schema dump` recognizes a sqlite3 database file as its first
argument and can extract the relevant schema from there.




## What now?

Having exported your schema versions into files like this, drift tools are able
Expand Down
163 changes: 92 additions & 71 deletions docs/docs/Migrations/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,33 +5,104 @@ description: Tooling and APIs to safely change the schema of your database.

---

The strict schema of tables and columns is what enables type-safe queries to
the database.
But since the schema is stored in the database too, changing it needs to happen
through migrations developed as part of your app. Drift provides APIs to make most
migrations easy to write, as well as command-line and testing tools to ensure
the migrations are correct.
Drift ensures type-safe queries through a strict schema. To change this schema, you must write migrations.
Drift provides a range of APIs, command-line tools, and testing utilities to make writing and verifying database migrations easier and more reliable.

## Guided Migrations

## Drift tools
Drift offers an all-in-one command for writing and testing migrations.
This tool helps you write your schema changes incrementally and generates tests to verify that your migrations are correct.

Writing correct migrations is crucial to ensure that your users won't end up with a broken database
in an inconsistent state.
For this reason, drift offers tools that make writing and testing migrations safe and easy. By exporting
each version of your schema to a json file, drift can reconstruct older versions of your database
schema. This is helpful to:
### Configuration

1. Test migrations between any two versions of your app.
2. Write migrations between older versions your app more easily, as the current schema generated by drift
would also include subsequent changes from newer versions.
To use the `make-migrations` command, you must add the location of your database(s) to your `build.yaml` file.

## Make Migration
```yaml title="build.yaml"
targets:
$default:
builders:
drift_dev:
options:
databases:
# Required: A name for the database and it's path
my_database: lib/database.dart

Drift offers an all-in-one command for writing and testing migrations.
The `make-migration` command will save the current state of your database definition, create a migration helper function and create a test to verify the migration does what it should.
Run `dart run drift_dev make-migrations --help` for a detailed explanation of the command.
# Optional: Add more databases
another_db: lib/database2.dart
```
You can also optionally specify the directory where the test files and schema (1) files are stored.
{ .annotate }
1. Drift will generate multiple schema files, one for each version of your database schema. These files are used to compare the current schema with the previous schema and generate the migration code.
```yaml title="build.yaml"
targets:
$default:
builders:
drift_dev:
options:
# The directory where the test files are stored:
test_dir: test/drift/ # (default)

# The directory where the schema files are stored:
schema_dir: drift_schemas/ # (default)
```
### Usage
Before you start making changes to your initial database schema, run this command to generate the initial schema file.
```bash
dart run drift_dev make-migrations
```
Once this initial schema file is saved, you can start making changes to your database schema.


Once you're happy with the changes, bump the `schemaVersion` in your database class and run the command again.

```bash
dart run drift_dev make-migrations
```

This command will generate the following files:

- A step-by-step migration file will be generated next to your database class. Use this function to write your migrations incrementally. See the [step-by-step migration guide](step_by_step.md) for more information.


- Drift will also generate a test file for your migrations. After you've written your migration, run the tests to verify that your migrations are written correctly.

- Drift will also generate a file which can be used to make the tests validate the data integrity of your migrations. These files should be filled in with before and after data for each migration.

If you get stuck along the way, don't hesitate to [open a discussion about it](https://github.com/simolus3/drift/discussions).


### Example

See the [example](https://github.com/simolus3/drift/tree/develop/examples/migrations_example) in the drift repository for a complete example of how to use the `make-migrations` command.

### Switching to `make-migrations`

If you've already been using the `schema` tools to write migrations, you can switch to `make-migrations` by following these steps:

1. Run the `make-migrations` command to generate the initial schema file.
2. Move all of your existing `schema` files into the schema directory for your database.
3. Run the `make-migrations` command again to generate the step-by-step migration file and test files.

## During development

During development, you might be changing your schema very often and don't want to write migrations for that
yet. You can just delete your apps' data and reinstall the app - the database will be deleted and all tables
will be created again. Please note that uninstalling is not enough sometimes - Android might have backed up
the database file and will re-create it when installing the app again.

You can also delete and re-create all tables every time your app is opened, see [this comment](https://github.com/simolus3/drift/issues/188#issuecomment-542682912)
on how that can be achieved.

## Manual setup
## Manual Migrations

!!! warning "Manual migrations are error-prone"
Writing migrations manually is error-prone and can lead to data loss. We recommend using the `make-migrations` command to generate migrations and tests.

Drift provides a migration API that can be used to gradually apply schema changes after bumping
the `schemaVersion` getter inside the `Database` class. To use it, override the `migration`
Expand All @@ -54,28 +125,7 @@ However, be aware that drift expects the latest schema when creating SQL stateme
For instance, when adding a new column to your database, you shouldn't run a `select` on that table before
you've actually added the column. In general, try to avoid running queries in migration callbacks if possible.

Writing migrations without any tooling support isn't easy. Since correct migrations are
essential for app updates to work smoothly, we strongly recommend using the tools and testing
framework provided by drift to ensure your migrations are correct.
To do that, [export old versions](exports.md) to then use easy
[step-by-step migrations](step_by_step.md) or [tests](tests.md).

## General tips

To ensure your schema stays consistent during a migration, you can wrap it in a `transaction` block.
However, be aware that some pragmas (including `foreign_keys`) can't be changed inside transactions.
Still, it can be useful to:

- always re-enable foreign keys before using the database, by enabling them in [`beforeOpen`](#post-migration-callbacks).
- disable foreign-keys before migrations
- run migrations inside a transaction
- make sure your migrations didn't introduce any inconsistencies with `PRAGMA foreign_key_check`.

With all of this combined, a migration callback can look like this:

{{ load_snippet('structured','lib/snippets/migrations/migrations.dart.excerpt.json') }}

## Post-migration callbacks
## Post-Migration callbacks

The `beforeOpen` parameter in `MigrationStrategy` can be used to populate data after the database has been created.
It runs after migrations, but before any other query. Note that it will be called whenever the database is opened,
Expand Down Expand Up @@ -114,32 +164,3 @@ beforeOpen: (details) async {
}
```

## During development

During development, you might be changing your schema very often and don't want to write migrations for that
yet. You can just delete your apps' data and reinstall the app - the database will be deleted and all tables
will be created again. Please note that uninstalling is not enough sometimes - Android might have backed up
the database file and will re-create it when installing the app again.

You can also delete and re-create all tables every time your app is opened, see [this comment](https://github.com/simolus3/drift/issues/188#issuecomment-542682912)
on how that can be achieved.

## Verifying a database schema at runtime

Instead (or in addition to) [writing tests](#verifying-a-database-schema-at-runtime) to ensure your migrations work as they should,
you can use a new API from `drift_dev` 1.5.0 to verify the current schema without any additional setup.



{{ load_snippet('(full)','lib/snippets/migrations/runtime_verification.dart.excerpt.json') }}

When you use `validateDatabaseSchema`, drift will transparently:

- collect information about your database by reading from `sqlite3_schema`.
- create a fresh in-memory instance of your database and create a reference schema with `Migrator.createAll()`.
- compare the two. Ideally, your actual schema at runtime should be identical to the fresh one even though it
grew through different versions of your app.

When a mismatch is found, an exception with a message explaining exactly where another value was expected will
be thrown.
This allows you to find issues with your schema migrations quickly.
62 changes: 33 additions & 29 deletions docs/docs/Migrations/step_by_step.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,6 @@ description: Use generated code reflecting over all schema versions to write mig

---



Database migrations are typically written incrementally, with one piece of code transforming
the database schema to the next version. By chaining these migrations, you can write
schema migrations even for very old app versions.
Expand All @@ -27,31 +25,6 @@ Sure, we could remember that the migration from 1 to 2 is now pointless and just
upgrades from 1 to 3 directly, but this adds a lot of complexity. For more complex migration scripts
spanning many versions, this can quickly lead to code that's hard to understand and maintain.

## Generating step-by-step code

Drift provides tools to [export old schema versions](exports.md). After exporting all
your schema versions, you can use the following command to generate code aiding with the implementation
of step-by-step migrations:

```
$ dart run drift_dev schema steps drift_schemas/ lib/database/schema_versions.dart
```

The first argument (`drift_schemas/`) is the folder storing exported schemas, the second argument is
the path of the file to generate. Typically, you'd generate a file next to your database class.

The generated file contains a `stepByStep` method which can be used to write migrations easily:

{{ load_snippet('stepbystep','lib/snippets/migrations/step_by_step.dart.excerpt.json') }}

`stepByStep` expects a callback for each schema upgrade responsible for running the partial migration.
That callback receives two parameters: A migrator `m` (similar to the regular migrator you'd get for
`onUpgrade` callbacks) and a `schema` parameter that gives you access to the schema at the version you're
migrating to.
For instance, in the `from1To2` function, `schema` provides getters for the database schema at version 2.
The migrator passed to the function is also set up to consider that specific version by default.
A call to `m.recreateAllViews()` would re-create views at the expected state of schema version 2, for instance.

## Customizing step-by-step migrations

The `stepByStep` function generated by the `drift_dev schema steps` command gives you an
Expand All @@ -64,8 +37,8 @@ shows:

{{ load_snippet('stepbystep2','lib/snippets/migrations/step_by_step.dart.excerpt.json') }}

Here, foreign keys are disabled before runnign the migration and re-enabled afterwards.
A check ensuring no inconsistencies occurred helps catching issues with the migration
Here, foreign keys are disabled before running the migration and re-enabled afterwards.
A check ensuring no inconsistencies occurred helps to catch issues with the migration
in debug modes.

## Moving to step-by-step migrations
Expand All @@ -84,3 +57,34 @@ this point. From now on, you can generate step-by-step migrations for each schem

If you did not do this, a user migrating from schema 1 directly to schema 3 would not properly walk migrations
and apply all migration changes required.

## Manual Generation

!!! warning "Important Note"

This command is specifically for generating the step by step migration helper.
If you are using the `make-migrations` command, this is already done for you.


Drift provides tools to [export old schema versions](exports.md). After exporting all
your schema versions, you can use the following command to generate code aiding with the implementation
of step-by-step migrations:

```
$ dart run drift_dev schema steps drift_schemas/ lib/database/schema_versions.dart
```

The first argument (`drift_schemas/`) is the folder storing exported schemas, the second argument is
the path of the file to generate. Typically, you'd generate a file next to your database class.

The generated file contains a `stepByStep` method which can be used to write migrations easily:

{{ load_snippet('stepbystep','lib/snippets/migrations/step_by_step.dart.excerpt.json') }}

`stepByStep` expects a callback for each schema upgrade responsible for running the partial migration.
That callback receives two parameters: A migrator `m` (similar to the regular migrator you'd get for
`onUpgrade` callbacks) and a `schema` parameter that gives you access to the schema at the version you're
migrating to.
For instance, in the `from1To2` function, `schema` provides getters for the database schema at version 2.
The migrator passed to the function is also set up to consider that specific version by default.
A call to `m.recreateAllViews()` would re-create views at the expected state of schema version 2, for instance.
Loading

0 comments on commit c02754c

Please sign in to comment.