diff --git a/README.md b/README.md index 972ff445..c6943414 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ You can think of it as the grandson of the legacy [ezxmlinstaller](https://githu In either `require` or `require-dev` at the end of the bundle list in the composer.json file add: - "kaliop/ezmigrationbundle": "^2.0" + "kaliop/ezmigrationbundle": "^3.0" Save it and run @@ -41,7 +41,7 @@ The `registerBundles` method should look similar to: ### Checking that the bundle is installed correctly -If you run `php ezpublish/console` you should see 4 new commands in the list: +If you run `php ezpublish/console` you should see the following new commands in the list: kaliop kaliop:migration:generate @@ -59,9 +59,20 @@ To get the latest version, you can update the bundle to the latest available ver composer update kaliop/ezmigrationbundle +### Upgrading from version 2.x to version 3 + +* Make sure you read carefully all the BC notes in the [release notes](WHATSNEW.md) + +* Nothing else is required, unless you have one of the following: + + - migrations definitions generated using extension versions 1.x or 2.x, yet to be applied + - code which extends the migration bundle code/apis + + For both cases, the fix is to apply manual changes to your code/migrations. + ### Upgrading from version 1.x to version 2 -Please read the [dedicated documentation page](doc/Upgrading/1.x_to_2.0.md) +Please read the [dedicated documentation page](Resources/doc/Upgrading/1.x_to_2.0.md) ## Getting started @@ -77,14 +88,14 @@ For example: php ezpublish/console kaliop:migration:generate --format=yml MyProjectBundle -The above command will place a new yml skeleton file in the `MigrationVersion` directory of the MyProjectBundle bundle. +The above command will place a new yml skeleton file in the `MigrationVersions` directory of the MyProjectBundle bundle. If the directory does not exists then the command will create it for you, as long as the bundle does exist and is registered. If the command is successful it will create a new yml file named with the following pattern: `YYYYMMDDHHMMSS_placeholder.yml`. You are encouraged to rename the file and change the `placeholder` part to something more meaningful, but please keep -the timestamp part and underscore, as well as the extension +the timestamp part and underscore, as well as the extension. -(the contents of the skeleton Yaml file are stored as twig template) +_(the contents of the skeleton Yaml file are stored as twig template)_ ### Listing all migrations and their status @@ -118,6 +129,16 @@ So far so good, but what kind of actions can be actually done using a migration? Each migration definition consists of a series of steps, where each step defines an action. +A simple example of a migration to create a 'folder' content is: + + - + mode: create + type: content + content_type: folder + parent_location: 2 + attributes: + name: hello world + In a Yaml migration, you can define the following types of actions: - creation, update and deletion of Contents - creation, update and deletion of Locations @@ -125,8 +146,11 @@ In a Yaml migration, you can define the following types of actions: - creation, update and deletion of UserGroups - creation, update and deletion of Roles - creation, update and deletion of ContentTypes +- creation and deletion of ContentTypeGroups +- creation, update and deletion of ObjectStates +- creation, update and deletion of ObjectStateGroups - creation and deletion of Languages -- creation of Tags (from the Netgen Tags Bundle) +- creation and deletion of of Tags (from the Netgen Tags Bundle) The docs describing all supported parameters are in the [DSL Language description](Resources/doc/DSL/README.md) @@ -187,9 +211,11 @@ After removing the information about the migration form the migrations table, ru ## Usage of transactions / rolling back changes By default the bundle runs each migration in a database transaction. -This means that if a step fails, all of the previous steps get rolled back, and the database is left at its previous state. -This is a safety feature built in by design; if you prefer the migration steps to be executed in separate transactions -the easiest way is to create a separate migration file for each step. +This means that if a step fails, all of the previous steps get rolled back, and the database is left in its original state. +This is a safety feature built in by design; +* if you prefer the migration steps to be executed in separate transactions the easiest way is to create a separate + migration file for each step +* you can use the command-line flag `-u` to disable usage of transactions by the migrate command Note also that by default the `migrate` command stops on the 1st failed migration, but it can be executed with a flag to allow it to continue and execute all available migrations even in case of failures. @@ -232,6 +258,8 @@ and the corresponding php class: ## Known Issues and limitations +* unlike the Symfony Migrations Bundle, this bundle does not support rollback of changes. Read above for the reason. + * if you get fatal errors when running a migration stating that a node or object has not been found, it is most likely related to how the dual-kernel works in eZPublish, and the fact that the legacy and Symfony kernels use a separate connection to the database. Since the migration bundle by default wraps all database changes for a migration in a diff --git a/Resources/doc/DSL/README.md b/Resources/doc/DSL/README.md index b151f9d8..27202656 100644 --- a/Resources/doc/DSL/README.md +++ b/Resources/doc/DSL/README.md @@ -7,6 +7,7 @@ Specific topics are covered below. *NB* For more examples of valid Yaml files describing migrations, have a look in the directory Tests/dsl/good + ## Content language By default the bundle uses eng-GB for creating all multilingual entities (contents, contentTypes, users, etc...). @@ -14,34 +15,30 @@ In order to create content in a different language, either specify it in your ym use a command-line switch. -## Importing binary files +## Notes on field types -To import binary files like images into attributes (ezimage, ezbinaryfile fields) the attribute needs to be defined as -a complex type to tell the system to handle it differently. +### Importing binary files -Below is a snippet of how a simple/primitive field type is defined in a content creation Yaml definition: +Below is an example snippet for creation/update of a content with a field of type ezimage: attributes: - - title: 'Example title' - -To define a complex attribute we need to indicate the type of the attribute and then provide the required data fields for -the attribute. + image: + path: /path/to/the/image.jpg + alt_text: 'Example alt text' -Below is an example snippet for ezimage: +A simplified form, which does not allow for setting the 'alt' attribute, is: attributes: - - image: - type: ezimage - path: /path/to/the/image.jpg - alt_text: 'Example alt text' + image: /path/to/the/image.jpg -Images __have__ to be placed in the `MigrationVersions/images` folder. -Binary files __have__ to be placed in the `MigrationVersions/files` folder. -The paths to files/images in the definition are relative paths from the MigrationVersions/images or -MigrationVersions/files folders. -For example using the path from the snippet above the system would look for the image file in -`MigrationVersions/images/path/to/the/image.jpg` in the bundle's directory. +The paths to files/images in the definition are either +* absolute paths, or +* relative paths from the MigrationVersions/images or MigrationVersions/files folders. + +For example using the path from the snippet above the system would look first for the image file in +`MigrationVersions/images/path/to/the/image.jpg` in the bundle's directory, and if no file is found there, look for +`/images/path/to/the/image.jpg` Please see the `ManageContent.yml` DSL definition file in the `Resources/doc/DSL` folder for more information. @@ -58,10 +55,10 @@ in other steps. For example, you could set a reference to a the location id of a folder that you create and then use that as the parent location for creating articles in that folder. -Here is another example on using references: +Here is an example on using references: the first step creates a new content type in the system and sets a reference to its id; -the second step adds a new policy to the editor role to allow editors to create objects of the new content -type under the location with id 2. +the second step adds a new policy to the editor role to allow editors to create objects of the new content type under +the location with id 2. - mode: create @@ -81,7 +78,6 @@ type under the location with id 2. - identifier: section_page_class attribute: content_type_id - - mode: update type: role @@ -116,6 +112,28 @@ associated to the reference found. *NB:* references are stored *in memory* only and will not propagate across different migrations, unless you execute the migrations in a single command (and without the 'separate processes' switch). +*NB:* please do not use the character `]` in your reference names. See below for the reason. + +### References in the XML for the eZXMLText Field + +To tell the system to look for references in the xml that is used to populate ezxmltext type fields the Yaml definition +will need to use the definition used for defining complex attributes. +Please see the importing binary files section above on how to define complex data type handling for an attribute. + +Below is an example snippet showing how to define references for ezxmltext. + + attributes: + - description: + type: ezxmltext + content: '
' + +*NB:* when using references in xml texts you must include the two extra characters `[` and `]`, which are not needed +when using them as part of other elements in the yml file. +This is done to minimize the chances that some random bits of text get modified by error (and because we need an +end-of-reference identifier character). + +### Complete list of available reference + *note:* the following lists are currently out of date - look in the single DSL files for the complete set. Currently you can use references to store the following values: @@ -160,21 +178,3 @@ You can use references to set the following values: - `user_group_id` For more information please see the DSL definitions in the `Resources/doc/DSL` folder. - - -## References in the XML for the eZXMLText Field - -To tell the system to look for references in the xml that is used to populate ezxmltext type fields the Yaml definition -will need to use the definition used for defining complex attributes. -Please see the importing binary files section above on how to define complex data type handling for an attribute. - -Below is an example snippet showing how to define references for ezxmltext. - - attributes: - - description: - type: ezxmltext - content: '
' - -*NB:* when using references in xml texts you must include the two extra characters `[` and `]`, which are not needed -when using them as part of other elements in the yml file. -This is done to minimize the chances that some random bits of text get modified by error. diff --git a/Resources/doc/Upgrading/2.x_to_3.0.md b/Resources/doc/Upgrading/2.x_to_3.0.md new file mode 100644 index 00000000..3b348600 --- /dev/null +++ b/Resources/doc/Upgrading/2.x_to_3.0.md @@ -0,0 +1,11 @@ +Upgrade instructions for upgrading from version 1.x to 2.0 +========================================================== + +1. Make sure you read carefully all the BC notes in the [release notes](WHATSNEW.md) + +2. Nothing else is required, unless you have one of the following: + + - migrations definitions generated using extension versions 1.x or 2.x, yet to be applied + - code which extends the migration bundle code/apis + + For both cases, the fix is to apply manual changes to your code/migrations. diff --git a/WHATSNEW.md b/WHATSNEW.md index 9ab9ddd5..9a5c2ac7 100644 --- a/WHATSNEW.md +++ b/WHATSNEW.md @@ -1,56 +1,12 @@ Version 3.0.0 ============= -* New: better error messages on content creation/update when content fields do not validate (issue #84) - -* Fix: when creating/updating contents, a NULL value is now accepted as valid by all (known) field types, including - object relation, image and binary file - -* New: added support for creating and deleting Content Type Groups - -* New: made it easier to allow custom references types to be substituted in xmltext and richtext fields - - -Version 3.0.0-beta6 -=================== - -* New: it is now possible to use a priority to the services tagged to act as complex field handlers - -* Changes to the YML definition language: - - * renamed the `main_Location` tag for content creation to `parent_location`. The old syntax works but is deprecated - - * renamed the `other_Locations` tag for content creation to `other_parent_locations`. The old syntax works but is deprecated - - -Version 3.0.0-beta5 -=================== - -* New: It is now possible to replace the executor and step definition in a BeforeStepExecutionEvent - -* New: it is now possible to set values to content fields of type eztags - - -Version 3.0.0-beta4 -=================== - -* Add a method for fast dropping of all migrations from the db - - -Version 3.0.0-beta3 -=================== - * New: it is now possible to store migration definitions in json format instead of yaml. The json format is not documented separately, as it is identical in structure to the yaml one. -* New: the 'migrate' command learned a `--separate-process` option, to run each migration it its own separate - php process. This should help when running many migrations in a single pass, and there are f.e. memory leaks. - * New: the 'migrate' command learned to give information ion the executed steps when using the `-v` option -* New: the 'migration' command learned a `--skip` option, to tag migrations as to be skipped - -* New: it is now possible to create, update and delete Object States and Object State Groups +* New: it is now possible to set values to content fields of type eztags * New: updating and deleting of Users, User Groups, Roles and Content Types now accepts more flexible definitions of the elements to act upon, and comprehensive resolution of references @@ -70,32 +26,29 @@ Version 3.0.0-beta3 * New: references are now resolved for user_id and group_id when assigning Roles -* New: the `parent_location` and `other_parent_locations` tags in Content creation, as well as `parent_location` in Location - creation will now try to identify the desired Location by its remote id when a non integer value is used +* New: the `parent_location` and `other_parent_locations` tags in Content creation, as well as `parent_location` in + Location creation will now try to identify the desired Location by its remote id when a non integer value is used * New: the 'roles' specified when creating user groups can be so via a reference * New: the content_type_group tags in ContentType creation will accept a Content Type Group identifier besides an id - -* New: added 2 events to which you can listen to implement custom logic just-before and just-after migration steps are - executed: - - * ez_migration.before_execution => listeners receive a BeforeStepExecutionEvent event instance - * ez_migration.step_executed => listeners receive a StepExecutedEvent event instance +* Fix: when creating/updating contents, a NULL value is now accepted as valid by all (known) field types, including + object relation, image and binary file * Fix: migrations will not silently swallow any more errors when creating users or assigning roles and a non-existing group is specified -* Fix: (issue 78) when using transactions, do not report 'Migration can not be ended' if there is an error during the - commit phase +* New: made it easier to allow custom references types to be substituted in xmltext and richtext fields -* Fix: (issue 76) when using transactions on either ezplatform or ezpublish with workflows, migrations might fail - when creating/editing contents that the anon user can not access +* New: it is now possible to use a priority to the services tagged to act as complex field handlers + +* New: added 2 events to which you can listen to implement custom logic just-before and just-after migration steps are + executed: -* Fix: throw correct exception when trying to access unset property in API/Value objects + * ez_migration.before_execution => listeners receive a BeforeStepExecutionEvent event instance -* Fix: properly set the 'enabled' field when creating languages + * ez_migration.step_executed => listeners receive a StepExecutedEvent event instance * New: it is now possible to add resolvers for custom references using tagged services. The tag to use is: `ez_migration_bundle.reference_resolver.customreference`. @@ -107,15 +60,20 @@ Version 3.0.0-beta3 This gives greater flexibility in deciding which types of references are resolved for each field when creating or updating contents -* Changed: removed unused Behat and Phpunit tests as well as one leftover Interface +* Changed: removed unused Behat and Phpunit tests * Changed: removed from the YML documentation the description of keys which had been deprecated in version 2. The keys are still accepted, but support for them will be dropped in a future version -* Changed: the service ez_migration_bundle.reference_resolver.content is now deprecated and will be removed in a future version +* Changed: the service `ez_migration_bundle.reference_resolver.content` is now deprecated and will be removed in a future + version; the service `ez_migration_bundle.helper.role_handler` has been removed * Changes to the YML definition language: + * renamed the `main_Location` tag for content creation to `parent_location`. The old syntax works but is deprecated + + * renamed the `other_Locations` tag for content creation to `other_parent_locations`. The old syntax works but is deprecated + * Creation and update of content: the format use to specify the attributes has been simplified. The old format is still working but is considered deprecated and will be removed in the future @@ -129,10 +87,10 @@ Version 3.0.0-beta3 * content update supports the following new tags: `creation_date` * location creation and update now support the tag `parent_location` as preferred form for `parent_location_id`. - The former variant is considered deprecated and will be desupported in a future version + The latter variant is considered deprecated and will be desupported in a future version * rich text fields in content creation/update can be specified using a string instead of an array with key 'content'. - References will still be resolved if found in the xml text. + References will still be resolved if found in the xml text * when specifying values for image and binary-file fields, the tags `filename` and `mime_type` can now be used @@ -150,13 +108,76 @@ Version 3.0.0-beta3 * references to 'tag:' and 'location:' will not be resolved any more in fields of type Object Relation and Object Relation List. On the other hand non-integer strings will be resolved as remote-content-ids - * changes for developers who extended the bundle: the MatcherInterface, ReferenceResolverInterface and - StorageHandlerInterface have a new method each; many Executor services now have a different constructor - signature; one Trait has been removed from the public API; the service ez_migration_bundle.helper.role_handler - has been renamed to ez_migration_bundle.helper.limitation_converter; the chainResolver will now apply reference - transformation from all matching sub-resolvers, not only from the 1st one; the interface - `StorageHandlerInterface` has acquired a new method; the method `endMigration` in interface - `StorageHandlerInterface` has acquired a new parameter + * changes for developers who extended the bundle: too many to be listed here. New interfaces where added, existing + interfaces modified, etc... A lot of service definitions have been modified as well. + + +Version 2.5.1 +============= + +* Fix: bring back the support for resolving 'location:' tags as was done in version 1.X of the extension + +* New: better error messages on content creation/update when content fields do not validate (issue #84) + + +Version 2.5 +=========== + +* New: add support for creating and deleting Content Type Groups + + +Version 2.4.2 +============= + +* Fix: improve detection of failed migrations when using separate processes and the child process dies without traces + + +Version 2.4.1 +============= + +* Improve fix for issues #76 and #78 + + +Version 2.4 +=========== + +* New: it is now possible to create, update and delete Object States and Object State Groups + +* Fix: (issue #78) when using transactions, do not report 'Migration can not be ended' if there is an error during the + commit phase + +* Fix: (issue #76) when using transactions on either ezplatform or ezpublish with workflows, migrations might fail + when creating/editing contents that the anon user can not access + +* Fix: throw correct exception when trying to access unset property in API/Value objects + +* Fix: properly set the 'enabled' field when creating languages + +* BC BREAK: for developers extending the bundle: the method `endMigration` in interface `StorageHandlerInterface` has + acquired a new parameter. + Also, the unused DefinitionHandlerInterface has been removed. + + +Version 2.3 +=========== + +* New: the 'migration' command learned a `--skip` option, to tag migrations as to be skipped + +* BC BREAK: for developers extending the bundle: the interface `StorageHandlerInterface` has acquired a new method + + +Version 2.2.1 +============= + +* Fix: when using the `--separate-process` option for the 'migrate' command allow the migration to last up to a day, + and do not wait until it is finished to print its output + + +Version 2.2.0 +============= + +* New: the 'migrate' command learned a `--separate-process` option, to run each migration it its own separate + php process. This should help when running many migrations in a single pass, and there are f.e. memory leaks. Version 2.1.0