-
Notifications
You must be signed in to change notification settings - Fork 0
Dirac v6r10
The are 3 main features introduced within this release:
-
The full use of the new version of the Request Management System, introduced first within release v6r9, together with the Transformation System and Data Management system.
-
Many changes are applied in the TransformationSystem, and particularly in the TransformationDB.
-
A new "package", named Workflow, has been introduced. This package makes it easier, for VOs, to define Job Workflows, introducing generic modules implementations that can be extended by the VOs.
Those new features that need special care when upgrading from previous versions to v6r10 are outlined below.
Using the new RMS system everywhere means using it for all the replication activities, that were using the old RMS system up to version v6r9. The old system, populating the old set of tables, will not be feed any more. System administrators might want to check that all the requests are treated within the old system before stopping the corresponding agents: this is the reason why, in v6r10 code (as well as v6r9), a mix of services and agents from the old and new system are present. The code treating the old system will be removed from the next minor, or major, release of DIRAC. If you are upgrading from DIRAC v6r9, no new agents nor services need to be started.
Changes have been applied in several places, including Accounting and Transformation System.
As part of the changes for the new RMS, a new DB, and a new set of tables will be created at the start of the FTSManagerService, provided that the FTSDB has been previously created. This DB, together with the new implementation of the service and clients, will substitute the old system. Old requests present in the old system will keep to be treated by the old services and agents.
Few code changes are:
- the TransformationClient now provides an easily extensible State Machine for Transformation Status and for the status of the Transformation Files.
- when deriving a transformation, the Transformation Files status will be deterministic.
- the methods for the Replicas table have been deleted. But the biggest changes are at the DB level: please read further.
The tables definition is switching from MyISAM to InnoDB, and few triggers are being introduced. The code will be fully compatible, so there is no need, for any VOs, to switch MyISAM to InnoDB. It is anyway highly recommended. Applying this change needs special care. It has to be noted that, for this release, both the tables definition, and the triggers, are defined in the TransformationDB.sql file, and that system administrators are required to manually intervene at the database level to apply them. The changes to apply follow:
- TransformationDB.Replicas can be fully removed, including its content.
- The primary key of TransformationDB.DataFiles should be only "FileID" (see https://github.com/DIRACGrid/DIRAC/pull/1724/files for details)
- The other changes are coded in https://github.com/DIRACGrid/DIRAC/pull/1731/files, and includes settings of PKs, FKs, and a trigger. Pay special attention to the TransformationFiles and TransformationTasks table.
In any case, to repeat: VOs can schedule the change of the schema at their favourite time, independently from the introduction of DIRAC v6r0, because tables defined within MyISAM will keep working.
The new workflow package provides four generic modules. Detailed instructions on how to use it and extend it can be found in the DIRAC.Workflow.Modules.ModuleBase module.
Some new policies have been introduced and are ready to be used. The configuration of the authorization should be changed from "get = all" to "select = all".
- A small change has been applied to the JobLoggingDB: the primary key changed, as well as a new Trigger has been introduced. Details can be seen in https://github.com/DIRACGrid/DIRAC/pull/1724/files
- A new JobPolicy is in place <>