Update the dataflow README.

Add information about the new stage and dataflow to the README.
PanDAWMS · Apr 19, 2019 · 7117242 · 7117242
1 parent 181fb14
commit 7117242
Showing 1 changed file with 11 additions and 0 deletions.
diff --git a/Utils/Dataflow/README b/Utils/Dataflow/README
@@ -43,6 +43,7 @@
 1.9     019_oracle2esFormat            Prepare data for bulk upload to ES
 2.5     025_chicagoES                  Get additional metadata from Chicago ES
 6.9     069_upload2es                  Upload data to ES
+7.1     071_esConsistency              Check data in ES
 9.1     091_datasetsRucio              Get dataset metadata from Rucio
 9.3     093_datasetsFormat             Update dataset metadata: add
                                          "data_format" field
@@ -79,6 +80,16 @@
                                           |
     ** ----> 019 -> 069                   | Prepare and upload metadata to ES
 
+3. Tasks metadata consistency check
+---
+
+009 ---> 016 ---> 071
+
+This is a simplified and slightly changed version of the previous dataflow,
+intended for making sure that information is consistent between ProdSys2 and
+ES. It gets a very basic set of metadata from ProdSys2, adds ES-related
+fields, and checks that it is present in ES rather than uploading it.
+
 ==============
 * REFERENCES *
 ==============