Skip to content

Commit

Permalink
Remove service fields before checking.
Browse files Browse the repository at this point in the history
Service fields are different from data fields and shouldn't be checked.
  • Loading branch information
Evildoor committed May 28, 2019
1 parent 8458030 commit 77e23fb
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions Utils/Dataflow/071_esConsistency/consistency.py
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,11 @@ def process(stage, message):
if field in data:
del data[field]

# Fields starting with an underscore are service fields. Some of them are
# treated in special way (see _id above). Service fields should not be
# checked, so they are removed.
data = {field: data[field] for field in data if field[0] != '_'}

# Do not check empty documents with valid _id and _type.
# It's unlikely that such documents will be produced in DKB. In general,
# such documents should be checked by es.exists(), and not es.get().
Expand Down

0 comments on commit 77e23fb

Please sign in to comment.