Skip to content

Commit

Permalink
Set EOP to "" to remove need for eop_filter.
Browse files Browse the repository at this point in the history
  • Loading branch information
Evildoor committed Sep 1, 2020
1 parent 130d600 commit 7ba11b6
Showing 1 changed file with 11 additions and 11 deletions.
22 changes: 11 additions & 11 deletions Utils/Dataflow/data4es/batch_stage/run.sh
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
source ../../shell_lib/eop_filter

cmd="./stage.py -m s"
cmd_batch2="./stage.py -b 2 -m s"
cmd_batch100="./stage.py -b 100 -m s"
# $1 has to be used as a workaround to pass empty string as a value for
# -E, since "... -E ''" will treat the single quotes literally due to double
# quotes around them.
cmd="./stage.py -m s -E $1"
cmd_batch2="./stage.py -b 2 -m s -E $1"
cmd_batch100="./stage.py -b 100 -m s -E $1"

# Various tests that should produce the same results.

# Stage chains.
# these differ by size without eop_filter at the end
cat inp | $cmd | eop_filter | $cmd | eop_filter > outp1
cat inp | $cmd_batch2 | eop_filter | $cmd_batch2 | eop_filter > outp2
cat inp | $cmd_batch100 | eop_filter | $cmd_batch100 | eop_filter > outp100
cat inp | $cmd | eop_filter | $cmd_batch2 | eop_filter > outp12
cat inp | $cmd_batch2 | eop_filter | $cmd | eop_filter > outp21
cat inp | $cmd "" | $cmd "" > outp1
cat inp | $cmd_batch2 "" | $cmd_batch2 "" > outp2
cat inp | $cmd_batch100 "" | $cmd_batch100 "" > outp100
cat inp | $cmd "" | $cmd_batch2 "" > outp12
cat inp | $cmd_batch2 "" | $cmd "" > outp21

0 comments on commit 7ba11b6

Please sign in to comment.