Fixes for workflow-multiplicities and setVarsFromALIEN_PROC_ID scripts

[shahor02]

No description provided.

[github-actions]

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2022-pp-apass4
async-2023-pbpb-apass3
async-2023-pbpb-apass4
async-2022-pp-apass6-2023-PbPb-apass2
async-2022-pp-apass4-accepted
async-2022-pp-apass6-2023-PbPb-apass2-accepted
async-2023-pbpb-apass3-accepted
async-2023-pbpb-apass4-accepted
async-2023-pp-apass4
async-2023-pp-apass4-accepted
async-2024-pp-apass1
async-2022-pp-apass7
async-2024-pp-cpass0
async-2024-pp-cpass0-accepted
async-2022-pp-apass7-accepted
async-2024-pp-apass1-accepted

[chiarazampolli]

I am not sure about this, did you test it? I did in the version in master.

[shahor02]

Yes, for me it was failing with space present.

[chiarazampolli]

The behaviour without the space is different. The idea was to take a negative value.

Can you show me the error you got?

Here a script that I just tested, it is the same as before your change (I just pass as parameter the ALIEN_PROC_ID) and for me it works:

#!/bin/bash

# Script to set a couple of variables that depend on ALIEN_PROC_ID.
# Since several scripts might need this, it is in a separate script.

# let's get the last 9 (for int) and 8 (for int16) digits of ALIEN_PROC_ID, to be passed to define NUMAID and shm-segment-id via O2JOBID, which are int and int16 respectively. Then we make them an int or int16
ALIEN_PROC_ID_MAX_NDIGITS_INT32=9
ALIEN_PROC_ID_MAX_NDIGITS_INT16=8

#########################
# for testing purposes only
export ALIEN_JDL_PACKAGES=test
ALIEN_PROC_ID=$1
#########################

echo "ALIEN_PROC_ID for current job = ${ALIEN_PROC_ID}"

if [[ -n ${ALIEN_JDL_PACKAGES} ]] && [[ ${#ALIEN_PROC_ID} -lt ${ALIEN_PROC_ID_MAX_NDIGITS_INT32} ]]; then # we are on the grid, and we expect to have the PROC_ID
  echo "We cannot determine O2JOBID, the job id is too short (${ALIEN_PROC_ID}), we need at least ${ALIEN_PROC_ID_MAX_NDIGITS_INT32} digits, returning error"
  #exit 2
fi

ALIEN_PROC_ID_OFFSET_INT32=$((10#${ALIEN_PROC_ID: -${ALIEN_PROC_ID_MAX_NDIGITS_INT32}}))
echo "ALIEN_PROC_ID_OFFSET_INT32 = $ALIEN_PROC_ID_OFFSET_INT32"

ALIEN_PROC_ID_OFFSET_INT16=$((10#${ALIEN_PROC_ID: -${ALIEN_PROC_ID_MAX_NDIGITS_INT16}}))
echo "ALIEN_PROC_ID_OFFSET_INT16 = $ALIEN_PROC_ID_OFFSET_INT16"

# let's make them int32 or int16, but not with the max possible value (which would be 0x7fffffff and 0x7fff respectively)
# but a bit less, to allow to add [0, 15] on top afterwards if needed (e.g. we usually add
# the NUMAID), see https://github.com/AliceO2Group/O2DPG/pull/993#pullrequestreview-1393401475
export O2JOBID=$(((ALIEN_PROC_ID_OFFSET_INT32 & 0x7ffffff) * 16))
export O2JOBSHMID=$(((ALIEN_PROC_ID_OFFSET_INT16 & 0x7ff) * 16))
echo "ALIEN_PROC_ID = $ALIEN_PROC_ID, we will set O2JOBID = $O2JOBID, SHMEMID = $O2JOBSHMID"

Running it with the job id I used in #1526, I get:

zampolli@pcalicecz5:~/work/O2/test_setVarsFromALIEN_PROC_ID$ source test.sh 3027145464
ALIEN_PROC_ID for current job = 3027145464
ALIEN_PROC_ID_OFFSET_INT32 = 27145464
ALIEN_PROC_ID_OFFSET_INT16 = 27145464
ALIEN_PROC_ID = 3027145464, we will set O2JOBID = 434327424, SHMEMID = 20352

With your change i get something different:

zampolli@pcalicecz5:~/work/O2/test_setVarsFromALIEN_PROC_ID$ source test.sh 3027145464
ALIEN_PROC_ID for current job = 3027145464
ALIEN_PROC_ID_OFFSET_INT32 = 3027145464
ALIEN_PROC_ID_OFFSET_INT16 = 3027145464
ALIEN_PROC_ID = 3027145464, we will set O2JOBID = 1189687168, SHMEMID = 12160

[shahor02]

Hi @chiarazampolli

With spaces I get:

export ALIEN_PROC_ID=1234
WORKFLOWMODE=print $O2DPG_ROOT/DATA/production/configurations/CTFSkimming/ctf-skim-workflow.sh -f fff -c list.list
ALIEN_PROC_ID for current job = 1234
/home/shahoian/alice/sw/ubuntu2204_x86-64/O2DPG/master-local1/DATA/production/common/setVarsFromALIEN_PROC_ID.sh: line 16: 10#: invalid integer constant (error token is "10#")
ALIEN_PROC_ID_OFFSET_INT32 = 
/home/shahoian/alice/sw/ubuntu2204_x86-64/O2DPG/master-local1/DATA/production/common/setVarsFromALIEN_PROC_ID.sh: line 19: 10#: invalid integer constant (error token is "10#")
ALIEN_PROC_ID_OFFSET_INT16 = 
ALIEN_PROC_ID = 1234, we will set O2JOBID = 0, SHMEMID = 0

[chiarazampolli]

Hello @shahor02 ,
Your ALIEN_PROC_ID is too short, with that we cannot extract the last 9 digits, since you have only 4. I think this is the issue. On the grid, we don't have such numbers. And we explicitly took the last 9 to insure unique IDs.
Chiara

[shahor02]

Hi @chiarazampolli,

I think the ${XX: -$YY} (with space) is not defined. e.g.

shahoian@alicers05a:~$ XX=123
shahoian@alicers05a:~$ echo '|${YY: -${XX}}|'" -> |${YY: -${XX}}| ,"'|${YY:-${XX}}|'" -> |${YY:-${XX}}| ,"'|${YY:--${XX}}|'" -> |${YY:--${XX}}|"
|${YY: -${XX}}| -> || ,|${YY:-${XX}}| -> |123| ,|${YY:--${XX}}| -> |-123|

If you want to assign the negative value, just use --.

[chiarazampolli]

For me it is defined, see the output of my test script above:

zampolli@pcalicecz5:~/work/O2/test_setVarsFromALIEN_PROC_ID$ source test.sh 3027145464
ALIEN_PROC_ID for current job = 3027145464
ALIEN_PROC_ID_OFFSET_INT32 = 27145464
ALIEN_PROC_ID_OFFSET_INT16 = 27145464
ALIEN_PROC_ID = 3027145464, we will set O2JOBID = 434327424, SHMEMID = 20352

[shahor02]

Ah, wait, so you were not assigning negative value but taking the 1st N characters, I think we've already discussed that but then I forgot it, sorry.
Will restore your spaces, but I don't like that one cannot run the skimming script w/o defining a fake ALIEN_PROC_ID, will add default

[shahor02]

#1634

[chiarazampolli]

I need to take the last N digits, yes.