2018UL SC/Electron/Photon V1

This is the release that was used to train the 2018UL regressions for superclusters, photons and electrons. Setup remains the same as the 2017UL release

export SCRAM_ARCH=slc6_amd64_gcc700
cmsrel CMSSW_10_4_0
cd CMSSW_10_4_0/src
cmsenv
cd - 
git clone [email protected]:cms-egamma/EgRegresTrainerLegacy.git
cd EgRegresTrainerLegacy 
gmake RegressionTrainerExe -j 8
gmake RegressionApplierExe -j 8
export PATH=$PATH:./bin/$SCRAM_ARCH #add the binary location to path
export PYTHON27PATH=$PYTHON27PATH:python
mkdir resultsEleV5
mkdir resultsPhoV5
mkdir resultsSCV5

The input files were the V5 regression trees as described at: https://github.com/cms-egamma/SHarper-UserCode/releases/tag/2018UL_SCElePho_V5a

There is no relevant change w.r.t to the 2017 trees except that 2018 samples were used.

The eos location of the input trees are
/eos/cms/store/group/phys_egamma/ReleaseInputsArchive/2018UL_SCReg/input_trees/ and /eos/cms/store/group/phys_egamma/ReleaseInputsArchive/2018UL_ElePhoReg/input_trees/

The outputs of the training are at
/eos/cms/store/group/phys_egamma/ReleaseInputsArchive/2018UL_SCReg/results and
/eos/cms/store/group/phys_egamma/ReleaseInputsArchive/2018UL_ElePhoReg/results

The outputted GBRForestDs in root files were converted and uploaded to the database with EgammaDBTools (release 2018UL_V1).

The scripts to run the regressions are

• scripts/runSCRegTrainings.py for mustache superclusters
• scripts/runEleRegTrainings.py for electrons
• scripts/runPhoRegTrainings.py for photons

To run the regressions, command line options need to be specificed these are:

• --era : year to produce for , valid options are 2016, 2017, 2018
• --input_dir (-i): input directory containing the trees
• --output_dir (-o): directory to output the results

Example commands:

scripts/runSCRegTrainings.py --era 2018 -i /eos/cms/store/group/phys_egamma/ReleaseInputsArchive/2018UL_SCReg/input_trees/ -o resultsSCV5
scripts/runEleRegTrainings.py --era 2018 -i /eos/cms/store/group/phys_egamma/ReleaseInputsArchive/2018UL_ElePhoReg/input_trees/ -o resultsEleV5
scripts/runPhoRegTrainings.py --era 2018 -i /eos/cms/store/group/phys_egamma/ReleaseInputsArchive/2018UL_ElePhoReg/input_trees/ -o resultsPhoV5

Also it is important to note that the SCRAM_ARCH got hardcoded in the python scripts, so if you are running with a different SCRAM_ARCH to the one mentioned above, you will need to change it manually. This will be fixed in a later update. The error will look a lot like

Traceback (most recent call last):
  File "scripts/runEleRegTrainings.py", line 98, in <module>
    main()
  File "scripts/runEleRegTrainings.py", line 40, in main
    if run_step1: regArgs.run_eb_and_ee()
  File "/mercury/data1/harper/test/reg/EgRegresTrainerLegacy/python/regtools.py", line 128, in run_eb_and_ee
    subprocess.Popen(["bin/slc6_amd64_gcc700/RegressionTrainerExe",self.cfg_name()]).communicate()
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-omkpbe4/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-omkpbe4/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Electron regression

The electron regression runs in 4 steps

1. trains the regression on Ideal IC inputs.
2. it then makes a new tree with the mean of the training saved as a new branch regIdealMean. This can segfault in older versions of root so make sure you have CMSSW_10_4_0 or higher setup. Future versions will find a better way to do this
3. applies regIdealMean to the energy estimate, fixes the mean to 1 and then trains for the sigma on the real intercalibration sample. At the same time, it makes a new tree with this saved as a new branch regRealSigma.
4. runs the e/p training using the ideal mean and real sigma. This used to have two extra sub steps which ran them in separate bins (low and high pt) but this was found not to be necessary.

Photon regression

This works exactly the same as the electrons except that of course it does not do step 4 , the track combination. Otherwise the only difference is that it is trained on photons.

Uploading the conditions

The conditions can be uploaded to the database with EgammaDBTools, specifically release 2018UL_V1

2017UL Electron/Photon V2

This is the release that was used to train the 2017ULV2 regressions for electrons and photons. The V1 regressions while uploaded to the database were never used as a bug was found in them (iPhiMod20 if you're interested).

export SCRAM_ARCH=slc6_amd64_gcc700
cmsrel CMSSW_10_4_0
cd CMSSW_10_4_0/src
cmsenv
cd - 
git clone [email protected]:cms-egamma/EgRegresTrainerLegacy.git
cd EgRegresTrainerLegacy 
gmake RegressionTrainerExe -j 8
gmake RegressionApplierExe -j 8
export PATH=$PATH:./bin/$SCRAM_ARCH #add the binary location to path
export PYTHON27PATH=$PYTHON27PATH:python
mkdir resultsEleV5
mkdir resultsPhoV5

The input files were the V5 regression trees as described at: https://github.com/cms-egamma/SHarper-UserCode/releases/tag/2017UL_ElePho_V5

The eos location of the input trees are
/eos/cms/store/group/phys_egamma/ReleaseInputsArchive/2017UL_ElePhoReg/input_trees/

The outputs of the training are at
/eos/cms/store/group/phys_egamma/ReleaseInputsArchive/2017UL_ElePhoReg/results

To run the electron regression, do
python scripts/runEleRegTrainings.py for electrons and python scripts/runPhoRegTrainings.py for photons.

This release also did the real IC resolution training for the supercluster regression with scripts/runSCRegJobRes.py. Note that it purposely reads in the existing supercluster regression trained in the previous release https://github.com/cms-egamma/EgRegresTrainerLegacy/releases/tag/2017UK_SC_V1

However you will need to edit those files to replace /mercury/data1/harper/EgRegsNtups/ with /eos/cms/store/group/phys_egamma/ReleaseInputsArchive/2017UL_ElePhoReg/input_trees/ as the training was done at RAL and I wanted to upload them as is.

Also it is important to note that the SCRAM_ARCH got hardcoded in the python scripts, so if you are running with a different SCRAM_ARCH to the one mentioned above, you will need to change it manually. This will be fixed in a later update. The error will look a lot like

Traceback (most recent call last):
  File "scripts/runEleRegTrainings.py", line 98, in <module>
    main()
  File "scripts/runEleRegTrainings.py", line 40, in main
    if run_step1: regArgs.run_eb_and_ee()
  File "/mercury/data1/harper/test/reg/EgRegresTrainerLegacy/python/regtools.py", line 128, in run_eb_and_ee
    subprocess.Popen(["bin/slc6_amd64_gcc700/RegressionTrainerExe",self.cfg_name()]).communicate()
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-omkpbe4/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc700/external/python/2.7.14-omkpbe4/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Electron regression

The electron regression runs in 4 steps

1. trains the regression on Ideal IC inputs.
2. it then makes a new tree with the mean of the training saved as a new branch regIdealMean. This can segfault in older versions of root so make sure you have CMSSW_10_4_0 or higher setup. Future versions will find a better way to do this
3. applies regIdealMean to the energy estimate, fixes the mean to 1 and then trains for the sigma on the real intercalibration sample. At the same time, it makes a new tree with this saved as a new branch regRealSigma.
4. runs the e/p training using the ideal mean and real sigma. This used to have two extra sub steps which ran them in separate bins (low and high pt) but this was found not to be necessary.

Photon regression

This works exactly the same as the electrons except that of course it does not do step 4 , the track combination. Otherwise the only difference is that it is trained on photons.

2017UL SC V1

This is the version used to train the 2017 ultra legacy supercluster regression for the CMSSW release. It was trained by running

cmsrel CMSSW_10_1_0
cd CMSSW_10_1_0/src
cmsenv
cd - 
git clone [email protected]:cms-egamma/EgRegresTrainerLegacy.git
cd EgRegresTrainerLegacy 
gmake RegressionTrainerExe -j 8
gmake RegressionApplierExe -j 8
export PATH=$PATH:./bin/$SCRAM_ARCH #add the binary location to path
scripts/runSCRegJob.py 

The data of the regression was 29-04-19. As it was run at RAL, the runSCRegJob, the path /mercury/data1/harper/EgRegsNtups/ needs to be changed to /eos/cms/store/group/phys_egamma/ReleaseInputsArchive/2017UL_SCReg/input_trees/

Note that there is a random seed here so re-running will not give identical trainings as before, just highly similar.