forked from desihub/LSS
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
110 changed files
with
12,440 additions
and
1,020 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| #pseudo pipeline describing how BGS LSS catalogs get created from 2nd gen mocks, very similar to LSSpipe_Y1mock_2ndgen.txt | ||
| #Set up | ||
| >> source /global/common/software/desi/desi_environment.sh main | ||
| In this documentation, $LSS is the directory where you have the LSS repository | ||
|
|
||
| #1), cutsky mocks get created from boxes and have the Y1 tiles list applied to geometry (Alex Smith) | ||
|
|
||
| #2) These are adapted to run LSS and AltMTL pipelines, giving the definition of BGS subtype and have columns added necessary for fiberassign, and have the targeting mask applied via | ||
| https://github.com/desihub/LSS/blob/main/scripts/mock_tools/prepare_mocks_Y1_bright.py | ||
|
|
||
| >> ./prepare_script_bright.sh | ||
|
|
||
| #3) CREATE FIBER COLLISION FILE AND CALCULATE POTA FILES | ||
| https://github.com/desihub/LSS/blob/main/scripts/getpotaY1_mock.py | ||
|
|
||
| >> ./getpota_Y1_bright_script.sh | ||
|
|
||
| ### AltMTL + LSS pipeline INSTRUCTIONS ### | ||
|
|
||
| 4) CREATE INITIAL LEDGERS | ||
| >> ./run_Y1SecondGen_initialledger_batch.sh (adapt to define the mock path and program dark/bright) | ||
|
|
||
|
|
||
| *) GOTO LSS/bin | ||
|
|
||
| 5) INITIALIZE ALTMTL DIRECTORY STRUCTURE FOR 1 MOCK | ||
| >> ./Y1ALTMTLRealizationsBRIGHT_mock_init.sh | ||
|
|
||
| 6) MAKE PARENT DIRECTORIES FOR OTHER REALIZATIONS | ||
| >> python initAMTL_bright.py | ||
|
|
||
| 8) MANUALLY ADD EXTRA TILES TO THE END OF mainsurvey-DARKobscon-TileTracker.ecsv | ||
| >> python /pscratch/sd/a/acarnero/codes/LSS/scripts/mock_tools/add_extra_tilesTracker.py | ||
|
|
||
| 9) RUN ALTMTL PROCESS ON MOCK REALIZATIONS | ||
| This process runs on regular queue that takes 12 hours but to complete the operation we need more time. Therefore, it needs to be run at least twice | ||
| >> nohup ./Y1ALTMTLRealizationsBRIGHT_mock.sh & | ||
|
|
||
| ** If you want to leave the second process in the queue, once you start running the first, you can put a second process in the queue stating to start after this first process ends, with the option --dependency=afterany:<JOBID> in dateLoopAltMTLBugFix_mock_batch.sh | ||
| <JOBID> can be found with squeue -u <USER> | ||
|
|
||
| 10) RUN A SECOND TIME | ||
| >> nohup ./Y1ALTMTLRealizationsBRIGHT_mock.sh & | ||
|
|
||
| 11) Combd calling run1_AMTLmock_combd_LSS.sh from abamtl_combd_cat_sbatch.sh (Change path accordingly) | ||
|
|
||
| 12) run1_AMTLmock_LSS_BGS.sh with abBGSamtl_cat_sbatch.sh | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
| # Now you have several pathways. Make clustering catalogs only from potential assigments (applying all the data masking) or mimic the entire data processing, staring from AltMTL to LSS pipeline. | ||
|
|
||
| ### POTENTIAL ASSIGNMENT ONLY INSTRUCTIONS ### | ||
|
|
||
| PYTHONPATH=$PYTHONPATH:$HOME/LSS/py | ||
|
|
||
| #Make complete LSS catalogs with goodhardware and imaging veto masks applied | ||
| #Code does randoms in serial, 4 are likely enough | ||
| >> python scripts/mock_tools/pota2clus_simp.py --veto _gtlimaging --realization 0 --maxr <max_num_ran_needed> | ||
|
|
||
| ----------------------------------------------- | ||
|
|
||
|
|
||
| ### AltMTL + LSS pipeline INSTRUCTIONS ### | ||
| ## CAUTION: NO PIP WEIGHTS YET TESTED OR IMPLEMENTED ## | ||
|
|
||
| #The output directory names are just suggestions. | ||
| #Generate initial ledgers | ||
| # First define an output directory <base_output> (like in your scratch or if in production for obscon=dark, this is $DESI_ROOT/survey/catalogs/Y1/mocks/SecondGenMocks/AbacusSummit). | ||
|
|
||
| # Create initial ledgers and save hplist to feed MockAltMTL production, both saved in <base_output>/initial_ledger<mock#> (~20 minutes) | ||
| >> python $LSS/scripts/mock_tools/run_mtl_ledger.py $DESI_ROOT/survey/catalogs/Y1/mocks/SecondGenMocks/AbacusSummit/forFA<mock#>.fits <base_output>/altmtl<mock#>/initled/ DARK | ||
|
|
||
| First need to create (mkdir) the directory <base_output>/altmtl<mock#>/initled | ||
| It will save the HP list in the same directory with the name hpxlist_<obscon>.txt | ||
|
|
||
| # Run AltMTL process (without generating bitweights yet) for 1 realization with $LSS/scripts/mock_tools/MockAltMTLScriptMain.sh (~24 hours) | ||
| # Modify by hand several options inside MockAltMTLScriptMain.sh | ||
| # This are the ones that you should worry (up to line 50. there are other options but you can ignore) | ||
| * mockNumber=<mock#> If running over production directories, set the realization here and can skip simName and targfile configuration. | ||
| * simName="<base_output>/altmtl<mock#>/initled/" This is the directory where to save all the AltMTL products for a given mock# | ||
| * path2LSS=/pscratch/sd/a/MYUSER/codes/LSS/bin/ This is the path to the LSS repo bin subdirectory | ||
| * ndir=1 Number of Alternative ledgers. In principle, we should run a single realization to get as data, and then another large run with ndir=128 or 256 to calculate PIP weights. If ndir = 1, then BitWeights are not calculated. | ||
| * obscon='DARK' Either DARK or BRIGHT | ||
| * endDate='--endDate=20220613' End date for Y1 campaign | ||
| * exampleLedgerBase=<base_output>/initial_ledger<mock#> This is the directory you indicated when creating the initial ledgers | ||
| * hpListFile="<base_output>/initial_ledger<mock#>/hpxlist_dark.txt" This is the hp file list created while running the initial ledgers | ||
| * targfile='--targfile=/global/cfs/cdirs/desi/survey/catalogs/Y1/mocks/SecondGenMocks/AbacusSummit/forFA<mock#>.fits' This is the path to the parent target sample | ||
|
|
||
| # This script takes quite some time to run (~24h). It generates ndir random realizations (given subpriority) and run fiber assigment creating potential assignments and assignments itself. | ||
| # Their outputs will serve as input for the LSS pipeline. | ||
|
|
||
|
|
||
| # Run LSS pipeline as follows. | ||
| # CAUTION: it has only been tested to run one mock by one mock with mockmin = 0 and mockmax = 1 | ||
| # Prepare and combine data for dark or bright time data. Better run in batch mode since it might die in between if not running in batch mode. | ||
| # We don't need randoms at this stage any more. They will be read directly from data randoms when creating the full catalogs | ||
| >> srun -N 1 -C cpu -t 04:00:00 --qos interactive --account desi python $LSS/scripts/mock_tools/mkCat_SecondGen.py --base_output <base_output> --simName <base_output>/altmtl_main_rea{MOCKNUM} --mockver ab_secondgen --mockmin 0 --mockmax 1 --survey Y1 --add_gtl y --specdata iron --tracer dark --combd y | ||
|
|
||
| [40 minutes] | ||
|
|
||
| Example for mock = 1: | ||
| srun -N 1 -C cpu -t 04:00:00 --qos interactive --account desi python mkCat_SecondGen.py --base_output /global/cfs/cdirs/desi/survey/catalogs/Y1/mocks/SecondGenMocks/AbacusSummit/altmtl1 --simName /global/cfs/cdirs/desi/survey/catalogs/Y1/mocks/SecondGenMocks/AbacusSummit/altmtl1 --mockver ab_secondgen --mockmin 1 --mockmax 2 --survey Y1 --add_gtl y --specdata iron --tracer dark --combd y | ||
|
|
||
|
|
||
|
|
||
| # Create the full catalogs for data and randoms and apply vetos. Calcualte the FKP weights and make the clustering catalogs. | ||
| #LRG | ||
| >> srun -N 1 -C cpu -t 02:00:00 --qos interactive --account desi python mkCat_SecondGen.py --base_output /global/cfs/cdirs/desi/survey/catalogs/Y1/mocks/SecondGenMocks/AbacusSummit/altmtl{MOCKNUM} --mockver ab_secondgen --mockmin 0 --mockmax 1 --survey Y1 --add_gtl y --specdata iron --tracer LRG --minr 0 --maxr 18 --fulld y --fullr y --apply_map_veto y --apply_veto y --use_map_veto _HPmapcut --mkclusran y --resamp y --nz y --getFKP y --mkclusdat y --targDir /global/cfs/cdirs/desi/survey/catalogs/Y1/mocks/SecondGenMocks/AbacusSummit | ||
|
|
||
| #ELG (ELG_LOP with notqso) | ||
| >> srun -N 1 -C cpu -t 02:00:00 --qos interactive --account desi python mkCat_SecondGen.py --base_output /global/cfs/cdirs/desi/survey/catalogs/Y1/mocks/SecondGenMocks/AbacusSummit/altmtl{MOCKNUM} --mockver ab_secondgen --mockmin 0 --mockmax 1 --survey Y1 --add_gtl y --specdata iron --tracer ELG_LOP --notqso y --minr 0 --maxr 18 --fulld y --fullr y --apply_map_veto y --apply_veto y --use_map_veto _HPmapcut --mkclusran y --resamp y --nz y --getFKP y --mkclusdat y --targDir /global/cfs/cdirs/desi/survey/catalogs/Y1/mocks/SecondGenMocks/AbacusSummit | ||
|
|
||
| #QSO | ||
| >> srun -N 1 -C cpu -t 02:00:00 --qos interactive --account desi python mkCat_SecondGen.py --base_output /global/cfs/cdirs/desi/survey/catalogs/Y1/mocks/SecondGenMocks/AbacusSummit/altmtl{MOCKNUM} --mockver ab_secondgen --mockmin 0 --mockmax 1 --survey Y1 --add_gtl y --specdata iron --tracer QSO --minr 0 --maxr 18 --fulld y --fullr y --apply_map_veto y --apply_veto y --use_map_veto _HPmapcut --mkclusran y --resamp y --nz y --getFKP y --mkclusdat y --targDir /global/cfs/cdirs/desi/survey/catalogs/Y1/mocks/SecondGenMocks/AbacusSummit |
Oops, something went wrong.