Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AnalysisQC] Change configuration logic #1590

Merged
merged 1 commit into from
Apr 10, 2024

Conversation

benedikt-voelkel
Copy link
Contributor

@benedikt-voelkel benedikt-voelkel commented Apr 10, 2024

  • disable EMCAL analysis
    (to enable, need to revise resource needs, crashes or hangs
    sporadically)

  • split config JSONs into executable-specific single JSONs Most of those are seen as configurations for "service wagons". Each analyses is forced to use those comon configurations and it is not possible to have analysis-specific configurations for service wagons.

  • possible to use special configuration strings in configs; this is done to avoid duplications of configs

    • "!ANALYSIS_QC_is_mc!" ("!ANALYSIS_QC_is_data!") will be replaced with "true"("false") or "false"("true") depending on whether the analyses are run on MC(data)
  • new default is to pipe all analyses together; to split into single analyses, run o2dpg_analysis_testing_workflow.py [...] --split-analyses

Copy link

REQUEST FOR PRODUCTION RELEASES:
To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available
async-2022-pp-apass4
async-2023-pbpb-apass
async-2023-pp-apass1
async-data
async-mc
async-2022-pp-apass6-2023-PbPb-apass2

@benedikt-voelkel
Copy link
Contributor Author

benedikt-voelkel commented Apr 10, 2024

@chiarazampolli @noferini @catalinristea
The default is now to merge all analyses together, see also the commit message.

At this point, there is no difference in configurations for pp and PbPb, it's all the same.
Tested on

  • pp MC
  • PbPb MC
  • pp data (/alice/data/2023/LHC23zs/539580/apass4_test_dev_0329/0340/o2_ctf_run00539580_orbit0275138592_tf0001956972_epn108/003/AO2D.root)

Note that the merged analyses pipe takes around 1.5 times as long as running all single analyses in parallel.

* remove unused analyses

* disable EMCAL analysis
  (to enable, need to revise resource needs, crashes or hangs
   sporadically)

* split config JSONs into executable-specific single JSONs
  Most of those are seen as configurations for "service wagons".
  Each analyses is forced to use those comon configurations and it is
  not possible to have analysis-specific configurations for service
  wagons.

* possible to use special configuration strings in configs; this is done
  to avoid duplications of configs
  * "!ANALYSIS_QC_is_mc!" ("!ANALYSIS_QC_is_data!") will be replaced
    with "true"("false") or "false"("true") depending on whether the
    analyses are run on MC(data)

* new default is to pipe all analyses together;
  to split into single analyses, run
  o2dpg_analysis_testing_workflow.py [...] --split-analyses
@benedikt-voelkel benedikt-voelkel merged commit 1df34c0 into AliceO2Group:master Apr 10, 2024
6 checks passed
@benedikt-voelkel benedikt-voelkel deleted the analysis-qc branch April 10, 2024 15:21
@benedikt-voelkel benedikt-voelkel added async-2022-pp-apass4 async-2023-pbpb-apass4 Request porting to async-2023-pbpb-apass4 labels Apr 17, 2024
benedikt-voelkel added a commit that referenced this pull request Apr 26, 2024
* remove unused analyses

* disable EMCAL analysis
  (to enable, need to revise resource needs, crashes or hangs
   sporadically)

* split config JSONs into executable-specific single JSONs
  Most of those are seen as configurations for "service wagons".
  Each analyses is forced to use those comon configurations and it is
  not possible to have analysis-specific configurations for service
  wagons.

* possible to use special configuration strings in configs; this is done
  to avoid duplications of configs
  * "!ANALYSIS_QC_is_mc!" ("!ANALYSIS_QC_is_data!") will be replaced
    with "true"("false") or "false"("true") depending on whether the
    analyses are run on MC(data)

* new default is to pipe all analyses together;
  to split into single analyses, run
  o2dpg_analysis_testing_workflow.py [...] --split-analyses

Co-authored-by: Benedikt Volkel <[email protected]>
benedikt-voelkel added a commit that referenced this pull request Apr 26, 2024
* remove unused analyses

* disable EMCAL analysis
  (to enable, need to revise resource needs, crashes or hangs
   sporadically)

* split config JSONs into executable-specific single JSONs
  Most of those are seen as configurations for "service wagons".
  Each analyses is forced to use those comon configurations and it is
  not possible to have analysis-specific configurations for service
  wagons.

* possible to use special configuration strings in configs; this is done
  to avoid duplications of configs
  * "!ANALYSIS_QC_is_mc!" ("!ANALYSIS_QC_is_data!") will be replaced
    with "true"("false") or "false"("true") depending on whether the
    analyses are run on MC(data)

* new default is to pipe all analyses together;
  to split into single analyses, run
  o2dpg_analysis_testing_workflow.py [...] --split-analyses

Co-authored-by: Benedikt Volkel <[email protected]>
@benedikt-voelkel benedikt-voelkel removed the async-2023-pbpb-apass4 Request porting to async-2023-pbpb-apass4 label May 7, 2024
benedikt-voelkel added a commit to benedikt-voelkel/O2DPG that referenced this pull request May 14, 2024
* remove unused analyses

* disable EMCAL analysis
  (to enable, need to revise resource needs, crashes or hangs
   sporadically)

* split config JSONs into executable-specific single JSONs
  Most of those are seen as configurations for "service wagons".
  Each analyses is forced to use those comon configurations and it is
  not possible to have analysis-specific configurations for service
  wagons.

* possible to use special configuration strings in configs; this is done
  to avoid duplications of configs
  * "!ANALYSIS_QC_is_mc!" ("!ANALYSIS_QC_is_data!") will be replaced
    with "true"("false") or "false"("true") depending on whether the
    analyses are run on MC(data)

* new default is to pipe all analyses together;
  to split into single analyses, run
  o2dpg_analysis_testing_workflow.py [...] --split-analyses

Co-authored-by: Benedikt Volkel <[email protected]>
benedikt-voelkel added a commit that referenced this pull request May 14, 2024
* remove unused analyses

* disable EMCAL analysis
  (to enable, need to revise resource needs, crashes or hangs
   sporadically)

* split config JSONs into executable-specific single JSONs
  Most of those are seen as configurations for "service wagons".
  Each analyses is forced to use those comon configurations and it is
  not possible to have analysis-specific configurations for service
  wagons.

* possible to use special configuration strings in configs; this is done
  to avoid duplications of configs
  * "!ANALYSIS_QC_is_mc!" ("!ANALYSIS_QC_is_data!") will be replaced
    with "true"("false") or "false"("true") depending on whether the
    analyses are run on MC(data)

* new default is to pipe all analyses together;
  to split into single analyses, run
  o2dpg_analysis_testing_workflow.py [...] --split-analyses

Co-authored-by: Benedikt Volkel <[email protected]>
(cherry picked from commit 1df34c0)
benedikt-voelkel added a commit that referenced this pull request May 16, 2024
* remove unused analyses

* disable EMCAL analysis
  (to enable, need to revise resource needs, crashes or hangs
   sporadically)

* split config JSONs into executable-specific single JSONs
  Most of those are seen as configurations for "service wagons".
  Each analyses is forced to use those comon configurations and it is
  not possible to have analysis-specific configurations for service
  wagons.

* possible to use special configuration strings in configs; this is done
  to avoid duplications of configs
  * "!ANALYSIS_QC_is_mc!" ("!ANALYSIS_QC_is_data!") will be replaced
    with "true"("false") or "false"("true") depending on whether the
    analyses are run on MC(data)

* new default is to pipe all analyses together;
  to split into single analyses, run
  o2dpg_analysis_testing_workflow.py [...] --split-analyses

Co-authored-by: Benedikt Volkel <[email protected]>
(cherry picked from commit 1df34c0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant