Merge pull request #39 from eic/38-update-production-related-faqs

Updating the questions and answers relevant to production WG

_documentation/faq.md

@@ -13,18 +13,50 @@ Please use the [Helpdesk](https://eic.cloud.mattermost.com/main/channels/helpdes
 Submit new questions and comments by email: [![emailicon](../assets/images/site/icons/email.png){: width="3%" }](mailto:[email protected])
 
 ------------------------
+**Q: How can we browse the simulation output from a specific campaign and locate certain output files?**
 
-**Q: How can we access simulation files on S3 storage?**
+A: Visit the [epicprod website](https://eic.github.io/epic-prod/campaigns/campaigns_reco.html) to view the list of avaiable campaigns. Pick a campaign that you want to view in detail. For example: [23.12.0](https://eic.github.io/epic-prod/RECO/23.12.0/). Here, you will find the output directories listed in a nested tree structure. The directory nomenclature usually follows the pattern:
 
-A: See [https://doc.athena-eic.org/en/latest/howto/s3_file_storage.html](https://doc.athena-eic.org/en/latest/howto/s3_file_storage.html)
+```<base directory>/<campaign tag>/<detector config>/<physics processes>/<generator release tag if available>/<electron momentum>x<proton momentum>/<q2 range>/```
 
-**Q: How can we locate specific files in S3 and better understand their directory structure?**
+The preferred method to list the files in a directory is to use the xrdfs interface within eic-shell container. For example:
+```
+    xrdfs root://dtn-eic.jlab.org   
+    ls /work/eic2/EPIC/RECO/23.09.1/epic_craterlake/DIS/NC/18x275/minQ2=1000
+```
+See more details [here](https://eic.github.io/epic-prod/documentation/faq.html).
+
+**Q: What input datasets are used in a particular production campaign and where can I find them?**
+
+A: The directory structure of the input datasets mimic the directory structure of the output files from `<physics processes>` onwards. Consider, the output files under `root://dtn-eic.jlab.org//work/eic2/EPIC/RECO/23.12.0/epic_craterlake/DIS/NC/10x100/minQ2=1`. The corresponding input datasets can be found in the following manner:
+
+```
+xrdfs root://dtn-eic.jlab.org
+ls /work/eic2/EPIC/EVGEN/DIS/NC/10x100/minQ2=1
+```
+
+**Q: What is the procedure to introduce a new dataset into a production campaign or replace an existing one?**
+
+A: Follow the input dataset creation [guidelines](https://eic.github.io/epic-prod/documentation/input_preprocessing.html) listed in the epic-prod website.
+
+**Q: How can we load files into ROOT directly from xrootd?**
 
-A: **TODO:** Ask Prod for 1-2 lines on the directory structure; tutorials cover how to navigate (one can use a web interface as well)
+A: Enter the full file path including server address using root TFile::Open. For example:
+```
+    auto f = TFile::Open("root://dtn-eic.jlab.org//work/eic2/EPIC/RECO/23.12.0/epic_craterlake/DIS/NC/18x275/minQ2=1000/pythia8NCDIS_18x275_minQ2=1000_beamEffects_xAngle=-0.025_hiDiv_1.0000.eicrecon.tree.edm4eic.root")
+```
+See more details [here](https://eic.github.io/epic-prod/documentation/faq.html).
 
-**Q: How can we determine the properties of simulated files, such as parameters, particles, and energies?**
+**Q: How can we copy files from xrootd?**
 
-A: **TODO:** Production knows best, but more or less by the file name.
+A: Use xrdcp with full path and local destination directory. For example:
+```
+xrdcp root://dtn-eic.jlab.org//work/eic2/EPIC/RECO/23.12.1/epic_craterlake/DIS/NC/18x275/minQ2=1000/pythia8NCDIS_18x275_minQ2=1000_beamEffects_xAngle=-0.025_hiDiv_1.0000.eicrecon.tree.edm4eic.root <local destination>
+```
+
+**Q: What is included in the campaign, and what are the major changes compared to before?**
+
+A: We version track the [epic](https://github.com/eic/epic/releases) and [eicrecon](https://github.com/eic/eicrecon/releases) repositories and the new features introduced in a campaign are tracked in the release notes. For example: [epic-23.12.0](https://github.com/eic/epic/releases/tag/23.12.0) and [eicrecon-1.9.0](https://github.com/eic/eicrecon/releases/tag/1.9.0) represent the software changes introduced in December campaign. As of December, 2023 we have also began enforcing version tracking on our input datasets. 
 
 **Q: How can we effectively use the information stored in EICRecon rootfile outputs, such as matching reconstructed particles to their MC particles?**
 
@@ -38,26 +70,6 @@ A: Beam effects are handled by the [afterburner](https://github.com/eic/afterbur
 
 A: **TODO:**Changing magnet configuration in reconstruction makes little sense. Checking and changing it for simulation should be answered by the prod group.
 
-**Q: What are the commands for browsing files on S3 and copying them to local directories?**
-
-**Q: How can we load files into ROOT directly from S3?**
-
-A: Recommended way currently is using the XrootD mirror because it doesn't need a password. See:
-```
-    auto f = TFile::Open("root://dtn-eic.jlab.org//work/eic2/EPIC/RECO/23.09.1/epic_craterlake/DIS/NC/18x275/minQ2=1000/pythia8NCDIS_18x275_minQ2=1000_beamEffects_xAngle=-0.025_hiDiv_1.0000.eicrecon.tree.edm4eic.root")
-```
-or, for browsing:
-```
-    xrdfs root://dtn-eic.jlab.org   
-    ls /work/eic2/EPIC/RECO/23.09.1/epic_craterlake/DIS/NC/18x275/minQ2=1000
-```
-To use S3, you need the minimal step:
-```
-    export S3_ACCESS_KEY=[ask in mattermost]
-    export S3_SECRET_KEY=[ask in mattermost]
-    mc config host add S3 https://eics3.sdcc.bnl.gov:9000 $S3_ACCESS_KEY $S3_SECRET_KEY
-    auto f = TFile::Open("s3https://eics3.sdcc.bnl.gov:9000/eictest/EPIC/RECO/23.09.1/epic_craterlake/DIS/NC/18x275/minQ2=1000/pythia8NCDIS_18x275_minQ2=1000_beamEffects_xAngle=-0.025_hiDiv_1.0000.eicrecon.tree.edm4eic.root")
-```
 
 **Q: What is the best practice for analyzing EICrecon output, especially regarding hadron PID likelihoods?**
 
@@ -67,8 +79,6 @@ A: **TODO:** Needs an updated analysis tutorial. I don't believe at this point w
 
 A: **TODO:** We originally planned a dedicated "Tutorial: Writing physics benchmarks that run automatically and reproducibly" - this needs to be revisited. Otherwise, Dmitry K, Wouter, Sylvester may be able to write up something short. This is a good FAQ candidate really, if we can get this answered.
 
-**Q: What is included in the campaign, and what are the major changes compared to before?**
-
 **Q: How can we access various (SI)DIS variables through different reconstruction methods?**
 
 A: **TODO:** Things like ``InclusiveKinematicsDA.cc`` are part of EicRecon, but I don't know how to use it. I don't believe it's calculated by default during campaigns, so one would still need an analysis level version. Would be good example analysis code, and I'm sure it exists in many forms on various people's laptops.