diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 0000000..0f8f8d9 --- /dev/null +++ b/.travis.yml @@ -0,0 +1,73 @@ +language: c +matrix: + include: + - os: linux + addons: + apt: + sources: + - ubuntu-toolchain-r-test + packages: + - g++-4.9 + env: + - MATRIX_EVAL="CC=gcc-4.9 && CXX=g++-4.9" + + - os: linux + addons: + apt: + sources: + - ubuntu-toolchain-r-test + packages: + - g++-5 + env: + - MATRIX_EVAL="CC=gcc-5 && CXX=g++-5" + + - os: linux + addons: + apt: + sources: + - ubuntu-toolchain-r-test + packages: + - g++-6 + env: + - MATRIX_EVAL="CC=gcc-6 && CXX=g++-6" + + - os: linux + addons: + apt: + sources: + - ubuntu-toolchain-r-test + packages: + - g++-7 + env: + - MATRIX_EVAL="CC=gcc-7 && CXX=g++-7" + + - os: linux + addons: + apt: + sources: + - ubuntu-toolchain-r-test + packages: + - g++-8 + env: + - MATRIX_EVAL="CC=gcc-8 && CXX=g++-8" + + - os: linux + addons: + apt: + sources: + - ubuntu-toolchain-r-test + packages: + - g++-9 + env: + - MATRIX_EVAL="CC=gcc-9 && CXX=g++-9" + +before_install: + - eval "${MATRIX_EVAL}" + +script: + - echo $CC + - $CC --version + - cd openmp-threading + - make + - ./rsbench -s small -t 4 -m history + - ./rsbench -s small -t 4 -m event diff --git a/README.md b/README.md new file mode 100644 index 0000000..be30937 --- /dev/null +++ b/README.md @@ -0,0 +1,202 @@ +![RSBench](docs/img/logo.png) + +[![Latest Github release](https://img.shields.io/github/release/ANL-CESAR/RSBench.svg)](https://github.com/ANL-CESAR/RSBench/releases/latest) +[![Build Status](https://travis-ci.com/ANL-CESAR/RSBench.svg?branch=master)](https://travis-ci.com/ANL-CESAR/RSBench) +[![Published in Proceedings of EASC 2014](https://img.shields.io/badge/Published%20in-Proceedings%20of%20EASC%202014-167DA4.svg)](https://doi.org/10.1007/978-3-319-15976-8_3) + +RSBench is a mini-app representing a key computational kernel of the Monte Carlo neutron transport algorithm. Specifically, RSBench represents the multipole method of perfoming continuous energy macroscopic neutron cross section lookups. The mulitpole method is a recently developed strategy for building microscopic cross section data "on-the-fly" that requires orders of magnitude less memory storage as compared to traditional methods (e.g., those represented in [XSBench](https://github.com/ANL-CESAR/XSBench)). RSBench serves as a useful performance stand-in for full neutron transport applications like [OpenMC](https://github.com/openmc-dev/openmc) that support multipole cross section representations. + +## Table of Contents + +1. [Selecting a Programming Model](#selecting-a-programming-model) +2. [Compilation](#Compilation) +3. [Running RSBench / Command Line Interface](#Running-RSBench) +4. [Verification Support](#Verification-Support) +5. [Theory & Algorithms](#Theory-&-Algorithms) + * [Transport Simulation Styles](#Transport-Simulation-Styles) + - [History-Based Transport](#History-Based-Transport) + - [Event-Based Transport](#Event-Based-Transport) + * [The Multiple Cross Section Lookup Method](#The-Multipole-Cross-Section-Lookup-Method) + * [Faddeeva Function Evaluation](#Faddeeva-Function-Evaluation) +6. [Optimized Kernels](#Optimized-Kernels) +7. [Citing RSBench](#Citing-RSBench) +8. [Development Team](#Development-Team) + +## Selecting A Programming Model + +RSBench has been implemented in multiple different languages to target a variety of computational architectures and accelerators. The available implementations can be found in their own directories: + +1. **RSBench/openmp-threading** +This is the "default" version of RSBench that is appropriate for serial and multicore CPU architectures. The method of parallelism is via the OpenMP threading model. + +2. **RSBench/openmp-offload** +This method of parallelism uses OpenMP 4.5 (or newer) to map program data to a remote accelerator memory space and run targeted kernels on the accelerator. This method of parallelism could be used for a wide variety of architectures (besides CPUs) that support OpenMP 4.5 targeting. NOTE: The Makefile will likely not work by default and will need to be adjusted to utilize your OpenMP accelerator compiler. + +3. **RSBench/cuda** +This version of RSBench is written in CUDA for use with NVIDIA GPU architectures. NOTE: You will likely want to specify in the makefile the SM version for the card you are running on. + +4. **RSBench/opencl** +This version of RSBench is written in OpenCL, and can be used for CPU, GPU, FPGA, or other architectures that support OpenCL. It was written with GPUs in mind, so if running on other architectures you may need to heavily re-optimize the code. You will also likely need to edit the makefile to supply the path to your OpenCL compiler. + +4. **RSBench/sycl** +This version of RSBench is written in SYCL, and can be used for CPU, GPU, FPGA, or other architectures that support OpenCL and SYCL. It was written with GPUs in mind, so if running on other architectures you may need to heavily re-optimize the code. You will also likely need to edit the makefile to supply the path to your SYCL compiler. + +## Compilation + +To compile RSBench with default settings, navigate to your selected source directory and use the following command: + +```bash +make +``` + + You can alter compiler settings in the included Makefile. Alternatively, for the OpenMP threading version of RSBench you may specify a compiler via the CC environment variable and then making as normal, e.g.: +```bash +export CC=clang +make +``` + +### Debugging, Optimization & Profiling + +There are also a number of switches that can be set in the makefile. Here is a sample of the control panel at the top of the makefile: + +```make +OPTIMIZE = yes +DEBUG = no +PROFILE = no +``` +- Optimization enables the -O3 optimization flag. +- Debugging enables the -g flag. +- Profiling enables the -pg flag. When profiling the code, you may +wish to significantly increase the number of lookups (with the -l +flag) in order to wash out the initialization phase of the code. + +## Running RSBench + +To run RSBench with default settings, use the following command: +```bash +./RSBench +``` +For non-default settings, RSBench supports the following command line options: + +| Argument |Description | Options | Default +|-------------|------------|---------------|------------| +|-t |# of OpenMP threads| integer value | System Default| +|-m |Simulation method| history, event| history| +|-s | Problem Size | small, large, XL, XXL | large| +-p | # of particle histories (if running using "history" method)| integer value | 500,000 | +-l | # of Cross-section (XS) lookups. If using using history based method, this is lookups per particle history. If using event-based method, this is total lookups. | integer value | (History: 34) (Event: 17,000,000) | +-p | # of avg poles per nuclide | integer value | 1,000 | +-w | # of windows per nuclide | integer value | 100 | +-d | Flag to disable Doppler broadening | || +-k | Optimized kernel ID | integer value | 0 + +- **-t [threads]** +Sets the number of OpenMP threads to run. By default, RSBench will run with 1 thread per hardware core. If the architecture supports hyperthreading, multiple threads will be run per core. If running in MPI mode, this will be the number of threads per MPI rank. This argument is only used by the OpenMP threading version of RSBench. + +- **-m [simulation method]** +Sets the simulation method, either "history" or "event". These options represent the history based or event based algorithms respectively. The default is the history based method. These two methods represent different methods of parallelizing the Monte Carlo transport method. In the history based method, the central mode of parallelism is expressed over particles, which each require some number of macroscopic cross sections to be executed in series and in a dependent order. The event based method expresses its parallelism over a large pool of independent macroscopic cross section lookups that can be executed in any order without dependence. They key difference between the two methods is the dependence/independence of the macroscopic cross section loop. See the [Transport Simulation Styles](#Transport-Simulation-Styles) section for more information. + +- **-s [size]** +Sets the size of the Hoogenboom-Martin reactor model. There are four options: 'small', 'large', 'XL', and 'XXL'. By default, the 'large' option is selected. The H-M size corresponds to the number of nuclides present in the fuel region. The small version has 34 fuel nuclides, whereas the large version has 321 fuel nuclides. This significantly slows down the runtime of the program as the data structures are much larger, and more lookups are required whenever a lookup occurs in a fuel material. Note that the program defaults to "Large" if no specification is made. The additional size options, "XL" and "XXL", do not directly correspond to any particular physical model. They are similar to the H-M "large" option, except the number of gridpoints per nuclide has been increased greatly. This creates an extremely large energy grid data structure (XL: 120GB, XXL: 252GB), which is unlikely to fit on a single node, but is useful for experimentation purposes on novel architectures. + +- **-p [particles]** +Sets the number of particle histories to simulate. By default, this value is set to 500,000. Users may want to increase this value if they wish to extend the runtime of RSBench, perhaps to produce more reliable performance counter data - as extending the run will decrease the percentage of runtime spent on initialization. Real MC simulations in a full application may use up to several billion particles per generation, so there is great flexibility in this variable. + +- **-l [lookups]** +Sets the number of cross-section (XS) lookups to perform per particle. By default, this value is set to 34, which represents the average number of XS lookups per particle over the course of its lifetime in a light water reactor problem. Users should only alter this value if they are trying to capture the behavior of a different type of reactor (e.g., one with a fast spectrum), where the number of lookups per history may be different. + +- **-p [poles]** +Sets the average number of poles per nuclide. + +- **-w [windows]** +Sets the number of windows per nuclide. + +- **-d** +This flag disables Doppler broadening, in effect making it single temperature 0K simulation. Doppler broadening is enabled by default. While Doppler broadening is likely to be used in most cases in a full application like OpenMC, in may be useful to disable in some cases so as to compare to traditional single temperature lookup algorithms. + +- **-k [kernel]** +For some of the RSBench code-bases (e.g., openmp-threading and cuda) there are several optimized variants of the main kernel. All source bases run basically the same "baseline" kernel as default. Optimized kernels can be selected at runtime with this argument. Default is "0" for the baseline, other variants are numbered 1, 2, ... etc. People interested in implementing their own optimized variants are encouraged to use this interface for convenience rather than writing over the main kernel. The baseline kernel is defined at the top of the "Simulation.c" source file, with the other variants being defined towards the end of the file after a large comment block delineation. The optimized variants are related to different ways of sorting the sampled values such that there is less thread divergence and much better cache re-usage when executing the lookup kernel on contiguous sorted elements. More details can be found in the [Optimized Kernels](#Optimized-Kernels) section. + +## Verification Support + +Legacy versions of RSBench had a special "Verification" compiler flag option to enable verification of the results. However, a much more performant and portable verification scheme was developed and is now used for all configurations -- therefore, it is not necessary to compile with or without the verification mode as it is always enabled by default. RSBench generates a hash of the results at the end of the simulation and displays it with the other data once the code has completed executing. This hash can then be verified against hashes that other versions or configurations of the code generate. For instance, running RSBench with 4 threads vs 8 threads (on a machine that supports that configuration) should generate the same hash number. Running on GPU vs CPU should not change the hash number. However, changing the model / run parameters is expected to generate a totally different hash number (i.e., increasing the number of particles, number of gridpoints, etc, will result in different hashes). However, changing the type of lookup performed (e.g., nuclide, unionized, or hash) should result in the same hash being generated. Changing the simulation mode (history or event) will generate different hashes. + +## Theory & Algorithms + +### Transport Simulation Styles + +#### History-Based Transport + +The default simulation model used in RSBench is the "history-based" model. In this model, parallelism is expressed over independent particle histories, with each particle being simulated in a serial fashion from birth to death: + +```c +for each particle do // Independent + while particle is alive do // Dependent + Move particle to collision site + Process particle collision +``` + +This method of parallelism is very memory efficient, as the total number of particles that must be kept in memory at once is equivalent to the total number of active threads being run in the simulation. However, as there are many different types of collision events, the history-based model means that there is no natural SIMD style parallelism available for work happening between different threads. + +#### Event-Based Transport + +An alternative simulation model is the "event-based" model. In this model, parallelism is instead expressed over different collision (or "event") types. To facilitate this, all particles in the simulation are stored in memory at once. Each event kernel is executed in parallel on vectors of particles that currently require that event to be executed: + +```c +Get vector of source particles +while any particles are alive do // Dependent + for each living particle do // Independent + Move particle to collision site + for each living particle do // Independent + Process particle collision + Sort/consolidate surviving particles +``` +This method of parallelism is requires more memory and requires an extra stream compaction kernel to sort and organize the particles periodically to ready them for the different event kernels. The benefit of this model is that kernels can potentially be execute in a SIMD manner and with higher cache efficiency due to the potential to sort particles by material and energy. On CPU architectures, the costs of sorting and buffering particles typically outweigh the benefits of the event-based model, but on accelerator architectures the tradeoff has been found to usually be more favorable. + +### The Multipole Cross Section (XS) Lookup Method + +RSBench represents the multipole macroscopic cross section lookup kernel. This kernel is responsible for adding together microscopic cross section data from all nuclides present in the material the neutron is travelling through, given a certain energy: + +

XS_Lookup_EQ

+ +Macroscopic cross section data is typically required for multiple reaction channels "c", such as the total cross section, fission cross section, etc. + +Historically, cross section data has been stored in pointwise format, sometimes requiring in excess of 100,000 energy level data points be stored for a single nuclide. There are a variety of methods for performing cross section lookups on traditional pointwise data, as represented in the mini-app [XSBench](https://github.com/ANL-CESAR/XSBench). However, a more memory and bandwidth efficient cross section representation method has recently been developed known as the "multipole" format that models the quantum mechanical resonances (or "poles") that underly the pointwise data. By this method, the resonances can be modeled mathematically and assembled on-the-fly while storing only a fraction of the data that is required for the traditional pointwise format. The tradeoff is that a greatly increased amount of floating point work must be performed when expanding the quantum mechanical "residues" into useable cross section data. + +More information regarding the mathematics and equations used in the multipole method can be found in: + +>C. Josey, P. Ducru, B. Forget, K. Smith, Windowed multipole for cross section Doppler broadening, Journal of Computational Physics, Volume 307, 2016, Pages 715-727. https://doi.org/10.1016/j.jcp.2015.08.013 + +#### Faddeeva Function Evaluation + +Doppler broadening of resonance data requires evaluation of the complex error function, also known as the Faddeeva Functon. This function is not typically available as a language supplied or standard library intrinsic, so use of the multipole method requires implementing our own or adding a library dependency. Typical libraries that implement the Faddeeva function (e.g., the [MIT Faddeeva Package](http://ab-initio.mit.edu/wiki/index.php/Faddeeva_Package) written by Steven Johnson), break up the phase space of the function into many different areas, with each area using its own evaluation technique. This minimizes the number of floating point operations that must be performed, but can create a lot of branching which often precludes high SIMD efficiency. An alternative lightweight formulation is available, known as the [Fast Nuclear Faddeeva (FNF) Function](https://github.com/jtramm/FNF). FNF is a much simpler source implementation that only has one possible branch, allowing it to be sepcialized for use in complex phase space commonly seen in light water reactor simulations. FNF is therefore used in RSBench to minimize code complexity while providing acceptable accuracy for the use case of neutron transport. + +## Optimized Kernels + +If using the event-based model, we will be executing the lookup kernel in RSBench across all particles at once. While SIMD execution is possible using this method, typically issues can arrise that greatly reduce SIMD efficiency. In particular, different materials in the simulation have very different numbers of nuclides in them. For instance, spent fuel has 300+ nuclides, while moderator regions only have 10 or so nuclides. This creates a significant load imbalance across lanes in a SIMD vector, as some particles may only need a few iterations to complete all nuclides while others would need hundreds. Therefore, efficient SIMD execution of the event-based model is not possible without some optimizations. + +One promising optimization for the event-based model is to perform a key-value sort of particles: first by material, and then by energy within each material. The first sort by material allows for adjacent particles in the vector to typically reside in the same type of material -- meaning that they will require the same number of nuclide lookup iterations. Then, the energy sort means that adjacent particles in the vector will be located close in energy space -- potentially allowing for many adjacent particles to access the same energy indices in each nuclide and therefore perform many or all of the same branching operations and read the same cache lines into memory at the same time. Once sorted, separate event kernels are then called for each material in the simulation. These two sorts can potentially boost both SIMD efficieny and cache efficiency, with effects being amplified as more particles are simulated at each event stage. The downside to this optimization is the introduction of the key-value particle sorting operations, which can be costly and potentially outweight any gains due to improved SIMD efficiency and cache performance. + +We have implemented this optimization in both the OpenMP threading and CUDA models. They are not enabled by default, but must be enabled with the "-k 1" or "-k 6" flags if running with OpenMP and CUDA respectively. These optimizations have not yet been implemented in the other programming models due to the lack of an efficient parallel sorting function being easily available without having to create an external library dependency. + +## Citing RSBench + +Papers citing the RSBench program in general should refer to: + +>Tramm J.R., Siegel A.R., Forget B., Josey C, "Performance Analysis of a Reduced Data Movement Algorithm for Neutron Cross Section Data in Monte Carlo Simulations," Presented at EASC 2014 - Solving Software Challenges for Exascale, Stockholm. [https://doi.org/10.1007/978-3-319-15976-8_3](https://doi.org/10.1007/978-3-319-15976-8_3) + +Bibtex Entry: + +```bibtex +@inproceedings{Tramm:rs, +author="Tramm, John R. and Siegel, Andrew R. and Forget, Benoit and Josey, Colin", +title="Performance Analysis of a Reduced Data Movement Algorithm for Neutron Cross Section Data in Monte Carlo Simulations", +booktitle = {{EASC} 2014 - Solving Software Challenges for Exascale}, +address = {Stockholm}, +year = "2014", +url = "https://doi.org/10.1007/978-3-319-15976-8_3" +} +``` + +## Development Team +Authored and maintained by John Tramm ([@jtramm](https://github.com/jtramm)) with help from Ron Rahaman, Amanda Lund, and other [contributors](https://github.com/ANL-CESAR/RSBench/graphs/contributors). diff --git a/README.txt b/README.txt deleted file mode 100644 index 66df96e..0000000 --- a/README.txt +++ /dev/null @@ -1,315 +0,0 @@ -============================================================================== - _____ _____ ____ _ - | __ \ / ____| _ \ | | - | |__) | (___ | |_) | ___ _ __ ___| |__ - | _ / \___ \| _ < / _ \ '_ \ / __| '_ \ - | | \ \ ____) | |_) | __/ | | | (__| | | | - |_| \_\_____/|____/ \___|_| |_|\___|_| |_| - - Version 12 - -============================================================================== -Contact Information -============================================================================== - -Organization: Computational Science Division - Argonne National Laboratory - -Development Lead: John Tramm - -Contributing Developers: Amanda Lund - Ron Rahaman - -============================================================================== -What is RSBench? -============================================================================== - -RSBench is mini-app to represent the multipole resonance representation lookup -cross section algorithm for Monte Carlo neutron transport. - -More information can be found in the following two publications: - -https://doi.org/10.1007/978-3-319-15976-8_3 - -https://doi.org/10.1109/Co-HPC.2014.9 - -============================================================================== -Quick Start Guide -============================================================================== - -Download---------------------------------------------------------------------- - - For the most up-to-date version of RSBench, we recommend that you - download from our git repository. This can be accomplished via - cloning the repository from the command line, or by downloading a zip - from our github page. Alternatively, you can download a tar file from - the CESAR website directly. - - Git Repository Clone: - - Use the following command to clone RSBench to your machine: - - >$ git clone https://github.com/jtramm/RSBench.git - - Once cloned, you can update the code to the newest version - using the following command (when in the RSBench directory): - - >$ git pull - - Git Zip Download: - - Simply use the "zip download" option on our webpage at: - - https://github.com/jtramm/RSBench - -Selecting A Source Version---------------------------------------------------- - - RSBench has been implemented in multiple different languages to target - a variety of computational architectures and accelerators. The available - implementations can be found in the "RSBench/src" directory: - - 1. RSBench/src/openmp-threading - - This is the "default" version of RSBench that is appropriate for serial - and multicore CPU architectures. The method of parallelism is via the - OpenMP threading model. - - 2. RSBench/src/openmp-offload - - This method of parallelism uses OpenMP 4.5 (or newer) to map program - data to a remote accelerator memory space and run targeted kernels on - the accelerator. This method of parallelism could be used for a wide - variety of architectures (besides CPUs) that support OpenMP 4.5 targeting. - - NOTE: The Makefile will likely not work by default and will need to be - adjusted to utilize your OpenMP accelerator compiler. - - 3. RSBench/src/cuda - - This version of RSBench is written in CUDA for use with NVIDIA GPU - architectures. - - NOTE: You will likely want to specify in the makefile the SM version - for the card you are running on. - - 4. RSBench/src/opencl - - This version of RSBench is written in OpenCL, and can be used for CPU, - GPU, FPGA, or other architectures that support OpenCL. It was written - with GPUs in mind, so if running on other architectures you may need to - heavily re-optimize the code. You will also likely need to edit the - makefile to supply the path to your OpenCL compiler. - - 4. RSBench/src/sycl - - This version of RSBench is written in SYCL, and can be used for CPU, - GPU, FPGA, or other architectures that support OpenCL and SYCL. - It was written with GPUs in mind, so if running on other architectures - you may need to heavily re-optimize the code. You will also likely need - to edit the makefile to supply the path to your SYCL compiler. - -Compilation------------------------------------------------------------------- - - To compile RSBench with default settings, use the following - command: - - >$ make - -Running RSBench--------------------------------------------------------------- - - To run RSBench with default settings, use the following command: - - >$ ./RSBench - - For non-default settings, RSBench supports the following command line - options: - - Usage: ./RSBench - Options include: - -m Simulation method (history, event) - -t Number of OpenMP threads to run - -s Size of H-M Benchmark to run (small, large) - -l Number of Cross-section (XS) lookups per particle - -p Number of particle histories to simulate - -P Average Number of Poles per Nuclide - -w Average Number of Windows per Nuclide - -d Disables Temperature Dependence (Doppler Broadening) - -k Optimized kernel id selector. The baseline kernel "0" is default. - Default is equivalent to: -s large -p 300000 -l 34 -P 1000 -w 100 -k 0 - - -m - - Sets the simulation method, either "history" or "event". These - options represent the history based or event based algorithms - respectively. The default is the history based method. These two - methods represent different methods of parallelizing the Monte - Carlo transport method. In the history based method, the central - mode of parallelism is expressed over particles, which each require - some number of macroscopic cross sections to be executed in series - and in a dependent order. The event based method expresses its - parallelism over a large pool of independent macroscopic cross - section lookups that can be executed in any order without dependence. - They key difference between the two methods is the dependence/independence - of the macroscopic cross section loop. - - -m - - Sets the simulation method, either "history" or "event". These - options represent the history based or event based algorithms - respectively. The default is the history based method. These two - methods represent different methods of parallelizing the Monte - Carlo transport method. In the history based method, the central - mode of parallelism is expressed over particles, which each require - some number of macroscopic cross sections to be executed in series - and in a dependent order. The event based method expresses its - parallelism over a large pool of independent macroscopic cross - section lookups that can be executed in any order without dependence. - They key difference between the two methods is the dependence/independence - of the macroscopic cross section loop. - - -t - - Sets the number of OpenMP threads to run. By default, RSBench - will run with 1 thread per hardware core. If the architecture - supports hyperthreading, multiple threads will be run per - core. This command is only available with the openmp-threading - programming model. - - -s - - Sets the size of the Hoogenboom-Martin reactor model. There - are two options: 'small' and 'large'. By default, - the 'large' option is selected. - - The H-M size corresponds to the number of nuclides present - in the fuel region. The small version has 34 fuel nuclides, - whereas the large version has 321 fuel nuclides. This - significantly slows down the runtime of the program as the - data structures are much larger, and more lookups are required - whenever a lookup occurs in a fuel material. Note that the - program defaults to "Large" if no specification is made. - - -p - - Sets the number of particle histories to simulate. - By default, this value is set to 300,000. Users may want to - increase this value if they wish to extend the runtime of - XSBench, perhaps to produce more reliable performance counter - data - as extending the run will decrease the percentage of - runtime spent on initialization. Real MC simulations in a full - application may use up to several billion particles per generation, - so there is great flexibility in this variable. - - -l - - Sets the number of cross-section (XS) lookups to perform per particle. - By default, this value is set to 34, which represents the average - number of XS lookups per particle over the course of its lifetime in - a light water reactor problem. Users should only alter this value if - they are trying to capture the behavior of a different type of reactor - (e.g., one with a fast spectrum), where the number of lookups per - history may be different. - - -P - - This is the average number of poles per nuclide. Default is - set to 1000. - - -w - - This is the number of windows per nuclide. Default is set to - 100. - - -d - This flag disables Doppler broadening (temperature dependence) in the - calculation. Doppler broadening represents a calculation where the - temperature of the materials in the reactor are considered and - Doppler broadening is accomplished via the evaluation of the Faddeeva - function for each pole within a window, which is accomplished by using - a fast approximation method for the Faddeeva function. Disabling - this feature represents a neutronics calculation done at 0K. - - -k - - For some of the XSBench code-bases (e.g., openmp-threading and cuda) - there are several optimized variants of the main kernel. All source - bases run basically the same "baseline" kernel as default. Optimized - kernels can be selected at runtime with this argument. Default is - "0" for the baseline, other variants are numbered 1, 2, ... etc. - People interested in implementing their own optimized variants are - encouraged to use this interface for convenience rather than writing - over the main kernel. The baseline kernel is defined at the top of - the "simulation.c" source file, with the other variants being defined - towards the end of the file after a large comment block delineation. - The optimized variants are related to different ways of sorting - the sampled values such that there is less thread divergence and - much better cache re-usage when executing the lookup kernel on - contiguous sorted elements. - -============================================================================== -Debugging, Optimization & Profiling -============================================================================== - -There are also a number of switches that can be set in the makefile. - -Here is a sample of the control panel at the top of the makefile: - -COMPILER = gnu -OPTIMIZE = yes -DEBUG = no -PROFILE = no - --> Optimization enables the -O3 optimization flag. - --> Debugging enables the -g flag. - --> Profiling enables the -pg flag. - -============================================================================== -Verification Support -============================================================================== - -Legacy versions of RSBench had a special "Verification" compiler flag option -to enable verification of the results. However, a much more performant and -portable verification scheme was developed and is now used for all -configurations -- therefore, it is not necessary to compile with or without -the verification mode as it is always enabled. - -RSBench generates a hash of the results at the end of the simulation and displays -it with the other data once the code has completed executing. This hash can -then be verified against hashes that other versions or configurations of -the code generate. For instance, running RSBench with 4 threads vs 8 threads -(on a machine that supports that configuration) should generate the -same hash number. Running on GPU vs CPU, or using a different programming model, - should not change the hash number. However, changing the simulation model or -run parameters is expected to generate a totally different hash number -(e.g., increasing the number of particles, number of poles, etc, will result -in different hashes). Changing the simulation mode (history or event) will -also generate different hashes. - -============================================================================== -Citing RSBench -============================================================================== - -Papers citing the RSBench program in general should refer to: - -Tramm J.R., Siegel A.R., Forget B., Josey C, "Performance Analysis of -a Reduced Data Movement Algorithm for Neutron Cross Section Data in -Monte Carlo Simulations," Prsented at EASC 2014 - Solving Software -Challenges for Exascale, Stockholm. https://doi.org/10.1007/978-3-319-15976-8_3 - -Bibtex Entry: - -@inproceedings{Tramm:rs, -author="Tramm, John R. -and Siegel, Andrew R. -and Forget, Benoit -and Josey, Colin", -title="Performance Analysis of a Reduced Data Movement Algorithm for Neutron Cross Section Data in Monte Carlo Simulations", -booktitle = {{EASC} 2014 - Solving Software Challenges for Exascale}, -address = {Stockholm}, -year = "2014", -url = "https://doi.org/10.1007/978-3-319-15976-8_3" -} - -============================================================================== diff --git a/src/cuda/Makefile b/cuda/Makefile similarity index 100% rename from src/cuda/Makefile rename to cuda/Makefile diff --git a/src/cuda/init.cu b/cuda/init.cu similarity index 100% rename from src/cuda/init.cu rename to cuda/init.cu diff --git a/src/cuda/io.cu b/cuda/io.cu similarity index 100% rename from src/cuda/io.cu rename to cuda/io.cu diff --git a/src/cuda/main.cu b/cuda/main.cu similarity index 100% rename from src/cuda/main.cu rename to cuda/main.cu diff --git a/src/cuda/material.cu b/cuda/material.cu similarity index 100% rename from src/cuda/material.cu rename to cuda/material.cu diff --git a/src/cuda/rsbench.cuh b/cuda/rsbench.cuh similarity index 100% rename from src/cuda/rsbench.cuh rename to cuda/rsbench.cuh diff --git a/src/cuda/simulation.cu b/cuda/simulation.cu similarity index 100% rename from src/cuda/simulation.cu rename to cuda/simulation.cu diff --git a/src/cuda/utils.cu b/cuda/utils.cu similarity index 100% rename from src/cuda/utils.cu rename to cuda/utils.cu diff --git a/docs/img/XS_equation.svg b/docs/img/XS_equation.svg new file mode 100644 index 0000000..843fcdd --- /dev/null +++ b/docs/img/XS_equation.svg @@ -0,0 +1,259 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/img/logo.png b/docs/img/logo.png new file mode 100644 index 0000000..62d1c24 Binary files /dev/null and b/docs/img/logo.png differ diff --git a/src/opencl/CLutils.c b/opencl/CLutils.c similarity index 100% rename from src/opencl/CLutils.c rename to opencl/CLutils.c diff --git a/src/opencl/Makefile b/opencl/Makefile similarity index 100% rename from src/opencl/Makefile rename to opencl/Makefile diff --git a/src/opencl/init.c b/opencl/init.c similarity index 100% rename from src/opencl/init.c rename to opencl/init.c diff --git a/src/opencl/io.c b/opencl/io.c similarity index 100% rename from src/opencl/io.c rename to opencl/io.c diff --git a/src/opencl/kernel.cl b/opencl/kernel.cl similarity index 100% rename from src/opencl/kernel.cl rename to opencl/kernel.cl diff --git a/src/opencl/main.c b/opencl/main.c similarity index 100% rename from src/opencl/main.c rename to opencl/main.c diff --git a/src/opencl/material.c b/opencl/material.c similarity index 100% rename from src/opencl/material.c rename to opencl/material.c diff --git a/src/opencl/rsbench.h b/opencl/rsbench.h similarity index 100% rename from src/opencl/rsbench.h rename to opencl/rsbench.h diff --git a/src/opencl/simulation.c b/opencl/simulation.c similarity index 100% rename from src/opencl/simulation.c rename to opencl/simulation.c diff --git a/src/opencl/utils.c b/opencl/utils.c similarity index 100% rename from src/opencl/utils.c rename to opencl/utils.c diff --git a/src/openmp-offload/Makefile b/openmp-offload/Makefile similarity index 100% rename from src/openmp-offload/Makefile rename to openmp-offload/Makefile diff --git a/src/openmp-offload/init.c b/openmp-offload/init.c similarity index 100% rename from src/openmp-offload/init.c rename to openmp-offload/init.c diff --git a/src/openmp-offload/io.c b/openmp-offload/io.c similarity index 100% rename from src/openmp-offload/io.c rename to openmp-offload/io.c diff --git a/src/openmp-offload/main.c b/openmp-offload/main.c similarity index 100% rename from src/openmp-offload/main.c rename to openmp-offload/main.c diff --git a/src/openmp-offload/material.c b/openmp-offload/material.c similarity index 100% rename from src/openmp-offload/material.c rename to openmp-offload/material.c diff --git a/src/openmp-offload/rsbench.h b/openmp-offload/rsbench.h similarity index 100% rename from src/openmp-offload/rsbench.h rename to openmp-offload/rsbench.h diff --git a/src/openmp-offload/simulation.c b/openmp-offload/simulation.c similarity index 100% rename from src/openmp-offload/simulation.c rename to openmp-offload/simulation.c diff --git a/src/openmp-offload/utils.c b/openmp-offload/utils.c similarity index 100% rename from src/openmp-offload/utils.c rename to openmp-offload/utils.c diff --git a/src/openmp-threading/Makefile b/openmp-threading/Makefile similarity index 83% rename from src/openmp-threading/Makefile rename to openmp-threading/Makefile index bc1fb66..85f06d7 100644 --- a/src/openmp-threading/Makefile +++ b/openmp-threading/Makefile @@ -2,11 +2,9 @@ # User Options #=============================================================================== -# Compiler options are "gnu", "llvm", and "intel". -# This can be set below, or with the "COMPILER" environment variable. -# Default is gnu. -ifeq ($(COMPILER),) -COMPILER = gnu +# Compiler can be set below, or via environment variable +ifeq ($(CC),) +CC = gcc endif OPTIMIZE = yes DEBUG = no @@ -32,24 +30,26 @@ obj = $(source:.c=.o) # Sets Flags #=============================================================================== +# Make sure we didn't pick up lower case cc +ifeq (cc,$(CC)) +CC = gcc +endif + # Standard Flags CFLAGS := -std=gnu99 -Wall # gcc Compiler -ifeq ($(COMPILER),gnu) - CC = gcc +ifneq (,$(findstring gcc,$(CC))) CFLAGS += -fopenmp -flto endif # LLVM Compiler -ifeq ($(COMPILER),llvm) - CC = clang +ifneq (,$(findstring clang,$(CC))) CFLAGS += -fopenmp -flto endif -# intel compiler -ifeq ($(COMPILER),intel) - CC = icc +# Intel Compiler +ifneq (,$(findstring intel,$(CC))) CFLAGS += -qopenmp -ipo endif diff --git a/src/openmp-threading/init.c b/openmp-threading/init.c similarity index 100% rename from src/openmp-threading/init.c rename to openmp-threading/init.c diff --git a/src/openmp-threading/io.c b/openmp-threading/io.c similarity index 100% rename from src/openmp-threading/io.c rename to openmp-threading/io.c diff --git a/src/openmp-threading/main.c b/openmp-threading/main.c similarity index 100% rename from src/openmp-threading/main.c rename to openmp-threading/main.c diff --git a/src/openmp-threading/material.c b/openmp-threading/material.c similarity index 100% rename from src/openmp-threading/material.c rename to openmp-threading/material.c diff --git a/src/openmp-threading/rsbench.h b/openmp-threading/rsbench.h similarity index 100% rename from src/openmp-threading/rsbench.h rename to openmp-threading/rsbench.h diff --git a/src/openmp-threading/simulation.c b/openmp-threading/simulation.c similarity index 100% rename from src/openmp-threading/simulation.c rename to openmp-threading/simulation.c diff --git a/src/openmp-threading/utils.c b/openmp-threading/utils.c similarity index 100% rename from src/openmp-threading/utils.c rename to openmp-threading/utils.c diff --git a/src/sycl/Makefile b/sycl/Makefile similarity index 100% rename from src/sycl/Makefile rename to sycl/Makefile diff --git a/src/sycl/init.cpp b/sycl/init.cpp similarity index 100% rename from src/sycl/init.cpp rename to sycl/init.cpp diff --git a/src/sycl/io.cpp b/sycl/io.cpp similarity index 100% rename from src/sycl/io.cpp rename to sycl/io.cpp diff --git a/src/sycl/main.cpp b/sycl/main.cpp similarity index 100% rename from src/sycl/main.cpp rename to sycl/main.cpp diff --git a/src/sycl/material.cpp b/sycl/material.cpp similarity index 100% rename from src/sycl/material.cpp rename to sycl/material.cpp diff --git a/src/sycl/rsbench.h b/sycl/rsbench.h similarity index 100% rename from src/sycl/rsbench.h rename to sycl/rsbench.h diff --git a/src/sycl/simulation.cpp b/sycl/simulation.cpp similarity index 100% rename from src/sycl/simulation.cpp rename to sycl/simulation.cpp diff --git a/src/sycl/utils.cpp b/sycl/utils.cpp similarity index 100% rename from src/sycl/utils.cpp rename to sycl/utils.cpp