From 9df546d09dbbf9e0b6b5c5f72ba7e0462eb2a704 Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 18:02:37 +0000 Subject: [PATCH 01/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 121 ++++++------------- 1 file changed, 35 insertions(+), 86 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 9ce93832..1f182cf6 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -1,45 +1,23 @@ --- title: Convert a C++ program +author: Timothy Kaler +date: 2022-07-20T16:22:55.620Z --- +A common application of OpenCilk is the parallelization of existing serial codes. Indeed, it is often advisable for programmers to write correct and efficient serial codes prior to introducing parallelism because of the notorious difficulty of writing correct parallel programs. In this section, we shall walk through the process of converting an existing serial C or C++ code to an OpenCilk parallel program and show how OpenCilk's suite of tools can be used to debug race-conditions and scalability bottlenecks. -{% alert "primary" %} -***Note:*** This page will be updated soon to include `cilk_scope`, introduced with OpenCilk 2.0. -{% endalert %} - -## Overview - -Here is the sequence of steps to create a parallel program using OpenCilk. - -* Typically, you will start with a serial C or C++ program that implements the basic - functions or algorithms that you want to parallelize. You will likely - be most successful if the serial program is correct to begin with! - Any bugs in the serial program will occur in the parallel program, but - they will be more difficult to identify and fix. -* Next, identify the program regions that will benefit from parallel - operation. Operations that are relatively long-running and which can - be performed independently are prime candidates. -* Use the three OpenCilk keywords to identify tasks that can execute in - parallel: - * `cilk_spawn` indicates a call to a function (a "child") that can proceed in parallel with the caller (the "parent").* - `cilk_sync` indicates that all spawned children must complete before proceeding. - * `cilk_for` identifies a loop for which all iterations can execute in parallel. -* Build the program: - - * **Linux* OS:** Use the `clang` or `clang++` compiler command. -* Run the program. If there are no ***race conditions***, the parallel program will produce the same result - as the serial program. -* Even if the parallel and serial program results are the same, there - may still be race conditions. Run the program under the ***cilksan - race detector*** to identify possible race - conditions introduced by parallel operations. -* ***Correct any race conditions*** with ***reducers***, locks, or recode to resolve - conflicts. -* Note that a traditional debugger can debug the *serialization* of a parallel program, which you can create - easily with OpenCilk. - -We will walk through this process in detail using a sort program as an example. - -## Start with a serial program +One typically begins with an existing serial C or C++ program that implements the functions or algorithms that are relevant to one's application. Ideally, the serial code you start with will be well-tested to verify it is correct and performance-engineered to be efficient in terms of the total work it performs. Any correctness bugs in the serial code will result in correctness bugs in the parallel program, but they will be more difficult to identify and fix! Similarly, inefficiency in the original sequential code will translate to inefficiency in its parallel equivalent.\ +\ +Next, one begins the process of introducing parallelism into the program. Typically one starts by identifying the regions of the program that will most benefit from parallel execution. Operations that are relatively long-running and/or tasks that can be performed independently are prime candidates for parallelization. The programmer then uses the three OpenCilk keywords to identify tasks that can execute in parallel: + +* `cilk_spawn` indicates a call to a function (a "child") that can proceed in parallel with the caller (the "parent"). +* `cilk_sync` indicates that all spawned children must complete before proceeding. +* `cilk_for` identifies a loop for which all iterations can execute in parallel. + +The programmer can then compile and test their new parallel program. On **Linux* OS** one can compile one's program using the OpenCilk compiler with the `clang` or `clang++` compiler commands. The program can then be run on the local machine and tested for correctness and performance. If the parallelization of the original (correct) serial program contains no ***race conditions***, then the parallel program will produce the same result as the serial program. \ +\ +The OpenCilk tools can be used to debug race conditions and scalability bottlenecks in parallelized codes. Verifying the absence of race conditions is particularly important when writing parallel code. Fortunately, OpenCilk provides the ***cilksan race detector*** which can identify all possible race conditions introduced by parallel operations when a program is run on a given input. Using the OpenCilk tools, the programmer can identify possible race conditions in their parallel code and correct them using a combination of **reducers,** locks, and recoding. + +## Example: Quicksort We'll demonstrate how to use write an OpenCilk program by parallelizing a simple implementation of ***Quicksort*** @@ -103,20 +81,9 @@ int main(int argc, char* argv[]) } ``` -## Convert to an OpenCilk program - -Converting the C++ code to OpenCilk C++ code is very simple. - -* Add a "`#include `" statement to the source. `cilk.h` - declares all the entry points to the OpenCilk runtime. - -The result is an OpenCilk program that has no parallelism yet. +### Compiling Quicksort with the OpenCilk compiler -Compile the program to ensure that the OpenCilk SDK development -environment is setup correctly. - -Typically, OpenCilk programs are built with optimized code for best -performance. +This quicksort code can be compiled using the OpenCilk C++ compiler by adding `#include ` statement to the source file. The `cilk.h` header file contains declarations of the OpenCilk runtime API and the keywords used to specify parallel control flow. After adding the `cilk.h` header file, one can compile the quicksort program using the OpenCilk compiler, as shown below. ##### Linux* OS @@ -124,21 +91,15 @@ performance. > clang++ qsort.cpp -o qsort –O3 -fopencilk ``` -## Add parallelism using `cilk_spawn` +### Add parallelism using `cilk_spawn` -We are now ready to introduce parallelism into our `qsort` program. +The next step is to actually introduce parallelism into our quicksort program. This can be accomplished through the judicious use of OpenCilk's three keywords for expressing parallelism: `cilk_spawn`, `cilk_sync`, and `cilk_for`. -The `cilk_spawn` keyword indicates that a function (the *child*) may be -executed in parallel with the code that follows the `cilk_spawn` -statement (the *parent*). Note that the keyword *allows* but does not -*require* parallel operation. The OpenCilk scheduler will dynamically -determine what actually gets executed in parallel when multiple -processors are available. The `cilk_sync` statement indicates that the -function may not continue until all `cilk_spawn` requests in the same -function have completed. `cilk_sync` does not affect parallel strands -spawned in other functions. +In this example, we shall make use of just the `cilk_spawn` and `cilk_sync` keywords. The `cilk_spawn` keyword indicates that a function (the *child*) may be executed in parallel with the code that follows the `cilk_spawn`statement (the *parent*). Note that the keyword *allows* but does not *require* parallel operation. The OpenCilk scheduler will dynamically determine what actually gets executed in parallel when multiple processors are available. The `cilk_sync` statement indicates that the function may not continue until all `cilk_spawn` requests in the same function have completed. `cilk_sync` does not affect parallel strands spawned in other functions. -```c +Let us walk through a version of the quicksort code that has been parallelized using OpenCilk. + +``` void sample_qsort(int * begin, int * end) { if (begin != end) { @@ -153,25 +114,15 @@ void sample_qsort(int * begin, int * end) } ``` -In line 8, we spawn a recursive invocation of `sample_qsort` that can -execute asynchronously. Thus, when we call `sample_qsort` again in line 9, the call at line 8 might not have completed. The `cilk_sync` -statement at line 10 indicates that this function will not continue -until all `cilk_spawn` requests in the same function have completed. +In the example code above, the serial quicksort code has been converted into a parallel OpenCilk code by adding the `cilk_spawn` keyword on line 8, and the `cilk_sync` keyword on line 10. The `cilk_spawn` keyword on line 8 indicates that the function call `sample_qsort(begin, middle)` is allowed to execute in-parallel with its ***continuation*** which includes the program instructions that are executed after the return of the function call on line 8 and the `cilk_sync` instruction on line 10. -There is an implicit `cilk_sync` at the end of every function that waits -until all tasks spawned in the function have returned, so the `cilk_sync` here is redundant, but written explicitly for clarity. +The `cilk_spawn` keyword can be thought of as allowing the recursive invocation of `sample_qsort` on line 8 to execute asynchronously. Thus, when we call `sample_qsort` again in line 9, the call at line 8 might not have completed. The `cilk_sync` statement at line 10 indicates that this function will not continue until all `cilk_spawn` requests in the same function have completed. There is an implicit `cilk_sync` at the end of every function that waits until all tasks spawned in the function have returned, so the `cilk_sync` here is redundant, but written explicitly for clarity. -The above change implements a typical divide-and-conquer strategy for -parallelizing recursive algorithms. At each level of recursion, we have -two-way parallelism; the parent strand (line 9) continues executing the -current function, while a child strand executes the other recursive -call. This recursion can expose quite a lot of parallelism. +The parallelization of quicksort provided in this example implements a typical divide-and-conquer strategy for parallelizing recursive algorithms. At each level of recursion, we have two-way parallelism; the parent strand (line 9) continues executing the current function, while a child strand executes the other recursive call. In general, recursive divide-and-conquer algorithms can expose significant parallelism. In the case of quicksort, however, parallelizing according to the standard recursive structure of the serial algorithm only exposes limited parallelism. The reason for this is due to the substantial amount of work performed by the serial `partition` function invoked on line 5. The partition algorithm may be parallelized for better scalability, but we shall leave this task as an exercise to the intrepid reader. -## Build, execute, and test +### Build, execute, and test -With these changes, you can now build and execute the OpenCilk version -of the qsort program. Build and run the program exactly as we did with -the previous example: +Now that you have introduced parallelism into the quicksort program, you can build and execute the OpenCilk version of the qsort program with the command shown below. ##### Linux* OS: @@ -179,7 +130,8 @@ the previous example: > clang++ qsort.cpp -o qsort –O3 -fopencilk ``` -### Run qsort from the command line +\ +The quicksort code can be run from then command line as shown below to verify correctness and measure its runtime performance. ```shell > qsort @@ -188,17 +140,13 @@ Sorting 10000000 integers Sort succeeded. ``` -By default, an OpenCilk program will query the operating system and use -all available cores. You can control the number of workers by setting -the CILK_NWORKERS environment variable: +By default, an OpenCilk program will execute in-parallel using all of the cores available on the machine. You can control the number of workers for a particular execution by setting the CILK_NWORKERS environment variable as shown below. ```shell CILK_NWORKERS=8 ./qsort ``` -### Observe speedup on a multicore system - -Run qsort using one and then two cores: +Using the CILK_NWORKERS environment one can measure the parallel speedup achieved by quicksort when varying the number of utilized cores. Below we show the result of running the quicksort program using one and two cores. ```powershell > CILK_NWORKERS=1 qsort @@ -210,4 +158,5 @@ Sorting 10000000 integers 1.468 seconds Sort succeeded. ``` -Alternately, run cilkscale to get a more detailed performance graph. \ No newline at end of file +\ +One can also use the cilkscale tool to generate more detailed performance graphs. \ No newline at end of file From 66b5916cb5fb9a286dec4d9e73df59620399d9bf Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 18:21:54 +0000 Subject: [PATCH 02/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 27 ++++++++++---------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 1f182cf6..dfd66cf0 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -3,28 +3,27 @@ title: Convert a C++ program author: Timothy Kaler date: 2022-07-20T16:22:55.620Z --- -A common application of OpenCilk is the parallelization of existing serial codes. Indeed, it is often advisable for programmers to write correct and efficient serial codes prior to introducing parallelism because of the notorious difficulty of writing correct parallel programs. In this section, we shall walk through the process of converting an existing serial C or C++ code to an OpenCilk parallel program and show how OpenCilk's suite of tools can be used to debug race-conditions and scalability bottlenecks. +A common application of OpenCilk is the parallelization of existing serial code. Indeed, it is often advisable for programmers to prioritize writing correct and efficient serial code before attempting parallelization due to the notorious difficulty of writing correct parallel code. In this section, we shall walk through the process of converting an existing serial C or C++ code to an OpenCilk parallel program and show how OpenCilk's suite of tools can be used to debug race-conditions and scalability bottlenecks. -One typically begins with an existing serial C or C++ program that implements the functions or algorithms that are relevant to one's application. Ideally, the serial code you start with will be well-tested to verify it is correct and performance-engineered to be efficient in terms of the total work it performs. Any correctness bugs in the serial code will result in correctness bugs in the parallel program, but they will be more difficult to identify and fix! Similarly, inefficiency in the original sequential code will translate to inefficiency in its parallel equivalent.\ -\ -Next, one begins the process of introducing parallelism into the program. Typically one starts by identifying the regions of the program that will most benefit from parallel execution. Operations that are relatively long-running and/or tasks that can be performed independently are prime candidates for parallelization. The programmer then uses the three OpenCilk keywords to identify tasks that can execute in parallel: +## General workflow + +One typically begins with an existing serial C or C++ program that implements the functions or algorithms that are relevant to one's application. Ideally, the serial code you start with will be well tested to verify it is correct and be performance engineered to achieve good performance when run serially. Any correctness bugs in the serial code will result in correctness bugs in the parallel program, but they will be more difficult to identify and fix! Similarly, inefficiency in the original sequential code will translate to inefficiency in its parallel equivalent. + +Next, one begins the process of introducing parallelism into the program. Typically one starts by identifying the regions of the program that will most benefit from parallel execution. Operations that are relatively long-running and/or tasks that can be performed independently are prime candidates for parallelization. The programmer can identify tasks in their code that can execute in parallel using the three OpenCilk keywords: * `cilk_spawn` indicates a call to a function (a "child") that can proceed in parallel with the caller (the "parent"). * `cilk_sync` indicates that all spawned children must complete before proceeding. * `cilk_for` identifies a loop for which all iterations can execute in parallel. -The programmer can then compile and test their new parallel program. On **Linux* OS** one can compile one's program using the OpenCilk compiler with the `clang` or `clang++` compiler commands. The program can then be run on the local machine and tested for correctness and performance. If the parallelization of the original (correct) serial program contains no ***race conditions***, then the parallel program will produce the same result as the serial program. \ -\ +The parallel version of the code can be compiled and tested using the OpenCilk compiler. On **Linux* OS** one invokes the OpenCilk compiler using the `clang` or `clang++` commands. One compiled, the program can be run on the local machine to tested for correctness and measure performance. If the parallelization of the original (correct) serial program contains no ***race conditions***, then the parallel program will produce the same result as the serial program. + The OpenCilk tools can be used to debug race conditions and scalability bottlenecks in parallelized codes. Verifying the absence of race conditions is particularly important when writing parallel code. Fortunately, OpenCilk provides the ***cilksan race detector*** which can identify all possible race conditions introduced by parallel operations when a program is run on a given input. Using the OpenCilk tools, the programmer can identify possible race conditions in their parallel code and correct them using a combination of **reducers,** locks, and recoding. ## Example: Quicksort -We'll demonstrate how to use write an OpenCilk program by parallelizing -a simple implementation of ***Quicksort*** -([http://en.wikipedia.org/wiki/Quicksort](http://en.wikipedia.org/wiki/Quicksort)). +Let us illustrate the process of parallelizing an existing serial code by walking through an example wherein we shall introduce parallelism into a serial implementation of ***Quicksort*** ([http://en.wikipedia.org/wiki/Quicksort](http://en.wikipedia.org/wiki/Quicksort)). -Note that the function name `sample_qsort` avoids confusion with the -Standard C Library `qsort` function. +Note that in this example we use the function name `sample_qsort` in order to avoid confusion with the Standard C Library `qsort` function. ```c #include @@ -83,7 +82,7 @@ int main(int argc, char* argv[]) ### Compiling Quicksort with the OpenCilk compiler -This quicksort code can be compiled using the OpenCilk C++ compiler by adding `#include ` statement to the source file. The `cilk.h` header file contains declarations of the OpenCilk runtime API and the keywords used to specify parallel control flow. After adding the `cilk.h` header file, one can compile the quicksort program using the OpenCilk compiler, as shown below. +This quicksort code can be compiled using the OpenCilk C++ compiler by adding `#include ` statement to the source file. The `cilk.h` header file contains declarations of the OpenCilk runtime API and the keywords used to specify parallel control flow. After adding the `cilk.h` header file, one can compile the quicksort program using the OpenCilk compiler. ##### Linux* OS @@ -95,11 +94,11 @@ This quicksort code can be compiled using the OpenCilk C++ compiler by adding `# The next step is to actually introduce parallelism into our quicksort program. This can be accomplished through the judicious use of OpenCilk's three keywords for expressing parallelism: `cilk_spawn`, `cilk_sync`, and `cilk_for`. -In this example, we shall make use of just the `cilk_spawn` and `cilk_sync` keywords. The `cilk_spawn` keyword indicates that a function (the *child*) may be executed in parallel with the code that follows the `cilk_spawn`statement (the *parent*). Note that the keyword *allows* but does not *require* parallel operation. The OpenCilk scheduler will dynamically determine what actually gets executed in parallel when multiple processors are available. The `cilk_sync` statement indicates that the function may not continue until all `cilk_spawn` requests in the same function have completed. `cilk_sync` does not affect parallel strands spawned in other functions. +In this example, we shall make use of just the `cilk_spawn` and `cilk_sync` keywords. The `cilk_spawn` keyword indicates that a function (the *child*) may be executed in parallel with the code that follows the `cilk_spawn` statement (the *parent*). Note that the keyword *allows* but does not *require* parallel operation. The OpenCilk scheduler will dynamically determine what actually gets executed in parallel when multiple processors are available. The `cilk_sync` statement indicates that the function may not continue until all `cilk_spawn` requests in the same function have completed. `cilk_sync` does not affect parallel strands spawned in other functions. Let us walk through a version of the quicksort code that has been parallelized using OpenCilk. -``` +```c void sample_qsort(int * begin, int * end) { if (begin != end) { From 295b900fd4f0011f363e057b624fa8b3bf3cab49 Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 18:33:06 +0000 Subject: [PATCH 03/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index dfd66cf0..79b77bc5 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -15,13 +15,13 @@ Next, one begins the process of introducing parallelism into the program. Typic * `cilk_sync` indicates that all spawned children must complete before proceeding. * `cilk_for` identifies a loop for which all iterations can execute in parallel. -The parallel version of the code can be compiled and tested using the OpenCilk compiler. On **Linux* OS** one invokes the OpenCilk compiler using the `clang` or `clang++` commands. One compiled, the program can be run on the local machine to tested for correctness and measure performance. If the parallelization of the original (correct) serial program contains no ***race conditions***, then the parallel program will produce the same result as the serial program. +The parallel version of the code can be compiled and tested using the OpenCilk compiler. On **Linux* OS** one invokes the OpenCilk compiler using the `clang` or `clang++` commands. One compiled, the program can be run on the local machine to test for correctness and measure performance. If the parallelization of the original (correct) serial program contains no ***race conditions***, then the parallel program will produce the same result as the serial program. -The OpenCilk tools can be used to debug race conditions and scalability bottlenecks in parallelized codes. Verifying the absence of race conditions is particularly important when writing parallel code. Fortunately, OpenCilk provides the ***cilksan race detector*** which can identify all possible race conditions introduced by parallel operations when a program is run on a given input. Using the OpenCilk tools, the programmer can identify possible race conditions in their parallel code and correct them using a combination of **reducers,** locks, and recoding. +The OpenCilk tools can be used to debug race conditions and scalability bottlenecks in parallelized codes. Verifying the absence of race conditions is particularly important as such errors can lead to non-deterministic (and often buggy) behavior. Fortunately, OpenCilk provides the ***cilksan race detector*** which can identify all possible race conditions introduced by parallel operations when a program is run on a given input. With the help of OpenCilk's tools, one can identify and resolve race conditions through the use of **reducers**, locks, and recoding. ## Example: Quicksort -Let us illustrate the process of parallelizing an existing serial code by walking through an example wherein we shall introduce parallelism into a serial implementation of ***Quicksort*** ([http://en.wikipedia.org/wiki/Quicksort](http://en.wikipedia.org/wiki/Quicksort)). +Let us illustrate the process of parallelizing an existing serial code by walking through an example wherein we shall expose parallelism in a serial implementation of ***Quicksort*** ([http://en.wikipedia.org/wiki/Quicksort](http://en.wikipedia.org/wiki/Quicksort)). Note that in this example we use the function name `sample_qsort` in order to avoid confusion with the Standard C Library `qsort` function. @@ -94,7 +94,7 @@ This quicksort code can be compiled using the OpenCilk C++ compiler by adding `# The next step is to actually introduce parallelism into our quicksort program. This can be accomplished through the judicious use of OpenCilk's three keywords for expressing parallelism: `cilk_spawn`, `cilk_sync`, and `cilk_for`. -In this example, we shall make use of just the `cilk_spawn` and `cilk_sync` keywords. The `cilk_spawn` keyword indicates that a function (the *child*) may be executed in parallel with the code that follows the `cilk_spawn` statement (the *parent*). Note that the keyword *allows* but does not *require* parallel operation. The OpenCilk scheduler will dynamically determine what actually gets executed in parallel when multiple processors are available. The `cilk_sync` statement indicates that the function may not continue until all `cilk_spawn` requests in the same function have completed. `cilk_sync` does not affect parallel strands spawned in other functions. +In this example, we shall make use of just the `cilk_spawn` and `cilk_sync` keywords. The `cilk_spawn` keyword indicates that a function (the *child*) may be executed in parallel with the code that follows the `cilk_spawn` statement (the *parent*). Note that the keyword *allows* but does not *require* parallel operation. The OpenCilk scheduler will dynamically determine what actually gets executed in parallel when multiple processors are available. The `cilk_sync` statement indicates that the function may not continue until all `cilk_spawn` requests in the same function have completed. The `cilk_sync` instruction does not affect parallel strands spawned in other functions. Let us walk through a version of the quicksort code that has been parallelized using OpenCilk. From c22972e45ff5c13b864122a3a0086d76dec417b2 Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 18:38:08 +0000 Subject: [PATCH 04/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 79b77bc5..524fad7a 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -3,7 +3,7 @@ title: Convert a C++ program author: Timothy Kaler date: 2022-07-20T16:22:55.620Z --- -A common application of OpenCilk is the parallelization of existing serial code. Indeed, it is often advisable for programmers to prioritize writing correct and efficient serial code before attempting parallelization due to the notorious difficulty of writing correct parallel code. In this section, we shall walk through the process of converting an existing serial C or C++ code to an OpenCilk parallel program and show how OpenCilk's suite of tools can be used to debug race-conditions and scalability bottlenecks. +A common application of OpenCilk is the parallelization of existing serial code. Indeed, it is often advisable for programmers to prioritize writing correct and efficient serial code before attempting parallelization because of the notorious difficulty of writing correct parallel code. In this section, we shall walk through the process of converting an existing serial C or C++ code to an OpenCilk parallel program and show how OpenCilk's suite of tools can be used to debug race-conditions and scalability bottlenecks. ## General workflow From b34618e60afe36ef470a4d35faff8d54c9912912 Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 19:00:52 +0000 Subject: [PATCH 05/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 30 ++++++++++++++++++- src/img/cilkscale-qsort-execution-time.png | Bin 0 -> 32850 bytes src/img/cilkscale-qsort-speedup.png | Bin 0 -> 35794 bytes 3 files changed, 29 insertions(+), 1 deletion(-) create mode 100644 src/img/cilkscale-qsort-execution-time.png create mode 100644 src/img/cilkscale-qsort-speedup.png diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 524fad7a..53f526de 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -157,5 +157,33 @@ Sorting 10000000 integers 1.468 seconds Sort succeeded. ``` +### Checking for race conditions using Cilksan + +The Cilksan race detector can be used to check for race conditions in the parallelized quicksort code. To run Cilksan on our parallel quicksort routine, we must compile the program with Cilksan enabled and then execute the instrumented program. + +```shell +> clang++ qsort.cpp -o qsort –Og -g -fopencilk -fsanitize=cilk +./qsort 10000000 +``` + +The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. + +### Measuring scalability using Cilkscale + +Cilkscale can be used to benchmark and analyze the parallelism, in terms of work and span, of an OpenCilk program. These measurements can be used to predict parallel performance on parallel processors.\ +\ +The Cilkscale visualizer tool can be used to illustrate the scalability of the quicksort program by compiling the program with the additional flag `-fcilktool=cilkscale` and then executing the program as shown below. + +```shell +> clang++ qsort.cpp -o qsort –O3 -fopencilk -fcilktool=cilkscale +./qsort 10000000 +``` + \ -One can also use the cilkscale tool to generate more detailed performance graphs. \ No newline at end of file +\ +\ +The plots generated by Cilkscale for the example quicksort program are shown below. + +![Cilkscale speedup for quicksort.](/img/cilkscale-qsort-speedup.png "Quicksort speedup") + +![Cilkscale execution time for quicksort.](/img/cilkscale-qsort-execution-time.png "Quicksort execution time") \ No newline at end of file diff --git a/src/img/cilkscale-qsort-execution-time.png b/src/img/cilkscale-qsort-execution-time.png new file mode 100644 index 0000000000000000000000000000000000000000..bd24718dbff6d3215f213c5554564560663fce91 GIT binary patch literal 32850 zcmZ5{byQSQ*SFFoE#07ilt{M>El5dshz#A`Fn}~t0umC^-Q6KQh;(;%_jd=M=Y7`t z{$R1@-gD2{XYYM}J5PwBywnRc60~Q}p1qKf7FT}u?0F3Emw=|Y!?C*K6P2T5c&wMsz#6?wH_4ZRy zQ{=}w<*&H;FN*?iiv>5{%KC>22Um-_hlSfkFC+WU5&hfK)*@77Ba zn%m36UYA9$V5i-Q<)h_JNt<~^6%|Oj_YJDZ-4Xmd4vq-Dj0_qO&0ofgiwh{{i^)Y> zuPlO?sVNN|r*Tz{8q1{7u{t{khtBl?WaG&aT}iB^wYBUAb$BF<_jY#WqaFv-6(;DC z5hQ_Dvo*v*Z5d+0(ELyI=QkM{8Iq+?MI36J9efuT7cSm8yIyihg;hEl8c=hTX5|-R zp$JlRbo6uYiBf%=vuAE@Zbz4wQW_dWkz{;EXK5b74_C(LGy0y#furzo-@nuH@)A9N z{=AQZod+DD(BcWF2Fktct*t2Z}0JFhoI&fNTsp3lA0`N9>2oAvbc zv^nKrjlS!0ARdx!%j?qRv+c2Q-w!ddvFxV9WKNNuk#46O`J2Vz0Riw7US~?x>5u89 ztq)rD*%$k>!IOF(FFY;}xFU;e*;|fQ!pdrDdi>y#kEV>H@-I0K+U<6lJvnb_>F8e- zwl|;DA9I}S*De-JE>*8R7N58sU0-SWJl^x9xvro(kuB;wI#$_rg+RTyJ+ExfXKLr{ zWxFkQAH}o^nbeBf{1MUPxVm~KGaD`TXR7OVOB(+^dR`vPGcYnLJ;FacPODE>P&cJH zemhP+x@mrVxZO!c*~G+&IgJ}>z0&dGqnAtcEp53$d%RtIgjSfyPR+~|w6BLUl_&=E znx=aNF@DO)Gq!2D{<5~YS=j49WxR+;sN<`y@AWeF^+)AdGT!YSS0zfw)QGU~0tBA@ zw!v}R-)t;rdD=8xbyh*>JZ?rmUcv~|NJ02Ino%c{cyF4<%BVkZ`>Q1DBw~;;@cZeoK=FnOl|{+4_afT{ffv_f1@5-UteDb zkW4|Tda2$A6-u`?9H+Fs)$^sL%YLR(GQ5)hBOx8{>-C-MT=~?363s9ImZPml3e^A; zGqd~#0?!xf>gwl2p_sLNM6thrOP?Js>Ng`VH2WO2Bbz@0f+!JBgbMvF(&RYXoBnjK z;I{Afk>S0!xA%??6T1I&uj@<1=);g%e4iHKz{#E-$<*E7-)Q51j=rlqXxRO0hmVYT zRjgIr5lt-?9Uc8M?eg}pRsHr_)8l44cYFxFma$E>eKN=QhzJc&|HsGxDM5?rgU!_( ze#&sFK;i0^@|s7}*&o@_1}BU2xjVtisPvC%v5d*Y#Kb#3U6<_^C)bwJ7`w)U`GKr0ma@HL+;`MA#s`ia6%d4wd58Q!kBO;F~v!|OQ>a%oobmA|t$L&au zS>xcF%wCS3)SvV-jXPK`w&V5y#aSLj!hVxiU79NtDo6A3euV#c6;x=DsZ>98^@7bbl=L_>6E{}piD+U$r=jP*5 zIP_IBbsyWg*^7CE^;Hwog|zrz3tox_AOR4GDgEJt@?@TNt+n}V)oVC-`0>>%&5+Pg zWv`8h(Cyb*h@JcKOzH(zp!SUyF=p{-)l!f(!S#KY1vUMjq&7`T^(eelYC$fRO@$i| zjfX9yok1@?_5Og*Vx)6qNwh69&^M_aTkDHerEJ}UxKt4WAX+XOJ~cg^Z_N8UQQ!L% zGd_S0nF$k2CJh!k_T&T4x$hS5Bz_V__jSp~kL^$FFv0x0k;-(q!N|HX|3<>J>nl$G zjiF#0Bmk3gr2Q-WUqlvG#Z9BoclnXg_}}phMHAuLe+n+H0(hF`oqnf*g@wf>ojjTI z^XL2Xv6ZEH?sVhpll7gOBmPnMm2yvo4X*Xa$~VJ-(Ta(3qvt0l?;@Z1`}^D4?S`?t zpMk5b&`=knQ0G(k9PI4a-Q~OPij)-P3UG0?>hJWEHFfZZ04U2J`H3nKG$O&nyd3f3 z@vWup2G1M79lVLz8n$QzAMZ}h-)8&V#mKTM1#s!U zBjvVK6#nAZ^G;$8?oQ#FP)xgNZ21Z9~Sq*hol7d9B;4IX1U9yQS*0 zjuuB@A4Ry+16b?(St|<+3bt8&~g4~lOZo%aiL_#JPdy5cxo zEcl>-kX}0|AYYXLM#S125a>HTGP9Prr--0Ji%83e-D(eGYcpzP8|x&pyfFQv&FkK|Kcue)=885lJ% zEV%EoRO`k^L-&=*sWZT;kv7bl72@P#Jn+c$4^|B|L~EPex_VrJE`3efxe}_xJ;qll zInjxVbIu2I`JN9|pIvo{us7c8yOsF6)s8$*HnOb&fp~4%t(B%%_=ttsqKpN7Sy^wc zlHv>w8aI+HROWs=7N=q?BI|Oowcb^XI?&`qzcp;@lG`sXkpK;jNUx8)5K-96%KKJ{ zgL~?p9ua-y~B~K}2 zpkM($a5`bpvdtU2+xPEEL9t3b<3cSq!^1* zukfATkLh;adwPEfJ|+@QgBy6r%ml3q8>chUg;ytT_FcA59$lWQPfp&?d`hEOPA^##7}%%19B7>s#4Ndo){OW!|9vt z&*RB{MCY}(5bv2LztNSm4A(qLxk}tj^N`;cAt#e7A>)>NVOzz?i*o98P2zE}=NBIz zj|b!AN`4X3rGwJ;2ChI2oCd}}_V~$=gL6=Z8896zX$#gsJ*w{eG zQHm=WrO*1?`CP1u+;oL(g@bx@#%%VHw$d8pd+L&$ay zyM+HN2K0x%7HHSMSEc3j3+23E&bo1rv@031g?9$ih}_0c9103=OJ9x*kDvgWG2`ul z&Ci^tk#};?PyPyhe!$>CdSH;L_ zY{(3suH>%BJkvdWEaO+1<1=1%O|G3U`3^g9jrdJQX_}MqGId_IZ>|}mBD<#{<2_^G z_rx3?db`<(y!+Yq;P|WLsVUery$}CWTdmDf7AV4Rf?;y=ZL3J8rveJepqSW&eb3#8m@1Xee(nbFkIrixrreAlQxa-NAhVPWxikP*pNp| z9Ngs;(BpNI|G;=)3A%jTs-?-7X%%ZnsnLc!v<0U>uZ*?m{HgA0zeb~M;^z>IRU6#g zTEOcPqzv{@o%Tf>{22PdLn@VqGSBT8kP_yAjj%ij4E3i%sRlvBu@OY`UPeetYfSh` zEPWYCE1o0ylPwebdcW05&dxN!VBK-20L%&Mz(p>~>^ESQm6HZs#^kEUnsNpvCgJq_ z1Klhf+>**RWi((y(ERb~hzJuqA2|&`5lROm5YixsVw(q|gixXQD>8{o1HT|z)F2vQ z`M+p5xT4qy8EXF@F;Aa@in0HFDiMnlll4o=4m?ZU#r#!tu7Vf8n^Ip{4WOIXBN-A+ zHY|YgD0Oc*HbEj>>(=2E&0*;=MKi`V#tVWC(FTU>-F~68Y@DLH36QR9Ie@z)6kKwt z#Uf*MdcWPSyvhL3ux!bD7`O)RQ*vEgkMI>un>zc-!kPYJtLDg~OCB}Aic&jUAcs^6 zwnfvI&%MKx3GjRcAE&7~#Rr+a0Fm6e4Gx|kG;deae zCNLI@C6+tCrRdhZu6YtMN+z#h(qdXUOdT9^=na_H$UNS&ys z1NaOhC8Pe)l@?uhmxUhkN*Lq(?SDQ+V2MeOmpCzu)_`forvGbtI=s1uKY?9nltIBn zLclYS71$p1t}lQcS67;bCY%DM8{C|H)YomnAfu4&<>mj*o)Uf6FsZ9aSBfwI5eMjU z5RJvd&2=RNC)Cu9&7*a%2TGO#blBDaAEQbu_Vy#{}E+AAoQ=@a*I+s6vb|hU~v%e@fsY zTg|?yDw_%Dg8L`S1Yk?wG((`y51HC@m<5r(LWcuEE46=mBvLw57&Fr z4>NmEOdMX6b)%Z-SYVfkx)F>o)xU-zbbNfofVbP-AJ1e61l(t7$xx_b({T@}>qe4k znaKb_*5#Rslr-k|>}BuGVrbDGb}}0QaK1oR`5e^OJB6jtW)GJti%HDOMW5DDZ7v|w zu{B;u*b_k-T<@^CvJ%GD0|dF|dqP~}IJj|}OT-FHT#a6Cp4p>}fIw4NoLwJag7#bD$S zJ3k_@=gMEh#&ncEcgJRdYg?mPL;}u5_7+J%P@3s=dm*i$z~lag*M99QF5xEw3h%3B z4)l;=3+o>l`wIb=G-laZ_eG|%6-ugz1PFxUGX6k#u3;o>FH6+oxw~l2{I}wak#K_ zO;)adW+n{+GBWMkw~8TyiR{L2JuWPb5A)rI%($wA@NrGEq@Gk_M?(6b<<{D`(EQ=D z_0jNTt*?im%ZBm|Hps}&^zJ?arnAeIfEXAV8AYwFODZ@S z=o}5pakR){^^zdK=JWRh(Frq6N7BBA7yT}9-^vJn>wV|?QiP9u{AEn7PJ-BMvn*lE zv;QtHC(X>vtU{J06-BWH%?Q8^g{55fXIs1)Lv38$-17G;D&ED@3>w=s9ZR0`^6>Cf zA3Cb2OA{d6p$-{gQJmw20r;L#^EWB_ZDnXI?zFP9a&%lA1RfbP&nw)ithB01#R~62 zNwCIhHYhwC1EyH&(%whAoR1LG5Qr9Ls7YyGw{FhW^N79clFPbt6zFnVi7k3(|1fm#RJ!#%{=fa#K&9?@t1pl zhrtU+a!xdUcE)~9cSAJq4v40N1$NA!bb)k2J|g_q5BSN|98D51OJgn(iyxDc$*I$z zT`OHgZkUwtL>{;Y%COY)y$@jDQ z=pnhaPv{D)t|r}bJjwzTJU-y*9?gTeLzorO z+D_!z$-5_{%3}nc7tuVlAvQjWo6_@$6A+NVSPS}SRsq6I*b8^oCd`Vhq^EhZ%)`k5 z3ER_3TlHRioZ9pnrqaujPsh0rO-9Wb!`8#cC^d*oK7Vj5R z(tqLu{{i6^K|W)0dbkuReB}DaBxrUZ9g_*6A?Utz()BRI%|9TtjhX zWD`D04>SCb*KBkyMUyPJ+Af3Pp`14WgCWV#wZJvM;)ducx0U6#XVTTMqnZ3E7qqg! z9|0sGm#QcA0tyQYM{jK~DTOhRk&$6`Es@ubhTCce($pjfU~qnY^fSB%cZ!*`)I@AD ze_F|O7?-_v3Z$_*_NFVx%RL8^cuA>~ ztWR;O<{y{LDMj!paTsAZ#t4`-vNcNArRQWqTIC15b#|*qP%*6hR)vjmb^Y#6ED=k~ z=lWjfFZms}e2a>hvvSRcC8ZatU6E%l7VYY5MbQ31id-s+jG-Yl5*9@`u=wcKbG0H3 zzlZxMZAoFcBRc0vt@uP&MCUKh2gWH_*tQC;NWsl5DuEdGyW)pw-BTrom}9 zsM=zZM93G&#JivG1m4|wW^M91Yz}w7`mW=jD&W#FHiqXi@ATb1AU+|1L`o$(At4Cx z9w-4zuMTH_z5d;JIEAlgVW)h^g{$i0Vpe4qvc0^!H_X}%{tbyHdUQsc+Lf*(&K_6?^Qb${Ux$)=ygAhNj8%=)Q zKdVt7n~FF~|1lY2QBsct-N!ToW6w$Y^&HxdFtsP;N+zI?;f<>=v6YfC*- zWgQV?|gf4WWA4^F5Q*T|E#=waMVJJ-?>6huZdVdYcXOOM1n&k#3 zBeAiuRW}>=Vy>gDme(%0GqKh!!Fd9yy z6vAR=_FsZjq#rB*rSe>{f?Dp2gCq5>J`&UdlFDQMs3(1qbCY>{Xd_nX(EAbB2+7zQ3^?uRS#U|!a_ieQ#ivopz;6!kbq}be5IZ&1`+r1H z)6ns+`gp3>O3$(P50JVAo3!h~jx!Z7)@*eFE<&&F$0}N)f@j#s!7J+t- z;~{6qH36hXU{jB}d2AL!AM`?otrw&jh3?;k1AFKQ`5L{&Z2y2Ulklnz+D?8P_|Iim zBo~)#MpJ&#M|)s9yRg+zV1c6R>+0;3OiOblwbPYc}>I&yccdh!xf zka2KRyTVZ%_1w`p1=O4Ham=19Uf6p7=zNA&#Aqa_p zr?@=j`9D%VX~YDM79J{r>|38gB`y%8%V4D^AIiLcrhGqAnib zwG$3Cdt!w*L6wkm-QOYj+kf$kIW5TOXrqA`T5}iv{6JWm@mqH>01X9UC?V}t5r=>q zfmWO%grlWs@Xs>>7F)tAbops@L&l=xiA@)#4LEu#q&T<%d}=n^pm&7HsRB)vmH%QG zxv$Yg%y!opGdJ5%%lYM`Z-)PyM4@#ZTyvoczRq8ZSTf^@#=#X^LfUZ(nB64!5`1y2zXPkQWiXhEE1+?dB1ybZSd|@cYl==5%!H>+7y} z$GvvlFV&X-aO<9{cTjyC;#mv_N^Fw%y6Jx` zhd`w}BQyd50pX|5<3mAdOMZMb8Do#oz#*h&7H-o0X6fyd|5qRu5wLGDQ6ySxJrB*2 z!UD>r$f^8J9lNEik+f2gL{fh#t8s8o`Ws}R&RUtJg~+&}|FmpJ(U_XL>ZuCEtlvT= zFE1}?bN@-3`ug*LCD`XNq@*Mbg^l?ikd7%CQyXq1w6=->Hb1)yqHqM*UMfMsIT+82 zqdl)SF~F@pQ5UQWD8EPkDbAP@Ei9sHr;QTy2g(^)0GMyhH`ba1wbD+Y@c5Zb)dxuS zw@kM@n+dYX;B#w#%{2A|5AsqGkglPjkG~#ouCDS>zXFpKyn6k*ASBe(-!3jYn{FI% zLo9&^yP~4PxK}-R2CdI!wk4>>oS)N_Qp2QwMGXL}@r)kOpF(xEIKVUtjpvq?VFT8> zd#OEu{qt{ZximpdP2Sv&>1kr1b$|!y1xfb1?nGT0w`4Om_ zk^ePVtTa`qvYI6*R4?U@EJ*%g-GItrI?P$)3lw(^0Z*l8BBTkZCf5N@m=e(Hu(D#5 zbvX#Ia>8|opzlv3a7vAh?JNH2U?iY}xw-UO)#ib^jn356)X(kg>8(*;tXI5>~2`)R4E#c8IiJrP%% z={~mSo;)wWH&HC9 z1WK24Q`!z>#?G1JcgRlE$DIxt${+Lqubdw^E#T+2OJ8(Zv}jTi>ueT6@$8RL4GJxM>Z>0-Vy3GxEv@5?19nHHe5=*JRFs1;0ysPnrwHahdMkxDm zu<-J*T_*nOjA1hF3FmwLPESwo4d$snx{R73(v9aFY4W(p*5O`=KZ%wepq3wj$x#Cq zQ`+U#Ro5jiKS}m%Z|Wv$YCGE4i89NZbF$zA?sb*f7}CPR%k|m$`T0?xF`&8!bdkyxG(-u;!kp%JBqJE%yN*&>~tc+?bj7g{xx2upu>=h@{4yDUC9GjmcK;br##hL)P#-lmaO;ro#$MPuGHQ0N&Zc&c$?+UVhN z-U%YW6P^WL3~@m_jBPtDnYtp-5$IH4(dP&AwV4mHvY)z%733dHkkpI`Vq&^*dC%=Z zE9j_oEAs=k$0?#)2AfeOB_&7O)?qPTM)5G|g#rwwr+XK1xfVB%8YR~@H?l#Zecr2y zF5`)+gH%Vge1U-BX7`r$Asdh1s+o~YBdEdL-8IV=udLXk54Y&)QQHWfm!D3O z0*v}M1+VB*h+KDzYYSdECv|vCn|x;+ScBDNW6^pe$vt{9Bb-F7jOg4R+)%a&{q685 zK)WMcbL<|poP=1O2c*Vi)G+fXQO$xoylhat2!vrN*tBLS7S%rhJB44~7Mm`FYLydo z>8*)1)JJX;984qtb9f8#l)@ozg*yU^VI3VWKbzj(UoNU9#ogpgfHEtgCe&5xoSwTk z0$NjzE5aiBam2T<2+-#fEw!PH3v2ZNE@KJy0vB&Ig*Rh^59UH$l?g_V=hW z10-thR$bVOijKC9?@!DKmY!q`oI_st)avS#)D#sN{2choayieB6EChyc?m%kJx&q0 zF^f$TOUGY%!Hq7oG$JEYPbnI?h(JjDd2+(HQYN5GZuw#gZC~{T<$)_XN7f~1wwlyu z7X-^}s(g%q(6O;$0lhcq8zbpa_qUg{e0-YKfElPRZQ&yXtyC@|vzXR&IW1y!6CQ^^ zFEp0sk+2_u=nVI`6H=#(ihLz~GoRj_DUnGdeMW!{g@q^RLF| zC-{cJh1jMJYTuR9YR1R0BZnhNot1+qc}&c&e4SbhutOhDg4zc(A;I#M)qhzh2@G$6 zDDddGm$Fl=)uPm9I6m6b{4^h}j?Mn+Ih!QC=xjAQMXu%t4FKgB0nse4C^JX=@o9Ue z`g;9jfK6(1I5oqJq{e#Q9I&A&Ow@Lv_WlR*Yni_0l4b=IMhmqXc#t)kwnD-f;4I~YS-hD>7*qaXc^GA6Vdb6&BCS&=9xoVrFUWf%E}YYROB8hVEP~K0cUQs@2tlQ{QeaMDu+Eg1vk?r#YbmufG~08Z#|^;z9%+E$qO<352SsK!&4I*+tx?CTy@geawP*VATS)ho{-6#hZy2VB>to;`+#+>DP(uv)m&U zBn!4NbG1LuLSbSKsBxgz!$4)ffECZgA#0qU*GzwoSEH-VCjl~D~rWMeXalA_erIIMd5MGaKIc{ zFRbs(@&63?)zmm|ZSNI@g+6`!6{U}u;SCcg9qZNrYWjWUz0&KRE5rg`na_ax1_1)d zF-b@^iO$3MI+*K}2116>HjCpi-X*?!=4cF*)Be4dn-j#D`TT;Q{c-6Oa&KPU#kRWo z1bH7xabnENI!q9&LP)$_v;+31uc6G$Z$;q_0G^`3-mw^vR>_boJZ!$mj;CzzR-33B z8yJ8|7FvZvosUEzLjiv9cEHf*#Nb>1wPkeZTM`*LrLJQMGha|g*pPh0{4JhZ${Wuq z^lOuDqE*VDbK1&pBw+l^o|)r)l&tzON^tA2a_pqzuk~ZC5~=2I89=xf6-fDGPcbpF zRdNvdH|=F5HMF5=B-**Ta2Qs9Bm{^%@#{t|Q0ecmLbr1?#6-F`L>}r!!X!@W!GVpy z14bC(8N%<2y%<@%6vTAK=%gRG`x&Tde4}i$)P@5qAwzP4QDnG|r+4A48iKS<=R>o5 zv&t;01d1R(33>?MFId0+nL9epx`5*D9iLntv}M@C ziqw!CiB#0So$*2#>f8Z^ds|!E_kf#H?s;u*Y%|WRQ-=*CTtyETn!o8q(1bkxil#z! zwY>>|5|~fD^@ay=>etDky_dz3mv?+GSsoWp_>V^W(;*(?o3rg6g11gPD&a)z9aA?v zN=iy~dlds^N3%BZ(SK_tKxvQXZeU7_tFyj(={i2lmi;}uS(^DbcC{TF0muznCSSY~Yi*3%=khyjZ==hKCkiEY-JB{S%!8kSBF? z=F_bj2j0Pt_=$u zb^$r=h~~|-t3g-rA(*J#7NJ5ma&p@J6`gpzoMxf61Wa8c_`|-<0f7%a#!2`$9fu*V z9h-)V#Bqs>!^J3pgF1L1xM;Vkoeu(Q5Lc^>>OZMZVKsgF26e`*0p#ssqHvI)#e{;(?O)a+~)5DNEJ)TE@OK=U2wuSnp5 zn6Mm|01bnq36^yR)HL5KceRxuhW2Iz!iD$OJVaEc*O&$|7%MA1i*{yuhDK+?9?+W* zB~wtxPyTt8a1$tQh6Jdcij|eM4roEg!NJL>VZtLJDL}|v81s4Y#WE=a^4Yje5~$&B z-_%vGe=|PKEe>0apK@%=Q1}Zi1{zUd<^giog>%xqu>q{?5^UJ#Jgs(Z;bl)55 zer6*{Fapp`!a+mF_2j*1yZz8w>t|VBWTIPG+0LJEs4X_i`~`SF*8oCmzO}ez2D4dc z9NZjc#_IR>ip|#Jab#w~isFz{0Ani+w70lTzJ=>?&usC@Yk0TV+nn3kgL-5tFMw5s zrNpI&{qO#Gp8XiZN;P$ol#>h}^+>T37BL%G4FX08V3@RB2TxM5+VJv~U|d;2&{FZ~cjc}H-l z`D8Ki$=Y6}>-m>ZI!efow$f9zbjC0WxU8&~z5Tb6I3$}u@O<}EflmLMQdMA{6-U|Z zAJ*T>EJFjb;n4kUDFF7OI|uDSxMe!CBVxnT@UFS3^}0V;pPqo~VRa_k7J$L%nH-s; z8=c%>fkGC{^>meumOX}k$MZVl(pIfd7fryY5`*mv1j$YiY}v2_f8xA*d6GNBI>IP* z_1AVv*BbfJFLH`^8d=Mi{Dx1!&_^bpUlRDj6ZS>Xg=TR@t=`O4%*;j768vmh&DT3B zT#wdp(Y==xb+@t}VDE`Uf``L%H2{-H1Ze20fp!i=AWjf3m1xGkb9tN6$V=-PYW=OM zJm8}&X;(8g((Pn@bH*X-a*&b#ZObQI#3{=Vw) z23gZ>N)8ZR3lDKlmo@>)#`mWc9U7FB)y}ZO+qFMRAD=$lID(_YR{$_ZX+idwnj2LZGqNPY zz(tl(2Xk+E86{#Juhf*K@99qkYEEI`G*Ih>=z+PLiIIj#A%9NBzKbpE#B72D8>{(UYk^DHPbhy(1+tGsl}1wx)75Rga>tK*N!0w9 zLsySLOpwH2*Ds;sO^t+9pP$%=tn{u`9N=Iez`@tGoP?9iO-M7^Bs9u}B^Ql8tKL&i zwE+XaSZ%j_xSwNLmJ$^(+dvfFGNDUea?MB zsvb-T4{X?Jv_d%i?D%Pu9#rOK$jmE$pfd|v@k*LWF-C$wcU*=cE$D=DDRrxOPWzI% zSt?#3U+zkK*>-OgWrRAdpp5p%A$g1K&gR!Wmf#~7J(A>I(4U!pGy>QqAlO-8g{?i^ zJxz2rP6Zz0-1|CZc&3JZ)=I^D>8KX(m&!@6YAt*fXizt|<=@4Hx|3*QWUShh9ahW& zcY2}%lyyO302GO=Aoo(ndFQ z>S*~rMDQit4_0oJL~H{xCDCc1A&C+{`;R}{WOo<)t}CJ;Y!(PG%WtH|7tkE@x`)}_ z@y`;mE7?YnRvgS=+5P2rbBK2O-OIKIt@N}To_;~a03 zo!xN+GzWY&zNwyd$;-^{b9S`7`}pVG^*e3?!fj9oP1G0-t&-zf9RsZQj|X5MyFjN) z0pBY}hxPb0uO&})cmA2#X-4%)U14GAxEUQl-LS=_cYKl61@qtz(NhUYgiM2dyoIf8 zU4m;Y2wCmy{9NQCpGbJXrp|{?Sxp(|e>E?15E(bOqrjOxTgX|BFNGG_SYh8kaBOMt z6E}8vVnQP|MThYQ$So;eevST%6Lo(=RMkiFWk1E_EQMm&_p*Kw*;>86{T|7Mj#YpH zY&yO~%+3xrF$?)ZK#++0G7ywc#*6|I0)Sbn?seUY|0V1DU@U$En;X5XTJs8>3WKpx zB#KU7A*KLm7Sp>wRaG^5*dMYX z6;k~P+@x9cT(&Dl_AEK~YyN^YDO`f$Nr#Q;wXZ~`Zb_yg`d*Jh(!-6B-7H_5UI7`| z*v2pe7XB=^NSg(QHekwpA-~l5x0iOX%BLdaU8}Fz!OFEA7E7FGLUz?zZIu#ZwT< zyK+|d#nd=%`M^l#Mja&fO~W%&Ru0&r4#07&Jpag2^E#LAW(E;0>$9}qEqu3eh(_a` zXxyom*l&fOttY8cYQIF3iZn1YqPgMcSF3BZv|^`ygLX^UO_W#5N4+Of3KtXjU#H3{4ujQo@x)h^}`bR!xPlQoQiO!NAe!x^>Lh3ML@S_?;gwrmIW*Ya8D~=uiQ`1`r@CVXHOE6(i`jP5#PW`eI{KRbK8_<0ZffT-9`HQ2iZe z)Y>E%CoKpZ!T_Mt252F1-2Q`TF1nA6KRVc8lqreTtx4>mqW1Cf4E}oZSEo7O@v24sOZ`~C}Mk}&ZOD^@&bTKy# zhYJwAjvU%i+}IIB6%g7pe<5Ptd;R7uxUub9B)>UN!@~=B4n$nOnH^Nk3mR^&AHaDt z>A%84RjzPhX#x}(85z_dO!5LGuUOSWwG5z)+%wU7d~yOBNMJc$@HiwEGRvB7mM;%x_9`$B?NM}e2JW_U{x}~daNMaC@L~9Os=+HyyVqb z+rMQGC0m;6#VJv5>^N0ZAY&2c=@ba0sVTz_i;X#oZWU?ZO+7SzZ|opiwm`XS7uv) zD?zqE*#_1(oycj1+utuw$-}D;fecOyN!8a$4pREXHSMsCSj13%!}?*(jZ(++%=&v{ zdqx2K-Qi{MDYhwO1dS&WuK#oZ=T?F_rPSX4lxz6w?K}DU?qQ_YXuU59J700&*+@wh zp0775Yd9*6)*QA4PD~K3?1dlu`DrAjU`!S<(|u3qPkAW{UIV&-qnDNp22%yp5xcxR zJ;U|#`a))FZKRx?Yi4qOzD0BeWLa>3KB-?;U9CE;pS$Xb8fffyaf421}z$Rw*m9i@V!$ciG#!VoE|?Mc=9u zdmKVhu=wBn=)1`j1)%Nfgv0jq&yRSMca96tPpm;s9~|d}%Fds89-JtdsL+Unt5mLh z#IzKAR_Ltu)l&c$(~8`{$4}_4_og$a@`bW@e*H@o@{6603Tm#Gr%s0Tvx}6e=cEvV z@Nz{RaQ#**@^->bYy+Zz&>OVE^B>`^_w1StTyLz|MZt+*qd9H=g#9^&HU$6tI0|)f z*4L2KB=IO189mA3RnA*nP3OdvYJQ2BOCfJtHznGwSXWefDYS5uv=nd@71`J-7fB)P z-k%w53Qry%*l>xe=VE)@Vg6} zi~iNd0MRM~mB>5FjMBi4l2Y=y5__?ehR>>ASFOIgd31z?okC2c`meiIlYUao?d}Ka z6`@KRPPgcW$Z;?+nGBt|ICcp&n4FedFW<$-!0isoK?n&798E;AcaU&mBs-R~f{&|* zChLxd&@-B|mYH%(!-Y&0M8!oV-alkKt2=iHcno+z@n$4aBQH`U(sSJ-0-|Z#mJZ_f z5rodi=RSL8^~m22B?|7HMC)16nCZ5^$HEt$Im$(Zc7WN~FdUpEr93>U(Xw%gxV6}v zo$HhrU9y6a(=cAB?oJBqL^+HMOMX_uIr+@uAw0lTX%g~8cJObJriMHc+QV<1Hbq6J#;5r(k4+up9ni*7EuBvca z3`x#OU(+e0FmA&?u~s>;GENkKXV4`A2U^azQl|IztStCG)5*|g6LMtF7+1!b;t`j8 zY)oYp#JCYz+mGlC2sr-skSKyhA@0kn?17FJXRy8lqP+;GQUTH^lD`+1wMTaX!c^wg z#v@8Ie-=3xk|&tNEkkp}q;=D8*q$GeZHKf;!5PdCYN--`@so~!&OL>SDng%U-2bP4 z={ApznXLliW$K~fQXVvWZq4wa!29J2-p|RRgtSUeT?{fu^tH{uDtFj~!BJURcIm9| zymWMI%i~|Mmi;Xz3dNW=>A~=Nm>+I`KI!hha@_X5i5S=a(%49)ZSfs#7R{EvR>7Os zgu7}npA%3)h%|H&_zCHUJhkclTO_w*8g%Qgi9Lo{T3l{BPPr9a^;YwZ7-|S-3Y--7 zJf^6v)Vup5zm3rv&JGaZl_aglPFQkNdPLSh+Zl|PhS7B-`}afo+oCL-9v zNhWA6armLQGCxBVdA%e(T2EYZ{UL>1%&(7?NvzJ=>n=x|CVh*BLx>rJZF{1xSXg*# zeGel-^K*Qc1ZrepAggc!_}Ak>zP2FGHo;J=oatoX#%;4l`5QiBpWNI|&00&$ZKF>; z#P_(lILRo$W)hHvc9Y&F*t5!1N(V?FqeJA;0w>S)0FN8{V6w6E zJR!6%J2(upGbic=%|eK;?vGI7vzlTnoEsyCy@r{Lw?#A)G0V}-X!udMJ67{u7 z&{9FR(;tG+M1te@xJ`5Vb3wA9F$-D!d-iPj!MovLcHCzS+;33ZhUJI04okIpXr0=V zu<;=-PGMjPBO2}<|I-ma_&2+QJR_m}M|kx$@H$6HQ1>^07-L1%kllX6Arz2t2;+gL)f z20p40CWV&?*?}uwxjc82C&8D+3n*TU_!nOLI1gVWlI`@s6wHm9G`5~pbF2;lVx-7i z8GjlAxNsBH>!8x2;OB6ZFCh1d9=tmFlv(H+nj1+>uhN-_L}VBkMSrrwr6nbmjTJ}# zk_mC8e@%k5<8K!mj9g-8*Z#7>WNdn&g(g0UAR-B)%t|}(YAbWBzXwV8%s6PPH|p=~ zi;@yos<+q*kRYRp7Oyw2+&44CZ2Nxu20v&O*$}8|d<6oFKfhKgpuRu?3kwL1B8x?} zM^7u$%wKzsD7deb`nD=Mug$$E>n`t>UQCFz4Bo>tZQfP$8<`ijrCi5c6!2UY)~QuP zs_(eP{C|DDRa6{Z7j~NjcXtaA+#$F_ut1PNSRcXtwk6I=p~ySoH;cXw#q;V<6r zKjVyZbNZ?;y2dE$UAxv^&zyD*%iB4qbYXOUCTg;ro*9PSi+VAw#28PG6`1()v@+VM ze*>O@7u;!|0?KCHgW{NTpOzeFc_2*4gX$b4t^oICELxSKXYx*a99)%FVs7`-vAi+n zNAO&$?!TJRr#a$Oj-2*#muYEs@sQlWatcTV&g}&ypmjQcb$5 zU~lBbsk>AKtv&nnm*76e+(4REkVe(`z#o3m2^V&>B?&&AUhu1y&U@!E%cDI|toOdV z7O1>5+8AHM$Q}#nxcNXduV;*wdF1^&Iqz8qqns{^US({Hy$9wenPkTqx$vJKqHo8wl@3 zj{cqnTK|>h(W-`zDptVp>aU}XNn%o`Ry|Kib#=wdx+voujKRcet9q6eHHVA8zIW9` zlRok|OkQCi6!R=oDw4~DQbN*od%|7vNYt5%<%kQ0C0MvD)|Sq}zl!a_=^G+K=NBEn zf&<*~=7%+>>$wLp-xZUlb(?Rp-9L1qXhWQ@^N&>3_qt&)`QNP2f8TF>C&rO`HAk@!yjC@g#8&yNHHIu)5+rF|&i> zCOX9J9Z(sj!(jXdMi1ZIHU#-HZeE8)xIp-MyIzHSd31NPCbMxK<_A8TBaM7`iM{K7 zW%E_9t@wUDNUtUJ#J^LI{be{xWV%r4pW`U|%``IVtabwX>pn&TXWwz9insk*A=*Wq zhQ{62I60%dd|&#VTT7{yfZb6Sq?j2~j-o*aFWO2#V49B|JAT)>eqytTk!#u1P&9x0 z14pMHl{NFE7Ex>M9NCf@UiHxO3R~YC9vUq4U;OcYHnuwzyG!SDEkdG5GG`@CFw!LK zbIWXgL6;CeQCU-Pk0dxOGJE11d_>$rkYptMfcs!uiMN|UH_{-V1zRmlhUP0a1={S& zr>^*cs<8kaG=82d7eyykYmfPkXF}KC)MV}5!Zr8kz=N;*^gI7NN?o7{zw8%ZQg6n; zq%R&k>2Unr-WAOoFICF8$eA4O5>Q7R>=7ta3B0$#76YL#f0N0?U6cB4 zNWWSy!>Y*j#2*BGuB8xVqLZKECgcU-BiEu4@8^IGODm2PLSmKKb}=ejmivVdmS-Lc*c(DmW{u z*nWZO*G&8h-bfy@d{106g~7pWd!FTOCbs0h$EhY@;s0E z%T{YEn=W3q=hd3)!HK3$)B_K1c6}~oYULd@On89!T-w6$Hq=Xj2!Sdl69po^79Kyp zbA7#(!ZWvyC)Cwi(qX&QLAgUNvpo0FiTlb|4=RceSlwL}(s=!uEufWc;9cX zt|cTa5W`YWN4Y`aHNY1y_HTWzz5dYq-+NE|`r5>5QPVa>^RIJ0+pwhdmeM{VLclz) z+c$tLfD|pjN>~lFFmI8Jg}SF=4CY03Qo)U$qLRlk(Mw(U_!Mmq3S*z+P)iu8KQ)Su zOA`P6@9(xBumIvA1YhTEqm)s!c5;*>m5t=dgv^y%i-}eL(-z)~58c*idHSeAK4ott zW6*HJM<~|G)$DHh0P3}nVKU=@ulm$7dOSSao@^8KHv9BzjrGFupsF>W{;!1?-lG2+TE~th-n^ax+L))n&#mXP1PLcEr)l?KA(DtdE!2#7#LK{i%ZC=^27tm6(^=+1Jast(v+~NqeY)E6P4hb{Ku0_4ombO};)#($iy; z&&*64tR+bq6%<6yJi1??B@^{rLD30Qz*9NFm-CTL)PDqEH)HTboL@$8x%XBzPIDb;VBd z;mYJdE{?^&DjPQ(i(3i8dZNxDKJ{z=r0eBFyWhdh|ELYNq0w?%-u4e@?WZxOe*O&| z`2+XCj}W1sIVLO}7hw;azTwaux;Qov&(&(ZF04y7&Pilq!#XQ=iyfTDSSyFpF{*YH z+qeK_5Z}E%-H$NAkyF{+YnxY67RKA#f(KP^P-aiEu8DCuArUFI;T}&Dpogs5->|Y1 zQek0Wzptr;9Uss=kdEr@jAUsM4701ofONE+7W+GqYDLzGucsL4t_<~R`Fw9 z`DG8_*@A@>wGM!@H~TWpgg&ddl*f&Gq+L``hKUJ@U{++=|Io1mr#`Z7xsuu#GBSje zv~M`6X4h(M0}2jgez2Q0IWjiWB3EhJC&v27QTAmb2gU?Z#RLg_!KLG8?S-nT-sHwP zm6?|B!8{ZE^HjWU=;}wg?MUTHAvd+8K#7*7!v4wm*LJYf!K%TyDtZ_Pr>(fH7rUcy zRv(og71*H`e@JfaXAv`C;H0W?4gyV@Z0U@DCy_tfr$kJ!>UI~y#dsJ!nbz8hmd+5t zo!B`u9BSoylVAmEC@I}Z+f`q`e|i2Vy<8qStM2O?b$!|G{cuKpdb(ez#rN#Iv2Ldc zDrID{uVns?cle>V?{H?g&q|UZ5(mfnYBw(u-&YG?OjToC-ZsHV!}@qBms_?~1A?C$ z`^`7{rYX38R;hlFkC!()cQxd2&%M= z4XCrFxkD>4jSIwl{2gaP!fvbI$+=ok1(N>W=BUW^H|BDDAVW}L79JfT9{|TE>|pWn zVgnU_{WCnz4fcO9q(%ZU8dBIOM|HFG2OoE<@pYoa^=DQPmB=KDo`Y1;)^bywod3Wv zYH84m3Z{ELy(;2s!Jk#GRvcQw3^6Bu{q*g98S+XLn_mA%TP5NM!(!&f?zD*c;K#U1 z%IE0>^sCpEiNVC`z;uGHda@Q9yS2$GWtU0ayLBlplbnGcSZ^rC#`zgv z8K{sHtWqDHH2yy_vX=dD~Ye(|3P-)j|R_A=~=uE;Ru< zWc(nBg}O*4)Mu9#+TB$aXHj#z(#ho3^2?Ajb3GpIL;I2zgh6HeteSoaR>g<{sAd?1a5hE$UsGX8@X+0CO-D7>yNA>K5eJwB@B zKBu?Ml>5CO8+49f8AcGf@Rt5IpfIsCHO6d{`EThl7q z;bvdDj6#~0WSM^L=4e??oE@mj!^u*W1W|l-%{`ha`#QhhblK9WoFAvGFGpb}RlE&D zEy)Uw0lSjkUAi5ICDN5jAN`ocnL%%w@3aXbLKKT><~t~mQ%}z-R@1tz4q4)~<+Cg~DWidc;lyJVnZLZ|OBN&1^qhN^ViyeiuxY%kF#O5bzgN)G zVQ?7TZX|5u%PrsrPGBXiClX0Wr!rz=w_^(nJT6t6bLW!OByrZHtil@kw2-6)a#4c! zk0(LFh&&bSJKVdu_@5YMc?_~Xg(9I*S_1_mzNI1&{-ME&HVumh2LmJzX875r`Jh%} z*g{+R!9R76IV;llwOuy7AQn0R=qj)Wtu6&?^PL^S8wa#TO ziernSWetL6j`eZb`jFEL<=8zdHd`##PCb;1PZk=O#2#wI;u}VyimFLe6YdZ(cN zJX`r-kQqa*v;P>A5Ttny?PcpJ0Y6PvLTW5Rbp}J_R~Nc#LQm%%){4)r7wM?r;*=K} zuuvhj+{JSjHCviBtIAWe(~w-KmDkR4ouI%%4;+h>={?FefN}qXSX%8g%6cG&W<$kbuH$;W>YZU zy4`yIcCM6nCC90CK%#x%ulmD7GG}3$*y^okS6L=g&RT5Gs)FG<-!x2FH?dOU_wy9G z2@prC7D$ z1q!%!*#I*ut)4XkKWiNb<|(4zYj54ZBIlFu!r}cdH~#8Q8|>Py&G6w_Sm}gO?sb=} z>(%dzt5~;ayLZvLts@u0WMW0{fBKqfAk>DQ!z8B5JTiQ1w4fDApyO8wS>JOWDgY-` zeM~m4I2-b&VPyWH-~Haj?9?$Isin`O2>)m5lp*rpfuZ;rpLRAM`_>*D(m9WPD5cMg zi|QZM+NY@fSq>BX(Dx$TxtnHyMy{k(hVJ*RpFm_S*M*xRU#LAN?m!<} zm?$bPNle8ndPrsOK~=l?pTY2XS=g+&e49 zdmDM`=9uD_T##zY0|z(xzuSKSdFZpl0ED$IPcX|d^90OI8e%1H%^rFQKd5Kenxz7C zra={i?9$Z+^Ek~G<-~jC%2Rh0_|1t3S`aI5mXxmsECwxL3sJOHP10pP)IMgeQO+6p zlqirX0D_9e`Q&wV+j&{D>)EuXz)qCcT0LHV?0Z+twU z2$hd;Ru8xy{(zq%lG$(|eM^1P`R2^UZO;|+MrA#-}| zyTQcr2OqS@qK0U`f?iT!Z6<;=(#VEdb-2+VeoT|cQ;K!%$jr|= z=vOM^mFjrDQ>mN!>10W<#IM9Vt45ia@1C{INA8z*a*HL#31t?t^!T6g``Y9%3PhBo zlf$KXXa*YVIgT~ehW$5>pat-i>Ysm=f=l^%lrA#C^*8| z!CjLK-omGm2TKJ;3frXEzh`Zku(MCIV7hCJ`f#3T9@H42?7u&yxQ=C)hpH}D#$0xG zmcqTdAnSR~8hSE$jGEnQMeqvVWIZbQ^ArLc949|xQN#p${WnFf8@Z0wY z1rxp_7(#ISETVxIMVVp?_SQQK)N$p97Ru9-dC1!5(5%NNMG10o)%g_VZ1;-rV&6Wh zqw&7(L0^gu(y&M(ADI4ooR!CJZK^GYUX)0R6Vi0;s$5Bl2qUwa4)m~73)puEaD~@q zGI-gRhOtPDMTYdZl3I4r-=4ekWPO)56e?5ywRk4_%9`IqW#r zGm(=^KPrwCp^A|ih(mhUmLAHtWxr}*d~e)Y*Ur>csH%>MjnIGEG?Sy{&6qqG)j9pA zsN$eZ3oh41HaIb8l_x1Y(Rf=5M{Hy*O0?(;CaIJEAIHcGYVzw$#7?y7;NXQQIp}Pj zht(tS%dX?13Ea?c@MuA#k2otGw!9z%qtM>XsC|1EN!eUOvD3iSbZ{P+>Q~0ME-{;I z_;7pEUrxU&2dy|ywSVC8jUN6rq#IeaEEt?qc=n_(p)CCgqB#G33Bzo!$}4Hl?%bYW z_WN}RrEt8t>Q6PO%kRjhZ1C&)x(b15-udbB!FL)S-9z|HIPW+zRk{(LM^>-CJ6i69 zC6KLs^W?W)E<<5qTM`EX^JdWp9d6l`eb+f`9B^c*$_NKm;z?zD52;W6BOM!p2O~W0 z2~*saQ}2V|@Tn2|mw8!(!RXz|F__L+7>4smJxvU#4f-71Qbu$RN?7KkhQIh>W6=+d zE>)5t$`bb(Jqh6zll7F;QXloT{Rn$nB|YhsawDTZJtyZ){%ceuE@fzXWL(tz_>vYe zx4~(6^ATopk6K5cPtADN;UE}tOaJv1C`cs#WLKK-y3OewJYE%^+?;>o04ms0?wG) zEHA8=xp7qSdl&BspO`!7z1*eq+{!xg{9>`0^6S7c9Rn`P{7?lpB(1Fu#rl&_^(QzjVcy z&4S5_DeC!IMQQn+J`my@Uzuh0)U_q*Xb3y0!|bHQ{0SD4LZ{kz@qvMSyi^j&|1yBb zUh120#>Ma@Vp``tr>bQsY5z+T%yc)YMhr~`T4X$5%=oR2DO#uj>YxW0xq6aEox@h_ zCQillKXgeA^U&9Evv-u_H4f5O*(q~?Gf(n;Ey z&9BdyX)eubEh}{%Qvu>~_ z+WJhBbY@_mX-iHj?mE^<0tc>{SHGbCVZ!)*HeUta7zg42G<2px` zP%E<-qyR-%fL!S^>K_1QAL-gn7|Yr4n)K*!zn8R?bIbC*636COh$@HH#^Jr%R(2*? zms*ToCvP7C8vz?FpSDXW70|SRhk=>By0LhXF_3ZByM+fjI+}D(${A_4$!Y~{pyoEz zykv3OYB>^7m!*Z3|7j1AYJo%DEubi8M3Wj*qO%r79|hQ}*_}#04aDIZ;dh)6H80z9 z-i$Sfz{BW~7C9sGDo0p(edKB@JY^f1zZzFtKx6Z+Vd#c$BnA4vpSI^v?l#+G|HK%C(tMu z;<UW-TGYW`igrOztY zVvFj$dd_1B@nOS8dwpOPs(Zl4CC9CZt(02t6roZ98bARuYnOZf%hNMN{yfCyFH>S8 zV|P4JX3sx6sm)2=q@0zKA2`hHnrtEeYBFXB1NTt+bk^2_D;fGX|E!$p{~6UT1gYvfcfo}c=>Ba@R^ z&@ryrawyHZS)BQ}c!VQZ>d<K@YkxPCC>W%aYs)3M zSj=k|^oetZ)b}~xU{Bs+^7HXxzCrye;}5Hf<=oEI?}2HhuzGMP{-cBe#pcImmvd>E z6JEreZu1uQm1S%CitNxgpShb^$Bm#*Ej zx=^2vovjxN(R1U~Zk8~O-jJj(O2#AUvp3}Vc=NG%y4YiFGVc$#&M5RH#d!j9Qcqac z?i0l}sXF3L`3VMQ;(=rzxu@4VyM8QR23*yfA+fc!xz+x~Z8+2(F+Ipvpl?g9zw++K ztnmMAfyX#0wuEQ+35jEt_+`5y^pJIXp?y} zH72G$ftfxYALevi*h0M z)nGLuj^DNpt9`Sfc$Fpy4~#UyxM#KgH$RX-SKsrwtJ!QXn?g)E_}6n51jk)hLHOs- zHVsbqlnt&6Zc&ftN&@xQAh`uz5+rS(4*E2eb&*UR-XE@>q1eO72aEy(&7CV(9dnYKS3Ta6Xx4OJ19FOYSz@SCK} zFGc{8xtOBn#>6weJTmdYOh~dwN{ZTkH7%ceOnbW6`!LhQxD2u%U6bv&$;fP9(8;Wy z-`PFg!;(&xW~3!92Mz7&0e@Hi^#E6vzvx7&NctSF1@PzGOyn=mrV>^!rcT;C*J94k z4GbP5N5R=hp_`|fr+tFpvra9o)vjS1j`M}lF;?F>FUENevZ0nFgWxg5C!8C+V?!GQ zaDC7*bRJF~9UDChirf@NIYBE%jS+2M7g<3}t7vyi4g#k!F&Xa8zagH5FfZ zJFW|z4{acIhgx!_8-6c;dA@tfeLgs=2DwsD*eaS)*yzMT+w{jlTOYJ5>*;WZn%IeF zHs`7)NNrzgG2Emf+S^TzPDGaoi^iSgCroXfLbfrhk?2n$nXoXWxk_GuYBjy-S}%v& zI?Puyy4&1v+n$qxEEnvlXdO4;%aSA+rfjiWU(JU_pL$%bhLI@oS1@?s-Sixbj(2{X z>6+|PR8;kw?9Ly|!4fUjdZQ}+B~XL?d1(srfbhxmg<{;nKxR^pL;z#JH9## z6B?a}K@%ze*+)4~_w!4E^n}=CZ_z@H>7hfq;^JxdkA|yiSrFC1kZ?0`cRPN}Pray~ ze`AuHH~IBjt1h>O$74op2HnuZ^MX)kCmv{Lryh7#t+Sh%S-us#`W!bw$No-v9O)E) z*6~l_%b{8B@*g^H{aBt*@5q_?QNtU>=LCU*aDnLo>0fr3*@clenc$iWoj31(c%4Z}RxOUAh= zGU(RBeMV$D0oHw7bGsIt%{ZE4w$WT<^`iZ}By7iN2KX;0_EC4X&}9#&^vhS>-`rm2 z_Rfc!%v3x*R;JuulJ?kKx>e*BSYhm}-Nb8)(h4>yn`0s2e{7iuZDO8@L7bjO5XmRf zt7c08h}uElhsbuo7AA!4EoQ{XEXV?^K)5xa52>->y;1X64ZL=w^U0_=Sx>wAc;s}I zRx1;A;-Z1H@+V`ek97P98*tr~yw)5q;dgLL6m>;Cq2)U5eIJrw?iP2fybi9Ygyz|? zI9?~0c2n;3FX57J_CB{fJc5&39N2Z2t_XFXP9&CVuBewQu3)u+T-|cRog{L)Fgl6w zNtnD7e|C2|YB8s$R(%D1Wc*J;AGwAGo12u8D6K!s$?67N5qdkO4HyfWE%{omcB(az z1v){hc1IMq239qh)1>t)&S?%XV97Ties)Y&T&48i1zKH#UV>tLIwKC#T)*BzkmbQS zybOZB-7l7>Ur+0q+vP7W3%6#Bm)apax2q$GS^zQnHe2-I9m>D7mxO@7mt9f%uV=I!v~Ey)0Y?4i}PJzBC^%YdfxB) zXl@HYQA*m)VB_H#9nF@iw|a1H0(^$mDPWHFmi7WV!1MC*^w&H5@p&CT*#gjLWPH~3 z-yf9J)x#!orNBmSuxsbTX@rc7jJ{Rn|97jF>shTgy2#|zl#-@q1O}z#dNlVc>rKYL zUsST=rCQu?x%(4<^@ab2hys{uC3On_G5^I@lxf5qxBp>K5d#7zb`L)oBwX6aTzXn~ zz#;&^uK+VaE|5g|_a9CBe<%O62#Z`sT3RHDS*PO1iDoX`8=4QGmgX<3CkzY?bpzya zR__PrT)42Ps4sv#1|1L2>`EmrA8F{RysZmRpHw&>e7|l0R8>9yaYLrxq9;J;M%f`o z&-Zy||Gl-f^&8x7dwUyS5Qp`nup31G4gD#JRsqo1s6GLt0st!psg- zR17&?Pqp7D7y*iS0PmC)D)Rj2fAm-XD*1=oGdt@){{Wbauh>&hXXkq$y{~qj5dVe% z*U(6L(9+fdu9S-c05_#WeG_y=F#)j`(hdM-m-8|TWOHsvmSTHFR^9hV-opB{w6uy# z$kTeBC$(SWq$08lrKGLX9i@+*fKkXq3|6d-q#3$z8YF^sw% z2UIvL#g@L}RswpubAUgoa_||YRPc7$>Cg*^M7X=0f=H&~2*+*{)O3~hv}RsHX6g0T zif_1UZn}_I{$B35tJSt=bNY(o?;9CS zK*P_B^a3EXVu7;k&lxNnSnh4uSk%cXF52Q20>Y?+famKw6zTuA@dU0oD8&2?p6AYe zbu@G~2OLQ5;Ft62p)*}jKj-h8)8;e7mw`NK+c6qBe*waV6E3} z2nvykKxx~$)uopP$)CBT0d9YP|32N0)m2^ZGI#I$w7I0Y7V-|UmXbf_xNB5VqUWiO zx&v=e7XbT6=t<}wXJ0EJAG4VhLv{$H)hZT)Nspj>2zVF5@PJ35!=d;ebTYf+Qs3kh zPUn5atCJ->z#E|Iibt*e7#vS)*L=eD1Y~;}0CXG#FfGm<{lRY67SJoR>+Wo1C;pPG z)5AhHReClFmo#{^In7BN^r$nbfY3Xq=UZB41h@zj@G(b+_7|!Pn+npayW4rQ%i6-` zgp*2oMhG^DA#UElX}>U%p?J!uwO+tf5IBbh)D8x|^(ak1mBs^x7nMDA&<$0a4SZ9Rba<6xXNlDi0TPMb3>REIprj*dmMz$K&@UoPYB3E2Iq%C}JiRBd98_?S>(l32Ucd20OKG~S^w3WK1>*hv z)Yh#S{^_+KS0_Aedw)%;8gdb&SSSvdpmB<9IQ^%)^TDnHt;V)&7rFDTCdo|io0O{5 zZ;?SzIy;SuHIlU3*Mc_d0Ke|gG_;ID(`(nVA|0=v2q6ZCxB)=ZsQ7vt+Srr|yd$nR zPe>N)${%n1wA$i6DDft81T7Vli(nM&$T1A2 z$9BqOK3$}YFXj+U_x1Jd7YVE52FseJF{ELlT9@ca$VLQXxO=xhp7I;nJ$KyE>R7i_ z7aj%I-oo#-L0~U|{Gj5JF`#JuiqFtbbqM7HJA9dcmtvnoR!Xg_*Hr7K-08@24I^Cr zmWu^Ket9pZweMn#g17}+=-72MjDaYkrI$HGOl&GGpOqfYMZGW3H5;Y2OnxUB$~%u( zV4uUzT!re>RVcfyVb{@^VGQz*Ic^XT>*+|?;;#9-2ta?%xCx%xFHtj~h-d~=Dk3aR z)L?bOo*Wk!7YPNgq0Z)c=POq=cbZ*M7Zb3wXeXUH=eVO16z|OGIQsOd)$P~8+AOxf z9PjW_aqYcz1_RCGWP*$+TPt=#KVlQVyA6SIDa)J9sf3R@jEA4!2$=I7`ep!RBp?Ck z!jvLs+{L4xi;mfa7HXl3ks&bWWyd(aFH z^-T>4j%I92QMiZ_-BFnsO{XJc;(>UWrd$wN8DYxyWbcP7B@F@;3|;GoB=^Dr(K}$o zP}N{Dt*9mMXf|2RK5x2+?6Wq`T138-NA8uCA#7CC9fDL$WkeEqM z?B~AM^NxEB&WOr+%-+5P&CRUO8h5o3n2mnBKRr-oD;PH7Qw#o0o6N?T)TD%+=^aPGFR*xC!U zIF6DUcTv$}1*IY|2#~GLaI`G}xGk4%@#LR_XT(gnxEXmaU%^>z>wCX}CJ8tnTe>c6 z!Asb{OdQRb^wEwns8*bGF%iO+IQzJ)0Vy^E{m`O}>l9C4J~v$z^QP61{khKdIq%32>SU?T7Vq}cD-Rz0c>D@L8dVlmo30Z#{ zf-ZLkgD}#QHH11j5nCdCZ}65?<@M=eH;^0m+I>ouAcAO%>grtz#CJ@NSyzi|RCo{& z4Lt$}jjrov+M!5A6p`XVkFTDex>$>ggOu{mOjXMW zQ3Q>Y%$tBM_ee_6M_Qf-oxbw?jON&4NXj>71klD`M-nU;t$rbWLjz=RErB~d8XseW zA;ZmXvpSp5a){x)uR)C$Pifxitu~}X;WopsW_rdd+m-GN>2Mt>?S-lOgYNIWqIk*t z#th`=DHwl>S`7p_!8N=DY5WR-c-=h>&lOU2?dcbxC6=;E#FclulO*g-*$aJ$El%T~M7trfZPw#iy=E@ZBk>SDn-<~!N1qbjbr|sy%4_PQmhXJbBRFHL<_Q3lj&I)z< zw2Dn82HJ|hRA$6eT^q>GLLm|#G}4NTK`gv7ksPxl;bz*b5jm2y%wEK@rs#L=LDNU9SJs(}6rj}o0`RT2jHS^&JybEf}Z3^&z?(+D-M#mLP6i?X!3O%fE#(GQG1 zW%c4NZOKH6>3k~&fd~}R)7R05vdBJ*R4$(#ZgL! zj7>D^XgWfLdX@fdl+#-MCUEwB60!b<4B`hSWKC9Qo62RlSd#@rE~ocQs#vCUShmXQ z4fMGhUo$~z7AcjCaA2G}fI|L@Y4?F$pfIEgH)6a2=r|bVSI0=?9NlN%BoVffQWB6O z-&h|A@G-3%1T>3K97%Va8#l68dhB&svxbGe-*IYC)fFPG_A8p$kz}fh*^g{0nZM0LfixZ#2N2G1a{3uexP$PM=f0;tRm}vpm5^B-^-X z#3q^YmT)EQjhb=PUg@7v}j=_8R8&F7qxPLCpm&k#V z^#A8NX2m5RJx$SXeEgP#1}ERe10xZf_-^BmAe>z_`pSS<7nLJJxplwGy+%Mg{T*?X za77MtDI#nwuQ^AZq3KUHSn(vH!1bV`!f}_r@28B?|9WHTkr0^ZP17sgD0+jAPOwzZ zcAV1kTfPnJ3kT<2q;i-F8)%17&EC6J$^|U~#`kpBir>OIa?~}bZc=uL2ypU=`M#Uy z{gnpK9yri@KBx~tJCh7Z2=xr*{+E-oEJf_+piX_B%(TRSlnn_rtR$s;eGFk?VJcj0 z(qsorX45a{lmKhGAxi?ugfm6)&BH1i&(YR(hqj@4;~aVZ^k${MP7{+*|Byb{g^k`9 zr2g>OzazX^zvio4E^{QkTL@n(@pgx2*|V1EcZqz)L*D$Fwwv`;4UZ$YA%Kb_N$B}X zIRRHTky*#1>sQWD5{s~b4le3E2xbWZ>QTERY}PO3aL8%HPl2H$XiVysNFoX;EX9@! z_BMr9(cMMDH0At$cv@GJ8qjs_4bA29M{Afwt+7&zn`4}7Lek-(PPJj|y6zCZ65luN zQWyX(LSU7X8VSyHs#S$daXLBNg*gkPw6bRiSxFD1fZzM@f8YK7zhfgz$Lo9cf3&w}Pn3qbA~69S0U8<_v9c0e3k?lD3H+tu;eaO%Y^{6XAG*7i zq8wWJFvAA;fMqB9Oco8TDuM9Y0vmkBcTqBMM?)j&Lj9rlIe)fBLyNIfhRbSuneJrc z`rTVOiW~`nM&l`^@_$yU`T@sk%D>gV=Sxlhw(4f5X`iIt3vgJW}# zg`0bzzVGv927*Q>q@wNC@Cj3baX`;Xt7PiCz}pMO$DSLo%*;&iF~%;}S2vK9^vQmZ zbAsl$5Jcz;R@?1m35jX_tXqlm^ObJOuH|LxxtxI>-@TSklXu5IpI@EQ=rB@1tSWvt zdw*>{=%qhwySsUQd9uZsZtNEI%&0-w}T$)Euq`H#TWtWlg;r~ zyO^-Z$U&q&*K1LtOEcc~2F%^CN45sNC z`*(#!Mc+gxjSKoJnsyr*%f$o)o-_HZv$QaO|I^UxIqR)>dA3)P8@l$_doBBLv!r6v zkuuU(*lXY1|Ml5PgVa17GSDpGqA0-W;qFDV_t7zO@cdxu#M@Zn?D}NVY;vw*FpbaX z*wd|tQg7XN=FQJ%$%%=1ZFbaBxVisL&ij~oEnv3<%rcXuK9{pEvT)=z7BdH|*YqIp?kw=JlCj6Q&Sm5^ z{j%wrw5^IZQ&XP}_J)%QqZ4DCs#?d%%_HZgorixt`&lB_Wjf8jEqxt+tQs$M`LbcJ z^-3H1`|OWn#fH1w)2Amw)8cDc1$)!gHmc%>{j4!FQa$ov!L7b$yKdmdZOH`gwVbK! z{HZ>EBXr()6WY^wn<-<|7|$Y;10YJ|^GNH@hB}zW-02~kBiCB zFdwhgJB+I&9JS&ydF3_VoKE{>A<_G$)Sn!{8x|tymv@`?(gyjP)XrSZkWRDHjIT_{ zW?KEPHg6Vdr+m(K%R(PPYnpNP`XpQLZm&*+&W8Q@ES3B|eth=d7a23}l~kX2iE~U? zUn|%$&n%yOHOXqG7FRxnT4(YkKKkd%K{=7HTX zgMX}OyA_T-H4fdW(4eRI$*&hH8CLP5ldL^8h3UMvssuC$l^SYebQ!Yh@Kkzw)MP~M4P zo_No{9_dVhtC@DkLd{^i8c9pgkZV8`U$D4XgarLKG?;RjD7}w~L$uFG>3{Qf=MUx1 z(pitAr7QXzx+lhbG#-^&Bne~mE~~i*%aa+3^@+0oTJ?;Lv!qs~pvfNbpTBmU&~`*c zA*%F<7#?Zh;@h^K@BNh>3rF7tf;%CblIeVPx^r?}KkK8KKuY&OiFt|J&EiMNi%;>q zhBf{5ipwE{{6A#r(^b^eVx=Mt8l{af^Kr@%<{N_MLSCW{zm)8*k#oOi{m)D!BS$KE zGVI-F3lgL@r&+6AmlSFr(_C{r&*A;~@!3yz!+NJO%;Wuqt{5EG8DpYeeaBnmSv;j$ zx&C<@E^?8ao_=fktH}5HEwPt*X*5*Ki;n_t{pLn@U?Gm9HUla59psqY>#X zAskE`0T(_9RzJ2FiEen?;L`Qch@Qm(H3Ww_(B)#DA43oBC5s|-osIzvtUpl0dG zK6h6gSLfe|JC}sYYhxjEW`0%4wS2;Hx6Wr#SINP-S8dkc$mg06jayzhZHO$FPJ9Uj zUWJ}9eCAt_s;+KY>#$ki&C0YD77y44DG}Gj?tJtZ*&gCA@w6hno)Ir&gZ#UR^4K5? zqGXWCGE83>Odt08Pt9nq3B)1@^LFh5Yx0cB*!g_>Mjp!(()n#(Z|1U7@?=PCZru(_ z(ly`ziZMM_8kty$gIIErnlv(oR}%eBP};`#Yn8sceqj>W%!#3L9XWXpEP$U~lp#sC z`#B?vT-L}94{^v$Gmfs3J#?}8Q($cI`6llovp&p8Fz2pHBQxkc1}}e4;6b;oS9ajt z&18Kao~{}^wq@~#baqnqUPKF2@S z1Q+HFXM!ZDzDKXj&No4tfpg)D%}pOF1pdy>;f05#@7 zA~WLlNJ_(T6h(h@#0Mu$n%Bh&3hFPCFS*yf9FE;l zl_ICFh1C|TJY4sQRH!2!HCb+UaEeS_TN56Fv*9e%ceK<%M3OlMi_E5!q&#xu`(D^t z=Xl>u7?9wSnCK%H<%7r9^|`czr8+#vdyFR#_>w zQ15cr;W8gavpU@`4?YuL&8WNdev~j*zys5F#(SVkd=_mrBY-uG7l67gmaE@0H%06s zo|~dOQQKDWWA$a)P#7~yye(~v$S`OOHuF1ra>`6IB?go@EFJgzdr{T1;3jf{VDrS@ z#g0F+HOBfZy<3M*0yE^TyvcrWp6?h^`C?3-bR|fytNll5P)cJ(w&FiSLuQGEbC?b6M60~?9O|7H$UWnUa-))xulmK|`;BeR zcUkuW&X5f`UW@o9iS0t=bN?+iCrW zM1>th`8_v>X6V%4&Dmivg_Xh?`LI}B^8c>wUYsjrQZAD3lXlW6D4R#k>vFetdzx9S zes@f*Tc{Txxez$tb)Cd&LXhd(T{zo7ATw5YHJal0^&l3-tRgVK>WsZ0`XKgzb19*o zgFs+1BYnOLs$kUPmh;{Dh1~?hAJ4cJy)GIr>*Nca{_K?}@w=U|*X~%rg8ch@V`qzv zw)(&`xc){5QRo!CtVF#nMCwh*%c-fUOPXBat?i3N`Une})EGAkNXSbz@0P3Crbg}_ zN{DX3Jf=seAF@fmW3e1hh^R}Dn{)*R% zkej+G+y7qIrS81lU{4{_Egq*yuII5 z^M@N>pI6%21i3YA$@gUKYfj6fTXZ!CfeXERqF`kMkrwYFl)KL8qt_q_5A3Jl1id~bNLQ(bJ2H3K8Ds#f{lJECPH@hm$GeLY2S{HgoZJ^q>$$zqUGAuMPjGy|?*>?bfB z{$^S9{>4gqeI4{>=eLeq-zsp|)f%uTOn~<;&PTma-U?F;YgFoE^vtro)Y1wT{@E1z z2=8MWstc)W$y?{*i*U(BX;UdxHs-BcO+=RX)Qf$PsD#0<9m!jg3-_yc+L z=~qZ#X84#O1RM;PG+Cly!Y}d>eg2?bzDzfoPK`Jl)}A`=1Z&zdf7| z@hEthGMzap7!noOcgIu9^Ev>aBR1wrn*JSnbgCPQsz0BZ*a=VC&2$O*GnO01nbUkvPAN{3_acFJ7Aza?nskR3I!viR+U^qX50A zH}@r4u0>Az`Fi%k+3=vYPkHp-?{xR{?EGj+SQ$>C>F@8~-@$jfz1$Muv$EQj;D9OQ z(HaVBllUKNI1{lsO>R(fz?v+!ddsSx1T^r{*1{hO2wYW99cQoqsV%8XV&ACN+0r#6 z@1-OjSQL9b1`oJcq4sCk-j~QrEw0)F%03v*M;5YvmcoE}KR&9TUq7>2Yf+*{S$l~q z%AScesf^WTS8sYcU{}C;O=LdD|8(jPNlLr)r1x&an!^;ek7VVj`z46VCjDq{J+I1< zg&#jUs1Gp*URQMqVIfwP8IGJOR7up;)#F<^`(D}%THS@?W_4{fUWglmAm;Wvp;E=_;P)DkKU4;i{CGfnLUp3S6G<(orO7{jPcWuA7mW)4pZQ`ukn4 zl27iNZ=31tA4H$pcyE>uR&3vYvm`Megjus!BYiFWTN*^3jK3v3HVe}TYJ2(@?FxY5 zsc+ON(e;USyF@w9yK6^dpThm|C{Mn*WOno?u@*V~ZRFPeo9)iudTx0`elBL{vy!TJ zd_hJ}Uo!IPiS*vkk^T-2m}Ui4%QDSPk2fRy$8S$y5@dpckV1*+mZpb0esmozMo z*0S7xVL%~qIHoC~AJS;})TJ+$DI^m@QuVQGXnRH90$<-j4ZHt(Z|dtOU2GXOk|+D^ zFlyS0C_ERE{c8|=>srw_+okiH9M59-cYFWEW%kpkXnGn+^2dP^Ibxg;)qiUNv($cY zE0j2Gy1QidBfNLvC-coNs8kOqK5_agH6!8x;6bC3cYopY5shgFIXo!l?)R&;D|Pp!!oZ zL1%}5Uc1dZj%UZl>CRb`Zz{qki_O|vNvxNDYsjMF|B1$;ISy$Num>m*UjFZK*W1xJ}EYPGJxkwkp+Vej{X^uy&*#!*7##|= z){6r7@835xnKFZ$`tFFn_I~wGCgnBRpl4poMeCBD78h19EjlCk;oe|+VwW7-UA$lB zc{e;t>aiiY4@dv1&3}HJJIwBD!{Z&-b<<8qI|q%0mHc#KPNlotleG_C{(#QW!o+61 z3P^q-f&vri(akmb7$Zc3yw1BW@JQDvsUgVA&61VsLH>6NadgE>Xp@K-hu@VJm%n>F z%0{!3-ER4J<^(xDD9oqRUJLJ(*^|%lnOdsHCb^QvzKe@# zj;kIZ;3YDa!>VHTnn#a&ra{HbgH2Ov1mdo>AGg%?zbqmdIh2)^8yg9NEu?op4rfbH z8=#jWJJ~&YY3*KKhdjcTwrbeV(dX-t$NG@>)ZnY_Q1sq8iTN~ z)C$@IE_*Dr3*OB|sG!Ec5-AywLC~K=^|1JUheTAmV+V8PEo1dV`K$e1Q~0+!`;t zx3;WAMk%3Lka5IZ*Zfrfc~^{3jpKGO+aXL*wV$dkvTfVjg2Lrk!&y{AUwNvOphZP# zQ7n(GC)6eBp^TAcuJp4V&MF?^6k+0;S2{5>&7I^W z(Z#E+<;s++wW!_*{_df7T^WL0O_q7okh93H$F*`2u zMc>VTZkoPC=iAM14~YEdZ=boxU(AV6@1j>2B!3>s7Sn5cac{6NAj2`+wcMzHNVmeY zjQ*926k!@?pSOuaLTVU5I0Vj{DhEVxv=}Io&^po7Kgc&?H2&C+GSru8^RT`ARh_Re zVdUIWbVa*g{6f^c9Zh^8jH0HFO2@M}QhtXoUWiMpwa{~K)-Wb!ja93BFxKoEo7y<9 z;u*fzz(>I!V|`dj^C#skr-qV~%pjIzyYDL3Kw-Y$`@Rv_l9iMf>Z8G_V+TaA zGbTTgOaiFt(~4=(*m3jRs3LP&IEy#29&e@JdP;vS6zSycN4<-k8eZ)1-9wXVjvT03PFIMUKIKDX^dPgMCe7L%pdE>m? zdJyUU^O@L&V`N0cmq#w7$h~qUEsoWVPdQ_`4R_a47dQL9Yo5=nsu#hT`wsRiv#wNf zHp?DISIZcGDMG-h6b3||<|a`qw(0XvyLaWjOL!-B+G4Z;g%K%8-yFY_vVA*%b_Ec> z&3V-kG91%Ib|S4T)gc}nVR*1psJ`HX{DFaivGP#uUn@dAw6rPraYxi1J$YhG$Uat- zNc(uP3#yvnIh+381aG9Qe)bFI<7^6XT?RdPkmH9<0l@-k;NPAM!i`KGez^laO}fu@ zoY?G--Pm6@i+{)sH(IrEJkpyuN0r^GJHpX$Cyk9kkg;4*&Rv2+iiEZ zKb^w#DZlRqqbH(IVCxZ!IMrVYh}gqJJL1Hcnaf`}RZmNq>a#59sgwp7QBFwEFQ4W# zf_@Rm$oZ)*@3^-zu&xX z_a3I>5OzAlV$?horcZUtGx5@DI~vbPtovKIW4F(7E>RG;4V*8Dy3>`;Us?AULHVgD z@S#p>-5z!qp)!K1eBTVsE9KxBwN3Y^lX3%Z4RCjpF-E(EGbGcGRX=6(hx*!YtHZDP zAjaa?>Oor`CfYq6{-+bu&R0xPjg9<`>#&>5$)`}Vm}D2blc1&;8r-jeyO1-g$vna| zG@eNpUr0h8p_QwbBrjx9o?Ujk-wf|1yB|msd~Qm7gtr)af?+I@AEgTUMefS0E@m1+ z8S$fOtqBNy(^D&G0yfA0sYPz~>#gsFF`-<7s=d+XQ2Bcl&JHp{6sb@Z0%=cwt+w>*HDSqYo?>y-Jr`lkqnBN-oOOW7Yy(Llf}X9X~u2W zp?TfQ<#*4Ig^c|UdQcgc8(Q=ZlfEsAyZKrOFODIOe<=nWqPm8sOS&(s_4{woEEDD{)<~_uG_^c+#H#)KPcuci1pSa4EkBFDDlyJs`p^H5t zH6)MX-GciM7|p@jE%};^aqw zZ4&yMD7u?0=*+f#wN^Yr6(5gH`fB^@xC2iQ%tk>EKeiWN3gOX*nww|#g*Y| zh!l?ZV-c7WRyDpLW=Vc|+pP~{qW#7oTEaAm3o#1;J6{L3t{jV;UwpvL<%Ucg8tY>u zftmM(4JF#5a`&@qipR9;;=NbG=U0KV=jUD~-aSIRBBd0QUsykOoe`x^yN^hIqQa?y zmtNQj|uMaziT<@#fdsFyg~$+k;Nog&f8{=R+@FEX>+0bpS!oWS52GxiyuHgWwrLg@4}Xs?rG@9}_G9>l=o1=*lsz^#=xyQwt((Q8jP=Q^C7L|q^~a)R&<3BQl72(8{HrlPl=KgE^!`7hx zQn1=WLNMlX+GQ9<+Dh+oEMZ#y_?Zmk$3lsVSm6?cN+|`$ZgsKcYEcxE8st-D0h%j^I%Td_PTsV=#K=htE zoZHAeTjym84a?X6Y=%;BIk(=21O-t>gv(jdNxz$k7+;+Dng0Wt8^h~Lz}%?p#~a0w z)4isUQ8i|)pC?mHHhgx+=)#wB3?0ngx?jA*$37vho17{;A(_U835{X9R;|7l`PgXA z|15jrKg;`KyDEC{^H)gz2C4cvg!B{K;Ke@%{DU6UxI7bl=n0g)wfCGWdpxRZvV?_h z@x!eX-I+7KSGr2hxSI##1`5tRP5bl$6G!16oC+|OMNWdd)gAZi0s{QJ3`vbLF@=hI72jt#loS+mNh?=$%K^0ZS@IT$>79`l&sO+r|rCg-n%WDHSuEPhQ9l)=&IN{ z-<)v8nD-J^5!$ZxHD)AM2TyyZ$S;O4u6-sy>iA8^j>>^TdBPm_P_;37p%hKU;K`9h zp)R#(-6<7C(k6CE)nxB;CKbOAIW`3cQKJ4==4P%!&w93enu&VbDnz1>r zHkv7}kmWvhDJ9M=gjM0?5LeEcI$B2d3&dz!XQlM7RJlT3@sR#2;on-@iMdnP?+!kD z^t5zD&r8VG8qb)`pPswO_cFu799^#R&7b|~jv2xGwQGsqTUR#yMhKT*dea*Hq~~0{ z(S@1J0Unmag33mZ-YCvEj(;g?6DWto3{aZx61qOwJ(xvFpp486P5%o>M^mh)9+nz> zAXJhv=~Gr8lO4`3KK}jOAzEXJPle|wuZbvQh+N-xz7M;Hck;Dr%p*ZU+e!?aC$`t+ z>>)u&KKEeRZ1fRkUeah^tOWFKGW z2~kyq_b<^M1v2f2FCMwH=)ZrJrMls6d+U5@B4VTL#buT} z$k2N6qJ=FYmlAmB3h%$(Jspxkqp`kEJ#s|GFk`FF*Nx-y>>?C96yegMjmh-LB~O>9 z_x-%bb$Eh2*7x64gF?v?ka8mv#BC_o|vWyllqp27slgz%PzEfe1kzPk!_ zjOj@9jigCogCXf@m1t+KW!dmH-CK645svLXTtnI6?8}CR?WMVnSe9bl1I9q3l~%%O z3SBiDLZT&7d^M8++SPHgGD}}7hy}29LJ?R9zqX3YRf*Myi#0k(!Q;k@(WZM4Di*S~ zP7kF!2%x~&tqu!EQ#@oZ8o-m+nZ}EJw+u?;^)v4{UJD<0d<@`_978 z-lSvx6pUGOBR)2;uovI!x^jPxWUBYXgN5`TWs=$RN?@hTQ4-3LN%B^<^uP-@$se=V zfF*nVK5aW1hy~zD$C&v%8`lu`{TWT7<6A-K;+Y~Z%{oox40;5upxiR4oZMG-I>iOs z1%^Oosjn}2DR{|Cv4>sH-1&A19;GcbQW3k988bgCa8ZN=ve;sFi|Dfjmk_Fktx?t+ zUUz(HFN#>{piUUsj;5Ao&1YZ@=&%Z;h9pEy?$aTNRKsIJE31|>+>yGoggnE{<&b)Y z-7kw~0T}BHR``N3lrvYIk_#<=2imun&KBDIXv7%4r8P?nWl9$RF#_<$mv1D4UOWwi zgUrCE84EHew-@P5J9{VN11 z_N$DYe8%SYU|Dl+b~6|p_=j7H9;&vhQzJe-Dc42#&_hqVW$g|dWgglw23Cq`=7H4x zDK3q5TR%*LEvEwZcxq>~RnkRSNv>AB<6n6>dLZ_y?gKKqa7iXd&D1nCqfc$jJhJk~ z9;w4n%8a8bJXb6$YTZvZm$VrdrL@v1omAO~th^QiLB~DQ!c}c$Z_-~5`R35fDYi*h zrWTo*Lq;NDrzMVu1oVw@sa!cqI~?Rk{hOg)R%V9R9^fuul8A7q<0Dii%2CNeRJ($xU7FIH|s6q$!*| zqNx0~3tlJ;5f%Nyg`-6oeSen_XsR?`e0STc1aXn&PXjhQTwa~Dcy;xKe8jvYI4(2# zL^BA2ub$?4A2<1X(C-^b0SXnLp8B8;_nvUSAep<=$yY9=_0tY$Yfwz|VxVd||M;cG zJ%;7AMGYfDM?6SJ8JyBE`k~fH%$aQaI$HnL3(n{*SED(Y!*j{nmL!sui4Am&Upb2& zpReR!->(m1gNA9-e;nZN08f(_q* zE@H85jYh^%7ip1Q-tI46dp_;t#jZ+mjFzwz1w|sHvv!e3%mWOYF$gJ$G-E+3W7re^ zpmizDTM=FvV*J^23u9Zm4HTbO3M*k|w# zYYkmB%&Ocs<`FJ`Suz~!KqkY>nK}&pjK+*0UZ}rG1412xp^6Xj23$>lMYxGcIpRk& z)fvqidYQ*x^{A}Km8B5T5c;yhS?D+r;Q0t0eJ3*i{q>G$99ehWQh>og43QI1bOMWP}uRhi+q@DRJ(^VEJXAvCg+`cU`>*IzstLe zGhN(LvYqFg+yfnPg$?&j8xNGFjA37+~a1ID&j7y1Fak}P5(4NPJpLK|0 zV~yGW4#N%>g}=hCr*j!X{xD9b>~d=L7l=cK3r&3mp0pQV5lexMx61^UyR&DJND;i85W+De0{Vx&ld z0=L~o7LH<34!-<=oF42k8PcJem+QLZ!oTxNTugcU+Bbwpxc4o%frPw?tT~V6u$LD! zN%gM&>Bo?JljN>{^%PDz;A~#|An_-AoDB4jC2}`exIgX-P)ja(-q2Qf=VsPF#s4oa zf&ZlHr&UbIItR4r;~x;vWJGAvnvY`58L$d|4muB6@0N)*dBFz{qf}^UXfQktrBtb# zqAI25*>H1nQ{mOOPmvz|S&uu)Om~skBXb($zyd3M1Uujd?A+d!HPzPFb=Vud%&>3i zMPt6f%Rh?~lvDjr9+@wREOm5l+PIebzj930rt)Nz>}r9gx*dZk`YrM+o8#K!>kv+S`LMJ=Z&)B|~re zZX~u-y8$u#%Z(sleq38nrqRO#&ATzF-~vwpumCv1iEOE5orDUM5~&tmSHHXv|A2+Duj390E$i0Bgb%n5L$tuQuH@ zxJX|>Zu37SuQ|lkcZ_1R5DpG4Du9bAoX8)^_(g@0#`?9_PjWf`yNSnaLn_GyIpQkV zVKfL$SMXYdX0dWsDi?9i3C`#qKpdT?=fif3Gs?Ly2=vImiu(*ed_mrBY5dFLCbc&lqw{2#Ef_p&zVs zm)a$ek7P47*!`AV?IghX_xRp9?kFxXJ)2S8>jOq13_u39pphlT(AuMQCcDX5unoqd zK#hwfdK(p%A}ka_h#D~j0RQ*L_8TJPqrw#SQQN1W(aFt=V#)}wx5=g{(G;TQ0ePb_ zP&oVbL|bO}cF&10A6MvKSCkJ@+)56r2?j&fJN`+O?9;>2ov5(oV11#U`G2CB2Ys#n z{((=qCJ^G!4FCFa(X#vGqi6AVBdICUkf>1j->5Pj;W~Fymkz-*qR0Fci7FAO>7$3j zJ8?$5N)Bcvx521-a;Syk;jxiF^r_Pv<>m#C)c>vBWo)fbb!m;x*F^vB@5;2Jw!**1 z8v5ZY;4m0`KK00Oi~rko8f4|B^cr#goX|1JRP*bvdk9T~f4WAF>r1#t{Uk<}p}r0v z&jG?!cFmN%M7PW-=K)?}VIyN*XY&YxC46 zYay^iZDNBASxbjI@h^;qo4IK|gsJ?$F}{z+hqXF1mv#yh7H7nkRsVHZR z8p0F^8Ypw%V0`>0F@l(W0FXR*0PvKK3eTu`_4E4dhO_S1I*cbzlo_c4UUiXai_ak{ zRro(~ql^iJv0#7|Y%0pYt-<%GDj0)m>!??MYLuz?jk%t^e~{D>io10{D`KkzsP{T* zprcQ^1b2J!pK^Q#z|L|bO1%xnj>A973z1PIZYuxihQ|Y1eZv;Nvl%*k`NvbzQ~<*F z|M>8%-`&kwt7QHJHa~y+iKxrGjQHl~0_&yT zxW*Biq4%OFHu<1)+YJXO@%cnVBPaukRrVwtXcP>V=90*$)38V5f}-&{gM;Sl{!kjT z=_-3LX-U8EtVO+W)Q#ZDG%Uf;nD3DO6)pAq#g*X@mr)8!Ti1({U>ic3Lh0?>y6c;>xyGWxLXb&)f*M?ycj@(I zvKS&BwKOiCQ&srG*MHbTafrtvG(q9eSeb5_p&O`&8UV8BXQ#se+Ej^@cDh{!&+M-e zb3hd`tIc5KOibm(1remoyVI|}wPd3bxZcg6I-zO$Q}M>6&Cj1rr=O>bZC zWCtupac(Z@nfZ@q~0Y+U280HP=*6|<1Y)&;wfgsZNkDD_tAP_p(`?MPH2o?w(gPmKyhO<&YAjaom_q<~ zOx^{w7(f-mwN`g|lZkwf*I5b_6Ha%EAEA0z7!Jt-DKw8ZGfw(?aKNAS!IXe@UA?kb zB!h%(Q35runf*hHqcs3Jc^gpn>bK#64vfgE4Q_q7{0)^;^#-LaD^-0=?>KY38 zY#o$WG9K(2)2m#fXGjUo7snDSSvVmIiyYudU*5HGnHjc@X@#!SVG0p;;|X4|!R7+* zS^!TGbFLL#smdNpsKWdd zC!_(8V~So+*S%+z+c+~6byNZ6Ml0t_3F%!V5^5w;=jJ8ZHZ(SPLS&wl6QXTqRzRl` zgYilSV5>>*-m!9Kze!$RUEQ=geg6J|a!**J;lGWd{E(6VpHe?Jue$|pM|VIEJm7MJ zb7QP1W^eTE<>^j|M-4X}YAHei2VV9!DvlM{htr}|aIuhhaIY#W1xJM_xum z4$bjF#y8XBK+-k7&cUi}2C1Urpn2;8nPcEbnW-EP<+nWfUDR3{0Hyo zL>>Bp+4)i!MIxo{U&GsPAg*XIHR&0{hxD^Uf~Ue**MUqo)5v z_D}O41uLlDolh-Lv??6|QT-@BE?^ECKHUOKn{r8PpcZZ!kSbqjq6H|B8Kb-mKBL+cP~A3I-!}-np(1eT zm+0z`7O+qu!^!|p#WAC3>;O7{>8gNl%zYSitG@J%Q7CpErAWBBRo*|ahk1K@(W=k-CeI|-5!MQc6+f#Z2;4!)rVp2-mvV-;mU_x z_$2~X?NjW2L2g2zsj-*8m{kaajeY|(FWK}z`wK{xV_$T5HpWZRAH4pf-NZnZ;C(OB zrUlu;-@GqpGO$G|1POaN{zd1KCh5^~;?u&sJQ?Qnh#q+C0G2n@3HG6(GIn4fAPNsk z1<14MD6;VMQgTl#r!gQ$yS1a^_Jc=x=-$Q_`1trfL}HW|9QWL>N*rUk}>R$-q)u@0Zp%C-G;fv+vg7$-Gz>m~gM|U7^9G{VBZ#oJ{ zNE4;s+{-W6?6$bg>z8_R&V+KcNMnBm%gORlo8+PAOTNs+X`^akb^ZhK zx+&eM`m8K|VZU?Nv1%RGZ-exE^!A<~wILt6@uMPo#Ls5jR{*ctxiiS0> z<`GI8wZy8FkYr+mOi@$?D3h~=d@^z9C^-ntYfJSZd#~`pd~B!xj29=19O-Rb1-uYn z#ZP#0xQAxY%&V?juEcxbH~p%FdwO#`DmmviMt~!E?SHfV_HAg~RtsdeyFN^>RN4-` zNBL8mOVMhNYuXzjcm!!`Ne?TSD@rQ)FK^upsz!A!XvZV7vJQaSZ}N|c0{q!QP{GBI zqQ%dEDtEW7vibp&;SrRN)`#c^^z(ixU@X9JyBJBI7B(&TqsgJ_ z7m5uG3@o$xRdtPTOEw3&>Fv)I-MGomY1{7*r6x_ftv9De290lur9NIuNKSwpe?{=A z%&#E~&?^5&0JNf>a%9I89oXd0<)Fni@P_^jxy&|uU(%$f_AMn*n|jHLOl)|xmQeEw z0H-55C;(b!>bT-zA&yb~_Q-G!()Bq2G<*X5bzu;p%(0GA)r0c>BT4d||D1~ELa6GJ90H|_F^V05WDM9-3pPQ)Q4dT?e;N!dE zHhhG|Bk+DKgSZym!&m=o9jY4FZ9W)R<<+ZFCRV92fYf}2$8uxK9-lhb2#}KjMH7+k zPo4zqjbp;`pIXw?P*N#<6Qk+6*FUXLP!ON}Zbxv}{4Wz1|^nf(~?5?T}dz83<23YV802yee0sGW3o%92e zs2bS#3#(gD1+b6=Rm{i6lxJ!1N-6`IzoQS-{&CX(@+CwL>0)K5q07A+%syMZS2P|_-Y@w2O{+%RM0?T7)MRGte#+e zxWvu@UX4UKTshxVIeo35c>Sd58H9s^r>J8!6!Eyw7C(Ik5V%NHo14{>R#p(hgGgSX zw#dc)Lyd1iwrxHu)KM%G+mZ%QZV;M$A2KZeo1FfP4L05qbnT4ln#94P7~AcW;_X@2 zo%PAb@qfMWp?a|#sJ`zu^Iu;vc1Td^)GJW%wM~k8Zt>^P-fhMRc=00>8HC$+XtfAS_ZSL25B942yApMDomdCj|*AyyqZE z;e^x!sN6u`@Z&9B@tvBVr=P#2f{Bu+2-_b;h#>+F&Zv!uKxmi<014nn@}b1%czyJ# z&*^sY>$fJh-!nc8mCt%D5qpCWdZl*K^d0I)E&e$^(Otr(^>lmkb7s~UARbi%XZG6A z1JHbwD@uMv*mJjZVlY+D zbVI^HTaQCvnhAb4cb0~0NwcD?5v}AZr;v~aJkC8bKU`gxGC-@JC^sp=I08`#04t0a zK_f*(FvC$qJu`?@i3+aqkq9DClMg)FEnjV6AQ0S|@aRy4zFOgB2zGu{n5WNj=N^BG z&VRZW@c?Et1yJ=z^=$53|I^H4$%2ZS*VDS7N*>0l5c|Ul{5VA_N*>86DcM2!^;c8# zTq*yzZg!M?x^_BdN^INvZ*hT}oDFWarY7hp`TucsmSItT(b`uUK_#TSYv}H9=n^D` zmhJ|TR!Wc%h8Co|q@+Vann92d=@u#JhIh~Ze0k6L$aOI@``OQ~wf4GyHypkm!;kn8 z5D&I{vtwDjhi?D~!F%=qnLvYhn))sE%48RSwP7YRqY@Z$Ag;etU~h=v0_ zmf1g1>O8m_EQo8A!?h?wm43ezhg?$!x$QSdq=uxC|Fy>j*(XD_Co#YA>(uX#S>>PO89!Y_7=N z<3c>V=y#+F8^}Ld4TskE)^ubLx0}WWHCj4us9!MeC+l&Sj^ZNKoe!s z+C3V)f*R}kWblydt-=Pf^W1DIGj$nhw}JyZz_P7l0h;&-okZ`w50C~NcSwyP2mK&& zBg54*r@BWRveRw_c*h~f!D9L1?>*v$k9U8oRzeFQ{ z3It{f$m}SqzD?yPrq4^q7~@gzU|=aNBL=N%mc_db^U?grOc3Ndqg6CwAgHJjZ~%>cmw7uiyV=bb z!;Y+Uu=&Lf0XrO65UsmMh(I9S0hL4c*(q2yez__!dZDk6aE~Qf@1*`WU64SXGaTgC z_k!LkWBQMR;B#s$x_z4{fZ;KnYHNUOFC7@L0C|&8`m*?ndXJhg;lp!%^Y#Oy@Y9W1 zJ+*P3GPeJJS@Z=8XKRr4>tLVA^M>Pax4k$zItox!?S~dv&^|5r z>Hm2$JpAo{y}}K?1gHg!_FHzih&~`eBFHq4BGhIqixWv3X#*h;G9Lk;FPweY|JhsNvIMBMaMwDTZSFj21mg$lY3*0KZ; z?-?Q&s*n79d<8&czz3p!aHj+GT5VBEqTu?WPlocL=vp*CNE7LhH&pzlH<~$sJlP#? zWz+25iBNU;s1xP%j160o&`N&j_4)2*a)$NDTV4V~X&w3}(ec$+=6r{TQHaDqmJ1l4 zU)1`-=i)8Ig++vuN!^a}frf~j;NFHk!8pX2eg~@Z=Cj-2pH!}$t%~!{)eJ^6dmB3k z3`Tll=T(g6;iL3}&*9)Qe;S@@*3wvFY)KE4OxGIpNTkmIk{pvEEUy3O1&GAG|4t^* z1YO9|v;uvc28a*J^d8Rv3J~}0_4zrK;1Jxmd~0x!CbNP!p1)HBBm)&*)Hvt!qojWO zJ?O}E9$_41M?{BT$zXL8x@NtWw^wy+@2Zjb=V~18rm9XNE8CkK8NO?4m99W^f^iYKq7E_Lk2^bU(&};!VM;^xYy@*sdU>A|mHobW z&DL79veh^r3#ChYb;M8zX!I=zY7mEW9#I|EE-#@I__>v=;|?UOuO%{_PZ%qU2+2-x zt1NMmBCbd8oEmzw@xElU9QH!f&u=9TcxmQ@oD*lW$KVAn$^?N<^)a3PZ50^}b*J^t zJwgW5b;c?ETuUorCMgPB$zl801XJuP^mEXm3;)mljc=~o=Xe}LW46T=g32#5>YXOE zj~<06LAhFY`I)?bVa=Y7n|T zdS!j4GT6?nzp4?N81nv+)$IwD?K%t#{V9pkE?W)343h@tobL9V(PJmWCS{A?{YvwP z?%Ee}xoV^mvh+KODu$f`vn{X7pN!ZCPB7M3=k-p;B#>&xg!y+{&yTJ#<^_dS44E-e z*topV8vOH_P770BIf+HZVy92VW=So|k*fGhUD2ybLq1Xc4W&)dbg_S1rs*8xm3h`z z?|+^e;x*4dY$kZElg3!D86UPr$XH>X7SkEn_5lYq`08>S9`w3}kvg&DfY4x=tm%N9 z+gveh@Ces+-n|c#pRvFu(kd&R?d4)7y}_2P@}JyXzO5ZmKgxu3wOA2=FN>8Du(X`uv;b73b9G-eZzwI5p{~g;dx*l7dtmHFoOgaVqRa7y-k$C=Bl3X%hJMn*V~e5^&3`{ z1Vo!zl2^H#B4Rl2MgJo==Bd(!mrSMJa46MsWGIUsy<`HP`#)%u9XTCGzu1hBOJ%Qc zbYkY6D1q;*f?nU%lgzgABgQUCF?DJxo;%3xPXa2nV|Z0gwkJnrAu z4@pcls#ZIHD+z)F+G5&+Dta3u&rLNKM7|orHPp0c=F2j53ADuJCj1Lj9cFZ}rN7~S zH|kTK=XLj^dAI{9lT9k{QctKy*u#T7S`1B_j;1w_7hC5uJeismYYV(Q&S>!1w7m1$ zTsj%Lt*3tJ*4mFVM@fq5F`Gy$--#SOwbvTMPmnR72Vo){aI&tbeSJYXcCrlib?7== zSyM9=dEpJ}Nlx7DVXxYqWMx{~mgLb3#z+Feg1XB)OqeIa4&$(oJQ;HZoM) zA*I{B6RryT$rH=FKA#2_Kz|eVj8vDDrYDGRdm{yP+=!G&G5o(OD!WOnmQ|$rTM%p` zdVEFbz%PnEm5~Hq0$mQ3+Og4;UzV1I|2E_#i5Lj8Z+gY}kC?oswh zR-!PEW+e*7z%Ps%_zw|TK8>*#yuU?)^ZT^Z20ymHt}w*nec!KiRD;M_;8mW`)ST%G z?y3foZUlZpc^iEQQNpB@vbLXy+={XEvkpVnnmr4qQWLl-<`&O@qEHm1C8Y-S^SPT8 z!+!A}1ugiG=^DGL(d0-!{Mw*CKLcUk&%J^l6M0Euwdg$2IqyFOU_0V5Q;;~vqpu*o zQf?VXTXwl~?u#lr!x{Ii6n{|GWStEPd{x5}h^vWhc8lq)k4hlR9{b@qD8cd5zfzD& z*2^?>ru#F|f|Ma~7$jaDN2oM@<9sxANupDEI3HyhL4Usn?gx@PY)_Iwa()OTH<^)| zbW2dBkPJ@l!yxeG0%ZXnpjbf}oS(h652|?Z*~FC?CrPnTj+(I}YqpmC9+guu51J%0 z9bQ1G(6;Ucmwa@rHsjL?v^k(hn43FqrSIYn38t7;JgUaDW)MU5$pkY38nuW)IF=@K zyh7sOlSGsgX{p}8NVbj2s(q9xPM2_w0u3=I8;f||4%gyr@CrvbL9lUKL00@<@28^t z=M~49Gfsbu$D|i`W_fHwdnqXx#yRr^^%CAV<+T)T21zy!h7`!T{&b}n9W`DJWuV7P00FU8b`ik96U->ebR9y(5=$lY?Y$7oRmpG^+l@Q z4e#OKtR4$&*JT*bQxRe{=l&2kDIbqHtFL;IUmv(s|7jR+c}hb4C2^j~AkPrs0NNL$ z{Pw>Y#~u`9tz#|7CXfr0`&^4)i%Y1b*mBM2TH4zdD^If-@cj+WFYsO11m_iSgY5w_ za4MZaw@6r7Jh&e+lK*B6#Pkli@hoyNR+{i9>d>KxOUKP&Hk!;lUOYW>18SwIckgWW zF24SaV~tWj79PCm+!`KnwH?a)C(p#iYeL+pwuUp3=FM{^Zo3j0zIgHNyD0pZm=^q_ zYRc{~r6bs3uGaCR)?X$vt;OIfi!#Y~OPCWFFlhHJ@M?5_#GOf2YKpfY?;jvlq|I}H zZqt(Z>s0hnt{Q&d{*V)X&zZz(+kl7>Uww;3l}46xc+77Zq1#-}*rwN1kd?x|U0gJr zrrM_@*N+zEj9)%%XM$C)8Dvw-nDv>pxpC7$B1hyn*ocNuGZ@_ zmQ+~v6Cs}5j^aDt@}1L97QH8uddZsbn3C5kjIklhQftT?gaU$0X~+Vn{IYAJMwtEKaS_On)Ewsz*FES?;L)%2sPi>F345N}crH;TV@D zMzxfsI1fky#zWkSgE&#ofQR{#pk(Skz0NN#^W+_O zzL#Vp+kb7LJG_hQnIK$T;+Uv%1)}TyL45KnNPxIQ?1HX`>lVo z{A8-oDCrf5(V9_x!?2aPQyXn+yW4KbRGR9uIlMeZo9yWu*Z0&WP*PQ2g2`JMo<#VP z@$Wa*ue5js82$hH%xmc#!=2`8)hD<$QrGtp4+hHEY*Q*Z@-k(ehK66B1U50-oWaP^ z0u2O{fAj0ghrVqaUd4}Qs&UNg74Wf{AI)d_YF3807!Hj+78;rl-WTqr@QNUy`|z+XZICy6Fd`rXTW%9vZg!FVDu(D==g|=g(kzs z1^(xKexYu@9$c79N#4qp2Y*&N@|{;uXC3-@Ijv{MXNLSIlx#>?h)1#2=kN%szf^5x zX=x^_9ZF)N%C|MA))lSt=D~l$9 zhH`wb5ZG^`w0u-Hn-?EkoSYZ!IEk%I;3`XVn@nh^CzgEDf_rFxXn7s^w28R8C4Gl+ z1L9hpx|XXLHk8KEKT!024zoru+>voD1lQ#7nI4PUs#xD(w>0rp+|^3k)?eOo?-Uw4 zyz0_ywPgEj^+Z8m#{mEHtWi~=;i2_p@%fLdkZ8lA^H7+IJFHocr$i>tNYS6=*;9@9 z(cw0n^bd8+!Lc1?_Ujr0$*=n{XlJV>KwtQP)!Gmi_crtHa=X@Q;cO#F!R;Gd?;cPc z-I(L~r9RVHV#FoXoPSI%%vFsJxM~uCYpBELcRZO&pTnyflqO4?;#$HnXyg!W8q=`72o~Y z%&5h#sFASClyh)3dq)Jucx%uPT@zSf#7RQo4kVIVt=dX(>jPz!h?tyQMwrTGVS25Y zYb#CQ@NZ7IyFmn3=ST;a>I6bFaUAnwN~x>g?Ru{$Ttf+7U`h4R9V6K1>*#taiz;l5 z$AI5D7U8gF^j+0(rateL&{iB3j~;3j&vAJ={0Ez?C!+|<#4HxHZo_!<@o)HS=B0H1 zQ3)AW`j)jrylA~miCU1iM?%R%Wf!trelG(Tj{~dp+gm3haC>J?uRUr|ZS&6S_6G-T zp!9*G?8$=~cF$x;G^~ij?S?Kfja1IKRfMgnQ-)bu+5pp(FK+>|PDX?H(Od1_829t* zEn|g!PK#_)#pl{-iV-PJ1AXSgb9McPbga=hY%N;#PKLG0CS*00mQ3xMEmr5lDSv*C z^EKSV)F&BaaGK1z9mVA~!1dhgeq%ueW+@S^#3kam`F(}%qx9oZ%E4AK%_G-=)bIJk z*KxbJ4}splDjRp2O zK%v$~l|%yeO76vXT9jJ5pUeiezI70!oE6M?|vX^*?^dFxtHq=ejFT} zXmK_gz$@=FGL27`yc<7+`+1kGFJv|LcxdR+S)0O3qr4fDYoN&& z535Ri@BLB@HT?YH1*Y+3suoeQO)ks?U6JB4h)PkoccG(A8uZ9ieWiVo#)9>RQC@NZA9GRP4PBglKF z!6o?=(GW09Y_N-ahI?~7{bk0vs&3Rtf0aziSfVE) ziiKWI0zigOjq2k=OzL-d@JlUT;0&uX?n)LkLQM5rT65V$?}LV~ILVV1X(k<5&FK-^ zKO=4M3Fd_Kj%Wt-^$Ar?2J3#Tv2ie!1r?;!L`q2*az3t2o7$V?Fd0ifUZeb~edIct zTJk(kAT-UYWq^U+`l@xYMeX=LjhhZLOlie39R5R0wibbys3nKY8B6>3`@KZV7KZ$W z`RVHA&#&w>*v|>gMm`V;m7ZITrLbO%c+5ANzW1KXq-^f|cxxE5&UTo^b8~F|ZEh}&CyyQ{qbhzR9xKsWW!t`= z2s*o8ntkFCBn1&OG;6FmZZXl;K52h5#*ulxNsghJ!R!+ruDm1U%(w(e_zP~tU1(mv zBq2u8rqVap`rYzxTQ16pn$4WETOikAu%x~7X;L2T4E<%8)nEs!k%^?d#cIs>x>$T` z&eyo#{mO=o0?OhH>hM;^Qlf5uX6nS8YkWN6<;eZ_)<4 zGj4j(kuCbN@nex@>Urk7Gf~^k;AH**Q+^>$4Nr`fC#9txG5kHQe@1pGMCR}jubcSG zrIXS-7*JX+pYHDOW{=))B`|Ef9Z)J=fnh6(pa%K(d)OUdsrYQ2a{E9YL82)FQZVM) zK^Lc4zIzoF@CU{SV>S~fRyq1Wu2&r_3~-TBHT+6 zFGEIb*xf|d(5OjIpHB8yiRfjx8bQ+-F^A-@rIf$-u|Z)W-3d|jW@QAdrr=o zdpx#yUpAbmZ58&hu@9eI;r_e7-SWP?aHaG?tldQL@=pBly^F%V8XWZa#Q#%zcoy+G zh&n!lVe`S{g2a-*Now~Oa4yqJa zRZ|^ebix?Dth&X?xK_)NO`e9Z(mSGWo=*WYFf~N+5o3dHMa<;{(xKw0dl_*=|2AXDJo>}1 z#b~or6X>=?`A<3!GNW%7n59g~=}@Y^WItS_+Qol(Zd>s%PIOvUz2C584@*XM%a@Z1 z-|pZhxa~OD)>slTa*twj9rp*K?fTlxG6S`Gc)?F zvnzce6{ydirA6wXg4%IeFG&2*e)sQ&pp1&_PSeyPV%xieWA8yWRb3VdVcA-)L3qMf z#lL?UVHY!U%3@MlgzMFKZgeXu^f~Hm*tD1IrMB=C8XEsO*r3Q*4N9DM#gqP8x1~gx z*B0kcB0OP!@!J@$(LY(Yg+B5+caoPa9~`<-(H*{4mpxh;?2u?><&D!-uM0{mb9mpwu6O&`fnzVHbBC^p>1_!(#)%&|MTqCMjY+Q=*Z+Z9@!Oggwy>4pC3M! zC{=h(($~)3f$x^8;&M(rAhC0l_-|g_yzwo|k;(FIn4wZ#H`$U5HfwsJu#DjWeYGF) zxVF3I)wI~0*cN#W8kQHuH4sV>ML4uANdK+ks`x1-UODll+c3RZTUCnzSBH73jjw zMfpK3e0=Sh*(dIy-$8Hag|v$HKk1TRkL+@fOFQv|%AI&6?s5AK)AyXvf(9*Nv38;vT8CgNX%SbN=rGb2KNXc^CBVkU)4ca;QG4?yq_NHz zJNS5~xFR)t1;?8sD?dK)WkbQKdg4jdw%GAUDFH*~d1+Zy^DpU53UyW6V=}0@X<~~~ z*u0A0-_$iygbx>4FV4)hIn>3pTUTtV^a_E%HtrBa-+Q+VFfmm;BMN$;XBOIA<^8+w z;V=CF9X2Jq4HJ;1RYy$fagD#47if)*=}-?KR1*o(sU zl`C~VS5{_*=j#FewELcbC>P4guHBLpEit>XYj6KrN>v*{J@*3s3voRKEy}#tB_p@f z_z&L?a%)VhIg^u)hlWh&n4flAP1GfRfOcE=oII}Oa}RIny2;I_vhLX)eEWco878e2 zoBw*KHOFzO&HUKPeZV4i3Jn#8t19f8vb{4ijAr{5VbrXxf6^vt8G-v6zgvHecpbukTcOiZ2PQ@apL|f3xPF9K zH{mLb`Ou2*ivh#S%yXgP9KToe+TU~1R+VY8*3}trDl7=G-52Zz|LiSNhHq5y##+>` z1n+M&!~OiqFw%Q=-VdKNwqbjgPv|Q{N8P;XYE;jfH`C!2iDKx&%MZ+y&E+>auwZmGwL0hStZndu31RGG8)E92v)=CnU{ zU8Kyv!D(4mutJ98c)Z-98}zn3q!R9VZNlDV7|GZUiRbZ~kw4V7ZU>@xVgMaZhpwKS z`}?zllY!FgEOwlklpjX9nXrFbD9#RvKlo%-1Gpc}QhzHhei$3B9e0kAJ2mR16l0_J z6NbesDX9!^Wma}b)$yjmA0B9-Rjo`Z<@PsG8n#HW_lC86)jDEmH);{u$n{dvIv1va z;YPpb<}y@i<)-~05!}=E_}YZaUvrsCNF4R_NIhfnJO2a=o-3H7iHPOJCHzPP;+5uu zE4(EK2BIi}5JXevZ!x(EaY+*w#Bib_wvc8Tt?OuJ>7d&>bi_T(oamvAncFO5_b&im zEW^@S+dFiB>TSNwUh$Qv@sZ;q_p!h^eWm`Vm}ak-ln4Fa1@84-^XZ8a8V<>EeFzWm z;Je3l(NM0BAw zo(#pWTei*4Li`I2w6yyfP4rsrx0$0H2nw0R?QS$*zee19)Gx4+O;;~}L?_VWPz&8h zYt4Temok69p#qELNxv`Fk1>ukuRs`7%3i!eO?`@OMTU4qU1yhYBj8gdu@57-rQ}#G z=!J?(NtuQsFzE@f&}uOGG8E$`o1hF^zoHPn?Nl7AC}`a^)U2BLYcQttxKo*^$I!^O zM?F{=YEX93JK2+;(j3`wVfB#0e(%-hDEMplPe5_L*drWKN!F;~|GIxF>g8~k`eMe} zzJT+adiR<}>Ap7CEoImqPt?J3=G>xaqi+)P;W&3sxQ!})Y`Z)b;U^}$8OfBjtMi9X zF(Q@KOdBhEgPQ{Td6XF)<4_UTk^BftdE*wh#HxQOJ>`age$09BG0iQ<5YHu&`JhTYXL3oC(KnW zr7=za6eB#zg9eExX*yFAEnamC#?qH?V%!)fQUfb%wE&Aes6b47jd1Lg(=-vSVY7NAwIsth+TD0jbyGB#;(u-&z%c5yOh8k@Xh|id_e;4TfI@7GCwZEwGuXuQ+>;+G za;42IZ$frQ=R8o@l{iYJYNu9^eViflPt)v8 z@nIbFC$?>Z&xs$KwY_}(#y~Ffo%TX7#F+I!JjA`z(ws6Qvtl(z)@5F+my2@T?imS7 zs<_#5*IGYFc-SZpeIWW>?iai9f>a^goStAEPeA%K@#R$b)Nx|gH8IquPv;X25sU4M zxR_>UBeC4&u|hkr|~hu4c|>#TDaF#1LGppx@iSP^4P-Y=kG zlv}%V$E0CVxc)lM=uBWy)4ElWYM0in_X(27qBEZ%eb2z`CX|O6s+B=Ma9PJzC z`lgDBo@(qO^^KGOpEd$9l7vY)e&OAPl3o|oUco!g6T==JrGzK@W$cW&)k&Gh&tQiNfEQ8B#<*l5GNNV;e+K$MibMI6j_-4MFI2mYD8Q zA9jU*ZFOi%(jRLY!H-nXBA5}?p#S(GsYW;^nX|lY*r*2Lrum(Ug5|PtQ&sOCLgsl& zid{)TKfHR@rW0@nRH4~mCjp)kp9u&WFuSN6^Ai~8C4o+@bTE<0ew%ME$QD$u(X+~1 zt#pSY1t!Cw{EGppJ309~rP^`5FAmCNODpK}zE-=L*E|}=hc9rgc?$?`T~S8aljJLh zSaMM?Y!?@(LPN2f#dm6h_WcGs*o}#aVg7e4fy*FB*qX2jmJF!0`ZI=fX~tM8k? z^nH1RDb6i^_EsjSqh9GBde1DfK9+afqp|LO7);rr7Vl}fDV`WAEJJ*x>I#(L`b2pn zSpvpD-P@tq^^_Rvk|s-B!jRB$@sn={L#vHCf*hpXJQ+$#T{V*b*{sz1PVIZL7m0=Z z>&X=L&~>SzR5q6ziipRiCTF)5CmvWMWKo%~wXU6XP$i3~!2z!ssimU+s0a`WT%gJ> z1u6R@hWzr4=x~@~zXT3V>!3bB*Jnna#Dwv`TCvA21J^l`c@$9YK@fhNrV( zHj%C7=6cjyx=ZkxhY0EC@l~0V_T6c-L9GDg!Jm=7iyCIuWRrVxQU6{TwjBtL@XR0TfCug%o(x&mVqINuo12|l+P|FIYSI6e$Nnq>H1{S)is{QO?R)yHwd zkx>cnE-xM2A$8s1K$XrQ%SP|odEC$^D)xGp73o5UtW*CaUMn7=3ss`ry-f#7VD#Eo z3stFyHYT6h*cM(ge$5xQqn@MHV}3%(LN;mvO>s$31{y{Zuusmcujj$HlPE9EY>IBA zf;&bG51d)8K~4A~+E;N$M5ZWED`uVVJW;^U#cbiA6-b((y<$!XXCPMDw{Aw(EJDav zU?f3wAEuhinaWI)kYzXBJ4aT1*ktf2cD?RJw)i<)jd%qpz*N zPw44j-N3S8#EZ|7a=FO15jHAtx;|iLjzY`(blWe*5?W z>A;QImg=9bT)xwZ&9VX9l} zR=&Z%1=3yi*#XZ2jWEGO21M{c3*&M0o@ID`$}>=r(!GgEwf4&hxSwn1w9l91g?m9W z*Jq+rlgCDq3chk|w1DvvR@j`cpi&u$)jaM?+3+T+qSm?@;4*-(fHf|slji-d*P+C6 z0`<~Kd^m-VFfokj4UUqxInr`8ad}`r|5c7K8SQ^;rsw z^@7Z2hcU6wGy_V!G}MyDY)0X;De^9$Q2ZOTAm&^6l%o>&p~O~#MyQywLNV^3T}E!8 zR8c3d$<5kQcqq$Yojz1KV;xOVf^wKaOs$2bXh73*Yv`(ym{k}z{|FrHFaWn=#ufnL z))Joh_{7(=_YF;E6sdF(qQl*7il9?iiR#Z&hdwzQZ6q4-Z-lk!92SH{vvA3Ajgd1v zFCIx8Wu^u@CLnU05fWfu2TUtUJoQ`ewbe+9S>v<1;{3@pNyt8)g*8}C#=RD&p2|%# zobs}-(0@1~EGDInkI(8?xBPT=4^NCrY}7zdwbL!%ws-+JX?opvv_9+5WnF6}RRU?{QCwc3R{vYIG`h%3^jl zbJlxs_TAYR_hM8FXp!LxVl$NZNp;8?34i@uZsS%w(P}SX2hWaFVX6me#hre73la7H zSGwvu&amg|$XypPHgQV!XT4t@|t!p+%qGf(azZi$9CA z86c#n(R}gjux&5liKbV4TOGgk>ru5!VXKIf5%r%rmvmr39J_Gmv-~dBZ&g#>Y)-06 zL6{(EwVhWuv6EuDdijlkMe2w0BS%K1#lV;AEw1qIgoMml)mk2Yek}!+zfA9k08OdP zlD-zx4)6A-)T-}yhT_FVW#&osq?n7+q|RbB63!q4D3w#7Wav-%YQ%V3 z>!s|H#z@9uJ205)Lp;VnE$T4vbppbmD6w)lLC>Xkm(lp5A;y5S{AbG0Clp6zNATf3 z@Zqq27NymMD89qEsEP9h(^cR62FjZq91j_XjyJ1o8=q>L`G&jkQ7je(Z~Ed=Tk`Wa%9{o|2p%aBvC#n)*u>lJr|ko7ZDZVH=w>^f645llvttX zUTYn?5e)h$3Hp^Jfyj|~(7-H{Elb*zL~~O_)WRz%AV4`Y#u1;J_mshEaqKwI7BrLe zd+9WNZNMms=yB`A)ypmZ!(Qv*n%PqL-43q7n<){tfFO9qkgzbY?CPs3(W9?@J!F`E z{!_x>bmozPgn!Y=<#o^@?!AG;c&Ue}P0>j8ls@=UfB93q1ZVo6OP{%NPoFcjo5HOG zDVY|`s32LrSckp6!#(J0)>A2*9f_a=Vk8RQ zaD6!q=3DqJ%(eaOIV9WC$(lsk3 zriV+>nlPJ5Yjo^M&*R4@OwcnzBhKRBA!68k8}4;LgvWE146tKAO+PVfc{5l$WK;Sc zMBFnTS=(stvsfs1kv>q)x&g5@q0 zPG3NYGT)^DO?!1bG^%}ZVx`={W1!8Yk$l-Xn(CZU5!Q-4hv3!IBYj9O0WD?&h%i$V z7!-$!)H4DURc!Tv`VYJ`NwB>mMI-wQt*Tpy&l`g**-1T5rH${G{$rMq5~;rUFaTUj z3joxE4`l8GZ{}P(2k{jt>5x3d{{KXT5n46im^J@qE!H8*X}(E|27uCsyD(`gTq;ct zyK%S&kU}$mumhVX39t2kOa*9G4Wz!r|8-Hfd_nO29a7{r(BQoI?fmBQxDcq+ka&+B z%hd1DB;|Jh>BHxo8wY_fpBpKT2Hq&TNM`*4O$EhFzu*$k=gS6HIvFz@mS(uLl=iN5 zH7vOGJoal>Q&JJ5R--sixfTN=()N8g0=p=7XXBub@8)xjd+bf6`zRCD(}vOZ#i4(V zeM>Xex7f`GU{RX{wiLwbN6{lX$PclghsvUCr3;rZ4vYv#k{0 z6KFpJ&yfP|+L!c!?)p@`%O}6!IK+C*@C&efc`OZlv9A?2KbN}Un?I1j6LR{M^U>Xw zmlmq1xU)9#X1g$zJz0FGq7DHEu!ZS!)(ap>9d!xD?CLJlEkE&N0kDuc!g*XOF&&d8 zAQe9aZg8YAWdGQ4NKCvEkT%`OXeQ(w;7+cn(S>hfSI{joxnduYAoZ3+u|P8q#Lv-f z0F>Y{sinAg_OzlFeflyl04OZL2Q`6;kqH7BIcIH37k14OaGX{H8t?%BkPLuM(QWbC zrdLgY9D<)r3tvT)K~8pNG?9MLOZm$Yya&LzCIQIj2OC+Mu!p{hJ@mFf@0CRwE}u3A z0zyMaeqpHn=w%fJld z3;Gp|CPg5W+Vg*Q1AIEmCvGa87g}_S)Y9{oqJ|z&=K*}(-o(picu2!Ro-zzT&H=Mn z>SZYu?d6XWQmhB$*NCc)^4`-Bwub(%dMPl5-F?5%y;h^CzRK0?V(myJ3yVCAE3BMA z5M;^E?ss98j}24-e!$iSU}F8`OhYNV*hIR#`2ukN>HTW3TW`sL^{c2O{xK;IAUHju z!p22Ak-GW!{BhfsEQmKg)=YTkBF)fE_0T`S zl}rYMd#q8I$L$N!nn@JELpH#ypERxn7=!D$58&O?_yspNj)^Uy-7jBg6fWM$)gInJCK>A^Iun8@puVFUatu zf!5}t*GGp z;Oe8L@o)cn2{i|lSw0t(Uksd`vheX~OHhAAdZqt|?)o-LBKacT_Gg2$5!E+RuR&{b z`!B7p-nF z)-mr>ze*5O4WR%j?H@aM1EKoz(=iUusu@v{0`Kb|Fo3lrVlV6 zg&JS1Sv6Z#S2f$($m#`Ece4}QJO|#0Ez`Y+jnIoVOIml^3?MWX`?JSBUedO59<#Fw zCiKt3l{+3!%O6<8q_1vm|H2Ayc>BJ8-?ltLB~>#{`^+bT`AARVs?mJ;T(c0}XNqE4 z`_Kd0g=R4Vd$5j6H?*&N#tjy2oR0 zIV-$mMj}C`1}BvcF@!eBz|U^@r`RSVLog?tYdD&k14PrhUZ#W=4800VcXP+5Q@sSn zd7MYSwYwx**b!s|OGs1nk?aGMP^zVb&bzB&A4ZW208P{lGvJEGgVSCDyJIjUTL^2; zLab?h&(dvmk`~^e^80&`H+4?>Tg>|9$SyZ!1H-q8VojRnovKzZY5stY@~Z22{|`(A ze@6U=gx%ZMU%5*nK|}l5;yzU$s1q3~I683NnWDr+$94u2(6RY?j0(^@9$J zln?$p0HArahfUe;f4-;;P4m|N_^4CL1r-gM(>kJh3<2uR#J*Sx#{leV?GpSgGsB3m zI-va^`6`dai2Eo&KT7(cT=&ufkb+c6`L{h}#KOEXw6wH5mEyzTiQKqmPHb8yEx6^B zWhQ>sfy>x?6en?{^!3Aqk;n0KBsQ-*W-G+dKY-fi`rR|Qf9AnoU47F+=_Mdf6o(033y7nYzVl%^>|^&tZzRAOUM87 z@Bcf1Kup65{3XBZiv;eeO2p;VS36%e zz`}4M`3QGp65sUx!B?C+|9iQ2u=0!aTg?3r5z1sZvebbmT1#2*6>`np38rO!|6qE0 z`jhKH*xPV9GkPC?A*kp1@b?r09(x$f$=tC(aUeI^vG0j)1{Wfrtb&7H53A{d= znlgODdV{(L@mTh&xd}0aVx8s22PmgK=1p6Eev&PVi8CVB)SQI<>Z~i3lH?(?WA!(Z z5E;=93k<7tk8|E@QK_@2a2NNc2K@U?DT4@SIM3tGGW2kaI9EScu0Fu6Z1 zT^zYx_wNdeQKsOic`HvcvU&-bx772{o>(*yG8(TiY*yj46S6R?c+)%KDW87jswQ+H zzIGjr{->oQ)D#*S%~Re(6DN9`H&!GboogM8+%DR+jWp4pEt*-t2OM{YiTCm&emMn&!P?2UQA zbe?ih*?N~vB-*r*wuk@C2HTl7^zVAjzt+uGmABdcYAY`~zTox-Jdr|emycY)gKZq} zzIZjYMzDUqP~y(u-OtzTC3`YID-aw?BgTI(6T*?0Gr^iA_$LtFGmeaFs&5CQBjbcY>d~^=t`Gc246NM$gAvdwd_~3-zBXWND_Vjh*r23wZ02HQKDe#U(v0MEK6Mq)7DcAbgg*4X_Cfb_}#ss64 z&aABSlQCynUii%Mm}}Pudg=F$baUx9A<Clqly6Xz@@b7INh=1QG2^68(BS@FX!V)dwX2O} z!wDhPI(r8RqW+my%ox4de!iGE%+;=AmMGf+h;@efxFuTNfIH`)%@``4w`sHTGq{hF z`w*$m<+BWb+9qvEeDtkg-~e|SXM19xlX;(LXP(&K1>{o_%xEX_NH{ATZrX+uAZd2(GH zgT9cQ9ELzi*6f~3ow7nargj>)8Lg|(X;c^k)+71j3!3Q2Po7DoIV#-gw^#(LUH(wwsMu@g*yU7KW)lGzgqGsZ{z1>uvsR1wg~(JWoh4jTHrK z^15qq`28Uft1ln~!xn({YFI%p6?}i&c$WF?RDMV6!=si$ z#kWN-CS#t5*_p1l)chHbv+h0KUWi+Euj~!37AslXSi-?LT(6_kacPwA`DONBYry_b zQ}StuR!x2toE#O;>FA%4d%63wHPLrFEp{4mh9BcGkyH?v=8NF&>j)Ko!R~VF#iq-h zV?rk}u$JEqrw@q?$N5^1hq_9)tH@)FM+jlBdiw6@i%N97Rw?le$v!)~v1g@D_4m)= zkxi2A?{?NYXmgBFw^ph7V{QRRqQv7O8jFk8SY%|?z7`d?qgUqt1;YV6{#<_l<9F>v z;g*$0M3~vTFgW`U`lX2o&{+`|68)!%bI=BW*aL$$fp20d2$+BW4*H;Q9I#;gds#E^ z(;<_tcZd^l54TYet)i*Ii)*Vm1kDi<1jU6|i3KUQZRGJrSs;S zZ~FF#w)rH^;C5{!c0qScViUwp5EzbAS(i8iKhskMIG;h^zcnaj-rZi=#bMPG&&Lu$ z0I+U&%&gz7ib_XA+-C&c5hq+(G^hgHX2dyMmlgjLASW(@yNlQeaS^O10`yodSRphO z;+pxaxYMvii0!!TZh?kag9Z)K^SA~CsI<>v7<-C|h*70rrcY0l`1p9=9&h<-X8yRO zn2jS04Boi9=+?&Up8*|9fn~Xz>-Kiwe5@M(-sn80g=k3-^FW+j5prLTc!dk^Ph0}a zh>gLWeSp9rSnldCM1<>eEeakJ7vVD^Dp(x6-}&dAr$U2BP+FWoI}Msx2;hRa5F4Uf z9SaTI*D#Fz!9>KUw)neu0s(?hR>_Z8IeRJyl`(VQ2wqG?48t&tea1w@Fbu=kXG}y4 f!!V3}MCAVgE$5o=LNV;Z00000NkvXXu0mjfM2`=W literal 0 HcmV?d00001 From 6e3a0f685fee0317870c15527d52e6c207527035 Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 19:06:25 +0000 Subject: [PATCH 06/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 29 ++++++++++---------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 53f526de..7f5e66e9 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -157,17 +157,6 @@ Sorting 10000000 integers 1.468 seconds Sort succeeded. ``` -### Checking for race conditions using Cilksan - -The Cilksan race detector can be used to check for race conditions in the parallelized quicksort code. To run Cilksan on our parallel quicksort routine, we must compile the program with Cilksan enabled and then execute the instrumented program. - -```shell -> clang++ qsort.cpp -o qsort –Og -g -fopencilk -fsanitize=cilk -./qsort 10000000 -``` - -The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. - ### Measuring scalability using Cilkscale Cilkscale can be used to benchmark and analyze the parallelism, in terms of work and span, of an OpenCilk program. These measurements can be used to predict parallel performance on parallel processors.\ @@ -179,11 +168,21 @@ The Cilkscale visualizer tool can be used to illustrate the scalability of the q ./qsort 10000000 ``` -\ -\ -\ The plots generated by Cilkscale for the example quicksort program are shown below. ![Cilkscale speedup for quicksort.](/img/cilkscale-qsort-speedup.png "Quicksort speedup") -![Cilkscale execution time for quicksort.](/img/cilkscale-qsort-execution-time.png "Quicksort execution time") \ No newline at end of file +![Cilkscale execution time for quicksort.](/img/cilkscale-qsort-execution-time.png "Quicksort execution time") + + + +### Checking for race conditions using Cilksan + +The Cilksan race detector can be used to check for race conditions in the parallelized quicksort code. To run Cilksan on our parallel quicksort routine, we must compile the program with Cilksan enabled and then execute the instrumented program. + +```shell +> clang++ qsort.cpp -o qsort –Og -g -fopencilk -fsanitize=cilk +./qsort 10000000 +``` + +The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. \ No newline at end of file From df466cf8119d0c50893635ceb92b35a8a19f08e4 Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 19:17:44 +0000 Subject: [PATCH 07/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 7f5e66e9..1ee4a0d7 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -166,9 +166,17 @@ The Cilkscale visualizer tool can be used to illustrate the scalability of the q ```shell > clang++ qsort.cpp -o qsort –O3 -fopencilk -fcilktool=cilkscale ./qsort 10000000 + +Sorting 10000000 integers +All sorts succeeded +tag,work (seconds),span (seconds),parallelism,burdened_span (seconds),burdened_parallelism +,14.511,0.191245,75.8764,0.191514,75.7699 ``` -The plots generated by Cilkscale for the example quicksort program are shown below. +\ +The CIlkscale tool reports on the total work, span, and parallelism in the execution of quicksort. The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](doc/users-guide/getting-started/#using-cilkscale). \ +\ +Using Cilkscale and the visualization tools provided in OpenCilk, we can produce plots that illustrate the speedup and observed runtime performance of our quicksort program when run on a varying number of processors. ![Cilkscale speedup for quicksort.](/img/cilkscale-qsort-speedup.png "Quicksort speedup") @@ -185,4 +193,4 @@ The Cilksan race detector can be used to check for race conditions in the parall ./qsort 10000000 ``` -The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. \ No newline at end of file +The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. More detailed instructions about the use of Cilksan can be found [here](doc/users-guide/getting-started/#using-cilksan). \ No newline at end of file From 28fff4cbf5e3e77ac2765682ecae27842510a226 Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 19:17:50 +0000 Subject: [PATCH 08/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 1ee4a0d7..0c544201 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -174,7 +174,7 @@ tag,work (seconds),span (seconds),parallelism,burdened_span (seconds),burdened_p ``` \ -The CIlkscale tool reports on the total work, span, and parallelism in the execution of quicksort. The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](doc/users-guide/getting-started/#using-cilkscale). \ +The Cilkscale tool reports on the total work, span, and parallelism in the execution of quicksort. The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](doc/users-guide/getting-started/#using-cilkscale). \ \ Using Cilkscale and the visualization tools provided in OpenCilk, we can produce plots that illustrate the speedup and observed runtime performance of our quicksort program when run on a varying number of processors. From dd5995cf142e75f05b4487210d38c9d22d783eb6 Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 19:23:16 +0000 Subject: [PATCH 09/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 0c544201..b6ba3aee 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -161,7 +161,7 @@ Sorting 10000000 integers Cilkscale can be used to benchmark and analyze the parallelism, in terms of work and span, of an OpenCilk program. These measurements can be used to predict parallel performance on parallel processors.\ \ -The Cilkscale visualizer tool can be used to illustrate the scalability of the quicksort program by compiling the program with the additional flag `-fcilktool=cilkscale` and then executing the program as shown below. +One can use Cilkscale to benchmark the parallel scalability of quicksort by compiling with the additional flag `-fcilktool=cilkscale` and then executing the program as shown below. ```shell > clang++ qsort.cpp -o qsort –O3 -fopencilk -fcilktool=cilkscale @@ -174,16 +174,14 @@ tag,work (seconds),span (seconds),parallelism,burdened_span (seconds),burdened_p ``` \ -The Cilkscale tool reports on the total work, span, and parallelism in the execution of quicksort. The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](doc/users-guide/getting-started/#using-cilkscale). \ -\ -Using Cilkscale and the visualization tools provided in OpenCilk, we can produce plots that illustrate the speedup and observed runtime performance of our quicksort program when run on a varying number of processors. +Cilkscale will report the total work, span, and parallelism in the code at the end of the program execution, as shown above. + +The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](doc/users-guide/getting-started/#using-cilkscale). Using Cilkscale and related tools, we can produce plots that illustrate the speedup and observed runtime performance of our quicksort program when run on a varying number of processors. The resultant plots are shown below. ![Cilkscale speedup for quicksort.](/img/cilkscale-qsort-speedup.png "Quicksort speedup") ![Cilkscale execution time for quicksort.](/img/cilkscale-qsort-execution-time.png "Quicksort execution time") - - ### Checking for race conditions using Cilksan The Cilksan race detector can be used to check for race conditions in the parallelized quicksort code. To run Cilksan on our parallel quicksort routine, we must compile the program with Cilksan enabled and then execute the instrumented program. From c1a275e160ed40209814e9645f9674ea77e83c6c Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 19:23:59 +0000 Subject: [PATCH 10/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index b6ba3aee..11c06954 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -176,7 +176,7 @@ tag,work (seconds),span (seconds),parallelism,burdened_span (seconds),burdened_p \ Cilkscale will report the total work, span, and parallelism in the code at the end of the program execution, as shown above. -The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](doc/users-guide/getting-started/#using-cilkscale). Using Cilkscale and related tools, we can produce plots that illustrate the speedup and observed runtime performance of our quicksort program when run on a varying number of processors. The resultant plots are shown below. +The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](/doc/users-guide/getting-started/#using-cilkscale). Using Cilkscale and related tools, we can produce plots that illustrate the speedup and observed runtime performance of our quicksort program when run on a varying number of processors. The resultant plots are shown below. ![Cilkscale speedup for quicksort.](/img/cilkscale-qsort-speedup.png "Quicksort speedup") @@ -191,4 +191,4 @@ The Cilksan race detector can be used to check for race conditions in the parall ./qsort 10000000 ``` -The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. More detailed instructions about the use of Cilksan can be found [here](doc/users-guide/getting-started/#using-cilksan). \ No newline at end of file +The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. More detailed instructions about the use of Cilksan can be found [here](/doc/users-guide/getting-started/#using-cilksan). \ No newline at end of file From 21af0cdbd782d4f1f559b42b86311bfdd07854b8 Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 19:26:07 +0000 Subject: [PATCH 11/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 11c06954..a8da1e42 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -159,8 +159,8 @@ Sorting 10000000 integers ### Measuring scalability using Cilkscale -Cilkscale can be used to benchmark and analyze the parallelism, in terms of work and span, of an OpenCilk program. These measurements can be used to predict parallel performance on parallel processors.\ -\ +Cilkscale can be used to benchmark and analyze the parallelism, in terms of work and span, of an OpenCilk program. These measurements can be used to predict parallel performance on parallel processors. + One can use Cilkscale to benchmark the parallel scalability of quicksort by compiling with the additional flag `-fcilktool=cilkscale` and then executing the program as shown below. ```shell @@ -173,10 +173,11 @@ tag,work (seconds),span (seconds),parallelism,burdened_span (seconds),burdened_p ,14.511,0.191245,75.8764,0.191514,75.7699 ``` -\ Cilkscale will report the total work, span, and parallelism in the code at the end of the program execution, as shown above. -The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](/doc/users-guide/getting-started/#using-cilkscale). Using Cilkscale and related tools, we can produce plots that illustrate the speedup and observed runtime performance of our quicksort program when run on a varying number of processors. The resultant plots are shown below. +The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](/doc/users-guide/getting-started/#using-cilkscale). Using Cilkscale and related tools, we can produce plots that illustrate the speedup and observed runtime performance of our quicksort program when run on a varying number of processors. + +Plots illustrating the parallel execution time and speedup of the quicksort program we have parallelized in this example are shown below. ![Cilkscale speedup for quicksort.](/img/cilkscale-qsort-speedup.png "Quicksort speedup") From f208e4c873da52e5de383ea792d558d507ccffbb Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 19:28:14 +0000 Subject: [PATCH 12/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index a8da1e42..26ea411b 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -159,7 +159,7 @@ Sorting 10000000 integers ### Measuring scalability using Cilkscale -Cilkscale can be used to benchmark and analyze the parallelism, in terms of work and span, of an OpenCilk program. These measurements can be used to predict parallel performance on parallel processors. +Cilkscale can be used to benchmark and analyze the parallelism, in terms of work and span, of an OpenCilk program. These measurements can be used to predict performance when running on a varying number of parallel processors. One can use Cilkscale to benchmark the parallel scalability of quicksort by compiling with the additional flag `-fcilktool=cilkscale` and then executing the program as shown below. @@ -175,7 +175,7 @@ tag,work (seconds),span (seconds),parallelism,burdened_span (seconds),burdened_p Cilkscale will report the total work, span, and parallelism in the code at the end of the program execution, as shown above. -The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](/doc/users-guide/getting-started/#using-cilkscale). Using Cilkscale and related tools, we can produce plots that illustrate the speedup and observed runtime performance of our quicksort program when run on a varying number of processors. +The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](/doc/users-guide/getting-started/#using-cilkscale). Plots illustrating the parallel execution time and speedup of the quicksort program we have parallelized in this example are shown below. From b8843156ea5a02124c8d84f16cde62ad21209857 Mon Sep 17 00:00:00 2001 From: tfk Date: Wed, 20 Jul 2022 19:39:59 +0000 Subject: [PATCH 13/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 26ea411b..4d208e11 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -190,6 +190,10 @@ The Cilksan race detector can be used to check for race conditions in the parall ```shell > clang++ qsort.cpp -o qsort –Og -g -fopencilk -fsanitize=cilk ./qsort 10000000 + +Cilksan detected 0 distinct races. +Cilksan suppressed 0 duplicate race reports. + ``` The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. More detailed instructions about the use of Cilksan can be found [here](/doc/users-guide/getting-started/#using-cilksan). \ No newline at end of file From fd4784725322aa85d652420cc0a7e66dfe5b47eb Mon Sep 17 00:00:00 2001 From: tfk Date: Fri, 16 Sep 2022 18:35:59 +0000 Subject: [PATCH 14/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 88 +++++++++++--------- 1 file changed, 48 insertions(+), 40 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 4d208e11..95283abc 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -2,22 +2,23 @@ title: Convert a C++ program author: Timothy Kaler date: 2022-07-20T16:22:55.620Z +attribution: true --- -A common application of OpenCilk is the parallelization of existing serial code. Indeed, it is often advisable for programmers to prioritize writing correct and efficient serial code before attempting parallelization because of the notorious difficulty of writing correct parallel code. In this section, we shall walk through the process of converting an existing serial C or C++ code to an OpenCilk parallel program and show how OpenCilk's suite of tools can be used to debug race-conditions and scalability bottlenecks. +OpenCilk can be used to add parallelism to existing serial codes without changing the original program's semantics. Let us walk through the process of converting an existing serial C or C++ code to an OpenCilk parallel program and show how OpenCilk's suite of tools can be used to debug race-conditions and scalability bottlenecks. ## General workflow -One typically begins with an existing serial C or C++ program that implements the functions or algorithms that are relevant to one's application. Ideally, the serial code you start with will be well tested to verify it is correct and be performance engineered to achieve good performance when run serially. Any correctness bugs in the serial code will result in correctness bugs in the parallel program, but they will be more difficult to identify and fix! Similarly, inefficiency in the original sequential code will translate to inefficiency in its parallel equivalent. +The typical process for adding parallelism to existing serial C or C++ programs using OpenCilk involves five steps: -Next, one begins the process of introducing parallelism into the program. Typically one starts by identifying the regions of the program that will most benefit from parallel execution. Operations that are relatively long-running and/or tasks that can be performed independently are prime candidates for parallelization. The programmer can identify tasks in their code that can execute in parallel using the three OpenCilk keywords: +1. **Debug serial code:** Verify the original program is correct. It is a good practice to write correct and well-tested serial code prior to attempting parallelization. Bugs that exist in the serial code will also exist after introducing parallelism, but may be more difficult to debug. +2. **Identify parallelism:** Identify regions of the code that could benefit from parallel execution. Typically, operations that are relatively long-running and/or tasks that can be performed independently are prime candidates for parallelization. +3. **Annotate parallelism:** Introduce parallelism to the code using the OpenCilk keywords \`cilk_for\`, \`cilk_spawn\`, and \`cilk_scope\`. These keywords are described in more depth in ???. A summary of their semantics is as follows: -* `cilk_spawn` indicates a call to a function (a "child") that can proceed in parallel with the caller (the "parent"). -* `cilk_sync` indicates that all spawned children must complete before proceeding. -* `cilk_for` identifies a loop for which all iterations can execute in parallel. - -The parallel version of the code can be compiled and tested using the OpenCilk compiler. On **Linux* OS** one invokes the OpenCilk compiler using the `clang` or `clang++` commands. One compiled, the program can be run on the local machine to test for correctness and measure performance. If the parallelization of the original (correct) serial program contains no ***race conditions***, then the parallel program will produce the same result as the serial program. - -The OpenCilk tools can be used to debug race conditions and scalability bottlenecks in parallelized codes. Verifying the absence of race conditions is particularly important as such errors can lead to non-deterministic (and often buggy) behavior. Fortunately, OpenCilk provides the ***cilksan race detector*** which can identify all possible race conditions introduced by parallel operations when a program is run on a given input. With the help of OpenCilk's tools, one can identify and resolve race conditions through the use of **reducers**, locks, and recoding. + * `cilk_for` identifies a loop for which all iterations can execute in parallel. + * `cilk_spawn` indicates a call to a function (a "child") that can proceed in parallel with the caller (the "parent"). + * `cilk_scope` indicates that all spawned children within the scoped region must complete before proceeding. +4. **Compile:** Compile the code using the OpenCilk compiler. On **Linux* OS** one invokes the OpenCilk compiler using the `clang` or `clang++` commands. One compiled, the program can be run on the local machine to test for correctness and measure performance. +5. **Verify absence of races:** Use OpenCilk's ***cilksan race detector*** to verify the absence of race conditions in the parallel program. If the parallelization of the original (correct) serial program contains no ***race conditions***, then the parallel program will produce the same result as the serial program. With the help of OpenCilk's tools, one can identify and resolve race conditions through the use of ***reducers***, locks, and recoding. ## Example: Quicksort @@ -80,21 +81,17 @@ int main(int argc, char* argv[]) } ``` -### Compiling Quicksort with the OpenCilk compiler -This quicksort code can be compiled using the OpenCilk C++ compiler by adding `#include ` statement to the source file. The `cilk.h` header file contains declarations of the OpenCilk runtime API and the keywords used to specify parallel control flow. After adding the `cilk.h` header file, one can compile the quicksort program using the OpenCilk compiler. -##### Linux* OS +## Identify parallelism -```shell -> clang++ qsort.cpp -o qsort –O3 -fopencilk -``` +The `sample_qsort` function is invoked recursively on two disjoint subarrays on line 16 and line 17. These independent tasks will be relatively long-running and are good candidates for parallelization. This proposed parallelization of quicksort represents a typical divide-and-conquer strategy for parallelizing recursive algorithms. An intrepid reader might also notice that the partition algorithm invoked on line 13 may also be parallelized for even greater scalability. -### Add parallelism using `cilk_spawn` +## Annotate parallelism -The next step is to actually introduce parallelism into our quicksort program. This can be accomplished through the judicious use of OpenCilk's three keywords for expressing parallelism: `cilk_spawn`, `cilk_sync`, and `cilk_for`. +The next step is to actually introduce parallelism into our quicksort program. This can be accomplished through the judicious use of OpenCilk's three keywords for expressing parallelism: `cilk_for`, `cilk_spawn`, and `cilk_scope`. -In this example, we shall make use of just the `cilk_spawn` and `cilk_sync` keywords. The `cilk_spawn` keyword indicates that a function (the *child*) may be executed in parallel with the code that follows the `cilk_spawn` statement (the *parent*). Note that the keyword *allows* but does not *require* parallel operation. The OpenCilk scheduler will dynamically determine what actually gets executed in parallel when multiple processors are available. The `cilk_sync` statement indicates that the function may not continue until all `cilk_spawn` requests in the same function have completed. The `cilk_sync` instruction does not affect parallel strands spawned in other functions. +In this example, we shall make use of just the `cilk_spawn` and `cilk_scope` keywords. The `cilk_spawn` keyword indicates that a function (the *child*) may be executed in parallel with the code that follows the `cilk_spawn` statement (the *parent*). Note that the keyword *allows* but does not *require* parallel operation. The OpenCilk scheduler will dynamically determine what actually gets executed in parallel when multiple processors are available. The `cilk_scope` statement indicates that the function may not continue until all `cilk_spawn` requests within the scoped region in the same function have completed. Let us walk through a version of the quicksort code that has been parallelized using OpenCilk. @@ -104,20 +101,31 @@ void sample_qsort(int * begin, int * end) if (begin != end) { --end; // Exclude last element (pivot) int * middle = std::partition(begin, end, - std::bind2nd(std::less(),*end)); + std::bind2nd(std::less(),*end)); std::swap(*end, *middle); // pivot to middle - cilk_spawn sample_qsort(begin, middle); - sample_qsort(++middle, ++end); // Exclude pivot - cilk_sync; + cilk_scope { + cilk_spawn sample_qsort(begin, middle); + sample_qsort(++middle, ++end); // Exclude pivot + } } } ``` -In the example code above, the serial quicksort code has been converted into a parallel OpenCilk code by adding the `cilk_spawn` keyword on line 8, and the `cilk_sync` keyword on line 10. The `cilk_spawn` keyword on line 8 indicates that the function call `sample_qsort(begin, middle)` is allowed to execute in-parallel with its ***continuation*** which includes the program instructions that are executed after the return of the function call on line 8 and the `cilk_sync` instruction on line 10. +In the example code above, the serial quicksort code has been converted into a parallel OpenCilk code by adding the `cilk_spawn` keyword on line 9, and defining the `cilk_scope` region to include lines 9--10. The `cilk_spawn` keyword on line 9 indicates that the function call `sample_qsort(begin, middle)` is allowed to execute in-parallel with its ***continuation*** which includes the function call `sample_qsort(++middle, ++end)` on line 10. + +The `cilk_spawn` keyword can be thought of as allowing the recursive invocation of `sample_qsort` on line 10 to execute asynchronously. Thus, when we call `sample_qsort` again in line 10, the call at line 9 might not have completed. The end of the `cilk_scope` region at line 11 indicates that this function will not continue until all `cilk_spawn` requests in the same scoped region have completed. There is an implicit `cilk_scope` surrounding the body of every function so that at the end of every function all tasks spawned in the function have returned. + +The parallelization of quicksort provided in this example implements a typical divide-and-conquer strategy for parallelizing recursive algorithms. At each level of recursion, we have two-way parallelism; the parent strand (line 10) continues executing the current function, while a child strand executes the other recursive call. In general, recursive divide-and-conquer algorithms can expose significant parallelism. In the case of quicksort, however, parallelizing according to the standard recursive structure of the serial algorithm only exposes limited parallelism. The reason for this is due to the substantial amount of work performed by the serial `partition` function invoked on line 5. The partition algorithm may be parallelized for better scalability, but we shall leave this task as an exercise to the intrepid reader. -The `cilk_spawn` keyword can be thought of as allowing the recursive invocation of `sample_qsort` on line 8 to execute asynchronously. Thus, when we call `sample_qsort` again in line 9, the call at line 8 might not have completed. The `cilk_sync` statement at line 10 indicates that this function will not continue until all `cilk_spawn` requests in the same function have completed. There is an implicit `cilk_sync` at the end of every function that waits until all tasks spawned in the function have returned, so the `cilk_sync` here is redundant, but written explicitly for clarity. +## Compile -The parallelization of quicksort provided in this example implements a typical divide-and-conquer strategy for parallelizing recursive algorithms. At each level of recursion, we have two-way parallelism; the parent strand (line 9) continues executing the current function, while a child strand executes the other recursive call. In general, recursive divide-and-conquer algorithms can expose significant parallelism. In the case of quicksort, however, parallelizing according to the standard recursive structure of the serial algorithm only exposes limited parallelism. The reason for this is due to the substantial amount of work performed by the serial `partition` function invoked on line 5. The partition algorithm may be parallelized for better scalability, but we shall leave this task as an exercise to the intrepid reader. +This quicksort code can be compiled using the OpenCilk C++ compiler by adding `#include ` statement to the source file. The `cilk.h` header file contains declarations of the OpenCilk runtime API and the keywords used to specify parallel control flow. After adding the `cilk.h` header file, one can compile the quicksort program using the OpenCilk compiler. + +##### Linux* OS + +```shell +> clang++ qsort.cpp -o qsort –O3 -fopencilk +``` ### Build, execute, and test @@ -157,6 +165,20 @@ Sorting 10000000 integers 1.468 seconds Sort succeeded. ``` +### Checking for race conditions using Cilksan + +The Cilksan race detector can be used to check for race conditions in the parallelized quicksort code. To run Cilksan on our parallel quicksort routine, we must compile the program with Cilksan enabled and then execute the instrumented program. + +```shell +> clang++ qsort.cpp -o qsort –Og -g -fopencilk -fsanitize=cilk +./qsort 10000000 + +Cilksan detected 0 distinct races. +Cilksan suppressed 0 duplicate race reports. +``` + +The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. More detailed instructions about the use of Cilksan can be found [here](/doc/users-guide/getting-started/#using-cilksan). + ### Measuring scalability using Cilkscale Cilkscale can be used to benchmark and analyze the parallelism, in terms of work and span, of an OpenCilk program. These measurements can be used to predict performance when running on a varying number of parallel processors. @@ -183,17 +205,3 @@ Plots illustrating the parallel execution time and speedup of the quicksort prog ![Cilkscale execution time for quicksort.](/img/cilkscale-qsort-execution-time.png "Quicksort execution time") -### Checking for race conditions using Cilksan - -The Cilksan race detector can be used to check for race conditions in the parallelized quicksort code. To run Cilksan on our parallel quicksort routine, we must compile the program with Cilksan enabled and then execute the instrumented program. - -```shell -> clang++ qsort.cpp -o qsort –Og -g -fopencilk -fsanitize=cilk -./qsort 10000000 - -Cilksan detected 0 distinct races. -Cilksan suppressed 0 duplicate race reports. - -``` - -The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. More detailed instructions about the use of Cilksan can be found [here](/doc/users-guide/getting-started/#using-cilksan). \ No newline at end of file From 721b0bcf56db02ca410ccdb40a90135e384085d9 Mon Sep 17 00:00:00 2001 From: tfk Date: Fri, 16 Sep 2022 18:39:39 +0000 Subject: [PATCH 15/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 95283abc..88bbb3c2 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -81,8 +81,6 @@ int main(int argc, char* argv[]) } ``` - - ## Identify parallelism The `sample_qsort` function is invoked recursively on two disjoint subarrays on line 16 and line 17. These independent tasks will be relatively long-running and are good candidates for parallelization. This proposed parallelization of quicksort represents a typical divide-and-conquer strategy for parallelizing recursive algorithms. An intrepid reader might also notice that the partition algorithm invoked on line 13 may also be parallelized for even greater scalability. @@ -91,7 +89,7 @@ The `sample_qsort` function is invoked recursively on two disjoint subarrays on The next step is to actually introduce parallelism into our quicksort program. This can be accomplished through the judicious use of OpenCilk's three keywords for expressing parallelism: `cilk_for`, `cilk_spawn`, and `cilk_scope`. -In this example, we shall make use of just the `cilk_spawn` and `cilk_scope` keywords. The `cilk_spawn` keyword indicates that a function (the *child*) may be executed in parallel with the code that follows the `cilk_spawn` statement (the *parent*). Note that the keyword *allows* but does not *require* parallel operation. The OpenCilk scheduler will dynamically determine what actually gets executed in parallel when multiple processors are available. The `cilk_scope` statement indicates that the function may not continue until all `cilk_spawn` requests within the scoped region in the same function have completed. +In this example, we shall make use of just the `cilk_spawn` and `cilk_scope` keywords. The `cilk_spawn` keyword indicates that a function (the *child*) may be executed in parallel with the code that follows the `cilk_spawn` statement (the *parent*). Note that the keyword *allows* but does not *require* parallel operation. The OpenCilk scheduler will dynamically determine what actually gets executed in parallel when multiple processors are available. The `cilk_scope` statement indicates that the function may not continue until all `cilk_spawn` requests within the scoped region have completed. Let us walk through a version of the quicksort code that has been parallelized using OpenCilk. @@ -203,5 +201,4 @@ Plots illustrating the parallel execution time and speedup of the quicksort prog ![Cilkscale speedup for quicksort.](/img/cilkscale-qsort-speedup.png "Quicksort speedup") -![Cilkscale execution time for quicksort.](/img/cilkscale-qsort-execution-time.png "Quicksort execution time") - +![Cilkscale execution time for quicksort.](/img/cilkscale-qsort-execution-time.png "Quicksort execution time") \ No newline at end of file From 49261b8c6bc6d1140999f55b7505bf0a3205e69c Mon Sep 17 00:00:00 2001 From: tfk Date: Fri, 16 Sep 2022 18:46:29 +0000 Subject: [PATCH 16/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 88bbb3c2..d542f60f 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -163,7 +163,7 @@ Sorting 10000000 integers 1.468 seconds Sort succeeded. ``` -### Checking for race conditions using Cilksan +## Verify absence of races using Cilksan The Cilksan race detector can be used to check for race conditions in the parallelized quicksort code. To run Cilksan on our parallel quicksort routine, we must compile the program with Cilksan enabled and then execute the instrumented program. @@ -177,7 +177,7 @@ Cilksan suppressed 0 duplicate race reports. The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. More detailed instructions about the use of Cilksan can be found [here](/doc/users-guide/getting-started/#using-cilksan). -### Measuring scalability using Cilkscale +## Measure scalability using Cilkscale Cilkscale can be used to benchmark and analyze the parallelism, in terms of work and span, of an OpenCilk program. These measurements can be used to predict performance when running on a varying number of parallel processors. From ead47e44540fa6e0ca2cfb658d4ec159a0f83ced Mon Sep 17 00:00:00 2001 From: tfk Date: Fri, 16 Sep 2022 18:47:40 +0000 Subject: [PATCH 17/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index d542f60f..4070c985 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -163,7 +163,7 @@ Sorting 10000000 integers 1.468 seconds Sort succeeded. ``` -## Verify absence of races using Cilksan +## Verify absence of races The Cilksan race detector can be used to check for race conditions in the parallelized quicksort code. To run Cilksan on our parallel quicksort routine, we must compile the program with Cilksan enabled and then execute the instrumented program. @@ -177,7 +177,7 @@ Cilksan suppressed 0 duplicate race reports. The Cilksan race detector will report any race conditions present in the program and verify the absence of races in a race-free program. More detailed instructions about the use of Cilksan can be found [here](/doc/users-guide/getting-started/#using-cilksan). -## Measure scalability using Cilkscale +## Measure scalability Cilkscale can be used to benchmark and analyze the parallelism, in terms of work and span, of an OpenCilk program. These measurements can be used to predict performance when running on a varying number of parallel processors. From a45836cae894e125f9b7b5c5ea3cc6597cd77c7d Mon Sep 17 00:00:00 2001 From: tfk Date: Fri, 16 Sep 2022 20:52:01 +0000 Subject: [PATCH 18/19] =?UTF-8?q?Update=20User's=20guide=20=E2=80=9Cconver?= =?UTF-8?q?t-a-c++-program=E2=80=9D?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- src/doc/users-guide/convert-a-c++-program.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index 4070c985..b0660f8a 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -26,7 +26,7 @@ Let us illustrate the process of parallelizing an existing serial code by walkin Note that in this example we use the function name `sample_qsort` in order to avoid confusion with the Standard C Library `qsort` function. -```c +```cilkc# #include #include #include @@ -93,7 +93,7 @@ In this example, we shall make use of just the `cilk_spawn` and `cilk_scope` key Let us walk through a version of the quicksort code that has been parallelized using OpenCilk. -```c +```cilkc# void sample_qsort(int * begin, int * end) { if (begin != end) { From 8d91f2cf2e457d9a5ead4a7cbb2d1bad372b6091 Mon Sep 17 00:00:00 2001 From: Bruce Hoppe <93284810+behoppe@users.noreply.github.com> Date: Tue, 20 Sep 2022 15:44:14 -0400 Subject: [PATCH 19/19] review convert a c++ program --- src/doc/users-guide/convert-a-c++-program.md | 176 ++++++++++--------- 1 file changed, 95 insertions(+), 81 deletions(-) diff --git a/src/doc/users-guide/convert-a-c++-program.md b/src/doc/users-guide/convert-a-c++-program.md index b0660f8a..010daa4a 100644 --- a/src/doc/users-guide/convert-a-c++-program.md +++ b/src/doc/users-guide/convert-a-c++-program.md @@ -4,27 +4,28 @@ author: Timothy Kaler date: 2022-07-20T16:22:55.620Z attribution: true --- -OpenCilk can be used to add parallelism to existing serial codes without changing the original program's semantics. Let us walk through the process of converting an existing serial C or C++ code to an OpenCilk parallel program and show how OpenCilk's suite of tools can be used to debug race-conditions and scalability bottlenecks. +OpenCilk can help you add parallelism to existing serial code without changing the original program's semantics. This guide will walk you through the process of converting an existing serial C or C++ code to an OpenCilk parallel program and show how OpenCilk's suite of tools can be used to debug race conditions and scalability bottlenecks. ## General workflow The typical process for adding parallelism to existing serial C or C++ programs using OpenCilk involves five steps: -1. **Debug serial code:** Verify the original program is correct. It is a good practice to write correct and well-tested serial code prior to attempting parallelization. Bugs that exist in the serial code will also exist after introducing parallelism, but may be more difficult to debug. +1. **Debug serial code:** Verify the original program is correct. It's good practice to write correct and well-tested serial code prior to attempting parallelization. Bugs that exist in the serial code will also exist after introducing parallelism, but may be more difficult to debug. 2. **Identify parallelism:** Identify regions of the code that could benefit from parallel execution. Typically, operations that are relatively long-running and/or tasks that can be performed independently are prime candidates for parallelization. -3. **Annotate parallelism:** Introduce parallelism to the code using the OpenCilk keywords \`cilk_for\`, \`cilk_spawn\`, and \`cilk_scope\`. These keywords are described in more depth in ???. A summary of their semantics is as follows: - +3. **Annotate parallelism:** Introduce parallelism to the code using the OpenCilk keywords {% defn "cilk_for" %}, {% defn "cilk_spawn" %}, and {% defn "cilk_scope" %}: * `cilk_for` identifies a loop for which all iterations can execute in parallel. * `cilk_spawn` indicates a call to a function (a "child") that can proceed in parallel with the caller (the "parent"). * `cilk_scope` indicates that all spawned children within the scoped region must complete before proceeding. -4. **Compile:** Compile the code using the OpenCilk compiler. On **Linux* OS** one invokes the OpenCilk compiler using the `clang` or `clang++` commands. One compiled, the program can be run on the local machine to test for correctness and measure performance. -5. **Verify absence of races:** Use OpenCilk's ***cilksan race detector*** to verify the absence of race conditions in the parallel program. If the parallelization of the original (correct) serial program contains no ***race conditions***, then the parallel program will produce the same result as the serial program. With the help of OpenCilk's tools, one can identify and resolve race conditions through the use of ***reducers***, locks, and recoding. +4. **Compile:** Compile the code using the [OpenCilk compiler](/doc/users-guide/getting-started/#using-the-compiler) (e.g., using the `clang` or `clang++` commands within your OpenCilk installation). One compiled, the program can be run on the local machine to test for correctness and measure performance. +5. **Verify absence of races:** Use OpenCilk's {% defn "Cilksan" %} race detector to verify the absence of race conditions in the parallel program. If the parallelization of the original (correct) serial program contains no {% defn "race condition", "race conditions" %}, then the parallel program will produce the same result as the serial program. With the help of OpenCilk's tools, one can identify and resolve race conditions through the use of {% defn "reducer", "reducers" %}, locks, and recoding. ## Example: Quicksort -Let us illustrate the process of parallelizing an existing serial code by walking through an example wherein we shall expose parallelism in a serial implementation of ***Quicksort*** ([http://en.wikipedia.org/wiki/Quicksort](http://en.wikipedia.org/wiki/Quicksort)). +We'll illustrate the process of parallelizing an existing serial code by walking through an example where we expose parallelism in a serial implementation of [quicksort](http://en.wikipedia.org/wiki/Quicksort). -Note that in this example we use the function name `sample_qsort` in order to avoid confusion with the Standard C Library `qsort` function. +{% alert "info" %} +***Note:*** We use the function name `sample_qsort` in order to avoid confusion with the Standard C Library `qsort` function. +{% endalert %} ```cilkc# #include @@ -37,53 +38,53 @@ Note that in this example we use the function name `sample_qsort` in order to av // This is pure C++ code before Cilk++ conversion. void sample_qsort(int * begin, int * end) { - if (begin != end) { - --end; // Exclude last element (pivot) - int * middle = std::partition(begin, end, - std::bind2nd(std::less(),*end)); - std::swap(*end, *middle); // pivot to middle - cilk_scope { - cilk_spawn sample_qsort(begin, middle); - sample_qsort(++middle, ++end); // Exclude pivot - } + if (begin != end) { + --end; // Exclude last element (pivot) + int * middle = std::partition(begin, end, + std::bind2nd(std::less(),*end)); + std::swap(*end, *middle); // pivot to middle + cilk_scope { + cilk_spawn sample_qsort(begin, middle); + sample_qsort(++middle, ++end); // Exclude pivot } + } } ``` -In the example code above, the serial quicksort code has been converted into a parallel OpenCilk code by adding the `cilk_spawn` keyword on line 9, and defining the `cilk_scope` region to include lines 9--10. The `cilk_spawn` keyword on line 9 indicates that the function call `sample_qsort(begin, middle)` is allowed to execute in-parallel with its ***continuation*** which includes the function call `sample_qsort(++middle, ++end)` on line 10. +In the example code above, the serial quicksort code has been converted into a parallel OpenCilk code by adding the `cilk_spawn` keyword on line 9, and defining the `cilk_scope` region to include lines 9-10. The `cilk_spawn` keyword on line 9 indicates that the function call `sample_qsort(begin, middle)` is allowed to execute in parallel with its ***continuation*** which includes the function call `sample_qsort(++middle, ++end)` on line 10. The `cilk_spawn` keyword can be thought of as allowing the recursive invocation of `sample_qsort` on line 10 to execute asynchronously. Thus, when we call `sample_qsort` again in line 10, the call at line 9 might not have completed. The end of the `cilk_scope` region at line 11 indicates that this function will not continue until all `cilk_spawn` requests in the same scoped region have completed. There is an implicit `cilk_scope` surrounding the body of every function so that at the end of every function all tasks spawned in the function have returned. -The parallelization of quicksort provided in this example implements a typical divide-and-conquer strategy for parallelizing recursive algorithms. At each level of recursion, we have two-way parallelism; the parent strand (line 10) continues executing the current function, while a child strand executes the other recursive call. In general, recursive divide-and-conquer algorithms can expose significant parallelism. In the case of quicksort, however, parallelizing according to the standard recursive structure of the serial algorithm only exposes limited parallelism. The reason for this is due to the substantial amount of work performed by the serial `partition` function invoked on line 5. The partition algorithm may be parallelized for better scalability, but we shall leave this task as an exercise to the intrepid reader. - ## Compile -This quicksort code can be compiled using the OpenCilk C++ compiler by adding `#include ` statement to the source file. The `cilk.h` header file contains declarations of the OpenCilk runtime API and the keywords used to specify parallel control flow. After adding the `cilk.h` header file, one can compile the quicksort program using the OpenCilk compiler. +Before you can compile your Cilk program with the OpenCilk compiler, you must add `#include ` to the source file. The `cilk.h` header file contains declarations of the OpenCilk runtime API and the keywords used to specify parallel control flow. After adding the `cilk.h` header file, you can compile the quicksort program using the `clang++` compiler in your [OpenCilk installation](/doc/users-guide/getting-started/#using-the-compiler). + +{% alert "info" %} + +_**Note:**_ This guide assumes that OpenCilk is installed within +`/opt/opencilk/` and that the OpenCilk C++ compiler can be invoked from the +terminal as `/opt/opencilk/bin/clang++`, as shown in [this +example](/doc/users-guide/install/#example). -##### Linux* OS +{% endalert %} -```shell -> clang++ qsort.cpp -o qsort –O3 -fopencilk +```shell-session +$ /opt/opencilk/bin/clang++ qsort.cpp -o qsort –O3 -fopencilk ``` ### Build, execute, and test Now that you have introduced parallelism into the quicksort program, you can build and execute the OpenCilk version of the qsort program with the command shown below. -##### Linux* OS: - -```shell -> clang++ qsort.cpp -o qsort –O3 -fopencilk +```shell-session +$ /opt/opencilk/bin/clang++ qsort.cpp -o qsort –O3 -fopencilk ``` -\ -The quicksort code can be run from then command line as shown below to verify correctness and measure its runtime performance. +The quicksort code can be run from the command line as shown below to verify correctness and measure its runtime performance. -```shell -> qsort +```shell-session +$ ./qsort Sorting 10000000 integers 5.641 seconds Sort succeeded. ``` -By default, an OpenCilk program will execute in-parallel using all of the cores available on the machine. You can control the number of workers for a particular execution by setting the CILK_NWORKERS environment variable as shown below. +By default, an OpenCilk program will execute in parallel using all of the cores available on the machine. You can control the number of workers for a particular execution by setting the `CILK_NWORKERS` environment variable as shown below. -```shell +```shell-session CILK_NWORKERS=8 ./qsort ``` -Using the CILK_NWORKERS environment one can measure the parallel speedup achieved by quicksort when varying the number of utilized cores. Below we show the result of running the quicksort program using one and two cores. +Using the `CILK_NWORKERS` environment variable, you can measure the parallel speedup achieved by quicksort when varying the number of utilized cores. Below we show the result of running the quicksort program using one and two cores. -```powershell -> CILK_NWORKERS=1 qsort +```shell-session +$ CILK_NWORKERS=1 qsort Sorting 10000000 integers 2.909 seconds Sort succeeded. -> CILK_NWORKERS=2 qsort +$ CILK_NWORKERS=2 qsort Sorting 10000000 integers 1.468 seconds Sort succeeded. ``` ## Verify absence of races -The Cilksan race detector can be used to check for race conditions in the parallelized quicksort code. To run Cilksan on our parallel quicksort routine, we must compile the program with Cilksan enabled and then execute the instrumented program. +The Cilksan race detector can be used to check for race conditions in the parallelized quicksort code. To run Cilksan on our parallel quicksort routine, you must compile the program with Cilksan enabled and then execute the instrumented program. -```shell -> clang++ qsort.cpp -o qsort –Og -g -fopencilk -fsanitize=cilk -./qsort 10000000 +```shell-session +$ /opt/opencilk/bin/clang++ qsort.cpp -o qsort –Og -g -fopencilk -fsanitize=cilk +$ ./qsort 10000000 Cilksan detected 0 distinct races. Cilksan suppressed 0 duplicate race reports. @@ -179,13 +182,13 @@ The Cilksan race detector will report any race conditions present in the program ## Measure scalability -Cilkscale can be used to benchmark and analyze the parallelism, in terms of work and span, of an OpenCilk program. These measurements can be used to predict performance when running on a varying number of parallel processors. +Cilkscale can be used to benchmark and analyze the parallelism, in terms of {% defn "work" %} and {% defn "span" %}, of an OpenCilk program. These measurements can be used to predict performance when running on a varying number of cores. One can use Cilkscale to benchmark the parallel scalability of quicksort by compiling with the additional flag `-fcilktool=cilkscale` and then executing the program as shown below. -```shell -> clang++ qsort.cpp -o qsort –O3 -fopencilk -fcilktool=cilkscale -./qsort 10000000 +```shell-session +$ /opt/opencilk/bin/clang++ qsort.cpp -o qsort –O3 -fopencilk -fcilktool=cilkscale +$ ./qsort 10000000 Sorting 10000000 integers All sorts succeeded @@ -195,10 +198,21 @@ tag,work (seconds),span (seconds),parallelism,burdened_span (seconds),burdened_p Cilkscale will report the total work, span, and parallelism in the code at the end of the program execution, as shown above. -The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](/doc/users-guide/getting-started/#using-cilkscale). +The Cilkscale tool can be used in conjunction with other benchmarking and visualization scripts provided as part of the OpenCilk toolbox. More information about the use of Cilkscale and related tools for benchmarking and visualizing parallel program performance can be found [here](/doc/users-guide/cilkscale). Plots illustrating the parallel execution time and speedup of the quicksort program we have parallelized in this example are shown below. ![Cilkscale speedup for quicksort.](/img/cilkscale-qsort-speedup.png "Quicksort speedup") -![Cilkscale execution time for quicksort.](/img/cilkscale-qsort-execution-time.png "Quicksort execution time") \ No newline at end of file +![Cilkscale execution time for quicksort.](/img/cilkscale-qsort-execution-time.png "Quicksort execution time") + +## Discussion + +We have seen how to convert a serial C++ program into a parallel Cilk program. So... what next? +We might seek to use the parallel processing enabled by OpenCilk to obtain additional performance improvements. +We will return to this topic with forthcoming +documentation and blog posts. Please [let us know](/contribute/contact/) if +you'd like to be notified about important updates to OpenCilk and its +documentation. + +The parallelization of quicksort provided in this example implements a typical divide-and-conquer strategy for parallelizing recursive algorithms. At each level of recursion, we have two-way parallelism; the parent strand continues executing the current function, while a child strand executes the other recursive call. In general, recursive divide-and-conquer algorithms can expose significant parallelism. In the case of quicksort, however, parallelizing according to the standard recursive structure of the serial algorithm only exposes limited parallelism. The reason for this is due to the substantial amount of work performed by the serial `partition` function. This function may be parallelized for better scalability, but we shall leave this task as an exercise to the intrepid reader.