diff --git a/.gitignore b/.gitignore
index 2691b61..1e2ced7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -159,8 +159,12 @@ cython_debug/
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
.idea/
+# MacOS
+.DS_Store
+
# Checkpoints and influence outputs
checkpoints/
analyses/
data/
-*.pth
\ No newline at end of file
+*.pth
+*.pt
\ No newline at end of file
diff --git a/DOCUMENTATION.md b/DOCUMENTATION.md
index 08c8d92..cb85859 100644
--- a/DOCUMENTATION.md
+++ b/DOCUMENTATION.md
@@ -1,5 +1,7 @@
# Kronfluence: Technical Documentation & FAQs
+For a detailed description of the methodology, see the [**paper**](https://arxiv.org/abs/2308.03296) *Studying Large Language Model Generalization with Influence Functions*.
+
## Requirements
Kronfluence has been tested on the following versions of [PyTorch](https://pytorch.org/):
@@ -10,7 +12,7 @@ Kronfluence has been tested on the following versions of [PyTorch](https://pytor
Kronfluence supports:
- Computing influence functions on selected PyTorch modules. At the moment, we support `nn.Linear` and `nn.Conv2d`;
-- Computing influence functions with several strategies: `identity`, `diagonal`, `KFAC`, and `EKFAC`;
+- Computing influence functions with several Hessian approximation strategies: `identity`, `diagonal`, `KFAC`, and `EKFAC`;
- Computing pairwise and self-influence scores.
> [!NOTE]
@@ -42,8 +44,8 @@ query_dataset = prepare_query_dataset()
**Define a Task.**
To compute influence scores, you need to define a [`Task`](https://github.com/pomonam/kronfluence/blob/main/kronfluence/task.py) class.
-This class encapsulates information about the trained model and how influence scores will be computed:
-(1) how to compute the training loss; (2) how to compute the measurement (f(θ) in the [paper](https://arxiv.org/abs/2308.03296));
+This class contains information about the trained model and how influence scores will be computed:
+(1) how to compute the training loss; (2) how to compute the measurable quantity (f(θ) in the [paper](https://arxiv.org/abs/2308.03296); see Equation 5);
(3) which modules to use for influence function computations; and (4) whether the model used [attention mask](https://huggingface.co/docs/transformers/en/glossary#attention-mask).
```python
@@ -59,6 +61,7 @@ class YourTask(Task):
model: nn.Module,
sample: bool = False,
) -> torch.Tensor:
+ # This will be used for computing the training gradient.
# TODO: Complete this method.
def compute_measurement(
@@ -66,6 +69,7 @@ class YourTask(Task):
batch: Any,
model: nn.Module,
) -> torch.Tensor:
+ # This will be used for computing the measurable quantity.
# TODO: Complete this method.
def tracked_modules(self) -> Optional[List[str]]:
@@ -129,7 +133,7 @@ scores = analyzer.compute_pairwise_scores(
...
```
-You can organize all factors and scores for the specific model with `factor_name` and `score_name`.
+You can organize all factors and scores for the specific model with `factors_name` and `scores_name`.
### FAQs
@@ -146,15 +150,15 @@ inspect `model.named_modules()` to determine what modules to use. You can specif
> [!NOTE]
> If the embedding layer for transformers are defined with `nn.Linear`, you must write
-> `task.tracked_modules` to avoid influence computations embedding matrices.
+> `task.tracked_modules` to avoid influence computations embedding matrices (it is too expensive).
**How should I implement Task.compute_train_loss?**
Implement the loss function used to train the model. Note that the function should return
the summed loss (over batches and tokens) and should not include regularizations.
**How should I implement Task.compute_measurement?**
-It depends on the analysis you would like to perform. Influence functions approximate the [local effect of downweighting/upweighting
-a data point on the query's measurable quantity](https://arxiv.org/abs/2209.05364). You can use the loss, [margin](https://arxiv.org/abs/2303.14186) (for classification),
+It depends on the analysis you would like to perform. Influence functions approximate the [effect of downweighting/upweighting
+a training data point on the query's measurable quantity](https://arxiv.org/abs/2209.05364). You can use the loss, [margin](https://arxiv.org/abs/2303.14186) (for classification),
or [conditional log-likelihood](https://arxiv.org/abs/2308.03296) (for language modeling).
**I encounter TrackedModuleNotFoundError when using DDP or FSDP.**
@@ -188,7 +192,7 @@ import torch
from kronfluence.arguments import FactorArguments
factor_args = FactorArguments(
- strategy="ekfac", # Choose from "identity", "diagonal", "KFAC", or "EKFAC".
+ strategy="ekfac", # Choose from "identity", "diagonal", "kfac", or "ekfac".
use_empirical_fisher=False,
immediate_gradient_removal=False,
ignore_bias=False,
@@ -217,8 +221,8 @@ analyzer.fit_all_factors(factors_name="initial_factor", dataset=train_dataset, f
```
You can change:
-- `strategy`: Selects the preconditioning strategy (`identity`, `diagonal`, `KFAC`, or `EKFAC`).
-- `use_empirical_fisher`: Determines whether to approximate the [empirical Fisher](https://arxiv.org/abs/1905.12558) (using actual labels from batch)
+- `strategy`: Selects the Hessian approximation strategy (`identity`, `diagonal`, `KFAC`, or `EKFAC`).
+- `use_empirical_fisher`: Determines whether to use the [empirical Fisher](https://arxiv.org/abs/1905.12558) (using actual labels from batch)
instead of the true Fisher (using sampled labels from model's predictions). It is recommended to be `False`.
- `immediate_gradient_removal`: Specifies whether to instantly set `param.grad = None` within module hooks. Generally,
recommended to be `False`, as it requires installing additional hooks. This should not affect the fitted factors, but
@@ -227,7 +231,7 @@ can potentially reduce peak memory.
### Fitting Covariance Matrices
-`KFAC` and `EKFAC` require computing the activation and pseudo-gradient covariance matrices.
+`KFAC` and `EKFAC` require computing the uncentered activation and pre-activation pseudo-gradient covariance matrices.
To fit covariance matrices, you can use `analyzer.fit_covariance_matrices`.
```python
# Fitting covariance matrices.
@@ -236,7 +240,7 @@ analyzer.fit_covariance_matrices(factors_name="initial_factor", dataset=train_da
covariance_matrices = analyzer.load_covariance_matrices(factors_name="initial_factor")
```
-You can tune:
+This step corresponds to Equation 16 in the paper. You can tune:
- `covariance_max_examples`: Controls the maximum number of data points for fitting covariance matrices. Setting it to `None`,
Kronfluence computes covariance matrices for all data points.
- `covariance_data_partition_size`: Number of data partitions to use for computing covariance matrices.
@@ -244,15 +248,13 @@ For example, when `covariance_data_partition_size = 2`, the dataset is split int
are separately computed for each chunk. These chunked covariance matrices are later aggregated. This is useful with GPU preemption as intermediate
covariance matrices will be saved in disk. It can be also helpful when launching multiple parallel jobs, where each GPU
can compute covariance matrices on some partitioned data (You can specify `target_data_partitions` in the parameter).
-This should not affect the quality of the fitted factors.
- `covariance_module_partition_size`: Number of module partitions to use for computing covariance matrices.
For example, when `covariance_module_partition_size = 2`, the module is split into 2 chunks and covariance matrices
are separately computed for each chunk. This is useful when the available GPU memory is limited (e.g., the total
covariance matrices cannot fit into memory). However, this will do multiple iterations over the dataset and can be slow.
-This should not affect the quality of the fitted factors.
- `activation_covariance_dtype`: `dtype` for computing activation covariance matrices. You can also use `torch.bfloat16`
or `torch.float16`.
-- `gradient_covariance_dtype`: `dtype` for computing activation covariance matrices. You can also use `torch.bfloat16`
+- `gradient_covariance_dtype`: `dtype` for computing pre-activation pseudo-gradient covariance matrices. You can also use `torch.bfloat16`
or `torch.float16`.
**Dealing with OOMs.** Here are some steps to fix Out of Memory (OOM) errors.
@@ -264,7 +266,7 @@ or `torch.float16`.
### Performing Eigendecomposition
-After computing the covariance matrices, `KFAC` and `EKFAC` require performing Eigendecomposition.
+After computing the covariance matrices, `KFAC` and `EKFAC` require performing eigendecomposition.
```python
# Performing Eigendecomposition.
@@ -273,13 +275,13 @@ analyzer.perform_eigendecomposition(factors_name="initial_factor", factor_args=f
eigen_factors = analyzer.load_eigendecomposition(factors_name="initial_factor")
```
-You can tune:
-- `eigendecomposition_dtype`: `dtype` for performing Eigendecomposition. You can also use `torch.float32`,
+This corresponds to Equation 18 in the paper. You can tune:
+- `eigendecomposition_dtype`: `dtype` for performing eigendecomposition. You can also use `torch.float32`,
but `torch.float64` is recommended.
### Fitting Lambda Matrices
-`EKFAC` and `diagonal` require computing the Lambda matrices for all modules.
+`EKFAC` and `diagonal` require computing the Lambda (eigenvalue) matrices for all modules.
```python
# Fitting Lambda matrices.
@@ -288,7 +290,7 @@ analyzer.fit_lambda_matrices(factors_name="initial_factor", dataset=train_datase
lambda_matrices = analyzer.load_lambda_matrices(factors_name="initial_factor")
```
-You can tune:
+This corresponds to Equation 20 in the paper. You can tune:
- `lambda_max_examples`: Controls the maximum number of data points for fitting Lambda matrices.
- `lambda_data_partition_size`: Number of data partitions to use for computing Lambda matrices.
- `lambda_module_partition_size`: Number of module partitions to use for computing Lambda matrices.
@@ -297,7 +299,7 @@ You can set `cached_activation_cpu_offload=True` to cache these activations in C
- `lambda_iterative_aggregate`: Whether to compute the Lambda matrices with for-loop instead of batched matrix multiplications.
This is helpful for reducing peak memory, as it avoids holding multiple copies of tensors with the same shape as the per-sample-gradient.
- `lambda_dtype`: `dtype` for computing Lambda matrices. You can also use `torch.bfloat16`
-or `torch.float16`, but `torch.float32` is generally recommended.
+or `torch.float16`.
**Dealing with OOMs.** Here are some steps to fix Out of Memory (OOM) errors.
@@ -312,7 +314,7 @@ or `torch.float16`, but `torch.float32` is generally recommended.
**I get different factors each time I run the code.**
This is expected as we sample labels from the model's prediction when computing covariance and Lambda matrices.
Using `use_empirical_fisher=True` could make the process more deterministic. Moreover, different hardware might compute
-different eigenvectors when performing Eigendecomposition.
+different eigenvectors when performing eigendecomposition.
**How should I select the batch size?**
You can use the largest possible batch size that does not result in OOM. Typically, the batch size for fitting Lambda
@@ -347,16 +349,14 @@ score_args = ScoreArguments(
- `damping`: A damping factor for the damped matrix-vector product. Uses a heuristic based on mean eigenvalues
(0.1 x mean eigenvalues) if None.
-- `immediate_gradient_removal`: Whether to immediately remove `param.grad` within a hook. This should be set to
-`False` in most cases.
+- `immediate_gradient_removal`: Whether to immediately remove `param.grad` within a hook.
- `data_partition_size`: Number of data partitions for computing influence scores.
- `module_partition_size`: Number of module partitions for computing influence scores.
- `per_module_score`: Whether to return a per-module influence scores. Instead of summing over influences across
all modules, this will keep track of intermediate module-wise scores.
-- `query_gradient_rank`: The rank for the query batching. If `None`, no query batching will be used.
-- `query_gradient_svd_dtype`: `dtype` for performing singular value decomposition (SVD) for query batch. You can use `torch.float32`,
-but `torch.float64` is recommended.
+- `query_gradient_rank`: The rank for the query batching (low-rank approximation to the query gradient; see Section 3.2.2). If `None`, no query batching will be used.
+- `query_gradient_svd_dtype`: `dtype` for performing singular value decomposition (SVD) for query batch. You can also use `torch.float32`.
- `cached_activation_cpu_offload`: Whether to offload cached activations to CPU.
- `score_dtype`: `dtype` for computing influence scores. You can use `torch.bfloat16` or `torch.float16`.
@@ -366,7 +366,7 @@ but `torch.float32` is recommended.
### Computing Influence Scores
-To compute pairwise influence scores, you can run:
+To compute pairwise influence scores (Equation 5 in the paper), you can run:
```python
# Computing pairwise influence scores.
analyzer.compute_pairwise_scores(scores_name="pairwise", factors_name="ekfac", score_args=score_args)
@@ -374,7 +374,7 @@ analyzer.compute_pairwise_scores(scores_name="pairwise", factors_name="ekfac", s
scores = analyzer.load_pairwise_scores(scores_name="pairwise")
```
-To compute self-influence scores, you can run:
+To compute self-influence scores (see Section 5.4 from [paper](https://arxiv.org/pdf/1703.04730.pdf)), you can run:
```python
# Computing pairwise influence scores.
analyzer.compute_self_scores(scores_name="self", factors_name="ekfac", score_args=score_args)
@@ -388,14 +388,14 @@ scores = analyzer.load_self_scores(scores_name="self")
3. Try using lower precision for `per_sample_gradient_dtype` and `score_dtype`.
4. Try setting `immediate_gradient_removal=True`.
5. Try setting `query_gradient_rank > 1`. The recommended values are `16`, `32`, `64`, `128`, and `256`. Note that query
-batching is only supported for computing pairwise influence scores.
+batching is only supported for computing pairwise influence scores, not self-infleucen scores.
6. Try setting `module_partition_size > 1`.
### FAQs
**Influence scores are very large in magnitude.**
Ideally, influence scores need to be divided by the total number of training data points. However, the code does
-not normalize the scores. If you would like, you can divide the scores with the total number of data points used to
+not normalize the scores. If you would like, you can divide the scores with the total number of data points (or tokens) used to
train the model.
## References
diff --git a/README.md b/README.md
index 85db38a..3756a4d 100644
--- a/README.md
+++ b/README.md
@@ -2,7 +2,11 @@
+
+
@@ -19,7 +23,7 @@
---
-> **Kronfluence** is a repository designed to compute [influence functions](https://arxiv.org/abs/1703.04730) using [Kronecker-factored Approximate Curvature (KFAC)](https://arxiv.org/abs/1503.05671) or [Eigenvalue-corrected KFAC (EKFAC)](https://arxiv.org/abs/1806.03884).
+> **Kronfluence** is a research repository designed to compute [influence functions](https://arxiv.org/abs/1703.04730) using [Kronecker-factored Approximate Curvature (KFAC)](https://arxiv.org/abs/1503.05671) or [Eigenvalue-corrected KFAC (EKFAC)](https://arxiv.org/abs/1806.03884).
For a detailed description of the methodology, see the [**paper**](https://arxiv.org/abs/2308.03296) *Studying Large Language Model Generalization with Influence Functions*.
---
@@ -53,10 +57,10 @@ pip install -e .
Kronfluence supports influence computations on `nn.Linear` and `nn.Conv2d` modules. See the [**Technical Documentation**](https://github.com/pomonam/kronfluence/blob/main/DOCUMENTATION.md)
page for a comprehensive guide.
-### Examples
+### Learn More
-The [examples](https://github.com/pomonam/kronfluence/tree/main/examples) folder contains several examples on how to use Kronfluence.
-We plan to add more language model examples. **TL;DR** You need to prepare the trained model and datasets, and pass them into the `Analyzer`.
+The [examples](https://github.com/pomonam/kronfluence/tree/main/examples) folder contains several examples.
+We plan to add more examples in the future. **TL;DR** You need to prepare the trained model and datasets, and pass them into `Analyzer`.
```python
import torch
@@ -90,27 +94,27 @@ eval_dataset = torchvision.datasets.MNIST(
train=True,
)
-# Initialize the task with relevant loss and measurement.
+# Define the task.
task = MnistTask()
-# Prepare the model for influence computation with the specified task.
+# Prepare the model for influence computation.
model = prepare_model(model=model, task=task)
analyzer = Analyzer(analysis_name="mnist", model=model, task=task)
-# Fit all EKFAC factors for the given model on the training dataset.
-analyzer.fit_all_factors(factors_name="ekfac", dataset=train_dataset)
+# Fit all EKFAC factors for the given model.
+analyzer.fit_all_factors(factors_name="my_factors", dataset=train_dataset)
-# Compute all pairwise influence scores using the computed factors.
+# Compute all pairwise influence scores with the computed factors.
analyzer.compute_pairwise_scores(
- scores_name="pairwise_scores",
- factors_name="ekfac",
+ scores_name="my_scores",
+ factors_name="my_factors",
query_dataset=eval_dataset,
train_dataset=train_dataset,
per_device_query_batch_size=1024,
)
# Load the scores with dimension `len(eval_dataset) x len(train_dataset)`.
-scores = analyzer.load_pairwise_scores(scores_name="pairwise_scoeres")
+scores = analyzer.load_pairwise_scores(scores_name="my_scores")
```
## Contributing
diff --git a/examples/_test_requirements.txt b/examples/_test_requirements.txt
deleted file mode 100644
index ec96d22..0000000
--- a/examples/_test_requirements.txt
+++ /dev/null
@@ -1,3 +0,0 @@
-scikit-learn
-jupyter
-evaluate
\ No newline at end of file
diff --git a/examples/cifar/README.md b/examples/cifar/README.md
new file mode 100644
index 0000000..67b16d5
--- /dev/null
+++ b/examples/cifar/README.md
@@ -0,0 +1,57 @@
+# CIFAR-10 & ResNet-9 Example
+
+This directory contains scripts for training ResNet-9 on CIFAR-10. The pipeline is motivated from
+[TRAK repository](https://github.com/MadryLab/trak/blob/main/examples/cifar_quickstart.ipynb).
+
+## Training
+
+To train ResNet-9 on CIFAR-10 dataset, run the following command:
+```bash
+python train.py --dataset_dir ./data \
+ --checkpoint_dir ./checkpoints \
+ --train_batch_size 512 \
+ --eval_batch_size 1024 \
+ --learning_rate 0.4 \
+ --weight_decay 0.0001 \
+ --num_train_epochs 25 \
+ --seed 1004
+```
+
+## Computing Pairwise Influence Scores
+
+To obtain pairwise influence scores on 2000 query data points using `ekfac`, run the following command:
+```bash
+python analyze.py --query_batch_size 1000 \
+ --dataset_dir ./data \
+ --checkpoint_dir ./checkpoints \
+ --factor_strategy ekfac
+```
+You can also use `identity`, `diagonal`, and `kfac`. On A100 (80GB), it takes roughly 1.5 minutes to compute the
+pairwise scores (including computing EKFAC factors).
+
+## Mislabeled Data Detection
+
+We can use self-influence scores (see Section 5.4 for the [paper](https://arxiv.org/pdf/1703.04730.pdf)) to detect mislabeled examples.
+First, train the model with 10% of training examples mislabeled by running the following command:
+```bash
+python train.py --dataset_dir ./data \
+ --corrupt_percentage 0.1 \
+ --checkpoint_dir ./checkpoints \
+ --train_batch_size 512 \
+ --eval_batch_size 1024 \
+ --learning_rate 0.4 \
+ --weight_decay 0.0001 \
+ --num_train_epochs 25 \
+ --seed 1004
+```
+
+Then, compute self-influence scores with the following command:
+```bash
+python detect_mislabeled_dataset.py --dataset_dir ./data \
+ --corrupt_percentage 0.1 \
+ --checkpoint_dir ./checkpoints \
+ --factor_strategy ekfac
+```
+
+On A100 (80GB), it takes roughly 1.5 minutes to compute the self-influence scores.
+We can detect around 82% of mislabeled data points by inspecting 10% of the dataset (96% by inspecting 20%).
\ No newline at end of file
diff --git a/examples/cifar/analyze.py b/examples/cifar/analyze.py
new file mode 100644
index 0000000..8669924
--- /dev/null
+++ b/examples/cifar/analyze.py
@@ -0,0 +1,157 @@
+import argparse
+import logging
+import os
+from typing import Tuple
+
+import torch
+import torch.nn.functional as F
+from torch import nn
+
+from examples.cifar.pipeline import construct_resnet9, get_cifar10_dataset
+from kronfluence.analyzer import Analyzer, prepare_model
+from kronfluence.arguments import FactorArguments
+from kronfluence.task import Task
+from kronfluence.utils.dataset import DataLoaderKwargs
+
+BATCH_TYPE = Tuple[torch.Tensor, torch.Tensor]
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description="Influence analysis on CIFAR-10 dataset.")
+
+ parser.add_argument(
+ "--corrupt_percentage",
+ type=float,
+ default=None,
+ help="Percentage of the training dataset to corrupt.",
+ )
+ parser.add_argument(
+ "--dataset_dir",
+ type=str,
+ default="./data",
+ help="A folder to download or load CIFAR-10 dataset.",
+ )
+ parser.add_argument(
+ "--checkpoint_dir",
+ type=str,
+ default="./checkpoints",
+ help="A path that is storing the final checkpoint of the model.",
+ )
+
+ parser.add_argument(
+ "--query_batch_size",
+ type=int,
+ default=1000,
+ help="Batch size for computing query gradients.",
+ )
+ parser.add_argument(
+ "--factor_strategy",
+ type=str,
+ default="ekfac",
+ help="Strategy to compute influence factors.",
+ )
+
+ args = parser.parse_args()
+
+ if args.checkpoint_dir is not None:
+ os.makedirs(args.checkpoint_dir, exist_ok=True)
+
+ return args
+
+
+class ClassificationTask(Task):
+ def compute_train_loss(
+ self,
+ batch: BATCH_TYPE,
+ model: nn.Module,
+ sample: bool = False,
+ ) -> torch.Tensor:
+ inputs, labels = batch
+ logits = model(inputs)
+ if not sample:
+ return F.cross_entropy(logits, labels, reduction="sum")
+ with torch.no_grad():
+ probs = torch.nn.functional.softmax(logits, dim=-1)
+ sampled_labels = torch.multinomial(
+ probs,
+ num_samples=1,
+ ).flatten()
+ return F.cross_entropy(logits, sampled_labels.detach(), reduction="sum")
+
+ def compute_measurement(
+ self,
+ batch: BATCH_TYPE,
+ model: nn.Module,
+ ) -> torch.Tensor:
+ # Copied from: https://github.com/MadryLab/trak/blob/main/trak/modelout_functions.py.
+ inputs, labels = batch
+ logits = model(inputs)
+
+ bindex = torch.arange(logits.shape[0]).to(device=logits.device, non_blocking=False)
+ logits_correct = logits[bindex, labels]
+
+ cloned_logits = logits.clone()
+ cloned_logits[bindex, labels] = torch.tensor(-torch.inf, device=logits.device, dtype=logits.dtype)
+
+ margins = logits_correct - cloned_logits.logsumexp(dim=-1)
+ return -margins.sum()
+
+
+def main():
+ args = parse_args()
+ logging.basicConfig(level=logging.INFO)
+
+ # Prepare the dataset.
+ train_dataset = get_cifar10_dataset(
+ split="eval_train", corrupt_percentage=args.corrupt_percentage, dataset_dir=args.dataset_dir
+ )
+ eval_dataset = get_cifar10_dataset(split="valid", dataset_dir=args.dataset_dir)
+
+ # Prepare the trained model.
+ model = construct_resnet9()
+ model_name = "model"
+ if args.corrupt_percentage is not None:
+ model_name += "_corrupt_" + str(args.corrupt_percentage)
+ checkpoint_path = os.path.join(args.checkpoint_dir, f"{model_name}.pth")
+ if not os.path.isfile(checkpoint_path):
+ raise ValueError(f"No checkpoint found at {checkpoint_path}.")
+ model.load_state_dict(torch.load(checkpoint_path))
+
+ # Define task and prepare model.
+ task = ClassificationTask()
+ model = prepare_model(model, task)
+
+ analyzer = Analyzer(
+ analysis_name="cifar10",
+ model=model,
+ task=task,
+ )
+ # Configure parameters for DataLoader.
+ dataloader_kwargs = DataLoaderKwargs(num_workers=4)
+ analyzer.set_dataloader_kwargs(dataloader_kwargs)
+
+ # Compute influence factors.
+ factor_args = FactorArguments(strategy=args.factor_strategy)
+ analyzer.fit_all_factors(
+ factors_name=args.factor_strategy,
+ dataset=train_dataset,
+ per_device_batch_size=None,
+ factor_args=factor_args,
+ overwrite_output_dir=False,
+ )
+ # Compute pairwise scores.
+ analyzer.compute_pairwise_scores(
+ scores_name=args.factor_strategy,
+ factors_name=args.factor_strategy,
+ query_dataset=eval_dataset,
+ query_indices=list(range(2000)),
+ train_dataset=train_dataset,
+ per_device_query_batch_size=args.query_batch_size,
+ overwrite_output_dir=False,
+ )
+ scores = analyzer.load_pairwise_scores(args.factor_strategy)["all_modules"]
+ logging.info(f"Scores shape: {scores.shape}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/cifar/detect_mislabeled_dataset.py b/examples/cifar/detect_mislabeled_dataset.py
new file mode 100644
index 0000000..efcd84f
--- /dev/null
+++ b/examples/cifar/detect_mislabeled_dataset.py
@@ -0,0 +1,117 @@
+import argparse
+import logging
+import os
+
+import torch
+
+from examples.cifar.analyze import ClassificationTask
+from examples.cifar.pipeline import construct_resnet9, get_cifar10_dataset
+from kronfluence.analyzer import Analyzer, prepare_model
+from kronfluence.arguments import FactorArguments
+from kronfluence.utils.dataset import DataLoaderKwargs
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description="Detecting mislabeled CIFAR-10 data points.")
+
+ parser.add_argument(
+ "--corrupt_percentage",
+ type=float,
+ default=0.1,
+ help="Percentage of the training dataset to corrupt.",
+ )
+ parser.add_argument(
+ "--dataset_dir",
+ type=str,
+ default="./data",
+ help="A folder to download or load CIFAR-10 dataset.",
+ )
+ parser.add_argument(
+ "--checkpoint_dir",
+ type=str,
+ default="./checkpoints",
+ help="A path that is storing the final checkpoint of the model.",
+ )
+
+ parser.add_argument(
+ "--factor_strategy",
+ type=str,
+ default="ekfac",
+ help="Strategy to compute influence factors.",
+ )
+
+ args = parser.parse_args()
+
+ if args.checkpoint_dir is not None:
+ os.makedirs(args.checkpoint_dir, exist_ok=True)
+
+ return args
+
+
+def main():
+ args = parse_args()
+ logging.basicConfig(level=logging.INFO)
+
+ # Prepare the dataset.
+ train_dataset = get_cifar10_dataset(
+ split="eval_train", corrupt_percentage=args.corrupt_percentage, dataset_dir=args.dataset_dir
+ )
+
+ # Prepare the trained model.
+ model = construct_resnet9()
+ model_name = "model"
+ if args.corrupt_percentage is not None:
+ model_name += "_corrupt_" + str(args.corrupt_percentage)
+ checkpoint_path = os.path.join(args.checkpoint_dir, f"{model_name}.pth")
+ if not os.path.isfile(checkpoint_path):
+ raise ValueError(f"No checkpoint found at {checkpoint_path}.")
+ model.load_state_dict(torch.load(checkpoint_path))
+
+ # Define task and prepare model.
+ task = ClassificationTask()
+ model = prepare_model(model, task)
+
+ analyzer = Analyzer(
+ analysis_name="mislabeled",
+ model=model,
+ task=task,
+ )
+ # Configure parameters for DataLoader.
+ dataloader_kwargs = DataLoaderKwargs(num_workers=4)
+ analyzer.set_dataloader_kwargs(dataloader_kwargs)
+
+ # Compute influence factors.
+ factor_args = FactorArguments(strategy=args.factor_strategy)
+ analyzer.fit_all_factors(
+ factors_name=args.factor_strategy,
+ dataset=train_dataset,
+ per_device_batch_size=None,
+ factor_args=factor_args,
+ overwrite_output_dir=False,
+ )
+ # Compute self-influence scores.
+ analyzer.compute_self_scores(
+ scores_name=args.factor_strategy,
+ factors_name=args.factor_strategy,
+ train_dataset=train_dataset,
+ overwrite_output_dir=True,
+ )
+ scores = analyzer.load_pairwise_scores(args.factor_strategy)["all_modules"]
+
+ total_corrupt_size = int(args.corrupt_percentage * len(train_dataset))
+ corrupted_indices = list(range(int(args.corrupt_percentage * len(train_dataset))))
+ intervals = torch.arange(0.1, 1, 0.1)
+
+ accuracies = []
+ for interval in intervals:
+ interval = interval.item()
+ predicted_indices = torch.argsort(scores, descending=True)[: int(interval * len(train_dataset))]
+ predicted_indices = list(predicted_indices.numpy())
+ accuracies.append(len(set(predicted_indices) & set(corrupted_indices)) / total_corrupt_size)
+
+ logging.info(f"Inspect Interval: {list(intervals.numpy())}")
+ logging.info(f"Detection Accuracy: {accuracies}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/cifar/pipeline.py b/examples/cifar/pipeline.py
index 420d1fd..dd1f68e 100644
--- a/examples/cifar/pipeline.py
+++ b/examples/cifar/pipeline.py
@@ -33,7 +33,7 @@ def forward(self, x: torch.Tensor) -> torch.Tensor:
def construct_resnet9() -> nn.Module:
- # ResNet-9 architecture from: https://github.com/MadryLab/trak/blob/main/examples/cifar_quickstart.ipynb.
+ # ResNet-9 architecture from https://github.com/MadryLab/trak/blob/main/examples/cifar_quickstart.ipynb.
def conv_bn(
channels_in: int,
channels_out: int,
@@ -81,7 +81,7 @@ def get_cifar10_dataset(
assert split in ["train", "eval_train", "valid"]
normalize = torchvision.transforms.Normalize(mean=(0.4914, 0.4822, 0.4465), std=(0.247, 0.243, 0.261))
- if split in ["train", "eval_train"]:
+ if split == "train":
transform_config = torchvision.transforms.Compose(
[
torchvision.transforms.RandomCrop(32, padding=4),
diff --git a/examples/cifar/train.py b/examples/cifar/train.py
index 2228a21..1d76f79 100644
--- a/examples/cifar/train.py
+++ b/examples/cifar/train.py
@@ -1,6 +1,7 @@
import argparse
import logging
import os
+import time
from typing import Tuple
import numpy as np
@@ -10,7 +11,6 @@
from torch import nn
from torch.optim import lr_scheduler
from torch.utils import data
-from tqdm import tqdm
from examples.cifar.pipeline import construct_resnet9, get_cifar10_dataset
@@ -55,7 +55,7 @@ def parse_args():
parser.add_argument(
"--weight_decay",
type=float,
- default=0.001,
+ default=0.0001,
help="Weight decay to train the model.",
)
parser.add_argument(
@@ -92,7 +92,6 @@ def train(
num_train_epochs: int,
learning_rate: float,
weight_decay: float,
- disable_tqdm: bool = False,
) -> nn.Module:
train_dataloader = data.DataLoader(
dataset=dataset,
@@ -105,7 +104,7 @@ def train(
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=weight_decay)
iters_per_epoch = len(train_dataloader)
- lr_peak_epoch = num_train_epochs // 4
+ lr_peak_epoch = num_train_epochs // 5
lr_schedule = np.interp(
np.arange((num_train_epochs + 1) * iters_per_epoch),
[0, lr_peak_epoch * iters_per_epoch, num_train_epochs * iters_per_epoch],
@@ -113,22 +112,24 @@ def train(
)
scheduler = lr_scheduler.LambdaLR(optimizer, lr_schedule.__getitem__)
+ start_time = time.time()
model.train()
for epoch in range(num_train_epochs):
total_loss = 0.0
- with tqdm(train_dataloader, unit="batch", disable=disable_tqdm) as tepoch:
- for batch in tepoch:
- tepoch.set_description(f"Epoch {epoch}")
- model.zero_grad()
- inputs, labels = batch
- inputs, labels = inputs.to(DEVICE), labels.to(DEVICE)
- outputs = model(inputs)
- loss = F.cross_entropy(outputs, labels)
- loss.backward()
- optimizer.step()
- scheduler.step()
- total_loss += loss.detach().float()
- tepoch.set_postfix(loss=total_loss.item() / len(train_dataloader))
+ for batch in train_dataloader:
+ model.zero_grad()
+ inputs, labels = batch
+ inputs, labels = inputs.to(DEVICE), labels.to(DEVICE)
+ outputs = model(inputs)
+ loss = F.cross_entropy(outputs, labels)
+ loss.backward()
+ optimizer.step()
+ scheduler.step()
+ total_loss += loss.detach().float()
+ logging.info(f"Epoch {epoch + 1} - Averaged Loss: {total_loss / len(dataset)}")
+ end_time = time.time()
+ elapsed_time = end_time - start_time
+ logging.info(f"Completed training in {elapsed_time:.2f} seconds.")
return model
diff --git a/examples/glue/README.md b/examples/glue/README.md
new file mode 100644
index 0000000..12398a1
--- /dev/null
+++ b/examples/glue/README.md
@@ -0,0 +1,60 @@
+# GLUE & BERT Example
+
+This directory contains scripts for fine-tuning BERT on GLUE benchmark. The pipeline is motivated from [HuggingFace Example](https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification).
+Please begin by installing necessary packages.
+```bash
+pip install -r requirements.txt
+```
+
+## Training
+
+To fine-tune BERT on some specific dataset, run the following command (we are using `SST2` dataset):
+```bash
+python train.py --dataset_name sst2 \
+ --checkpoint_dir ./checkpoints \
+ --train_batch_size 32 \
+ --eval_batch_size 32 \
+ --learning_rate 3e-05 \
+ --weight_decay 0.01 \
+ --num_train_epochs 3 \
+ --seed 1004
+```
+
+## Computing Pairwise Influence Scores
+
+To obtain a pairwise influence scores on maximum of 2000 query data points using `ekfac`, run the following command:
+```bash
+python analyze.py --dataset_name sst2 \
+ --query_batch_size 175 \
+ --train_batch_size 128 \
+ --checkpoint_dir ./checkpoints \
+ --factor_strategy ekfac
+```
+On A100 (80GB), it takes roughly 80 minutes to compute the pairwise scores for SST2 with around 900 query data points
+(including computing EKFAC factors).
+
+We can also use query batching (low-rank approximation to the query gradient; see Section 3.2.2 from the [paper](https://arxiv.org/pdf/2308.03296.pdf)) to compute influence scores with a
+larger query batch size.
+```bash
+python analyze.py --dataset_name sst2 \
+ --query_gradient_rank 32 \
+ --query_batch_size 436 \
+ --train_batch_size 256 \
+ --checkpoint_dir ./checkpoints \
+ --factor_strategy ekfac
+```
+Note that query batching is slower in this case (140 minutes in total), as the number of training data points is small and the cost of performing SVD dominates the overall cost.
+Assuming that you ran above two commands, `query_batching_analysis.py` contains code to compute the correlations between the full rank and low-rank scores.
+
+
+
+The averaged correlations between the low-rank and full rank scores for 100 data points is 0.98.
+
+## Counterfactual Evaluation
+
+We plan to add a simple demo for counterfactual evaluation on the RTE dataset soon.
+
+
+
\ No newline at end of file
diff --git a/examples/glue/analyze.py b/examples/glue/analyze.py
new file mode 100644
index 0000000..022dfd9
--- /dev/null
+++ b/examples/glue/analyze.py
@@ -0,0 +1,187 @@
+import argparse
+import logging
+import os
+from typing import Dict, Optional
+
+import torch
+import torch.nn.functional as F
+from torch import nn
+from transformers import default_data_collator
+
+from examples.glue.pipeline import construct_bert, get_glue_dataset
+from kronfluence.analyzer import Analyzer, prepare_model
+from kronfluence.arguments import FactorArguments, ScoreArguments
+from kronfluence.task import Task
+from kronfluence.utils.dataset import DataLoaderKwargs
+
+BATCH_TYPE = Dict[str, torch.Tensor]
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description="Influence analysis on GLUE dataset.")
+
+ parser.add_argument(
+ "--dataset_name",
+ type=str,
+ default="sst2",
+ help="A name of GLUE dataset.",
+ )
+ parser.add_argument(
+ "--checkpoint_dir",
+ type=str,
+ default="./checkpoints",
+ help="A path that is storing the final checkpoint of the model.",
+ )
+
+ parser.add_argument(
+ "--query_gradient_rank",
+ type=int,
+ default=-1,
+ help="Rank for the low-rank query gradient approximation.",
+ )
+ parser.add_argument(
+ "--query_batch_size",
+ type=int,
+ default=100,
+ help="Batch size for computing query gradients.",
+ )
+ parser.add_argument(
+ "--train_batch_size",
+ type=int,
+ default=128,
+ help="Batch size for computing training gradients.",
+ )
+ parser.add_argument(
+ "--factor_strategy",
+ type=str,
+ default="ekfac",
+ help="Strategy to compute influence factors.",
+ )
+
+ args = parser.parse_args()
+
+ if args.checkpoint_dir is not None:
+ os.makedirs(args.checkpoint_dir, exist_ok=True)
+ args.checkpoint_dir = os.path.join(args.checkpoint_dir, args.dataset_name)
+ os.makedirs(args.checkpoint_dir, exist_ok=True)
+
+ return args
+
+
+class TextClassificationTask(Task):
+ def compute_train_loss(
+ self,
+ batch: BATCH_TYPE,
+ model: nn.Module,
+ sample: bool = False,
+ ) -> torch.Tensor:
+ logits = model(
+ input_ids=batch["input_ids"],
+ attention_mask=batch["attention_mask"],
+ token_type_ids=batch["token_type_ids"],
+ ).logits
+
+ if not sample:
+ return F.cross_entropy(logits, batch["labels"], reduction="sum")
+ with torch.no_grad():
+ probs = torch.nn.functional.softmax(logits, dim=-1)
+ sampled_labels = torch.multinomial(
+ probs,
+ num_samples=1,
+ ).flatten()
+ return F.cross_entropy(logits, sampled_labels.detach(), reduction="sum")
+
+ def compute_measurement(
+ self,
+ batch: BATCH_TYPE,
+ model: nn.Module,
+ ) -> torch.Tensor:
+ # Copied from: https://github.com/MadryLab/trak/blob/main/trak/modelout_functions.py.
+ logits = model(
+ input_ids=batch["input_ids"],
+ attention_mask=batch["attention_mask"],
+ token_type_ids=batch["token_type_ids"],
+ ).logits
+
+ labels = batch["labels"]
+ bindex = torch.arange(logits.shape[0]).to(device=logits.device, non_blocking=False)
+ logits_correct = logits[bindex, labels]
+
+ cloned_logits = logits.clone()
+ cloned_logits[bindex, labels] = torch.tensor(-torch.inf, device=logits.device, dtype=logits.dtype)
+
+ margins = logits_correct - cloned_logits.logsumexp(dim=-1)
+ return -margins.sum()
+
+ def get_attention_mask(self, batch: BATCH_TYPE) -> Optional[torch.Tensor]:
+ return batch["attention_mask"]
+
+
+def main():
+ args = parse_args()
+ logging.basicConfig(level=logging.INFO)
+
+ # Prepare the dataset.
+ train_dataset = get_glue_dataset(
+ data_name=args.dataset_name,
+ split="eval_train",
+ )
+ eval_dataset = get_glue_dataset(
+ data_name=args.dataset_name,
+ split="valid",
+ )
+
+ # Prepare the trained model.
+ model = construct_bert()
+ checkpoint_path = os.path.join(args.checkpoint_dir, "model.pth")
+ if not os.path.isfile(checkpoint_path):
+ raise ValueError(f"No checkpoint found at {checkpoint_path}.")
+ model.load_state_dict(torch.load(checkpoint_path))
+
+ # Define task and prepare model.
+ task = TextClassificationTask()
+ model = prepare_model(model, task)
+
+ analyzer = Analyzer(
+ analysis_name=args.dataset_name,
+ model=model,
+ task=task,
+ cpu=False,
+ )
+ # Configure parameters for DataLoader.
+ dataloader_kwargs = DataLoaderKwargs(collate_fn=default_data_collator)
+ analyzer.set_dataloader_kwargs(dataloader_kwargs)
+
+ # Compute influence factors.
+ factor_args = FactorArguments(strategy=args.factor_strategy)
+ analyzer.fit_all_factors(
+ factors_name=args.factor_strategy,
+ dataset=train_dataset,
+ per_device_batch_size=None,
+ factor_args=factor_args,
+ overwrite_output_dir=True,
+ initial_per_device_batch_size_attempt=512,
+ )
+ # Compute pairwise scores.
+ rank = args.query_gradient_rank if args.query_gradient_rank != -1 else None
+ score_args = ScoreArguments(query_gradient_rank=rank, query_gradient_svd_dtype=torch.float32)
+ scores_name = args.factor_strategy
+ if rank is not None:
+ scores_name += f"_qlr{rank}"
+ analyzer.compute_pairwise_scores(
+ score_args=score_args,
+ scores_name=scores_name,
+ factors_name=args.factor_strategy,
+ query_dataset=eval_dataset,
+ query_indices=list(range(min([len(eval_dataset), 2000]))),
+ train_dataset=train_dataset,
+ per_device_query_batch_size=args.query_batch_size,
+ per_device_train_batch_size=args.train_batch_size,
+ overwrite_output_dir=True,
+ )
+ scores = analyzer.load_pairwise_scores(scores_name)["all_modules"]
+ logging.info(f"Scores shape: {scores.shape}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/glue/figure/counterfactual.png b/examples/glue/figure/counterfactual.png
new file mode 100644
index 0000000..ab1954e
Binary files /dev/null and b/examples/glue/figure/counterfactual.png differ
diff --git a/examples/glue/figure/query_batching.png b/examples/glue/figure/query_batching.png
new file mode 100644
index 0000000..8fdb5fd
Binary files /dev/null and b/examples/glue/figure/query_batching.png differ
diff --git a/examples/glue/pipeline.py b/examples/glue/pipeline.py
index d38d44e..3db7e55 100644
--- a/examples/glue/pipeline.py
+++ b/examples/glue/pipeline.py
@@ -1,13 +1,11 @@
-import os
from typing import List
-import torch
-import torch.nn as nn
-import torchvision
from datasets import load_dataset
+from torch import nn
from torch.utils.data import Dataset
from transformers import AutoConfig, AutoModelForSequenceClassification, AutoTokenizer
+# Copied from https://github.com/huggingface/transformers/blob/main/examples/pytorch/text-classification/run_glue.py.
GLUE_TASK_TO_KEYS = {
"cola": ("sentence", None),
"mnli": ("premise", "hypothesis"),
@@ -41,14 +39,12 @@ def get_glue_dataset(
data_name: str,
split: str,
indices: List[int] = None,
- dataset_dir: str = "data/",
) -> Dataset:
assert split in ["train", "eval_train", "valid"]
raw_datasets = load_dataset(
path="glue",
name=data_name,
- # data_dir=dataset_dir,
)
label_list = raw_datasets["train"].features["label"].names
num_labels = len(label_list)
@@ -75,7 +71,7 @@ def preprocess_function(examples):
load_from_cache_file=(not False),
)
- if split == "train" or split == "eval_train":
+ if split in ["train", "eval_train"]:
train_dataset = raw_datasets["train"]
ds = train_dataset
else:
@@ -86,3 +82,10 @@ def preprocess_function(examples):
ds = ds.select(indices)
return ds
+
+
+if __name__ == "__main__":
+ from kronfluence import Analyzer
+
+ model = construct_bert()
+ print(Analyzer.get_module_summary(model))
diff --git a/examples/glue/query_batching_analysis.py b/examples/glue/query_batching_analysis.py
new file mode 100644
index 0000000..ad00ff4
--- /dev/null
+++ b/examples/glue/query_batching_analysis.py
@@ -0,0 +1,35 @@
+import logging
+
+import matplotlib.pyplot as plt
+import numpy as np
+from scipy.stats import spearmanr
+from tueplots import markers
+
+from kronfluence.analyzer import Analyzer
+
+
+def main():
+ logging.basicConfig(level=logging.INFO)
+
+ # Load the scores. You might need to modify the path.
+ full_scores = Analyzer.load_file("scores_ekfac/pairwise_scores.safetensors")["all_modules"]
+ lr_scores = Analyzer.load_file("scores_ekfac_qlr32/pairwise_scores.safetensors")["all_modules"]
+
+ # Only plot first 1000 points to avoid clutter.
+ plt.rcParams.update({"figure.dpi": 150})
+ plt.rcParams.update(markers.with_edge())
+ plt.rcParams["axes.axisbelow"] = True
+ plt.scatter(lr_scores[0][:1000], full_scores[0][:1000], edgecolor="k")
+ plt.grid()
+ plt.xlabel("Full Rank Score")
+ plt.ylabel("Low Rank (32) Score")
+ plt.show()
+
+ all_corr = []
+ for i in range(100):
+ all_corr.append(spearmanr(full_scores[i], lr_scores[i])[0])
+ logging.info(f"Averaged Spearman Correlation: {np.array(all_corr).mean()}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/glue/requirements.txt b/examples/glue/requirements.txt
new file mode 100644
index 0000000..7b0c55a
--- /dev/null
+++ b/examples/glue/requirements.txt
@@ -0,0 +1,4 @@
+transformers
+evaluate
+datasets
+scikit-learn
\ No newline at end of file
diff --git a/examples/glue/train.py b/examples/glue/train.py
index 675233f..b6dacf6 100644
--- a/examples/glue/train.py
+++ b/examples/glue/train.py
@@ -1,6 +1,7 @@
import argparse
import logging
import os
+import time
from typing import Tuple
import evaluate
@@ -9,7 +10,6 @@
from accelerate.utils import set_seed
from torch import nn
from torch.utils import data
-from tqdm import tqdm
from transformers import default_data_collator
from examples.glue.pipeline import construct_bert, get_glue_dataset
@@ -26,12 +26,6 @@ def parse_args():
default="sst2",
help="A name of GLUE dataset.",
)
- parser.add_argument(
- "--dataset_dir",
- type=str,
- default="./data",
- help="A folder to download or load GLUE dataset.",
- )
parser.add_argument(
"--train_batch_size",
@@ -94,7 +88,6 @@ def train(
num_train_epochs: int,
learning_rate: float,
weight_decay: float,
- disable_tqdm: bool = False,
) -> nn.Module:
train_dataloader = data.DataLoader(
dataset=dataset,
@@ -103,26 +96,29 @@ def train(
drop_last=True,
collate_fn=default_data_collator,
)
+
model = construct_bert().to(DEVICE)
optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate, weight_decay=weight_decay)
+ start_time = time.time()
model.train()
for epoch in range(num_train_epochs):
total_loss = 0.0
- with tqdm(train_dataloader, unit="batch", disable=disable_tqdm) as tepoch:
- for batch in tepoch:
- tepoch.set_description(f"Epoch {epoch}")
- model.zero_grad()
- outputs = model(
- input_ids=batch["input_ids"].to(device=DEVICE),
- attention_mask=batch["attention_mask"].to(device=DEVICE),
- token_type_ids=batch["token_type_ids"].to(device=DEVICE),
- ).logits
- loss = F.cross_entropy(outputs, batch["labels"].to(device=DEVICE))
- total_loss += loss.detach().float()
- loss.backward()
- optimizer.step()
- tepoch.set_postfix(loss=total_loss.item() / len(train_dataloader))
+ for batch in train_dataloader:
+ loss = model(
+ input_ids=batch["input_ids"].to(device=DEVICE),
+ attention_mask=batch["attention_mask"].to(device=DEVICE),
+ token_type_ids=batch["token_type_ids"].to(device=DEVICE),
+ labels=batch["labels"].to(device=DEVICE),
+ ).loss
+ optimizer.zero_grad()
+ loss.backward()
+ optimizer.step()
+ total_loss += loss.detach().float()
+ logging.info(f"Epoch {epoch + 1} - Averaged Loss: {total_loss / len(dataset)}")
+ end_time = time.time()
+ elapsed_time = end_time - start_time
+ logging.info(f"Completed training in {elapsed_time:.2f} seconds.")
return model
@@ -136,14 +132,14 @@ def evaluate_model(model: nn.Module, dataset: data.Dataset, batch_size: int) ->
total_loss = 0.0
for batch in dataloader:
with torch.no_grad():
- outputs = model(
- batch["input_ids"].to(device=DEVICE),
- batch["token_type_ids"].to(device=DEVICE),
- batch["attention_mask"].to(device=DEVICE),
- )
+ logits = model(
+ input_ids=batch["input_ids"].to(device=DEVICE),
+ attention_mask=batch["attention_mask"].to(device=DEVICE),
+ token_type_ids=batch["token_type_ids"].to(device=DEVICE),
+ ).logits
labels = batch["labels"].to(device=DEVICE)
- total_loss += F.cross_entropy(outputs, labels, reduction="sum").detach().item()
- predictions = outputs.argmax(dim=-1)
+ total_loss += F.cross_entropy(logits, labels, reduction="sum").detach()
+ predictions = logits.argmax(dim=-1)
metric.add_batch(
predictions=predictions,
references=labels,
@@ -160,7 +156,7 @@ def main():
if args.seed is not None:
set_seed(args.seed)
- train_dataset = get_glue_dataset(data_name=args.dataset_name, split="train", dataset_dir=args.dataset_dir)
+ train_dataset = get_glue_dataset(data_name=args.dataset_name, split="train")
model = train(
dataset=train_dataset,
batch_size=args.train_batch_size,
@@ -169,11 +165,11 @@ def main():
weight_decay=args.weight_decay,
)
- eval_train_dataset = get_glue_dataset(data_name=args.dataset_name, split="eval_train", dataset_dir=args.dataset_dir)
+ eval_train_dataset = get_glue_dataset(data_name=args.dataset_name, split="eval_train")
train_loss, train_acc = evaluate_model(model=model, dataset=eval_train_dataset, batch_size=args.eval_batch_size)
logger.info(f"Train loss: {train_loss}, Train Accuracy: {train_acc}")
- eval_dataset = get_glue_dataset(data_name=args.dataset_name, split="valid", dataset_dir=args.dataset_dir)
+ eval_dataset = get_glue_dataset(data_name=args.dataset_name, split="valid")
eval_loss, eval_acc = evaluate_model(model=model, dataset=eval_dataset, batch_size=args.eval_batch_size)
logger.info(f"Evaluation loss: {eval_loss}, Evaluation Accuracy: {eval_acc}")
diff --git a/examples/imagenet/README.md b/examples/imagenet/README.md
index 4333276..399f60d 100644
--- a/examples/imagenet/README.md
+++ b/examples/imagenet/README.md
@@ -1,3 +1,54 @@
+# ImageNet & ResNet-50 Example
+
+This directory contains scripts for training ResNet-50 on ImageNet. Please begin by installing necessary packages.
```bash
-torchrun --standalone --nnodes=1 --nproc-per-node=4 ddp_analyze.py
-```
\ No newline at end of file
+pip install -r requirements.txt
+```
+
+## Training
+
+We will use the pre-trained model from `torchvision.models.resnet50`.
+
+## Computing Pairwise Influence Scores
+
+To obtain a pairwise influence scores on 1000 query data points using `ekfac`, run the following command:
+```bash
+python analyze.py --dataset_dir PATH_TO_IMAGENET \
+ --query_gradient_rank -1 \
+ --query_batch_size 100 \
+ --train_batch_size 512 \
+ --factor_strategy ekfac
+```
+On A100 (80GB), it takes approximately 12 hours to compute the pairwise scores (including computing EKFAC factors).
+
+We can also use query batching (low-rank approximation to the query gradient; see Section 3.2.2 from the [paper](https://arxiv.org/pdf/2308.03296.pdf)) to compute influence scores with a
+larger query batch size.
+```bash
+python analyze.py --dataset_dir PATH_TO_IMAGENET \
+ --query_gradient_rank 32 \
+ --query_batch_size 500 \
+ --train_batch_size 512 \
+ --factor_strategy ekfac
+```
+On A100 (80GB), it takes roughly 4 hours to compute the pairwise scores with query batching (including computing EKFAC factors).
+Assuming that you ran above two commands, `query_batching_analysis.py`
+contains code to compute the correlations between the full rank and low-rank scores.
+
+
+
+The averaged correlations between the low-rank and full rank scores for 100 data points is 0.95.
+
+## Computing Pairwise Influence Scores with DDP
+
+You can also use DistributedDataParallel (DDP) to speed up influence computations. You can run:
+```bash
+torchrun --standalone --nnodes=1 --nproc-per-node=2 ddp_analyze.py --dataset_dir PATH_TO_IMAGENET \
+ --query_gradient_rank -1 \
+ --factor_batch_size 512 \
+ --query_batch_size 100 \
+ --train_batch_size 512 \
+ --factor_strategy ekfac
+```
+On 2 A100 (80GB), it takes approximately 6 hours to compute the pairwise scores. When available, you can use more GPUs
+to speed up influence computations.
diff --git a/examples/imagenet/analyze.py b/examples/imagenet/analyze.py
index e2786ab..d831a9d 100644
--- a/examples/imagenet/analyze.py
+++ b/examples/imagenet/analyze.py
@@ -4,18 +4,19 @@
import torch
import torch.nn.functional as F
-from analyzer import Analyzer, prepare_model
-from arguments import FactorArguments
-from task import Task
from torch import nn
from examples.imagenet.pipeline import construct_resnet50, get_imagenet_dataset
+from kronfluence.analyzer import Analyzer, prepare_model
+from kronfluence.arguments import FactorArguments, ScoreArguments
+from kronfluence.task import Task
+from kronfluence.utils.dataset import DataLoaderKwargs
-BATCH_DTYPE = Tuple[torch.Tensor, torch.Tensor]
+BATCH_TYPE = Tuple[torch.Tensor, torch.Tensor]
def parse_args():
- parser = argparse.ArgumentParser(description="Influence analysis on ImageNet datasets.")
+ parser = argparse.ArgumentParser(description="Influence analysis on ImageNet dataset.")
parser.add_argument(
"--dataset_dir",
@@ -25,29 +26,28 @@ def parse_args():
)
parser.add_argument(
- "--factor_strategy",
- type=str,
- default="ekfac",
- help="Strategy to compute preconditioning factors.",
+ "--query_gradient_rank",
+ type=int,
+ default=-1,
+ help="Rank for the low-rank query gradient approximation.",
)
parser.add_argument(
- "--batch_size",
+ "--query_batch_size",
type=int,
- default=512,
- help="Batch size for compute factors and scores.",
+ default=100,
+ help="Batch size for computing query gradients.",
)
parser.add_argument(
- "--analysis_name",
- type=str,
- default="imagenet",
- help="Name of the influence analysis.",
+ "--train_batch_size",
+ type=int,
+ default=128,
+ help="Batch size for computing training gradient.",
)
-
parser.add_argument(
- "--checkpoint_dir",
+ "--factor_strategy",
type=str,
- default="./checkpoints",
- help="A path to store the final checkpoint.",
+ default="ekfac",
+ help="Strategy to compute preconditioning factors.",
)
args = parser.parse_args()
@@ -55,40 +55,38 @@ def parse_args():
class ClassificationTask(Task):
- def compute_model_output(self, batch: BATCH_DTYPE, model: nn.Module) -> torch.Tensor:
- inputs, _ = batch
- return model(inputs)
-
def compute_train_loss(
self,
- batch: BATCH_DTYPE,
- outputs: torch.Tensor,
+ batch: BATCH_TYPE,
+ model: nn.Module,
sample: bool = False,
) -> torch.Tensor:
- _, labels = batch
-
+ inputs, labels = batch
+ logits = model(inputs)
if not sample:
- return F.cross_entropy(outputs, labels, reduction="sum")
+ return F.cross_entropy(logits, labels, reduction="sum")
with torch.no_grad():
- probs = torch.nn.functional.softmax(outputs, dim=-1)
+ probs = torch.nn.functional.softmax(logits, dim=-1)
sampled_labels = torch.multinomial(
probs,
num_samples=1,
).flatten()
- return F.cross_entropy(outputs, sampled_labels.detach(), reduction="sum")
+ return F.cross_entropy(logits, sampled_labels.detach(), reduction="sum")
def compute_measurement(
self,
- batch: BATCH_DTYPE,
- outputs: torch.Tensor,
+ batch: BATCH_TYPE,
+ model: nn.Module,
) -> torch.Tensor:
- _, labels = batch
+ # Copied from: https://github.com/MadryLab/trak/blob/main/trak/modelout_functions.py.
+ inputs, labels = batch
+ logits = model(inputs)
- bindex = torch.arange(outputs.shape[0]).to(device=outputs.device, non_blocking=False)
- logits_correct = outputs[bindex, labels]
+ bindex = torch.arange(logits.shape[0]).to(device=logits.device, non_blocking=False)
+ logits_correct = logits[bindex, labels]
- cloned_logits = outputs.clone()
- cloned_logits[bindex, labels] = torch.tensor(-torch.inf, device=outputs.device, dtype=outputs.dtype)
+ cloned_logits = logits.clone()
+ cloned_logits[bindex, labels] = torch.tensor(-torch.inf, device=logits.device, dtype=logits.dtype)
margins = logits_correct - cloned_logits.logsumexp(dim=-1)
return -margins.sum()
@@ -96,35 +94,59 @@ def compute_measurement(
def main():
args = parse_args()
-
logging.basicConfig(level=logging.INFO)
- train_dataset = get_imagenet_dataset(split="eval_train", data_path=args.dataset_dir)
- # eval_dataset = get_imagenet_dataset(split="valid", data_path=args.dataset_dir)
+ # Prepare the dataset.
+ train_dataset = get_imagenet_dataset(split="eval_train", dataset_dir=args.dataset_dir)
+ eval_dataset = get_imagenet_dataset(split="valid", dataset_dir=args.dataset_dir)
+ # Prepare the trained model.
model = construct_resnet50()
-
task = ClassificationTask()
+
+ # Define task and prepare model.
model = prepare_model(model, task)
analyzer = Analyzer(
- analysis_name=args.analysis_name,
+ analysis_name="imagenet",
model=model,
task=task,
)
-
- factor_args = FactorArguments(
- strategy=args.factor_strategy,
- covariance_data_partition_size=1,
- covariance_module_partition_size=1,
+ # Configure parameters for DataLoader.
+ dataloader_kwargs = DataLoaderKwargs(
+ num_workers=4,
)
- analyzer.fit_covariance_matrices(
+ analyzer.set_dataloader_kwargs(dataloader_kwargs)
+
+ # Compute influence factors.
+ factor_args = FactorArguments(strategy=args.factor_strategy)
+ analyzer.fit_all_factors(
factors_name=args.factor_strategy,
dataset=train_dataset,
+ per_device_batch_size=None,
factor_args=factor_args,
- per_device_batch_size=1024,
+ overwrite_output_dir=False,
+ )
+
+ # Compute pairwise scores.
+ rank = args.query_gradient_rank if args.query_gradient_rank != -1 else None
+ score_args = ScoreArguments(query_gradient_rank=rank, query_gradient_svd_dtype=torch.float32)
+ scores_name = args.factor_strategy
+ if rank is not None:
+ scores_name += f"_qlr{rank}"
+ analyzer.compute_pairwise_scores(
+ score_args=score_args,
+ scores_name=scores_name,
+ factors_name=args.factor_strategy,
+ query_dataset=eval_dataset,
+ query_indices=list(range(1000)),
+ train_dataset=train_dataset,
+ per_device_query_batch_size=args.query_batch_size,
+ per_device_train_batch_size=args.train_batch_size,
overwrite_output_dir=True,
)
+ scores = analyzer.load_pairwise_scores(scores_name)["all_modules"]
+ logging.info(f"Scores shape: {scores.shape}")
if __name__ == "__main__":
diff --git a/examples/imagenet/ddp_analyze.py b/examples/imagenet/ddp_analyze.py
index 0e3d0fd..5d64551 100644
--- a/examples/imagenet/ddp_analyze.py
+++ b/examples/imagenet/ddp_analyze.py
@@ -5,14 +5,12 @@
import torch
import torch.distributed as dist
-import torch.nn.functional as F
-from torch import nn
from torch.nn.parallel.distributed import DistributedDataParallel
+from examples.imagenet.analyze import ClassificationTask
from examples.imagenet.pipeline import construct_resnet50, get_imagenet_dataset
from kronfluence.analyzer import Analyzer, prepare_model
-from kronfluence.arguments import FactorArguments
-from kronfluence.task import Task
+from kronfluence.arguments import FactorArguments, ScoreArguments
from kronfluence.utils.dataset import DataLoaderKwargs
torch.backends.cudnn.benchmark = True
@@ -33,28 +31,22 @@ def parse_args():
)
parser.add_argument(
- "--factor_strategy",
- type=str,
- default="ekfac",
- help="Strategy to compute preconditioning factors.",
- )
- parser.add_argument(
- "--covariance_batch_size",
+ "--query_gradient_rank",
type=int,
- default=512,
- help="Batch size for computing covariance matrices.",
+ default=-1,
+ help="Rank for the low-rank query gradient approximation.",
)
parser.add_argument(
- "--lambda_batch_size",
+ "--factor_batch_size",
type=int,
- default=256,
- help="Batch size for computing Lambda matrices.",
+ default=512,
+ help="Batch size for computing influence factors.",
)
parser.add_argument(
"--query_batch_size",
type=int,
- default=64,
- help="Batch size for computing query gradient.",
+ default=100,
+ help="Batch size for computing query gradients.",
)
parser.add_argument(
"--train_batch_size",
@@ -62,54 +54,22 @@ def parse_args():
default=128,
help="Batch size for computing training gradient.",
)
+ parser.add_argument(
+ "--factor_strategy",
+ type=str,
+ default="ekfac",
+ help="Strategy to compute preconditioning factors.",
+ )
args = parser.parse_args()
return args
-class ClassificationTask(Task):
- def compute_train_loss(
- self,
- batch: BATCH_DTYPE,
- model: nn.Module,
- sample: bool = False,
- ) -> torch.Tensor:
- inputs, labels = batch
- logits = model(inputs)
-
- if not sample:
- return F.cross_entropy(logits, labels, reduction="sum")
- with torch.no_grad():
- probs = torch.nn.functional.softmax(logits, dim=-1)
- sampled_labels = torch.multinomial(
- probs,
- num_samples=1,
- ).flatten()
- return F.cross_entropy(logits, sampled_labels.detach(), reduction="sum")
-
- def compute_measurement(
- self,
- batch: BATCH_DTYPE,
- model: nn.Module,
- ) -> torch.Tensor:
- # Copied from https://github.com/MadryLab/trak/blob/main/trak/modelout_functions.py.
- inputs, labels = batch
- logits = model(inputs)
-
- bindex = torch.arange(logits.shape[0]).to(device=logits.device, non_blocking=False)
- logits_correct = logits[bindex, labels]
-
- cloned_logits = logits.clone()
- cloned_logits[bindex, labels] = torch.tensor(-torch.inf, device=logits.device, dtype=logits.dtype)
-
- margins = logits_correct - cloned_logits.logsumexp(dim=-1)
- return -margins.sum()
-
-
def main():
args = parse_args()
logging.basicConfig(level=logging.INFO)
+ # Prepare the dataset.
train_dataset = get_imagenet_dataset(split="eval_train", dataset_dir=args.dataset_dir)
eval_dataset = get_imagenet_dataset(split="valid", dataset_dir=args.dataset_dir)
@@ -117,60 +77,59 @@ def main():
device = torch.device("cuda:{}".format(LOCAL_RANK))
torch.cuda.set_device(LOCAL_RANK)
+ # Prepare the trained model.
model = construct_resnet50()
task = ClassificationTask()
+
+ # Define task and prepare model.
model = prepare_model(model, task)
model = model.to(device=device)
+
+ # Apply DDP.
model = DistributedDataParallel(model, device_ids=[LOCAL_RANK], output_device=LOCAL_RANK)
analyzer = Analyzer(
- analysis_name="ddp",
+ analysis_name="imagenet_ddp",
model=model,
task=task,
- profile=True,
- disable_model_save=True,
)
+ # Configure parameters for DataLoader.
dataloader_kwargs = DataLoaderKwargs(
- num_workers=2,
- pin_memory=True,
- prefetch_factor=2,
+ num_workers=4,
)
+ analyzer.set_dataloader_kwargs(dataloader_kwargs)
+ # Compute influence factors.
factor_args = FactorArguments(
strategy=args.factor_strategy,
)
- analyzer.fit_covariance_matrices(
- factors_name=args.factor_strategy,
- dataset=train_dataset,
- factor_args=factor_args,
- per_device_batch_size=args.covariance_batch_size,
- dataloader_kwargs=dataloader_kwargs,
- overwrite_output_dir=False,
- )
- analyzer.perform_eigendecomposition(
- factors_name=args.factor_strategy,
- factor_args=factor_args,
- overwrite_output_dir=False,
- )
- analyzer.fit_lambda_matrices(
+ analyzer.fit_all_factors(
factors_name=args.factor_strategy,
dataset=train_dataset,
+ per_device_batch_size=args.factor_batch_size,
factor_args=factor_args,
- per_device_batch_size=args.lambda_batch_size,
- dataloader_kwargs=dataloader_kwargs,
overwrite_output_dir=False,
)
- scores = analyzer.compute_pairwise_scores(
- scores_name="pairwise",
+
+ # Compute pairwise scores.
+ rank = args.query_gradient_rank if args.query_gradient_rank != -1 else None
+ score_args = ScoreArguments(query_gradient_rank=rank)
+ scores_name = args.factor_strategy
+ if rank is not None:
+ scores_name += f"_qlr{rank}"
+ analyzer.compute_pairwise_scores(
+ score_args=score_args,
+ scores_name=scores_name,
factors_name=args.factor_strategy,
query_dataset=eval_dataset,
+ query_indices=list(range(1000)),
train_dataset=train_dataset,
- per_device_train_batch_size=args.train_batch_size,
per_device_query_batch_size=args.query_batch_size,
- query_indices=list(range(1000)),
+ per_device_train_batch_size=args.train_batch_size,
overwrite_output_dir=False,
)
- logging.info(f"Scores: {scores}")
+ scores = analyzer.load_pairwise_scores(scores_name)["all_modules"]
+ logging.info(f"Scores shape: {scores.shape}")
if __name__ == "__main__":
diff --git a/examples/imagenet/figure/query_batching.png b/examples/imagenet/figure/query_batching.png
new file mode 100644
index 0000000..bc865c7
Binary files /dev/null and b/examples/imagenet/figure/query_batching.png differ
diff --git a/examples/imagenet/pipeline.py b/examples/imagenet/pipeline.py
index e3f0fe2..51aa6ae 100644
--- a/examples/imagenet/pipeline.py
+++ b/examples/imagenet/pipeline.py
@@ -2,8 +2,8 @@
from typing import List
import torch
-import torch.nn as nn
import torchvision
+from torch import nn
from torch.utils.data import Dataset
diff --git a/examples/imagenet/query_batching_analysis.py b/examples/imagenet/query_batching_analysis.py
new file mode 100644
index 0000000..c0b5480
--- /dev/null
+++ b/examples/imagenet/query_batching_analysis.py
@@ -0,0 +1,36 @@
+import logging
+
+import matplotlib.pyplot as plt
+import numpy as np
+from scipy.stats import spearmanr
+from tueplots import markers
+
+from kronfluence.analyzer import Analyzer
+
+
+def main():
+ logging.basicConfig(level=logging.INFO)
+
+ # Load the scores. You might need to modify the path.
+ full_scores = Analyzer.load_file("scores_ekfac/pairwise_scores.safetensors")["all_modules"]
+ lr_scores = Analyzer.load_file("scores_ekfac_qlr32/pairwise_scores.safetensors")["all_modules"]
+
+ # Only plot first 1000 points to avoid clutter.
+ plt.rcParams.update({"figure.dpi": 150})
+ plt.rcParams.update(markers.with_edge())
+ plt.rcParams["axes.axisbelow"] = True
+ plt.scatter(lr_scores[0][:1000], full_scores[0][:1000], edgecolor="k")
+ plt.grid()
+ plt.xlabel("Full Rank Score")
+ plt.ylabel("Low Rank (32) Score")
+ plt.show()
+
+ # Compute the averaged spearman correlation.
+ all_corr = []
+ for i in range(100):
+ all_corr.append(spearmanr(full_scores[i], lr_scores[i])[0])
+ logging.info(f"Averaged Spearman Correlation: {np.array(all_corr).mean()}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/imagenet/requirements.txt b/examples/imagenet/requirements.txt
new file mode 100644
index 0000000..a667c1f
--- /dev/null
+++ b/examples/imagenet/requirements.txt
@@ -0,0 +1,3 @@
+scikit-learn
+matplotlib
+tueplots
\ No newline at end of file
diff --git a/examples/requirements.txt b/examples/requirements.txt
deleted file mode 100644
index e69de29..0000000
diff --git a/examples/uci/README.md b/examples/uci/README.md
index ee30203..7cc0e56 100644
--- a/examples/uci/README.md
+++ b/examples/uci/README.md
@@ -1,8 +1,7 @@
# UCI Regression Example
-This directory contains scripts designed for training a regression model and conducting influence analysis with
-datasets from the [UCI Machine Learning Repository](https://archive.ics.uci.edu/datasets). Install all necessary packages:
-
+This directory contains scripts for training a regression model and conducting influence analysis with
+datasets from the [UCI Machine Learning Repository](https://archive.ics.uci.edu/datasets). Please begin by installing necessary packages.
```bash
pip install -r requirements.txt
```
@@ -11,7 +10,7 @@ pip install -r requirements.txt
To train a regression model on the Concrete dataset, run the following command:
```bash
-python train.py --dataset_name concrete \
+python train.py --dataset_name concrete \
--dataset_dir ./data \
--output_dir ./checkpoints \
--train_batch_size 32 \
@@ -22,16 +21,21 @@ python train.py --dataset_name concrete \
--seed 1004
```
-# Computing Pairwise Influence Scores
+## Computing Pairwise Influence Scores
-To obtain a pairwise influence scores using EKFAC, run the following command:
+To obtain a pairwise influence scores using `ekfac`, run the following command:
```bash
-python analyze.py --dataset_name concrete \
+python analyze.py --dataset_name concrete \
--dataset_dir ./data \
--checkpoint_dir ./checkpoints \
--factor_strategy ekfac
```
+You can also use `identity`, `diagonal`, and `kfac`.
+
+## Counterfactual Evaluation
-# Counterfactual Evaluation
+You can check the notebook `tutorial.ipynb` to run the counterfactual evaluation.
-You can check the notebook `tutorial.ipynb` for running the counterfactual evaluation.
\ No newline at end of file
+
+
diff --git a/examples/uci/analyze.py b/examples/uci/analyze.py
index a8d963f..653886e 100644
--- a/examples/uci/analyze.py
+++ b/examples/uci/analyze.py
@@ -6,14 +6,14 @@
import torch
import torch.nn.functional as F
-from arguments import FactorArguments
from torch import nn
from examples.uci.pipeline import construct_regression_mlp, get_regression_dataset
from kronfluence.analyzer import Analyzer, prepare_model
+from kronfluence.arguments import FactorArguments
from kronfluence.task import Task
-BATCH_DTYPE = Tuple[torch.Tensor, torch.Tensor]
+BATCH_TYPE = Tuple[torch.Tensor, torch.Tensor]
def parse_args():
@@ -35,16 +35,15 @@ def parse_args():
"--checkpoint_dir",
type=str,
default="./checkpoints",
- help="A path to store the final checkpoint.",
+ help="A path that is storing the final checkpoint of the model.",
)
parser.add_argument(
"--factor_strategy",
type=str,
default="ekfac",
- help="Strategy to compute preconditioning factors.",
+ help="Strategy to compute influence factors.",
)
-
args = parser.parse_args()
if args.checkpoint_dir is not None:
@@ -58,7 +57,7 @@ def parse_args():
class RegressionTask(Task):
def compute_train_loss(
self,
- batch: BATCH_DTYPE,
+ batch: BATCH_TYPE,
model: nn.Module,
sample: bool = False,
) -> torch.Tensor:
@@ -72,7 +71,7 @@ def compute_train_loss(
def compute_measurement(
self,
- batch: BATCH_DTYPE,
+ batch: BATCH_TYPE,
model: nn.Module,
) -> torch.Tensor:
# The measurement function is set as a training loss.
@@ -83,17 +82,20 @@ def main():
args = parse_args()
logging.basicConfig(level=logging.INFO)
+ # Prepare the dataset.
train_dataset = get_regression_dataset(
data_name=args.dataset_name, split="eval_train", dataset_dir=args.dataset_dir
)
eval_dataset = get_regression_dataset(data_name=args.dataset_name, split="valid", dataset_dir=args.dataset_dir)
+ # Prepare the trained model.
model = construct_regression_mlp()
checkpoint_path = os.path.join(args.checkpoint_dir, "model.pth")
if not os.path.isfile(checkpoint_path):
raise ValueError(f"No checkpoint found at {checkpoint_path}.")
model.load_state_dict(torch.load(checkpoint_path))
+ # Define task and prepare model.
task = RegressionTask()
model = prepare_model(model, task)
@@ -103,25 +105,27 @@ def main():
task=task,
cpu=True,
)
-
+ # Compute influence factors.
factor_args = FactorArguments(strategy=args.factor_strategy)
analyzer.fit_all_factors(
factors_name=args.factor_strategy,
dataset=train_dataset,
per_device_batch_size=None,
factor_args=factor_args,
- overwrite_output_dir=True,
+ overwrite_output_dir=False,
)
+ # Compute pairwise scores.
analyzer.compute_pairwise_scores(
- scores_name="pairwise",
+ scores_name=args.factor_strategy,
factors_name=args.factor_strategy,
query_dataset=eval_dataset,
train_dataset=train_dataset,
+ # Use full batch for computing query gradient.
per_device_query_batch_size=len(eval_dataset),
- overwrite_output_dir=True,
+ overwrite_output_dir=False,
)
- scores = analyzer.load_pairwise_scores("pairwise")
- print(scores)
+ scores = analyzer.load_pairwise_scores(args.factor_strategy)["all_modules"]
+ logging.info(f"Scores shape: {scores.shape}")
if __name__ == "__main__":
diff --git a/examples/uci/data/concrete.data b/examples/uci/data/concrete.data
new file mode 100644
index 0000000..51d5ce3
--- /dev/null
+++ b/examples/uci/data/concrete.data
@@ -0,0 +1,1030 @@
+540.0 0.0 0.0 162.0 2.5 1040.0 676.0 28 79.99
+540.0 0.0 0.0 162.0 2.5 1055.0 676.0 28 61.89
+332.5 142.5 0.0 228.0 0.0 932.0 594.0 270 40.27
+332.5 142.5 0.0 228.0 0.0 932.0 594.0 365 41.05
+198.6 132.4 0.0 192.0 0.0 978.4 825.5 360 44.30
+266.0 114.0 0.0 228.0 0.0 932.0 670.0 90 47.03
+380.0 95.0 0.0 228.0 0.0 932.0 594.0 365 43.70
+380.0 95.0 0.0 228.0 0.0 932.0 594.0 28 36.45
+266.0 114.0 0.0 228.0 0.0 932.0 670.0 28 45.85
+475.0 0.0 0.0 228.0 0.0 932.0 594.0 28 39.29
+198.6 132.4 0.0 192.0 0.0 978.4 825.5 90 38.07
+198.6 132.4 0.0 192.0 0.0 978.4 825.5 28 28.02
+427.5 47.5 0.0 228.0 0.0 932.0 594.0 270 43.01
+190.0 190.0 0.0 228.0 0.0 932.0 670.0 90 42.33
+304.0 76.0 0.0 228.0 0.0 932.0 670.0 28 47.81
+380.0 0.0 0.0 228.0 0.0 932.0 670.0 90 52.91
+139.6 209.4 0.0 192.0 0.0 1047.0 806.9 90 39.36
+342.0 38.0 0.0 228.0 0.0 932.0 670.0 365 56.14
+380.0 95.0 0.0 228.0 0.0 932.0 594.0 90 40.56
+475.0 0.0 0.0 228.0 0.0 932.0 594.0 180 42.62
+427.5 47.5 0.0 228.0 0.0 932.0 594.0 180 41.84
+139.6 209.4 0.0 192.0 0.0 1047.0 806.9 28 28.24
+139.6 209.4 0.0 192.0 0.0 1047.0 806.9 3 8.06
+139.6 209.4 0.0 192.0 0.0 1047.0 806.9 180 44.21
+380.0 0.0 0.0 228.0 0.0 932.0 670.0 365 52.52
+380.0 0.0 0.0 228.0 0.0 932.0 670.0 270 53.30
+380.0 95.0 0.0 228.0 0.0 932.0 594.0 270 41.15
+342.0 38.0 0.0 228.0 0.0 932.0 670.0 180 52.12
+427.5 47.5 0.0 228.0 0.0 932.0 594.0 28 37.43
+475.0 0.0 0.0 228.0 0.0 932.0 594.0 7 38.60
+304.0 76.0 0.0 228.0 0.0 932.0 670.0 365 55.26
+266.0 114.0 0.0 228.0 0.0 932.0 670.0 365 52.91
+198.6 132.4 0.0 192.0 0.0 978.4 825.5 180 41.72
+475.0 0.0 0.0 228.0 0.0 932.0 594.0 270 42.13
+190.0 190.0 0.0 228.0 0.0 932.0 670.0 365 53.69
+237.5 237.5 0.0 228.0 0.0 932.0 594.0 270 38.41
+237.5 237.5 0.0 228.0 0.0 932.0 594.0 28 30.08
+332.5 142.5 0.0 228.0 0.0 932.0 594.0 90 37.72
+475.0 0.0 0.0 228.0 0.0 932.0 594.0 90 42.23
+237.5 237.5 0.0 228.0 0.0 932.0 594.0 180 36.25
+342.0 38.0 0.0 228.0 0.0 932.0 670.0 90 50.46
+427.5 47.5 0.0 228.0 0.0 932.0 594.0 365 43.70
+237.5 237.5 0.0 228.0 0.0 932.0 594.0 365 39.00
+380.0 0.0 0.0 228.0 0.0 932.0 670.0 180 53.10
+427.5 47.5 0.0 228.0 0.0 932.0 594.0 90 41.54
+427.5 47.5 0.0 228.0 0.0 932.0 594.0 7 35.08
+349.0 0.0 0.0 192.0 0.0 1047.0 806.9 3 15.05
+380.0 95.0 0.0 228.0 0.0 932.0 594.0 180 40.76
+237.5 237.5 0.0 228.0 0.0 932.0 594.0 7 26.26
+380.0 95.0 0.0 228.0 0.0 932.0 594.0 7 32.82
+332.5 142.5 0.0 228.0 0.0 932.0 594.0 180 39.78
+190.0 190.0 0.0 228.0 0.0 932.0 670.0 180 46.93
+237.5 237.5 0.0 228.0 0.0 932.0 594.0 90 33.12
+304.0 76.0 0.0 228.0 0.0 932.0 670.0 90 49.19
+139.6 209.4 0.0 192.0 0.0 1047.0 806.9 7 14.59
+198.6 132.4 0.0 192.0 0.0 978.4 825.5 7 14.64
+475.0 0.0 0.0 228.0 0.0 932.0 594.0 365 41.93
+198.6 132.4 0.0 192.0 0.0 978.4 825.5 3 9.13
+304.0 76.0 0.0 228.0 0.0 932.0 670.0 180 50.95
+332.5 142.5 0.0 228.0 0.0 932.0 594.0 28 33.02
+304.0 76.0 0.0 228.0 0.0 932.0 670.0 270 54.38
+266.0 114.0 0.0 228.0 0.0 932.0 670.0 270 51.73
+310.0 0.0 0.0 192.0 0.0 971.0 850.6 3 9.87
+190.0 190.0 0.0 228.0 0.0 932.0 670.0 270 50.66
+266.0 114.0 0.0 228.0 0.0 932.0 670.0 180 48.70
+342.0 38.0 0.0 228.0 0.0 932.0 670.0 270 55.06
+139.6 209.4 0.0 192.0 0.0 1047.0 806.9 360 44.70
+332.5 142.5 0.0 228.0 0.0 932.0 594.0 7 30.28
+190.0 190.0 0.0 228.0 0.0 932.0 670.0 28 40.86
+485.0 0.0 0.0 146.0 0.0 1120.0 800.0 28 71.99
+374.0 189.2 0.0 170.1 10.1 926.1 756.7 3 34.40
+313.3 262.2 0.0 175.5 8.6 1046.9 611.8 3 28.80
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 3 33.40
+425.0 106.3 0.0 151.4 18.6 936.0 803.7 3 36.30
+375.0 93.8 0.0 126.6 23.4 852.1 992.6 3 29.00
+475.0 118.8 0.0 181.1 8.9 852.1 781.5 3 37.80
+469.0 117.2 0.0 137.8 32.2 852.1 840.5 3 40.20
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 3 33.40
+388.6 97.1 0.0 157.9 12.1 852.1 925.7 3 28.10
+531.3 0.0 0.0 141.8 28.2 852.1 893.7 3 41.30
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 3 33.40
+318.8 212.5 0.0 155.7 14.3 852.1 880.4 3 25.20
+401.8 94.7 0.0 147.4 11.4 946.8 852.1 3 41.10
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 3 35.30
+323.7 282.8 0.0 183.8 10.3 942.7 659.9 3 28.30
+379.5 151.2 0.0 153.9 15.9 1134.3 605.0 3 28.60
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 3 35.30
+286.3 200.9 0.0 144.7 11.2 1004.6 803.7 3 24.40
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 3 35.30
+439.0 177.0 0.0 186.0 11.1 884.9 707.9 3 39.30
+389.9 189.0 0.0 145.9 22.0 944.7 755.8 3 40.60
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 3 35.30
+337.9 189.0 0.0 174.9 9.5 944.7 755.8 3 24.10
+374.0 189.2 0.0 170.1 10.1 926.1 756.7 7 46.20
+313.3 262.2 0.0 175.5 8.6 1046.9 611.8 7 42.80
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 7 49.20
+425.0 106.3 0.0 151.4 18.6 936.0 803.7 7 46.80
+375.0 93.8 0.0 126.6 23.4 852.1 992.6 7 45.70
+475.0 118.8 0.0 181.1 8.9 852.1 781.5 7 55.60
+469.0 117.2 0.0 137.8 32.2 852.1 840.5 7 54.90
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 7 49.20
+388.6 97.1 0.0 157.9 12.1 852.1 925.7 7 34.90
+531.3 0.0 0.0 141.8 28.2 852.1 893.7 7 46.90
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 7 49.20
+318.8 212.5 0.0 155.7 14.3 852.1 880.4 7 33.40
+401.8 94.7 0.0 147.4 11.4 946.8 852.1 7 54.10
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 7 55.90
+323.7 282.8 0.0 183.8 10.3 942.7 659.9 7 49.80
+379.5 151.2 0.0 153.9 15.9 1134.3 605.0 7 47.10
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 7 55.90
+286.3 200.9 0.0 144.7 11.2 1004.6 803.7 7 38.00
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 7 55.90
+439.0 177.0 0.0 186.0 11.1 884.9 707.9 7 56.10
+389.9 189.0 0.0 145.9 22.0 944.7 755.8 7 59.09
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 7 22.90
+337.9 189.0 0.0 174.9 9.5 944.7 755.8 7 35.10
+374.0 189.2 0.0 170.1 10.1 926.1 756.7 28 61.09
+313.3 262.2 0.0 175.5 8.6 1046.9 611.8 28 59.80
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 28 60.29
+425.0 106.3 0.0 151.4 18.6 936.0 803.7 28 61.80
+375.0 93.8 0.0 126.6 23.4 852.1 992.6 28 56.70
+475.0 118.8 0.0 181.1 8.9 852.1 781.5 28 68.30
+469.0 117.2 0.0 137.8 32.2 852.1 840.5 28 66.90
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 28 60.29
+388.6 97.1 0.0 157.9 12.1 852.1 925.7 28 50.70
+531.3 0.0 0.0 141.8 28.2 852.1 893.7 28 56.40
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 28 60.29
+318.8 212.5 0.0 155.7 14.3 852.1 880.4 28 55.50
+401.8 94.7 0.0 147.4 11.4 946.8 852.1 28 68.50
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 28 71.30
+323.7 282.8 0.0 183.8 10.3 942.7 659.9 28 74.70
+379.5 151.2 0.0 153.9 15.9 1134.3 605.0 28 52.20
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 28 71.30
+286.3 200.9 0.0 144.7 11.2 1004.6 803.7 28 67.70
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 28 71.30
+439.0 177.0 0.0 186.0 11.1 884.9 707.9 28 66.00
+389.9 189.0 0.0 145.9 22.0 944.7 755.8 28 74.50
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 28 71.30
+337.9 189.0 0.0 174.9 9.5 944.7 755.8 28 49.90
+374.0 189.2 0.0 170.1 10.1 926.1 756.7 56 63.40
+313.3 262.2 0.0 175.5 8.6 1046.9 611.8 56 64.90
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 56 64.30
+425.0 106.3 0.0 151.4 18.6 936.0 803.7 56 64.90
+375.0 93.8 0.0 126.6 23.4 852.1 992.6 56 60.20
+475.0 118.8 0.0 181.1 8.9 852.1 781.5 56 72.30
+469.0 117.2 0.0 137.8 32.2 852.1 840.5 56 69.30
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 56 64.30
+388.6 97.1 0.0 157.9 12.1 852.1 925.7 56 55.20
+531.3 0.0 0.0 141.8 28.2 852.1 893.7 56 58.80
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 56 64.30
+318.8 212.5 0.0 155.7 14.3 852.1 880.4 56 66.10
+401.8 94.7 0.0 147.4 11.4 946.8 852.1 56 73.70
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 56 77.30
+323.7 282.8 0.0 183.8 10.3 942.7 659.9 56 80.20
+379.5 151.2 0.0 153.9 15.9 1134.3 605.0 56 54.90
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 56 77.30
+286.3 200.9 0.0 144.7 11.2 1004.6 803.7 56 72.99
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 56 77.30
+439.0 177.0 0.0 186.0 11.1 884.9 707.9 56 71.70
+389.9 189.0 0.0 145.9 22.0 944.7 755.8 56 79.40
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 56 77.30
+337.9 189.0 0.0 174.9 9.5 944.7 755.8 56 59.89
+374.0 189.2 0.0 170.1 10.1 926.1 756.7 91 64.90
+313.3 262.2 0.0 175.5 8.6 1046.9 611.8 91 66.60
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 91 65.20
+425.0 106.3 0.0 151.4 18.6 936.0 803.7 91 66.70
+375.0 93.8 0.0 126.6 23.4 852.1 992.6 91 62.50
+475.0 118.8 0.0 181.1 8.9 852.1 781.5 91 74.19
+469.0 117.2 0.0 137.8 32.2 852.1 840.5 91 70.70
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 91 65.20
+388.6 97.1 0.0 157.9 12.1 852.1 925.7 91 57.60
+531.3 0.0 0.0 141.8 28.2 852.1 893.7 91 59.20
+425.0 106.3 0.0 153.5 16.5 852.1 887.1 91 65.20
+318.8 212.5 0.0 155.7 14.3 852.1 880.4 91 68.10
+401.8 94.7 0.0 147.4 11.4 946.8 852.1 91 75.50
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 91 79.30
+379.5 151.2 0.0 153.9 15.9 1134.3 605.0 91 56.50
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 91 79.30
+286.3 200.9 0.0 144.7 11.2 1004.6 803.7 91 76.80
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 91 79.30
+439.0 177.0 0.0 186.0 11.1 884.9 707.9 91 73.30
+389.9 189.0 0.0 145.9 22.0 944.7 755.8 91 82.60
+362.6 189.0 0.0 164.9 11.6 944.7 755.8 91 79.30
+337.9 189.0 0.0 174.9 9.5 944.7 755.8 91 67.80
+222.4 0.0 96.7 189.3 4.5 967.1 870.3 3 11.58
+222.4 0.0 96.7 189.3 4.5 967.1 870.3 14 24.45
+222.4 0.0 96.7 189.3 4.5 967.1 870.3 28 24.89
+222.4 0.0 96.7 189.3 4.5 967.1 870.3 56 29.45
+222.4 0.0 96.7 189.3 4.5 967.1 870.3 100 40.71
+233.8 0.0 94.6 197.9 4.6 947.0 852.2 3 10.38
+233.8 0.0 94.6 197.9 4.6 947.0 852.2 14 22.14
+233.8 0.0 94.6 197.9 4.6 947.0 852.2 28 22.84
+233.8 0.0 94.6 197.9 4.6 947.0 852.2 56 27.66
+233.8 0.0 94.6 197.9 4.6 947.0 852.2 100 34.56
+194.7 0.0 100.5 165.6 7.5 1006.4 905.9 3 12.45
+194.7 0.0 100.5 165.6 7.5 1006.4 905.9 14 24.99
+194.7 0.0 100.5 165.6 7.5 1006.4 905.9 28 25.72
+194.7 0.0 100.5 165.6 7.5 1006.4 905.9 56 33.96
+194.7 0.0 100.5 165.6 7.5 1006.4 905.9 100 37.34
+190.7 0.0 125.4 162.1 7.8 1090.0 804.0 3 15.04
+190.7 0.0 125.4 162.1 7.8 1090.0 804.0 14 21.06
+190.7 0.0 125.4 162.1 7.8 1090.0 804.0 28 26.40
+190.7 0.0 125.4 162.1 7.8 1090.0 804.0 56 35.34
+190.7 0.0 125.4 162.1 7.8 1090.0 804.0 100 40.57
+212.1 0.0 121.6 180.3 5.7 1057.6 779.3 3 12.47
+212.1 0.0 121.6 180.3 5.7 1057.6 779.3 14 20.92
+212.1 0.0 121.6 180.3 5.7 1057.6 779.3 28 24.90
+212.1 0.0 121.6 180.3 5.7 1057.6 779.3 56 34.20
+212.1 0.0 121.6 180.3 5.7 1057.6 779.3 100 39.61
+230.0 0.0 118.3 195.5 4.6 1029.4 758.6 3 10.03
+230.0 0.0 118.3 195.5 4.6 1029.4 758.6 14 20.08
+230.0 0.0 118.3 195.5 4.6 1029.4 758.6 28 24.48
+230.0 0.0 118.3 195.5 4.6 1029.4 758.6 56 31.54
+230.0 0.0 118.3 195.5 4.6 1029.4 758.6 100 35.34
+190.3 0.0 125.2 161.9 9.9 1088.1 802.6 3 9.45
+190.3 0.0 125.2 161.9 9.9 1088.1 802.6 14 22.72
+190.3 0.0 125.2 161.9 9.9 1088.1 802.6 28 28.47
+190.3 0.0 125.2 161.9 9.9 1088.1 802.6 56 38.56
+190.3 0.0 125.2 161.9 9.9 1088.1 802.6 100 40.39
+166.1 0.0 163.3 176.5 4.5 1058.6 780.1 3 10.76
+166.1 0.0 163.3 176.5 4.5 1058.6 780.1 14 25.48
+166.1 0.0 163.3 176.5 4.5 1058.6 780.1 28 21.54
+166.1 0.0 163.3 176.5 4.5 1058.6 780.1 56 28.63
+166.1 0.0 163.3 176.5 4.5 1058.6 780.1 100 33.54
+168.0 42.1 163.8 121.8 5.7 1058.7 780.1 3 7.75
+168.0 42.1 163.8 121.8 5.7 1058.7 780.1 14 17.82
+168.0 42.1 163.8 121.8 5.7 1058.7 780.1 28 24.24
+168.0 42.1 163.8 121.8 5.7 1058.7 780.1 56 32.85
+168.0 42.1 163.8 121.8 5.7 1058.7 780.1 100 39.23
+213.7 98.1 24.5 181.7 6.9 1065.8 785.4 3 18.00
+213.7 98.1 24.5 181.7 6.9 1065.8 785.4 14 30.39
+213.7 98.1 24.5 181.7 6.9 1065.8 785.4 28 45.71
+213.7 98.1 24.5 181.7 6.9 1065.8 785.4 56 50.77
+213.7 98.1 24.5 181.7 6.9 1065.8 785.4 100 53.90
+213.8 98.1 24.5 181.7 6.7 1066.0 785.5 3 13.18
+213.8 98.1 24.5 181.7 6.7 1066.0 785.5 14 17.84
+213.8 98.1 24.5 181.7 6.7 1066.0 785.5 28 40.23
+213.8 98.1 24.5 181.7 6.7 1066.0 785.5 56 47.13
+213.8 98.1 24.5 181.7 6.7 1066.0 785.5 100 49.97
+229.7 0.0 118.2 195.2 6.1 1028.1 757.6 3 13.36
+229.7 0.0 118.2 195.2 6.1 1028.1 757.6 14 22.32
+229.7 0.0 118.2 195.2 6.1 1028.1 757.6 28 24.54
+229.7 0.0 118.2 195.2 6.1 1028.1 757.6 56 31.35
+229.7 0.0 118.2 195.2 6.1 1028.1 757.6 100 40.86
+238.1 0.0 94.1 186.7 7.0 949.9 847.0 3 19.93
+238.1 0.0 94.1 186.7 7.0 949.9 847.0 14 25.69
+238.1 0.0 94.1 186.7 7.0 949.9 847.0 28 30.23
+238.1 0.0 94.1 186.7 7.0 949.9 847.0 56 39.59
+238.1 0.0 94.1 186.7 7.0 949.9 847.0 100 44.30
+250.0 0.0 95.7 187.4 5.5 956.9 861.2 3 13.82
+250.0 0.0 95.7 187.4 5.5 956.9 861.2 14 24.92
+250.0 0.0 95.7 187.4 5.5 956.9 861.2 28 29.22
+250.0 0.0 95.7 187.4 5.5 956.9 861.2 56 38.33
+250.0 0.0 95.7 187.4 5.5 956.9 861.2 100 42.35
+212.5 0.0 100.4 159.3 8.7 1007.8 903.6 3 13.54
+212.5 0.0 100.4 159.3 8.7 1007.8 903.6 14 26.31
+212.5 0.0 100.4 159.3 8.7 1007.8 903.6 28 31.64
+212.5 0.0 100.4 159.3 8.7 1007.8 903.6 56 42.55
+212.5 0.0 100.4 159.3 8.7 1007.8 903.6 100 42.92
+212.6 0.0 100.4 159.4 10.4 1003.8 903.8 3 13.33
+212.6 0.0 100.4 159.4 10.4 1003.8 903.8 14 25.37
+212.6 0.0 100.4 159.4 10.4 1003.8 903.8 28 37.40
+212.6 0.0 100.4 159.4 10.4 1003.8 903.8 56 44.40
+212.6 0.0 100.4 159.4 10.4 1003.8 903.8 100 47.74
+212.0 0.0 124.8 159.0 7.8 1085.4 799.5 3 19.52
+212.0 0.0 124.8 159.0 7.8 1085.4 799.5 14 31.35
+212.0 0.0 124.8 159.0 7.8 1085.4 799.5 28 38.50
+212.0 0.0 124.8 159.0 7.8 1085.4 799.5 56 45.08
+212.0 0.0 124.8 159.0 7.8 1085.4 799.5 100 47.82
+231.8 0.0 121.6 174.0 6.7 1056.4 778.5 3 15.44
+231.8 0.0 121.6 174.0 6.7 1056.4 778.5 14 26.77
+231.8 0.0 121.6 174.0 6.7 1056.4 778.5 28 33.73
+231.8 0.0 121.6 174.0 6.7 1056.4 778.5 56 42.70
+231.8 0.0 121.6 174.0 6.7 1056.4 778.5 100 45.84
+251.4 0.0 118.3 188.5 5.8 1028.4 757.7 3 17.22
+251.4 0.0 118.3 188.5 5.8 1028.4 757.7 14 29.93
+251.4 0.0 118.3 188.5 5.8 1028.4 757.7 28 29.65
+251.4 0.0 118.3 188.5 5.8 1028.4 757.7 56 36.97
+251.4 0.0 118.3 188.5 5.8 1028.4 757.7 100 43.58
+251.4 0.0 118.3 188.5 6.4 1028.4 757.7 3 13.12
+251.4 0.0 118.3 188.5 6.4 1028.4 757.7 14 24.43
+251.4 0.0 118.3 188.5 6.4 1028.4 757.7 28 32.66
+251.4 0.0 118.3 188.5 6.4 1028.4 757.7 56 36.64
+251.4 0.0 118.3 188.5 6.4 1028.4 757.7 100 44.21
+181.4 0.0 167.0 169.6 7.6 1055.6 777.8 3 13.62
+181.4 0.0 167.0 169.6 7.6 1055.6 777.8 14 21.60
+181.4 0.0 167.0 169.6 7.6 1055.6 777.8 28 27.77
+181.4 0.0 167.0 169.6 7.6 1055.6 777.8 56 35.57
+181.4 0.0 167.0 169.6 7.6 1055.6 777.8 100 45.37
+182.0 45.2 122.0 170.2 8.2 1059.4 780.7 3 7.32
+182.0 45.2 122.0 170.2 8.2 1059.4 780.7 14 21.50
+182.0 45.2 122.0 170.2 8.2 1059.4 780.7 28 31.27
+182.0 45.2 122.0 170.2 8.2 1059.4 780.7 56 43.50
+182.0 45.2 122.0 170.2 8.2 1059.4 780.7 100 48.67
+168.9 42.2 124.3 158.3 10.8 1080.8 796.2 3 7.40
+168.9 42.2 124.3 158.3 10.8 1080.8 796.2 14 23.51
+168.9 42.2 124.3 158.3 10.8 1080.8 796.2 28 31.12
+168.9 42.2 124.3 158.3 10.8 1080.8 796.2 56 39.15
+168.9 42.2 124.3 158.3 10.8 1080.8 796.2 100 48.15
+290.4 0.0 96.2 168.1 9.4 961.2 865.0 3 22.50
+290.4 0.0 96.2 168.1 9.4 961.2 865.0 14 34.67
+290.4 0.0 96.2 168.1 9.4 961.2 865.0 28 34.74
+290.4 0.0 96.2 168.1 9.4 961.2 865.0 56 45.08
+290.4 0.0 96.2 168.1 9.4 961.2 865.0 100 48.97
+277.1 0.0 97.4 160.6 11.8 973.9 875.6 3 23.14
+277.1 0.0 97.4 160.6 11.8 973.9 875.6 14 41.89
+277.1 0.0 97.4 160.6 11.8 973.9 875.6 28 48.28
+277.1 0.0 97.4 160.6 11.8 973.9 875.6 56 51.04
+277.1 0.0 97.4 160.6 11.8 973.9 875.6 100 55.64
+295.7 0.0 95.6 171.5 8.9 955.1 859.2 3 22.95
+295.7 0.0 95.6 171.5 8.9 955.1 859.2 14 35.23
+295.7 0.0 95.6 171.5 8.9 955.1 859.2 28 39.94
+295.7 0.0 95.6 171.5 8.9 955.1 859.2 56 48.72
+295.7 0.0 95.6 171.5 8.9 955.1 859.2 100 52.04
+251.8 0.0 99.9 146.1 12.4 1006.0 899.8 3 21.02
+251.8 0.0 99.9 146.1 12.4 1006.0 899.8 14 33.36
+251.8 0.0 99.9 146.1 12.4 1006.0 899.8 28 33.94
+251.8 0.0 99.9 146.1 12.4 1006.0 899.8 56 44.14
+251.8 0.0 99.9 146.1 12.4 1006.0 899.8 100 45.37
+249.1 0.0 98.8 158.1 12.8 987.8 889.0 3 15.36
+249.1 0.0 98.8 158.1 12.8 987.8 889.0 14 28.68
+249.1 0.0 98.8 158.1 12.8 987.8 889.0 28 30.85
+249.1 0.0 98.8 158.1 12.8 987.8 889.0 56 42.03
+249.1 0.0 98.8 158.1 12.8 987.8 889.0 100 51.06
+252.3 0.0 98.8 146.3 14.2 987.8 889.0 3 21.78
+252.3 0.0 98.8 146.3 14.2 987.8 889.0 14 42.29
+252.3 0.0 98.8 146.3 14.2 987.8 889.0 28 50.60
+252.3 0.0 98.8 146.3 14.2 987.8 889.0 56 55.83
+252.3 0.0 98.8 146.3 14.2 987.8 889.0 100 60.95
+246.8 0.0 125.1 143.3 12.0 1086.8 800.9 3 23.52
+246.8 0.0 125.1 143.3 12.0 1086.8 800.9 14 42.22
+246.8 0.0 125.1 143.3 12.0 1086.8 800.9 28 52.50
+246.8 0.0 125.1 143.3 12.0 1086.8 800.9 56 60.32
+246.8 0.0 125.1 143.3 12.0 1086.8 800.9 100 66.42
+275.1 0.0 121.4 159.5 9.9 1053.6 777.5 3 23.80
+275.1 0.0 121.4 159.5 9.9 1053.6 777.5 14 38.77
+275.1 0.0 121.4 159.5 9.9 1053.6 777.5 28 51.33
+275.1 0.0 121.4 159.5 9.9 1053.6 777.5 56 56.85
+275.1 0.0 121.4 159.5 9.9 1053.6 777.5 100 58.61
+297.2 0.0 117.5 174.8 9.5 1022.8 753.5 3 21.91
+297.2 0.0 117.5 174.8 9.5 1022.8 753.5 14 36.99
+297.2 0.0 117.5 174.8 9.5 1022.8 753.5 28 47.40
+297.2 0.0 117.5 174.8 9.5 1022.8 753.5 56 51.96
+297.2 0.0 117.5 174.8 9.5 1022.8 753.5 100 56.74
+213.7 0.0 174.7 154.8 10.2 1053.5 776.4 3 17.57
+213.7 0.0 174.7 154.8 10.2 1053.5 776.4 14 33.73
+213.7 0.0 174.7 154.8 10.2 1053.5 776.4 28 40.15
+213.7 0.0 174.7 154.8 10.2 1053.5 776.4 56 46.64
+213.7 0.0 174.7 154.8 10.2 1053.5 776.4 100 50.08
+213.5 0.0 174.2 154.6 11.7 1052.3 775.5 3 17.37
+213.5 0.0 174.2 154.6 11.7 1052.3 775.5 14 33.70
+213.5 0.0 174.2 154.6 11.7 1052.3 775.5 28 45.94
+213.5 0.0 174.2 154.6 11.7 1052.3 775.5 56 51.43
+213.5 0.0 174.2 154.6 11.7 1052.3 775.5 100 59.30
+277.2 97.8 24.5 160.7 11.2 1061.7 782.5 3 30.45
+277.2 97.8 24.5 160.7 11.2 1061.7 782.5 14 47.71
+277.2 97.8 24.5 160.7 11.2 1061.7 782.5 28 63.14
+277.2 97.8 24.5 160.7 11.2 1061.7 782.5 56 66.82
+277.2 97.8 24.5 160.7 11.2 1061.7 782.5 100 66.95
+218.2 54.6 123.8 140.8 11.9 1075.7 792.7 3 27.42
+218.2 54.6 123.8 140.8 11.9 1075.7 792.7 14 35.96
+218.2 54.6 123.8 140.8 11.9 1075.7 792.7 28 55.51
+218.2 54.6 123.8 140.8 11.9 1075.7 792.7 56 61.99
+218.2 54.6 123.8 140.8 11.9 1075.7 792.7 100 63.53
+214.9 53.8 121.9 155.6 9.6 1014.3 780.6 3 18.02
+214.9 53.8 121.9 155.6 9.6 1014.3 780.6 14 38.60
+214.9 53.8 121.9 155.6 9.6 1014.3 780.6 28 52.20
+214.9 53.8 121.9 155.6 9.6 1014.3 780.6 56 53.96
+214.9 53.8 121.9 155.6 9.6 1014.3 780.6 100 56.63
+218.9 0.0 124.1 158.5 11.3 1078.7 794.9 3 15.34
+218.9 0.0 124.1 158.5 11.3 1078.7 794.9 14 26.05
+218.9 0.0 124.1 158.5 11.3 1078.7 794.9 28 30.22
+218.9 0.0 124.1 158.5 11.3 1078.7 794.9 56 37.27
+218.9 0.0 124.1 158.5 11.3 1078.7 794.9 100 46.23
+376.0 0.0 0.0 214.6 0.0 1003.5 762.4 3 16.28
+376.0 0.0 0.0 214.6 0.0 1003.5 762.4 14 25.62
+376.0 0.0 0.0 214.6 0.0 1003.5 762.4 28 31.97
+376.0 0.0 0.0 214.6 0.0 1003.5 762.4 56 36.30
+376.0 0.0 0.0 214.6 0.0 1003.5 762.4 100 43.06
+500.0 0.0 0.0 140.0 4.0 966.0 853.0 28 67.57
+475.0 0.0 59.0 142.0 1.9 1098.0 641.0 28 57.23
+315.0 137.0 0.0 145.0 5.9 1130.0 745.0 28 81.75
+505.0 0.0 60.0 195.0 0.0 1030.0 630.0 28 64.02
+451.0 0.0 0.0 165.0 11.3 1030.0 745.0 28 78.80
+516.0 0.0 0.0 162.0 8.2 801.0 802.0 28 41.37
+520.0 0.0 0.0 170.0 5.2 855.0 855.0 28 60.28
+528.0 0.0 0.0 185.0 6.9 920.0 720.0 28 56.83
+520.0 0.0 0.0 175.0 5.2 870.0 805.0 28 51.02
+385.0 0.0 136.0 158.0 20.0 903.0 768.0 28 55.55
+500.1 0.0 0.0 200.0 3.0 1124.4 613.2 28 44.13
+450.1 50.0 0.0 200.0 3.0 1124.4 613.2 28 39.38
+397.0 17.2 158.0 167.0 20.8 967.0 633.0 28 55.65
+333.0 17.5 163.0 167.0 17.9 996.0 652.0 28 47.28
+334.0 17.6 158.0 189.0 15.3 967.0 633.0 28 44.33
+405.0 0.0 0.0 175.0 0.0 1120.0 695.0 28 52.30
+200.0 200.0 0.0 190.0 0.0 1145.0 660.0 28 49.25
+516.0 0.0 0.0 162.0 8.3 801.0 802.0 28 41.37
+145.0 116.0 119.0 184.0 5.7 833.0 880.0 28 29.16
+160.0 128.0 122.0 182.0 6.4 824.0 879.0 28 39.40
+234.0 156.0 0.0 189.0 5.9 981.0 760.0 28 39.30
+250.0 180.0 95.0 159.0 9.5 860.0 800.0 28 67.87
+475.0 0.0 0.0 162.0 9.5 1044.0 662.0 28 58.52
+285.0 190.0 0.0 163.0 7.6 1031.0 685.0 28 53.58
+356.0 119.0 0.0 160.0 9.0 1061.0 657.0 28 59.00
+275.0 180.0 120.0 162.0 10.4 830.0 765.0 28 76.24
+500.0 0.0 0.0 151.0 9.0 1033.0 655.0 28 69.84
+165.0 0.0 143.6 163.8 0.0 1005.6 900.9 3 14.40
+165.0 128.5 132.1 175.1 8.1 1005.8 746.6 3 19.42
+178.0 129.8 118.6 179.9 3.6 1007.3 746.8 3 20.73
+167.4 129.9 128.6 175.5 7.8 1006.3 746.6 3 14.94
+172.4 13.6 172.4 156.8 4.1 1006.3 856.4 3 21.29
+173.5 50.1 173.5 164.8 6.5 1006.2 793.5 3 23.08
+167.0 75.4 167.0 164.0 7.9 1007.3 770.1 3 15.52
+173.8 93.4 159.9 172.3 9.7 1007.2 746.6 3 15.82
+190.3 0.0 125.2 166.6 9.9 1079.0 798.9 3 12.55
+250.0 0.0 95.7 191.8 5.3 948.9 857.2 3 8.49
+213.5 0.0 174.2 159.2 11.7 1043.6 771.9 3 15.61
+194.7 0.0 100.5 170.2 7.5 998.0 901.8 3 12.18
+251.4 0.0 118.3 192.9 5.8 1043.6 754.3 3 11.98
+165.0 0.0 143.6 163.8 0.0 1005.6 900.9 14 16.88
+165.0 128.5 132.1 175.1 8.1 1005.8 746.6 14 33.09
+178.0 129.8 118.6 179.9 3.6 1007.3 746.8 14 34.24
+167.4 129.9 128.6 175.5 7.8 1006.3 746.6 14 31.81
+172.4 13.6 172.4 156.8 4.1 1006.3 856.4 14 29.75
+173.5 50.1 173.5 164.8 6.5 1006.2 793.5 14 33.01
+167.0 75.4 167.0 164.0 7.9 1007.3 770.1 14 32.90
+173.8 93.4 159.9 172.3 9.7 1007.2 746.6 14 29.55
+190.3 0.0 125.2 166.6 9.9 1079.0 798.9 14 19.42
+250.0 0.0 95.7 191.8 5.3 948.9 857.2 14 24.66
+213.5 0.0 174.2 159.2 11.7 1043.6 771.9 14 29.59
+194.7 0.0 100.5 170.2 7.5 998.0 901.8 14 24.28
+251.4 0.0 118.3 192.9 5.8 1043.6 754.3 14 20.73
+165.0 0.0 143.6 163.8 0.0 1005.6 900.9 28 26.20
+165.0 128.5 132.1 175.1 8.1 1005.8 746.6 28 46.39
+178.0 129.8 118.6 179.9 3.6 1007.3 746.8 28 39.16
+167.4 129.9 128.6 175.5 7.8 1006.3 746.6 28 41.20
+172.4 13.6 172.4 156.8 4.1 1006.3 856.4 28 33.69
+173.5 50.1 173.5 164.8 6.5 1006.2 793.5 28 38.20
+167.0 75.4 167.0 164.0 7.9 1007.3 770.1 28 41.41
+173.8 93.4 159.9 172.3 9.7 1007.2 746.6 28 37.81
+190.3 0.0 125.2 166.6 9.9 1079.0 798.9 28 24.85
+250.0 0.0 95.7 191.8 5.3 948.9 857.2 28 27.22
+213.5 0.0 174.2 159.2 11.7 1043.6 771.9 28 44.64
+194.7 0.0 100.5 170.2 7.5 998.0 901.8 28 37.27
+251.4 0.0 118.3 192.9 5.8 1043.6 754.3 28 33.27
+165.0 0.0 143.6 163.8 0.0 1005.6 900.9 56 36.56
+165.0 128.5 132.1 175.1 8.1 1005.8 746.6 56 53.72
+178.0 129.8 118.6 179.9 3.6 1007.3 746.8 56 48.59
+167.4 129.9 128.6 175.5 7.8 1006.3 746.6 56 51.72
+172.4 13.6 172.4 156.8 4.1 1006.3 856.4 56 35.85
+173.5 50.1 173.5 164.8 6.5 1006.2 793.5 56 53.77
+167.0 75.4 167.0 164.0 7.9 1007.3 770.1 56 53.46
+173.8 93.4 159.9 172.3 9.7 1007.2 746.6 56 48.99
+190.3 0.0 125.2 166.6 9.9 1079.0 798.9 56 31.72
+250.0 0.0 95.7 191.8 5.3 948.9 857.2 56 39.64
+213.5 0.0 174.2 159.2 11.7 1043.6 771.9 56 51.26
+194.7 0.0 100.5 170.2 7.5 998.0 901.8 56 43.39
+251.4 0.0 118.3 192.9 5.8 1043.6 754.3 56 39.27
+165.0 0.0 143.6 163.8 0.0 1005.6 900.9 100 37.96
+165.0 128.5 132.1 175.1 8.1 1005.8 746.6 100 55.02
+178.0 129.8 118.6 179.9 3.6 1007.3 746.8 100 49.99
+167.4 129.9 128.6 175.5 7.8 1006.3 746.6 100 53.66
+172.4 13.6 172.4 156.8 4.1 1006.3 856.4 100 37.68
+173.5 50.1 173.5 164.8 6.5 1006.2 793.5 100 56.06
+167.0 75.4 167.0 164.0 7.9 1007.3 770.1 100 56.81
+173.8 93.4 159.9 172.3 9.7 1007.2 746.6 100 50.94
+190.3 0.0 125.2 166.6 9.9 1079.0 798.9 100 33.56
+250.0 0.0 95.7 191.8 5.3 948.9 857.2 100 41.16
+213.5 0.0 174.2 159.2 11.7 1043.6 771.9 100 52.96
+194.7 0.0 100.5 170.2 7.5 998.0 901.8 100 44.28
+251.4 0.0 118.3 192.9 5.8 1043.6 754.3 100 40.15
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 28 57.03
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 28 44.42
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 28 51.02
+446.0 24.0 79.0 162.0 10.3 967.0 712.0 28 53.39
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 3 35.36
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 3 25.02
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 3 23.35
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 7 52.01
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 7 38.02
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 7 39.30
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 56 61.07
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 56 56.14
+446.0 24.0 79.0 162.0 11.6 967.0 712.0 56 55.25
+446.0 24.0 79.0 162.0 10.3 967.0 712.0 56 54.77
+387.0 20.0 94.0 157.0 14.3 938.0 845.0 28 50.24
+387.0 20.0 94.0 157.0 13.9 938.0 845.0 28 46.68
+387.0 20.0 94.0 157.0 11.6 938.0 845.0 28 46.68
+387.0 20.0 94.0 157.0 14.3 938.0 845.0 3 22.75
+387.0 20.0 94.0 157.0 13.9 938.0 845.0 3 25.51
+387.0 20.0 94.0 157.0 11.6 938.0 845.0 3 34.77
+387.0 20.0 94.0 157.0 14.3 938.0 845.0 7 36.84
+387.0 20.0 94.0 157.0 13.9 938.0 845.0 7 45.90
+387.0 20.0 94.0 157.0 11.6 938.0 845.0 7 41.67
+387.0 20.0 94.0 157.0 14.3 938.0 845.0 56 56.34
+387.0 20.0 94.0 157.0 13.9 938.0 845.0 56 47.97
+387.0 20.0 94.0 157.0 11.6 938.0 845.0 56 61.46
+355.0 19.0 97.0 145.0 13.1 967.0 871.0 28 44.03
+355.0 19.0 97.0 145.0 12.3 967.0 871.0 28 55.45
+491.0 26.0 123.0 210.0 3.9 882.0 699.0 28 55.55
+491.0 26.0 123.0 201.0 3.9 822.0 699.0 28 57.92
+491.0 26.0 123.0 210.0 3.9 882.0 699.0 3 25.61
+491.0 26.0 123.0 210.0 3.9 882.0 699.0 7 33.49
+491.0 26.0 123.0 210.0 3.9 882.0 699.0 56 59.59
+491.0 26.0 123.0 201.0 3.9 822.0 699.0 3 29.55
+491.0 26.0 123.0 201.0 3.9 822.0 699.0 7 37.92
+491.0 26.0 123.0 201.0 3.9 822.0 699.0 56 61.86
+424.0 22.0 132.0 178.0 8.5 822.0 750.0 28 62.05
+424.0 22.0 132.0 178.0 8.5 882.0 750.0 3 32.01
+424.0 22.0 132.0 168.0 8.9 822.0 750.0 28 72.10
+424.0 22.0 132.0 178.0 8.5 822.0 750.0 7 39.00
+424.0 22.0 132.0 178.0 8.5 822.0 750.0 56 65.70
+424.0 22.0 132.0 168.0 8.9 822.0 750.0 3 32.11
+424.0 22.0 132.0 168.0 8.9 822.0 750.0 7 40.29
+424.0 22.0 132.0 168.0 8.9 822.0 750.0 56 74.36
+202.0 11.0 141.0 206.0 1.7 942.0 801.0 28 21.97
+202.0 11.0 141.0 206.0 1.7 942.0 801.0 3 9.85
+202.0 11.0 141.0 206.0 1.7 942.0 801.0 7 15.07
+202.0 11.0 141.0 206.0 1.7 942.0 801.0 56 23.25
+284.0 15.0 141.0 179.0 5.5 842.0 801.0 28 43.73
+284.0 15.0 141.0 179.0 5.5 842.0 801.0 3 13.40
+284.0 15.0 141.0 179.0 5.5 842.0 801.0 7 24.13
+284.0 15.0 141.0 179.0 5.5 842.0 801.0 56 44.52
+359.0 19.0 141.0 154.0 10.9 942.0 801.0 28 62.94
+359.0 19.0 141.0 154.0 10.9 942.0 801.0 28 59.49
+359.0 19.0 141.0 154.0 10.9 942.0 801.0 3 25.12
+359.0 19.0 141.0 154.0 10.9 942.0 801.0 3 23.64
+359.0 19.0 141.0 154.0 10.9 942.0 801.0 7 35.75
+359.0 19.0 141.0 154.0 10.9 942.0 801.0 7 38.61
+359.0 19.0 141.0 154.0 10.9 942.0 801.0 56 68.75
+359.0 19.0 141.0 154.0 10.9 942.0 801.0 56 66.78
+436.0 0.0 0.0 218.0 0.0 838.4 719.7 28 23.85
+289.0 0.0 0.0 192.0 0.0 913.2 895.3 90 32.07
+289.0 0.0 0.0 192.0 0.0 913.2 895.3 3 11.65
+393.0 0.0 0.0 192.0 0.0 940.6 785.6 3 19.20
+393.0 0.0 0.0 192.0 0.0 940.6 785.6 90 48.85
+393.0 0.0 0.0 192.0 0.0 940.6 785.6 28 39.60
+480.0 0.0 0.0 192.0 0.0 936.2 712.2 28 43.94
+480.0 0.0 0.0 192.0 0.0 936.2 712.2 7 34.57
+480.0 0.0 0.0 192.0 0.0 936.2 712.2 90 54.32
+480.0 0.0 0.0 192.0 0.0 936.2 712.2 3 24.40
+333.0 0.0 0.0 192.0 0.0 931.2 842.6 3 15.62
+255.0 0.0 0.0 192.0 0.0 889.8 945.0 90 21.86
+255.0 0.0 0.0 192.0 0.0 889.8 945.0 7 10.22
+289.0 0.0 0.0 192.0 0.0 913.2 895.3 7 14.60
+255.0 0.0 0.0 192.0 0.0 889.8 945.0 28 18.75
+333.0 0.0 0.0 192.0 0.0 931.2 842.6 28 31.97
+333.0 0.0 0.0 192.0 0.0 931.2 842.6 7 23.40
+289.0 0.0 0.0 192.0 0.0 913.2 895.3 28 25.57
+333.0 0.0 0.0 192.0 0.0 931.2 842.6 90 41.68
+393.0 0.0 0.0 192.0 0.0 940.6 785.6 7 27.74
+255.0 0.0 0.0 192.0 0.0 889.8 945.0 3 8.20
+158.8 238.2 0.0 185.7 0.0 1040.6 734.3 7 9.62
+239.6 359.4 0.0 185.7 0.0 941.6 664.3 7 25.42
+238.2 158.8 0.0 185.7 0.0 1040.6 734.3 7 15.69
+181.9 272.8 0.0 185.7 0.0 1012.4 714.3 28 27.94
+193.5 290.2 0.0 185.7 0.0 998.2 704.3 28 32.63
+255.5 170.3 0.0 185.7 0.0 1026.6 724.3 7 17.24
+272.8 181.9 0.0 185.7 0.0 1012.4 714.3 7 19.77
+239.6 359.4 0.0 185.7 0.0 941.6 664.3 28 39.44
+220.8 147.2 0.0 185.7 0.0 1055.0 744.3 28 25.75
+397.0 0.0 0.0 185.7 0.0 1040.6 734.3 28 33.08
+382.5 0.0 0.0 185.7 0.0 1047.8 739.3 7 24.07
+210.7 316.1 0.0 185.7 0.0 977.0 689.3 7 21.82
+158.8 238.2 0.0 185.7 0.0 1040.6 734.3 28 21.07
+295.8 0.0 0.0 185.7 0.0 1091.4 769.3 7 14.84
+255.5 170.3 0.0 185.7 0.0 1026.6 724.3 28 32.05
+203.5 135.7 0.0 185.7 0.0 1076.2 759.3 7 11.96
+397.0 0.0 0.0 185.7 0.0 1040.6 734.3 7 25.45
+381.4 0.0 0.0 185.7 0.0 1104.6 784.3 28 22.49
+295.8 0.0 0.0 185.7 0.0 1091.4 769.3 28 25.22
+228.0 342.1 0.0 185.7 0.0 955.8 674.3 28 39.70
+220.8 147.2 0.0 185.7 0.0 1055.0 744.3 7 13.09
+316.1 210.7 0.0 185.7 0.0 977.0 689.3 28 38.70
+135.7 203.5 0.0 185.7 0.0 1076.2 759.3 7 7.51
+238.1 0.0 0.0 185.7 0.0 1118.8 789.3 28 17.58
+339.2 0.0 0.0 185.7 0.0 1069.2 754.3 7 21.18
+135.7 203.5 0.0 185.7 0.0 1076.2 759.3 28 18.20
+193.5 290.2 0.0 185.7 0.0 998.2 704.3 7 17.20
+203.5 135.7 0.0 185.7 0.0 1076.2 759.3 28 22.63
+290.2 193.5 0.0 185.7 0.0 998.2 704.3 7 21.86
+181.9 272.8 0.0 185.7 0.0 1012.4 714.3 7 12.37
+170.3 155.5 0.0 185.7 0.0 1026.6 724.3 28 25.73
+210.7 316.1 0.0 185.7 0.0 977.0 689.3 28 37.81
+228.0 342.1 0.0 185.7 0.0 955.8 674.3 7 21.92
+290.2 193.5 0.0 185.7 0.0 998.2 704.3 28 33.04
+381.4 0.0 0.0 185.7 0.0 1104.6 784.3 7 14.54
+238.2 158.8 0.0 185.7 0.0 1040.6 734.3 28 26.91
+186.2 124.1 0.0 185.7 0.0 1083.4 764.3 7 8.00
+339.2 0.0 0.0 185.7 0.0 1069.2 754.3 28 31.90
+238.1 0.0 0.0 185.7 0.0 1118.8 789.3 7 10.34
+252.5 0.0 0.0 185.7 0.0 1111.6 784.3 28 19.77
+382.5 0.0 0.0 185.7 0.0 1047.8 739.3 28 37.44
+252.5 0.0 0.0 185.7 0.0 1111.6 784.3 7 11.48
+316.1 210.7 0.0 185.7 0.0 977.0 689.3 7 24.44
+186.2 124.1 0.0 185.7 0.0 1083.4 764.3 28 17.60
+170.3 155.5 0.0 185.7 0.0 1026.6 724.3 7 10.73
+272.8 181.9 0.0 185.7 0.0 1012.4 714.3 28 31.38
+339.0 0.0 0.0 197.0 0.0 968.0 781.0 3 13.22
+339.0 0.0 0.0 197.0 0.0 968.0 781.0 7 20.97
+339.0 0.0 0.0 197.0 0.0 968.0 781.0 14 27.04
+339.0 0.0 0.0 197.0 0.0 968.0 781.0 28 32.04
+339.0 0.0 0.0 197.0 0.0 968.0 781.0 90 35.17
+339.0 0.0 0.0 197.0 0.0 968.0 781.0 180 36.45
+339.0 0.0 0.0 197.0 0.0 968.0 781.0 365 38.89
+236.0 0.0 0.0 194.0 0.0 968.0 885.0 3 6.47
+236.0 0.0 0.0 194.0 0.0 968.0 885.0 14 12.84
+236.0 0.0 0.0 194.0 0.0 968.0 885.0 28 18.42
+236.0 0.0 0.0 194.0 0.0 968.0 885.0 90 21.95
+236.0 0.0 0.0 193.0 0.0 968.0 885.0 180 24.10
+236.0 0.0 0.0 193.0 0.0 968.0 885.0 365 25.08
+277.0 0.0 0.0 191.0 0.0 968.0 856.0 14 21.26
+277.0 0.0 0.0 191.0 0.0 968.0 856.0 28 25.97
+277.0 0.0 0.0 191.0 0.0 968.0 856.0 3 11.36
+277.0 0.0 0.0 191.0 0.0 968.0 856.0 90 31.25
+277.0 0.0 0.0 191.0 0.0 968.0 856.0 180 32.33
+277.0 0.0 0.0 191.0 0.0 968.0 856.0 360 33.70
+254.0 0.0 0.0 198.0 0.0 968.0 863.0 3 9.31
+254.0 0.0 0.0 198.0 0.0 968.0 863.0 90 26.94
+254.0 0.0 0.0 198.0 0.0 968.0 863.0 180 27.63
+254.0 0.0 0.0 198.0 0.0 968.0 863.0 365 29.79
+307.0 0.0 0.0 193.0 0.0 968.0 812.0 180 34.49
+307.0 0.0 0.0 193.0 0.0 968.0 812.0 365 36.15
+307.0 0.0 0.0 193.0 0.0 968.0 812.0 3 12.54
+307.0 0.0 0.0 193.0 0.0 968.0 812.0 28 27.53
+307.0 0.0 0.0 193.0 0.0 968.0 812.0 90 32.92
+236.0 0.0 0.0 193.0 0.0 968.0 885.0 7 9.99
+200.0 0.0 0.0 180.0 0.0 1125.0 845.0 7 7.84
+200.0 0.0 0.0 180.0 0.0 1125.0 845.0 28 12.25
+225.0 0.0 0.0 181.0 0.0 1113.0 833.0 7 11.17
+225.0 0.0 0.0 181.0 0.0 1113.0 833.0 28 17.34
+325.0 0.0 0.0 184.0 0.0 1063.0 783.0 7 17.54
+325.0 0.0 0.0 184.0 0.0 1063.0 783.0 28 30.57
+275.0 0.0 0.0 183.0 0.0 1088.0 808.0 7 14.20
+275.0 0.0 0.0 183.0 0.0 1088.0 808.0 28 24.50
+300.0 0.0 0.0 184.0 0.0 1075.0 795.0 7 15.58
+300.0 0.0 0.0 184.0 0.0 1075.0 795.0 28 26.85
+375.0 0.0 0.0 186.0 0.0 1038.0 758.0 7 26.06
+375.0 0.0 0.0 186.0 0.0 1038.0 758.0 28 38.21
+400.0 0.0 0.0 187.0 0.0 1025.0 745.0 28 43.70
+400.0 0.0 0.0 187.0 0.0 1025.0 745.0 7 30.14
+250.0 0.0 0.0 182.0 0.0 1100.0 820.0 7 12.73
+250.0 0.0 0.0 182.0 0.0 1100.0 820.0 28 20.87
+350.0 0.0 0.0 186.0 0.0 1050.0 770.0 7 20.28
+350.0 0.0 0.0 186.0 0.0 1050.0 770.0 28 34.29
+203.5 305.3 0.0 203.5 0.0 963.4 630.0 7 19.54
+250.2 166.8 0.0 203.5 0.0 977.6 694.1 90 47.71
+157.0 236.0 0.0 192.0 0.0 935.4 781.2 90 43.38
+141.3 212.0 0.0 203.5 0.0 971.8 748.5 28 29.89
+166.8 250.2 0.0 203.5 0.0 975.6 692.6 3 6.90
+122.6 183.9 0.0 203.5 0.0 958.2 800.1 90 33.19
+183.9 122.6 0.0 203.5 0.0 959.2 800.0 3 4.90
+102.0 153.0 0.0 192.0 0.0 887.0 942.0 3 4.57
+102.0 153.0 0.0 192.0 0.0 887.0 942.0 90 25.46
+122.6 183.9 0.0 203.5 0.0 958.2 800.1 28 24.29
+166.8 250.2 0.0 203.5 0.0 975.6 692.6 28 33.95
+200.0 133.0 0.0 192.0 0.0 965.4 806.2 3 11.41
+108.3 162.4 0.0 203.5 0.0 938.2 849.0 28 20.59
+305.3 203.5 0.0 203.5 0.0 965.4 631.0 7 25.89
+108.3 162.4 0.0 203.5 0.0 938.2 849.0 90 29.23
+116.0 173.0 0.0 192.0 0.0 909.8 891.9 90 31.02
+141.3 212.0 0.0 203.5 0.0 971.8 748.5 7 10.39
+157.0 236.0 0.0 192.0 0.0 935.4 781.2 28 33.66
+133.0 200.0 0.0 192.0 0.0 927.4 839.2 28 27.87
+250.2 166.8 0.0 203.5 0.0 977.6 694.1 7 19.35
+173.0 116.0 0.0 192.0 0.0 946.8 856.8 7 11.39
+192.0 288.0 0.0 192.0 0.0 929.8 716.1 3 12.79
+192.0 288.0 0.0 192.0 0.0 929.8 716.1 28 39.32
+153.0 102.0 0.0 192.0 0.0 888.0 943.1 3 4.78
+288.0 192.0 0.0 192.0 0.0 932.0 717.8 3 16.11
+305.3 203.5 0.0 203.5 0.0 965.4 631.0 28 43.38
+236.0 157.0 0.0 192.0 0.0 972.6 749.1 7 20.42
+173.0 116.0 0.0 192.0 0.0 946.8 856.8 3 6.94
+212.0 141.3 0.0 203.5 0.0 973.4 750.0 7 15.03
+236.0 157.0 0.0 192.0 0.0 972.6 749.1 3 13.57
+183.9 122.6 0.0 203.5 0.0 959.2 800.0 90 32.53
+166.8 250.2 0.0 203.5 0.0 975.6 692.6 7 15.75
+102.0 153.0 0.0 192.0 0.0 887.0 942.0 7 7.68
+288.0 192.0 0.0 192.0 0.0 932.0 717.8 28 38.80
+212.0 141.3 0.0 203.5 0.0 973.4 750.0 28 33.00
+102.0 153.0 0.0 192.0 0.0 887.0 942.0 28 17.28
+173.0 116.0 0.0 192.0 0.0 946.8 856.8 28 24.28
+183.9 122.6 0.0 203.5 0.0 959.2 800.0 28 24.05
+133.0 200.0 0.0 192.0 0.0 927.4 839.2 90 36.59
+192.0 288.0 0.0 192.0 0.0 929.8 716.1 90 50.73
+133.0 200.0 0.0 192.0 0.0 927.4 839.2 7 13.66
+305.3 203.5 0.0 203.5 0.0 965.4 631.0 3 14.14
+236.0 157.0 0.0 192.0 0.0 972.6 749.1 90 47.78
+108.3 162.4 0.0 203.5 0.0 938.2 849.0 3 2.33
+157.0 236.0 0.0 192.0 0.0 935.4 781.2 7 16.89
+288.0 192.0 0.0 192.0 0.0 932.0 717.8 7 23.52
+212.0 141.3 0.0 203.5 0.0 973.4 750.0 3 6.81
+212.0 141.3 0.0 203.5 0.0 973.4 750.0 90 39.70
+153.0 102.0 0.0 192.0 0.0 888.0 943.1 28 17.96
+236.0 157.0 0.0 192.0 0.0 972.6 749.1 28 32.88
+116.0 173.0 0.0 192.0 0.0 909.8 891.9 28 22.35
+183.9 122.6 0.0 203.5 0.0 959.2 800.0 7 10.79
+108.3 162.4 0.0 203.5 0.0 938.2 849.0 7 7.72
+203.5 305.3 0.0 203.5 0.0 963.4 630.0 28 41.68
+203.5 305.3 0.0 203.5 0.0 963.4 630.0 3 9.56
+133.0 200.0 0.0 192.0 0.0 927.4 839.2 3 6.88
+288.0 192.0 0.0 192.0 0.0 932.0 717.8 90 50.53
+200.0 133.0 0.0 192.0 0.0 965.4 806.2 7 17.17
+200.0 133.0 0.0 192.0 0.0 965.4 806.2 28 30.44
+250.2 166.8 0.0 203.5 0.0 977.6 694.1 3 9.73
+122.6 183.9 0.0 203.5 0.0 958.2 800.1 3 3.32
+153.0 102.0 0.0 192.0 0.0 888.0 943.1 90 26.32
+200.0 133.0 0.0 192.0 0.0 965.4 806.2 90 43.25
+116.0 173.0 0.0 192.0 0.0 909.8 891.9 3 6.28
+173.0 116.0 0.0 192.0 0.0 946.8 856.8 90 32.10
+250.2 166.8 0.0 203.5 0.0 977.6 694.1 28 36.96
+305.3 203.5 0.0 203.5 0.0 965.4 631.0 90 54.60
+192.0 288.0 0.0 192.0 0.0 929.8 716.1 7 21.48
+157.0 236.0 0.0 192.0 0.0 935.4 781.2 3 9.69
+153.0 102.0 0.0 192.0 0.0 888.0 943.1 7 8.37
+141.3 212.0 0.0 203.5 0.0 971.8 748.5 90 39.66
+116.0 173.0 0.0 192.0 0.0 909.8 891.9 7 10.09
+141.3 212.0 0.0 203.5 0.0 971.8 748.5 3 4.83
+122.6 183.9 0.0 203.5 0.0 958.2 800.1 7 10.35
+166.8 250.2 0.0 203.5 0.0 975.6 692.6 90 43.57
+203.5 305.3 0.0 203.5 0.0 963.4 630.0 90 51.86
+310.0 0.0 0.0 192.0 0.0 1012.0 830.0 3 11.85
+310.0 0.0 0.0 192.0 0.0 1012.0 830.0 7 17.24
+310.0 0.0 0.0 192.0 0.0 1012.0 830.0 28 27.83
+310.0 0.0 0.0 192.0 0.0 1012.0 830.0 90 35.76
+310.0 0.0 0.0 192.0 0.0 1012.0 830.0 120 38.70
+331.0 0.0 0.0 192.0 0.0 1025.0 821.0 3 14.31
+331.0 0.0 0.0 192.0 0.0 1025.0 821.0 7 17.44
+331.0 0.0 0.0 192.0 0.0 1025.0 821.0 28 31.74
+331.0 0.0 0.0 192.0 0.0 1025.0 821.0 90 37.91
+331.0 0.0 0.0 192.0 0.0 1025.0 821.0 120 39.38
+349.0 0.0 0.0 192.0 0.0 1056.0 809.0 3 15.87
+349.0 0.0 0.0 192.0 0.0 1056.0 809.0 7 9.01
+349.0 0.0 0.0 192.0 0.0 1056.0 809.0 28 33.61
+349.0 0.0 0.0 192.0 0.0 1056.0 809.0 90 40.66
+349.0 0.0 0.0 192.0 0.0 1056.0 809.0 120 40.86
+238.0 0.0 0.0 186.0 0.0 1119.0 789.0 7 12.05
+238.0 0.0 0.0 186.0 0.0 1119.0 789.0 28 17.54
+296.0 0.0 0.0 186.0 0.0 1090.0 769.0 7 18.91
+296.0 0.0 0.0 186.0 0.0 1090.0 769.0 28 25.18
+297.0 0.0 0.0 186.0 0.0 1040.0 734.0 7 30.96
+480.0 0.0 0.0 192.0 0.0 936.0 721.0 28 43.89
+480.0 0.0 0.0 192.0 0.0 936.0 721.0 90 54.28
+397.0 0.0 0.0 186.0 0.0 1040.0 734.0 28 36.94
+281.0 0.0 0.0 186.0 0.0 1104.0 774.0 7 14.50
+281.0 0.0 0.0 185.0 0.0 1104.0 774.0 28 22.44
+500.0 0.0 0.0 200.0 0.0 1125.0 613.0 1 12.64
+500.0 0.0 0.0 200.0 0.0 1125.0 613.0 3 26.06
+500.0 0.0 0.0 200.0 0.0 1125.0 613.0 7 33.21
+500.0 0.0 0.0 200.0 0.0 1125.0 613.0 14 36.94
+500.0 0.0 0.0 200.0 0.0 1125.0 613.0 28 44.09
+540.0 0.0 0.0 173.0 0.0 1125.0 613.0 7 52.61
+540.0 0.0 0.0 173.0 0.0 1125.0 613.0 14 59.76
+540.0 0.0 0.0 173.0 0.0 1125.0 613.0 28 67.31
+540.0 0.0 0.0 173.0 0.0 1125.0 613.0 90 69.66
+540.0 0.0 0.0 173.0 0.0 1125.0 613.0 180 71.62
+540.0 0.0 0.0 173.0 0.0 1125.0 613.0 270 74.17
+350.0 0.0 0.0 203.0 0.0 974.0 775.0 7 18.13
+350.0 0.0 0.0 203.0 0.0 974.0 775.0 14 22.53
+350.0 0.0 0.0 203.0 0.0 974.0 775.0 28 27.34
+350.0 0.0 0.0 203.0 0.0 974.0 775.0 56 29.98
+350.0 0.0 0.0 203.0 0.0 974.0 775.0 90 31.35
+350.0 0.0 0.0 203.0 0.0 974.0 775.0 180 32.72
+385.0 0.0 0.0 186.0 0.0 966.0 763.0 1 6.27
+385.0 0.0 0.0 186.0 0.0 966.0 763.0 3 14.70
+385.0 0.0 0.0 186.0 0.0 966.0 763.0 7 23.22
+385.0 0.0 0.0 186.0 0.0 966.0 763.0 14 27.92
+385.0 0.0 0.0 186.0 0.0 966.0 763.0 28 31.35
+331.0 0.0 0.0 192.0 0.0 978.0 825.0 180 39.00
+331.0 0.0 0.0 192.0 0.0 978.0 825.0 360 41.24
+349.0 0.0 0.0 192.0 0.0 1047.0 806.0 3 14.99
+331.0 0.0 0.0 192.0 0.0 978.0 825.0 3 13.52
+382.0 0.0 0.0 186.0 0.0 1047.0 739.0 7 24.00
+382.0 0.0 0.0 186.0 0.0 1047.0 739.0 28 37.42
+382.0 0.0 0.0 186.0 0.0 1111.0 784.0 7 11.47
+281.0 0.0 0.0 186.0 0.0 1104.0 774.0 28 22.44
+339.0 0.0 0.0 185.0 0.0 1069.0 754.0 7 21.16
+339.0 0.0 0.0 185.0 0.0 1069.0 754.0 28 31.84
+295.0 0.0 0.0 185.0 0.0 1069.0 769.0 7 14.80
+295.0 0.0 0.0 185.0 0.0 1069.0 769.0 28 25.18
+238.0 0.0 0.0 185.0 0.0 1118.0 789.0 28 17.54
+296.0 0.0 0.0 192.0 0.0 1085.0 765.0 7 14.20
+296.0 0.0 0.0 192.0 0.0 1085.0 765.0 28 21.65
+296.0 0.0 0.0 192.0 0.0 1085.0 765.0 90 29.39
+331.0 0.0 0.0 192.0 0.0 879.0 825.0 3 13.52
+331.0 0.0 0.0 192.0 0.0 978.0 825.0 7 16.26
+331.0 0.0 0.0 192.0 0.0 978.0 825.0 28 31.45
+331.0 0.0 0.0 192.0 0.0 978.0 825.0 90 37.23
+349.0 0.0 0.0 192.0 0.0 1047.0 806.0 7 18.13
+349.0 0.0 0.0 192.0 0.0 1047.0 806.0 28 32.72
+349.0 0.0 0.0 192.0 0.0 1047.0 806.0 90 39.49
+349.0 0.0 0.0 192.0 0.0 1047.0 806.0 180 41.05
+349.0 0.0 0.0 192.0 0.0 1047.0 806.0 360 42.13
+302.0 0.0 0.0 203.0 0.0 974.0 817.0 14 18.13
+302.0 0.0 0.0 203.0 0.0 974.0 817.0 180 26.74
+525.0 0.0 0.0 189.0 0.0 1125.0 613.0 180 61.92
+500.0 0.0 0.0 200.0 0.0 1125.0 613.0 90 47.22
+500.0 0.0 0.0 200.0 0.0 1125.0 613.0 180 51.04
+500.0 0.0 0.0 200.0 0.0 1125.0 613.0 270 55.16
+540.0 0.0 0.0 173.0 0.0 1125.0 613.0 3 41.64
+252.0 0.0 0.0 185.0 0.0 1111.0 784.0 7 13.71
+252.0 0.0 0.0 185.0 0.0 1111.0 784.0 28 19.69
+339.0 0.0 0.0 185.0 0.0 1060.0 754.0 28 31.65
+393.0 0.0 0.0 192.0 0.0 940.0 758.0 3 19.11
+393.0 0.0 0.0 192.0 0.0 940.0 758.0 28 39.58
+393.0 0.0 0.0 192.0 0.0 940.0 758.0 90 48.79
+382.0 0.0 0.0 185.0 0.0 1047.0 739.0 7 24.00
+382.0 0.0 0.0 185.0 0.0 1047.0 739.0 28 37.42
+252.0 0.0 0.0 186.0 0.0 1111.0 784.0 7 11.47
+252.0 0.0 0.0 185.0 0.0 1111.0 784.0 28 19.69
+310.0 0.0 0.0 192.0 0.0 970.0 850.0 7 14.99
+310.0 0.0 0.0 192.0 0.0 970.0 850.0 28 27.92
+310.0 0.0 0.0 192.0 0.0 970.0 850.0 90 34.68
+310.0 0.0 0.0 192.0 0.0 970.0 850.0 180 37.33
+310.0 0.0 0.0 192.0 0.0 970.0 850.0 360 38.11
+525.0 0.0 0.0 189.0 0.0 1125.0 613.0 3 33.80
+525.0 0.0 0.0 189.0 0.0 1125.0 613.0 7 42.42
+525.0 0.0 0.0 189.0 0.0 1125.0 613.0 14 48.40
+525.0 0.0 0.0 189.0 0.0 1125.0 613.0 28 55.94
+525.0 0.0 0.0 189.0 0.0 1125.0 613.0 90 58.78
+525.0 0.0 0.0 189.0 0.0 1125.0 613.0 270 67.11
+322.0 0.0 0.0 203.0 0.0 974.0 800.0 14 20.77
+322.0 0.0 0.0 203.0 0.0 974.0 800.0 28 25.18
+322.0 0.0 0.0 203.0 0.0 974.0 800.0 180 29.59
+302.0 0.0 0.0 203.0 0.0 974.0 817.0 28 21.75
+397.0 0.0 0.0 185.0 0.0 1040.0 734.0 28 39.09
+480.0 0.0 0.0 192.0 0.0 936.0 721.0 3 24.39
+522.0 0.0 0.0 146.0 0.0 896.0 896.0 7 50.51
+522.0 0.0 0.0 146.0 0.0 896.0 896.0 28 74.99
+273.0 105.0 82.0 210.0 9.0 904.0 680.0 28 37.17
+162.0 190.0 148.0 179.0 19.0 838.0 741.0 28 33.76
+154.0 144.0 112.0 220.0 10.0 923.0 658.0 28 16.50
+147.0 115.0 89.0 202.0 9.0 860.0 829.0 28 19.99
+152.0 178.0 139.0 168.0 18.0 944.0 695.0 28 36.35
+310.0 143.0 111.0 168.0 22.0 914.0 651.0 28 33.69
+144.0 0.0 175.0 158.0 18.0 943.0 844.0 28 15.42
+304.0 140.0 0.0 214.0 6.0 895.0 722.0 28 33.42
+374.0 0.0 0.0 190.0 7.0 1013.0 730.0 28 39.05
+159.0 149.0 116.0 175.0 15.0 953.0 720.0 28 27.68
+153.0 239.0 0.0 200.0 6.0 1002.0 684.0 28 26.86
+310.0 143.0 0.0 168.0 10.0 914.0 804.0 28 45.30
+305.0 0.0 100.0 196.0 10.0 959.0 705.0 28 30.12
+151.0 0.0 184.0 167.0 12.0 991.0 772.0 28 15.57
+142.0 167.0 130.0 174.0 11.0 883.0 785.0 28 44.61
+298.0 137.0 107.0 201.0 6.0 878.0 655.0 28 53.52
+321.0 164.0 0.0 190.0 5.0 870.0 774.0 28 57.21
+366.0 187.0 0.0 191.0 7.0 824.0 757.0 28 65.91
+280.0 129.0 100.0 172.0 9.0 825.0 805.0 28 52.82
+252.0 97.0 76.0 194.0 8.0 835.0 821.0 28 33.40
+165.0 0.0 150.0 182.0 12.0 1023.0 729.0 28 18.03
+156.0 243.0 0.0 180.0 11.0 1022.0 698.0 28 37.36
+160.0 188.0 146.0 203.0 11.0 829.0 710.0 28 32.84
+298.0 0.0 107.0 186.0 6.0 879.0 815.0 28 42.64
+318.0 0.0 126.0 210.0 6.0 861.0 737.0 28 40.06
+287.0 121.0 94.0 188.0 9.0 904.0 696.0 28 41.94
+326.0 166.0 0.0 174.0 9.0 882.0 790.0 28 61.23
+356.0 0.0 142.0 193.0 11.0 801.0 778.0 28 40.87
+132.0 207.0 161.0 179.0 5.0 867.0 736.0 28 33.30
+322.0 149.0 0.0 186.0 8.0 951.0 709.0 28 52.42
+164.0 0.0 200.0 181.0 13.0 849.0 846.0 28 15.09
+314.0 0.0 113.0 170.0 10.0 925.0 783.0 28 38.46
+321.0 0.0 128.0 182.0 11.0 870.0 780.0 28 37.26
+140.0 164.0 128.0 237.0 6.0 869.0 656.0 28 35.23
+288.0 121.0 0.0 177.0 7.0 908.0 829.0 28 42.13
+298.0 0.0 107.0 210.0 11.0 880.0 744.0 28 31.87
+265.0 111.0 86.0 195.0 6.0 833.0 790.0 28 41.54
+160.0 250.0 0.0 168.0 12.0 1049.0 688.0 28 39.45
+166.0 260.0 0.0 183.0 13.0 859.0 827.0 28 37.91
+276.0 116.0 90.0 180.0 9.0 870.0 768.0 28 44.28
+322.0 0.0 116.0 196.0 10.0 818.0 813.0 28 31.18
+149.0 139.0 109.0 193.0 6.0 892.0 780.0 28 23.69
+159.0 187.0 0.0 176.0 11.0 990.0 789.0 28 32.76
+261.0 100.0 78.0 201.0 9.0 864.0 761.0 28 32.40
+237.0 92.0 71.0 247.0 6.0 853.0 695.0 28 28.63
+313.0 0.0 113.0 178.0 8.0 1002.0 689.0 28 36.80
+155.0 183.0 0.0 193.0 9.0 1047.0 697.0 28 18.28
+146.0 230.0 0.0 202.0 3.0 827.0 872.0 28 33.06
+296.0 0.0 107.0 221.0 11.0 819.0 778.0 28 31.42
+133.0 210.0 0.0 196.0 3.0 949.0 795.0 28 31.03
+313.0 145.0 0.0 178.0 8.0 867.0 824.0 28 44.39
+152.0 0.0 112.0 184.0 8.0 992.0 816.0 28 12.18
+153.0 145.0 113.0 178.0 8.0 1002.0 689.0 28 25.56
+140.0 133.0 103.0 200.0 7.0 916.0 753.0 28 36.44
+149.0 236.0 0.0 176.0 13.0 847.0 893.0 28 32.96
+300.0 0.0 120.0 212.0 10.0 878.0 728.0 28 23.84
+153.0 145.0 113.0 178.0 8.0 867.0 824.0 28 26.23
+148.0 0.0 137.0 158.0 16.0 1002.0 830.0 28 17.95
+326.0 0.0 138.0 199.0 11.0 801.0 792.0 28 40.68
+153.0 145.0 0.0 178.0 8.0 1000.0 822.0 28 19.01
+262.0 111.0 86.0 195.0 5.0 895.0 733.0 28 33.72
+158.0 0.0 195.0 220.0 11.0 898.0 713.0 28 8.54
+151.0 0.0 185.0 167.0 16.0 1074.0 678.0 28 13.46
+273.0 0.0 90.0 199.0 11.0 931.0 762.0 28 32.24
+149.0 118.0 92.0 183.0 7.0 953.0 780.0 28 23.52
+143.0 169.0 143.0 191.0 8.0 967.0 643.0 28 29.72
+260.0 101.0 78.0 171.0 10.0 936.0 763.0 28 49.77
+313.0 161.0 0.0 178.0 10.0 917.0 759.0 28 52.44
+284.0 120.0 0.0 168.0 7.0 970.0 794.0 28 40.93
+336.0 0.0 0.0 182.0 3.0 986.0 817.0 28 44.86
+145.0 0.0 134.0 181.0 11.0 979.0 812.0 28 13.20
+150.0 237.0 0.0 174.0 12.0 1069.0 675.0 28 37.43
+144.0 170.0 133.0 192.0 8.0 814.0 805.0 28 29.87
+331.0 170.0 0.0 195.0 8.0 811.0 802.0 28 56.61
+155.0 0.0 143.0 193.0 9.0 1047.0 697.0 28 12.46
+155.0 183.0 0.0 193.0 9.0 877.0 868.0 28 23.79
+135.0 0.0 166.0 180.0 10.0 961.0 805.0 28 13.29
+266.0 112.0 87.0 178.0 10.0 910.0 745.0 28 39.42
+314.0 145.0 113.0 179.0 8.0 869.0 690.0 28 46.23
+313.0 145.0 0.0 127.0 8.0 1000.0 822.0 28 44.52
+146.0 173.0 0.0 182.0 3.0 986.0 817.0 28 23.74
+144.0 136.0 106.0 178.0 7.0 941.0 774.0 28 26.14
+148.0 0.0 182.0 181.0 15.0 839.0 884.0 28 15.52
+277.0 117.0 91.0 191.0 7.0 946.0 666.0 28 43.57
+298.0 0.0 107.0 164.0 13.0 953.0 784.0 28 35.86
+313.0 145.0 0.0 178.0 8.0 1002.0 689.0 28 41.05
+155.0 184.0 143.0 194.0 9.0 880.0 699.0 28 28.99
+289.0 134.0 0.0 195.0 6.0 924.0 760.0 28 46.24
+148.0 175.0 0.0 171.0 2.0 1000.0 828.0 28 26.92
+145.0 0.0 179.0 202.0 8.0 824.0 869.0 28 10.54
+313.0 0.0 0.0 178.0 8.0 1000.0 822.0 28 25.10
+136.0 162.0 126.0 172.0 10.0 923.0 764.0 28 29.07
+155.0 0.0 143.0 193.0 9.0 877.0 868.0 28 9.74
+255.0 99.0 77.0 189.0 6.0 919.0 749.0 28 33.80
+162.0 207.0 172.0 216.0 10.0 822.0 638.0 28 39.84
+136.0 196.0 98.0 199.0 6.0 847.0 783.0 28 26.97
+164.0 163.0 128.0 197.0 8.0 961.0 641.0 28 27.23
+162.0 214.0 164.0 202.0 10.0 820.0 680.0 28 30.65
+157.0 214.0 152.0 200.0 9.0 819.0 704.0 28 33.05
+149.0 153.0 194.0 192.0 8.0 935.0 623.0 28 24.58
+135.0 105.0 193.0 196.0 6.0 965.0 643.0 28 21.91
+159.0 209.0 161.0 201.0 7.0 848.0 669.0 28 30.88
+144.0 15.0 195.0 176.0 6.0 1021.0 709.0 28 15.34
+154.0 174.0 185.0 228.0 7.0 845.0 612.0 28 24.34
+167.0 187.0 195.0 185.0 7.0 898.0 636.0 28 23.89
+184.0 86.0 190.0 213.0 6.0 923.0 623.0 28 22.93
+156.0 178.0 187.0 221.0 7.0 854.0 614.0 28 29.41
+236.9 91.7 71.5 246.9 6.0 852.9 695.4 28 28.63
+313.3 0.0 113.0 178.5 8.0 1001.9 688.7 28 36.80
+154.8 183.4 0.0 193.3 9.1 1047.4 696.7 28 18.29
+145.9 230.5 0.0 202.5 3.4 827.0 871.8 28 32.72
+296.0 0.0 106.7 221.4 10.5 819.2 778.4 28 31.42
+133.1 210.2 0.0 195.7 3.1 949.4 795.3 28 28.94
+313.3 145.0 0.0 178.5 8.0 867.2 824.0 28 40.93
+151.6 0.0 111.9 184.4 7.9 992.0 815.9 28 12.18
+153.1 145.0 113.0 178.5 8.0 1001.9 688.7 28 25.56
+139.9 132.6 103.3 200.3 7.4 916.0 753.4 28 36.44
+149.5 236.0 0.0 175.8 12.6 846.8 892.7 28 32.96
+299.8 0.0 119.8 211.5 9.9 878.2 727.6 28 23.84
+153.1 145.0 113.0 178.5 8.0 867.2 824.0 28 26.23
+148.1 0.0 136.6 158.1 16.1 1001.8 830.1 28 17.96
+326.5 0.0 137.9 199.0 10.8 801.1 792.5 28 38.63
+152.7 144.7 0.0 178.1 8.0 999.7 822.2 28 19.01
+261.9 110.5 86.1 195.4 5.0 895.2 732.6 28 33.72
+158.4 0.0 194.9 219.7 11.0 897.7 712.9 28 8.54
+150.7 0.0 185.3 166.7 15.6 1074.5 678.0 28 13.46
+272.6 0.0 89.6 198.7 10.6 931.3 762.2 28 32.25
+149.0 117.6 91.7 182.9 7.1 953.4 780.3 28 23.52
+143.0 169.4 142.7 190.7 8.4 967.4 643.5 28 29.73
+259.9 100.6 78.4 170.6 10.4 935.7 762.9 28 49.77
+312.9 160.5 0.0 177.6 9.6 916.6 759.5 28 52.45
+284.0 119.7 0.0 168.3 7.2 970.4 794.2 28 40.93
+336.5 0.0 0.0 181.9 3.4 985.8 816.8 28 44.87
+144.8 0.0 133.6 180.8 11.1 979.5 811.5 28 13.20
+150.0 236.8 0.0 173.8 11.9 1069.3 674.8 28 37.43
+143.7 170.2 132.6 191.6 8.5 814.1 805.3 28 29.87
+330.5 169.6 0.0 194.9 8.1 811.0 802.3 28 56.62
+154.8 0.0 142.8 193.3 9.1 1047.4 696.7 28 12.46
+154.8 183.4 0.0 193.3 9.1 877.2 867.7 28 23.79
+134.7 0.0 165.7 180.2 10.0 961.0 804.9 28 13.29
+266.2 112.3 87.5 177.9 10.4 909.7 744.5 28 39.42
+314.0 145.3 113.2 178.9 8.0 869.1 690.2 28 46.23
+312.7 144.7 0.0 127.3 8.0 999.7 822.2 28 44.52
+145.7 172.6 0.0 181.9 3.4 985.8 816.8 28 23.74
+143.8 136.3 106.2 178.1 7.5 941.5 774.3 28 26.15
+148.1 0.0 182.1 181.4 15.0 838.9 884.3 28 15.53
+277.0 116.8 91.0 190.6 7.0 946.5 665.6 28 43.58
+298.1 0.0 107.5 163.6 12.8 953.2 784.0 28 35.87
+313.3 145.0 0.0 178.5 8.0 1001.9 688.7 28 41.05
+155.2 183.9 143.2 193.8 9.2 879.6 698.5 28 28.99
+289.0 133.7 0.0 194.9 5.5 924.1 760.1 28 46.25
+147.8 175.1 0.0 171.2 2.2 1000.0 828.5 28 26.92
+145.4 0.0 178.9 201.7 7.8 824.0 868.7 28 10.54
+312.7 0.0 0.0 178.1 8.0 999.7 822.2 28 25.10
+136.4 161.6 125.8 171.6 10.4 922.6 764.4 28 29.07
+154.8 0.0 142.8 193.3 9.1 877.2 867.7 28 9.74
+255.3 98.8 77.0 188.6 6.5 919.0 749.3 28 33.80
+272.8 105.1 81.8 209.7 9.0 904.0 679.7 28 37.17
+162.0 190.1 148.1 178.8 18.8 838.1 741.4 28 33.76
+153.6 144.2 112.3 220.1 10.1 923.2 657.9 28 16.50
+146.5 114.6 89.3 201.9 8.8 860.0 829.5 28 19.99
+151.8 178.1 138.7 167.5 18.3 944.0 694.6 28 36.35
+309.9 142.8 111.2 167.8 22.1 913.9 651.2 28 38.22
+143.6 0.0 174.9 158.4 17.9 942.7 844.5 28 15.42
+303.6 139.9 0.0 213.5 6.2 895.5 722.5 28 33.42
+374.3 0.0 0.0 190.2 6.7 1013.2 730.4 28 39.06
+158.6 148.9 116.0 175.1 15.0 953.3 719.7 28 27.68
+152.6 238.7 0.0 200.0 6.3 1001.8 683.9 28 26.86
+310.0 142.8 0.0 167.9 10.0 914.3 804.0 28 45.30
+304.8 0.0 99.6 196.0 9.8 959.4 705.2 28 30.12
+150.9 0.0 183.9 166.6 11.6 991.2 772.2 28 15.57
+141.9 166.6 129.7 173.5 10.9 882.6 785.3 28 44.61
+297.8 137.2 106.9 201.3 6.0 878.4 655.3 28 53.52
+321.3 164.2 0.0 190.5 4.6 870.0 774.0 28 57.22
+366.0 187.0 0.0 191.3 6.6 824.3 756.9 28 65.91
+279.8 128.9 100.4 172.4 9.5 825.1 804.9 28 52.83
+252.1 97.1 75.6 193.8 8.3 835.5 821.4 28 33.40
+164.6 0.0 150.4 181.6 11.7 1023.3 728.9 28 18.03
+155.6 243.5 0.0 180.3 10.7 1022.0 697.7 28 37.36
+160.2 188.0 146.4 203.2 11.3 828.7 709.7 28 35.31
+298.1 0.0 107.0 186.4 6.1 879.0 815.2 28 42.64
+317.9 0.0 126.5 209.7 5.7 860.5 736.6 28 40.06
+287.3 120.5 93.9 187.6 9.2 904.4 695.9 28 43.80
+325.6 166.4 0.0 174.0 8.9 881.6 790.0 28 61.24
+355.9 0.0 141.6 193.3 11.0 801.4 778.4 28 40.87
+132.0 206.5 160.9 178.9 5.5 866.9 735.6 28 33.31
+322.5 148.6 0.0 185.8 8.5 951.0 709.5 28 52.43
+164.2 0.0 200.1 181.2 12.6 849.3 846.0 28 15.09
+313.8 0.0 112.6 169.9 10.1 925.3 782.9 28 38.46
+321.4 0.0 127.9 182.5 11.5 870.1 779.7 28 37.27
+139.7 163.9 127.7 236.7 5.8 868.6 655.6 28 35.23
+288.4 121.0 0.0 177.4 7.0 907.9 829.5 28 42.14
+298.2 0.0 107.0 209.7 11.1 879.6 744.2 28 31.88
+264.5 111.0 86.5 195.5 5.9 832.6 790.4 28 41.54
+159.8 250.0 0.0 168.4 12.2 1049.3 688.2 28 39.46
+166.0 259.7 0.0 183.2 12.7 858.8 826.8 28 37.92
+276.4 116.0 90.3 179.6 8.9 870.1 768.3 28 44.28
+322.2 0.0 115.6 196.0 10.4 817.9 813.4 28 31.18
+148.5 139.4 108.6 192.7 6.1 892.4 780.0 28 23.70
+159.1 186.7 0.0 175.6 11.3 989.6 788.9 28 32.77
+260.9 100.5 78.3 200.6 8.6 864.5 761.5 28 32.40
\ No newline at end of file
diff --git a/examples/uci/figure/counterfactual.png b/examples/uci/figure/counterfactual.png
new file mode 100644
index 0000000..bac49e7
Binary files /dev/null and b/examples/uci/figure/counterfactual.png differ
diff --git a/examples/uci/requirements.txt b/examples/uci/requirements.txt
index 4a67121..5a65422 100644
--- a/examples/uci/requirements.txt
+++ b/examples/uci/requirements.txt
@@ -1,2 +1,4 @@
scikit-learn
-jupyter
\ No newline at end of file
+jupyter
+matplotlib
+tueplots
\ No newline at end of file
diff --git a/examples/uci/tutorial.ipynb b/examples/uci/tutorial.ipynb
index 912c9b9..789fd6d 100644
--- a/examples/uci/tutorial.ipynb
+++ b/examples/uci/tutorial.ipynb
@@ -1,11 +1,20 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "id": "60a4df59-3c1e-47aa-bc08-12b883bc6579",
+ "metadata": {},
+ "source": [
+ "# Kronfluence Tutorial\n",
+ "\n",
+ "Kronfluence is a repository designed to compute influence functions using Kronecker-factored Approximate Curvature (KFAC) or Eigenvalue-corrected KFAC (EKFAC). In this short tutorial, we will introduce some functionalities of Kronfluence on the UCI regression pipeline (it is quick to run and does not require GPUs)."
+ ]
+ },
{
"cell_type": "code",
"execution_count": 1,
"id": "initial_id",
"metadata": {
- "collapsed": true,
"ExecuteTime": {
"end_time": "2024-03-12T10:46:20.005159Z",
"start_time": "2024-03-12T10:46:19.995640Z"
@@ -13,67 +22,57 @@
},
"outputs": [],
"source": [
- "import kronfluence"
+ "import math\n",
+ "import random\n",
+ "from typing import List, Optional, Tuple\n",
+ "\n",
+ "import matplotlib.pyplot as plt\n",
+ "import torch\n",
+ "import torch.nn.functional as F\n",
+ "from random import shuffle\n",
+ "from torch import nn\n",
+ "from torch.utils import data\n",
+ "from tueplots import cycler, markers\n",
+ "from tueplots.constants import markers as marker_constants\n",
+ "from tueplots.constants.color import palettes\n",
+ "from tqdm import tqdm\n",
+ "\n",
+ "from examples.uci.pipeline import construct_regression_mlp, get_regression_dataset\n",
+ "\n",
+ "plt.rcParams.update({\"figure.dpi\": 150})\n",
+ "plt.rcParams.update(markers.with_edge())"
]
},
{
- "cell_type": "code",
- "execution_count": 2,
- "outputs": [],
+ "cell_type": "markdown",
+ "id": "7d13fb5e-1ffb-463d-bcd4-dbf09559abb1",
+ "metadata": {},
"source": [
- "from examples.uci.train import train, evaluate"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T10:47:03.741170Z",
- "start_time": "2024-03-12T10:47:02.235222Z"
- }
- },
- "id": "4e56f0f1d6e34e62"
+ "## Setting up the Model and Dataset"
+ ]
},
{
- "cell_type": "code",
- "execution_count": 43,
- "outputs": [],
+ "cell_type": "markdown",
+ "id": "8c4246ae-3f8f-460f-a987-ad89c1bdc5cd",
+ "metadata": {},
"source": [
- "from kronfluence.analyzer import Analyzer, prepare_model\n",
- "from kronfluence.arguments import ScoreArguments\n",
- "from kronfluence.task import Task\n",
- "from typing import Tuple\n",
- "import torch\n",
- "from torch import nn\n",
- "import math\n",
- "import torch.nn.functional as F"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:02:46.505618Z",
- "start_time": "2024-03-12T21:02:46.503015Z"
- }
- },
- "id": "6dc3ab20b6cb4050"
+ "Before computing influence scores, we need to prepare the trained model and datasets. Let's define the hyperparameters that we will use to train the model."
+ ]
},
{
"cell_type": "code",
- "execution_count": 4,
- "outputs": [],
- "source": [
- "from examples.uci.pipeline import construct_regression_mlp, get_regression_dataset"
- ],
+ "execution_count": 2,
+ "id": "4e56f0f1d6e34e62",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T11:40:46.724609Z",
- "start_time": "2024-03-12T11:40:46.722860Z"
+ "end_time": "2024-03-12T10:47:03.741170Z",
+ "start_time": "2024-03-12T10:47:02.235222Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "f3ed29a0d098c6dd"
- },
- {
- "cell_type": "code",
- "execution_count": 12,
"outputs": [],
"source": [
"dataset_name = \"concrete\"\n",
@@ -83,70 +82,163 @@
"num_train_epochs = 40\n",
"learning_rate = 0.03\n",
"weight_decay = 1e-05"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "aafc7037-a40d-47d0-877b-91e2bc71ec7a",
+ "metadata": {},
+ "source": [
+ "After loading training and query (validation) datasets, we will train the model."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "id": "3ec0486b-4abe-461e-a258-8d24d9c1b10b",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "((927, 8), (927, 1))"
+ ]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T20:55:43.240827Z",
- "start_time": "2024-03-12T20:55:43.238178Z"
+ "source": [
+ "train_dataset = get_regression_dataset(data_name=dataset_name, split=\"train\", dataset_dir=dataset_dir)\n",
+ "query_dataset = get_regression_dataset(data_name=dataset_name, split=\"valid\", dataset_dir=dataset_dir)\n",
+ "train_dataset.data_x.shape, train_dataset.data_y.shape"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "id": "54b2c746-0432-4917-aeb7-17e91e72553b",
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "Sequential(\n",
+ " (0): Linear(in_features=8, out_features=128, bias=True)\n",
+ " (1): ReLU()\n",
+ " (2): Linear(in_features=128, out_features=128, bias=True)\n",
+ " (3): ReLU()\n",
+ " (4): Linear(in_features=128, out_features=128, bias=True)\n",
+ " (5): ReLU()\n",
+ " (6): Linear(in_features=128, out_features=1, bias=True)\n",
+ ")"
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
}
- },
- "id": "cd2af4deeea3afd7"
+ ],
+ "source": [
+ "construct_regression_mlp()"
+ ]
},
{
"cell_type": "code",
- "execution_count": 13,
+ "execution_count": 5,
+ "id": "d8f275b5-7767-4d26-84ff-8a39aa74df08",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def train(\n",
+ " dataset: data.Dataset,\n",
+ " batch_size: int,\n",
+ " num_train_epochs: int,\n",
+ " learning_rate: float,\n",
+ " weight_decay: float,\n",
+ " disable_tqdm: bool = False,\n",
+ ") -> nn.Module:\n",
+ " train_dataloader = data.DataLoader(\n",
+ " dataset=dataset,\n",
+ " batch_size=batch_size,\n",
+ " shuffle=True,\n",
+ " drop_last=True,\n",
+ " )\n",
+ " model = construct_regression_mlp()\n",
+ " optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=weight_decay)\n",
+ "\n",
+ " model.train()\n",
+ " for epoch in range(num_train_epochs):\n",
+ " total_loss = 0.0\n",
+ " with tqdm(train_dataloader, unit=\"batch\", disable=disable_tqdm) as tepoch:\n",
+ " for batch in tepoch:\n",
+ " tepoch.set_description(f\"Epoch {epoch}\")\n",
+ " model.zero_grad()\n",
+ " inputs, targets = batch\n",
+ " outputs = model(inputs)\n",
+ " loss = F.mse_loss(outputs, targets)\n",
+ " total_loss += loss.detach().float()\n",
+ " loss.backward()\n",
+ " optimizer.step()\n",
+ " tepoch.set_postfix(loss=total_loss.item() / len(train_dataloader))\n",
+ " return model"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "id": "6a1337cb-6fe5-4edc-9821-de27bbef9074",
+ "metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "Epoch 0: 100%|██████████| 28/28 [00:00<00:00, 545.57batch/s, loss=0.889]\n",
- "Epoch 1: 100%|██████████| 28/28 [00:00<00:00, 582.99batch/s, loss=0.56]\n",
- "Epoch 2: 100%|██████████| 28/28 [00:00<00:00, 576.00batch/s, loss=0.436]\n",
- "Epoch 3: 100%|██████████| 28/28 [00:00<00:00, 574.70batch/s, loss=0.365]\n",
- "Epoch 4: 100%|██████████| 28/28 [00:00<00:00, 554.65batch/s, loss=0.309]\n",
- "Epoch 5: 100%|██████████| 28/28 [00:00<00:00, 539.66batch/s, loss=0.251]\n",
- "Epoch 6: 100%|██████████| 28/28 [00:00<00:00, 531.06batch/s, loss=0.224]\n",
- "Epoch 7: 100%|██████████| 28/28 [00:00<00:00, 547.77batch/s, loss=0.217]\n",
- "Epoch 8: 100%|██████████| 28/28 [00:00<00:00, 574.51batch/s, loss=0.186]\n",
- "Epoch 9: 100%|██████████| 28/28 [00:00<00:00, 546.78batch/s, loss=0.21]\n",
- "Epoch 10: 100%|██████████| 28/28 [00:00<00:00, 540.93batch/s, loss=0.189]\n",
- "Epoch 11: 100%|██████████| 28/28 [00:00<00:00, 534.31batch/s, loss=0.181]\n",
- "Epoch 12: 100%|██████████| 28/28 [00:00<00:00, 551.12batch/s, loss=0.171]\n",
- "Epoch 13: 100%|██████████| 28/28 [00:00<00:00, 548.24batch/s, loss=0.147]\n",
- "Epoch 14: 100%|██████████| 28/28 [00:00<00:00, 561.72batch/s, loss=0.15]\n",
- "Epoch 15: 100%|██████████| 28/28 [00:00<00:00, 563.34batch/s, loss=0.143]\n",
- "Epoch 16: 100%|██████████| 28/28 [00:00<00:00, 575.86batch/s, loss=0.143]\n",
- "Epoch 17: 100%|██████████| 28/28 [00:00<00:00, 560.37batch/s, loss=0.15]\n",
- "Epoch 18: 100%|██████████| 28/28 [00:00<00:00, 511.37batch/s, loss=0.142]\n",
- "Epoch 19: 100%|██████████| 28/28 [00:00<00:00, 506.99batch/s, loss=0.119]\n",
- "Epoch 20: 100%|██████████| 28/28 [00:00<00:00, 521.90batch/s, loss=0.118]\n",
- "Epoch 21: 100%|██████████| 28/28 [00:00<00:00, 531.20batch/s, loss=0.112]\n",
- "Epoch 22: 100%|██████████| 28/28 [00:00<00:00, 489.00batch/s, loss=0.122]\n",
- "Epoch 23: 100%|██████████| 28/28 [00:00<00:00, 557.58batch/s, loss=0.12]\n",
- "Epoch 24: 100%|██████████| 28/28 [00:00<00:00, 555.09batch/s, loss=0.104]\n",
- "Epoch 25: 100%|██████████| 28/28 [00:00<00:00, 564.86batch/s, loss=0.0998]\n",
- "Epoch 26: 100%|██████████| 28/28 [00:00<00:00, 565.11batch/s, loss=0.126]\n",
- "Epoch 27: 100%|██████████| 28/28 [00:00<00:00, 540.83batch/s, loss=0.105]\n",
- "Epoch 28: 100%|██████████| 28/28 [00:00<00:00, 561.48batch/s, loss=0.0981]\n",
- "Epoch 29: 100%|██████████| 28/28 [00:00<00:00, 524.37batch/s, loss=0.0991]\n",
- "Epoch 30: 100%|██████████| 28/28 [00:00<00:00, 513.95batch/s, loss=0.104]\n",
- "Epoch 31: 100%|██████████| 28/28 [00:00<00:00, 517.93batch/s, loss=0.0894]\n",
- "Epoch 32: 100%|██████████| 28/28 [00:00<00:00, 522.07batch/s, loss=0.115]\n",
- "Epoch 33: 100%|██████████| 28/28 [00:00<00:00, 521.34batch/s, loss=0.0845]\n",
- "Epoch 34: 100%|██████████| 28/28 [00:00<00:00, 520.06batch/s, loss=0.0926]\n",
- "Epoch 35: 100%|██████████| 28/28 [00:00<00:00, 534.94batch/s, loss=0.0833]\n",
- "Epoch 36: 100%|██████████| 28/28 [00:00<00:00, 517.63batch/s, loss=0.091]\n",
- "Epoch 37: 100%|██████████| 28/28 [00:00<00:00, 520.73batch/s, loss=0.0929]\n",
- "Epoch 38: 100%|██████████| 28/28 [00:00<00:00, 544.11batch/s, loss=0.0883]\n",
- "Epoch 39: 100%|██████████| 28/28 [00:00<00:00, 400.69batch/s, loss=0.0901]\n"
+ "Epoch 0: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 433.68batch/s, loss=0.942]\n",
+ "Epoch 1: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 578.97batch/s, loss=0.663]\n",
+ "Epoch 2: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 507.52batch/s, loss=0.436]\n",
+ "Epoch 3: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 590.30batch/s, loss=0.351]\n",
+ "Epoch 4: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 509.56batch/s, loss=0.293]\n",
+ "Epoch 5: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 474.25batch/s, loss=0.25]\n",
+ "Epoch 6: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 537.15batch/s, loss=0.222]\n",
+ "Epoch 7: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 644.86batch/s, loss=0.206]\n",
+ "Epoch 8: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 741.23batch/s, loss=0.185]\n",
+ "Epoch 9: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 669.30batch/s, loss=0.194]\n",
+ "Epoch 10: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 483.44batch/s, loss=0.17]\n",
+ "Epoch 11: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 644.04batch/s, loss=0.177]\n",
+ "Epoch 12: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 578.84batch/s, loss=0.158]\n",
+ "Epoch 13: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 506.81batch/s, loss=0.173]\n",
+ "Epoch 14: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 546.14batch/s, loss=0.145]\n",
+ "Epoch 15: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 130.30batch/s, loss=0.136]\n",
+ "Epoch 16: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 208.50batch/s, loss=0.14]\n",
+ "Epoch 17: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 584.80batch/s, loss=0.141]\n",
+ "Epoch 18: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 620.74batch/s, loss=0.133]\n",
+ "Epoch 19: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 740.27batch/s, loss=0.142]\n",
+ "Epoch 20: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 676.98batch/s, loss=0.115]\n",
+ "Epoch 21: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 502.51batch/s, loss=0.117]\n",
+ "Epoch 22: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 545.23batch/s, loss=0.117]\n",
+ "Epoch 23: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 638.29batch/s, loss=0.109]\n",
+ "Epoch 24: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 626.93batch/s, loss=0.112]\n",
+ "Epoch 25: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 611.66batch/s, loss=0.106]\n",
+ "Epoch 26: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 621.82batch/s, loss=0.103]\n",
+ "Epoch 27: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 633.66batch/s, loss=0.105]\n",
+ "Epoch 28: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 493.41batch/s, loss=0.0992]\n",
+ "Epoch 29: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 634.15batch/s, loss=0.101]\n",
+ "Epoch 30: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 749.29batch/s, loss=0.101]\n",
+ "Epoch 31: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 575.25batch/s, loss=0.0898]\n",
+ "Epoch 32: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 502.94batch/s, loss=0.0888]\n",
+ "Epoch 33: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 605.97batch/s, loss=0.0852]\n",
+ "Epoch 34: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 131.50batch/s, loss=0.0891]\n",
+ "Epoch 35: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 688.25batch/s, loss=0.0881]\n",
+ "Epoch 36: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 676.97batch/s, loss=0.0883]\n",
+ "Epoch 37: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 713.74batch/s, loss=0.0779]\n",
+ "Epoch 38: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 528.26batch/s, loss=0.0904]\n",
+ "Epoch 39: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 605.93batch/s, loss=0.0747]\n"
]
}
],
"source": [
- "train_dataset = get_regression_dataset(data_name=dataset_name, split=\"train\", dataset_dir=dataset_dir)\n",
- "\n",
"model = train(\n",
" dataset=train_dataset,\n",
" batch_size=train_batch_size,\n",
@@ -154,54 +246,142 @@
" learning_rate=learning_rate,\n",
" weight_decay=weight_decay,\n",
")"
- ],
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "23d2dc0b-bcfa-4c4b-ab5c-4d96d1d22e03",
+ "metadata": {},
+ "source": [
+ "We can compute the loss on the query dataset after training the model."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "id": "6dc3ab20b6cb4050",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T20:55:46.006660Z",
- "start_time": "2024-03-12T20:55:43.876876Z"
+ "end_time": "2024-03-12T21:02:46.505618Z",
+ "start_time": "2024-03-12T21:02:46.503015Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "c75658f17d06a7ab"
- },
- {
- "cell_type": "code",
- "execution_count": 14,
"outputs": [
{
"data": {
- "text/plain": "0.16043664876697133"
+ "text/plain": [
+ "0.14518585019898647"
+ ]
},
- "execution_count": 14,
+ "execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
- "query_dataset = get_regression_dataset(data_name=dataset_name, split=\"valid\", dataset_dir=dataset_dir)\n",
+ "def evaluate(model: nn.Module, dataset: data.Dataset, batch_size: int) -> float:\n",
+ " dataloader = data.DataLoader(\n",
+ " dataset=dataset,\n",
+ " batch_size=batch_size,\n",
+ " shuffle=False,\n",
+ " drop_last=False,\n",
+ " )\n",
+ "\n",
+ " model.eval()\n",
+ " total_loss = 0.0\n",
+ " for batch in dataloader:\n",
+ " with torch.no_grad():\n",
+ " inputs, targets = batch\n",
+ " outputs = model(inputs)\n",
+ " loss = F.mse_loss(outputs, targets, reduction=\"sum\")\n",
+ " total_loss += loss.detach().float()\n",
+ "\n",
+ " return total_loss.item() / len(dataloader.dataset)\n",
"\n",
"evaluate(model=model, dataset=query_dataset, batch_size=eval_batch_size)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "368fb7a4-8615-4452-8772-4c4d4fcb2944",
+ "metadata": {},
+ "source": [
+ "## Defining a Task"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6f86702a-6e8f-425a-b47b-de3252b88599",
+ "metadata": {},
+ "source": [
+ "Before computing influence scores, we need to define a `Task` class. This class encapsulates information about the trained model and how influence scores will be computed: (1) how to compute the training loss; (2) how to compute the measurement; (3) which modules to use for influence function computations; and (4) whether the model used attention mask."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "id": "03dad2b0-dd52-47ec-92d1-503914f78951",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from kronfluence.analyzer import Analyzer, prepare_model\n",
+ "from kronfluence.task import Task"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7a158910-7f1d-4d47-b59f-8df708d2f358",
+ "metadata": {},
+ "source": [
+ "We can optionally use `Analyzer.get_module_summary` to easily get the name of available modules."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "id": "97fdf9f1-5296-4e08-916a-1f593990e620",
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "==Model Summary==\n",
+ "Module Name: `0`, Module: Linear(in_features=8, out_features=128, bias=True)\n",
+ "Module Name: `2`, Module: Linear(in_features=128, out_features=128, bias=True)\n",
+ "Module Name: `4`, Module: Linear(in_features=128, out_features=128, bias=True)\n",
+ "Module Name: `6`, Module: Linear(in_features=128, out_features=1, bias=True)\n"
+ ]
+ }
],
+ "source": [
+ "print(Analyzer.get_module_summary(model))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "id": "41238e1b9bcec5cc",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T20:55:47.948299Z",
- "start_time": "2024-03-12T20:55:47.939642Z"
+ "end_time": "2024-03-12T20:58:22.611435Z",
+ "start_time": "2024-03-12T20:58:22.608172Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "eed8dcc003228fc4"
- },
- {
- "cell_type": "code",
- "execution_count": 23,
"outputs": [],
"source": [
- "BATCH_DTYPE = Tuple[torch.Tensor, torch.Tensor]\n",
- "\n",
"class RegressionTask(Task):\n",
" def compute_train_loss(\n",
" self,\n",
- " batch: BATCH_DTYPE,\n",
+ " batch: Tuple[torch.Tensor, torch.Tensor],\n",
" model: nn.Module,\n",
" sample: bool = False,\n",
" ) -> torch.Tensor:\n",
@@ -209,1283 +389,987 @@
" outputs = model(inputs)\n",
" if not sample:\n",
" return F.mse_loss(outputs, targets, reduction=\"sum\")\n",
+ " # Sample the outputs from the model's prediction for true Fisher.\n",
" with torch.no_grad():\n",
" sampled_targets = torch.normal(outputs, std=math.sqrt(0.5))\n",
" return F.mse_loss(outputs, sampled_targets.detach(), reduction=\"sum\")\n",
"\n",
" def compute_measurement(\n",
" self,\n",
- " batch: BATCH_DTYPE,\n",
+ " batch: Tuple[torch.Tensor, torch.Tensor],\n",
" model: nn.Module,\n",
" ) -> torch.Tensor:\n",
" # The measurement function is set as a training loss.\n",
- " return self.compute_train_loss(batch, model, sample=False)"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T20:58:22.611435Z",
- "start_time": "2024-03-12T20:58:22.608172Z"
- }
- },
- "id": "41238e1b9bcec5cc"
+ " return self.compute_train_loss(batch, model, sample=False)\n",
+ "\n",
+ " def tracked_modules(self) -> Optional[List[str]]:\n",
+ " # These are the module names we will use to compute influence functions.\n",
+ " return [\"0\", \"2\", \"4\", \"6\"]"
+ ]
},
{
- "cell_type": "code",
- "execution_count": 24,
- "outputs": [],
+ "cell_type": "markdown",
+ "id": "03d4f582-74b8-41a3-8e09-ebfbe72be849",
+ "metadata": {},
"source": [
- "task = RegressionTask()\n",
- "model = prepare_model(model, task)\n",
- "analyzer = Analyzer(\n",
- " analysis_name=\"tutorial\",\n",
- " model=model,\n",
- " task=task,\n",
- " cpu=True,\n",
- ")"
- ],
+ "Kronfluence wraps all supported modules within the model with `TrackedModule`. This wrapper will be used for computing the factors and influence scores. Once your model is ready and the task is defined, prepare your model with:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "id": "1192b28fd4535410",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
"end_time": "2024-03-12T20:58:24.595047Z",
"start_time": "2024-03-12T20:58:24.588848Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "1192b28fd4535410"
+ "outputs": [],
+ "source": [
+ "task = RegressionTask()\n",
+ "model = prepare_model(model, task)"
+ ]
},
{
- "cell_type": "code",
- "execution_count": 27,
- "outputs": [
- {
- "data": {
- "text/plain": "[('',\n Sequential(\n (0): TrackedLinear(\n (original_module): Linear(in_features=8, out_features=128, bias=True)\n )\n (1): ReLU()\n (2): TrackedLinear(\n (original_module): Linear(in_features=128, out_features=128, bias=True)\n )\n (3): ReLU()\n (4): TrackedLinear(\n (original_module): Linear(in_features=128, out_features=128, bias=True)\n )\n (5): ReLU()\n (6): TrackedLinear(\n (original_module): Linear(in_features=128, out_features=1, bias=True)\n )\n )),\n ('0',\n TrackedLinear(\n (original_module): Linear(in_features=8, out_features=128, bias=True)\n )),\n ('0.original_module', Linear(in_features=8, out_features=128, bias=True)),\n ('1', ReLU()),\n ('2',\n TrackedLinear(\n (original_module): Linear(in_features=128, out_features=128, bias=True)\n )),\n ('2.original_module', Linear(in_features=128, out_features=128, bias=True)),\n ('3', ReLU()),\n ('4',\n TrackedLinear(\n (original_module): Linear(in_features=128, out_features=128, bias=True)\n )),\n ('4.original_module', Linear(in_features=128, out_features=128, bias=True)),\n ('5', ReLU()),\n ('6',\n TrackedLinear(\n (original_module): Linear(in_features=128, out_features=1, bias=True)\n )),\n ('6.original_module', Linear(in_features=128, out_features=1, bias=True))]"
- },
- "execution_count": 27,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
+ "cell_type": "markdown",
+ "id": "fe331fda-fd5f-4555-b952-2cf62aca05ab",
+ "metadata": {},
"source": [
- "list(model.named_modules())"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T20:59:23.401098Z",
- "start_time": "2024-03-12T20:59:23.396531Z"
- }
- },
- "id": "4a38704b80ff26b2"
+ "You can see that the `TrackedModule` are now installed."
+ ]
},
{
"cell_type": "code",
- "execution_count": 25,
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "Fitting covariance matrices [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n",
- "Fitting covariance matrices [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n"
- ]
- }
- ],
- "source": [
- "covariance_matrices = analyzer.fit_covariance_matrices(\n",
- " factors_name=\"ekfac\",\n",
- " dataset=train_dataset,\n",
- " per_device_batch_size=None,\n",
- " overwrite_output_dir=True,\n",
- ")"
- ],
+ "execution_count": 12,
+ "id": "4a38704b80ff26b2",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T20:59:08.113601Z",
- "start_time": "2024-03-12T20:59:08.079505Z"
+ "end_time": "2024-03-12T20:59:23.401098Z",
+ "start_time": "2024-03-12T20:59:23.396531Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "87964b32dd38d4ff"
- },
- {
- "cell_type": "code",
- "execution_count": 33,
"outputs": [
{
"data": {
- "text/plain": "torch.Size([129, 129])"
+ "text/plain": [
+ "Sequential(\n",
+ " (0): TrackedLinear(\n",
+ " (original_module): Linear(in_features=8, out_features=128, bias=True)\n",
+ " )\n",
+ " (1): ReLU()\n",
+ " (2): TrackedLinear(\n",
+ " (original_module): Linear(in_features=128, out_features=128, bias=True)\n",
+ " )\n",
+ " (3): ReLU()\n",
+ " (4): TrackedLinear(\n",
+ " (original_module): Linear(in_features=128, out_features=128, bias=True)\n",
+ " )\n",
+ " (5): ReLU()\n",
+ " (6): TrackedLinear(\n",
+ " (original_module): Linear(in_features=128, out_features=1, bias=True)\n",
+ " )\n",
+ ")"
+ ]
},
- "execution_count": 33,
+ "execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
- "covariance_matrices[\"activation_covariance\"][\"2\"].shape"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T20:59:59.570656Z",
- "start_time": "2024-03-12T20:59:59.567837Z"
- }
- },
- "id": "481af2c88737df72"
+ "model"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "190902fc-fe59-42c7-a94f-07c577dedbf7",
+ "metadata": {},
+ "source": [
+ "We can now create the `Analyzer` instance to compute influence scores. The `analysis_name` is used to organize the results."
+ ]
},
{
"cell_type": "code",
- "execution_count": 34,
+ "execution_count": 13,
+ "id": "19d3800e-a518-4bfe-a0b1-02ebf16b8f3e",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "analyzer = Analyzer(\n",
+ " analysis_name=\"tutorial\",\n",
+ " model=model,\n",
+ " task=task,\n",
+ " cpu=True,\n",
+ ")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "d5c4fb98-4dd2-4d49-b856-8a989d5a530e",
+ "metadata": {},
+ "source": [
+ "## Computing Influence Factors"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b05772c8-28f7-4471-acfe-762e55344790",
+ "metadata": {},
+ "source": [
+ "We can compute the activation and pseudo-activation covariance matrices with `fit_covariance_matrices`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "id": "87964b32dd38d4ff",
+ "metadata": {
+ "ExecuteTime": {
+ "end_time": "2024-03-12T20:59:08.113601Z",
+ "start_time": "2024-03-12T20:59:08.079505Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
+ }
+ },
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "Performing Eigendecomposition [4/4] 100%|██████████ [time left: 00:00, time spent: 00:00]\n"
+ "Fitting covariance matrices [1/1] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n",
+ "Fitting covariance matrices [1/1] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n"
]
}
],
"source": [
- "eigen_factors = analyzer.perform_eigendecomposition(\n",
- " factors_name=\"ekfac\",\n",
+ "analyzer.fit_covariance_matrices(\n",
+ " factors_name=\"tutorial_factor\",\n",
+ " dataset=train_dataset,\n",
+ " per_device_batch_size=None,\n",
" overwrite_output_dir=True,\n",
")"
- ],
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "319928b0-3e4a-4a60-8b5e-dd70f5230f82",
+ "metadata": {},
+ "source": [
+ "You can load the computed covariance matrix with `load_covariance_matrices`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "id": "481af2c88737df72",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T21:00:31.570596Z",
- "start_time": "2024-03-12T21:00:31.553063Z"
+ "end_time": "2024-03-12T20:59:59.570656Z",
+ "start_time": "2024-03-12T20:59:59.567837Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "31da64ca7f237819"
- },
- {
- "cell_type": "code",
- "execution_count": 36,
"outputs": [
{
"data": {
- "text/plain": "torch.Size([129, 129])"
+ "text/plain": [
+ ""
+ ]
},
- "execution_count": 36,
+ "execution_count": 15,
"metadata": {},
"output_type": "execute_result"
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoAAAAJYCAYAAADyoFdhAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAABcSAAAXEgFnn9JSAAEAAElEQVR4nOz9eXQd1Zn1j++qupplWbIleZJHecA22GBodxgcOwQSwkwCdKb1BSchoZPulyEkv7U6pNNJOuvtjhlCh3RDuhOSXiQkL5AQhtANJIQQZjMYPOBZ8mxLtmRrlm7V+f0h6579PFcS873Yej5rea0rnbpVp06dKpXr2bV34JxzMAzDMAzDMEYMYb47YBiGYRiGYeQWuwE0DMMwDMMYYdgNoGEYhmEYxgjDbgANwzAMwzBGGHYDaBiGYRiGMcKwG0DDMAzDMIwRht0AGoZhGIZhjDDsBtAwDMMwDGOEYTeAhmEYhmEYIwy7ATQMwzAMwxhh2A2gYRiGYRjGCMNuAA3DMAzDMEYYdgNoGIZhGIYxwkjluwOGYRiGYYwszj//fGzevDmn26yvr8f999+f022+n7EbQMMwDMMwcsrmzZuxadM6zJxWkJPtbWroy8l2jiTsBtAwDMMwjJwzc1oBXntiak62ddzSxpxs50jCbgANwzAMw8gDDgmSnG3LkNhLIIZhGIZhGCMMewJoGIZhGEbOcQBil5sngA5AkJMtHTnYE0DDMAzDMIwRht0AGoZhGIZhjDCsBGwYhmEYRl5I7OWMvGFPAA3DMAzDMEYY9gTQMAzDMIyc44Cc2cDYSyDZ2BNAwzAMwzCMEYY9ATQMwzAMIw84xC5XGkDTGmrsCaBhGIZhGMYIw54AGoZhGIaRF+wt4PxhTwANwzAMwzBGGPYE0DAMwzCMnOMAxDl6AmjPGbOxJ4CGYRiGYRgjDHsCaBiGYRhGXjANYP6wJ4CGYRiGYRgjDLsBNAzDMAzDGGFYCdgwDMMwjJzjgJwZQVuhORt7Aqjo6urCP/7jP2L27NkoLi7GxIkT8bnPfQ47d+7Md9feMzo7O3Hffffh85//PObMmYPi4mKUlZVh4cKF+M53voP29vYhv/uzn/0MixcvRnl5OcaMGYOzzz4bTz/9dA57nxv279+P2tpaBEGAmTNnDrvsSBiTpqYmXHfddZgzZw5KSkowZswYLFq0CF/72tcGXf6BBx7A0qVLUVFRgYqKCixbtgwPPfRQjnv93vHCCy/g0ksvxcSJE1FQUIDKykosWbIEd9xxB9wgf+DiOMbNN9+M4447DiUlJaipqcGll16KdevW5aH3b58XX3wR//Iv/4KPf/zjqKurQxAECII3Tlx9O+fIU089hbPPPhtjxoxBeXk5Fi9ejP/+7/9+t3blXeOtjEmSJHjyySfx9a9/HSeeeCJGjRqFoqIi1NfX48orr8TWrVuH3daRMibG+xRnZOjq6nIf+MAHHAA3YcIEd+mll7rFixc7AK6mpsZt3rw53118T/jP//xPh/7/ILm5c+e6Sy65xH30ox91o0aNcgDcMccc4/bu3Zv1vauuusoBcCUlJe6CCy5wH/3oR10qlXJRFLnf/va3ud+R95DLLrvMBUHgALj6+vohlxsJY7Jy5Uo3duxYB8DNnz/f/c3f/I372Mc+5qZOneqiKMpa/uabb3YAXCqVcmeddZa74IILXElJiQPgfvjDH+ZhD95d7rnnHhdFkQPgFi1a5C699FL3oQ99yKVSKQfAffrTnxbLx3HsLrroIgfAVVZWuk984hNu6dKlLggCV1pa6p577rk87clb54ILLshcO/jfcLydc2RgjIMgcEuXLnWf+MQnXGVlpQPgvvrVr74He/b2eStjsnHjxkz7+PHj3fnnn+8uuugiN2nSJAfAjRo1yj355JODfvdIGpPBmDdvnps9O+V27piQk3+zZ6fcvHnz8r3b7yvsBpD4xje+4QC4k08+2bW1tWV+f+ONNzoAbunSpfnr3HvIz372M/fFL37RrV27Vvx+165d7oQTTnAA3Kc+9SnR9uijjzoAbuzYsW7Dhg2Z3z/99NOusLDQVVZWupaWllx0/z3nsccecwDcF7/4xWFvAEfCmOzbt89VV1e70tJS97vf/S6rXd+8vP766y6KIldUVOSefvrpzO/Xr1/vxo4d61KplNu4ceN73u/3ir6+PldbW+sAuF/84heibe3atW7MmDEOgPvjH/+Y+f3Af7hmzZrl9uzZk/n9Pffc4wC4mTNnur6+vpztwzvhX/7lX9w3v/lNd//997vdu3e7oqKiYW8A3845sn//fldRUeEAuHvvvTfz+z179riZM2c6AO7xxx9/t3ftbfNWxmTTpk3uzDPPdH/4wx9ckiSZ33d3d7vLL7/cAXBTpkxxvb294ntH2pgMht0A5h+7ATxMT0+PGz16tAPgXnrppaz2BQsWOABu5cqVeehd/nj66acdAFdUVOR6enoyv//Yxz7mALibb7456zv/5//8HwfA3XDDDTns6XtDZ2enq6+vd/PmzXMbNmwY9gZwJIzJ3/7t3zoA7kc/+tFbWv6qq67KarvpppscAPd3f/d373Ivc8drr73mALg5c+YM2j5w3P/1X/8187u5c+c6AIM+7Tr//PMdAHfPPfe8V11+T3mjG8C3c47867/+qwPgLrjggqzv/OY3v3EA3LnnnvtOu/6e8UZjMhSdnZ2Zv0l/+tOfRNuRPibO9d8Azpqdctt2jM/Jv1l2A5iFaQAP89RTT+HgwYOor6/HCSeckNV+8cUXA+jXMo0kFi5cCADo6enB/v37AfTrJP/4xz8C8OPCHE1j9e1vfxtbtmzBbbfdhoKCgiGXGwlj0tXVhTvvvBNlZWVYvnz5m/rOgM7vaB2ToqKiN7Xc2LFjAQBbt27FunXrUFJSgnPOOSdruaNhTIbi7Z4jw82hc845B8XFxXjsscfQ3d39bnc5r5SUlGD27NkAgF27dom2kTomxruL3QAeZtWqVQCARYsWDdo+8PtXX301Z316P7BlyxYAQEFBAcaMGQMAWL9+PXp6elBTU4O6urqs7xwtY/Xqq6/ixhtvxPLly7FkyZJhlx0JY7Jy5Uq0tbXhhBNOQElJCR5++GFce+21+PKXv4wf/OAHWX+kWltbsW3bNgAY9D9VkydPRnV1NRobG3Ho0KGc7MO7zYwZM1BfX4/169fjl7/8pWhbt24d7rzzTlRVVeGiiy4C4K8zxx577KD/oTga5slQvN1zZLhrc2FhIY499lh0d3djw4YN70Gv80eSJGhsbAQAjB8/XrQdTWMSu9z8M7KxG8DDDPyhGuzCxL8fOCFHCrfccgsA4Kyzzso87XijsSorK0NlZSVaWlrQ1taWm46+yyRJgi984QuorKzE97///TdcfiSMydq1awEAtbW1uPDCC3H22Wfj5ptvxn/8x3/gmmuuwcyZM3HXXXdllh8Yk6qqKpSVlQ26ziP9vIqiCD//+c9RWVmJz3zmMzjxxBPxyU9+EqeffjoWLFiAuro6/OEPf8j852kkX2fezjly6NAhHDx4cNjvHa1jdtddd2Hfvn2oqanBKaeckvn9SB4T493FbgAPM2B1UlpaOmj7wB+wI/WP99vh97//PX7yk5+goKAA3/3udzO/f6OxAo788frhD3+IF154AStWrMiU74ZjJIxJS0sLAOD+++/H//zP/+BHP/oR9u3bh4aGBlx33XXo6urCZZddhldeeQXAyBgTADj11FPxxBNPYMaMGXjppZfw61//Go8//jjCMMSZZ56JGTNmZJYdydeZtzMf2IJqJI3Z9u3bcfXVVwMAvvOd7wipwdE2JkmO/hnZ2A2gMSivv/46PvvZz8I5hxUrVmS0gCOBbdu24frrr8fSpUtx+eWX57s77xuSpP8ymk6n8Z3vfAdf/vKXUVNTg6lTp2LFihW45JJL0NfXhxUrVuS5p7nlrrvuwuLFizF58mQ899xzaG9vx4YNG3D55ZfjxhtvxOmnn46enp58d9M4Qujo6MDHP/5xNDc348ILL8SVV16Z7y4ZRyl2A3iY8vJyAP2myIPR0dEBABg1alTO+pQvdu7cibPOOgstLS249tprcdVVV4n2Nxor4Mger6985Svo7e3Fbbfd9qa/c7SPCeD3EcCgL4EM/O6JJ54Qyx/NY7Jx40ZcdtllqK6uxoMPPojFixejrKwMs2bNwu23345zzz0XL730En76058CGNnXmbczH3jOjYQx6+vrwyWXXIKVK1fitNNOy9KVAiNvTIz3DrsBPMyUKVMAADt27Bi0feD3U6dOzVmf8sGBAwfwkY98BI2NjVi+fDluuOGGrGXeaKw6OjrQ2tqKqqqqI/IC9OCDD6K0tBRXXnklli1blvn3yU9+EkD/DfLA7/bs2QPg6B8TwM/90tJS1NTUZLVPmzYNALBv3z4AfkxaWloyf5A0R/p59atf/Qp9fX0466yzxB/mAS699FIAwJ///GcAI/s683bOkYqKCowePXrY7x0tY5YkCS677DI8/PDDOP744/HAAw+gpKQka7mjaUwcgBhBTv7ZeyDZ2A3gYQZKnC+99NKg7QO/X7BgQc76lGva29vxsY99DGvXrsXHP/5x/Od//uegEUZz5sxBUVERmpqaBo3IOxrGqrW1FU888YT499xzzwEAuru7M78bsFkYCWMy8CZvV1fXoCXNAwcOAPBPKCorKzN/9F9++eWs5bdv347m5mZMnToVFRUV71W331MG/tAO/EHWDPx+QD85cJ1ZvXo1+vr6spY/GubJULzdc2S4a3NfXx9Wr16N4uLijGXKkcrf//3f46677sLs2bPxv//7v6isrBxy2ZEyJsZ7i90AHubUU0/F6NGjsXnz5oyInbnnnnsAAOedd16Oe5Ybenp6cMEFF+D555/HRz/6Udx1112IomjQZUtKSnD66acDAO6+++6s9iN9rFy/QXrWv4Fczvr6+szvBp56He1jAvQ/wVm4cCGcc5kyLzPwO7Z8GfC6G9h/5mgYkwF7jpUrVw7a/sILLwDwT0enT5+OuXPnoqura9As5KNhTIbi7Z4jw82hBx98EN3d3TjjjDNQXFz8bnc5Z1x//fX493//d0yZMgWPPvooamtrh13+qBkTByQ5+mePAAchXw7U70cGouBOOeUU197envn90R4Fl06nM9mkS5YscR0dHW/4neEinYqKio6K2DPN1q1b33YU3NEyJr/4xS8cAHfccce5Xbt2ZX7/8ssvZ2LP/t//+3+Z33MU3DPPPJP5/YYNG46KKLgXX3xx4E+L+/d//3fR9swzz7iysjIHwD366KOZ33MUHGds33vvvUdcFJzmnUTBDXWODBV7tnfv3iMi9uyNxmQgEWf8+PFiTIbjSB8T5/qTQGbOSrnXt03Iyb+ZsywJRBM45+y++DDd3d1YtmwZnnvuOUyYMAFLlixBY2MjnnvuOdTU1ODZZ58Vlg5HC7fcckvGcuCiiy4ashx3ww03oLq6OvPz1VdfjVtuuQWlpaU488wz0dvbi0cffRTOOdxzzz248MILc9D73NHQ0IDp06ejvr4emzZtGnSZkTAml19+ecb77pRTTkFXVxeefvpp9PT04IorrsCPf/xjsfzNN9+Ma6+9FqlUCmeeeSYKCwvxyCOPoKurC//2b/+Gv//7v8/Tnrw7fO1rX8toZefPn4958+Zh165deOaZZ5AkCb74xS/i9ttvzyyfJAkuvvhi/Pa3v0VVVRU+/OEPo7m5GU888QSKi4vx+OOP46//+q/ztTtviYceekhYRD3//PNwzon+f/Ob3xSpJ2/nHLn33ntx6aWXwjmHZcuWYezYsXjsscfQ2tqKa6+9FjfeeON7up9vhbcyJq+88goWLVoE5xxOPvnkIUu2X/jCF3DaaaeJ3x1JYzIY8+fPR0/fBtz/h+Gfdr5bnP/hfSgqmI01a9bkZHtHBPm683y/0tnZ6b75zW+6+vp6V1hY6MaPH+8uv/xyt3379nx37T3jW9/6VuYpxnD/tm7dmvXdO+64w5144omutLTUVVZWurPOOss99dRTud+JHPBGTwAHONrHJEkS9+Mf/zizj2VlZe7kk092P/vZz4b8zv333++WLFniysvLXXl5uVuyZIl74IEHctjr95bf/OY37iMf+UjmqWZVVZX70Ic+5H75y18Ounw6nXY33nijmz9/visuLnZjx451F198sVuzZk2Oe/7OuOOOO97wunHHHXcM+r23eo785S9/cWeddZarrKx0paWl7qSTThp2zuWLtzImjz/++Ju69g42hs4dOWMyGPPmzXP1s1JuzbaJOflXb08As7AngIZhGIZh5JSBJ4C//cO4nGzvog/vtSeACnsJxDAMwzAMY4SRyncHDMMwDMMYeTgAicu2GnuvtmVI7AmgYRiGYRjGCMOeABqGYRiGkRdi5OYJoJGNPQE0DMMwDMMYYdgNoGEYhmEYxgjDSsCGYRiGYeQchwBxjp5DOSs1Z2FPAA3DMAzDMEYY9gTQMAzDMIy8kCsbGCMbewJoGIZhGIYxwrAngIZhGIZh5ByH3NnAmBF0NvYEcAjmz5+P+fPn57sb7ytsTLKxMRkcG5dsbEyysTHJxsbEyBVH5A1gV1cX/vEf/xGzZ89GcXExJk6ciM997nPYuXNnvrtmGIZhGMabJHZhTv4Z2Rxxo9Ld3Y3TTz8d3/3ud9He3o4LLrgAkydPxh133IETTjgBW7ZsyXcXDcMwDMMw3tcccTeA//zP/4xnn30WJ598MjZs2IBf//rXeO6553DjjTeiqakJn/vc5/LdRcMwDMMw3pAACcKc/IP5AGZxRN0A9vb24tZbbwUA/OhHP0J5eXmm7dprr8WCBQvwxBNP4MUXX8xXFw3DMAzDMN73HFE3gE899RQOHjyI+vp6nHDCCVntF198MQDggQceyHXXDMMwDMN4Cwy8BZyLf/YWcDZHlA3MqlWrAACLFi0atH3g96+++urb3sb48ePR0dGBvr4+ALC3sYjNmzcDsDFhbEwGx8YlGxuTbGxMssnXmGzbtg1lZWXYs2dPTrdr5I8j6gZw27ZtAIC6urpB2wd+39jY+La30dHRgY6uLhSOru5f19YmuUBvX+ZjEEZDrse5RP4iTrhRtoX+QWwQDb3OQbbiP/H3qI9Z2wuUDoJ/juQD4YC+5oIAKYwCADRu2gf00Tikhp5GSYHcn7Av7dcZqzEqLPDrpLakSK2jh9aRjkWbKy/26+iVbaBt63HgH+PSQgxFkMhjl0qXAgAa1+/OPq680kIaI7UO3lenjgF6ev1nPTcSGj8aOwByX3Ubf4+XU7iyIt/H9m7ZGNK+Fcj1B3GMAhweF33+8L7rbdMYBbEcIxf7ZYNIzrek0I9L2CvXKeaHOB5qTHh/EjUv+ZxXx8Dx/qRUW4E/lmF3GgUoAwBs27AHLqZ+qXkT0Hi6SJ2vdK1Adw99SZ27tD8uVHOK91U/FtFzWKyUvqevMURC8wYAoi6/rKPrQdAXizFJioa+jgR9NF76HOny45BUlMhtd9K203q+0RzQ6+ym806Pn7im0TirYxWkaR7p6w0df318UsHh6+yWfbIfUNe3tJynAc31rEsRHfOkUI5zurS/ra+rC8bI4oi6AWxvbwcAlJaWDtpeVtZ/MWlra3vDdQ31v6u+vj4Ujq7GrP/v/wcAGP9cu2iP1m/PfA4qK+SX6axzXfIPZnKg1bf1yZM6GuXXE1SPGbrT+qymEz6uHe3XsWGb/Fqv315QKG9ugiL6uWq0bOMLivqDmTT4cYgmTRiynz1Tx4qmIrohcAcPya9Nm5T5HB7qzHzunF0jlitds9v3Y/8B0db5oWMzn8s2tch+7d7nPxeocaA/AAdPm4ahSHXKi27pCw3+B3Vc+YYtPXuyX8dBeaENWvw4uCo5p+LXN/vVqeMj5tjMKXKd22iM6uV/mMJDfvtJww7ZZ/pD0bNkQeZz4f+8IBcrLfM/TJ8s25r9uKenjRNt0SHq8w75pMFNp+PfJsco2evnTVgj51TX7NrM55J1cp3xPvpeif/jmUyXYxJX+PmQalU3u5v8+RSq89N10rI1VaKtc6o/XmWvbBdtCc19Pj8BIJri+5aMkjc0cbm/uYpe2Zj5HNC+9f/sv+fK5TrSlf76yTdnABD00M/qpiUpov+gbZLXmICWbTtjrmireM7ve99Ufy4XbN0rlus6zu93kJbXuqLtNKdqRsltP7s687njoyeJttFP+YcBcVOzaAunT/VtY8pEW/Di63650XJ7bpKfby7lrxvpCnnjW7ivI/M5KZZ/aqNmOv7l8u8Zr9Ot2Sjauj/opU8lu+TfpnCvvxa6PnVc6W9mz8xa0bbrtP65s/U//hVT1LmVC8yiJX8cUTeAuSLVlWTd+A2QtPuTWv9BTvb5C0xYoS5S+okDt5XSxVrdFPETk6BcXqT4D0dc7v+AFarl4gPqxoS3zf1s7xRt/PTEdcs/ivzHNKvP9MegcJ38Qy6efOgnYZ30P3n6n25JY6tcBz350H88yzbTsuoJI39PP30KRvkXisq2yXGIWv3PrkQ9Hayu9J/37ZfrpItuwd6Dfh0F6glWi+9zqP6QpyaO98vpG2Yay3BXk2rz+x62yWMX0HEO9JMPevpUeMB/LyxWNxi0b0Gn+s9Oh19/aqe8QeenZEksn9AGW72PpwvlzYfroWOnnvry00JH5ycAgJ+00JhEe+SxCnfRXKkoF22O/+OlnwZV+vMn6JBzvWQ7/SHXT1foHHFqHFyJv5Hg/wgBQNjqr0sJ90X95zA54G+YglJ57MJuepq6W94U8X8QXafadq2/OUjUecdbr3hZ3YQ3+7Eu4Lmv+ly0j84zVTlwZX4fCna3yvXTf7T0uSu2MUx1hf9jDwAJP0FV/5kPD9ADBlp/eFBdG1r9+ZrMkv/hiLgve+S5y1UGp57sluz28zs8KOc6X6OTmfI/ZdHe1sznwr3yb1vpnv75Fg5dDDCOUo6oG8CBt3471YVpgI6O/hNi1KhRg7Yza9asGfT38+fPzy5bGYZhGIbxruIAJBYFlzeOqGevU6b0l7l27NgxaPvA76dOnTpou2EYhmEYhnGEPQFcuHAhAOCll14atH3g9wsWLBi03TAMwzCM9wsB4pw9hzIjaM0RdQN46qmnYvTo0di8eTNeeeUVHH/88aL9nnvuAQCcd95572xDvX0ZTUiiNEVhmdfrpRukbiRVW535LPRmAJIO0m6ol1hcp9cHaU2b0K20SU0b9y21kgTHSlMUFpE4Welgkj3+xQitRRLrUNq0hHUx+nu0jbBSvbhAb7XqMQrpZYiA3qLm8QHkGDmlI8QuEpartqR7aE1beo//XtQjj0FAmjDHL5IASEjbpd+GjukFlWisf3lAzykev3jnbtEUkp5Tjxcfr+Rg25Bt4U5l60DHJ1Fvcwb0pmS00T9p128qJof8sQp79FwniYbWt9HPWW+88xxTuiuxjkNyXwufJ21kkdRh8fwISBjPuksAcs6WyfOTzyfXJvVTrG/jYwwAAWtVleY0GULGAgDxmvW+L/qFN9IO8jUlC1ouapXa0WA77UOZ0gvT9QDayYDmrZ7roi9KoymuHTQmPHYAELKGTetdSQunNbTcz0i9AJduPYghoXMm0POmhTSU2jlBvOxH16Z2pRunuVfQIF94SWgOO3X+8Nhqja5bsynzOetqTbrFcOsu0RQf8n0LmuQ6x7X1z8WN7UO/2W0cnRxRJeDCwkL83d/9HQDgK1/5SkbzBwA33XQTXn31VSxduhQnnnhivrpoGIZhGMabwKH/LeBc/DMNYDZH1BNAALj++uvx2GOP4emnn8asWbOwZMkSNDY24rnnnkNNTQ1++tOf5ruLhmEYhmEY72uOuBvA4uJiPP744/i///f/4pe//CXuu+8+jBkzBpdffjm++93vDmkS/VYIwijj8aetXkTZ96R5oi1+aZ1fR5H0hOKfdTmILUjCUdKCIr2bSpN10+X3pnj/PbaLcGXS98tto3KAKjGzf5gu67C9izZtZrsIXSKJhykVsZVNrMvdXHIa58vpSMkH1QGXYF5bL9u4tKvK1hGVjmKyyQCAVJ33oNPjh4ND+0pG1d4aQ3tvpcZ7v62YyrC6nB5zOVWbatPPukQfjSb/SGX9k7Dn2Sz1UlQHlcK1tySZIHPpO9kly43R5Im0A6pfvH+6vFVAx195OPJR1lIINvENxlTKdXJ5Wpfv2QCX+qLLtQmXClVZPJrgvQxZMgEAqcn+ehOrNlGS03ORjl2sypSpqd7CIx4jfSGjFiod8phoo+Fiuv6oYxDSvNTl1JDOQT2neGzTu6WsQI8nI653BbT++bNkn+kalipR1zC25tFl0e0km1A+dinKi9c+gGz3k96swgPI5F8bSLPFjpinStLA1/L0ZOllGr3u9zWsln3m8nCiZB8RjwNJjgBpx5SMlfMmpPcmA2Xcvn9p/xxO3z+0Vdl7SXJkFSKPKo64G0AAKCkpwXe+8x185zvfyXdXDMMwDMMwjjjs1tswDMMwDGOEYTeAhmEYhmHkHOcCxDn659y7YwPzwgsv4NJLL8XEiRNRUFCAyspKLFmyBHfccYdMDTpMHMe4+eabcdxxx6GkpAQ1NTW49NJLsW7dukHW7nnggQewdOlSVFRUoKKiAsuWLcNDDz30ruzDAEdkCTgnHD6QHO8GSKsX1vwBQHDs7MxnHeEUb/MiDG2Nwnq6ROmBItK06KipZDNZC9SQHkRp1jhqTus/WLemc3W5Xzraju0wEq0x4sxVpUUJKK9S6wNFFjHpsMKDcv3pXV5/lGUlwigtF2vOQpUWk97hY8hS42V+bfd8r/MqXiNNyPl4ac1UusFbUqQod9QpWw62+sgak1F+nZGyRolbW/06tE6J9j3V1CrahJWJ0i3yBYzzn3UeatLkLTx0zGHClhM6ao76xTo4AHBTfOxduEPaZggLHG2HQ2MU6Kg2mh8RHXM9zqzl7J4zXrQVPkPaXpV7nF7ns1pTM6bJfhWTtcj+VtEW8/hp+yCyAgo7VDwj26GwHRIUdE6G8+fINooe03OW9YhOazRJxxwp3ZqwdKmfJNpSpCsV14P1W2W/SGsZN8ttO86CLpb6atatolVe+9JNQ6c6caxjuECOUfLKWt82WmY8x3P8HIiayUpG2d+wXUyk8r9ZnxwrS6Kwwo9RpLSjjuZD0CTHiC2pQnVO8vzQmsbRG/r1vVGvsv0xsrj33nvxN3/zN4jjGIsWLcKSJUvQ1NSEJ598En/5y1/w2GOP4Re/+EVm+SRJcMkll+C3v/0tKisrcc4556C5uRn33HMPHnroITz++ONYvHhx1nZ+8IMf4JprrkEqlcIZZ5yBoqIiPPLIIzj33HPxwx/+MOOG8k6xG0DDMAzDMPJC7oyg3xnpdBpf/vKXEccxfvGLX+DTn/50pm3dunU47bTT8Mtf/hJf+MIX8KEPfQgA8NOf/hS//e1vMWvWLDz55JMYN67/Pzn33nsvLr74YnzmM5/BunXrkKL/BK5fvx7XXXcdioqK8Pjjj+Pkk08GAGzYsAGnnHIKrrnmGpx11lmYOXPmO96nI2PkDcMwDMMw8sTrr7+Offv2Yc6cOeLmDwDmzp2Lz372swD6S8QD3HTTTQCA73//+5mbPwD4xCc+gfPPPx+bNm3C7373O7GuW265BXEc48orr8zc/AHA7Nmz8Y1vfAPpdBq33HLLu7JP9gRwEJxL4A6X27JKn1R+0lYvXPbdc8YE0VbzU0pVGC8tAbrH+zJMwWOyxBhNpvUckOVhTjloXTIt83nUZlkGiZqp5KjsItoW+/Jm6W+UjQWVz9I7pbN8RGWlSPULXH5oke74ffV+fwq2yySAruneSuLgdL+OcfdtFsuJ8oZOqKAS1sELjhdtox9o9V8bK8s6XMaMJ0l7haLt/nudC2UJsHQDlXJ2KWuMcd5uo298ZeZzz3xZYi597FXf/1ly/V0T/NxQphwA2cfoUi5LB3RpnyUIgbId4vKw41LhhFqxXEjWHumt0kIjENY1qs9kJ8R2JAAQHKB5q6QKXL5PnzRbtO073kscJv1C2gJx2TKZ4a1roj3SBoitN1IdKh2F0jKaTpZzo7bLXw8OfECWjov3+zEqXLtBtPF56NIqsea04/1iK19XffFygZCsUvS1iMvDvePkzCkiWyhta8PnvFZMhVWV9IOyZqLth5t3ijZODQpGk7WQKkW2LPbXhopqKQ9I7fLXiiypCksJKtX3UnQN2yvLwWEblWW1dovtpGqUZVCBX2ffNN9W1KwSi8i+Ki6T87l7vj8/y+5/Ua6frmHhNGVrxufhHrk/nHS0/4Py70/1b/yY9Z1QL9o6a/tL4cm6/DwPStyR8RyqSJ1jQzF2bP+x3bp1K9atW4eSkhKcc845WctdfPHFuP/++/HAAw/gE5/4ROb3Azq/iy++eNDvXHvttXjggQfwwx/+8O3shuDIGHnDMAzDMIw8MWPGDNTX12P9+vX45S9/KdrWrVuHO++8E1VVVbjooosAAKtWrQIAHHvssSgoyPZYXLRoEQDg1Vf9A4DW1lZs29avHT/hhBOyvjN58mRUV1ejsbERhw4dymp/q9gNoGEYhmEYOcchQIwwJ/9c1jPtt0YURfj5z3+OyspKfOYzn8GJJ56IT37ykzj99NOxYMEC1NXV4Q9/+APGjOl/KjxwIzdUOMXA7xsbfQVl4DtVVVUoK8uq+Qz5vbeLlYANwzAMwzjq2bx5M+bPnz9o25o1a97w+6eeeiqeeOIJXHTRRXjppZfw0ksvAQAKCwtx5plnYsaMGZll29v75TSl5PLADNzgtbW1venvDPW9t4vdAA5GnCA50ApgEIuLDoo5U5oAtnphzR8gdVHJ2o2irXAjaaZ0rBpph4KCQtHG0Wmj/+CXYxsOAEinvaZJ26aUUUSZ3raOrBP9WrfJ/5AooVcw9P+0ArK4SDtpO1BAbTUUUZWoqDFtYyDWT8ek4t6Voo01RwlZtGiyrDdozIq2yP91JazDUmPLVh/h4fkEACVqv7lfwatybpRtJEudTtkvsQ5leYO9XtuVZXmzhyxWQmWjQ31jm5Zk6/Yhl9Pz0vVRPJaaC6xbE7GKap1Zmiwieuo18fOk1dTPg7IsIubKS97aI2sGsWXHJqlhY4uT6l/JSDyOM6z4lfwezwc3zDhrUqT703Pd0bnt6LzW1jh8Tha9uEk08Tq1hRPIZkaf1xyf6PT3xKaHPl+DjQ2+QZ0vox/0xzUrfo2WHXauq+Mfd0vrJNEv0n3qaEgx/zZuEU2pnf6Pc0R90fvN41egbHMitttR+8p/K2IdUUdk2eEQ1fetlb+gWMLoaXmjU3lYDx11qmjOHBG/S/58ueCuu+7C8uXL8YEPfAB33XUX5s+fj127duGGG27AjTfeiMcffxxPP/30m9YL5hu7ATQMwzAM46invr7+TT3pG4yNGzfisssuQ21tLR588EGUH86ZnjVrFm6//Xbs2rULDz74IH7605/ib//2bzPtnUP8x73j8MOkUfQf9Df6zlDfe7uYBtAwDMMwjJzjACQIc/Jv6JrCm+NXv/oV+vr6cNZZZ2Vu1JhLL70UAPDnP/8ZADBlyhQAwI4dO7KW5d9PnepDAga+09LSkrnRezPfe7vYE8DBcE6WsQhObdAlUrbX0FYvXPZNTZR2Ea7Ml8Xi9bJcw9vT5TTuIzv6R8rd37UPPpEAICCbG23nAQxTpqJElPReaSUhEkSUk31AJ06iHfBp/OLxZNOyWo4JuNQSyP/DsP0Fp28AQHpLg9+W0lgkXEJTpSkeW13m5fKNUyUgtqsR61DJD1ySDQpkm5vuUxWCTbJsLUproZoboPJjr5zLIVlcBFSSBWSZOaZyWmqctGzhBJlY2XKIsrIqI/L6I5WIw+OeqDnLJcdIJbUk1WRro8qDAfVT7Ks6VjG/UVcl+8VlPkyS23ab/DmTmiqtN1yntwWJ1TkyHAH9z961yNIkzzFRYh5OFqHGOW70pfcgpZOBhi4D8jzSsoKEEyp0mg2do+GYSt+PJmkDFU72Nj3YsVu08bVWz/Wwgq67uhTO10wlK+DrVKT2JyZ9VVRZKdoCtpqh7bGVEACEbLGkroMRlQjTu9S+0rHMSlwhKYm+Tol1TJNpLG6dt9IKy+Q5j4GkqMCeBw3HwI3X6NGjB20f+H3L4XN24cKFAIDVq1ejr68v603gAf3gggULMr+rrKzElClTsG3bNrz88ss47bTTxHe2b9+O5uZmTJ06FRUV0vLo7WBH3DAMwzAMYxjGj+9/cLNy5cpB2wcMoKdNmwYAmD59OubOnYuurq5BM3zvueceAMB5550nfj/gGTjQ/ma+83axG0DDMAzDMPJC7MKc/HunXHDBBQD6S7z/8R//IdqeffZZ3HzzzQCkgfO1114LAPj617+Offt8FeA3v/kN7r//fsycOTOz3gGuuuoqRFGE2267Dc8++2zm9xs3bsT3vvc9pFIpXHXVVe94fwC7ATQMwzAMwxiWRYsW4brrrgMAfPnLX8axxx6LSy+9FKeddhpOPfVUdHR04Itf/CLOOOOMzHc+97nP4aKLLsLGjRtxzDHH4JJLLsGHPvQhXHzxxSgpKcGdd94pcoABYM6cOVixYgV6enqwZMkSnH322bjwwguxcOFC7N+/HzfddNO7kgMMmAZwcMIQ0aj++npQKvUSrOvRMVqsKeF4N0BZvSgNxqHjKBZog4o9G+e1hHGl3F64oSHzuXeyX0e6TB7Wki1Ko0X01XodQaA1gKwJ0W/qk6YlNVFqnzjKCBOVXosjpJQGsHeGX7a7xo/lqM4pctv7KBZKa8U4AmuyjHtLURSU1vJFNTS2idQKxdO9ZjO1V8beiWHR+iPW/VAclhsjNSQhaYe0tVDHZK9NKmsc+nTVWr4w8uMcKMEy6/f0/I6KlNXQwHcqlOiZ16+sN1ib5HqVjnQ4nVG1j9UK9TGg45qMkXqtg3P8HK7ao7SdFG0nIgrHyEg3ng3dU+W8KSENYG+tHIfCbj/30zVSkxO1UKwaa7cUws4FQHq2128VNGqLHb8sx/bp48/nQaxi1aIu0id3SZuUmC2jlD5QaMfUcWS9a6g1zmyxQnM/pTTNfC3SbSFF7iXasoXPGaXLjsb6OSWuS5CRjyIuE0DAb2FGcl/5Ohy1+Hka1ii9Xo3f1ySlxquPtJz7muW2+dxSNwcc8RgUaIsy3+emE+U1pnYHaa8ny+PTOq//fEo35/52wCFA8g4Nmt/Ktt4pK1aswCmnnILbbrsNL774ItavX49Ro0Zh6dKluOKKK/CpT31KLB+GIe6++27ccsst+OlPf4oHH3wQZWVl+MQnPoFvf/vbmDdv3qDbueaaazBz5kysWLECTz75JADgpJNOwte//nWce+6573g/BrAbQMMwDMMwjDfBRRddlIl7ezNEUYRrr702Uw5+s5x33nnvmtZvKOwG0DAMwzCMvPBu6POMt4fdAA5CEEUIDpejnE4WEFYIshzEJYaCx6T3D6dsaKsXLvvqcmq83acLRGn56B5jfKkqemGd37YqI2p7EiYgl/tIW31QuTssk6U17leWHQFtj0s3AOCafTnaKcuOgtd8XwqoxBSMVoaXNO6uRZZ1UnW+fOZeluV0HodYlb5Csl7QpZzUntbM52SPtPNwMSViqNQY3lduS6gUDUhbC23ZUvzQC77PysYiYhsAZTMR7/PbSFVVyu1RiSmtElG4NM7zIVbSBN52VDdRtLmDZIehLIl4bujyfUjlLp00wf0Ktu8VbRXrKKlhnLRf4vVwik+kjr+jsS15fY9sozSJ8M+viLZY9EsmgYCshkJlxeLIZsQpB5fgKb8NN2uGbBNz2J+f2uJE2Byt2yrXQedPMkUeu4jsV4JiOad4HGJ1XeRUClekgu/puAZtVFpVZd7U2gbfLz03eCzHy+sUJzDpucjnq74OpvZQsolO/+G0FFU65mvFcPZRnP4TkbwBkJZbUbWUIzgqP4vUHgAh58MqK6sUzf2au1aJNk4pCdbJc3lU+dz+fvS9U6c840jDbgANwzAMw8g5DkCco3dR7fY2G3v2ahiGYRiGMcKwJ4CGYRiGYeQeByQuN28B2yPAbOwG8I1QdhSs80jvlvoMfkU/mqy0fGs3ZD7rGDJh9aJ1RMfNyXx07V2iKWnw2pdw1jTfsFtqzIRWMVR2BBzp1jh4ZmF/x6R+hu0JgslSd4ND3p7CpVSsGumKtN6t79jpmc8FTWTf0aH2m+xjdNQca9/CepWVuMXr3aLR0hpD9EWNkSP9XnzCHNFW0ODnQKzsKVhrF5IGSGsFQVqhQM0N7hdrMgEVX6Y0U6yFjHdJTRvHS2k7HKFjYmuPYYLHk/1yvxOO0VJxRRwFx7ZJgLTACbTmlLRcQbHUuAYUz+a27RJtrPuKakgfqCLq+BzRc9Y1e9uhYJG0bQgb/dgm05QdUuyvHQFZNgEquk1ZqrBezO1RelGKUmNSE6Q+mGPJgklKO8za2y1SA+oSr2nVsWqsvYtUn+PWVmpTsYQ9g0dWptW8TE2t89tSOlw0+zmWbtgumoSGVmk7OVYPSnPs2ukaUyt1eCCNo7ZmCkhTy3uajJZ61+AgnZPanmbebP+9TQ2iLaTthUqHGZBmM35dasmFbc+xs0Qb1vpl3dx60eSi/r2w+6ORh5WADcMwDMMwRhj2BNAwDMMwjDwQ5OwlkOw0A8NuAIfisK1CoG0s2nyJJKqbLpoCLlUeUIkRBVTuUiUS4SyvrF647Nu0RKZqjK2kvh30pbW+Y6eJ5aIuWfoQbWSFEKl9Taicoq0XhJVJgZxGbJ3TdaIsw5a9RiXNZlkCTpf7EkbU7UsffXXSQiNI/DgUPL9etumSJrdROShpkWXLiEtoqvTVNcfve+lqVWKkhIywXJYtg25Kr2BLGFXCZJL9MrUlmObLYknTftEWsbVIgUrw4BLjsTI2qK/QH69IW1xU+DHqG1+Z+ZxqVzYZE33JLEjLtI9oH1n9dMrvsXQgUePsar2tUbhLlT7ZIkYl6YTUt2RGnWgLVnvpBdt5hFMnieW4RNcxV1rJlJb77UUtstQOKlVHe1tlG1mzJPWTRVPU5JeNm4dO6un569ni56ImKqHv8ucLl24BIOBzUiVZ9E304+ymKsujF/z5pM953h8t34jo3Do0TyapsAggXU5l/rFSVuC20LmlZBJBFSWIKHlAwpY02j6I+hWqdfYe70uhhU3quDo/nomew7Qet933OVDjnFT4fqYrZSm3sMGnf0TKpontdthmCgDCVpJXaPuYcf5Ytk+XFmWjttPPnXKM9p7RP7bxq3aDNNKwG0DDMAzDMHKOA5DkyAjaNI7ZmAbQMAzDMAxjhGFPAA3DMAzDyAuxafPyht0ADooD0v02EVrrwhFFwRRl+7DZW4I4bR9DmhLXJ9cZskXEGKmfYasXofnTkF1EgdIDoZU0MioyKqb90VoU1tO4NqmR4Ri3YG+zaEuorXSj1K0JawllJVG6iuwd2GamT2qF4lKvd9ORYWxXErSoGD8+BjoejzVzSivE/WILGgAIOkiPptbpyKJG2J+oPmfNFSKk46+tS8QYqT4Lq5xKOX5Bj1+PHr/koO9LwXayPFLaxBTr8LRtDtlyBFofyst2qzi+zTTOeozoPEwpWyC239EK0DSPbULn3T45LxNaf/lKZRdC+8qxYwAQkp1QouLRmEjPjT6/DZdW2yMbnYJD6lpxiGLCaJ06spJtdNxWaZuS6vPXrWS00q2yvk0dO76OcKweIK2FKta1yu81+eOTOuCPfzJOxqMJaxyt5SugfpVKPR3YpqdCrhM8zkpPV7TZWwslTfIaJq6TSrOdkHUOX8uD7VIfzNZCBWmp12NbIx47QGqv9d8KtvdhvS4AoWOteFla7MRt3vImVONQubl/+6HclDECsBtAwzAMwzByjkOQQw2gPWnUmAbQMAzDMAxjhGFPAAfBRRHi2sOvxpdLe43USl/64HIMAKDGP+ZvXTJNNI3+g7ej0NYyvZMpQeSFdaJNJHwcVNujsu+2vz0283ny/0iLk7CLyilFcn/aP+Qd40f971r5PUqliJtliSSa6S1wAuWwH42lMrZKiXB13sIlpUo5HfO9FUv7RD81xz2qEkq2tPr1KdsXtoto+StpqTP6d6/4bddpGxC/DwdPkskmFat9ianl9GmirbTJl12KV8lUhaCUSodLT8h87hkjy7VlD/l+9XxogWhLUv5/rWUbZNky3rgl83m4dBl0qXIafXaV0mIHVB7qm+Lnc6pYzhsu5QaU/AJIK5tonLQS4RIjp6MAgKMSY1QrrVjSO3xCTu9MKb3YddqMzOep/yltgYT1xzGU4rJDpviwfVDPMfL4F768OfO5+fK/Em1jV/vrQUedtKcJqNJW/vtVok2XzZnuZcdlPpes3CLaHJ1bwrapVpYRHcky2s5dKNrKG6h8+9pG2ecJ/vyMVUpI/IH5mc8qy0Ye85T8s8LygWiutyQKdslryoELaP0dUhZR1OLPz8JVW2WfKVGm8zh5Xpds9defgMqnABCPq8x8DpXUJ6SSczhWynJ66/y8TYr8vIm6VNrHHm87tP94uY7wWP9zxW9ekm10XIMaWTruOsZf00pel2XeZHND5vPurywWbZMe8WPUMVP2pfm4/n2I/4y8YBrA/GFPAA3DMAzDMEYY9gTQMAzDMIy8kCsNoJGNjbxhGIZhGMYIw54ADkZvH4IN/fqXQh2PxtYLKpIKB73GZNRmqTdJSCelI9fSZf4wFBSpmLDdXuenI97Y7oV1f20zK8RyFR3eViDokzqVURu97kpb3rgub7fBOhtA6f6UBjDZudsvN0/GkLGlgoiTA1Bc7fsdF3kLhbha7g/459WbRFO6wVtelE2sFG0h6d3ifXLb0USvran4s9RdodZrfsY8JDWaQbnvpzZzYbuKAorYKqqReq2Y5lTR46+KtvTJXhfltu0Ubaz705YQInpuoowQZAJlccExeKm1DX65UcqKh/RTwe59oi2s8hqj9K7doo2tf3Q0F0ePJXvkOtmWI2qX83Tci16HFaiYMNdDy5LNDOqkPjTZ1Jj5nGpX87nex8vVPqki6uh8Gr1P2Q6RNUuW0Q+Pu7JmKjzgz9dksuxntNvr5hKyTXHagoQsiEa/JPWOwtJpqozOS29qyHxOTZTbDl7x55q2UQrH+nPEjZLXt4j3j49HjdSiVa324xXuV5Y6KdL6jlb2J2nfl5KnVTQknReJuvaJCMO0bONrXyKlt+AroVNRikw808f/VWyV1kWpZtLNqv2JD7RmPkdOWrYUP+81m2681MmyJVHJfmUFttPPgTJlA1N2WKcbDp0Yahyl2A2gYRiGYRg5xwGILQoub1gJ2DAMwzAMY4RhTwAHw7lMOTQ+oOwBqETrtsmyC9t+RM2yhJEmt3/XLl30S7b4cl1WaYXc5KMu9YyebDPY6oVLvgBEsoVTZbdgj69v6EQK7kuoSsCixFgsy9YBJ1R0KgsSSnGIVSkiiP32yzf60kqwU1l2sL0KFFQyKdgjyzMxWdIEkfy/T7zdl1e5lKL77JTFhSgxatf+9d4+hMu1jko8mkDZ2hTu9VKCRI0Xlw7jZpnUwVYSUKV21HpriWS3HFu2JwmojMg2OQAQNfmx1XMqobKY3h8xx7QVCo2ltkpimUHUIuUVxdu9HYarlMcu6fDWSSGVhxNlcSI7Kc+DaB/ta7mUfSS7/LbDCbLUzukVKFHfo2QYnTSBld6OKaqVNiCO+sbrCIulpVJCc1bb7XCijLZwCijFJSv1hq8BOh2DrF5CdQxAxzygOexUghCvMdbJJnzdVeVaPccYRykxOnElbCeJS4Gcw7yNQM19se+cZhTKMYl20zlZK8vd4jp4UM5n0Q9Vthb7vqVRtPE5X7FV/g3gZJ1QJSSNWdv/NybqUdeXnBAgyZkNjNnNaOwJoGEYhmEYxgjDngAahmEYhpFzTAOYX+wGcDCCIOut1wxcblBvzYoygiqRDFemGJZhEgNEqYoSPvSbvlyiS4pkqSPifoZDPyIPdOmTSxGqbM1jFKiypUtnFW19G/UlTIYuR7jOriHbgpTfv0Bti0tAQYEsyXG5Wx97N8zY8tuJTgW1uITe9Cug8dPrILLGmeeNPj5cTlVvC4JL3IE6zWlsnSp3BtTGb1DqN705MSRrvIR0QEkOxA9qf7j8rEpy4vzRpXBRhnt7f0xYEtCn0n/C7f48dwXleNPQPugzK6CSHIYpkzstkyga/Lqkj0FCMhOnJBpCHqDOEZ5/Th1zfoten4NCupJS1zoqOTsxL4cpyemEHzp/XI+UlfB6tBzF0VvtYp5AjkvQLa/lwyEkD7w/+jjS+CUl8hzk625WaZqdINR85rf2s98s98s6da3gdWr5S1LY32833PEwjkrsBtAwDMMwjNzjgMTl6MbTHgFmYRpAwzAMwzCMEYY9ATQMwzAMI+c4AHGOnkPZA8Bs7AZwMIIgo7UJdMICJRQImwwAjuwC2hZLh/2ynd4uQq+zr9bbJgQbZQpFSDYQ0Z4W0RaTzqf9Q7MynzndA5BWL5HSebSfNDXzufghlehAupH4kFxnNGuGX3+btLUJSkinckhaHLgJPu0hUlqhXtJetc73YzTm0Va5juE0gKQVaj9WJhmU7vO2Ezo5IyC7nURZNoTN3gakd4a0+iggR3/XJCMDUpMm+HWO9uvvGS8tTgqfeM2vY/ok0dZb7a1LilulJUh6h7euCQqUNoz1TlozRe44bDPUv1JKl2jz+6YTNlibFO9RVjJElsaQdZ6kKQMA163si7iNLHbi8fL47D/O70P13atlN1nnRekfQZO0zWFrpsImOZ95TNrrZZ9HpadkPndMlW2FB8nW5ulVok1o7XQKxSkL/efXVCoNLRuW0TwqkJdy1l7Go5RNE+17oiyJAraM0rpP1qMpHRlrNF3DDtHG64n4PFPncfeSeZnPRQf0nPUC26BbWUvx3BxTKbtM23YqXSZs9+vUc0+k7IyR8y0pomM3zZ/jYZuav8PomDsWTMx8LiIrIQBIqM+pqkrR5sb4ORbskvsTt/i/D4emHyPaxr7mz7v0lFrR1lXTf8xdyjSAIw0rARuGYRiGYYww7AmgYRiGYRh5IMjdSyBmBJ1F4HSNZoQzf/58NG7ai9OmfR4AELRLb4+YnOV1CYvd8HVZZ7jwdyYaJx/Px1RWjFQ6AidicOKBU/Y0wi5E2QPwslH9NPk9ShrRNiBJF5U7nLaqGNrGYLiyiLD6YKsFnVZAJZ+0SrmIZk7zP6gyH1tjaCsEYUEyTZbvHaVGBCrRQYTG637S/oQU+J5QaRWQthairAeVGKETKioocUEd1/iQ30ZUJecpSxDSW2WaAM/TqNqXnOMmOc7hKL+OsGas7DPN2XCsLJ+ByneJkgck1BZVyNJ0TOkiUbXaHrWFY2XqRazK8kPB0oEsyyYaEzHv1bKBKsMGdCxdp7yO8PUhy9aE0NcDPheSNj9+WQlC5TR+6pzjlJUsu6vh7Jeoz1mlYzqfApWkI44PX8O0HRKXkbtkeTig+YbRam7Q+RnOmyXaQG2JKh3z9VQfV5Yc6JQVca3gUr46Blz6DmdMEW3J5gbfpseLUlD035Gopsa3qbI1WwTpawxbdWmrqWRB/5g98+qtmDq9BmvWrEEumD9/PvZ278DnfvORnGzvpx9/BOOK63K2f0cC9gTQMAzDMIy8kJgSLW/kbeQ7Oztx33334fOf/zzmzJmD4uJilJWVYeHChfjOd76D9vb2Ib/7s5/9DIsXL0Z5eTnGjBmDs88+G08//XQOe28YhmEYhnHkkrcbwF/+8pe46KKL8NOf/hRRFOH888/HkiVLsHXrVnzrW9/CX/3VX2Hfvn1Z37v66quxfPlyrF69GmeccQYWL16MRx99FB/84Adx33335X5HDMMwDMN4y/RHwQU5+Wdat2zyVgIuKCjAF7/4RVx99dWYO3du5ve7d+/GOeecg5dffhlXX301fvnLX2baHnvsMdxyyy0YO3YsnnnmGcyadVi78MwzWLZsGZYvX45ly5ahsrLyHfUtcD5GbNjosoPKboX1NDoWSOhdho6yGs7iROtU2N6DbQu0fkZrUxgRO9Qq90f0S9t5kOYn6VVxUrSs1tqJtSi9EW+D16H1bYkad7FtOl5ZmhzWweh+cXxZm8p0Y42O0vnpqC7ZSHFppLsRYw4g4X7q6CeKttNaS54PYZnUJor4Ld3nZtJG6vgnto9h/ZFajueYPg9E1Fy7slQZZrxC0na67qHjvlyP1Liy7i/rnBwigjFUFk7DwXZLoY4aIw2t1lYFZHOUaKsXrdPl7/H8UOd8MlSE3DDr1/NNHJNS9WeR15+lF1a2MNzG0YCJvlZQP+k4xmpuRKzfVFGArHfTx4DPi1Brtnmc3dDXGz0fYo5n07F0evsDv9dzjbWCB6XeVcwVNV4y2lCvk2LvtJaPdX96nSJSUl5jogH7Gv0d46gnb08AL7vsMtx+++3i5g8AJkyYgB/96EcAgN/85jfopRP4pptuAgBcf/31mZs/ADj55JNx5ZVXorW1FT/5yU9y0HvDMAzDMN4piQty8s/I5n2pvly4sN8ItaenB/v397/F19XVhT/+8Y8AgIsvvjjrOwO/e+CBB3LUS8MwDMMwjCOT9+VbwFu29LvfFxQUYMyY/rLA+vXr0dPTg5qaGtTV1WV9Z9GiRQCAV1999R1v3wUB3GEbkiyHeCoV6PIfp0ukd+568xuk/5yEOnGBS5PjpSWEowSOuNmnXGhrh5B+1mUDTviI1PdESXaiTMCI12/y6y+Vfebyk5szXbRhrf+etr/gElDcQrYfc2eK5YK9fl+hS4VUptIlpoDKJ4natkgyaJPlmr6FPvUkenataHNUKoq0BQlZZbBNT6LWz5YqiX75aRjLIGFPkZUEQokbyqbF0dhqhJVJLX2PUgayvlM69JzNLpn7tkRJFaK5/qm+a9wp2oaTOIASeEKVnBA3+bnClhp6Pos+lsv9Cbdu931Wdi5iW7qkSVZG0USZSpPs9ukpWUkgPDenymtdRPY+6b2kkdalfCrnafkGj1EyVlqQuNd98og+r4eziOHyat882efCbWSPRdKLKJDPHxK6FjllEcPnLpQdFp8jTpVahQxEy1jGUbKOtquivuhjzukcPL/CCjmWIlVF/R1J6Gdt78X2Ytp+SaTzqBSfqNhbLqVVOo84lmoc4lGHryNhPp6SBUhcrp5D2VNAzfvyCeAtt9wCADjrrLNQdFi/sm1bv5/TYDd/AFBWVobKykq0tLSgTf2BNQzDMAzDMDzvuyeAv//97/GTn/wEBQUF+O53v5v5/YAtTKn+XylRVlaG1tZWtLW1YdSoUUMuB/SbUA7G5s2bkcLw3zUMwzAM453hAMQ5ejJnr7hk8756Avj666/js5/9LJxzWLFiRUYLaBiGYRiGYbx7vG+eAO7cuRNnnXUWWlpacO211+Kqq64S7eWHo406h9HgdHT0a3De6OkfgCHjYObPn4/GdTuRNPTrfrQ9AFtvaK0d6zOi+XNEW7zOa99StdWiDbSNeLvUPrFmL1GxZ2zLEM30WrtAx7Y1Dx1RF83y+rakYYfcNul1WPMHAKm6SZnP6V17RFtI2pdg83bRFpCOUVuEBKRpc3892X9evUUsh2FsbRKK6otmSf2hiF/ST5JZF6P0dAVrfJwUlO6GLUk48gqQcWa8bzoeLd3g1x9Vqei0CRT9tH23aGJdYdih7FZ4OTVvWO8UqRgqtl/h8dL6RmHnsVsefxHppbVc1BZpu6Ydfj3D6eJS02SsVt8EGrNVG+X26BzlyC13SGot49ZWv/6JE2S/6ByMZtfLtj1+bINqNUY0N4aL3NOaLD63kp1ybGO2/iF7l0RpzFj7FqnISj5ewQGl7WSNprbwYVsjbS1Dxyv13Dq1SrJpGVPp16/Ol3DWNP/DbjVnqV/aIkjsn7J34uOfZb1DtldOW2yR/UpEfe5fKcUl1tL5qS2I2NZIzY0UzcV4l9TrcQydvh6I80KdI2mOKT1O/v1x67f6Nq057Dm8HkuFHXG8L24ADxw4gI985CNobGzE8uXLccMNN2QtM2VK/0V/x44dWW1A/81fa2srqqqq3tQNoGEYhmEY+cUsWvJH3kvA7e3t+NjHPoa1a9fi4x//OP7zP/8TgX6jDcCcOXNQVFSEpqYm7Ny5M6v9pZdeAgAsWLDgPe+zYRiGYRjGkUxenwD29PTgggsuwPPPP4+PfvSjuOuuuxAN4dxfUlKC008/HQ8//DDuvvtuXH311aL9nnvuAQCcd95577hfQSqFaFJ/GSgr5YDTJNSNKluLRAeU1QYlMwj7BqiSk3adnzzR/1CgkgbIDkWUfVUJWDjXq/JpQFYyWUkTlPChS6Zc9tU2LeBS5fga0eT2+zJFrMpPwaJ5mc8FO/b770yWJbmgg2xAmvaLNpGAoMqPbK+QVX6mkoy2ykERlZFKZOmLj0GWPQmthxNegiJZYo5qaIz0MdhMpcO34NSfGu9tTnQpj+2KYjUXuWQW0Tp0mV+Ut4cpBw4noYiVtUw0zssDQnUecFoKSy0AIMV2G5Cw1Qz/b1fbxaRG+zHRViIifaFDHeNSlcDC0PkalsmyG4/RcKkgfKwAIKRSYrzTn2d8vAHAkTWOU7KZqM5fU4azTQlV6ZjPi+HmTTB1kmjj0j5fM3WiRtBCJdnxSibTRDIWnY5CP2tJQ1Tj1xMraxS+BoRK4iDmKdk5AUBUPkQiir7WsW0Op+8AAM0/neLj+vx8zjpHyCop2bJNtPE1OuhREgo+f+rkXOmp6f+e25j750HOIWc2MFbhziZvTwDjOManPvUp/PGPf8SSJUvwm9/8BoVKU6e59tprAQD//M//jI0bvdbnmWeewe23347Kykp8/vOff0/7bRiGYRiGcaSTtyeAt956K377298CAKqrq/HlL3950OVuuOEGVFf3/y/ujDPOwFVXXYVbbrkFxx9/PM4880z09vbi0UcfhXMOd9xxxzvOATYMwzAMIzckZtCcN/J2A9hCj7YHbgQH45/+6Z8yN4AA8IMf/ADHH388br31Vjz66KMoLCzEGWecgW9+85s45ZRT3tM+G4ZhGIZhHA3k7Qbwn/7pn/BP//RPb+u7l19+OS6//PJ3tT9ZDAgG9AsppE3S0U9C70S6JL0ebR8j7BC0xQnZVWg9YsI2MGQtkuyUdiFC06Y1hiXUL6XlCjhiSemU2OoFyp6EbW2csrEQthzl0lIlPEDaLrJzcErrEtN+B2p/QrZN0JFHpOWJqqS+KdnvNTrh7BmiTdiTzJBJNAFZXGjrErG9GWRdkpbHmI9/NElGhoWsFVPRgxxZp48P6/7YLgaQ88Gp+cY6xqDba5MCpT9NSAsXqLnOli1aRxaSBU2gLDt4HPT3RD+1RpPn6TAWQQHp9bSGTVhjKK1lTOMcKQsnnt9BlbLUobmZbFMvrrHWU81Td8hvLytejq1sqJ+J0kXyOrW+jTWUw8USZtma0HqCEqlb4/VEe5plG8Ubsv5Vb5uj2YJ2qbUUmkl1nomxrJa2KQlZrGTNDYqiY42uhvWuAAC+TrIWVttvkd45VNpEPj6J0pXyeaGvyegbWjvKlkupdmWXRvsaHpRzquhwX4L0MFF/7xkB4py9BWxPGjV5fwvYMAzDMAzDyC3vCx9AwzAMwzBGFg45fAs4J1s5srAbwEFICiL0TB0LAChcJx/Pc2lK24UkY+kxf4sqyfA6lDUGJvrX8sMu6XLvUr6M1HXiVNFWupEsUKjcEMyTtixBp19noMuIVG7ihAhAliLcHJmqIRI+tP0BlcX2fVb6Mo573Jfe3FaZEtI7xZeYUvv9uB+88HixXMUmKimt2yra2FqCxw6Qjv665MOl40BZ+Oz4/LGZz3W/kX2OyWIjNXWyaGObiaCbyjU6jYXKvskeVZokuwjX0irbqFwX1MnSsbCIWKxyrw/6fY+U7QyXV9OzfLk7Umkswp5E2dqEsT+OibLQ4BQHfUFuP/+EzOeKJ2TyDKeGBOr86avxfUmlZEoIdlIJkI5H95kniMUKD/nyXXeN3J/ytf48CzqVBQkdy0DZDqGQEkSOkQkinOqiky04QaL35NmirXgPzanX/dzTZdGwgkrFxcpdodyXb7v/Wvar6H9e8n3WSTrbd/ltq2tfVD0283n338gUipqXfJ87q3xfSjdVyn6xJKBblTdZ9qGvRVupvN6qLXxonarUniz2tlNdVXKMih983n9NXTPjem9LVbDdz430BFl+7pnnr+tlr6sUJ7ruigQUAAH9DUh2S+savn5H8+Tc4O+5IinLCCkpJimXll47z+g/X9M7BrdgM45erARsGIZhGIYxwrAngIZhGIZh5AWLgssf9gTQMAzDMAxjhGFPAAch7EujaGu/ZiPLJqPHa1OCchnvFOz1ViJ99Sq+jKxZAmV/kpBOxanIoKDY653KXlO6GNZTcbyPsq7h+CqnLEjcBB+/BaXJE4qZtVKTFYz33+N4N0BavbDmDwCS0V5/EpImDwCCFt/Pzulew1T1jLLQoPFK9PGhMembIbWJBc2kk9QRXqSFS0jPBACTHiIdmda7kU1DvEvqdUKOF2MdVqGcN8kGr6+L1LaTcWS90SdtJoTG8JCKtiMLknCzsukZTfOvQO4Px4ZFL6/361C2LGxzlKiYONawaYTFxSgZczb6eX+csyx1yCImqJdRY32jfF8KtkjrEp7DbIdStlqOSUKxhOV16twl/Vk8QVqqhGSxk1QpW6MOPxfjdRtFG9tycEwkAGGjVLpG9pOvP3yeD2uNorS9rL0tU5rjhDS0Wu8o7HfCoZ/aTHxoh9we2c6k+JxX+tDeBdMyn4valHURn6+dss+gaySUFU9wgMahXfa5oMFfmwq2SJ1fOqTzR1m4BC2kvRzl53pql7x2hz3+e/FYFenHkZLN8voZD2OHxDYwWXrUvV5nmF4k9YEpOuaBumaObuj/ORw6kfA9xYyg84c9ATQMwzAMwxhh2BNAwzAMwzByTr8NTG6eAJoNTDZ2AzgILk68C75KBeASY6yc2Nkage0BACDNJUZl5wH62Sk3eXZ7d83qGT2VkVKlvgzCpQAAiOOhHd4jskrJWiqhlIseZU9D5cf4gCxhcMKHtnoRZV81tsEu3+8ycqVPb5MlpSDlS366RB9ROa2wUR2D1la/nCrrxJS4klKlfbZmYVsWAEj4+PTJ4xNTukiKy6La6meYtI9gt09VEOU/yCQYt0+mL7g0zSNlMxLRsUw6ZamNx5PLyMkhZa9BPzudZEEJEllJBsKWQ9lr0Djr5BEm2iPnW+kBPx9ifW7RHAtpnXGLtPrhvkS7pWwh7vKlNm2UwUkdUbMsPwvbETVGcEMnlnBJPau0OwR6ueQgJXMUSEsQTtJx2maE5BtJk5xT4tgpmQlfD9Aqx5YTSyL6HNP5CABFjf6czLpGch/1vvLxKZUWQfFBNW/5e/sPDNnG80GfdyApBlsSuXZl/UX7ENVIaQdonbG2SqLriFNJMKASfdIk2/hvReEmKR2I2a5opzzmFYctd6LuNzfXjKMHuwE0DMMwDCMPBDkzgrYouGxMA2gYhmEYhjHCsCeAhmEYhmHkHpdDH0ATAWZhN4CDUVgAN63faiJQlgNhC+l8dIxWlX9lv2u6tIsoIBsY/Wp/7wxv4VLwmozc6jvWxx6ly6WWp3SV19d1zPdRYMXVyrYg9jPfKe1Ob7nX/BSsUnow0i1FY+X+sD1NsGieaAsPeC0Mx7sB0uqFNX8AsO8Cb11Q1Oa3XVE4V66jh/Rte5ROiWg/VsajlZEdRVAqrUpSpFtMj68UbfG06sznvnJ5ypTu8PsXbVd2KKSniumYxKXSeqWQdHdJtYyTikf7cU5tkHY4AWn7OAIPAMJxZJXSJrVC8VSKHlyjIt5I/+gm+P0OW6W+yZGNRdguY/VYWxWOUpZHdAwQSUVdMGeG/6xi1cJGrwNN10k9Vdc4P0bSbEPq6QLSfaYq5JKObG26p1aKtpINXhN4aJG0iBlF3zu4oFq0pbr89aG0S1p2cBRh0qEsTyZP9J+hIB2jo7i/aJLsl6N1ds+TtjkFE/34hQfltrG/1W9b6WSTMX7Mwv1S78jHvPdUGT1YvN4fg95p3pqpYI3cdN94f10MxyhLnUM0x5Rmjq2YXJm0dwqn+n13O6QuLqj30ZqJOieDl9f5zypSsm+Gv66EvTH9XtpOsaaue5S8drOGsmSl1H2DdeCF8nt9833UYeq5daKNbY72njNDtNXe6+df9+JZoq3p+P597/svux0YadgRNwzDMAwj5zjkzgfQHgBmYxpAwzAMwzCMEYY9ARyEIE4QHuovjSTl0lYg4LLvOFny4Uf3B6fLR/c1bEExXpb5umuoDBvIe/KCJl8yi7plX9iZv30irb9Ilk/KN5KzfCLL1q3zfVlnzDrZ54BKwNo2w/31ZN/HHdJuBZT2kNovy4Oc8MFWL4As++79K/+/wtEvyXU4LncqS4iQSnut9XJ6lz1LyQlFal/JfqW7Wo5z2Va/73GRKjKyvYdOJSn3ZWYu+3ZXy3JT9AyN3+Ra0dYy25e0aner5JluX9YJdfpGO5X2VHnY0fEJq2WJnu0psNOX6JNJsryVHu3XWbBPHn9ObXAqvYTL4hgjpRAhJdZAJdbEtK8H6+W+dkz0+1O+Vp4/nNYTU1kx2tsqlmM7FDetUm67yh/zzmq5/vJSPw69ZbKNS8BxkxojtrlRSSB9E/24FOyWpVZO9eHjytY7gJRo9JXLEmbx695GxynblICTbiI1ll00N1RZlFNdijdLaYfrItkH2cBo2cKBuf64jn1NSg6SUb6fYaTmLMkFesfJa19Bkypx8zrpnNQWPgnb4dRUirZ0GZXhqbRbtFduKy7zbW1T5PWmoN2PQ4m65rM1UzhhmmiL2nupbZxoS8gKqmOSeqpGfyuCtHwW1juq/2enPY6Mox67ATQMwzAMIy/k7CUQIwsrARuGYRiGYYww7AngICRFETpn95e8ShpbRRu/vYeUvH8OD/q3Msfdt1muk8uDqzeJtlGd/s2uYLQqMVLJp69OlsyCPr/suEf9W5KxfgtYOb8zYx5t9T+oFBIOfI/mzhRNbrV/e9RNVm8g0tuJBy88XrRVPePfZNUJH/y2L5d9d39Urr/2Be/uH/TqdBTf57r75VuzoLcF4waVUDLLv21d/uI20bbnPN82/hG5TkdJA4FOCaFyWsHaxsznQvUGMur8W5/JGjlvajZSyoV6k1SkNtRKWQE2+X3oOU2+pV2ynsqdKlCej3nvsb7MHz2xSixWyOUn9baoSJDRqRBUrkvWbRRt6Q+f6Nf/jHzDMarw2xjzmHxzuXiRf5szaJdluITOV/eSX2fb+SeK5YqbvJwj1anSZVr8Ma69S84NfqO/+vey9InRVI6k+QUA2O/LsIlKxAm7/Bi1z5VvPJdu8290B/Q2bKDKtZwoM+q5RtHWN92/xeoi+fQl2ubndzhjimgDlfpjVXKOqn0/m5fIt47HvqgkIodJT5aygtpn/ZvESYksmXKJNq6tFG3BWj8fitTx5+tBrOQi/OCpb4JcZ7idzvO18npd3O3nG6jsOuAckfm5wB+TcQ9ulW0kcXAT5DEOad7EG+T1IJrr3+DVfU6RrGD6vVpy4MezoE1eM6Pew9fFPL0lYU8A84c9ATQMwzAMwxhh2BNAwzAMwzByjkOQsyeAzqLgsrAngIZhGIZhGCMMewI4CGFPGqVr+l3jXbdMAnGkOWObFABIU+pAqBzcHetPlF0Ia2ugkxNaWv32Evnav0iU2OKXg9YAsi1Hp7JUoZ85BQIAEtL5BHtV4gbtg7CmABCTlrBik7RzQOj/zxGklBULJXyw1Qtr/gCge5zfn1KVJuEO0bLjpcYoYG2a0l2xlQRbJgDA+N973RcfD43bKzVg7MyfJhudUGkteU45nYDBy/XIuRix5vCgTPtISPtUvPWAaOub6PsVrlRpAjQHCl/xuiU1Y4XFTaDOEdkROc5O6bCYolUN/mvqHGFrjJSyril9ar3/YUyl3B6PJ1k4lT/TIJej84AtVABpgeN2yrQXPnZQx8ftIH3oqKHtg1yf1GSF63zfikqk9jY6QFo4Gnct34pqyf5mnzx3C0gP1jtVWlkFNPfj0TJVI9pNujV9DaPvVa1VljRt/tgV0LUiGSWvN9jlE1cCPU84uUXZR/E1BSrpSCRpqGtFapvfnk5j4fkXqO8llErD8yvcLHXFhXT9cRXyuo7dtK8qbSomDTXU3xg0ee1owUGVzlNB46nsvtJ7vO43Um0VW/u/Fyo5da4wDWD+sCeAhmEYhmEYIwx7AmgYhmEYRl7IVRSckU3gnH7GPLKZP38+GtftxKmlFwBQJR6o0qGTj9K5VCASDyBLWAiViz4HvOsSYIlK/+C+cMkpeuc27nrbb5ZQJ03QegJVCufScaJKoRHZiXCpUB8DHuesPtMxCUpkCYsTCaDc98X31LET5ZQCuT+idKQSHfg4a0mA2PQwZVHup0v3Db2YPv7DfC8kGxoxL9X3ArKE0SVzcQx0OXCYfom+qGMgzh9la5Jw6knxMOeEtvqgn4PU0P/f5Ta2UMlarkCuI6EybKTKfLyepF1JIYa77IY8DkOf1292LNnaB4CUoOjjQ+daqM+fIcrp/T/6/dGpNCIJhMY5USXzkGya9DFww5RkeRyyzl36XtZ5RuViva98XmTNG56nXHIe9ljJbQ93DRvuPH+z567+u5F0eIlIWKYShQ5fv//ScjemzpmANWvWDLn9d5P58+djW+cenPbzz+Zke3+57E5MKR2fs/07ErAngIZhGIZh5ByH3GkA7UlXNqYBNAzDMAzDGGHYDaBhGIZhGMYIw0rAg+DKi9H5oWMBAGWbW2XjLv86vbaLYBIVgRWwvkXrCunnVJ2ME4r3eWsRrX0JyVoiqPKRVGkVcyb0bdp6hTRN4dQ62UZxRTomLjng7Qj0/oRVPpYs0PpA1kxpzRyvg20flLUDW73obXedtSjzuXyljMBKk14vqlCaH4pZ614io9OK91D8VrfS09F8QJ8aIx6zOT4KLNglbTlcix/LaLyy+iFrmVBHyJGOLBxfK5p4DkT100Rb0O3HLFHxcmK5Y7wFiVuzXjaGft5EM1TMWbO3nXETZL9C1r7tkbY5QcEwOslmsttQkXtsSRN2KUuNJj/WQmNYKo+/2Laas/GO3X45peUMWL9VIPVnbrrf90hZhLB+S2soWS/KVkL92yAN3R6yMdF6uhKv8wqUNQ5IX+dK1TWM9K46Vg8UbZg0qXg31rhNl9ewcDudI7U+9iwiKxQASOoperCpVa6f5k08XsUevrQ287H7jAWiqeQPr/lVqHFOTfXbQ1rZDpFukeP+AAB0rRW2Vj3quk5/A5L5M0RbQFY/0Ri1P0SsrKWCOh+LmaU4pPHU53U0zs/FIJTPfQ6e2h9tl/yv0k/mApdDGxirAWdhTwANwzAMwzBGGPYE0DAMwzCMvGBG0PnDbgAHIeiNUbbpcFlOW0KwHYa2aKGS38ELjhdNFfeuzHxOTZ8qvzbZlwDcy5tFW1gvlxX9bPGO+y1/NT7zuWxipViuYI8vRQSq1NF+LH3vqY2ijUsmcbtMmohm+bJfmGXFQikHKVmo6Jvh3fELG2UZifvSWu+nZt39O+X6yWE/UGUkLvs2nSXLLtUPUXmrTKUcsAVNJC9IIaVsdMyTJdpStkrZKst8qckTM5/jQr8/XSdNE8sV/5HSHcZVirbeY/yYlKzfK9oSKm+KBBQAqUm+VORaZTJDPNWvM1JlK1HK2+uPT2raFLnt0WQl0aCOD50jQZNMIeG0jGB0hWqjvhQMnaTTsWiyaOuo9WNb++g20SZsQaZSaVL1y5E8wI2TZdcw5aURrYtkSbtytS/fH5xXKdqKWnyfC16Rx0dIMZw8f5IFs/y2d6nx4/IdlbSjWpV6Q9YlvZPHirbCrZRCocu8bHmk7E+4XByqxCJHxy5oUQkVbMVCJWxtqdM2y8s+ykpkW3SIpCP75ViCypvJIXUtmjnN93lTg2xjaYmy5WFLGlTKeZqQfITTmKJ2dS5V+f3pmiCvN5g4N/Ox/MlNcv10rY3GyeOarvJyh2irSqWhc6T58hNFW82vVmc+9yyeLdraJvfPo2RoBYZxlGI3gIZhGIZh5ByHIIc2MPakUWMaQMMwDMMwjBGGPQE0DMMwDCMvONMA5g2LglPMnz8fja/vwmmjPwFA2pYAMpIqUhYN8X6v19GWHVmxPYSI5hkuVotsX/o7R3YOpBvRtgXxfrJs0bFgFIGUFUnE8Us6TorXobd3oDXzORrGgiJubRVNwg6B9DlsoQMAIF1hvHO3aOJIKn18eLy4j1nLKksNEeOkLWloWR33xOuMW7wOMysqi61s1PEfNmaNdGRZ0U+stVP2QQEtmxyU+kARIcfbVpY9UYXXRem5LS2P5HxzfUonNeT31HJ0meJt9//C71+itKpDxWpFSn/I/dT7w8dRj5eIG0xUfBmtU/dDRIHpecPnk5r7rtPbk3AMYZZFFGlcY2VJxVF6Oi4xpv1LTVCWRM1eE+rUOSL6rGLpRAQjoS2ihL5aXW9EvKSe6we9JlDrCnlOxXqusz2Wjpcb5rwTekGOe1T7E9B13bUp3eIwsaHiuqj+PPM4Z0UWJkPHWcZsNaWvi9X9P/9ly39h6szanEbBNXTsxV/95PKcbO+Fz/8M08rGWRQcYU8ADcMwDMPIC4lp8/KGaQANwzAMwzBGGPYEcDCCwLv6q7Iol09iSsMAZDJHOFa6uycN3p5Cl4e5/BBrB3cqVSUtcntcpuAEEU4P6V8/udUXyJIPJ25kpZfQ97KSBmgfnLaIIad8LlkBMoFBl+ECXmeRL2/GKtmEy7y6dMMJH9rqJb3N25Xo0jSX71NTZJKB20dpEirZApQSoUt5PD+EjYqaU+ld3s4hqpGWHVwSdmpuJB0dtJgsWXEpLKukSfYePJb9v/ClSi4jBZEsb4n5oEp+3K9IyQOSLv+//eFkEjodgY+PTvEQyR1aakF942OuJQBcvtMlZi7fRzOUHU6jn1OhaguoJBdv3CLa9FwR66Tj7FTZMtRyiIHltHSApCupCeNFG89LXZrkcUj2Swsa3rbTJUbqZ9ZxpVI4n/Oxup7xOeLUtYjPg6RN2sxwKTfU84askrSMIaDklrBs6FJ4VlIH2+OMosQVZanD1z6dXsPriPdIKyues2GJKitTyT7eI22huFycqlByIZauKHufgQSTwNRgIw67ATQMwzAMI+c45M4I2m5vs7ESsGEYhmEYxgjDngAahmEYhpEXzAYmf9gN4CAEgde/BSryKE26C9bdAUB6h9cDBYVD5+okylomqvHbCJU2iO0wIqXlEbFdfaSzmag0P9t9v7RWSOxfqOwb2AZGae1Yb6IjvVg7FFYp/QxbsRySWp5Uue9LQBYHIcXOARAaukTpA1k/pTWGwpZF6ZtS03y8mNZdiWO5v1W1kU5O23lM8RFi8bYd/vfVUucX0vr1toXFyTB2K1p3xdYbodreUNsGICPYSMOWpSOsrvbb6pAaULak0TYwrOXSGjChr21RGjDC9al1jhraRonti1j3l1IRWxwFl7WvrGndIW2HovFeE+r2yWhDPncDrbuin9laCgAiivHLspahfQjDoQs4PGeFDg7qnFS6OBFDVlMt22huJko7GPE6lYULnxfielajjgGdu4HSjgYcE6iuRbx/rl1eU1i/FzfLc15onDuUVpl12Xousn0MX+vKy8RifG3V5whYf5hl2+XXqeciaxyz7FzYdqhcXg9YVxiomMWBqEC3V13jjaMeuwE0DMMwDCP3uNxpAE0EmI1pAA3DMAzDMEYY9gRwEOLSQhw8bRoAoGybKgdRKcIp64DUeO+cH0+S5ZOQSky6hMEO+KG2AeEyjyodg8o8B0+amPlc8WdpORFSKVQ7xCe1vqQQjK2U32vzfc5ysqcShi5vhbNn+LYDsnySUDkypUom6fF++93VvhxY/uI2sRzbGOj96V4yz/c5kv+zLP3Ler8KbfVC5a2dnz1GtNXd25j53HLqZNFW+RqVlTY2iLakyZcE+z640P++R5bdwuf8cvHJx4m2wm2+vBWmZanIkR1GUCGlCgGPbST/n8fjHLz4ulxnL6VLnODHIdooS+0ixaFWlaK2+zJpsmCmaAq7SKrQKst1PF7RMfWiLV63MfO5b95U0ZYu9eeTMohB0kCl93oqydPvASAki5hwnDx3QSXA3lPmi6biLd5yqfd4KVXoK/fztPQPq1XHqLSr0mVEGVuV0ONj/L4HqzZkPg9nVZLMmyaagnZ/HQnbZfnZ1fvzwh2Qx6d3zuzM56JVDfJ7dF3UVkkBJ5ZMpfOuSUoADi3x41eyR8lk6JwJelSqCiWU7P2beaJt3J/IEkvbdlGJWyfPsCVWqk4moiRUXu2r8Z9T7SplaZuXC7WcPVe0jWrwJeeCrdLOheUJOkWl6xgv7ylZL7/HySDpGmkDU9Du19kzS+7P7g/0X2v71uZDixfkUANoWkONPQE0DMMwDMMYYdgTQMMwDMMwco75AOYXewJoGIZhGIYxwrAngG9A1KqsN1hrdVDq4rrne41R0fZW0eZI9xco7Vs83es6UnvU90jn1zVHamtKV3ldVsVqsqBQmqyALA5ckdTMhc1e66Kjn0Dawb6FM0RTwRrS5al1YoePNtvx+WNF06SHvG4lURFI8TSvySnb6vu15zyprRr/e9q2ssko3uPtFsKD0nohYTuKfdIag20zWPMHAGv/0esr594ovwe2/jh2llzndj8ORXtJr3dI9svN8Lqu8DUVGTaR9DrKZoQtb4LpStO4xmvmus5aJNrKXmjw36tVejfSn7XM8jqiym0ykoqjxlBaLNoCiqGK1jaINnBEndK07r3M6yTH37NJtLG+Fluk9qltmY8QK9khLS7YmsPt8t9r/Rs5JsUtfr/3z5XrmHKnP15Fe+Q5z3O4UFvqTCL7mPlS0xi83uD7qO2XSGPWOVVaGRUd8GPGViVJS6vcNq3TzZB2K3GZP5Zdk6RWrPjRl/0PysKp6AWaf3pf6Tzc+kmpMat7wu9D+yS/7apX5CpY91fQpOLedtO1YvIE0eTI1mT876VWNb2TbHuU5c2BJf56nS6ST6HG/OxZv351nT/4AX89qHzVXzO7J8qxTJ88zS+3Rmqhw73+e26U/HuQVPqfwy07RVtxo9cx9tTLvwdFW73eMeqUesSkptKvs1uOQ9+o/mdjLk+PgyyBLn+8b54A7t+/H7W1tQiCADNnzhx22Z/97GdYvHgxysvLMWbMGJx99tl4+umnc9RTwzAMwzCMI5v3zQ3gV7/6VTQ3N7/hcldffTWWL1+O1atX44wzzsDixYvx6KOP4oMf/CDuu+++976jhmEYhmGMaJqamnDddddhzpw5KCkpwZgxY7Bo0SJ87WtfG3T5Bx54AEuXLkVFRQUqKiqwbNkyPPTQQ8NuY82aNbjkkktQU1ODkpISHHfccfjBD36ARFW+3i7vixLwH/7wB/z85z/HF7/4Rfz4xz8ecrnHHnsMt9xyC8aOHYtnnnkGs2b1l9yeeeYZLFu2DMuXL8eyZctQWVn5jvoTJA6pzv4BdiWyvOl27xvsKwCA4jXeWqJzobQLKdriy4o6VSO1l8qwuix6wpzM59LVu0Qbl31aTp+W+TzmoXWyz2yb0ifTKnpn+HJNQae0HGALiujZtbKNXe5LVHlwhi+t1P1GlmS4BB1WVYo2ts2Ii3w5Zfwjsgwiyl3K1T7o9n3umCdLUSVkM6KtKjjhQ1u9cNl394fl98Y/Rdt/bb1oA6VzdFOpLZkmy3qlz27OfO76a1lGLt1ANhYVssTkyP4i2CjL1iGVlUsfXyP7RfseqyQVTgyo2ESlw77eIZeDsvpJ03/komPk03wufwcl0rRl/N3e1gTKiiVe78coOHa2aHOcPLFdJnXElGwRzvNjO+YhdaxqvT1R6XpZIotpnA8tlXKESjqfDi5UfaayYuXd8hhwyo5O5wlTfmxL/qSO3expg34vnFonl6OypVPJHCmy4ileKa8VnF7jVAJG7yJ/LItel+dkTHKE6Xc3ibZk49bM5yqyiIpf3yyW677wpMznnmp53S1P+Wum2yavg2yV0vFBaeFUStempFFa/1T9nvZdSSEcpexoyU7Fej+2nVPJmkm9yzDqVX8t75tQKdo6J9P5+YQ6Bu1kv5XI+mjbcf6cLP/9KtEWk8Sld5a8ThWv8rKZzg9KOU/Zzv6Oh/JPQ85IjkB7lhdffBEf/ehHsX//fsyfPx8XXHABDh06hLVr1+Lmm2/GihUrxPI/+MEPcM011yCVSuGMM85AUVERHnnkEZx77rn44Q9/iL/7u7/L2sYzzzyDD3/4w+jq6sLixYsxbdo0/PnPf8Y111yDp59+Gr/+9a8RaAupt0jenwB2dXXhS1/6EubNm4frrrtu2GVvuukmAMD111+fufkDgJNPPhlXXnklWltb8ZOf/OQ97a9hGIZhGCOTpqYmnHXWWejq6sLvfvc7rF69Gr/61a/w+9//Hg0NDVlytPXr1+O6665DUVER/vznP+Phhx/Gfffdh1deeQVjx47FNddcg02bpOa5r68Pn/nMZ9DV1YWbbroJzz33HH79619j48aNOPnkk3H33Xfj5z//+Tvel7zfAH7729/Gli1bcNttt6FAPc1hurq68Mc//hEAcPHFF2e1D/zugQceeG86ahiGYRjGu4pzQU7+vVt861vfQnNzM1asWIHzzz8/q33x4sXi51tuuQVxHOPKK6/EySefnPn97Nmz8Y1vfAPpdBq33HKL+M5vf/tbbN26FQsXLsQ111yT+X15eTluvfVWAMCNN974jvclrzeAr776Km688UYsX74cS5YsGXbZ9evXo6enBzU1Nairq8tqX7RoUWadhmEYhmEY7yZdXV248847UVZWhuXLl7+p7wzo/N7Kg6vhvrNo0SLMmDEDq1evRkNDw1vpfhZ50wAmSYIvfOELqKysxPe///03XH7btn4Nw2A3fwBQVlaGyspKtLS0oK2tDaNGjRp0uTdD0NWH0gGrjOpK2W/Sm0TVMraN44NKN0hrjIT0Ti4txRb8fxOO8wGAggZvXeF03FeHX2dpk19nUC6X45gmpKT+sKDZ2y3o2CHui0tLWwER/bRXWarQ9+JOaaMTkrYmUfFLpTvIvoa8AbQNA5N0qPWT1Uep0j4lPWQ7skNqxThSTsS7AcJ+RWj+0B8bOEBK2WYwrMEJykpFW5q0ViWbldULxw02St0VR4gl6tilyPonGCXnA0hjJLR8kHY46CNtWoHUZAVs/aM0bMLXYaeKq6Lj7zqUHQ7Nh1B5Q7DeLWqW2rQqjrDKilmkCDHqS0zRfwAQkFYwmCM1Ujw3KjYrexI6BqNfHlof7FRkIdumuC4ZxxYeIg1YkdTXBk2tmc8x6cOCLmmpkya9a6pV2sAMhzvkz7W4tVW0FbT4OZUcUuckW6zskdcDoXGk+LdorLSrqnjJn5ParsoVkX2QOucDOs/L1qpjQFZDWXop0v1pfSDHc8ZKlx3RsSvd4dsC1eeEtKPJFLmvpQ1+/iUqZpPPA93nipe9tZR+DYDnaVwizwPuW8VadX0L+vuWDw3gkWYEvXLlSrS1teG0005DSUkJHn74YTz66KPo7u7G7Nmzcemll2LiRG8T1Nramrl3OeGEE7LWN3nyZFRXV6OxsRGHDh1CRUW/PnzVqn5958CDLc2iRYuwZcsWvPrqq5g2bdrb3p+83QD+8Ic/xAsvvIA77rgDY8eOfcPl29v7L7ylpaVDLlNWVobW1tY3dQM4f/78QX+/efNmpDD0NgzDMAzDGHmsXdv/MmRtbS0uvPBC/O53vxPt//AP/4Cf/OQn+NSnPgXAP7iqqqpCWZl8mWiAuro6NDc3o7GxEccdd5z43lAPvAZ+39jYOGj7myUvN4Dbtm3D9ddfj6VLl+Lyyy/PRxcMwzAMw8gnLodG0K7/Ac9QD3/WrFkz6O+Zlpb+J9j3338/oijCj370I1xyySXo7OzErbfeihtuuAGXXXYZ5s6di+OPP/5NP7gCgDZ6EvxG3xvsO2+HvNwAfuUrX0Fvby9uu+22N/2d8sNlzU5VUmQ6DpeU3kz5d6iDPX/+fDS+vsvbXqj0hYAsVVyfLIuyXUC8a49so9KUtn0QZQrlsB8f8CWTsFxNBloPlxizziey8HBq+ByVipJeuT+MLtdwuTurdEwl7tRUaakSU4nWKWuRiJIzeN8CZRfj9pLNhHL3Bx0Tt1VZ0HASiC7D888bG+Q6OeFDWb1w2Tcok7YmCY2to/Izjzkg50a8ZZtoS030KTHpjqHnfqhKhendfpyj+qmizW33NhpBpGTAdDUOemm8umWZUtjCpORlhM+RWF2gQipvJWqdqTqfZhLvkaVjLm9xaQ0AIj6uen+ohBa3UPmxRpZFef/iCjmWXEwL1sv/cTsqBya6lFvs1xOOltY/CZWgAyVVcK3UpmQfjuYAyzJ0mZLPi2hfi2xjGchYKVtIq4QcwSY/N3WKi6BWVnTE+PH1TZU3E9628jljyyBto8XX03i7lEkIGxWn1lnh1xlWjhZt8d6hy/liOUpPCktUIg7N2aJGOWf570Ogzh8+l9nGCACSBn8MojFKckKWRGWvq+PItkM75N+m0qr+vyth37vjLXc0M+C/l06n8b3vfQ9f/vKXM20rVqxAY2Mj7r77bqxYsQK/+MUv8tXNN01ebgAffPBBVFZW4sorrxS/7z58Ed65cyeWLVsGAPjVr36F8ePHY8qU/rinHTuUVuMwHR0daG1tRVVV1TvS/xmGYRiGkRvezTd034j6+vo39aRvKMpJXz/YSyDLly/H3XffjSeeeEIs/1YfXJWXl6OlpWXI772Vh13DkTcNYGtra2aQNN3d3Zm2gZvCOXPmoKioCE1NTdi5cycmTZLZpy+99BIAYMGCBe9hrw3DMAzDGIlMndpfTSktLUVNTfbLVQMvZOzb1/8EeeDBVUtLCzo6OgbVAQ481BpY98D3WlpasGPHjkHvaQb7ztshLzYwzrlB/23d2u8YX19fn/ndwICWlJTg9NNPBwDcfffdWeu85557AADnnXdebnbCMAzDMIwRw8CbvF1dXejpyZZBHDjQLwkYePJXWVmZuQl8+eWXs5bfvn07mpubMXXq1MwbwACwcOFCAP7BlubdeuD1voiCe7Nce+21ePjhh/HP//zPOOecc0QU3O23347Kykp8/vOff+cbCoKMnUSgRJis+UiNl3E7adZnjJNtMem+QqXzA2lHXLN8RZ81WkG31Mw5+l5QSjYzpcqChmK0XCJtLFKTJvg2rSMizUx8oFU0RaxNUhoWXtYpDUs42j+y5rEEpGbGkd4xUfYKIekRtRaJdYypyRNFG1t9sLYSkBFYidbosTZxGDFv1vem0TrXeaf31Dj5P0fuSxCqUzI9tDcDa6F0rFrAWr5OqU1zZLiu9UcCsvPQOs+QLJBijubTfRlGH6pJaByylqOfQ2W3E9f58XQvyVgtob1l2xSlp3VUaole2yLXweentiCp8+dPpPTCbOcRNymrJJ7ral+TaX7eRq3yfGXLmJDtnpSuOCgg2xR1nrHeLWmT62c9YjhajrPjc01pbyP644W9KgqOLYnY6kfp7jgaUl/DAo6l1HYr2pJG9IuuN2qehlu9FjbRtkBke6Q120PqA0NlqUTbdmqcxb6rG4n4EC2rdIsp/ruizt2ku9V/rVxdD3b6a1io9Ny9Jf3r0ZGBueHdNWl+o229U6ZMmYKFCxdi1apVeOKJJ/CRj3xEtA9ULtny5ZxzzsF//Md/4J577sFpp50mlh/qwdU555yD//7v/8Y999yD66+/XrS9/PLL2LJlC4499th3ZAEDvA+SQN4KZ5xxBq666irs378fxx9/PC688EKcffbZ+OAHP4h0Oo077rjjHecAG4ZhGIZhDMbXv/51AMB1112H3bu9d+Urr7ySSefg9xuuuuoqRFGE2267Dc8++2zm9xs3bsT3vvc9pFIpXHXVVWIbF110EaZPn45Vq1bh5ptvzvy+o6MDX/nKVwAAX/3qV9/xvhxRTwCB/lDl448/HrfeeiseffRRFBYW4owzzsA3v/lNnHLKKfnunmEYhmEYb5JcGUG/W3z605/GI488gp///OeYN28eTjnlFHR1deHpp59GT08PrrjiClxyySWZ5efMmYMVK1bg2muvxZIlS3DmmWeisLAQjzzyCLq6uvBv//ZvmDlzpthGQUEB7rzzTpxxxhm49tpr8etf/xpTp07Fk08+id27d+Piiy/GZZdd9o735X11Azht2jS4N2EKdPnll7+3/oGFKaRn99uXFOyVqQNshxLvlK/Tp6Z7QWbf+ErRFlJZlO1iAABU1tE2MGE1lTtVeTihslW81D9yLnhBWpWEXLYsUGWD0b6MFKpyHZcHs9z3i6ks1im/F82Y4pfTZetiSs5QiRhxtS8jccJGwVppvcHJGVnl9DnT/ToK5b6y/Ulq2hTRFm/zb5f3fXChaCva60sy3ZPkW1fCfkeVcrjs23smucC3yOVCKj+5udNFm2vzcyOlSj7pnX5/smxtqPSpS2QhjXu8X1mEUMmJy+KhTlWhklZYoSxO2G5lsnxZiy2PoGwzQOkyIaU0AEC8w+8rJsgSevc4SsSpkmXFhMr+0XTfF9cs91uUHyvk+Zls8CXhvqVybhQ++7rv/snHiLZ0uT8GpQ+vkusc5q3AcJcvoerSJJcOuaSZlXJBZf74+FmiKdjsn1oE6noQUtky0fKNGl/21+Xb9C6/zqh+mtxeH5VMp3tbqPTq18Vy6Q+fmPlctEvut+OyqCrJcok2fbL0eCt43m/DqeQhpPw+hKrPyeaGzGe+ngFAusbP977RZPWyX1kl7Wv1XZ4pz4OAUnaCDdL6KeI5rOU1JHcI1ffYhijqUZZenMBUJa2F0gOpIXoOGUNyxx134NRTT8Xtt9+OP/3pTwiCAIsWLcKXvvSlQW/MrrnmGsycORMrVqzAk08+CQA46aST8PWvfx3nnnvuoNs45ZRT8MILL+Bb3/oW/vSnP2HVqlWor6/H1772NVx11VXZ5/zb4H11A2gYhmEYxsjAIXdG0O/mZoIgwBVXXIErrrjiTX/nvPPOe8svqc6fPz+jE3wvOKI0gIZhGIZhGMY7x54ADkbikDrYX9Z0umRKZRHt/M4O/j3zx4m2Eiqt6Tcv3Rh6K0+9Qccl4aBYBcNTybFnjF+uqEY68Tt+g7dPlgp7xvtyV+HGraKNHel1YHlICQI6BB1pKtHotzkLqbymAuy57Ntd7T8XqjdvQ04s0Wksu/zbll0nTRNtJVxyVuMQ0VutSY96o/KQP+bJNFnuDGidOuFDvO1LZd+eGjlvSvhNVZVk4Yrp+KvxEv911m+BjvJlHv0Goov9XMyaw7welWwh1k/fSw4dGnI5p9/QpHHXBQxRoutVaQ+c9lEg29om+fOpWJUtRZmE3tIMlPyA5ykfb0C+sRsXqzc9J/mklrhE9ivV6cdZywNEuU09AknPpLeAX5F9YYmMSH9RSTCc1BL2qrlB45A+JOdGxCkbKo3DcYLIQXlceYzQImUzvO8RvRHstKShlNY/UZ5nRXxN0W/b03pSbUpeQVICpyQ7oOtIoMaB9ydQ14pUK5Xvh3lzlt9kDrvkdap7gj8/i19X40xyGy0XSpfTdVElovCb2V1TZIm+hNI/+sbIud9X3j8fXJ4eB+XSCNqQ2BNAwzAMwzCMEYY9ATQMwzAMI/e4HD4BzJHW8EjCngAahmEYhmGMMAL3ZnxXRhDz58/Htg17cFrtpwEAiXKPF1okpW8TditKP5N0D62n4u9pq4JwdIVePENMWhth56D0YMMS+P8DuD65bdbW6KQJ1gRGKhMxafX9ikgjBUg7Dz1+qfFeNxk3e51PVCcTPeLde+mHYfZVa2QIp3Q9bCeTdVxnUN6isk0Rx0Bvj/RBok0txxrKRGmrxDEJh9b8cHJB/xeH1pxyWopOGghSfhyE5UiT1KaK76hts45Qj4meY0xqAunplOWRWEda6qk4HSG9r1ktLPcv851JE9UvSIepjkFMtjZZ40xoPSV4Tik9JVt26D5ywoe2euDLtUhnydKDUeqJ0pXyuZxlXUT2NGGx1oeSpnGY8y41bbL4mRNyeK7rdKGQNM6cbKSXHXZOqXNEJMGoucf7x8k2AJDesZNWIo9BNIoSPjjZRCVs8N+A9G6pPxRjqxJEeHtJh9SAcsKUTnjha3kWNMd0Pwf0vH/Z8XNMmTUOa9asGXo97yLz58/HpkNNmHbL3+Vkew1X3YqZFTU5278jAXsCaBiGYRiGMcKwG0DDMAzDMIwRhr0EMgguCuGq+kuvuqwT7/SO96EqrYnkjFmyDBK8utF/VtYyAVk46BJwQOXhrEf+RM+HFmQ+Fz3+qlwHlUGyLGgoHSHYJJ3luTSRtCu7iCoKilclLC77JntkcDrbrXBJCQCSalrnZCp1rNks+8xWIsqGIeIy8rhK0YZVPiEl0lY5lLgQn3ycaAtf80kQXX8tUxVKNvvyVrxFjl8Q+rHmhA9t9RK87PvlTpRpEqntVNJUsgJRVtJlRDo+2qaFkwZibdlB5dWg3M+9sFOl11ApUs+pmLYXqUQPcczVseOyb0Q2QwCQ3uvnESfuAEDXTL+Nwida5fZornBpkpNfACXf0KVcKsmlTztWNBVtIHuNqUoKUejPu+jPMglEdlKqcNhiKd24XbRxuTumVBWdzCGsf2ZPE23Bfprrra1y/VyGb5Ll9HD2DP/Drr2iLSbZR6JsWlgukiojeyw1ZzmxRFvXpMgORV9TXExl35PmyT43+n5mXT8pLcWp6xtfk/XYsoVQMobslpTVC7bT34oF8rxOyN4JK9fKPtP8i3TKDl0X0arOXS5Hq+0F2/w8TSbKedo7tn97yb6hJTPvJWYDkz/sCaBhGIZhGMYIw54AGoZhGIaRH+w11LxhTwANwzAMwzBGGPYEcDB6ehG/3q87S02UNiYhR/Mo7RNG+bauCVIzVbbR6zpYdwcAHZO9rUDxQy+INtZ1BNPqZF8avI4pSZFO6eT5YrnCvV6D45SFQm+117OkXu0UbWwJkpXYPcHrSJLNjbJfZKMSVlWKtmSctyAIdkuNUTzaj1HLbG8DUbOxQCzH/2vRdi4xRen1HiOPXQFbV2gbCxqXwm2yX26i1y2VbpB2KEmZ76eeKxwv5tq87QfHuwFAQvYUQvMHYO/ZXu827oEtok3bU4g+s03PmKohl0tNkJGFjixiXCfFds2bIZYLemnf1ssIQbbiSO+RWjHW2mk7ioii8zgOq/8Xfv71jZearO4qfx6WjK8VbQlbjXCMmorYYg1bx5lzRdvolV4P1lklj10RHQNXIP8/XXDA74PTkWFsv5TIucgRYpEao4S0fWzvkihLFdZy9qhrUekOf0zY0gQA0hxFqfqF3b4tUBYxCKlfKhoyTEi/RzrfSNkt9dH+hI27RBvoPNPaYY5cdI1yvjmKBnSJitw7yWsOi16X23MUDSdi7gD0TPHHpKDFnyPB/la5jnFem3romErRVraDvqfGma14tN41WOfPNY57BKRVzq5l8pyvu4esuQ5Ka5nuOf1zwEX50eKZBjB/2BNAwzAMwzCMEYY9ATQMwzAMI+c4ZBeX3sttGRK7ARyMKMpYZSQHpYWGI+sF7YYfkbu/Ms0Qj/W13UpZoz8MsTobuAyn7RW4RFO2wbe5bTvVYkNbbxS3+jJFWp+JQ6QoAIAjiwOo0opIDFBJKkEflRh7pOVNaoPvd+1uP4KxSlEQ6QWqDMolxpL1shzEBaesZAZOQEjLEiP20bhXyJIZGn2f07psSaQ4VaVLpS9weoGyeuGyr+tTNhPDXDk5zSQrVYOPq0oP4NKhSKRYK8vPolScHrpfWbZDdM44fW5xSTAZeu6lNkgLl9GdZP2j7ElEKgknrqjjz7YfFX+RJe30Pl/6HFWqbKHIKqVQ2TsJ+xhV7hwWmm+xsi4JKcWDE12yUmjouJatlSkUMZWRs2ynSPaRVZqmZXWCCF+LuOwKAAktG+30x0db0BTs82VLnV6SUMKLvu7yMYhUoodIvVH7U7DSW3O54eQUap4WbaLzjo65Lk1jr+9X5ZPq2kDnRVpfw9gWTJ27AVnE8N8GAGL/6u5pkE0kcQjVNaxyZf8xCbvfwhw1jgrsBtAwDMMwjDwQ5FADaFpDjWkADcMwDMMwRhh2A2gYhmEYhjHCsBLwYCRJRiOm9Sb8czRaxvQITYuK3xLrGEYPpKN/RKyWiiRiHUm80Wu0RKwVILVi2gphh9ewZcUOkU4qVLYPyTB6N7YgCZXNBOuDhD4HQED6yqDbb1vbMERkLcMRYf0b8PuXqCgr1jclHVKnJOL42mQsFI+Da1Y6zGG0Q6zLSu8km4lhtHsi3k2vX0eGlWmlqYf3Tx9X0L7GTdLWhrcn9K59UivG2w7JogNQ+kBtt0PENE/6t03/Hx1Of6r1m6/4KC19jsQc8cfHLlSaOdJPJSpii61S4nUbRZuwvNkq7ZB4LHl+9W/EH0s9ttxPrWkTui/W3Sl9G28vrWPvqI2jEwEgPtDi26qklUjClip6PtD1QZ+vwraJ9YDq2pBsbvDLqWskrz8aUym/R8crSxfHmmd9rpKmNtaaRjrXsuYp/zzMuRwU+HEI1fGP6TzX57G4NnXLuc7zLSyW6+S5H+9TVlZ0vFjvDACpKf32YkGu3sZgHIBclYDtLZAs7AmgYRiGYRjGCMOeABqGYRiGkRfy8eDR6MduAAejsACYOQUAEO5SyQ8H/eP/QKUJcAIGlGWHsGxQViwBWTugRDnss6VCgSytBIW+pCkSFtRr/sIGRJXWuEyhLWJEaU9tO1QlVEakl9TJdIzgEJWRVJkioJJGWOb3J6lVSRbkZK/tL0JKgnCHZOkmoGMQq9Ikj19QIR32A0puCTbKMl/S5UtOuszDx1WUtLSsgNNMhil9DkdQN0H+YsNm/5kSCQAAlHKQVaKnsm/IJSVVKhbjrqxkOP3F6XlCpc9YpbikJk/0bdrOhaxmAnWOpGr9/uljLmxT6FhFM6fJfnGShrZUYdmEKh2LcVA2MMLOQ5U0XTKMDITLvmPl3A9TbDtCJWBVwgzZLkSVPrlE66qkPCCg8QtGy7kR0Pi5tFonXwvVfAuafVlZlIeV1U9YS/utUzVo/JLJ6ppC18hAlYcDto/REgqa34GycImb/feyzmu6NvH5oq9Foi/d8viwPVJYI8v8PEaxsqBh2ZGbopKHtvmycqDmIv+t0OdP15x+GUDSZLcDIw074oZhGIZh5Ad7Apg3TANoGIZhGIYxwrAngIZhGIZh5IXcGUEbGrsBHIy+NIJt/VFnOt6HdTfaZoR1fpHSdYDsShyUbo1ij+J9UmuVqvP6MygdUUIxa+G4Gv97FR8Usj5HRSyxfkvHTonopGHsTlLKSoLtIpItMvaOdTI6QiwcR/vaTlYFKjov4b4o/Vm6Ybvv1ySpi+MILK2RcaRv0tpOt8Zbf4Qzpoq2VIf/Xnq31K2xrQLvdzBKagwdjXuorDfYUkfryIRdBGv+IOdNerPULfLcTNql5c1QViyB0oAKPVWXtN6IW7zmS1tcCK2a0juyXUk0Ro4D25NAn5McbabsiRKy0eC+xGq8hM2R0nLxuKemTJJtdPw5ag6Q0X2ONYaAiuNT5xZdR/T5E5EmkMcyNUHqwfjaoOdzUOD1YOnX1TiQPizeKS2JItKZuh27RRtfO0IV8chzTOgi1XEMSXvLOjsAwg6FbX8AWUEMlIUT66Gd0pwKW6Bg6OtiovSVQ53XWr+d7PHX/FDZE0V1Xu+abtwu2sQ6tWa71Gta41XrZBvbzsyYIvtCFjthrdRoFu/sv8YE6benPzaOXOwG0DAMwzCM/GAawLxhGkDDMAzDMIwRhj0BHIzCAiT1/e7oYZt0Yg+5LDJLlQObWjOfE1VOZbsNtkkBgKDclz5SZKEBAPEu2t6xM+U6K8mmoYvsCCbKkiz2Ulk5UIecnfmrZJmCEx20VUFC6+SSL6BSQhbPl33eTKUjZY2ANloPWcL0nDZPLFa81Y9tsmmraIvqp/n+t0oLBS6taDf8kK03VJm866xFmc+lj6+R66RyblQv50PQSWUrstfQSSNsCZGoBBkuhcaqtC8SPpT1Bpd9ez+ySLSVrvapJFFlpWhji5q+ub6MlHpBlptEaa1ClhhTrX5f9dyIanw/03tkyTyaN9t/b4M8rsKKgywtACCZ6OUPYbssRwfbfNINl8zDY4+R6yerkq5j60RbyTo/ZxNtXUQlufQuVTKl8ykiixsAcC0+vSJRSRMBlXnjKWNUP/28DajMK0rkAAJOIVHl1KSSJAhTa0VbsLbBf66U33Mky9AJLyFdw3r/arZoK97oj3NSRdsukscRrTRX2uX5yWXsSM11t9WXUANlgcVyGN3nvtOPz3wu2SSlN6Isq22O6FiKUrgeE7KkStfKa2uwakPms5bQ8PyOd8pSu0ipWXycaIv2USKKuoaF0yb7HzrkObLr3H5ZQ3qXsj/KEaYBzB/2BNAwDMMwDGOEYTeAhmEYhmEYIwwrARuGYRiGkXsccvcSiL1skoXdAA5GkiA81K+TCJQWRVhEdEh9IGu79Gv/rHcSlhOQdhGB0m5wLFRfoYr36fGak2FVFPzav4pfAklKgnKl+WH9DGmWAACh76eOnmNNGw5KvQlGk2ZO2SvEU70WhjUsJeulVqxvotdFRbtkn4Nur6+Mp0prjLDB62mi1NBTPz2+Uvxc9kKD/2GC1EyxVslt3yWaHNltcLSdtr9gCw2tw5QrVMeOdXGHpK6QrV5Y8wcAXcf4cSn80yrRxjFUhZtI+1Yg9VqO50oir6wu7edlqKK5HGkCOX4PAED62kjpWNkiJhwv9ahhA+3fRHV8SJclYu82NYjFEprPxQ1Sa+lozobaPojO+UhZbwQ0DjqijvultWnxdq9bLChV0ZBkZcK2JqHS+QnrJ3V8wjaKdCtQ8WWkaU10TBzZu4Td8trHUWOF++U1U0RDdtE6Dg4TJ6mj01j3mZbjxdfkpEKNAx8Dtc6S1/18c8rKSKxDW9KQVjmgORWklIaOtp0eJS2CikkLq8NwRQyijrokTXCwQVoEgfTjcbmKr3tpi19lndSjFh3s336ghtU4+rEbQMMwDMMw8oS9BJIvTANoGIZhGIYxwrAngIPRl0bS0F9y0iVZTvsI1aN7cClXpT0gpCSIkhLRJKwkGoZOzohaZRmWkwAclZwDlSyQUEKFU30OqZ/prTIxYrj0j2i0Lw/FlHICyJJWpMuW5FafZcWyxpcpwmpf5nWdyopnpbckSVSaSNLll410IsFBb7HidFmMSkzBi6+LtoBK6HGDdO1niwg9V7gUHu+ndAyVGMCl3ViV2lMTqBSq7CjiJm9dIcqbkOkL2uqFy77hWGkzwjYTbDsTKOsVR2WqLBsTTntokvYaPKcCVd7i8jOXfPtXROkLjarUzgkVG6V9DJfQuMwXjZOlYk5xSSnZh7A1UsdYlBV71Vyk0qdTcgdxbqlzMprs00bcdmUDQvNb2BqpYyCOv5obMZXhg53yT0Ca+6mSbUQyUKhKk300PzbK60hCpeqgk0qt6voSsPxBl8UpVSPQ8g0q33NqDwDwWnSSjqPrg06QEctp2y6aA3z8E7V+lu8UUQlWL6tL0yKBJ5Kl4zTZwohrA+T1O+VUEgidW079HRn7TP94Rl2y/znDtHl5w54AGoZhGIZhjDDsCaBhGIZhGPnBngDmDXsCaBiGYRiGMcKwJ4BDER7Wpyj7i4B1HspShfV1Ou6LdR1a+xYVeX2G1kWxhknbrSQH6b9ObC2iNSWkkQm0DQzvj9b8sb5N21/QNli7BShLBaXXcZ3DxUmRdQXr98JgyOXigzI6TaA0OUJD56Rei4+z61V6IO6n0uEJbZzWhDJ0/PV+BynattI0chyfbuPjlaUj4n7q7dHxSpQeSOiraN6wvhWQ85Q1f/0rpXEYRkeqx0HY3AT6/6Y0fn3yuIb7WzOftZOFXjbze2VjElL0oNbJ8nmedexYW6XbWO+orZ9YA6a1aWwZo7SX6JL9zhAOHeOVpfvVtia8LGkAQ637TNN8y7L+IUsaZYEF0v0J/Zw6Nk5FHTLiGqPGmef+cMdHf0/MN3WdB82PrH4W0nyg/c66Durrz1Co65vrHdq6hudpQlGAgDoP1Xkn9kFrbzPneZ7exrUouLxhTwANwzAMwzBGGPYE0DAMwzCMvDBc4cR4b7EbwEFwZUXoWbIAAFB4QJZcoo3eniKokFYvjkombo+0vxC2KcOULbU9BZeA+lRCRcF2/yi/b4q3KkmtbRDLBWQ7EqgyskgyqK4WbaKEWivTF5LNfhvReGlHEHSTrc2sOtEWvbzeL6dKEW4CbX+nH7/eYyeL5Qpf8VYfgSqfBMfM9D/s3S/XTyUgXa7hMl9wwjGiqWWWH7OKTTXye320zl5V8tnTnPkYTaFxUGU8Ts7ISmOh8llYruYblYeyym5U3uqbKy0hOOEj1mU3KvseuOjYzOcx/+9luRwfu+NmiaZgLaUOqEQPPkcGrJYybdO8/UmqqVW2sX2IshZKT/XnTKpYlqPTjXS+soyhWtrfoNj3q326PEdKH/a2OdGkCaKNrV8CVfJLT/H9CrtlCT3c6W1NtAWJSOOoVMd8hk9xCOhcSk2W6Q6cJpHMny7aumv89aBsgzz+qXa/78l+NTcWzvH9VykUbEPTtmSmaKt43lsnpSf560jUopJAWEJTrqyy+Nyikj8AJGRDFS6cK79HlkGJKlt3Hz8t87mwVSWPvCKtoEQ3Z/vzKdrj7Z3iiSqhZpe//vROl9f1ws2cQqIKcZQoE6+X9jHBdH8tDJQ9FiePJKPLRFNQUe+3F8rt7Tiz/9qR/q+hZQTG0YmVgA3DMAzDMEYY9gTQMAzDMIz8YCXgvGFPAA3DMAzDMEYY9gRwEIL2bhT+zwsAgFDbN5B+ItklNSzhaK+fCSZIzUey1etgUkrn50hLGG/YLNdJMU46oipNGp1UMVnJqOgnYeGh4qoCsoRIN+qYM9LXtbSIpogixNK79og21lpFq6WGhbVqySEZXxW2ej1iMslr7aInVonltNUH49aQLmqa1L6BYsmCSGrF2Jon2ijHoXIbLdsnY6E42k5bi7AdThgObXUQ03EMO6V2x82b4fu8Vo6lo77EKnItoH6lXlgn2hKyvNARbzxXWPcX1kh9KMdjuZfk+h1FhiWNylKH7Eqicrmv2OL1ejpWi49Pqm6SaIv2+3nj1JxijWhY6ud6rCPjqM9lTmpA2V4jKVcxfhv83A9VNFdqm9empfc1izahY9Qq+CavHXM71bnFqyBboPR2GY/H+xMdkNepEtIOKlOo7Mg63vbqTf4HraElyh9ZLX6O6ZoT0XHVFkQRzbFERy6ydZGOkCyjebReHlfetoiyA1Cy2R8Tt19e32LqZ1RRIdoC1i6yrlifn9Tn1CGl1yMNbazGAfRzpKIa43X+GGj9s7AWqlsoN/eq/16oLL0m/67/mDe2xYCcwu89LsidDYzZzWRhTwANwzAMwzBGGPYE0DAMwzCMvBCYBjBv2A3gYIQBwtL+skJQKm05kkPewiVS1gsJlW5CnUDASRAqVQHkjq/LDWIVE2UZLlXmrRK4NB2Pq5Srb6ISQ6yKPvS9UJWORaKHhsrDkbLDSTqo9KlsZ0SygSrXuRJfakuP9p8LVWnNkVVKskXaUSD0U1pbIfDYJqrUxRY4gbISEW78OgmEUlx0eTis9rYQCdntsC1PFqocyPYXTpXvufSVZalD69HHgMfPkV1I1nroM5d8AQDVVX4xnWTB6SVZqTSUJqHWmWXNMwSuQpWO2YpFna84ROVh2l6ojgGXFYO0OkfG0dzoVHYhVZX+B50YQWk2OlWDxzbpkCXagErjOrXDtfllE5JlBCpxBwX+e4GeNyTD0HMxoVI129EAEOVO1y77zOdWoPtMNjfcz2i8srziPqrjGFT5Poc9aj5zapAa5xRfF5XlkeNUEp0Swik7an4HZOOkZR9iuVJ/fe6rkONceNCPn77usqVOVhIIW3oNc750jZZtEV0PghJpsdN2TH+ZOdlpNjAjjfdFCbipqQnXXXcd5syZg5KSEowZMwaLFi3C1772tUGXf+CBB7B06VJUVFSgoqICy5Ytw0MPPZTjXhuGYRiG8Y5wOfpnZJH3G8AXX3wRc+fOxY033oiCggJccMEF+MAHPoADBw7g5ptvzlr+Bz/4Ac4//3w8/fTTOPXUU3H66afj+eefx7nnnotbb701D3tgGIZhGIZxZJHXEnBTUxPOOussdHV14Xe/+x3OP/980f7888+Ln9evX4/rrrsORUVFePzxx3HyyScDADZs2IBTTjkF11xzDc466yzMnCmd6N8yBQXAYcd17bYecukwK3jclx/SWxtlG72VGSuH/YCSQaI6VVamt9OySlNU3gio1BXs3icW4/QFXTaI9/gSoEirAOCoX7q0Fu/2byeKEHKoN0uL5PYSemM4K6S+3ZeOC/ZRisdo9RbeMOHv0QxKPWjYKbfNbw+qUq7jMlytSokopfLNAfXGHm9fleFiCmsPufx8SCbB8NzQpTxHbzXyW58AEJIEQO8PqHyfVTKlRAQuNwHqWFLCh37Tl8u+cb16K/d1P/dDlQTC2473qTeXadt6ToWU1IK98o1akaQxRfaF54fr9OOnU1Wiykq/XEqV+SnRQZfhHb8pr+fzGF8mx9hK0QYqR+pjzuMSHSOvZYGj8j1dR7IkAFxeT8tzhBN3ss5BWlaUVgGgks5DVQJOON1mlkzuCRtJEkDnUtAt18/XG31eB5zioSQaMZWto7lyvETajEqQ4WtTlnSArgdZ8gfuM0lqAvVWO7/lHMQyQSjeQ+klatsRlej1G8JRDa1Hl6apTF76ipTGODqfdPKMi/L8dqy9nZs38voE8Fvf+haam5uxYsWKrJs/AFi8eLH4+ZZbbkEcx7jyyiszN38AMHv2bHzjG99AOp3GLbfc8p732zAMwzAM40gmbzeAXV1duPPOO1FWVobly5e/qe8M6PwuvvjirLaB3z3wwAPvXicNwzAMwzCOQvJWAl65ciXa2tpw2mmnoaSkBA8//DAeffRRdHd3Y/bs2bj00ksxcaIvh7a2tmLbtv7H2ieccELW+iZPnozq6mo0Njbi0KFDqBjmbVrDMAzDMN4H2AsaeSNvN4Br164FANTW1uLCCy/E7373O9H+D//wD/jJT36CT33qUwCQufmrqqpCWZnSNB2mrq4Ozc3NaGxsxHHHHfe2+xbEMcLmfu1dovQS/HOkLRRYh6e1XGwREqrX7dlu46DUZLFGK9qnbAzIUiPhNImqKrEca1Gy7A54ObKx6f8FaWb097jP2gaEdJJhLPV0bBHBmh9A7gNbKIh+vBFs9ZDIK4tIdFDWG8IWZPtu+T2yUUk360QHsldQx5ztFhKVpCJWwbo4pQ8Uc0XptViPFrIdCYCYtpdqVXY7pPPSWjseM042cCpFgbfNmj8AAK2TtU56/VrTyBotrZNlIqVjFbYWe6SukO1XErLsCLVei+fiVqkdZa2YTq/gY+7UfHOkr9XaW0eWMfr8YU1Y0CVtQIQmmM9/PZ/pGqmTesJmf54HdRNEm9AEq/SXNFsuqfnAurVgnxwjoTNsoWsRFHxNUTZNSRPZ0yidJycPQSWniHQMNc68rNO2XcOQkLY3YQsibdkypjLzObWuQbTFpCsM9LnLqSSB1McJG7JaqSt0PLbKwkv0UyWBlG/pvz6EfW/hOmscFeTtBrDl8B+o+++/H1EU4Uc/+hEuueQSdHZ24tZbb8UNN9yAyy67DHPnzsXxxx+P9vb+m6tSLdYlBm4M25SwfTDmz58/6O83b96MAgy9DcMwDMMw3iXsCWDeyJsGMDn8VCedTuM73/kOvvzlL6OmpgZTp07FihUrcMkll6Cvrw8rVqzIVxcNwzAMwzCOSvL2BLCcbBgGewlk+fLluPvuu/HEE0+I5TtVEDjTcbgMMko5qw/GmjVrBv39/Pnz0bi1Celp/fYVqZ2qFMWlUFWmCISzvFoxP8rPKqfRo/fyodMrnNp3fpQfjfOu+uldqoRJpRWdSCASI8bK0rFw+9f2F1Qy0akKnDSQHGjFUGRZfVDyAKelOB2WzmVKbX8xwY9D0KRK5tQXLlkBsqSZLJBWEtHaBv9Z2XJgpy+ZxfrJM5Vhosm+bOVUAgpLB6JaVXbjMqKWFbDFyTAlwERZdnBpyjXJkqlIeCELl6RRWUdQuUtbvQiLi5nT5OqpHK3LqfxzapK0QxJlTJ2qwaU9NR848YXnW9wkS/ksm9Bjyf3SqQ18PQh1csZov6zbIc9JYUmiynx8bnHJF5CpLjHNr6zkDL42FMu5znYlXDYE5DyFKmlGM6b476n94fSfqGzo8jqPs7ZD4pJzVCrTKsR1V1lzcbk4PmGOXCedu7FOaqH9y0rVoGOSZXNFcyw1liQu6jjyHAtValTEUgiVDMVJRK5NpQtN8xY7yfZdss/R0EkeYj5QQhEA9JUd3vcwD3YsuTRptieNWeTtCeDUqVMB9Jd0a2pqstqnTZsGANi3r/+PyZQp/ReflpaWzI2eZseOHWLdhmEYhmEYRjZ5uwEceJO3q6sLPepJGgAcOND/9GbgyV9lZWXmJvDll1/OWn779u1obm7G1KlT7Q1gwzAMwzgScEFu/hlZ5O0GcMqUKVi4cCGcc5kyLzPwO7Z8OeeccwAA99xzT9byA78777zz3ovuGoZhGIZhHDXkNQru61//Oj7zmc/guuuuw//+7/9iwoR+S4JXXnkFN954IwDgyiuvzCx/1VVX4cc//jFuu+02fPKTn8QHPvABAMDGjRvxve99D6lUClddddU771jiEB06rDNJKY0ZaVGCAmVHwzq5WL5SH7IliNYxcoyXslvhZbM0YBwFx7ooHXPGmjm5ZQSsW1GaH96HrGgm0puIGCgAIfXLdQ+tk+QoOwBI2shGhzQ52qJBWjuoeDS2nFBPltmKJ+lSI8Fari5lT0L6meCQlB9wxFKoIqOE7QSPg94f6pfT84a0XcNZ+GjLG9ZFaTsPoe0Mhv6fMUcIZlkXsV5IbVtoNJUFjdDvuaFFOVonKTRzOgKLzq3hYruEBY3qs7DXULYcaYoaCyN1brEOU2stC/y+Zu2pOEeHjpQMtD5QW5kM0g8AcB1+Oa0rFdcmfexYC6tsmkK6/iRqnvJ56LqVRo+3QfrnUFsQ0b7qGDpxfRtiDAAg7JZtySDVpcyydJzjA0PbNGVZYJHmWdjVaDssmouuTOqkxTrVnGUNoD7vArYJU1pL1q4GKi6P4yZdp7SIyTeBafPyRl5vAD/96U/jkUcewc9//nPMmzcPp5xyCrq6uvD000+jp6cHV1xxBS655JLM8nPmzMGKFStw7bXXYsmSJTjzzDNRWFiIRx55BF1dXfi3f/u3d54DbBiGYRiGcZST1xtAALjjjjtw6qmn4vbbb8ef/vQnBEGARYsW4Utf+hIuu+yyrOWvueYazJw5EytWrMCTTz4JADjppJPw9a9/Heeee26uu28YhmEYxtvFngDmjbzfAAZBgCuuuAJXXHHFm/7Oeeed995q/frSwI5+24lkmNKnTiuIRvuXT8LxtaIt3bDdL6csSBht2cGl0ESXaDnZoNrbEWirF1FqVSUlYdlCDvdZ/VLO8lFlZeZzrFIuhB2OWk/ANhpOOc+zjcEY6te6jUP2K6s0SUkQwWj1MhDbeSjbDN6HqLVdtPG4i9QJyLJLokpfAiobZRVd2Q5DWTGEZDOhk1PYdkZbXPDYspUMoJImtI0OzfekYUfmc6Tsibi8Fe+TVjJcDtRWL1z2ZUsTvazT5WFOXNGlrxKyzVBjJCyQaK7Hyv6GrZj0tjlxA+OkY0GycWvmc1Qv3Qd4PVmyj2EQlkR96npA6xGl1bS6NvB5oZIf0tt80ok+/gEd16zrDacNKeun+BB975A8f+QG/Fjq1JsUXcOcsukZ7h6By5vhfrnONJ8Xyn6Lr7VZSS2c1KGuI0KeUkzfa1aWPXyedchrQ0BzMb1bnp8sXYnUtrm8rv9WsDwgNUampcR7/XzXx65gW/9YB28hDcU4OsjbSyCGYRiGYRhGfrAbQMMwDMMwjBGG3QAahmEYhmGMMPKuAXxfUpiCm96voQi27pRtpJ/Qd89uyvjM5+DAMDYWOrKH9XvK6oVjiFytjGoLN3tdoWuluCWlPwRbKijrFaEpUZqfsMxrWKK5s+Q6d/hoLo6hA6SWq/38E0Tb6Of9eHJkGAAEc2b4bVO0VPrDJ4rlilY1+G0dlOMckPUGtJUEjaXWZIXF/rgmTftF297LFmY+j797g2hjrVCqTupuEraWoL5oq5LUBD9v4malKyXNWZYFCFmJpFTUVHob6ffmzZbf2+mPXVYEFs1TN432Z8sOsRh/T0dl8TqyNICEbus8c0Hmc+mf1ok21s06Ze0RT/Y2J6keFVlImkDWa8YfWiSWK2jx50FHnYx0K3uWrJ86lIUG7WvQrWyAyr1eNDVjmuzzTopSU1YsIqpvgrRwCVtI97nLa8eytLak+3NKA5iaSPNtvLymJCtXZz5rrXJMYxmpSDw+fw5ecLxoG73ef69vlJ8rBRuU/pS0lkGZ0ujSuTTceZYoXXaWzpi/N3ea71elnMMFj6z0q1C2Ken6CX65bV6r6CYq3XeVP/6Fm2R0Hl93UxNklCJfo/U5EhSR3nGGSrzi2EBlVxRSLGGg9LyHFvXvT/yI0o7nCLOByR/2BNAwDMMwDGOEYU8ADcMwDMPIDxbTljfsBnAQgtghbOt/7O+ULQco9UKX8sId9Dp/gXqczpYQOjGAEwpUeYtLIeEuaV3BLvdRrS8VJrq0So/8A90vIqqQpS9O8XCNshTO5chQlbTZbqPiiU1Dfi9QpSlhQ5D2ZbfCZ2Q5kK15gmiYh9hqX/l4RWNk6Stu8aWW6Jh60Tb+HtqHcaokR8c1VnYrYow4jaFXjheXfaOxsl+ug0rViSpnUXmLy4GA3L9kw1bZNtGXnLhUDECWlZta/TqGSV/QJWC2R0pNkqVpTvjQditc9g3UOCdko8TWOAAQbfJzU1eT2CqHLTVST74q94FKmuVN0uonJplBpKxrhCVIkTq3yPIk2SvPXU6C0CkeooypS+h0XDlBJlQlWV5/0Ns3dNv6RtEUVo/161cJP6kpdb7tkLZK8stWPSuvFbzvhZS+kVZSixTJX/Rx5JJzoixiEtq/aMYU0RbSeaGtrMJ1DZnPJaosOpwhSkEjHUtOCVLygILtVPatlnOWE17iZjkOnPikZR8hzeH0xi2yjc7DQNlcsVQmUlZWZVv7j2XYN3S53Dg6sRtAwzAMwzByj0PujKBNa5iFaQANwzAMwzBGGPYE0DAMwzCM/GBP5vKG3QAOgovTGd2K1uSJ5ZQ+gzVzOiZOLKdtLDh2SOnp2ColUBYxrGlL7yDdjY7RIo1MVvRXn9QxCkinoqPTeB90XB5rmjgyDgDiYWxBwkavR4tJUxRVyDgktnDR1w7X7I9Jlm0K92OY4xOr6LnUeK+Zi9dvlttjPZW2VKHtxzt2DdkvjrJK75X6zaxjOQQcvwZI24xQafSE7i9r/WQDQ/YXw0WZhVqbyNvatUf+gm051LbZdoQ1fwCENjFWY6TPC7k90mgOEwUojomy/eBzJL1VaubE+jc3iKaQtVYqglHr/obqi7aF4uPKJB3q+FDsmduj4ix5naofrJnM2kbXMFGHhNMRcrwOpX9m0jv9OZIV8cgxbmoshzsGw54/NG+H7bMeE/oej4mOWOPj6IaZN1n7yovpvxWsMw6UpVdM8Y/blW0TXWPi3fKcDAf2T2tFjaMeKwEbhmEYhmGMMOwJoGEYhmEYecGMoPOH3QAOQhClENYctkNIq/Im2VgE5NgPAKAyQvokmb4QPfWa/zxeOr8nY8ilfbu08wiKKQmkTL6+nyLbgd6Z3p0+apdl3YjSAxDLch0nAYRbd4k2x+kVykIhNc3bLbg2aQkBKjEFqiwS1HsX/2iPLGel67wFxcF6X6Ib85i0O0iRpUKiSowh2Ux0LJos2kqfXO/7USrH0vX58kffPOWwv8Ufk+BYdVybfVkxUXYOYRVZukygRI8CWfJJXn098zk1XW67b7wvi6Y2yLIO23TocRbHuVDak4Tj/TgHjeqYsxUPlWt1+oKrINuMvdKWI+JSuNo2l8WDMmm9weUubfXCZd94mUzxcJEvCRZvkuXhhCx2Muc0gGS/nHuiBDhX7ms0tjLzuXeytOkpet2PH5+DABDQeehWvS7bUn5ctAwjNdnbrQxXwozJDkVLO3h+J5PHy7Zt3p4kGCWtnwKeU7q0nvLbcLrkzMdVf4/nwwSflhGpY9CzYFrmc/EWZZvDZV81Jo6ug8k0ua/RvtbMZ22VxNZMrl3uD+9DNEGuk+d+mPJFtPCgLG9zAkvn7LGiraiJJC4b5XnNJWedssTHVfeZ00U6zz5etJU9tsavY6qc3/tP7O9b/LsHYYws7AbQMAzDMIz8YE8A84ZpAA3DMAzDMEYY9gTQMAzDMIz8YE8A84bdAA5CUhiha3a/9iKI5ewsfP5Q5nMQqtfwR3ltyL7jpcZs0mpvZZJUjxZtB+f4top1Uu8WTPJ6wVBpPtgSYtdpMzKfx70o9UDF20knp6wD9h/nNUBjX5YWLUKH1SctAvomeC1USuu8SKPTVyMjqvpG+WVLD8jvdY3zOraOib6fxYukLq70qfUYClfutYMdtXJ6l7JOqlhao4COXbpUjl/bMtI7qmjAqrX+50jZu8R1XvfXPc7Ph7ZJsl/j93qdT9dMGYHWXeWXHd2ptKOvrM18TtXK77EuM5lYI9rCBrKkqZO6tXB/a+ZzeirrtZTOkywjtB4sYPsTbXlDba5ERchN9vvA8W6AiseK5DHo+Kqft4V/0yraOAYxrvbnWedCqesq2+zP684p0nao7Flv/XPgDPm9mg4/tgfrld418T9XvaatS4a2ztnxCT/fJj2kbHRICxdRbJu2c2H9btsseQ6O7iZt4k6piwOdI4mywwlIvxf0SN0iW5DEtVInGXZ4bVzPJH/tKz4o7VV2nebnQ+lMqVOrfsXPv+iAnItBEdkolctrStRM1zs15r11/voWdctjHpIuMxktdZLts/w+OFp91CPH2dFpHihHqoOz/Lwc06j6TJpDVyrnVNsxvs+FB+VKoyde9uufJq8xZTP9nGo+vlK0NS/uH5f4URgjDLsBNAzDMAwjL9hbwPnDNICGYRiGYRgjDHsCOAhhbxol6/pLL04711O5AaoMwiXhSb+QZcrkIJWOleN61R5KJRgny3Vumy/XJTPqRBsXKqf+J1mclKmUg0oqb6iydfXdq/33lPWGoz6zvQoAYJVPy8hK46ByVCo1RbQVbPHrjFtaRRsXUMrX+n4GqvQNst9Jdkgbk7DL23nUPrpNtMV8LHWyBfVZFu+Bkh1Uotm+WzZSycxFShLw0rrM59IqKn2pOZUm65XCJ1rltsf7spu2seDkDLYnAmRZNmyXpTxM9Ot0G7eKJi4kptiCSK0/4CSNKbJchz1k4aGTZyi9hucXAKR6qKyModFWL1z2dcdMk9t72R8DUBJI2VqdNOHLg4WVx4gmPp8mPCATHdK7/TGp3lIp10n7Hig7j4T64hK5t+N/8LT/YdYM0RbQXBkuOSMkWUblC2rO0jriOVJeEW7yliSBlnZQyVaXh/m4QpXo2e6naDcdc76WAph++ybfr33SBiYa7a9hQmIAeW0tapB/0hI6Pjp9Jer0fY52SQunNB2fQCWgjOqe6H8gyYQuw4OsmXrnyWt32Z/9vjpta8NjqxKLSrf4a1pYoSx8Jvl+TXpUWjM5um5Vb5XyitLmOQCApjYHyOr9e48D4II3XOxd25YhsCeAhmEYhmEYIwx7AmgYhmEYRn6wJ3N5w54AGoZhGIZhjDDsCeAguHTsNSjaXoH0OoGyBGHNT1gkLS4cRyUpS5WELDs4DguQupVg9QbRlqa+sObHKW1iomObiJB0XnGT1MFwvBTHTgFAQPFOiYqJEyibCfGfPaV94lg3tu/I0huRhslp6xWOxyqUGiOXpnFXdjh8LJMGGc0UspWI1oQmND8CpX1iSw36XqCWc2RPIaLYACQHWn2bmosxazS1LopioYJtUvMj5rTW6LHGrJH0YHquH6I5q/rFczFR85nR8WWsCYzbpOaQrYU43g2Qc0Vo/qDsY/gcVNF5fHxSqzaLtpj0bbrPbC3i1DxlXWGsx2GYiLeQ9JVuu9S4ivWw7i+RxyDhS8BueQ7yMQmVDjcmy5agQJ0/HFmnNIc8LsFaaWWV0DiAtKSxui5FpJPV48xzPVLXDZ5/sYqGFNpENebBOq9/HdqUZ5DzlaMP6VruVMxmSOdMwXMyCjDRekFeP8+3lNRhinNNXSvSB/xcCQ9KfWDS7s9Xvc7S5xv6f98l/y4ZRz92A2gYhmEYRs4JkDsbmBy9anJEYSVgwzAMwzCMEYY9ARyMIEB4uESkH+tz+VaXt6JR3sgkmTFRtOEln9qgbQzAJbMOWWKMarwtDFtH9P+CyhvHzPGfN28Xi4XKFoYJ6ijZYP2WIZfTpdaQbBn0/yKCUkp7UCVTThcJC+T0C2id8RhfwmA7lf5f+GPCJT5AlY6mKnuSNd4qJyIrGQCIqdQa1UvLBkf2K+G8WXJ7VOKOW1pEG8sFounUF2XFgw2+5JiaNlm2UdkqULYZcbMv2esyfEjWG3pOhTRP3TDl+4COD5clAWXn0qkkDVx+VlIILvuHZGMDAI76ybYfABDT3A9rxso2SviAOkd43kaTfOpJepss84sSZqk8P3mMkhPmiLZwjS8jutnS8ijooXLdalkCHA4uT8fKBoSPQ9JNpdxiWdLm61YwWV6LWBKQKIsTsX41NyK2idJyBH1tEn3xy0Z8LWpT8hpKGom0fIPKxVqGwccumqjSckgukGjrJ16Huo5wiTtQczFgeyRKHgpblRUTSQLC8dIGiFNw4jUq2Yi2zfITQM7NtCrti2vFZJnwE2xsyHzWY+QGriuteXoeZC+B5A17AmgYhmEYhjHCsCeAhmEYhmHkBYuCyx92AzgYhQVIpveXAaM98s3YhN6ai1RyBpcRoj2yHCgKqPoNsDHV/nuqJMNv94W6pLmP+raDygF1MrA+2SITMUSfmw4M2RZSKSrQZQN6C1SnhMR7fVJD95kniLay1d6RPm6RZaNUhS9NRntbM5/bzj9RLFf+TEPmc6JKZKJ8p/aNy4pc8gWAFCWwxOot4Na/WZT5POYhWa4RbyfWyBSXgMpDrtnPB53UkiIH/3jb0G8g63Idl4qimdNkv6isHB4rky2wqcF/TyVUOCrfotrP71glhvDcCMvlG4firWn19jiXxeMmmfYQf8iPc+rJV+U6uTS5X55bnQv9fNcJH1xO5bJvxycWi+XKNw/zVjMl97hX5Jv4KPDyjbBBJm5wWkpWEshwb3fX+/M8mCJLeeFufsvdbzum8xGQxwdqrkdjfNwDlzABIN7oZSB6bnA6h35LN6r2ZfmuE6eLttKNvs/JKH9+pnSZl0qYTqVc8Bvi0Sy5fpZoxHtkSozagPgxqPNjm4xWMpkXvWRHJ9aggkrqO/1bx06lOPXM8esveGq1aGOHgtSMaXL9dJ7H6o33iMYlOHGeaAvX+5SaoEX2mVNDXLGUZRxc0H/skt12OzDSsCNuGIZhGEZ+sCeAecM0gIZhGIZhGCMMewJoGIZhGEbuccjdE0B70piF3QAORhggrujXaIS7lEc8pzso6wPWwSSHhk4yiA9JfUYkFpOzNBxFWhidEjFEQkGyqRFDEUTyoS/btAQFb346xK2tmc+p0aNEG2vACg8pi5ADpN9ycmwdp5mQxUFxU7VcjuwVsmxgSJPlVNIAW4KESgfDy4bKIqa4hTRatdKCJKDxE/o5AI5sJ4ROUlnqgPqi7VbYXiPbUoX6pXRebAsSNkvNXEI2NNpKQoxLMSUG6KQJ2reoslK2cUJJVZVso7nveuXxL2gh+5hRck7xsXPd0n6pbDOdT4lcp0hgoXOENX8AEJd7TVbB7lbZRnNd6x05CSJQlh3JaL+sPgaMTlkJG/0xCdQ8Zc0h65G11ZNIr9BJFmVehxe0y3NEzDGVuMEaWq1HTQ76610Qq7+0XWwLNHj/ASCkecl97F+YjqvSgCaUGqN1uI7asv7+k07OjZHHTiS86IQPvg6TpVdcLo9V0R6/bZ38IY6PSm7ivzEifQVAMtr3M9wk9cL8t0NcZyGvk/pvQPGB/v3JOm7GUY+VgA3DMAzDMN4i+/fvR21tLYIgwMyZM4dd9mc/+xkWL16M8vJyjBkzBmeffTaefvrpYb/z1FNP4eyzz8aYMWNQXl6OxYsX47//+7/ftf7bDaBhGIZhGHkhcLn5917w1a9+Fc3NzW+43NVXX43ly5dj9erVOOOMM7B48WI8+uij+OAHP4j77rtv0O/ce++9WLp0Kf7nf/4HCxYswFlnnYWNGzfisssuw3XXXfeu9N9KwIORJEi1Hi5bKDsCUWrpleXN7jnejiLVIduiTd59HxR6DgDdU32ZrOR1FWae8uWAjrmyvFG+0m+j5xhvTZBqV6HeVBroK5fWC4VN5Kq/VSaIiFUou4jURG9x4A6qcjeVjrpr5PbKyXoh2i0tG7qnVvp1TvOfU52qfELlzUSVA7mc6sZJm57kFW/tEFVId38uaYbjZMl5/1xftipdr/oyZ0bmc1whS0DRa95Sw1X40k1wSCYZiPErkYkOHWfOzXyu+Iu0YhESBGXLwT93HSuTTYobvLVESpUAuYzUPt2XYcuctJIJ0lQiS6n/R271c93pZBsqhWu5Q0edP9fKm1QJkMr+mCvtkDqn+GNZWCn7mVrl7XBEQo0qyXHZd9PnpI3SrB/7ZTuPkXZIJc96W5h4vJQHJCX+8uoK5JwKSS6gS9psEdI3Uc7hqIvO7d3yWsHEVPpsv1hZ3mynZJMJlaIt9aIviyYq2SYa7/c9UqXw9M5dfnuT5Dlf1OSvb+0z/LEapRJx0mX+e9GWXaItrPHjp9NrIpIZBKp0LPZBlcJ3X+pTfSo3yVJrIc3NUElc9p/qr7VVq1r96tNKflDq96flU38l2sauJGucHfI4BnT9SU1WqUQNNC6T5FwM2/y53Hy6TKWpWuOPa9Al/z7sOq3/upVeZWm5b4U//OEP+PnPf44vfvGL+PGPfzzkco899hhuueUWjB07Fs888wxmzeqfd8888wyWLVuG5cuXY9myZagkKc2BAwfwuc99DnEc495778XHP/5xAMDevXtx2mmn4cYbb8S5556LZcuWvaN9sCeAhmEYhmEYb5Kuri586Utfwrx5897wadxNN90EALj++uszN38AcPLJJ+PKK69Ea2srfvKTn4jv/Nd//RcOHTqECy64IHPzBwDjxo3D97//fQDAjTfe+I73w24ADcMwDMMw3iTf/va3sWXLFtx2220ooBezNF1dXfjjH/8IALj44ouz2gd+98ADD4jfP/TQQ0N+55xzzkFxcTEee+wxdKsXD98qdgNoGIZhGEZ+cDn69y7x6quv4sYbb8Ty5cuxZMmSYZddv349enp6UFNTg7q6uqz2RYsWZdbJrFq1SrQzhYWFOPbYY9Hd3Y0NGzZktb8VTAM4GL19wKb++DStU+LX+aMJUoNR+My6zOegTNoKxM0+ti1S2sESjprSViL0vdJyqW9hq4TCl73WKamXEy3a57Vi4XZlOcCWJEojE5OFRqj1gWy/omwsWH9UvlZG6QW0r7GykijZ4DWBcRXFwrVIjWEyibSQyoon3uHjuMKUGgeK7tMxdBHrMlXblDv9OPBxBKReS6nwRAxZssHrAQMVgcX6Q30MRq/0usL0PhmdFrFVirLsYNuUknUyosyRPUmirHLYbqP04VW+W9qChnRKwS45Jijy+6etktL7KMpMzZuyZ/25FStdKdthRGMr1fdI56fsUGKySmLtWKjOQbZ6Yc0fACSkfy15YbNoY4uQsEHq1kDHVUenORrnRP8vnsYspc55NPt+JnTMXbu0tWHLltGPvi63TTYm0TR5jrC1VDRJxtAle/3809YoHC9X82c5Do50phXNNB8K5ZOTiKxyEnX8Q5rPWRZBbMWk7ZAq/Xmt4x8n/s7bZWlLGt473Zcxv/dxkOLc1bY5FJ85tlNdd0n/HBTJ60G8ucG3qWsFx4iGNBcAwNH5NOZVed4lr1KEpdK1T3iqfzx3dJoNzBuRJAm+8IUvoLKyMlOKHY5t2/rvIwa7+QOAsrIyVFZWoqWlBW1tbRg1ahQOHTqEgwcPDvu9uro6rFy5Eo2NjViwYMHb3Bu7ATQMwzAMI0+8V2/oDsbmzZsxf/78QdvWrFnzht//4Q9/iBdeeAF33HEHxo4d+4bLtx/+j1mp8ndlysrK0NramrkBbKf/zA31vbLDD5ja2toGbX+zWAnYMAzDMAxjGLZt24brr78eS5cuxeWXX57v7rwr2BPAQQiiCGH14XKhsipwbf7uPNkjbUyC6ZMzn5tOlrYP1b8iOwz1+n5vLSUG/PkVuc5F8zKfoxZpqRFv807wzZd7m4HaJ2Wp0FEZyRXIx//t9b5EUvq7laKNUyFEqQNANLve97FD2TLUUnmwU5a34gm+DKtLpocW+ZJTZ7Uf99q7tonl3E5vm6DTSwIqK7UuqhVtFff69UQzpE2Co9Jx7ynyf4js6H9o6XS5TkqUCNbLBBZOF+lbujDzOS5WTvyP+FJr+rRjRVtnld+fUaXSIiZet9H/EMrRTE3xVinJPulTFfKY6WQYljhQCTApl9sOOr11iVPlVC77hqpcF/L2xklbI55HUYX8XnqrH9veyTJd5MAZ3rZlwgPyGIiEnBPm+D6/IrUznPCRZfVCZd8dl0ubmcn3eGnEvtNluaaw3RcSR/32RdHmuFyoyv7BZBp3kg4AQDTB7yvvWzROWtckJFU4eOHxoq3qGW/Tk2yW48XHK71NJk2E8/34Req8Tm9pyHxOlSipClkBJXV0/ry2XizXdtGJmc+jGlRCyX5/DqYbpBxFJJTMlud1tHNoyYGwZiqREodgmx+jcKo8rp31/hrmUn6dRfulnU/Y4vfh0FyViHOcX8eoXz8rv0dPfYI6VYYf5c9Dt0bKEZIev/2mi+pF24S9/lzrnidtlNrr+svMSUGebGBy+ASwvr7+TT3pG4yvfOUr6O3txW233famv1N++LrSqf5+Mh2Hz49Rh8+9cr4WdXaiQlmWDfadt4vdABqGYRiGYQzDgw8+iMrKSlx55ZXi9wNv4u7cuTPjy/erX/0K48ePx5Qp/f8h2bFD/mdqgI6ODrS2tqKqqipzM1dRUYHRo0fj4MGD2LFjB+bNm5f1vYH1TZ069R3tk90AGoZhGIaRH46gd09aW1vxxBNPDNrW3d2daRu4KZwzZw6KiorQ1NSEnTt3YtIk+fT1pZdeAoCsFzkW/v/be/MoO8rz3Petqr17ngeNLak1gxokIYxiRstc2eEgEyaBb058rzDm5GKHLLPkLHL+sM/xMrBOjmNsk0NyFuvY4HglOAkCjBHGMfIkh9FCloTmsVtSt6Qe1PO4d1XdP7bU3/O+u7sBI3rb9PNbi8Xu/nZNX31fdanep55nxQrZunWrbN++PesGMJVKye7du6WgoECWLFnyvo6HGkBCCCGEkAmI43jM/44dyyQ0LVy4cPR39fX1IiJSWFgo119/vYiIPP3001nr3LRpk4iI3HTTTer369atU+3I5s2bZWhoSNauXSsFBQVZ7e8FPgEcgziKJT6vcanQNXa0AbExPWnQZE0b1HoQZUdxWOtu8oacziM0dhF+E8QEGYsLH7Qv1budzsYDmwcRkahl/Mio0rTTzETWqmJkxH7dcRp0hkVG8wMavSit98UHXRzazIiIlIJ+r6TIaXLQysHuV9hmbFng2Ct26yirGLRJUVOzagtmOL1gwVGtoUStZ4XpW9RvobWHiNbv5L3urDi82UavBZ/zD+pzlQ/rD9tM5iTo/rJsRkBP59nzAzrWiWLIVNSh2S8PI92shg0tgkzkGmoMo0M62k5i1xNZ9hdgx5S/X9uM1PY7fVP61Jlx1+nvge1Z89aUO1aMd8vss1sHav5ERKIKp9eZtkW34TjVI2NifW2cdOfSr9ZRcGhlo/SvZp6hdrniF1pHGJc67VtslvPQZsRo+Xywc0k36WP1ku582XEagzbNP+DOgWfmdcU2uG6UaxutGDR6volLxP5DzZ+Itsryus2YanXXjrDjrGrywfYoPNyo2gqaXDkPx7pXovdZ4LjLh7VOFo8nShpbKJyTh4wGdJHTUMZmDHswRyoP6Ws36pHzW3pUW1hQISIifvoP6FHcHxAbN26Ul156SR566CFZt26dioJ7/PHHpaKiQj73uc+pZe655x55+OGH5fnnn5dnn312NA2ktbVVHnjgARHJ5BC/X/gEkBBCCCGTT5yxgZmM/3JVal67dq188YtflI6ODlm5cqXccsstcuONN8p1110n6XRannzySZUDLCJSVVUlTzzxhPi+L+vXr5frr79e7rjjDlm6dKkcPnxYNm7c+L5zgEX4BJAQQggh5APj29/+tqxcuVIee+wxefnllyUvL0/Wrl0rX/nKV+Sqq64ac5nbb79dtm7dKg899JC8/vrrMjIyIsuWLZP77rtPNmzYcEH2izeAY5EIRGozr+1nWZxgmoSxgUksqB/9fPajusxX9i+u5JiYp1/tT9e6Uq53Qpcmo3r33eBMl27rdo/y++tcqaO8VT/i901iCdI/z5VhChuNpQqUN2y51qvRpSnVVgnHY8vRla5kFrTr/exe7uxjRordw+maH+uSLJZWAuNqL1BO6V5WoZrKGl3pxrc2MFAOGlmprV7yoDTdvULb+5T/1o2ByCSbBLDO4SudfUhYqMu1xVBOT83T1ihx0vVDnrG8QWsUsXY4kC6QNhIAZYFjLFywROtBeTBrDOF5NWk5aHHil+rzg6kNwUL9Bps3BDKJfF3ewnSEkUV6/nQvdCXBmqMVensDbv7GYBHiN+p0FCzfhTO0wSsmfFirFyz7nvnjOaotv8f1Q8nTxpppUF9X1L4MuXNik1QwcSOOIAHDJHNgCs3QCj3WC3e5fc5KeIFxlGWbAvMgMVf3Q9js+jOYOUu1RSjTWOzOebhjr/re4DXOuqSwRfdPosNdK+yDHCzXpufo+ZNocaXdyEhafLAaCi5apPf5KFhGLV2g2lLVbqyMVLh5lter51LQ47aXrtD9PAz2TiWNppQbujnvG6uk4dkVo58TjfrN0jjttj9UrdeZB3N5ZJa2FRkuz1yP4uDDbwPzQVBfX5+VGDYWd91113v2D7z66qvlpZde+h337J3JeQn4N7/5jdx5550ya9YsSSaTUlFRIddee608+eSTY3ZqGIbyrW99Sy699FIpLCyU2tpaufPOO2Xfvn1jrJ0QQgghhFhy+gTwmWeekU9/+tMShqGsWrVKrr32Wmlra5Nf//rX8h//8R+yZcsW+ed//ufR70dRJHfccYc899xzUlFRIevWrZP29nbZtGmTvPjii/KLX/xCVq9encMjIoQQQsi7ZTKj4IgmZ08A0+m0fOELX5AwDOWf//mf5a233pJ//dd/lZ///Oeya9cuqaqqkqeeekp+8YtfjC7zxBNPyHPPPSeLFy+W/fv3y6ZNm+SXv/ylPP300zIwMCB/9md/Jmn7NhwhhBBCCFHk7Ang/v37pbW1VZYuXSr/+T//Z9V28cUXy2c+8xn5u7/7O/nNb34jH//4x0VE5Jvf/KaIiHz961+X6dOdJun222+XP/mTP5Ef/ehH8vzzz8vtt9/+vvYtTvoycE4bV3hC3yN7+Iq+1cgUOD1IQYexVwCbDtQliYgEnaAJM+HPUQj/PJpAZ+CBBCju1to6D+0CjHVAXrfTqXjF2sbAS0DUVJ6xzRgGPY3Rn8VoSZKn2/x+6D9zPInBaMzPUm50ZCchANscTzzf2bnkd+pzgJFlnrFsQcuOVImZFrOdTjLMH18ng9Yedp3pEtcniQFjCgJ9G+VpfWDyLFiEGLsV/Nk3Wq4YbE2CSm234aXBisXoojxYZ3qu68vEca13jeH8+1U65io+DVYs5SYKDs6XlXhgZKE9VrQk8frMPkdgC2LscATOszcM+kY7z2CMRYXm/IPNCMa7iehzjJo/EZGREncMgYlsUppae1pBX2ktT9DCRcw8V4AOMzLarhg1ulY7CGM4NrrfuMCdO89qR2F7qZl6PCS6Icat2F0jPTueofvsPMDIQnvu4l63/pEq3V/BMdCVRub6CcearjAWW/jdtLYyCovc+MD4tFSxHjeJjvF1nnnd7hxbyyNl2zSsx3pY4Polr1jb9ESDbl+KTplYOtA7xqbbR0ozv4hz9TiITwBzRs6eAOZb8fE4VFdnBNnHjh2Tffv2SWFh4ahJIrJ+/XoREXnhhRcu3E4SQgghhHwIydkN4IIFC2ThwoVy4MABeeqpp1Tbvn375J/+6Z+ksrJSbr31VhER2blzp4iIXHLJJZK0Jq4ismrVKhER2bVr1we854QQQgi5IMST9B/JImcl4CAI5B//8R/lU5/6lPzZn/2ZPPLII7J48WJpbW2VX//617Js2TL53ve+J1VVGbuR48czr+XX1dWNub7zv29qahqz/b3gD6WleEfGKsHaNcQTlGeko2v0Y95ekyYAqQ3hGV1O88665WzqhXewcfRztFDbTARg2VHy451uW2LA8plp8l6F5UwJGFM8glna1gYtSHy73HFnZRNctFC1hZCWYkvARWCjohI+FmtbFg/KaWG7dv4PjjiLi+SOXtWGWwuNwz5a3hT9bLdersEdQ8XTe3QblG8wmUVEJyIUvQT9PKzLM9jvwdadug1kBlbfihYeNtEhhjEVzNG2HHGP6xe7Lx5EC/lgR5Ju1f3soySgukKvA/okPqntVvAc2AQMtFGKzhjrH0xc2blfNVW+7dq86dNUW4jHt3v/uN/z211qTJzUVj8o3yh97i3VhgVUa/Wiyr5ztHVNgGk5xmoK51awWFuQeD0uxSWEa5NvJA0RHHfRm0dUm6CFU5W+3oSNYBFjyunecXcuQ7PPKsXFzGssaSabIXFjlu6TwuffdD/4etsRXGvtfkVD7rpR8Mu3VVuIperYlO+L3DqTLToJJJ2CtCFzrSiAc5CE65SVWmAKTn67/luRhr8BsSlp436GxgYo/xdd7msJ/ec7AKsmf6c+53hG8v9DX8NqejLJFMGQzashH3Zy+hbw1VdfLb/61a/k1ltvle3bt48GI+fl5cknPvEJWbDAXfz6+jKTrshOlnMUn/sj2tvbO2a7paGhYczfHzlyRJJSPGYbIYQQQi4MnkzeW8A5cjn8vSanPoA/+MEPZPXq1TJnzhx54403pK+vTw4ePCh33XWXPPLII3L99dfLsHlCQQghhBBC3h85ewJ46NAh2bBhg0ybNk02b94sJSWZt/AWL14sjz/+uLS0tMjmzZvliSeekM9//vOj7QOmbHSe/v7MG2ul5m278dizZ8+Yv29oaJDjJvieEEIIIeTDRM5uAP/lX/5FUqmU3HDDDaM3d8idd94pmzdvlq1bt8rnP/95mTs3E2d08uTJrO/i7+fNmzdm+3shDkMXs2asClC3FGRpvkC3lmVx8O70FbEpYaO2K2jr0m1g9SE+PMydYJ89q/kCHUmWjgwsLqJTZ1Sb0mRZ70U41viE1oCJB/sZa3sFtMfx0Fqmo1P0F8e3xlHHmjAvC+F+WT0d6mmMnsrb3wi7rPcZvxtZ+x3Qwim9mz0/JkJOr2SCh/RgVRFHE5yDTq0jQm2S3RfsF78ZtKp2/KIOq13rp2IVE2eWm+B4ME5MTD+riLoJzquNThvPOikCjaTFH9LnI4ZzbK1E0PrH6oXR6gU1fyLakggjvESMxs3s53hjJSs1Cfo5MnYuPurrUtpmZMKxAec1tsvBOoND+hqN/YDH5lkNNWzP6vzw+pZ1vcFrkbW1getIbGIppcVd07I0jRMQdcB4j8BSydrywDnwzbhXtmDmeCa8HsD5sSM7Brsda4ekxoCxL0u0Z5bz0jnQAE7mCxp8ESSLnJWAz9+wlZeXj9l+/vednZk//itWrBARkd27d0sqlcr6/nn94PLlyy/4vhJCCCGEfJjI2Q3gjBmZt0q3bds2ZvtvfvMbEckELYuIzJ8/Xy6++GIZHByUF198Mev7mzZtEhGRm2666QPYW0IIIYRccGgDkzNyVgK++eab5Wtf+5ps3bpV/vf//t/y+c9/frTt9ddfl29961si4gyeRUQ2btwo/+W//Bd54IEH5KqrrpJp0zJWDs8++6z86Ec/kkWLFsnNN9/8/ncujkfLn1klP8C+oq/Lqe9+xGEJKLZJdlA6CG2pLZ39JDTTYMtB8MjflAZU+WGCGL3sMi+UH02ahGobMi/xROP3Z9Q/MOb3IlN2yyo/YRuer6zOHB+0kshKocDyky1NQrnGs8kw4/VnVlkyGrctnqC/JuoHPIbIygpUH5ntwT7HeD5sqb0fynrGjkIdt5VCyPjHo0raE8y7Cc+/TXsY73tm/Xju7JidaGxYKxu9UvieKTGOO3dF919oSsBqjL3LOSih2eceV6rMKovCeMsav8aaZdzl+nWfqG1Av0f22oDHY86x6q8JEpEmnBOGCUutvwNZYy92+xza8jDO6/fQz+qcmDEcTiC9UCXnEbPcOenFuNcr8qElZ08AV61aJX/1V38lIiJf+MIX5JJLLpE777xTrrnmGrn66qulv79f/vzP/1zWrl07uszdd98tt956qxw6dEguuugiueOOO+TjH/+4rF+/XgoLC+Wf/umfJJHIqbMNIYQQQt4lXjw5/5FscmoD87d/+7fy7LPPyic/+Uk5ffq0PPfcc7J371752Mc+Jk899ZQ8/vjj6vu+78vTTz8tjzzyiMyaNUs2b94sb7/9ttx+++2ybds2+aM/+qMcHQkhhBBCyB8OOX9cduutt47Gvb0bgiCQjRs3ysaNGz/AvSKEEELIBw6fzuWMnN8A/j7iJZMSzM1Ey8WF+aot3HNg9HNino5mQxuL6JqVqi2xDWKojFdhesls1/bKDtUWYGyTAS1jhtZcOvo576zRtmzb65axerqrVrhtv6pjyFTUmLGPSdTNlvHAqDGv2CS3gPVD1KKtMTyILEvNcm+H+4Nam+Lva3TrMHojjCiLli9WbcHB467N6H+C2RBLZXV+Je4Y/IR+aO73uO3HXVrnE9W74/FbXExYepGOZgt2udgmv7pSbxviqqS1Q7WF7RBDVVOt2gRsRjyzzvAERPXN0edRnbtSsGdq09v2SlxaTtiqI9B8SOuxGjMPzg9aoYiI+FUVo5+V1YZofVJijo6DPHn73NHPM7796rj7grYj4UJ93H6TswRB+x4REQGtr2ci3eIk2JoMGTsXOHaMdxOZ2Aak+88+Ovq5+lcnVFs8DBq3aALLFhjffq0ZG2CVkj7ZrJpwzou5VgQwjtImztIHKy+rCcWx6Fe6dXiBnksDK915jI0MrugY6K3NPEANnWdiKdE+Jjyt9xmj02KjtYzzsvPmR1eJ8wL73djaRCUQq9jZp9rCae76hjZTdp+tJZUscn0kodHlQtThyA1XqLait915HpmvYxB7FmSi+sIfbhEyteANICGEEEJyAvV5uSOnGkBCCCGEEDL58AngGMSBJ1Fp5rE4lvhEdEkprNJJID6UIz0o+YoYe41OnWyRbHIlp3jxAr3caVdeG/6jJXq5HlcOKtx2dPRzNGeG+l4wrcatzzjlx2+75bzpujQgWCadp8tuESQb+GW6pI3O/yNX6n0u2uPK5FlpHPA5ecqVU/su1iWs/MJFo5+DV3Wkn1/tSuZ+iykjYkk7y5YB+sWUJgfmufNc+Eu9PVynV6YTbYIuV/bBlIBgh05mwHJgukmX/AI4ntCURVXZ15R5o6Ou3B3O1TKCJJSVs5JasIRe4Y4nNkkWPpT2g4sWqTZv0MkFIpPi4qE9TUr3QzzTjVMxFks+lhWNDcjsF2Hf7Pw50TL6GfvPm6tLuR4keqRm6f5KlBS6fT54VLXheLMpJD6UBAOzX5jwYa1esOzbv0LLBYoOu/6Mm1zihp3XPozFuFeXH+PZbp6nlq5Sbfmv7hv97C3Ucz4+BaV+m2xR5Pqo/4p61VZ8yPV7XOiudWGB/vNTtNMdd5w2Nj0wZsN6c+4OuPK6Z0qmSmZgrFHSMypGPwdJvS/R4WOwASP7qHTLhWdcnwRmDgrsc1Surw3eHif78OYaOQ2ODTvnIZ0nnlWr2xqWjn4uPKGvbzFYGeUd06XwoY/UZ74zgcsP+XDCG0BCCCGE5AaWgHMGS8CEEEIIIVMMPgEkhBBCyOQzmTFtfNKYBW8Ax8L3JSzJaIL8Lq2fwdf+g04TsQVaEWt/Eve49WTFiaXAssNGVIE1Rn6b0SOiBQnoT4JT7Xr9oJny8o3FBerwjPYtAuuXoMfYGKDNhLGqCcqdZq7gtNF5DU8Q1QQ6HA+is4qO63UEZ52+JSswDLU8vn7Ajbo/P1/b+6ANRHjRPNWWfxYscJbUqzavzS2XFYGFfVThbB9iG7824I41YXSYEei3/MJC1YbL+cZ6A/VIfruJKwN7kqw4LPg5XuD0ZzbQLe6FKLhYj3XU/Vl9KOo+PROj5uN8qjLWNWe1llABc9IztjMhjGHU7/pmjkjSaR+DQRPT1t7l2mZqfW086M5BYPY5LnWWJJ6ZP9FEEYIwR1DzJyISVkJfH3DrSMyYrr6H6/AqtFY5hv4qaDIaTThfcUeXapOZTnNmzx3OrfxOM8chwtKDa0Wi21wbKmCs2Fi1AXesQbM+dyGcg2iB1i36reOPm6ATNNs2Eg+v5SY+U8C+Bq2L4nI91v1OuE7N0GMjAB1hbHWyYLEkRh8qcC79DqPzw2u5OecC14eoplw1JXszfe0Zxxny4Yc3gIQQQgjJCfYfl2TyoAaQEEIIIWSKwSeAYzE0LMGOQyIiEpkyYtTvyhbWxiQecaWPrHJdGsq8pqSANg1YzrB4LdqdPg3l4gDKBpHZr8iUaxAfnPNt2oegPYl1/ocSKiagiOi0EW+/2bY3/r/3YrAuEbDl8EwZJBxy+4n9KiISodu/6eescifgw3n2dh7Uu4ypDaZEH8Kx2n3BdISws8v93paf4ZzjsWW27fYrMiV6TILIkhXAufRg2yI6nSOrvIXb/q1LvcnqZ7Ayio1VBZY0JxrPNpUmbHFpHHFqfKlA2KZLgGiHk25u0V/23fFF0LeeSXqIsI9OacubCMez6S8PJAdxpEuaAmXRifrB2tqo8wpWLyKiyr5+gbMZSZ8+o78Hx+31mf1Smza2KRMca4xpMJERX0CJ0d+2TzWFON5gX2xKDF7DLOqaZvcL9/nAMdUWov2WmSNolWQtYtTPsb5m4ThVGBsglYjSrtNLIrgOZklCcD7Z6yWkoET2eOA67w/o8YbXGM9YYE0/kykJH+qbQJ7zQUJtXs7gE0BCCCGEkCkGnwASQgghJCcwCi538AZwLDzfBcfb8gx+zQawY5sp84kqW5oEDCwXm+0l4K3D2Lrc46P8aa4MhukHIrpUlBV0DyWsrFIx7qc51mho/DcQ8c3VrDI5lmRMWzDbOfzjsdnQeNVDxqVfvbk8TTvle1BCteUgtR/mbU4sD/omEQVTL2zYvMA21FixYwPfVK3Qb+hFUP7OKslhqdCU73Hc2DdoMcUj6tVvsmPpMDHHvQWcNmMKy1t2v1AmgW/eiph+T+t9xvPql+o3KiNM2bFla1ynb9qgj3AehOatXN+8ta/2q899N5iu3wIWVWIc/zVK385dmOcoHRERNdfsOnGuYdk3MW+O+h6WKX3zdiq+6Zs+2qj3EyQh9nqQmO/ejk836sQanMtZ8hc1NmHcmPQNAemAZ+agByXNrHI3fq/AJG7ku7dhI/tGLb59HZlkExhGWdfrJFxDYR34Zq+ISKjehjf7BX2UPtmsdwvGKV5nRfS5C81yak5Or1FtguNhhnEaOJdYEp9lFMhUgzeAhBBCCMkNfAKYM6gBJIQQQgiZYvAGkBBCCCFkisES8Bh4vjeq0YhsAgGmDhSMr+XK0vVY2wRA2ccYzVTU4zRaVjODaSPxGWeN4RcWqO9FkKoRGUsIH+ww0LbE7pd15kctFyaZZH7hvuuXGUd6sFeIuk2SCuq8QAdjdVDBNKdTClu1JYhfCOkLJh3DL3a6m9jaraAtSMJqk+DfSWaf0xNYY6jzBUkTYi1O0EqmV2vTlP2K0TuizjTrnINu0WpVfbQM6tPbQ5TdhR2/SRh7ZqyjjsyO2bgf9FT++JYqYjWasH2vyGrMYP5YexfYNRxHtr+wj0Kji8Rkm8jYeeDY8Mw68bxG1mIJz6W1AcEEGaMdU0k60H/WmgR1f5G1J4FrQFCtU3xwXiTqZus2SPSwtikBznNzDvwU9AvOA6OLRG1s3GOuDXDcVnMcwjnxrP4Qxqa1MlL7aHS50TDYdlnNLmp78+CaYlKOlPbSaqHh+NDGSCT7fI23bfu3AvsT/x6ITGwRI0Vm3E42LAHnDD4BJIQQQgiZYvAJICGEEEJyAm1gcgdvAMcg9n2JSzKP9j3zeDzoAusVU9bxG5aOfh6Zrl3t8986PPrZM1YfYQ2UT/ZpJ3tvNthOWDuUY86KofdTK0Y/l2835SAIYI9N2TosdT8Huw6rNiyLWLf6AI4hNnYRfjVuz9jOQMksSOpS0dAyV3JKlbjyVukbTXqfsezr6/KmB+HsI3N0acX/9W9HP6NNiohOl4iW1au2eIGzk4nN9hJdri0wwfMROPqHKxe7/RgxpeLd0O9L9LaHZ7pxVLxXJ1Skj7uUiNiUeTFQ3rMWJ1CaDIzdCp7nqGG++95ZLR1ASx1J6+NJt7j9DAr0WA+mgT2FLQ/Dz55JPYlPu21Ec/S5613sjqHiNzqVRk65ueCBrY1YSxDov771q1VT+cv7Rz9337JStVX84ujo56EVc1VbFLh1Fr15RLehFCM0JfpasHQykgCvwl0rsJRrrV6wjNh38+WqrWwvlHLbu/T6oYxt7UmCixaNfk706u3hd/0VF+t1NjkLofAiZyUT7G1U3xtucFY2+Sf1fvkwxiJjt4Rl33CWnvPeQUj7sJZRV13q9uWsvoZ5B915DWq1pUr/pW4cJftcaTcY0CXgoNOdn67Lddm66BRYP736tl4OrmFZJe1CSEgy0psYxkP3jQ2qrWK767OwXF8P2lZlzmV4kjYwUw3eABJCCCFk8oll8jSAfNKYBTWAhBBCCCFTDD4BJIQQQsik48nkaQC9d/7KlIM3gGPhe5KuyOgk/CETA3TCaXJspI6cbhv9mH9cR2dhnFDYpGOUgkGnI/NKte2DgBYqNUvHIyVSLjqtpNHpP+JerddScVglWpvotYEeyLShpsS3MUenQOdVN0u1Kd1SidGwgG4xNDqvJOh3CvY7PV1qvtZ8JUE7iPuR+YWzQsg7prVCEUaB2Xi0Ste3Xp/WdobFTneTGBzfSkISWkODsW7eEadN83yj5QSdj9fRo9qKTjoNW2j0YKjRtBF/HvRRVKHHlN/rbCBCawsEesGhWtdfhb89oL+HGtCZWiPloy2HGVN4ftLHtcYsMQvOs7GBUTFXx7XOr3wItFcpPV/RfsWD7dm4vxgsgkpOaJuMGNZZ+Zre57jUHV/hLhOPVgna3hptt+KDhUvUo8859lE8W19jrNbzPBgRJqL1gUrzJyL9C92xlwwZSyKYk0FFhWqKYB54x/SxoiVNulTrjJOgvVNaUhNXVrDXaVqjGnN+Ctx4tjZAIWhtAzN3Vb+YMeX3uWPHSEdLullfy4sxqg8tg2br6xRavZQ06fOTOOWuP7GNiYN5nT5xUrUFoDO39kH496j02PiaXW/EzJFE5nhYIZ168AaQEEIIIbmBd545gxpAQgghhJApBp8AjkUsEpwr9XmntKO6QMpBbJM5MGHBWBWoUlRC258IOP9Hc3U5VY46G4N4nnGMx9f53z7kPs+r0/s1gWVHBHYYgS1pF4ElSLVO9PCghBqbdIwIyiJDf7RQtRVDqSU+Zexqup0VQwz2O3Ggy14j81zpKGhtU224nNdnrB2g1BqbtAdVJu8bUk2Ds135pmDbPhmXal22iqBkiykX6R5TygVrnrCrS7WhTYtN3AggEUGVG0UkvR9sR+aZMmLSleu85vEvAcUHXWnNZDYouxBrEeTVOWlC3GlSDcCCxgt0yTycAWX4A9r6R6UvGJlE3Axl8qXzVJsPiSgRzLO4RFth4FiJZlaotqDezafoiN4vlHbYxAgPE2yqtB0OpsHEpmyNliqppatUW0ETzDtI40gfbdT7jAkfxuoFy76dq2eqtoqX9rr1m2tR1jkZZ3sD03Q/BINuPTj2sAQrIiIwL6K9h1QTprGkl+lzHOyCUq6VAEAZNho0CRgFICsY1hKKic4rWlvFZ+F7Jn1DprnrtWeSlEL4++Av0PZBmLhi04ziRlcS9ubppJYIpCWJZp1Yo9JFjAVScEVm3tGPb+rBG0BCCCGE5ATeeOYOloAJIYQQQqYYfAJICCGEkNzAJ4A5gzeAYxHH4g2f0wAaew2l3Yi1MipEnYXRN2EMUZzS2pcwDdYLJ7XFRRy5bSR+o604vDywRpjp9GDpw41m005DZzUluA6JjNILdIvx/qO6DTRZsbVXAN1a/k+2q7aoYHzrEunocm35ri0wdiF4DJHRxaljMMcTdju7Dd9E4mE0V7xQa2sKXnYRcsFcra9Eq4d0q9YAYb/7YEcRWO0b2FjYiLr0GadxtNpRtLLxerSm0QeNnmcit1BDlzZxhjH8nOgrHfP3IiIRHKtvdKXhaafJC+bovhRlR6HPXbRtt1tnjda7hqDZ9Aa1RhPnmn9Y22aE/WBlVOR0f+EhPZ5R55V4S9uy4BjzTXSeh2PfaoJR29lobFNwuchY3sC+5L+qNadoaxKDrswvNhZOQ6A5LtRxlngOUPMnIuKhjtVci9C2JWoydjidbiyWvKzXGfU5bd9EmlYPznlimraIEZjziYN62yGsP87SgLprstWcem87nWxoxxTY2tjl4haIF4QxZc9juM/pGINlS/T6QS8en9A2M3jdstfWAOyEItCHZ/YZ/sbkGZ056HS9Aj0eqt/O9F8wbJW+5MMObwAJIYQQkhv4BDBnUANICCGEEDLF4BPA8Tjn9h4PaCsRgbKvTTmIoZSX5ddvygNqU1Das4/nsZRjk0fQhiaEcoBKVBCRCKwwYpO+gT9jukPmF1DCLNK2GVgytcvFUGoNFs9Xbd4AWN60mZIpWD1I4P5tYm0SwnJn5+LvOqzXgdYvptydgDJ51KHTEYJaV3KKz2qbFoGUEGtrYm1bEL/cLYf9ZUvTPiaUmD7BcRObMRRgekm5Lk2GzS4hxasw5w4tY6wVD5SObB8hWEaOh409DfSlDJnSMYyNyLQFmC5iS3K4bXNeowFn7+GZ0peXdFICtAEJphvLIyinRp06JSaY7axS0sd1idnHFBdfz3pM0rFlRME0ibS2LlFWOQuN5ABlErDOyFynEnWu9I62MiI64cNavWDZN31xvV5utyubx2l9HUnMAuufMn1dDM5CGRaO2yvW41KNIyM5wDJ/VG/saSBJJejQUogYxkpsrFhU6olJ41ByHoO+hrp+t5KWxLw57gc4byIiIZwvv36OaouONMLGzD7j2Cg31lwojTH7j+MjMDZKqYqM5CD2s/5qffDEk/gWMJ80ZsEngIQQQgghUww+ASSEEEJIbuCTuZzBJ4CEEEIIIVMMPgEcC8+TKD+jJfKnaTsKQS2X0T75YN/gV1aoNrTssPYHfjFElA1p7ZPS/Vk9SJfTvoQfbXC7v8Po4kCbkq3zA+sAs19K62ItYlALaa1lIBYqshYH+F1f//sjqnI6Nm8Q9qVVxxoFp5wWLjLRTwL2MRgLJ6JjvHwT74TatJGl2rIh/zdu+yOrFqm2ZCfESx3WtgxKv1nrxlGc0HowtNTwlyxQbXLK2cDY84O6Qs/EXAUYx2Yj8XDcGl1hnAYd0IqlbpndekwpG5MKE0MHetTA6Dd9PP9GY4g2PQljt4MxbmL6z5sJc8TEEqLlEsaVhSZCEKPGMGJPRCQCKx6/Yalq86FvrfVGXADa3uPG3gnHra+PJwArlviU3k+ZWeva2ty4TMzX9icYJxZcpMdsVAxWTDbeDaxeUPMnIpJe6daTeEtbUqVBS5qIanUb9HUwDdrM9ROj07DvREQkBMubI9pSJ4R5kTIxfskWWK5XrxOvhZGxUULtaJZmF/S2qNm215uwyelFwysu1vt1FrZt4gV90JaHJjYSIznjfUdUG17T4mX6OhIcc9fhaJY+P0OVmTlpIzcnC8/+XSOTBp8AEkIIIYRMMXgDSAghhBAyxWAJeCxGUuKdK+fZpAksd6ZPnVZtAaYXmPImJinYNAFMCcEymIhI4I1foo0g5QCLG9Y9XllODOhSIdqtRKZ8NhGYVhCeadVtaI1hysNoLYP7JSLid8CxQ5nP9gken7XXiKAs5hu7AzwHsbFsiCBpIn9no95nKO3l79eWGlg6srYmWF5VpXdbpsTz1aJtWVSJyaZ2QHlTlW5FJAY7j6ykFpQZmPKjSpc4CCVt019ocaJkESLausakSUShK6f5JqFCpUSY0pfadv/4Je3Ijm88vonGDZSYgxIzbmCfgwEt0Ug3uXKkLVt7IKEI+/V+2TQgtU6cT56+jnho9wL9nLZJIyDRSPSa9JJj5rsAyhGyrF6g7GvtdgT2KyvhA8cOjK/Q2u2gZYu1YYHjiYydC57LYJ+WYaTRpsmUGr0Bt19eUaFqi9rhOlJo7FbQqguvYWe71PewJBsc09dIlNdkXddhnNo5IodcudguF4KsINGiZTNoa+MbK7DKtsz4CAb07ycNVoBzBp8AEkIIIYRMMfgEkBBCCCE5YdKMoEkWfAJICCGEEDLF4BPAcTgfWWT/cYK6O7SVEBEJQTfiGZsRvQ6tYcL4Lb9AL4dRY4HRDqLmEGO7fLNf2Gb1YKifsXo91D6hBkdEx1fZCCTU0yldpCE22jHcTw+ONWsdqMNs0TpMgf3KiigDHV6W1hLj3qw9BdhAhDaibIKIv6DMaYfSLU4LZ/sLyYqg8kELZ7aFEXI2ljDE8WA0bajf8lPGugT6D8+jBY8tMn2C/RxZ7RvoysIeY4cExxMNjR8FJyY6Dcdplv4Mxjf2rR1TqH9NN2vroqDKjY300Ua9frALCZuN1Qtq1cz8UdpEe17hfFltGsYs4jywcXJq7JkoONy2vYbFoMvDeDcRbfUiJnpO9afZFzwHMV4brCUVXlMCo6GOndYuMVPbmGCkZJbFFuiRbVxeiDZKE8Sg2eXiceaF1ZX6cE3BOE4RrRee6O+IPa84R4KaGtUWw98mG+OI68nSiBdkxnBOouBEqAHMIXwCSAghhBAyxeATQEIIIYRMPvEkagD5pDEL3gCOQVScL71rM87tZb+1JcbxS36ycPboR/+IsQtBCwpTavVnzRj9HOfrklwANgM9yypVW9m+Lvc9LAeV6nKgj0kNJkUhbnRu9baM6EEJK7VMW1wk3tjnvjdvtmoLTruSzKlP6+SEWS+67Ykpd45c7dJMCo649ID2a/X6K/e6MphvSrky333X69RWIlhy9ouK9HJQ/lDJEiJy7P92yRDznzbJDHCsYlNjIEEiWFjvft+pj9uDFAq0sRER8TDZxJTMVSl5ui4H+VDGHrlCJ5vkdUBJ65BJIYCyXO+1Lvmh5Ke79baLof8Wz9FtrVBqLdZjCpNuImP10n3zytHPla/r+YOpKnb+hNNgXpg0A2+vTrM4z+Dl8/X3IGmib7Yu0ddudSXhRKEuyYZQfgxmzlJtqZluv2zaQXDIzYMsWxs4vv4r6lVbfqc7r/42Nwd9s19oXeSv0CkU6VInMxmYpiUnJS/vdftVps8dJnzYUiuWbxvva1BNs3/lZABDta5vSw906XXAmPWHzbiBsY+JQSIiAufAzt34FFrq6LEx8slV7nOpLoaVPP2GW8xINlJXuv4sOOzWP1Kv52Aq4daZ125K5sfh74q5bgQwl6PDjarNm+/mWmxKzl4MpWRj9ZIYcn0b1ujS+7FbMj+n/hdvB6YaPOOEEEIIyQ18MpczqAEkhBBCCJli8AaQEEIIIWSKwRLwGASDKSl7IxOXhK/ki+honjilX9FPgF2ItcZQr/ObeKf4LEQiWSsW0MXoQCIRaXPLpeG1/8DojTB+yVpOqEgio8lDW4O849pWIESLmJNaJxn1OW1X7XatW4t7XVtsIp0KDrj1xINON1T9ltHF9To9TTiobUb8E86qwtodoG4pMhoZDy0ojCar7leg0Tt0TK8TtqEVOdoCxUs5rZCNdMPIMGu94kdOCxWZ5XAUee06VgvPQcEhHS+n+sGMYYEotbI3XWRYaPoLdWt+k2mD8RfZGL9o/HpP+QH33ehM27jfs7F0PthfeEZzGMGcxHNVdKhdfU9gvua3aa1t3AcxZ/1mPMM5sfrNBFjLWDsXtCCx1xHUbxUf0vNO2t3PGP1lx5SfAougJm1rk4TrTzCodYs4boKz+nqQbnXnxOri8FqBmj8RkWSr64fkGTj/JjothnMQGusVvGYmOrR+N4TxFp/Qx6r61lwXCw+7MVBo9HR45bDnB3V/eM6T5rohqL3u6FJNEeqRO4wVE8YZmmtYdNRF3fnWRgcx4wHtq7yz+lox78UFIiLS0heJaIedDxxPJu8lkByZ3Pxec0GeAL711lvyN3/zN3LbbbdJXV2deJ436qM3Ed/73vdk9erVUlJSIlVVVXLjjTfKq6++OuEyr7zyitx4441SVVUlJSUlsnr1avn+979/IQ6DEEIIIWRKcEGeAD744IPy/PPPv6dl7r//fnn00UelsLBQPvnJT8rQ0JC8/PLL8tOf/lQ2bdokt9xyS9YyzzzzjHz605+WKIrkuuuuk5qaGvnZz34mGzZskF27dsk3vvGNC3E4hBBCCJkM+BJIzrggN4BXXnmlLF++XK644gq54oorpL6+XobNI2hky5Yt8uijj0p1dbW89tprsnjxYhERee2112TNmjXy2c9+VtasWSMVFRWjy5w9e1buvvtuCcNQnnnmGbnttttEROTMmTNyzTXXyCOPPCKf+tSnZM2aNe/7eOJkIKl5mWfhyaTpIrCjkKR294+q3SN/r9ykLxxqHP3sV1XodVa5R/lY3hTR1izpEm3ZkDgLpZyLnWWHmCQLD5MSjMN+UOr2E8uzmQXdU1y7HB6DfdqL1iUDlbpUlIDlAlMOHKl39QdvglJhEkurpmSOlgqxPXdgqRCU64I6lkWtrU3fbNfvlUsW6HVCGR4TNkREPLTKAPuGwKZjQL8niqertqjGlSODZlPKhTlmS3I+JNFElWYsDsKxDuh9wQSG9GzXl4FNd0DboaIC1YY2N5iGkNkZKGmZcZMCe5K8ygq9GFrgGKuP4dlu/uSfMrZAPZA8AdY1UakuyXpgv9S3QI+NsnZ3PFGdto/xD4AkYPE81ZYuduck2axLuSo1wpT5sM/iQlNqrXFWH56yBTJjPemOJ7xI71dwFsrPSV36xLQhO6+DaVAfNPMT0zHQ6kVEl32H5laMfi40NkDRxa5vE6dMkgWM9bjA9AmMfW+OTi9Bm6vIyEVSM8YvofoH3bHb6/XwAtcPSbDcivKNCMR3cymcqdeRdxKOz8pRytw5iM31zauEfbZSHyjzps05TzSCNKZSj+/W1ZntpQ/xlYCpxgU543/9138tX/va1+Smm26SGTNmvOP3v/nNb4qIyJe//OXRmz+RzI3kvffeK11dXfLd735XLfOd73xHenp65Oabbx69+RMRmT59unz9618XEZFHHnnkQhwOIYQQQiYBL56c/0g2k37LPzg4KD//+c9FRGT9+vVZ7ed/98ILL6jfv/jii+Mus27dOikoKJAtW7bI0EQZooQQQgghZPJvAA8cOCDDw8NSW1srdXV1We2rVmXc2Xft2qV+v3PnTtWO5OXlySWXXCJDQ0Ny8ODBD2CvCSGEEHLBiePJ+Y9kMek2MMePZ15jH+vmT0SkuLhYKioqpLOzU3p7e6W0tFR6enqku7t7wuXq6upk27Zt0tTUJMuXL39f++ilQkkeO6eZMAMHbWG8hsWqTUAPFBnNlIDmJ7R2Eai1MbqbdIuzRvGqdQRSNB30QC1ga1FrbCzOQJtZfzQwvp4OrSoCY10TgWWMV6C1iWhlUnS4QrWh9UPY1aWakntgN2Gd6TnamyBC3c3xk6otgOgnz2oAYZ1hp7ZCCGphG226rXKH+xzuP6KXq4b4JdO3aNOQ3r1/9LPV9ShrlFjrSlF7Z/sL9VoS6eVitOkx8YJ+N2jHzD6j5UXQCfFbxiIomOF0eN6Q1pzijIl6tCYPtYmhaUsedHMkbeaI2naHPj8FYLci+VofFqI1R6/T2iWMZjLq7Br9XOqbfxejtvPtA6rJg3Mc7tir27BvZ2ltmgd2UtGQsQUCHWZYoMdKonts+5issQ7jIdjbqNsgaszvM+cOtbDFJi4RtMVZ8wf6ISviDeY86v5S87VcKHHQ2Q5ZmyYfNbsmjjNOwXdPafsgD8abZ+LrEr3Q74ePq7YIrvvhmVbVloRrJl7rsiI+QcvpT6tSbeHJU67N2rmA3jE0tlAe/OzPn6vbYEwFnVpLrq4HrXpuFZ3J7Jtv/mSRDz+TfgPYd85nqshmsQLFxcXS1dU1egPYB95U4y1XfM7/q9dMmPFoaGgY8/dHjhyRpBSP2UYIIYSQC8Rk6vP4EDALvvZDCCGEEDLFmPQngCUlGUuKAev0DvSfc9svPVfiOr/M+eXKyrIyMbKWeSf27Nkz5u8bGhqksaldBi/NlJrzW/V++vCSiddjjmGms/DoXK1LPuWb33brmKPd91PT3PEkTLkmMc+VvOOjxuUeyoNnb3ZPNCt369LaRJbcQ9cuG/1c+Ob45c2sUt7ierf+TrM9KDGJsY8ZWe6Wy2/S5xFtGc5e7J70TnvdpCG0uJIMlhRFRKKFzm6ld7EeC+U/cSW6RL0un2AKQc+12uqj8LQryQzd8hHVVrbdlXKiVp0u4YOVSfr/utx9LtJ2EUU/2z36OVypZQUp6L9kqy7tR0ca3bbA/kZExC8D65cunV6BKFsJEYkhaQLlD0FtjYxHbNI+UO6QtRyURRM1uiwW+9BmymnpZjf2h2EMiYi0XOPGwPzHD+tdgeOLwT4mNtIOH8q+aN8iIhJAykrvrZertoptTqIxeM1C1YZPHAqff1O3YXnY7MvASjc2i3aeUG1xhRvTAVhEiW9mOcyL4YY5qqlgL8gmjBWLVwM2SsZOCi2WAmu/hJIXsxzOLWX1clAfW/P/e9Ho58oDugRcdARKzuaa4kF5uO+j9aqtdLsbN3ac+iOwz8ZayG9xtilevZYdjUxzcytd7MZ6fodJ34DPw9X6OuXVOZlS4Q5dfvbAmitRref8SB3MmSFds43fcte3wZv0OC3d4fatv0GX3rsWZY4h1MN+8uCTuZwx6U8A587NXNxOnjw5Znt/f790dXVJZWXl6M1cWVmZlJeXT7jc+d/PmzdvzHZCCCGEEJJh0m8Aly5dKvn5+dLW1ibNzc1Z7du3bxcRyXqRY8WKFaodSaVSsnv3bikoKJAlS5Z8AHtNCCGEEPLhYdJLwIWFhXL99dfLSy+9JE8//bTcf//9qn3Tpk0iInLTTTep369bt062bt0qmzZtks985jOqbfPmzTI0NCSf+tSnpKDApBL8jnjpzHNp65SP4faJQhPwDuWzshqTNIElEngDTES/BawSD0TEr4XSnkmawLfFkv3uObrfoUsdIZY+TOh5/llw2DdO+ZgmkBVYD2/bxTN0mc/rg/WYN0TzIekE37wUEfGrXOmj+m1XmooKTcIG9KV6u01Egja3zuJCM7yhTBabt1rxbU4s+YqIJNvcvgzX6DpJjG+d2jdxISEjvwXOwSw9NrwiN478EZMK0eRKWJ4pfakxZcLm1RucfeaNQHzr1KRQqO+VuP2KGnW5zoeXsWKzDkxtCIoKTZsbD3GbLpnjPmdVhXw3bguO6jc9ixa55Jaw1bwFiuVoePM3xhK5iMTFbj8DI7WI4C3j0kbdl1G5K8MWtuj5E+XBXPP1vMP9ilOmZApfjc0brzaBY3R9VebNf0hAyT/ZpVcB6TLR3kOqLTEN5rJJc4oL4Hpg5w+MTX9Yv0QXgtwHEz7sm75Y9h2q1P1VCNtWjgei31AvPGVSdgrhLWD7pjS+DWtL2nh+TujrdX6nO748KMmnK/QLiol2dw6KW/U1OYXJIDY1C/4ehCf1saaWuVJ1XqMe63iFLmg35Xvsa6MWKDibGVP++JeCDxQveufvkA+GnLwEsnHjRhEReeihh+TQIXcBeu211+Txxx+XiooK+dznPqeWueeee6SsrEyef/55efbZZ0d/39raKg888ICIiHzpS1+ahL0nhBBCCPnD5oLcAL744ovy0Y9+dPS/kXN+S/i780keIiJr166VL37xi9LR0SErV66UW265RW688Ua57rrrJJ1Oy5NPPqlygEVEqqqq5IknnhDf92X9+vVy/fXXyx133CFLly6Vw4cPy8aNGy9IDjAhhBBCJol4kv4jWVyQEnBbW5u88cYbWb/H37W16cfV3/72t2XlypXy2GOPycsvvyx5eXmydu1a+cpXviJXXXXVmNu5/fbbZevWrfLQQw/J66+/LiMjI7Js2TK57777ZMOGDRfiUAghhBBCPvRckBvAu+66S+66665JWe7qq6+Wl1566T1v673gpULJP5GxHYiLtaYwhkSMLB3RGXeTm2jRbuuR0vxoPZ0/6DQgvjW6BgsKa9nhgUYvvxM0HgmjN8qfQAfTDZomY6ETo+WNtZlA3VebtmlBbVpsdD6e0YQhfg847JeCq327sRkpA91Nv7HiQeuSHqNhwn43ujUPtGnBsBHDQLpIienbON/1p2c0oR4cewx2G/lG1xVCUkKixJiQgzYty2YG9K6xTZ7Bc2fW6YGWNDzdatqcTs5DmwyTnIFj0TO6tAi1fbafYdv2H+XhWTfWA2vnFMF6THpJzQ7Xt0G51lei/jWGsZJlXQP6Td9Y1/jQt16HNppHjVmiw9rhuGtFVGi0yWA7E6eN5ckxSNkpMssNgKUK7teA0b7BePZtcgbo6Wx/KS2c0QtLOH5iTRyDvtbomAWvmaB388220eoFNX8iInGe2y/faGFR22s1tB6mpZjj8fA60qTdJbKuk7gvQ9DW6vo52aXHBuq5Uyu0RRDqiu3fg3gI9JWmn4uOdsH39PUtgL9Hw/n6WJNwXcR1iIgM1Jwb7zl6SjZpRtAkCxpBE0IIIYRMMSb9LWBCCCGEkIw+b5IeAfJJYxa8ARyLwJd0baY8kDzVpZpiLA2YUoRf4MpBUYcui6LlgS2nRlD68mdoR/o02G8kTDg72ozk7TzmGspNKRdLssZywIMyglen00vweMSUEbH8gE7/dnveUp2qIQOwPVu2hFKoHzjH+3Bahf5e2pVFPFMWDWc4i4sgq1zn+ivq1QkIWO7yhnVJTua4fomPG4sQKCvaElOAFjGqZK7lAbhcZEqyaHNj7VaCqgq33Bzt7h/tcKkAwXRd0hQoCdoAe7WfYC0TmeQefxjGWIFJOZjrbFmwZJnZ9viJ84k6t1xkLGImSs4IzrpzacvwAZSnQyjJBYvNuOxwc9DaIXlQjk5bOxwYU/bviwdyDjs2VMne/gEEe5KwXs/JoBn6BdaZPn1GfS8xw6US2TGFMoz0Mm2cnzjovFmjep1Y5B9xxx6Zsn9iZq1rq9LXn0QH2PsUwJywFjdwPbVWL1j27b9cp/gUboXSvpEHxF2uLB+ZkqkPPwdGXqPkCLZMjtd9vLbacTPD9UmyQ5fFo6Mu/cNbpsvD3jE4B1ZCAzZa6fl6zida3N+c/G3a3kfJBVJ6nUEqYzXGUuzUgzeAhBBCCMkJvPHMHdQAEkIIIYRMMXgDSAghhBAyxWAJeCwGh8V7fbeIiITWCgFey7cRQWgl4VndyBmnw/HLtN5EwKYlPG7sCEAvGFnrinanFUL7DqutUfojo5HB+K0QdCkioi0IrO4KotMiowHE5VDPIiIiYF1il/OhHwTtG/Ye1esAC43IWGjIdqd9k+laT4mRXtZeIQTNmdeuNXoxRuJZnQ/GyxmNXgRxXNhmdXcYBRaHerwlpjsdUdg2fgSWZ6w38GzFx7RuTdl7+CZeDnRREY7ZYqMBBZuR0Ogw0ULDak7197S1DGphrfYJx1/cb84BaC3tHFH9jlZMLVozF/W6cxVUmlg11A6aeY3aSDV+RSSGdUZDZo6YeahAvdaBJtUUwviLx4mFExEJ8dpgIytBnxzs0pFhYR/oKXtMpOSI+67VNCrNphmnIZw7vE7FRouG18zIRM2h1Qtq/rLYe1jvF/ZRZOxwQL8b2XGK481oB708aFOWV7ovfTXPjAUN6GbjvUf0PsP1wDfRpulmp0EOerTGOQ3jNDFH6zfDE24530RWVr2SuUYHA2bOTRYsAecMPgEkhBBCCJli8AkgIYQQQiYdTybvJZAJnrlPWXgDOAZRWaH0//FHRESk+LguNwQHoUxaW60XRCf4Cl0q8jGRwJQUsNwZ1OlH98pixVq4lDmrlIFLnYVG4asHZDw8Y9khYCXiz9QlU78PkxN0uQGtELJsEmqghGbc8aXSfTcwKQcxpF6MTHeu9vl9puQD5TNbdhlau3z0c9SjLUcSUDLxq2yZz5W+znx6mWqb8WNXQu2/7iLVVrzXlUnDE6bcDaSvbHD70WvO/2/3u88f0duOm1ypMqjR4y2G9AcPzqOIiAc2N5h4ICISlYEtxx5tF4ElU3/Fxa7hwDH1PYFEj+DiRbqt+fTox/CyparJH3LnxDfJGWidFCzQVh/hkUb3vXptf5EugUScRn1JC1vcvgSznDWKTUAJal2p3SvWJdMY7ImiJXq/0JYlPadWtY1UufFd8Mu39TrR3ielx7AH5XbPlOuiBXWuDc6JV6BTibDsG87S4yboAysmm0Kx1NnCWBul1MwK17ZPy0VwHnrmOhJD+dEDSyU5peNB+z5aP/q58JQu82PCh7V6wbJv+/+zSjVNf9lJakJT9seSs2+kHdjr/nR9XvE6la5ycylrXp902+u5boFqKj3kStxBW5eMR2SkHdEfuTmZOK6txgKUo5hrq5KSzNa2UCevyfRD6h91+gr58MMbQEIIIYTkhskygiZZUANICCGEEDLF4BNAQgghhOQEGkHnDt4AjkEwkJLyV87ZL5jH02mwJ0iUaN1Nus1pWhIJ/dp/iDYQRsMSVDstX1ZsE8RQ4WcREQEbhcJjzkLDar5isNew+kMf7DZi0G6JaNsHa3GCdg5BrdaURKi1MTYj3lmnrgmNrtCf53SMyTbQ/VnND+jPxOxX4c9Aa7WoXm8b7XaMVUUAmsDpv9TapHSzs/spKjQaSozEM7YcAZyH5JtO5+dP0/2FVkN+k9YpoQVJljUKaH5sJJ4P247atVbIB5sgE8alzqvX5LRbodl2Aq14GrV1EVpjBHsbVRvabaSN/gyteXyj18J5GLR2qaagHfbllOk/GMO2H9T30LKls1O1+WB5pKLYROvBMIpLRCQ4BlY5NtIrOcGlF/o2Nsv5rW7fQrSLydc6XKXJO6j1enh9iIyViN/prm+xsStKtrhzkO7q0suB5jA+ZeIM4TzHMFY8Y5tTut2Nt9jMM9QqYrybiLZ6Qc2fiEgMmmfPRHeGrW6ex2Ysos1N2KztvnCeJyAK0DNxmXhtKn9T64MxEs+eA7ROik10Yt4JsEoyY11da/O1ni884441KNX7WbMr00dHh3knNtXgDSAhhBBCJp9YJs8HkPe3WVADSAghhBAyxeATwDGI02lXIrRJIIAtIyLpM23jttmyMrre21KrQAnIz9OP9dFKwoMyQmStHTAtwzPJD1BytmWQiZJAsMQUnjZlNzwG+4ZX3/huTPHJU2P+PjRlEDwntkSibEwON+r1YDnNpAKEWB48q0uA+N2oySS1YHnapot0drkmLKebUjvagKAVSmaV4ycZKDsce36wZGbGFMoAbP9h2WqiFAXVX+a4cZ2hHVOIPR5sMrYcattZ5WGwVLHzB/oFUzuyxjP+YCQHIdjAYPKLiIjXDaU8Y0mkzp3town6RVnUmOXUOvBcwT6KTDzn1Tm2yTY4t4ykwe+F64+9hg1MINnAcwDb82x/gVWWLZHjfkXWRgvGUZbVC5R9J7pWZI2HCa59atP9TqLhGZsudS0y+4zz1VpZTbQ9LEdnpexA2khwVF+ncBxFJhko/5x8xBs0+zFJUAOYO/gEkBBCCCHkHRgYGJAf/vCH8rnPfU6WLl0qBQUFUlxcLCtWrJCvfe1r0gd+spbvfe97snr1aikpKZGqqiq58cYb5dVXX51we6+88orceOONUlVVJSUlJbJ69Wr5/ve/f8GOhzeAhBBCCCHvwFNPPSW33nqrPPHEExIEgfzJn/yJXHvttXLs2DH57//9v8sVV1whra2tWcvdf//98tnPflZ2794ta9euldWrV8vLL78s1113nfzwhz8cc1vPPPOMfOxjH5Of/OQnsnz5crnhhhvk0KFDsmHDBvmrv/qrC3I8vAEkhBBCSA6IRaJJ+u8CvAWSTCblz//8z2Xv3r2yd+9e+bd/+zf5yU9+IgcOHJDLLrtM9u/fL/fff79aZsuWLfLoo49KdXW17Ny5U374wx/KT37yE9m6dasEQSCf/exnpcu8VX/27Fm5++67JQxD2bRpk/zyl7+UTZs2yf79+2XRokXyyCOPyC9/+cv3fTzUAI5FXlL8+fPGbkPrkjJtAxMXOj2Q36u1NTFo9NBqQ0RH8yROa/0ZRpSNrFyo2vKPQAzZ9Aq3/rTWDfl9bl9iEwWn4t6sjgi0KN50bV0iYMUQm7gi1BxFq3W0WbLR7bPVu3kLXZ9HRa6PYiMpShx360if0f/aSsybAyvUC0aNzg7DS+pzgFoh39janL3WxW9V/nif3hmwdPHKdISYf8zZWkjCWYmIsfaIB+HfYUmt80x/ZLFr2qZj29AGCK1KRETC9g73tetXqrbC/U6DGA9b/ZEbO0Mr690yR4z9CVpq5Ou+xCi4LNuhyorRz1kxVxe77fn7GvU6IRLN2uiM1DkbpWDAxKrtMxF2539fN1P/otON51N3LlZNs55vGv0clxmrj1bXz76xXxKYazaaS0CrFmHco4gEM11kXXpGhW7rdPM1Ogr2Lv742troqkvVz34f9FGB0dq9fcT9YC1oYIx5A+YaBudy5JM6jq3wsBs7qRluHTY6zR8BjR70q4i2rsnS0/W7PvFMLKWyejH60M4NHx39XPMbfS0K9x502ysqUm2p1S4OMm8f2NoYyyu0COq9VMfjlb3WOPo56tdRlziO4iE9NnDOBA06ZtHrdn8r+i6brdqKjrnx7XdqG52Td9aLiEj6HzcLmZgNGzbIhg0bsn4/c+ZM+fu//3u56qqr5Nlnn5WRkRHJO/d3vnzx9AAAMytJREFU/pvf/KaIiHz5y1+WxYvdteXKK6+Ue++9V/7u7/5Ovvvd78qXvvSl0bbvfOc70tPTIzfffLPcdttto7+fPn26fP3rX5fbbrtNHnnkEVmzZs37Oh4+ASSEEEJIbogn6b8PmBUrVoiIyPDwsHR0ZP4BMzg4KD//+c9FRGT9+vVZy5z/3QsvvKB+/+KLL467zLp166SgoEC2bNkiQ+YfCO8V3gASQgghhLwPjh49KiKZMnFVVaYqceDAARkeHpba2lqpq6vLWmbVqszT8l27dqnf79y5U7UjeXl5cskll8jQ0JAcPHgwq/29wBLwWAS+hFWZUk9wQL8y70G5K32kSbX5y+GRvHmVPwRrkcAkegTtUE415QAsMea1mZIZ2ND4aCVg7A48KCt6Q8Z6Af4FYUtRPrjc20SPGL7rQ5KJiEg84ErOg5W6VJQ8Or6tBZZ9sU9SMyv096CPMIFARETS41s7oJWEX6yXi/qhTG5sGdL5UF4z5Ue0hbFl2AhsLfyF9W4/evSbYl6xKzHF5i2y/P2QjmDtScAGwgtNv0IZvvCwtiSKwYoja7zBWMnrAruYDiNNgHKaZ0pkaHGSJXeAeWDbUhWuZFpoUhWUdU2f3udgyJX9ghZdOsRe8SDZIio3+1zltldxWJ//COx8PJNQEYKMIbhokWpLV7htJE1KSNg/vs0NSjECa4eC9jFoVRLp+elDykZwVveXNwjjZticH5jXgZG4YGKFV6TnD9rjjJTqfSkcz0rrsE4okZlQJjUpJGmYZ0GlmWcwD3xja6Psdsz1AMu+A/N06Th/D8hfTGKJkthUuX1Jl5trEdjolL2p/44oeY3pS4EUKZsEg/KAuNVYRoGdVMmOFt0GpeN4WrVqKz+a6SM/Ny4wHxobmEcffVRERG644QbJPzdmjh/PjPGxbv5ERIqLi6WiokI6Ozult7dXSktLpaenR7q7uydcrq6uTrZt2yZNTU2yfPny33mfeQNICCGEkA89R44ckYaGhjHb9uzZ8zuv98c//rF897vflWQyKQ8++ODo78/bwhSZfyQjxcXF0tXVNXoDiFYy4y1XXJz5B2tvb++Y7e8W3gASQgghZPKJZULj6wu+rQ+A/fv3y2c+8xmJ41j+9m//dlQL+IcAbwAJIYQQ8qFn4cKF7+tJn6W5uVluuOEG6ezslI0bN8oXv/hF1V5SkpFRDAwMjLW4iIj0n0uTKT0nDTu/zPnlysrK3nGZ3xXeAI7F0Ih4b+0XEZHI2CvEnaCF8rW2Jdqx1/1QYGwf4F85oXls68HgyIrmAh1ZViwUrNMH7VZsdTB2nQBaHKCGREQkxDipHm0dgMdutYNIweY31c9p7DNzPN5vncVKBPow/0Sz+l40QYQTRk35RruDurWwWx+PipqCaD4Rkarvve7WYdaJOp/QWNKg1Ux0pNH93mjfMEbL6o1ioxfUjbDtdhNLCBrAdJPWH00IaEK9Hfvd+u0YQj1iv7EBGu97Bqu1TP502+jnCQLklJZPRMSHcZs2505/EcbeW3t1G4zFPDumcNvH9VjEMaZsWUTEh7GRTr17gVUMkY/RYWNjg7FusM+ekdlFw27uegePjr8te16hj0LTl2o8t2utJVLy9BvqZzRf8Q+68RCZfvYxxs3oBjEaLrRRjRg1Z/ZFRdvZqEawekHNX2ZnsB+6dNMr7ucI1m9HOkbphTZCEPWo7+VNTrB+iow+ECPxrFXORDrJ4nP97vebaLlJ4g9VA3j27Fn55Cc/KU1NTfLZz35WvvGNb2R9Z+7cuSIicvLkyaw2kcyNXFdXl1RWVo7ezJWVlUl5ebl0d3fLyZMnZdmyZVnLnV/fvHnj2NW9S/gWMCGEEELIu6Svr0/+03/6T7J371657bbb5P/8n/+jc+HPsXTpUsnPz5e2tjZpbm7Oat++fbuISNaLHOfLyOfbkVQqJbt375aCggJZsmTJ+zoO3gASQgghJDf8gfkADg8Py8033yxvvvmm/PEf/7H84Ac/kGCcN90LCwvl+uuvFxGRp59+Oqt906ZNIiJy0003qd+vW7dOtSObN2+WoaEhWbt2rRTYSuN7hCXgsfB98cszj2NjU97Eu3xbPvHLK90PtdoaRQ65MkxQUaHbIFXBlh+xJBhZHQHsi1/tth2Z6oyXMqUC3Ocq2OfIFFCwLGq2HVRVjH4OTYII2kfExp7EB6d+m0Lhgf1BVOvWL3sP6++NZyshxoqlwmgnoAwb4HGLSNjp+j1RN121xZD+4hl7kvB0du7j6HJQcg4WzHXrSOlxEx2HsrWxksFycWzK1iglsOVuLPmosqGI+Gg7Y8qwcWrs4mtgdCg49u06EFvSFrSPMakNaB9kieBYg5kzdFu5G2/eBPMVt2f7Esfp+bk/un44//48bcsQHm50+7V0gd5psCQKD41fhp0Qc+7U2I/HvxZhvwcm2Sbd3DLm97LWb8E0lkJ97pSMwUgccEz5eN2wkol617fxiVPj7kZgxw2UO/3ptaotbIb12JIzyF9sP2DZNzF7lt4B/BtQAhYu7V0yHp5NgsHroimnY2nXHiv+rfDNfMFz4Ndoq5f4tCuv22vY6HVyaIJzT0REJAxD+dM//VP5+c9/Ltdee608++yzo4kf47Fx40Z56aWX5KGHHpJ169aNpoG89tpr8vjjj0tFRYV87nOfU8vcc8898vDDD8vzzz8vzz777GgaSGtrqzzwwAMiIio55HeFN4CEEEIIIe/AY489Js8995yIiNTU1MgXvvCFMb/3jW98Q2pqMv/wWrt2rXzxi1+URx99VFauXCmf+MQnZGRkRF5++WWJ41iefPJJqTAPhaqqquSJJ56QO++8U9avXy9r1qyR6upq2bJli3R1dcnGjRvfdwycCG8ACSGEEJIjvMmygbkAdMJLoOdvBMfiq1/96ugNoIjIt7/9bVm5cqU89thj8vLLL0teXp6sXbtWvvKVr8hVV1015jpuv/122bp1qzz00EPy+uuvy8jIiCxbtkzuu+++MfOIfxd4A0gIIYQQ8g589atfla9+9au/07J33XWX3HXXXe9pmauvvlpeeuml32l77wbeAI5FMpB4diaWyD9rnLbRcsDYEYRL57i2pNZTJJpBb2K0aWGF0zD5VstTWeE+gz2EiLadGKlzmsMsRQJqVuw68mEImOPxCpwuJgH7YdcZlBiNIehpwoUz9WKdoCU02sTUAqftShe7/SoY0q+6Y/xaFhATFZWZiCX8l6axEkFLjahEu693f9RpgMoO6PEQWN0kgBqndK0754kuo6cELyeMhRMRGZ7rzmv+YfMvZdDF2eXwX9X+HKNh6nW2LV6gNWZxnjueeAnoFju11YvV2iEqOs1qyiBe0GrF0jBWkk06vg71Z3GZ1jD1LXbnvHRIH2t8xtnj4HiWMv09H3RqHVfrtqofHxj9PLBQa3sLYCymqo0+tAjGsLHziSBCzuKXwvXAzjs4XyHYpqBFS6bR6Q/7L9XHU4zXgwKj14N12hgy1EZ6Rnwew1hMXXmxais47ObB8AKn0UsaDdvINHfc+Z02ChDmqxmzXp4b67GJeEQrFktq9UVuHWmjVQarF2tl1POR2aOfC9qc/jAo0n2J6xyaro8nvwXGA8wXEZHETBhjCTN/QDvo2RhMYGiJ1skWYLyc0SN2fCSjFwxbc3Q7MP4llHzA8C1gQgghhJApBp8AEkIIIWTS8eLJ0wD+oRpOf5DwBnBMPIkT5x6OmsGJKRvW/iJohzJIvS4VBVhOMS7tQSeUlU0bFh/iEy2qDZM7onx4xG/TENBKwpb86l3ZzbdlUSBs00kTwbTacb4pImADkzyhLQ7iUiiFF+pShD8SwvegVN2qt61sJUxJW3wokZmSjI9JEKW6JONBSS5Vq8upFbtc28A8bdNSdNKVt8IJynqpctgXkwoQHIPzWFWi2pKdUCZL2rI1lL7suMEEhJPaUsODkrNdDu1EgtMgCcD0FRGJIb3AWv2gjUWiWs+DGOxpbPkxeRzOszlWTJvxE2YM448dXaoNZQYxlvabT+vvQTmtcqdeB9prxInxEx1GKkzCS9J9N9lmvJki3Z96pW658IwuhaONim4w+5XnjifZp2UlWK6NzxrJiQqf11IFVfadIOEFS74iIhEcexLmXWQSkdLFbszmVZiIq1a41hqZDI7ZdJWeuwm4xkQmsSZvH0hJqvS8xoQPZfUiuuw7XOPOeVGzPqc+SC0S5loUloJNj2jQDslKO6Ia2M8jOuEHU5CS3VqigX8Tokrdt4Vtmf7zQ94hTTV4A0gIIYSQ3MD7zpxBDSAhhBBCyBSDTwDHIA48SZdlHtH73ebtuj54e8sGloPzf367cWmHElrUo0sffi24tpt1RuWuZGLf2PSgJBwMjp3gkNkAlGvM+v1eLOXpEgaWEX2bBIFJCjN0OdiDEmB6pk7cSLS4Mmncp9+MTMEbgvlnoOxWP1t9z8fSh3kL14N0kaDPJI3AW6Beny5v+eCOn+jTbz8OzYKSial8eflufPimpI3l6PyO8d+a9aFMGg/qbXtQ0rSlVvW9Cd62teVblbhi3zqHN3PDWW5cenvHT7KwyQIelpVtqRC3167fOo9nTXOL9ev5g33rd+tSXjAM58eWqjHhowtKnzYxogTGhnkjVOBN0vwOU2qHY8/rNW+1w5vsQaUpMcL8yXpTFY41qNbzJ8aUEijrBeZtYUzZCQZMUsts94aod0bLK7A0nZXogW+PmvQfHH8j9Tp5JNnv5hpKVTwjOcG+TVfo0mcSz92gHhsoxUn0jn9+vGHTBvMzXa7LvGrUmoQPfNsXy76pMvMW8LDrr0S7vuZHUFZWb6eLLrVH5g3h1CKXUmTELzoNyI5hOD/pcr29wZrMeYiC8cv6Hyh/QD6AHzb4BJAQQgghZIrBG0BCCCGEkCkGS8CEEEIIyQm0Z8kdvAEcAy8dSV7rOZ1RV49uBL0OOvaLiLblMFYVqK2xy4W1Th/kW20N6J2iMq2LwfUEp50eKFw0R30vOAW6O+PurxIWbBtoZGwKCdoTxEaHF3U4bdfwsumqzR8GLaHRtwRDTh8WFrvtxUndl3mgOcTkAhFjgWPsDrxipweKTQqBh+kLx/U601fWj34u3WUsLtqdxYXVTHllsP3WLrdt48SvdIwntGVLPB30VEavhekPnrEHiU67/fRnTFNtAlrVoKZaNaGlht/iji22x1bkNEyRsR1CqxJrH4R6Lb9Ij+d0JViX2H6A/YqNRUyMP1prIdC/4jkfXqoTavJPO41WZCw7glluDPudWjsqoCsLerTWLtEBY8xoE5U1U6znXVQCx2DGit/prkeooQs7tJ4SUzsCk+ISowZ5mj7/4b5Dbv/n6etICKknSm8mIn6l0yqm7LUPk4/gmoLLiIhgD2Vp5vogvcZojn3Uv57Ucxf73WqcVWqI0WFOlCCCGlG0ekHNn4jI4Gx3Dor3atshv9et3zN/D+JuSPgxYz2vucv9MNPM69POMijKM3/awUYp0a21kDJHn0sydeANICGEEEJyA18CyRnUABJCCCGETDH4BHAsPE+ignOvxi+uU03JRldiSM/RpYig25V8sIQpIpLEUpspU0VQMglqdHKCQOpFukIvl0y78mDHSldOKTumy5syzbVFheOf8oQ/V/3sQSkCkx9ERGLcz3adgOGXu5JP8X6dZBBWQ2mqVpefhiD9o3eu+zx98zG9bUgaiY2NSdSwYPTz4Ext7VC8zx2Pb20zICWg80YdZl+xx5U4UzP1ctFc1w/5TSb1pNfZ3ISLnJWNb6xewlOuPOQvv0i19Vzktlfxa2Ndg2U4k+jhVzhZQXqatiBJl7oSZ/728e1dRua7ElOiR5//VJkbi16o50FiX6PbjzmzVFtcDMv163XmHYayr5kH8bGm0c8DS/S48WAIjCwz8/WN/W5foBSefGW3+h7aNHX+6RWqrRpsVHou1mXL8mGwPKoYv5SW325KuVACDru1zMTvdOMmKjdykRmwfZAf+GVGjgLzoutyLcMoaXJz0DOlzmDZEveDSVUJr3DzIjhmpBAg58hrN2VyWE8I88efps/xcLXrv+JW3SepFQtHPyc7dElbIOGn57oFqqn8zWb3NTNHei9146HsTZ2qEYJ0wDNl+KHpYBkFcgFbtsay76F79bis3+yu0Yk9+vqGYGqPiEgMT8vQdkxEREBycvpKvdysPrDSMk/c2i/L/Bz++7i78cERi3jju1td8G0RDZ8AEkIIIYRMMfgEkBBCCCE5IJ5EDSAfAVr4BJAQQgghZIrBJ4Bj4IWhBO0ZDUpgo9nAQiHYb7QuoO0batA6pQBjm6yFQsppOdKgdRLRmpy8Rm2pEaLVxyVOG5Ro1xFrGKsVmGiu/uVOo+W/uUcvBzYTkdEAJkDnJ0ZPp+xwbOxdIRz7sImogn1L9sE6rNbllDtuG4Hmgf5MZmktn4rBMzFUApYqpY0mhuyM0zgOzJmn2ooanVYpbDFWD6DD8+AcD83Ueq2CAxD9VKC1o8UnYV/MPqN20MZqBXXuvHo7D+rt1TrtqI2CQ/KOwPGYfs4DeyIchyIiIVi9BNZOA7R2XoXWJqLO1LMRcjCm8tv0WOxe7DRZxVsPqzbU9sUw9rIse2B71dtMPBqMt+hSo02EdQ5X6nOX1w363TO6j9S4jfT4DkGz6e05otow8i2CffYKtd4V513RKa19S5xyljH23AlEp4UD+vqWPOv2y1pGRaCvC47reYAWLnkn3VwKT2qrH69u+ehnq7VNtoEu8uhxvRxEqZUe0pZEcYE7z7F50lT2WuO4bWo+mQjG/BbQ9pa6bWO8m4i2ekHNn4jIUC3EyfUZTSPoQ4MCrT8MZ7vx5+/Q8zoCS6KCDm3hI81Ou55atUg1TX8z8/8zRjo+afDBXM64IE8A33rrLfmbv/kbue2226Surk48z8u+gJ8jiiL59a9/LQ888IBcfvnlUlpaKvn5+bJw4UK599575dix8QWxIiKvvPKK3HjjjVJVVSUlJSWyevVq+f73v38hDoMQQgghZEpwQZ4APvjgg/L888+/q+8ePXpUrrvuOhERmTFjhlx//fUSBIG8+eab8vjjj8tTTz0lP/7xj+Waa67JWvaZZ56RT3/60xJFkVx33XVSU1MjP/vZz2TDhg2ya9cu+cY3vnEhDocQQgghk4BHH8Cc4cX22ffvwP/8n/9T+vv75YorrpArrrhC6uvrZXh4OOuxuojIkSNH5POf/7z81//6X+XjH//46JPC4eFhuffee+V73/uezJ07Vw4fPizJpCupnD17VubPny89PT3yzDPPyG233SYiImfOnJFrrrlGDh8+LL/4xS9kzZo17+tYGhoapOlIq1yz+P/L/OK0tjGJwLLBpiiEmGxhXeex1GaejnoJd5y+KYtFPbA9W2qFMlLY6spWmAKQ2Wewc0mbtA8sN1TqbSt3/EA/LMZ1+sW69BFBydlfXK/XCZYQkUk9UfsN+xXPNLYfA67UETU1q7ZgOqSEDJqaBhxPZMouqlyb0OXOuARST+x46AXXflOG9aGcFmMZySSnYHkzMlYVqjxoxg0mafjGUifd5GwtEjO0DQimMeC4ybS5bfggaQht2gfYU9jj9vKh7JbS4y3ud2XFyJQREzPdfobtxlIH7JAClB+IiEBKjU14weSRGEpkiQX1eh0gR4iMbAGPx+6XSmOxaTlQZsb9EDHJJsPGwgcSa7zZM/R+QuIHjmG7Drw2hWaeBWgZM11b+MQnWty2jYVP1OjGlD2eoBpK4yZdBOc8jpXs5CEY3+Z48PzLIm1XFe91ZXJro4XnMjIlbb/E9QMm24iIhFCy900Z1sohRtdRoKU9mPARm/mD586fb46n2ZXQ7T6ruZyvZQxqbnXrMYwpKPY6n1qWkbW8se1RmVdfI3v2aCnQB0VDQ4M0HWuTq1b85aRs79Wd/0vmza+dtOP7Q+CCPAH867/+63f93YULF8pPf/rTrN/n5+fLP/zDP8hzzz0nx48fl1dffVU+9rGPjbZ/5zvfkZ6eHrn55ptHb/5ERKZPny5f//rX5bbbbpNHHnnkfd8AEkIIIYR82Pm9egu4sLBQlizJvPTQ0tKi2l588UUREVm/fn3WcuvWrZOCggLZsmWLDJmXFQghhBDye0ocT85/JIvfqxvAKIqkqSnzFuyMGbr0sXPnThERWbVqVdZyeXl5cskll8jQ0JAcPHgwq50QQgghhDh+r2xgfvCDH0hra6vU1tbKVVddNfr7np4e6e7OaCjq6urGXLaurk62bdsmTU1Nsnz58jG/867xROJz8Ww2KklZo1iLkzKnTYqM5kNpX4z1hgf6utjqVMAyxsaxRYPuZ9SbWc3PeOsT0TqsyERSoQ7PahPjFGirUkb7BMfjDWotD2rJcB0iIoL7Av3g2zgstIGIx88Rsjo/pVvy9L99wg5nT5EwuqiowvVt0KfPD67T9m3Y4+wiUHeDWp1Mo9MU+TYmEPvBnjuIr8uyNUGdktGmhS3OEsL2XzwCmsMFoE2yGibQPgZmbISgU8vSReHPRueF2kSrKxxv2yIiwUz3j8WJNIAYGSaDplIA/eVNr1FN4ZFGtwrQXYpozaYX6jGldGWmn3Huqv0SUf0gZi57qCuFMWt1anj9Caoq9DrwmmJiHFGfGsFxi+hrTGQ0zqiNDEz/4TnxIK7Mnn/UzFm9azzkjsc7pnW/kb2O4HK4DfMUyMd9SYyt6xMZS6vqdIbq2Mw5iI0OT4Ea52ZjHwV6ysjomFFTa302IoietOMNNZpW4xwMnft7lIunZLGIMAouZ/ze3ACeOHFC7r//fhER+drXvib5cJHq63MDu8hcgM9TfE443ds7waQDGhoaxvz9kSNHJOGVjtlGCCGEEPJh4PfiBrC/v19uu+02aW9vl1tuuUXuvffeXO8SIYQQQj5gaAOTO3J+A5hKpeSOO+6Qbdu2yTXXXCNPPfVU1ndK4HX9gYEBKSsry/pO/7lyWGnpu3t6N96r4A0NDdK096TEew6JiEhsSoVY3rSWHQGU7/x6XaoOj7iED2sfg+Wu9Okzqg1LO7FxpMcSqgfpDoEt66GFgq+PJwHWMulTuhSBpamwTdufBJUueSTs7FRtWGKKTpnjgXJhbB3woVTpz6x36z+o0xBUqcKUisIzbj8DU8pNn3SlI5VIIrrUau1j/KNuudhKAmD7oT0ePA9wjrGMJ2LKZ2ZsJ+a75JGsch1IAkJbvodth806cUGVOwNjJQFjOjxw1C1SXTX+90x5GM9J3GvKczCmrJ0Lli2tTAJLecH0abqtCEpvHbqPlMUSyiRMX+JcSszRcxfL617dTNUmh1wf+Wa8obVMVh9NBJRhQ3M8qiQM/WyTehCbEpI+cdK1mVI72oXYkiBKGqxUAc9XdLhR7wCmscD2QlOtSVTDNeWksSeC8WxLsniNzJJ9TJB0g5KaLEsaIMt2CEqoXrGrSEVoAyYiXhLmvPm7hAkf9pqPZd+gqlK3wTy3sg9lcwTXDRFzHTbLBS2Z6w+mFZGpQU5fAomiSDZs2CAvvfSSrFy5Ul544QUpNBcrEZGysjIpL8/cOJw8eTKrHX8/b968MdsJIYQQ8nsG3wLOGTm9AfzLv/xL+cEPfiBLliyRf//3f5eKiopxv7tixQoREdm+fXtWWyqVkt27d0tBQcGojQwhhBBCCBmbnN0AfvnLX5Z/+Id/kLlz58rLL78s06ZNm/D769atExGRTZs2ZbVt3rxZhoaGZO3atVJgXdsJIYQQ8vsJnwDmjJxoAL/1rW/Jww8/LDNmzJAtW7bI3Llz33GZe+65Rx5++GF5/vnn5dlnnx1NA2ltbZUHHnhARES+9KUvXZD9i0sKZOi6y0REpPCU0ZTsOTz6WUUqibH3mDn+DW1WNBPEeGEMlIiIN9fFMfldWjODNjSDFzkrjII3D+n9Ah2MZ+0VqpwmL0gbDYiPWq4+1YTrDC5erJcDzaFndIvYJsYiJNXgxkHQ5/RTWetvc5pDa5uDGq10pX5jPIC3ya0uygetDfaliEhBk9te76Va51X2W4htajyu2hKgVQvr3HLpEq3BydvnNEXRHD1uvH3HYJ/NP24m0NOhBYm1JMJjTRt9oA/nxJs/x+3/vsP6e7AvQa2xzYH4Qr9+jmrzMJbQ6NYwcs03xxOCTsrGdvVe5PSJRUf1OUBNGy5n525UDvYqjdqEHjVsUak+B8Gi+aOfh2dX6MUK3PnJ/0XXuOvM0qlB1FnQ3KrbKqBfWp12VNmr2PUbvWvQsNS1NWpJjYpSM3rXqBq2fahJtaEOD8eNiEgE58QDOyTPaABH6ty2U8v0PCg62gXL6TmfbnbnK/qji1Vb3gk3d60WFvWCwUwTl4gRhiYGU+CaFtW440kt0uvIa3b7bGNRw9nuWO0fYbR6sdZcfTdfPvq59Kd7VZvSXpt4ToFY1aHV+nraOycz79LParso8uHngtwAvvjii/Lggw+O/jxyznvrox/96OjvvvKVr8i6detkx44dozdq8+fPl4cffnjMdd5zzz1yzTXXjP5cVVUlTzzxhNx5552yfv16WbNmjVRXV8uWLVukq6tLNm7cyBg4QgghhJB3wQW5AWxra5M33ngj6/f4u7Zzb5F2dXWN/mvotddek9dee23Mda5Zs0bdAIqI3H777bJ161Z56KGH5PXXX5eRkRFZtmyZ3HfffbJhw4YLcSiEEEIImSwmywiaZHFBbgDvuusuueuuu97Vd9esWZP1OPy9cPXVV8tLL730Oy//bvDSkRS2ZMqFfrcuAasi6TTteO+1gWXDaW2bgqU1tOEQEfHg8bwYo+twP5Sca/T20Mm+cL8rRcYzjB3FUVeusT3vtbgSU1ZCCVqXlOqSmaDtgy27QfpCsEy/lOMNuLJf1Kb7NvHGvtHPPpRkUjMr1PeS3a6UG7brchD2bHBM29qgjU5orBfQ5qTwgG4bXujKUSU/3qna8NplLRvQ+sU/6Pooz5x/lWxhE2Sg323KBY6peK4uW4c7XV/K6kv1OmFfEqb0hVYWeK48X0sHlAWFGTfBNDf+ohOmnAoSB2vZ4S9wb/CnwV7l3A6MfoxNGktet9u+b0q7KDlIgxWGd/kyve3DUAqdrfvEb+9y296jLYlimLsJU07NgzJcbO1W8AeTqiGha41n6bnsd7iSIKZxhCd1OgYmlngpY3EC119v3mzVpMq1pgwf73PHrhJWRF+bYjO+VYoQbNufb6Q/Q+5c5TXq6yfaDqXn67EegBQmcVzb5qD9SWySR1QpvNUsB7Ywvpl3Sj5y5MTox6wCKsiAPCOv8XdAZKmxD8KZZq1esOw7fOVS1YbSH6+jS+8LnMvCPXqsDFXXZ5bhjdiUI+c+gIQQQgiZgsTx5BlB80WQLHJqA0MIIYQQQiYfPgEkhBBCSG7gk7mcwRvAMfDSofhnMpoQa1WB1iiokRLRET4TRTNZIrCPSVg9CERZxdN1hBzqm6IjjW4XjXYHI7DERNthjJuNqIvA+iUo0Pq29Fm3nG90ixg95w1q3U0EUW1WR+SDDi9qdVFQiaTRT5XB8ZwyD7FPja9pVLFx9qIDmh8buZd/zO1zaHWSqNFJ6bZoqMt9HoTxEGk9kJd067B9oiyDzHIBaquOa70jrjNoNTFkGP93TNt5oJWEpCewKsFlwNpDRCTuhO0ZPVjc73R/WevsgDGVr61L8JzYORn86rduX2bPUm3ps6BBRP3ZAX3cqEv2jc1IDHM+MjoyLF/Faa21iwbdctZmJAY7nNCOm9373XINWuelItfAAslGumGcnI1c9Gc4bVpkoiExKtKuE8+J3Wc8r16sYwMVMA+srVH8ltO32dGGtj2JFq3XS6Odi9GqquhL32ivQUuMUYAiouZaZKym1Nfgmm/HrNKBG5seHEdev14/XnetbhHnp7X7Qk16bDTo6vpj+qjkZEZT6ad5IzbV4A0gIYQQQnIDnwDmDGoACSGEEEKmGHwCOAZx7NzYo0Xa1d4/Bq7z1abUmufKtR3XzVRtNT905Y24XlsvtF3uSmi1P9A2I3KJc23vm68tLjCF4tRfrB79XNihS5hlx9zj/9g8/u+Zf9Ho58p/NTnLkASQNrYp/qWuNOUN64JNAmw64nxtjpBe5Wxh8g5rZ/4z6xaMfu6f7fZz/jMd6ntY3vJNGQlLHe13Xa7apm1ypbWEKcnEJa6Mna7VbcGAK+2NLNYJBWGhKysV729XbXEJJE8Mu3UMztUl0/yfuXPuL79ItbWscaX3uk2NettQJvdMmdxf4Cw2YpNkEJa4MlIiNlYcUCZX6Rh1K9TXBstdibloh7EBGtS2GWr1YF2TqNLzABMXPCMrSJ9wFisDN65Ubd317thnv6zPgd/t5ow3x81Jr1MnLEQgaWi/XvdJ1S5X0m67daFqqzzkSodD1XqsF51y5Tt/p7aPUaXxWM/XkRuuGP1ceELvJyaBoD2JN11bRMVnXD9039ig2kqPuZJpolnPrRiuYTZBJl7m5meiRS8XdUBZdsTYzkAZM32Rs/oJOnXpc/AmN18L2nVJdjgfUlW26dJnYo4r+8dFJi0Hrj/BUW3T03eZG38lO7RdkQ+2M76RxgwtcTY0yW4orab1eYzy3Lg8faW+phR0uL8rFf9mrrswHhLz56kmTPiwVi9Y9t3/vxaptou/6s5X/0X6GnbijzN9O3LYlM8nCz4BzBl8AkgIIYQQMsXgE0BCCCGE5AYaUOcMPgEkhBBCCJli8AngGHi+N6pBCs50qbawx72i72tJiYp3qnlWWxXgq/cYqSQiMu2k0ylFNhZqr4uCKz2hNYAh2AXM/inYkTRrvR5aDlirguq3QfNltw06pSyrlwPH3Ge7HFjN+MayI3HMRSdZS5Vpz8B3MYYuaXSEqEc0+pFgutO31P7LbtUWYuSaZ/QusM/JPm3FE9VWjH4u2Kn1bl4+9LvpB68ZrFmg3wtPasuWCDQ/nrFzqdvkdFhZmiywrvDydB8pW6B6rWP1truYtchETSkLnzKndwt2HVbfC9D+xIwpjNGy2kS0pwnPmLjEcqeTisAmxS5XvGWPaiteBHrHE1pXGvW5OeIdanTbMpFxaHlSucfoA3cdGP0884weGzFYeOSZ8YxRjVblhDF4nrHKKXrbRXVl2VDBfiqtZYuZ82BdUrG9VbV5cH7smML5ZO1PAtA/h3YswrEnhoyVEehyE40QzWbslkp3gOWJ0REmcb9MP4cQN2httNQYM1rLomPuPFurHLxmxkb/XJBw5yvuGt/yCC1vZvUZvStco0Nz3cBYSoyyExERvBYauy/UP6PmT0Qkhn0u/q2+hs0fyuxba28sMoGDD/nwwRtAQgghhEw6XiyTFgXn8V2TLFgCJoQQQgiZYvAJ4BhEeQkZXpQpJead6VNtXhtYVZjSJJZBUpdpu4jgVVe28uFVfhGRaI6zFfBMeTi+GNYzoF3hfSih9C9ydiHFprTig+WFZyxB0nNdydTfZZzl0UXf/CsN00XiOp1y4He7ckpUokvHHpQ7PFOqHlrtLG88cKVP9uqSUjBByoEH+zy8eolqy9/hytaSsKVJdy6HF5vjGXL7PHDdAtVWtteV+mNT2lXJJpWu5Jiq0n2St89pCaJZuoQVQF/6Nk0AkyCMHY4PqQDSr21ZgjqwzbAlQChjxdCXvi3lFroxHJskAyz72nSZGKxLrIWPB2MqKNRzJDzl+tabp8tp7SsrRj/XHGtWbZikE8xy5zUu0GVrnBfeoLExgXLx0DK97fwWN7dGZumSXAwqg/z/0GVrJQkZ0SXAkfluTuYd0+XbqMZZCHndbtuY7iGiLWLCcjMHR6CEikkzIuIVuHMSlOoyOY5N35RoUQYS1mibIw8sduJK6KNWXabsb3DXQTEKjaKjbj+9lN62D9eDcLa2wwlK3ZiKQH4ioq+L8TQ9Tn0or+O4FNFWM1Glm5Ppcj2mEt1wvTbXz9QqZ9OS93ajavOgXCtGooHXyMI9eqzjmLJWL1j27bmyXrX1zMtsL9xBG5ipBp8AEkIIIYRMMfgEkBBCCCE5IFaBAx/4toiCTwAJIYQQQqYYfAI4BukiT1quyeg8ik5rXcf0Xqd36vhYnWorP+j0WgPTtHajotpp9KRI65u6ljkdSWnJxaotDpwu48xara2pOOJ0K+2Xwmv+07X+o2ov2MzkaauCwVqnkSqRxaot6HW2AmGp0ZhB/NtwrdYY5YOmo3mt9hUob3RaoTJjF9G20vXZSKlbRzCiz0HZMafJqXrpoGrrvtpFJ/XO0cc6+6TbF29Yb3tkjuvLUx/Vx5qCfSlutvYxbp1FlbofRgrd9EpDZFyqRP+7K7/RbW+kWm97aKkbGxXbtGYyMdeNv8GlWrdY0OxsVFo+ZXRr3e54ql8z2j7Q/Z38hBtvc57X+tPei9xx4xgVESk56radKtbzAEke17FtPatcVFvxMa299cHCp+NyrddqX+00YEXtS1Vb0ZuNbj/Bsqd7uV5HwVk3nluu0eNt5ivuHPTV6eMJCypGPw+X6/E2Uur6paZHz61EuzuesFlb1/QscNeHoY/Uq7Zkrzt308+48xPZCDT4uW2V0Y4m3H4FV1Sqtuq3Xb+nKnQ/DFWCVU6bXqcUuH45dou+Ts170elmW1e75YrO6GtD1yLXfwVn9dOagRqn7QtS+txVveK0cCev0TrMml3uGPJb9Xg7eWf96Ofyo9paphhtdSr0Ojs+4rZf2OaWG6wxf07nuG23X6aPZ/qb7rMX6ri3YMitMzCRe71zXD8PVdertpKTTkN5Pt7tPOetXkSc5u88Ox/4BxERufTFThHRfTspUAOYM/gEkBBCCCFkiuHFMW+/kdLSUukfHJRkVeZfnL7+h6Ek+twbaOki/RZwMOKeRERJfW8dDMATJ8+8iVsEb02m9OnAn8JC/aTFh1WG8ADANy8xBsPw1qwxQI7haYCfMpk8qM3wzZMvNAI2bxZ7EIqeLtH/2sR9xn/pioikysDkFhczI1Sto08/yYvgiVOkT48keuENZDPs4yQ8oSsyfQSHZ8cD/mz7L8Y+g36PzT+7gkG3EvuEFp+u+aa/8BgweF7EnINivU4PXjrFbZ9rdcuVuuWw70REonxjeguofrDjBreUMoa+Be6EZY1FeOs0LNVPpkJ48IpPyETMG70wTqMC01+hWy5tzn9iAPo5aeZgGufB+OMmGNLHg+cntsbGZW4yx6abPVhNAsZ+bE2IcX1FZkzh+szcUtcKc+7w+IIB8xYwfDdVrvs22QdjsRDeLDdDL4SHq77xlsedztpn2JdUqbkmD7sve4P6WpGudOPI103i98MT74TuvxCe7PswbqJg/LEemge0CXgxH8eQiKhrq5fSHZEuccfnmSmC6xkp0/uC8yLM120LazJvaR9pTEkyr1h60TD/A6ShoUGOHz4j18y9e1K29x/Hn5C5i6bLnj173vnLUwSWgA3FxZnyYqqnS0REFixcOMG3pxZHjmQsahb+ofbJ9Hf+ynvlD75P3i3vse9G+6V+kvul8p2/kiumzFix1I7fNGX7ZAI+kD6ZMOEjU/ZN5h0f/ftHpga8ATScPp3xG2toaBAR4b8WAPZJNuyTsWG/ZMM+yYZ9kg37hEwWvAEkhBBCyOQTy+S9BEKxWxZ8CYQQQgghZIrBJ4CEEEIIyQ2TZgRNLHwCSAghhBAyxeATQEIIIYTkgFgkjt75axdqW0RBH0BCCCGETCoNDQ1y/NBpuWb2hknZ3n80/6PMXTyDb1cDfAJICCGEkNzAZ1A5gxpAQgghhJApBp8AEkIIIWTyiWXy3gLmg8Ys+ASQEEIIIWSKwSeAhBBCCMkN1ADmDD4BJIQQQgiZYvAGkBBCCCFkisESMCGEEEJyA0vAOYNPAAkhhBBCphh8AkgIIYSQ3MAngDmDTwAJIYQQQqYYfAJICCGEkBwQi0TR5G2LKPgEkBBCCCFkisEngIQQQgiZfGKZPA0gHwBmwSeAhBBCCCFTDD4BJIQQQkhu4FvAOYNPAAkhhBBCphh8AkgIIYSQ3BDxCWCu4BNAQgghhJApBm8ACSGEEEKmGCwBE0IIIWTSiSWWOJ4cI+iYPjBZ8AkgIYQQQsgUg08ACSGEEDL5xDJ5L4HwAWAWfAJICCGEEDLF4BNAQgghhOQGGkHnDD4BJIQQQgiZYvAJICGEEEJyQzQ5bwGTbPgEkBBCCCFkisEngIQQQgjJDdQA5gw+ASSEEEIImWLwBpAQQgghZIrBEjAhhBBCJp84lniyXgJhqTkLPgEkhBBCCJli8AkgIYQQQnIDn8zlDD4BJIQQQgiZYvAJICGEEEJyQ8QngLmCTwAJIYQQQqYYfAJICCGEkNwQMwouV/AJICGEEELIFINPAAkhhBAy+cSxxJOlAeTbxlnwCSAhhBBCyLtgcHBQ/tt/+2+yZMkSKSgokFmzZsndd98tzc3Nud619wxvAAkhhBCSG+Jocv67AAwNDcn1118vDz74oPT19cnNN98sc+bMkSeffFIuu+wyOXr06AXZzmTBG0BCCCGEkHfgoYcektdff12uvPJKOXjwoPzrv/6rvPHGG/LII49IW1ub3H333bnexfcEbwAJIYQQQiZgZGREHnvsMRER+fu//3spKSkZbdu4caMsX75cfvWrX8lbb72Vq118z/AGkBBCCCE5IY7iSfnv/fLKK69Id3e3LFy4UC677LKs9vXr14uIyAsvvPC+tzVZ8AaQEEIIIWQCdu7cKSIiq1atGrP9/O937do1afv0fqENDCGEEEJywx+IEfTx48dFRKSurm7M9vO/b2pqmrR9er/wBpAQQgghk86A9Mlr8U8nbVtHjhyRhoaGMdv37Nkz4fJ9fX0iIlJUVDRme3FxsYiI9Pb2vo+9nFx4A0gIIYSQSWXhwoWTvs3zT/FIBt4AEkIIIWRS+dGPfpTrXXhPnH/rd2BgYMz2/v5+EREpLS2dtH16v/AlEEIIIYSQCZg7d66IiJw8eXLM9vO/nzdv3qTt0/uFN4CEEEIIIROwYsUKERHZvn37mO3nf798+fJJ26f3C28ACSGEEEIm4Oqrr5by8nI5cuSI7NixI6t906ZNIiJy0003TfKe/e7wBpAQQgghZALy8vLkvvvuExGRv/iLvxjV/ImIfPOb35Rdu3bJxz72Mbn88stztYvvGS+O4/dvkU0IIYQQ8iFmaGhI1qxZI2+88YbMnDlTrr32WmlqapI33nhDamtr5fXXX5cFCxbkejffNbwBJIQQQgh5FwwODsr/+B//Q5566ik5ceKEVFVVyQ033CAPPvjguCbRv6/wBpAQQgghZIpBDSAhhBBCyBSDN4CEEEIIIVMM3gASQgghhEwxeANICCGEEDLF4A0gIYQQQsgUgzeAhBBCCCFTDN4AEkIIIYRMMXgDSAghhBAyxeANICGEEELIFIM3gIQQQgghUwzeABJCCCGETDF4A0gIIYQQMsXgDSAhhBBCyBSDN4CEEEIIIVMM3gASQgghhEwxeANICCGEEDLF4A0gIYQQQsgUgzeAhBBCCCFTjP8flTHdkK/PU8cAAAAASUVORK5CYII=",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
}
],
"source": [
- "eigen_factors[\"activation_eigenvectors\"][\"2\"].shape"
- ],
+ "covariance_factors = analyzer.load_covariance_matrices(factors_name=\"tutorial_factor\")\n",
+ "plt.matshow(covariance_factors[\"activation_covariance\"][\"2\"])\n",
+ "plt.colorbar()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "a8eb2c2d-32e4-46c8-b53e-c608f2f32091",
+ "metadata": {},
+ "source": [
+ "We can perform Eigendecomposition after fitting covariance matrices."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "id": "31da64ca7f237819",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T21:00:45.840072Z",
- "start_time": "2024-03-12T21:00:45.835288Z"
+ "end_time": "2024-03-12T21:00:31.570596Z",
+ "start_time": "2024-03-12T21:00:31.553063Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "f0178839635f6aff"
- },
- {
- "cell_type": "code",
- "execution_count": 37,
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "Fitting Lambda matrices [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n",
- "Fitting Lambda matrices [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n"
+ "Performing Eigendecomposition [4/4] 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n"
]
}
],
"source": [
- "lambda_matrices = analyzer.fit_lambda_matrices(\n",
- " factors_name=\"ekfac\",\n",
- " dataset=train_dataset,\n",
- " per_device_batch_size=None,\n",
+ "analyzer.perform_eigendecomposition(\n",
+ " factors_name=\"tutorial_factor\",\n",
" overwrite_output_dir=True,\n",
")"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:01:00.458530Z",
- "start_time": "2024-03-12T21:01:00.113019Z"
- }
- },
- "id": "582867d0b427db3"
+ ]
},
{
- "cell_type": "code",
- "execution_count": 38,
- "outputs": [
- {
- "data": {
- "text/plain": "torch.Size([128, 129])"
- },
- "execution_count": 38,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
+ "cell_type": "markdown",
+ "id": "39c411f2-c1c0-4483-9ec8-2d7ca966237a",
+ "metadata": {},
"source": [
- "lambda_matrices[\"lambda_matrix\"][\"2\"].shape"
- ],
+ "Next, we can fit the Lambda (corrected-eigenvalues for EKFAC) matrices with:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "id": "582867d0b427db3",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T21:01:07.857067Z",
- "start_time": "2024-03-12T21:01:07.852205Z"
+ "end_time": "2024-03-12T21:01:00.458530Z",
+ "start_time": "2024-03-12T21:01:00.113019Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "526f2e3cb469d6a0"
- },
- {
- "cell_type": "code",
- "execution_count": 39,
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "Computing pairwise influence scores [0/1] 0%| [time left: ?, time spent: 00:00]\n",
- "Computing dot products on training dataset [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n",
- "Computing pairwise influence scores [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n",
- "Computing pairwise influence scores [0/1] 0%| [time left: ?, time spent: 00:00]\n",
- "Computing dot products on training dataset [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n",
- "Computing pairwise influence scores [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n"
+ "Fitting Lambda matrices [1/1] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n",
+ "Fitting Lambda matrices [1/1] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n"
]
}
],
"source": [
- "scores = analyzer.compute_pairwise_scores(\n",
- " scores_name=\"pairwise\",\n",
- " factors_name=\"ekfac\",\n",
- " query_dataset=query_dataset,\n",
- " train_dataset=train_dataset,\n",
- " per_device_query_batch_size=len(query_dataset),\n",
+ "analyzer.fit_lambda_matrices(\n",
+ " factors_name=\"tutorial_factor\",\n",
+ " dataset=train_dataset,\n",
+ " per_device_batch_size=None,\n",
" overwrite_output_dir=True,\n",
")"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:01:40.623332Z",
- "start_time": "2024-03-12T21:01:40.490302Z"
- }
- },
- "id": "8bdb3aa3d8aade4a"
+ ]
},
{
- "cell_type": "code",
- "execution_count": 41,
- "outputs": [
- {
- "data": {
- "text/plain": "tensor([[ 5.0061e-02, -3.6199e-02, -1.2702e+00, ..., 2.3542e-03,\n 5.4149e-01, 9.2300e-01],\n [-4.9688e-01, -2.7395e-02, -3.8494e+00, ..., 5.6789e-01,\n -2.4231e+01, -2.9652e+00],\n [-7.3917e-01, -2.1874e-01, -4.9690e+01, ..., -2.0099e-01,\n -7.6540e-01, -1.0384e+00],\n ...,\n [ 6.8861e-01, 6.6297e-01, 9.4206e-01, ..., -8.3435e-01,\n 6.2277e+00, 7.0717e-02],\n [-1.9911e+00, 6.5527e-01, 6.2227e+00, ..., 9.6123e-01,\n -2.0532e+01, 7.6572e+00],\n [ 1.5878e+00, -2.6181e+00, -5.0869e+00, ..., -1.7254e-01,\n -3.7389e+00, 1.4460e+01]])"
- },
- "execution_count": 41,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
+ "cell_type": "markdown",
+ "id": "dfad35d4-e0f7-4483-a915-fb28a717c2ed",
+ "metadata": {},
"source": [
- "scores[\"all_modules\"]"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:01:47.451459Z",
- "start_time": "2024-03-12T21:01:47.448032Z"
- }
- },
- "id": "51d4f1a9ad39cfab"
+ "While we sequentially called `fit_covariance_matrices`, `perform_eigendecomposition`, and `fit_lambda_matrices` in the above example, we can instead fit all required factors with:"
+ ]
},
{
"cell_type": "code",
- "execution_count": 44,
+ "execution_count": 18,
+ "id": "0f00c60c-c932-42f4-b72f-2a59025c9417",
+ "metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "Computing pairwise influence scores [0/1] 0%| [time left: ?, time spent: 00:00]\n",
- "Computing dot products on training dataset [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n",
- "Computing pairwise influence scores [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n",
- "Computing pairwise influence scores [0/1] 0%| [time left: ?, time spent: 00:00]\n",
- "Computing dot products on training dataset [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n",
- "Computing pairwise influence scores [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n"
+ "Fitting covariance matrices [1/1] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n",
+ "Fitting covariance matrices [1/1] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n",
+ "Performing Eigendecomposition [4/4] 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n",
+ "Fitting Lambda matrices [1/1] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n",
+ "Fitting Lambda matrices [1/1] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n"
]
}
],
"source": [
- "score_args = ScoreArguments(per_module_score=True)\n",
- "\n",
- "per_module_scores = analyzer.compute_pairwise_scores(\n",
- " scores_name=\"per_module\",\n",
- " factors_name=\"ekfac\",\n",
- " query_dataset=query_dataset,\n",
- " train_dataset=train_dataset,\n",
- " score_args=score_args,\n",
- " per_device_query_batch_size=len(query_dataset),\n",
+ "analyzer.fit_all_factors(\n",
+ " factors_name=\"tutorial_factor\",\n",
+ " dataset=train_dataset,\n",
+ " per_device_batch_size=None,\n",
" overwrite_output_dir=True,\n",
")"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:02:49.326940Z",
- "start_time": "2024-03-12T21:02:49.187921Z"
- }
- },
- "id": "a1edac308383f728"
+ ]
},
{
- "cell_type": "code",
- "execution_count": 45,
- "outputs": [
- {
- "data": {
- "text/plain": "{'0': tensor([[-2.6266e-02, -1.8842e-02, 1.1664e-01, ..., 2.4500e-03,\n 1.3508e-01, -4.2287e-02],\n [ 3.0197e-01, -1.4576e-01, 3.2981e-01, ..., -1.7151e-01,\n 2.1003e+00, -3.2144e+00],\n [ 1.1908e-02, -1.3091e-02, -3.8957e+00, ..., 4.9687e-03,\n -1.2466e+00, -7.2670e-03],\n ...,\n [ 8.1234e-01, -6.6214e-01, -7.0342e-02, ..., -2.6749e-01,\n 6.9416e-02, 2.1124e-02],\n [ 1.4178e+00, 3.2288e+00, -9.8175e-01, ..., 1.6657e-01,\n -4.7388e+00, -3.4997e+00],\n [-2.0218e-01, -1.7329e+00, -3.7108e+00, ..., -3.1222e-01,\n 1.6311e+00, 7.7284e+00]]),\n '2': tensor([[ 0.1232, -0.0557, -0.7605, ..., 0.1088, 0.3196, 0.7115],\n [ -1.1788, 0.1449, -2.5917, ..., 0.3668, -2.0784, 1.3429],\n [ -0.1844, -0.1768, -23.9932, ..., 0.3725, -0.4919, -1.0844],\n ...,\n [ -0.3336, 0.6093, 0.3979, ..., -0.9447, 3.0782, -0.2292],\n [ -2.0491, -3.4502, 1.9648, ..., -0.1059, -13.8598, 4.9501],\n [ 2.2986, -0.7076, -3.8665, ..., -0.3188, -0.5854, -6.6292]]),\n '4': tensor([[ -0.0514, 0.0343, -0.6465, ..., -0.0979, 0.2112, 0.2504],\n [ 0.5855, 0.0650, -2.1663, ..., 0.3206, -25.5599, -0.6195],\n [ -0.5678, -0.0391, -20.6871, ..., -0.6139, 1.4205, 0.0583],\n ...,\n [ 0.1469, 0.7100, 0.4384, ..., 0.2733, 2.7951, 0.3150],\n [ -1.4954, 0.8342, 5.7622, ..., 1.0790, -2.8883, 5.4326],\n [ -0.3669, -0.1428, 3.1085, ..., 0.4055, -4.4859, 12.0807]]),\n '6': tensor([[ 4.5021e-03, 4.0613e-03, 2.0121e-02, ..., -1.1026e-02,\n -1.2443e-01, 3.3208e-03],\n [-2.0554e-01, -9.1504e-02, 5.7879e-01, ..., 5.1960e-02,\n 1.3072e+00, -4.7417e-01],\n [ 1.1173e-03, 1.0219e-02, -1.1136e+00, ..., 3.5497e-02,\n -4.4738e-01, -4.9912e-03],\n ...,\n [ 6.2971e-02, 5.7528e-03, 1.7607e-01, ..., 1.0451e-01,\n 2.8499e-01, -3.6171e-02],\n [ 1.3561e-01, 4.2407e-02, -5.2260e-01, ..., -1.7835e-01,\n 9.5490e-01, 7.7421e-01],\n [-1.4177e-01, -3.4834e-02, -6.1821e-01, ..., 5.3028e-02,\n -2.9870e-01, 1.2801e+00]])}"
- },
- "execution_count": 45,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
+ "cell_type": "markdown",
+ "id": "8a7387c5-a4ba-41f1-8365-95b5785b742b",
+ "metadata": {},
"source": [
- "per_module_scores"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:02:52.412327Z",
- "start_time": "2024-03-12T21:02:52.406994Z"
- }
- },
- "id": "b439b36290e6a12c"
+ "## Computing Influence Scores"
+ ]
},
{
- "cell_type": "code",
- "execution_count": 47,
- "outputs": [
- {
- "data": {
- "text/plain": "torch.Size([103, 927])"
- },
- "execution_count": 47,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
+ "cell_type": "markdown",
+ "id": "b57f42e9-ff52-4d13-8493-e00f7fcf3425",
+ "metadata": {},
"source": [
- "per_module_scores[\"2\"].shape"
- ],
+ "After computing all neccessary factors, we now compute the pairwise influence scores."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "id": "8bdb3aa3d8aade4a",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T21:03:05.160179Z",
- "start_time": "2024-03-12T21:03:05.155732Z"
+ "end_time": "2024-03-12T21:01:40.623332Z",
+ "start_time": "2024-03-12T21:01:40.490302Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "d4d5dc4917ec0ca5"
- },
- {
- "cell_type": "code",
- "execution_count": 50,
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "Computing self-influence scores [9/9] 100%|██████████ [time left: 00:00, time spent: 00:00]\n"
+ "Computing pairwise scores (query gradient) [0/1] 0%| [time left: ?, time spent: 00:00]\n",
+ "Computing pairwise scores (training gradient) [1/1] 100%|███████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\u001b[A\n",
+ "Computing pairwise scores (query gradient) [1/1] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n",
+ "Computing pairwise scores (query gradient) [0/1] 0%| [time left: ?, time spent: 00:00]\n",
+ "Computing pairwise scores (training gradient) [1/1] 100%|███████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\u001b[A\n",
+ "Computing pairwise scores (query gradient) [1/1] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n"
]
}
],
"source": [
- "self_scores = analyzer.compute_self_scores(\n",
- " scores_name=\"self\",\n",
- " factors_name=\"ekfac\",\n",
+ "analyzer.compute_pairwise_scores(\n",
+ " scores_name=\"tutorial_score\",\n",
+ " factors_name=\"tutorial_factor\",\n",
+ " query_dataset=query_dataset,\n",
" train_dataset=train_dataset,\n",
- " per_device_train_batch_size=len(query_dataset),\n",
+ " per_device_query_batch_size=len(query_dataset),\n",
" overwrite_output_dir=True,\n",
")"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:03:40.288919Z",
- "start_time": "2024-03-12T21:03:40.046890Z"
- }
- },
- "id": "c19a85a06f41213e"
- },
- {
- "cell_type": "code",
- "execution_count": 53,
- "outputs": [
- {
- "data": {
- "text/plain": "torch.Size([927])"
- },
- "execution_count": 53,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "self_scores[\"all_modules\"].shape"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:03:50.609012Z",
- "start_time": "2024-03-12T21:03:50.604147Z"
- }
- },
- "id": "7707710849ac2d0f"
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "outputs": [],
- "source": [],
- "metadata": {
- "collapsed": false
- },
- "id": "7d861065aa7bea66"
+ ]
},
{
"cell_type": "markdown",
+ "id": "b9194624-2311-44c6-adbf-06a24ad41ebd",
+ "metadata": {},
"source": [
- "## Counterfactual"
- ],
- "metadata": {
- "collapsed": false
- },
- "id": "261a508a6c02267d"
+ "You can load the pairwise scores with `load_pairwise_scores`. The pairwise score will have the dimension `query_dataset_size x train_dataset_size`."
+ ]
},
{
"cell_type": "code",
- "execution_count": 56,
- "outputs": [],
- "source": [
- "small_query_dataset = torch.utils.data.Subset(query_dataset, list(range(10)))"
- ],
+ "execution_count": 20,
+ "id": "51d4f1a9ad39cfab",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T21:05:54.279349Z",
- "start_time": "2024-03-12T21:05:54.269910Z"
+ "end_time": "2024-03-12T21:01:47.451459Z",
+ "start_time": "2024-03-12T21:01:47.448032Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "4522bd15c0f00882"
- },
- {
- "cell_type": "code",
- "execution_count": 57,
"outputs": [
{
"data": {
- "text/plain": "10"
+ "text/plain": [
+ "torch.Size([103, 927])"
+ ]
},
- "execution_count": 57,
+ "execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
- "len(small_query_dataset)"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:06:00.421397Z",
- "start_time": "2024-03-12T21:06:00.416579Z"
- }
- },
- "id": "407672278a2eb7d7"
+ "scores = analyzer.load_pairwise_scores(scores_name=\"tutorial_score\")\n",
+ "scores[\"all_modules\"].shape"
+ ]
},
{
- "cell_type": "code",
- "execution_count": 58,
- "outputs": [
- {
- "data": {
- "text/plain": "0.10985381603240967"
- },
- "execution_count": 58,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
+ "cell_type": "markdown",
+ "id": "5c914f47-4cf8-47d6-a638-1d0cf10357ed",
+ "metadata": {},
"source": [
- "evaluate(model=model, dataset=small_query_dataset, batch_size=eval_batch_size)"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:06:27.168938Z",
- "start_time": "2024-03-12T21:06:27.161268Z"
- }
- },
- "id": "19a50d9bd3a7573"
+ "If you would like to obtain the influence score for each module, you can pass in `ScoreArguments` to `compute_pairwise_scores`."
+ ]
},
{
"cell_type": "code",
- "execution_count": 59,
+ "execution_count": 21,
+ "id": "a3a7267f-664f-42c5-9a0f-4f462c5a0bb9",
+ "metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "Computing pairwise influence scores [0/1] 0%| [time left: ?, time spent: 00:00]\n",
- "Computing dot products on training dataset [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n",
- "Computing pairwise influence scores [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n",
- "Computing pairwise influence scores [0/1] 0%| [time left: ?, time spent: 00:00]\n",
- "Computing dot products on training dataset [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n",
- "Computing pairwise influence scores [1/1] 100%|██████████ [time left: 00:00, time spent: 00:00]\n"
- ]
- }
- ],
- "source": [
- "scores = analyzer.compute_pairwise_scores(\n",
- " scores_name=\"counterfactual\",\n",
- " factors_name=\"ekfac\",\n",
- " query_dataset=small_query_dataset,\n",
- " train_dataset=train_dataset,\n",
- " per_device_query_batch_size=len(small_query_dataset),\n",
- " overwrite_output_dir=True,\n",
- ")"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:06:42.141007Z",
- "start_time": "2024-03-12T21:06:42.067707Z"
- }
- },
- "id": "8b3eb92fd287b0cc"
- },
- {
- "cell_type": "code",
- "execution_count": 62,
- "outputs": [],
- "source": [
- "summed_scores = scores[\"all_modules\"].sum(dim=0)"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:08:01.250153Z",
- "start_time": "2024-03-12T21:08:01.244362Z"
+ "Computing pairwise scores (query gradient) [0/1] 0%| [time left: ?, time spent: 00:00]\n",
+ "Computing pairwise scores (training gradient) [1/1] 100%|███████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\u001b[A\n",
+ "Computing pairwise scores (query gradient) [1/1] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n",
+ "Computing pairwise scores (query gradient) [0/1] 0%| [time left: ?, time spent: 00:00]\n",
+ "Computing pairwise scores (training gradient) [1/1] 100%|███████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\u001b[A\n",
+ "Computing pairwise scores (query gradient) [1/1] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████ [time left: 00:00, time spent: 00:00]\n"
+ ]
}
- },
- "id": "86fd82e97c1c0af9"
+ ],
+ "source": [
+ "from kronfluence import ScoreArguments\n",
+ "\n",
+ "score_args = ScoreArguments(per_module_score=True)\n",
+ "analyzer.compute_pairwise_scores(\n",
+ " score_args=score_args,\n",
+ " scores_name=\"tutorial_per_module_score\",\n",
+ " factors_name=\"tutorial_factor\",\n",
+ " query_dataset=query_dataset,\n",
+ " train_dataset=train_dataset,\n",
+ " per_device_query_batch_size=len(query_dataset),\n",
+ " overwrite_output_dir=True,\n",
+ ")"
+ ]
},
{
"cell_type": "code",
- "execution_count": 64,
+ "execution_count": 22,
+ "id": "0156e383-a2da-4ea4-bccf-a0183c78e712",
+ "metadata": {},
"outputs": [
{
"data": {
- "text/plain": "tensor([647, 503, 326])"
+ "text/plain": [
+ "dict_keys(['0', '2', '4', '6'])"
+ ]
},
- "execution_count": 64,
+ "execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
- "torch.topk(summed_scores, 3).indices"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:08:07.688741Z",
- "start_time": "2024-03-12T21:08:07.682234Z"
- }
- },
- "id": "67c3033c9935f72"
+ "per_module_scores = analyzer.load_pairwise_scores(scores_name=\"tutorial_per_module_score\")\n",
+ "per_module_scores.keys()"
+ ]
},
{
- "cell_type": "code",
- "execution_count": 65,
- "outputs": [],
+ "cell_type": "markdown",
+ "id": "95a869ff-70f0-49a4-a8f3-92f5c81a8ecf",
+ "metadata": {},
"source": [
- "def get_top_k_indices(current_score, top_k=1):\n",
- " return torch.topk(current_score, top_k).indices"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:08:23.455884Z",
- "start_time": "2024-03-12T21:08:23.450194Z"
- }
- },
- "id": "fc57a82158420f4d"
+ "We can also visualize the score matrix for the last module."
+ ]
},
{
"cell_type": "code",
- "execution_count": 66,
+ "execution_count": 23,
+ "id": "065666a3-6dc5-49dc-9dff-d0ca3886a619",
+ "metadata": {},
"outputs": [
{
"data": {
- "text/plain": "tensor([647, 503, 326, 256, 217, 36, 221, 550, 288, 240])"
+ "text/plain": [
+ ""
+ ]
},
- "execution_count": 66,
+ "execution_count": 23,
"metadata": {},
"output_type": "execute_result"
+ },
+ {
+ "data": {
+ "image/png": "iVBORw0KGgoAAAANSUhEUgAABugAAAEZCAYAAAB1paMzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAABcSAAAXEgFnn9JSAAEAAElEQVR4nOz9ebgmRZUnjn8i3+VuRdWlimJfhGKTUhC0sQt+1SCNiiigDm7TPgOl7TJqK+IyPSMqov040wiKgo1jCziDOygttG2LirRf2RRQhhLZKWSTqqK2W3d575sZvz8yI/JE5InMfN97b92q8nx43odbmRkRJyJOnHMi4sQJpbXWEAgEAoFAIBAIBAKBQCAQCAQCgUAgEAgE2wTRfBMgEAgEAoFAIBAIBAKBQCAQCAQCgUAgEPw5QTboBAKBQCAQCAQCgUAgEAgEAoFAIBAIBIJtCNmgEwgEAoFAIBAIBAKBQCAQCAQCgUAgEAi2IWSDTiAQCAQCgUAgEAgEAoFAIBAIBAKBQCDYhpANOoFAIBAIBAKBQCAQCAQCgUAgEAgEAoFgG0I26AQCgUAgEAgEAoFAIBAIBAKBQCAQCASCbQjZoBMIBAKBQCAQCAQCgUAgEAgEAoFAIBAItiFkg04gEAgEAoFAIBAIBAKBQCAQCAQCgUAg2IaQDTqBQCAQCAQCgUAgEAgEAoFAIBAIBAKBYBtCNugEAoFAIBAIBAKBQCAQCAQCgUAgEAgEgm0I2aATCAQCgUAgEAgEAoFAIBAIBAKBQCAQCLYhZINOIBAIBAKBQCAQCAQCgUAgEAgEAoFAINiGkA06gUAgEAgEAoFAIBAIBAKBQCAQCAQCgWAbQjboBAKBQCAQCAQCgUAgEAgEAoFAIBAIBIJtCNmgEwgEAoFAIBAIBAKBQCAQCAQCgUAgEDiYmJjAJz7xCRx66KEYHBzE3nvvjbe97W148skne85rw4YN+MAHPoADDjgAAwMDOOCAA3DOOedg48aN7Pdnn302lFLB3+WXXx4s6/rrr8cJJ5yAhQsXYuHChTjxxBPxr//6rz3TPNdQWms930QIBAKBQCAQCAQCgUAgEAgEAoFAIBAItg9MTk7iZS97GW677TbstddeWLlyJR577DHccccdWLp0KW677TYcdNBBtfJat24dVqxYgYceeggHHXQQXvKSl2D16tVYvXo1Dj30UNx6661YvHixk+bss8/G17/+dbzyla/EnnvuWcjzrLPOwste9rLC8y984Qv44Ac/iGaziZNPPhkDAwP4yU9+gomJCXzpS1/C+973vv4aZA4gG3QCgUAgEAgEAoFAIBAIBAKBQCAQCAQCi/POOw//8A//gBUrVuAnP/kJFixYAAC4+OKL8aEPfQgnnHACfvGLX9TK661vfSu+8Y1v4PWvfz2+853voNlsAgDe//7340tf+hLOOussXHXVVU4as0F300034cQTT6xVzv3334/ly5ej2WzipptuwooVKwAADzzwAI477jhs2rQJ9913Hw4++OBa+c01JMSlQCAQCAQCgUAgEAgEAoFAIBAIBAKBAADQ6XRw6aWXAgAuu+wyuzkHAOeeey6OPPJI3Hzzzbjzzjsr83r66afxrW99C+12G1/+8pft5hwAXHjhhVi6dCmuvvpqPPvsszOm+5JLLkEcx3j3u99tN+cA4NBDD8XHPvYxdLtdXHLJJTMuZ7YgG3QCgUAgEAgEAoFAIBAIBAKBQCAQCAQCAMCvfvUrbNq0CcuWLcPRRx9deH/mmWcCSO96q8KPf/xjJEmClStXYo899nDeDQwM4LTTTkMcx/jRj340Y7rNPXOGvn5p3lZoVn8iEAgEAoFAIBAIBAKBQCAQCAQCgUAg+HPA7373OwDAMcccw743z++5555ZyeuKK64I5vX9738f1157LeI4xoEHHojTTjsNhx9+eOG7jRs34vHHHwcAdlNxv/32w2677YY1a9Zg8+bNWLhwYSXtcw05QUcwMTGBT3ziEzj00EMxODiIvffeG29729vw5JNPzjdpAoFAUAvj4+O47rrr8Pa3vx2HHXYYBgcHMTIygqOOOgoXXHABxsbGgmmvuuoqHHvssViwYAEWL16MU089Fbfccktpeb/61a9w6qmnYvHixViwYAGOPfZY/J//839mu1oCgUDQF9avX4/dd98dSqnK+PIiAwUCwc6EtWvX4sMf/jAOO+wwDA0NYfHixTjmmGPwkY98hP3++uuvxwknnICFCxdi4cKFOPHEE633cQirV6/GG97wBixduhRDQ0N44QtfiC984QtIkmQuqiQQCAS18Otf/xpvfOMbsffee6PVamF0dBQrV67ElVdeCa114fs4jvH5z38eL3zhCzE0NISlS5fijW98I+67777ScvqRmwKBQNAvTj/9dCxfvryn3+mnnz6jMs1G17777su+N8/XrFkz53l96UtfwqWXXop/+qd/wkc/+lEcccQReO9734tut8uWs+uuu2JkZGTGdG8LyAZdhsnJSZx00kn49Kc/jbGxMZxxxhnYb7/9cOWVV+Loo4/GI488Mt8kCgQCQSW++c1v4nWvex2uuOIKNBoNnH766Vi5ciUeffRRfPKTn8Rf/MVfsPGczznnHKxatQr33nsvTj75ZBx77LG48cYb8Vd/9Ve47rrr2LKuvfZanHDCCfjxj3+MI488EqeccgoefPBBnHXWWfjwhz88xzUVCASCanzoQx/CunXrKr8TGSgQCHYm3HnnnXj+85+Piy66CK1WC2eccQb+8i//Es899xw+//nPF77/whe+gNNPPx233HILjj/+eJx00km444478JrXvMbeO+Lj1ltvxV/8xV/gmmuuwUEHHYTTTz8d69atwwc/+EG8+c1vZhfBBQKBYK5x7bXXYsWKFfje976HvfbaC69//etxzDHH4LbbbsPb3vY2vPWtb3W+T5IEb3jDG3DuuefiiSeewKtf/WosX74c11xzDV7ykpfgjjvuYMvpR24KBALBTPDwww/joYfuA7oP1fo99NB9+MlPfhLcvKsD4+Q/PDzMvjcbYFu2bJmzvI4++mhcfvnleOCBBzA+Po5HHnkEl112GUZHR/HlL3+54HxWVU6vdG8TaIHWWuuPfexjGoBesWKF3rJli31+0UUXaQD6hBNOmD/iBAKBoCauuuoq/c53vlP//ve/d54/9dRT+uijj9YA9Fve8hbn3Y033qgB6CVLlugHHnjAPr/lllt0u93Wo6OjesOGDU6a9evX64ULF2oA+tprr7XPn3nmGX3wwQdrAPqmm26a9foJBAJBXfz0pz/VAPQ73/lODUAvW7aM/U5koEAg2Jnw7LPP6t12200PDw/rf/mXfym8v/32251//+EPf9CNRkMPDAzoW265xT6///779ZIlS3Sz2dQPPvigk6bT6egDDzxQA9AXX3yxfb5lyxa9YsUKDUBfeeWVs1sxgUAgqMD09LTefffdNQD9jW98w3n3+9//Xi9evFgD0D//+c/t869+9asagD7kkEP0M888Y59fc801GoA++OCD9fT0tJNXP3JTIBAIZoojjjhCP//Qlp586sBav+cf2tIDAwP6iCOOYH918I53vEMD0B/72MfY9w8++KCVoVV4+ctfrgHor371q+x7My9/+ctfXou2e++9V7fbbd1sNvXjjz9un//qV7/SAPQ+++wTTHv88cdrAPpXv/pVrbLmGnKCDkCn07EeLpdddhkWLFhg35177rk48sgjcfPNN+POO++cLxIFAoGgFs466yx85StfwfOf/3zn+V577YXLLrsMQBq3udPp2HcXX3wxAOC8887DIYccYp+vWLEC7373u7Fx40Z87Wtfc/L753/+Z2zevBlnnHEGXv/619vne+yxB/7xH/8RAHDRRRfNbuUEAoGgJiYmJvCud70LRxxxROVpNpGBAoFgZ8InP/lJrFu3DhdeeCEb1ujYY491/n3JJZcgjmO8+93vxooVK+zzQw89FB/72MfQ7XZxySWXOGl+8IMf4NFHH8VRRx2FD37wg/b5ggUL7LxaZKBAINjW+MMf/oBnn30Whx12GP7zf/7PzrvnP//59vTcr3/9a/vc2IH/+I//iD322MM+/0//6T/h9NNPx0MPPYR/+Zd/cfLqR24KBALBbCGBrvUDgGXLlmH16tXsrw7MHsn4+Dj7fuvWrQCAXXbZZZvmBcCG8Ox2u/jZz35Wu5x+yppryAYd0vtDNm3ahGXLlrGXB5555pkA0vjSAoFAsKPiqKOOAgBMTU1h/fr1ANJF7J///OcAcllHEZJ/JrY+l+bVr341BgcH8dOf/hSTk5OzVwGBQCCoiU996lN45JFHcPnll6PVagW/ExkoEAh2JkxMTODqq6/GyMgIVq1aVStNmTzrRwYec8wxOOigg3Dvvffiscce64V8gUAgmBEGBgZqfbdkyRIAwKOPPor77rsPQ0NDePWrX134rh8ZKOuHAoFgLqEBJDX/m41g4/vvvz8A4IknnmDfm+cHHHDANs3LwDjYPv3004VyNmzYYDfiZqOsuYRs0AH43e9+ByCdTHAwz++5555tRpNAIBDMNsxdmq1WC4sXLwYA3H///ZiamsLSpUvZi1pD8q9MbrbbbbzgBS/A5OQkHnjggVmtg0AgEFThnnvuwUUXXYRVq1Zh5cqVpd+KDBQIBDsTfvOb32DLli04+uijMTQ0hH/7t3/Dueeei/e85z34whe+gKeeesr5fuPGjXj88ccBgHVU3W+//bDbbrthzZo12Lx5s30u82eBQLA94qCDDsKyZctw//3345vf/Kbz7r777sPVV1+NXXfdFa973esA5LLsBS94AevQxcmyfuWmQCAQzA40Yl3vh1nYojOO/nfddRf73jw/8sgjt2leBhs2bACQ3ykHAKOjo3aT7u677y6k+eMf/4h169bhgAMOwMKFC2uXNZeQDTrAKlduYYY+X7NmzTajSSAQCGYbJszGKaecYr0Lq+TfyMgIRkdHsWHDBnt56ubNm7Fp06bSdCI3BQLBfCBJEvzt3/4tRkdHbajJMogMFAgEOxN+//vfAwB23313vPa1r8Wpp56Kz3/+8/inf/onfPCDH8TBBx+Mb33rW/Z7IwN33XVXZ2GDgpNnMn8WCATbIxqNBr7+9a9jdHQUf/M3f4MXv/jFePOb34yTTjoJRx55JPbdd1/87Gc/s86q/ciyfuWmQCAQzBZ6CXE5Uxx//PFYtGgRHn74Yfz2t78tvL/mmmsAAKeddlplXqeccgqiKMIvf/lLPPvss867qakpXH/99Wg0Gjj11FNr0TY1NWVPNPtOY+ZUtKGvX5q3FWSDDsDY2BgAYHh4mH1vlK5ZmBEIBIIdDT/60Y/wta99Da1WC5/+9Kft8yr5BxRloElTlk7kpkAgmA986Utfwq9//WtceOGFNnxRGUQGCgSCnQnGi/iHP/whfvzjH+Oyyy7Ds88+i8ceewwf/vCHMTExgbPOOssusPQjA+ukExkoEAjmC8cffzxuvvlmHHTQQbjrrrvwne98BzfddBOiKMLLX/5yHHTQQfbbfmRZv3JTIBAIZgMaQAxd6zcbW3Ttdhvve9/7AADvfe97nZCRF198Me655x6ccMIJePGLX2yfX3rppTj88MPx3//7f3fy2muvvfCWt7wFnU4H73nPe9Dtdu27j370o1i7di3e+ta3Yvfdd7fP//CHP+D//t//i6mpKSevtWvX4s1vfjP++Mc/4qijjsLxxx/vvP/ABz6ARqOByy+/HLfddpt9/uCDD+If/uEf0Gw28YEPfGAGLTO7kA06gUAg2Mnxhz/8AW9961uhtcaFF15oj5ULBALBzoTHH38c5513Hk444QScffbZ802OQCAQbHMkSQIA6Ha7uOCCC/Ce97wHS5cuxQEHHIALL7wQb3jDGzA9PY0LL7xwnikVCASCucG3vvUtHHvssdhvv/1w++23Y2xsDA888ADOPvtsXHTRRTjppJMKC70CgUCwI2FbnqADgPPOOw8vfelLccstt+CQQw7Bm970JvzlX/4lPvShD2Hp0qW44oornO/XrVuH+++/37kXzuALX/gCli1bhmuvvRaHH3443vzmN+OFL3whvvjFL+KQQw7BxRdf7Hz/zDPP4L/8l/+CvfbaC694xSvwN3/zN3jZy16GZcuW4brrrsO+++6L7373u1BKOekOO+wwXHjhhZiamsLKlStx6qmn4rWvfS2OOuoorF+/HhdffDEOPvjgWWujmUI26AAsWLAAADA+Ps6+N7vDu+yyyzajSSAQCGYDTz75JE455RRs2LAB5557bsFDpEr+AUUZaNKUpRO5KRAItjXe+973otPp4PLLL6+dRmSgQCDYmUDl06pVqwrvzbObb77Z+b4XGVgnnchAgUAwH3jwwQdx1llnYbfddsMNN9yAY489FiMjIzjkkEPwla98Ba95zWtw11132cXkfmRZv3JTIBAIZgMaqH0H3Wxt0Q0ODuKmm27Cxz/+cQwPD+O6667DmjVrcPbZZ+Ouu+5yTiZXYbfddsMdd9yBv/u7v0On08EPfvADbNq0Ce9///txxx132BDEBoceeijOOeccHHbYYfh//+//4Xvf+x5+85vf4JBDDsEnP/lJ3HPPPTj00EPZsj74wQ/ihz/8IVasWIFf/vKX+NnPfoaXvOQluP766/F3f/d3M2qT2UZzvgnYHmAuDnziiSfY9+b5AQccsM1oEggEgpniueeewyte8QqsWbMGq1atwuc+97nCN1Xyb+vWrdi4cSN23XVXO8FYuHAhFi1ahE2bNuGJJ57AEUccUUgnclMgEGxr3HDDDRgdHcW73/1u5/nk5CSA1GHhxBNPBAB8+9vfxp577ikyUCAQ7FQwMmd4eBhLly4tvH/e854HAPbeDyMDN2zYgK1bt7L3KXHybP/998eGDRvwxBNP4Mgjj6yVRiAQCOYa3/72tzE9PY1TTjnFcVgweOMb34gbbrgB//Ef/4H/+l//a19rgf3KTYFAIJgtJPNQ5tDQEC644AJccMEFld+ef/75OP/884PvFy9ejC9+8Yv44he/WJnX3nvvjc9//vO9kOrgtNNO267umgtBTtABNtzbXXfdxb43z7nJh0AgEGyPGBsbw6te9Sr8/ve/x+tf/3p89atfLRz5BtJj3wMDA1i7di2efPLJwvuQ/CuTm9PT07j33nsxODgY9GQRCASCucDGjRtx8803O7/bb78dQLpRZ56ZTTuRgQKBYGfC0UcfDQCYmJhgQ7g999xzAPITIKOjo3ax+e677y58/8c//hHr1q3DAQccgIULF9rnMn8WCATbI8zG2KJFi9j35rm5r9PIsnvvvRfT09OF7zlZ1q/cFAgEgtnAtr6DTrBtIBt0SC+RXbRoER5++GF7YTbFNddcAwA7xI6rQCAQTE1N4YwzzsAdd9yBV77ylfjWt76FRqPBfjs0NISTTjoJAPC9732v8D4k/1796lc77yluuOEGTE5O4uSTT8bg4OCM6iIQCAR1obVmf48++igAYNmyZfaZOUUiMlAgEOxM2H///XHUUUdBa23DWFKYZ2YjDyiXZ/3IwLvvvhuPPPIIXvCCF1hZKxAIBNsCe+65JwDgN7/5Dfv+17/+NYD8NPGBBx6I5z//+ZiYmMC//uu/Fr7vRwbK+qFAIJhrTOt6P8EOBC3QWmv9sY99TAPQxx13nB4bG7PPL7roIg1An3DCCfNHnEAgENREt9vVr3vd6zQAvXLlSr1169bKNDfeeKMGoJcsWaIfeOAB+/yWW27RAwMDenR0VG/YsMFJs379er1w4UINQF977bX2+Z/+9Cd98MEHawD6pptumq1qCQQCQd949NFHNQC9bNky9r3IQIFAsDPhG9/4hgagX/jCF+qnnnrKPr/77rv14sWLNQD93e9+1z7/wx/+oBuNhh4YGNC33nqrff7AAw/oJUuW6GazqR988EGnjE6now888EANQF988cX2+djYmF6xYoUGoK+88sq5q6RAIBAwuPPOOzXSAyb6y1/+svPu1ltv1SMjIxqAvvHGG+3zr371qxqAPuSQQ/Sf/vQn+/zaa6/VAPTBBx+sp6ennbz6kZsCgUAwUxxxxBH64EOb+oE/7lXrd/ChTX3EEUfMN9mCGlBaa9lTRRr26MQTT8Ttt9+OvfbaCytXrsSaNWtw++23Y+nSpbjtttt6uvRQIBAI5gOXXHIJzjnnHADA6173umBYjc997nPYbbfd7L/POeccXHLJJRgeHsbLX/5ydDod3HjjjdBa45prrsFrX/vaQh7XXnst3vjGN0JrjRNPPBFLlizBT3/6U2zcuBHnnnsuLrroormookAgEPSExx57DAceeCCWLVuGhx56iP1GZKBAINiZcPbZZ+PrX/86RkdHcdxxx2FiYgK33HILpqam8I53vAP/+3//b+f7z3/+8zj33HPRbDbx8pe/HO12Gz/5yU8wMTGBL37xi/i7v/u7Qhm33HILTj75ZExMTOClL30pDjjgAPzyl7/E008/jTPPPBPf/e532fDqAoFAMJf4yEc+Yu9eX758OY444gg89dRTuPXWW5EkCd75znfiK1/5iv0+SRKceeaZ+MEPfoBdd90Vf/3Xf41169bh5ptvxuDgIG666Sa89KUvLZTTj9wUCASCmWD58uXoTD+AG35WvGeYw2v+ei3arUOxevXqOaZMMFPIBh3BxMQEPvvZz+Kb3/wm/vjHP2Lx4sU45ZRT8OlPfxr77rvvfJMnEAgElTj//PPxqU99qvK7Rx99tBB26KqrrsKll16K++67D+12G3/5l3+Jj3/84zjuuOOC+fzqV7/CZz7zGdx2223odDo44ogj8L73vQ9nnXXWTKsiEAgEs4I6G3SAyECBQLDzQGuNf/7nf8ZXvvIV3HfffVBK4cgjj8S73vWuoHy6/vrrceGFF9o7lY4++mh89KMfxWte85pgOatXr8YnP/lJ/OIXv8DWrVuxbNkyvP3tb8cHPvABRJHcpiEQCOYHP/jBD3D55ZfjzjvvxKZNm7DLLrvgRS96Ed7xjnfgLW95S+H7OI5xySWX4IorrsDDDz+MkZERvOxlL8OnPvUpHHHEEcFy+pGbAoFA0C+WL1+OqekH8MOf7V7r+9P/+lkMyAbdDgHZoBMIBAKBQCAQCAQCgUAgEAgEAoFAINgOYTbofvCzPWp9/7q//pNs0O0gaM43AQKBQCAQCAQCgUAgEAgEAoFAIBAIBAIeGkCi64UQlxNZOw5kg04gEAgEAoFAIBAIBAKBQCAQCAQCgWA7Rgy543dng2zQCQQCgUAgEAgEAoFAIBAIBAKBQCAQbKfQUIhR755fLRt5Owxkg04gEAgEAoFAIBAIBAKBQCAQCAQCgWA7Rt0Ql4IdB7JBJxAIBAKBQCAQCAQCgUAgEAgEAoFAsJ1CA+joRu1vBTsGZINOIBAIBAKBQCAQCAQCgUAgEAgEAoFgu4VCUjPEJSTE5Q4D2aATCAQCgUAgEAgEAoFAIBAIBAKBQCDYTqEBxDU33uQE3Y4D2aATCAQCgUAgEAgEAoFAIBAIBAKBQCDYjhHruifoBDsKZINOIBAIBAKBQCAQCAQCgUAgEAgEAoFgO4UGkMgJup0OskEnEAgEAoFAIBAIBAKBQCAQCAQCgUCw3UIhljvodjrskGciJyYm8IlPfAKHHnooBgcHsffee+Ntb3sbnnzyyVnJf/ny5Vi+fPms5CUQCAQ7GkQGCgSCP2eIDBQIBH/OEBkoEAj+nCEyUCAQbM/QSENc1vnJCbodBzvcBt3k5CROOukkfPrTn8bY2BjOOOMM7Lfffrjyyitx9NFH45FHHplvEgUCgUAgEAgEAoFAIBAIBAKBQCAQCGYNCaJaP8GOgx2utz7zmc/gtttuw4oVK/DAAw/gO9/5Dm6//XZcdNFFWLt2Ld72trfNN4kCgUAgEAgEAoFAIBAIBAKBQCAQCASzAq0V4po/rSXE5Y6CHWqDrtPp4NJLLwUAXHbZZViwYIF9d+655+LII4/EzTffjDvvvHO+SBQIBAKBQCAQCAQCgUAgEAgEAoFAIJhVxIhq/QQ7Dnao3vrVr36FTZs2YdmyZTj66KML788880wAwPXXX7+tSRMIBAKBQCAQCAQCgUAgEAgEAoFAIJh1aChM62atn4acoNtR0JxvAnrB7373OwDAMcccw743z++5555tRpNAIBAIBAKBQCAQCAQCgUAgEAgEAsFcIpbQlTsddqgNuscffxwAsO+++7LvzfM1a9b0Xcaee+6JtWvXotVqYfny5X3nIxAIBDsqHn74YQAQGSgQCP4sITJQIBD8OUNkoEAg+HOGyECBoB4ef/xxjIyM4JlnnplvUv6soAEkNQMi6rklRTCL2KE26MbGxgAAw8PD7PuRkREAwJYtWyrzCinbtWvXIlEKyegoHly/Pn+hUM7ZVe97gdkI1xXPIgDJDMvpleZe24GjO5Qu9J0i7xPved28y2gK1SmUv++oUKcNaVllqFunKvTat7NVbt3yNcrbgks/m2NsNlDGR3Vpnes61cmf6/vRUQBwZWDo217Kqot+x3avefl9xn23rcfGn3v+/ZYz2/3UC+roRVR8UyedLzc5XaS8f/cKWua2aNOZ2lZV9e2Rj1T2rQ7JwBnm3xc4XVmma3q1abjvmPxt29C8QvbqTNqkrL6hb+vw62zrr+3NJpltKOQ29zw56Ko4/b9uhD7I/l+3H7anPtvWsraMDq7cKhlYd65aVm7db3tN369tOJu2J5dnP3Xut52iLEEyi4N3rvl0NtYMZosGP/+qZpyJvRUqa65kw0zHLpffXNJXZ21pLtrJyMDn1s/+/BaYPb3VrzyarW/N/sBM1iRnG3Nhm9Ypc3tZu5qJfuyD5umJid4SCGYNsd6hbiwT1MAOtUG3LdBqtZCMjmLf//YROCdGyWaYIkLLfKMbQDSd/V0xTmx6Hfg2tBkHOMovGUoQTdQblHZhxVvcU4n7XCV5+VoV0+kG0JhM/5G0GOldYUQp7dHgfacS5r0CkoGU0GiS1LeuQeBvZNK6IusDQrOts8k/tGBv8plW0I1yTdbopB/Hbf47S4tZhOhTmdN8/L4tTxguV8XV9ePyswsrDIs2OirYFsH0/S60qvzftC3YMdELtvEGXXDslEC3NNR0RaIKA9xpp7IxV7MuhX6g49CjScUV8pQuHnP5oAeamXZQREaX8XMdOHIlMM4ABMdaXX7ldBFtc6Xz8kN1MWWpblHW60beFlXgaI5Mnk3N6tL8Q/Q22aupD0J8Uta+lWNvG23QRR2FhMpNv/+I/gYY/V5RD8rvVQtDdeWR8mjy0+imtnzGZ8CX77y3mfHvo0z/6qYulK8Sd5w0JjJdPZBl6Y3HAp/M0eTbjsFYFW0tf+PE30AjMobWb6YbdM2tph2BeEDn7/08mTbpRX/pBmxljFxkbUPAHfdV9etDfwX1Skka+zpQ515sD1M+tMoX3Mkjjr5Smhk6fFDejqYyec3Yaw5vVZVRoXPo97RdBp5LE0wtzjvN0ZU9yijd1lCdeoxY1U91+zGo5/pchJ+x7ernE/HzBVoOa09zcyTzvpvPG6rmfdC910lpMl8KjOfKOVDNuXYwbV0Z10M/h+bqZWlCSEZSQy3amjaUr+sKurCOjK6gxdh3pu975lF/fu7xo6N3S+gI0ZY0tUOn+TdHQyH/Kj5i1oiQKPvvvsoiPBPi577GTtnYjfO/7XivWgeomPPXpcuW6Y9Df7x5/a8S9LV+wrUdK/camNHmE+VdbfhAoXp+bjMAK28MQvNKFqRtq+pfxXu2/MweVFMzVEo1UMXv0bRns3h8U8WnvdpPBVldYY8bODZbVT/0A9J3zvwfRX4uyEN/zZux3/xx8sT/uhD7L1nSJ7GCfqGhkNT0oJM76HYc7FAbdAsWLAAAjI+Ps++3bt0KANhll10q81q9ejX7fPny5XjwufW5QCVGirULM+GvyaJJNF3DKPEMiuD3jJLV2cRcEU+4aKq+FcRNZJKhBNF4VPiuvSl91llIJsSGpgTupMvLVyWEfsUrFmOcOItyZYpIA41sI9JRXqYfqha0tfvMKiCrsVzjw9Bl+9czjPIFE9jyrUHAbVoCiAfz56qbpSNNb9qx0sCtmNAmxkiKlV2UjId0qWHpKF+uvyo259iJXoWx3B3SiLruex3BqV9hAZ0xMhyavUVF7Y83apBwC2s9LrT6fUXpa22OML2L2+iFTRIASQO2HXqZkLc352OU29ix7TOtkAwym9u2Du6ky/B8aNLgjzk64a+qsy3D+7cxorlF+qRFHALapEyOT0PjiWvDLH00raxsTVrps8ZUvgmiNW1XowyY/AJoZov93SHSN4FNx7JxppsayNqHXUSJ8r5RWrNjz/JcW+fylC74MmNPM4uyUZdfIDALuVTW2fKdvqFtwchwukhA4C8oFGSAJ+OczSw6aTLy3aubrXOjyI9aBWglNNtNIENnlG8GURlR6vHrySCTp9EruqXdSRGjL6guq7tw7NNp8qGLeYAnQ4l9RPMrjHdPrucfZ3WcVmFnGITHu/8NrUchfabz2QUHbyx2FxgdSvIg7cTVT1H7xysn1N+Vk29mQm+ziYkuIe3D2nkD+QZow4zRts7HJNH/HKi86u5CFjxMGrpAVtKPANxFt5KF8FQnug1D2zlp544nzbFMxg7r4DjKMyk+ck4FMpt9Ri+o2BsHJXnaV+Q72mdc+SGHPWPzJ01d4F0gYCtyNhFHn/J0JbPomQxq5xnNL17cRWND0+ZlyqTjIMlmmcZpxCGTWfT1HSOndksza2cbdZ3RJLfNyXzEzIfopp2GqycAQE2pcicFwsfsBpBXPz+N+TfgPrPt0M11tbPpSenM0iVt3naz9DNjiLMztQKS4fSjxlbO+AjIDtp3JlOy8VImd+PdOog2pIPHWTymPMbwczKU1bnC+TSaVkiyyV1oDJQtdhYcF6x9wKfn8rd/kv52xqZnywR1IYGhKeqofCG/5oKxj8ZYw6HDsZGZtoEG4J9U9eUzRzfhQ7sxx/CTY1OE6qTd51T/OO0b0p9M++YL5bls4DbLrI3c0lbGOGskjC7XUT5HcewMg5aG9jbufAdFdq5MbQXfgddbWDf2obOOQea6VXMwSkvoG+23l998RIZwc5zgpqD9wH3e3JLp9ZHcHivbXI4Xd9F6NpU3cbuoK1l+B4rrPcjlnooDfMz1o0bBodHXJUYO0Pl1PJLJ5bEIOivXzMuiaaKrNFybLfufs55W07mA2kzchrfTtmZMkDax85J2Ph9RZgx4+pujpcz29Q8emG/jYV4vWFoG8kaxNgXDZ9F0vknPbcQCLp+xdJYtBfjzUgDJcGydJPSeU8C6zPsvsB5lNsvigZz3ufKDjofcHMyvC+UnIrdtFl3Cu3Q+Q/MlvFNiZgq2AeQE3c6HHWqDbv/99wcAPPHEE+x78/yAAw6YWUFk8cEoXIVcgBlFQRfw4uGE3USisIsLRujW9OIEYBcHHVDFxy220EmHmSCQbFobGrkXNKFxanHmdUeMCGrgK+ohxyhfR074Nl3kLWRWTFYABD0l6SSdflsKndfLtoVXB9/rxl8g9CfHKiHGQcAwciZ93OI5VdTmT8444IwIYgSaibxu6HRjDt5iHpNnpXdOxUkWbjEytHhs4CxWBerHLh7TZ36ZETFkOD7wTnIUK1LyztDJTKA4nuvsGhc2nLg25zY7QrTQ7zqLEvudnTgxhqHSQGPcbMYwedJJl84Xb+jCkUO3N+boGHbqXGbAeihsbCAfD9zYCo0HuoFiJ0I6MBc0sphuqGdtESudy1tOXvSA7jDTNw2wRJVNWqKOytuUoyMp9qUPy3MdxRr+zkTWyD6GpzhPTa2A5IDJ9B9/GsjJMjKIeFVSWe5PlCjtBY9VpswyJAN556eT1/QfdsE5sPDE8SPAj7O8MHeC6H9nnRE8upXnbe6fOoiHirYGK7sD/MRuljF9atMQXZe0inZD1Mknt6w8oZ6ipB3YhRUqt8u8czVK5XbVBl5os4zKSCe9twBYtVBLZbjm7DCHGPJnxWaWeZ+0dIFPaJu4/JQ9I4us0WRkeT+mPFpmc1A6qLwyQ4dz7vI9a6kdCndTDSALK8xCqWMbEl1g+XAq39Tt7JFWOtrSqO357JRF+YCRN/Q0sv227iJ+YMGT9j1rB9EsqIzsUW9UjQ2lAWU2zpi6UJuKy6f9dCt3PjFt45XZ8E/gJWS8N3X55no31+tTe6SEqCmFxni2gDWYO8c5C0wlMq4KlN/Zk+JUniXFNG4FchpsOzR1rhcDJ9HN8zLbjX4Xsufp383NDVu+TU9lgLcJCbg0F4QD5RdmLjPw+AC6C4idyukqxqGhuYXQWaLf4sHczmM3on17wZtDUhnjODnUlB10DkPr5oxnTwaadE4aH6aZyaKrPZUWcUwWBudAW1Y/3SD2Ty8n4KicM5s0nCMq83fwlLGxeSJ3XIc29IL/pq8Y+8RfnAeA5pYo178VUW+g800Ws6ZD84sH8w0H1iRQRTmgladjyGaleW+QtHgZWuZEEpoXOWl9O85zCuEiCZTN+QsRUqrmuEuydajJogzUTPrmulwXqbjcpnR0PkOrkXu6oQGOjzmbSSF3+tPFNHTDI38INDdm8q5N5q1mLkt401nfoFmQ+UqZTUVlaWevVOlHm8gScIVooY7EzmYYacsC6JpLTZ3c2BrxfbLVrG258tasczlyq2Ri5IwXzg6D2+ZsNjXFsMmnualp+6Tx+GDuABzI33xrHHwL0aaMKRVad+TGu9dPzhqdYvQjicJlvgmiN7UkmGVoALHcQbfTYYfaoDvqqKMAAHfddRf73jw/8sgjtxlNAoFAIBAIBAKBQCAQCAQCgUAgEAgEcwYNJHU90WSHbofBDrVBd/zxx2PRokV4+OGH8dvf/hYvetGLnPfXXHMNAOC0006btTK5cFbcabbGeFR+OgfEg4F4WdX27qwaVNwJpwpv7STQ+5z3jn1HT2qEPKBKvB7TTLL/MV5cwfuNSrytlPes4MnjtQPn6eN4z3heYI4XE1M/x7OaIZ2+4Dy//PJ7RShPLu/KEFQcKsIlBPMLnVAAer9fCow3uFdm6DRaKA54zyfCfSemMs/0Xi5lrxov9FPvJINmvCsp2PHgl03HR+heyaxMls9tm6rCs37BnW5hW5TQz8nqUFx+1svZnJZOXBnQ15ih9GVpnVPI5ISfoaf0FFBE9E4opG/ZKaTAt04ZJZ6mdaCfHSg8syfnIuJRzaXlvBdnakiGeKNCL/UVor1CBtuivbwL4d38MCPcyXkONFTJLHk6cjzCeuYG8mf1c9V45LKskNO17gap6VVf+Lsm+jm5VdUXTjhFG0eoopyAfiyc0tIV39HXjeIdgfT0XOgElxM+DiiETS7lpSoeIX+b0EH+87r5svfNcDYTyp9VgTs96rwP2SSmz1GDvkBfhKAVnEgQBboq9BLHR0EZx4XlrMpf5emdsGDDxTxtkn5sO4oa8snKRmPzeEYWF47ZDeNs/p6B4V8Djn0Rso+Yb0v1JuXhEtuoO5LUtx9ImY7e0d433rOy8VRAVRSQHsHZLCGaZpp/6OqE6ky8fwZOejplcVEN+mlnrs49zPu4dQZnPtCDvUvTOGHbUPw7LbuaTieNsdPIPN/Q19jSqLwmgtW7Xpi9YNqATcWd3iwrj5YZsh1YPqfpG/m33KmeQqhOCuX2adSpEOL+Ooxy58B1+UIxp1K5kPtcmXXLSfNEsZ808jl3BKDsRDTlfW49CvVtqmhz/aVflv96aYdeRVcoHypvnZPXaQGUp0qjnVT1V4Q8ksIsrWPQOXvSqqbB1CX2o7L4MHSG5CqxHQs0UXs0MF5LT0cKtiOo2ifo5trmFMwedqigpe12G+973/sAAO9973vtnXMAcPHFF+Oee+7BCSecgBe/+MWzXjYVUI0JhcaEco//NuAu5pZmhuqJqY8Ihd7iFtCC0CgI+XjX4qUQdnIWWjj23sMzEEJlOe9oWR5CmwxcnioOf09/hfeN9Kd0YLOO6UcnL0NLVneVpJtDNIzZbKBq0Z5rE93UxVjxFFx/VaHmgr/Pj6Yd6/azD78fqsZLcNKhCA+RvPtBiGfMO4u6k9AAb5d97/N2aBwYmkrlklmUC413uO/Lyur3Am0zHkMoHVtV7aZ53rQg9W9MqTS0iQbiBQniBYkj78r6PogEhXZxJpG07iVj0yzwld5TVyIvChM+pqyq+tn3ZbKdWeA1xj/HhypWhU0obrz2hVCfk/7kngfbsU7dNf9dlS5is2RkQ+k49vmsj/Yrc4DoFzMNjV9Zj14dPTIZZvjZXxDuZ5zXtv16yTPjGUprbV1BecYPfVuWnpMLzN2gHJ3+s0KePdgdVXV12prwfqXNxOVF0nDpOX4o1SnOh245db+tKsuhiWmrKp3K5c/Sl5SPB3Nfo6U/sBgcrHudRXCiQ2yoPmKH+HWdzXEYlNvGBgvpZK8tjB1sNoGjaeVcIRDMv0Ku1HHsoRsSMxkb+UPkvMboHZuml7JIE8aDOr/Hlubjy4MZOGU438yCniubb4a+rcRsrKP59euhruycd6bgHMN8m9DoYGau4duxPW981vw+aWkkAzoPIefRVsiW0c+GvnhRfwsDVC6X6Rv/rtwZLaaHbGMg3bgoa3NS/9Ac0cojLv/Ey5+RLWx55p8Vm6AhqGlVcB4KjueqcVBz7Ssv3LXzHNnJrZsQnrDfMWuDlZjhmkNl31CamTleVXml5XtzNFaXltUvMBfN6UV/62QlcOqU3U0eqqehQcXVdoJdgwytyfQxX3DyJrQJtl9oAImOav1maVov2AbYoU7QAcB5552Hn/70p7jllltwyCGHYOXKlVizZg1uv/12LF26FFdcccV8kygQCAQCgUAgEAgEAoFAIBAIBAKBQDAr0FCYrumRp+UE3Q6DHeoEHQAMDg7ipptuwsc//nEMDw/juuuuw5o1a3D22WfjrrvuwkEHHTQn5VIPAuPhVwgtU9Nbobb3LQXnqdhLCCbGK4oec6fvS72wuMtIe/TI7svzbRZRegqoF3gnLko9WbdVnUl/sPXswfPJybPqkzk4QdCr12aV5+BcH9MvnBqYAWbsCZlhm46zOSirF8/koAd/Td50vPLGozSMGzlRMxdjuK6XclW5s3Gyr64M66WsKl03Jyd+a6JwUiajdbbGXh30bAv06AFfLLBGwtlu8yoP/oo6zTbv05PdBZ7vRd7UpG+mmPHpDL/9+/UAr/Oeg5E1M+Vdn465QE17tba9Mwt15k74VfFDLxEdqk481T4BN1unbALlU7nMeZtvVyg5udSTTTPbdJR9Os/zMgtqm5TQX2nvV+nV7aGuZZiDsVSpK2exzNr8VHJKmpWxM5Aztfm7h/UcgJFBJF00VvMos08Cc/q1r3WGPtqKLaOGnUrnSvOOmnZs0tZI2uFGmlO56OkFg+C8wK+TRn9rOtsSkU5/c4S++kejOMaztjWn9Ob0qFHFqUczjpKWLg9vTNYgZx1y1GqHQgxV6yfYcbDDnaADgKGhIVxwwQW44IIL5psUgUAgEAgEAoFAIBAIBAKBQCAQCASCOUUy6yckBPONHXKDbt5BTzwQRNldHaUXtaJPbwemzJ48laiXCJPej+1eSkfZgYcaNHEX3c8HQuXPNX3mxEjV3S79gPMwni+Y+rGX09egzcQt7ze2fLFQWP7v18tPe+OHPptNz8HKfIwu3l495wIyEoBzUbv5N8DXuS8ZV4XAhcocn6mYHBmbiVwLyM26nv8qVpX3z1VnYj6emYzj7pDQjfzExoz7keZrZGVZHP7ZwmwXUaErgXLenxUSTP7GE7uh+9M7cy1vyuRFBe/Mt56bK+jW7PD+rHq0l/RTXVqA3uUV+45EfKidZxUC/Ndv/lFX8fOBPsdTFS390GrT9EZKlrjfhL3D2gzboWe3bpbL1ar3s4lexjvll1LeCdhMswq/f2d4srpSn88B787V6SFfj+/MmEt9Xrt//FNK5s9+bNtZ5LOe5DvVMZ4u9edg2wNmOnbMfesAap+A2pan/XopK6TrClE/YmVPp/Wl82ej7hV22naorivXau3aU3MmxlEJ6t7fN8fzw1nDNrQDBUVoAHFNgS7dtONANug4RIAe0FBTuVRsb47QWZhKS2os278bKB5FDmymUeHvCOASRZcMpB9GU5G9NDQeTuzijUOTmQiSY+VJK/1/YypfMNANUpbRQw1Y7a26qmjYmQtUATQmFDq7pSvM0XgmHBIFUKXn1UVp2HACqqsKCkgxhpWOeIOCU7JRRyEeSXL6GRrst1nb0YugVQy0Nqd1mc762w9lqrg2Y+pr6IsH8kRRRwElF+aavFRM/jZhDAB+Qh2l/QoA3eGs7rHKnw1puxFQe0JOQXjT5KMb5O+I2WhQeWE0rIrtZzJB4OhQCXL+9Z/7ZWbv2xsjTC3O+yyazl4bPhnUdrzQsHqWjri8TaKOckNheJvealpBm/d07NONRlJ/Ww9irDnjAChsZGlmYc+m12QTx9BPDSdPHtkyDf8m1Txhxxypp+GzuJ3Xz5kINtxnuknoV0B7/zEAwNTjCwrlcTKKwmkT0jamTVSS91k0lfeDkYeUb5PBTMZORvnYHUxsY/iXsQPZOCI6wKdTkXrmBal84yl7n7Q0oqm0AgW9gEz+G7HeUfxCAVOW4b2kpR2e014/+fmY9snzyeusplUhLKWa9njVfGtksMrlbN43ZOwlRd4HYHWwwwekjNINc7K5qiPkY9OUTxZMaV6m7wuhN/2x68slv0/oGAvYzsZmiDqqkCcob2Xpo45y7Aw69k1FKB/avwnvmL85HUDleqguDszGn6nHpHLkqc2c2CKFRQai35x+JjTHhHcorYBr03BQtB0AKLORTOWGGYfDcT7es/EYTfP955Tv8ZatF6plhKmXT5MZu3Rzno5x244ez1g6jEkW6cJF847+JHqDcx5SMZFXWZpkINelTtg00wxtDT2cftzYlE0zknInA+WNl0Kb0PdNosuZ+lM+sfxO0kDldTF1jUeS1H6FOwaSjPw0jFnFwo0nV+PhxOavugwfEf2sBzRUx+2nqJO3WdICDjvycQDAA3ftn9fdjDeSv9XTLc2OXcOHUZfYmdM5HysiI31dpRWxbUcSRJPegNaZjif/pv+nNoGjv817OlfSeftHZNxS29OChIMz75v7jCN+Ytilg4ydgk1nPvEeJW2NaDLK6bN9UhwvoPRxvEn5yIw7T3cW5IRy60z1pvmOygYre4ht5+TPzEVpXZoTZj6RPZtWucMADZ2q83rktkz2zrOXOLlP65GY942cDi7MW0G/ZHX2SGKdwOh7NV2tl0p5M8n53NCy7wufwR9X72npZ50TG24aH45Oj4zcJfqdyjUq70vysnRO5f1YF0p7/Orzlvbm4r5eCum6LL2OApvXxLawazGMTVRYZwk8t2X6djKd02dMmAwltu8a4xGvg4zc6Cprv9kxRPojmlT5GgMdz2b9gcgfZ97HtJ+Vzx3lynB/3ujbxf7zgN1s+rk5ruzYpzxo7CzW8SRCbq+HeIzIQH/+T+2MsrRpAe6YANK2aUykmSYtzc6hbVZGv00rO22MB/J1Et9uTjMtym0nT8obRgaRdcQq25r2eWPCzD+1w5u2XEt0UW6F4LQHJZ8bp8zYoPqBkweObUTTM23WyGyGZCC/KsjYqI6MIXZEbq/meSZtXRh7VEYYOOMe+ZohJxdSua4L6Z31Jk/u+vrNX5ukNrxWxfxVV7m0ZDo2X9dz1wT8PomIPGDXnpq8HejMJyp0HRf6veermgRzBIWktodx3e8E8w3ZoBMIBAKBQCAQCAQCgUAgEAgEAoFAINhOoSEn6HZGyAYdhwTO6TkA9vQc4Holas+TIgTHy8UcSWdChYVgPIO0Il4ZxNuWnojgQ7VlZHqnzfxTf0rDOeFVOC2gco9WPazzk3Om+Eb5qUCt4JxsK3j/UA84h37Pcxl8PZO2zj2rAh65mnjm2e8IffbknOexFaTJr7OhhXge67ahOeABSDzL8vyL9IdCkcbGs470XeJ5PvsIhuIzNIH87XvgUd5n+qxwOsZ7T09KcHQ43wfK9OsxtWuS9znxbs3djcrHRogWA9/T2q+T7wmZn/SgxBbzLfWA8+vJpY+4j8n/ufalzoeMd3GQnAYZM9n/Le9xTloczZQfNTD1RPHknP10IEFjPJd9BXq8061Oucj6hJxSs6+9E6kAEHWKgieazk+r0dMlVB6Wht6i49amIR6C9vQD8WxOinWlHsbBPmLGIz394Iydkn6mJ5Zp3vZRxNBAvPq0Iu/NaW1V9B6lz6gu5GS1Q3NIbhgXfOtlqp1vCqfN/TY2zR8Ke8LoQr8faT5B+Uq9q7mTcwYaBXkXD2r+5IDK29nRe3DTV9nwURc8P4fsFDOOaD38sqJ8DHLyIJpiPMzJ33Fbs31ubZrAKSGbjdfmBU/QSEOZk1OTDfgItZljU/k6VnlVqQjPVyiDevsO5KewfK9zP0+HR62MI/KKnDLm9AZ3koF6NNPT4vY9TWLtGIVokze9iEo6yc9HIx/PBWHkniJ2xhs39sx4iImXMkj9q0KOG3lAPfxDsp7hbdYznbHJVEcV0ictnZ+Qi72TcwaGJGJTJa0SOuHKXXo6suDhH2mWd+2JTu40tz/ePPhRAwoRM5T7rc6ih+g4st/T8cJ5fht0nxwmMjL7X6ysjuBOz/k0ACm/Ww94UgbLM0yWjh0R5RmwNo2vF7M8qS7gTmKwcoo7gcbMUQA4crE75PKPbuQyjnrqK01sP3t6M/+OPUlI6SDj1daJiTASmouxc0VCepkurgqdz9nbDm+SUxfm+R/v3ZPPn/RdWShwgMr43H5hTyTpIn18JQAkGb/3eHrOZEHn3ZZskpXDR2ZscJEGGJtGMf3tvKdrMb5NyNBaOsfiZDGlyci1aWX5MJSnPRkdo3CC3pdP3Mk1zcgehye40/iUpTgZwKRnQ9IH1hEMz04vLNpUuq3ZdSRanj1lpRHk7/SDYvmFU0X+2PWL9EI7qthdz6rkE6R1cqIWcacvCR/Xkhm0DoR3UxnqzkFUVzmRRUzu8SCpB+VNz34I0sO0WbAd/X7y5Io9bU4jFnC2nGNc5Y+4E/i0ftZmMf1Pxxsj650TbiR/mp9/6j50gpyLSMSNF2esMXLX73d/fPi6wF+jdOYLxDbvjuRyhRuztk88mq0NFDFtasaLH93It12n8/EUihK2vYXJ/bOFBpK6R2l7NwEE8wTZoBMIBAKBQCAQCAQCgUAgEAgEAoFAINhOoQHEqLdbKvtzOw5kg64fzDDubu7x47+oTjObSLi4xH7sZt/bnDkJ4aPq5FlpPUN39TAeW8E2KakD6w1X4Q0Tek9P4FWdgrKnHRkPQq6MED2sR2ggH9bzuYqPZkl6sx7O/ZYTOgFYpo+IR1DodMpcovIkWsibrSx9wJOzmpgadNTNqoTuoJd0Bcq8rKOJqD6NHJ+HZDXDG5QOTt7U9ZZ3Xvcy3mZLr9Sho4qPymipOAlTWm4P73ul2fcmDaV3TuipwLczoKdU3lTpmgCsR2RA9/ZrH8yabKyQMZzXvYHjTdsPKpL73txlp1PpHbsz0hGzqF+CcqeuvJ0LGaPdvwtlV9lEdVAiZ7RiD9aVgzmZDPB2EoV/giNMFPNIwe2nHtMDqD7h3qON7UOVnOIN2ftV9ksVetHpfpQO7pue0IP+ChZZYe9z86ba6THz9u0Harpmp9SdTwTGWy8otQ8iXvaUyfpZRVXfBE6C1M6+at5XJ4+KEz/9oLajfpV+qiIt9H629HKZLtTV6xzO6Uk/L1JOMsCss1QhYCdyY3Q25/T0EFT+MPt/D+M5+F0PfNzz3KFiXSeE0hOtPdgxVd85fZf9mQSiQ4A52UUjYlTKg1lYXylDcH2C+65X2eWvZ3m6sI4urTy5zZW1rdCLvdZLZLay50zEgtDJYIOYnvANzDcs+l2bEswSVP0TdHIH3Q4DOaAagNLub2BDBHM56OC6CIPrIi88miqkoeMgIossrS0RWluinlpfxfnFrPaSUpI+6qq0DDr2FHkeZd+T9wsO2pTnRdDYexyNvcedslSS/qJphWRRF8mirg3XqBJS5yj/qQRIFnaRLCS3ikd5XXRT5/SZsieVLVM30h+itFxznNtvZ6fNAeiWdkKCRNOKrSdNY+qBCEiGEyTDnmYk6ZsT6SXqUSf7TSuHvkL+Kv9FJCwpzdNvB6eeMWkTlS9OmXak7d7eFKG9KXLy1C1t0ycjMZIRb+ahmB99HvHtTNM3phQaU8UxoDTyC2ZJ9Ro0PKvpZy9Pn6e0F0KytM0IT3K8kwwm0E0N3dRuOxOaClBuXobPLP2ElOY+40wGbl5QCKa3MoIuNpB+oOPK1MO/+FvpNF/Dr6F2on1TGM9w+6fQ/0YORWmdaVumFyMHeMYgEKKI9pNpcyoPbHLCR2qPKag9pgDk/Bh1FNqbI7Q30xiEcNs/+yXt9KfbGslQgmQoydsjyevptIMnb30ZY2UJeZ4MJuz74PjK8ubKtDxO5JAJIebQbGS/SseeXtKBXtJhZZDVI+RH5ZiOcp6w5Sh+HBs+trLKC59sx2DMy0sD3dT52GPa2ekT0s5OnkaOLJ6GXjztpI26efsZ/Q64493wBO0TKteTwSSVKxkPOfQbPhvQTpvu94JnsN8LnnHbmeRv8rR1GkocPtRR+qOykI79gtymvAEU+kQrRmdTkoguouPQ1DnqqkI/6FbeD3ogf2HHVqzsePXblbaLn69Jb2WCJ0tsm9A02uVzzmayunKXGMkusdOmvl6zstK0Y4A3C3XyQzgz+tfkOXrIc/bvRif9hXT1wociLHwocsZwe1NkaX3VyrvxqpV3u+VSUkidbDsMJbZ8Y1/4oaR0BMfmUgksrcbmaI0plveoDLF80tRImmmoIq7vW2M54fEuMeJdYld2cF1AbAFqNxg7TiUpf2oSckolrj3oyzgnPBvDR1FXoTme/mif0/okC2L7K7xXrrwq6EKvflbGZzY8a8eQ/HVDI9pzMv118nxpm5txQmWF+W5wbeSMY98eVUnevtTOoW3ut5nTTgnsfOPE4+7Ficfdm7aZJxfpM93Wts11S5O6Mv3H6Fpri1G7NlbOuPfbCcrlHSPvnTmC+bubz6EcED1tynVsN0YWmjFG7Tin/8h42+WxCLs8FjltztmzlD8Mzcb+1m1dyJvai7Q9d/+1Wz7tU18GUDt8YH2EgfUpE1C94NjJ2bgwjnicfG5MeTLes4NA9CalkcojyidKw7Gj6ZyffsfNBZ18plW+WG/6lMpAIncMjCykSEOIuvq7WKBbDyo36TgwMjoZToq6hrSJU9eKso0usvXWXpnZePXncKb9jNx15r8tjda+W9Had2tuZ9HqhuYZnN4l7ywf0PmhyZ/YTE7fEhnxyVOvwSdPvSaXQQNkXkH0J5UX1Cbz57LO2IM7TzEyLW+w4thz+nzfrQXZ4fxNyjR0JsNJYewmi7rOeCz0v8rtfTq/p+D6nOoZOpc1/Uzbjq5nNSYVGpN5mxqb3cDmxbQJHRetzRFam10+tfpJe3zMyDOap1+mw5+7T+XtYMYzkfXUrjPrPIDLc60xlY7hROUhwHefsnnbPjH61+eJLEk8kiAeSZznppzCmDbzpuwHePrX0EzeUxm8YE2EBWsil8eJLKbjwPQrpT+aUoimlO3vqKusrlU6tw8cm4rTr6ZPWtq2k+ExgOh0quuJrqMorJOU2NFKAwPP5bx52D89a+2z0LyGyuPCWqRTUNaGXp8Zu8G0GbtmQGwSx07M2t7Yyf4aIqXXysUpFR4Xgm2CBFGtn2DHgZygEwgEAoFAIBAIBAKBQCAQCAQCgUAg2E6hAUwnEuJyZ4PSWkt/ESxfvhwPPrcee/+PjzghDvSAtqFujAdR4h0B1ma7k/EkUDr3iLGXpYbGk/F28LykQnmb/AHvaLPK8zCeKzR0W7IgRjTWcN+3tA2ppBs694pgHKK0Iidr6KEsEjLIeNlEE8a1xvVGKYSbSph2IWkqwwhovl1pevO38U6Lh7UbSiDO688X4v0z87wD3PalfeL87bWp/94vJ9RezikF5T5LmpknErIQG6b+xkPWC7cQutC7UC6lz/P6AVwPKuNlH02rQr81Oso9Qk9oMXn7vEUvnecQdRUbmtB482mQMUvGBq1D1aW3Jq+kpYu8SPJMBhNEk9UKU+k8DS2b8ogTVsy0Lx2jHO/MFFQGcfLIfEbkGq1zGU1+O5sTndFWxt1f5RchOxfcmz8j5LxtLqOmp2I0Q0sgz+GnU6LG98y91hpbI+upSC+mdkgskxe07RgZbk4qx08PF+tGs6FyjXuf8PzjXKxN/jbjQHmerDaNLj6zPEl4wg0RWmwHTtc5/UHrZJIxskZ1ifd1XYuFGeMhOoFcXho9njQ1TyuVx0ZGOfoje8fU2eZl/jb9YOSx7/HK9INtF//0SZae60dbHtHvoXY0pwQb41FBr4RkjNXzVOaZYiLSJi1tvzHt7+g3TmSSMc5BxSo/vcV9F6X8Y/LnbCFbj+EEjbGoQEuZfobKyw21k+1/Op4CPFFIQ8CNAZUw6UmbO/lX2JHOe0bu5zJA5f3HyQ2S3siaaCqXy8HwbZSPjexgxljSDNgCVfUjXumWfmL7UvvIb38qA2zZIXuT9LdKcnnDjidmnHB5aeIpXRm6LmCnWRC5a8pM2poNp2pPWGVlOuHLBzSi7FSR0w4c71TIUnZeRNqHjmGavmBPEzuRk4EhvcCh7lxNxe7Y9OUAN//z8y1tH49mVjZwspmUSfW/gbF/fJupACrDqdwracdoWuX2FaPLAN6mcew4pi50TmvpT4p5NaYU4gEdTG9OGgKoDCXGzrWIjAvZZhwf+Plw35n8Tb62TRLVU0g6P1/aJ5Qnq3RZmU1RVaYDossV6QfHJqbz1lA+Jh2KMrOsfIcnbQIUbE/d0IDJfyrXBVG29hMPakfG+7QkA9rK6mQkRrSlEabNs0/sd4SmQj/5OtnahDXnhaGxWzLHsGVl7zlbxOqKqKijnPkrZ7MYukyengw0J/yATFd7utgZjwr5SfgppqCQfvTa1xkjqLADK+Zlzt+M/lfZCUJbF5MtPfFt5glZmOagHVQGBXaO4ugKrs6UN0LzDeaZPQXaLB/PtLyqdTADup5Waa+T/HzeofTRE2WhdcfKNRdmDmv4OOoqdBdmax5mXZTyu87bzMjA4LpvaO7k6SWWRzy559fVRifxv7UFAj7vP/k/L8QhS5Zg9erVPMGCWcfy5cvxzOST+M/fe3Wt77/5hn/FnoP7zEofTUxM4LOf/Sy+/e1v4/HHH8fixYtxyimn4NOf/jT22WefnvLasGEDzj//fFx33XV45plnsOeee+J1r3sdzj//fIyOjjrfTk9P46abbsIPf/hD/OIXv8AjjzwCrTWe97zn4dWvfjX+23/7b1i6dGmhjKuuugqrVq0K0vCmN70J3/72t3uiey4hJ+g46KJAo0reMfrtByg1nJ1Fuap1e844qjDKq+6m4BajzOac8167hlUVrazgpyFUJrwMfOXrIXRfWU9xwCsmr6YMe/eN17ali30o5q8bmg8rrQJ/M3Us67/acabJMxWTzQe6CMKFXgm0bxVPOY/9blb5BJvLh92c8/IvjMEKQ5Tj8WBZ3KMaDihBur0862zOAa6BXkWLE8+djtGaY6Mn6MDfHE0ZaJ17GePsxhwpOwlsjAFweZsu6AQWQsryHN8zz8zce6MVgEZJ+ShORP2yOFoN4qeGiw+5bCoWwwrjr+TfOmIWv0DaiWs7bjHHT8+0Q6Ws42Ss9v5v8qlYGArm4z8O9Bd3F1oVrez9THXlO5DfTcoh0A9Vdxhw/WjLq3HPkNHZIf3FocGksYscpI0U2Rixn1WJyirbp6HLv/EdAgJ6AkjHfa/6uVTW+M97aFP2fWCRgvuubvrg+1Idyegff7xmUGTRjkPoThB/0dt1BuDzqqyfMYloGCoampOMj7L2q5SF5L1m7jyrGifBb+ven1b1GbOww951CBQXtAnUlKp9x3CVLA3m4y8AeukL6QjvswvBPegRliaqiyltJXKgzvyvtH38OUddm4+OHW4cBWymAvpoR9ahyiuf1VEBPvS/pXeoc/ZRYZ7u/bOXhW12fkT7MbCgX8YHveQfbJMK+Pk6oYm5fgj0U5lNUVWmSwDzncfrde+krSUv/eI5WcbMdVRXASRCnsmTtRHJ2LeylOgRszkXpI1xDA3NCyvnrXXnhaEmrrvOFJiPlOmKkAwK0uWvyyg4TpJ+//ttyG7MceWUPK81fmvOy5y/mbGnG5pvH2ezzi2s5825rLwynR2sc2BNoGreWMYTLHkBPROUnT3Mt0x+rCOUWa+i47kHee2X4fyfIGlq/l5dxnGmcjyG5k4VNmWBNmYN0LFpao4XwfxAA4hDkxHm29nA5OQkTjrpJNx2223Ya6+9cMYZZ+Cxxx7DlVdeiRtuuAG33XYbDjrooFp5rVu3DitWrMBDDz2Egw46CK997WuxevVqXHLJJfi3f/s33HrrrVi8eLH9/uabb8YrX/lKAMDznvc8vOpVr8L09DRuvfVWXHTRRfjGN76BX/ziFzjssMPY8o466ii86EUvKjx/6Utf2ntDzCFkg04gEAgEAoFAIBAIBAKBQCAQCAQCgWA7RjInnvphfOYzn8Ftt92GFStW4Cc/+QkWLFgAALj44ovxoQ99CG9729vwi1/8olZe55xzDh566CG8/vWvx3e+8x00m+nW1Pvf/3586UtfwrnnnourrrrKfh9FEd74xjfiQx/6EI499lj7fNOmTXjTm96Ef//3f8eqVatwyy23sOW99rWvxfnnn99Xvbcl6h3z+DOEOe5rLkpe8ruc+QfXRhhcG6UXRmeeTq19t9oLYtmLQ1X+SxZ2kSz0LjBVcC4k9Y9zJ6NdJKNdNy8CU7Ye0M4F7eZbe1E9wdDzttj39hLnprb1A5BfSpxdqhrtOenSZ8rS5LfnZPqLVaFc+qwxSS6lz/LUu3UKbRZNK/dS2exb/+Jle9Es6UMn/I1yn9ML1mnbN7cqNLeqnKZm3qa0zrbNG+6Fzn6fmHbVTW0vJKf0+3xgj6eTS6ppnxYu6yb004vcaZtb3htKbAgz+14X29SkU7FK+c7wCWlz+yxJwwOaEIEmT6cu5oL3DCasoU+HQdQlvJHVzV72ruBe9p6hOaGcfrSXHmd1a47n6RuBC94tz5WdbAHcviB9Ymml6cnYp33p85PbGPn3Js9kOLGX9g5siDCwwU1UuNi4RJ7YenaVc/G3nxfgjTPzN8mT1pnSmgwnrIbxx6x/2Tct04yrqKPcMcDIWHqpPZVL9qJk2u6GD6byck2bqgQY2H8MA/uPObKHG6/O2KH8pIo/esG530f0wmali+NFN3K5RGUgBb3k2vxtL3mmJ0YapE0IKH1+nlD5xdNRR7GX2u91xLPY64hnXaKWTgFLp9LQhtl4tBeMe+2QE5L+qF5wQnl4erbQ5hmSXafdZ9nfVn+P533f3Kpy+ujF3KT/7IXYRq5Qj1HSvqZtuLHnP3P42MglUo6R11Q+0zrR/tVNDarL6aXzQVns6RUgtxU43m6N5W1G+8Beek/7lLETqFw0dBi6De1+ezv9wDme+rrEs2namyLbjrqpLe8ProswuM7NMFnUtbYWZ19QXU7b0dc1Rl9F3skHh2dL6kTbLxlM3MveVdoPtB2tDHsuwsBzkXPRvG6T9g3UyeZFxpbTnubPLP3wgZsLcs3h8yS3HU3bJwvcfmqNKacetO90I+dDK9dVLreShV3bPq19t6K171YAcNqpYD+QUw2O7GTsK9Ou1Laj+s/2PSPLrR0awZUnUbG9DS0qSfvO6f/sO2PT6baGOVHW3hyhvTlydYVnKyTDiXM6kOZreZPwhG7qgh2rEmIzMnbKgsejvM9MiFBPV7Y3RWhvivI6EDvIHx+2P0yehCYjt5UGFqyJsGBN5LSfKZPyVGsznRCQHy3Le8/pIvrTUa4LHVlZwge0HZpbc5uCtje1rYyOdPjRjM0Ero4y5Js2J/U0MsBJQ97b+Z1vH2b00b6zdiIdL0YmNoppnXkflTEVY2fhIxEWPhK5dWZ0UXNcWfr1gC7YZ77tYOtqaG7r/D3hCTveBry5vCerdUs7bQkFGwLR1Nn+SN84cywvfzqe6fzdjHfH5qo6LYxim1FdyenaaFqxcpmz2bh86LdWF9F22HOyQJsJj+232cD6CAPrI8e2AnI+5uzVkG3sjDOG54xtbXi8NabQ3hihvTGXb/4aDc17n+V/wj7L/5S3YyfXESced2+enhmjaSYozCnN2HPSURlMdCWdz5i5nG1fOuaRl8f1o7WZNkeFNMlwkttsmZynbd+YUkgWxOnVKVykBkIfHYcGuumOYb2kA72kk9ezqR1+NjaXlcsD2h3vZi2Es7eJ/Dr43Ntw8Lm3OaSatjd62dpUZhxwYPQLldFWvsfK6g/dyOmMiO3f3GcczX3S6w9Mmw+ujZy5h2m7E4+7Fyced29qC3E2lUef0una39DztvDf0bqQNqO8m+w6nc6vyHvbHwTJcMK2uVOUb18TuUrloZX1VO4iz9dZz2PmXQZUhzl6lK6b+uuaXpeH7GfAldtW7i7K13tbW3KZSNc+aNsbvbfwoQgLH0rHoq070yc+DdZ2zfrEX0uw8xZiu/u8bXS0Wev0105omyUL4lwOCbY5tAYSHdX6zcalZp1OB5deeikA4LLLLrObcwBw7rnn4sgjj8TNN9+MO++8szKvp59+Gt/61rfQbrfx5S9/2W7OAcCFF16IpUuX4uqrr8azz+ZrSyeddBK+853vOJtzALBo0SJcccUVAIBbb70Va9asmVE95xuyQScQCAQCgUAgEAgEAoFAIBAIBAKBQLAdI4Gq9ZsN/OpXv8KmTZuwbNkyHH300YX3Z555JgDg+uuvr8zrxz/+MZIkwcqVK7HHHns47wYGBnDaaachjmP86Ec/qkXb3nvvbe+fe+qpp2ql2V4hIS4DsPeUZXdOrT8qfze5lLrzp//r/nEEKuP9qpjN0Wam2TXc+N4eoo1N51sfNkY9jZseEfqGi4kmHtsl/9Z4y3SVUz97f4i5IP6ZQXdbN3HTAwCeGUz/39Doelcsae+Zf1mtWtcmH2dFtHTgzolinVSc08LdEZB+lP6PvU9MAXHbo8n38jXeWEn+PGHyonewmb/d+2+Y8qlHG3OKi423TZ2EO3mZ9j2JbV64E9B8a7/PCTB9TnnPiYGu8/LH93JdjPw7IfxY2Ny9Y5TmpKkLl9JrcucE13TxoGbvflF0DGR/c31Paeb600HC94XNq11spxDdVfdwmTpF45Hln6ldA/Rz4J6bejY1qwUK/GPA3TNm+ofUmY2xzuUHIB4yjFT8xvHMNrImMK6dPiVyKXTXJODeUzK1a/7B1OOZRxCRPewda75nKkOXJamZ01GI9+6fdGDuX+yO5O3E8Z4dL4187JgyqQxTMX+XBqWPo6M7FB4TWgFP/3734ou1A2k+yOPqV95VSA6vsfUM3W3pkRdtaLHvHf2dwbYtPN6lPBmZ/4dlPeDJeL+fPX7xbQXKwzrS+X1wVOdRNmf6196X0tYsHzq8xYwpx1bw6J9ekL9TXeXc9WTJ4+yY7D2VqzkdHu1Er+QvvP8TFHSJSZ9l21mUOHrP8D43BqLNzTx94K5C7j4g7p4N3/Of0kTpDN7XZeT+ZJTTmv3P6Qdi80wtznmb1bFl92UiMLYYuTz+6EL7jLvTT0e57Ri695DWwS9LxYzNqvM+izY3LX9MPzFiP6HtVLAf6P1LjA3rP7cylOqVrP50DJXJsKTl3jdSdtcU7TvnnjDv1CkAdBZSz/wiLSH969hSQDpeTZ26qmD36Ii3GQ3G9k/yPjP9rV0+6yxKCvXg7gK0oPqH0DS5NP927HlGcMGOEyOXp3NnXnRGc/oqbS9PbhTeZ4h0uS4M5W/agT6n7U3bzOhI7h4yHfH9q5m7co0MCKUJ3g+ZuDSrjsptJTpejPxndAodv6H7I7nxsPkgwttUHnnfdoe1e2dkxd2mtq6G5g65PzEw7yrM5el3jFyLhzTPR43ieOPy18qbE2fP6Xi3aWu4V5fd98zpWh1pJJ7ZBJTbbP5cz3xL5ZmFWRsgZYbynlqS8Z6nbw0fc/ZqyDZ2xhnDc9T+A4B4IMuM/M9JS0VZW+PJ1dnCYpQ/M/34i1tekH8bus+ascOsvKA8w8grOp7ofMa2b+C+q7L71ToLkwKNTj5+5J+s7Ggs5a/QXX+GPjWtCuWrrns/sVrfLr73T7BmdQFQuA/VroXQNKZMMj986OK/LJBKZbVjU5XdLc/ZplT3k7p1hzIaY9i2pDzafTJfJDP9y81bAMJfUcCm8ujTyl37K/vWfA/AkWF0bsXZVwbReMS2uVOUzwfePcCWvxmeo/my63lMGfFAPqd3eJqOQZ/WCpqdd0RuW3tsU77IMr1L3o/sfZs6l3mbD85lAHs/cmiOYmxXc+IuLr4DXPvL520nTVLe9mbcy5108wWFuHaIy7rfhfG73/0OAHDMMcew783ze+65Z1byuuKKK2rlBQAbN27Ehg0bAAB77rkn+82dd96Jj3zkI9i8eTP23HNPnHTSSTjhhBNq5b8tIRt0HMxkgAobIrSVJ3z9v+eMJqBnAcjR6uTJ5VdRls2TWUBTZIHOhM3x8yqlqQc6OPiXtVsYWhnD1KfJGEp1L+CldQ6VWReKMdSD+TsJs//Pk4Ks3ac95NdrXklLW8OGpnfyCSw8Ft7XaEe/zk64Dq7MqjxD45H7lNus6uG98y3Td1X96bwvq1+NOtmN8H4uvO4FdeWaX+d+xhaXhjzzQ1IgUeWL5z3wRi30wpP+dz3ojZ5kfQUtM5UxBb3VKx9sI8xo7CqSvl/6++F35lunvcveqx7lTR/vbbgeZnHZ+SyB3QTxN3hmhDJbICo+V7qcT0P6sZd2YPOfRV01W+OI45PQQmxpPhVONfUyycuvnaRiHDh5EzsaKNoRvkNbv+hn7BloBcvPdHPYvqe2K2dv96jLfLmtvLXjMv6KplVhE9AZO73QQu3JWdLFobafsTwMzCNsmio+no35BOVnbuxFvIytK8NmTN8M4dj7dDxwTkXzZVPU1Ht9ZV3BT7NlrxVk/Qx4s5d5ZV36TRg+AD3N9R1dzIz/vvR7qJy688rQHNaDbunCBjWrJzhagFnlkxDqzmsr59IzoM9ZD/P+Nnlybd5TmT2Oh5BtORc2eHC81aS53/Q96RAE7JQSOnrliZDNUbXmMOO53GyipM37Wa8TzB400hCXdb+dKR5//HEAwL777su+N8/rhJiczbyANORmt9vFC1/4Qhx44IHsNzfccANuuOEG++8LLrgAJ5xwAr7zne8UTvHNJ+YtxOX4+Diuu+46vP3tb8dhhx2GwcFBjIyM4KijjsIFF1yAsbGxYNqrrroKxx57LBYsWIDFixfj1FNPDV4GKBAIBAKBQCAQCAQCgUAgEAgEAoFAsCMj0arWDwAefvhhLF++nP3VgdmfGR4eZt+PjKRRTbZsKbnncg7yuvvuu/GZz3wGAPC//tf/Krzfa6+9cP755+Puu+/Gpk2b8Mwzz+CHP/whDj/8cNx88814zWtegzie65MC9TFvG3Tf/OY38brXvQ5XXHEFGo0GTj/9dKxcuRKPPvooPvnJT+Iv/uIvnEsBDc455xysWrUK9957L04++WQce+yxuPHGG/FXf/VXuO6662aHOA17Ea2BIhez6oh4zGXPkqGEvSTVQeZp5lwUXBfUS82QSSIYBS8r1Vl9vAu0AbAnVvz6+Zel0suPVZznb2mi4QRMaEXajpmnCvUUonTTC27NRamFi0kNaczFz9DIL5M1dBoaSL8qnV+QChCa6MWwJk9FaOGQKIcWW9du+qOXrUbT5BJbkn+BtxRy+qeLl9CnBWQ/45mt8zZRmlwYrfLy6QWxlk6d5+VcrkvanLt4O++ItA2QuJcX0/b1+89e/O61s30fkTJVMT1LhhcKil7KbH/24uyYHTOWT0KeWfSi5JZOQwhktOlGzkdOuAeSp18mpa3slIn51vBh0oINRUMvjqbtZ/ioDM0J5YxHm56OUdL3ls/NmDLeYNm4K/ARVyfvknRu7Fg6Gi5PK+3yYXMrdWvL60vlnZXLhGaTZ9RR9n3SBJJm9j7j3airoLJfiAfZsWnGIzMGqAxMBjWSQe2OByaUh4oBtccU1B5TfJ/6Y8TvB+89d9k5d7k2TWfeaU5G0nYmbWLkwsCGyKUhyy+XS4RRsmeDa0kaImNouzt97o+txH1G+5eeNDP1dPqKgX2f0Ufbp7Ulp5XVxRk/RoSPlAb0gIYe8D/O3/vgeAMABtdFGFwXpeMw4y2uPymfcWVSXeK8Jmk4OcbpRysLG16Zpv2y/oy6KtVxPu84maW/BiMnWmOq1PbSrdwOUTGRd4xeSZqkLlQXMGM/pCu5NkOS/eh3ph00+dvoEpWN+Titn8mrNabSf/t8QHnSQ9RRBd62p4A8eWpkXWOKjDP6nthZNi/ONopyHZTbi3ye7Bgm9o8Zt7qpHb1mvyV153SEJamrWN4ybaOb2rGPGh2V8huhz9SV2v1liDoKzbEIzbHIaSPOnjrqxQ/bv6n+tXQASAY0kgF+PKkklzNWrgTEma1nBDQmFRqTGU95fBR18/FCx6huaOiGRryoW2xTndvgvk0IuPqjOaGcPufGkLUv2jr9qTQdTas04ZcFuW0Xc/3j6QgWRi+ZesCVp7bcTCeH5L1Bw7dfyDjXvjwwNNI86RxI5eOUtq1j25v8yTOHJiIP7TM6Dpk5im8HOHMo76Se0qltxtn21t4j81M6nuPRLuLRrkMHQMZpJgOiaWKjMbqy0KYefZQmRzaRuUIhDUFrS1SwSeIRmij/k/YD109Ultk6xczY8E+SBWxS+4mx2Qyf0Tow6ei8hdZ5cG2EwbURO3eh5Th0VcC0gx8az/azeR+5aZy5h2/vEn3A9ZnefcptN9L2jixppbLNyresfo4eNeUQm8JZU8h0BdWl3BiLphWSoSTVJYYmMld1bG8qLzI0OsqRU8WGLj5SOtcrrp2S8jOV0U47GTvWT4eMLmPTcfO5KJ13J8MxmhPU6ErTtDdGeT0Mr3tzDUp/oc8a3tgyfEJkgLO2BY8/GF7wZYe1s722pLI9pFP89QljE1l51sn1LzeXSgaTwthQMZmfMjQ5z+j8jaPTk2U54enPrHdw/ZFWCu64zPjBn2dXtZMjDxNl9b7JszmhyuUN0XusvTet8rUxY1sQmqMuoTVAJzcOnSr4fERoaG/K+dzYtj7PNccVmuMq75NAWVQu+W0YmotSGm3fMOsbVP+ntrEuzOWsLVAxdxbMPbblHXTbI/70pz/h9a9/PSYnJ3HOOefgVa96VeGbV77ylfjkJz+JF73oRVi4cCH22GMPnHbaafj1r3+NQw89FL/5zW/w3e9+dx6o5zFvIS5brRbe+c534pxzzsHzn/98+/zpp5/Gq1/9atx9990455xz8M1vftO+++lPf4pLLrkES5Yswa233opDDjkEAHDrrbfixBNPxKpVq3DiiSdidHR0xvQVwtZFuiCQlc6/iyaj8AaOzTT7HxP2sRLsgkNefugYeP68XmGFfLwj1rqt+bugSDF+iCifJjuhZJScc7cKXXBj6pcM5O3oHD/3FuroxMQYngD4kHJm8gySp4ZrAHm0pIZ4/q19zoTuC8Vmd46/mzK5O1C8dEBuuBtaDMwdEHYjFQCK19I4bcfe+6MCfW2QAMju/3DyYu6rM6D3F4W+oW3RM5K8fMpv5u/GWKN0PAeNryjnOWexw6eT62YuT0auhNLoRh7C01lUbuc0+TCLdgB/3wu9lw9w28qWS/qejrlCmcQQLb2fr+GWWRbaMeoyZRLanLsjAnkW8tf5ogANO2bHqcrTOH0TGoYl/Rdnm2Cqq3K52UQeLrnj8RBHb0YTnhnI/y4Qwf+bDWWS8HcpsH2WFN9zYc0At672Pbkz0Q/PRmP9OzHus2eTSxOHzwp3zHgLTj5NyYAbhseUrzge1eQ+t0A/h+48AdJ7BUpDZ2ZyI6G6SAFqyiWayiCgqEvZNgAwuVtxYTB0fx/Lr9x4Zspx6CN1CW0cAumE2CnTkErbhCnLeZbVhbtnojuiy8NBEh5IvE0SmjeQTUizPgneYevRlC4kFPMqbWeA5SMu/+mF+dix9pHH+3YcmcVBet9aM28fu9nQIYvalN+y9okb9BmK9ifte2ZizoV6o3fIOnYY/Yaxf9xCi99aPZ+ooh1FQHmH8jG9x5HqP/M9DTPk32PVoHesMEjaOg+ZTu8QYcbm7+5cxuZB71D1yzfPgUxW6Oxb0ych1Ur60+RvNoVp/kkr7zPn/siMjsZ4i7Vp7f0/DZ3LBtPORH849kfAZsoXjwnNA7rw3sr3sZx5uXvCKscoKYvqJe5blenkgoOTN8eL26QdIl3kTyJD0oyzT9m7q8ndMlSvM3e7Vd33FrLR8/T+RBjOArh/DxRNmzT5Mp35DBnv5tvm+laWXru2J+Ef+++qOUJgcdVP49iLzPycq8f0LkkhL+e+xsB8wJFrvr2dkL72ZXz2ns0/gAJfVug0Nc1vdnL3X7H3R5J8Q7rIXkMQ599x80buLj4VF20iWma6SUT+9pHdieyQy3wXdRQw1czLt3qFyEDCh3ogLTSJ8wbO750v0knLjQc0GtldWFQnOff6lsynYmLnsvO3Bgq6lOo357nVpcVxohuAAjMvYtLHzDoDknTeDXh3eGZ2YGfUtdeTrE3tXEW5uthHY1J5d3rm5eYPPXoVHKcIX+4paIePHTub5mGJcMuk7xLujnQyp6d3o1Gd6qwx+nfgRq7ty/W/zYd8x9okoXm4pwupAzJr09BsIo2E44USOLZjQ1tbwvR94f5X02emTQkNIf2jqM0Ed40syRywQunroLC2RCZ+8aC2m7XselOS17GwxuSXY5w+/HvRa9CvFdL1XC+N/SedF3m2ceEbVb6OJZhbaCh0aw40DYVly5Zh9erVfZe3YEF60fP4+Dj7fuvWrQCAXXapuOtylvLasmULTj31VDz22GN4wxvegIsuuqiyXJ+G97///Xjf+96Hf//3f8db3vKWntLPFcqW3ecUZ511Fr7yla84m3NAegTxsssuAwB8//vfR6fTse8uvvhiAMB5551nN+cAYMWKFXj3u9+NjRs34mtf+9o2oF4gEAgEAoFAIBAIBAKBQCAQCAQCgWAbQPcQ4rKGY08V9t9/fwDAE088wb43zw844IA5z2tychKnn3467rrrLrziFa/A1VdfjSjqfWvL7Ck9/fTTPaedK8zbBl0ZjjrqKADA1NQU1q9fDwCYmJjAz3/+cwDAmWeeWUhjnl1//fUzJ0CheEqKtJQJCePDhoLRKHih+Z6OKua/CXnvc2ExqtJSWrjQhiaMCM1TaUAPx9DDrssVPYJtwqM5oTdoPSMNRDpYFxOmh/NyZo+9G+eTkvZx4IdGoCH1AMS7dhHv2nVCXTnhHLKyaPn0qLvpv2hKIcqOyZeGGVDIw6bQNuP6kYSPccJ2cM1i+pbkZcM8UJqd+lU0Ihcqq8Izplaf+GlKTlwAKLQ/1w5++AET8pHNS+ftorN2Kg3Fwj1XeXqufIpQyMYCaQE6QnnXCaXh5O/RXMjPz8Pn/RAozdnfTp3L2tYvsiR8rG6gPMQk/TcNdUbGUWNCoTGRp1eZRyr1zDVlWY805Y6roLytGhuJck+HwesL094V/eSHM+ZAw+v44WlcolDaz1woGZuOepz6aKBwQtfpO6sgc5ramyO0N7uhkyr7maGTTeeHssnec/pHxW5IJfb0WZnoVN7fRkZ4+tfXwTZ8SyjPhk5/5vuqEJWAw1OFLBO3HmxIU/MnRxYXAoeUz6UJeX87Y6wPWHuEhLDk3vvPbFsZm4KYOo5HMJFLpbYZ4e3gd349Q38zaZJdum6flrTX0LMKQ8968obw3Mg+WzCyTxrL34agpH3O6BXHpjW6tGLmYE5jqRg2cfo3oT8UUonUPSR3HTkZkV8gnfmOD/WeP+NkcDyU5KESPZ1Q6TVMaCqVHYYfGSTtBEk7KdBqk9O+MRjtpL8aM7x4QYw4CwtZqL9nR5bNi9g6AYji9MfRzIY6LZy8zH4LutALuvBhQtE5PEDp98dkj2ECDb2VqLDjbDskKh/PFfYWaxNUyIugfTCd/pwknH4nfxuaaThqh2ZTJtOPfrhEjl425H5VuESSv5UxDHR2CilkJ4fmIFwaJwSZyZ+Rm9Q253RNT6B2GDO/pHQFUWIThWwWVi5X8JbDp9m3nC7i7PXCSWuTl9HPnuxRi6fSX931AFp8yLb1kLR0bieSb2k9Le8q4Iy/uAtn/MVdQX5myyS8UVg7UmDnDiwq2kAtnio+C5mcJkQyU2wyEjt0siDhlllSSZvZZ8QOdMZeO0l/hraK+W4wdCODkA1snhu5p30+rtIVJeM8HkkcOWLmBdw6lbPOQGgtzEsBJIu6SBZlepGGRCzREyZcYU+ga0OhdSKmzKqrA/y5qqZ2HPL+ca7toGUxbc5d4WBBZUAmS2g5HG/3s8ZFQfWusbn9tWKnn70QnVXlF+Y5np3I1ilWvDzy8i3wXuC9YP6gUT/E5Wx0l9mjueuuu9j35vmRRx45p3l1u1286U1vwi9+8Qscd9xx+P73v492u11dAQYbNmwAkN95tz1gu9yge+SRRwCkYTAXL14MALj//vsxNTWFpUuXYt999y2kOeaYYwAA99xzz7YjVCAQCAQCgUAgEAgEAoFAIBAIBAKBYI5R+wTdLOD444/HokWL8PDDD+O3v/1t4f0111wDADjttNMq8zrllFMQRRF++ctf4tlnn3XeTU1N4frrr0ej0cCpp57qvNNaY9WqVfjhD3+IF73oRfjXf/3XGW2uXXvttQDyvaTtAdvlBt0ll1wCIO24gYE0Zvjjjz8OAOzmHJDueo6OjmLDhg3YsmXLzAhgPDJaW3LGpqeU8mfEy53xNPBjgvuXHwO5hwjr+U09BY1nGfFKCZ2+MM8570l0vHj5mWe0moigTCx9xqPHePU7p7T6APWMox5Fhe9icnE047VYQJL+rJdKArdPOxHQifL01DNOg/UOck5VGO/SbnYP4ADv5kLzpHeR5B/kNPmedw6/BGSq0ye+h71yeca2BeOB7aTP2s4vpwzBfA2fMi4byXA507AnMXzvHa/MiB4IZbrE4f8EhZMyhZNNBQLCtPre4FUnDJw8a3rXBj2VStKnns86vzsvkK/1kmI8WllvQ1omkR0Fmhje8etBL8T2y0y9Bd2OdNKHPHqj/Je0stOVhs6AFxjXj473LvEwY0/f1oRzYsekDcgdtm4heUAucbbyhOmH0ClvLn/KD9bDMOCxrqZV8f7P7BRSiA/jtkbc1jzvwG2fqnYu6AP//p0sH3ry2aaNc3leBcfjk9Bs24fQ4XuDFyoQq/THIdPx/v2nZSct/fRF4r30FR7BxdMn9RmdHU+sJ2dFPiFHWKvf+ZNrTj2Ntz5js9CT18lAkp86JifoSk8ha7C87dDky+g6zWh1snJP/Pp5Ebo6izQ6izwZQN6PrR/G2PrhcJGcl2ycP8+9mOsoLGR2iLInVB3aS/SSM5Y4e5j+u6vSH20bLluSp3tKzDAqeBnc0ukPRd6pPAlJdGXVmA2e9NEKKjt5rFvaOZkEwPUsN88mmtAT4SvGqzyn7XcxEC9IEC9I2HZ1vPADXu1JA4V7aNjxxPWbIhE7tjShtjQzuojtNx2lv8x25U6AUp1XuLuqzP4i8x/ncRkf0HyJ3WDawTllVTV/4sYIyZeL6OCfgLQnLbI5iH8KjD0Rk9FvaNYRyIlYUl8z12L0d6UNrOHON7i5ZgmijmJ5lzuRQp/b4gOn0bk2CdJkaDb2ph8RI8DTVSc2fVDbrhcU7AfKa6G8Mp6rWsdwC0KhTkYXBWH6vq4tY+TJ1haSra36dlAoS2rbFuYAymk7/+SStdmz308ePRw/efRwd97pzRlCdXJODxF7UUe6eLdUKL+ScpKtodAyXD4pQSyfeXRySbn1A5cY5DrRT+/NydVUA2oqVxz05BEhNUek3fFWItfp2hDXz1RnVfJxCZzoGEyjRt30zrP0Ps2cfu5Uf7BPsjlEyA7l0pRGfKqDGM78ytLCpGf5mL7350heXoVTqf6Q4HRotkZZVScjS1I6c97zZUsvcoZd7yJDUDc0v8bI8HbUSX89n1wjUcxUQBdXrduak7S6RLY4fSeYV2zLDbp2u433ve99AID3vve99p44IL2K7J577sEJJ5yAF7/4xfb5pZdeisMPPxz//b//dyevvfbaC295y1vQ6XTwnve8B91uvgjz0Y9+FGvXrsVb3/pW7L777k66c845B1dffTUOP/xw/OQnP8Ho6Ggl3Z/97Gexbt0659n09DQ+9alP4Xvf+x6GhoawatWq2u0w1wjP4OYJP/rRj/C1r30NrVYLn/70p+3zsbExAMDwcHhxYWRkBBs3bsSWLVsqLydcvnw5+/zhhx8GSEcbfp5ekEsgTtlE03xYABZZcrsAnaHsAlw2m9Alzdy3jICOOnloMasYzEKHRyvNJ9pKjCbGgGcvsqfv6QWpXvkOVF6mXQigk6bA5bhOeBSgcIm9ucibXnbugAv759UByC64B6CmVOlkVOm8ztaI9L8hG2l+uT6flMHZkOPq14cipeFYQ3QULgxXpH8YBa6myitUZ1z4/eqHLLTPA/1ctTBchUJbBMovzyTwuCQsXE/Z1whFVpkHNVTJZk0/9PRaZmNK5bxVwYPsxFCjsKAJuPTnsi97RhY7NSnXubSelFlaH2Zzoif665ZD81VMXowc554b+DKUlp20dWEjztBYJKqYnpZpQgiVTaRDKOger05cSCiajsrVpOSy7CAt/ndl/RugiVtMduCH2vLbnS4Ees8Ki7VM/mV1Culv7ltOf9F2LsszaZXLYuO05Ke3z6ochMh4cHSpUcnTOe/QML0NYycwk2P6PMSjPdmDXD8aOjY3WBo0k4baqaZNHSeVzcTkn8Gk2t805vLj6GPfB94Zva38jXZTZpav2VSvswHm2wyAZx8E2t/SRMYW4OkCzq6j/RShXEY2wOdBbCVOB3G62Ni4ddo+GmsUntHvGluZcWDsgC6jS0jfpBvFOvie7TNvE6GxJaOP2JhWXmsgmnSNHN92LIzTXhaUuO8UinaVtw5bKJO0gzY0hPIHo8tqzBXL9I6hO/SdVrBOUtSW9x1Py2guK7uWLDRdSucSXFkVdlSoTThdx9kCwe+9Z2Y9ncIZDyHMQO7WodF579nO4Q+z/6vApwH7wtLAzDGpLrJJ69qNCLdjNN6/X3loDBX0a4yczxgZr5Uri6f+uKDw3m4otjSqeNZHaFGdjlHFlMXlz7VXcOyUNK3REyFw88ReUOgDf40gZF/Qf3P2YZmdEXA2onLP4eO648jQQOzJaCux47w5pKXFZMOMW2d9g/BmtIXR3xV0RmSNTnl5BjeU6VyWm1sx+sGEHe0XdiMcNXiW2rlbwzYNfd7L2CjQFfiWfebrUU9/F/RP9nl3hJffvi6u0gsqzttJEblU5uysWzrv8wp+Ctmugm0Djfqbb7pX76AAzjvvPPz0pz/FLbfcgkMOOQQrV67EmjVrcPvtt2Pp0qW44oornO/XrVuH+++/n73j7Qtf+AJuu+02XHvttTj88MPxkpe8BKtXr8a9996LQw45BBdffLHz/b/8y7/gi1/8IgBgv/32w0c+8hGWxr//+7/H4Ycfbv/9P/7H/8CnPvUpvOQlL8F+++2HzZs347e//S2eeuopDA4O4uqrr8Y+++wz06aZNWxXG3R/+MMf8Na3vhVaa1x44YU2NqlAIBAIBAKBQCAQCAQCgUAgEAgEAsGfK2brdFxdDA4O4qabbsJnP/tZfPOb38R1112HxYsX4+yzz8anP/3pYLRDDrvtthvuuOMOnH/++bjuuuvwgx/8AHvssQfe//7341Of+lThdJy5Lw4AbrzxxmC+Z599trNB94lPfAK33nor7r//ftx1113QWmPffffFu971Lnzwgx/EYYcdVr8BtgGU1nq72Pd+8skncfzxx2PNmjU499xzcdFFFznvf/jDH+KMM87A0UcfHbxMcNddd8XGjRuxefPmyhN0ISxfvhwPrl+Pff6e35EVCAQCgUAgEAgEAoFAIBAIBAKB4M8RT/7PC3HIkiVYvXr1fJPyZ4Ply5fj8fFn8P/7+ltrff//nXU19h/eU/poB8B2cQfdc889h1e84hVYs2YNVq1ahc997nOFb/bff38AwBNPPMHmsXXrVmzcuBG77rpr35tzZaBHgendGPnDHmI4e0fx+0UUCOFVP4M+01XRP0v1s9npHto2kN59gFmlb1bzmk1w9eyn7jX4pOf+2S4kz86P0B0+2xPmmr7gvYuFD5GPjZr3wc0qZlsubQ+oqNOc9n2g3Mo7rbYn+O23jXlye5cdfy6o2w9z3l8RepeNCj3Ltp7q0EPes9U+vfTHfKBwb8tMMENd2HOb17BZa9+htx3Jrn54b1vJ31AZszrv6iHdjta3PaEfO69P23A+9Tdbdo86gKV9O7STS+WtCZnfI82V91PWzmgW8sAs6pO6mI85WAbKe3M5hma6tpVmgm0mT2Yt/Vyj7rxzDupR1Ze99HXUVTNf550FOgTbNzTq30En3b7jYN6XycfGxvCqV70Kv//97/H6178eX/3qV6FUUSAddthhGBgYwNq1a/Hkk08W3ptTdUceeeSc0ywQCAQCgUAgEAgEAoFAIBAIBAKBQLBNoOtv0MkO3Y6Ded2gm5qawhlnnIE77rgDr3zlK/Gtb30LjQbvpjM0NISTTjoJAPC9732v8P6aa64BAJx22mlzQit36a1z0fq0Yi8b5sCewKtKw3jZxAN1XHi9H4GO+PSO95Dx4iJ5UM+uSu8f3wuqwpuaemmZvLVK29/vA93S6UWmPv2kfQuXr5oLlannknexMOe5VunNVuHpWfDYqeAT/3RmXbDlzPR0Z4UXn+2jEs/nQvmz4RlYE1zdOa+kGXs3MxdUV6JHCWx5ifJjiaehbgT6hryn6av4JDTmTNo6zwrvzS/0vkxeBGihaVias/fG89Vecmw+o3/PkpcdJ2/o+LRlBspqdBQanTAhjqxm3lE6qjzzqMwq7cOKU4eGJj2g2bwMbzon1LPvkqHEedZzPzCn5EKyyl4w7+k355sSL+kgHxr0MMbcjOHUQ3X59MlQ4rRXCNzJQb/tKa11bRqHj8vKpLrItHmVh3nFCUJfhpXSpALf0k/qnrilaWbpdEzo9IjpByvLQuQxMoS2de2TVb7NZpBkP2a8+vXwEbKfuDprVXxfsKMM6spq1QM/A9V6tVHd5zoCdFNDN3Xlt82t1YSV8gydj3QUojJdEeBX/7luACpOf6X2XLCgYpv7/FiAb6tNK0TTyikzaWkkRuZ69nylx3mdOZrJr0IWBHVyxju0b3riPTIOdM02rytPuDyMDq6jnx25xI2Dsr710lfqdDK2fdmlq07zRuVtFpyP1bQzuLmqb2ex5dY9GcXR0KMdmjTTHy3b/t2DjC6zoYMosetN+WX9w42XOu1WlieVsf2MJ/Z9BCDS6Y/akUZ2ENuezZPjU+WOLfp3Ib1y27KnuWcdm8m8ojqA6/co521u7NN5B/deD2hX3hpbg7YprZ9Hf2GOUGcc036itFC5R2VQyfy3HzhrWuRvzk7j5kg9gbYHNzYD7cXyFpc+xE8V+VeS7dskTX4u6aQpkbGhNRHWtuwDVTo+aenqOYbVoRq6UaOjQ3Ups2HJuKqaE6hYue3jkz7DNhPMDBoK3SSq9dPSWTsM5m2DLo5jvOUtb8HPf/5zrFy5Et///vfRbrdL05x77rkAgM985jN48MEH7fNbb70VX/nKVzA6Ooq3v/3tc0q3QCAQCAQCgUAgEAgEAoFAIBAIBALBtoTWqtZPsOOgWf3J3ODSSy/FD37wAwDAbrvthve85z3sd5/73Oew2267AQBOPvlkfOADH8All1yCF73oRXj5y1+OTqeDG2+8EVprXHnllRgdHZ0bgqkHh++1AyBp1ndn6WeMcGlUXCdh9q3xAKJettmpPzdT1zGmcHJQu3mwdaEZ+F4snhdQwYuF1Ml5x3l0TfMNqX2u9k/DGc8V2yhemabOvldoCDW8qEs9l7g0vreTISMheYbSZWnstxXtCLBNURtaBejy27UHcLwx0zT0PTdefS/vqvIL7/upb48nCW2+NH9ymrWMJg4qdmVDFd2hMefQVvHMf186tBRKZViQHpJGl3naxfTvNC/d0J4QLCEwQBcH48nsyG0ubaC8uF1OiBl7KiFe97RO5jsFoMIrr8Bn1Ds8JP+5U2eGjilG1yD33NPI28V8F03kwoTygYX3rKof6EkEU6ijy5n6OcWV6NuycQGgfIz2gJDcp21laUqK37MnB33M4HRzsI9NnrT+ZowydFJQfnZoy9JE3XI7pHJsUT5S4D1WGV3sFDVLNp0j2xj9bfgsxEfWE9bXRdm/o25Nemk7M2OM9iNLR0TS1RyXobIqbcw6z0veBctJiu9tnyCQhilTZSe6qr7tjlRbX5qxTblnCETHsGmIfLdyN+LtcbbMmnYW1+YOP9eYwxhbzTlBQE7J+TpOE9uOnTcl1X1h3zOygNIR1Mn+2A3Me+raDv68jEOpPCCnXgsRRTL6ymgIjlGm/9h5h5NZ4G+uLJqvX1bVXKqin1mbIkBTkB4jA81DegqmXjZhzIA2A7/NqJ4Nym0uH86+qZr3MrqIlSGhfgigir/K+pzKWFZu2kK8x8w4pmPLnNp12tfIjipZzfGpR5NvGwPIeY3a+5zpEhoHnJ3HtYWZV8Tl3UTHAycXtMplMys3phSvt0J6x/u0MJ7r8JT2ZDCXJ+mzSju/BM7JQmr7Mrat0z4VtmftsVOpQAKPTZ9WzI9A3/ciuyrqZ07KG52vatzJVmqf0Dk/kSVlNstsIpoO8LklirS1+Yza0xwC9kVPuqws+4au5kPBvCKRk3E7HeZtg27Dhg32b7NRx+H888+3G3QA8IUvfAEvetGLcOmll+LGG29Eu93GySefjI9//OM47rjj5pRmDjvKhnTtxfOZGgE9YM7armyBUaO4AOcv9DITgBnRWjWRLEvnP+rhzGsv9M+0L3qhq1Z+s7TgOdt5bc/ol+b5rmvd8vvux5qTBbuw50/aZomuWg4VMwBHcyg8Ru225BYXAu/7wYzahJtM10nmbz72mH4m2JZjrY5M9nmmb/oqFkpLk1Ys0AVpqloIDmZY8u9taP9UofaCOUHl4mXZouQMaXI/7C//vsqaYzjydL5v7a6ry3rop8o69diXtcrugz+CPF0nBFSNfOYjfa1NLO7fdVHlbDFLcq3vudKOvuhnFtqLezHhJP3wzw7YPpUbtQY91G1W5W8vzhzct2RssZuRGfpZMHeKrtoQqfh2xvPfmv04G/q6pzxmaUwkzfLBW5uPK6CZTZZedOWsrUf1idpjr49N0jJwG3KzVf95acca8rAwR+1FRpZsYnNl1c94lvIRzDo00jvo6n4r2DEwbxt0559/Ps4///y+0p599tk4++yzZ5UegUAgEAgEAoFAIBAIBAKBQCAQCASC7RESvnLnw3z7g26fUCh61wcuvfX/Xefi1rqXxvvll17kydDcz6Wy/sXo9rJVkz+5CJi9hFS5z7hyG1MKjSnl5mvAcGTostNQm5TWVQG6raHb5KJZ/9tEpT+ShuZZSK/Atr9TrGnHiovLe7pI17QJoc9cwhy8zDV0aWxJuaEL4v1v/O9K61kleQJllCahHoW0+xg6nEtvmfxrjc2sLdjLhSv60Cmba9PAeLY/eslwFe9VjP1K2VLxLTsOTNtUeHArnY8n9j2REaEL0qvoLW0fki93sTSt10wv5nYubmbel2bTS7mMPKL1UjrXQaH0pZfO99AW9FL1QvsqIOooRJ1wA/TV5ga9WjdEx4X0UKXcrurLwJhmPw21fxUIv3J5UjihXXyaSvQBOw4V87dfJieX67ZdgZCKdKH3ZRenN8rlAZs/anxbkZ59bewsepl8RRojn2k9IiccIOypVZa/AnZWaZ+yxJO8sl9Q3pTl0Qtvqop+qBp7nm1WNvaiaZWGKgoVRXiIPiv7tiqvyjRRxftA3o697refCc9exza0D6vL7UXvGR0R0s9Gx3D83Lfu4Oglf4doCdl0bF+VtW1Nm4cdw74dGpDLhSLryjAqI0pkaei5Q6+Zf4XoKsm/MMfx61dDf7G2bx82mdGjoblqXwj1I0VVWX465noO87dvm87I9gLftuw8o4pPfH4n6w+9yLle6hN1lZXxlfNOIqPyh6TMuAYzVaDUHme+c571Ypdk/OTrLWtLlOg93YAr1wrEwZ23+jT3OsfyeL8w9mqM44IMYWiuOz9m7W0mJLtDmz9frtnPtMx+xmmZnVMrr15kXK/2g08TZwdSPmJoqbIDrQwh31EZWNmmPegIX67qpq7WtWYc1h3vJfMZMxcoK4fP3C1HsB1Dpyfo6vzkCN2Og3k7QScQCAQCgUAgEAgEAoFAIBAIBAKBQCCogurhBN3MHUYE2wayQcfBePURtLYodIfTh8lg6voQTUauJx71CiyBiakcjySIxnP3BXPSIJpK39OYz43sWRLyQmYeV43X1pYIndGsLtPps3g4QWOCOz6Q/W9a2fqracVfXmySaCDJOMzeNaSAeDCr5zTj5t/N621fJQrNyez1UP69oVlTjy2dpzd3U0RTkXNhc2tj+oHOaEta2rkI15746JL8STtEkyqnH8D0rrHTjxwa+42n9D89zMeJt5cncw2J/K6D7L1z74bO2z8ZyOrchcMTlk9DHkumH5lq6AbJi3hduXHqs39kdDWmVN7PHVXg22Q4RjQWdktTce61ZmkiF92n37htoWIFHeX9bNo0mowsbZbmSDuXfBeQKFsXnsDcA4q77Jxe/mzbnsSgp23Htj03nqkXVisbg12V0xG4SLj0wmQNgLuwm9BJ/7Zjjty54Iwd8nehTiRveueVKjlBFS/uItrYtGWlGQTo4+5XUnk/c7HsFeGpeDglujERWd5KmjqX17ukDeWMdeXKNkMTBzM2ufoWZCWRZ/SbUP6KyL3muML0AiNji99qBTSzOpgLuJ08FXk+kOmH8YZth+a4QndBYr81dNI+t7LbeNtS3iRldRflzBd5ekcluYxGArZNKFpjmTxeoNnvrMeokRuKyA6N/K4NTiyp/L2lI4IrO8jYB4p6yablZBin10mfhvrenI4y9aD3uamusrrMZtlA6b1EKgGSlsmbIcn36jd0Gf01nXvoGZrjocTKYB0V8x18NkJn18y2YtpBN+Cc+iq0wXRRv7gZ5H2qkoxG5G2WjOYyRg/HiDPC1WSaSHWU249UHyHrQ//Sd04/MHTZz73kWlGai3WLunydjf5LdWFGZuax6xJT34507ok0Mo7wPivrqQ1tdEJXlcow59s4lTMAML1LcTw3plQuo8jpr7L66Qj5KYdIF3QVEp6uZNjIwMj2vdEV0VTezs54IDYBPcFITxOb7zj7waHbtDntb5rnQD72rIwzPO6dTPfbP2nmD6OY8Ln5PgaS3Trp+7XtNC05YREP52M7J4hmUKyT8uZXBTuJ0KAjQO0/kb5/Yih7lvOzbgDNLWn5xt6kcpnrz9aWCNO7JE6Z/rd0XgeAlfPB9AE7zKZJclvEjFffTlKeXHfaUfNtpXz9BLdtOfuIprH2zdYGnz+jVqlssGUZnUa+9G0+P0+aKbWDqG3sy1M1rXLe10TcmOgv2tUn7AkXarv57+kzRqzoiNBn5ipkPCtqHxCdUSkDfRpAZJxvq5fwGZDLUDNvVmR+7dv2QNam1JbgSGLmoJzNY3URiUbDnsTwozt4dXagc8LonN6hj5MjjIxNRsL8Hg9oq9O5CA/+eDI2jV1TIDakY4MH+IDVob5+QrpmBGS6iJl30Xl4PrYCioWDP5cCGY8JADOeOLkew5HLbN6WJ1w+tJ9QGeXZWb5N5a89aWqjk7zK7rlOyAoonQtSmu06UqzYu8HKxnPSojJMQ2UfmfW81iaFqd0SS2+BZjpGmXWYpAk0svUoNDVLHwdKa9LObUYA0Fmkp/QlCrajbmg2X1OneEjntiGdwzA6WSuGVk/uJmYOOhVldMKxt834pLLezLXNWqajB8i8Uk3nhVPeK7NJHFoZODLO/E34Ml33zOtX+Fbla8CGD+Ihvs25uy+pDDVyKRnUUGZdOUsTdfL1Mq6f2s9F6OxKeNPM8YztyenLOjpNMOvQkDvodkbIBl1NdEdyAWmEvjO57jW8EYDG1sgR/EbRcIapWYzod3RxY3d6YZIblmYPYzxyNwI8I0k3NVFqvOLijCj63oBdVOOM6oZGd7iYPrihYebp3kKcQXfEK5fmSYwgbhOCLprGxgjw+pFD8thI+keL70Bu88C+Iwa4phcaMzwX2jAtbJz478v2F2tcRq29i5bjgdxI4/q5saVR2mZ0sylkYPv9mrRdI8b0Pzd2fGPePgvkXSyc4X06OeD4OGQs1wwVoWJiRHWZMdiHbAj1QehCanYTrOy7OmWW0N94rplP5KhcouOxJH3ZuLL5mDnt1nTA64Yb/tHwQmNrLvfzlxX9R/lxkpHvdJJONyE4OqvqkSWaXqBLN9wBspjJbk5ni7UAMJVtXDS0bYfOrglrpLPykuTJ9VPDbHYy7Ug3AQBmgc4rx1/IdxYuSDodkgdlfJQwbRlYtC3daPfau0yvs/zuLYyYyTXXjwmnayrsFGfRsUo2UN42tgvT3s2xhtUBKi7mO7FnUrqIwi0C0/JDm3N0scgZW55spzKmsSnfiE7o5gadAPsLQ5yu4MZACdiFFbpY5i/MtPgJO+uEQNuP0MrSV6VDPDrSPMvTsptRFfnrCHZxgLM3je3lpFEodapJnX6K70My0i5yUIeMrC5GPlY51QAAt3neT9tzocW0AqKJnBbLk4GQ0cXNbUBHxbHr5E825uzz7O/C5lwhg2oaSp9rQD+ZbcwRWUntkHgofw4gKOPMwllnNKkv4wJ5lel8Tsb5tpH2N/T9/H0+rWHbseOQ8htnx1Mn0C2NnL4ALwAI2rk5HWTRT4W/Cz1n5T0jT5MBDW482fFWo6z8ZfFvx77g2t/jQ5NGce3DbBzUlrUo2WSoyMPOdQMOc+w8rmrq4831CvmatgjootJ5H7dgTR9F5EVI3tbs58ZmYtv6crELGGUcHAuE96jjMOD2V0z4NDhHDG1Get81xnJHJ5b3Sd/Q9ZPaYD61a0RUf1fNZav6pob+K8hoTwb4/RLcNC3hhzK7037TZfjAnzvUyJ+Oh27m4NMdyp3cnfpz8xaFghOGivM+rxo7Ifgb0P7mZ77ZmvMWt45kDjAU5HcPjmrm37TP6XorUBwrnP2vKtZ+cscK3sZ36O8RTpoyPRooE5rQ3S75Dqhcm7M27ETuSGbHRUO7fePJk6ndkjx/zR8c8emW3Z/5Q+jwhWDHhWzQCQQCgUAgEAgEAoFAIBAIBAKBQCAQbMdI5PjiTgfZoONgvOkdL+E81JkJI6BomBzGm64Kfki6Um+NCs+lftKlR9WzuhjviSYKIaCc93C9eLgQMIkNEakK5XOhnGrVgfNqLPsepMq0L4kXkg1jR7zRVUKeU88cWg/Pk0dNk/YJeJeYI/fUyyQURtAHfeacTmTSFELzeGU6fUPB8ElZmEK/T3zvG+fECWmz3JNQ8Z59vXiWe3n6oc7802aahP3g2q8Obzm8XzbeVeC5/1lg/LNhWsjpzsLpwBplsN7KdeVPHfg8USNPGoaI87ilp19YPqSh1rgyFZ+e/5Yf72UhNCvbj3p/+mMSyEM0dVTwdGutckDkGgkHGcpLl52kop6yTEyixmTEey36Ybm8PLmQsKw3IUezL09QHMN+/klLuzxlxnmITr9c3+uyjKfpiV+mH/sJlcLBPxVY4O2q/Dy5VKCrTy/coF5B8WSzHxYsmkYpv1bJUjaUNkibVISTonI1GdRQnTxfjqYyOVBH3lk6uDSBehb0ekAGOafuyPv8tF/2LOQ5ztmLVe0fkltmHJbI9wJIn/nhiRz9TDyznXasaksiA2hobCCTB0z9TYjKkG1RetpeEbFWYV+wp3QJzdT2ZqNcUFAe5cQx4ScnNDEjo/M5grbvqE7sSd55dWJDgfse6CQsmn1P9XOFh76fb9Sdma3DhekreK2XzLsKfJx9x4UJDNWZDRtWR/absnx5QsJ+VZ1AZE8IcGVTuz+QHy2/7tzQfhcr9rSe823ZHKYCoZDxLE0V8wKn7ar0BtOmhfBidVFi57FtFqKNCXlXd67khEv0bbI66Wib9aIrK+xI7iSgY4eY9FwUBy8UpQ2DaNZOIjdN6anBBolAQGhiZX1gjObXgeRpuKsVKuGHlQSh3ZtrlZ1go9FeKhHoU39tKQ1DH9bV/tiyVxaMM5FPSNlWhgT4sjRkPbxn3nvaTk7oQXJqjcpyG5a8lfMTPflUOMFI5U1FdIyQDVzoJ69v2XHM2NvsicWAXuolzG/ZfLhqLhyMjsDU2Zlj97iWGwRnr4bsPJqsLFQs933ItqR2PzM3sOD6yBvD9m+O34yMmYFdJegfGkCc1BO4M2VpwbaDbNCF4CmW1qbIhotw7ihjBGDVIodRBElT5wadyoUxq+iokUgnV1yYQ8ZgcSYQWb6NiSgP42SFt865QhODkyySmDSNyai4OBHBhtdRdALFhAmgC8lsbHpDhvIWUWwGOZ1cnvndU+63pk5RthCXtPP0SGDvnlM0tB5jEJnY8kmbn3TQOsXm3kISO94xRulkxNBRYmD7xrBpWCfEhXlPywrIcOXxkaaGH00XWJywBqfZMEpUfgdcp7iIpVuaNyipYWuME9petK+9xen2RoXJpbrwre0nEtBdJaoY6i7x5gr+BCTJH+qGdtsfcDeT2nnM7zJjkS52BUMC0cUsJtxj6aQBAEy4wlB4L47nKhbW8gzysZkMJ3mcc2ZSYrOg9VAoXXzQLTrDyct08iujWeWb4xGJwW7lUZuEgySTXyq38oX8Ip1akXvAuDuhjK6g91iROtg8wdxvQL/z+omFaaY436jON+3Is1jlIdBCY9Cvc5xvIKoYRcuhajGNlkMXBQ0dcTHUC70PrXKhEyjeu9QAlLkzkSy42Lj8JJSXSmDvSim9z8whgPxNJyfefZy2fBQnZz3dR+OXY/5N/u8sCNCyrE6sXiCzi02EZ9nFdxpmqU3GBiMj8sQ5LYafoo7K+7nCacNZ7DL5dBVU2SZJBCTkbjY69gEgGUrs/YcL9t2MLU/vYr8Fcrlhq0MXrODd5eB/Q+gGiE7zedcjn258FDb34baD0w9mXCpt+VlTXUhsy7LNHu7eOUOLoddfLFLeGDB3FVMbr3Lhg+blhXOiNCt6Bx3ZdCwsYFFEuW1nywBphyahnyzYOGFk/cUJ36nDt/Eb+d0saRlFwpzy/RBWCRniTW3vY2lk9/dqRXRMrGz6Rhb20oYy9kD7wcqDCEV7XpE7YIyNR+qZDCZ5mGLSdmWhPwt2TEC2m/dWf5s7SptuCOeC3CcLybqtgYxu29/kbs7QYllBLkfEtiM2m7NhTuYIvm3qyGV615+5g0653+ahpYgu4brS1In8rZvkObPQ7rCsGQ+KyMhEFe931sj1muW38n525AYzNgA4doFv26qY14+2SaaBmF5PrPJ0Jh+2/UmerO3JzStpf3uOAZTmZCjJ70oKhbWk44RTxaad4qITQQFl8o6k081An5nvqH7ybQrAdegifF5IH8HKyIQ6gxgZzsyVC3ME4xCQ3b3qzK81U6eE5w8OKsl1NZ2f+nWi4a2juJi50m5YzGYW/t6mc2SktuHtaTaWPyIQBsu/MTYRvdvNOkIlCpy8za9JIc7jnBMJkZEcKO/ReYed60S6YOel86rsb6rLvHzTP8j8mbNz6Hg1j6hNlH1TyJvIU3rXdBnaG1NC7N3VNM/E7QdWV5ENa99+Slo6vxOzq6x9a2wzFec8Q9vMhnX07IWCsyjRNbSuVXPigmxGXreCnebPZzTyuwjJ2LB2NXEadq7loDKCvSuRVILqRerIZx6qnDe5tUHTpuy1QWTeRzfGSx3lFPOMvvb0C52D2Sy5ORR1UCEy2PBHczx9xvGmn77MPqBrU5YGKpcVrNw230UdlYeSJ++5dSZ7t2lJGwnmEBr1Q1xKH+0wkA06gUAgEAgEAoFAIBAIBAKBQCAQCASC7Rh6JmEhBNslZIOOA/VgzNBdkHswWk8E6hFRcfkxt2sdTSsnTZ1LbZ33gZ1wLj3nNZcM6MIFrr53W0I8TQ0UDfnje0klxNNVF/MDkJ/OoJ6ujPeJQz8XjoKpPw3tYL25vLobr5DYnJzzPB2Nx24o7Eh+yWpWnWniIc148imdX+7M0QO4XljFSrn0mWeOd06j2D7UM80+m+aZq1K2V4RPoB6HAICGdi+j9VDrsvCKMv12nNwtcetqPBi5/kSRdwsnBtn3eTuzHknmb+o1X3HyvLLtqTec9fTUhfdcGoBvf+c7+m1NWpxHxitwqpi4jiyKJsMNRE+vOB659iEpg/N8jpGHRSFjx3jROukZGUfzo6edKcpOXCkSyoSTR46sJEO44AHp9VMZnFBsxsM80c77kGzMP8roI16oRt7FA7n3rJVbgPV65OCUQ9rZ9C899WfTNDydWFF/J4wwUn7kTqFbj1XiDZ9625cXUBYOiv6bDQ3t82j2jgsRnX/AlOF7LXv9R51LkxYZK4SHS73NFfGErQoPQ2izfOw9Bzydzuj3eFg7J5sKHr1EV3Chd1g7hKYnJ5+h4Xj4AqluNnXa+sgiRF75hdNmnsdvnROXVSfHynhLc7qIRk+g/WBOjEZ+hobW9P/BkxRc6EumbR0e4vJBwHu5qqlIm2svuoNTLm3zmiHX4JyWIukDp0vyE09M/rnYLtDup63iDycsGMN7BqqrEE03nDTQuU1Hx1k8UGw7t9Bi+aydkADRdFR4b09sTEb8fKTM5AjwAyuXNOzpEyc6BDlR6st9R4/TUGJGBJCQu0EZ5/OSf0qHC/NkX/Jy2WZVxQ+KnhrJeNOXmxz9jB3jhIRjeAoef9ns/YgXCvnpDzoeuPobfgqcdgmOUyqvPXDzwnhQsyE+Q5Eq7N/c6VGGDhbeiQga/QJI9SAXjrynUKQmLe2DQJqqcJP+vLkg35T7vHAym8uTa196ktTYjozOdOwEkDS0fzzec2wWxdSph3VJmrfVT6QdDGgkAPfELGPbAeiOJHk6oCCDCidqlCtDSnUx+ZbO3zme4q480E1d5J9Q5CUbDYbUk/Rj4ked8fJkdRmlhYviYtrGp8enUQd0VCBvKvtDUAlsZCw+5KtCNJ3NF2roisKaAJWpZH5haE3ayE/kUjs343FfXhSiFiRg5xBs23N62kvHlsmt33Bz/UAo8sKpv0CerG2GPAJBrjfz+WeIJ0rnMEk+VeXWFatC/3IIzakq1z+YU9YqyeuaLCihCTm/VYZPp3xJ+oPKI9+OjgeJ3NABXVNTlwnmGqqHDboeFKZgXiEbdAKBQCAQCAQCgUAgEAgEAoFAIBAIBNsxEjlBt9Oh4myHQOns53sDGi/JzPtKR2l87qirbJogEpXe0eV7w6uwF4SK83sBHDo8Oh1PtKa2z3UD8O8xoR49Ds1ZHPk0pnf6t4nnrxvutypx22bhI1EaM93cH0Bi0qcJgGQoRjIU2zuvnLbi2k3D5um0SVU7U884UycTq1nn720fE88S3/PGqWfWf4amkGeVk2cz+0VAY+9xNPYed8oxf0fTyr2XUIHlPfq3bpC6kvZujik0x1yeUV3Feo1GHZXfOVRHzlOvMe09z961N0Y2vnsBOm9Trh+jaYWhP0UY+hM9buSWU0gX8XnZcRWRcVAh+ZwTHQEsvTP9Gex6H/FuZPiSjt8q3mXf0zpnlfJ5l8uH8jlbjjeGix/B9qn9luZp/qYnozJZyPFS5N33UUa/f8I4ovdJ+mTSephxwNRPNzWSVn63gG7qVFYSGatbOj1pEwEDz0UYeC4qlc8AXw+bp2lDr52c9iT5+GhMlve1uTeAniAGkHrFZXcnUR3B6QOqK2xb0e+yvCjPmDZJhhPE2c9BJit92WTzjjzPXALd1Bh43hYMPG9LJZ+kCeDqRNru9LOsTs6JV9K2zQmF5gTjIdrW+akeBpw88nnG92C3cpfkUQoy3pwyyTOjQ6KOQmMq/ZnvkiXTbN87+fs6G/l4UKaNfTrJM2ozGX1rx1hAv3N5mXx2eYQQw42/kP4l5dM86Thx7tsA7N0rhRMk1Cbw7AZLg28z0Peclz4z9ikOf/EaHP7iNem9gR5vObxL7UIqY6hN4JdJvqX3nLAyLEFB7rP8Q3mbyFNrcySubUzHXFA3aaA1ptAaU67coeUzbc+ixAPenhAukRfpP9Kftc0qxmsyErs6wDxf2EWy0Fx2nP6C90yR+llaAmOwuVWhuVWVj3Fapk7vkUsGE8s31CZTSXriOGlrtm04e92XR36ZQK4Xg/aH+Vbn5Tvf6eKPtf2IvGbnchXw53S5/eWWmSxyDUZ/vrbnEc+y+Vo7yYOVN2a8+bKDkfsGI09ERJcTWUa/Vd7Pf+/BKcuMgaZ2+MW+z+yEqKPK25zKJZ3e35YMJSw/2DtySJsoTfiIyZ+W79TP9F/E8x431+ZsP3Nqwfm1dG7zkDJbWyK0tkTOs8LfHuj8mc4L8w/4dE4beOmcNqFtYfjMsxP9+XOp3eDLAk9Ppv3l1s+3N+lzJ2/u77pyn9DikMzN82nZTJ249/aebK3ydQbSd8lI7LRv6M5LR75Q+0KT+UiAXzhdamwuDoNri0SkJ2Iz3iVzZcv7xHbi7HGVEHuW0wtJ3iZcv9mTlkZckXUkpcHKSVp33ai20X20NkWWZnuSktrrzbzNaZ9W2Sz5g/xPJ8qQqWfglLHTDtSO9eS2oZtGayrQxcj6pKULc65e5lfB9UhdnO8kzTxyA1eWpT+gn5Kmdk+de7xT0NGBNSUKjiY7XjTsvMnYRpXzMh9emxd4IuMjU45DRw25XpiDhPq8AnY8V82huPeK2KPM2BFsO2gAWtf8zTexgtpQWte+WvDPAsuXL8eD69djn7//yHyTssND6ZrKRiAQCOYZdMIwH2WLrBQIBIK5gcjYPz9In2/fmE+bSyDYFtgpZZDZ3J2NrEQGCAQ7BZ78nxfikCVLsHr16vkm5c8Gy5cvx0Nb1uKgS95b6/tHPnAZDt5lqfTRDgAJcSkQCAQCgUAgEAgEAoFAIBAIBAKBQLC9QqP+HXRyJGuHgYS4DMCEvTBHiBv0wlkaasNJxDwzr8igMGFowoUX8+GOlDf3GS++p5/QME5MyBsa+oKGVrAhwPwwFxltjUmFRnZxu08XTcOFDDAhlXRTp6E7vBAvHE26raEH0p8TUoEcYy870u6EeVBpeIfBtREbhs8PFcGFjjBhQ5KRGMlIjOaEQqOjLI+woZtMSAJTnte/fqgzp04MfQDc0FM2I9hRbUJDUT4IhmYoCXPIhorz+dMPd8HxDkF7c7noCYWno+X7ITwoH5mQGE4YIS9EZjkBgccMz9nx4oUu5NJw5XBjuzTsk88PTNgyg2Q4YctnQ6kEaKX5++Fv/BAneSgbRh4FwPE+pdOOLRo+hgEn4xqTCslwgsQPvWgLoBmADRVRFrq4MpwGE7aDC0mnowq9QMJiBEOrkLBhBZq9bEeejDDyZFTMw+tfJxwNpZ8Zm1x4mea4QnNcOeFhKO/o3TrQu3WCbUb7tCx8jJ/Of++UH2pHEtrZ9BEnw4NhZMqQpW14fG5DF1E6zTsm5N7Qs3yf0e/Y8DOUDtKnZSFxnDaj9DFj3wmB6YP2OTMMC2GzfBkXCklH8iwbgz7vOaFwEqYeGR+H9D/N1z5j2omTm6ysDfEUDcvt50/bxGsrlXhjlNqBHr/Zcrz0Tj0JL1mbgoNPBw1jqFEIG1vWZyasUFCuMratihVfvwy6wYf9srIyVqy8aW+OrL1iniW7TiPZdbpy3NMxHLS/QvMJuHRwNk/6QfqjtnN7U4T2phrTOwUMbIgwsCEqyvsMVIb7vJuMdkvHk8PbjNzgeN/nv7JQ6bqRh6Cmzyx9rWJ4WKf6JTZx6Lsq/Uj/5sa40iQMcSDEZRnPOOFrqdyvsN1oXQogMpqrLy3Thk2s4P2qkHQA3FCxxhYiY5jOq4oZlZdfaj8bvUFkFPtdSZ5+qHcjX8t0aqn9wNkz5s9+bA4UbbIyeYoolXWlui5Ev0+XaV+uD/xHAVvFWYeg9Af6pg5C/ezLsGSXGFg6lf68scGlNzxrdQnhjWRAs3MIVp7Q+tO+q2mPV/FJpSzkQtpleTfHc/3K2WSOvGD6NGg7EhlAeaag68rCkfr0E7rLoDQwuC7C4LrAXCgLn1uY/9YEtW2pPrb5RF6bmVCyjJ535tfUJiDy0qHb7wPKryVXHDjfhsC8D4330vkIl7Uvezw70h8vRh/Rd3SNkqXdm4uacpUGe9VLSB6ydeZC2NLrPzoq7186Hsjf5poFE/a50NYeHxbWDhP3x/WlbmrX/vD7lJEBQZ4oWZcRbBvomj/BjgM5QReAvRssE0hOvObQglEJ99PN7e5IxTBhXvt3ogBA98nh4nv6WUJiU3v3twFunSx9Ov/WxB33EQ+SdP5dLX4aP72GvQNNabixpQM0KaJ8nPjWgb99OHeuaWBy98T+7dNYxwnB0NgYSy2c7hDTBj6yT5IsxnUhTyZuuo1VH6CPvfuO8Ob0Am2fVdUrdI8ekPUx04/OP/3JRYB3DDqLkvLxwpXple/fpUd5SSsUF7JC8d4ZqECbcTxnxwu9gyfi0xQz5Mc22x+cXPCNJu+baDwqr4euHkdsXbT3b59mTh4FwPE+LS/O3jtyiQEn4+JBjWi8jLnJn4E292WU87pqDZSRhU5/kz9L9QKxroL9SfI1vG+NZi/rrfsUlZjm+IhOgI3YbOQTdJpGMQtp3eGw3tQKiNa207/N/RN+mbH7vZtB/X+n92sUyHNB6seWWdL+lXyevY89XlfevTJOezL8PLG7Wwl/PAI5/UoH2qxK39F6cncBMe2gyL2pBZA82PtX/Hr6uiU0xnTFe/OZVw9LI7dYSPqjUmfS91w7VcnVgAz289RRMf+QzM/bguhPkr+ZyIdo59qSjgdrU3BweBCAFz2/lp2UIZrk9RYtq6B/G7ownhwkgEqK7618V7y86SxMCs8az7Uq60BpK/22Qi45tjm36WHMPGI7dxbVvGhNA1OLiT3MdK/VS4zciDY2S8eTw9uc/GR43+fB6ZK6qBiYWuLa8yomvNEt3vXtlFVz3bBw/xij/1hdH7KzApvIwfT0FdWVJTQCcO9OLPmOlV8URO5bPquoQshuCspC039kDPv60k1UXn5BlrF6hWvIennSfrD8Nh3gt5o6d8ZpPPg2WZUMorKOTRfSVQxdtecwgTkxZ+9XzSurEBqvvgyLtjSALWWXeLrpjV51eN7wc0exYyEoOwwptO84O57OJ2rappWy0Lxn5srdYV1q5znPuDltwHZkywej6ypkVNVcmINWRH9weXu82WsYTqr/O6NJsf50naGhbR05OyU0nliblWn/0LyqkNb7lgWXfYA+bn2jNGu/X83Y4eaiirm/XLtrlBxNZfpTt4vyiOU1gLd5OCdE0jZJWxd42x8bxlZuZGsXwfJJnqHy0gdFmlRXsfIqz7Q8vf+thKidX9Q+QSfYYbDdnKBbv349dt99dyilcPDBB5d+e9VVV+HYY4/FggULsHjxYpx66qm45ZZbthGlAoFAIBAIBAKBQCAQCAQCgUAgEAgE2w46UbV+gh0H280G3Yc+9CGsW7eu8rtzzjkHq1atwr333ouTTz4Zxx57LG688Ub81V/9Fa677rq5JzSAneV4b2WolwrH3DqhYuwx+xnKil7DDdh0JeEce4HuoQ590cp5+vheYrOEftuyb9Thk3lEnX7lQmzMJWaDZ2dMw1zwSQlPc6FQdlRsszHmeRyzbVZTjnDhqIzs7meM+NBR+uM8OuvkPRPsULwU6i/muQ231Y91NwdtEgp1Vhf99rMTDqeH/KvsgzkdxzXHZSEsaE3MmgxV3t9+nrPRPly+5pkOvJ8JeqB5Lmy/WrbzLEMZG7Ms76r3cwy2rUnf17Xn68qDvmniyqpL01xhLvsuVP85mqNsa8xU71SFk+yp/J2kTVnMk2yZzbG3o89NLErsyfnCXJdv5iD8y9ksiPxdMZ4rw8P2IQ9m3I5VZc7V2hTTFjvNeKvA9lTP0mtwdpL17x0VGoDWNX/zTaygNraLDbqf/exn+PrXv453vOMdpd/99Kc/xSWXXIIlS5bgd7/7Ha677jr8+Mc/xn/8x3+g0Whg1apV2Lhx47YhWiAQCAQCgUAgEAgEAoFAIBAIBAKBYM6hoHW9387r6bPzYd436CYmJvCud70LRxxxBD784Q+XfnvxxRcDAM477zwccsgh9vmKFSvw7ne/Gxs3bsTXvva1WaHLegRk3m7xcO46QC/8NWiMq2qvGO3+yi4h9cFdesqlcTwcVP7jLmr1L4/2ve65y4sbk+5lrOwl5zQNExfcXJDKeWNwnuEhj0R7+iJ0iinkgW3+ZC61Nxf3lo4ML1/dCJRFkDTTX1WbVV7USzxiuW9pW9l2pqeQmMuBAd5Th16EWwXnMlrF804xkfej9Hh82Kvns3/prr2Emb7n0pS8d2hBsc20zxOFxOVlVoLm713+HsxLlXs+1eI5Ajrm/DTOhdibI7Q3V6sXcymzf8lxXmAFTfQ5N26J7GM9CH0eNO3Rwynbut7SnKcmHaOl+ZScIDA0WPoJ3Vw9qGxgZbV356C9X8XIy7hIq0oCfZ6o9EdpNbrInIQJnfYK1I/SVgr/kmuSFzv+GT6pPTYYOebLE06GFS6DD9DM5RmE0RF1TjiEdKSvCzz5Xstb1nxDdaqno6iOCY23oHcz5Xdm7Du8S/rR178OHylAdVX6C/CJybdSP3LtSHmLqzPV74Qf/XZPBvOEjuwg7cDaN8QOdNJwtDMwsrq5lQ5EsPxmyjI2R0HGVfCeitMflcucDHXakdafkVH274CdyPU5Kw8YWRsCPXnM0twDTPvrAQ09YIiC00ZKh+28Qp0ob0U8XX7/OfkwdQjpMrbvGXlQzBC2rU15yXCMZLj62HXlHKlG+gIfMc8iT15Ye5+8d2jK+JT2EzsvDPEJN1+g+duCkY/vmrKehScj2TTWDtT5vUMV+XNy2akzqSc7T2hp6JYuzI0BuPoK9cec6RPOvqjSv0qT8Z64c+Xa8xcj45v530sOXY8lh66vmYFLT2hew9YlAsp4q4znrIw0Ooazx2vomoIu82033xbxs+PozvKiY9fqEiKL7HhU/Dw9VPdoWiGi95+SPwfWR44u5mgz7w0/B22iwBhm6TL2Qw/yLhlMkAwmqW4xMoppp7J25+xcIFx/moa+p+OQs+MK+ceu7CjY1mwiIFkQI1mQ6ZIQz9ZdKWX0vPPOINO9li8yfjb1c/jJjEdm3tWvPK9ct6OPS8aTqWNhbkDTM2uPZQjaHyH5HXm/EM22gPxXqV/7jbbl8Yxvgxdo8f82jxjeCJVTWefQc8tfiqfN2uNk3p49c8ZYLzpOMLvQyAVl5W++iRXUxbxv0H3qU5/CI488gssvvxytViv43cTEBH7+858DAM4888zCe/Ps+uuvnzlRiiwCJOmv/VzDCqB4QCMe0M4ETLdgFxRCRogdH8bwIJebauVuXPlKIWmlP0MfFNxJiTEwqSHlGLNFoqKJqChUNTHSFAqLJEkDiPYZR7TPOFSSXtBKL2lVsbILXCY/55JdupmlUFRk3iKK0rALQs6iECVZeX97yrU5kU+UVFehtf9WtPbfatuULj7Yi789mp3my96P3q8wer+CWjJVSEPphwL0QAI9kF4SbCewpO2ToQTJUBJcgMr/kaexfEQWYsy/k2a+mJM0c1oaUwqNqbIZM5M/IyV8/rZ0kHYwfMBtvqg98jbjJuTQuUGQDGgkA7qwWdPeFKG9KSeO9hOd3LJEJ6qwyVW54K/cBULTvoZ+avBGXSY9NdLMhMwzLPP0jAENlBqMdNHYfh97soPUxYyR0hAf4OUZtwAYdfO6dBYm6CxM+AU88kxHQLwgQbyAr1S8ZDrnE06u0rzoRC5rp2QoKUzIncm/zn+UH5rjCs1x5UwqjVx02mFA85uWhj4jP72/bZtlEzHdAOIFMeIFgUVHwvuK0OyUlY03rfK6OJMb0w4qXwhgQ2CR9qELFqaeoc207pC2l1vbrMyCA9WLkU5/AIafijD8VID5vMUUn/f9tm50FBpE1qhpMpkg7Ze0deFSbSqjaB2dTXjt5kP535loe+1C2y5Z6AoGlSj3Qnjq5EAXfbNfsmSabysC2+ekzTn9BRR5UiX5+9BimH3mLxQEJpe6qfNyqB2U/VpbIn7jyOQTCJFpx/iAhuoCypO5zmKPyuVj0tTp5eiGH8iY01HOH8mS6by9af39ca5VZTuTT90L5kN5IrcHk6UdR16rJLPdaJ4ZfzW3KDS3KGfsUj6Nd+sg3q3j0sJt6AdsGaOL40FmlkcfkfFqbA5jS5TqGlO/aWX1luEdEwpXxZmutfZDOoZ0A4gmFaLMLvYdaJJBjeaEQnNCuQ5ARm62dcGeBYBDj3kchx7zeJpH07WtNJ0DcM5lSf4tXcB12oHa7n56TWxDOgy6Kv2RvBxbxtibgbY29dARsN/yZ7Df8mccO5+ivddWtPfayi+UEpvD0bOU77w8jR2uuorV3047RsDCgzZi4UEbkQxpJEa/ZI4fSgPR3hOI9p5gF6sSf46lsrSWGJQuptl+8p45jicKmU2f60rD87k9Sux9jVxeZfNLwLWhre1rbDfqNGNke8yPJ2dzhLHHHdlK9JZj3yj3R3kj6BxmxqPZmCdzWXbh0LOnuXmRkYFa5fnTuaOaVlDT6RzBwNjOVF7QOjlymtnA5Oy0EHy57awJNHSud2iepM/L9Cadpz133xI8d9+SekQRRFMKEcnHmSv7zkAZH9P3voxy5jB0zFNZm9l3VhfRb+kmlrExySaK0lRvpfxA9b9WZL7A2cA+LTYhCrYHWyezhkM31MkYoXKN/t04cAyNA8fyMeSNMWo7F8ZrN5+jGX6mdogjj7ixR+dwlJ+yfuDmVqENk2giSm0LR79mZSzuOO3j876OSNsGxruVmwR0jDjz62764zbtCrYWPBkMoDERoTHhNbZPElkbUxr52CQyhvJx2SYENwdIiJ209DByhU+WTzSd2hC6CYc/3Y2RrG2apP6mrJjnCfYZ4RF3vCLonAN4fWp0ScanTr6EX+imj+FD2l++zWBsCX9eRdva6Xs6X/B0qdJkjZKxq2k/cWsPVL86c24K6jzAyG9fNtHxlgwmtn/oWimda1mbL+MNFVfINX8ju8QhwDxrTuRjPJ3zazsvBzLHAqK3jG1M+cQ6ARB5IZgf1A1xKdhxMK8bdPfccw8uuugirFq1CitXriz99v7778fU1BSWLl2Kfffdt/D+mGOOsXkKBAKBQCAQCAQCgUAgEAgEAoFAIBDsNNA1f4IdBs3qT+YGSZLgb//2bzE6Oop//Md/rPz+8cdTD1Zucw4ARkZGMDo6ig0bNmDLli3YZZdd+ieOeopk/++OaOv1Zr1MqOec56lZyJI+yz6NppTzXBGP2EJ64tlm0lNvPWfgmb8VeRwx3ylt/03paG5OXVh0S+feLKbMtgaeGk4fNYuehkkrz9N4UTrlaliPPt9726fPtEM8oJG08zrRvFh4deoO6dwbCRrdRxYAACJKG3ENNjSb/iichso+3Xh4ln7tgOOpb8kwbZcAjbHI1sk5kZEhGo+ystKXCQkB6vADrXP2PtL5N+YUgaKOitMkj0ao0cL5I4Lr8Yli01taFfl/VpbxDnOKWTtQWb7heYdfyPvOaOKmaWjoOO8zhxZDuz0Zo6E6uXen/S5QP+6h9dq13n057yNReVvT07gmKzIenX62Y5vvJ5OuvT5lruldPO8v4kFlYE9w6SLvaeX1nSf3Cl5b5r3n+Z4STVzNTDtysszj52iCeCfCLa+xrm29uli5StrMnILSDW37JhqP8rakdSZyy7YZ4bPpBdT9LaMl88Z0ZHbAO5lDIXwwgZpWiMpikiGnmW1TBesxqWLknsqWN/O/oXJ542aSv7c8Q/L3y/P/bbxFHZ0QKN+827p/+nE0rYrtFwGa6NoqGmLfY9STdfaUXNZn9BQd/dsZD3S8eGPH4SHCh9x4NmhsbHo6n3wL5CcRGPoBQG30jsL6bYpyPvHh6/eCDGS8Qx36PfXue6T7eXKCtbOIChOmn5uaTWe+izoqOCb8MjlaGlsaTl5G3qj1aVvrBlzZ7fNpgD4HjA5gaXUeZknXtvNHnB1H/t3dJR+DVq7Febpmlpdu5CfFVKLyUzyhw6yGJ4xObyB4stGWScPcMTT7NoPzd0Nbu8vRlczYMPWIuiicjAWISpqI8hO+lCearp3h0/TAnfunf0RkbDX5dvaRnqjK24ztfyI3rBwg/eGcljThEElZtp3iXIay8p2A6sonf7uXUycf00+OpGVSm4DQGfLq9utH7TCFsE739cfmR0ZTmrs5zdFkrov1H9P5iHH+Tlra5tscjyr1gt8njnzz7F5Dk8nDkbVURphxQserJt9yY4JJz/FU1C2RA8jmDmbOQmwqrn+p3ObsG0qzpcWME27ckjTmND/AnEAwxRIZwH1r+ZSR21FH2fHuNCMJuUXlmc9r6UkM8s6XT77e8PJw7GljI06qwglcwJOrtB4leg1kPlFHl3OgJwsBV8ZwY1RNK3Yuy9HnyAOjS6ZzXWx0ERKvThnsvI5EaaFrAiaygDm5Zsqkc+kQbcUXJk+Szjs57qSPNBKunZh5lW4A8aPpmkKDjvssr8ndXfsmTUweRUSGk/mbqT/lfXo63KYnczj6zHzTGIuKofVD6wCmHGb+HK1r2/fJQOKehsxodtaeOHudnpb0aW5U2ARAbqdQeUBpp6fXmHFYiFIQAdGWZrEcwi+Uj8vgX7cCuGt86+9dmtc/+188kI9xKk+c9UTKr74cUrDrGIZeAAB3ZQQdAy1SJ6bNWRC5xfIPlRvUTuPscLI25qd35jIJCjI2HVukfRr5t6b8OFujpBEquPpRWWLtxC6K7ezB0RseuHlfPJjkJ+VI5ADnxDJpp8ZWV8aF5L+dp3t2mI3soL3/k3KcaDfKLR8ApnZL3HU/Ti8Qm0swv9CVA1iwo2HeNui+9KUv4de//jWuvPJKLFlSHbphbGwMADA8PBz8ZmRkBBs3bqy1Qbd8+XL2+cMPPwyMjtp/c4s8ocWY2uOjwgCui0qhWEFzSOg7ixx+OiKo2TAgjhIuX1hjQSdKjFJxNxgDNHJgJhNuPmTiYI7uh5R0WVsGjPlQWAQfdGOmLn84BhE3EfS/7RWcEdVLvkwaurBVubFRMemyedIwdaE0ZvKnA+X3YGcUJhM+73tlhsDydtl38DbmuPLrTl4VX6zzLc1f5X/mH2f/Y5wIaqGMDyJduhjlkEHK9yePhW89o999yf+7n7FTtchhEZCVfCL+Q25jjP13ZTl5UaGFNbb8ksVn0EkLnRdwYVyN0a9qi80w6DCsO4mgE8U+9E3ZYlqh37lFjDKENnjMgnlcXn4xYc1n5lUVvZzOrsiz8r3fB8wiS1k+ock5CxLiJXc8qEhTh60CoYNKQeVi3XHILXCR9qH2RdkGG9fOoTSscxK3gOb3W0nxIWeGso3MSlkbWpQMpacLQoE8K+WjBrsgzEElCto4o4Q2Spk69KWX6AZn2T09irFzgLwvqSNS7cI9/Vsx9rlNWQqffjre/c25Qpk15VJQpgZ0cuFxL2O4iqSK/lY6QG8P9mWt+SKxB3vKm6QvG8/cIj9F0q6wmfqhqZf3jE1TeedVDbrqll8rD992rpFnaJ7g5ON959jbNcdWXXus4Pw8k3YhyjTohJiBlXtsnoHNoLL0VTZN3XmDlz64YcChZG2LzSfU3+S9zZLZpADgOJ7M2BYum+OEkjA2SdmGNN0wBlAp78rsY1bP+ms3vr6hc6Vepohl33jOJLXRr/7i0pWV78kY1gmQS1ZmT/r2MCMPe5G3vcrmYJj+AO/5jpNBOkJjezb613NgkP2f7RyzZF8Kth/Mywbd448/jvPOOw8nnHACzj777PkgQSAQCAQCgUAgEAgEAoFAIBAIBAKBYIeAnKDb+TAvG3Tvfe970el0cPnll9dOs2BBGkJgfHw8+M3WrVsBoFZ4y9WrV7PPly9fjgfXry9NS8MQUiRDqctBVHExbV+o6anopzEeS/FIRhsJfaMHdB6ijebPeeN7njxAwKOCeOeEPNedI+lefXo5AVAVyosLXeh4gpB6WtoU2OP3NC/leTnRetTxirehTEm4A3u8Pi7yFhdKJeT55YfnpLSX0Zcfuc/zZ8sn9ag6XWIunDfhiBwQ71uuHioBkuwIvuFRJzwJV2bISzn7LhlK7NisPJVQ8V7pPMQsDXVi/o6mi6FuejrJwY13QpO5QJ6GyPDLArw2JeVTr7SqkDYOL5SMOVrnUq9EKnfKyjVl1vREDdbZk2FRNw+NFDrlasaRucgZ8EKtgkln/s94r9mLs03+RB4B1TJPt/LL4uuMx0JbKK+sMlpJ/sGTlB6irmI9RC0dAR7NTynlJzFs28R5iCsudF2/J9xC6bUJBzUVYMiaXp82e19eodh3fsi6OvKB5XMn5F3WZi2dh0rLnjWmVCozaoCVF9SDvcbYM3D0Clc8d8rI8CANrxZqZ84Dnnry1zyFC408PEw/Xqgh1GwzCnOS39EjjK7j+onmq5u5DHP4jZZb4o3u5FnSjgU7pUQX1xmjhXBUlPdIHlW86ZTdi/1s8idykw15WIJkYReNTUwILa64KEAcY5uH5LYN9+lFgeC+LZZP/5F9FxgDts2Z8K4F/VTG+yj/DuBt3zI4HvLcaYEeeIDTy45NYcLherLeH9t+aEEuJJ1bSAWtJe+Tlrb9T/Nnx4n9oLycqMPbdmyoc4Mac0FWBtawQwFv/qiK+VN7nEPUUTaaij/3AKptModWUw9SpjPvJHZzpb1tsubGaI9rcIWrB/zTI34ZIRlZpQv70G/0u1L9TfKgYy84FykZG2xIOQa1xqNp04a283aOH3UrcJ2CLSxMR61vfL2bPaNz+gIfEJuAguP9OusH5ruy0MfcXCxYDzD8r1E5/7c2U8VJVkXGKzgZQduR9tkMbMKQPU3t1MK80OMTS5bpx7Z2IgfNJULj2beJfFneS8h9FlRe0X975XO8E1ozSIa9tVqNfAxHGrFZy+XWrnoB1f8MLXXWCw19/eQ/EzvCP91Z+0SxYNsjNK8OfSvYITAvG3Q33HADRkdH8e53v9t5Pjk5CQB48sknceKJJwIAvv3tb2PPPffE/vun90A88cQTbJ5bt27Fxo0bseuuu87s/jmBQCAQCAQCgUAgEAgEAoFAIBAIBILtCbJzutNh3u6g27hxI26++Wb23eTkpH1nNu0OO+wwDAwMYO3atXjyySexzz77OGnuuusuAMCRRx45c+KM5xvZaW5M5p7n9BJq6mEwY2+LMpLIBekWVbGkiRcXd7FyNKmKHjt+XH1vzFNPSrZIzxtJL55Onz/XsjRRr0vupAelv0AH45XveNiTdA6VtNkyL6t4MO9H6gVk28Sj0SJx3aCqTqTYbzI6zGkQ/+QeLRNwaarr1dnakn7QHc49m2vJbeuSVexbNmZ3hReG0u548Ps5dCE0pUdNuWmTIe1ekl3h9e/n29zcyHm3F+9Rr16m/Gg6+zurZjSd/50M6Fredslwkl8IzHjxIlHumM++od7MM/XCsvwYureNeO1zFzLbfAZ08SQQ06bRhHL42D9FRKEViqcGAv1k26Sr3PHo1d85jaXSC8/T8pVNb08gJCq9NBo1PABDJ+eytM6JHuvJyqf1x0vUUY6u4cqhMobz1HTj8BvP//whe+II5Bn5zveEde6DCXnc+v2oYT3YrfcioSNpaXuHkJpWxT73+YDxrGRPNQTi+zc3pR8P/SmlZeyAxGm/ZJeUQRub0gwcuc95a9O7vzKPzcZY5Ah5O7aTqkGcwj/96Z/mLow17yTN9GjMnvCnOi7OdHa0oVWoi0p6d8ArnI7gPKMbRX40iDpEBtL0ZIz7nsX07yjO3ZS1AtTuqWLRzw7Y8h36eoRKwuM4J8bLP9CIlN+MF29jPCraXIH0zrjP/m6ORdbWibrud5amMvp8z3UvTchLOXQy3jxz5A2KY5U7FUjzcU6FmLxLaHKiMzB8qOL8RI2jNxwvdsb2KxkQaiK3OVSiXB0GFHUBOXFkiqO2f6ncRj7erTxt6MqTIg1zSrmpc1uAnK7gZIvT5oVM4bRZwVbw2t4/TVfoe/PejDHt5hl5cyPntBg3tnsQYM0Jb65g0nt50b4zNFDQU206Iu3X9WijZZDnfjsnns1EdbJzei7J+YDLnz1VT8au1e80UgOlrWS8RVPlc8VCOnOYITA/NCfcFYlkENGII16fNCZUwdaj7+ORxJ7QZ4milz6Rfig77Rc65etHC/HBRmhRxe91S1s7PRlMKtcc/LE7/HSE8T0T55lTD4XyEylE3lWdkAudivPXHFIhpJ1vgNxOU12AkycmwgpdxyjUy0NjSqGb3d/NnTwq2FP2hVs2LS/qqCIfRPnYVklOXzDCgNdPvgyMF6WZRVua+XecPCPP6FytcOopdCdWJi+SdlI8FZgoRKbt27rQFklbY+TxtNDxvUJHr8n/A6fATZ5cm1StCbl1yV5zAXyI/GWjBgVkeRXPs+scRkZPK2g/Eojm8+LKVF2VR0gyLzvK9pNu6KK8CI3BmpEQnPUFIsucuwSj/NtiReBmxpXZo16ma0eUz6ltVZqVAtRUSjTlYRvxqaNyudqHzeASW3zUmFKYXhhnZZkFI6/9/BP2PZy8dlAlF02eXlQjm9ykifM5WPAkZb9tJJgdSPvvdJi7HaUSaK3Z36OPPgoAWLZsmX32vOc9DwAwNDSEk046CQDwve99r5DnNddcAwA47bTTZoHAXKk2OgqNjsLoH/LXg2sjDK6N0BzPDDMFtPbZmk6gMyXpGBOA/Q4qDXuTLOzCgWeA+hOyeMk04iXTbl7UiGjq9NfWaa8aAzH7tjGp0Jh0JerA87bkeZl8GhqDz0YYfDZKla+pU1elBsJ+Ey59piw6qdtrMv0lCq0n2mg90c7r1lVobk1/jQmFpJmGCjR5Jks7he6IppXTfuZbQ1PUJeWbCYT22jFLq1Vep+aEQnNC5eHnsrxbYwqtMZXT1NLQjdwQ0C2d/kybR+kkIfLa17xXOi8z6ipE0+lmjoqVszANpGWZSW0yqJEMZmU305+te2Y4mTAzpv7TixJML0rcTdA4b79kKLGhWO17DZsBpcn8nSzq5u2Ttbnl76z84aciDD/lx3PLecv0s63ncFzYNKAKXmWGlzW+ADS2RrlBaBYUSPrmhMr5QKd8E00rqBhZuDyS16RyeRZw+CAKhLmzNEduOpPW5OnwguE9Qzf5ReNRoZ5Abvia0IZKZyE6szq1N0Vob4oK7VeoUybLVOy2tck/6io0xyI0x9y+o/moaZX+dN7/9Eg9V+dkOEk3Jbhm1Mqh1cgDp8yM96JplY8tI2tiQh/JvzEeoTFODGsj+8YVGuN5+pSG9NeYVGhMRGhMRBh8toHBZxtQMdA+YAztA8YQTal87DHjlbav0/amz7O0aaiPNL1ukr4yvDmt0olBJ22HwnhppgtOjQnlyEDSpLZPVawsT9kyaWifCHyfE/pM+9jxpvLyVcaDdCFQxcCez38Wez7/Wbevl04BS6egmxrRlEoX7QYTJIMJdDMfoyDh3ex40HnfO84MRq4G2twgXjLtPDN1GfxThME/pfqb8qApd/Pzu9j8/K6TBirdmGtsath2sovHGnm4YqRjKspkpOmnaDxKw0t7fWf63PJqJ9c70bSy8to8s44J3jgx+gVEvzWmlEM/VBrSxZE3ZkyYeiig/WQb7SfbDv3mR8ep0W0OvxIZyIXdoDLC0JH2JWlTo1eM3Mrq5fevyT/exdMlJj3p23g4QTycpHy4ZhDRmkFrx1G5GI92rX1GaaI/q/cN/REsb1P9bPiAwtHfVJeZKhmdrvM8u7vEuf1m7M0xImuJPBl4LsLAcxHSMD4xkuEY3YVJ3naJAhKXN52xRWxY2k+W/kz+Dz9vc85Tht+pzZGkC3ZJW1seTha4/WTkPl3ksXZKE2iOKzTHiawhMihe1LX0t/bZmtrfKm+zdBxkMtDKWlde5UyZ/nRLQ3XhOGSkC63E5iFt7vO7o3+zPBvjEXRb24U5bmwYPh9Yz0/HTPvptrbyorUlQmtL5Iw92k/dhTG6C+NCaGDaFsYuTBrZr+XpUKQ0GnnN6c+RP+Y2melvX1e2NkdobXbbwfRN1FVF55EG4YNWPvcwcltl5Y780TgdZhuXGW+3tuR5tjdy3ixe39PnCk4/d0c0uiNkrGftZOcw2bikuojjA9VVth2aZOxaewp5m6kkb3Nqu9m5gQb0QAI9kA84s5Gruvl40hEQL+wiXpiOlXgkSa86ILI0GdCpY5XZ3NZ5H+gG6bumtnW1+sPvM9qmUe7A5sOmj7z2yuq68JEICx9JO8bUWenUsSsiNn5zXCEeShAPJekCI2lLINMrRNfS+ZjVM6bOpC50vPn2NK2nDe9K3je3kr6lfJDxpo7yv+k3VtaQ8WwcjHQEO94bxLnGGacB+Lbp2LK8UzhdG03l83MK085U3lFdRWF41+gipx32mizQGI/kwpjmNbAuwsC6yJFhWuV83NgapXNCynuMbQzkdlY8kNuh3Bwkl68K7Q0R2hvMrjGcfnbsoq7C3i/4E/Z+wZ/sd9GkQtJKZcXK41eTds7lrh2nZDxa2jXQ3hyhvTmVr40NLTQ2tIo2EFx5aecIpO+dMW8bBAXepusUA8/ljWr6Od4lRjyoEQ+69o9BcyzC2CHTGDtkuhBiF0jrauhz1slsPbQzNvTiDvTiTi6r2trhZ6vrjf5ua0eXmPUTyg/O2M9+B597Ow4+93aHViP3QGwqlZBxwIH0qa0TldFkPubI0kwvRsT2b+w9jsbe6XU+Rh60Nyl2Drpy5b1YufJea583txZ1qqHP8MDQAVswdMAWthqUH834Tm0W88Bdj7TjyJO/QOaYGOn0x+hXAI5NrHRqh9FvzTgy626IinIXyNdqVVyUAbRMuvZjdSLI2KR6Oynm5dRZ8/Xx5yDxaL7eG03law5mbYLKU2pL7fJohF0ejSz/J54dybU5QOyrbIz4DtFm3BrbPZrI5+X2u4jYPAO5/efwePYsWRBbe1gwT7AMVPET7DCYlw26fnHuuecCAD7zmc/gwQcftM9vvfVWfOUrX8Ho6Cje/va3zxd5AoFAIBAIBAKBQCAQCAQCgUAgEAgEsw6t6/0EOw6U1ttPlz322GM48MADsWzZMjz00EPsN+eccw4uueQSDA8P4+Uvfzk6nQ5uvPFGaK1xzTXX4LWvfe2MaFi+fDkefG499vn7jziedclQknrAI39GPTiiLgnnUdGi9DLSYBgJPx+mTP8SbgN2k5xJb7w2nM90oHyGfuvZ7udb9Z7U0w9D4IQhMjAe/iU0lcJ4SnLPDUry5drJ0oUs7GfVpcRMGAI29BPn8EvK59LyBYKtU+30BNF0jVA1XJhCjmczNDrKhq/rG37+fp3L3ivS5n04lQR5Yi7zIfTbUBeBUCVOGoNQc9N2KumzXlDKZ14/9co7PZdvxlQo/JxJQ8YgvWB9Jm3h0BGS4eZZVTkVbeLI1aqLuWu2L0t/IJ0JlUbHNQ0/RmW8pW2W+C3N2M0r6vJymeU3omP6lQtlPF+pn+2HOR3mxHJK89yYaT7NBf3v949nc9Rqo5D+rft+FuCEsKantYA0zJHlTZWf5vTtsWDm5O8+6hFqR8szSR7aqCokEdULnE1Rpf9Z24TaH+bkZ7daf/uexYVwzVVymZMNhA5DkwlTl5joEaE8A/1U2SYltlklerEzQjYbjQJhThcQfuBkCxeyvVCW+dPI44Cu4+S6BTMH8cvLvbX59wXSvPHg6GKkNLL2T91xOBvyxp83MWHkfDjhwalNihJ6Au/r2vEznS/QvvPDmgZlRJaPOf1YWS7csh0ZVmFHhkLR2bz7mNf10rZ+nUPpaJ515cmszDFq2r55ocj7hDmBUoceG4KSCZfr2GSMLC/QAhR5fybymOYTsm19hNYOyHMTncgJhWs+JTZFL/1ZFvo/iP8/e+8ebFlR3Y9/ep9z750XwzCAryCDvERGJWJCRFDRQmMJqAhqEq3C+I2J8WuCksS8v6lfvvFbRkRB/eYbq4wmlo9YgkYBja9gNAoxiEqcIC91UKPCAPOe+zhn9++Ps7v36u7Vj33OuXPvwPpU3Zoze+/uXt29evXq7rVWc/st9RhtxYwdGomD5U1Pd03uueTWMMqb12NlTgF+6FzAbftieQRGn6ZtxvWDn09uXvDmd1157cSMXQ7JOT0Cp2+aMZ3b95oGYjqp/4y2mbMHyWba/BNbJ0+yLs3MOzRMcQ7maoSqwFs6SVJuvIKRq2PoST9+y+U46cgjsW3btvEIFXTG1q1bcefOHXj0X/5e0fc/+V9X4KRNR0kfHQI4pDzoAODKK6/E+9//fjzhCU/A5z//edx4440499xz8eUvf3niwzmBQCAQCAQCgUAgEAgEAoFAIBAIBIJVhdLwlhLm8pBCP//JwcNxxx2HEoe+V73qVXjVq161fIRQbwZjje/dy+CjNvHoS7KPWHXohNUGa4Wbsh4MMk+U5z/L1MPSH7EOduqXKleHdWYtvUqt/2KIWCFF6+lZHUbb1lgKFlgROXdAce1urFMZy6DYbwd+nSIkjSObc95zQMTKL5FsYu85Ln+fd/333rvObUHyp3cvON4nHfPs9D21dspZJjNpir4Z12KsS1rvO9ayfkL2iMnJZBpqoZ/xIh6LDi6vLrItJ5epXM2Z3pTOVR36ZDing+8c2cjI+Em8WAN49MXkMiurusylseIT6Yrz9GXUGJ5zXazdA2+XnAwdp51yVVh+I1x3PHhzpPOORjWYVv1yySPl2OcZ3ZKmN3e/mLssgm8zci2qE3rpi+Zvv319Xs7JPq4Ihrede9ZSeY6pE03kqdFFJ4jRR+Ul821KnywpK8cTyb4ukAfWSr1UZfFlEkMf6znHrZu49h9jvAb5eHkUeRZ11JFT78dty+JyfA8DT/dNlWM9W5fUWOtS5x65jGdT7n2y6JzczYCrs71/KPVtoTyZik7Ude1M9z7GpMXUj85FlhxOH4zlbbYZ9Hjtl6VzkvWG9/1wbaj72k/HjH7QyXPOFsY8GtPzO5BxmT2MonVkhzHp35+6XPvKirlvnra9o1PldAV/7vLu/g4TxDJKP6drPSdyVyGrjbP/Eh3Hy4zcfNM+JD9zYy613+jlBXTc28kUXS0WzIvm2wPTEXbZ8QqmLQ5eFwumgCBiieCQxyHnQXdQoFC+OUOTMZeJTit///JZU16yGDJgewsKPU8R8S8ILhngylPC6OXW5lJ0Jy+vrkq3Fzaz5XVpl3H6SSG4ONvPSw1Gf53K70JL4luun4vQKGfcJcIH4+JW9rLcFE/ROpa23YSHDiasg6GtM4IFC9xL4+nvgkPNESFj0IFR6AMT/mBiTMq7De/ROpuLrdksPLKT3+py+nRft+FIErznXCpPYC7upqiWVCfZGB0DqYN1la+nM6ZL5AOXV0XSdpSd2fqryPxH60fysodIvXh/xNKXIil7mDasllTQ32PNiwph++bq0UUGMt86bUrmEJ/+ZFv7tDD0j4vYhepF6BJisuB9jo7UBfBBvn7ZVI8Yt74eKi4cTkx/YMYTlR1G96H0sbybQUmagJaYjBgz/1i6WJuP2xdjyfUG/QNdhGzkMW2zQqbqwns2/8wcFSunFKl1QSqNRURuU54O0kxp06IoRCSFR2vQThPK0060EBqKx7u32atN+LSDsJYohe6Bn6saBOsgP/1cpE5TrCddF3febVmG3Rkqe6e5ocfpTOxc1EGu23yYfQ9VY7w2LYTS5XNVb15Z3dFJ32X+X46xZXTOGt3zj9Q7WSc6v3NlqXS7dOVHP59O9TS8o5Dlo+wapRAxXmJ1CW5vSTF1Hnf+nRDjzOVd8k6VOY25tBooVE0YeBpCm4ahToZPngS14nmOW9cNVLkxNk1XuoZh0nA0ZfNaRXrBwxa68E9wyGBVedAJBAKBQCAQCAQCgUAgEAgEAoFAIBAIPEjoyoccxIOOg26tJ4ylRUW8b6inhkG9pi625NAzOulhw1oqMKffTgiACoFVhh8uwA8ZQC2+HPf9hFVh3UfrpaVaukx6NchYolRe/X3LIM4SclJrOZJnzMrKsaTpj/7sO996hPMuKLROUBlLhqzVHmNZ5XjfEELtz0m8FwCH1ztZ5STqGYQX4Szq/LbIWTNmPCVoqE6t0vUo8ppZUqM/0ybE67LU6qnI45ahqb+7h/5uM8iQlDusV6VHg5V3niVoF9A6p/jYt4g1HrVOXrRvCscWzYd69gZ0RkLHDNZpDNZpxzJbm/ByBeVzIVCsN1Nk3Jh2GsnOhLWc0m2bMu0aWCZzaNKOeDUsK+cVkrSwq4HZLXsxu2WvW+RcjXqudixQ6xndjkVCU4CKtA/jcZsbo9QrUw2JZxxTptItXbQfS0O2a2qJy8h42s8Utk+b7ytvDPm6QrWoWH50PK9T80smbJJjOc7IaGecKi9dpMxqoLLtmJTFGXlUIkOpTmJlAyOjtCLf5sY9mVvZ1zneyemLRgb1dCDDY3WuFhQqEymB6gSGJqLbmHYomYt9moYbCSOZNAmdalS2dviUk/v2WyKDTT6UN515/Ygl1EcsNf9Roz9af5OnCp859HbQj2zfZtIM1up8P9tMI49NmxHejM2vVOcr1UfpHNUVgTxg6ml0Cir3Kf05fdLyCamTExHEW0PEwuEV660d1B9HftPnxqvPjOHK060iHhFB/jndjdMpqIxn+oS2T269Yb3KuXkn0079fePpkVw7cVDDDB9VXsP4Xo0L/Fyak+sp+LqtXRd7Xp5FIP1i9iE6g1urEtqCz7Ub4aa0DahcN+PcmYvMd0QPzK51TT9Qr78aqGdGf0aPK0F2nNE5Ro/m0dTYo3kN12kMDh9gcLgbdsf0fb22YG+I4UNuruOJaenm1k31TJh/1puLeeakYWQX3VuJrv2aNmHDb/fcPIMoMYwcc3UXBd3TNrxhss/NGmAI9Pcq9Pe6mdNoGhwfR5Eab5GoQYbPKP/R9VlsjhllCivX6NjlQMeoPnIR+sjFsjoZkmnf0Kg3RF6y7c3xfqlOxCDgnSYvKiNz86bhEytL4MnoCehzSGPapVpqy8rtc1TD0V+UpgidRVeOeWPYWZebTyLr1iCrKQVzEkyAuvBvijhw4AD+1//6Xzj55JOxZs0aPOYxj8GrX/1q/PjHP+6c14MPPohLL70UW7ZswdzcHLZs2YI3vOEN2LlzZzTNcDjEO97xDjzpSU/C2rVrcfTRR+NlL3sZbrvttmRZ1157LZ71rGdh48aN2LhxI8455xxcf/31nWlebogHXQRGIJlYy6xyTBboaqCK7y1SZgKNLPBZYWieUUG/SMrMbNp2OlwnZRmF2twTBY32wCYyGZuDCl3BhowbrG0X5g5NmvzGKK2tk2r/NZt0NLa0WaD7By9+WzkTDKmTyctpx8hkR++l8+9NKmpnk35AyKML5czigd6P1RYW5mOUjWrJzdsemJo7JSKHnYrJkzuwceqp3Palz4Pfpk8ii1VafhDaZKiSscW1cou0NBmllChAuq8Bf0wz/BgSaDIPx4Gq2wzYzXjNPBso9rDetgPZOKO8PWwWLmpRJft01PfxNqsGqs2fUdAcWlQ75mwMeyoDa4YPIjKuzZynjxoLcMqfed9bVJYWOkaoom3HKxfWcLFlmlq1/Wl4vp7RRQtkky54RWQFvafEhiBseFDPugtL7qCvXjPKrJoPK0JllBoy6Xtk7PfAChyHPn/sxBb05PnCPRuC94bW3oKyd9TZhSeVm3SDuCm7v0dhsL79yO/H3JxWDdpvdE/bfC0f0DMGMrbqOb6dzcLXN3QBXD7k+MH285xun2vybQPnbgcy79n3s5qd681m6GCddmUot6DjZDSRN/39bV4+nLozcxE9OHXa3rweqlG4NJAxWqtwM9UnW7ffBvJCG9nrlUXEsxPWsekL7h6NatDyaalBEKdzTAM09I6pjAnTTOU0lRc1adt2w5Tob0Q340L4mDZTQ2Xbhx7WmlJ7u3uuQVdAfPuxLYfjZZqebBZoWqYni0z9DKqdM+Q/RIb6JNH5n5E3ms4VmXnD6ZuM7mYqoGf4sWvlzrohqn1MLC2Gt4LNPrhy2xzCRg8FSH8a/UMtqaCsakm1PMXoIdw9TpQfjeHDiJZw3VMNyHiOjKFBcxhcLVb2vRNO3NPdzYGIXxY3/7Igctm0TbWgnDqZ30YP6x2osve8OLqVt4bwYduayjqOD5gxmA2RT9uErlmZ/B2aSfuZz6yeNKud/gfCuchmQ8ZVG7qVzOuUflI81clGD2DlWXLzuvnN3YXolO/XL6ZPN99Vg1ZG0rWqM8Y9uel8G5EbXJtZ3ZDqjpyM9FB588VwXd3ea0R1ASoPGP2G030onY7cHHjfOnoQ6Wczhn2Z6OtpBLpP5jIz5/t3tCbGiT5qEdV9s27+OiyrniEV5Gijj5YUeoOeUyc1UKgbPaK/q8fOdTb9kKwByWEzbQdzQFQz9zzT/QHLw3R+HIT8oat2T4u7FxA9BDJKweU9bi1rZHV/Ty8wYnf2kxZDPa53QLV3+dVAtb9q28KkJ33m06/72r330ONtFlphaWNt6TeoiQG51VfrVh/n1gCGrmhRMzrQ51Xd0k/3N+xXylOzfQOACk4/6SSftWnUA7MBfVGdxpcxOnNvPJWvni5g03TQk4MD44Fy914auikfp/YojbEQ4NWDjB12v20MOHMK0dFrTz8L1vzN78EGss/D6QxEx2b3NxJrHXroWw1UKIapWOjBrpe5+cn/XnCQQdY2Rd9OAfPz83jOc56Dm266CY9+9KPxohe9CD/4wQ/w/ve/H9dddx1uuukmHH/88UV57dixA2eeeSbuuusuHH/88Xjxi1+Mbdu24aqrrsJnPvMZ3Hjjjdi8ebOTpq5rvPSlL8UnPvEJbNq0Ceeddx527NiBq6++Gtdffz1uuOEGnHHGGUFZV155Jd74xjei3+/j3HPPxdzcHD73uc/h/PPPx7ve9S68/vWvn0r7TAPiQScQCAQCgUAgEAgEAoFAIBAIBAKBQLCa4btNxv6mhL/6q7/CTTfdhDPPPBN33HEHPvrRj+Lf//3fccUVV+C+++7Dq1/96uK83vCGN+Cuu+7CS17yEtx+++346Ec/iu985zv4nd/5Hdxxxx247LLLgjTve9/78IlPfAInnXQSvvvd7+Lqq6/Gl770JXzsYx/D/v378YpXvAKDgevVfvvtt+P3f//3MTc3hy9/+cv4zGc+g3/6p3/Ct771LRx55JF44xvfiLvuumvitpkWlNZazr0Jtm7dijvvvx8/90d/EP0mZkFw0NHREmXq6ZejzJWgSSAYAwddDsjYWFZwHgKOV8cKyvuxvaEPJlKW4auQdzlPxYc7lqVNVrjvO9cp4+GwGvBw5d3VIIsFLR6ufGgg/BhiUl3lod6mKQ+9aJqHeJuUYlJ5U5p+qnKNeDmNgy60HNJ8Mg09cQzdbZzxuOwYw8t20jyzyQ9l3lqlOKTatIB/fvyWy3HSkUdi27ZtB4UkQXNm8cD9+Lk/jZ9ZUPz4zZfjpM2T9dHi4iIe8YhHYNeuXbjlllvwlKc8xXl/2mmn4dZbb8XNN9+Mpz71qcm8fvKTn+CYY45Bv9/HPffcg0c+8pH23cLCAh772MfigQcewH//93/jEY94hH136qmn4rbbbsMnPvEJvPjFL3byfNGLXoRPfepTuPrqq3HRRRfZ56973evw//7f/8Oll16KK6+80knzjne8A5dddhle//rX413velfHFlkerKYpSSAQCAQCgUAgEAgEAoFAIBAIBAKBQOBDF/5NAV/96lexa9cunHDCCcHhHABcfPHFAEZ3veXwz//8z6jrGs94xjOcwzkAmJubwwUXXIDhcIhPf/rT9vn3v/993HbbbVi7di3OO++84vLNPXPm/bg0HyzIAV0h/Ps6ovfElVhCNN9NHLN3wvSxS+Z1X4fxw+0HyNJPLx4NLoLWzDP6OhL3n4V36XeKDvpMz+nRXUCZ9G2iSF6mHirTZhFapnURrRPzm96RN6OD+826XPxN0xTR4NclVb8p1DuoS6osP172cl9q26F+ugfnThUg0k9k7OhZ7cabH7c9yXiOlkuRGjNe7PlSPkt9m5VHsTzJ2GRh6kHqTy9JphdGG3k/zthxaDLpaTtlLrM2mLaFHcdzKRSNM04BzPCWudTcuWyeaadxEJO39Zoa9Zo6ef9P8WXnDFJtGzxP5Bmlgf0YyXameeb4ODt2EvnG8qPfhR+guG056L5Olq/9/L2yHN4bY15w2nRac7pTQPioC+86dcrwCduOGZ0sqtv477u2C9Ezg3sVI/p3krdzYyeWnpnzUjpsNP/cZ7H8Sudn5f0ubG8jD8fJ09wx3RWlPGHlkOL1WYDoQk0/lcynneksfF6W6Xg6zaTI8WyuTnZeY2Qpl24s7w4AaqDa+6ASfFyvrbPjYdz1TvGaZ5w8x5gjdDUlD59cuR3WUM7rwrnE+T/VwQsRzOUdUeyJtqTQ36fs3b7JbzVsXYyMUmSt5t87xqbP0Zyan0j2LJ+M01ZddWOTxvzF7rjvoFv6ZeX4q0iXYZBcC/lt3pX3aN/MMcTQ/CvSPtxaKpZ/6d5LaZ4lxZbqjonnqby59DGdgdULcntP/h/FMuyKU52kpC3qtTXqtXV+nR4Zb36bjaWr0Dk5w5OT7IsIDj18+9vfBgCcfvrp7Hvz/NZbb12WvEyaJz7xiZiZmSlKs3PnTtxzzz0AwB4qPvaxj8VRRx2F7du3Y/fu3Vm6Dwb6K03AoQJH6TG/vcWOWVT4l98m85xEsKl0enp5MEebGraXrdILYu1Fuz2EF2t7myVBmUPm0ncfTUKltXtJOaGjCJlvOfp0BaBpC+7ScD4j7xvl/Qu4lxNzWXj1DH6bR6a/yCXPuX52LsMmfMptVETpM3wwCBWJokWN6T9K5zAxHnKLkr5Gb35UmegFtyp8xtLK8dYU9nBs+5r61eHF19k8YgfV3NjSbf/aRV+ONxQZ21TeRGRYtq87bmArf+wA7kX3OT6oMmOL1N+p5xj16DWXktez2qZXw3b3KXdxNDteiNykMtSAvcya6VOuHcd9rzTshc8cdF87cgBoDygNrZ0PDJk0dPFB33HtFx3b9FOv/XVFeIK0qelnXblpzHi2iz1/fucu6eboYNrWjnN/LJCL2334oVZNPikZB7QX3KuBcsdEk2eMFvue9LOlMzMXsM9NnYbu2PTz8o2f/DKh+fax3w1UXDY06Z26eG3u8B7ReUz6XH9zl7YDYGVsrh2d90ZGDsn3kbJsuip87+gEnB5Y821hQelPzekUZCOl04xI+3/IvCIbZUGb5GRgwdjJvbd1KtzEUUMFVDrIOzlXpMrPleeN95J5QS01H1dA1Ry41cb4J6OvDtfwxGXlBdWHPTlAZRCV4WpBuWktEU0Gdg1DeHucuYrSWSjvgnKYud753oxDI6uXFNtmsXZMtW81UKzOTPUXm948K5hfq+Y6D8tbZK3nyFBGBpWsYUw/18whLIWhvbe/4tdV5rsh8nogp5Ml8nQQ2Yy27UP1h7pdK3SSA1bejD6sltq5roi3E3zIpo99Z37XiOsq/jPSDlYno99l1hLOnEbnuoweltyfyMl6Q9qchi7cHdOKrMcobc2japBh/YjccNZGKXlKv2X0LG5edPZpKIjOwoV+TMo4IsujfZuSi94zfxwG64cUzw3HG8cB72gvn27LezdvclDL9g1dF3NrWYZ3qY7Eyl2kx4ODSN9w6WmbBLpnph9zCPo5sUfojD2Q9yndXbd6LKeboc68L0CwFu23QlRXbb+pZl7Qfe20fzVf2d9JMGu1cejl4KwLKr6fg/XKFMoVjIcuh6R33303tm7dyr4rCX1pDrqOOeYY9r15vn379mXJa5I0RxxxBNavXx9Nt2PHDmzfvh1PetKTsrQvN+SATiAQCAQCgUAgEAgEAoFAIBAIBAKBYDXjIF5kuHfvXgDAunXr2PfmAGzPnj3LktdypOlK98GAHNBxMNZPGtaiqF43RLW3MTdkrIOqJZW1/LPfLrZWE6w1HWdxwlm3MFYyALHaIJZnnKcGtbJ0rN2IFVV2zHtW4rqnHStr1uvEWKVUCC1+B6Fll+6Bt8jKWKnS7xzrXN9rIZaeWPZyVvImvdP3EcslazHb16wFoO0/zrMllmfO4raL5fUg7GiTb29RYTibzszyF80nYWXf368wWBd/ryJWwLY8xrIs8Ej13/fy1pc50PYN+ryj95xDYwEcCzZjeb2Yz4Dlc+pF44xN7z2lUaN8zJXQg1GdOO9NxysiNU45y2f/fYpmUn9Lh4a1ePY9TQzNHFiLXcai1rEo5qytGTo5byr/veN14HnKBFaJiTZ3vGiJfHY8VhgaZnePKrO4sa1MZbweZlpvaTP/1dCut7WfZ8TaOWZZ7fNJtURkiNPPzXdUrleAbjKOeVbn5oukpWpkLgl4tkI79qrWOtS0U91H6F3t0UT7j7V2z3j/WC9uakFJ3lnrT8Yy2wHxyodn9RrQE+N5YHyrTJKn8cKtCR9aD7WBq3+Y+tWZOc+C0Q1GGZBPMlb9nIeB08++XPZ5n5En1Aq4bcuwTjl+cLxvm/Jn9lZYPLyO1ol+a9474xHgowXQOrG6o5s3APQOjAoYrtFOnoEVc0aGAhGdivHCLfXE0D1dHP4vNg4cj1arlDHzEglvlrWmJW1i1yM9zfJ8r3nP6YB0bNO2871SAaBaaunkPKtinkFURo8ewPU88zxezTdOWgoqYzOIjltfhhXkUzfzjhPyl1j/pzxCuuhbdIzF1gPUa2P0Ia/f2L4bhLREvWjN+zltvR5L6LdzDdFNWJqYsek/p/nZbzN85ueZWyNEPdA4nunxfZL12LXqSzNG6Zq+ZF705F3Um82A6HlUD2I9tGkxhCYz11rvWjoPDzydzHvv5BXjfSo7GOTaxZ/LdAVHtthn/hjxaXP0i9G/QyYcXbUENlIEhQqbyvFuDb7zeZ2TkbSe/vexsUvkgW1/wg/sPhGtBxethaMzAoePGtlh5qQSL+WYLpFCvWGUqNrba8cG0SMcnarj/gH1VFRLjMex307cOGP4nbaznb/nyBrLeDtHZEy/0ZkGa+mkQPJnZBQ3xmuyh0m9Fx103CsI+tnTuag8ckDWKspvM/q9avk/Fk2kayQkB4qpMm3mGW0955w9JIYP6vUNb+7rdYtoRWgBwnUnfWfKo2skIBzDmmnLwCN6gmYTTACffzLfnnDCCUWecoKVhRzQcdBEoDUCvLernxTaXQR6PWcWIOCVTG6RzoTCogdDnDLrbKyQ9MlDHu1+62/kKm+i4zYz7eQXWaxwoS/sdzO63dhryq6WFK+w0LrS0A5+GCbttluvUfyGph/8cA/+ZpiGo/iaNqjNee2sZhf8nMJCDyC5jRVax9wmEKuAk7VPTTZGcuH5kOCT4Zx2DxB8eojCxm6q0s2B5t+lwzQbCs6nBwAflotZHFN+qkjYMz+M6+gDnndT+dNv2feRTQ4OdgGwBF4GxDYSDU9x9xp4mzAAnDA/0c0Ichjl90lw0Nv8rgatgu7IE0/hrfvuYgHw5JIG1EJly/fL6c0ru9CPHUhzC+42tC+/kKwIT1GeAUYbkeZ+v8rbvAfg3B0TPaTw21zD5WmThTns8GVQrP8ToBuAPs/Tg51RP3HM3f6sIqHwgNH8xfHf0mGhEKxN3w3b/Gn4NHsvAzkY4RA7NE21Uz2jHblj5gVlNop75LCmjiz6KJiNpBhPA/wi06GZA6FDM/0YGHAwi1duQ4WW7cgob67z52xfnvkHH344TC6EpVM8M15j/BSl2UO1qOyGfaxMu8lDwwiZtvROXupZlz7dA8tfNn9m8UtDnSHDG7HNwLbNFZRPQESumfzVEI6e0T5vxlCVmUsjcs3I4npW59vE0zl8PSk33kvDgg/XknoOyKdem/f2KwzWhxnYQ4hFwtD0p9FtuAOqyFzEjcF6TY3egSpI19IRhhZ23lcA1wCl4Swp7FzQA6rm93Am1ImUbvucGguY8dbbX7VrkMjYU15oxJGRmrJ0RDebLa1mgm7LoeHbzQGi2fgP1kh+uE4/dDEnr8z8HVYpijbMnGLvl3P0IL+tNE+HlfsazqZxzmjBkV1N+vZl+F4NycEK4XMa1pIL/euEyvQOatUBxZbvzB9MyFwa2j7Fz/SwR9H2o3KD6AfcAV40RFwB1EABhudMPYd0LgHPQOZZTG/UzG8/bSkSB9GxA8bsNRPEsAUI240eqATkUN050vbBRrA7PYfItRlBYBxgCXO/q5Zafb+ea+c6Spvd0J4lhiHMAdfSYXV43YQ3Hvx+qOhhjifPgNEYzeqZGSTDq7J82w74erY1xLZ9Q8I5R8NJ030Uur/SlE3TmLUDYrofNzaYNVY0JJ8haW94qmrknr/3kwTlLVMOCW040skMcYROxsCENeaIGKnT+S0IC87pdjVa42inH8I1cQysQTrd56Jtn1ij+Hn4qAaM8btq5YgaKFZ39vP21x2p+pXoUalQ707+zHiq5lXw3LmOiBqf7Rk9dPYIvbk8KDNSbgDt1sPXk/SMty5LtXMHoyfBMqGrbjABNmzYAADYv38/+37fvn0AgMMOO2xZ8lqONF3pPhjoqJIKBAKBQCAQCAQCgUAgEAgEAoFAIBAIDiaMh2/ubxo49thjAQA/+tGP2Pfm+ZYtW5Ylr0nSPPjgg/YgbhK6DwbEgy6CwNou4yHXyfrOC+sYKzOXvxNSkyGP81oLnjNWhRS+Ra9vrZWkOfaOs9wyoBZDxosjYxUHePXj+oo8Mp5z0e99b7GIq3fSA4yUmbN89unzy8k+o/kwllmxcBNuJmFe1FowaVEUscRkf3u05egB+P7keMKxvuuHY8N3xx+LdxPlO+2QaW/OI9bPi0VqguV4KBfOTsEJv1Nqge+EKqN84suPjIW8X74PGyYnQltMxsVkrH3NWLhTy3HHk0DH0+RknFtoPE3A4ynL6wi4MRWVVRlFja1rynsTkbbg6kytLhfKGI7lk5x1uf+5sUhmwhAFNJbSlPltn3XQD7i2KpJHfhquTF9WpzyjI+WydU3VPxKGLMdP7Yfp+udCUdIxHPwGXPlH579Cy+loGxXO6dDpMlh9JtYejNxzf5fJ+hg/Z+dtWqZPdofFoVbp77uMEfOc854DqDeZpxcaB4GM51ySJur9MF+lLbcj3nOlcikng5xvCW/beY/RiWg/ON7UXjikEppsWhqCt6BuLP8T3kiFX3e8qGP5x8ZkMlEin15YZtb7I0KHI19Tct1Pl5JdVJ6S+ZkNYZrQ7Zz3nIwvkIFc36ZC28e+i3oRZTwIknXKlU95nxkjsbKT/RejfUoba1kZEaOZ0Rliewq+F1GUllKdyF97d1xrJvOKfVYRuR/Jk3r2sHLQeMckvKJjdMTkKh2jxfNCpJ2L+99Uk8g19kqMkihSMTmR+bYE0f20wj5n03p6SDJ9woML8PRUMkbYSAiJtVws/+y35p2/n2XL7zCXl86psbZP6bv+Po6v32mP/wrm0uhabUz5n4ymlqln9NoBTo/NtVlM983pH+Z5Yr4Krk9J9bl4z608ytSmqeC0004DANxyyy3se/P8yU9+8rLkZdJ85zvfwdLSEmZmZrJpNm3ahGOPPRb33HMPvvnNb+Lss8920vzwhz/Ejh07sGXLFmzcuDFL98GAeNAJBAKBQCAQCAQCgUAgEAgEAoFAIBCsYpgw37m/aeCss87C4Ycfjrvvvhvf+ta3gvdXX301AOCCCy7I5vX85z8fVVXhK1/5Cu69917n3cLCAq699lr0ej284AUvsM8f97jH4QlPeAIOHDiA66+/vrj88847z3k/Ls0HC3JAdwhjWoNt2dDRIyFIPkWBMs28ljNPgaAInCXpIYppjaNDejw+RPpSMAU8hMb2oYiDrSscsjLrUMO4Y2pK4zHXz6uWD5j658bIVMfQJO3/cJWjq2AO8e8rT77vmO+qHSuChyWSPPkQlEHTCpU2ygxR+SDjXPCwxiqYxwWHADRGbpJFf5MXNzs7i9e//vUAgP/5P/+nEzLy7W9/O2699VY861nPwlOf+lT7/N3vfjdOOeUU/PEf/7GT16Mf/Wj86q/+KhYXF/G6170Og0F7kfib3vQm3HfffXjlK1+JRzziEU66yy67zH5DD/Y+/vGP41Of+hROPPFEvOhFL3LSXHrppej1evjbv/1b3HTTTfb5nXfeiTe/+c3o9/u49NJLx22WqWNVHNDdd999+P3f/308/vGPx9q1a7F582acfvrp+IM/+AP2+2uvvRbPetazsHHjRmzcuBHnnHMOe4r6UEfXkFwHHRMKAiNPVitWO32ChzA0DqpL+3JiWuPokB6PD5G+FEwBD6GxfShiOeRIKs9DVmYdahh3TE1pPHYKU7uawNQ/N0amOoYmaf+HqxxdBXNINtTXmPxxSOt5gockkjz5UJRB0xx/EVkl41zwsMcqmMcFhwh04d+U8Gd/9mf4pV/6JXzta1/DSSedhJe//OV42tOeht/7vd/D0Ucfjfe9733O9zt27MDtt9+On/zkJ0FeV155JU444QRcc801OOWUU/Arv/IreNKTnoR3vvOdOOmkk/D2t789SPPqV78aF154Ie68806ccsopeOlLX4pnP/vZuPjii7F27Vp88IMfRL/v3uL2+Mc/HpdffjkWFhbwjGc8Ay94wQvw4he/GKeddhruv/9+vP3tb8eJJ544vUaaECt+QPeNb3wDT3jCE3DFFVdgZmYGL3rRi/C0pz0NDzzwAN7xjncE31955ZV44QtfiK997Ws466yz8JznPAdf//rXcf755+Pd7373CtRAIBAIBAKBQCAQCAQCgUAgEAgEAoFgeaBQHuJyWjYPa9aswQ033IA///M/x7p16/BP//RP2L59O171qlfhlltuwfHHH1+c11FHHYWvf/3r+J3f+R0sLi7iE5/4BHbt2oXf/d3fxde//nVs3rw5SFNVFT72sY/hiiuuwGMe8xhcd911+M///E9cdNFFuPnmm/FLv/RLbFlvfOMb8alPfQpnnnkmvvKVr+CLX/wifuEXfgHXXnstfud3fmfs9lgOKK31ip3P33fffTj11FOxf/9+fOQjH8ELX/hC5/3Xv/51nHHGGfb/t99+O7Zu3Yp+v48bbrgBZ555JgDgjjvuwNOf/nTs2rULt91220QnoFu3bsWd99+PY970B+4lyDPaXoxu3O7pxZ7VQDmXBqdg03cYKVVzKWrd90ZYqqwK7eWdXBrmEm9FLq1VzKXxqgZ0cyitK41qYfSxuWxU0VP6iAWUbi73VsPQ3ZaWT991ajPvglql23SqBnSPyZ9cfG3ecxf1UlqcdjSPIvTRUA2pOjj9TNJmra97Jn1bhhqq5h3Ji7ngm9I3tsVa5pLa4PNMnUrq7NNcr6lRzSdsDmi4AMKnxXWn4wlhWzp8RulnLkyO0ldIU6wf7XuanvJ+ItTQOH2vNFA38qAauOMsRR9Fr7mMmF6A3hZA8orQztJt+mQ4Rj8wZcTK75R2zPKnjkybOp9ydWfmDQCtuQ8dI9OSK2O0mT+X2Doz8p+WFW2bDC1j1bVr/WJtT96bC9QduU91FoNx27RrX5bqK+PCk8tB8b5c9vQL+r4aKOhq9MLoOTHLQ04PjBPhlumkj7Vn81z3tO1Trky/fhxYGVi174yuk513OvB4b6HRZYxe7OmDQV4+b+fGo/ddCW9x9GfrxMi1YnSUtX4/6gptOwyB/r7Rf5Y2aksTq1t2kSv02458GuM9Vg+i6Yh+kGt/q4+b9tej9RiA0ZqM0znouqzvNQLls4zssEmGylnjZPvS138icrtUBnSRm9E27zB2U2ugnAyy39VoZRhXptcmncempy9PrJ+VlJlBb0GhbvTYWJ6l+Tt1isigifUrpkyDkjY333H8wrZt7a6XouUAts7VfNumlgag05jgdJCobmu+ycl9uu4n6w32u0J9OYox5YClraPepWfb/S52v4ipU9GantnbSX0P5OeSNkH5XOuA9sM4elqKJMLvLB90kGFR/cBAd6B1En6KybBI26XW5w7vj6GzxPY0uoxdv0x2Lus6vyTq4u+xAgndp5F7apH/wOpEqjt/5mS98z4mQ7x++PFbLsdJRx6Jbdu2dSNGMDa2bt2Ku3bcj2Pf8IdF399z5V/jxKOkjw4FrKgH3V/8xV9gx44duPzyy4PDOQDO4RwAXHXVVRgOh3jta19rD+cA4OSTT8af/umfYjAY4KqrrpoKbcEBxlIrwazrPRFYdV9HN3KCvMcQpnVftwtNjbKy6OTEpWHS23pFJiRtFrT1aCNQ97SzEajV6BtdxeuoBmq04ZQp339e3GZeeppOV6PJRHmbyIZmIHzP0ejXszTkT64OTj8z9MeghrCHEUaB8vsGcOvJ0Tc2tPeX+zxTVtEmgkdz8nAODV0N7/r8UVR3jyf8tnT4zF8QliidHWiK9SObPjGeS8pKwWzuOgdhBfRRDGc1fzgHWLpTtLNo+nmsfmDKmKR9/H5YcWTa1PmU+y5WBzO2cukpFMLFkl/WmG3mzyWWJ2N5knHC0pyhZSw+6Vq/3Lcxua+89pikTTsn4suc2h0jmQ2uQC4n9IO6r1s+YeYKP19fD4wTEX6X5ZcmjTmci5Xp14/7Y+kwetwwTB8lqQOPD+c0hnM6qg+ytMH7f4ms6sDPXF7Z/Bm5VowustbTJWw/k7lsaYPG0gbt0JTt5wIa7bcd+TTGe9x7Jx3RD3LtY/QLSptaUq3BZN1uELHrsqBw8ruwX/01TvZ7X//JyZBoRvG00SRT0PNsm6fyyWzw0bURn5E7B3Qem5nySzCWPEhgOKezeXbRvSwiMmjidVskv9I2N99x/MK2AyfjYmjqXPtrg3H0F0YHieq23vxYkmdqvET3ErrMK2PKgWL9xINaVGGZJXtHGXq66IGlcwmlaazxQPth0r0nn6TcHNBBhnFzrc/XxbROwk+xciJtx4Lj/TF0lk75c2UV0jzNdR3tt6zus6iih3MAsnutSRIzst55H+uXXD8IDhpKPegEhw5W7IDuwIED+OAHP4j169fj13/914vSmHvmLr744uCdeXbttddOj0iBQCAQCAQCgUAgEAgEAoFAIBAIBIKVhn/oHPsTHDLo5z9ZHtx8883Ys2cPzj77bKxduxaf+cxn8PnPfx7z8/M4+eST8bKXvQyPecxj7Pc7d+7EPffcAwB4ylOeEuT32Mc+FkcddRS2b9+O3bt3Y+PGjctGu+NWbMJljRHWq4vbNBcyLhpCRLXvk/kz4V1yISahPRr8unax2CoML8OWE/2wDa1kvrUhlhqYkHw2z4zgKgk9YMB9Vw3a8Dh++3H0m++4cJdsKJE6zFMrLEsIjdJQnV3yTKb106d4W8ENg1S5aYzFLlBgrdZjLCIpb3N8TviZhkTi6MuNlxidKfp1XzveFssJNnzNUAEmPJwvpxBv8+KwaoysHX2QKEe1/UhDG5rwr6h0axVIx1aCZwJ5UMjndjzPaNaKNBX+pWTcce2o50YP1YKXyJMNfp3YvEiosWRIHyBsE4730XoHqTqU034+QT8wcq8TYvNPpD9Lw22xYORZ15A/AMpoprKF+XbaYbEAXj/xx6VfJv3NhvAm457js9R4cmjzxk4qNHE0jy4hhe1DU2gs0/h7hzdSefjpMnTGQpFR2QYwMtTPNyfrFklYsjH0gJy86x9QGKzVAc2l4RjdFw3NS0w4RA8m5FBFdMxcmal6ON8O8/LM4Xkg2qa5MO2UPvqt9sZsNASlIml8mmOyvrCfonWp239pOqvPkzCtrWcPo4eNC+X9S8r061msGxt4JFZLbj/7ITatbGT4xQnZS/Lh2t8W7/dJwZjtFPYrMj9y/JGqm/MdXJ2weC1aKJcpn0X1ce8agWpJZfWYYhD6uesKKF0GsfVC7H2qzOA5PJ2L6jGc7CsMY9tbVGwEjeScSsovop/kWcKzvm7JtT8NzW+v7aDrL7P/QPSX4rWO4n/TscHtD/CZIs13XN+SdZMts0/kXcXL+GI6yG9nvBqM0bdOHh30aRPSvFpUgcxxQp7TcNORtSa7rjPfVOF8HOUDwm9JnqGhByec5krDLef2FdWwZVg1VHZsGAQyKrN2qMw4m2v1SWf92bHPnT4pTFuyPo6lMyje66Vt1vzDhuqOZlA+NovXNZEQl+a9GnagTzB9EJ2w6FvBIYEVO6D7r//6LwDAIx7xCLz4xS/GJz/5Sef9n/zJn+Dv/u7v8Ku/+qsAYA/njjjiCKxfv57N85hjjsGOHTuwfft2POlJT0qWv3XrVvb53XffDWza1KUqAoFAIBAIBAKBQCAQCAQCgUAgEAgEywYJX/nQw4od0D344IMAgE996lPo9Xr4v//3/+KlL30p9u/fj3e/+91429vehksuuQRPeMIT8PM///PYu3cvAGDdunXRPM3B3Z49eyYjzlgQEqv94eEDVLv77XugvacEocVVkGXGehc6bc3gWHsbq8Fha93Doe4D1YDJkyQJLEE0sQZcVNBNlbW1lGjT1LMaMBZj5L2x7NJVmxdrfcLQrGo4ni6jh/zHMSuW4RpDYJve1L8aEJqa99WSshZL9axuL6BvLHc02r4JrMDQWLyYfqC0NGXWM9paAem+dq1SmgLMs4pY5RmLFMd61lhgUSuZHloLPnJXohq4dI6IiVhiZizkiy3ziTWXk8a3EM5ZBJE2sY9IPqwl5qMOYPiTtU0BlPj239rUu1Z2bFBPDc7rg5ZPrbj9tlRLbj+ZNjF9Uy2p0CKXFtDBo5R6QFlaifWm4yHGWMjb7zwLL67NqTUW501mPCWqRYXaH7tg+sobQyaNlVUx63LG20tpYkAYsfD3G13VZLwQCzzjeVvP6KwHQCmoBXvNlGksbnVPOzLKgMb8d+rGDB3WUtPEr6cWq72wrf06UQ+EUZ66vfOoF3pqKg3UjdxVC6qdD0ibsZ7ZJk9OFhArXudOHzpeKZ95XiPVomppGrZ1Ga4bFUplpXMXXgdF17FsNqJlxqUjlicX2z93Mbfuaag6tNK231XtHELvbnE84JkxGi3fPK/D546M1GHj0b6j+djySR0cjyB/rlhUrWcuqbqdxyugWgrLaj8kRWlYnQZk/jcW/Gt2VFg8nFhMA9Azus2CsU51+JDhB8CdA0xadk6lMobR0+xnhDcceWh4gwjGmJeNaXMNIocY2aZoBABqeWz6nxnX9axGb97oPM0zzzuS83ihPGXos7oTqYfV8ehzHZExtBzOSt/KE81/a/6tEd4HopAVF44nf8r7lT4iup1tk1phZvfo44XNbamtvt3OIWa8YMhkDq9NGHlK1xvWmt/IE1qHCO2cl7NmeEeB5zlHDzM0mTliSbmyvqkrx8O9BWBgloxM+XpGu/NApF7Ki4IBr050DLIegsy6webll6lJXW05ZDwrkgUVu6bvyRrNmVeNbOjpUAZrlxR/DURpMeuKegaODs2t21re9cYBRv/3778c0dem4WAjo6yp0dtf2byKvSI83cQpG94ai9F3qZ6lvUJ6C628ozLE8Twq1SOZ+SPqMRvJK1kGw5PcPE/L9ecdU4aRDZRfzFykSZQK6i1n91Z87znDH81/HW/pmnjM0nvD0D5L1bkmXmCOVxy3RqHj0Z//dDsHqSWFem3t5Bl49nh1gmrzrJZazy36np3fnbro4L2zLjTimolA4KShcqX5bqQ7moHWprN09nUwF7p6SKh3sH3nVSq51hyGupuT3ptfOQ/DIM8BoNeSunltUs/oVmerXBkcQIc6neMNRXiTyl9HhvrzdwRWdnlr+hLZorTrdZ7aj9IKrdwn3uLcGsPOVZ7yxcll6nHa1rl9b6AVwkgMpJ6x6A9+nRy5QMcjXXPTcv25jKSh6aKeniZ/QyeVB3RsU36xvEF028ZrMJgKvXklR5MT9Ya2DTevGVQAhm16H1rROTunBQuWHdIFDzms2AFdXY9G/GAwwJvf/Ga87nWvs+8uv/xybN++HR/72Mdw+eWX40Mf+tDUy9+2bRv7fOvWrbjz/vuDibj34Ex42MQtruBOGLEN0PZj8rMgXJemi8uYUCSTQhCyxtHwGLoU2USiB5BEkbd5MgtbXTEKN81+qOzij730mnOljmy8xJSkYAEBdyKzdNMFBlGiKrOxxE3IQKDM61iILarYMgeVXF3YtqPKGhe2Usf7IkBuIo0oHDne5MJBsuOkAQ1LximVrMKQIb3+77WOEsgtrs0mRTTMQaIMfzNBeUo0HY/O5jizccQu+COh4Axo+ByqLLMH+rSfE+EOYgcz7QOebvp/w3vcgUB0jJLfqXAMQag38x0dwpyMcxR0V96E70cYrmkW2d7hE3sgEauPBxruiAuTo5338XyczZQCRSw1dujhMZuns3ETyi3FbPpSPmDp4OZKulBjQqHA73vvtR9ezW+/elY7fW4PSUx4Oi/vXFiQ7PhhFk1siKmEjIrNDzZvbjOZooa7kezTmUtONrCicy03trl5xehO88puZrHzu0q3fS5EXSw0IDUkshtnNewGs0lDx+j80bVzaD76gBITllMN4rILaMZGYg5w8uTmd4T5081t+s4xeuHmEHIwz4ZDZmji5hIw49LfHB2uDfkwqb/5c75mximV+9yGBNM/7CEJg6jMTLFfTp2qEPKTB8f4i6Qz+dNNkIWjQp7kDJ1ycsLk6YRfB8Nnnm7cviB0cjLOr4fzAdp+zKyLRnw6+m3nTyIvAH5sGQzWEz3TfEfnXGrQRuWaXye/7zLjMziM83Uers24sZE5pHLyzeiP3DiOhtTzD6JJGnbTk+pUnEEYM39E9Z1IWGa/Tr0DFW9sUqAfAYl5ntLvb2rSNAydSxt1y691O7boOKhN2HGujQn8kIOjjMrqUAQ6VXDGlFw7RuS3U5fmJyeX6Pq2ZE8EgD2cM/Sl9ORce9ADCWddnRmPrEEl2Yep5t0TNj88pjXcpGE/qdzym8pbn0bXHojzJD3YSqahz42xJDUmNTST7xR5z+XJrkkVb3zI7oGR+isqV6jcZ/qEjlOO/1g9ivIu0w/s/BpZ3/r1DvY5fBnpy9AGZg/OGS/9Vo+tyQGXU36C/x3e4+aKjL5v1vdqSbkHc953YcHhe0q3bcuccTJtR0Int3bw6xTjTUeHjq29EPZrVicM9Ac+TfQqkObb6oAZBCQNM9f6a0Wrfvh6iEcnt36l8p++Z0PqUj1lkrlIMDHEg+6hh4IjoeXBhg0b7O9f//VfD96bZ//6r//qfL9///5onvv27QMAHHbYYVOjUyAQCAQCgUAgEAgEpSUnVQAAnN5JREFUAoFAIBAIBAKBYEWhC/8EhwxWzINuy5YtAEYhK48++ujg/XHHHQcAuPfeewEAxx57LIBRaMx9+/ax99D96Ec/cvKeJnRfd2bunDXXRNZvHcGWFbEoLvG0cfIAnw9LR08nw/hFQ4GOIVii9fAscbJeRNmCCr4h1mBd889ZyE0DybzGEeq5NDFrrRJ6Yll61tyBBZ5vRTXhZJW0Vivs5y71ZMMArlJMizeLrGwj4zgFLrwcDT+X9RgaAzlPIPvdpPKvJExqqZU058EWa4fS8KzMGB2Lt8fsD9b7gD6PvOdCwRSXRfLQiffxjMqfx+qXzQtpS+2uMHTQcIQsSBg7tk8jeop9HZG1rQW4csZw4OzijfXcOC3VF8aSF4Xtn/S2TSDLu6XDkPOmok4U3dXlACwvGt1tqOCHlyvJb+J5yThKdJF1mXBVpR5+AO81OQnqvo7qwWlCJiu3k96TCVvLeeRyXp2sR+yYNOUQhHfzZFTW0ycTUj73npabKm9aHlhOmSRE5FhCIOfRQZ8th0qc03MI2DB8tk944rJe8CY17dspzstsWR3X8enMppDHMuZdzPOZNimVm46H8jgm8ROWP05ZWS/ocbPPRSPh0uT0dbTvp72nltMts+lrvi1z+hv1Umrr3+qxbPSDDFLRSNIJmzIj4bKTeZJDgXH5lW1/LsLAakbHtaLv+Vbqbew/y/FuKmqA/3/2SppDZ0vqoQ/pi4ccVsyD7ilPeQoA4MCBA1hYWAjeP/DAAwBaz7lNmzbZQ7pvfvObwfc//OEPsWPHDmzZsgUbN25cLrIFAoFAIBAIBAKBQCAQCAQCgUAgEAgOHvTosLXkTw7yDh2s2AHdsccei9NOOw1aaxvGksI8Mwd5AHDeeecBAK6++urge/PsggsuWA5yHaZ2mN2gXgEzjhLrSeX9OR+EacxdBPQSVicfjKyCnAua/QFfJWhTkfZrUBOfTucbjv5IOUHeXpxlR1Bp973SXv3M86FqLYhSbcrAWGFpVWCNploaWfoLyqIx22msc2XiRKfyo/0c6SOuzIOBXDlZqzzKe4wF21TrUSoPOlqSt/Qz/DhO/oyHT7IdEmObs7DL9hkdh2NAexeAl/ah7qO9cLxpPzpenDbNjJlkuTm5y433CKpFZS/nLqajaVv63PD/uF44wQXviXYyNOk5zcvlRNvqfpvGn4OAAsvJyHtOvjv0RPhRV2g9b31r3pkco4/+ovNrIVTNp9EzOk8DwnR+XmN70BXOhVAR+unY8xGZC505NTbOFFzvAzXSMepY7AhPPyntm6792AXOnRuZfmZ1l4weyNLty6gETWx/+uV18Vrg9Ey4fVLC6z4on3D0B+1AvrXvarT3bZTIbcJvuW978xOuIXw9NlGeoyeqtr+CdP79f17fxCz5UzIu4MdctWr3D/B0W38TYsx5LVcP7nl0Tvba05f7qbkSyMjDyHtalv1Okf9n2juIHACmbf15uNEHszLEf56gqVpSNppBqs04ncDNKD0OonxI5moDrm1Qtc+dNWVOX4zpHyl0laN+Ou5VhK856L4OoxyMU09651LHORbAaI0Rk1VevhxS+nOQD5U3TP/bZxXTPqpD/RJ1AZox3eielA5ObkfnsUi5UZ0pQWsRH8Z4rll3cPqXntNQA9XOKRPKyORalbYTo887cpvyATcOEoh5Tef4op4Z/Tk0eXdb2/Sp/TYnA5T1rf/Y8NUA9n7NUllE+8eun/xv/LEV6/OmzLrvyRaOT0plLNVpllGHd9YtVZ4+u7bh5h0/39ScnhojirzP9CW7bqfl9PO0CJYZuvBPcMhgxQ7oAOBNb3oTAOD3f//38ZOf/MQ+/9a3voUrrrgCAPDa177WPr/00kvR6/Xwt3/7t7jpppvs8zvvvBNvfvOb0e/3cemlly4LrRVZiLKCMxPGZlJwk0duIWrdlmODMxbqhUyktp5NeqXdze3UpjCfeXrioXWi37ATmtks8Yvw8nbS0c0uuuFK0zKKhO7pNlSFaQtDU2YUOZtVsXah771FdhehGmuz1Oaym0H7ni4AcmV2VW7GCWk3bigDh4fNM2ZTd6ohjgrDGXYNNWHHDsOPHGIb+mOVD0THHMDXORv+ZcI2pxuEsQUAC1oP037cM/83g6wSHSRof5oxWpK+ntX85dweHbnDLEce5ZD7rrBt1IJy5Yk/N3FFD9o0zlwWI7VQBnF8ojK0jBImaM2FreLkaoSWZDbeJg0tvyR0VnSTKfK+WK6XKv86LLOo7Wk5HpQJkZnQQ+xCVCM8nPY2B8y32Q0gQn9nGdABzkXymX5mdZeMHsjSrSO/m80gZ+MlVvdxFoZEzwxeUVkQMxLI5M3J26g+anQxboxyelqszIbfct8O52LCpf1jacnplhxZkfoGz2t37PgyLKbnxDaS7cFHB74w6w1ad249wBnBsWuMSduJPI/OyYQ/gnYg76NlkXHA0ptLz1YmTGMQPZhR4Z+TT6MzseXGNiBJerYfiW6bajN/8zyo0zDdJjE+LN3c5daq/jiJldt5rugqRwmNsX2CGF+z+dCDE5+m4GPEZZW/1k61BSPXdU+zY8uvQyxPZ6yWzhs51My8TNdqOf0uoz9kdV+NcIx672Pp/HdZ3i/lw5g8aXiCNZBdUNl1rbO/kTMyTK1VaT6MjOV09NEhFTMOUsh8GjtUzdUtN5fwiTLfRd5TPk7xcmwv0DFeGEPH9+edQK5zfEINcHNlMeuyScAaDRA+rAb5tqR5TXJomOIjZcaHWT958pCW67QL14+xNZjgoIEzpmINrASHDFbsDjoA+LVf+zV87nOfwz/8wz/g1FNPxdOf/nQcOHAAX/va17CwsIDXvOY1eOlLX2q/f/zjH4/LL78cl112GZ7xjGfguc99LmZnZ/G5z30OBw4cwDvf+U6ceOKJK1gjgUAgEAgEAoFAIBAIBAKBQCAQCASCKUMO3x5yWNEDOgB4//vfj7POOgvvec978KUvfQlKKZx++un4rd/6LVxyySXB92984xtx4okn4vLLL8dXvvIVAMAv/MIv4E1vehPOP//8ZaOz9sIXAAitj4glxrTBWop53l8sjJUQCdVgX9WtFQelnXpd+emo9RNryV+7liFcm5RecE4xSZsGXgMmNM6Qf5+FaQvz/3oKfW8yi/BWMZj0NOxNlM5IuhJS/LA6QXruXe7C4Qo2nALHJ1yZufHgh1maZD4bhVIaEWBkgz+eSnih9Luu3xokrVMB1lptXBTxGRBY8eTKTI4t4jlgx/OYfuEm/A21so1akZUiw2T0susccm1KLfW5uSnlKeXQ1NetlSgnD8ZpB0qTyacHVEvNb67P/LkqUy5nOezMawzfce9jfVLKhxxiabl5OYWoDKDj2dAfmZdy/Ujfl4znOLHNP8R6NNbOKTpz5VrvnCYvlhQvPJCTJ01T83ySQvQ96Qdbdhfdh0mfmqedfGkdpigjufRd+ClfQPMtSeeEMgIp35MnJZhYhnvtn5uTu8wfvu7Mlc+2JX3PtFWyTE3KVGQOpKGjxwh7O468iKap0+9TcyirJ05xfTaWfkDGI12LOPULp1+23Jhubb1QcmvVpnw9Zh+PU2cKbl0YhJsEomNgEt2xZOwm5x1mvaGG/BrArBV8j1OWPpOe5GXSUe+dkvbPebx06j+U6WH+WtZJ02XRFRnP9vUY842TPlW2x6/ZPGu+f/MFjcoK9IKOe0sp3ZZLF5trrGe8DnXTcdafMTpLdGj/26LyKZ95+mfSQ7EgT+3nXwCH96ncz9SJmwti5U6q042F0nUhldHjbriM0eZ+O0e/5dZ/kbViKXJ7tXRf1ZEbDO9m227cNuXozNQ5tz6GiUolWBlMwAuC1YkVP6BTSuE1r3kNXvOa1xSnueCCC5bvrjlgJGQqxDe4mAnfEbol+aNQ2TX5m0MAqqBz6elmDFHC2EmjB3YiS4WIcZQAjfCwq+CONWexktlIHgfZyXWSjdjE90l6qALObbyBvAezYGczbunhDtMMrxSFW2QUAU6x5ECVSGdseAs1N1GGHqKkWBIJb7BtksnTV8KSiNWZKOvaHOR03BTzaerybVcX9eh4KtxgLC3Dx3DDSCBU+8IYcX559dpRA1YHylcYHL9HZY/P21RG6vAzgMi1QXn7jLNR7cjNhDIe3bw1STIywl9c+pthNH3FhNDTypOXTJtym0ipua4akPcMH0YPdsbgV3azQhXyTwm4BStdlOXmktIFew982B6mfoopt4v8cOStGcaxcGIJPanutxv/qtLBojN7wFYrIDWHeUZDqXoA4bzvGxakwnL55dr3qY2TiG6V23y26WmfG5rJJjzlMyeNdtMA7b0mzoEllS10c5hpS+eO4JyBk8nTlJXRIyitTp+YV2QjUw3bF8WHz55OUnw4TvvJmzeyRnKEnxw+TB2+Jso3xiPV/CgD3dOubCN3DpfAN8KrmdDjyXZq1klcvobm9mH4HR1vWdbIjDc2zaQbRrHNUeU+0j2kQ5LR95rvHxq+MzdvJA+OZnR7qEn0C/Zgq1SeczqTp6M7zxGXQTFwsik1XhXhg3pOB6F/g7HpzzveWtZH8UEUfeSF/rVlmXGlI/1AyaRytfl23Y9HGR94ZNthY/E2palgrmwLG6Oowj2BoMyELqDN3gwQnX+yBx6Fz+qZVp91xoFPmwbqudp+69epXlej2h9vBDr/Gt277munLIfnmTr670d5kGd+GNkIOdVCm7HV50kZpXsSsb2pFBvl9sNy49XR3XQZ/43uNGz1KM6Aysmn4zjw9eGgHyvvfeU9J3PB6KP229GHcA5Vk6C8m9E9uyCpT9fl9EWNGDP9GKxlc2tpMm/YMJhM2HMAyb26FJIH7kTuK3o1Ep1f6RyKcGxw67hJdJ0iYw9mDWgN7Ss90vPHmCcEk0NFdMrYt4JDAyt+QCcQCAQCgUAgEAgEAoFAIBAIBAKBQCBIQA7eHnKQA7oYPAsXGtrRWurNtNbgNBxFFJ71y6TWnaxFj29N61tiEOvEei1j2aWA2Z2jZ4uH1+Fpu3+JsPeehntApUPrmhoYrnWtzRzyI23Ce5+opm4uEYE1mWf9WfveiJ4FvvU8m9FBeidfwiPG8s2m8ekhlpVs6EZDCgn5FrUCb8qmfGQ9HYzV31Chmh89G86g9eCLWU3mLoZOQOn2E8cTIWE5pfu69dShY42xGnToYCwFk+1Ey69IPTkLetrPsTpTPkiE4JzZU2Fxo9uovqcG0HiXmHwyVqQ0vR2jm2p2HDiWoGsaD7V5hlDPO8XyD7FGdr063LKo1d/MngpLTZ2N51yJZZZv+Tx62JRZAdV8U+YsKZPrMwJH3nEy0vTDQEE3VmyDdSPae/MV6qq1qrR1Hbr8GpTJVKO/XzV5k77hLCUT+QKuhTjbphVpR07uUmvzWW29FbnQzZr8dt5RC0n/dQ30mjyHa0hdqUcP+RYY1aHivHypdT+VJ1ydzCOm7atF1YYrJfT3Gm+u4Szf4KwnINDOa5w3V9VaH1tPCLT6gePZ4s/RpE7+nG4tqq1cJ2mpRTNpG1aeRMB5SmgyNtBzy6XeXJyHPuflrIZoPQhIXR1ra8bSlLUy5VCwMCm1oAdaWmg7sh5Tho97LU9Ta99ouMnmGRuJgZm3evMKwzmvkqRNqDykFsE2m5k2ZG3/AOF92nd+nahXHuO9y1lrBt/5Hi/UmrpG1jo50D0VbPvVs6087O9rZOx6nZ0Xsl4JlCZqkQ24UQ2oY3hKbs+QelCvRapvUp2Dk2PEq6LVWcmczek6tqA0fdCtp0U14GVsoMeR/OrDl9B7YMalw9OtTHo2nHGtmI4mIHrwmh2jhAtH1E4/GXldLbRehX545KDaCdno8DGdi9A+88duteTWy7egr5ZaD7FqQNYYdA5i+NB45kSjCzBjyPF+JfUcrm/0m31hXj7v+/NWtdRWlobPZL3BjFw5YgnqgZn2mR9pQrffqmGrTw/XMXVmdJ5qUbXzIuf1T9Z1QNhnzpqd0f0V0nwCT0aavH3Pf1OWocl6NkU8Li3PLCirj88fPUYMUsTnuNh30HDWqgDcuSSSDxuqlZmLqOd3bP0e6AcDN2Qr9USl37WFeO/pKzIH2eQMb+kZzUYt4ubyeqbVfU0aOn/SdVdO54hdpRLMO6RNqwHRcwlPGXnrhBmkYNqJenoG61pFPi2YV4IIOBE9zybxdF8zry9taHVQxxvdy6s+cgn9n84CGOk3vjc564FUtXOqJusmys8sH0f0SZOXlUtUR+6FvKtqIpf3VkTnaOa0oXL7ztNJoNs+U0sqrVMxbUvXpbnrFmib0Osg/PnV0WcjET+49ot6nJrnJNKOsydh2nyu5d3UHqsaqnYPMKKHUZ2PzSchY4Bw7NTrh+2eyNEL0PfNjb6L7EeZdbGVB7E1yiKRoQyfuImYOhh+YqLFxfR5J0saSrtgHSZYPohn3EMPckAnEAgEAoFAIBAIBAKBQCAQCAQCgUCwmiEHdA85yAEdB42A2R1r/cY6xrVcLTAh8C1QCyzcDayHlmcZysUe14x1B2etx1pl6pHnXJAXLZNa2nDWM9QizjeOIxYvHGIeN+yzzN1qXDtQLyALL75z7XvBMXUYEZtI44NaIXLGsDrxLkILax1ELEmHa41pWSbLnHXlGPdf5YZDtcTE9Uc6TepuRMDjHS4fz/OT9dDrgJT3i+89B/DWYrG7flgLQ/Ld4hG1/Y4bB/Rb1nPOfOdfWp/xag28VUnWS4fV8bGSQDAegdbCcEjGFmd9GumzrBeRGRrEYtXK2J5mPWrHgWOhyPE7Rcryl4xrtk611zeJflBLKmjT5G/v/7HxnrTgilplcnNp84/Pth15i/NmBuKec5bnIv2dnG/q9BwQvZ8o1WYKrucZEFgZx+a4GALL7ZQMjHnpUFoy+QOIW1fS8ZyYo4ru+crUWTG8HfPa42RHal5xvBpzHrGF/UTzGsa8bDL52DoT2eHwaKHXPK2TaRs1UAG/F+kupXejkT6jbW/nTVqnWZIwNf/Hhj1tPyY6hO+1GnyXQOwuYfZ3pB9rxvs0a+2esfD2abRpOH09db/j7hlWhrDeCmPo8KgB1aRb3BSuSxQinjgdxllAU8ZqnJM3Pu9zuqHDB4Xrhey9vBnvQEcPbLwSou2R8nSoAI6ZkrrrbrK9EIu6wng6pPRVCupVzN5hm7gTEIjL6uK7aDnPstj6lbnTKk5Ykz2V+5k7YKNZFa5xWBnUoaxsnUw5NFpHyd3oaHQgf+8EiX5KrZ0iMs6/U1INFB/ZgymHvQ+VlFEtKl7PJeWz7cfJMKZP6r5mxxH1NkvR79MySpT5PpNnVDfssEYYrPVe++t/73v1wExWV3LSN6gbj10uakyU1Eie/prDn0u49jUyXhMvY1WH+30mDx/Wi6rDPFdzO7+ZOjuewaSeyXIjMiSVxo/qZb6lc6HD51zUqoR+xN2z6+RZoFt33Ssy3nMARt5zfrt484b18MuMs+ias4Q+nf6u053sHdtDMH2IB91DD12G4MMTjBDTyt1AAMqVzdG3ZAE3AR0cTZ0Wo1x+Tb1M3ZQf0pI8M9/5gsF5ViHkMu6ZIckLE1N68SV113Z++59ptOGnFP9trk7+e6WRrBNFrt+5slN1Udp7X0hHFwShOBVTf4Y2///cJmkqTfhB+lunbRXatmB4o5ivUvDHJG2TTP6mPfwN+BxsO2bkgZsI6UMhyjOl+XLjJrHBk8wqMc51D2mepvKC8FhWFjL0G7mgFZHRKp0mB1o3Vq6Uyjg1CntRFRwWOnKZ43dVUG7T5izN3IEqgMFajcFa92PbpjG5VyugVoE8MTLV6dO+DsYLKy850Lbg5GYHsGVG2soWz9DO5uX9Dvg4QjPVKVL1i83Zfl2i4y0RjjCWp897fjmUT1nQ994cYGVoas6Plc/IOpYPc2Boss8LeSyrRxla6ZzG0IwazmadpvK9i14Q08+aPOnGQGk7BQt+QiufANG5yNks4sYWt4GWmWv9OpTo6VGeJ2OwntGoZ3RUbhqY8KNFZZG2KebTTJ7JzSBPx1VDNfrzx5aRr2Z+UbD9TNdN/vyUNVoB7FwBjA6KKxJinAsvOQkojdy4NGDn5Nj8yvF7pO5B+wDdZIs/10XSOPMrx++JMZpdA9Q8gU45hfWgv2nb2Dkgx0fI6IQRPTIFEzqepimaXzPrUxYRWdh17JesS7g+j+oXuc15f+8gUgeWliHYEPW2H5m1aA5V7NDNb18qw2Ig5dd9PdpUp3Q16YdzOs17/jgF2vCpjJ7ZdX2f5Hkvr6QektNlKtJmERqdecE+HP3Z9ZZ5V2k3lHeun2PrlcwYqRaVPeSyoLxLdaqMTsDuDRV8Z+hvP0Aoq5m5NkBOpzKfzWo3/L+Kr1EsEvpYkqbS9/Q7A65NUusEU6XCdTU7Lktp7LJ+JN9qbrz7eTVjk+XNTP5d3rP7HCYJ5eM6wrOC1QNd+Cc4ZCAedAKBQCAQCAQCgUAgEAgEAoFAIBAIBKsVXQ7f5JDukIEc0EVgrPKMxUBvn7LeAdqE1lhQ7vfGujRyKS0NawcwFsXmPXfBKRM74cSn/BB3fuuxNn+bJzcAw+To71MYrNduekW+MVZNlAwAa+4bEb5wVO18a2AvyB2okBaSP7XG4C7rpu1THz4yl612tSxLL61NCZ16be2EEZjZMyLWhEaiLu9Ke/kaOshlrqYrTGiE3t7KXnCuPUtMgFj7IbxM3n5LLLTM/9kwQWjzpG789j0J09drLLKHa9MX+Y7q2qQjl0SbPBW9hDphEebnay+YHyAIORB45XlZq6FyaPE/sB6QQBsOaKFNoxu6KWjoQlOGeW7q0MUDVXF1YZ6xbWP5QQXPAGL9pfh0ll9IO/g8A4zGDR0zQR1ql2aOVtpO/sXbTv0YK3aW16ks1AzvEdiLzuGOSzaECgkDZMdDrVA/YmGU7n4SC42RhxR27KzR6boQa9BcaKNo+L8GRq7OHxWa/qm6DXnEhUXl5IOlD6O2UcQidP2PR2XtfWzEzNCTR45VJrFqpe1IL/EO6OfCxAEYHjZs0/h8pN3f/njOjVc6F9JvTTtWiyTMbkVo5MY1zTdmjUgt/eGORzvOdZon/LHhy80jv6lw/2mMVbFn/W3K98vJhmAkz633FRANYWXbtBEx1SCsk+4RmgbK8o+5IL3uZ0LS0tA/pH1yMpSWT3WzYH7vub/NfNDfO0pUz2o+vB83L5G+o+3IydU2o/Zb+p2heWaPsuH9jFcKlYGUz7k5Vy3xIbZUjG4AiIRent01ypSGQefqwep5Hu+zbWL4af0QvT2hqbqVZ/22z+h4RkpWDxUro3zdy9BqeLK/b0TU0oa2wWojt/b1kmEGdUXkVqV5Od28HqzT7JjsNRbUg7WtUma98XQ79mg4zfU/bOT7sWkzcl2N9CYAqOeY8isApqyavDRzAO1nMoZyaxB/reR8RsKnqhrY9N1Rwt2Pa8jwPLJn9jbts47QZIrqa9v+HB8mxwBC+Wue+bznzMncGsKXiaasQatbcWOfrgUdWpj25epC62TalFrks/MfwrGp6tGYG1W6eUd1cCYtlSFUbjuyXjPpqNwk7RPUrfL4h5E3jm7m6XyO7saFq2RA69HbXwVypp7R6ZDoCvz6nlkDBfLUEtg+Syx7R/RsbNbNJtwoaXRHf2r+ndlZYamR7e4aIF6mM9dVpH/IGDB94rQP7RumIvUckXU29F9LmJ3TvD2P1LziRDqK1G9UOFDPhPRzYfDqPtDf36wXCM1Wjzig2iUsuR7FkUe+nl2T/YV9VRAy10cwdujY0ETnYurMrS+5fZi1P62w/zHN/MnoeWrQZqAGujhEnrOWJGPD2UdC2M/Wu8rwxpK3Z8CNKVMO0ZeqA27f+aBr3mzYXK9NaqKnVIsqDMdI1yhET4vR6uv2dR9274nKFpYeOj/1SdtS3Zz5llv/sntPXKhphximTglRCcBZy3O6K/WAo8/mdpA9SqYcqycvtjS3ewahfhDQaXUeps7eeiVIrzR0z2NOwq/QLV3DRl9VSyqYv0Yvmn/8/UKiJwdpTDXX1ujta/ZFe3EdBfDmevsQPE2CFUFuKAkOPcgBXQT+Ame4ppVAXGxyZ5OE2ySiwi6mbKXSM5OeOZxz0sQEJfN8uIa8ppuilL5AGwfmj66d//ug9+UFh5E6Mml5CmaQJ3PIUHqvj7NhrIHBBu8Db3EWKK7expKBndzo5MUskunz6Kavf4DlpWUVGa4scthk76Aj6enBj5NVr/3Wz1PPhBs3gVLv1V+r+ELclp/oc90Ly3Q/CNPVs24aWydf0XAyIf8rmeHIoiA15riNZA7Rzf7M4oYbo1xeqcM5m4aO/Yzs4cYctwlTemehVnDvrPS/ZcZW7I4YdsHZ01AP0EuKwnT0W8DdIIyN/RR9FLmFiJlL9KxmD+ZsOaqlOTe/sM+9zZh9xxTGAUtsPpt8Ddj7csgBD0djby+RoV6e3IZjlA4GdNFCZZBi7nOlizKfDptfaqFIF5IMfdzGjqERaMsO0ng00MM553s6hyX4lTvED75hDFNyacxBOn8wQuuvg7kueg+nQWbzNNio9csnRgimPId2stmkBrBh9GIbNilaYu+S9cvM74P12jmQAfjxAngy0G7sEPnNbGwU0wn+YI6dB5nf/sFNqs7V3l60rwE48y+9gyVJviYyymmf9j3XPkuH6eB9ZQ4PC8eSyTNGF+BtmBPYjcohXyBXRrF812QjnJMNNUJDKVIOK3dyfKC83z5J3v02O0/RQf6Uz42RIXuHGtl05GRwdgxwIoDqBDH5mJKbkbHJrhdytNDXzLeUn7l5j85f0bzgzgVOGQyd9ndk/sjqKvQZ5T1/Po7MZe1ayKPZX8N0DBXo08gZ2mTvKy5Yy3Fg5UjBpmi129X/6X1u3Ibu0uF1dq7yfzvrvwzNzt3jEfrtgUhsnyXoSO+/ifZ15iAqz/xQj3DXr1aP9A7eze/av9eYKK81Mfrh6FQMbwPkbjKi+8fgjDnAlVGRhubayTk09PYMDjyyTt5HOloLhXNF8I33npZD8618eaXhHhx5tPh7BlFDeJpnTfZJIm1cer9iLi1rAEPXDZk7VBVZo5jXDm0x2cLkWSKn7M+UQTnVI2PreG+uNOlKyo7SlJm3nT1KBq0xETMuc0a19Fuuzkw9Hd7WyjlUNd9xh9PGeCrqeBHRo+wBY2K8VgcqXmeLIVG+YBVA+uIhBzmgEwgEAoFAIBAIBAKBQCAQCAQCgUAgWMWQ+wEfepADukL41vbhB10ym5CYKSFmsZP1XMrlG7Ey5N6vCEqt0TLoYt16sMF6rWE8q9GyAiM0dPh+OWiYelmlhunLzePTrNOU8hq3ztl0hfRFrbQL0zvW9l3GSSL/XN0Cy9uDhIMmgzMeWKWeluOgc/ouHgrMs3HoXTZ5HCuvgMaxPAtKPWZjlqCTIGM5nLWCzegrK43YXD52XgcZnax0x8l/nDEY0Xdz3i3B+ynWqYtnUTavMeavnIffSoHzuEmtx1Z8fZHDQWzPLrJv6ijUlZOYZC06jfJXEjm5NGGezuODNGZKypnEYylPQOFnETpzob7t/6kXTiJCiE07Le+UCWXLOF710yq/KOpAhzFRPAceLDkx4diL7umshE7XpcwJ23c1yaZp5B29ooPzoJ9wPbLqdSHB9HCo6zuCAHJAJxAIBAKBQCAQCAQCgUAgEAgEAoFAsIohHnQPPaxCe+FVAC6musq8x2iALNsgydE0TpZDPgNaj1SdlIkJ3pEOVafvjOHo4D8Iy3ZoytAWy5+jz/m2Gv05z1Jl+bwzbaiWltK+m6Ss0jr49HQuqmNa59vKfe7nU8J/y4nl6Jux82v4uawQ7y/zvqSddV+3d42k8suRxrVpLD3z3Bk7hTJqtWPSMRjLM1XWsmFMed41n4OJiWTcagJtU2ZcJZOOqUdkZUNORtH/k9/LzsccSdwclRq7uTpH2n8cnYo+L5YnHeT2WIjI7SQic5xNX8BHZl4oLdNp+9x8UtJWnO7JpB+HviCvRP/Z/MdYQXYdW5xuW5z/MvHfQZfb3Ppz2nUrzTMh651nHeaAsejo+q2fNDbvTFJ+1T7Ljq1pYBXpMgG68FOH9P7YW5a5mq5hYnxCZXFizyC2z5ItP0WX/zuV1Rjtw81VWTlM54LYvJFZVyX3LyYZp+Oi4/pz4rLMzwg/+XSNu75LrpWngcK1WnYtGWl/NVTjjauGtiQvl6wBJmmrLmuM0jVv1zVQwXtu3ZDcCxasHHTHP8EhAfGg48AwsXPJpyLfmUd6md2JuUG1TAONhiTiLlbnLnt3QNpH0W/NO7PZwbSZ7msbak4nvovSrkLFxk8b0OTnkdt08MO2KQSXsroZkjIzWY8F7baVfbwcoaXGyItt5xjvJNBpjJVeSDxFOPRVSLqcO7xd2k+Et506mYujY5dcmzQcX1dwLhJmxwaVd+Y3VxZT55IQDdWiin6raqBuZqlc6JtOIbrMMyKPnH6gcbXI5myyHA7kgmzbtj3YdsrJIgqzIMmFzKGIXQQ/iWyIys1e8yPCG+ZibM0t5JkNFz8cYq79g+exMZipc7RPGFqDd5H3XfqZpukiG4q+KyjTgAtHGZPBTt90pCEX9tIpk+ZPxzCbMZM+S0z3+UIRfXHckDSp0FK6x8i+WHtl2n6scMJkHMXkSfCuw0HKOPMz1cep3EnOdZH50RH1mbr4+eZ4S3PzSwSx0EcOf3P9kOCdHCh9bNjPjKyd2VVh6bD0yaPfpoHuUho2nMrCVJ957ydBjkdL52RftzI8m9NpWH4es07ZujC8NdYYNXJ3qDrpKga9+VFhw7ncRI2J+ncqoQX9+XEY4XPvG6DjXBHj52VYUGbXAKWI6UkT1IHK20nmjxyqRWXzH85qnkY6XnJ7Q5nyOBnGreU5PYvVDwg6tU9mrkrlRXk/Oza59xUAUo9gPZHhm3HlDSdjo/s1iXEQnUtp2tTaoMroLxzNXfa2CG3cWi2Xp6PjknUtiwn0UFW7c31Qfs+db0vn0tLyS95PsnelhkgfmoHUadLreHJ0Jt4XhZwtpUOwrFAoPySVrjp0IAd0AoFAIBAIBAKBQCAQCAQCgUAgEAgEqxnL4v0hWEnIAV0hUpeSB+8PEUStjaZoVRi0S8bi2HjPOUm6eMakvi983zmNnsC7Z8roasW+nMi1Wdf00+zXSS/fzZZfahXexQo4NnZyVlaputZunrmLuZNljRkSMkWfVt0s48IMyt9TC0JHNk5iqct5co7p3TmOdSibz4SW51FkeIPt55glKvOuc/uPy48d55vsu0SeWavE0jaYUn/m5o+x65EsdDKaOrf9lL0QHMvpZcBE8g9T0D0mteKdcrogLZWnE87rXWkq+r6Qv6K0T2AFXVZw+5P1Rksg5z3n5MWhJAR2ysOPyX+aul1W3o3pzW7HdKadD0ZdUu+z+mDKAp96OXQYV1nPuYKyVwpRrznqPTlOnx7Eumb7fKLMJ/s2u1aZEuqZMSvMyKVo+H6aLDf2EmuQSfUDt9DEqy7yI+atnooIUZflES1/zPURNx7H1rPHWCPQdWGpHj2OHp5bx2fznKBvusDvj6B8xmu5WwETvh8DTt9xnpq+1+K06zQhDsX97YcbJMzoQw8rfgfdf/zHf+BlL3sZHvOYx2BmZgabNm3CM57xDLz//e+H1iHHDYdDvOMd78CTnvQkrF27FkcffTRe9rKX4bbbblsB6lvI4BAIBIJDF1qJIjoJxrkLQSAQCASrAzIHHmRorMqDJsF0MfG4WmYjEMHyQXRigUDAQfQtwdSgC/8EhwxW1IPummuuwctf/nIMh0OcfvrpeMYznoH77rsPX/nKV/Bv//Zv+MIXvoAPfehD9vu6rvHSl74Un/jEJ7Bp0yacd9552LFjB66++mpcf/31uOGGG3DGGWesYI0EAoFAIBAIBAKBQCAQCAQCgUAgEAimDDl8e8hhxTzoBoMBXve612E4HOJDH/oQvvGNb+CjH/0o/uVf/gW33norNm/ejA9/+MO44YYbbJr3ve99+MQnPoGTTjoJ3/3ud3H11VfjS1/6Ej72sY9h//79eMUrXoHBYLD8xDfWbM4lvqrcg0D39SjsgW85QfItycdcUE7TOu/pBbBMnrELSNVQjfJWo2+C75pLYqM00ktkufxrkq9Ht7281qeHa5NIOf63frp6TY16TdxHP6izH0Kn6b/eokJvUUHP6KyF4+hiW9h2i9Yl03ZsGsoGlHaOplj+pRaaDO+zn2n3z0eq/U05Nm2ENr+fgj7z0jk0DcMMUzzD0efn69CYu+SZtksH61hLf66fyfdZUJ7I0ZLiT8OLXttkaUqUqXvte5aXlPdbtWUoDehZnZanNF8i1xwZRevn153jgww4GRfN38u7aF4gdbHPvLlAz2noWCipTJv6vy1qNfoj0DPaykc7r5C8ewsKvQWVlOOl44TLn/2Om9MoCueV1JiL9ZMf7ihFi6rDvKK8Q0H5mJv3x/B0pDJu0vQBuQN+fk+lcTOIFUr+uHwbaBrOKjY/Z5DlqRR9EdrY/svJ5+Y95ROqZ7H5c/MmgcNzOd2PocWkCUJ9TTpvkW8tb/oyIPYthwgfOfpiQ3M1UKgGKisDqO4a08dz/MbWj7Yd174d9Mh6XY16XR3qFU0+qbmCbesx5GIqvdGduTHGzWV+OwfZTxqei6yLSuflgCZvbHQuP6Nfx9IlZWFufjU6VbP+4fQMJ08mlFeKNq5Odt3EJfHHQeH6mY4nm39E5+HoZPOiNMX0xCl70KXWRjH6cmMzqm9n1n1R3Zxbf/rI6LzZNu84VyTpMyQtKVSLo78SPSqQy1QNpdd2xMos7UdOH4+sQTrLB0IHl1b3kJ5fmfWfn3dufo6tN3Lfmu87rWWbb+waJVWGJ2PYT7m5hjyr14YTj88zSbnO8ZY/Zyv+u6BcTl/tMsYTc0mRHMjAyYebYwr3uWL0+PM3he5l+qFDnbi5MBbylqszpTOVJrouzPFrQv+oliaZsAQHFTrk2difHOQdOlixA7rvfve7uPfee/H4xz8ev/Zrv+a8e8ITnoBXvvKVAEYhMA3e/va3AwDe+ta34pGPfKR9ftFFF+GFL3wh7rrrLnzyk588CNQLBAKBQCAQCAQCgUAgEAgEAoFAIBAcJOjCP8EhgxULcTk3N1f03ZFHHgkA+P73v4/bbrsNa9euxXnnnRd8d/HFF+NTn/oUrr32Wlx00UVTpRVorBQag4JqcfSDXiRM35emDy6nNVaDhYYLekbby1qNNYl/GSm1RBwR3b4frq9RHWiIMFZLNTA8bHRDabW/cqyZDI2m3jFLKF017+vwvdJAPUNo9gRGtcRcEtsrb+dR+ST/hmbOAsq2TeXSYduKsdLTqrUsseUsqbb9IxfAVsSxk730lrMMVuFv55shyc/P0/mwbYs60TcBTfSZdp8Hl9p6z+l7zvqnt79q24ors6CffeheSCdAxkYfbTtTKyaGPp8nwg/Izwj9SVppf3GWdww/OelSl21735txGr1Em2urcY2nfGs0Jh//mR1HzMXIShM+Z8ZlNB9jRbqorAWrsZL0xxO9pNvmQy0Ftfve5OHXha1rj9DkpbX5w/2mpM040H62eVXuv6P/ANW8iuar6GXllM+pFSBTfj1r5oX2WbVgaEJbcZq3kXHDUC4GZeR43eNv3ddQ1ApQ+XSqlk/qRj4A0X4O2srrR9s+Nf9+VI6bSd14fijGWlETrxjDw9HLw5m5hJNhPu8HcobwO/3WzkWRi+wNn1cD5r1Kd11tvDH8PDm5yiH2nhtn3rwKAKBtT+pHxy433m3W4yx6InKfkyf1unqkixWkn3tg9N3CETX7rXr0/Cjvn64J9RDweboeIs1nmfmRWmoafbFaVK5+lJhrfHkV0KLbb6g+aq2TOdoUWn2ZlGvlJjSri3B6rpEhvt5mSTZJqI7OvQepa6Q9rbxaUkDFfGTXAG2m9dzoYTVfZccPrV+wJgGZKxg6WZ2C6tuatG+qbxIwuvPw8GZdsqfnzBG6YQpVtXMalWs+n3F17AQdyYPyjtFFBoSOoS94EOUf+0nN6G5UhlEvgkQ+Wo281YHReOH0dHbsG5or3epXg7adaTl+ntWiQh3xFmgTMnUyZRI9hktTz9XtWozIOKojczpVWw74OifkEp3//DUa4OlembaNwVmzm7FN+IjWI6evB2O2QHdW/rcZHTd43vzm5iJd6VZO+fpSJE9f5xhuJHIgQVMWiT4ZrtG2IRwPOJqctO1RW+8DANx3x1FB3kVzXmLedXTP3LoT4fv6sKFtqxz8dYODvoYetrSxdaD6NLO/lZsDaPsM1zdzGNF9HD3M120Hys4FxdB8/1K5N7uT0am4rBQC/YHqSZweofu6LV8RGcvoPBpg91nqdU07Haii+wY+rE5B657QcZ08ST/SdUtMXpoXsf0Hv07BGsXsnZE65+pny4rsc1nd8bBWltjlKbOGCfi9kM1Mmt4BhWGz1uvvq0byxcuHbXfTdpzeR+rhr7uyez5N2eweoiEtN3eD18cFKwNVj7MQFaxmrNgB3fHHH48TTjgBt99+Oz784Q87XnS33XYbPvjBD+KII47AhRdeCAD49re/DQB44hOfiJmZmSC/008/HQBw6623LjvtnODKHhpxC3IPqU0g9nuyqcmmoRtjzNit5sMdV62A/u6RMlf3NatkKbooYlAtmFmjLZTSqbgNTEIzB65tLB3+JOR/S9PUsIeSwUGeR19OAaaLT65Pc5v3OaWcTcssaGle9CDWLm6Jgm/6xm+z5EKvQO6neDa6oUuUef87RyHSzDOEba7Jgjwoy+STULKcTceCOqfGa29BYTg7/oSZU3bMxkTd19H2M89s+A5mfUE3Uh3FN7aA8sYc7ZPefGGdvcVV9NChoTlFE128V0vNZ1Xb56ODFzchVUbVMGzr2CFEkEeEJgq7KUYPIQYIN7t6Ol0msyANaCKL34A+kr6ecQ0GgnxiMjizGWTyjMor5oDV8Iuzro6M966o5ivXuMPOOyrIP3bgUDwXE3mbHLu+IUYmjEhgzBPhA9o3dEGfQ8xIyOQVfB/rezL2wkTeb26zqdDggCK1cZGkQbmPHOOpgSIHPiR5bn7rSn/Jd0au7qvCDXlCE6V/4cjRB45uSDZ68cO1o39NWO4CehSRV6Vyjx6MODpHTH8JMog8NmnIeI3pNFyerJ5GNw25sUXGaCADc3KZ8Hv0swzZziaYp7PGdCar25dMx8aIgtmkUQNebreJ+eeUPm5T1gmdxm30Epjx2H9wtFyt+9r5xtSV29AumcunBkancQ5USDtkD1bM84TuBvB9E8xrzXu7KciF74+NN2Zs0HbmDo4Mhmtq9jDPLYChOXeQa3kz07G5vlfovOYO1h/cWjO3rsu1ueFdKncjOkWO7iKdxKPFkbHNu6zhHq1Tk5eZiyi/VQutTpYySAtoI9/0druHTV3GtzMXJjAKC12e8f3/efToBxMqURN5xR5i0Hkpo3+xtJL03Pzc290rb6PEd2aeAHh9mcracTfrKZ39Xc0+FDWspDzptelwfe0Yk5Qam7LG7WTNSPk4l2cw3qhcXuQUarDr89j+D1cu3ccr1c+iTgIpMHOubzzv0zjKP02MXyd/jFoaIwf6rC6UWFfRNb+RJV32b8fBYF1ruDHYUGfXfdbIsvBA24c9QC01gmPKKKkzt8cqWBmMZSgqWNVYsQO6Xq+Hf/iHf8D555+PV7ziFbjiiitw0kkn4d5778VXvvIVnHrqqfj7v/97bN68GQBwzz33AACOOeYYNj/zfPv27UXlb926lX1+9913A5s2dayNQCAQCAQCgUAgEAgEAoFAIBAIBALBMkEO6B5yWLEDOgA466yz8K//+q+48MILccstt+CWW24BAMzOzuK5z30ujj/+ePvt3r17AQDr1q1j81q/fj0AYM+ePVOhLRoGDZNbU8SQyrc0pJqbYeR3BnUm/F9LVGihMbqwNG/JHH2Vsyilz3LlxPLxLXY4Cx6gyKLpoCNWZUMTZ3lEk4/RZsuCMSwFs5ZwGYvbfAGZ9xlreSozir3nxuwOx3sz5+mSsBjNWa8H3/thBMl4onVOWiYzlpfAeFaXUY9KrnzGytdJw3k6MJaaXcD2E6W5dDyWfJbz5DDFD8erS9bKMGcFnLMs71heDiyvAnGPlglEIzevTCPvnJWjU35Jfsulu4xpiZt81iDmSTmtOdmVIXq8vlqOaZXKqMJxwnolEPlvQzMSGUgRt4LuCCpjvfB8UwHNclKdpmTeb77rOl9Omy+48Pq2qIysjYKZF202HUJ9j0NLybg1nlK2zn5+MXluHma8qEqjJXTy1MnoPXS9FP0eyHv7MLRH257LK7eeSOWv0rKhWlLjzUulvKsmG15d1pr8h+EjNVCsPBqnfpxu2rXCE4Ud67JG49qCCy9K2mZcr65xvO0tCrznbN4dZLsJVT7OGNOpbwqgMzIsK8P9vICRt1KqrXJyp0O6GPw5zq+nX57yQmhPpBPT9RL1Bi9dA3GI9VOGJztFhyjEpOEIszIuB06eMXtYDiKe3111f1216XVhPpNGc3GufVgsnxdzyPHEiu5VCg4eyD5VybeCQwMrGjX2Ix/5CM444ww89rGPxb//+79j7969uOOOO/CqV70KV1xxBZ7znOdgYWFhWcretm0b+3fCCScsS3kCgUAgEAgEAoFAIBAIBAKBQCAQCARjQRf+CQ4ZrNgB3Z133olLLrkERx11FK677jqcccYZWL9+PU466SS85z3vwfnnn49bbrkF73vf+wAAGzZsAADs37+fzW/fvn0AgMMOO2xy4qhlYGMZ3yPxo9VQjS7hpJbLdfuXzd5857d+U5YqPA1fs8O9PDdIl8lLM1a4wOjSWXPxrF8nVQMzx+zDzDGj9vatiliPFb/cHlovNZ9+5iJdpYGZvQozexWbJkzglt3fp5z2qdfWqNfWwXdReO1o2mR2Z4XZnVU0L5rG1FkZi176B0D39eivwEqLra9B1fwpQM9q6Fnt8mmkzWy5tB7Nb7+vislKjIl6TX6gOOOkCumuFpW1KAeQvsvMz1uT9imFb12e4H1nzHSxbiHp/Xbzx7bfTxyqRZWWSx15zdaFyCpKs0GKj/161DOjP64cPaeT9XPyMrwP91lvXqE3r1wFhcmzWlKolpSVQ2qorOWcVrzcyo0NPaNDOcuNR+XK3QDG6i9j5Wfbw2sLn//qNXU4Bpk+5WiOobeg7J17NhlpR593AKBaUKj8NNxc5tePocnOyw38uwdtekZRrQaq+zgdY7xz/OM8y1noHrUY0uDNT3Ts5eqUej+6V4/oOj5tHbTHoO8JzdViJH+09UlnXkAAHQ9e/1M+qWe1rXN9xBLqI5ayWfv8zbVnip9zPKceOR/mE2l7K+uAtk2rNl29foh6/dB9z7RfbJwZuZHVcxUsI9p61vHy/Po5+nak7exvRh+P0dTfp9DfF9EjKzh8Yp8ftTD6I6gGysoMp85+kUQHivYz6ScOps9iuiNXfn34APXhg3ibkPRWH44gOS8V5O/3X67NfHm4ZsserNmyx52ziE4YzGVEtmR110k3LPz+MHUmfZ76Y/MCnDFjxlqn+YnTbxqe5fJSQ8XKyNi60niDpGRY3ffSlJJeWE/avsFzps2Sc53Hn6XfBahKOygOR16aZ8Pmr2P2/f0K/f1UUUH8twp1Jp+mONEkv+a3MxelEPvE8GPtflDP1ajn6rHWVTl5ZeXGjGb1ffs5mS+UBtTRC1BHLzjrb9sOC4pfmxA9hJMLvj4LwM0nM5ea9/VcKLuTOl8N1otJb15qaWT2aRyamLkstlaMPeuqu/ryoHQ/jlujUX2V8nGW57x+dnRLb46CavqY2Q+kstro8/7aydRRz2noOe3SldOzGN6K1o3Jy+xXOWB43Nd/km1n5hy/z8w4o/tJzNihdNI9Sg5GnhpZ4tfXpzV59yazDvVB9Vlosh8Y41FT54ZOp55c+T4NmbkZcPdFubo4NCs+T0dPmXzqE4wJhbB/on8rTaygGCsW4vIf//EfsbS0hOc///n28I3iZS97Ga677jp8+ctfxm//9m/j2GOPBQD86Ec/YvMzz7ds2TI5cUT4m3+Ha7RVWowg0lSAE7fpbNgQ82/BZacGZrFDDyEWjmgLyoXrMIqmc4gRSTN37+jjwXrdXkpv6lwBg3tG4UTB1Hk04TSKRU8H4SiUBrSV7ipwxVbMAker0SWrPsxEGFtUGAw2aIe+6oCvRbhtbzaMaYgF7uBxcdMoUW9/1dYjEmLB5t9r+8B5bxbOw7btWH7i8lTtN+0F3Ar9vSNinHCLZrHDXO7rF0XbbBxwF9gbVIvkcJnW2SShygKnzGsSrouWZ77leFu173Wv7Wcn9F0HJcOne8Tbo3e9AwqD9drJn743bV73gWoAlw6k2w4AZneN2m9xY215h72weUajXtPQRw5CbJ8P3ZA8rJzg6mwLaGmkdablBCFkvZArnGJn22lRsfVjaTPtWLX1qAYhn4w+bt4vtfUfNpuQ1YHKpnE3sZhsOHlK0szsHvXTYC1ZQNUIw/RoRi4RaAVUQ/f/DipAk/lJe2PG4a1ZjWqBi3/L1IUBx8dagQ+bUpEx4M8Vug0NRPO0eddw6Ajmktqrl8cf/f1VK/vaqcYufGn40brPaK5m0df8pnWldBja7IK6astsaWseeTwU8JS/5vVoVQ/MuukTGx264vUCp8/89948Z+rihKE11RvyfMCBDbdsnkUMhXL6FC2TC5PryBU651L+ApzQSNVCO/Z798+EtDNzBN3cickDv/6+MQOXxtJ5z9rAkMRve/N76bBGhi2pVrT02g9m7x0pkoN1vH7B8suwpd/M27oPVn916mnGgdE5fN0uUX49o8N5gbR9PaPtvGDmz2HBYcxgA5m3vDIRkzf3zbUk2Hm7+UFDgTF8Ws+QvmbmPId3Ff++t3vUZ7rXGhRRHdK2E81/bz9KE+Dpu/OVLdMP7+7oy0x/1XO1TR/L35d9lGYuJKAvEw/8eLQ+nNs5KmeJ6PMO/WQ9wMmbog03RHQWZgNs9CJMP+rTcI6xm5c1r/P68wmAtr8XImGxSJ6WNsLHNM1wfW3z8usxkhFMXUjfKU+3dtYdnHyfq1HtqwI6WBlHw/AanWAx0UkYtaHNl+qwhpYK7Rqd6VPlt7fVz5q1WKWTfKB7JI1unylurjHP6CZ2ZF1lyqgGdKCE5ZdguMYbW1Rn0u5zWwBdDyXodGg2ImLY5mvmIiOngVGdrC6TW79z5Sqgt88o9+V6A12ucHpokFOtWp5iNtwp79d9YOb7awCQfQqSZLChdtogILdu5lPAvaKCW89RQq0eplodxJcDGLVX1zDQnAxUD860z/q8vKBzHb8ecEh34H/faw6WzZqS6hfcmnG4eYD+vY3ONsvPAZZOOr8285tzHQFZf3N8zOm7Srey1ewdVZ5uGcwblcZwbbuPZMrlwjlzMlT3AJCDvOwepCm2kUdUZ4qGjrW0Nv/Wo7UVMNpbavcSQjodGUnJT+11rtXuHDVsnwPu/DV64S1C4O1R+mUPlZU5RpY4Y43yWYzORDtzY21p0xCVabNNA1R7Gp2OWa9oBbuXYfchBort/2qp1SO4/aOU0fpgA1kDmENPAtq3MYNrZ+9MsLLoJuIFhwBWzIPOHKgdfvjh7Hvz/MEHHwQAnHbaaQCA73znO1haCq2azf11T37yk6dOq0AgEAgEAoFAIBAIBAKBQCAQCAQCwUqh1INuteGrX/0qXvCCF2Dz5s3YsGEDzjjjDHzgAx8YO79rr70Wz3rWs7Bx40Zs3LgR55xzDq6//nr22+3bt+Nd73oXnv/85+NRj3oUZmZmcNRRR+H5z38+PvWpT0XLOO6446CUiv5997vfHZt+ihXzoHvUox4FALj55pvZ9//xH/8BYNQQAPC4xz0OT3jCE3Dbbbfh+uuvx4tf/GLn+6uvvhoAcMEFFywLvWrQWjvUnMVX7lJdwLH6ZF9nrDqD73MeJcRajvPSqeZ5q8yljaE1Fme9EfOO0ZVrjULTawX05kOLJbYcL18fbD8AcQsgEMso/zmxHrIeNxErWdaqMmdBkrDkdD7rk7JZMzOGDmopSKy8BsbiiHre9HWQJmuVOa5QT6WLWAoms1P8bwPqVZC07gOgayaPjvX06ab5Ua9N7r31fhhG2j9imW2wcOSoU9VS/lL6al4FzzjLY0NPCv6YM2EagHyd24fe7xT/6TR/OHWiIUKWyDMu/4ZmWh9jsa172mmHnHcQpdWHtail456zJo+kN6gGmfJjsprLa0ElrUupdXD0Pfeb48NIWEGTNtm2mfGZq6fjOUz4zMp3f74ulbcRWF7i5HKH8LtsnvaB+1/WqjHjfZush+b7hPUa7GI1meOnTNtH0zXfsR4cMfr8+nk8bPuqUC5TGa5j45njXW6uojxjdCdq2Z2yYkU7/3PPADcSgUp55NL8qZVvIo3TB7RPjD6oPX2wUMbQPA2qpVZ3Hc51mLhznyb6jKUro/PH5HZOl3GeUU+REprA6/vRNKRP/OgOfp/66B2o8vQzVugx/cMHjchB1yWcVz3nRRKlKYGS77JzAeMB0UnnbV5zuptbEJM0MrfTvEp1TqfvGA+LlB7R20d4IzaumLkm5zlnSZvVyfEX1a1N0RFZXextxJXtrytM/Sr3m6LsZ0N9uiv8/imSB8y6N6tTUn27Aec15ngpZcYrjQJCvZRLx47ziM71BeuyagDAeKfEeNyM0SUmwg+dx5fSjacL9o5S61rd48dBSsbm9lnYjdzM3Ojky80JJXtkBL63eyw6kUHvgb71vi3WHRHZR6KeiISPc/LED5XpeJAxew5Ktx58fmSZAIr36HLm+kJ5MeSiyuTSkjah6XPrCfZxIo2/L2nHZGQuZOVBqp+It3h07yS35u8ol3t7K8u/vZ191iOa84AzV7gEMsiITk7PAFidKwCzJg8+ScgdwSqDPvQ66ZprrsHLX/5y1HWNZz7zmTjqqKPwxS9+EZdccgluvfVWvO1tb+uU35VXXok3vvGN6Pf7OPfcczE3N4fPfe5zOP/88/Gud70Lr3/9653vX/GKV+CrX/0q5ubm8LSnPQ2PetSj8L3vfQ+f/exn8dnPfhZvfOMb8fa3vz1a3iWXXMI+jzmedYXSemV69ZZbbsFTn/pUAMDf/M3f4Ld/+7ftu5tuugnnnnsu9u3bh89//vM499xzAQDvfe978ZrXvAYnnXQS/u3f/g2PeMQjAAAf//jHcdFFF+HEE0/Ebbfdhn5//HPHrVu34s7778fP/dEfdErHHValvgXKv3/YIjc5jDl5ZNt/mco9GEiFC5x2OalFz3iZTpZPlzE4MXK0TqtNYtl3kCFTlTcp3l/mOmcxxXF50GR0F5rHqF8sFMtyyQSuLIPkIRy3xtJ8XpyxSBd+lPl3dYALSzlWPuMeGrKZ4eDLsIMwV6TaZRzdFeje1gdzfmZDdHUsv0hOTLHvaHi8ZHmYXplO1gV8AsA5XPbDDI8etu+7bMgWlWtAw0UebHlO+3yZ1wKxuuXqXNomTlitZWzHGG+ldIbc+ozlvci3yz5eIod94+YLrC79ZEVoWsXr7AApPdYP/ey/1wjCCx7UtSyD7HjNHSZ1KYuEzi3W7bl8YuuFWOjGMTCWvJqwnIM19jqVc5DG5kqPg0kxcd8dSjIQwI/fcjlOOvJIbNu2baVJedhg69at+P6P7sdp55WdWXz7+svxuGNWvo8eeOABPO5xj8Pu3btxzTXX4CUveQkA4Gc/+xnOPvts3HXXXbjhhhtwzjnnFOV3++23Y+vWrej3+7jhhhtw5plnAgDuuOMOPP3pT8euXbtw22234cQTT7RpfuVXfgVnn302LrnkEhx22GH2uXECGwwG+OxnP4vnPe95TlnHHXcctm/fjuU+PluxEJenn346fv/3fx8A8LrXvQ5PfOIT8bKXvQxnn302zjrrLOzbtw+/+Zu/aQ/nAODVr341LrzwQtx555045ZRT8NKXvhTPfvazcfHFF2Pt2rX44Ac/ONHhnEAgEAgEAoFAIBAIBAKBQCAQCAQCwWqDGpb9rRa8973vxe7du/GiF73IHs4BwCMf+Ui89a1vBQBcccUVxfldddVVGA6HeO1rX2sP5wDg5JNPxp/+6Z9iMBjgqquuctL84z/+I17/+tc7h3MAcN555+HVr341AOAjH/lI57pNCyt2QAcAl19+OT7+8Y/jec97Hn7605/iE5/4BP7rv/4Lz3rWs/DhD38Y73nPe5zvq6rCxz72MVxxxRV4zGMeg+uuuw7/+Z//iYsuugg333wzfumXfungEE4tRBt0sZCIhhjpSsaULPXSHyHpLp7Lc6K4tzkXbPKeljNxzN1MuaOwO+XZURq6pBsnf8Nby2V1ZNo0GsqrOCPm2YT87Fu1LWvMZSZ/p6wOVn9jFV+Ve57k+KFTO6XGRsyLKYNJ+iloc0b+FJfZUdZNA51kSU4ecklM33uhNJZjbHB8Nok8KkrTlR+xvPIxhtUa/53DctLqzFVEhqXKzNHSlceSZdU4+HKgo0dsSd/4OkEq3Ti6a27uWW5P/hKemDTPJE8ZHonRMQYPFS2ex5gDsuVyOh1XNG0PjTgtGiPPuSnpumw/TDH/zsjoHNMA7ROuX3LyrlQe0u+Wc15MhVbsXGbT5r4MYmVcbi03rXVCajx0zWoF9JMclntNyReK6Y+tjjpCii+c9x6tTroCueqnX+n+j45Xo7Mx7TLpWjYos3Ds2s8ja4+u+fjv2DwzzybFwZLLY6NwbE4qV5drrXqwwPUdJ1uWSwZmZVjHvUzBKoYu/FslMPfCXXzxxcG78847D2vWrMEXvvAFzM/PT5yfeXbttdcW03faaacBAP77v/+7OM20seLuZhdeeCEuvPDC4u97vR4uu+wyXHbZZctIlUAgEAgEAoFAIBAIBAKBQCAQCAQCwerAoXaI/e1vfxvAKJqij9nZWTzxiU/EzTffjDvuuANPfvKTk3nt3LkT99xzDwDgKU95SvD+sY99LI466ihs374du3fvxsaNG7P0fe973wMAPOpRj4p+c/nll+Puu+/G3Nwctm7digsvvBBHH310Nu9SrKgH3aGEnKfG8hOA0Gtvwt5L3V9g69vREni5PbdizziLo7Etj8axfk5YolAaJu2zXP6U9uW27OoEv03HGD9dJ6CsBfyE8PPn4tIvF9RAsZewd85HA2qooIZleU3i6TIuSsdWF5i+o/nScg6WxfBBs47saJ06dhnTqEskn2l6nid5NVL+snqDF+JQU8JLkerb0nuUplHW6AOsKktDiuIx4Fnwdx07WeveZfBg64LlyH+qeRoe8nWzCXGwPJ9pecuJLlE8VsSS/mB70zZg+2SFaLHFZ9ZgU0Opd4anE07kjbfaPFSWC1OeS6eBcdve8t4yzQXFXqqxb6rR32rlrbHGbqK+aqhaWR3zypmiDJvWemAS3XOaGNuTaUptulz7RdPIY5pzYa6dD4YMTO0djYNUdIykDCuNxCQnBasbGoDWhX8rTSywe/du7Nq1CwBwzDHHsN+Y59u3b8/mZw7njjjiCKxfv37i/Hbu3IkPfOADAIAXvehF0e/e9KY34T3veQ/e+c534rd+67dw3HHH4X3ve182/1KsuAfdqoQR/hpWMA1na1Tzo/+wF8UOFVDp4DmHqtlY1z3Nf2ueZQaSGsIJEWXSFB8CmTp6cGjiJjNKH0OrbR+qnHLvmc0LWif6jt0jyk1qpGzuIC8KejhJ/x8pV9WA7vHf0m+A5rtEeIjSzUjj+u+noQcMnZDguWpJoZ7JMKPpM3rgQenzkvcPKAzWpnzvw7p0qZOzcNIuPeTRZEi0me6hKATTuIpZPTsqNBsWKzLGbfkVoGnDMP3ofJ9S9kidbd9nyAOAupmFYnXJjS0OuXFi82ZkpdIA6lZGTw2Gj+kz2t6Fcr+oKHPgauYkb1zWM+a7kD5HVpsxWDCnzO4cfbR4eFtYtdS0Y7+d68yzeoYoi5RPc/MemV/8gwia3gnbRvKPzrkmXUSGTnLZPJ2nk2O+Qjv2qrZ/TJq6n++zSfnI0lozbdArkDk+SJ1ybT8OSmRotdjw3Kx2+QdePTXQa74dzhUSRXknwmPsXJ155tBpviF6BNsPqk1jNyKIzlMsFyObB6ZtegeUM85LUA0U6n7bpkauF81hgKNv0j6z/TWrnTz9+pXwSbEeRqqe+tbnrUkw2gA1jBbS4sjozDzu0xikb6CVJ6+DxPn8DS3KzAVkTo2FS+Pg8CTpZ0fGI5Ef/a6QdcfWpznQ+SOzxmCRW0MxKP0u+i3ho5ycCN7NaNvnXUD5MEs/Iw9i4PJi9ZrCPuHKU3U+zy59Ystq5GZnozxfJ8q0kxN2usM8T+tk5cVsuAiItg+bKSWML8t/l82S1N+Re55up3u61aG5/L31qdnTMfObJjJmZq/CYJ1uy+KyS+yJjDPu/byLx25KnwSc9Z2fvu7r5D6HyuwN+3ma9o+uwVL6PvM+hnrDiNGrvb32IbcuK8grIJHwu8P7XH+QNov2M1NnIxPqGV08dnsLaR23WDelsrTRD8aR+X7Z7N4gnYs5smN9Bvd7Xbl7QTZP5vdyoO4D1aClZfQj8u2aUaWr+ap4DeGgVCfSjJ5H27keT/YIDh66HCzffffd2Lp1K/tu27ZtU6Iojr1799rf69atY78xB2179uwpzi+WV9f8Xvva1+K+++7D0572NDbC4wtf+EI8+9nPxlOf+lQcffTR+N73vof3ve99uOqqq/Abv/EbOPLII5MHe6WQAzqBQCAQCAQCgUAgEAgEAoFAIBAIBILVjCnakpfgwgsvxG233dYpzQc+8AGcccYZy0TRdPDXf/3X+OhHP4rNmzfjQx/6EJQKT6Tf+c53Ov/funUrrrjiCpxyyin4zd/8TfzhH/6hHNAtG3RraaUaS4e53T0sHTYaAYd9b2T2sPcYjbqxQFEaGBoLEmPlRi2rh8pamPSaOw8H1BNTEWt5xsqqXtdY+ezrtVbS1OJm2FrE+GHbgNYqQg2UTX/mL30XN914ivOeektVi6qtH7Em00csjZ49MBNa9AwVhkctjup5/4y1LBvOtlYZ9ZGj99WOWeupYj09IlZEtnylgzKpRYxWrSWP9XgcEms+1dLfu3e2bTNjcaSA3rxrXaRnNfq7R3kN12hbp2qUDQZrdWuRNNtKSdfTo0mz0NJH+8S3cFdDhcFhI6J6+yvLE9bwVqNtu5621kPWQr0GZhqaBxs0KkPfhhFDqAVX6Jj6141Xm1powx4O19WBVRS14FdDhdn7R+8XjmwYrgK0IZZYvpsKDNYRPiXWxKae1ZIKLDCrJWKBTy01zTPqfaJarwlTzmBDbccmtcozz6jlFmt1SMqkVmTWm21J2XxnH6xaDwNq5etbas4QS00ybq3HxxwxO6za57oOLTJ9+n06zf/pezVQttx6VocecJ61nGMhaerU/J7dWWFx4+gDy9tDFSoOngWctSajlmHEutWMrcG6xppsqR0vrCcBeVYNFNb+dPRg32PNB0Dd5NXf2Wst/4Zt3w43NzJiZ996+Nm27xFLUeLRxFnDOR4dRBZzltA2/74O+aQH9Pd7lsm0TCq3ZrXlQ+t9WLVl1n3dlkX5x4znuh1nNH/Kk7bOpn4DYP7Ro46s9reZ9g+M0sw/aoj+rp5Nb9L0mvdLG4kMNrKO0Kw0aQvjcbmkHK91a2XejNHh4QNUu/ttOxoZOTfqiJmdPSvj+/sUFjcPnfJpPaDovN62DZUBzrwPOOPZ8JhWsLIYAPQjFkZJ7p0bPRi4fGT62qTRs61XgiYedq1HrbL9BBV62qoh8YbWjJUyoU/P6MBqsneglct0DjDjqbenZ/vEYDhXo1ogbUk8BE3Zjgz0dQrf8tjXeTYN0Nvdc+g06cx3dt5bauXd7K52TrfztlYYrB99oBs+qfb1HI9UyvOmHainKODxK/VKIHKNW09xlrzHPem/cc83fm7UPA2dvQOtbka9Mo66ZfTv/U9WrXV0T0M1PHHqL/wAALDtG8e5MsawzIDUk/SPqZ8hev6RA6tfGfhzpuHD4WFNO+5XGK5pdR4z9qk8czwADc8O2rY1dAzX1Db/pU2NbnygtTIO9I/mWTVPxobRx+38oltVZNjSUh/WyLXdfZv/0MxF85XricF0qrWCXqhCbzXd8kxvf+W2haHlkSMZoe+da2WHL5/hjpPjn/RjAMBddz46kGO0TXRfQ29qFNmdM62eOmiTDNd6esySsu0wu7PC4ibXHc3Xm/r7mn7aQGSRYTOiM1lPR+p5VQH12oYRmjFYzVetjNHK6Z9R/p7SY+jivBfob7LusLxvxu3AbWvjtWnS1LOajQRg5EY9qzGzd/S7N68wf3Sr25v3VBZT71eH9gZmfnZCkzO8N2xkWW8fETgkidVtBm46I5tN3w3X8HLP0FktKStDbdNTHYfqfJRk6vHRPF/349HD/cfUUPNk3tNumuG6utUpZnVQ/3rDEP0H+va91YXIvGDnb+LZy60RbLMstvOf0q3+Y3WjGe2uSzyPlpFnFVm3eVB0/qtb/caMYd1r6fLnGA6+bjpcW7ty2/BXU/9qUVkZ4+iZ1IPPkyGGrlGiVr87+ubRo/vIFTPDI5dQPWjCNzRJ5lyafE+OauDuSWzaej8AYOd/HdnSWRi2zomsYl+284LV03pAvdbMW5WjS5j2qM1ctqRw5yv/HwDg+E/8ln1msHT8PNTPRvqdI/+ZdSHVq/uNvBis16GeFlRw9I/De82zI05+AA/cuTko318vKd3q4zS6jaFt9tH7sHhf46GwSAdxU/aiwmDjqAIzu3vOXogpR5O1ulp0O2K4rsbs/SNGGqzTGB4+EjS9Pa1nG/UWC8b7Gj2Sc039LU8bfZbuLRH5esw/j9L85Kw2r96+th2O/sbo2Y6nhPJSKyIDPN0dGO2nmL2W4ZFL6O0Y7Tn5egQA9PdUVj8yY6iaVzb/3rwi81Krr//pC68BAPzV9S9px+4aQwBpIELb0lHNnLmn50TfAkbrKxpxJZiDati+qXb3Hb3AR0X0PKMnVANPTydjCmg8Mcm8ZnU+qs8SfZTKLqCRkWaP0sgaStOisu2z+cQHAAAP3LG51XHR6jx2/Rnx/I15tVldhokS0T9A5kq6RqQydC2ZtwFAuXnZMdu0k4I7h/kevdx+VkXGn55p97Lts4r0gwLqytM5NFlj1SoumwTLDgVW5Y1+e8IJJ0zsKff9738ft99+e6c0+/fvBwBs2LDBecbdCbdv3z4AwGGHHZbN1+Rn8udQkt8HP/hB/PEf/zHWr1+P66+/Hscff3y2bIr/8T/+B/7sz/4Mt99+O37wgx/guOOO65Teh9Jay6gi2Lp1K+68/3783B/9wUqT4iA3EaxKqFAJzCYhdcq5VE8cviX6cfNvZmRMkz6BoBMKeXTqxS4DT08rz0NlvMUOf1dDbPCHElb9/BjDCo1tW/yUx1G0HzrMswezHw+2jFvtfLra6SvGuDJ2SuMx144HtZ1pnXL1Y95Hwwz776fB75O0/wrNqyuui2Ta7GDQFwsx7bznQnytAtpXA7qshVN5jJNO0A3Jdh5HBq3y9UCnUKXZzJp/GfkAHEQZtdxY5X3aBQdNrjyE2sygE8+t8FqwK378lstx0pFHHpQwiYIRtm7dih/88H6c/pzfK/r+ln+5Asc9duX7aNOmTdi1axe2bduGU089NXj/i7/4i7j55pvx7W9/G09+8pOTee3cuRNHHHEEgFG4S+4euqOPPho7duzArl272APB6667DhdeeCGqqsK1116L5z3veWPV6+lPfzpuvPFGfO1rX8OZZ545Vh4G05piBQKBQCAQCAQCgUAgEAgEAoFAIBAIBMsApcv+VgtOO+00AMAtt9wSvFtaWsJ3vvMdrFmzBieffHI2r02bNuHYY48FAHzzm98M3v/whz/Ejh07sGXLFvZw7l//9V/x0pe+FFprfPjDHx77cA4AHnzwQQBgDwm7Qg7oCmFCdQEYtZrXctVAFQ8APadHfz3vea8JGzJUbugSYBTuonafUZdsjiZKi6pdd2pDM/3Whr3j6tGEhqgGCvXa2rpg+98q3X4LNCEAYh5mQ2XpNvnUJOiqSdtbVKiWlFtfwAkd5eTr1bXyQq3oGW3Dd1n6Cc3md6odoEYhJGd2V6iPWGLb39CvNJw2c8pqUK+pR6GQmHyiIN9ygliTMA85pIQ45dPYN36b+7wXlPeo+WJ6uPYCmnAsxE2fK2f0YvSnapIv4T02SSZkivmG/vn0+3XpBKbO0TzIt8F49OXIGKB5xsZckr7MdykZYfuoIm0dKcfJp2kTOs59GgI6mvHijAXCM0Y+O2m8/5fAD8OnathQHZqEcHXCd5YoWESWmjkmmiYlZ0i5tM7ZOYKBocORp0SG2jHM0cLwf6pMf47w+dQfr/44sX0RkTeldU59p72wP+y3evQ3CtXm0hKV59z8VRfQzPAZHUcx2VYynnJlm7yrRcWWY5Cz1M7Ws+LHroGdl71vjZ4wKgQsTwCwodKmvvhJlBl9zqSPzUWsTGHKdMJuGt21UK9w8mHCN8ZQMSGAKZx5jZMnXJp6lG9Fw7QBrSyM6Hz1piXUJgwkHVs0HF2MTjp/TDoV03FAdexm3qBzVxZGxuhRKMB6wygGEzuOHzk/+qN6gNFthwihPBni5alJf7HwdeN1Q9Trhm3fwR3PwbwdkSPmPZ0jsm3F0Un4JDYncnpSl02TrOxMyFujT1jvSEM/mfN9ncuf9wL6SJ2tntKFn2PTFpH7tryBcsLa+rKm5b12fgx0toSe6OghE6BkjcInTH/LyYu6766Ni2lcUk7ouYnbJDYHUn4sHNucDPX1Q04/ySE1dhw5lNAJqiVl5xJufvTHrh3v3BgaKLY+Kf5x1qqZ+XkcPi6aI3zdk6ylUmvAFB+xe1tGVnlgeT43zmj7JnRKW3RF1igk/bT0OSq76vU09vHor7eonDazfc61U5Wmy3me2tvw+a2wzqXzW5SmcaD5cRqbdw3Mfltq/RLQZtok085OmQw/ptaQQLsnWNI2vQNtiFAWZH/EJcIrn8ki0LG9ND26txajlVljCFYAuuPfKsB5550HALj66quDd9dddx3m5+dx7rnnYs2aNRPnZ55dcMEFwbtbbrkFL3zhC7GwsID3vve9uOiii4rr4GPbtm24/fbbsW7dOpxyyilj52MgB3QCgUAgEAgEAoFAIBAIBAKBQCAQCASrFhpqWPa3Wk7ofuM3fgMbN27EJz/5SXz84x+3z++991686U1vAgD83u+FYTtPOeUUnHLKKfjxj3/sPL/00kvR6/Xwt3/7t7jpppvs8zvvvBNvfvOb0e/3cemllzppbr/9djz/+c/H7t27cdVVV+FVr3pVlu5Pf/rT+Jd/+Zfg+a233mq98H7jN34Ds7Oz2bxyGMMm62EEBcvLg/Xa/raXZJPjTe7SWicrYhGnvAvG7Tf2guAwI2uJQ145l/AylkKUPu1b32J06SprAesQ1fxj7pLraXuxuZOvIY+0GYD2CJhYSdqLbnvaPm8vPSVFN/kMZ1trTTVMW9SPLDzdMp0LwGtCXvOjntOOlaG98JumI81vfi9tbC5yfaC9iNa/IxAY0dvb22vrzPCJuSTbuUQ6A3rxuOM5hMbabam1Ns72s6Gf9Ke1iiFp6eW3Jk81DC+IpZdxc3XRP22tInyrRgDuJcXUMpOyvH8JdQ9s2xprXFS6pUVpdszYvHJeYuQC95xlfOwellRa/3L35Hvafr5xYq8dO0V3emS+LUWqfv4YMVaSLI/W7jgKC4LLH4BjJaSWVJYGA2NZH7RDQjZhyNfVtygzVnP+d3UjOlRN+NSUS9OTS91jsBeEz7YXk7N3vABJ3nfypJehU/IKrXbpRfCa4y3jATxk8iR961jtkvRUXlp5YNJ4c0VgLe3IHd22b4T3k3VWhJaUdbzn3ZO6m43SZOfHmGcRw/tFc4j1SCGyjMzZrOwmFuDmtxnD9AJ4tr+o/G6801TOAjMnt2i/MmXp2opqtr17+6v2ObW4rePj0S0/LyzHukuIm/eIPubn6ZdjHtM5gvaJ07dmzNRhGgqrJ0XaI0nTksrPL5afwg9o/rTN6VxoZUCFQMbpCtANYbT9qL5n0lNaq50zJJM2Lz8fDroHx0Mo2e+1AhK8pIakfMLzdN7K3m3KOC9Ue5rMKk/vMt//9xq3TKIb6irkaTpPOpb2lI7UePKeW93ZDGIFVEtt/jZLTwe2+Xo6tZ5tB0d2HHKyi8hFh8+p7mzawPCryvMMbcegLow+EZDqeUQE6ehcat5X4WdOXUhadg5i5JGTR0I0agXroezoaXTdFSkL8OZsMG2m2rFB5QnVQ1K6W+7+RN2DlTG0nC55cs+p97rpB6qH5dJT1GvMItdkjrYdaV0K5XJUN6JjvHAu5+SCJnqUk4zS6dc10o8siIyNtSmAkedmSt/09CRKC+C17VxtvaFoesqvvp4XHTuc3OTGgPNBmC671mPSqCXlyjCvfRTt+xifmPmN5k/lJmkHMy+bNDR/ln6vHc3aipsrWoJg10o0r+i6KbGGcdaqjKys9pJJu/luONdm5MjQPtFTTAST2BqJI7Um7xNrSbpfF60zwvfc/kRsPHNIjldP362Y/VZnWovo9H4a53tGhlDeze7pJNrK0bdN39XuM1u9zL6cv8cVfsA/DvYRmWwcHZtZY9D9Zac9uLVmyXpGsLzIL0NXFTZv3oz3ve99eNnLXoaLL74Y55xzDo488kh84QtfwM6dO3HZZZfhnHPOCdLdfvvtAEZhMCke//jH4/LLL8dll12GZzzjGXjuc5+L2dlZfO5zn8OBAwfwzne+EyeeeKKT5ld+5Vdw33334eijj8Y3vvEN9oDulFNOwR/90R/Z/3/961/H//f//X/YsmULTjvtNKxbtw7f+973cMstt2AwGOCcc87BW97ylskbCHJAlwaVX0vtIUTdTKpqoFpBiPSkE4QTwUh5dpQDslkZpOE27JlNCJ9+bqPY1Ku/T2G4zmgKzfsZDdCQGP4GYd0q/WpRhUqcX55PElmwm4nXyZ857FG6VdbopOFsLpJ9I7OQV02mzkYtycsuVLxNTzNxORuAdCJtnveavls6rA7ypHWCAoabR8Kk2t3nJ09uocwsANhFPOEDq0AvtaGe6CQf49HSQwybD92EYTaVqgHpx5o5wKMHiLQcpq6xAxzjol9TZZa0ieWD2GGf+ZYLwRlR0uyzng54PbsBx4FZnMXgKH7kEJrbDKX5J4svUUapHDJjzoR+4zb00NLJ5c9uDERQHz5A78G+m7UnS7nNLMo7dGPN1snsSRIZZA8MBsqObWp4MWxC4VXzlctniQ0F7vCZLrQc3uQW/DZxZjxqWCOOalG1G82RxbvZMIgdLpi8qDwxeVVLyrZVbBPK1juzWLP5VOGmphq05ccOm+hz02d2jogtDpk2CcIIebTTbzi5y4X35RY1Tj1j3ybGjr/g9g8V/f7IbthXbiXpwU5ONhTrOX2wBgVGlszurKyxCw2Xwh0EdtlUdDY9nc0Fl/eH62pUB5oF/ZxGbQ9xGv1hUbk8QXSNKJiNEZ8m+iypO85oe7Cb2rwE+Hy4sdVlI9UZG2azpBe+p2XW/XZjzD4umR9JWb7+wtXTLYDQwrVTX1sjBk5PM+HlfVpM+Ee6wWZlOZk/2E3N2s3TzpsM/XpG84Z2ptiIrm/1I3q47G0QtZm5ZUK1IV57+6qgf9QQGG5o5r394Q6ZntWh0QFzuOPkSXmP6LGxzbrhEY3u/OCMzY+WYQ8oyVxB+d2np7egXONGrlhTP+ZQTYHZACOgh7KcXAf4+Tc5julaLyJbaFr7OjPe6VrT8EG1v4rOfYAvS71y4LZd7vDArAGcuYbo5Y5OERhzhusKwCuTHuRzsjE1h2XamcKsQeih3ehBWA57SJNBTAbk0ldUjwX49Sl5HjWAZXQOzsCDInZQ6fez7mlnDZ47nObK5PQDNgmRsY4xhv9dD60+vaBYGWXWl9VQhXVS7e+Z3RUG61x9OmesQPOi65nhejJGyXtfj83xhSMjYmUzc4Wju/qHikSPcQ7wmDo5fMYYb0Mj3DOg6wqA3xOjRRUYO0cNRE05nr7OtlnKAIfI7ZromYbfewuN8b8lyKNd8fWk83z0kCtGsPfM8HkQNtz71qGDKdMaKcy0iSgfjCP3oFz9xtJBeY6ZS80ebTXv6gaAq39wjgvZA+8O4IwA2LHhjbfWCMUQwuef3PtB22f1LJGxdAySA0R/PTWzp8Kg0f0cPbLDfpng4OFQDDN60UUX4ctf/jL+6q/+CjfddBMWFxdx6qmn4vWvfz0uueSSzvm98Y1vxIknnojLL78cX/nKVwAAv/ALv4A3velNOP/884PvzX1x9913H/7hH/6BzfNZz3qWc0D3y7/8y/jhD3+I//iP/8BXv/pV7Nq1Cxs3bsTZZ5+NV7ziFfj1X/919Hpj3H3DQA7oBAKBQCAQCAQCgUAgEAgEAoFAIBAIVis0AF14QrfKDvLOOussfOYznyn+XmfqecEFF7B3zXH4wQ9+UFyuwZlnnokzzzyzc7pxIAd0OTRWAbO7FBY2jxhjZvfI/KHut9Yi9bratTDlsmosPYaHjcwafItVznPOoF7bWLkQj7v+PoXBWmK5BzfEVDVoPR04S8elRyyh2t13ylRLCvrIxdHvB2ftmKdef5bemoRoI5YaRgBU5IJqG6ZgOKIbGFkO+d509awOLIbUMBJukgmDRMOatBYhCqqpSD2roY1FdFN33yLEt8hRA9eSxfw2bQ8Qr0pqjUcsVapd/faZ16ZOf9NwCQw/sB4fxCOHhnM485nbAABf++qpti0tnb7V81zIX5Qmn5bAq8Gz8KzpJfIklKlF3V6+zVmH6lkN3fRDRTyZqTVWEAZLo23zAYBe04+m6ZdUG+qs0m25jLVYNMQI+dZYM/aaEFGUdyktNimxnHJ42xtD9H2s/Hpdw8P7emx4mTYEY+v5RMcm59GjFeOFRPLSilrbNc9o+FRNrM/q0KrN1s2zqFWJUDP9+2ZsO1sLMNpPNMwQCVVmvlWVDsIcKQ0ow1PUwt4OImDpiBFR1b6e9dRkw2bUYK0e7Ws63pjwcTaMS1+3nsnMGHS8MDivN8JvVC7GwvQMmwvLTWhdwO1nX4bSUCs05HDAb4CTrl7X9N1Cxc5Bsw+Oyl/aWIeWlj244Wkz1oS+V0Q923rpqrqVA+2cqBxLyjZkb/vMqZMf0ofI3eAy++Z94L3p0R5Y73pW+9TrEgDquRo94+1FrLSpDLVeOkzIVOrRU/d16EldtXOYntFsOG8O1oO8avNsPR2IXGDmssUjarfOCWtw1ouKzj8R3nDk7RpXnvT29toxNVCobAVIOQlL1Zk9FZYO810beDocEJr89lV12w9qsfX4cepJ6Au8CQlvVEsROUDCblGrZ5ueWhmb342MqvZXoRwiaaqldl6D9cZtw6NHrW0NzYuqlWO0nUw66pVPLPyNhx9n4d070OrbNNS48aSollrCaN3M/K5VS0vvQDtGLOlc6MJaud4rnrzUtM0WVdgPcEN99bzw67pyQ0DZEOGM9wcFlXFm3aJ7YVgx3UNr9U/mL1POzM7KegBQS/xkmDrfuJR4INj3xDNZ7R4xUt3MWb29PfT3jmgabKgx3Dh6rho6KxIusZon3uSmv2kbe5bttn6W6PaZUw+qZzb5mDTVoF0nGQ8mNWj5WfcAPec2UCz0MdUPWK97hr56jug/hDesfjTHhE6uCZ/3NB/NhdNnGbnrRApIeFk53tDUs5p69c0RPvb0g+GGIfqEZp+/Rm3Qjr1cCDcDPzIHhe8R6c9RvXnFr0tJ+lJPg3pt653aRgNxE+c8ppzwgPDamZk3qKcFF/2hWlRhhBkFd81sZbQhktSJetM3tPXmqzDcGoCZPaMfi0fUrJcb5yHleFpyHkfUo4dZq9voQnsV+o1cnD+qDr5T9UinNfWw80XToDQyx+IjBtb72hl3Hi8F+Rs9babdJ6FjlNazJKQdRT2j2zWFtzYw9Pkhh+u5ur2KQ4c8N4rigZY+v88qIo/6CPYEaLST0fq++d3wC93P4nS+3oLCwKwVS8JpU8JHP8JHJKx0NHSweUSiQ7X7YW2bcfuDw7Xa7icuHjm0c227bvLGuxeSXmmyVqP6PO1TRkbQtrXpGd3SSU/ec96nNgS1Jxd9mn2afPpohBY1JOqdGQ/e1T62rUiUCbs0iK1bmP5zIn5E1mv+t7H1td3naYif3V1hsYkSoobEs3mOJmzzMnsea34y6qiljTXr8TzcNBpQak+fDZfpzoUerUT55Obpxc3Ddi6gujeh07bZDNjyBQcPh6IHnSANOaDLoWH6hc2tNjFsFl9UYOYO54B2gqoimxQpxd1fwFE6AG8h1aCeIUocs5Cq9vDdX+1oLzf0JxrnTqF+KJQdpaxWwd15uhopJeaZTzc9YKQ0sHeacAdczEKMLhqqJQW9swmZRwUaOVCwG3gkdA4Xs9sWGbsXyKPX0MQqDdZ9v6UjFbYj4BV/8gTw1RtPHT2ifGrupvJ153n+uaU5UUWtECjbJp3zLyXX2Wxj3lM+48IIMc/oXYLOAZvhg752SeE2y1JKvdf37cZWq0yyMGOIKTsIP5aaZMm7mQdGPDykh27ct5VGnQhj4G+kDb3NLH9DIQwz5C5K7LcF9zLRfPwyDeghC4tIOCN7MDYMN1qcDTayGKFKebXPNBq/UeNmGNJtwI03J4QGkY+cjLffMSFVnfcV3BCZ/reEt3QPqBbCAUR5g5PBzsYOl9YLWQOQeTEiVxY3hYPfoYPyUSTEWwzu4rH9v/LmNFuWIbVQ0eWMa6LfRLo2SOcvbv2wvguuUU8gQxXhuYhKwm2GUVlpeTIWpogBG2qMypDUAZrmx35uI9PKoFi+lBYamsk/6CdhPqulkNbAWMMrKzic48CMDSR0Cq0wuivLR+Lgw9IKb4xG5Lc5XNJEBrIbPDS8Hzm89e8P9g1crGGN2aBR/GGRW5nmH7rZxG0acvNLhl+pTuUeVLY0c4zk0GxoMYctGX7VfXcjNTVX0I199zCMzGte6GEnLzqXmY39WHuTRHQz0p/raN9zbWtD5AN8mDoONa/Pc/og3XBX+9qDUjrmzB16bEgzrp18JHjfIDh8Nd/4tDe//XmfHuSiTs/1FNyBLbdR59C6qNi2oHe0BsYwPu97bRLc/wTEQ3kyB4jsGoauqyL1c/jYK7+3v+J1s5iM8NdtDA8CBfoeyc+f3x0ZEymrFHRzPrbGzM6RjOFHMN7o+9gYpoaXDOyGfcbIkL7n7nofJRz9s7Rh9KN3wOtnOi8zSK0Xo/3gPR+s0zYsJfedVnDuSvQN2ugaw64lEOkvykdUTHh7J4DL05y+nguta1Byl3G4R1A5Rg5cHuYATg2ZuZ4e6HNrfrJu0SoMD1j3vYWHV6/hnLdWjITio3nSAySKqIzNrM8Dfd0zgGxlZ5uRmcuqhdboKLYfw4X91Mwc4X9DoYhRqfM+osayOhO3Jo/oQqyRLtmj8+mj/ezkQ9Yljk7rHWJqlZ9fuYO11B5kkD6hK3DyxxzOjQhGcF+6D7NuXjw81imjf8z1Hw5N4OsXZNHzxpPPJ0vEiJ7ZT6PllLSZYJkxho4hWN2QAzqBQCAQCAQCgUAgEAgEAoFAIBAIBIJVDFUa4lJwyEAO6DgY6ydi2U2tDk14m3qWWLRuXkJ1/4yTjW+FY6xreo/dDwAY/midY4Hne7M4Vn1NSBRqkdObb0NcGq+var5qvUPIJaB+mDcAqA8fWAsMailJ3d/9sGTDtbX1vlADYjlHwmJYr7teGK4SICEu1+ngAno/vI2h3XoLOh4zaJ9FrMQtTSZNBTzilPsAAPfefnSbPuPxQ9vEWLb39zTh2Ta1oQk4UKu9uu+GvzB0BmFRqDVMJF96wWsbh8KYl2rUh48K6u3sW0viunL7y9DXusI3eVPL4z7jxaRbGllLVGKZpn3LNwD1YUNr+cx6rRGPF+Ph4Hu4BaFIqcUP+dZa+Sy2FmrUioyzNmQvoNdoLbKJRbENO9Uj1mIxi2IOjHeFH9rOx9CERohYBbeWTYq19qft6Ix3LjwMaVPfApJaQTtjMGKd7EPpNhwUIqEd/TA9uTyBlo/rNdqGwzJtpmq046RqPQgUCQFVmzC4e3otn0c8du1P1pq8HY+xMEPmWRtSVwVjIhZOygFtH+/ib8fTsW7DLJvQOw5UWBfHgr4PJ7SXraoJS0L6cc2WPQCAhe9tbMcxKWfzf46+vf80HXpjaziezVxYzRhPB1XSbZs44d9Im7TzlqmQlwfXvzR/M2bo/OuLzTntWsZ7lqK0TtVAYeiFY9R93YY9Y8abJvRXS+3Yaa1kVXauc8LX+rKLsZ40tAIRC3sFvs/ov4SffCt3NeBD/znRASJ9buhMecDTucz+H+3YoWE7g3o19Fnejlmu+jKSzp++d46hveH9atCOLSrXubDfhk+Ga4hn1JJqrf0JHamQOHR+czy/aEhHP/yS17c23FXTNr2jFqB/smb0jMoYNqJAy6dcOEiQPqW80dvXhIhivA8dD7zK45/mPQ0RbklZ2+ioByo7V5vQvAtHtAOE8xJyvNqYfjbPR/+SuZobW6Se3HjTJLSzobm3v+J513iNqjZMUrWzH+hUuiJrC6rjcnqWkf8D5UTGYGmlc6pPXwU3xKUX2krVwNyOxsL8CI0Nj38QALD7+5ua8t0+NYjqD5x3q03EfK+Inszp4CoMz6aIvlr3tePZ55ftzCWcqI6tFak88cO315FxRMrkPJOcvjPzI1lDOHOe6TOTD1nLRT2HOE8Gcl1DfeRoQKoH2rV1zosn5q3vp6f6rNN/KRDedEL/c9/Sdia8kQVZt9C5GHBlNitv/Kz8JYzvDW6+M7J0rUZ1wDBSk8Zbd9nnpAwnTH3zwu45zPC6L50fnHCLzU8rY4+s23rTcUP4LdsOdL3itwmTtjevbLh/6q1L50e6NzJs1gsm7GW1pFqvw6OWoHbOtPWj/0agFfEwnG29sOkYza2HuHFI37Fjj+rDXp71uhozO1sv5sAbjS4lIrKUtlmg8wyU3edSA9WGCabeZDRai0c/FxbT1NWkD7wCBwrDtW7EJPMcGPGu7YeZcA0SVDGhzw8PH6C3y91yHUUaaGWs5VPzWWbvQA3buZbySSq9rshcveh5STF1y4XRDfR1Tsf3843IY5uGmQ8U0QmcIglP2DQZDzK/nRQJZUojg2W9whh+8K9/AQAakr1eo9Hb60ZfctYwqu2fTdtG3+0+UbN1aj2XFTjP9dwao2bqT+WGrtv3OZ6i5QlWAB0jDAlWPwoDGKXxjW98A295y1vwkpe8BMcccwyUUlAqP1L//u//HmeccQY2bNiAzZs34wUveAG+9rWvJdN89atfxQte8AJs3rwZGzZswBlnnIEPfOAD06iGQCAQCAQCgUAgEAgEAoFAIBAIBALBqoLSgNK68G+lqRWUYioedP/7f/9vfPKTn+yU5g1veAOuuuoqrF27Fs973vMwPz+Pz3/+8/jc5z6Hq6++Gi9+8YuDNNdccw1e/vKXo65rPPOZz8RRRx2FL37xi7jkkktw66234m1ve9s0qsODWoqSi3Dt650zodUETU4sRYc/XNfmaZCxhnLuDGreD9Zre2rea6xbnVj9xEp4SO5ksqBxwokliNq8MPp931xgEdE7UGG4rrXOtZZvhHbrobCoAutopUeXPo9oUYHnGb2E2pI5p/PWcpzFjalrrZx48T+793Cbl607+e3XCZ5ldUXbGqP/+5bHAByrffO+GujQYlwjvJjbs8JheYJaLVorNlL+/p6lmfNktMWT+teNoZ9/Z09gLeflw1qOk0uDffQf7Kfveqjb9I5VXzzFiO+Yy3BN39WzxKOzT+gilkO2fozXH7USdkg13g206FJLoohFq/WS8bxdWGs2Mna5eO7UkrUlkCR3LK9Ck7CU90jUCrqw/jrSpvb9DLlXkLmTgY5d1xOwGW/znswz+XL3OxivRAX0drdjh4uHz1mjc3WmFqF07AQ0KbherZ7lefS+FwLazxW9i5H8O/pAtZfNc5a3xNrOqSd1oMh58zVY+N7G0WfEuwOk7fceQ+SiNxdpoPX2Yqz+c/C91dr5uxmvA0VkAPHULLS+dcqid0XZxvOsmIHRpfMExkrY90YCRv1oPSFN29U8EY7XnxkPszocW0o7dQrmBYWkFbUilqDO2LOe9Pwcw8otajGauisqNk+kxpNDs1cn7/4l99u206zn0GI4/znpK237RSGcCxwwOgfn1aAVbL2XjmwjHZg61zPu/GTKW/+T0Y+9xxL5Te6a2PyE+wEA999+JD/2GcthXbW6hjpgZDFTNb9tTP2aK431f6/xPPTi5ug0ooDzlrEItve2LFatN7Zmsq0w8iYEANXKG3oXUvbOxSbPhSONewR5xbQn9SqILYzteqLmPxgYb+75ys5rnPeko/usGaXRCxVv8UxpNV7Wum1LalVvvBI5me94uJu29XSCwMNee7o3mN/m06rlheoR8wCA+qdrsLC5GWc9jZ33bxglN/pmj3jwUa8Ekz/1gvLqY+lI6IbQCKz6dUXoJ95qdl3Q11YPVTXae4E53UQRbwHGe9JZGzDRC6hOQutuI5esDz2CHS8pAuq1YGlmoqUo6q3NPGPXMBEZ2W9kzHBWQz04E7y3Og3Vc5h5Rfd4j1hL34BXKlhZb8cYyavm+4HjidK7XGleNE1vPuQDzovJR84T0/dmq+apokPmeZOEWRbF+nUdmYvouqq9J7wZ10uK7b+l5t6mymtz9aiRHADxxs7BuZvV0205DA6rMTis+Y5EZnHyJB4lr3raVwEA//Cvz2hpasrp/XQ2uCcs54000ieZ7zkZGVl32no+Yh76p2uYhEz5JlLDjA51gUo7UUjYNb/Jp0bI85q0GTM/O9EhKkCva5SOeSID6J6KJ9dHfMToEpxcszJCOzq18uQmtMvHOXARZmw9yX4b58mvhq0HPbsm5oqvNAabSLQXO9AStKp2TlWLKhjuQbF03mxoM2tltcDvIdqihu0cyEXB8OkCPN7RrX4WRKlo3pPllqWX9fqn9SFjAwD0T9eQOTvkmZJ9R5q/L5dn9yksbhw97O+u2nHk6RkAnH2u3ceny7LrDo9m4wHK3tVM0lZLpE3puhtwdVtfZzK06vbbmHe24CBB2v4hB6X15IFL//qv/xr79u3DL/7iL+IXf/EXcdxxx2FhYQGxrL/whS/guc99Lo488kjceOONOOmkkwAAN954I8455xysW7cO3//+97Fp0yab5oEHHsDjHvc47N69G9dccw1e8pKXAAB+9rOf4eyzz8Zdd92FG264Aeecc85Eddm6dSvuvP9+/Nwf/YHzvCSMwsHGctHEhQnklQMkN9dZJc3PP0kIUzZdfHL58Gsud3OACzdlflL3f+4i3BidmTJLkQz7VJKe2bTjFiWd+Iapn897nfONtdkE0L12wyM6NpiF4HKADfOQK69DmyRDyoHng2ixiQ0JTlGl8NOMI49SvNMlz2K5VZL/GGO3FOzGgM6P+ewYy4XqmlJdxmqzcWTklGWENbxQzEKPfjeOjJwiJi2/U/op8XkyfOsyt+M09aDoRvKkfLiM8oQrxz9k9zeSnQOFLjKS/psLv5SYK6aK0npE9FE/hDGQnwu78DY3NmLfTaSncWXGNoKZkG/R+dc70HY2uCJhO6cKn3djOglt5wZ+KFg27wnG5ErPFTFk1xG5McNttnYcBz4/c3I1Z3SUwjTk/tT6z9RzmGjzVYBl28fgdM8Mj5WudR2jmg7rGjavcdcTyQ/RHmxxG+JjyJiAzg5zNTCddVksb2BkDMmGpCeY2hphihh7/wOYio6cKjfbT1Nqn6LrEpYbK7g+n4aeNcl+16R7M8XlAZP17xj89uO3XI6TjjwS27Ztm6BgQRds3boV23+wA0/7hUuLvr/p5quw5bijpI8OAUxFlfzDP/xD/OVf/iUuuOACPOpRj8p+//a3vx0A8Gd/9mf2cA4AzjzzTLz2ta/Fzp078Xd/93dOmve+973YvXs3XvSiF9nDOQB45CMfibe+9a0AgCuuuGIa1REIBAKBQCAQCAQCgUAgEAgEAoFAIFg90LrsT3DI4KDbeh04cAD/8i//AgC4+OKLg/fm2bXXXus8v/7666NpzjvvPKxZswZf+MIXMD8/P22SAbTh0wCMWi3iGXYw47uWWl+Y38Flrn4oR8D1MMLI2saG5/Q/j3lqNH/GOtcpV2Hkll3zYSPgua+H8TTIzx6cMBDmPdsPJi9Cn3mmq/ZP6ZFLvXWr5+C3RazPKf1c+3FJSJ1sPTxrOJXKk5RpvvXDVUWtyDPW1k4xHXiPRe79GBZ/JgwXEFoQ0tAVqg6t/8aC32ZeO7f/QZklUu4bkmc9o5MhI7Rq/3JyieMJkzb4tuFP7r2TR9M2JfKQHcdMnk4/Mr8dmrjxnst/qKxMtHwSkycEyfe07Gbe0JXbP1oxbceUEesTixpOeC1KGx3DtE7ZOSsmYzhwfE5kbbIsTiZ1lBHsXEOz6+nRX0wGmu8y7yed5zl55JTP9HNRX5n0Xej3+qxLOWyZnnWoVgA4PcAvKzU/pp77Y4eiGW9qqMJyYnKbk0MFbZFts8I5gPIDN3ZLEcw/tgC09W1oCsYMJ9c9nckPA8vVnepV43iVUHnslzUOn0ZpMHqg4scefcbONak66NHaoe7n+9EPT+aXCaDlaU4P9HXSkjlfJcoCGdOR+XVsz4xCfZiWlf2M6PAGjvccp5NkeD+GLB94eQV8qsD3W6ScXP7+t9kxl2tT5r2tK2lHrhx23cHVMdJ+xfNcQftl8+DaaZx8zXioiD7rzzFmjDFzPovYuo+jtZTMCddt0feN7pkFsxbNlTlNj8TAK6207VLjLUK/naPoWrN0H8CXHYUycFqgax/2PRea11vXxNJF197+nO7LSD9PTsZm2ncSeZMqO/qZpw9T2RnMtZnx7lyBQNcQ4+hBHfip5WOij004dnLlO3MIN/YoH40hA7Nr9oI8ct6Qsfy76EzFa7mYvIitd0p419szyOkcghWEbvWK3J+Ewjx0cNAP6G6//XYsLCzg6KOPxjHHHBO8P/300wEAt956q/P829/+tvOeYnZ2Fk984hMxPz+PO+64YxmoFggEAoFAIBAIBAKBQCAQCAQCgUAgWAlolHvQyQndoYKDfkB3zz33AAB7OAcA69evx6ZNm/Dggw9iz549AIDdu3dj165dyXTm+fbt2ycnkloKNL+rJfJ7QaFacD3AdM+1CEtZi3NW0s5vzop5Ro/uRiPPzUWofv7tB+3HxqqPesMNN1Az6OZPA/UajXqNHnFHM56txQqxPukfUIH3i3OPB+dpo+F4MqS80Wx9qvbPsTykVnvGMql262rrS/Ifbl7CcPMS6hmgngH0nHY8g2Z3Vpjd2Q4N3YPrNWnapPmrlhRrFVkN1OiuMNWmcUYctapsrJNM2wCEn7zLWumdIKpu20XVsHWqZzTq2dGfntO2rHE86KJ9Qp7ZNmesZzkLm95CzMSOoSNhuUPz7h1QLn1+uoq0c4WAvwKPDp8Mf0z7Xla+pSTznf/M0OXztvOesb6t52rUcy7DOfQZby0il6J14to84iVsx5w3Ju09EWYcDpRj8ZdCPVujnq15udfXTv6Gd6l3oPldr61Rrx21Sf+AQv/AKDOOFiu3SJ8PNw0w3DSIynWnHwjv+WPKkfXE0k8NRn9QpP3MuJzRSQsnxzKbgwL6+xX6+5s6+23mjRfj1RGzKGX7n/CUX1cA6C0q9DzPY1P/mAefkbXcfZ+Vzz++xWyd6Qfvfb1hOPpbV6NeV8d5nhlv9LfjSUDqZMB6QdN3pM3tXOvX0cvfpqm99vV0CserIeMxBIzmroq0fWABHbO+VLzscDyezLuaz9Npp4RXge+tZX+b/pjRvM5lxl5FOyCUB8782uhZNM9qSbn5k/qXWpAGuozXz3beZGQc910MRjelqGfa9MO1NYZrR7xv+7522yLoB6K/mDS0/XxP7VYfcD1W2XxjU0QFRxcy35u86Jix9SBzEsezVn9WbhsaPU33RvON9u4so8+cseWPMaYuxnNu5LWfng+pXHX0dVp/06Z0PDFr7npOozZ6H1eWWU9oWHkYo6+3oEb6Gjf/UT0q1Z9w+SPpvenxkukfWye4csvM26YedD6rNw7cedXXTWl0EKYd7Tt/XddEAXHmZBXmZfXNMlXIwq5HOFlO12V0/iM84evjpq6GDwOvCNXyub9GBeiYpkR6TUV0f60SugrN0/AhzdbXtxidCxjpe8lIB1VYjxLEdG9DA83f/mz4sd4wRDVoonnE9H1mfmdBpy3qbUbTcjxLwfFOqkjj+eeNQWfcelDajTpjxy2Zi7h5y4zXWCQQw5tGpwcavbVZ19Lyi1F5dfFkF7f+48YDQMbYUGF4xBKGRyy5c4WZF4ft/gCnT7L6Nl0b0HUryd/pF7re88YdbatScDp+PdvOO3T9TtuMyp0AZP7mojL5azxo98/X5+nawqRl19WU9zz9y9EnIzKE42OOH7j0zly0tg7Ga2yfxPC4U2eixzqynraZv35gZIye0e3elFOByPdWnrf6WCzSg+VnMm/6awwgslakfER0X6uvm/9T3mH0PLYfvXpZPdLIEvq+ZLhwvBUpC4C7xuqRCEhU5yFtqueafTtGT2LXmt5cYfrXroEojYYHVViPmCzVMf0oUWfBQYYu/BMcMujnP5ku9u7dCwBYt25d9Jv169dj586d2LNnDw477DCbJpVu/fr1AGAP9XLYunUr+/zuu+8GNm1qGbn5t57VdlPCKjxUaDaHVwDYxRh74W8F6Jr5hlOCmQ3M3v7K2cwJQDemjOA3izsAvX2Vqww1tFXzqs3Tr5Nufw/W6lBpJRsxzmW0vrs0eFlhNkCdMunE5W2EO99577mLcJUGqp0zbqHehtbCEd7BRyYcou5rtv1peB1H4eXyMIoPVZqY+tl28DZo6KGd/YDhp1R4CYc4b8K3v0mfOM88WhWhievn6KKB0pEYDw4tDQbrdfttzfR/7fWJVxf/IDlXXjB2yNhSQ1eBDqDIvwxvsyHKSJv2DlQBTQ59TZ6VDukO4G0CRp8RBc6RWw1N1YDU2fBzgUJQLSYYheFtKsNo3aoDbaMN6aFHFWZsF3mkfaq9PZunIjzT5kNoSvCKM17pc0YW2u+WFHQ//h66DeEaCxc1WNe2OSfXSzcJ6Xh2YOR6Y0jg08JttDgLR9LmJu3Sxrr9zm/HnvfAkwfBwSIzjh3+aPqXmzdGFQjIj49JQoefF5WxzqEUQvlrFv6pud8hcVZHwwtx5bPI9Jnlc+6w2K+z36e0z6gs536btAPF6lQ2G27+Ib/VIr8BHtv88PmK8p4iuoCVZX4b+fzDjRfvGTdvxmj1aY7JO/qdPfAl7cjJMDpvmL6PbShZkPQmf27TO9BTvMxS48cH3VQIZBlG/WSzN/0Z24gnfMa1o9EJRvNX2AD04CpZb24TZeiNjcQ4r5bIXNeMYV8PMJvgsTnAlm/0yEgb0/VENW8mcEaP1cBwLhyblJ9LZVfOaCsmT+t1o0ZXZEOPyq1qd99NT8Zeb1c/DMFYt+3KzTssTd6cb2SIMyeT8ll5yOUbmZMDGU6z4daMkfT82ohk1iN9q8N1Hy0noEm544Hq/gHNDLTi17Wl+n5vfxV9BzQ6eOZAip2XEnQE5Ri52/Bjb3ePLZPVSQp0YzafDumU4U86p2cLIWXZjJp/agT6k1aAZtZzdI2nSXqrb+9Pd44dY0SvV0MAniFBp8NvKq+oiPTXr3VBvnbdp9Hb1Xdp8WQFyxOMzhY1uqjCTuHWexzYMZYDI4vVoDVKt3MCpaNmyeTzZPV99xm7v8PKs3QdUu2kFdoDT86gVHt8zOgiHG0GdK6pFkKCzaGrn6WhpSS8q6MrFMgGtaTgGz45/8Z0hkjfWv6g/ydzgJ2rSb5DRl7ExltgMMXtjeTyich2Z2x0kcvcnhHJ239eN8ZQQNOOvjGU8uTBotuolKTsuNdeaG+TD9MGdH0e6GS1avWDyLzTSfYKlhVK7pd7yOGgH9AJBAKBQCAQCAQCgUAgEAgEAoFAIBAIOkAO6B5yOOgHdBs2bAAA7N+/P/rNvn37AACHHXaYk8ak27hxYzZNDtu2bWOfb926FXfef79jlQyMLI9joRgMOAt++i7wBCmitEnDWRP10Xo1pCyXSt43Fh26rx1PD9aCh1qK+pa+vuWVX8kKwKD97bdZ1JK81JoNoaWna/naWiEbKzDq+QOEFlW+hXngzUdpi1gG57wKW+LDNNSyLGs5TS7OrZtyaP1ifJDkXcYiiHvWGb7lFvWy5CwYCT+xZXv8ZnnTtAOxUC+mDV6epPyA90n5dZ+3OA4sNSNWviYkQkzmmBBfioTaZb1Iqog1vnmvMnUCeJ42iNSZ83i1STzeMeORWn21HggqtCLz+sPKDsYSkspdx1qa8rux6DV0zLYWr2oAzOwd/Wfx8FgMvrYsm2eDuplhqUeIQxP1mGCs3g16i4r1ZDCoFlUb9o0Jtep7OxkLPWpJyckb+86xttOhtSGRN05IQmMpWLXzivW+jKQx0BXp0yUVyktvvAeWnT0E8yNAvICIRw3QhtTUjRmwDSkFl09cU9fmH9qnnDyLjEETsk0tMp0OuN5saGQY8QxOynUV8qSe0UDCalRpoudXCNrcGbu0/U2YRM5jhNDB0tzX7Ngx71lZypRB6xHQTL1fjcdU0w6OofqSckJUJYskVsKBt3ZHK2QOjjecp4sEluTceGXmmJwXE+dN51iBE8vewLJ5TrceiJrJX7eZ6p7Oz1sDt1NpP1L9wHoCZkIqR70iDG+QKBiuybJLl58nh7btW34KvCLMt0aux8J+e151Prp4o4WJ2591fyRfAK+dmUgUqbI5XZHm6VhuR3Qr+6wGMGh5zqRxdBYjt0k7x0KsOXRMAjOemLkAdTt2rccjXeuQeSm3CHTmTyI3U33KrjX9ucynWfO87PQdkScmn9QY7u9TGKzNrJlpP3qf5vKv59qoNjRihcHMngpLh9Vtem8u1T3iAUnLTazPqiXV6uRclAm4zwI9WDPPIqjX1KzXTScUrtuMB1w1T/Rtfxwizne0n5Lyhq7lDT8NXI8eTl7472LvHVq5pqM6jSmnx8hWquIycybdB+BA39VzOu357OmxlDZTpuZ0a072Rtosuq8So8mL7mTK7joXRMHU2Z+f/XndkZscyQsKw2a8qwWV5tnMXBCL8pQbB0GUD7JH5/C+5T1N6sc3IO0nq7MaHbXn0pjsU6LPOW2b0VktyFqPk5Gx8WjX1XQvI6WvD928fI/sVJjlrnDkFtGzSvX04u9Infp7FBaPHg3k3r4mogVgI3VUi6rtErM+9HTDlIwE+HmHA40AEHj/kshDRR7FgpWDRpl3vPlWcEjgoB/QHXvssQCAH/3oR+z7ffv2YefOnTjiiCPsYdvGjRtx+OGHY9euXfjRj36EU089NUhn8tuyZctU6Aw2++hmxBjKiaMITEvQlS7CY6BpbGyCeBlc+anwMeb/Dn2eklJCd9T1veNzU6Y5CODCwIyeuw8CXugQqsn/pkuIktzkyoIsrnIhIpyyEm1WxFvc4jqnzCbqxR6E5tohdn5CDi2LN0gjZTnpHKYG1ADBxkg2z1h/+CEK/GTevVFR0PHGleuPwUy7BGVRnZtT1gvALUy8pi2ih9ugZfkwIuPYjVQFDNZEZKOXngPLB5TM3BgxRWTCl+gZXRyqdUQDN4kl8qdyKyLD2DnQHHYp8KE9cwcvdYqoTNrI/BgL/8aF7cy2Zalcj21sxQ7mDDKHkpaMyKZrUJ4fWqWj3I62BxNG1iISki13WGSzHuTHR24uceSU2RDh6peRu0G+aGgvXBx10tPqdvO5a/oSme4flJakT7WPz8uBXuUVlp23Gp7KbYrm7nbLlkf5vePiNdsflc7mWSoDuEMMn4YuB8CjD8O0zuvcvDOB/l7ynWaMBPxvNQ3TmCpzmhsTGf3OGjxy8q50IwVw508/nxgII8fCA7cPI7/9NEQnytFing1nM3Rm0sfeG8RC1hosrfdCtvqbmMyhHved88qTf8FGvlceNXKwdBaOBxv6PUNTF0QP2Mx6QuX7pEu+wXec4Z+nOxSvRaO6bYIAbv6IhDbPydocnTaM/iBzB2VmHySmO5fyLns4FzO+TuiOuTVdJ92mgM6crhqk7+uorhkrf6Rb8etGNllXPdTvN05ecbpzZC4L9C8mZGcOWpXrTE66nNFupH9YI+MOayWtvMc5fapgX5E1BsqtRRkU8zz5briGD+/LrUtiumHRfkEBUjKky3iWA7yVh4S4fOjhoB/QPf7xj8fc3Bzuu+8+/PjHP8bP/dzPOe9vueUWAMCTn/xk5/lpp52GL3/5y7jllluCA7qlpSV85zvfwZo1a3DyySdPRN8999yDpQMH8OO3XD5RPgcFY2woFOeLZcx7JeVISfnLWf9c/tMse7nrMS0cKnR2wXLz+aRtthxtvtJje7lAD6a64lDk7XH6sWTjpGs54+RZkv9KyveDgdz8cijx4sMZpXy0Wvp0peg4mOPNnwMeCnrapHK2azkaK1/n5YZfv9UyRh9K6Mq3XeRpaZ7LiYM1LmlZK11nioNF08FYq62mdk0hx3MrMdd2kaGrkY8NVpoPpjn/ruZ2PlTBtWmunVdZPyzt2IF7mmh2goMMOaB7yOGgH9CtXbsWz3nOc/CZz3wGH/vYx/CGN7zBeX/11VcDAC644ALn+XnnnYcvf/nLuPrqq/HKV77SeXfddddhfn4e559/PtasWTMRfevXr8f+/ftR7dyJE044YaK8BAKB4FDE3XffDQAiAwUCwcMSIgMFAsHDGSIDBQLBwxkiAwWCMtyzbx/Wr1+/0mQ8PDGUA7qHGpTW0z92XbNmDRYWFhDL+gtf+AKe+9zn4sgjj8SNN96Ik046CQBw44034tnPfjbWrl2L73//+9i0aZNN88ADD+Bxj3scdu/ejWuuuQYveclLAAD33nsvzjrrLNx111244YYbcM4550xM/9atWwHE76kTCASChzJEBgoEgoczRAYKBIKHM0QGCgSChzNEBgoEgtWKrVu3Yvvd9+LsU3676Pt/++7/w5YTHiHy7BDANK7LxvXXX4+nPe1p9m9xcREAnGfXX3+9/f7cc8/FpZdeivvvvx8///M/jxe/+MV4wQtegGc+85kYDAZ4//vf7xzOAcDmzZvxvve9D1VV4eKLL8ZznvMcvPSlL8XjH/943HXXXbjsssumcjgnEAgEAoFAIBAIBAKBQCAQCAQCgUCwqqB12Z/gkMFUQlzed999+Pd///fgOX123333Oe+uvPJK/PzP/zze/e534/Of/zxmZ2dx7rnn4s///M/x9Kc/nS3noosuwpe//GX81V/9FW666SYsLi7i1FNPxetf/3pccskl06iKQCAQCAQCgUAgEAgEAoFAIBAIBALB6oIcvj3kMJUDule96lV41atedVDSnXXWWfjMZz7TuSyBQCAQCAQCgUAgEAgEAoFAIBAIBIJDEnJA95DDVA7oBAKBQCAQCAQCgUAgEAgEAoFAIBAIBMuEeqUJEEwbckAnEAgEAoFAIBAIBAKBQCAQCAQCgUCwSqE0oAo96JQ42h0ykAM6Btu2bVtpEgQCgWDFIDJQIBA8nCEyUCAQPJwhMlAgEDycITJQIBCsekiIy4cc5IBOIBAIBAKBQCAQCAQCgUAgEAgEAoFg1UIDdekBnRzkHSqQAzqBQCAQCAQCgUAgEAgEAoFAIBAIBILVjFouoXuoQQ7oBAKBQCAQCAQCgUAgEAgEAoFAIBAIVis0ykNcigPdIQM5oBMIBAKBQCAQCAQCgUAgEAgEAoFAIFjNKA5xKThUIAd0AoFAIBAIBAKBQCAQCAQCgUAgEAgEqxYa0KUhLuUg71CBHNAJBAKBQCAQCAQCgUAgEAgEAoFAIBCsZpSGuBQcMpADOoFAIBAIBAKBQCAQCAQCgUAgEAgEgtUKjfIQl3KOd8hADugEAoFAIBAIBAKBQCAQCAQCgUAgEAhWM8SD7iEHOaATCAQCgUAgEAgEAoFAIBAIBAKBQCBYzZADuocc5IBOIBAIBAKBQCAQCAQCgUAgEAgEAoFgNUMO6B5ykAM6gUAgEAgEAoFAIBAIBAKBQCAQCASCVQsN1HX5t4JDAnJAJxAIBAKBQCAQCAQCgUAgEAgEAoFAsFqhUX5AJ+dzhwzkgE4gEAgEAoFAIBAIBAKBQCAQCAQCgWA1o5aTt4ca5IBOIBAIBAKBQCAQCAQCgUAgEAgEAoFglUJDQ+syDzotLnSHDOSATiAQCAQCgUAgEAgEAoFAIBAIBAKBYLVCo9yDTs7nDhnIAZ1AIBAIBAKBQCAQCAQCgUAgEAgEAsFqhpaTt4ca5IBOIBAIBAKBQCAQCAQCgUAgEAgEAoFgNaMuC3EpOHRQrTQBAoFAIBAIBAKBQCAQCAQCgUAgEAgEggS0LvtbZfjqV7+KF7zgBdi8eTM2bNiAM844Ax/4wAfGzu/aa6/Fs571LGzcuBEbN27EOeecg+uvv5799gc/+AGUUtG/Rz3qUdFyHnzwQVx66aXYsmUL5ubmsGXLFrzhDW/Azp07x6bdh3jQCQQCgUAgEAgEAoFAIBAIBAKBQCAQrFZoDV3qQbeKDumuueYavPzlL0dd13jmM5+Jo446Cl/84hdxySWX4NZbb8Xb3va2TvldeeWVeOMb34h+v49zzz0Xc3Nz+NznPofzzz8f73rXu/D617+eTffIRz4Sz3/+84Pnhx9+OPv9jh07cOaZZ+Kuu+7C8ccfjxe/+MXYtm0brrrqKnzmM5/BjTfeiM2bN3einYPSehX1lkAgEAgEAoFAIBAIBAKBQCAQCAQCgQAAsHXrVmy/7cc4a+0Li77/6oFPYcsTfg7btm1bZsrSeOCBB/C4xz0Ou3fvxjXXXIOXvOQlAICf/exnOPvss3HXXXfhhhtuwDnnnFOU3+23346tW7ei3+/jhhtuwJlnngkAuOOOO/D0pz8du3btwm233YYTTzzRpvnBD36Axz3ucXjWs56FL33pS8W0v/KVr8SHPvQhvOQlL8FHP/pR9PsjX7ff/d3fxbve9S5ccskl+Pu///vi/GKQEJcCgUAgEAgEAoFAIBAIBAKBQCAQCASrGbUu+1sleO9734vdu3fjRS96kT2cA0bebG9961sBAFdccUVxfldddRWGwyFe+9rX2sM5ADj55JPxp3/6pxgMBrjqqqsmpvsnP/kJPvKRj2B2dhZ/8zd/Yw/nAODyyy/H0UcfjQ9+8IO49957Jy5LDugEAoFAIBAIBAKBQCAQCAQCgUAgEAhWK7SGHg6L/lZLiEtzL9zFF18cvDvvvPOwZs0afOELX8D8/PzE+Zln11577bjkWvzzP/8z6rrGM57xDDzykY903s3NzeGCCy7AcDjEpz/96YnLkgM6gUAgEAgEAoFAIBAIBAKBQCAQCASC1Qxdl/2tEnz7298GAJx++unBu9nZWTzxiU/E/Pw87rjjjmxeO3fuxD333AMAeMpTnhK8f+xjH4ujjjoK27dvx+7du4P3P/vZz/AXf/EX+M3f/E38wR/8Aa6++mosLi52pps+v/XWW7N059DPfyIQCAQCgUAgEAgEAoFAIBAIBAKBQCBYKehVFL4yh927d2PXrl0AgGOOOYb95phjjsHNN9+M7du348lPfnIyP3M4d8QRR2D9+vXR/Hbs2IHt27fjSU96kvPuu9/9Lv7yL//SeXbsscfiYx/7GM444wy2rBTdALB9+/YkzSWQAzqBQCAQCAQCgUAgEAgEAoFAIBAIBIJViv3Yixvrfy7+9u6778bWrVvZ99u2bZsmaSz27t1rf69bt479xhy07dmzpzi/WF6x/Obm5vDbv/3bePnLX44nPOEJWLt2LbZt24b//b//Nz796U/jl3/5l/Gtb30LW7ZsKS6rC905yAGdQCAQCAQCgUAgEAgEAoFAIBAIBALBKsQJJ5zQOY3xApsEF154IW677bZOaT7wgQ8EHmkriUc/+tH4m7/5G+fZ0572NFx//fV4xStegQ9/+MP4P//n/+A973nPitAnB3QCgUAgEAgEAoFAIBAIBAKBQCAQCASrEJ/61KdWpNzvf//7uP322zul2b9/PwBgw4YNzrONGzcG3+7btw8AcNhhh2XzNfmZ/Dl0yQ8A/uRP/gQf/vCH8dnPfrZTWV3LSUEO6AQCgUAgEAgEAoFAIBAIBAKBQCAQCAQW3/rWt8ZOu3HjRhx++OHYtWsXfvSjH+HUU08NvvnRj34EAE54yRiOPfZYAMCDDz6Iffv2sffQdckPAE466SQAwE9+8hO2LJPfpOWkUE2cg0AgEAgEAoFAIBAIBAKBQCAQCAQCgUDQ4LTTTgMA3HLLLcG7paUlfOc738GaNWtw8sknZ/PatGmTPTj75je/Gbz/4Q9/iB07dmDLli2stx6HBx98EACCw74U3fT5k5/85KJyUpADOoFAIBAIBAKBQCAQCAQCgUAgEAgEAsHUcN555wEArr766uDdddddh/n5eZx77rlYs2bNxPmZZxdccEExfddccw0A4PTTT3eeP//5z0dVVfjKV76Ce++913m3sLCAa6+9Fr1eDy94wQuKy4pBDugEAoFAIBAIBAKBQCAQCAQCgUAg+P/bu1+QutoAjuM/8WKUoajYBFEuC4JgMN6gzbIyGAgWyzZFULwYFbQoIpgEs1FR1sSyINxo2EQQMRm8DrEJIt6VF0HcXt6XjZM+n/j8Oec5T/2GA/w1k5OTaW1tzcHBQfb29p7H6/V6qtVqkmRubu7VvnK5nHK5nKurqxfjMzMzaW5uztbWVmq12vP4+fl5VlZWUiqVMjMz82LP9vZ2zs7OXr1jb28vCwsLSZLPnz+/mOvu7s6HDx/y8PCQT58+5fHx8XmuWq3m5uYm4+Pj6ezs/K9X8VtNjUaj8cdPAQAAAAAAgH/s7u7m/fv3aTQaqVQqaW9vz9HRUe7u7jI7O5v19fVXe5qampIkl5eX6enpeTG3sbGR2dnZlEqljI6OpqWlJYeHh7m/v8/m5mamp6dfrK9UKvn69WsGBgbS39+fp6ennJ6ePke7+fn5rK6uvjrDjx8/Mjw8nIuLi/T29mZoaCjfv3/Pt2/f0tfXl1qtlra2tj++H4EOAAAAAACAv+74+DjLy8up1Wp5eHjI27dvMzU1lYmJiV+u/7dAlyRfvnzJ2tra87/oBgcHU61WMzY29mrtzs5O9vf3c3Jyknq9nvv7+3R0dGR4eDgfP37MyMjIb899e3ubxcXF7O/v5/r6Ol1dXXn37l2Wlpby5s2b/38Rv/pWgQ4AAAAAAACK4x90AAAAAAAAUCCBDgAAAAAAAAok0AEAAAAAAECBBDoAAAAAAAAokEAHAAAAAAAABRLoAAAAAAAAoEACHQAAAAAAABRIoAMAAAAAAIACCXQAAAAAAABQIIEOAAAAAAAACiTQAQAAAAAAQIEEOgAAAAAAACiQQAcAAAAAAAAFEugAAAAAAACgQAIdAAAAAAAAFEigAwAAAAAAgAIJdAAAAAAAAFAggQ4AAAAAAAAK9BO2bzs9zDglVgAAAABJRU5ErkJggg==",
+ "text/plain": [
+ ""
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
}
],
"source": [
- "get_top_k_indices(summed_scores, top_k=10)"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:08:31.179881Z",
- "start_time": "2024-03-12T21:08:31.158234Z"
- }
- },
- "id": "ced4a88e4f734621"
+ "plt.matshow(per_module_scores[\"6\"] / len(train_dataset))\n",
+ "plt.colorbar()"
+ ]
},
{
- "cell_type": "code",
- "execution_count": 74,
- "outputs": [],
+ "cell_type": "markdown",
+ "id": "7c83722c-47fd-446b-96e5-cb6b8b264a1a",
+ "metadata": {},
"source": [
- "def get_keep_indices(remove_indices):\n",
- " remove_indices = [tensor.item() for tensor in remove_indices]\n",
- " return list(set(list(range(len(train_dataset)))) - set(remove_indices))"
- ],
+ "Note that the scores were divided by `len(train_dataset)`, as `compute_pairwise_scores` does not normalize the scores by the total number of training dataset."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "261a508a6c02267d",
"metadata": {
"collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:11:01.144295Z",
- "start_time": "2024-03-12T21:11:01.134291Z"
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "877e7b1d0b81a3c"
+ "source": [
+ "## Counterfactual Experiments"
+ ]
},
{
- "cell_type": "code",
- "execution_count": 76,
- "outputs": [],
+ "cell_type": "markdown",
+ "id": "35970418-4649-4a8f-886f-ecd4f8e7dbad",
+ "metadata": {},
"source": [
- "keep_indices = get_top_k_indices(summed_scores, top_k=10)"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:11:39.837132Z",
- "start_time": "2024-03-12T21:11:39.833193Z"
- }
- },
- "id": "98f26bc51559a58d"
+ "How would the model's behaviors on some query data points change if one or more data points were removed from the training dataset? We can use influence functions to identify influential training data points for a randomly selected query dataset."
+ ]
},
{
"cell_type": "code",
- "execution_count": 72,
- "outputs": [
- {
- "data": {
- "text/plain": "{0,\n 1,\n 2,\n 3,\n 4,\n 5,\n 6,\n 7,\n 8,\n 9,\n 10,\n 11,\n 12,\n 13,\n 14,\n 15,\n 16,\n 17,\n 18,\n 19,\n 20,\n 21,\n 22,\n 23,\n 24,\n 25,\n 26,\n 27,\n 28,\n 29,\n 30,\n 31,\n 32,\n 33,\n 34,\n 35,\n 36,\n 37,\n 38,\n 39,\n 40,\n 41,\n 42,\n 43,\n 44,\n 45,\n 46,\n 47,\n 48,\n 49,\n 50,\n 51,\n 52,\n 53,\n 54,\n 55,\n 56,\n 57,\n 58,\n 59,\n 60,\n 61,\n 62,\n 63,\n 64,\n 65,\n 66,\n 67,\n 68,\n 69,\n 70,\n 71,\n 72,\n 73,\n 74,\n 75,\n 76,\n 77,\n 78,\n 79,\n 80,\n 81,\n 82,\n 83,\n 84,\n 85,\n 86,\n 87,\n 88,\n 89,\n 90,\n 91,\n 92,\n 93,\n 94,\n 95,\n 96,\n 97,\n 98,\n 99,\n 100,\n 101,\n 102,\n 103,\n 104,\n 105,\n 106,\n 107,\n 108,\n 109,\n 110,\n 111,\n 112,\n 113,\n 114,\n 115,\n 116,\n 117,\n 118,\n 119,\n 120,\n 121,\n 122,\n 123,\n 124,\n 125,\n 126,\n 127,\n 128,\n 129,\n 130,\n 131,\n 132,\n 133,\n 134,\n 135,\n 136,\n 137,\n 138,\n 139,\n 140,\n 141,\n 142,\n 143,\n 144,\n 145,\n 146,\n 147,\n 148,\n 149,\n 150,\n 151,\n 152,\n 153,\n 154,\n 155,\n 156,\n 157,\n 158,\n 159,\n 160,\n 161,\n 162,\n 163,\n 164,\n 165,\n 166,\n 167,\n 168,\n 169,\n 170,\n 171,\n 172,\n 173,\n 174,\n 175,\n 176,\n 177,\n 178,\n 179,\n 180,\n 181,\n 182,\n 183,\n 184,\n 185,\n 186,\n 187,\n 188,\n 189,\n 190,\n 191,\n 192,\n 193,\n 194,\n 195,\n 196,\n 197,\n 198,\n 199,\n 200,\n 201,\n 202,\n 203,\n 204,\n 205,\n 206,\n 207,\n 208,\n 209,\n 210,\n 211,\n 212,\n 213,\n 214,\n 215,\n 216,\n 217,\n 218,\n 219,\n 220,\n 221,\n 222,\n 223,\n 224,\n 225,\n 226,\n 227,\n 228,\n 229,\n 230,\n 231,\n 232,\n 233,\n 234,\n 235,\n 236,\n 237,\n 238,\n 239,\n 240,\n 241,\n 242,\n 243,\n 244,\n 245,\n 246,\n 247,\n 248,\n 249,\n 250,\n 251,\n 252,\n 253,\n 254,\n 255,\n 256,\n 257,\n 258,\n 259,\n 260,\n 261,\n 262,\n 263,\n 264,\n 265,\n 266,\n 267,\n 268,\n 269,\n 270,\n 271,\n 272,\n 273,\n 274,\n 275,\n 276,\n 277,\n 278,\n 279,\n 280,\n 281,\n 282,\n 283,\n 284,\n 285,\n 286,\n 287,\n 288,\n 289,\n 290,\n 291,\n 292,\n 293,\n 294,\n 295,\n 296,\n 297,\n 298,\n 299,\n 300,\n 301,\n 302,\n 303,\n 304,\n 305,\n 306,\n 307,\n 308,\n 309,\n 310,\n 311,\n 312,\n 313,\n 314,\n 315,\n 316,\n 317,\n 318,\n 319,\n 320,\n 321,\n 322,\n 323,\n 324,\n 325,\n 326,\n 327,\n 328,\n 329,\n 330,\n 331,\n 332,\n 333,\n 334,\n 335,\n 336,\n 337,\n 338,\n 339,\n 340,\n 341,\n 342,\n 343,\n 344,\n 345,\n 346,\n 347,\n 348,\n 349,\n 350,\n 351,\n 352,\n 353,\n 354,\n 355,\n 356,\n 357,\n 358,\n 359,\n 360,\n 361,\n 362,\n 363,\n 364,\n 365,\n 366,\n 367,\n 368,\n 369,\n 370,\n 371,\n 372,\n 373,\n 374,\n 375,\n 376,\n 377,\n 378,\n 379,\n 380,\n 381,\n 382,\n 383,\n 384,\n 385,\n 386,\n 387,\n 388,\n 389,\n 390,\n 391,\n 392,\n 393,\n 394,\n 395,\n 396,\n 397,\n 398,\n 399,\n 400,\n 401,\n 402,\n 403,\n 404,\n 405,\n 406,\n 407,\n 408,\n 409,\n 410,\n 411,\n 412,\n 413,\n 414,\n 415,\n 416,\n 417,\n 418,\n 419,\n 420,\n 421,\n 422,\n 423,\n 424,\n 425,\n 426,\n 427,\n 428,\n 429,\n 430,\n 431,\n 432,\n 433,\n 434,\n 435,\n 436,\n 437,\n 438,\n 439,\n 440,\n 441,\n 442,\n 443,\n 444,\n 445,\n 446,\n 447,\n 448,\n 449,\n 450,\n 451,\n 452,\n 453,\n 454,\n 455,\n 456,\n 457,\n 458,\n 459,\n 460,\n 461,\n 462,\n 463,\n 464,\n 465,\n 466,\n 467,\n 468,\n 469,\n 470,\n 471,\n 472,\n 473,\n 474,\n 475,\n 476,\n 477,\n 478,\n 479,\n 480,\n 481,\n 482,\n 483,\n 484,\n 485,\n 486,\n 487,\n 488,\n 489,\n 490,\n 491,\n 492,\n 493,\n 494,\n 495,\n 496,\n 497,\n 498,\n 499,\n 500,\n 501,\n 502,\n 503,\n 504,\n 505,\n 506,\n 507,\n 508,\n 509,\n 510,\n 511,\n 512,\n 513,\n 514,\n 515,\n 516,\n 517,\n 518,\n 519,\n 520,\n 521,\n 522,\n 523,\n 524,\n 525,\n 526,\n 527,\n 528,\n 529,\n 530,\n 531,\n 532,\n 533,\n 534,\n 535,\n 536,\n 537,\n 538,\n 539,\n 540,\n 541,\n 542,\n 543,\n 544,\n 545,\n 546,\n 547,\n 548,\n 549,\n 550,\n 551,\n 552,\n 553,\n 554,\n 555,\n 556,\n 557,\n 558,\n 559,\n 560,\n 561,\n 562,\n 563,\n 564,\n 565,\n 566,\n 567,\n 568,\n 569,\n 570,\n 571,\n 572,\n 573,\n 574,\n 575,\n 576,\n 577,\n 578,\n 579,\n 580,\n 581,\n 582,\n 583,\n 584,\n 585,\n 586,\n 587,\n 588,\n 589,\n 590,\n 591,\n 592,\n 593,\n 594,\n 595,\n 596,\n 597,\n 598,\n 599,\n 600,\n 601,\n 602,\n 603,\n 604,\n 605,\n 606,\n 607,\n 608,\n 609,\n 610,\n 611,\n 612,\n 613,\n 614,\n 615,\n 616,\n 617,\n 618,\n 619,\n 620,\n 621,\n 622,\n 623,\n 624,\n 625,\n 626,\n 627,\n 628,\n 629,\n 630,\n 631,\n 632,\n 633,\n 634,\n 635,\n 636,\n 637,\n 638,\n 639,\n 640,\n 641,\n 642,\n 643,\n 644,\n 645,\n 646,\n 647,\n 648,\n 649,\n 650,\n 651,\n 652,\n 653,\n 654,\n 655,\n 656,\n 657,\n 658,\n 659,\n 660,\n 661,\n 662,\n 663,\n 664,\n 665,\n 666,\n 667,\n 668,\n 669,\n 670,\n 671,\n 672,\n 673,\n 674,\n 675,\n 676,\n 677,\n 678,\n 679,\n 680,\n 681,\n 682,\n 683,\n 684,\n 685,\n 686,\n 687,\n 688,\n 689,\n 690,\n 691,\n 692,\n 693,\n 694,\n 695,\n 696,\n 697,\n 698,\n 699,\n 700,\n 701,\n 702,\n 703,\n 704,\n 705,\n 706,\n 707,\n 708,\n 709,\n 710,\n 711,\n 712,\n 713,\n 714,\n 715,\n 716,\n 717,\n 718,\n 719,\n 720,\n 721,\n 722,\n 723,\n 724,\n 725,\n 726,\n 727,\n 728,\n 729,\n 730,\n 731,\n 732,\n 733,\n 734,\n 735,\n 736,\n 737,\n 738,\n 739,\n 740,\n 741,\n 742,\n 743,\n 744,\n 745,\n 746,\n 747,\n 748,\n 749,\n 750,\n 751,\n 752,\n 753,\n 754,\n 755,\n 756,\n 757,\n 758,\n 759,\n 760,\n 761,\n 762,\n 763,\n 764,\n 765,\n 766,\n 767,\n 768,\n 769,\n 770,\n 771,\n 772,\n 773,\n 774,\n 775,\n 776,\n 777,\n 778,\n 779,\n 780,\n 781,\n 782,\n 783,\n 784,\n 785,\n 786,\n 787,\n 788,\n 789,\n 790,\n 791,\n 792,\n 793,\n 794,\n 795,\n 796,\n 797,\n 798,\n 799,\n 800,\n 801,\n 802,\n 803,\n 804,\n 805,\n 806,\n 807,\n 808,\n 809,\n 810,\n 811,\n 812,\n 813,\n 814,\n 815,\n 816,\n 817,\n 818,\n 819,\n 820,\n 821,\n 822,\n 823,\n 824,\n 825,\n 826,\n 827,\n 828,\n 829,\n 830,\n 831,\n 832,\n 833,\n 834,\n 835,\n 836,\n 837,\n 838,\n 839,\n 840,\n 841,\n 842,\n 843,\n 844,\n 845,\n 846,\n 847,\n 848,\n 849,\n 850,\n 851,\n 852,\n 853,\n 854,\n 855,\n 856,\n 857,\n 858,\n 859,\n 860,\n 861,\n 862,\n 863,\n 864,\n 865,\n 866,\n 867,\n 868,\n 869,\n 870,\n 871,\n 872,\n 873,\n 874,\n 875,\n 876,\n 877,\n 878,\n 879,\n 880,\n 881,\n 882,\n 883,\n 884,\n 885,\n 886,\n 887,\n 888,\n 889,\n 890,\n 891,\n 892,\n 893,\n 894,\n 895,\n 896,\n 897,\n 898,\n 899,\n 900,\n 901,\n 902,\n 903,\n 904,\n 905,\n 906,\n 907,\n 908,\n 909,\n 910,\n 911,\n 912,\n 913,\n 914,\n 915,\n 916,\n 917,\n 918,\n 919,\n 920,\n 921,\n 922,\n 923,\n 924,\n 925,\n 926}"
- },
- "execution_count": 72,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [],
+ "execution_count": 24,
+ "id": "4522bd15c0f00882",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T21:09:57.285981Z",
- "start_time": "2024-03-12T21:09:57.281092Z"
+ "end_time": "2024-03-12T21:05:54.279349Z",
+ "start_time": "2024-03-12T21:05:54.269910Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "92b8efa50738e47f"
- },
- {
- "cell_type": "code",
- "execution_count": 73,
"outputs": [
{
"data": {
- "text/plain": "{tensor(36),\n tensor(217),\n tensor(221),\n tensor(240),\n tensor(256),\n tensor(288),\n tensor(326),\n tensor(503),\n tensor(550),\n tensor(647)}"
+ "text/plain": [
+ "1"
+ ]
},
- "execution_count": 73,
+ "execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
- "set(get_top_k_indices(summed_scores, top_k=10))"
- ],
+ "single_query_dataset = torch.utils.data.Subset(query_dataset, list(range(1)))\n",
+ "len(single_query_dataset)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "760504c5-2b97-4c64-a80a-ce3ba0167d5e",
+ "metadata": {},
+ "source": [
+ "We can compute the averaged loss of this selected query data point over multiple random seeds (e.g., initialization, data ordering)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "id": "d705577ebd58b8c8",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T21:10:05.051237Z",
- "start_time": "2024-03-12T21:10:05.035876Z"
+ "end_time": "2024-03-12T21:13:13.194949Z",
+ "start_time": "2024-03-12T21:13:13.178273Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "91540047fceda6b9"
- },
- {
- "cell_type": "code",
- "execution_count": 77,
"outputs": [],
"source": [
- "def train_and_evaluate(current_train_dataset, current_eval_dataset):\n",
+ "def train_and_evaluate(modified_train_dataset: data.Dataset, query_dataset: data.Dataset) -> float:\n",
" current_model = train(\n",
- " dataset=current_train_dataset,\n",
+ " dataset=modified_train_dataset,\n",
" batch_size=train_batch_size,\n",
" num_train_epochs=num_train_epochs,\n",
" learning_rate=learning_rate,\n",
" weight_decay=weight_decay,\n",
+ " disable_tqdm=True,\n",
" )\n",
- " return evaluate(model=current_model, dataset=current_eval_dataset, batch_size=eval_batch_size)"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:13:13.194949Z",
- "start_time": "2024-03-12T21:13:13.178273Z"
- }
- },
- "id": "d705577ebd58b8c8"
+ " return evaluate(model=current_model, dataset=query_dataset, batch_size=len(query_dataset))"
+ ]
},
{
"cell_type": "code",
- "execution_count": 78,
+ "execution_count": 26,
+ "id": "be1ab297-9260-47a9-82ec-56d1349fa862",
+ "metadata": {},
"outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "Epoch 0: 100%|██████████| 28/28 [00:00<00:00, 585.87batch/s, loss=0.911]\n",
- "Epoch 1: 100%|██████████| 28/28 [00:00<00:00, 638.57batch/s, loss=0.603]\n",
- "Epoch 2: 100%|██████████| 28/28 [00:00<00:00, 648.91batch/s, loss=0.42]\n",
- "Epoch 3: 100%|██████████| 28/28 [00:00<00:00, 635.11batch/s, loss=0.346]\n",
- "Epoch 4: 100%|██████████| 28/28 [00:00<00:00, 629.13batch/s, loss=0.296]\n",
- "Epoch 5: 100%|██████████| 28/28 [00:00<00:00, 619.84batch/s, loss=0.254]\n",
- "Epoch 6: 100%|██████████| 28/28 [00:00<00:00, 576.69batch/s, loss=0.221]\n",
- "Epoch 7: 100%|██████████| 28/28 [00:00<00:00, 671.24batch/s, loss=0.22]\n",
- "Epoch 8: 100%|██████████| 28/28 [00:00<00:00, 635.33batch/s, loss=0.204]\n",
- "Epoch 9: 100%|██████████| 28/28 [00:00<00:00, 637.90batch/s, loss=0.175]\n",
- "Epoch 10: 100%|██████████| 28/28 [00:00<00:00, 633.97batch/s, loss=0.169]\n",
- "Epoch 11: 100%|██████████| 28/28 [00:00<00:00, 661.69batch/s, loss=0.164]\n",
- "Epoch 12: 100%|██████████| 28/28 [00:00<00:00, 651.29batch/s, loss=0.157]\n",
- "Epoch 13: 100%|██████████| 28/28 [00:00<00:00, 666.77batch/s, loss=0.147]\n",
- "Epoch 14: 100%|██████████| 28/28 [00:00<00:00, 627.28batch/s, loss=0.139]\n",
- "Epoch 15: 100%|██████████| 28/28 [00:00<00:00, 613.12batch/s, loss=0.135]\n",
- "Epoch 16: 100%|██████████| 28/28 [00:00<00:00, 575.97batch/s, loss=0.126]\n",
- "Epoch 17: 100%|██████████| 28/28 [00:00<00:00, 592.14batch/s, loss=0.131]\n",
- "Epoch 18: 100%|██████████| 28/28 [00:00<00:00, 588.64batch/s, loss=0.121]\n",
- "Epoch 19: 100%|██████████| 28/28 [00:00<00:00, 574.55batch/s, loss=0.122]\n",
- "Epoch 20: 100%|██████████| 28/28 [00:00<00:00, 566.52batch/s, loss=0.112]\n",
- "Epoch 21: 100%|██████████| 28/28 [00:00<00:00, 538.18batch/s, loss=0.113]\n",
- "Epoch 22: 100%|██████████| 28/28 [00:00<00:00, 564.85batch/s, loss=0.13]\n",
- "Epoch 23: 100%|██████████| 28/28 [00:00<00:00, 560.32batch/s, loss=0.102]\n",
- "Epoch 24: 100%|██████████| 28/28 [00:00<00:00, 588.20batch/s, loss=0.101]\n",
- "Epoch 25: 100%|██████████| 28/28 [00:00<00:00, 444.86batch/s, loss=0.104]\n",
- "Epoch 26: 100%|██████████| 28/28 [00:00<00:00, 617.74batch/s, loss=0.0965]\n",
- "Epoch 27: 100%|██████████| 28/28 [00:00<00:00, 584.56batch/s, loss=0.0993]\n",
- "Epoch 28: 100%|██████████| 28/28 [00:00<00:00, 560.72batch/s, loss=0.101]\n",
- "Epoch 29: 100%|██████████| 28/28 [00:00<00:00, 598.80batch/s, loss=0.0893]\n",
- "Epoch 30: 100%|██████████| 28/28 [00:00<00:00, 598.30batch/s, loss=0.0995]\n",
- "Epoch 31: 100%|██████████| 28/28 [00:00<00:00, 548.75batch/s, loss=0.0889]\n",
- "Epoch 32: 100%|██████████| 28/28 [00:00<00:00, 676.57batch/s, loss=0.101]\n",
- "Epoch 33: 100%|██████████| 28/28 [00:00<00:00, 637.71batch/s, loss=0.089]\n",
- "Epoch 34: 100%|██████████| 28/28 [00:00<00:00, 626.70batch/s, loss=0.086]\n",
- "Epoch 35: 100%|██████████| 28/28 [00:00<00:00, 584.80batch/s, loss=0.0824]\n",
- "Epoch 36: 100%|██████████| 28/28 [00:00<00:00, 606.39batch/s, loss=0.0985]\n",
- "Epoch 37: 100%|██████████| 28/28 [00:00<00:00, 578.46batch/s, loss=0.0771]\n",
- "Epoch 38: 100%|██████████| 28/28 [00:00<00:00, 577.56batch/s, loss=0.074]\n",
- "Epoch 39: 100%|██████████| 28/28 [00:00<00:00, 631.95batch/s, loss=0.0846]\n"
- ]
- },
{
"data": {
- "text/plain": "0.07795017957687378"
+ "text/plain": [
+ "0.02686239752540132"
+ ]
},
- "execution_count": 78,
+ "execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
- "train_and_evaluate(train_dataset, current_eval_dataset=small_query_dataset)"
- ],
+ "num_iter = 25\n",
+ "base_loss = 0.0\n",
+ "for _ in range(num_iter):\n",
+ " base_loss += train_and_evaluate(modified_train_dataset=train_dataset, query_dataset=single_query_dataset)\n",
+ "base_loss /= num_iter\n",
+ "base_loss"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "520de0b1-e6a9-45fc-ae52-ae6f935a4da8",
+ "metadata": {},
+ "source": [
+ "We repeat the procedure above to identify the top influential training data points for this data point."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "id": "8b3eb92fd287b0cc",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T21:13:29.415683Z",
- "start_time": "2024-03-12T21:13:27.270171Z"
+ "end_time": "2024-03-12T21:06:42.141007Z",
+ "start_time": "2024-03-12T21:06:42.067707Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "16be6ab429b5fcd3"
- },
- {
- "cell_type": "code",
- "execution_count": 79,
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "Epoch 0: 100%|██████████| 28/28 [00:00<00:00, 626.29batch/s, loss=0.916]\n",
- "Epoch 1: 100%|██████████| 28/28 [00:00<00:00, 605.61batch/s, loss=0.577]\n",
- "Epoch 2: 100%|██████████| 28/28 [00:00<00:00, 658.10batch/s, loss=0.399]\n",
- "Epoch 3: 100%|██████████| 28/28 [00:00<00:00, 642.07batch/s, loss=0.346]\n",
- "Epoch 4: 100%|██████████| 28/28 [00:00<00:00, 646.28batch/s, loss=0.299]\n",
- "Epoch 5: 100%|██████████| 28/28 [00:00<00:00, 640.60batch/s, loss=0.257]\n",
- "Epoch 6: 100%|██████████| 28/28 [00:00<00:00, 609.95batch/s, loss=0.23]\n",
- "Epoch 7: 100%|██████████| 28/28 [00:00<00:00, 644.70batch/s, loss=0.205]\n",
- "Epoch 8: 100%|██████████| 28/28 [00:00<00:00, 672.63batch/s, loss=0.191]\n",
- "Epoch 9: 100%|██████████| 28/28 [00:00<00:00, 643.22batch/s, loss=0.188]\n",
- "Epoch 10: 100%|██████████| 28/28 [00:00<00:00, 577.76batch/s, loss=0.191]\n",
- "Epoch 11: 100%|██████████| 28/28 [00:00<00:00, 639.78batch/s, loss=0.158]\n",
- "Epoch 12: 100%|██████████| 28/28 [00:00<00:00, 659.67batch/s, loss=0.157]\n",
- "Epoch 13: 100%|██████████| 28/28 [00:00<00:00, 651.89batch/s, loss=0.148]\n",
- "Epoch 14: 100%|██████████| 28/28 [00:00<00:00, 631.47batch/s, loss=0.15]\n",
- "Epoch 15: 100%|██████████| 28/28 [00:00<00:00, 642.57batch/s, loss=0.142]\n",
- "Epoch 16: 100%|██████████| 28/28 [00:00<00:00, 659.21batch/s, loss=0.134]\n",
- "Epoch 17: 100%|██████████| 28/28 [00:00<00:00, 701.93batch/s, loss=0.127]\n",
- "Epoch 18: 100%|██████████| 28/28 [00:00<00:00, 645.56batch/s, loss=0.127]\n",
- "Epoch 19: 100%|██████████| 28/28 [00:00<00:00, 637.15batch/s, loss=0.121]\n",
- "Epoch 20: 100%|██████████| 28/28 [00:00<00:00, 674.00batch/s, loss=0.119]\n",
- "Epoch 21: 100%|██████████| 28/28 [00:00<00:00, 588.97batch/s, loss=0.109]\n",
- "Epoch 22: 100%|██████████| 28/28 [00:00<00:00, 583.82batch/s, loss=0.113]\n",
- "Epoch 23: 100%|██████████| 28/28 [00:00<00:00, 597.69batch/s, loss=0.11]\n",
- "Epoch 24: 100%|██████████| 28/28 [00:00<00:00, 624.01batch/s, loss=0.0941]\n",
- "Epoch 25: 100%|██████████| 28/28 [00:00<00:00, 585.46batch/s, loss=0.103]\n",
- "Epoch 26: 100%|██████████| 28/28 [00:00<00:00, 600.24batch/s, loss=0.105]\n",
- "Epoch 27: 100%|██████████| 28/28 [00:00<00:00, 608.82batch/s, loss=0.105]\n",
- "Epoch 28: 100%|██████████| 28/28 [00:00<00:00, 663.98batch/s, loss=0.0937]\n",
- "Epoch 29: 100%|██████████| 28/28 [00:00<00:00, 699.09batch/s, loss=0.0936]\n",
- "Epoch 30: 100%|██████████| 28/28 [00:00<00:00, 685.94batch/s, loss=0.114]\n",
- "Epoch 31: 100%|██████████| 28/28 [00:00<00:00, 689.95batch/s, loss=0.0866]\n",
- "Epoch 32: 100%|██████████| 28/28 [00:00<00:00, 666.33batch/s, loss=0.0854]\n",
- "Epoch 33: 100%|██████████| 28/28 [00:00<00:00, 712.76batch/s, loss=0.0911]\n",
- "Epoch 34: 100%|██████████| 28/28 [00:00<00:00, 710.28batch/s, loss=0.107]\n",
- "Epoch 35: 100%|██████████| 28/28 [00:00<00:00, 666.09batch/s, loss=0.0805]\n",
- "Epoch 36: 100%|██████████| 28/28 [00:00<00:00, 611.20batch/s, loss=0.0826]\n",
- "Epoch 37: 100%|██████████| 28/28 [00:00<00:00, 604.48batch/s, loss=0.08]\n",
- "Epoch 38: 100%|██████████| 28/28 [00:00<00:00, 580.85batch/s, loss=0.0743]\n",
- "Epoch 39: 100%|██████████| 28/28 [00:00<00:00, 604.00batch/s, loss=0.084]\n",
- "Epoch 0: 100%|██████████| 28/28 [00:00<00:00, 583.44batch/s, loss=0.869]\n",
- "Epoch 1: 100%|██████████| 28/28 [00:00<00:00, 577.74batch/s, loss=0.567]\n",
- "Epoch 2: 100%|██████████| 28/28 [00:00<00:00, 661.26batch/s, loss=0.403]\n",
- "Epoch 3: 100%|██████████| 28/28 [00:00<00:00, 668.64batch/s, loss=0.342]\n",
- "Epoch 4: 100%|██████████| 28/28 [00:00<00:00, 679.75batch/s, loss=0.296]\n",
- "Epoch 5: 100%|██████████| 28/28 [00:00<00:00, 640.19batch/s, loss=0.256]\n",
- "Epoch 6: 100%|██████████| 28/28 [00:00<00:00, 680.29batch/s, loss=0.224]\n",
- "Epoch 7: 100%|██████████| 28/28 [00:00<00:00, 655.48batch/s, loss=0.222]\n",
- "Epoch 8: 100%|██████████| 28/28 [00:00<00:00, 694.15batch/s, loss=0.185]\n",
- "Epoch 9: 100%|██████████| 28/28 [00:00<00:00, 665.41batch/s, loss=0.18]\n",
- "Epoch 10: 0%| | 0/28 [00:00"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
+ }
+ ],
+ "source": [
+ "plt.plot(sorted(scores[\"all_modules\"].sum(dim=0) / len(train_dataset)))\n",
+ "plt.ylabel(\"Scores\")\n",
+ "plt.grid()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "16c3c019-a9b3-4176-8a30-4f1ce4b3e7e7",
+ "metadata": {},
+ "source": [
+ "What happens if we train the model without positively influential data points? Intuitively, the query loss should increase if the model was trained without these top influential data points."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 29,
+ "id": "b07f18b5-7009-4458-aec2-4a49fd595348",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def get_topk_indices(current_score: torch.Tensor, topk: int = 1) -> torch.Tensor:\n",
+ " return torch.topk(current_score, topk).indices\n",
+ "\n",
+ "def get_topk_keep_indices(current_score: torch.Tensor, topk: int = 1) -> List[int]:\n",
+ " remove_indices = get_topk_indices(current_score, topk)\n",
+ " remove_indices = [tensor.item() for tensor in remove_indices]\n",
+ " return list(set(list(range(len(train_dataset)))) - set(remove_indices))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c0960cc9-9606-41ed-b7aa-9971789611d3",
+ "metadata": {},
+ "source": [
+ "We define `get_topk_keep_indices`, which returns dataset indices with `topk` positively influential data points removed."
+ ]
},
{
"cell_type": "code",
- "execution_count": 80,
+ "execution_count": 30,
+ "id": "1a22d21e-3d87-4a88-9665-517487f71ef6",
+ "metadata": {},
"outputs": [
{
"data": {
- "text/plain": "0.1237425720691681"
+ "text/plain": [
+ "(917, 907)"
+ ]
},
- "execution_count": 80,
+ "execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
- "base_loss"
- ],
+ "len(get_topk_keep_indices(scores[\"all_modules\"].sum(dim=0), topk=10)), len(get_topk_keep_indices(scores[\"all_modules\"].sum(dim=0), topk=20))"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 31,
+ "id": "b90e2a6a-e6bc-4865-a0f8-4e8d6d8012c6",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "topk_lst = [5, 10, 15, 20, 25, 30] \n",
+ "if_removed_loss_lst = []\n",
+ "\n",
+ "for topk in topk_lst:\n",
+ " keep_indices = get_topk_keep_indices(scores[\"all_modules\"].sum(dim=0), topk=topk)\n",
+ " \n",
+ " new_loss = 0.\n",
+ " for _ in range(num_iter):\n",
+ " new_loss += train_and_evaluate(modified_train_dataset=torch.utils.data.Subset(train_dataset, keep_indices), query_dataset=single_query_dataset)\n",
+ " new_loss /= num_iter\n",
+ " if_removed_loss_lst.append(new_loss)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "6194cf30-e728-4484-b4a0-60736016feb6",
+ "metadata": {},
+ "source": [
+ "We compare the results with the random baseline, where the same number of data points are removed."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 32,
+ "id": "868b1ff7796cfafa",
"metadata": {
- "collapsed": false,
"ExecuteTime": {
- "end_time": "2024-03-12T21:14:39.883335Z",
- "start_time": "2024-03-12T21:14:39.876048Z"
+ "end_time": "2024-03-12T21:16:31.778208Z",
+ "start_time": "2024-03-12T21:16:22.922187Z"
+ },
+ "collapsed": false,
+ "jupyter": {
+ "outputs_hidden": false
}
},
- "id": "4a2ab1743a8cf346"
+ "outputs": [],
+ "source": [
+ "random_indices = list(range(len(train_dataset)))\n",
+ "shuffle(random_indices)\n",
+ "random_removed_loss_lst = []\n",
+ "\n",
+ "for topk in topk_lst:\n",
+ " keep_indices = random_indices[topk:]\n",
+ " \n",
+ " new_loss = 0\n",
+ " for _ in range(num_iter):\n",
+ " new_loss += train_and_evaluate(modified_train_dataset=torch.utils.data.Subset(train_dataset, keep_indices), query_dataset=single_query_dataset)\n",
+ " new_loss /= num_iter\n",
+ " random_removed_loss_lst.append(new_loss)"
+ ]
},
{
"cell_type": "code",
- "execution_count": 84,
+ "execution_count": 33,
+ "id": "9e6fd656-c1e5-42cb-87f4-d7690f473fe9",
+ "metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "Epoch 0: 100%|██████████| 27/27 [00:00<00:00, 570.47batch/s, loss=0.926]\n",
- "Epoch 1: 100%|██████████| 27/27 [00:00<00:00, 567.78batch/s, loss=0.569]\n",
- "Epoch 2: 100%|██████████| 27/27 [00:00<00:00, 570.60batch/s, loss=0.386]\n",
- "Epoch 3: 100%|██████████| 27/27 [00:00<00:00, 584.24batch/s, loss=0.335]\n",
- "Epoch 4: 100%|██████████| 27/27 [00:00<00:00, 543.31batch/s, loss=0.276]\n",
- "Epoch 5: 100%|██████████| 27/27 [00:00<00:00, 611.63batch/s, loss=0.239]\n",
- "Epoch 6: 100%|██████████| 27/27 [00:00<00:00, 579.03batch/s, loss=0.213]\n",
- "Epoch 7: 100%|██████████| 27/27 [00:00<00:00, 587.25batch/s, loss=0.193]\n",
- "Epoch 8: 100%|██████████| 27/27 [00:00<00:00, 632.51batch/s, loss=0.175]\n",
- "Epoch 9: 100%|██████████| 27/27 [00:00<00:00, 581.18batch/s, loss=0.177]\n",
- "Epoch 10: 100%|██████████| 27/27 [00:00<00:00, 577.28batch/s, loss=0.158]\n",
- "Epoch 11: 100%|██████████| 27/27 [00:00<00:00, 419.68batch/s, loss=0.148]\n",
- "Epoch 12: 100%|██████████| 27/27 [00:00<00:00, 531.87batch/s, loss=0.157]\n",
- "Epoch 13: 100%|██████████| 27/27 [00:00<00:00, 535.44batch/s, loss=0.136]\n",
- "Epoch 14: 100%|██████████| 27/27 [00:00<00:00, 543.30batch/s, loss=0.134]\n",
- "Epoch 15: 100%|██████████| 27/27 [00:00<00:00, 586.75batch/s, loss=0.125]\n",
- "Epoch 16: 100%|██████████| 27/27 [00:00<00:00, 590.17batch/s, loss=0.13]\n",
- "Epoch 17: 100%|██████████| 27/27 [00:00<00:00, 407.74batch/s, loss=0.115]\n",
- "Epoch 18: 100%|██████████| 27/27 [00:00<00:00, 629.78batch/s, loss=0.124]\n",
- "Epoch 19: 100%|██████████| 27/27 [00:00<00:00, 629.34batch/s, loss=0.0994]\n",
- "Epoch 20: 100%|██████████| 27/27 [00:00<00:00, 682.78batch/s, loss=0.113]\n",
- "Epoch 21: 100%|██████████| 27/27 [00:00<00:00, 695.72batch/s, loss=0.103]\n",
- "Epoch 22: 100%|██████████| 27/27 [00:00<00:00, 654.56batch/s, loss=0.0986]\n",
- "Epoch 23: 100%|██████████| 27/27 [00:00<00:00, 657.21batch/s, loss=0.102]\n",
- "Epoch 24: 100%|██████████| 27/27 [00:00<00:00, 673.11batch/s, loss=0.0881]\n",
- "Epoch 25: 100%|██████████| 27/27 [00:00<00:00, 681.68batch/s, loss=0.0965]\n",
- "Epoch 26: 100%|██████████| 27/27 [00:00<00:00, 625.14batch/s, loss=0.0915]\n",
- "Epoch 27: 100%|██████████| 27/27 [00:00<00:00, 718.77batch/s, loss=0.0846]\n",
- "Epoch 28: 100%|██████████| 27/27 [00:00<00:00, 668.65batch/s, loss=0.0907]\n",
- "Epoch 29: 100%|██████████| 27/27 [00:00<00:00, 642.95batch/s, loss=0.0831]\n",
- "Epoch 30: 100%|██████████| 27/27 [00:00<00:00, 660.58batch/s, loss=0.083]\n",
- "Epoch 31: 100%|██████████| 27/27 [00:00<00:00, 720.57batch/s, loss=0.078]\n",
- "Epoch 32: 100%|██████████| 27/27 [00:00<00:00, 694.94batch/s, loss=0.0732]\n",
- "Epoch 33: 100%|██████████| 27/27 [00:00<00:00, 675.04batch/s, loss=0.0756]\n",
- "Epoch 34: 100%|██████████| 27/27 [00:00<00:00, 687.41batch/s, loss=0.0781]\n",
- "Epoch 35: 100%|██████████| 27/27 [00:00<00:00, 599.25batch/s, loss=0.076]\n",
- "Epoch 36: 100%|██████████| 27/27 [00:00<00:00, 679.97batch/s, loss=0.0754]\n",
- "Epoch 37: 100%|██████████| 27/27 [00:00<00:00, 693.84batch/s, loss=0.0751]\n",
- "Epoch 38: 100%|██████████| 27/27 [00:00<00:00, 676.10batch/s, loss=0.0732]\n",
- "Epoch 39: 100%|██████████| 27/27 [00:00<00:00, 586.66batch/s, loss=0.0777]\n",
- "Epoch 0: 100%|██████████| 27/27 [00:00<00:00, 680.81batch/s, loss=0.92]\n",
- "Epoch 1: 100%|██████████| 27/27 [00:00<00:00, 645.01batch/s, loss=0.606]\n",
- "Epoch 2: 100%|██████████| 27/27 [00:00<00:00, 659.31batch/s, loss=0.402]\n",
- "Epoch 3: 100%|██████████| 27/27 [00:00<00:00, 693.00batch/s, loss=0.345]\n",
- "Epoch 4: 100%|██████████| 27/27 [00:00<00:00, 680.05batch/s, loss=0.294]\n",
- "Epoch 5: 100%|██████████| 27/27 [00:00<00:00, 640.39batch/s, loss=0.258]\n",
- "Epoch 6: 100%|██████████| 27/27 [00:00<00:00, 653.25batch/s, loss=0.232]\n",
- "Epoch 7: 100%|██████████| 27/27 [00:00<00:00, 676.15batch/s, loss=0.196]\n",
- "Epoch 8: 100%|██████████| 27/27 [00:00<00:00, 662.82batch/s, loss=0.172]\n",
- "Epoch 9: 100%|██████████| 27/27 [00:00<00:00, 663.18batch/s, loss=0.157]\n",
- "Epoch 10: 100%|██████████| 27/27 [00:00<00:00, 665.84batch/s, loss=0.162]\n",
- "Epoch 11: 100%|██████████| 27/27 [00:00<00:00, 659.72batch/s, loss=0.143]\n",
- "Epoch 12: 0%| | 0/27 [00:00"
+ ]
+ },
+ "metadata": {},
+ "output_type": "display_data"
}
],
"source": [
- "new_loss"
- ],
- "metadata": {
- "collapsed": false,
- "ExecuteTime": {
- "end_time": "2024-03-12T21:16:35.056487Z",
- "start_time": "2024-03-12T21:16:35.049529Z"
- }
- },
- "id": "86370ac8eb37aee5"
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "outputs": [],
- "source": [],
- "metadata": {
- "collapsed": false
- },
- "id": "669ce001a286fe6f"
+ "plt.plot([0] + topk_lst, [base_loss] + random_removed_loss_lst, \"o-\", label=\"Random\")\n",
+ "plt.plot([0] + topk_lst, [base_loss] + id_removed_loss_lst, \"o-\", label=\"TracIn (Identity)\")\n",
+ "plt.plot([0] + topk_lst, [base_loss] + if_removed_loss_lst, \"o-\", label=\"IF (EKFAC)\")\n",
+ "plt.grid()\n",
+ "plt.legend()\n",
+ "plt.ylabel(\"Query Loss\")\n",
+ "plt.xlabel(\"Number of Training Samples Removed\")"
+ ]
}
],
"metadata": {
"kernelspec": {
- "display_name": "Python 3",
+ "display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
- "version": 2
+ "version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.6"
+ "pygments_lexer": "ipython3",
+ "version": "3.9.18"
}
},
"nbformat": 4,
diff --git a/examples/wikitext/README.md b/examples/wikitext/README.md
new file mode 100644
index 0000000..c08cd48
--- /dev/null
+++ b/examples/wikitext/README.md
@@ -0,0 +1,48 @@
+# WikiText & GPT-2 Example
+
+This directory contains scripts for fine-tuning GPT-2 on WikiText2 dataset. The pipeline is motivated from
+[HuggingFace Example](https://github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling).
+Please begin by installing necessary packages.
+```bash
+pip install -r requirements.txt
+```
+
+## Training
+
+To fine-tune GPT-2, run the following command:
+```bash
+python train.py --checkpoint_dir ./checkpoints \
+ --train_batch_size 8 \
+ --eval_batch_size 16 \
+ --learning_rate 3e-05 \
+ --weight_decay 0.01 \
+ --num_train_epochs 3
+```
+
+## Computing Pairwise Influence Scores
+
+To obtain a pairwise influence scores on 481 query data points using `ekfac`, run the following command:
+```bash
+python analyze.py --query_batch_size 32 \
+ --train_batch_size 64 \
+ --checkpoint_dir ./checkpoints \
+ --factor_strategy ekfac
+```
+You can also use `identity`, `diagonal`, and `kfac`. On A100 (80GB), it takes roughly 50 minutes to compute the
+pairwise scores (including computing EKFAC factors).
+
+
+## Counterfactual Experiment
+
+We can conduct counterfactual experiment by observing the increase in validation perplexity when removing top influential sequences.
+We show a simple demo in `run_counterfactual.py` (the code assumes that you have computed the pairwise influence scores with `ekfac` and `identity`).
+
+
+
+
+
+## Computing Linear Datamodeling Score
+
+We can also compute the [Linear Datamodeling Score (LDS)](https://arxiv.org/abs/2303.14186). The code in `evaluate_lds.py` measures the LDS obtained by
+retraining the network 600 times with different subsets of the dataset (5 repeats and 120 masks). We can obtain `0.37` LDS.
\ No newline at end of file
diff --git a/examples/wikitext/analyze.py b/examples/wikitext/analyze.py
new file mode 100644
index 0000000..d19b5e9
--- /dev/null
+++ b/examples/wikitext/analyze.py
@@ -0,0 +1,202 @@
+import argparse
+import logging
+import os
+from typing import Dict, List, Optional
+
+import torch
+import torch.nn.functional as F
+from torch import nn
+from transformers import default_data_collator
+
+from examples.wikitext.pipeline import construct_gpt2, get_wikitext_dataset
+from kronfluence.analyzer import Analyzer, prepare_model
+from kronfluence.arguments import FactorArguments, ScoreArguments
+from kronfluence.task import Task
+from kronfluence.utils.dataset import DataLoaderKwargs
+
+BATCH_TYPE = Dict[str, torch.Tensor]
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description="Influence analysis on WikiText dataset.")
+
+ parser.add_argument(
+ "--checkpoint_dir",
+ type=str,
+ default="./checkpoints",
+ help="A path that is storing the final checkpoint of the model.",
+ )
+
+ parser.add_argument(
+ "--query_gradient_rank",
+ type=int,
+ default=-1,
+ help="Rank for the low-rank query gradient approximation.",
+ )
+ parser.add_argument(
+ "--use_half_precision",
+ type=bool,
+ default=False,
+ help="Whether to use half precision for computing factors and scores.",
+ )
+ parser.add_argument(
+ "--query_batch_size",
+ type=int,
+ default=8,
+ help="Batch size for computing query gradients.",
+ )
+ parser.add_argument(
+ "--train_batch_size",
+ type=int,
+ default=8,
+ help="Batch size for computing query gradients.",
+ )
+ parser.add_argument(
+ "--factor_strategy",
+ type=str,
+ default="ekfac",
+ help="Strategy to compute influence factors.",
+ )
+
+ args = parser.parse_args()
+
+ if args.checkpoint_dir is not None:
+ os.makedirs(args.checkpoint_dir, exist_ok=True)
+ return args
+
+
+class LanguageModelingTask(Task):
+ def compute_train_loss(
+ self,
+ batch: BATCH_TYPE,
+ model: nn.Module,
+ sample: bool = False,
+ ) -> torch.Tensor:
+ logits = model(
+ input_ids=batch["input_ids"],
+ attention_mask=batch["attention_mask"],
+ ).logits
+
+ shift_logits = logits[..., :-1, :].contiguous()
+
+ if not sample:
+ labels = batch["labels"]
+ shift_labels = labels[..., 1:].contiguous()
+ reshaped_shift_logits = shift_logits.view(-1, shift_logits.size(-1))
+ summed_loss = F.cross_entropy(reshaped_shift_logits, shift_labels.view(-1), reduction="sum")
+ else:
+ reshaped_shift_logits = shift_logits.view(-1, shift_logits.size(-1))
+ with torch.no_grad():
+ probs = torch.nn.functional.softmax(reshaped_shift_logits, dim=-1)
+ sampled_labels = torch.multinomial(
+ probs,
+ num_samples=1,
+ ).flatten()
+ summed_loss = F.cross_entropy(reshaped_shift_logits, sampled_labels.detach(), reduction="sum")
+ return summed_loss
+
+ def compute_measurement(
+ self,
+ batch: BATCH_TYPE,
+ model: nn.Module,
+ ) -> torch.Tensor:
+ # We could also compute the log-likelihood or averaged margin.
+ return self.compute_train_loss(batch, model)
+
+ def tracked_modules(self) -> List[str]:
+ total_modules = []
+
+ for i in range(12):
+ total_modules.append(f"transformer.h.{i}.attn.c_attn")
+ total_modules.append(f"transformer.h.{i}.attn.c_proj")
+
+ for i in range(12):
+ total_modules.append(f"transformer.h.{i}.mlp.c_fc")
+ total_modules.append(f"transformer.h.{i}.mlp.c_proj")
+
+ return total_modules
+
+ def get_attention_mask(self, batch: BATCH_TYPE) -> Optional[torch.Tensor]:
+ return batch["attention_mask"]
+
+
+def main():
+ args = parse_args()
+ logging.basicConfig(level=logging.INFO)
+
+ # Prepare the dataset.
+ train_dataset = get_wikitext_dataset(
+ split="eval_train",
+ )
+ eval_dataset = get_wikitext_dataset(
+ split="valid",
+ )
+
+ # Prepare the trained model.
+ model = construct_gpt2()
+ checkpoint_path = os.path.join(args.checkpoint_dir, "model.pth")
+ if not os.path.isfile(checkpoint_path):
+ raise ValueError(f"No checkpoint found at {checkpoint_path}.")
+ model.load_state_dict(torch.load(checkpoint_path))
+
+ # Define task and prepare model.
+ task = LanguageModelingTask()
+ model = prepare_model(model, task)
+
+ analyzer = Analyzer(
+ analysis_name="wikitext",
+ model=model,
+ task=task,
+ )
+ # Configure parameters for DataLoader.
+ dataloader_kwargs = DataLoaderKwargs(collate_fn=default_data_collator)
+ analyzer.set_dataloader_kwargs(dataloader_kwargs)
+
+ # Compute influence factors.
+ factors_name = args.factor_strategy
+ factor_args = FactorArguments(strategy=args.factor_strategy)
+ if args.use_half_precision:
+ factor_args.activation_covariance_dtype = torch.bfloat16
+ factor_args.gradient_covariance_dtype = torch.bfloat16
+ factor_args.lambda_dtype = torch.bfloat16
+ factors_name += "_half"
+
+ analyzer.fit_all_factors(
+ factors_name=factors_name,
+ dataset=train_dataset,
+ per_device_batch_size=None,
+ factor_args=factor_args,
+ overwrite_output_dir=False,
+ initial_per_device_batch_size_attempt=128,
+ )
+
+ # Compute pairwise scores.
+ rank = args.query_gradient_rank if args.query_gradient_rank != -1 else None
+ score_args = ScoreArguments(query_gradient_rank=rank, query_gradient_svd_dtype=torch.float32)
+ scores_name = f"{factor_args.strategy}_pairwise"
+ if rank is not None:
+ scores_name += f"_qlr{rank}"
+
+ if args.use_half_precision:
+ score_args.per_sample_gradient_dtype = torch.bfloat16
+ score_args.score_dtype = torch.bfloat16
+ score_args.cached_activation_cpu_offload = True
+ scores_name += "_half"
+
+ analyzer.compute_pairwise_scores(
+ scores_name=scores_name,
+ score_args=score_args,
+ factors_name=args.factor_strategy,
+ query_dataset=eval_dataset,
+ query_indices=list(range(min([len(eval_dataset), 2000]))),
+ train_dataset=train_dataset,
+ per_device_query_batch_size=args.query_batch_size,
+ per_device_train_batch_size=args.train_batch_size,
+ overwrite_output_dir=True,
+ )
+ scores = analyzer.load_pairwise_scores(scores_name)["all_modules"]
+ logging.info(f"Scores shape: {scores.shape}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/wikitext/evaluate_lds.py b/examples/wikitext/evaluate_lds.py
new file mode 100644
index 0000000..407e510
--- /dev/null
+++ b/examples/wikitext/evaluate_lds.py
@@ -0,0 +1,29 @@
+import logging
+
+import numpy as np
+import torch
+from scipy.stats import spearmanr
+
+from kronfluence.analyzer import Analyzer
+
+
+def main():
+ logging.basicConfig(level=logging.INFO)
+
+ results = torch.load("files/lds_results.pt")
+ diff_loss = torch.from_numpy(results["diff_loss"])
+ mask = torch.from_numpy(results["mask"]).float()
+ mask = ((mask + 1) % 2).to(dtype=torch.float64).t()
+
+ # You might need to change the path.
+ scores = Analyzer.load_file("scores_pairwise/ekfac_pairwise.safetensors")["all_modules"].to(dtype=torch.float64)
+ preds = (scores @ mask).t().numpy()
+
+ corr_lst = []
+ for i in range(diff_loss.shape[1]):
+ corr_lst.append(spearmanr(diff_loss[:, i], preds[:, i])[0])
+ logging.info(f"LDS: {np.mean(corr_lst)}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/wikitext/figure/counterfactual.png b/examples/wikitext/figure/counterfactual.png
new file mode 100644
index 0000000..060e0fd
Binary files /dev/null and b/examples/wikitext/figure/counterfactual.png differ
diff --git a/examples/wikitext/pipeline.py b/examples/wikitext/pipeline.py
new file mode 100644
index 0000000..b0f5986
--- /dev/null
+++ b/examples/wikitext/pipeline.py
@@ -0,0 +1,102 @@
+from itertools import chain
+from typing import List
+
+from datasets import load_dataset
+from torch import nn
+from torch.utils import data
+from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
+from transformers.pytorch_utils import Conv1D
+
+
+def replace_conv1d_modules(model: nn.Module) -> None:
+ # GPT-2 is defined in terms of Conv1D. However, this does not work for Kronfluence.
+ # Here, we convert these Conv1D modules to linear modules recursively.
+ for name, module in model.named_children():
+ if len(list(module.children())) > 0:
+ replace_conv1d_modules(module)
+
+ if isinstance(module, Conv1D):
+ new_module = nn.Linear(in_features=module.weight.shape[0], out_features=module.weight.shape[1])
+ new_module.weight.data.copy_(module.weight.data.t())
+ new_module.bias.data.copy_(module.bias.data)
+ setattr(model, name, new_module)
+
+
+def construct_gpt2() -> nn.Module:
+ config = AutoConfig.from_pretrained(
+ "gpt2",
+ trust_remote_code=True,
+ )
+ model = AutoModelForCausalLM.from_pretrained(
+ "gpt2",
+ from_tf=False,
+ config=config,
+ ignore_mismatched_sizes=False,
+ trust_remote_code=True,
+ )
+ replace_conv1d_modules(model)
+ return model
+
+
+def get_wikitext_dataset(
+ split: str,
+ indices: List[int] = None,
+) -> data.Dataset:
+ assert split in ["train", "eval_train", "valid"]
+
+ raw_datasets = load_dataset("wikitext", "wikitext-2-raw-v1")
+ tokenizer = AutoTokenizer.from_pretrained("gpt2", use_fast=True, trust_remote_code=True)
+
+ column_names = raw_datasets["train"].column_names
+ text_column_name = "text" if "text" in column_names else column_names[0]
+
+ def tokenize_function(examples):
+ return tokenizer(examples[text_column_name])
+
+ tokenized_datasets = raw_datasets.map(
+ tokenize_function,
+ batched=True,
+ num_proc=None,
+ remove_columns=column_names,
+ load_from_cache_file=True,
+ desc="Running tokenizer on dataset",
+ )
+ block_size = 512
+
+ def group_texts(examples):
+ concatenated_examples = {k: list(chain(*examples[k])) for k in examples.keys()}
+ total_length = len(concatenated_examples[list(examples.keys())[0]])
+ total_length = (total_length // block_size) * block_size
+ result = {
+ k: [t[i : i + block_size] for i in range(0, total_length, block_size)]
+ for k, t in concatenated_examples.items()
+ }
+ result["labels"] = result["input_ids"].copy()
+ return result
+
+ lm_datasets = tokenized_datasets.map(
+ group_texts,
+ batched=True,
+ num_proc=None,
+ load_from_cache_file=True,
+ desc=f"Grouping texts in chunks of {block_size}",
+ )
+
+ if split in ["train", "eval_train"]:
+ train_dataset = lm_datasets["train"]
+ ds = train_dataset
+ else:
+ eval_dataset = lm_datasets["validation"]
+ ds = eval_dataset
+
+ if indices is not None:
+ ds = ds.select(indices)
+
+ return ds
+
+
+if __name__ == "__main__":
+ from kronfluence import Analyzer
+
+ model = construct_gpt2()
+ print(Analyzer.get_module_summary(model))
diff --git a/examples/wikitext/requirements.txt b/examples/wikitext/requirements.txt
new file mode 100644
index 0000000..cd62543
--- /dev/null
+++ b/examples/wikitext/requirements.txt
@@ -0,0 +1,4 @@
+transformers
+datasets
+matplotlib
+tueplots
\ No newline at end of file
diff --git a/examples/wikitext/run_counterfactual.py b/examples/wikitext/run_counterfactual.py
new file mode 100644
index 0000000..d682b7d
--- /dev/null
+++ b/examples/wikitext/run_counterfactual.py
@@ -0,0 +1,109 @@
+import logging
+import math
+from random import shuffle
+from typing import List
+
+import matplotlib.pyplot as plt
+import torch
+from tueplots import markers
+
+from examples.wikitext.pipeline import get_wikitext_dataset
+from examples.wikitext.train import evaluate_model, train
+from kronfluence.analyzer import Analyzer
+
+
+def main():
+ logging.basicConfig(level=logging.INFO)
+
+ train_dataset = get_wikitext_dataset(split="train")
+ # You might need to change the path.
+ identity_scores = Analyzer.load_file("analyses/wikitext/scores_identity_pairwise/pairwise_scores.safetensors")[
+ "all_modules"
+ ][:50].sum(dim=0)
+ ekfac_scores = Analyzer.load_file("analyses/wikitext/scores_ekfac_pairwise/pairwise_scores.safetensors")[
+ "all_modules"
+ ][:50].sum(dim=0)
+
+ def get_topk_indices(current_score: torch.Tensor, topk: int = 1) -> torch.Tensor:
+ return torch.topk(current_score, topk).indices
+
+ def get_topk_keep_indices(current_score: torch.Tensor, topk: int = 1) -> List[int]:
+ remove_indices = get_topk_indices(current_score, topk)
+ remove_indices = [tensor.item() for tensor in remove_indices]
+ return list(set(list(range(len(train_dataset)))) - set(remove_indices))
+
+ eval_train_dataset = get_wikitext_dataset(split="valid", indices=list(range(50)))
+
+ def train_and_evaluate(indices):
+ train_dataset = get_wikitext_dataset(split="train", indices=indices)
+ model = train(
+ dataset=train_dataset,
+ batch_size=8,
+ num_train_epochs=3,
+ learning_rate=3e-05,
+ weight_decay=0.01,
+ )
+ return evaluate_model(model, eval_train_dataset, batch_size=16)
+
+ num_iter = 1
+ topk_lst = [0, 50, 100, 150, 200]
+
+ ekfac_remove_perp_lst = []
+ for topk in topk_lst:
+ keep_indices = get_topk_keep_indices(ekfac_scores, topk=topk)
+
+ perp = 0.0
+ for _ in range(num_iter):
+ new_loss = train_and_evaluate(indices=keep_indices)
+ perp += math.exp(new_loss)
+ perp /= num_iter
+ ekfac_remove_perp_lst.append(perp)
+
+ logging.info(f"Removed {topk} data points. Perplexity: {perp}")
+ logging.info(f"EKFAC: {ekfac_remove_perp_lst}")
+
+ id_remove_perp_lst = []
+ for topk in topk_lst:
+ keep_indices = get_topk_keep_indices(identity_scores, topk=topk)
+
+ perp = 0.0
+ for _ in range(num_iter):
+ new_loss = train_and_evaluate(indices=keep_indices)
+ perp += math.exp(new_loss)
+ perp /= num_iter
+ id_remove_perp_lst.append(perp)
+
+ logging.info(f"Removed {topk} data points. Perplexity: {perp}")
+ logging.info(f"TracIn: {id_remove_perp_lst}")
+
+ random_indices = list(range(4656))
+ shuffle(random_indices)
+ random_remove_perp_lst = []
+ for topk in topk_lst:
+ keep_indices = random_indices[topk:]
+
+ perp = 0.0
+ for _ in range(num_iter):
+ new_loss = train_and_evaluate(indices=keep_indices)
+ perp += math.exp(new_loss)
+ perp /= num_iter
+ random_remove_perp_lst.append(perp)
+
+ logging.info(f"Removed {topk} data points. Perplexity: {perp}")
+ logging.info(f"Random: {random_remove_perp_lst}")
+
+ plt.rcParams.update({"figure.dpi": 150})
+ plt.rcParams.update(markers.with_edge())
+ plt.rcParams["axes.axisbelow"] = True
+ plt.plot(topk_lst, [ekfac_remove_perp_lst[0]] + random_remove_perp_lst[1:], "o-", label="Random")
+ plt.plot(topk_lst, [ekfac_remove_perp_lst[0]] + id_remove_perp_lst[1:], "o-", label="TracIn (Identity)")
+ plt.plot(topk_lst, ekfac_remove_perp_lst, "o-", label="IF (EKFAC)")
+ plt.grid()
+ plt.legend()
+ plt.xlabel("Number of Training Samples Removed")
+ plt.ylabel("Mean Query Perplexity")
+ plt.show()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/wikitext/train.py b/examples/wikitext/train.py
new file mode 100644
index 0000000..ebb880c
--- /dev/null
+++ b/examples/wikitext/train.py
@@ -0,0 +1,175 @@
+import argparse
+import logging
+import math
+import os
+import time
+
+import torch
+import torch.nn.functional as F
+from accelerate.utils import set_seed
+from torch import nn
+from torch.nn import CrossEntropyLoss
+from torch.utils import data
+from transformers import default_data_collator
+
+from examples.wikitext.pipeline import construct_gpt2, get_wikitext_dataset
+
+DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description="Fine-tune GPT-2 on WikiText dataset.")
+
+ parser.add_argument(
+ "--train_batch_size",
+ type=int,
+ default=8,
+ help="Batch size for the training dataloader.",
+ )
+ parser.add_argument(
+ "--eval_batch_size",
+ type=int,
+ default=16,
+ help="Batch size for the evaluation dataloader.",
+ )
+
+ parser.add_argument(
+ "--learning_rate",
+ type=float,
+ default=3e-05,
+ help="Fixed learning rate to train the model.",
+ )
+ parser.add_argument(
+ "--weight_decay",
+ type=float,
+ default=0.01,
+ help="Weight decay to train the model.",
+ )
+ parser.add_argument(
+ "--num_train_epochs",
+ type=int,
+ default=3,
+ help="Total number of epochs to train the model.",
+ )
+
+ parser.add_argument(
+ "--seed",
+ type=int,
+ default=0,
+ help="A seed for reproducible training pipeline.",
+ )
+ parser.add_argument(
+ "--checkpoint_dir",
+ type=str,
+ default="./checkpoints",
+ help="A path to store the final checkpoint.",
+ )
+
+ args = parser.parse_args()
+
+ if args.checkpoint_dir is not None:
+ os.makedirs(args.checkpoint_dir, exist_ok=True)
+
+ return args
+
+
+def train(
+ dataset: data.Dataset,
+ batch_size: int,
+ num_train_epochs: int,
+ learning_rate: float,
+ weight_decay: float,
+) -> nn.Module:
+ train_dataloader = data.DataLoader(
+ dataset=dataset,
+ batch_size=batch_size,
+ shuffle=True,
+ drop_last=True,
+ collate_fn=default_data_collator,
+ )
+
+ model = construct_gpt2().to(DEVICE)
+ optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate, weight_decay=weight_decay)
+ loss_fn = CrossEntropyLoss(reduction="mean")
+
+ start_time = time.time()
+ model.eval()
+ for epoch in range(num_train_epochs):
+ total_loss = 0.0
+ for batch in train_dataloader:
+ model.zero_grad()
+ lm_logits = model(
+ input_ids=batch["input_ids"].to(device=DEVICE),
+ attention_mask=batch["attention_mask"].to(device=DEVICE),
+ ).logits
+ labels = batch["labels"].to(device=DEVICE)
+ shift_logits = lm_logits[..., :-1, :].contiguous()
+ shift_labels = labels[..., 1:].contiguous()
+ loss = loss_fn(shift_logits.view(-1, shift_logits.size(-1)), shift_labels.view(-1))
+ loss.backward()
+ optimizer.step()
+ total_loss += loss.detach().float()
+ logging.info(f"Epoch {epoch + 1} - Averaged Loss: {total_loss / len(dataset)}")
+ end_time = time.time()
+ elapsed_time = end_time - start_time
+ logging.info(f"Completed training in {elapsed_time:.2f} seconds.")
+ return model
+
+
+def evaluate_model(model: nn.Module, dataset: data.Dataset, batch_size: int) -> float:
+ dataloader = data.DataLoader(
+ dataset=dataset, batch_size=batch_size, shuffle=False, drop_last=False, collate_fn=default_data_collator
+ )
+
+ model.eval()
+ total_loss = 0.0
+ total_num = 0
+ for batch in dataloader:
+ with torch.no_grad():
+ lm_logits = model(
+ input_ids=batch["input_ids"].to(device=DEVICE),
+ attention_mask=batch["attention_mask"].to(device=DEVICE),
+ ).logits
+ labels = batch["labels"].to(device=DEVICE)
+ shift_logits = lm_logits[..., :-1, :].contiguous()
+ shift_labels = labels[..., 1:].contiguous()
+ reshaped_shift_logits = shift_logits.view(-1, shift_logits.size(-1))
+ loss = F.cross_entropy(reshaped_shift_logits, shift_labels.view(-1), reduction="sum").detach().float()
+ total_loss += loss
+ total_num += reshaped_shift_logits.shape[0]
+ return total_loss.item() / total_num
+
+
+def main():
+ args = parse_args()
+ logging.basicConfig(level=logging.INFO)
+ logger = logging.getLogger()
+
+ if args.seed is not None:
+ set_seed(args.seed)
+
+ train_dataset = get_wikitext_dataset(split="train")
+ model = train(
+ dataset=train_dataset,
+ batch_size=args.train_batch_size,
+ num_train_epochs=args.num_train_epochs,
+ learning_rate=args.learning_rate,
+ weight_decay=args.weight_decay,
+ )
+
+ eval_train_dataset = get_wikitext_dataset(split="eval_train")
+ train_loss = evaluate_model(model=model, dataset=eval_train_dataset, batch_size=args.eval_batch_size)
+ train_perplexity = math.exp(train_loss)
+ logger.info(f"Train perplexity: {train_perplexity}")
+
+ eval_dataset = get_wikitext_dataset(split="valid")
+ eval_loss = evaluate_model(model=model, dataset=eval_dataset, batch_size=args.eval_batch_size)
+ eval_perplexity = math.exp(eval_loss)
+ logger.info(f"Evaluation perplexity: {eval_perplexity}")
+
+ if args.checkpoint_dir is not None:
+ torch.save(model.state_dict(), os.path.join(args.checkpoint_dir, "model.pth"))
+
+
+if __name__ == "__main__":
+ main()
diff --git a/kronfluence/__init__.py b/kronfluence/__init__.py
index fc62365..8b98251 100644
--- a/kronfluence/__init__.py
+++ b/kronfluence/__init__.py
@@ -1,11 +1,12 @@
from . import utils
-from .analyzer import Analyzer
+from .analyzer import Analyzer, prepare_model
from .arguments import FactorArguments, ScoreArguments
from .task import Task
from .version import __version__
__all__ = [
"Analyzer",
+ "prepare_model",
"FactorArguments",
"ScoreArguments",
"Task",
diff --git a/kronfluence/analyzer.py b/kronfluence/analyzer.py
index 7b8713c..710eb26 100644
--- a/kronfluence/analyzer.py
+++ b/kronfluence/analyzer.py
@@ -1,5 +1,5 @@
from pathlib import Path
-from typing import Dict, Optional
+from typing import Dict, Optional, Union
import torch
from accelerate.utils import extract_model_from_parallel
@@ -196,7 +196,7 @@ def fit_all_factors(
)
@staticmethod
- def load_file(path: Path) -> Dict[str, torch.Tensor]:
+ def load_file(path: Union[str, Path]) -> Dict[str, torch.Tensor]:
"""Loads the `.safetensors` file at the given path from disk.
See https://github.com/huggingface/safetensors.
@@ -209,6 +209,8 @@ def load_file(path: Path) -> Dict[str, torch.Tensor]:
Dict[str, torch.Tensor]:
The contents of the file, which is the dictionary mapping string to tensors.
"""
+ if isinstance(path, str):
+ path = Path(path).resolve()
if not path.exists():
raise FileNotFoundError(f"File does not exists at `{path}`.")
return load_file(path)
diff --git a/kronfluence/computer/computer.py b/kronfluence/computer/computer.py
index 49d6f7a..212d697 100644
--- a/kronfluence/computer/computer.py
+++ b/kronfluence/computer/computer.py
@@ -36,7 +36,6 @@
from kronfluence.utils.exceptions import (
FactorsNotFoundError,
TrackedModuleNotFoundError,
- UnsupportableModuleError,
)
from kronfluence.utils.logger import PassThroughProfiler, Profiler, get_logger
from kronfluence.utils.save import (
@@ -74,16 +73,15 @@ def __init__(
self.model = model
self.task = task
- try:
- tracked_module_names = get_tracked_module_names(self.model)
- except TrackedModuleNotFoundError as e:
+ tracked_module_names = get_tracked_module_names(self.model)
+ if len(tracked_module_names) == 0:
error_msg = (
f"No tracked modules found in the provided model: {self.model}. "
f"Please make sure to run `prepare_model` before passing it in to the "
f"Analyzer."
)
self.logger.error(error_msg)
- raise UnsupportableModuleError(error_msg) from e
+ raise TrackedModuleNotFoundError(error_msg)
self.logger.info(f"Tracking modules with names: {tracked_module_names}.")
if self.state.use_distributed and not isinstance(model, (DDP, FSDP)):
diff --git a/kronfluence/computer/score_computer.py b/kronfluence/computer/score_computer.py
index f435e99..628a23c 100644
--- a/kronfluence/computer/score_computer.py
+++ b/kronfluence/computer/score_computer.py
@@ -301,8 +301,11 @@ def compute_pairwise_scores(
)
if query_indices is not None:
query_dataset = data.Subset(dataset=query_dataset, indices=query_indices)
+ del query_indices
+
if train_indices is not None:
train_dataset = data.Subset(dataset=train_dataset, indices=train_indices)
+ del train_indices
with self.profiler.profile("Load All Factors"):
loaded_factors = self.load_all_factors(
@@ -592,6 +595,7 @@ def compute_self_scores(
)
if train_indices is not None:
train_dataset = data.Subset(dataset=train_dataset, indices=train_indices)
+ del train_indices
with self.profiler.profile("Load All Factors"):
loaded_factors = self.load_all_factors(
diff --git a/kronfluence/version.py b/kronfluence/version.py
index b1a19e3..3b93d0b 100644
--- a/kronfluence/version.py
+++ b/kronfluence/version.py
@@ -1 +1 @@
-__version__ = "0.0.5"
+__version__ = "0.0.2"
diff --git a/pyproject.toml b/pyproject.toml
index f6b3690..a8ddb26 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -17,7 +17,7 @@ docstring-code-line-length = "dynamic"
[tool.pylint.format]
max-line-length = "120"
-max-locals = 40
+max-locals = 45
max-args = 20
max-branches = 30
max-statements = 90
diff --git a/tests/factors/test_covariances.py b/tests/factors/test_covariances.py
index f1eba04..c731a08 100644
--- a/tests/factors/test_covariances.py
+++ b/tests/factors/test_covariances.py
@@ -8,13 +8,13 @@
from kronfluence.analyzer import Analyzer, prepare_model
from kronfluence.arguments import FactorArguments
+from kronfluence.task import Task
from kronfluence.utils.constants import (
ACTIVATION_COVARIANCE_MATRIX_NAME,
COVARIANCE_FACTOR_NAMES,
GRADIENT_COVARIANCE_MATRIX_NAME,
NUM_COVARIANCE_PROCESSED,
)
-from kronfluence.task import Task
from kronfluence.utils.dataset import DataLoaderKwargs
from tests.utils import ATOL, RTOL, check_tensor_dict_equivalence, prepare_test
diff --git a/tests/factors/test_eigens.py b/tests/factors/test_eigens.py
index 39544e6..bb83049 100644
--- a/tests/factors/test_eigens.py
+++ b/tests/factors/test_eigens.py
@@ -8,6 +8,7 @@
from kronfluence.analyzer import Analyzer, prepare_model
from kronfluence.arguments import FactorArguments
+from kronfluence.task import Task
from kronfluence.utils.constants import (
ACTIVATION_EIGENVECTORS_NAME,
EIGENDECOMPOSITION_FACTOR_NAMES,
@@ -16,7 +17,6 @@
LAMBDA_MATRIX_NAME,
NUM_LAMBDA_PROCESSED,
)
-from kronfluence.task import Task
from kronfluence.utils.dataset import DataLoaderKwargs
from tests.utils import ATOL, RTOL, check_tensor_dict_equivalence, prepare_test
diff --git a/tests/gpu_tests/ddp_variation_test.py b/tests/gpu_tests/ddp_variation_test.py
index b2a6d3c..7d2dbb3 100644
--- a/tests/gpu_tests/ddp_variation_test.py
+++ b/tests/gpu_tests/ddp_variation_test.py
@@ -14,7 +14,7 @@
from kronfluence.arguments import FactorArguments, ScoreArguments
from kronfluence.task import Task
from tests.gpu_tests.ddp_test import OLD_FACTOR_NAME
-from tests.gpu_tests.pipeline import BATCH_DTYPE, construct_test_mlp, get_mnist_dataset
+from tests.gpu_tests.pipeline import BATCH_TYPE, construct_test_mlp, get_mnist_dataset
LOCAL_RANK = int(os.environ["LOCAL_RANK"])
WORLD_RANK = int(os.environ["RANK"])
@@ -27,7 +27,7 @@
class GpuVariationTask(Task):
def compute_train_loss(
self,
- batch: BATCH_DTYPE,
+ batch: BATCH_TYPE,
model: nn.Module,
sample: bool = False,
) -> torch.Tensor:
@@ -45,7 +45,7 @@ def compute_train_loss(
def compute_measurement(
self,
- batch: BATCH_DTYPE,
+ batch: BATCH_TYPE,
model: nn.Module,
) -> torch.Tensor:
inputs, labels = batch
diff --git a/tests/gpu_tests/pipeline.py b/tests/gpu_tests/pipeline.py
index b4b991e..117ec52 100644
--- a/tests/gpu_tests/pipeline.py
+++ b/tests/gpu_tests/pipeline.py
@@ -10,13 +10,13 @@
from kronfluence.task import Task
-BATCH_DTYPE = Tuple[torch.Tensor, torch.Tensor]
+BATCH_TYPE = Tuple[torch.Tensor, torch.Tensor]
class GpuTestTask(Task):
def compute_train_loss(
self,
- batch: BATCH_DTYPE,
+ batch: BATCH_TYPE,
model: nn.Module,
sample: bool = False,
) -> torch.Tensor:
@@ -34,7 +34,7 @@ def compute_train_loss(
def compute_measurement(
self,
- batch: BATCH_DTYPE,
+ batch: BATCH_TYPE,
model: nn.Module,
) -> torch.Tensor:
inputs, labels = batch
diff --git a/tests/scores/test_pairwise_scores.py b/tests/scores/test_pairwise_scores.py
index 3e04016..19045a4 100644
--- a/tests/scores/test_pairwise_scores.py
+++ b/tests/scores/test_pairwise_scores.py
@@ -8,8 +8,8 @@
from kronfluence.analyzer import Analyzer, prepare_model
from kronfluence.arguments import FactorArguments, ScoreArguments
-from kronfluence.utils.constants import ALL_MODULE_NAME
from kronfluence.task import Task
+from kronfluence.utils.constants import ALL_MODULE_NAME
from kronfluence.utils.dataset import DataLoaderKwargs
from tests.utils import ATOL, RTOL, check_tensor_dict_equivalence, prepare_test
diff --git a/tests/scores/test_self_scores.py b/tests/scores/test_self_scores.py
index 4803706..e4e40fa 100644
--- a/tests/scores/test_self_scores.py
+++ b/tests/scores/test_self_scores.py
@@ -8,8 +8,8 @@
from kronfluence.analyzer import Analyzer, prepare_model
from kronfluence.arguments import FactorArguments, ScoreArguments
-from kronfluence.utils.constants import ALL_MODULE_NAME
from kronfluence.task import Task
+from kronfluence.utils.constants import ALL_MODULE_NAME
from kronfluence.utils.dataset import DataLoaderKwargs
from tests.utils import ATOL, RTOL, check_tensor_dict_equivalence, prepare_test
diff --git a/tests/test_per_sample_gradients.py b/tests/test_per_sample_gradients.py
index 18c8cd8..b3634ec 100644
--- a/tests/test_per_sample_gradients.py
+++ b/tests/test_per_sample_gradients.py
@@ -13,13 +13,10 @@
from kronfluence.analyzer import Analyzer, prepare_model
from kronfluence.arguments import FactorArguments
-from kronfluence.utils.constants import (
- LAMBDA_MATRIX_NAME,
- PRECONDITIONED_GRADIENT_NAME,
-)
from kronfluence.module.tracked_module import ModuleMode, TrackedModule
from kronfluence.module.utils import set_mode, update_factor_args
from kronfluence.task import Task
+from kronfluence.utils.constants import LAMBDA_MATRIX_NAME, PRECONDITIONED_GRADIENT_NAME
from kronfluence.utils.dataset import DataLoaderKwargs
from tests.utils import (
ATOL,
@@ -98,6 +95,7 @@ def for_loop_per_sample_gradient(
"conv",
"conv_bn",
"bert",
+ "gpt",
],
)
@pytest.mark.parametrize("use_measurement", [True, False])
@@ -180,7 +178,11 @@ def test_for_loop_per_sample_gradient_equivalence(
task=task,
use_measurement=use_measurement,
)
+
for i in range(num_batches):
+ if "lm_head" in for_loop_per_sample_gradients[i]:
+ del for_loop_per_sample_gradients[i]["lm_head"]
+
assert check_tensor_dict_equivalence(
per_sample_gradients[i],
for_loop_per_sample_gradients[i],
@@ -195,6 +197,7 @@ def test_for_loop_per_sample_gradient_equivalence(
"mlp",
"repeated_mlp",
"conv",
+ "gpt",
],
)
@pytest.mark.parametrize("train_size", [32])
@@ -274,9 +277,8 @@ def test_lambda_equivalence(
)
-@pytest.mark.parametrize("seed", [0])
def test_precondition_gradient(
- seed: int,
+ seed: int = 0,
) -> None:
input_dim = 128
output_dim = 256
@@ -329,9 +331,8 @@ def test_precondition_gradient(
assert torch.allclose(raw_results, results, atol=1e-5, rtol=1e-3)
-@pytest.mark.parametrize("seed", [0])
def test_query_gradient_svd(
- seed: int,
+ seed: int = 0,
) -> None:
input_dim = 2048
output_dim = 1024
@@ -391,7 +392,6 @@ def test_query_gradient_svd(
assert torch.allclose(score, lr_score_reconst_matmul)
# These should be able to avoid explicit reconstruction.
-
# This should be used when input_dim > output_dim.
intermediate = opt_einsum.contract("qki,toi->qtko", right_mat, new_gradient)
final = opt_einsum.contract("qtko,qok->qt", intermediate, left_mat)
@@ -477,9 +477,8 @@ def test_query_gradient_svd_reconst(
assert intermediate.numel() <= reconst_numel
-@pytest.mark.parametrize("seed", [0])
def test_compute_score_matmul(
- seed: int,
+ seed: int = 0,
) -> None:
input_dim = 1024
output_dim = 2048
@@ -498,29 +497,3 @@ def test_compute_score_matmul(
assert torch.allclose(score, unsqueeze_score)
path = opt_einsum.contract_path("t...,q...->tq", gradient, new_gradient)
print(path)
-
-
-@pytest.mark.parametrize("seed", [0])
-def test_compute_score_fast_matmul(
- seed: int,
-) -> None:
- input_dim = 512
- output_dim = 1024
- seq_len = 32
- batch_dim = 8
- query_batch_dim = 16
-
- set_seed(seed)
-
- input_activation = torch.rand(size=(batch_dim, seq_len, input_dim), dtype=torch.float64)
- output_gradient = torch.rand(size=(batch_dim, seq_len, output_dim), dtype=torch.float64)
- per_sample_gradient = opt_einsum.contract("b...i,b...o->bio", output_gradient, input_activation)
- gradient = torch.rand(size=(query_batch_dim, output_dim, input_dim), dtype=torch.float64)
- score = opt_einsum.contract("toi,qoi->tq", per_sample_gradient, gradient)
- print(score)
-
- all_score = opt_einsum.contract("tco,tci,qoi->tq", output_gradient, input_activation, gradient)
- assert torch.allclose(score, all_score)
-
- path = opt_einsum.contract_path("tco,tci,qoi->tq", output_gradient, input_activation, gradient, optimize="optimal")
- print(path)
diff --git a/tests/test_regression.py b/tests/test_regression.py
new file mode 100644
index 0000000..bc9aaea
--- /dev/null
+++ b/tests/test_regression.py
@@ -0,0 +1,146 @@
+# pylint: skip-file
+
+import torch
+
+from kronfluence.analyzer import Analyzer, prepare_model
+from kronfluence.arguments import FactorArguments
+from kronfluence.utils.dataset import DataLoaderKwargs
+from tests.utils import prepare_test
+
+
+def test_mlp_regression(
+ test_name: str = "mlp",
+ strategy: str = "ekfac",
+ seed: int = 0,
+ train_size: int = 32,
+ query_size: int = 16,
+) -> None:
+ model, train_dataset, query_dataset, data_collator, task = prepare_test(
+ test_name=test_name,
+ train_size=train_size,
+ query_size=query_size,
+ seed=seed,
+ )
+ assert round(list(model.named_parameters())[0][1].sum().item(), 2) == -2.45
+
+ model = prepare_model(model=model, task=task)
+ analyzer = Analyzer(
+ analysis_name=f"pytest_regression_{test_name}",
+ model=model,
+ task=task,
+ disable_model_save=True,
+ cpu=True,
+ )
+ kwargs = DataLoaderKwargs(collate_fn=data_collator)
+ factor_args = FactorArguments(strategy=strategy, use_empirical_fisher=True)
+ analyzer.fit_covariance_matrices(
+ factors_name=f"pytest_{test_name}",
+ dataset=train_dataset,
+ per_device_batch_size=1,
+ dataloader_kwargs=kwargs,
+ factor_args=factor_args,
+ overwrite_output_dir=True,
+ )
+ covariance_matrices = analyzer.load_covariance_matrices(f"pytest_{test_name}")
+ assert round(torch.sum(covariance_matrices["activation_covariance"]["0"] / train_size).item(), 2) == 7.81
+
+ analyzer.perform_eigendecomposition(
+ factors_name=f"pytest_{test_name}",
+ factor_args=factor_args,
+ overwrite_output_dir=True,
+ )
+ eigen_factors = analyzer.load_eigendecomposition(f"pytest_{test_name}")
+ assert round(eigen_factors["activation_eigenvectors"]["0"].sum().item(), 2) == 2.64
+
+ analyzer.fit_lambda_matrices(
+ factors_name=f"pytest_{test_name}",
+ dataset=train_dataset,
+ per_device_batch_size=1,
+ dataloader_kwargs=kwargs,
+ factor_args=factor_args,
+ overwrite_output_dir=True,
+ )
+ lambda_matrices = analyzer.load_lambda_matrices(f"pytest_{test_name}")
+ assert round((lambda_matrices["lambda_matrix"]["0"] / train_size).sum().item(), 2) == 15.14
+
+ analyzer.compute_pairwise_scores(
+ scores_name="pairwise",
+ factors_name=f"pytest_{test_name}",
+ query_dataset=query_dataset,
+ per_device_query_batch_size=1,
+ train_dataset=train_dataset,
+ per_device_train_batch_size=1,
+ dataloader_kwargs=kwargs,
+ overwrite_output_dir=True,
+ )
+ scores = analyzer.load_pairwise_scores("pairwise")
+ assert round(scores["all_modules"].sum().item(), 2) == 145.53
+
+
+def test_conv_regression(
+ test_name: str = "conv",
+ strategy: str = "ekfac",
+ seed: int = 0,
+ train_size: int = 32,
+ query_size: int = 16,
+) -> None:
+ model, train_dataset, query_dataset, data_collator, task = prepare_test(
+ test_name=test_name,
+ train_size=train_size,
+ query_size=query_size,
+ seed=seed,
+ )
+ assert round(list(model.named_parameters())[0][1].sum().item(), 2) == -0.75
+
+ model = prepare_model(model=model, task=task)
+ analyzer = Analyzer(
+ analysis_name=f"pytest_regression_{test_name}",
+ model=model,
+ task=task,
+ disable_model_save=True,
+ cpu=True,
+ )
+ kwargs = DataLoaderKwargs(collate_fn=data_collator)
+ factor_args = FactorArguments(strategy=strategy, use_empirical_fisher=True)
+ analyzer.fit_covariance_matrices(
+ factors_name=f"pytest_{test_name}",
+ dataset=train_dataset,
+ per_device_batch_size=1,
+ dataloader_kwargs=kwargs,
+ factor_args=factor_args,
+ overwrite_output_dir=True,
+ )
+ covariance_matrices = analyzer.load_covariance_matrices(f"pytest_{test_name}")
+ assert round(torch.sum(covariance_matrices["activation_covariance"]["0"] / train_size).item(), 2) == 42299.42
+
+ analyzer.perform_eigendecomposition(
+ factors_name=f"pytest_{test_name}",
+ factor_args=factor_args,
+ overwrite_output_dir=True,
+ )
+ eigen_factors = analyzer.load_eigendecomposition(f"pytest_{test_name}")
+ assert round(eigen_factors["activation_eigenvectors"]["0"].sum().item(), 2) == 4.34
+
+ analyzer.fit_lambda_matrices(
+ factors_name=f"pytest_{test_name}",
+ dataset=train_dataset,
+ per_device_batch_size=1,
+ dataloader_kwargs=kwargs,
+ factor_args=factor_args,
+ overwrite_output_dir=True,
+ )
+ lambda_matrices = analyzer.load_lambda_matrices(f"pytest_{test_name}")
+ assert round((lambda_matrices["lambda_matrix"]["0"] / train_size).sum().item(), 2) == 0.18
+
+ analyzer.compute_pairwise_scores(
+ scores_name="pairwise",
+ factors_name=f"pytest_{test_name}",
+ query_dataset=query_dataset,
+ per_device_query_batch_size=1,
+ train_dataset=train_dataset,
+ per_device_train_batch_size=1,
+ dataloader_kwargs=kwargs,
+ overwrite_output_dir=True,
+ )
+ scores = analyzer.load_pairwise_scores("pairwise")
+ assert round(scores["all_modules"].sum().item(), 2) == 6268.84
diff --git a/tests/testable_tasks/classification.py b/tests/testable_tasks/classification.py
index a8d18c2..ad80a09 100644
--- a/tests/testable_tasks/classification.py
+++ b/tests/testable_tasks/classification.py
@@ -6,10 +6,11 @@
import torch.nn.functional as F
import torchvision
from accelerate.utils import set_seed
-from kronfluence.task import Task
from torch import nn
from torch.utils import data
+from kronfluence.task import Task
+
BATCH_TYPE = Tuple[torch.Tensor, torch.Tensor]
diff --git a/tests/testable_tasks/language_modeling.py b/tests/testable_tasks/language_modeling.py
index 39cfd26..72d2f5a 100644
--- a/tests/testable_tasks/language_modeling.py
+++ b/tests/testable_tasks/language_modeling.py
@@ -6,11 +6,12 @@
import torch
import torch.nn.functional as F
from datasets import load_dataset
-from kronfluence.task import Task
from torch import nn
from torch.utils import data
from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer, Conv1D
+from kronfluence.task import Task
+
BATCH_TYPE = Dict[str, torch.Tensor]
@@ -130,11 +131,11 @@ def compute_measurement(
def tracked_modules(self) -> List[str]:
total_modules = []
- for i in range(4):
+ for i in range(5):
total_modules.append(f"transformer.h.{i}.attn.c_attn")
total_modules.append(f"transformer.h.{i}.attn.c_proj")
- for i in range(4):
+ for i in range(5):
total_modules.append(f"transformer.h.{i}.mlp.c_fc")
total_modules.append(f"transformer.h.{i}.mlp.c_proj")
diff --git a/tests/testable_tasks/text_classification.py b/tests/testable_tasks/text_classification.py
index 893c546..30fe9c3 100644
--- a/tests/testable_tasks/text_classification.py
+++ b/tests/testable_tasks/text_classification.py
@@ -5,11 +5,12 @@
import torch
import torch.nn.functional as F
from datasets import load_dataset
-from kronfluence.task import Task
from torch import nn
from torch.utils import data
from transformers import AutoConfig, AutoModelForSequenceClassification, AutoTokenizer
+from kronfluence.task import Task
+
BATCH_TYPE = Dict[str, torch.Tensor]
diff --git a/tests/utils.py b/tests/utils.py
index 2079921..7ecc0e0 100644
--- a/tests/utils.py
+++ b/tests/utils.py
@@ -108,6 +108,9 @@ def reshape_parameter_gradient_to_module_matrix(
remove_gradient: bool = True,
) -> torch.Tensor:
if isinstance(module, nn.Linear):
+ if module_name == "lm_head":
+ # Edge case for small GPT model.
+ return
gradient_matrix = gradient_dict[module_name + ".weight"]
if remove_gradient:
del gradient_dict[module_name + ".weight"]
+
@@ -19,7 +23,7 @@
---
-> **Kronfluence** is a repository designed to compute [influence functions](https://arxiv.org/abs/1703.04730) using [Kronecker-factored Approximate Curvature (KFAC)](https://arxiv.org/abs/1503.05671) or [Eigenvalue-corrected KFAC (EKFAC)](https://arxiv.org/abs/1806.03884).
+> **Kronfluence** is a research repository designed to compute [influence functions](https://arxiv.org/abs/1703.04730) using [Kronecker-factored Approximate Curvature (KFAC)](https://arxiv.org/abs/1503.05671) or [Eigenvalue-corrected KFAC (EKFAC)](https://arxiv.org/abs/1806.03884).
For a detailed description of the methodology, see the [**paper**](https://arxiv.org/abs/2308.03296) *Studying Large Language Model Generalization with Influence Functions*.
---
@@ -53,10 +57,10 @@ pip install -e .
Kronfluence supports influence computations on `nn.Linear` and `nn.Conv2d` modules. See the [**Technical Documentation**](https://github.com/pomonam/kronfluence/blob/main/DOCUMENTATION.md)
page for a comprehensive guide.
-### Examples
+### Learn More
-The [examples](https://github.com/pomonam/kronfluence/tree/main/examples) folder contains several examples on how to use Kronfluence.
-We plan to add more language model examples. **TL;DR** You need to prepare the trained model and datasets, and pass them into the `Analyzer`.
+The [examples](https://github.com/pomonam/kronfluence/tree/main/examples) folder contains several examples.
+We plan to add more examples in the future. **TL;DR** You need to prepare the trained model and datasets, and pass them into `Analyzer`.
```python
import torch
@@ -90,27 +94,27 @@ eval_dataset = torchvision.datasets.MNIST(
train=True,
)
-# Initialize the task with relevant loss and measurement.
+# Define the task.
task = MnistTask()
-# Prepare the model for influence computation with the specified task.
+# Prepare the model for influence computation.
model = prepare_model(model=model, task=task)
analyzer = Analyzer(analysis_name="mnist", model=model, task=task)
-# Fit all EKFAC factors for the given model on the training dataset.
-analyzer.fit_all_factors(factors_name="ekfac", dataset=train_dataset)
+# Fit all EKFAC factors for the given model.
+analyzer.fit_all_factors(factors_name="my_factors", dataset=train_dataset)
-# Compute all pairwise influence scores using the computed factors.
+# Compute all pairwise influence scores with the computed factors.
analyzer.compute_pairwise_scores(
- scores_name="pairwise_scores",
- factors_name="ekfac",
+ scores_name="my_scores",
+ factors_name="my_factors",
query_dataset=eval_dataset,
train_dataset=train_dataset,
per_device_query_batch_size=1024,
)
# Load the scores with dimension `len(eval_dataset) x len(train_dataset)`.
-scores = analyzer.load_pairwise_scores(scores_name="pairwise_scoeres")
+scores = analyzer.load_pairwise_scores(scores_name="my_scores")
```
## Contributing
diff --git a/examples/_test_requirements.txt b/examples/_test_requirements.txt
deleted file mode 100644
index ec96d22..0000000
--- a/examples/_test_requirements.txt
+++ /dev/null
@@ -1,3 +0,0 @@
-scikit-learn
-jupyter
-evaluate
\ No newline at end of file
diff --git a/examples/cifar/README.md b/examples/cifar/README.md
new file mode 100644
index 0000000..67b16d5
--- /dev/null
+++ b/examples/cifar/README.md
@@ -0,0 +1,57 @@
+# CIFAR-10 & ResNet-9 Example
+
+This directory contains scripts for training ResNet-9 on CIFAR-10. The pipeline is motivated from
+[TRAK repository](https://github.com/MadryLab/trak/blob/main/examples/cifar_quickstart.ipynb).
+
+## Training
+
+To train ResNet-9 on CIFAR-10 dataset, run the following command:
+```bash
+python train.py --dataset_dir ./data \
+ --checkpoint_dir ./checkpoints \
+ --train_batch_size 512 \
+ --eval_batch_size 1024 \
+ --learning_rate 0.4 \
+ --weight_decay 0.0001 \
+ --num_train_epochs 25 \
+ --seed 1004
+```
+
+## Computing Pairwise Influence Scores
+
+To obtain pairwise influence scores on 2000 query data points using `ekfac`, run the following command:
+```bash
+python analyze.py --query_batch_size 1000 \
+ --dataset_dir ./data \
+ --checkpoint_dir ./checkpoints \
+ --factor_strategy ekfac
+```
+You can also use `identity`, `diagonal`, and `kfac`. On A100 (80GB), it takes roughly 1.5 minutes to compute the
+pairwise scores (including computing EKFAC factors).
+
+## Mislabeled Data Detection
+
+We can use self-influence scores (see Section 5.4 for the [paper](https://arxiv.org/pdf/1703.04730.pdf)) to detect mislabeled examples.
+First, train the model with 10% of training examples mislabeled by running the following command:
+```bash
+python train.py --dataset_dir ./data \
+ --corrupt_percentage 0.1 \
+ --checkpoint_dir ./checkpoints \
+ --train_batch_size 512 \
+ --eval_batch_size 1024 \
+ --learning_rate 0.4 \
+ --weight_decay 0.0001 \
+ --num_train_epochs 25 \
+ --seed 1004
+```
+
+Then, compute self-influence scores with the following command:
+```bash
+python detect_mislabeled_dataset.py --dataset_dir ./data \
+ --corrupt_percentage 0.1 \
+ --checkpoint_dir ./checkpoints \
+ --factor_strategy ekfac
+```
+
+On A100 (80GB), it takes roughly 1.5 minutes to compute the self-influence scores.
+We can detect around 82% of mislabeled data points by inspecting 10% of the dataset (96% by inspecting 20%).
\ No newline at end of file
diff --git a/examples/cifar/analyze.py b/examples/cifar/analyze.py
new file mode 100644
index 0000000..8669924
--- /dev/null
+++ b/examples/cifar/analyze.py
@@ -0,0 +1,157 @@
+import argparse
+import logging
+import os
+from typing import Tuple
+
+import torch
+import torch.nn.functional as F
+from torch import nn
+
+from examples.cifar.pipeline import construct_resnet9, get_cifar10_dataset
+from kronfluence.analyzer import Analyzer, prepare_model
+from kronfluence.arguments import FactorArguments
+from kronfluence.task import Task
+from kronfluence.utils.dataset import DataLoaderKwargs
+
+BATCH_TYPE = Tuple[torch.Tensor, torch.Tensor]
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description="Influence analysis on CIFAR-10 dataset.")
+
+ parser.add_argument(
+ "--corrupt_percentage",
+ type=float,
+ default=None,
+ help="Percentage of the training dataset to corrupt.",
+ )
+ parser.add_argument(
+ "--dataset_dir",
+ type=str,
+ default="./data",
+ help="A folder to download or load CIFAR-10 dataset.",
+ )
+ parser.add_argument(
+ "--checkpoint_dir",
+ type=str,
+ default="./checkpoints",
+ help="A path that is storing the final checkpoint of the model.",
+ )
+
+ parser.add_argument(
+ "--query_batch_size",
+ type=int,
+ default=1000,
+ help="Batch size for computing query gradients.",
+ )
+ parser.add_argument(
+ "--factor_strategy",
+ type=str,
+ default="ekfac",
+ help="Strategy to compute influence factors.",
+ )
+
+ args = parser.parse_args()
+
+ if args.checkpoint_dir is not None:
+ os.makedirs(args.checkpoint_dir, exist_ok=True)
+
+ return args
+
+
+class ClassificationTask(Task):
+ def compute_train_loss(
+ self,
+ batch: BATCH_TYPE,
+ model: nn.Module,
+ sample: bool = False,
+ ) -> torch.Tensor:
+ inputs, labels = batch
+ logits = model(inputs)
+ if not sample:
+ return F.cross_entropy(logits, labels, reduction="sum")
+ with torch.no_grad():
+ probs = torch.nn.functional.softmax(logits, dim=-1)
+ sampled_labels = torch.multinomial(
+ probs,
+ num_samples=1,
+ ).flatten()
+ return F.cross_entropy(logits, sampled_labels.detach(), reduction="sum")
+
+ def compute_measurement(
+ self,
+ batch: BATCH_TYPE,
+ model: nn.Module,
+ ) -> torch.Tensor:
+ # Copied from: https://github.com/MadryLab/trak/blob/main/trak/modelout_functions.py.
+ inputs, labels = batch
+ logits = model(inputs)
+
+ bindex = torch.arange(logits.shape[0]).to(device=logits.device, non_blocking=False)
+ logits_correct = logits[bindex, labels]
+
+ cloned_logits = logits.clone()
+ cloned_logits[bindex, labels] = torch.tensor(-torch.inf, device=logits.device, dtype=logits.dtype)
+
+ margins = logits_correct - cloned_logits.logsumexp(dim=-1)
+ return -margins.sum()
+
+
+def main():
+ args = parse_args()
+ logging.basicConfig(level=logging.INFO)
+
+ # Prepare the dataset.
+ train_dataset = get_cifar10_dataset(
+ split="eval_train", corrupt_percentage=args.corrupt_percentage, dataset_dir=args.dataset_dir
+ )
+ eval_dataset = get_cifar10_dataset(split="valid", dataset_dir=args.dataset_dir)
+
+ # Prepare the trained model.
+ model = construct_resnet9()
+ model_name = "model"
+ if args.corrupt_percentage is not None:
+ model_name += "_corrupt_" + str(args.corrupt_percentage)
+ checkpoint_path = os.path.join(args.checkpoint_dir, f"{model_name}.pth")
+ if not os.path.isfile(checkpoint_path):
+ raise ValueError(f"No checkpoint found at {checkpoint_path}.")
+ model.load_state_dict(torch.load(checkpoint_path))
+
+ # Define task and prepare model.
+ task = ClassificationTask()
+ model = prepare_model(model, task)
+
+ analyzer = Analyzer(
+ analysis_name="cifar10",
+ model=model,
+ task=task,
+ )
+ # Configure parameters for DataLoader.
+ dataloader_kwargs = DataLoaderKwargs(num_workers=4)
+ analyzer.set_dataloader_kwargs(dataloader_kwargs)
+
+ # Compute influence factors.
+ factor_args = FactorArguments(strategy=args.factor_strategy)
+ analyzer.fit_all_factors(
+ factors_name=args.factor_strategy,
+ dataset=train_dataset,
+ per_device_batch_size=None,
+ factor_args=factor_args,
+ overwrite_output_dir=False,
+ )
+ # Compute pairwise scores.
+ analyzer.compute_pairwise_scores(
+ scores_name=args.factor_strategy,
+ factors_name=args.factor_strategy,
+ query_dataset=eval_dataset,
+ query_indices=list(range(2000)),
+ train_dataset=train_dataset,
+ per_device_query_batch_size=args.query_batch_size,
+ overwrite_output_dir=False,
+ )
+ scores = analyzer.load_pairwise_scores(args.factor_strategy)["all_modules"]
+ logging.info(f"Scores shape: {scores.shape}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/cifar/detect_mislabeled_dataset.py b/examples/cifar/detect_mislabeled_dataset.py
new file mode 100644
index 0000000..efcd84f
--- /dev/null
+++ b/examples/cifar/detect_mislabeled_dataset.py
@@ -0,0 +1,117 @@
+import argparse
+import logging
+import os
+
+import torch
+
+from examples.cifar.analyze import ClassificationTask
+from examples.cifar.pipeline import construct_resnet9, get_cifar10_dataset
+from kronfluence.analyzer import Analyzer, prepare_model
+from kronfluence.arguments import FactorArguments
+from kronfluence.utils.dataset import DataLoaderKwargs
+
+
+def parse_args():
+ parser = argparse.ArgumentParser(description="Detecting mislabeled CIFAR-10 data points.")
+
+ parser.add_argument(
+ "--corrupt_percentage",
+ type=float,
+ default=0.1,
+ help="Percentage of the training dataset to corrupt.",
+ )
+ parser.add_argument(
+ "--dataset_dir",
+ type=str,
+ default="./data",
+ help="A folder to download or load CIFAR-10 dataset.",
+ )
+ parser.add_argument(
+ "--checkpoint_dir",
+ type=str,
+ default="./checkpoints",
+ help="A path that is storing the final checkpoint of the model.",
+ )
+
+ parser.add_argument(
+ "--factor_strategy",
+ type=str,
+ default="ekfac",
+ help="Strategy to compute influence factors.",
+ )
+
+ args = parser.parse_args()
+
+ if args.checkpoint_dir is not None:
+ os.makedirs(args.checkpoint_dir, exist_ok=True)
+
+ return args
+
+
+def main():
+ args = parse_args()
+ logging.basicConfig(level=logging.INFO)
+
+ # Prepare the dataset.
+ train_dataset = get_cifar10_dataset(
+ split="eval_train", corrupt_percentage=args.corrupt_percentage, dataset_dir=args.dataset_dir
+ )
+
+ # Prepare the trained model.
+ model = construct_resnet9()
+ model_name = "model"
+ if args.corrupt_percentage is not None:
+ model_name += "_corrupt_" + str(args.corrupt_percentage)
+ checkpoint_path = os.path.join(args.checkpoint_dir, f"{model_name}.pth")
+ if not os.path.isfile(checkpoint_path):
+ raise ValueError(f"No checkpoint found at {checkpoint_path}.")
+ model.load_state_dict(torch.load(checkpoint_path))
+
+ # Define task and prepare model.
+ task = ClassificationTask()
+ model = prepare_model(model, task)
+
+ analyzer = Analyzer(
+ analysis_name="mislabeled",
+ model=model,
+ task=task,
+ )
+ # Configure parameters for DataLoader.
+ dataloader_kwargs = DataLoaderKwargs(num_workers=4)
+ analyzer.set_dataloader_kwargs(dataloader_kwargs)
+
+ # Compute influence factors.
+ factor_args = FactorArguments(strategy=args.factor_strategy)
+ analyzer.fit_all_factors(
+ factors_name=args.factor_strategy,
+ dataset=train_dataset,
+ per_device_batch_size=None,
+ factor_args=factor_args,
+ overwrite_output_dir=False,
+ )
+ # Compute self-influence scores.
+ analyzer.compute_self_scores(
+ scores_name=args.factor_strategy,
+ factors_name=args.factor_strategy,
+ train_dataset=train_dataset,
+ overwrite_output_dir=True,
+ )
+ scores = analyzer.load_pairwise_scores(args.factor_strategy)["all_modules"]
+
+ total_corrupt_size = int(args.corrupt_percentage * len(train_dataset))
+ corrupted_indices = list(range(int(args.corrupt_percentage * len(train_dataset))))
+ intervals = torch.arange(0.1, 1, 0.1)
+
+ accuracies = []
+ for interval in intervals:
+ interval = interval.item()
+ predicted_indices = torch.argsort(scores, descending=True)[: int(interval * len(train_dataset))]
+ predicted_indices = list(predicted_indices.numpy())
+ accuracies.append(len(set(predicted_indices) & set(corrupted_indices)) / total_corrupt_size)
+
+ logging.info(f"Inspect Interval: {list(intervals.numpy())}")
+ logging.info(f"Detection Accuracy: {accuracies}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/examples/cifar/pipeline.py b/examples/cifar/pipeline.py
index 420d1fd..dd1f68e 100644
--- a/examples/cifar/pipeline.py
+++ b/examples/cifar/pipeline.py
@@ -33,7 +33,7 @@ def forward(self, x: torch.Tensor) -> torch.Tensor:
def construct_resnet9() -> nn.Module:
- # ResNet-9 architecture from: https://github.com/MadryLab/trak/blob/main/examples/cifar_quickstart.ipynb.
+ # ResNet-9 architecture from https://github.com/MadryLab/trak/blob/main/examples/cifar_quickstart.ipynb.
def conv_bn(
channels_in: int,
channels_out: int,
@@ -81,7 +81,7 @@ def get_cifar10_dataset(
assert split in ["train", "eval_train", "valid"]
normalize = torchvision.transforms.Normalize(mean=(0.4914, 0.4822, 0.4465), std=(0.247, 0.243, 0.261))
- if split in ["train", "eval_train"]:
+ if split == "train":
transform_config = torchvision.transforms.Compose(
[
torchvision.transforms.RandomCrop(32, padding=4),
diff --git a/examples/cifar/train.py b/examples/cifar/train.py
index 2228a21..1d76f79 100644
--- a/examples/cifar/train.py
+++ b/examples/cifar/train.py
@@ -1,6 +1,7 @@
import argparse
import logging
import os
+import time
from typing import Tuple
import numpy as np
@@ -10,7 +11,6 @@
from torch import nn
from torch.optim import lr_scheduler
from torch.utils import data
-from tqdm import tqdm
from examples.cifar.pipeline import construct_resnet9, get_cifar10_dataset
@@ -55,7 +55,7 @@ def parse_args():
parser.add_argument(
"--weight_decay",
type=float,
- default=0.001,
+ default=0.0001,
help="Weight decay to train the model.",
)
parser.add_argument(
@@ -92,7 +92,6 @@ def train(
num_train_epochs: int,
learning_rate: float,
weight_decay: float,
- disable_tqdm: bool = False,
) -> nn.Module:
train_dataloader = data.DataLoader(
dataset=dataset,
@@ -105,7 +104,7 @@ def train(
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate, weight_decay=weight_decay)
iters_per_epoch = len(train_dataloader)
- lr_peak_epoch = num_train_epochs // 4
+ lr_peak_epoch = num_train_epochs // 5
lr_schedule = np.interp(
np.arange((num_train_epochs + 1) * iters_per_epoch),
[0, lr_peak_epoch * iters_per_epoch, num_train_epochs * iters_per_epoch],
@@ -113,22 +112,24 @@ def train(
)
scheduler = lr_scheduler.LambdaLR(optimizer, lr_schedule.__getitem__)
+ start_time = time.time()
model.train()
for epoch in range(num_train_epochs):
total_loss = 0.0
- with tqdm(train_dataloader, unit="batch", disable=disable_tqdm) as tepoch:
- for batch in tepoch:
- tepoch.set_description(f"Epoch {epoch}")
- model.zero_grad()
- inputs, labels = batch
- inputs, labels = inputs.to(DEVICE), labels.to(DEVICE)
- outputs = model(inputs)
- loss = F.cross_entropy(outputs, labels)
- loss.backward()
- optimizer.step()
- scheduler.step()
- total_loss += loss.detach().float()
- tepoch.set_postfix(loss=total_loss.item() / len(train_dataloader))
+ for batch in train_dataloader:
+ model.zero_grad()
+ inputs, labels = batch
+ inputs, labels = inputs.to(DEVICE), labels.to(DEVICE)
+ outputs = model(inputs)
+ loss = F.cross_entropy(outputs, labels)
+ loss.backward()
+ optimizer.step()
+ scheduler.step()
+ total_loss += loss.detach().float()
+ logging.info(f"Epoch {epoch + 1} - Averaged Loss: {total_loss / len(dataset)}")
+ end_time = time.time()
+ elapsed_time = end_time - start_time
+ logging.info(f"Completed training in {elapsed_time:.2f} seconds.")
return model
diff --git a/examples/glue/README.md b/examples/glue/README.md
new file mode 100644
index 0000000..12398a1
--- /dev/null
+++ b/examples/glue/README.md
@@ -0,0 +1,60 @@
+# GLUE & BERT Example
+
+This directory contains scripts for fine-tuning BERT on GLUE benchmark. The pipeline is motivated from [HuggingFace Example](https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-classification).
+Please begin by installing necessary packages.
+```bash
+pip install -r requirements.txt
+```
+
+## Training
+
+To fine-tune BERT on some specific dataset, run the following command (we are using `SST2` dataset):
+```bash
+python train.py --dataset_name sst2 \
+ --checkpoint_dir ./checkpoints \
+ --train_batch_size 32 \
+ --eval_batch_size 32 \
+ --learning_rate 3e-05 \
+ --weight_decay 0.01 \
+ --num_train_epochs 3 \
+ --seed 1004
+```
+
+## Computing Pairwise Influence Scores
+
+To obtain a pairwise influence scores on maximum of 2000 query data points using `ekfac`, run the following command:
+```bash
+python analyze.py --dataset_name sst2 \
+ --query_batch_size 175 \
+ --train_batch_size 128 \
+ --checkpoint_dir ./checkpoints \
+ --factor_strategy ekfac
+```
+On A100 (80GB), it takes roughly 80 minutes to compute the pairwise scores for SST2 with around 900 query data points
+(including computing EKFAC factors).
+
+We can also use query batching (low-rank approximation to the query gradient; see Section 3.2.2 from the [paper](https://arxiv.org/pdf/2308.03296.pdf)) to compute influence scores with a
+larger query batch size.
+```bash
+python analyze.py --dataset_name sst2 \
+ --query_gradient_rank 32 \
+ --query_batch_size 436 \
+ --train_batch_size 256 \
+ --checkpoint_dir ./checkpoints \
+ --factor_strategy ekfac
+```
+Note that query batching is slower in this case (140 minutes in total), as the number of training data points is small and the cost of performing SVD dominates the overall cost.
+Assuming that you ran above two commands, `query_batching_analysis.py` contains code to compute the correlations between the full rank and low-rank scores.
+
+
+
+