diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 0000000..6759ed0 --- /dev/null +++ b/.nojekyll @@ -0,0 +1 @@ +557f7733 \ No newline at end of file diff --git a/AoGPlots.html b/AoGPlots.html new file mode 100644 index 0000000..0b6d0a6 --- /dev/null +++ b/AoGPlots.html @@ -0,0 +1,1327 @@ + + + + + + + + + + + +Creating multi-panel plots – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Creating multi-panel plots

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

This notebook shows creating a multi-panel plot similar to Figure 2 of Fühner et al. (2021).

+

The data are available from the SMLP2024 example datasets.

+
+
+Code +
using Arrow
+using AlgebraOfGraphics
+using CairoMakie   # for displaying static plots
+using DataFrames
+using Statistics
+using StatsBase
+using SMLP2024: dataset
+
+CairoMakie.activate!(; type="svg") # use SVG (other options include PNG)
+
+
+
+
tbl = dataset("fggk21")
+
+
Arrow.Table with 525126 rows, 7 columns, and schema:
+ :Cohort  String
+ :School  String
+ :Child   String
+ :Sex     String
+ :age     Float64
+ :Test    String
+ :score   Float64
+
+
+
+
typeof(tbl)
+
+
Arrow.Table
+
+
+
+
df = DataFrame(tbl)
+typeof(df)
+
+
DataFrame
+
+
+
+

1 Creating a summary data frame

+

The response to be plotted is the mean score by Test and Sex and age, rounded to the nearest 0.1 years.

+

The first task is to round the age to 1 digit after the decimal place, which can be done with select applied to a DataFrame. In some ways this is the most complicated expression in creating the plot so we will break it down. select is applied to DataFrame(dat), which is the conversion of the Arrow.Table, dat, to a DataFrame. This is necessary because an Arrow.Table is immutable but a DataFrame can be modified.

+

The arguments after the DataFrame describe how to modify the contents. The first : indicates that all the existing columns should be included. The other expression can be pairs (created with the => operator) of the form :col => function or of the form :col => function => :newname. (See the documentation of the DataFrames package for details.)

+

In this case the function is an anonymous function of the form round.(x, digits=1) where “dot-broadcasting” is used to apply to the entire column (see this documentation for details).

+
+
transform!(df, :age, :age => (x -> x .- 8.5) => :a1) # centered age (linear)
+select!(groupby(df, :Test), :, :score => zscore => :zScore) # z-score
+tlabels = [     # establish order and labels of tbl.Test
+  "Run" => "Endurance",
+  "Star_r" => "Coordination",
+  "S20_r" => "Speed",
+  "SLJ" => "PowerLOW",
+  "BPT" => "PowerUP",
+];
+
+

The next stage is a group-apply-combine operation to group the rows by Sex, Test and rnd_age then apply mean to the zScore and also apply length to zScore to record the number in each group.

+
+
df2 = combine(
+  groupby(
+    select(df, :, :age => ByRow(x -> round(x; digits=1)) => :age),
+    [:Sex, :Test, :age],
+  ),
+  :zScore => mean => :zScore,
+  :zScore => length => :n,
+)
+
+
120×5 DataFrame
95 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowSexTestagezScoren
StringStringFloat64Float64Int64
1maleS20_r8.0-0.02651381223
2maleBPT8.00.0269731227
3maleSLJ8.00.1216091227
4maleStar_r8.0-0.05717261186
5maleRun8.00.2926951210
6femaleS20_r8.0-0.351641411
7femaleBPT8.0-0.6103551417
8femaleSLJ8.0-0.2798721418
9femaleStar_r8.0-0.2682211381
10femaleRun8.0-0.2455731387
11maleS20_r8.10.06083973042
12maleBPT8.10.09554133069
13maleSLJ8.10.1230993069
⋮⋮⋮⋮⋮⋮
109maleStar_r9.00.2549734049
110maleRun9.00.2580824034
111femaleS20_r9.1-0.02861721154
112femaleBPT9.1-0.07523011186
113femaleSLJ9.1-0.0945871174
114femaleStar_r9.10.002762521162
115femaleRun9.1-0.2355911150
116maleS20_r9.10.3257451303
117maleBPT9.10.6164161320
118maleSLJ9.10.2675771310
119maleStar_r9.10.2543421297
120maleRun9.10.2510451294
+
+
+
+
+
+

2 Creating the plot

+

The AlgebraOfGraphics package applies operators to the results of functions such as data (specify the data table to be used), mapping (designate the roles of columns), and visual (type of visual presentation).

+
+
let
+  design = mapping(:age, :zScore; color=:Sex, col=:Test)
+  lines = design * linear()
+  means = design * visual(Scatter; markersize=5)
+  draw(data(df2) * means + data(df) * lines)
+end
+
+
+
+

+
+
+
+
+
    +
  • TBD: Relabel factor levels (Boys, Girls; fitness components for Test)
    • +
  • TBD: Relevel factors; why not levels from Tables?
    • +
  • TBD: Set range (7.8 to 9.2 and tick marks (8, 8.5, 9) of axes.
    • +
  • TBD: Move legend in plot?
    • +
+
+
+Fühner, T., Granacher, U., Golle, K., & Kliegl, R. (2021). Age and sex effects in physical fitness components of 108,295 third graders including 515 primary schools and 9 cohorts. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-97000-4 +
+
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/AoGPlots_files/figure-html/cell-8-output-1.svg b/AoGPlots_files/figure-html/cell-8-output-1.svg new file mode 100644 index 0000000..f019a0d --- /dev/null +++ b/AoGPlots_files/figure-html/cell-8-output-1.svg @@ -0,0 +1,693 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/about.html b/about.html new file mode 100644 index 0000000..bc467b0 --- /dev/null +++ b/about.html @@ -0,0 +1,919 @@ + + + + + + + + + + + +About – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

About

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

About this site

+ + + + Back to top
+ +
+ + + + + \ No newline at end of file diff --git a/arrow.html b/arrow.html new file mode 100644 index 0000000..af7c9ee --- /dev/null +++ b/arrow.html @@ -0,0 +1,3720 @@ + + + + + + + + + + + +Saving data with Arrow – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Saving data with Arrow

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +
+

1 The Arrow storage format

+

The Arrow storage format provides a language-agnostic storage and memory specification for columnar data tables, which just means “something that looks like a data frame in R”. That is, an arrow table is an ordered, named collection of columns, all of the same length.

+

The columns can be of different types including numeric values, character strings, and factor-like representations - called DictEncoded.

+

An Arrow file can be read or written from R, Python, Julia and many other languages. Somewhat confusingly in R and Python the name feather, which refers to an earlier version of the storage format, is used in some function names like read_feather.

+

Internally, the SMLP2024 package uses Arrow to store all of its datasets.

+
+
+

2 The Emotikon data

+

The SMLP2021 repository contains a version of the data from Fühner et al. (2021) in notebooks/data/fggk21.arrow. After that file was created there were changes in the master RDS file on the osf.io site for the project. We will recreate the Arrow file here then split it into two separate tables, one with a row for each child in the study and one with a row for each test result.

+

The Arrow package for Julia does not export any function names, which means that the function to read an Arrow file must be called as Arrow.Table. It returns a column table, as described in the Tables package. This is like a read-only data frame, which can be easily converted to a full-fledged DataFrame if desired.

+

This arrangement allows for the Arrow package not to depend on the DataFrames package, which is a heavy-weight dependency, but still easily produce a DataFrame if warranted.

+

Load the packages to be used.

+
+
+Code +
using AlgebraOfGraphics
+using Arrow
+using CairoMakie
+using Chain
+using DataFrameMacros
+using DataFrames
+using Downloads
+using KernelDensity
+using RCall   # access R from within Julia
+using StatsBase
+
+CairoMakie.activate!(; type="svg")
+using AlgebraOfGraphics: density
+
+
+
+
+

3 Downloading and importing the RDS file

+

This is similar to some of the code shown by Julius Krumbiegel on Monday. In the data directory of the emotikon project on osf.io under Data, the url for the rds data file is found to be [https://osf.io/xawdb/]. Note that we want version 2 of this file.

+
+
fn = Downloads.download("https://osf.io/xawdb/download?version=2");
+
+
+
dfrm = rcopy(R"readRDS($fn)")
+
+
525126×7 DataFrame
525101 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowCohortSchoolChildSexageTestscore
Cat…Cat…Cat…Cat…Float64Cat…Float64
12013S100067C002352male7.99452S20_r5.26316
22013S100067C002352male7.99452BPT3.7
32013S100067C002352male7.99452SLJ125.0
42013S100067C002352male7.99452Star_r2.47146
52013S100067C002352male7.99452Run1053.0
62013S100067C002353male7.99452S20_r5.0
72013S100067C002353male7.99452BPT4.1
82013S100067C002353male7.99452SLJ116.0
92013S100067C002353male7.99452Star_r1.76778
102013S100067C002353male7.99452Run1089.0
112013S100067C002354male7.99452S20_r4.54545
122013S100067C002354male7.99452BPT3.9
132013S100067C002354male7.99452SLJ111.0
⋮⋮⋮⋮⋮⋮⋮⋮
5251152018S401470C117964male9.10609Star_r1.63704
5251162018S401470C117964male9.10609Run864.0
5251172018S401470C117965female9.10609S20_r4.65116
5251182018S401470C117965female9.10609BPT3.8
5251192018S401470C117965female9.10609SLJ123.0
5251202018S401470C117965female9.10609Star_r1.52889
5251212018S401470C117965female9.10609Run1080.0
5251222018S800200C117966male9.10609S20_r4.54545
5251232018S800200C117966male9.10609BPT3.8
5251242018S800200C117966male9.10609SLJ100.0
5251252018S800200C117966male9.10609Star_r2.18506
5251262018S800200C117966male9.10609Run990.0
+
+
+
+

Now write this file as a Arrow file and read it back in.

+
+
arrowfn = joinpath("data", "fggk21.arrow")
+Arrow.write(arrowfn, dfrm; compress=:lz4)
+tbl = Arrow.Table(arrowfn)
+
+
Arrow.Table with 525126 rows, 7 columns, and schema:
+ :Cohort  String
+ :School  String
+ :Child   String
+ :Sex     String
+ :age     Float64
+ :Test    String
+ :score   Float64
+
+
+
+
filesize(arrowfn)
+
+
3077850
+
+
+
+
df = DataFrame(tbl)
+
+
525126×7 DataFrame
525101 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowCohortSchoolChildSexageTestscore
StringStringStringStringFloat64StringFloat64
12013S100067C002352male7.99452S20_r5.26316
22013S100067C002352male7.99452BPT3.7
32013S100067C002352male7.99452SLJ125.0
42013S100067C002352male7.99452Star_r2.47146
52013S100067C002352male7.99452Run1053.0
62013S100067C002353male7.99452S20_r5.0
72013S100067C002353male7.99452BPT4.1
82013S100067C002353male7.99452SLJ116.0
92013S100067C002353male7.99452Star_r1.76778
102013S100067C002353male7.99452Run1089.0
112013S100067C002354male7.99452S20_r4.54545
122013S100067C002354male7.99452BPT3.9
132013S100067C002354male7.99452SLJ111.0
⋮⋮⋮⋮⋮⋮⋮⋮
5251152018S401470C117964male9.10609Star_r1.63704
5251162018S401470C117964male9.10609Run864.0
5251172018S401470C117965female9.10609S20_r4.65116
5251182018S401470C117965female9.10609BPT3.8
5251192018S401470C117965female9.10609SLJ123.0
5251202018S401470C117965female9.10609Star_r1.52889
5251212018S401470C117965female9.10609Run1080.0
5251222018S800200C117966male9.10609S20_r4.54545
5251232018S800200C117966male9.10609BPT3.8
5251242018S800200C117966male9.10609SLJ100.0
5251252018S800200C117966male9.10609Star_r2.18506
5251262018S800200C117966male9.10609Run990.0
+
+
+
+
+
+

4 Avoiding needless repetition

+

One of the principles of relational database design is that information should not be repeated needlessly. Each row of df is determined by a combination of Child and Test, together producing a score, which can be converted to a zScore.

+

The other columns in the table, Cohort, School, age, and Sex, are properties of the Child.

+

Storing these values redundantly in the full table takes up space but, more importantly, allows for inconsistency. As it stands, a given Child could be recorded as being in one Cohort for the Run test and in another Cohort for the S20_r test and nothing about the table would detect this as being an error.

+

The approach used in relational databases is to store the information for score in one table that contains only Child, Test and score, store the information for the Child in another table including Cohort, School, age and Sex. These tables can then be combined to create the table to be used for analysis by joining the different tables together.

+

The maintainers of the DataFrames package have put in a lot of work over the past few years to make joins quite efficient in Julia. Thus the processing penalty of reassembling the big table from three smaller tables is minimal.

+

It is important to note that the main advantage of using smaller tables that are joined together to produce the analysis table is the fact that the information in the analysis table is consistent by design.

+
+
+

5 Creating the smaller table

+
+
Child = unique(select(df, :Child, :School, :Cohort, :Sex, :age))
+
+
108295×5 DataFrame
108270 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowChildSchoolCohortSexage
StringStringStringStringFloat64
1C002352S1000672013male7.99452
2C002353S1000672013male7.99452
3C002354S1000672013male7.99452
4C002355S1001222013female7.99452
5C002356S1001462013male7.99452
6C002357S1001462013male7.99452
7C002358S1001462013male7.99452
8C002359S1001832013female7.99452
9C002360S1001952013female7.99452
10C002361S1002132013male7.99452
11C002362S1002372013female7.99452
12C002363S1002372013female7.99452
13C002364S1002502013female7.99452
⋮⋮⋮⋮⋮⋮
108284C117943S1305392018female9.10609
108285C117944S1305392018male9.10609
108286C117945S1305392018male9.10609
108287C117946S1305392018male9.10609
108288C117956S4005802018male9.10609
108289C117957S4009192018male9.10609
108290C117958S4009192018male9.10609
108291C117959S4009192018male9.10609
108292C117962S4012502018female9.10609
108293C117964S4014702018male9.10609
108294C117965S4014702018female9.10609
108295C117966S8002002018male9.10609
+
+
+
+
+
length(unique(Child.Child))  # should be 108295
+
+
108295
+
+
+
+
filesize(
+  Arrow.write("./data/fggk21_Child.arrow", Child; compress=:lz4)
+)
+
+
1774946
+
+
+
+
filesize(
+  Arrow.write(
+    "./data/fggk21_Score.arrow",
+    select(df, :Child, :Test, :score);
+    compress=:lz4,
+  ),
+)
+
+
2794058
+
+
+
+
+
+ +
+
+Note +
+
+
+

A careful examination of the file sizes versus that of ./data/fggk21.arrow will show that the separate tables combined take up more space than the original because of the compression. Compression algorithms are often more successful when applied to larger files.

+
+
+

Now read the Arrow tables in and reassemble the original table.

+
+
Score = DataFrame(Arrow.Table("./data/fggk21_Score.arrow"))
+
+
525126×3 DataFrame
525101 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowChildTestscore
StringStringFloat64
1C002352S20_r5.26316
2C002352BPT3.7
3C002352SLJ125.0
4C002352Star_r2.47146
5C002352Run1053.0
6C002353S20_r5.0
7C002353BPT4.1
8C002353SLJ116.0
9C002353Star_r1.76778
10C002353Run1089.0
11C002354S20_r4.54545
12C002354BPT3.9
13C002354SLJ111.0
⋮⋮⋮⋮
525115C117964Star_r1.63704
525116C117964Run864.0
525117C117965S20_r4.65116
525118C117965BPT3.8
525119C117965SLJ123.0
525120C117965Star_r1.52889
525121C117965Run1080.0
525122C117966S20_r4.54545
525123C117966BPT3.8
525124C117966SLJ100.0
525125C117966Star_r2.18506
525126C117966Run990.0
+
+
+
+

At this point we can create the z-score column by standardizing the scores for each Test. The code to do this follows Julius’s presentation on Monday.

+
+
@transform!(groupby(Score, :Test), :zScore = @bycol zscore(:score))
+
+
525126×4 DataFrame
525101 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowChildTestscorezScore
StringStringFloat64Float64
1C002352S20_r5.263161.7913
2C002352BPT3.7-0.0622317
3C002352SLJ125.0-0.0336567
4C002352Star_r2.471461.46874
5C002352Run1053.00.331058
6C002353S20_r5.01.15471
7C002353BPT4.10.498354
8C002353SLJ116.0-0.498822
9C002353Star_r1.76778-0.9773
10C002353Run1089.00.574056
11C002354S20_r4.545450.0551481
12C002354BPT3.90.218061
13C002354SLJ111.0-0.757248
⋮⋮⋮⋮⋮
525115C117964Star_r1.63704-1.43175
525116C117964Run864.0-0.944681
525117C117965S20_r4.651160.31086
525118C117965BPT3.80.0779146
525119C117965SLJ123.0-0.137027
525120C117965Star_r1.52889-1.8077
525121C117965Run1080.00.513306
525122C117966S20_r4.545450.0551481
525123C117966BPT3.80.0779146
525124C117966SLJ100.0-1.32578
525125C117966Star_r2.185060.473217
525126C117966Run990.0-0.0941883
+
+
+
+
+
Child = DataFrame(Arrow.Table("./data/fggk21_Child.arrow"))
+
+
108295×5 DataFrame
108270 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowChildSchoolCohortSexage
StringStringStringStringFloat64
1C002352S1000672013male7.99452
2C002353S1000672013male7.99452
3C002354S1000672013male7.99452
4C002355S1001222013female7.99452
5C002356S1001462013male7.99452
6C002357S1001462013male7.99452
7C002358S1001462013male7.99452
8C002359S1001832013female7.99452
9C002360S1001952013female7.99452
10C002361S1002132013male7.99452
11C002362S1002372013female7.99452
12C002363S1002372013female7.99452
13C002364S1002502013female7.99452
⋮⋮⋮⋮⋮⋮
108284C117943S1305392018female9.10609
108285C117944S1305392018male9.10609
108286C117945S1305392018male9.10609
108287C117946S1305392018male9.10609
108288C117956S4005802018male9.10609
108289C117957S4009192018male9.10609
108290C117958S4009192018male9.10609
108291C117959S4009192018male9.10609
108292C117962S4012502018female9.10609
108293C117964S4014702018male9.10609
108294C117965S4014702018female9.10609
108295C117966S8002002018male9.10609
+
+
+
+
+
df1 = disallowmissing!(leftjoin(Score, Child; on=:Child))
+
+
525126×8 DataFrame
525101 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowChildTestscorezScoreSchoolCohortSexage
StringStringFloat64Float64StringStringStringFloat64
1C002352S20_r5.263161.7913S1000672013male7.99452
2C002352BPT3.7-0.0622317S1000672013male7.99452
3C002352SLJ125.0-0.0336567S1000672013male7.99452
4C002352Star_r2.471461.46874S1000672013male7.99452
5C002352Run1053.00.331058S1000672013male7.99452
6C002353S20_r5.01.15471S1000672013male7.99452
7C002353BPT4.10.498354S1000672013male7.99452
8C002353SLJ116.0-0.498822S1000672013male7.99452
9C002353Star_r1.76778-0.9773S1000672013male7.99452
10C002353Run1089.00.574056S1000672013male7.99452
11C002354S20_r4.545450.0551481S1000672013male7.99452
12C002354BPT3.90.218061S1000672013male7.99452
13C002354SLJ111.0-0.757248S1000672013male7.99452
⋮⋮⋮⋮⋮⋮⋮⋮⋮
525115C117964Star_r1.63704-1.43175S4014702018male9.10609
525116C117964Run864.0-0.944681S4014702018male9.10609
525117C117965S20_r4.651160.31086S4014702018female9.10609
525118C117965BPT3.80.0779146S4014702018female9.10609
525119C117965SLJ123.0-0.137027S4014702018female9.10609
525120C117965Star_r1.52889-1.8077S4014702018female9.10609
525121C117965Run1080.00.513306S4014702018female9.10609
525122C117966S20_r4.545450.0551481S8002002018male9.10609
525123C117966BPT3.80.0779146S8002002018male9.10609
525124C117966SLJ100.0-1.32578S8002002018male9.10609
525125C117966Star_r2.185060.473217S8002002018male9.10609
525126C117966Run990.0-0.0941883S8002002018male9.10609
+
+
+
+
+
+
+ +
+
+Note +
+
+
+

The call to disallowmissing! is because the join will create columns that allow for missing values but we know that we should not get missing values in the result. This call will fail if, for some reason, missing values were created.

+
+
+
+
+

6 Discovering patterns in the data

+

One of the motivations for creating the Child table was to be able to bin the ages according to the age of each child, not the age of each Child-Test combination. Not all children have all 5 test results. We can check the number of results by grouping on :Child and evaluate the number of rows in each group.

+
+
nobsChild = combine(groupby(Score, :Child), nrow => :ntest)
+
+
108295×2 DataFrame
108270 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowChildntest
StringInt64
1C0023525
2C0023535
3C0023545
4C0023555
5C0023565
6C0023575
7C0023585
8C0023594
9C0023605
10C0023614
11C0023625
12C0023635
13C0023645
⋮⋮⋮
108284C1179435
108285C1179445
108286C1179455
108287C1179465
108288C1179565
108289C1179574
108290C1179585
108291C1179595
108292C1179625
108293C1179645
108294C1179655
108295C1179665
+
+
+
+

Now create a table of the number of children with 1, 2, …, 5 test scores.

+
+
combine(groupby(nobsChild, :ntest), nrow)
+
+
5×2 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowntestnrow
Int64Int64
11462
22729
331739
448836
5596529
+
+
+
+

A natural question at this point is whether there is something about those students who have few observations. For example, are they from only a few schools?

+

One approach to examining properties like is to add the number of observations for each child to the :Child table. Later we can group the table according to this :ntest to look at properties of :Child by :ntest.

+
+
gdf = groupby(
+  disallowmissing!(leftjoin(Child, nobsChild; on=:Child)), :ntest
+)
+
+

GroupedDataFrame with 5 groups based on key: ntest

First Group (462 rows): ntest = 1
437 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowChildSchoolCohortSexagentest
StringStringStringStringFloat64Int64
1C002452S1011752013male7.994521
2C002625S1033292013male7.994521
3C002754S1048142013female7.994521
4C003269S1022582012female7.997261
5C003599S1058432012female7.997261
6C003807S1007542011male8.01
7C003985S1029452011male8.01
8C004086S1042552011male8.01
9C004657S1014002014male8.038331
10C005036S1059092014male8.038331
11C005440S1010232019male8.052021
12C005523S1018252019female8.052021
13C005697S1036152019male8.052021
⋮⋮⋮⋮⋮⋮⋮
451C112638S1037182015female9.04861
452C114749S1129382017male9.065021
453C115460S1019532015male9.081451
454C115569S1005722017male9.084191
455C115587S1007542017female9.084191
456C117108S1031962018female9.106091
457C117229S1036152018male9.106091
458C117230S1036152018male9.106091
459C117419S1049542018female9.106091
460C117437S1050042018male9.106091
461C117539S1056362018male9.106091
462C117659S1064832018female9.106091
+

⋮

Last Group (96529 rows): ntest = 5
96504 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowChildSchoolCohortSexagentest
StringStringStringStringFloat64Int64
1C002352S1000672013male7.994525
2C002353S1000672013male7.994525
3C002354S1000672013male7.994525
4C002355S1001222013female7.994525
5C002356S1001462013male7.994525
6C002357S1001462013male7.994525
7C002358S1001462013male7.994525
8C002360S1001952013female7.994525
9C002362S1002372013female7.994525
10C002363S1002372013female7.994525
11C002364S1002502013female7.994525
12C002365S1003042013male7.994525
13C002366S1003042013male7.994525
⋮⋮⋮⋮⋮⋮⋮
96518C117942S1305392018male9.106095
96519C117943S1305392018female9.106095
96520C117944S1305392018male9.106095
96521C117945S1305392018male9.106095
96522C117946S1305392018male9.106095
96523C117956S4005802018male9.106095
96524C117958S4009192018male9.106095
96525C117959S4009192018male9.106095
96526C117962S4012502018female9.106095
96527C117964S4014702018male9.106095
96528C117965S4014702018female9.106095
96529C117966S8002002018male9.106095
+
+
+
+

Are the sexes represented more-or-less equally?

+
+
combine(groupby(first(gdf), :Sex), nrow => :nchild)
+
+
2×2 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + +
RowSexnchild
StringInt64
1male230
2female232
+
+
+
+
+
combine(groupby(last(gdf), :Sex), nrow => :nchild)
+
+
2×2 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + +
RowSexnchild
StringInt64
1male47552
2female48977
+
+
+
+
+
+

7 Reading Arrow files in other languages

+

There are Arrow implementations for R (the arrow package) and for Python (pyarrow).

+
#| eval: false
+import pyarrow.feather: read_table
+read_table("./data/fggk21.arrow")
+
#| eval: false
+library("arrow")
+fggk21 <- read_feather("./data/fggk21.arrow")
+nrow(fggk21)
+
+
+

8 References

+
+
+Fühner, T., Granacher, U., Golle, K., & Kliegl, R. (2021). Age and sex effects in physical fitness components of 108,295 third graders including 515 primary schools and 9 cohorts. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-97000-4 +
+
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/bootstrap.html b/bootstrap.html new file mode 100644 index 0000000..2988ad9 --- /dev/null +++ b/bootstrap.html @@ -0,0 +1,1908 @@ + + + + + + + + + + + +Parametric bootstrap for mixed-effects models – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Parametric bootstrap for mixed-effects models

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

The speed of MixedModels.jl relative to its predecessors makes the parametric bootstrap much more computationally tractable. This is valuable because the parametric bootstrap can be used to produce more accurate confidence intervals than methods based on standard errors or profiling of the likelihood surface.

+

This page is adapted from the MixedModels.jl docs

+
+

1 The parametric bootstrap

+

Bootstrapping is a family of procedures for generating sample values of a statistic, allowing for visualization of the distribution of the statistic or for inference from this sample of values. Bootstrapping also belongs to a larger family of procedures called resampling, which are based on creating new samples of data from an existing one, then computing statistics on the new samples, in order to examine the distribution of the relevant statistics.

+

A parametric bootstrap is used with a parametric model, m, that has been fit to data. The procedure is to simulate n response vectors from m using the estimated parameter values and refit m to these responses in turn, accumulating the statistics of interest at each iteration.

+

The parameters of a LinearMixedModel object are the fixed-effects parameters, β, the standard deviation, σ, of the per-observation noise, and the covariance parameter, θ, that defines the variance-covariance matrices of the random effects. A technical description of the covariance parameter can be found in the MixedModels.jl docs. Lisa Schwetlick and Daniel Backhaus have provided a more beginner-friendly description of the covariance parameter in the documentation for MixedModelsSim.jl. For today’s purposes – looking at the uncertainty in the estimates from a fitted model – we can simply use values from the fitted model, but we will revisit the parametric bootstrap as a convenient way to simulate new data, potentially with different parameter values, for power analysis.

+

Attach the packages to be used

+
+
+Code +
using AlgebraOfGraphics
+using CairoMakie
+using DataFrames
+using MixedModels
+using MixedModelsMakie
+using Random
+using SMLP2024: dataset
+
+using AlgebraOfGraphics: AlgebraOfGraphics as AoG
+CairoMakie.activate!(; type="svg") # use SVG (other options include PNG)
+
+import ProgressMeter
+ProgressMeter.ijulia_behavior(:clear);
+
+
+

Note that the precise stream of random numbers generated for a given seed can change between Julia versions. For exact reproducibility, you either need to have the exact same Julia version or use the StableRNGs package.

+
+
+

2 A model of moderate complexity

+

The kb07 data (Kronmüller & Barr, 2007) are one of the datasets provided by the MixedModels package.

+
+
kb07 = dataset(:kb07)
+
+
Arrow.Table with 1789 rows, 7 columns, and schema:
+ :subj      String
+ :item      String
+ :spkr      String
+ :prec      String
+ :load      String
+ :rt_trunc  Int16
+ :rt_raw    Int16
+
+
+

Convert the table to a DataFrame for summary.

+
+
kb07 = DataFrame(kb07)
+describe(kb07)
+
+
7×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1subjS030S1030String
2itemI01I320String
3spkrnewold0String
4precbreakmaintain0String
5loadnoyes0String
6rt_trunc2182.25791940.051710Int16
7rt_raw2226.245791940.0159230Int16
+
+
+
+

The experimental factors; spkr, prec, and load, are two-level factors.

+
+
contrasts = Dict(:spkr => EffectsCoding(),
+                 :prec => EffectsCoding(),
+                 :load => EffectsCoding(),
+                 :subj => Grouping(),
+                 :item => Grouping())
+
+

The EffectsCoding contrast is used with these to create a ±1 encoding. Furthermore, Grouping contrasts are assigned to the subj and item factors. This is not a contrast per-se but an indication that these factors will be used as grouping factors for random effects and, therefore, there is no need to create a contrast matrix. For large numbers of levels in a grouping factor, an attempt to create a contrast matrix may cause memory overflow.

+

It is not important in these cases but a good practice in any case.

+

We can look at an initial fit of moderate complexity:

+
+
form = @formula(rt_trunc ~ 1 + spkr * prec * load +
+                          (1 + spkr + prec + load | subj) +
+                          (1 + spkr + prec + load | item))
+m0 = fit(MixedModel, form, kb07; contrasts)
+
+
Minimizing 2    Time: 0:00:00 ( 0.42  s/it)
+  objective:  29353.753287212847
+Minimizing 614    Time: 0:00:01 ( 2.82 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_subjσ_item
(Intercept)2181.675077.302928.22<1e-99301.8414362.1701
spkr: old67.749018.28243.710.000243.039640.5288
prec: maintain-333.920647.1621-7.08<1e-1162.1035246.9365
load: yes78.768119.52474.03<1e-0465.147342.1755
spkr: old & prec: maintain-21.963415.8062-1.390.1647
spkr: old & load: yes18.383815.80621.160.2448
prec: maintain & load: yes4.533315.80620.290.7743
spkr: old & prec: maintain & load: yes23.605215.80621.490.1353
Residual668.5061
+
+
+

The default display in Quarto uses the pretty MIME show method for the model and omits the estimated correlations of the random effects.

+

The VarCorr extractor displays these.

+
+
VarCorr(m0)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
subj(Intercept)91108.2428301.8414
spkr: old1852.409843.0396+0.78
prec: maintain3856.840862.1035-0.59+0.02
load: yes4244.168965.1473+0.36+0.82+0.53
item(Intercept)131167.1572362.1701
spkr: old1642.582540.5288+0.44
prec: maintain60977.6107246.9365-0.69+0.35
load: yes1778.776342.1755+0.32+0.23-0.15
Residual446900.4651668.5061
+
+
+

None of the two-factor or three-factor interaction terms in the fixed-effects are significant. In the random-effects terms only the scalar random effects and the prec random effect for item appear to be warranted, leading to the reduced formula

+
+
# formula f4 from https://doi.org/10.33016/nextjournal.100002
+form = @formula(rt_trunc ~ 1 + spkr * prec * load + (1 | subj) + (1 + prec | item))
+
+m1 = fit(MixedModel, form, kb07; contrasts)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)2181.758277.470928.16<1e-99364.7286298.1109
spkr: old67.811416.05264.22<1e-04
prec: maintain-333.858247.4629-7.03<1e-11252.6687
load: yes78.684916.05254.90<1e-06
spkr: old & prec: maintain-21.880216.0525-1.360.1729
spkr: old & load: yes18.321416.05261.140.2537
prec: maintain & load: yes4.471016.05260.280.7806
spkr: old & prec: maintain & load: yes23.521916.05251.470.1428
Residual678.9318
+
+
+
+
VarCorr(m1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)133026.916364.729
prec: maintain63841.497252.669-0.70
subj(Intercept)88870.081298.111
Residual460948.432678.932
+
+
+

These two models are nested and can be compared with a likelihood-ratio test.

+
+
MixedModels.likelihoodratiotest(m0, m1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
rt_trunc ~ 1 + spkr + prec + load + spkr & prec + spkr & load + prec & load + spkr & prec & load + (1 | subj) + (1 + prec | item)1328658
rt_trunc ~ 1 + spkr + prec + load + spkr & prec + spkr & load + prec & load + spkr & prec & load + (1 + spkr + prec + load | subj) + (1 + spkr + prec + load | item)292863721160.1655
+
+
+

The p-value of approximately 20% leads us to prefer the simpler model, m1, to the more complex, m0.

+
+
+

3 Bootstrap basics

+

To bootstrap the model parameters, first initialize a random number generator then create a bootstrap sample and extract the table of parameter estimates from it.

+
+
const RNG = MersenneTwister(42)
+samp = parametricbootstrap(RNG, 5_000, m1)
+tbl = samp.tbl
+
+
Table with 18 columns and 5000 rows:
+      obj      β1       β2       β3        β4       β5        β6        ⋯
+    ┌────────────────────────────────────────────────────────────────────
+ 1  │ 28691.9  2049.88  71.6398  -268.333  75.1566  -17.8459  -1.90968  ⋯
+ 2  │ 28642.6  2197.6   73.0832  -313.409  68.018   -18.062   30.2431   ⋯
+ 3  │ 28748.1  2175.11  85.4784  -311.786  48.8342  -27.3483  4.97874   ⋯
+ 4  │ 28683.7  2158.48  86.0498  -301.013  71.0117  -45.3329  27.3747   ⋯
+ 5  │ 28692.6  2342.6   81.9722  -305.764  78.9518  -37.0137  -5.59876  ⋯
+ 6  │ 28691.5  2198.64  90.4229  -387.855  93.4767  -37.2244  3.13351   ⋯
+ 7  │ 28687.7  2239.49  74.5619  -364.151  93.9154  -43.441   19.1572   ⋯
+ 8  │ 28673.4  2217.89  76.4077  -409.228  52.5047  -11.5777  26.4348   ⋯
+ 9  │ 28688.6  2184.89  81.3252  -298.693  89.8482  -5.99902  21.3162   ⋯
+ 10 │ 28597.6  2227.36  57.3436  -332.635  102.565  -3.85194  7.23222   ⋯
+ 11 │ 28700.8  2131.61  58.1253  -363.018  88.0576  -24.1592  2.26268   ⋯
+ 12 │ 28644.8  2145.05  94.7216  -242.292  103.158  -27.2788  27.0664   ⋯
+ 13 │ 28678.8  2113.56  55.4491  -382.425  89.8252  -12.9669  -2.74384  ⋯
+ 14 │ 28773.5  2123.38  117.383  -258.883  78.8998  -63.4576  26.0778   ⋯
+ 15 │ 28566.3  2168.89  52.8287  -289.968  71.5994  -14.7045  23.0165   ⋯
+ 16 │ 28764.1  2344.66  54.5199  -356.183  54.3082  -36.4798  32.3478   ⋯
+ 17 │ 28547.7  2230.3   56.2337  -329.448  88.8918  -27.7721  23.5031   ⋯
+ ⋮  │    ⋮        ⋮        ⋮        ⋮         ⋮        ⋮         ⋮      ⋱
+
+
+

An empirical density plot of the estimates of the residual standard deviation is obtained as

+
+
plt = data(tbl) * mapping(:σ) * AoG.density()
+draw(plt; axis=(;title="Parametric bootstrap estimates of σ"))
+
+
+
+

+
+
+
+
+

A density plot of the estimates of the standard deviation of the random effects is obtained as

+
+
plt = data(tbl) * mapping(
+  [:σ1, :σ2, :σ3] .=> "Bootstrap replicates of standard deviations";
+  color=dims(1) => renamer(["Item intercept", "Item speaker", "Subj"])
+) * AoG.density()
+draw(plt; figure=(;supertitle="Parametric bootstrap estimates of variance components"))
+
+
+
+

+
+
+
+
+

The bootstrap sample can be used to generate intervals that cover a certain percentage of the bootstrapped values. We refer to these as “coverage intervals”, similar to a confidence interval. The shortest such intervals, obtained with the shortestcovint extractor, correspond to a highest posterior density interval in Bayesian inference.

+

We generate these for all random and fixed effects:

+
+
confint(samp)
+
+
DictTable with 2 columns and 13 rows:
+ par   lower      upper
+ ────┬─────────────────────
+ β1  │ 2032.48    2340.42
+ β2  │ 34.3982    97.8079
+ β3  │ -428.09    -244.474
+ β4  │ 48.642     110.179
+ β5  │ -53.4336   8.42791
+ β6  │ -11.3574   51.5299
+ β7  │ -27.1799   36.3332
+ β8  │ -9.28905   53.5917
+ ρ1  │ -0.912786  -0.471123
+ σ   │ 654.965    700.928
+ σ1  │ 265.276    456.967
+ σ2  │ 179.402    321.013
+ σ3  │ 232.096    359.745
+
+
+
+
draw(
+  data(samp.β) * mapping(:β; color=:coefname) * AoG.density();
+  figure=(; resolution=(800, 450)),
+)
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+

+
+
+
+
+

For the fixed effects, MixedModelsMakie provides a convenience interface to plot the combined coverage intervals and density plots

+
+
ridgeplot(samp)
+
+
+
+

+
+
+
+
+

Often the intercept will be on a different scale and potentially less interesting, so we can stop it from being included in the plot:

+
+
ridgeplot(samp; show_intercept=false)
+
+
+
+

+
+
+
+
+
+
+

4 Singularity

+

Let’s consider the classic dysetuff dataset:

+
+
dyestuff = dataset(:dyestuff)
+mdye = fit(MixedModel, @formula(yield ~ 1 + (1 | batch)), dyestuff)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_batch
(Intercept)1527.500017.694686.33<1e-9937.2603
Residual49.5101
+
+
+
+
sampdye = parametricbootstrap(MersenneTwister(1234321), 10_000, mdye)
+tbldye = sampdye.tbl
+
+
┌ Warning: NLopt was roundoff limited
+└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/optsummary.jl:160
+┌ Warning: NLopt was roundoff limited
+└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/optsummary.jl:160
+┌ Warning: NLopt was roundoff limited
+└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/optsummary.jl:160
+┌ Warning: NLopt was roundoff limited
+└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/optsummary.jl:160
+┌ Warning: NLopt was roundoff limited
+└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/optsummary.jl:160
+
+
+
Table with 5 columns and 10000 rows:
+      obj      β1       σ        σ1        θ1
+    ┌────────────────────────────────────────────────
+ 1  │ 339.022  1509.13  67.4315  14.312    0.212245
+ 2  │ 322.689  1538.08  47.9831  25.5673   0.53284
+ 3  │ 324.002  1508.02  50.1346  21.7622   0.434076
+ 4  │ 331.887  1538.47  53.2238  41.0559   0.771383
+ 5  │ 317.771  1520.62  45.2975  19.1802   0.423428
+ 6  │ 315.181  1536.94  36.7556  49.1832   1.33812
+ 7  │ 333.641  1519.88  53.8161  46.712    0.867993
+ 8  │ 325.729  1528.43  47.8989  37.6367   0.785752
+ 9  │ 311.601  1497.46  41.4     15.1257   0.365355
+ 10 │ 335.244  1532.65  64.616   0.0       0.0
+ 11 │ 327.935  1552.54  57.2036  0.485275  0.00848329
+ 12 │ 323.861  1519.28  49.355   24.3703   0.493776
+ 13 │ 332.736  1509.04  59.6272  18.2905   0.306747
+ 14 │ 328.243  1531.7   51.5431  32.4743   0.630042
+ 15 │ 336.186  1536.17  64.0205  15.243    0.238096
+ 16 │ 329.468  1526.42  58.6856  0.0       0.0
+ 17 │ 320.086  1517.67  43.218   35.9663   0.832207
+ ⋮  │    ⋮        ⋮        ⋮        ⋮          ⋮
+
+
+
+
plt = data(tbldye) * mapping(:σ1) * AoG.density()
+draw(plt; axis=(;title="Parametric bootstrap estimates of σ_batch"))
+
+
+
+

+
+
+
+
+

Notice that this density plot has a spike, or mode, at zero. Although this mode appears to be diffuse, this is an artifact of the way that density plots are created. In fact, it is a pulse, as can be seen from a histogram.

+
+
plt = data(tbldye) * mapping(:σ1) * AoG.histogram(;bins=100)
+draw(plt; axis=(;title="Parametric bootstrap estimates of σ_batch"))
+
+
+
+

+
+
+
+
+

A value of zero for the standard deviation of the random effects is an example of a singular covariance. It is easy to detect the singularity in the case of a scalar random-effects term. However, it is not as straightforward to detect singularity in vector-valued random-effects terms.

+

For example, if we bootstrap a model fit to the sleepstudy data

+
+
sleepstudy = dataset(:sleepstudy)
+msleep = fit(MixedModel, @formula(reaction ~ 1 + days + (1 + days | subj)),
+             sleepstudy)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_subj
(Intercept)251.40516.632337.91<1e-9923.7805
days10.46731.50226.97<1e-115.7168
Residual25.5918
+
+
+
+
sampsleep = parametricbootstrap(MersenneTwister(666), 10_000, msleep)
+tblsleep = sampsleep.tbl
+
+
Table with 10 columns and 10000 rows:
+      obj      β1       β2       σ        σ1       σ2       ρ1         ⋯
+    ┌───────────────────────────────────────────────────────────────────
+ 1  │ 1721.95  252.488  11.0328  22.4544  29.6185  6.33343  0.233383   ⋯
+ 2  │ 1760.85  260.763  8.55352  27.3833  20.8074  4.32877  0.914412   ⋯
+ 3  │ 1750.88  246.709  12.4613  25.9951  15.8702  6.33404  0.200358   ⋯
+ 4  │ 1777.33  247.683  12.9824  27.7966  27.5413  4.9878   0.121411   ⋯
+ 5  │ 1738.05  245.649  10.5792  25.3596  21.5208  4.26131  0.052677   ⋯
+ 6  │ 1751.25  255.669  10.1984  26.1432  22.5389  4.58209  0.225968   ⋯
+ 7  │ 1727.51  248.986  7.62095  24.6451  19.0858  4.34881  0.212916   ⋯
+ 8  │ 1754.18  246.075  11.0469  26.9407  19.8341  4.55961  -0.202146  ⋯
+ 9  │ 1757.47  245.407  13.7475  25.8265  20.0014  7.7647   -0.266385  ⋯
+ 10 │ 1752.8   253.911  11.4977  25.7077  20.6409  6.27298  0.171494   ⋯
+ 11 │ 1707.8   248.887  10.1608  23.9684  10.5923  4.32044  1.0        ⋯
+ 12 │ 1773.69  252.542  10.7379  26.8795  27.7956  6.20553  0.156472   ⋯
+ 13 │ 1761.27  254.712  11.0373  25.7998  23.2005  7.30831  0.368175   ⋯
+ 14 │ 1737.0   260.299  10.5659  24.6504  29.0113  4.26877  -0.078572  ⋯
+ 15 │ 1760.12  258.949  10.1464  27.2088  8.03014  7.01905  0.72704    ⋯
+ 16 │ 1723.7   249.204  11.7868  24.9861  18.6887  3.08433  0.633218   ⋯
+ 17 │ 1734.14  262.586  8.96611  24.001   26.7969  5.376    0.297087   ⋯
+ ⋮  │    ⋮        ⋮        ⋮        ⋮        ⋮        ⋮         ⋮      ⋱
+
+
+

the singularity can be exhibited as a standard deviation of zero or as a correlation of ±1.

+
+
confint(sampsleep)
+
+
DictTable with 2 columns and 6 rows:
+ par   lower      upper
+ ────┬───────────────────
+ β1  │ 237.905    264.231
+ β2  │ 7.44545    13.3516
+ ρ1  │ -0.409926  1.0
+ σ   │ 22.6345    28.5125
+ σ1  │ 10.6755    33.3567
+ σ2  │ 3.07053    7.82205
+
+
+

A histogram of the estimated correlations from the bootstrap sample has a spike at +1.

+
+
plt = data(tblsleep) * mapping(:ρ1) * AoG.histogram(;bins=100)
+draw(plt; axis=(;title="Parametric bootstrap samples of correlation of random effects"))
+
+
+
+

+
+
+
+
+

or, as a count,

+
+
count(tblsleep.ρ1  .≈ 1)
+
+
292
+
+
+

Close examination of the histogram shows a few values of -1.

+
+
count(tblsleep.ρ1 .≈ -1)
+
+
2
+
+
+

Furthermore there are even a few cases where the estimate of the standard deviation of the random effect for the intercept is zero.

+
+
count(tblsleep.σ1 .≈ 0)
+
+
6
+
+
+

There is a general condition to check for singularity of an estimated covariance matrix or matrices in a bootstrap sample. The parameter optimized in the estimation is θ, the relative covariance parameter. Some of the elements of this parameter vector must be non-negative and, when one of these components is approximately zero, one of the covariance matrices will be singular.

+

The issingular method for a MixedModel object that tests if a parameter vector θ corresponds to a boundary or singular fit.

+

This operation is encapsulated in a method for the issingular function that works on MixedModelBootstrap objects.

+
+
count(issingular(sampsleep))
+
+
300
+
+
+
+
+

5 References

+
+
+Kronmüller, E., & Barr, D. J. (2007). Perspective-free pragmatics: Broken precedents and the recovery-from-preemption hypothesis. Journal of Memory and Language, 56(3), 436–455. https://doi.org/10.1016/j.jml.2006.05.002 +
+
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/bootstrap_files/figure-html/cell-12-output-1.svg b/bootstrap_files/figure-html/cell-12-output-1.svg new file mode 100644 index 0000000..7f7ccbf --- /dev/null +++ b/bootstrap_files/figure-html/cell-12-output-1.svg @@ -0,0 +1,386 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/bootstrap_files/figure-html/cell-13-output-1.svg b/bootstrap_files/figure-html/cell-13-output-1.svg new file mode 100644 index 0000000..6838788 --- /dev/null +++ b/bootstrap_files/figure-html/cell-13-output-1.svg @@ -0,0 +1,523 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/bootstrap_files/figure-html/cell-15-output-2.svg b/bootstrap_files/figure-html/cell-15-output-2.svg new file mode 100644 index 0000000..0bc53ef --- /dev/null +++ b/bootstrap_files/figure-html/cell-15-output-2.svg @@ -0,0 +1,884 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/bootstrap_files/figure-html/cell-16-output-1.svg b/bootstrap_files/figure-html/cell-16-output-1.svg new file mode 100644 index 0000000..1a20fba --- /dev/null +++ b/bootstrap_files/figure-html/cell-16-output-1.svg @@ -0,0 +1,1214 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/bootstrap_files/figure-html/cell-17-output-1.svg b/bootstrap_files/figure-html/cell-17-output-1.svg new file mode 100644 index 0000000..be47245 --- /dev/null +++ b/bootstrap_files/figure-html/cell-17-output-1.svg @@ -0,0 +1,1139 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/bootstrap_files/figure-html/cell-20-output-1.svg b/bootstrap_files/figure-html/cell-20-output-1.svg new file mode 100644 index 0000000..3e25864 --- /dev/null +++ b/bootstrap_files/figure-html/cell-20-output-1.svg @@ -0,0 +1,367 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/bootstrap_files/figure-html/cell-21-output-1.svg b/bootstrap_files/figure-html/cell-21-output-1.svg new file mode 100644 index 0000000..c7e7d62 --- /dev/null +++ b/bootstrap_files/figure-html/cell-21-output-1.svg @@ -0,0 +1,458 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/bootstrap_files/figure-html/cell-25-output-1.svg b/bootstrap_files/figure-html/cell-25-output-1.svg new file mode 100644 index 0000000..b8505a9 --- /dev/null +++ b/bootstrap_files/figure-html/cell-25-output-1.svg @@ -0,0 +1,611 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/check_emotikon_transform.html b/check_emotikon_transform.html new file mode 100644 index 0000000..adcf305 --- /dev/null +++ b/check_emotikon_transform.html @@ -0,0 +1,1697 @@ + + + + + + + + + + + +Transformed and original metrics in Emotikon – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Transformed and original metrics in Emotikon

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

In Fühner et al. (2021) the original metric of two tasks (Star, S20) is time, but they were transformed to speed scores in the publication prior to computing z-scores. The critical result is the absence of evidence for the age x Sex x Test interaction. Is this interaction significant if we analyse all tasks in their original metric?

+

Fitting the LMM of the publication takes time, roughly 1 hour. However, if you save the model parameters (and other relevant information), you can restore the fitted model object very quickly. The notebook also illustrates this procedure.

+
+

1 Getting the packages and data

+
+
+Code +
using AlgebraOfGraphics
+using Arrow
+using CairoMakie
+using DataFrames
+using DataFrameMacros
+using MixedModels
+using MixedModelsMakie
+using RCall
+using Serialization
+using StatsBase
+
+CairoMakie.activate!(; type="svg")
+datadir = joinpath(@__DIR__, "data");
+
+
+
+

1.1 Data and figure in publication

+
+
dat = DataFrame(Arrow.Table(joinpath(datadir, "fggk21.arrow")))
+@transform!(dat, :a1 = :age - 8.5);
+select!(groupby(dat, :Test), :, :score => zscore => :zScore);
+describe(dat)
+
+
9×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1TestBPTStar_r0String
2Cohort201120190String
3SchoolS100043S8002000String
4ChildC002352C1179660String
5Sexfemalemale0String
6age8.560737.994528.558529.106090Float64
7score226.1411.141524.651161530.00Float64
8a10.0607297-0.5054760.05852160.6060920Float64
9zScore-3.91914e-13-3.15420.000310883.550780Float64
+
+
+
+
+
+

1.2 Data and figure with z-scores based on original metric

+
+
# dat_om = rcopy(R"readRDS('./data/fggk21_om.rds')");  #Don't know what the _om is
+# @transform!(dat_om, :a1 = :age - 8.5);
+# select!(groupby(dat_om, :Test), :, :score => zscore => :zScore);
+# describe(dat_om)
+
+
+
+
+

2 LMMs

+
+

2.1 Contrasts

+
+
contrasts = Dict(
+  :Test => SeqDiffCoding(),
+  :Sex => HelmertCoding(),
+  :School => Grouping(),
+  :Child => Grouping(),
+  :Cohort => Grouping(),
+);
+
+
+
+

2.2 Formula

+
+
f1 = @formula zScore ~
+  1 +
+  Test * a1 * Sex +
+  (1 + Test + a1 + Sex | School) +
+  (1 + Test | Child) +
+  zerocorr(1 + Test | Cohort);
+
+
+
+

2.3 Restore LMM m1 from publication

+
    +
  • Command for fitting LMM m1 = fit(MixedModel, f1, dat, contrasts=contr)
    • +
  • Fit statistics for LMM m1: Minimizing 5179 Time: 0 Time: 1:00:38 ( 0.70 s/it)
    • +
+
+
m1x = LinearMixedModel(f1, dat; contrasts)
+restoreoptsum!(m1x, "./fits/fggk21_m1_optsum.json")
+
+
┌ Warning: optsum was saved with an older version of MixedModels.jl: consider resaving.
+└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/serialization.jl:28
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Childσ_Schoolσ_Cohort
(Intercept)-0.03830.0108-3.560.00040.59390.20240.0157
Test: Run-0.02280.0274-0.830.40520.83840.35880.0651
Test: S20_r-0.01470.0405-0.360.71710.58250.35960.1107
Test: SLJ0.03280.03300.990.31980.41270.30270.0896
Test: Star_r0.00060.01970.030.97630.55740.36200.0313
a10.27130.008631.63<1e-990.0966
Sex: male0.20640.002486.55<1e-990.0245
Test: Run & a1-0.44640.0131-34.05<1e-99
Test: S20_r & a10.14730.011412.97<1e-37
Test: SLJ & a1-0.00680.0103-0.660.5116
Test: Star_r & a10.07610.01116.84<1e-11
Test: Run & Sex: male-0.09000.0037-24.10<1e-99
Test: S20_r & Sex: male-0.09120.0032-28.23<1e-99
Test: SLJ & Sex: male0.03300.002911.24<1e-28
Test: Star_r & Sex: male-0.07200.0032-22.65<1e-99
a1 & Sex: male0.00100.00690.140.8876
Test: Run & a1 & Sex: male-0.01540.0126-1.220.2233
Test: S20_r & a1 & Sex: male0.01290.01091.180.2380
Test: SLJ & a1 & Sex: male-0.00980.0100-0.980.3256
Test: Star_r & a1 & Sex: male0.01660.01081.540.1241
Residual0.5880
+
+
+
+
VarCorr(m1x)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Child(Intercept)0.35272940.5939103
Test: Run0.70290030.8383915+0.11
Test: S20_r0.33933560.5825252+0.19-0.53
Test: SLJ0.17029000.4126621+0.05-0.14-0.29
Test: Star_r0.31072270.5574251-0.10+0.01-0.13-0.42
School(Intercept)0.04096400.2023957
Test: Run0.12876900.3588440+0.26
Test: S20_r0.12933510.3596319+0.01-0.57
Test: SLJ0.09165220.3027411-0.13+0.01-0.53
Test: Star_r0.13105750.3620187+0.26+0.09-0.06-0.28
a10.00934120.0966499+0.48+0.25-0.15-0.01+0.12
Sex: male0.00059990.0244934+0.09+0.13-0.01+0.05-0.19+0.25
Cohort(Intercept)0.00024520.0156587
Test: Run0.00423890.0651068.
Test: S20_r0.01225350.1106954..
Test: SLJ0.00802100.0895599...
Test: Star_r0.00098280.0313498....
Residual0.34568720.5879517
+
+
+
+
+

2.4 Restore new LMM m1_om Star and S20 in original metric

+
    +
  • Command for fitting LMM m1_om = fit(MixedModel, f1, dat_om, contrasts=contr)
    • +
  • Minimizing 10502 Time: 0 Time: 2:09:40 ( 0.74 s/it)
    • +
  • Store with: julia> saveoptsum(“./fits/fggk21_m1_om_optsum.json”, m1_om)
    • +
  • Only for short-term and when desperate: julia> serialize(“./fits/m1_om.jls”, m1_om);
    • +
+
+

2.4.1 … restoreoptsum!()

+
+
m1_om = LinearMixedModel(f1, dat; contrasts=contr);
+restoreoptsum!(m1_om, "./fits/fggk21_m1_om_optsum.json");
+
+
+
+

2.4.2 … deserialize()

+
+
m1x_om = deserialize("./fits/m1_om.jls")
+
+
+
VarCorr(m1x_om)
+
+
+
+
+

2.5 Residual diagnostics for LMM m1

+

Residual plots for published LMM

+
+
#scatter(fitted(m1x), residuals(m1x)
+
+
+
#qqnorm(m1x)
+
+
+
+

2.6 Residual diagnostics for LMM m1_om

+

Residual plots for LMM with Star and Speed in original metric.

+
+
#scatter(fitted(m1_om_v2), residuals(m1_om_v2)
+
+
+
#qqnorm(m1_om_v2)
+
+
+
+Fühner, T., Granacher, U., Golle, K., & Kliegl, R. (2021). Age and sex effects in physical fitness components of 108,295 third graders including 515 primary schools and 9 cohorts. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-97000-4 +
+
+ + +
+
+ + Back to top
+ +
+ + + + + \ No newline at end of file diff --git a/contrasts_fggk21.html b/contrasts_fggk21.html new file mode 100644 index 0000000..6c5420f --- /dev/null +++ b/contrasts_fggk21.html @@ -0,0 +1,17665 @@ + + + + + + + + + + + +Mixed Models Tutorial: Contrast Coding – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Mixed Models Tutorial: Contrast Coding

+
+ + + +
+ +
+
Author
+
+

Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

This script uses a subset of data reported in Fühner et al. (2021).

+

To circumvent delays associated with model fitting we work with models that are less complex than those in the reference publication. All the data to reproduce the models in the publication are used here, too; the script requires only a few changes to specify the more complex models in the paper.

+

All children were between 6.0 and 6.99 years at legal keydate (30 September) of school enrollment, that is in their ninth year of life in the third grade. To avoid delays associated with model fitting we work with a reduced data set and less complex models than those in the reference publication. The script requires only a few changes to specify the more complex models in the paper.

+

The script is structured in three main sections:

+
    +

  1. Setup with reading and examining the data

    2. +

  2. Contrasts coding

    3. +
+ +
    +
  1. Other topics
    2. +
+ +
+

1 Setup

+
+

1.1 Packages and functions

+
+
+Code +
using AlgebraOfGraphics
+using CairoMakie
+using Chain
+using CategoricalArrays
+using DataFrames
+using DataFrameMacros
+using MixedModels
+using ProgressMeter
+using SMLP2024: dataset
+using Statistics
+using StatsBase
+
+ProgressMeter.ijulia_behavior(:clear);
+
+
+
+
+

1.2 Readme for dataset("fggk21")

+

Number of scores: 525126

+
    +

  1. Cohort: 9 levels; 2011-2019

    2. +

  2. School: 515 levels

    3. +

  3. Child: 108295 levels; all children are between 8.0 and 8.99 years old

    4. +

  4. Sex: “Girls” (n=55,086), “Boys” (n= 53,209)

    5. +

  5. age: testdate - middle of month of birthdate

    6. +

  6. Test: 5 levels

    +
      +
    • Endurance (Run): 6 minute endurance run [m]; to nearest 9m in 9x18m field
      • +
    • Coordination (Star_r): star coordination run [m/s]; 9x9m field, 4 x diagonal = 50.912 m
      • +
    • Speed(S20_r): 20-meters sprint [m/s]
      • +
    • Muscle power low (SLJ): standing long jump [cm]
      • +
    • Muscle power up (BPT): 1-kg medicine ball push test [m]
      • +
    7. +

  7. score - see units

    8. +
+
+
+

1.3 Preprocessing

+
+

1.3.1 Read data

+
+
tbl = dataset(:fggk21)
+
+
Arrow.Table with 525126 rows, 7 columns, and schema:
+ :Cohort  String
+ :School  String
+ :Child   String
+ :Sex     String
+ :age     Float64
+ :Test    String
+ :score   Float64
+
+
+
+
df = DataFrame(tbl)
+describe(df)
+
+
7×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1Cohort201120190String
2SchoolS100043S8002000String
3ChildC002352C1179660String
4Sexfemalemale0String
5age8.560737.994528.558529.106090Float64
6TestBPTStar_r0String
7score226.1411.141524.651161530.00Float64
+
+
+
+
+
+

1.3.2 Extract a stratified subsample

+

We extract a random sample of 500 children from the Sex (2) x Test (5) cells of the design. Cohort and School are random.

+
+
dat = @chain df begin
+  @transform(:Sex = :Sex == "female" ? "Girls" : "Boys")
+  @groupby(:Test, :Sex)
+  combine(x -> x[sample(1:nrow(x), 500), :])
+end
+
+
5000×7 DataFrame
4975 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowTestSexCohortSchoolChildagescore
StringStringStringStringStringFloat64Float64
1S20_rBoys2014S100833C0734388.703634.87805
2S20_rBoys2018S102982C1098039.021224.7619
3S20_rBoys2018S106173C1018518.939084.87805
4S20_rBoys2018S101023C1094899.021224.34783
5S20_rBoys2019S105016C0853178.802194.87805
6S20_rBoys2011S100195C0878878.832315.71429
7S20_rBoys2018S101138C0902878.854214.54545
8S20_rBoys2018S101369C1168669.106094.87805
9S20_rBoys2012S101424C0503448.501034.7619
10S20_rBoys2015S103597C0731958.695414.65116
11S20_rBoys2017S101000C0386468.394255.12821
12S20_rBoys2019S105910C0575838.555784.34783
13S20_rBoys2012S105983C0892628.832315.12821
⋮⋮⋮⋮⋮⋮⋮⋮
4989RunGirls2018S130606C0731528.69268963.0
4990RunGirls2014S101710C0544348.542091190.0
4991RunGirls2016S104711C0980998.9117981.0
4992RunGirls2015S100511C0945278.88159927.0
4993RunGirls2014S101503C0923208.870641099.0
4994RunGirls2019S100511C0756218.71732828.0
4995RunGirls2011S111442C0612668.58042966.0
4996RunGirls2019S105041C0380378.388771053.0
4997RunGirls2012S100547C0313858.33402990.0
4998RunGirls2013S104632C0027317.99452855.0
4999RunGirls2016S103640C0242048.271051000.0
5000RunGirls2019S102131C0954048.88433891.0
+
+
+
+
+
+

1.3.3 Transformations

+
+
transform!(dat, :age, :age => (x -> x .- 8.5) => :a1) # centered age (linear)
+select!(groupby(dat, :Test), :, :score => zscore => :zScore) # z-score
+
+
5000×9 DataFrame
4975 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowTestSexCohortSchoolChildagescorea1zScore
StringStringStringStringStringFloat64Float64Float64Float64
1S20_rBoys2014S100833C0734388.703634.878050.2036280.817771
2S20_rBoys2018S102982C1098039.021224.76190.5212180.540353
3S20_rBoys2018S106173C1018518.939084.878050.4390830.817771
4S20_rBoys2018S101023C1094899.021224.347830.521218-0.448701
5S20_rBoys2019S105016C0853178.802194.878050.302190.817771
6S20_rBoys2011S100195C0878878.832315.714290.3323072.81518
7S20_rBoys2018S101138C0902878.854214.545450.3542090.0233473
8S20_rBoys2018S101369C1168669.106094.878050.6060920.817771
9S20_rBoys2012S101424C0503448.501034.76190.001026690.540353
10S20_rBoys2015S103597C0731958.695414.651160.1954140.275838
11S20_rBoys2017S101000C0386468.394255.12821-0.1057491.41529
12S20_rBoys2019S105910C0575838.555784.347830.0557837-0.448701
13S20_rBoys2012S105983C0892628.832315.128210.3323071.41529
⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮
4989RunGirls2018S130606C0731528.69268963.00.192676-0.333028
4990RunGirls2014S101710C0544348.542091190.00.04209451.20792
4991RunGirls2016S104711C0980998.9117981.00.411704-0.210838
4992RunGirls2015S100511C0945278.88159927.00.381588-0.577407
4993RunGirls2014S101503C0923208.870641099.00.3706370.590183
4994RunGirls2019S100511C0756218.71732828.00.217317-1.24945
4995RunGirls2011S111442C0612668.58042966.00.0804244-0.312663
4996RunGirls2019S105041C0380378.388771053.0-0.1112250.27792
4997RunGirls2012S100547C0313858.33402990.0-0.165982-0.149743
4998RunGirls2013S104632C0027317.99452855.0-0.505476-1.06617
4999RunGirls2016S103640C0242048.271051000.0-0.228953-0.0818603
5000RunGirls2019S102131C0954048.88433891.00.384326-0.821786
+
+
+
+
+
dat2 = combine(
+  groupby(dat, [:Test, :Sex]),
+  :score => mean,
+  :score => std,
+  :zScore => mean,
+  :zScore => std,
+)
+
+
10×6 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowTestSexscore_meanscore_stdzScore_meanzScore_std
StringStringFloat64Float64Float64Float64
1S20_rBoys4.601420.4225120.1570341.0092
2BPTBoys3.97560.7128960.3797160.979017
3SLJBoys128.41419.63430.2013230.993676
4Star_rBoys2.072460.3065980.09758781.06545
5RunBoys1044.39150.5080.2194871.02169
6S20_rGirls4.469940.404632-0.1570340.96649
7BPTGirls3.42260.632542-0.3797160.868667
8SLJGirls120.45819.0898-0.2013230.96612
9Star_rGirls2.016290.26495-0.09758780.920723
10RunGirls979.726136.739-0.2194870.928228
+
+
+
+
+
+

1.3.4 Figure of age x Sex x Test interactions

+

The main results of relevance here are shown in Figure 2 of Scientific Reports 11:17566.

+
+
+
+
+

2 Contrast coding

+

Contrast coding is part of StatsModels.jl. Here is the primary author’s (i.e., Dave Kleinschmidt’s) documentation of Modeling Categorical Data.

+

The random factors Child, School, and Cohort are assigned a Grouping contrast. This contrast is needed when the number of groups (i.e., units, levels) is very large. This is the case for Child (i.e., the 108,925 children in the full and probably also the 11,566 children in the reduced data set). The assignment is not necessary for the typical sample size of experiments. However, we use this coding of random factors irrespective of the number of units associated with them to be transparent about the distinction between random and fixed factors.

+

A couple of general remarks about the following examples. First, all contrasts defined in this tutorial return an estimate of the Grand Mean (GM) in the intercept, that is they are so-called sum-to-zero contrasts. In both Julia and R the default contrast is Dummy coding which is not a sum-to-zero contrast, but returns the mean of the reference (control) group - unfortunately for (quasi-)experimentally minded scientists.

+

Second, The factor Sex has only two levels. We use EffectCoding (also known as Sum coding in R) to estimate the difference of the levels from the Grand Mean. Unlike in R, the default sign of the effect is for the second level (base is the first, not the last level), but this can be changed with the base kwarg in the command. Effect coding is a sum-to-zero contrast, but when applied to factors with more than two levels does not yield orthogonal contrasts.

+

Finally, contrasts for the five levels of the fixed factor Test represent the hypotheses about differences between them. In this tutorial, we use this factor to illustrate various options.

+

We (initially) include only Test as fixed factor and Child as random factor. More complex LMMs can be specified by simply adding other fixed or random factors to the formula.

+
+

2.1 SeqDiffCoding: contr1

+

SeqDiffCoding was used in the publication. This specification tests pairwise differences between the five neighboring levels of Test, that is:

+
    +
  • SDC1: 2-1
    • +
  • SDC2: 3-2
    • +
  • SDC3: 4-3
    • +
  • SDC4: 5-4
    • +
+

The levels were sorted such that these contrasts map onto four a priori hypotheses; in other words, they are theoretically motivated pairwise comparisons. The motivation also encompasses theoretically motivated interactions with Sex. The order of levels can also be explicitly specified during contrast construction. This is very useful if levels are in a different order in the dataframe. We recommend the explicit specification to increase transparency of the code.

+

The statistical disadvantage of SeqDiffCoding is that the contrasts are not orthogonal, that is the contrasts are correlated. This is obvious from the fact that levels 2, 3, and 4 are all used in two contrasts. One consequence of this is that correlation parameters estimated between neighboring contrasts (e.g., 2-1 and 3-2) are difficult to interpret. Usually, they will be negative because assuming some practical limitation on the overall range (e.g., between levels 1 and 3), a small “2-1” effect “correlates” negatively with a larger “3-2” effect for mathematical reasons.

+

Obviously, the tradeoff between theoretical motivation and statistical purity is something that must be considered carefully when planning the analysis.

+
+
contr1 = merge(
+  Dict(nm => Grouping() for nm in (:School, :Child, :Cohort)),
+  Dict(
+    :Sex => EffectsCoding(; levels=["Girls", "Boys"]),
+    :Test => SeqDiffCoding(;
+      levels=["Run", "Star_r", "S20_r", "SLJ", "BPT"]
+    ),
+  ),
+)
+
+
Dict{Symbol, StatsModels.AbstractContrasts} with 5 entries:
+  :Child  => Grouping()
+  :School => Grouping()
+  :Test   => SeqDiffCoding(["Run", "Star_r", "S20_r", "SLJ", "BPT"])
+  :Cohort => Grouping()
+  :Sex    => EffectsCoding(nothing, ["Girls", "Boys"])
+
+
+
+
f_ovi_1 = @formula zScore ~ 1 + Test + (1 | Child);
+
+
+
m_ovi_SeqDiff_1 = fit(MixedModel, f_ovi_1, dat; contrasts=contr1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)0.00010.01420.010.99450.7264
Test: Star_r-0.00150.0444-0.030.9732
Test: S20_r-0.00320.0443-0.070.9424
Test: SLJ0.00320.04440.070.9421
Test: BPT0.00370.04440.080.9334
Residual0.6867
+
+
+

In this case, any differences between tests identified by the contrasts would be spurious because each test was standardized (i.e., M=0, \(SD\)=1). The differences could also be due to an imbalance in the number of boys and girls or in the number of missing observations for each test.

+

The primary interest in this study related to interactions of the test contrasts with and age and Sex. We start with age (linear) and its interaction with the four test contrasts.

+
+
m_ovi_SeqDiff_2 = let
+  form = @formula zScore ~ 1 + Test * a1 + (1 | Child)
+  fit(MixedModel, form, dat; contrasts=contr1)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01970.0144-1.360.17240.7196
Test: Star_r-0.01810.0450-0.400.6876
Test: S20_r-0.00790.0449-0.170.8612
Test: SLJ0.01200.04490.270.7899
Test: BPT-0.02360.0448-0.530.5991
a10.34980.04947.09<1e-11
Test: Star_r & a10.29030.15431.880.0599
Test: S20_r & a10.07130.15520.460.6461
Test: SLJ & a1-0.08240.1545-0.530.5937
Test: BPT & a10.35680.15022.380.0175
Residual0.6839
+
+
+

The difference between older and younger childrend is larger for Star_r than for Run (0.2473). S20_r did not differ significantly from Star_r (-0.0377) and SLJ (-0.0113) The largest difference in developmental gain was between BPT and SLJ (0.3355).

+

Please note that standard errors of this LMM are anti-conservative because the LMM is missing a lot of information in the RES (e..g., contrast-related VCs snd CPs for Child, School, and Cohort.

+

Next we add the main effect of Sex and its interaction with the four test contrasts.

+
+
m_ovi_SeqDiff_3 = let
+  form = @formula zScore ~ 1 + Test * (a1 + Sex) + (1 | Child)
+  fit(MixedModel, form, dat; contrasts=contr1)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01750.0140-1.250.21230.6907
Test: Star_r-0.01680.0438-0.380.7023
Test: S20_r-0.00950.0438-0.220.8274
Test: SLJ0.01220.04370.280.7808
Test: BPT-0.02130.0437-0.490.6259
a10.32230.04816.70<1e-10
Sex: Boys0.20820.013815.10<1e-50
Test: Star_r & a10.28370.15051.890.0594
Test: S20_r & a10.07950.15140.520.5996
Test: SLJ & a1-0.08860.1506-0.590.5561
Test: BPT & a10.31070.14642.120.0339
Test: Star_r & Sex: Boys-0.11880.0430-2.760.0057
Test: S20_r & Sex: Boys0.05490.04291.280.2011
Test: SLJ & Sex: Boys0.04810.04301.120.2635
Test: BPT & Sex: Boys0.17400.04304.05<1e-04
Residual0.6758
+
+
+

The significant interactions with Sex reflect mostly differences related to muscle power, where the physiological constitution gives boys an advantage. The sex difference is smaller when coordination and cognition play a role – as in the Star_r test. (Caveat: SEs are estimated with an underspecified RES.)

+

The final step in this first series is to add the interactions between the three covariates. A significant interaction between any of the four Test contrasts and age (linear) x Sex was hypothesized to reflect a prepubertal signal (i.e., hormones start to rise in girls’ ninth year of life). However, this hypothesis is linked to a specific shape of the interaction: Girls would need to gain more than boys in tests of muscular power.

+
+
f_ovi = @formula zScore ~ 1 + Test * a1 * Sex + (1 | Child)
+m_ovi_SeqDiff = fit(MixedModel, f_ovi, dat; contrasts=contr1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01810.0140-1.290.19700.6929
Test: Star_r-0.01770.0439-0.400.6865
Test: S20_r-0.00860.0438-0.200.8445
Test: SLJ0.01290.04370.300.7677
Test: BPT-0.02340.0437-0.540.5918
a10.32160.04816.69<1e-10
Sex: Boys0.20510.014014.61<1e-47
Test: Star_r & a10.28780.15061.910.0560
Test: S20_r & a10.07910.15130.520.6013
Test: SLJ & a1-0.08900.1505-0.590.5542
Test: BPT & a10.31110.14642.120.0336
Test: Star_r & Sex: Boys-0.11800.0439-2.690.0071
Test: S20_r & Sex: Boys0.05650.04381.290.1972
Test: SLJ & Sex: Boys0.05220.04371.190.2327
Test: BPT & Sex: Boys0.16510.04373.780.0002
a1 & Sex: Boys0.05100.04811.060.2887
Test: Star_r & a1 & Sex: Boys-0.01450.1506-0.100.9231
Test: S20_r & a1 & Sex: Boys-0.03030.1513-0.200.8411
Test: SLJ & a1 & Sex: Boys-0.07670.1505-0.510.6104
Test: BPT & a1 & Sex: Boys0.15250.14641.040.2974
Residual0.6732
+
+
+

The results are very clear: Despite an abundance of statistical power there is no evidence for the differences between boys and girls in how much they gain in the ninth year of life in these five tests. The authors argue that, in this case, absence of evidence looks very much like evidence of absence of a hypothesized interaction.

+

In the next two sections we use different contrasts. Does this have a bearing on this result? We still ignore for now that we are looking at anti-conservative test statistics.

+
+
+

2.2 HelmertCoding: contr2

+

The second set of contrasts uses HelmertCoding. Helmert coding codes each level as the difference from the average of the lower levels. With the default order of Test levels we get the following test statistics which we describe in reverse order of appearance in model output

+
    +
  • HeC4: 5 - mean(1,2,3,4)
    • +
  • HeC3: 4 - mean(1,2,3)
    • +
  • HeC2: 3 - mean(1,2)
    • +
  • HeC1: 2 - 1
    • +
+

In the model output, HeC1 will be reported first and HeC4 last.

+

There is some justification for the HeC4 specification in a post-hoc manner because the fifth test (BPT) turned out to be different from the other four tests in that high performance is most likely not only related to physical fitness, but also to overweight/obesity, that is for a subset of children high scores on this test might be indicative of physical unfitness. A priori the SDC4 contrast 5-4 between BPT (5) and SLJ (4) was motivated because conceptually both are tests of the physical fitness component Muscular Power, BPT for upper limbs and SLJ for lower limbs, respectively.

+

One could argue that there is justification for HeC3 because Run (1), Star_r (2), and S20 (3) involve running but SLJ (4) does not. Sports scientists, however, recoil. For them it does not make much sense to average the different running tests, because they draw on completely different physiological resources; it is a variant of the old apples-and-oranges problem.

+

The justification for HeC3 is thatRun (1) and Star_r (2) draw more strongly on cardiosrespiratory Endurance than S20 (3) due to the longer duration of the runs compared to sprinting for 20 m which is a pure measure of the physical-fitness component Speed. Again, sports scientists are not very happy with this proposal.

+

Finally, HeC1 contrasts the fitness components Endurance, indicated best by Run (1), and Coordination, indicated by Star_r (2). Endurance (i.e., running for 6 minutes) is considered to be the best indicator of health-related status among the five tests because it is a rather pure measure of cardiorespiratory fitness. The Star_r test requires execution of a pre-instructed sequence of forward, sideways, and backward runs. This coordination of body movements implies a demand on working memory (i.e., remembering the order of these subruns) and executive control processes, but performats also depends on endurance. HeC1 yields a measure of Coordination “corrected” for the contribution of Endurance.

+

The statistical advantage of HelmertCoding is that the resulting contrasts are orthogonal (uncorrelated). This allows for optimal partitioning of variance and statistical power. It is also more efficient to estimate “orthogonal” than “non-orthogonal” random-effect structures.

+
+
contr2 = Dict(
+  :School => Grouping(),
+  :Child => Grouping(),
+  :Cohort => Grouping(),
+  :Sex => EffectsCoding(; levels=["Girls", "Boys"]),
+  :Test => HelmertCoding(;
+    levels=["Run", "Star_r", "S20_r", "SLJ", "BPT"],
+  ),
+);
+
+
+
m_ovi_Helmert = fit(MixedModel, f_ovi, dat; contrasts=contr2)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01810.0140-1.290.19700.6929
Test: Star_r-0.00880.0219-0.400.6865
Test: S20_r-0.00580.0126-0.460.6454
Test: SLJ0.00030.00890.040.9713
Test: BPT-0.00450.0069-0.650.5170
a10.32160.04816.69<1e-10
Sex: Boys0.20510.014014.61<1e-47
Test: Star_r & a10.14390.07531.910.0560
Test: S20_r & a10.07430.04421.680.0929
Test: SLJ & a10.01490.03020.490.6217
Test: BPT & a10.07120.02333.050.0023
Test: Star_r & Sex: Boys-0.05900.0219-2.690.0071
Test: S20_r & Sex: Boys-0.00080.0126-0.070.9472
Test: SLJ & Sex: Boys0.01260.00891.420.1567
Test: BPT & Sex: Boys0.04060.00695.85<1e-08
a1 & Sex: Boys0.05100.04811.060.2887
Test: Star_r & a1 & Sex: Boys-0.00730.0753-0.100.9231
Test: S20_r & a1 & Sex: Boys-0.01250.0442-0.280.7769
Test: SLJ & a1 & Sex: Boys-0.02540.0302-0.840.3995
Test: BPT & a1 & Sex: Boys0.01520.02330.650.5132
Residual0.6732
+
+
+

We forego a detailed discussion of the effects, but note that again none of the interactions between age x Sex with the four test contrasts was significant.

+

The default labeling of Helmert contrasts may lead to confusions with other contrasts. Therefore, we could provide our own labels:

+

labels=["c2.1", "c3.12", "c4.123", "c5.1234"]

+

Once the order of levels is memorized the proposed labelling is very transparent.

+
+
+

2.3 HypothesisCoding: contr3

+

The third set of contrasts uses HypothesisCoding. Hypothesis coding allows the user to specify their own a priori contrast matrix, subject to the mathematical constraint that the matrix has full rank. For example, sport scientists agree that the first four tests can be contrasted with BPT, because the difference is akin to a correction of overall physical fitness. However, they want to keep the pairwise comparisons for the first four tests.

+
    +
  • HyC1: BPT - mean(1,2,3,4)
    • +
  • HyC2: Star_r - Run_r
    • +
  • HyC3: Run_r - S20_r
    • +
  • HyC4: S20_r - SLJ
    • +
+
+
contr3 = Dict(
+  :School => Grouping(),
+  :Child => Grouping(),
+  :Cohort => Grouping(),
+  :Sex => EffectsCoding(; levels=["Girls", "Boys"]),
+  :Test => HypothesisCoding(
+    [
+      -1 -1 -1 -1 +4
+      -1 +1 0 0 0
+       0 -1 +1 0 0
+       0 0 -1 +1 0
+    ];
+    levels=["Run", "Star_r", "S20_r", "SLJ", "BPT"],
+    labels=["BPT-other", "Star-End", "S20-Star", "SLJ-S20"],
+  ),
+);
+
+
+
m_ovi_Hypo = fit(MixedModel, f_ovi, dat; contrasts=contr3)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01810.0140-1.290.19700.6929
Test: BPT-other-0.08990.1387-0.650.5170
Test: Star-End-0.01770.0439-0.400.6865
Test: S20-Star-0.00860.0438-0.200.8445
Test: SLJ-S200.01290.04370.300.7677
a10.32160.04816.69<1e-10
Sex: Boys0.20510.014014.61<1e-47
Test: BPT-other & a11.42310.46633.050.0023
Test: Star-End & a10.28780.15061.910.0560
Test: S20-Star & a10.07910.15130.520.6013
Test: SLJ-S20 & a1-0.08900.1505-0.590.5542
Test: BPT-other & Sex: Boys0.81200.13875.85<1e-08
Test: Star-End & Sex: Boys-0.11800.0439-2.690.0071
Test: S20-Star & Sex: Boys0.05650.04381.290.1972
Test: SLJ-S20 & Sex: Boys0.05220.04371.190.2327
a1 & Sex: Boys0.05100.04811.060.2887
Test: BPT-other & a1 & Sex: Boys0.30490.46630.650.5132
Test: Star-End & a1 & Sex: Boys-0.01450.1506-0.100.9231
Test: S20-Star & a1 & Sex: Boys-0.03030.1513-0.200.8411
Test: SLJ-S20 & a1 & Sex: Boys-0.07670.1505-0.510.6104
Residual0.6732
+
+
+

With HypothesisCoding we must generate our own labels for the contrasts. The default labeling of contrasts is usually not interpretable. Therefore, we provide our own.

+

Anyway, none of the interactions between age x Sex with the four Test contrasts was significant for these contrasts.

+
+
contr1b = Dict(
+  :School => Grouping(),
+  :Child => Grouping(),
+  :Cohort => Grouping(),
+  :Sex => EffectsCoding(; levels=["Girls", "Boys"]),
+  :Test => HypothesisCoding(
+    [
+      -1 +1 0 0 0
+      0 -1 +1 0 0
+      0 0 -1 +1 0
+      0 0 0 -1 +1
+    ];
+    levels=["Run", "Star_r", "S20_r", "SLJ", "BPT"],
+    labels=["Star-Run", "S20-Star", "SLJ-S20", "BPT-SLJ"],
+  ),
+);
+
+
+
m_ovi_SeqDiff_v2 = fit(MixedModel, f_ovi, dat; contrasts=contr1b)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01810.0140-1.290.19700.6929
Test: Star-Run-0.01770.0439-0.400.6865
Test: S20-Star-0.00860.0438-0.200.8445
Test: SLJ-S200.01290.04370.300.7677
Test: BPT-SLJ-0.02340.0437-0.540.5918
a10.32160.04816.69<1e-10
Sex: Boys0.20510.014014.61<1e-47
Test: Star-Run & a10.28780.15061.910.0560
Test: S20-Star & a10.07910.15130.520.6013
Test: SLJ-S20 & a1-0.08900.1505-0.590.5542
Test: BPT-SLJ & a10.31110.14642.120.0336
Test: Star-Run & Sex: Boys-0.11800.0439-2.690.0071
Test: S20-Star & Sex: Boys0.05650.04381.290.1972
Test: SLJ-S20 & Sex: Boys0.05220.04371.190.2327
Test: BPT-SLJ & Sex: Boys0.16510.04373.780.0002
a1 & Sex: Boys0.05100.04811.060.2887
Test: Star-Run & a1 & Sex: Boys-0.01450.1506-0.100.9231
Test: S20-Star & a1 & Sex: Boys-0.03030.1513-0.200.8411
Test: SLJ-S20 & a1 & Sex: Boys-0.07670.1505-0.510.6104
Test: BPT-SLJ & a1 & Sex: Boys0.15250.14641.040.2974
Residual0.6732
+
+
+
+
m_zcp_SeqD = let
+  form = @formula(
+    zScore ~ 1 + Test * a1 * Sex + zerocorr(1 + Test | Child)
+  )
+  fit(MixedModel, form, dat; contrasts=contr1b)
+end
+
+
Minimizing 10    Time: 0:00:00 (10.34 ms/it)
+  objective:  13854.925652136491
+Minimizing 149    Time: 0:00:01 (10.57 ms/it)
+
+Minimizing 150    Time: 0:00:01 (10.67 ms/it)
+  objective:  13796.200602278586
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01780.0141-1.270.20550.7122
Test: Star-Run-0.01430.0448-0.320.74940.0703
Test: S20-Star-0.01070.0437-0.240.80670.7214
Test: SLJ-S200.00760.04280.180.85950.4677
Test: BPT-SLJ-0.01070.0431-0.250.80330.0000
a10.32140.04816.67<1e-10
Sex: Boys0.20530.014114.61<1e-47
Test: Star-Run & a10.28990.15281.900.0578
Test: S20-Star & a10.07650.15050.510.6111
Test: SLJ-S20 & a1-0.08960.1472-0.610.5427
Test: BPT-SLJ & a10.31610.14422.190.0284
Test: Star-Run & Sex: Boys-0.11220.0448-2.510.0122
Test: S20-Star & Sex: Boys0.05310.04371.220.2239
Test: SLJ-S20 & Sex: Boys0.05010.04281.170.2417
Test: BPT-SLJ & Sex: Boys0.17300.04314.01<1e-04
a1 & Sex: Boys0.04930.04811.020.3060
Test: Star-Run & a1 & Sex: Boys-0.00770.1528-0.050.9600
Test: S20-Star & a1 & Sex: Boys-0.02560.1505-0.170.8647
Test: SLJ-S20 & a1 & Sex: Boys-0.07720.1472-0.520.5998
Test: BPT-SLJ & a1 & Sex: Boys0.16450.14421.140.2540
Residual0.4996
+
+
+
+
m_zcp_SeqD_2 = let
+  form = @formula(
+    zScore ~ 1 + Test * a1 * Sex + (0 + Test | Child)
+  )
+  fit(MixedModel, form, dat; contrasts=contr1b)
+end
+
+
Minimizing 14    Time: 0:00:00 ( 7.45 ms/it)
+  objective:  13982.580514043671
+Minimizing 26    Time: 0:00:00 ( 7.94 ms/it)
+  objective:  14030.232041216525
+Minimizing 40    Time: 0:00:00 ( 7.75 ms/it)
+  objective:  13809.467531689255
+Minimizing 54    Time: 0:00:00 ( 7.68 ms/it)
+  objective:  13808.92595938738
+Minimizing 67    Time: 0:00:00 ( 7.69 ms/it)
+  objective:  13806.975111543254
+Minimizing 80    Time: 0:00:00 ( 7.70 ms/it)
+  objective:  13802.26846900888
+Minimizing 93    Time: 0:00:00 ( 7.75 ms/it)
+  objective:  13794.493144568278
+Minimizing 106    Time: 0:00:00 ( 7.77 ms/it)
+  objective:  13788.910178096115
+Minimizing 120    Time: 0:00:00 ( 7.76 ms/it)
+  objective:  13785.10690625065
+Minimizing 134    Time: 0:00:01 ( 7.73 ms/it)
+  objective:  13772.45481229144
+Minimizing 148    Time: 0:00:01 ( 7.73 ms/it)
+  objective:  13761.769858317157
+Minimizing 162    Time: 0:00:01 ( 7.73 ms/it)
+  objective:  13749.914617499626
+Minimizing 176    Time: 0:00:01 ( 7.70 ms/it)
+  objective:  13748.901412092946
+Minimizing 190    Time: 0:00:01 ( 7.69 ms/it)
+  objective:  13745.493223607971
+Minimizing 204    Time: 0:00:01 ( 7.69 ms/it)
+  objective:  13739.913892056678
+Minimizing 218    Time: 0:00:01 ( 7.68 ms/it)
+  objective:  13738.798251254031
+Minimizing 232    Time: 0:00:01 ( 7.65 ms/it)
+  objective:  13731.696418338688
+Minimizing 246    Time: 0:00:01 ( 7.70 ms/it)
+  objective:  13725.829887995456
+Minimizing 260    Time: 0:00:02 ( 7.69 ms/it)
+  objective:  13715.992145258002
+Minimizing 273    Time: 0:00:02 ( 7.72 ms/it)
+  objective:  13713.529683296727
+Minimizing 286    Time: 0:00:02 ( 7.74 ms/it)
+  objective:  13707.94234615188
+Minimizing 299    Time: 0:00:02 ( 7.75 ms/it)
+  objective:  13707.469665698492
+Minimizing 315    Time: 0:00:02 ( 7.79 ms/it)
+  objective:  13702.526901215406
+Minimizing 330    Time: 0:00:02 ( 7.79 ms/it)
+  objective:  13701.407982621175
+Minimizing 344    Time: 0:00:02 ( 7.78 ms/it)
+  objective:  13698.046594204667
+Minimizing 357    Time: 0:00:02 ( 7.77 ms/it)
+  objective:  13687.964903666249
+Minimizing 371    Time: 0:00:02 ( 7.77 ms/it)
+  objective:  13681.454854918124
+Minimizing 384    Time: 0:00:02 ( 7.76 ms/it)
+  objective:  13675.896375570843
+Minimizing 398    Time: 0:00:03 ( 7.76 ms/it)
+  objective:  13669.884022295657
+Minimizing 412    Time: 0:00:03 ( 7.76 ms/it)
+  objective:  13666.58031997356
+Minimizing 426    Time: 0:00:03 ( 7.76 ms/it)
+  objective:  13654.153332034997
+Minimizing 440    Time: 0:00:03 ( 7.75 ms/it)
+  objective:  13647.775301708025
+Minimizing 454    Time: 0:00:03 ( 7.74 ms/it)
+  objective:  13643.014192075638
+Minimizing 468    Time: 0:00:03 ( 7.74 ms/it)
+  objective:  13638.312250214072
+Minimizing 482    Time: 0:00:03 ( 7.74 ms/it)
+  objective:  13634.764313388223
+Minimizing 496    Time: 0:00:03 ( 7.73 ms/it)
+  objective:  13629.911540988593
+Minimizing 510    Time: 0:00:03 ( 7.73 ms/it)
+  objective:  13618.510459066481
+Minimizing 523    Time: 0:00:04 ( 7.73 ms/it)
+  objective:  13617.441944210696
+Minimizing 536    Time: 0:00:04 ( 7.75 ms/it)
+  objective:  13615.944580738964
+Minimizing 549    Time: 0:00:04 ( 7.76 ms/it)
+  objective:  13614.323330292325
+Minimizing 562    Time: 0:00:04 ( 7.76 ms/it)
+  objective:  13609.210957579151
+Minimizing 575    Time: 0:00:04 ( 7.77 ms/it)
+  objective:  13608.746693222904
+Minimizing 588    Time: 0:00:04 ( 7.77 ms/it)
+  objective:  13600.905356946085
+Minimizing 601    Time: 0:00:04 ( 7.77 ms/it)
+  objective:  13600.28561465062
+Minimizing 615    Time: 0:00:04 ( 7.76 ms/it)
+  objective:  13599.201972133687
+Minimizing 629    Time: 0:00:04 ( 7.75 ms/it)
+  objective:  13597.630840084275
+Minimizing 643    Time: 0:00:04 ( 7.74 ms/it)
+  objective:  13587.61837899172
+Minimizing 658    Time: 0:00:05 ( 7.73 ms/it)
+  objective:  13586.009605659594
+Minimizing 672    Time: 0:00:05 ( 7.72 ms/it)
+  objective:  13584.544374166664
+Minimizing 686    Time: 0:00:05 ( 7.71 ms/it)
+  objective:  13583.630001859437
+Minimizing 700    Time: 0:00:05 ( 7.70 ms/it)
+  objective:  13582.914338085047
+Minimizing 714    Time: 0:00:05 ( 7.70 ms/it)
+  objective:  13578.729620405957
+Minimizing 728    Time: 0:00:05 ( 7.69 ms/it)
+  objective:  13576.410341664843
+Minimizing 743    Time: 0:00:05 ( 7.68 ms/it)
+  objective:  13575.535008026138
+Minimizing 758    Time: 0:00:05 ( 7.66 ms/it)
+  objective:  13569.203202734127
+Minimizing 773    Time: 0:00:05 ( 7.65 ms/it)
+  objective:  13566.083881071609
+Minimizing 788    Time: 0:00:06 ( 7.64 ms/it)
+  objective:  13562.327394175594
+Minimizing 803    Time: 0:00:06 ( 7.63 ms/it)
+  objective:  13559.755657492522
+Minimizing 818    Time: 0:00:06 ( 7.62 ms/it)
+  objective:  13550.265769897971
+Minimizing 832    Time: 0:00:06 ( 7.62 ms/it)
+  objective:  13549.204684198947
+Minimizing 846    Time: 0:00:06 ( 7.62 ms/it)
+  objective:  13543.78672801541
+Minimizing 860    Time: 0:00:06 ( 7.62 ms/it)
+  objective:  13540.083343502723
+Minimizing 874    Time: 0:00:06 ( 7.62 ms/it)
+  objective:  13532.954282739014
+Minimizing 888    Time: 0:00:06 ( 7.62 ms/it)
+  objective:  13518.578757075324
+Minimizing 903    Time: 0:00:06 ( 7.61 ms/it)
+  objective:  13516.583189945406
+Minimizing 918    Time: 0:00:06 ( 7.60 ms/it)
+  objective:  13511.137765243213
+Minimizing 933    Time: 0:00:07 ( 7.59 ms/it)
+  objective:  13508.982220839549
+Minimizing 948    Time: 0:00:07 ( 7.59 ms/it)
+  objective:  13508.106646485307
+Minimizing 963    Time: 0:00:07 ( 7.58 ms/it)
+  objective:  13507.38564326617
+Minimizing 978    Time: 0:00:07 ( 7.57 ms/it)
+  objective:  13506.100735023087
+Minimizing 992    Time: 0:00:07 ( 7.57 ms/it)
+  objective:  13505.848916881616
+Minimizing 1007    Time: 0:00:07 ( 7.56 ms/it)
+  objective:  13503.662529956928
+Minimizing 1022    Time: 0:00:07 ( 7.55 ms/it)
+  objective:  13503.198231182243
+Minimizing 1037    Time: 0:00:07 ( 7.54 ms/it)
+  objective:  13499.10908455214
+Minimizing 1052    Time: 0:00:07 ( 7.53 ms/it)
+  objective:  13498.270363498952
+Minimizing 1067    Time: 0:00:08 ( 7.52 ms/it)
+  objective:  13494.61716876943
+Minimizing 1082    Time: 0:00:08 ( 7.52 ms/it)
+  objective:  13494.426303182518
+Minimizing 1097    Time: 0:00:08 ( 7.51 ms/it)
+  objective:  13493.949820444395
+Minimizing 1112    Time: 0:00:08 ( 7.51 ms/it)
+  objective:  13492.876060336108
+Minimizing 1126    Time: 0:00:08 ( 7.50 ms/it)
+  objective:  13492.319088075921
+Minimizing 1140    Time: 0:00:08 ( 7.50 ms/it)
+  objective:  13492.043682367279
+Minimizing 1154    Time: 0:00:08 ( 7.50 ms/it)
+  objective:  13488.238151284473
+Minimizing 1169    Time: 0:00:08 ( 7.49 ms/it)
+  objective:  13487.95914316905
+Minimizing 1184    Time: 0:00:08 ( 7.49 ms/it)
+  objective:  13485.29773348049
+Minimizing 1199    Time: 0:00:08 ( 7.49 ms/it)
+  objective:  13484.964179269911
+Minimizing 1214    Time: 0:00:09 ( 7.48 ms/it)
+  objective:  13484.472892658916
+Minimizing 1228    Time: 0:00:09 ( 7.48 ms/it)
+  objective:  13484.206739258865
+Minimizing 1242    Time: 0:00:09 ( 7.48 ms/it)
+  objective:  13483.872450555144
+Minimizing 1256    Time: 0:00:09 ( 7.48 ms/it)
+  objective:  13483.427564764825
+Minimizing 1270    Time: 0:00:09 ( 7.48 ms/it)
+  objective:  13482.660347165816
+Minimizing 1285    Time: 0:00:09 ( 7.47 ms/it)
+  objective:  13480.934818693167
+Minimizing 1300    Time: 0:00:09 ( 7.47 ms/it)
+  objective:  13480.415742812082
+Minimizing 1315    Time: 0:00:09 ( 7.46 ms/it)
+  objective:  13478.599813143883
+Minimizing 1330    Time: 0:00:09 ( 7.46 ms/it)
+  objective:  13478.152315558713
+Minimizing 1345    Time: 0:00:10 ( 7.46 ms/it)
+  objective:  13477.37579554788
+Minimizing 1360    Time: 0:00:10 ( 7.45 ms/it)
+  objective:  13476.929710265875
+Minimizing 1375    Time: 0:00:10 ( 7.45 ms/it)
+  objective:  13476.701806857083
+Minimizing 1390    Time: 0:00:10 ( 7.44 ms/it)
+  objective:  13475.606446069483
+Minimizing 1405    Time: 0:00:10 ( 7.44 ms/it)
+  objective:  13474.631828306214
+Minimizing 1420    Time: 0:00:10 ( 7.44 ms/it)
+  objective:  13473.865552192306
+Minimizing 1435    Time: 0:00:10 ( 7.43 ms/it)
+  objective:  13472.428076141157
+Minimizing 1450    Time: 0:00:10 ( 7.43 ms/it)
+  objective:  13471.530112313987
+Minimizing 1465    Time: 0:00:10 ( 7.42 ms/it)
+  objective:  13470.350915251656
+Minimizing 1480    Time: 0:00:10 ( 7.42 ms/it)
+  objective:  13470.168450050362
+Minimizing 1495    Time: 0:00:11 ( 7.41 ms/it)
+  objective:  13469.701008994685
+Minimizing 1510    Time: 0:00:11 ( 7.41 ms/it)
+  objective:  13468.923778066266
+Minimizing 1525    Time: 0:00:11 ( 7.41 ms/it)
+  objective:  13464.681036416456
+Minimizing 1540    Time: 0:00:11 ( 7.40 ms/it)
+  objective:  13464.42236593872
+Minimizing 1555    Time: 0:00:11 ( 7.40 ms/it)
+  objective:  13463.852645869476
+Minimizing 1570    Time: 0:00:11 ( 7.39 ms/it)
+  objective:  13463.362889527627
+Minimizing 1585    Time: 0:00:11 ( 7.39 ms/it)
+  objective:  13463.10204613948
+Minimizing 1600    Time: 0:00:11 ( 7.38 ms/it)
+  objective:  13462.543359438168
+Minimizing 1615    Time: 0:00:11 ( 7.38 ms/it)
+  objective:  13462.172461736023
+Minimizing 1630    Time: 0:00:12 ( 7.38 ms/it)
+  objective:  13461.449374484051
+Minimizing 1645    Time: 0:00:12 ( 7.37 ms/it)
+  objective:  13461.190844090655
+Minimizing 1660    Time: 0:00:12 ( 7.37 ms/it)
+  objective:  13458.674228341144
+Minimizing 1675    Time: 0:00:12 ( 7.37 ms/it)
+  objective:  13458.559413870433
+Minimizing 1690    Time: 0:00:12 ( 7.37 ms/it)
+  objective:  13457.809587834956
+Minimizing 1703    Time: 0:00:12 ( 7.37 ms/it)
+  objective:  13457.679375278443
+Minimizing 1717    Time: 0:00:12 ( 7.38 ms/it)
+  objective:  13456.458978047303
+Minimizing 1731    Time: 0:00:12 ( 7.38 ms/it)
+  objective:  13456.31949736413
+Minimizing 1744    Time: 0:00:12 ( 7.38 ms/it)
+  objective:  13455.712940619225
+Minimizing 1758    Time: 0:00:12 ( 7.38 ms/it)
+  objective:  13453.67528533141
+Minimizing 1772    Time: 0:00:13 ( 7.38 ms/it)
+  objective:  13450.595171898414
+Minimizing 1786    Time: 0:00:13 ( 7.38 ms/it)
+  objective:  13450.533882012489
+Minimizing 1800    Time: 0:00:13 ( 7.38 ms/it)
+  objective:  13450.346449755307
+Minimizing 1815    Time: 0:00:13 ( 7.37 ms/it)
+  objective:  13450.059858557048
+Minimizing 1830    Time: 0:00:13 ( 7.37 ms/it)
+  objective:  13449.955287495359
+Minimizing 1845    Time: 0:00:13 ( 7.37 ms/it)
+  objective:  13449.832268378392
+Minimizing 1860    Time: 0:00:13 ( 7.37 ms/it)
+  objective:  13449.482962102222
+Minimizing 1874    Time: 0:00:13 ( 7.37 ms/it)
+  objective:  13449.140915889853
+Minimizing 1888    Time: 0:00:13 ( 7.37 ms/it)
+  objective:  13448.870489174398
+Minimizing 1904    Time: 0:00:14 ( 7.37 ms/it)
+  objective:  13447.857731424949
+Minimizing 1920    Time: 0:00:14 ( 7.37 ms/it)
+  objective:  13447.409554398771
+Minimizing 1934    Time: 0:00:14 ( 7.37 ms/it)
+  objective:  13445.769244726864
+Minimizing 1948    Time: 0:00:14 ( 7.37 ms/it)
+  objective:  13444.935851343813
+Minimizing 1963    Time: 0:00:14 ( 7.36 ms/it)
+  objective:  13444.37249914655
+Minimizing 1977    Time: 0:00:14 ( 7.36 ms/it)
+  objective:  13442.32665539231
+Minimizing 1991    Time: 0:00:14 ( 7.36 ms/it)
+  objective:  13442.196883789904
+Minimizing 2005    Time: 0:00:14 ( 7.36 ms/it)
+  objective:  13441.999703874462
+Minimizing 2019    Time: 0:00:14 ( 7.36 ms/it)
+  objective:  13441.833223438814
+Minimizing 2033    Time: 0:00:14 ( 7.36 ms/it)
+  objective:  13441.773748989865
+Minimizing 2048    Time: 0:00:15 ( 7.36 ms/it)
+  objective:  13441.294223560086
+Minimizing 2063    Time: 0:00:15 ( 7.36 ms/it)
+  objective:  13441.258701306593
+Minimizing 2077    Time: 0:00:15 ( 7.36 ms/it)
+  objective:  13440.924170162842
+Minimizing 2091    Time: 0:00:15 ( 7.36 ms/it)
+  objective:  13440.8509067376
+Minimizing 2105    Time: 0:00:15 ( 7.36 ms/it)
+  objective:  13440.546523816636
+Minimizing 2120    Time: 0:00:15 ( 7.36 ms/it)
+  objective:  13440.203010858146
+Minimizing 2134    Time: 0:00:15 ( 7.35 ms/it)
+  objective:  13439.926439225594
+Minimizing 2149    Time: 0:00:15 ( 7.35 ms/it)
+  objective:  13438.816505054885
+Minimizing 2163    Time: 0:00:15 ( 7.35 ms/it)
+  objective:  13436.848866931141
+Minimizing 2177    Time: 0:00:16 ( 7.35 ms/it)
+  objective:  13436.647630267777
+Minimizing 2191    Time: 0:00:16 ( 7.35 ms/it)
+  objective:  13435.633653786492
+Minimizing 2205    Time: 0:00:16 ( 7.35 ms/it)
+  objective:  13435.44165971814
+Minimizing 2219    Time: 0:00:16 ( 7.35 ms/it)
+  objective:  13434.875721214747
+Minimizing 2233    Time: 0:00:16 ( 7.35 ms/it)
+  objective:  13434.795945264086
+Minimizing 2247    Time: 0:00:16 ( 7.35 ms/it)
+  objective:  13434.623026371628
+Minimizing 2261    Time: 0:00:16 ( 7.35 ms/it)
+  objective:  13434.420038974298
+Minimizing 2275    Time: 0:00:16 ( 7.35 ms/it)
+  objective:  13434.062963390024
+Minimizing 2289    Time: 0:00:16 ( 7.35 ms/it)
+  objective:  13433.588839592507
+Minimizing 2303    Time: 0:00:16 ( 7.35 ms/it)
+  objective:  13433.334731066388
+Minimizing 2316    Time: 0:00:17 ( 7.36 ms/it)
+  objective:  13432.535262732941
+Minimizing 2330    Time: 0:00:17 ( 7.36 ms/it)
+  objective:  13432.033842502817
+Minimizing 2343    Time: 0:00:17 ( 7.36 ms/it)
+  objective:  13431.8695811086
+Minimizing 2357    Time: 0:00:17 ( 7.36 ms/it)
+  objective:  13431.454834187258
+Minimizing 2371    Time: 0:00:17 ( 7.36 ms/it)
+  objective:  13430.793475601196
+Minimizing 2384    Time: 0:00:17 ( 7.37 ms/it)
+  objective:  13430.310440057801
+Minimizing 2397    Time: 0:00:17 ( 7.37 ms/it)
+  objective:  13430.055505730837
+Minimizing 2411    Time: 0:00:17 ( 7.37 ms/it)
+  objective:  13428.816399686963
+Minimizing 2425    Time: 0:00:17 ( 7.37 ms/it)
+  objective:  13428.590357249224
+Minimizing 2439    Time: 0:00:17 ( 7.37 ms/it)
+  objective:  13428.28594486956
+Minimizing 2453    Time: 0:00:18 ( 7.36 ms/it)
+  objective:  13427.991430509304
+Minimizing 2467    Time: 0:00:18 ( 7.36 ms/it)
+  objective:  13427.555431311215
+Minimizing 2481    Time: 0:00:18 ( 7.36 ms/it)
+  objective:  13427.129482832817
+Minimizing 2495    Time: 0:00:18 ( 7.36 ms/it)
+  objective:  13426.882725501258
+Minimizing 2509    Time: 0:00:18 ( 7.36 ms/it)
+  objective:  13426.628293662565
+Minimizing 2523    Time: 0:00:18 ( 7.36 ms/it)
+  objective:  13426.43264934884
+Minimizing 2537    Time: 0:00:18 ( 7.36 ms/it)
+  objective:  13425.966598576852
+Minimizing 2551    Time: 0:00:18 ( 7.36 ms/it)
+  objective:  13425.918839314189
+Minimizing 2565    Time: 0:00:18 ( 7.36 ms/it)
+  objective:  13425.549786421514
+Minimizing 2579    Time: 0:00:18 ( 7.36 ms/it)
+  objective:  13425.351238154704
+Minimizing 2593    Time: 0:00:19 ( 7.36 ms/it)
+  objective:  13425.20175381493
+Minimizing 2607    Time: 0:00:19 ( 7.36 ms/it)
+  objective:  13424.70773917163
+Minimizing 2621    Time: 0:00:19 ( 7.36 ms/it)
+  objective:  13424.567450979332
+Minimizing 2635    Time: 0:00:19 ( 7.36 ms/it)
+  objective:  13423.971690599428
+Minimizing 2649    Time: 0:00:19 ( 7.36 ms/it)
+  objective:  13421.990083325189
+Minimizing 2663    Time: 0:00:19 ( 7.36 ms/it)
+  objective:  13421.634796240483
+Minimizing 2677    Time: 0:00:19 ( 7.36 ms/it)
+  objective:  13420.922178233464
+Minimizing 2691    Time: 0:00:19 ( 7.36 ms/it)
+  objective:  13420.907396531918
+Minimizing 2705    Time: 0:00:19 ( 7.36 ms/it)
+  objective:  13420.664771435695
+Minimizing 2719    Time: 0:00:20 ( 7.36 ms/it)
+  objective:  13420.3551541278
+Minimizing 2733    Time: 0:00:20 ( 7.36 ms/it)
+  objective:  13420.288588365496
+Minimizing 2747    Time: 0:00:20 ( 7.36 ms/it)
+  objective:  13420.146606798058
+Minimizing 2761    Time: 0:00:20 ( 7.36 ms/it)
+  objective:  13419.13964667276
+Minimizing 2775    Time: 0:00:20 ( 7.36 ms/it)
+  objective:  13417.111267245295
+Minimizing 2789    Time: 0:00:20 ( 7.36 ms/it)
+  objective:  13417.001009317311
+Minimizing 2803    Time: 0:00:20 ( 7.36 ms/it)
+  objective:  13416.528035796677
+Minimizing 2817    Time: 0:00:20 ( 7.36 ms/it)
+  objective:  13416.130682686024
+Minimizing 2831    Time: 0:00:20 ( 7.36 ms/it)
+  objective:  13415.491576423505
+Minimizing 2845    Time: 0:00:20 ( 7.36 ms/it)
+  objective:  13415.464207226556
+Minimizing 2859    Time: 0:00:21 ( 7.36 ms/it)
+  objective:  13414.690570045917
+Minimizing 2872    Time: 0:00:21 ( 7.36 ms/it)
+  objective:  13414.590236106473
+Minimizing 2886    Time: 0:00:21 ( 7.36 ms/it)
+  objective:  13414.368719160782
+Minimizing 2900    Time: 0:00:21 ( 7.36 ms/it)
+  objective:  13413.691742222043
+Minimizing 2914    Time: 0:00:21 ( 7.37 ms/it)
+  objective:  13411.294449664012
+Minimizing 2928    Time: 0:00:21 ( 7.37 ms/it)
+  objective:  13411.011912399037
+Minimizing 2942    Time: 0:00:21 ( 7.37 ms/it)
+  objective:  13410.80828546598
+Minimizing 2956    Time: 0:00:21 ( 7.37 ms/it)
+  objective:  13410.423020476803
+Minimizing 2970    Time: 0:00:21 ( 7.37 ms/it)
+  objective:  13410.353191679213
+Minimizing 2984    Time: 0:00:21 ( 7.37 ms/it)
+  objective:  13410.266555457682
+Minimizing 2998    Time: 0:00:22 ( 7.37 ms/it)
+  objective:  13410.230006247453
+Minimizing 3012    Time: 0:00:22 ( 7.37 ms/it)
+  objective:  13410.166164442227
+Minimizing 3026    Time: 0:00:22 ( 7.37 ms/it)
+  objective:  13409.98275589857
+Minimizing 3040    Time: 0:00:22 ( 7.37 ms/it)
+  objective:  13409.864079267034
+Minimizing 3054    Time: 0:00:22 ( 7.37 ms/it)
+  objective:  13409.65827262085
+Minimizing 3068    Time: 0:00:22 ( 7.37 ms/it)
+  objective:  13409.497854791545
+Minimizing 3083    Time: 0:00:22 ( 7.37 ms/it)
+  objective:  13409.049622542632
+Minimizing 3096    Time: 0:00:22 ( 7.38 ms/it)
+  objective:  13408.989122638908
+Minimizing 3108    Time: 0:00:22 ( 7.38 ms/it)
+  objective:  13408.866659970707
+Minimizing 3120    Time: 0:00:23 ( 7.39 ms/it)
+  objective:  13408.818536220337
+Minimizing 3132    Time: 0:00:23 ( 7.39 ms/it)
+  objective:  13408.74400325802
+Minimizing 3144    Time: 0:00:23 ( 7.40 ms/it)
+  objective:  13408.59428742277
+Minimizing 3156    Time: 0:00:23 ( 7.41 ms/it)
+  objective:  13408.458097941693
+Minimizing 3168    Time: 0:00:23 ( 7.41 ms/it)
+  objective:  13408.161666971784
+Minimizing 3180    Time: 0:00:23 ( 7.42 ms/it)
+  objective:  13407.714453381945
+Minimizing 3192    Time: 0:00:23 ( 7.42 ms/it)
+  objective:  13407.532956535375
+Minimizing 3206    Time: 0:00:23 ( 7.42 ms/it)
+  objective:  13407.49938616791
+Minimizing 3221    Time: 0:00:23 ( 7.42 ms/it)
+  objective:  13407.366364307403
+Minimizing 3236    Time: 0:00:23 ( 7.42 ms/it)
+  objective:  13407.302256937379
+Minimizing 3251    Time: 0:00:24 ( 7.41 ms/it)
+  objective:  13407.178771271014
+Minimizing 3266    Time: 0:00:24 ( 7.41 ms/it)
+  objective:  13407.105039817245
+Minimizing 3281    Time: 0:00:24 ( 7.41 ms/it)
+  objective:  13406.806173895304
+Minimizing 3296    Time: 0:00:24 ( 7.41 ms/it)
+  objective:  13406.735740287513
+Minimizing 3311    Time: 0:00:24 ( 7.41 ms/it)
+  objective:  13406.380681041876
+Minimizing 3325    Time: 0:00:24 ( 7.41 ms/it)
+  objective:  13406.33203176374
+Minimizing 3339    Time: 0:00:24 ( 7.40 ms/it)
+  objective:  13406.245764444386
+Minimizing 3354    Time: 0:00:24 ( 7.40 ms/it)
+  objective:  13406.169515524845
+Minimizing 3369    Time: 0:00:24 ( 7.40 ms/it)
+  objective:  13405.978776179181
+Minimizing 3384    Time: 0:00:25 ( 7.40 ms/it)
+  objective:  13405.954334776397
+Minimizing 3399    Time: 0:00:25 ( 7.40 ms/it)
+  objective:  13405.567426425652
+Minimizing 3414    Time: 0:00:25 ( 7.39 ms/it)
+  objective:  13405.087683400248
+Minimizing 3429    Time: 0:00:25 ( 7.39 ms/it)
+  objective:  13405.015193295352
+Minimizing 3444    Time: 0:00:25 ( 7.39 ms/it)
+  objective:  13404.81997576858
+Minimizing 3459    Time: 0:00:25 ( 7.39 ms/it)
+  objective:  13404.572166834747
+Minimizing 3474    Time: 0:00:25 ( 7.39 ms/it)
+  objective:  13404.525812108724
+Minimizing 3489    Time: 0:00:25 ( 7.38 ms/it)
+  objective:  13404.250175499408
+Minimizing 3504    Time: 0:00:25 ( 7.38 ms/it)
+  objective:  13404.120031650426
+Minimizing 3519    Time: 0:00:25 ( 7.38 ms/it)
+  objective:  13404.008628592579
+Minimizing 3534    Time: 0:00:26 ( 7.38 ms/it)
+  objective:  13403.695365184198
+Minimizing 3549    Time: 0:00:26 ( 7.38 ms/it)
+  objective:  13403.451990218942
+Minimizing 3564    Time: 0:00:26 ( 7.38 ms/it)
+  objective:  13403.039052210072
+Minimizing 3578    Time: 0:00:26 ( 7.37 ms/it)
+  objective:  13402.757584794817
+Minimizing 3593    Time: 0:00:26 ( 7.37 ms/it)
+  objective:  13402.601892271661
+Minimizing 3608    Time: 0:00:26 ( 7.37 ms/it)
+  objective:  13402.22529925144
+Minimizing 3623    Time: 0:00:26 ( 7.37 ms/it)
+  objective:  13402.014501068501
+Minimizing 3638    Time: 0:00:26 ( 7.37 ms/it)
+  objective:  13401.59879583976
+Minimizing 3653    Time: 0:00:26 ( 7.37 ms/it)
+  objective:  13401.425982988963
+Minimizing 3667    Time: 0:00:27 ( 7.37 ms/it)
+  objective:  13401.106119077202
+Minimizing 3682    Time: 0:00:27 ( 7.36 ms/it)
+  objective:  13400.95897524237
+Minimizing 3697    Time: 0:00:27 ( 7.36 ms/it)
+  objective:  13400.84641682946
+Minimizing 3712    Time: 0:00:27 ( 7.36 ms/it)
+  objective:  13400.622923207833
+Minimizing 3727    Time: 0:00:27 ( 7.36 ms/it)
+  objective:  13400.531820088043
+Minimizing 3742    Time: 0:00:27 ( 7.36 ms/it)
+  objective:  13400.403974232504
+Minimizing 3757    Time: 0:00:27 ( 7.36 ms/it)
+  objective:  13400.027524652294
+Minimizing 3772    Time: 0:00:27 ( 7.36 ms/it)
+  objective:  13399.956298843907
+Minimizing 3787    Time: 0:00:27 ( 7.35 ms/it)
+  objective:  13399.795414080261
+Minimizing 3802    Time: 0:00:27 ( 7.35 ms/it)
+  objective:  13399.30786695311
+Minimizing 3817    Time: 0:00:28 ( 7.35 ms/it)
+  objective:  13399.015874677112
+Minimizing 3832    Time: 0:00:28 ( 7.35 ms/it)
+  objective:  13398.982769213508
+Minimizing 3847    Time: 0:00:28 ( 7.35 ms/it)
+  objective:  13398.751440909647
+Minimizing 3862    Time: 0:00:28 ( 7.35 ms/it)
+  objective:  13398.372395532657
+Minimizing 3877    Time: 0:00:28 ( 7.35 ms/it)
+  objective:  13398.338531781177
+Minimizing 3892    Time: 0:00:28 ( 7.34 ms/it)
+  objective:  13398.185657952337
+Minimizing 3907    Time: 0:00:28 ( 7.34 ms/it)
+  objective:  13397.906844047262
+Minimizing 3922    Time: 0:00:28 ( 7.34 ms/it)
+  objective:  13397.802749459792
+Minimizing 3937    Time: 0:00:28 ( 7.34 ms/it)
+  objective:  13397.628794346805
+Minimizing 3952    Time: 0:00:29 ( 7.34 ms/it)
+  objective:  13397.438395749821
+Minimizing 3967    Time: 0:00:29 ( 7.34 ms/it)
+  objective:  13396.853688099436
+Minimizing 3982    Time: 0:00:29 ( 7.33 ms/it)
+  objective:  13396.742808832169
+Minimizing 3997    Time: 0:00:29 ( 7.33 ms/it)
+  objective:  13396.691638782577
+Minimizing 4012    Time: 0:00:29 ( 7.33 ms/it)
+  objective:  13396.607579192801
+Minimizing 4027    Time: 0:00:29 ( 7.33 ms/it)
+  objective:  13396.434557549088
+Minimizing 4042    Time: 0:00:29 ( 7.33 ms/it)
+  objective:  13396.2596687127
+Minimizing 4057    Time: 0:00:29 ( 7.33 ms/it)
+  objective:  13396.050723842935
+Minimizing 4072    Time: 0:00:29 ( 7.33 ms/it)
+  objective:  13395.983879727624
+Minimizing 4087    Time: 0:00:29 ( 7.33 ms/it)
+  objective:  13395.615054056798
+Minimizing 4101    Time: 0:00:30 ( 7.33 ms/it)
+  objective:  13395.519329171548
+Minimizing 4116    Time: 0:00:30 ( 7.32 ms/it)
+  objective:  13395.224520234457
+Minimizing 4131    Time: 0:00:30 ( 7.32 ms/it)
+  objective:  13394.9658075478
+Minimizing 4146    Time: 0:00:30 ( 7.32 ms/it)
+  objective:  13394.791124196367
+Minimizing 4161    Time: 0:00:30 ( 7.32 ms/it)
+  objective:  13394.542863835995
+Minimizing 4176    Time: 0:00:30 ( 7.32 ms/it)
+  objective:  13394.444356962282
+Minimizing 4191    Time: 0:00:30 ( 7.32 ms/it)
+  objective:  13394.215587366845
+Minimizing 4204    Time: 0:00:30 ( 7.33 ms/it)
+  objective:  13394.052118449872
+Minimizing 4219    Time: 0:00:30 ( 7.32 ms/it)
+  objective:  13393.735517275018
+Minimizing 4234    Time: 0:00:31 ( 7.32 ms/it)
+  objective:  13393.44722601975
+Minimizing 4249    Time: 0:00:31 ( 7.32 ms/it)
+  objective:  13392.689064977443
+Minimizing 4264    Time: 0:00:31 ( 7.32 ms/it)
+  objective:  13392.5093225917
+Minimizing 4279    Time: 0:00:31 ( 7.32 ms/it)
+  objective:  13392.44059960045
+Minimizing 4294    Time: 0:00:31 ( 7.32 ms/it)
+  objective:  13392.393695754188
+Minimizing 4309    Time: 0:00:31 ( 7.32 ms/it)
+  objective:  13392.362321211876
+Minimizing 4324    Time: 0:00:31 ( 7.31 ms/it)
+  objective:  13392.171233814814
+Minimizing 4339    Time: 0:00:31 ( 7.31 ms/it)
+  objective:  13392.014931868012
+Minimizing 4354    Time: 0:00:31 ( 7.31 ms/it)
+  objective:  13391.834840869022
+Minimizing 4369    Time: 0:00:31 ( 7.31 ms/it)
+  objective:  13391.60572624347
+Minimizing 4384    Time: 0:00:32 ( 7.31 ms/it)
+  objective:  13391.4851080519
+Minimizing 4399    Time: 0:00:32 ( 7.31 ms/it)
+  objective:  13391.351281604431
+Minimizing 4414    Time: 0:00:32 ( 7.31 ms/it)
+  objective:  13391.02561016132
+Minimizing 4429    Time: 0:00:32 ( 7.31 ms/it)
+  objective:  13391.005123049501
+Minimizing 4443    Time: 0:00:32 ( 7.31 ms/it)
+  objective:  13390.969986950528
+Minimizing 4458    Time: 0:00:32 ( 7.31 ms/it)
+  objective:  13390.839823986207
+Minimizing 4473    Time: 0:00:32 ( 7.31 ms/it)
+  objective:  13390.77843726243
+Minimizing 4487    Time: 0:00:32 ( 7.31 ms/it)
+  objective:  13390.753674325308
+Minimizing 4501    Time: 0:00:32 ( 7.31 ms/it)
+  objective:  13390.628302796264
+Minimizing 4515    Time: 0:00:32 ( 7.31 ms/it)
+  objective:  13390.489636480692
+Minimizing 4529    Time: 0:00:33 ( 7.31 ms/it)
+  objective:  13390.268351402643
+Minimizing 4543    Time: 0:00:33 ( 7.31 ms/it)
+  objective:  13390.070050332106
+Minimizing 4557    Time: 0:00:33 ( 7.31 ms/it)
+  objective:  13390.031498748845
+Minimizing 4571    Time: 0:00:33 ( 7.31 ms/it)
+  objective:  13390.01326925619
+Minimizing 4585    Time: 0:00:33 ( 7.31 ms/it)
+  objective:  13389.683349396742
+Minimizing 4599    Time: 0:00:33 ( 7.31 ms/it)
+  objective:  13389.635133653057
+Minimizing 4613    Time: 0:00:33 ( 7.31 ms/it)
+  objective:  13389.504639993414
+Minimizing 4627    Time: 0:00:33 ( 7.31 ms/it)
+  objective:  13389.36226139724
+Minimizing 4641    Time: 0:00:33 ( 7.31 ms/it)
+  objective:  13388.960163631331
+Minimizing 4655    Time: 0:00:34 ( 7.31 ms/it)
+  objective:  13388.944437128237
+Minimizing 4669    Time: 0:00:34 ( 7.31 ms/it)
+  objective:  13388.830255904271
+Minimizing 4683    Time: 0:00:34 ( 7.32 ms/it)
+  objective:  13388.816706349484
+Minimizing 4697    Time: 0:00:34 ( 7.32 ms/it)
+  objective:  13388.750937512305
+Minimizing 4711    Time: 0:00:34 ( 7.32 ms/it)
+  objective:  13388.732745979316
+Minimizing 4725    Time: 0:00:34 ( 7.32 ms/it)
+  objective:  13388.477876255201
+Minimizing 4739    Time: 0:00:34 ( 7.32 ms/it)
+  objective:  13388.457039927416
+Minimizing 4752    Time: 0:00:34 ( 7.32 ms/it)
+  objective:  13388.279306965618
+Minimizing 4766    Time: 0:00:34 ( 7.32 ms/it)
+  objective:  13388.190556589718
+Minimizing 4780    Time: 0:00:34 ( 7.32 ms/it)
+  objective:  13387.951612964542
+Minimizing 4794    Time: 0:00:35 ( 7.32 ms/it)
+  objective:  13387.899844082938
+Minimizing 4808    Time: 0:00:35 ( 7.32 ms/it)
+  objective:  13387.474673898134
+Minimizing 4822    Time: 0:00:35 ( 7.32 ms/it)
+  objective:  13387.356360838254
+Minimizing 4836    Time: 0:00:35 ( 7.32 ms/it)
+  objective:  13387.319944968192
+Minimizing 4850    Time: 0:00:35 ( 7.32 ms/it)
+  objective:  13387.220607053066
+Minimizing 4864    Time: 0:00:35 ( 7.33 ms/it)
+  objective:  13387.093900436113
+Minimizing 4878    Time: 0:00:35 ( 7.33 ms/it)
+  objective:  13386.941327755601
+Minimizing 4892    Time: 0:00:35 ( 7.33 ms/it)
+  objective:  13386.874838433992
+Minimizing 4906    Time: 0:00:35 ( 7.33 ms/it)
+  objective:  13386.416810829352
+Minimizing 4920    Time: 0:00:36 ( 7.33 ms/it)
+  objective:  13386.320938527162
+Minimizing 4934    Time: 0:00:36 ( 7.33 ms/it)
+  objective:  13386.205973585209
+Minimizing 4948    Time: 0:00:36 ( 7.33 ms/it)
+  objective:  13385.97963379023
+Minimizing 4962    Time: 0:00:36 ( 7.33 ms/it)
+  objective:  13385.919946994349
+Minimizing 4976    Time: 0:00:36 ( 7.33 ms/it)
+  objective:  13385.133674770092
+Minimizing 4990    Time: 0:00:36 ( 7.33 ms/it)
+  objective:  13384.961282848453
+Minimizing 5004    Time: 0:00:36 ( 7.33 ms/it)
+  objective:  13384.939090435284
+Minimizing 5018    Time: 0:00:36 ( 7.33 ms/it)
+  objective:  13384.787272787049
+Minimizing 5032    Time: 0:00:36 ( 7.33 ms/it)
+  objective:  13384.660998351355
+Minimizing 5046    Time: 0:00:37 ( 7.33 ms/it)
+  objective:  13384.516221564649
+Minimizing 5060    Time: 0:00:37 ( 7.33 ms/it)
+  objective:  13384.667419611193
+Minimizing 5074    Time: 0:00:37 ( 7.34 ms/it)
+  objective:  13384.112936695135
+Minimizing 5087    Time: 0:00:37 ( 7.34 ms/it)
+  objective:  13383.952917105904
+Minimizing 5100    Time: 0:00:37 ( 7.34 ms/it)
+  objective:  13383.883000572481
+Minimizing 5113    Time: 0:00:37 ( 7.34 ms/it)
+  objective:  13383.869112148881
+Minimizing 5127    Time: 0:00:37 ( 7.34 ms/it)
+  objective:  13383.784772963409
+Minimizing 5141    Time: 0:00:37 ( 7.34 ms/it)
+  objective:  13383.676031899835
+Minimizing 5154    Time: 0:00:37 ( 7.34 ms/it)
+  objective:  13383.596635321861
+Minimizing 5167    Time: 0:00:37 ( 7.34 ms/it)
+  objective:  13383.420207179115
+Minimizing 5180    Time: 0:00:38 ( 7.35 ms/it)
+  objective:  13383.35937420951
+Minimizing 5194    Time: 0:00:38 ( 7.35 ms/it)
+  objective:  13383.412563334445
+Minimizing 5208    Time: 0:00:38 ( 7.35 ms/it)
+  objective:  13383.232362762981
+Minimizing 5221    Time: 0:00:38 ( 7.35 ms/it)
+  objective:  13383.104242076442
+Minimizing 5234    Time: 0:00:38 ( 7.35 ms/it)
+  objective:  13383.0929790402
+Minimizing 5248    Time: 0:00:38 ( 7.35 ms/it)
+  objective:  13382.859099446185
+Minimizing 5262    Time: 0:00:38 ( 7.35 ms/it)
+  objective:  13382.777078811254
+Minimizing 5276    Time: 0:00:38 ( 7.35 ms/it)
+  objective:  13382.586290260464
+Minimizing 5290    Time: 0:00:38 ( 7.35 ms/it)
+  objective:  13382.339263743459
+Minimizing 5303    Time: 0:00:39 ( 7.35 ms/it)
+  objective:  13382.330729079556
+Minimizing 5317    Time: 0:00:39 ( 7.36 ms/it)
+  objective:  13382.204917918061
+Minimizing 5331    Time: 0:00:39 ( 7.36 ms/it)
+  objective:  13382.121964126476
+Minimizing 5345    Time: 0:00:39 ( 7.36 ms/it)
+  objective:  13381.873865505397
+Minimizing 5358    Time: 0:00:39 ( 7.36 ms/it)
+  objective:  13381.855182459127
+Minimizing 5371    Time: 0:00:39 ( 7.36 ms/it)
+  objective:  13381.750769383762
+Minimizing 5385    Time: 0:00:39 ( 7.36 ms/it)
+  objective:  13381.710138183313
+Minimizing 5398    Time: 0:00:39 ( 7.36 ms/it)
+  objective:  13381.620214353155
+Minimizing 5411    Time: 0:00:39 ( 7.36 ms/it)
+  objective:  13381.466242282484
+Minimizing 5424    Time: 0:00:39 ( 7.36 ms/it)
+  objective:  13380.88099694821
+Minimizing 5437    Time: 0:00:40 ( 7.36 ms/it)
+  objective:  13380.852482779796
+Minimizing 5450    Time: 0:00:40 ( 7.37 ms/it)
+  objective:  13380.819890855964
+Minimizing 5463    Time: 0:00:40 ( 7.37 ms/it)
+  objective:  13380.736262859835
+Minimizing 5476    Time: 0:00:40 ( 7.37 ms/it)
+  objective:  13380.672026715503
+Minimizing 5489    Time: 0:00:40 ( 7.37 ms/it)
+  objective:  13380.506751959038
+Minimizing 5502    Time: 0:00:40 ( 7.37 ms/it)
+  objective:  13380.438781375276
+Minimizing 5515    Time: 0:00:40 ( 7.37 ms/it)
+  objective:  13380.368614276093
+Minimizing 5528    Time: 0:00:40 ( 7.37 ms/it)
+  objective:  13380.185128732526
+Minimizing 5541    Time: 0:00:40 ( 7.37 ms/it)
+  objective:  13380.136516668797
+Minimizing 5554    Time: 0:00:40 ( 7.38 ms/it)
+  objective:  13379.972482921621
+Minimizing 5567    Time: 0:00:41 ( 7.38 ms/it)
+  objective:  13379.939443298987
+Minimizing 5579    Time: 0:00:41 ( 7.38 ms/it)
+  objective:  13379.905332863302
+Minimizing 5591    Time: 0:00:41 ( 7.38 ms/it)
+  objective:  13379.86842277123
+Minimizing 5603    Time: 0:00:41 ( 7.39 ms/it)
+  objective:  13379.525399804494
+Minimizing 5614    Time: 0:00:41 ( 7.39 ms/it)
+  objective:  13379.403896043608
+Minimizing 5627    Time: 0:00:41 ( 7.39 ms/it)
+  objective:  13378.956253257915
+Minimizing 5640    Time: 0:00:41 ( 7.39 ms/it)
+  objective:  13378.045818564373
+Minimizing 5653    Time: 0:00:41 ( 7.40 ms/it)
+  objective:  13377.79493796228
+Minimizing 5666    Time: 0:00:41 ( 7.40 ms/it)
+  objective:  13377.668320128949
+Minimizing 5679    Time: 0:00:42 ( 7.40 ms/it)
+  objective:  13377.658958734275
+Minimizing 5691    Time: 0:00:42 ( 7.40 ms/it)
+  objective:  13377.637708129208
+Minimizing 5703    Time: 0:00:42 ( 7.40 ms/it)
+  objective:  13377.625346839159
+Minimizing 5715    Time: 0:00:42 ( 7.41 ms/it)
+  objective:  13377.617333501265
+Minimizing 5727    Time: 0:00:42 ( 7.41 ms/it)
+  objective:  13377.54948557925
+Minimizing 5739    Time: 0:00:42 ( 7.41 ms/it)
+  objective:  13377.502837544722
+Minimizing 5752    Time: 0:00:42 ( 7.41 ms/it)
+  objective:  13377.466585695089
+Minimizing 5765    Time: 0:00:42 ( 7.42 ms/it)
+  objective:  13377.456472188496
+Minimizing 5778    Time: 0:00:42 ( 7.42 ms/it)
+  objective:  13377.420870036578
+Minimizing 5791    Time: 0:00:42 ( 7.42 ms/it)
+  objective:  13377.406921252266
+Minimizing 5804    Time: 0:00:43 ( 7.42 ms/it)
+  objective:  13377.370124584479
+Minimizing 5817    Time: 0:00:43 ( 7.42 ms/it)
+  objective:  13377.35302146474
+Minimizing 5830    Time: 0:00:43 ( 7.42 ms/it)
+  objective:  13377.349389796611
+Minimizing 5843    Time: 0:00:43 ( 7.42 ms/it)
+  objective:  13377.34361501415
+Minimizing 5856    Time: 0:00:43 ( 7.42 ms/it)
+  objective:  13377.325122791328
+Minimizing 5869    Time: 0:00:43 ( 7.42 ms/it)
+  objective:  13377.29036253311
+Minimizing 5882    Time: 0:00:43 ( 7.42 ms/it)
+  objective:  13377.280163644704
+Minimizing 5895    Time: 0:00:43 ( 7.42 ms/it)
+  objective:  13377.246976870345
+Minimizing 5908    Time: 0:00:43 ( 7.43 ms/it)
+  objective:  13377.227608630747
+Minimizing 5921    Time: 0:00:43 ( 7.43 ms/it)
+  objective:  13377.198846318679
+Minimizing 5934    Time: 0:00:44 ( 7.43 ms/it)
+  objective:  13377.149085639976
+Minimizing 5947    Time: 0:00:44 ( 7.43 ms/it)
+  objective:  13377.130973619605
+Minimizing 5960    Time: 0:00:44 ( 7.43 ms/it)
+  objective:  13377.069916933811
+Minimizing 5973    Time: 0:00:44 ( 7.43 ms/it)
+  objective:  13377.054915751825
+Minimizing 5986    Time: 0:00:44 ( 7.43 ms/it)
+  objective:  13376.868824740704
+Minimizing 5999    Time: 0:00:44 ( 7.43 ms/it)
+  objective:  13376.79084285604
+Minimizing 6012    Time: 0:00:44 ( 7.43 ms/it)
+  objective:  13376.749280155498
+Minimizing 6025    Time: 0:00:44 ( 7.43 ms/it)
+  objective:  13376.682422649399
+Minimizing 6038    Time: 0:00:44 ( 7.44 ms/it)
+  objective:  13376.655282042768
+Minimizing 6051    Time: 0:00:44 ( 7.44 ms/it)
+  objective:  13376.585088006861
+Minimizing 6064    Time: 0:00:45 ( 7.44 ms/it)
+  objective:  13376.565631842575
+Minimizing 6077    Time: 0:00:45 ( 7.44 ms/it)
+  objective:  13376.46982149381
+Minimizing 6090    Time: 0:00:45 ( 7.44 ms/it)
+  objective:  13376.448142946712
+Minimizing 6103    Time: 0:00:45 ( 7.44 ms/it)
+  objective:  13376.434987296474
+Minimizing 6116    Time: 0:00:45 ( 7.44 ms/it)
+  objective:  13376.339550814751
+Minimizing 6129    Time: 0:00:45 ( 7.44 ms/it)
+  objective:  13376.333471791411
+Minimizing 6142    Time: 0:00:45 ( 7.44 ms/it)
+  objective:  13376.153261703344
+Minimizing 6155    Time: 0:00:45 ( 7.44 ms/it)
+  objective:  13376.124881930067
+Minimizing 6168    Time: 0:00:45 ( 7.45 ms/it)
+  objective:  13376.04707555881
+Minimizing 6181    Time: 0:00:46 ( 7.45 ms/it)
+  objective:  13375.978992237346
+Minimizing 6194    Time: 0:00:46 ( 7.45 ms/it)
+  objective:  13375.948807614506
+Minimizing 6207    Time: 0:00:46 ( 7.45 ms/it)
+  objective:  13375.688983930486
+Minimizing 6220    Time: 0:00:46 ( 7.45 ms/it)
+  objective:  13375.644490464692
+Minimizing 6233    Time: 0:00:46 ( 7.45 ms/it)
+  objective:  13375.639326058139
+Minimizing 6246    Time: 0:00:46 ( 7.45 ms/it)
+  objective:  13375.591860274195
+Minimizing 6259    Time: 0:00:46 ( 7.45 ms/it)
+  objective:  13375.568646805637
+Minimizing 6272    Time: 0:00:46 ( 7.45 ms/it)
+  objective:  13375.490601102661
+Minimizing 6285    Time: 0:00:46 ( 7.45 ms/it)
+  objective:  13375.474220636897
+Minimizing 6298    Time: 0:00:46 ( 7.46 ms/it)
+  objective:  13375.448782051913
+Minimizing 6311    Time: 0:00:47 ( 7.46 ms/it)
+  objective:  13375.42697524641
+Minimizing 6324    Time: 0:00:47 ( 7.46 ms/it)
+  objective:  13375.371696402435
+Minimizing 6337    Time: 0:00:47 ( 7.46 ms/it)
+  objective:  13375.357651158694
+Minimizing 6350    Time: 0:00:47 ( 7.46 ms/it)
+  objective:  13375.296076487008
+Minimizing 6363    Time: 0:00:47 ( 7.46 ms/it)
+  objective:  13375.24285208687
+Minimizing 6376    Time: 0:00:47 ( 7.46 ms/it)
+  objective:  13375.224310229685
+Minimizing 6389    Time: 0:00:47 ( 7.46 ms/it)
+  objective:  13375.201005784875
+Minimizing 6402    Time: 0:00:47 ( 7.46 ms/it)
+  objective:  13375.154346305404
+Minimizing 6414    Time: 0:00:47 ( 7.47 ms/it)
+  objective:  13375.116168044297
+Minimizing 6426    Time: 0:00:47 ( 7.47 ms/it)
+  objective:  13375.020508141875
+Minimizing 6438    Time: 0:00:48 ( 7.47 ms/it)
+  objective:  13374.981308338458
+Minimizing 6450    Time: 0:00:48 ( 7.47 ms/it)
+  objective:  13374.87269590405
+Minimizing 6463    Time: 0:00:48 ( 7.47 ms/it)
+  objective:  13374.66669194882
+Minimizing 6476    Time: 0:00:48 ( 7.47 ms/it)
+  objective:  13374.607414354025
+Minimizing 6489    Time: 0:00:48 ( 7.47 ms/it)
+  objective:  13374.557310891229
+Minimizing 6502    Time: 0:00:48 ( 7.48 ms/it)
+  objective:  13374.520006731414
+Minimizing 6515    Time: 0:00:48 ( 7.48 ms/it)
+  objective:  13374.427641703267
+Minimizing 6528    Time: 0:00:48 ( 7.48 ms/it)
+  objective:  13374.404839222785
+Minimizing 6541    Time: 0:00:48 ( 7.48 ms/it)
+  objective:  13374.331827232549
+Minimizing 6554    Time: 0:00:49 ( 7.48 ms/it)
+  objective:  13374.310622945588
+Minimizing 6567    Time: 0:00:49 ( 7.48 ms/it)
+  objective:  13374.224152422714
+Minimizing 6580    Time: 0:00:49 ( 7.48 ms/it)
+  objective:  13374.077757193983
+Minimizing 6593    Time: 0:00:49 ( 7.48 ms/it)
+  objective:  13373.99385817087
+Minimizing 6606    Time: 0:00:49 ( 7.48 ms/it)
+  objective:  13373.822935274242
+Minimizing 6619    Time: 0:00:49 ( 7.48 ms/it)
+  objective:  13373.782629826317
+Minimizing 6632    Time: 0:00:49 ( 7.48 ms/it)
+  objective:  13373.749947493518
+Minimizing 6645    Time: 0:00:49 ( 7.48 ms/it)
+  objective:  13373.715758730126
+Minimizing 6658    Time: 0:00:49 ( 7.49 ms/it)
+  objective:  13373.683631695414
+Minimizing 6671    Time: 0:00:49 ( 7.49 ms/it)
+  objective:  13373.606736935537
+Minimizing 6684    Time: 0:00:50 ( 7.49 ms/it)
+  objective:  13373.544731723981
+Minimizing 6697    Time: 0:00:50 ( 7.49 ms/it)
+  objective:  13373.455664094727
+Minimizing 6710    Time: 0:00:50 ( 7.49 ms/it)
+  objective:  13373.41295664743
+Minimizing 6723    Time: 0:00:50 ( 7.49 ms/it)
+  objective:  13373.321973085425
+Minimizing 6736    Time: 0:00:50 ( 7.49 ms/it)
+  objective:  13373.271715735558
+Minimizing 6749    Time: 0:00:50 ( 7.49 ms/it)
+  objective:  13372.982856866809
+Minimizing 6762    Time: 0:00:50 ( 7.49 ms/it)
+  objective:  13373.33130436045
+Minimizing 6776    Time: 0:00:50 ( 7.49 ms/it)
+  objective:  13371.965579579111
+Minimizing 6789    Time: 0:00:50 ( 7.49 ms/it)
+  objective:  13371.936472508583
+Minimizing 6802    Time: 0:00:50 ( 7.49 ms/it)
+  objective:  13371.898116064549
+Minimizing 6815    Time: 0:00:51 ( 7.49 ms/it)
+  objective:  13371.839962030695
+Minimizing 6828    Time: 0:00:51 ( 7.50 ms/it)
+  objective:  13371.786364898093
+Minimizing 6841    Time: 0:00:51 ( 7.50 ms/it)
+  objective:  13371.779027829594
+Minimizing 6854    Time: 0:00:51 ( 7.50 ms/it)
+  objective:  13371.652209540618
+Minimizing 6867    Time: 0:00:51 ( 7.50 ms/it)
+  objective:  13371.542213625144
+Minimizing 6880    Time: 0:00:51 ( 7.50 ms/it)
+  objective:  13371.45131608536
+Minimizing 6893    Time: 0:00:51 ( 7.50 ms/it)
+  objective:  13371.370562589334
+Minimizing 6906    Time: 0:00:51 ( 7.50 ms/it)
+  objective:  13371.364184471095
+Minimizing 6919    Time: 0:00:51 ( 7.50 ms/it)
+  objective:  13371.103810562956
+Minimizing 6932    Time: 0:00:51 ( 7.50 ms/it)
+  objective:  13371.697612452008
+Minimizing 6945    Time: 0:00:52 ( 7.50 ms/it)
+  objective:  13370.611521213672
+Minimizing 6958    Time: 0:00:52 ( 7.50 ms/it)
+  objective:  13370.590431240962
+Minimizing 6971    Time: 0:00:52 ( 7.50 ms/it)
+  objective:  13370.579600251302
+Minimizing 6984    Time: 0:00:52 ( 7.50 ms/it)
+  objective:  13370.569385500923
+Minimizing 6997    Time: 0:00:52 ( 7.50 ms/it)
+  objective:  13370.50814392767
+Minimizing 7010    Time: 0:00:52 ( 7.50 ms/it)
+  objective:  13370.485559016597
+Minimizing 7023    Time: 0:00:52 ( 7.51 ms/it)
+  objective:  13370.424913073977
+Minimizing 7036    Time: 0:00:52 ( 7.51 ms/it)
+  objective:  13370.15801874495
+Minimizing 7049    Time: 0:00:52 ( 7.51 ms/it)
+  objective:  13370.131222709017
+Minimizing 7062    Time: 0:00:53 ( 7.51 ms/it)
+  objective:  13370.115180201748
+Minimizing 7075    Time: 0:00:53 ( 7.51 ms/it)
+  objective:  13370.108699870281
+Minimizing 7088    Time: 0:00:53 ( 7.51 ms/it)
+  objective:  13370.081590758731
+Minimizing 7101    Time: 0:00:53 ( 7.51 ms/it)
+  objective:  13370.075186012284
+Minimizing 7114    Time: 0:00:53 ( 7.51 ms/it)
+  objective:  13370.026447433615
+Minimizing 7127    Time: 0:00:53 ( 7.51 ms/it)
+  objective:  13370.012837310256
+Minimizing 7140    Time: 0:00:53 ( 7.51 ms/it)
+  objective:  13369.980851301902
+Minimizing 7153    Time: 0:00:53 ( 7.51 ms/it)
+  objective:  13369.964754558721
+Minimizing 7166    Time: 0:00:53 ( 7.51 ms/it)
+  objective:  13369.913095109303
+Minimizing 7179    Time: 0:00:53 ( 7.51 ms/it)
+  objective:  13369.89427704984
+Minimizing 7192    Time: 0:00:54 ( 7.51 ms/it)
+  objective:  13370.178522857874
+Minimizing 7205    Time: 0:00:54 ( 7.51 ms/it)
+  objective:  13369.756503244644
+Minimizing 7218    Time: 0:00:54 ( 7.52 ms/it)
+  objective:  13369.743102026863
+Minimizing 7231    Time: 0:00:54 ( 7.52 ms/it)
+  objective:  13369.702159832486
+Minimizing 7244    Time: 0:00:54 ( 7.52 ms/it)
+  objective:  13369.651449666802
+Minimizing 7257    Time: 0:00:54 ( 7.52 ms/it)
+  objective:  13369.5892650795
+Minimizing 7270    Time: 0:00:54 ( 7.52 ms/it)
+  objective:  13369.580042344474
+Minimizing 7283    Time: 0:00:54 ( 7.52 ms/it)
+  objective:  13369.550205761981
+Minimizing 7296    Time: 0:00:54 ( 7.52 ms/it)
+  objective:  13369.53847638701
+Minimizing 7309    Time: 0:00:54 ( 7.52 ms/it)
+  objective:  13369.501306758371
+Minimizing 7322    Time: 0:00:55 ( 7.52 ms/it)
+  objective:  13369.475342705751
+Minimizing 7335    Time: 0:00:55 ( 7.52 ms/it)
+  objective:  13369.470985072418
+Minimizing 7348    Time: 0:00:55 ( 7.52 ms/it)
+  objective:  13369.41372521386
+Minimizing 7361    Time: 0:00:55 ( 7.52 ms/it)
+  objective:  13369.385378006635
+Minimizing 7374    Time: 0:00:55 ( 7.52 ms/it)
+  objective:  13369.357674377228
+Minimizing 7387    Time: 0:00:55 ( 7.52 ms/it)
+  objective:  13369.245185528023
+Minimizing 7400    Time: 0:00:55 ( 7.52 ms/it)
+  objective:  13369.232401075598
+Minimizing 7413    Time: 0:00:55 ( 7.52 ms/it)
+  objective:  13369.115761863104
+Minimizing 7426    Time: 0:00:55 ( 7.52 ms/it)
+  objective:  13369.022431451041
+Minimizing 7439    Time: 0:00:55 ( 7.53 ms/it)
+  objective:  13369.01399809988
+Minimizing 7452    Time: 0:00:56 ( 7.53 ms/it)
+  objective:  13368.9586116817
+Minimizing 7465    Time: 0:00:56 ( 7.53 ms/it)
+  objective:  13368.904309428835
+Minimizing 7478    Time: 0:00:56 ( 7.53 ms/it)
+  objective:  13368.849177516495
+Minimizing 7491    Time: 0:00:56 ( 7.53 ms/it)
+  objective:  13368.844352713299
+Minimizing 7504    Time: 0:00:56 ( 7.53 ms/it)
+  objective:  13368.88491520103
+Minimizing 7517    Time: 0:00:56 ( 7.53 ms/it)
+  objective:  13368.504069727554
+Minimizing 7530    Time: 0:00:56 ( 7.53 ms/it)
+  objective:  13368.316080250326
+Minimizing 7543    Time: 0:00:56 ( 7.53 ms/it)
+  objective:  13368.30677029764
+Minimizing 7556    Time: 0:00:56 ( 7.53 ms/it)
+  objective:  13368.265283500674
+Minimizing 7569    Time: 0:00:57 ( 7.53 ms/it)
+  objective:  13368.162191943964
+Minimizing 7582    Time: 0:00:57 ( 7.53 ms/it)
+  objective:  13368.042496879469
+Minimizing 7595    Time: 0:00:57 ( 7.53 ms/it)
+  objective:  13367.991488810767
+Minimizing 7608    Time: 0:00:57 ( 7.53 ms/it)
+  objective:  13367.96163360044
+Minimizing 7621    Time: 0:00:57 ( 7.53 ms/it)
+  objective:  13367.92223936331
+Minimizing 7634    Time: 0:00:57 ( 7.53 ms/it)
+  objective:  13367.907177319496
+Minimizing 7647    Time: 0:00:57 ( 7.53 ms/it)
+  objective:  13367.877608048024
+Minimizing 7660    Time: 0:00:57 ( 7.53 ms/it)
+  objective:  13367.702937814407
+Minimizing 7673    Time: 0:00:57 ( 7.54 ms/it)
+  objective:  13367.619025732143
+Minimizing 7686    Time: 0:00:57 ( 7.54 ms/it)
+  objective:  13367.421147783243
+Minimizing 7699    Time: 0:00:58 ( 7.54 ms/it)
+  objective:  13367.32194098596
+Minimizing 7712    Time: 0:00:58 ( 7.54 ms/it)
+  objective:  13367.252398044264
+Minimizing 7725    Time: 0:00:58 ( 7.54 ms/it)
+  objective:  13367.19987273607
+Minimizing 7738    Time: 0:00:58 ( 7.54 ms/it)
+  objective:  13367.188365503913
+Minimizing 7751    Time: 0:00:58 ( 7.54 ms/it)
+  objective:  13367.150803571705
+Minimizing 7764    Time: 0:00:58 ( 7.54 ms/it)
+  objective:  13367.144301395849
+Minimizing 7777    Time: 0:00:58 ( 7.54 ms/it)
+  objective:  13367.069368550663
+Minimizing 7790    Time: 0:00:58 ( 7.54 ms/it)
+  objective:  13367.058622049291
+Minimizing 7803    Time: 0:00:58 ( 7.54 ms/it)
+  objective:  13367.050423728862
+Minimizing 7816    Time: 0:00:58 ( 7.54 ms/it)
+  objective:  13367.025317448417
+Minimizing 7829    Time: 0:00:59 ( 7.54 ms/it)
+  objective:  13367.004923522029
+Minimizing 7842    Time: 0:00:59 ( 7.54 ms/it)
+  objective:  13366.958180232345
+Minimizing 7855    Time: 0:00:59 ( 7.54 ms/it)
+  objective:  13366.889445437599
+Minimizing 7868    Time: 0:00:59 ( 7.54 ms/it)
+  objective:  13366.873283626002
+Minimizing 7881    Time: 0:00:59 ( 7.54 ms/it)
+  objective:  13366.849938914107
+Minimizing 7894    Time: 0:00:59 ( 7.55 ms/it)
+  objective:  13366.813290796767
+Minimizing 7907    Time: 0:00:59 ( 7.55 ms/it)
+  objective:  13366.790500041658
+Minimizing 7920    Time: 0:00:59 ( 7.55 ms/it)
+  objective:  13366.751413991842
+Minimizing 7933    Time: 0:00:59 ( 7.55 ms/it)
+  objective:  13366.735362938518
+Minimizing 7946    Time: 0:00:59 ( 7.55 ms/it)
+  objective:  13366.707570033046
+Minimizing 7959    Time: 0:01:00 ( 7.55 ms/it)
+  objective:  13366.676525630457
+Minimizing 7972    Time: 0:01:00 ( 7.55 ms/it)
+  objective:  13366.57840019682
+Minimizing 7985    Time: 0:01:00 ( 7.55 ms/it)
+  objective:  13366.485061246203
+Minimizing 7998    Time: 0:01:00 ( 7.55 ms/it)
+  objective:  13366.457160909682
+Minimizing 8011    Time: 0:01:00 ( 7.55 ms/it)
+  objective:  13366.416319651384
+Minimizing 8024    Time: 0:01:00 ( 7.55 ms/it)
+  objective:  13366.403211679273
+Minimizing 8037    Time: 0:01:00 ( 7.55 ms/it)
+  objective:  13366.224660476837
+Minimizing 8050    Time: 0:01:00 ( 7.55 ms/it)
+  objective:  13366.120157493737
+Minimizing 8063    Time: 0:01:00 ( 7.55 ms/it)
+  objective:  13366.07068632837
+Minimizing 8076    Time: 0:01:00 ( 7.55 ms/it)
+  objective:  13365.858129255721
+Minimizing 8089    Time: 0:01:01 ( 7.55 ms/it)
+  objective:  13365.748974905742
+Minimizing 8102    Time: 0:01:01 ( 7.55 ms/it)
+  objective:  13365.729972563946
+Minimizing 8115    Time: 0:01:01 ( 7.55 ms/it)
+  objective:  13365.693606154346
+Minimizing 8128    Time: 0:01:01 ( 7.55 ms/it)
+  objective:  13365.601513470836
+Minimizing 8141    Time: 0:01:01 ( 7.56 ms/it)
+  objective:  13365.545644254496
+Minimizing 8154    Time: 0:01:01 ( 7.56 ms/it)
+  objective:  13365.535512039569
+Minimizing 8167    Time: 0:01:01 ( 7.56 ms/it)
+  objective:  13365.486857557771
+Minimizing 8180    Time: 0:01:01 ( 7.56 ms/it)
+  objective:  13365.404535042624
+Minimizing 8193    Time: 0:01:01 ( 7.56 ms/it)
+  objective:  13365.309493955465
+Minimizing 8206    Time: 0:01:02 ( 7.56 ms/it)
+  objective:  13365.282111891465
+Minimizing 8219    Time: 0:01:02 ( 7.56 ms/it)
+  objective:  13365.22045874424
+Minimizing 8232    Time: 0:01:02 ( 7.56 ms/it)
+  objective:  13365.202491006356
+Minimizing 8245    Time: 0:01:02 ( 7.56 ms/it)
+  objective:  13365.147682533097
+Minimizing 8258    Time: 0:01:02 ( 7.56 ms/it)
+  objective:  13365.01881806591
+Minimizing 8271    Time: 0:01:02 ( 7.56 ms/it)
+  objective:  13364.91352119024
+Minimizing 8284    Time: 0:01:02 ( 7.56 ms/it)
+  objective:  13364.909899063488
+Minimizing 8297    Time: 0:01:02 ( 7.56 ms/it)
+  objective:  13364.84132873697
+Minimizing 8310    Time: 0:01:02 ( 7.56 ms/it)
+  objective:  13364.798171947936
+Minimizing 8323    Time: 0:01:02 ( 7.56 ms/it)
+  objective:  13364.78950660913
+Minimizing 8336    Time: 0:01:03 ( 7.56 ms/it)
+  objective:  13364.737842341732
+Minimizing 8349    Time: 0:01:03 ( 7.56 ms/it)
+  objective:  13364.655399637355
+Minimizing 8362    Time: 0:01:03 ( 7.56 ms/it)
+  objective:  13364.584928252123
+Minimizing 8375    Time: 0:01:03 ( 7.56 ms/it)
+  objective:  13364.454427006647
+Minimizing 8388    Time: 0:01:03 ( 7.57 ms/it)
+  objective:  13364.449407192842
+Minimizing 8401    Time: 0:01:03 ( 7.57 ms/it)
+  objective:  13364.43269589034
+Minimizing 8414    Time: 0:01:03 ( 7.57 ms/it)
+  objective:  13364.387105424234
+Minimizing 8427    Time: 0:01:03 ( 7.57 ms/it)
+  objective:  13364.336911130813
+Minimizing 8440    Time: 0:01:03 ( 7.57 ms/it)
+  objective:  13364.31954021076
+Minimizing 8453    Time: 0:01:03 ( 7.57 ms/it)
+  objective:  13364.174568437382
+Minimizing 8466    Time: 0:01:04 ( 7.57 ms/it)
+  objective:  13363.990813515316
+Minimizing 8479    Time: 0:01:04 ( 7.57 ms/it)
+  objective:  13363.880959640148
+Minimizing 8492    Time: 0:01:04 ( 7.57 ms/it)
+  objective:  13363.872811041736
+Minimizing 8505    Time: 0:01:04 ( 7.57 ms/it)
+  objective:  13363.830706508677
+Minimizing 8518    Time: 0:01:04 ( 7.57 ms/it)
+  objective:  13363.791285763582
+Minimizing 8531    Time: 0:01:04 ( 7.57 ms/it)
+  objective:  13363.780501061185
+Minimizing 8544    Time: 0:01:04 ( 7.57 ms/it)
+  objective:  13363.698922377065
+Minimizing 8557    Time: 0:01:04 ( 7.57 ms/it)
+  objective:  13363.573010990069
+Minimizing 8570    Time: 0:01:04 ( 7.57 ms/it)
+  objective:  13363.559485774109
+Minimizing 8583    Time: 0:01:04 ( 7.57 ms/it)
+  objective:  13363.519351053219
+Minimizing 8596    Time: 0:01:05 ( 7.57 ms/it)
+  objective:  13363.469903991027
+Minimizing 8609    Time: 0:01:05 ( 7.57 ms/it)
+  objective:  13363.146869581506
+Minimizing 8622    Time: 0:01:05 ( 7.57 ms/it)
+  objective:  13362.974004896634
+Minimizing 8635    Time: 0:01:05 ( 7.57 ms/it)
+  objective:  13362.966726134808
+Minimizing 8648    Time: 0:01:05 ( 7.57 ms/it)
+  objective:  13362.876816482087
+Minimizing 8661    Time: 0:01:05 ( 7.58 ms/it)
+  objective:  13362.840602212113
+Minimizing 8674    Time: 0:01:05 ( 7.58 ms/it)
+  objective:  13362.808330337837
+Minimizing 8687    Time: 0:01:05 ( 7.58 ms/it)
+  objective:  13362.713331427221
+Minimizing 8700    Time: 0:01:05 ( 7.58 ms/it)
+  objective:  13362.52541540561
+Minimizing 8713    Time: 0:01:06 ( 7.58 ms/it)
+  objective:  13362.50936264579
+Minimizing 8727    Time: 0:01:06 ( 7.58 ms/it)
+  objective:  13362.481698098643
+Minimizing 8741    Time: 0:01:06 ( 7.58 ms/it)
+  objective:  13362.41400605131
+Minimizing 8755    Time: 0:01:06 ( 7.58 ms/it)
+  objective:  13362.333785017021
+Minimizing 8769    Time: 0:01:06 ( 7.58 ms/it)
+  objective:  13362.275017465407
+Minimizing 8782    Time: 0:01:06 ( 7.58 ms/it)
+  objective:  13362.192109590025
+Minimizing 8796    Time: 0:01:06 ( 7.58 ms/it)
+  objective:  13361.886057994947
+Minimizing 8810    Time: 0:01:06 ( 7.58 ms/it)
+  objective:  13361.8578144684
+Minimizing 8824    Time: 0:01:06 ( 7.58 ms/it)
+  objective:  13361.837550150405
+Minimizing 8838    Time: 0:01:06 ( 7.58 ms/it)
+  objective:  13361.801359127734
+Minimizing 8852    Time: 0:01:07 ( 7.58 ms/it)
+  objective:  13361.722740446581
+Minimizing 8866    Time: 0:01:07 ( 7.58 ms/it)
+  objective:  13361.805018484301
+Minimizing 8880    Time: 0:01:07 ( 7.58 ms/it)
+  objective:  13361.610564986404
+Minimizing 8894    Time: 0:01:07 ( 7.58 ms/it)
+  objective:  13361.481391794805
+Minimizing 8907    Time: 0:01:07 ( 7.58 ms/it)
+  objective:  13361.187647229715
+Minimizing 8921    Time: 0:01:07 ( 7.58 ms/it)
+  objective:  13361.118564894452
+Minimizing 8935    Time: 0:01:07 ( 7.58 ms/it)
+  objective:  13361.095143978411
+Minimizing 8949    Time: 0:01:07 ( 7.58 ms/it)
+  objective:  13361.060010275804
+Minimizing 8963    Time: 0:01:07 ( 7.58 ms/it)
+  objective:  13361.028610137604
+Minimizing 8976    Time: 0:01:08 ( 7.58 ms/it)
+  objective:  13361.016030356215
+Minimizing 8990    Time: 0:01:08 ( 7.58 ms/it)
+  objective:  13360.87569488967
+Minimizing 9004    Time: 0:01:08 ( 7.58 ms/it)
+  objective:  13360.86403823574
+Minimizing 9018    Time: 0:01:08 ( 7.58 ms/it)
+  objective:  13360.805560081775
+Minimizing 9032    Time: 0:01:08 ( 7.58 ms/it)
+  objective:  13360.797934487855
+Minimizing 9045    Time: 0:01:08 ( 7.58 ms/it)
+  objective:  13360.777648840398
+Minimizing 9059    Time: 0:01:08 ( 7.58 ms/it)
+  objective:  13360.719084503071
+Minimizing 9073    Time: 0:01:08 ( 7.58 ms/it)
+  objective:  13360.64059255163
+Minimizing 9087    Time: 0:01:08 ( 7.58 ms/it)
+  objective:  13360.606309391704
+Minimizing 9100    Time: 0:01:08 ( 7.58 ms/it)
+  objective:  13360.548142264728
+Minimizing 9114    Time: 0:01:09 ( 7.58 ms/it)
+  objective:  13360.528536697653
+Minimizing 9128    Time: 0:01:09 ( 7.58 ms/it)
+  objective:  13360.352454508524
+Minimizing 9142    Time: 0:01:09 ( 7.58 ms/it)
+  objective:  13360.243168861205
+Minimizing 9156    Time: 0:01:09 ( 7.58 ms/it)
+  objective:  13360.214255512903
+Minimizing 9169    Time: 0:01:09 ( 7.58 ms/it)
+  objective:  13360.04095263296
+Minimizing 9182    Time: 0:01:09 ( 7.58 ms/it)
+  objective:  13359.878291541914
+Minimizing 9196    Time: 0:01:09 ( 7.58 ms/it)
+  objective:  13359.864816122397
+Minimizing 9210    Time: 0:01:09 ( 7.58 ms/it)
+  objective:  13359.761715725013
+Minimizing 9224    Time: 0:01:09 ( 7.58 ms/it)
+  objective:  13359.654031072569
+Minimizing 9238    Time: 0:01:10 ( 7.58 ms/it)
+  objective:  13359.625091722635
+Minimizing 9252    Time: 0:01:10 ( 7.58 ms/it)
+  objective:  13359.345031662822
+Minimizing 9266    Time: 0:01:10 ( 7.58 ms/it)
+  objective:  13359.332598017907
+Minimizing 9280    Time: 0:01:10 ( 7.58 ms/it)
+  objective:  13359.21155468587
+Minimizing 9293    Time: 0:01:10 ( 7.58 ms/it)
+  objective:  13359.196035343644
+Minimizing 9306    Time: 0:01:10 ( 7.58 ms/it)
+  objective:  13358.865382311938
+Minimizing 9320    Time: 0:01:10 ( 7.58 ms/it)
+  objective:  13358.705446093038
+Minimizing 9334    Time: 0:01:10 ( 7.58 ms/it)
+  objective:  13358.687842992964
+Minimizing 9348    Time: 0:01:10 ( 7.58 ms/it)
+  objective:  13358.64371003148
+Minimizing 9362    Time: 0:01:10 ( 7.58 ms/it)
+  objective:  13358.573556317038
+Minimizing 9376    Time: 0:01:11 ( 7.58 ms/it)
+  objective:  13358.553746106649
+Minimizing 9390    Time: 0:01:11 ( 7.58 ms/it)
+  objective:  13358.633592758219
+Minimizing 9404    Time: 0:01:11 ( 7.58 ms/it)
+  objective:  13358.510627141965
+Minimizing 9418    Time: 0:01:11 ( 7.58 ms/it)
+  objective:  13358.470516051631
+Minimizing 9431    Time: 0:01:11 ( 7.58 ms/it)
+  objective:  13358.406529329048
+Minimizing 9445    Time: 0:01:11 ( 7.58 ms/it)
+  objective:  13358.200210589042
+Minimizing 9459    Time: 0:01:11 ( 7.58 ms/it)
+  objective:  13358.18182786044
+Minimizing 9473    Time: 0:01:11 ( 7.58 ms/it)
+  objective:  13358.1702544445
+Minimizing 9487    Time: 0:01:11 ( 7.58 ms/it)
+  objective:  13358.168786482529
+Minimizing 9501    Time: 0:01:12 ( 7.58 ms/it)
+  objective:  13358.14951156122
+Minimizing 9515    Time: 0:01:12 ( 7.59 ms/it)
+  objective:  13358.139993503879
+Minimizing 9529    Time: 0:01:12 ( 7.59 ms/it)
+  objective:  13358.03832674495
+Minimizing 9543    Time: 0:01:12 ( 7.59 ms/it)
+  objective:  13358.012150576302
+Minimizing 9556    Time: 0:01:12 ( 7.59 ms/it)
+  objective:  13358.004605254464
+Minimizing 9569    Time: 0:01:12 ( 7.59 ms/it)
+  objective:  13357.972735239047
+Minimizing 9583    Time: 0:01:12 ( 7.59 ms/it)
+  objective:  13357.964050785093
+Minimizing 9597    Time: 0:01:12 ( 7.59 ms/it)
+  objective:  13357.935744345072
+Minimizing 9611    Time: 0:01:12 ( 7.59 ms/it)
+  objective:  13357.888114224654
+Minimizing 9625    Time: 0:01:13 ( 7.59 ms/it)
+  objective:  13357.850452371327
+Minimizing 9639    Time: 0:01:13 ( 7.59 ms/it)
+  objective:  13357.7471777795
+Minimizing 9653    Time: 0:01:13 ( 7.59 ms/it)
+  objective:  13357.743734510303
+Minimizing 9667    Time: 0:01:13 ( 7.59 ms/it)
+  objective:  13357.671639362277
+Minimizing 9681    Time: 0:01:13 ( 7.59 ms/it)
+  objective:  13357.643879831463
+Minimizing 9694    Time: 0:01:13 ( 7.59 ms/it)
+  objective:  13357.58037450962
+Minimizing 9707    Time: 0:01:13 ( 7.59 ms/it)
+  objective:  13357.569773701442
+Minimizing 9721    Time: 0:01:13 ( 7.59 ms/it)
+  objective:  13357.48113185275
+Minimizing 9735    Time: 0:01:13 ( 7.59 ms/it)
+  objective:  13357.457171079514
+Minimizing 9748    Time: 0:01:13 ( 7.59 ms/it)
+  objective:  13357.38336146011
+Minimizing 9762    Time: 0:01:14 ( 7.59 ms/it)
+  objective:  13357.369791779747
+Minimizing 9776    Time: 0:01:14 ( 7.59 ms/it)
+  objective:  13357.289511126604
+Minimizing 9790    Time: 0:01:14 ( 7.59 ms/it)
+  objective:  13357.273580989931
+Minimizing 9804    Time: 0:01:14 ( 7.59 ms/it)
+  objective:  13357.108316384503
+Minimizing 9818    Time: 0:01:14 ( 7.59 ms/it)
+  objective:  13356.840164088062
+Minimizing 9831    Time: 0:01:14 ( 7.59 ms/it)
+  objective:  13356.831514002537
+Minimizing 9845    Time: 0:01:14 ( 7.59 ms/it)
+  objective:  13356.795490035292
+Minimizing 9859    Time: 0:01:14 ( 7.59 ms/it)
+  objective:  13356.785360096372
+Minimizing 9873    Time: 0:01:14 ( 7.59 ms/it)
+  objective:  13356.770529168338
+Minimizing 9886    Time: 0:01:15 ( 7.59 ms/it)
+  objective:  13356.732596082504
+Minimizing 9900    Time: 0:01:15 ( 7.59 ms/it)
+  objective:  13356.698430170953
+Minimizing 9914    Time: 0:01:15 ( 7.59 ms/it)
+  objective:  13356.675562484845
+Minimizing 9928    Time: 0:01:15 ( 7.59 ms/it)
+  objective:  13356.613038422038
+Minimizing 9941    Time: 0:01:15 ( 7.59 ms/it)
+  objective:  13356.508367548384
+Minimizing 9954    Time: 0:01:15 ( 7.59 ms/it)
+  objective:  13356.501531919435
+Minimizing 9968    Time: 0:01:15 ( 7.59 ms/it)
+  objective:  13356.470603181515
+Minimizing 9981    Time: 0:01:15 ( 7.59 ms/it)
+  objective:  13356.457082423112
+Minimizing 9995    Time: 0:01:15 ( 7.59 ms/it)
+  objective:  13356.426862269138
+Minimizing 10008    Time: 0:01:15 ( 7.59 ms/it)
+  objective:  13356.378150232973
+Minimizing 10022    Time: 0:01:16 ( 7.59 ms/it)
+  objective:  13356.373316467514
+Minimizing 10036    Time: 0:01:16 ( 7.59 ms/it)
+  objective:  13356.327605296028
+Minimizing 10050    Time: 0:01:16 ( 7.59 ms/it)
+  objective:  13356.317276413269
+Minimizing 10064    Time: 0:01:16 ( 7.59 ms/it)
+  objective:  13356.263535978207
+Minimizing 10078    Time: 0:01:16 ( 7.59 ms/it)
+  objective:  13356.251746193746
+Minimizing 10091    Time: 0:01:16 ( 7.59 ms/it)
+  objective:  13356.18602528506
+Minimizing 10105    Time: 0:01:16 ( 7.59 ms/it)
+  objective:  13356.071692888007
+Minimizing 10119    Time: 0:01:16 ( 7.59 ms/it)
+  objective:  13356.058297816235
+Minimizing 10133    Time: 0:01:16 ( 7.59 ms/it)
+  objective:  13355.980068275981
+Minimizing 10147    Time: 0:01:17 ( 7.59 ms/it)
+  objective:  13355.840684352894
+Minimizing 10160    Time: 0:01:17 ( 7.59 ms/it)
+  objective:  13355.825145810355
+Minimizing 10173    Time: 0:01:17 ( 7.59 ms/it)
+  objective:  13355.806880024276
+Minimizing 10185    Time: 0:01:17 ( 7.59 ms/it)
+  objective:  13355.7900044595
+Minimizing 10198    Time: 0:01:17 ( 7.59 ms/it)
+  objective:  13355.770173163284
+Minimizing 10211    Time: 0:01:17 ( 7.59 ms/it)
+  objective:  13355.731432885666
+Minimizing 10225    Time: 0:01:17 ( 7.60 ms/it)
+  objective:  13355.684629451498
+Minimizing 10239    Time: 0:01:17 ( 7.60 ms/it)
+  objective:  13355.639947712865
+Minimizing 10253    Time: 0:01:17 ( 7.60 ms/it)
+  objective:  13355.614689875074
+Minimizing 10267    Time: 0:01:17 ( 7.60 ms/it)
+  objective:  13355.559434175084
+Minimizing 10281    Time: 0:01:18 ( 7.60 ms/it)
+  objective:  13355.506833567335
+Minimizing 10295    Time: 0:01:18 ( 7.60 ms/it)
+  objective:  13355.493535417307
+Minimizing 10309    Time: 0:01:18 ( 7.60 ms/it)
+  objective:  13355.384513433572
+Minimizing 10323    Time: 0:01:18 ( 7.60 ms/it)
+  objective:  13355.224537038004
+Minimizing 10335    Time: 0:01:18 ( 7.60 ms/it)
+  objective:  13355.15117978509
+Minimizing 10348    Time: 0:01:18 ( 7.60 ms/it)
+  objective:  13355.119377250514
+Minimizing 10362    Time: 0:01:18 ( 7.60 ms/it)
+  objective:  13355.082610773745
+Minimizing 10375    Time: 0:01:18 ( 7.60 ms/it)
+  objective:  13355.034399186057
+Minimizing 10388    Time: 0:01:18 ( 7.60 ms/it)
+  objective:  13355.019166930098
+Minimizing 10401    Time: 0:01:19 ( 7.60 ms/it)
+  objective:  13354.954053528636
+Minimizing 10415    Time: 0:01:19 ( 7.60 ms/it)
+  objective:  13354.838118348758
+Minimizing 10428    Time: 0:01:19 ( 7.60 ms/it)
+  objective:  13354.829268505811
+Minimizing 10440    Time: 0:01:19 ( 7.60 ms/it)
+  objective:  13354.782246079063
+Minimizing 10453    Time: 0:01:19 ( 7.60 ms/it)
+  objective:  13354.692261538716
+Minimizing 10465    Time: 0:01:19 ( 7.60 ms/it)
+  objective:  13354.678863413472
+Minimizing 10477    Time: 0:01:19 ( 7.60 ms/it)
+  objective:  13354.622113764111
+Minimizing 10490    Time: 0:01:19 ( 7.60 ms/it)
+  objective:  13354.617954296547
+Minimizing 10503    Time: 0:01:19 ( 7.60 ms/it)
+  objective:  13354.568550540847
+Minimizing 10516    Time: 0:01:19 ( 7.61 ms/it)
+  objective:  13354.52995235283
+Minimizing 10529    Time: 0:01:20 ( 7.61 ms/it)
+  objective:  13354.48342526373
+Minimizing 10542    Time: 0:01:20 ( 7.61 ms/it)
+  objective:  13354.472504086138
+Minimizing 10555    Time: 0:01:20 ( 7.61 ms/it)
+  objective:  13354.431711489764
+Minimizing 10568    Time: 0:01:20 ( 7.61 ms/it)
+  objective:  13354.425110274613
+Minimizing 10581    Time: 0:01:20 ( 7.61 ms/it)
+  objective:  13354.415004948743
+Minimizing 10593    Time: 0:01:20 ( 7.61 ms/it)
+  objective:  13354.376122501155
+Minimizing 10606    Time: 0:01:20 ( 7.61 ms/it)
+  objective:  13354.258328140495
+Minimizing 10618    Time: 0:01:20 ( 7.61 ms/it)
+  objective:  13354.24409545277
+Minimizing 10631    Time: 0:01:20 ( 7.61 ms/it)
+  objective:  13354.213722589324
+Minimizing 10644    Time: 0:01:21 ( 7.61 ms/it)
+  objective:  13354.19897918767
+Minimizing 10657    Time: 0:01:21 ( 7.61 ms/it)
+  objective:  13354.17219520708
+Minimizing 10670    Time: 0:01:21 ( 7.61 ms/it)
+  objective:  13354.136471808779
+Minimizing 10683    Time: 0:01:21 ( 7.62 ms/it)
+  objective:  13354.096014953531
+Minimizing 10696    Time: 0:01:21 ( 7.62 ms/it)
+  objective:  13354.03708133263
+Minimizing 10708    Time: 0:01:21 ( 7.62 ms/it)
+  objective:  13354.03386071061
+Minimizing 10720    Time: 0:01:21 ( 7.62 ms/it)
+  objective:  13353.989327666772
+Minimizing 10733    Time: 0:01:21 ( 7.62 ms/it)
+  objective:  13353.959281073992
+Minimizing 10746    Time: 0:01:21 ( 7.62 ms/it)
+  objective:  13353.831018776487
+Minimizing 10759    Time: 0:01:21 ( 7.62 ms/it)
+  objective:  13353.726796034593
+Minimizing 10772    Time: 0:01:22 ( 7.62 ms/it)
+  objective:  13353.718224597455
+Minimizing 10785    Time: 0:01:22 ( 7.62 ms/it)
+  objective:  13353.676532591191
+Minimizing 10798    Time: 0:01:22 ( 7.62 ms/it)
+  objective:  13353.594094531705
+Minimizing 10811    Time: 0:01:22 ( 7.62 ms/it)
+  objective:  13353.458199080647
+Minimizing 10824    Time: 0:01:22 ( 7.62 ms/it)
+  objective:  13353.112552658386
+Minimizing 10836    Time: 0:01:22 ( 7.62 ms/it)
+  objective:  13353.060939681724
+Minimizing 10849    Time: 0:01:22 ( 7.63 ms/it)
+  objective:  13352.891486021508
+Minimizing 10863    Time: 0:01:22 ( 7.63 ms/it)
+  objective:  13352.846662323944
+Minimizing 10877    Time: 0:01:22 ( 7.63 ms/it)
+  objective:  13352.799414215078
+Minimizing 10891    Time: 0:01:23 ( 7.63 ms/it)
+  objective:  13352.728226970918
+Minimizing 10905    Time: 0:01:23 ( 7.63 ms/it)
+  objective:  13352.709538284384
+Minimizing 10919    Time: 0:01:23 ( 7.63 ms/it)
+  objective:  13352.591482143092
+Minimizing 10933    Time: 0:01:23 ( 7.63 ms/it)
+  objective:  13352.587567225084
+Minimizing 10947    Time: 0:01:23 ( 7.63 ms/it)
+  objective:  13352.559554775478
+Minimizing 10960    Time: 0:01:23 ( 7.63 ms/it)
+  objective:  13352.50164528464
+Minimizing 10974    Time: 0:01:23 ( 7.63 ms/it)
+  objective:  13352.496030790957
+Minimizing 10988    Time: 0:01:23 ( 7.63 ms/it)
+  objective:  13352.433264243053
+Minimizing 11001    Time: 0:01:23 ( 7.63 ms/it)
+  objective:  13352.392610200703
+Minimizing 11014    Time: 0:01:23 ( 7.63 ms/it)
+  objective:  13352.386814995974
+Minimizing 11028    Time: 0:01:24 ( 7.63 ms/it)
+  objective:  13352.369685569334
+Minimizing 11042    Time: 0:01:24 ( 7.63 ms/it)
+  objective:  13352.328664215842
+Minimizing 11056    Time: 0:01:24 ( 7.63 ms/it)
+  objective:  13352.294814843583
+Minimizing 11070    Time: 0:01:24 ( 7.63 ms/it)
+  objective:  13352.254762036617
+Minimizing 11084    Time: 0:01:24 ( 7.63 ms/it)
+  objective:  13352.209804358186
+Minimizing 11097    Time: 0:01:24 ( 7.63 ms/it)
+  objective:  13352.201201765914
+Minimizing 11111    Time: 0:01:24 ( 7.63 ms/it)
+  objective:  13352.187951210006
+Minimizing 11124    Time: 0:01:24 ( 7.63 ms/it)
+  objective:  13352.12432845711
+Minimizing 11138    Time: 0:01:24 ( 7.63 ms/it)
+  objective:  13352.113502813649
+Minimizing 11152    Time: 0:01:25 ( 7.63 ms/it)
+  objective:  13352.044690670089
+Minimizing 11165    Time: 0:01:25 ( 7.63 ms/it)
+  objective:  13351.995198748715
+Minimizing 11178    Time: 0:01:25 ( 7.63 ms/it)
+  objective:  13351.850530314987
+Minimizing 11192    Time: 0:01:25 ( 7.63 ms/it)
+  objective:  13351.781319274436
+Minimizing 11206    Time: 0:01:25 ( 7.63 ms/it)
+  objective:  13351.505125089992
+Minimizing 11219    Time: 0:01:25 ( 7.63 ms/it)
+  objective:  13351.461666813433
+Minimizing 11233    Time: 0:01:25 ( 7.63 ms/it)
+  objective:  13351.375305019152
+Minimizing 11247    Time: 0:01:25 ( 7.63 ms/it)
+  objective:  13351.268682597401
+Minimizing 11261    Time: 0:01:25 ( 7.63 ms/it)
+  objective:  13351.205823858822
+Minimizing 11275    Time: 0:01:25 ( 7.63 ms/it)
+  objective:  13351.158611692532
+Minimizing 11288    Time: 0:01:26 ( 7.63 ms/it)
+  objective:  13350.962384291226
+Minimizing 11302    Time: 0:01:26 ( 7.63 ms/it)
+  objective:  13350.932136916672
+Minimizing 11316    Time: 0:01:26 ( 7.63 ms/it)
+  objective:  13350.944612548308
+Minimizing 11330    Time: 0:01:26 ( 7.63 ms/it)
+  objective:  13350.482541061938
+Minimizing 11344    Time: 0:01:26 ( 7.63 ms/it)
+  objective:  13350.479360717298
+Minimizing 11357    Time: 0:01:26 ( 7.63 ms/it)
+  objective:  13350.434829527068
+Minimizing 11371    Time: 0:01:26 ( 7.63 ms/it)
+  objective:  13350.423136170532
+Minimizing 11385    Time: 0:01:26 ( 7.63 ms/it)
+  objective:  13350.323159644351
+Minimizing 11399    Time: 0:01:26 ( 7.63 ms/it)
+  objective:  13350.294377892336
+Minimizing 11413    Time: 0:01:27 ( 7.63 ms/it)
+  objective:  13350.259972299973
+Minimizing 11426    Time: 0:01:27 ( 7.63 ms/it)
+  objective:  13350.188672974036
+Minimizing 11440    Time: 0:01:27 ( 7.63 ms/it)
+  objective:  13350.00813112098
+Minimizing 11454    Time: 0:01:27 ( 7.63 ms/it)
+  objective:  13349.668297528224
+Minimizing 11468    Time: 0:01:27 ( 7.63 ms/it)
+  objective:  13349.4084963766
+Minimizing 11482    Time: 0:01:27 ( 7.63 ms/it)
+  objective:  13349.397833178671
+Minimizing 11495    Time: 0:01:27 ( 7.63 ms/it)
+  objective:  13349.391424331967
+Minimizing 11509    Time: 0:01:27 ( 7.63 ms/it)
+  objective:  13349.388827066301
+Minimizing 11523    Time: 0:01:27 ( 7.63 ms/it)
+  objective:  13349.32275659265
+Minimizing 11537    Time: 0:01:28 ( 7.63 ms/it)
+  objective:  13349.307151949433
+Minimizing 11551    Time: 0:01:28 ( 7.63 ms/it)
+  objective:  13349.287918451038
+Minimizing 11564    Time: 0:01:28 ( 7.63 ms/it)
+  objective:  13349.253492136515
+Minimizing 11577    Time: 0:01:28 ( 7.63 ms/it)
+  objective:  13349.231129859218
+Minimizing 11590    Time: 0:01:28 ( 7.63 ms/it)
+  objective:  13349.167494999376
+Minimizing 11603    Time: 0:01:28 ( 7.63 ms/it)
+  objective:  13349.150583653682
+Minimizing 11616    Time: 0:01:28 ( 7.63 ms/it)
+  objective:  13349.130223942819
+Minimizing 11630    Time: 0:01:28 ( 7.63 ms/it)
+  objective:  13349.04290873139
+Minimizing 11644    Time: 0:01:28 ( 7.63 ms/it)
+  objective:  13349.836668721175
+Minimizing 11657    Time: 0:01:28 ( 7.63 ms/it)
+  objective:  13348.74733190942
+Minimizing 11670    Time: 0:01:29 ( 7.63 ms/it)
+  objective:  13348.741253256907
+Minimizing 11684    Time: 0:01:29 ( 7.63 ms/it)
+  objective:  13348.713494788324
+Minimizing 11698    Time: 0:01:29 ( 7.63 ms/it)
+  objective:  13348.637519376694
+Minimizing 11712    Time: 0:01:29 ( 7.63 ms/it)
+  objective:  13348.630322087061
+Minimizing 11726    Time: 0:01:29 ( 7.63 ms/it)
+  objective:  13348.610383664243
+Minimizing 11739    Time: 0:01:29 ( 7.63 ms/it)
+  objective:  13348.639888595266
+Minimizing 11753    Time: 0:01:29 ( 7.63 ms/it)
+  objective:  13348.533745111956
+Minimizing 11766    Time: 0:01:29 ( 7.63 ms/it)
+  objective:  13348.527297904584
+Minimizing 11780    Time: 0:01:29 ( 7.63 ms/it)
+  objective:  13348.49504227205
+Minimizing 11794    Time: 0:01:30 ( 7.63 ms/it)
+  objective:  13348.481850959346
+Minimizing 11807    Time: 0:01:30 ( 7.63 ms/it)
+  objective:  13348.448432735393
+Minimizing 11821    Time: 0:01:30 ( 7.63 ms/it)
+  objective:  13348.338265718514
+Minimizing 11835    Time: 0:01:30 ( 7.63 ms/it)
+  objective:  13348.222146126012
+Minimizing 11848    Time: 0:01:30 ( 7.63 ms/it)
+  objective:  13348.21849025182
+Minimizing 11862    Time: 0:01:30 ( 7.63 ms/it)
+  objective:  13348.20912309162
+Minimizing 11875    Time: 0:01:30 ( 7.63 ms/it)
+  objective:  13348.187486413255
+Minimizing 11889    Time: 0:01:30 ( 7.63 ms/it)
+  objective:  13348.153741997252
+Minimizing 11903    Time: 0:01:30 ( 7.63 ms/it)
+  objective:  13348.126700332003
+Minimizing 11917    Time: 0:01:30 ( 7.63 ms/it)
+  objective:  13348.115368754363
+Minimizing 11931    Time: 0:01:31 ( 7.63 ms/it)
+  objective:  13348.082805935053
+Minimizing 11945    Time: 0:01:31 ( 7.63 ms/it)
+  objective:  13347.97463443457
+Minimizing 11959    Time: 0:01:31 ( 7.63 ms/it)
+  objective:  13347.89535710009
+Minimizing 11973    Time: 0:01:31 ( 7.63 ms/it)
+  objective:  13347.884248236129
+Minimizing 11987    Time: 0:01:31 ( 7.63 ms/it)
+  objective:  13347.864381234576
+Minimizing 12000    Time: 0:01:31 ( 7.63 ms/it)
+  objective:  13347.841470582796
+Minimizing 12014    Time: 0:01:31 ( 7.63 ms/it)
+  objective:  13347.780131875239
+Minimizing 12028    Time: 0:01:31 ( 7.63 ms/it)
+  objective:  13347.594960365634
+Minimizing 12042    Time: 0:01:31 ( 7.63 ms/it)
+  objective:  13347.574958612902
+Minimizing 12056    Time: 0:01:32 ( 7.63 ms/it)
+  objective:  13347.567593316824
+Minimizing 12070    Time: 0:01:32 ( 7.63 ms/it)
+  objective:  13347.55401920464
+Minimizing 12084    Time: 0:01:32 ( 7.63 ms/it)
+  objective:  13347.493032846134
+Minimizing 12098    Time: 0:01:32 ( 7.63 ms/it)
+  objective:  13347.48857371571
+Minimizing 12112    Time: 0:01:32 ( 7.63 ms/it)
+  objective:  13347.47405385167
+Minimizing 12125    Time: 0:01:32 ( 7.63 ms/it)
+  objective:  13347.41180824591
+Minimizing 12138    Time: 0:01:32 ( 7.64 ms/it)
+  objective:  13347.401719889378
+Minimizing 12152    Time: 0:01:32 ( 7.64 ms/it)
+  objective:  13347.369559940118
+Minimizing 12166    Time: 0:01:32 ( 7.64 ms/it)
+  objective:  13347.215503778454
+Minimizing 12180    Time: 0:01:32 ( 7.64 ms/it)
+  objective:  13347.202552838964
+Minimizing 12194    Time: 0:01:33 ( 7.64 ms/it)
+  objective:  13347.187421355935
+Minimizing 12207    Time: 0:01:33 ( 7.64 ms/it)
+  objective:  13347.184415022144
+Minimizing 12221    Time: 0:01:33 ( 7.64 ms/it)
+  objective:  13347.1580186075
+Minimizing 12235    Time: 0:01:33 ( 7.64 ms/it)
+  objective:  13347.136503558628
+Minimizing 12249    Time: 0:01:33 ( 7.64 ms/it)
+  objective:  13347.105996971026
+Minimizing 12262    Time: 0:01:33 ( 7.64 ms/it)
+  objective:  13347.096451258105
+Minimizing 12276    Time: 0:01:33 ( 7.64 ms/it)
+  objective:  13347.090186884801
+Minimizing 12290    Time: 0:01:33 ( 7.64 ms/it)
+  objective:  13347.025860355083
+Minimizing 12304    Time: 0:01:33 ( 7.64 ms/it)
+  objective:  13346.985643587235
+Minimizing 12318    Time: 0:01:34 ( 7.64 ms/it)
+  objective:  13346.962589302864
+Minimizing 12331    Time: 0:01:34 ( 7.64 ms/it)
+  objective:  13346.938159191901
+Minimizing 12344    Time: 0:01:34 ( 7.64 ms/it)
+  objective:  13346.927731188873
+Minimizing 12358    Time: 0:01:34 ( 7.64 ms/it)
+  objective:  13346.866783892154
+Minimizing 12372    Time: 0:01:34 ( 7.64 ms/it)
+  objective:  13346.85582349066
+Minimizing 12386    Time: 0:01:34 ( 7.64 ms/it)
+  objective:  13346.82110853323
+Minimizing 12399    Time: 0:01:34 ( 7.64 ms/it)
+  objective:  13346.764748544527
+Minimizing 12412    Time: 0:01:34 ( 7.64 ms/it)
+  objective:  13346.654866133002
+Minimizing 12423    Time: 0:01:34 ( 7.64 ms/it)
+  objective:  13346.48642662655
+Minimizing 12436    Time: 0:01:35 ( 7.64 ms/it)
+  objective:  13346.748483082294
+Minimizing 12450    Time: 0:01:35 ( 7.64 ms/it)
+  objective:  13345.628570482884
+Minimizing 12465    Time: 0:01:35 ( 7.64 ms/it)
+  objective:  13345.622866117956
+Minimizing 12480    Time: 0:01:35 ( 7.64 ms/it)
+  objective:  13345.622864899582
+Minimizing 12495    Time: 0:01:35 ( 7.64 ms/it)
+  objective:  13345.613333107416
+Minimizing 12510    Time: 0:01:35 ( 7.64 ms/it)
+  objective:  13345.6076133643
+Minimizing 12525    Time: 0:01:35 ( 7.64 ms/it)
+  objective:  13345.840923904347
+Minimizing 12539    Time: 0:01:35 ( 7.64 ms/it)
+  objective:  13345.546683230008
+Minimizing 12554    Time: 0:01:35 ( 7.64 ms/it)
+  objective:  13345.520414555409
+Minimizing 12568    Time: 0:01:35 ( 7.64 ms/it)
+  objective:  13345.482579864685
+Minimizing 12582    Time: 0:01:36 ( 7.64 ms/it)
+  objective:  13345.453370938521
+Minimizing 12596    Time: 0:01:36 ( 7.64 ms/it)
+  objective:  13345.414262193037
+Minimizing 12610    Time: 0:01:36 ( 7.64 ms/it)
+  objective:  13345.40551057463
+Minimizing 12624    Time: 0:01:36 ( 7.64 ms/it)
+  objective:  13345.330549550505
+Minimizing 12638    Time: 0:01:36 ( 7.64 ms/it)
+  objective:  13345.322551443474
+Minimizing 12652    Time: 0:01:36 ( 7.63 ms/it)
+  objective:  13345.309063311943
+Minimizing 12666    Time: 0:01:36 ( 7.63 ms/it)
+  objective:  13345.28478984227
+Minimizing 12680    Time: 0:01:36 ( 7.64 ms/it)
+  objective:  13345.271494036868
+Minimizing 12693    Time: 0:01:36 ( 7.64 ms/it)
+  objective:  13345.241204354403
+Minimizing 12706    Time: 0:01:37 ( 7.64 ms/it)
+  objective:  13345.151447869328
+Minimizing 12719    Time: 0:01:37 ( 7.64 ms/it)
+  objective:  13345.148512951804
+Minimizing 12732    Time: 0:01:37 ( 7.64 ms/it)
+  objective:  13345.144331022173
+Minimizing 12746    Time: 0:01:37 ( 7.64 ms/it)
+  objective:  13345.141916181929
+Minimizing 12760    Time: 0:01:37 ( 7.64 ms/it)
+  objective:  13345.128527242618
+Minimizing 12774    Time: 0:01:37 ( 7.64 ms/it)
+  objective:  13345.113935825902
+Minimizing 12788    Time: 0:01:37 ( 7.64 ms/it)
+  objective:  13345.108617722697
+Minimizing 12802    Time: 0:01:37 ( 7.64 ms/it)
+  objective:  13345.095125132742
+Minimizing 12816    Time: 0:01:37 ( 7.64 ms/it)
+  objective:  13345.082263493765
+Minimizing 12830    Time: 0:01:37 ( 7.63 ms/it)
+  objective:  13345.075697140193
+Minimizing 12844    Time: 0:01:38 ( 7.63 ms/it)
+  objective:  13345.039722544592
+Minimizing 12858    Time: 0:01:38 ( 7.63 ms/it)
+  objective:  13345.037761111911
+Minimizing 12872    Time: 0:01:38 ( 7.63 ms/it)
+  objective:  13345.1666793997
+Minimizing 12886    Time: 0:01:38 ( 7.63 ms/it)
+  objective:  13344.92859330037
+Minimizing 12900    Time: 0:01:38 ( 7.63 ms/it)
+  objective:  13344.925149030074
+Minimizing 12914    Time: 0:01:38 ( 7.63 ms/it)
+  objective:  13344.89630675107
+Minimizing 12928    Time: 0:01:38 ( 7.63 ms/it)
+  objective:  13344.892204139382
+Minimizing 12942    Time: 0:01:38 ( 7.63 ms/it)
+  objective:  13344.886592941577
+Minimizing 12956    Time: 0:01:38 ( 7.63 ms/it)
+  objective:  13344.872839413656
+Minimizing 12970    Time: 0:01:38 ( 7.63 ms/it)
+  objective:  13344.874315630397
+Minimizing 12984    Time: 0:01:39 ( 7.63 ms/it)
+  objective:  13344.79732289783
+Minimizing 12997    Time: 0:01:39 ( 7.63 ms/it)
+  objective:  13344.767966937921
+Minimizing 13010    Time: 0:01:39 ( 7.63 ms/it)
+  objective:  13344.763125665617
+Minimizing 13024    Time: 0:01:39 ( 7.63 ms/it)
+  objective:  13344.753555207179
+Minimizing 13038    Time: 0:01:39 ( 7.63 ms/it)
+  objective:  13344.736951753956
+Minimizing 13052    Time: 0:01:39 ( 7.63 ms/it)
+  objective:  13344.724748820467
+Minimizing 13065    Time: 0:01:39 ( 7.63 ms/it)
+  objective:  13344.667618019303
+Minimizing 13079    Time: 0:01:39 ( 7.63 ms/it)
+  objective:  13344.588659589892
+Minimizing 13093    Time: 0:01:39 ( 7.63 ms/it)
+  objective:  13344.576174037065
+Minimizing 13107    Time: 0:01:40 ( 7.63 ms/it)
+  objective:  13344.562048646847
+Minimizing 13121    Time: 0:01:40 ( 7.63 ms/it)
+  objective:  13344.48850606852
+Minimizing 13135    Time: 0:01:40 ( 7.63 ms/it)
+  objective:  13344.479955785238
+Minimizing 13149    Time: 0:01:40 ( 7.63 ms/it)
+  objective:  13344.459435019693
+Minimizing 13163    Time: 0:01:40 ( 7.63 ms/it)
+  objective:  13344.394226295124
+Minimizing 13177    Time: 0:01:40 ( 7.63 ms/it)
+  objective:  13344.38527770467
+Minimizing 13191    Time: 0:01:40 ( 7.63 ms/it)
+  objective:  13344.352889354923
+Minimizing 13205    Time: 0:01:40 ( 7.63 ms/it)
+  objective:  13344.332406485839
+Minimizing 13219    Time: 0:01:40 ( 7.63 ms/it)
+  objective:  13344.307621613509
+Minimizing 13233    Time: 0:01:40 ( 7.63 ms/it)
+  objective:  13344.23706030583
+Minimizing 13247    Time: 0:01:41 ( 7.63 ms/it)
+  objective:  13344.03560074504
+Minimizing 13261    Time: 0:01:41 ( 7.63 ms/it)
+  objective:  13344.00726643725
+Minimizing 13274    Time: 0:01:41 ( 7.63 ms/it)
+  objective:  13343.991815181456
+Minimizing 13288    Time: 0:01:41 ( 7.63 ms/it)
+  objective:  13343.967246389439
+Minimizing 13302    Time: 0:01:41 ( 7.63 ms/it)
+  objective:  13343.947982783458
+Minimizing 13316    Time: 0:01:41 ( 7.63 ms/it)
+  objective:  13343.898241744522
+Minimizing 13330    Time: 0:01:41 ( 7.63 ms/it)
+  objective:  13343.892410396104
+Minimizing 13343    Time: 0:01:41 ( 7.63 ms/it)
+  objective:  13343.857907028098
+Minimizing 13355    Time: 0:01:41 ( 7.63 ms/it)
+  objective:  13343.856922619932
+Minimizing 13369    Time: 0:01:42 ( 7.63 ms/it)
+  objective:  13343.806901450924
+Minimizing 13383    Time: 0:01:42 ( 7.63 ms/it)
+  objective:  13343.742183542854
+Minimizing 13397    Time: 0:01:42 ( 7.63 ms/it)
+  objective:  13343.588368684308
+Minimizing 13410    Time: 0:01:42 ( 7.63 ms/it)
+  objective:  13343.520113863007
+Minimizing 13423    Time: 0:01:42 ( 7.63 ms/it)
+  objective:  13343.515912747447
+Minimizing 13436    Time: 0:01:42 ( 7.63 ms/it)
+  objective:  13343.50793322963
+Minimizing 13450    Time: 0:01:42 ( 7.63 ms/it)
+  objective:  13343.503540939193
+Minimizing 13465    Time: 0:01:42 ( 7.63 ms/it)
+  objective:  13343.496950676708
+Minimizing 13478    Time: 0:01:42 ( 7.63 ms/it)
+  objective:  13343.449574027218
+Minimizing 13492    Time: 0:01:42 ( 7.63 ms/it)
+  objective:  13343.446848708649
+Minimizing 13506    Time: 0:01:43 ( 7.63 ms/it)
+  objective:  13343.43172562078
+Minimizing 13520    Time: 0:01:43 ( 7.63 ms/it)
+  objective:  13343.409884976587
+Minimizing 13534    Time: 0:01:43 ( 7.63 ms/it)
+  objective:  13343.384692148713
+Minimizing 13548    Time: 0:01:43 ( 7.63 ms/it)
+  objective:  13343.340063787291
+Minimizing 13562    Time: 0:01:43 ( 7.63 ms/it)
+  objective:  13343.326074487719
+Minimizing 13577    Time: 0:01:43 ( 7.63 ms/it)
+  objective:  13344.274033716938
+Minimizing 13592    Time: 0:01:43 ( 7.63 ms/it)
+  objective:  13342.755559848862
+Minimizing 13606    Time: 0:01:43 ( 7.63 ms/it)
+  objective:  13342.752462506818
+Minimizing 13620    Time: 0:01:43 ( 7.63 ms/it)
+  objective:  13342.751353947933
+Minimizing 13634    Time: 0:01:43 ( 7.63 ms/it)
+  objective:  13342.750873648554
+Minimizing 13648    Time: 0:01:44 ( 7.63 ms/it)
+  objective:  13342.74727714977
+Minimizing 13662    Time: 0:01:44 ( 7.63 ms/it)
+  objective:  13342.741142379062
+Minimizing 13676    Time: 0:01:44 ( 7.63 ms/it)
+  objective:  13342.69873160727
+Minimizing 13690    Time: 0:01:44 ( 7.63 ms/it)
+  objective:  13342.63023211118
+Minimizing 13704    Time: 0:01:44 ( 7.63 ms/it)
+  objective:  13342.629421813945
+Minimizing 13718    Time: 0:01:44 ( 7.63 ms/it)
+  objective:  13342.623464409451
+Minimizing 13733    Time: 0:01:44 ( 7.62 ms/it)
+  objective:  13342.611700424197
+Minimizing 13748    Time: 0:01:44 ( 7.62 ms/it)
+  objective:  13342.609370531165
+Minimizing 13763    Time: 0:01:44 ( 7.62 ms/it)
+  objective:  13342.603192153394
+Minimizing 13777    Time: 0:01:45 ( 7.62 ms/it)
+  objective:  13342.596185176415
+Minimizing 13791    Time: 0:01:45 ( 7.62 ms/it)
+  objective:  13342.5587861093
+Minimizing 13805    Time: 0:01:45 ( 7.62 ms/it)
+  objective:  13342.558206853093
+Minimizing 13819    Time: 0:01:45 ( 7.62 ms/it)
+  objective:  13342.556372652864
+Minimizing 13833    Time: 0:01:45 ( 7.62 ms/it)
+  objective:  13342.546611815378
+Minimizing 13847    Time: 0:01:45 ( 7.62 ms/it)
+  objective:  13342.532979705124
+Minimizing 13861    Time: 0:01:45 ( 7.62 ms/it)
+  objective:  13342.501951278515
+Minimizing 13875    Time: 0:01:45 ( 7.62 ms/it)
+  objective:  13342.438715714576
+Minimizing 13889    Time: 0:01:45 ( 7.62 ms/it)
+  objective:  13342.395686358097
+Minimizing 13903    Time: 0:01:45 ( 7.62 ms/it)
+  objective:  13342.384038808741
+Minimizing 13917    Time: 0:01:46 ( 7.62 ms/it)
+  objective:  13342.368030303965
+Minimizing 13931    Time: 0:01:46 ( 7.62 ms/it)
+  objective:  13342.34443888953
+Minimizing 13945    Time: 0:01:46 ( 7.62 ms/it)
+  objective:  13342.34313225605
+Minimizing 13959    Time: 0:01:46 ( 7.62 ms/it)
+  objective:  13342.334988698909
+Minimizing 13974    Time: 0:01:46 ( 7.62 ms/it)
+  objective:  13342.311733475282
+Minimizing 13989    Time: 0:01:46 ( 7.62 ms/it)
+  objective:  13342.289983820163
+Minimizing 14003    Time: 0:01:46 ( 7.62 ms/it)
+  objective:  13342.246444806078
+Minimizing 14017    Time: 0:01:46 ( 7.62 ms/it)
+  objective:  13342.244921382386
+Minimizing 14031    Time: 0:01:46 ( 7.62 ms/it)
+  objective:  13342.24441355453
+Minimizing 14045    Time: 0:01:47 ( 7.62 ms/it)
+  objective:  13342.218828023273
+Minimizing 14059    Time: 0:01:47 ( 7.62 ms/it)
+  objective:  13342.17934070385
+Minimizing 14073    Time: 0:01:47 ( 7.62 ms/it)
+  objective:  13342.06056912605
+Minimizing 14087    Time: 0:01:47 ( 7.62 ms/it)
+  objective:  13342.05049441877
+Minimizing 14101    Time: 0:01:47 ( 7.62 ms/it)
+  objective:  13342.026868573339
+Minimizing 14115    Time: 0:01:47 ( 7.62 ms/it)
+  objective:  13342.002058981561
+Minimizing 14130    Time: 0:01:47 ( 7.62 ms/it)
+  objective:  13342.001102257585
+Minimizing 14144    Time: 0:01:47 ( 7.62 ms/it)
+  objective:  13342.009124869204
+Minimizing 14158    Time: 0:01:47 ( 7.62 ms/it)
+  objective:  13341.959557608854
+Minimizing 14172    Time: 0:01:47 ( 7.62 ms/it)
+  objective:  13341.94567407461
+Minimizing 14185    Time: 0:01:48 ( 7.62 ms/it)
+  objective:  13341.928842195688
+Minimizing 14199    Time: 0:01:48 ( 7.62 ms/it)
+  objective:  13341.876274474751
+Minimizing 14213    Time: 0:01:48 ( 7.62 ms/it)
+  objective:  13341.87338260821
+Minimizing 14227    Time: 0:01:48 ( 7.62 ms/it)
+  objective:  13341.859648768921
+Minimizing 14241    Time: 0:01:48 ( 7.62 ms/it)
+  objective:  13341.833293218537
+Minimizing 14255    Time: 0:01:48 ( 7.62 ms/it)
+  objective:  13341.823698525688
+Minimizing 14268    Time: 0:01:48 ( 7.62 ms/it)
+  objective:  13341.779450060836
+Minimizing 14280    Time: 0:01:48 ( 7.62 ms/it)
+  objective:  13341.778788315685
+Minimizing 14294    Time: 0:01:48 ( 7.62 ms/it)
+  objective:  13341.749770237446
+Minimizing 14308    Time: 0:01:49 ( 7.62 ms/it)
+  objective:  13341.728708525225
+Minimizing 14322    Time: 0:01:49 ( 7.62 ms/it)
+  objective:  13341.718182395067
+Minimizing 14335    Time: 0:01:49 ( 7.62 ms/it)
+  objective:  13341.692137467515
+Minimizing 14348    Time: 0:01:49 ( 7.62 ms/it)
+  objective:  13341.679060573493
+Minimizing 14361    Time: 0:01:49 ( 7.62 ms/it)
+  objective:  13341.658373906903
+Minimizing 14374    Time: 0:01:49 ( 7.62 ms/it)
+  objective:  13341.634779269712
+Minimizing 14387    Time: 0:01:49 ( 7.62 ms/it)
+  objective:  13341.605974389611
+Minimizing 14401    Time: 0:01:49 ( 7.62 ms/it)
+  objective:  13341.482110165569
+Minimizing 14416    Time: 0:01:49 ( 7.62 ms/it)
+  objective:  13341.362884887116
+Minimizing 14431    Time: 0:01:49 ( 7.62 ms/it)
+  objective:  13341.361616337701
+Minimizing 14446    Time: 0:01:50 ( 7.62 ms/it)
+  objective:  13341.328550239748
+Minimizing 14461    Time: 0:01:50 ( 7.62 ms/it)
+  objective:  13341.318476155859
+Minimizing 14476    Time: 0:01:50 ( 7.62 ms/it)
+  objective:  13341.28721405701
+Minimizing 14491    Time: 0:01:50 ( 7.62 ms/it)
+  objective:  13341.206464117902
+Minimizing 14506    Time: 0:01:50 ( 7.61 ms/it)
+  objective:  13341.125584964371
+Minimizing 14521    Time: 0:01:50 ( 7.61 ms/it)
+  objective:  13341.124352856175
+Minimizing 14536    Time: 0:01:50 ( 7.61 ms/it)
+  objective:  13341.116067249786
+Minimizing 14551    Time: 0:01:50 ( 7.61 ms/it)
+  objective:  13341.108300017237
+Minimizing 14566    Time: 0:01:50 ( 7.61 ms/it)
+  objective:  13341.100408753897
+Minimizing 14581    Time: 0:01:50 ( 7.61 ms/it)
+  objective:  13341.054358721369
+Minimizing 14596    Time: 0:01:51 ( 7.61 ms/it)
+  objective:  13340.997305768331
+Minimizing 14611    Time: 0:01:51 ( 7.61 ms/it)
+  objective:  13340.989243322867
+Minimizing 14626    Time: 0:01:51 ( 7.61 ms/it)
+  objective:  13340.904651490418
+Minimizing 14641    Time: 0:01:51 ( 7.61 ms/it)
+  objective:  13340.85213009418
+Minimizing 14656    Time: 0:01:51 ( 7.61 ms/it)
+  objective:  13340.788736080816
+Minimizing 14671    Time: 0:01:51 ( 7.61 ms/it)
+  objective:  13340.631001280948
+Minimizing 14686    Time: 0:01:51 ( 7.61 ms/it)
+  objective:  13340.098106181962
+Minimizing 14699    Time: 0:01:51 ( 7.61 ms/it)
+  objective:  13339.802541688165
+Minimizing 14712    Time: 0:01:51 ( 7.61 ms/it)
+  objective:  13339.7996311221
+Minimizing 14726    Time: 0:01:52 ( 7.61 ms/it)
+  objective:  13339.786178103866
+Minimizing 14739    Time: 0:01:52 ( 7.61 ms/it)
+  objective:  13339.783297457383
+Minimizing 14754    Time: 0:01:52 ( 7.61 ms/it)
+  objective:  13339.746047789711
+Minimizing 14768    Time: 0:01:52 ( 7.61 ms/it)
+  objective:  13339.731428983258
+Minimizing 14782    Time: 0:01:52 ( 7.61 ms/it)
+  objective:  13339.67051133667
+Minimizing 14796    Time: 0:01:52 ( 7.61 ms/it)
+  objective:  13339.617804636975
+Minimizing 14810    Time: 0:01:52 ( 7.61 ms/it)
+  objective:  13339.538325454938
+Minimizing 14824    Time: 0:01:52 ( 7.61 ms/it)
+  objective:  13339.414503420601
+Minimizing 14839    Time: 0:01:52 ( 7.61 ms/it)
+  objective:  13339.408859688148
+Minimizing 14853    Time: 0:01:52 ( 7.61 ms/it)
+  objective:  13339.390946794207
+Minimizing 14867    Time: 0:01:53 ( 7.61 ms/it)
+  objective:  13339.378136657353
+Minimizing 14881    Time: 0:01:53 ( 7.61 ms/it)
+  objective:  13339.367340318568
+Minimizing 14892    Time: 0:01:53 ( 7.61 ms/it)
+  objective:  13339.354375949617
+Minimizing 14905    Time: 0:01:53 ( 7.61 ms/it)
+  objective:  13339.322476467445
+Minimizing 14918    Time: 0:01:53 ( 7.61 ms/it)
+  objective:  13339.22538221386
+Minimizing 14931    Time: 0:01:53 ( 7.61 ms/it)
+  objective:  13339.106615442135
+Minimizing 14944    Time: 0:01:53 ( 7.61 ms/it)
+  objective:  13339.093225269928
+Minimizing 14958    Time: 0:01:53 ( 7.61 ms/it)
+  objective:  13339.063518638402
+Minimizing 14973    Time: 0:01:53 ( 7.61 ms/it)
+  objective:  13339.046285189797
+Minimizing 14987    Time: 0:01:54 ( 7.61 ms/it)
+  objective:  13339.038613203244
+Minimizing 15002    Time: 0:01:54 ( 7.61 ms/it)
+  objective:  13338.997378755244
+Minimizing 15017    Time: 0:01:54 ( 7.61 ms/it)
+  objective:  13338.932883187394
+Minimizing 15030    Time: 0:01:54 ( 7.61 ms/it)
+  objective:  13339.051062597959
+Minimizing 15043    Time: 0:01:54 ( 7.61 ms/it)
+  objective:  13338.621232862097
+Minimizing 15056    Time: 0:01:54 ( 7.61 ms/it)
+  objective:  13338.617861841056
+Minimizing 15069    Time: 0:01:54 ( 7.61 ms/it)
+  objective:  13338.598513041637
+Minimizing 15082    Time: 0:01:54 ( 7.61 ms/it)
+  objective:  13338.590907214624
+Minimizing 15095    Time: 0:01:54 ( 7.61 ms/it)
+  objective:  13338.571448459938
+Minimizing 15109    Time: 0:01:54 ( 7.61 ms/it)
+  objective:  13338.55328101055
+Minimizing 15123    Time: 0:01:55 ( 7.61 ms/it)
+  objective:  13338.54889314367
+Minimizing 15137    Time: 0:01:55 ( 7.61 ms/it)
+  objective:  13338.452531246578
+Minimizing 15151    Time: 0:01:55 ( 7.61 ms/it)
+  objective:  13338.433243870313
+Minimizing 15165    Time: 0:01:55 ( 7.61 ms/it)
+  objective:  13338.416701821036
+Minimizing 15179    Time: 0:01:55 ( 7.61 ms/it)
+  objective:  13338.404845290723
+Minimizing 15193    Time: 0:01:55 ( 7.61 ms/it)
+  objective:  13338.385556007946
+Minimizing 15207    Time: 0:01:55 ( 7.61 ms/it)
+  objective:  13338.383354799196
+Minimizing 15221    Time: 0:01:55 ( 7.61 ms/it)
+  objective:  13338.37973805867
+Minimizing 15236    Time: 0:01:55 ( 7.61 ms/it)
+  objective:  13338.363546701294
+Minimizing 15251    Time: 0:01:56 ( 7.61 ms/it)
+  objective:  13338.349669530842
+Minimizing 15266    Time: 0:01:56 ( 7.61 ms/it)
+  objective:  13338.316654233975
+Minimizing 15281    Time: 0:01:56 ( 7.61 ms/it)
+  objective:  13338.312177241954
+Minimizing 15296    Time: 0:01:56 ( 7.61 ms/it)
+  objective:  13338.291933278197
+Minimizing 15310    Time: 0:01:56 ( 7.61 ms/it)
+  objective:  13338.186993749754
+Minimizing 15324    Time: 0:01:56 ( 7.61 ms/it)
+  objective:  13338.00936621119
+Minimizing 15338    Time: 0:01:56 ( 7.61 ms/it)
+  objective:  13337.995048239733
+Minimizing 15352    Time: 0:01:56 ( 7.61 ms/it)
+  objective:  13337.987054526275
+Minimizing 15366    Time: 0:01:56 ( 7.61 ms/it)
+  objective:  13337.976820763608
+Minimizing 15379    Time: 0:01:56 ( 7.61 ms/it)
+  objective:  13337.960073338872
+Minimizing 15394    Time: 0:01:57 ( 7.61 ms/it)
+  objective:  13337.935898769712
+Minimizing 15409    Time: 0:01:57 ( 7.61 ms/it)
+  objective:  13337.924520801054
+Minimizing 15423    Time: 0:01:57 ( 7.61 ms/it)
+  objective:  13337.921618246626
+Minimizing 15438    Time: 0:01:57 ( 7.61 ms/it)
+  objective:  13337.91258408071
+Minimizing 15453    Time: 0:01:57 ( 7.60 ms/it)
+  objective:  13337.893248043074
+Minimizing 15468    Time: 0:01:57 ( 7.60 ms/it)
+  objective:  13337.866187868603
+Minimizing 15482    Time: 0:01:57 ( 7.60 ms/it)
+  objective:  13337.82368052403
+Minimizing 15497    Time: 0:01:57 ( 7.60 ms/it)
+  objective:  13337.802456079604
+Minimizing 15512    Time: 0:01:57 ( 7.60 ms/it)
+  objective:  13337.778319405399
+Minimizing 15527    Time: 0:01:58 ( 7.60 ms/it)
+  objective:  13337.723085633355
+Minimizing 15542    Time: 0:01:58 ( 7.60 ms/it)
+  objective:  13337.681336951027
+Minimizing 15557    Time: 0:01:58 ( 7.60 ms/it)
+  objective:  13337.623714578207
+Minimizing 15571    Time: 0:01:58 ( 7.60 ms/it)
+  objective:  13337.620116607162
+Minimizing 15585    Time: 0:01:58 ( 7.60 ms/it)
+  objective:  13337.60039693924
+Minimizing 15599    Time: 0:01:58 ( 7.60 ms/it)
+  objective:  13337.53667073154
+Minimizing 15613    Time: 0:01:58 ( 7.60 ms/it)
+  objective:  13337.511657091229
+Minimizing 15628    Time: 0:01:58 ( 7.60 ms/it)
+  objective:  13337.37685004936
+Minimizing 15643    Time: 0:01:58 ( 7.60 ms/it)
+  objective:  13337.256223656761
+Minimizing 15657    Time: 0:01:58 ( 7.60 ms/it)
+  objective:  13337.224781952937
+Minimizing 15671    Time: 0:01:59 ( 7.60 ms/it)
+  objective:  13337.216337961538
+Minimizing 15686    Time: 0:01:59 ( 7.60 ms/it)
+  objective:  13337.19972683298
+Minimizing 15701    Time: 0:01:59 ( 7.60 ms/it)
+  objective:  13337.18508632273
+Minimizing 15716    Time: 0:01:59 ( 7.60 ms/it)
+  objective:  13337.110989585926
+Minimizing 15731    Time: 0:01:59 ( 7.60 ms/it)
+  objective:  13337.00356813491
+Minimizing 15746    Time: 0:01:59 ( 7.59 ms/it)
+  objective:  13336.990551020863
+Minimizing 15761    Time: 0:01:59 ( 7.59 ms/it)
+  objective:  13336.94006346217
+Minimizing 15776    Time: 0:01:59 ( 7.59 ms/it)
+  objective:  13336.933684914082
+Minimizing 15791    Time: 0:01:59 ( 7.59 ms/it)
+  objective:  13336.913998128693
+Minimizing 15806    Time: 0:02:00 ( 7.59 ms/it)
+  objective:  13336.87868722422
+Minimizing 15820    Time: 0:02:00 ( 7.59 ms/it)
+  objective:  13336.82005116382
+Minimizing 15834    Time: 0:02:00 ( 7.59 ms/it)
+  objective:  13336.70335415849
+Minimizing 15848    Time: 0:02:00 ( 7.59 ms/it)
+  objective:  13336.304934579377
+Minimizing 15861    Time: 0:02:00 ( 7.59 ms/it)
+  objective:  13336.26960745341
+Minimizing 15875    Time: 0:02:00 ( 7.59 ms/it)
+  objective:  13336.25961314018
+Minimizing 15890    Time: 0:02:00 ( 7.59 ms/it)
+  objective:  13336.257396265588
+Minimizing 15905    Time: 0:02:00 ( 7.59 ms/it)
+  objective:  13336.254043617424
+Minimizing 15919    Time: 0:02:00 ( 7.59 ms/it)
+  objective:  13336.246404504054
+Minimizing 15933    Time: 0:02:00 ( 7.59 ms/it)
+  objective:  13336.24520488202
+Minimizing 15945    Time: 0:02:01 ( 7.59 ms/it)
+  objective:  13336.232147575625
+Minimizing 15957    Time: 0:02:01 ( 7.59 ms/it)
+  objective:  13336.212331401934
+Minimizing 15969    Time: 0:02:01 ( 7.60 ms/it)
+  objective:  13336.1841625686
+Minimizing 15981    Time: 0:02:01 ( 7.60 ms/it)
+  objective:  13336.181296923438
+Minimizing 15993    Time: 0:02:01 ( 7.60 ms/it)
+  objective:  13336.156511995301
+Minimizing 16005    Time: 0:02:01 ( 7.60 ms/it)
+  objective:  13336.14404342165
+Minimizing 16017    Time: 0:02:01 ( 7.60 ms/it)
+  objective:  13336.022815213975
+Minimizing 16029    Time: 0:02:01 ( 7.60 ms/it)
+  objective:  13335.92158471718
+Minimizing 16044    Time: 0:02:01 ( 7.60 ms/it)
+  objective:  13335.919924256574
+Minimizing 16059    Time: 0:02:02 ( 7.60 ms/it)
+  objective:  13335.89393574206
+Minimizing 16074    Time: 0:02:02 ( 7.60 ms/it)
+  objective:  13335.844296660238
+Minimizing 16089    Time: 0:02:02 ( 7.60 ms/it)
+  objective:  13335.746457684778
+Minimizing 16104    Time: 0:02:02 ( 7.60 ms/it)
+  objective:  13335.745703669192
+Minimizing 16119    Time: 0:02:02 ( 7.60 ms/it)
+  objective:  13335.701548804107
+Minimizing 16133    Time: 0:02:02 ( 7.60 ms/it)
+  objective:  13335.626890877931
+Minimizing 16147    Time: 0:02:02 ( 7.60 ms/it)
+  objective:  13335.619251927536
+Minimizing 16161    Time: 0:02:02 ( 7.60 ms/it)
+  objective:  13335.549841101958
+Minimizing 16175    Time: 0:02:02 ( 7.60 ms/it)
+  objective:  13335.410669319033
+Minimizing 16189    Time: 0:02:02 ( 7.60 ms/it)
+  objective:  13335.223710334918
+Minimizing 16204    Time: 0:02:03 ( 7.60 ms/it)
+  objective:  13335.220345227215
+Minimizing 16219    Time: 0:02:03 ( 7.59 ms/it)
+  objective:  13335.218919295177
+Minimizing 16234    Time: 0:02:03 ( 7.59 ms/it)
+  objective:  13335.218114378069
+Minimizing 16249    Time: 0:02:03 ( 7.59 ms/it)
+  objective:  13335.21483029914
+Minimizing 16263    Time: 0:02:03 ( 7.59 ms/it)
+  objective:  13335.196398677857
+Minimizing 16277    Time: 0:02:03 ( 7.59 ms/it)
+  objective:  13335.12331996901
+Minimizing 16291    Time: 0:02:03 ( 7.59 ms/it)
+  objective:  13335.122550750682
+Minimizing 16305    Time: 0:02:03 ( 7.59 ms/it)
+  objective:  13335.120941846872
+Minimizing 16319    Time: 0:02:03 ( 7.59 ms/it)
+  objective:  13335.111804640095
+Minimizing 16333    Time: 0:02:04 ( 7.59 ms/it)
+  objective:  13335.10552670512
+Minimizing 16347    Time: 0:02:04 ( 7.59 ms/it)
+  objective:  13335.09730867549
+Minimizing 16361    Time: 0:02:04 ( 7.59 ms/it)
+  objective:  13335.08002516294
+Minimizing 16375    Time: 0:02:04 ( 7.59 ms/it)
+  objective:  13335.062649100168
+Minimizing 16390    Time: 0:02:04 ( 7.59 ms/it)
+  objective:  13335.003975012754
+Minimizing 16405    Time: 0:02:04 ( 7.59 ms/it)
+  objective:  13334.9908595706
+Minimizing 16419    Time: 0:02:04 ( 7.59 ms/it)
+  objective:  13334.923182506012
+Minimizing 16433    Time: 0:02:04 ( 7.59 ms/it)
+  objective:  13334.84935115552
+Minimizing 16447    Time: 0:02:04 ( 7.59 ms/it)
+  objective:  13334.843690938163
+Minimizing 16461    Time: 0:02:04 ( 7.59 ms/it)
+  objective:  13334.837000614687
+Minimizing 16475    Time: 0:02:05 ( 7.59 ms/it)
+  objective:  13334.827620651718
+Minimizing 16489    Time: 0:02:05 ( 7.59 ms/it)
+  objective:  13334.808317676594
+Minimizing 16503    Time: 0:02:05 ( 7.59 ms/it)
+  objective:  13334.803130001128
+Minimizing 16517    Time: 0:02:05 ( 7.59 ms/it)
+  objective:  13334.778763942137
+Minimizing 16531    Time: 0:02:05 ( 7.59 ms/it)
+  objective:  13334.752197105197
+Minimizing 16545    Time: 0:02:05 ( 7.59 ms/it)
+  objective:  13334.73026654141
+Minimizing 16560    Time: 0:02:05 ( 7.59 ms/it)
+  objective:  13334.667550521219
+Minimizing 16575    Time: 0:02:05 ( 7.59 ms/it)
+  objective:  13334.583685550962
+Minimizing 16590    Time: 0:02:05 ( 7.59 ms/it)
+  objective:  13334.58317779962
+Minimizing 16605    Time: 0:02:05 ( 7.59 ms/it)
+  objective:  13334.57324292342
+Minimizing 16620    Time: 0:02:06 ( 7.59 ms/it)
+  objective:  13334.567395581005
+Minimizing 16635    Time: 0:02:06 ( 7.58 ms/it)
+  objective:  13334.545578969643
+Minimizing 16650    Time: 0:02:06 ( 7.58 ms/it)
+  objective:  13334.543618434232
+Minimizing 16665    Time: 0:02:06 ( 7.58 ms/it)
+  objective:  13334.50757851838
+Minimizing 16680    Time: 0:02:06 ( 7.58 ms/it)
+  objective:  13334.475236578328
+Minimizing 16695    Time: 0:02:06 ( 7.58 ms/it)
+  objective:  13334.42317210642
+Minimizing 16710    Time: 0:02:06 ( 7.58 ms/it)
+  objective:  13334.364084989596
+Minimizing 16724    Time: 0:02:06 ( 7.58 ms/it)
+  objective:  13334.360189547035
+Minimizing 16739    Time: 0:02:06 ( 7.58 ms/it)
+  objective:  13334.341324531386
+Minimizing 16754    Time: 0:02:07 ( 7.58 ms/it)
+  objective:  13334.31600599354
+Minimizing 16769    Time: 0:02:07 ( 7.58 ms/it)
+  objective:  13334.305818915644
+Minimizing 16784    Time: 0:02:07 ( 7.58 ms/it)
+  objective:  13334.218941403124
+Minimizing 16799    Time: 0:02:07 ( 7.58 ms/it)
+  objective:  13334.08812966587
+Minimizing 16814    Time: 0:02:07 ( 7.58 ms/it)
+  objective:  13334.054647581615
+Minimizing 16829    Time: 0:02:07 ( 7.58 ms/it)
+  objective:  13334.051145102552
+Minimizing 16844    Time: 0:02:07 ( 7.58 ms/it)
+  objective:  13334.022545259388
+Minimizing 16859    Time: 0:02:07 ( 7.58 ms/it)
+  objective:  13334.013131942163
+Minimizing 16873    Time: 0:02:07 ( 7.58 ms/it)
+  objective:  13333.988855241048
+Minimizing 16887    Time: 0:02:07 ( 7.58 ms/it)
+  objective:  13333.895507994574
+Minimizing 16901    Time: 0:02:08 ( 7.58 ms/it)
+  objective:  13333.861465142676
+Minimizing 16914    Time: 0:02:08 ( 7.58 ms/it)
+  objective:  13333.846570221343
+Minimizing 16928    Time: 0:02:08 ( 7.58 ms/it)
+  objective:  13333.844384498356
+Minimizing 16943    Time: 0:02:08 ( 7.58 ms/it)
+  objective:  13333.841598127896
+Minimizing 16958    Time: 0:02:08 ( 7.58 ms/it)
+  objective:  13333.838511294707
+Minimizing 16973    Time: 0:02:08 ( 7.58 ms/it)
+  objective:  13333.831727143472
+Minimizing 16987    Time: 0:02:08 ( 7.58 ms/it)
+  objective:  13333.761543254921
+Minimizing 17001    Time: 0:02:08 ( 7.58 ms/it)
+  objective:  13333.759007582317
+Minimizing 17016    Time: 0:02:08 ( 7.57 ms/it)
+  objective:  13333.750043274238
+Minimizing 17031    Time: 0:02:08 ( 7.57 ms/it)
+  objective:  13333.742699564245
+Minimizing 17046    Time: 0:02:09 ( 7.57 ms/it)
+  objective:  13333.738548880305
+Minimizing 17061    Time: 0:02:09 ( 7.57 ms/it)
+  objective:  13333.70332397725
+Minimizing 17076    Time: 0:02:09 ( 7.57 ms/it)
+  objective:  13333.658183210646
+Minimizing 17091    Time: 0:02:09 ( 7.57 ms/it)
+  objective:  13333.609504180356
+Minimizing 17106    Time: 0:02:09 ( 7.57 ms/it)
+  objective:  13333.608415762683
+Minimizing 17121    Time: 0:02:09 ( 7.57 ms/it)
+  objective:  13333.602898274534
+Minimizing 17136    Time: 0:02:09 ( 7.57 ms/it)
+  objective:  13333.59507447989
+Minimizing 17150    Time: 0:02:09 ( 7.57 ms/it)
+  objective:  13333.589845285947
+Minimizing 17164    Time: 0:02:09 ( 7.57 ms/it)
+  objective:  13333.57636132562
+Minimizing 17178    Time: 0:02:10 ( 7.57 ms/it)
+  objective:  13333.559629796793
+Minimizing 17192    Time: 0:02:10 ( 7.57 ms/it)
+  objective:  13333.556620464973
+Minimizing 17206    Time: 0:02:10 ( 7.57 ms/it)
+  objective:  13333.543691341416
+Minimizing 17220    Time: 0:02:10 ( 7.57 ms/it)
+  objective:  13333.5359053734
+Minimizing 17235    Time: 0:02:10 ( 7.57 ms/it)
+  objective:  13333.500998866839
+Minimizing 17250    Time: 0:02:10 ( 7.57 ms/it)
+  objective:  13333.49816048437
+Minimizing 17265    Time: 0:02:10 ( 7.57 ms/it)
+  objective:  13333.462573128883
+Minimizing 17280    Time: 0:02:10 ( 7.57 ms/it)
+  objective:  13333.450415160492
+Minimizing 17295    Time: 0:02:10 ( 7.57 ms/it)
+  objective:  13333.349900711793
+Minimizing 17310    Time: 0:02:10 ( 7.57 ms/it)
+  objective:  13333.290778406292
+Minimizing 17325    Time: 0:02:11 ( 7.57 ms/it)
+  objective:  13333.28956611346
+Minimizing 17340    Time: 0:02:11 ( 7.56 ms/it)
+  objective:  13333.284034536722
+Minimizing 17355    Time: 0:02:11 ( 7.56 ms/it)
+  objective:  13333.270325968595
+Minimizing 17370    Time: 0:02:11 ( 7.56 ms/it)
+  objective:  13333.238687290366
+Minimizing 17385    Time: 0:02:11 ( 7.56 ms/it)
+  objective:  13333.228471864837
+Minimizing 17400    Time: 0:02:11 ( 7.56 ms/it)
+  objective:  13333.21777517206
+Minimizing 17415    Time: 0:02:11 ( 7.56 ms/it)
+  objective:  13333.20684214843
+Minimizing 17430    Time: 0:02:11 ( 7.56 ms/it)
+  objective:  13333.1543679713
+Minimizing 17445    Time: 0:02:11 ( 7.56 ms/it)
+  objective:  13333.142051582137
+Minimizing 17460    Time: 0:02:11 ( 7.56 ms/it)
+  objective:  13333.137439363796
+Minimizing 17475    Time: 0:02:12 ( 7.56 ms/it)
+  objective:  13333.116010241953
+Minimizing 17490    Time: 0:02:12 ( 7.56 ms/it)
+  objective:  13333.060280116428
+Minimizing 17505    Time: 0:02:12 ( 7.56 ms/it)
+  objective:  13333.050604128272
+Minimizing 17520    Time: 0:02:12 ( 7.56 ms/it)
+  objective:  13333.045945147009
+Minimizing 17535    Time: 0:02:12 ( 7.56 ms/it)
+  objective:  13333.027130369293
+Minimizing 17550    Time: 0:02:12 ( 7.56 ms/it)
+  objective:  13333.015321131978
+Minimizing 17565    Time: 0:02:12 ( 7.55 ms/it)
+  objective:  13332.97880722637
+Minimizing 17580    Time: 0:02:12 ( 7.55 ms/it)
+  objective:  13332.97565210861
+Minimizing 17595    Time: 0:02:12 ( 7.55 ms/it)
+  objective:  13332.983109679197
+Minimizing 17610    Time: 0:02:13 ( 7.55 ms/it)
+  objective:  13332.955121745945
+Minimizing 17625    Time: 0:02:13 ( 7.55 ms/it)
+  objective:  13332.937700654504
+Minimizing 17640    Time: 0:02:13 ( 7.55 ms/it)
+  objective:  13332.882778018167
+Minimizing 17655    Time: 0:02:13 ( 7.55 ms/it)
+  objective:  13332.88179781165
+Minimizing 17670    Time: 0:02:13 ( 7.55 ms/it)
+  objective:  13332.86334774716
+Minimizing 17685    Time: 0:02:13 ( 7.55 ms/it)
+  objective:  13332.816742069132
+Minimizing 17700    Time: 0:02:13 ( 7.55 ms/it)
+  objective:  13332.703862158785
+Minimizing 17715    Time: 0:02:13 ( 7.55 ms/it)
+  objective:  13332.701447899948
+Minimizing 17730    Time: 0:02:13 ( 7.55 ms/it)
+  objective:  13332.70036918491
+Minimizing 17745    Time: 0:02:13 ( 7.55 ms/it)
+  objective:  13332.691952049987
+Minimizing 17760    Time: 0:02:14 ( 7.55 ms/it)
+  objective:  13332.687567772737
+Minimizing 17775    Time: 0:02:14 ( 7.55 ms/it)
+  objective:  13332.678697258212
+Minimizing 17790    Time: 0:02:14 ( 7.55 ms/it)
+  objective:  13332.667736668933
+Minimizing 17805    Time: 0:02:14 ( 7.54 ms/it)
+  objective:  13332.646707279593
+Minimizing 17820    Time: 0:02:14 ( 7.54 ms/it)
+  objective:  13332.645596742994
+Minimizing 17835    Time: 0:02:14 ( 7.54 ms/it)
+  objective:  13332.641323786644
+Minimizing 17850    Time: 0:02:14 ( 7.54 ms/it)
+  objective:  13332.633087447546
+Minimizing 17865    Time: 0:02:14 ( 7.54 ms/it)
+  objective:  13332.619992940177
+Minimizing 17880    Time: 0:02:14 ( 7.54 ms/it)
+  objective:  13332.608462807628
+Minimizing 17895    Time: 0:02:14 ( 7.54 ms/it)
+  objective:  13332.600785487033
+Minimizing 17910    Time: 0:02:15 ( 7.54 ms/it)
+  objective:  13332.549919360208
+Minimizing 17925    Time: 0:02:15 ( 7.54 ms/it)
+  objective:  13332.501710666329
+Minimizing 17940    Time: 0:02:15 ( 7.54 ms/it)
+  objective:  13332.499659351568
+Minimizing 17955    Time: 0:02:15 ( 7.54 ms/it)
+  objective:  13332.491402559594
+Minimizing 17970    Time: 0:02:15 ( 7.54 ms/it)
+  objective:  13332.487239124843
+Minimizing 17985    Time: 0:02:15 ( 7.54 ms/it)
+  objective:  13332.45934803372
+Minimizing 18000    Time: 0:02:15 ( 7.54 ms/it)
+  objective:  13332.453125551532
+Minimizing 18015    Time: 0:02:15 ( 7.54 ms/it)
+  objective:  13332.446531414462
+Minimizing 18030    Time: 0:02:15 ( 7.54 ms/it)
+  objective:  13332.439426579731
+Minimizing 18045    Time: 0:02:15 ( 7.54 ms/it)
+  objective:  13332.417197071183
+Minimizing 18060    Time: 0:02:16 ( 7.54 ms/it)
+  objective:  13332.399875101924
+Minimizing 18075    Time: 0:02:16 ( 7.53 ms/it)
+  objective:  13332.34404242313
+Minimizing 18090    Time: 0:02:16 ( 7.53 ms/it)
+  objective:  13332.343236198773
+Minimizing 18104    Time: 0:02:16 ( 7.53 ms/it)
+  objective:  13332.333125607394
+Minimizing 18118    Time: 0:02:16 ( 7.53 ms/it)
+  objective:  13332.316439239708
+Minimizing 18132    Time: 0:02:16 ( 7.53 ms/it)
+  objective:  13332.291426513337
+Minimizing 18146    Time: 0:02:16 ( 7.53 ms/it)
+  objective:  13332.281114748686
+Minimizing 18160    Time: 0:02:16 ( 7.53 ms/it)
+  objective:  13332.237256910972
+Minimizing 18173    Time: 0:02:16 ( 7.53 ms/it)
+  objective:  13332.186687238827
+Minimizing 18188    Time: 0:02:17 ( 7.53 ms/it)
+  objective:  13332.179816500524
+Minimizing 18203    Time: 0:02:17 ( 7.53 ms/it)
+  objective:  13332.137391900345
+Minimizing 18218    Time: 0:02:17 ( 7.53 ms/it)
+  objective:  13332.135785362836
+Minimizing 18233    Time: 0:02:17 ( 7.53 ms/it)
+  objective:  13332.128262044927
+Minimizing 18248    Time: 0:02:17 ( 7.53 ms/it)
+  objective:  13332.116552123509
+Minimizing 18263    Time: 0:02:17 ( 7.53 ms/it)
+  objective:  13332.10702422502
+Minimizing 18278    Time: 0:02:17 ( 7.53 ms/it)
+  objective:  13332.093792357344
+Minimizing 18293    Time: 0:02:17 ( 7.53 ms/it)
+  objective:  13332.079644275545
+Minimizing 18308    Time: 0:02:17 ( 7.53 ms/it)
+  objective:  13332.077538074613
+Minimizing 18323    Time: 0:02:17 ( 7.53 ms/it)
+  objective:  13332.060810137089
+Minimizing 18338    Time: 0:02:18 ( 7.53 ms/it)
+  objective:  13332.032678512092
+Minimizing 18353    Time: 0:02:18 ( 7.53 ms/it)
+  objective:  13332.001288815278
+Minimizing 18368    Time: 0:02:18 ( 7.53 ms/it)
+  objective:  13331.841489251798
+Minimizing 18383    Time: 0:02:18 ( 7.53 ms/it)
+  objective:  13331.829448723205
+Minimizing 18398    Time: 0:02:18 ( 7.53 ms/it)
+  objective:  13331.820960955032
+Minimizing 18413    Time: 0:02:18 ( 7.53 ms/it)
+  objective:  13331.806649847997
+Minimizing 18428    Time: 0:02:18 ( 7.53 ms/it)
+  objective:  13331.791495228303
+Minimizing 18443    Time: 0:02:18 ( 7.52 ms/it)
+  objective:  13331.76630928168
+Minimizing 18458    Time: 0:02:18 ( 7.52 ms/it)
+  objective:  13331.743554202214
+Minimizing 18473    Time: 0:02:18 ( 7.52 ms/it)
+  objective:  13331.730589842344
+Minimizing 18488    Time: 0:02:19 ( 7.52 ms/it)
+  objective:  13331.728928041397
+Minimizing 18503    Time: 0:02:19 ( 7.52 ms/it)
+  objective:  13331.71955521925
+Minimizing 18518    Time: 0:02:19 ( 7.52 ms/it)
+  objective:  13331.68633211519
+Minimizing 18533    Time: 0:02:19 ( 7.52 ms/it)
+  objective:  13331.642191328901
+Minimizing 18548    Time: 0:02:19 ( 7.52 ms/it)
+  objective:  13331.483301410866
+Minimizing 18563    Time: 0:02:19 ( 7.52 ms/it)
+  objective:  13331.41677610945
+Minimizing 18578    Time: 0:02:19 ( 7.52 ms/it)
+  objective:  13331.409212166298
+Minimizing 18593    Time: 0:02:19 ( 7.52 ms/it)
+  objective:  13331.406243641904
+Minimizing 18608    Time: 0:02:19 ( 7.52 ms/it)
+  objective:  13331.402349978707
+Minimizing 18623    Time: 0:02:20 ( 7.52 ms/it)
+  objective:  13331.397021771743
+Minimizing 18638    Time: 0:02:20 ( 7.52 ms/it)
+  objective:  13331.387051208018
+Minimizing 18652    Time: 0:02:20 ( 7.52 ms/it)
+  objective:  13331.374546016901
+Minimizing 18666    Time: 0:02:20 ( 7.52 ms/it)
+  objective:  13331.30951760261
+Minimizing 18680    Time: 0:02:20 ( 7.52 ms/it)
+  objective:  13331.016718128223
+Minimizing 18694    Time: 0:02:20 ( 7.52 ms/it)
+  objective:  13331.016200164442
+Minimizing 18709    Time: 0:02:20 ( 7.52 ms/it)
+  objective:  13331.014856694055
+Minimizing 18724    Time: 0:02:20 ( 7.52 ms/it)
+  objective:  13331.000092066279
+Minimizing 18739    Time: 0:02:20 ( 7.52 ms/it)
+  objective:  13330.98174820219
+Minimizing 18754    Time: 0:02:20 ( 7.52 ms/it)
+  objective:  13330.923246875565
+Minimizing 18769    Time: 0:02:21 ( 7.52 ms/it)
+  objective:  13331.057292039288
+Minimizing 18784    Time: 0:02:21 ( 7.52 ms/it)
+  objective:  13330.820249156663
+Minimizing 18799    Time: 0:02:21 ( 7.51 ms/it)
+  objective:  13330.819560777869
+Minimizing 18814    Time: 0:02:21 ( 7.51 ms/it)
+  objective:  13330.81742589306
+Minimizing 18829    Time: 0:02:21 ( 7.51 ms/it)
+  objective:  13330.813268829435
+Minimizing 18844    Time: 0:02:21 ( 7.51 ms/it)
+  objective:  13330.809176184055
+Minimizing 18859    Time: 0:02:21 ( 7.51 ms/it)
+  objective:  13330.780886687935
+Minimizing 18874    Time: 0:02:21 ( 7.51 ms/it)
+  objective:  13330.735498194335
+Minimizing 18889    Time: 0:02:21 ( 7.51 ms/it)
+  objective:  13330.717466289127
+Minimizing 18904    Time: 0:02:21 ( 7.51 ms/it)
+  objective:  13330.671095346988
+Minimizing 18919    Time: 0:02:22 ( 7.51 ms/it)
+  objective:  13330.658830652581
+Minimizing 18934    Time: 0:02:22 ( 7.51 ms/it)
+  objective:  13330.646016764695
+Minimizing 18949    Time: 0:02:22 ( 7.51 ms/it)
+  objective:  13330.629924788125
+Minimizing 18964    Time: 0:02:22 ( 7.51 ms/it)
+  objective:  13330.575998203167
+Minimizing 18979    Time: 0:02:22 ( 7.51 ms/it)
+  objective:  13330.5184689651
+Minimizing 18994    Time: 0:02:22 ( 7.51 ms/it)
+  objective:  13330.51788424806
+Minimizing 19009    Time: 0:02:22 ( 7.51 ms/it)
+  objective:  13330.51448597193
+Minimizing 19024    Time: 0:02:22 ( 7.51 ms/it)
+  objective:  13330.509704314725
+Minimizing 19039    Time: 0:02:22 ( 7.51 ms/it)
+  objective:  13330.503051528925
+Minimizing 19054    Time: 0:02:23 ( 7.51 ms/it)
+  objective:  13330.479735019435
+Minimizing 19069    Time: 0:02:23 ( 7.51 ms/it)
+  objective:  13330.46355300842
+Minimizing 19084    Time: 0:02:23 ( 7.51 ms/it)
+  objective:  13330.432544731171
+Minimizing 19099    Time: 0:02:23 ( 7.51 ms/it)
+  objective:  13330.422039944147
+Minimizing 19114    Time: 0:02:23 ( 7.51 ms/it)
+  objective:  13330.358435354035
+Minimizing 19129    Time: 0:02:23 ( 7.50 ms/it)
+  objective:  13330.335687806124
+Minimizing 19144    Time: 0:02:23 ( 7.50 ms/it)
+  objective:  13330.334375663617
+Minimizing 19159    Time: 0:02:23 ( 7.50 ms/it)
+  objective:  13330.322470435196
+Minimizing 19174    Time: 0:02:23 ( 7.50 ms/it)
+  objective:  13330.277306498145
+Minimizing 19189    Time: 0:02:23 ( 7.50 ms/it)
+  objective:  13330.270544234161
+Minimizing 19204    Time: 0:02:24 ( 7.50 ms/it)
+  objective:  13330.209340675552
+Minimizing 19219    Time: 0:02:24 ( 7.50 ms/it)
+  objective:  13330.205449058165
+Minimizing 19234    Time: 0:02:24 ( 7.50 ms/it)
+  objective:  13330.199347486465
+Minimizing 19249    Time: 0:02:24 ( 7.50 ms/it)
+  objective:  13330.186147581022
+Minimizing 19264    Time: 0:02:24 ( 7.50 ms/it)
+  objective:  13330.15832003655
+Minimizing 19279    Time: 0:02:24 ( 7.50 ms/it)
+  objective:  13330.074843961767
+Minimizing 19294    Time: 0:02:24 ( 7.50 ms/it)
+  objective:  13330.02981807918
+Minimizing 19309    Time: 0:02:24 ( 7.50 ms/it)
+  objective:  13330.02634915684
+Minimizing 19324    Time: 0:02:24 ( 7.50 ms/it)
+  objective:  13329.999798483856
+Minimizing 19339    Time: 0:02:24 ( 7.50 ms/it)
+  objective:  13329.969057325463
+Minimizing 19354    Time: 0:02:25 ( 7.50 ms/it)
+  objective:  13329.963734783938
+Minimizing 19369    Time: 0:02:25 ( 7.50 ms/it)
+  objective:  13329.95293151198
+Minimizing 19384    Time: 0:02:25 ( 7.49 ms/it)
+  objective:  13329.933692490544
+Minimizing 19399    Time: 0:02:25 ( 7.49 ms/it)
+  objective:  13329.91544806432
+Minimizing 19414    Time: 0:02:25 ( 7.49 ms/it)
+  objective:  13329.897226474255
+Minimizing 19429    Time: 0:02:25 ( 7.49 ms/it)
+  objective:  13329.833230712793
+Minimizing 19444    Time: 0:02:25 ( 7.49 ms/it)
+  objective:  13329.83249528308
+Minimizing 19459    Time: 0:02:25 ( 7.49 ms/it)
+  objective:  13329.828962245141
+Minimizing 19474    Time: 0:02:25 ( 7.49 ms/it)
+  objective:  13329.825982439135
+Minimizing 19489    Time: 0:02:26 ( 7.49 ms/it)
+  objective:  13329.821616552857
+Minimizing 19504    Time: 0:02:26 ( 7.49 ms/it)
+  objective:  13329.814144659802
+Minimizing 19519    Time: 0:02:26 ( 7.49 ms/it)
+  objective:  13329.79126935944
+Minimizing 19534    Time: 0:02:26 ( 7.49 ms/it)
+  objective:  13329.789456750936
+Minimizing 19549    Time: 0:02:26 ( 7.49 ms/it)
+  objective:  13329.755567503045
+Minimizing 19564    Time: 0:02:26 ( 7.49 ms/it)
+  objective:  13329.75065324807
+Minimizing 19579    Time: 0:02:26 ( 7.49 ms/it)
+  objective:  13329.74617634451
+Minimizing 19594    Time: 0:02:26 ( 7.49 ms/it)
+  objective:  13329.729968831707
+Minimizing 19609    Time: 0:02:26 ( 7.49 ms/it)
+  objective:  13329.718980641403
+Minimizing 19624    Time: 0:02:26 ( 7.49 ms/it)
+  objective:  13329.652253531873
+Minimizing 19639    Time: 0:02:27 ( 7.49 ms/it)
+  objective:  13329.650557134548
+Minimizing 19654    Time: 0:02:27 ( 7.49 ms/it)
+  objective:  13329.647233110918
+Minimizing 19669    Time: 0:02:27 ( 7.49 ms/it)
+  objective:  13329.637596124892
+Minimizing 19684    Time: 0:02:27 ( 7.49 ms/it)
+  objective:  13329.629141556885
+Minimizing 19699    Time: 0:02:27 ( 7.49 ms/it)
+  objective:  13329.597380038285
+Minimizing 19714    Time: 0:02:27 ( 7.49 ms/it)
+  objective:  13329.590077110573
+Minimizing 19728    Time: 0:02:27 ( 7.49 ms/it)
+  objective:  13329.550889750011
+Minimizing 19743    Time: 0:02:27 ( 7.49 ms/it)
+  objective:  13329.535516698132
+Minimizing 19758    Time: 0:02:27 ( 7.48 ms/it)
+  objective:  13329.529940206885
+Minimizing 19773    Time: 0:02:27 ( 7.48 ms/it)
+  objective:  13329.501745272908
+Minimizing 19788    Time: 0:02:28 ( 7.48 ms/it)
+  objective:  13329.470498716539
+Minimizing 19803    Time: 0:02:28 ( 7.48 ms/it)
+  objective:  13329.4678260597
+Minimizing 19818    Time: 0:02:28 ( 7.48 ms/it)
+  objective:  13329.465515735588
+Minimizing 19833    Time: 0:02:28 ( 7.48 ms/it)
+  objective:  13329.45995635241
+Minimizing 19848    Time: 0:02:28 ( 7.48 ms/it)
+  objective:  13329.455773567352
+Minimizing 19863    Time: 0:02:28 ( 7.48 ms/it)
+  objective:  13329.448911258849
+Minimizing 19878    Time: 0:02:28 ( 7.48 ms/it)
+  objective:  13329.436440422476
+Minimizing 19893    Time: 0:02:28 ( 7.48 ms/it)
+  objective:  13329.426360743557
+Minimizing 19908    Time: 0:02:28 ( 7.48 ms/it)
+  objective:  13329.352069562941
+Minimizing 19923    Time: 0:02:29 ( 7.48 ms/it)
+  objective:  13329.347344352376
+Minimizing 19938    Time: 0:02:29 ( 7.48 ms/it)
+  objective:  13329.34617265124
+Minimizing 19953    Time: 0:02:29 ( 7.48 ms/it)
+  objective:  13329.34246769736
+Minimizing 19968    Time: 0:02:29 ( 7.48 ms/it)
+  objective:  13329.33301840727
+Minimizing 19983    Time: 0:02:29 ( 7.48 ms/it)
+  objective:  13329.325547087668
+Minimizing 19998    Time: 0:02:29 ( 7.48 ms/it)
+  objective:  13329.297596178694
+Minimizing 20013    Time: 0:02:29 ( 7.48 ms/it)
+  objective:  13329.266890222869
+Minimizing 20028    Time: 0:02:29 ( 7.48 ms/it)
+  objective:  13329.262394117955
+Minimizing 20043    Time: 0:02:29 ( 7.47 ms/it)
+  objective:  13329.25246840615
+Minimizing 20058    Time: 0:02:29 ( 7.47 ms/it)
+  objective:  13329.247770604656
+Minimizing 20073    Time: 0:02:30 ( 7.47 ms/it)
+  objective:  13329.20921632585
+Minimizing 20088    Time: 0:02:30 ( 7.47 ms/it)
+  objective:  13329.166166704345
+Minimizing 20103    Time: 0:02:30 ( 7.47 ms/it)
+  objective:  13329.163100854916
+Minimizing 20118    Time: 0:02:30 ( 7.47 ms/it)
+  objective:  13329.155168256664
+Minimizing 20133    Time: 0:02:30 ( 7.47 ms/it)
+  objective:  13329.144308616567
+Minimizing 20148    Time: 0:02:30 ( 7.47 ms/it)
+  objective:  13329.120819068099
+Minimizing 20163    Time: 0:02:30 ( 7.47 ms/it)
+  objective:  13329.10461898391
+Minimizing 20178    Time: 0:02:30 ( 7.47 ms/it)
+  objective:  13329.101833808185
+Minimizing 20193    Time: 0:02:30 ( 7.47 ms/it)
+  objective:  13329.0721810158
+Minimizing 20208    Time: 0:02:30 ( 7.47 ms/it)
+  objective:  13329.071358179608
+Minimizing 20223    Time: 0:02:31 ( 7.47 ms/it)
+  objective:  13329.058252399627
+Minimizing 20238    Time: 0:02:31 ( 7.47 ms/it)
+  objective:  13329.045935743467
+Minimizing 20253    Time: 0:02:31 ( 7.47 ms/it)
+  objective:  13328.997317338435
+Minimizing 20268    Time: 0:02:31 ( 7.47 ms/it)
+  objective:  13328.927473802381
+Minimizing 20283    Time: 0:02:31 ( 7.47 ms/it)
+  objective:  13328.92687589582
+Minimizing 20298    Time: 0:02:31 ( 7.47 ms/it)
+  objective:  13328.921026396762
+Minimizing 20313    Time: 0:02:31 ( 7.47 ms/it)
+  objective:  13328.913358183323
+Minimizing 20328    Time: 0:02:31 ( 7.47 ms/it)
+  objective:  13328.909590178911
+Minimizing 20343    Time: 0:02:31 ( 7.47 ms/it)
+  objective:  13328.902042924135
+Minimizing 20358    Time: 0:02:31 ( 7.47 ms/it)
+  objective:  13328.893171037555
+Minimizing 20373    Time: 0:02:32 ( 7.46 ms/it)
+  objective:  13328.884132216313
+Minimizing 20388    Time: 0:02:32 ( 7.46 ms/it)
+  objective:  13328.822744374469
+Minimizing 20403    Time: 0:02:32 ( 7.46 ms/it)
+  objective:  13328.816520422188
+Minimizing 20418    Time: 0:02:32 ( 7.46 ms/it)
+  objective:  13328.815997290687
+Minimizing 20433    Time: 0:02:32 ( 7.46 ms/it)
+  objective:  13328.815274382294
+Minimizing 20448    Time: 0:02:32 ( 7.46 ms/it)
+  objective:  13328.807729363056
+Minimizing 20463    Time: 0:02:32 ( 7.46 ms/it)
+  objective:  13328.797776250714
+Minimizing 20478    Time: 0:02:32 ( 7.46 ms/it)
+  objective:  13328.79612250832
+Minimizing 20493    Time: 0:02:32 ( 7.46 ms/it)
+  objective:  13328.789174281701
+Minimizing 20508    Time: 0:02:33 ( 7.46 ms/it)
+  objective:  13328.782602541985
+Minimizing 20523    Time: 0:02:33 ( 7.46 ms/it)
+  objective:  13328.75391608329
+Minimizing 20538    Time: 0:02:33 ( 7.46 ms/it)
+  objective:  13328.745396715341
+Minimizing 20553    Time: 0:02:33 ( 7.46 ms/it)
+  objective:  13328.74368186434
+Minimizing 20568    Time: 0:02:33 ( 7.46 ms/it)
+  objective:  13328.724947656156
+Minimizing 20583    Time: 0:02:33 ( 7.46 ms/it)
+  objective:  13328.719728334865
+Minimizing 20598    Time: 0:02:33 ( 7.46 ms/it)
+  objective:  13328.702692766936
+Minimizing 20613    Time: 0:02:33 ( 7.46 ms/it)
+  objective:  13328.696137179315
+Minimizing 20628    Time: 0:02:33 ( 7.46 ms/it)
+  objective:  13328.647686955097
+Minimizing 20643    Time: 0:02:33 ( 7.46 ms/it)
+  objective:  13328.613671381274
+Minimizing 20658    Time: 0:02:34 ( 7.46 ms/it)
+  objective:  13328.61181658777
+Minimizing 20673    Time: 0:02:34 ( 7.46 ms/it)
+  objective:  13328.57300651681
+Minimizing 20688    Time: 0:02:34 ( 7.46 ms/it)
+  objective:  13328.563083097935
+Minimizing 20703    Time: 0:02:34 ( 7.46 ms/it)
+  objective:  13328.562512474178
+Minimizing 20718    Time: 0:02:34 ( 7.46 ms/it)
+  objective:  13328.556037998474
+Minimizing 20733    Time: 0:02:34 ( 7.45 ms/it)
+  objective:  13328.552198662284
+Minimizing 20748    Time: 0:02:34 ( 7.45 ms/it)
+  objective:  13328.540902033885
+Minimizing 20763    Time: 0:02:34 ( 7.45 ms/it)
+  objective:  13328.540337993203
+Minimizing 20777    Time: 0:02:34 ( 7.45 ms/it)
+  objective:  13328.539066489924
+Minimizing 20791    Time: 0:02:34 ( 7.45 ms/it)
+  objective:  13328.536263043956
+Minimizing 20806    Time: 0:02:35 ( 7.45 ms/it)
+  objective:  13328.527693420467
+Minimizing 20821    Time: 0:02:35 ( 7.45 ms/it)
+  objective:  13328.509336638373
+Minimizing 20836    Time: 0:02:35 ( 7.45 ms/it)
+  objective:  13328.50008276588
+Minimizing 20851    Time: 0:02:35 ( 7.45 ms/it)
+  objective:  13328.497502201288
+Minimizing 20866    Time: 0:02:35 ( 7.45 ms/it)
+  objective:  13328.48402564686
+Minimizing 20880    Time: 0:02:35 ( 7.45 ms/it)
+  objective:  13328.479154114124
+Minimizing 20894    Time: 0:02:35 ( 7.45 ms/it)
+  objective:  13328.468692663213
+Minimizing 20908    Time: 0:02:35 ( 7.45 ms/it)
+  objective:  13328.467854868082
+Minimizing 20922    Time: 0:02:35 ( 7.45 ms/it)
+  objective:  13328.465649320882
+Minimizing 20936    Time: 0:02:36 ( 7.45 ms/it)
+  objective:  13328.459417663093
+Minimizing 20950    Time: 0:02:36 ( 7.45 ms/it)
+  objective:  13328.451719262754
+Minimizing 20964    Time: 0:02:36 ( 7.45 ms/it)
+  objective:  13328.433648257269
+Minimizing 20978    Time: 0:02:36 ( 7.45 ms/it)
+  objective:  13328.43263866804
+Minimizing 20992    Time: 0:02:36 ( 7.45 ms/it)
+  objective:  13328.423442030493
+Minimizing 21006    Time: 0:02:36 ( 7.45 ms/it)
+  objective:  13328.402658757783
+Minimizing 21020    Time: 0:02:36 ( 7.45 ms/it)
+  objective:  13328.380116310786
+Minimizing 21034    Time: 0:02:36 ( 7.45 ms/it)
+  objective:  13328.379635922072
+Minimizing 21048    Time: 0:02:36 ( 7.45 ms/it)
+  objective:  13328.374140634354
+Minimizing 21062    Time: 0:02:36 ( 7.45 ms/it)
+  objective:  13328.368476607291
+Minimizing 21076    Time: 0:02:37 ( 7.45 ms/it)
+  objective:  13328.35838525051
+Minimizing 21090    Time: 0:02:37 ( 7.45 ms/it)
+  objective:  13328.343381699531
+Minimizing 21104    Time: 0:02:37 ( 7.45 ms/it)
+  objective:  13328.339691008441
+Minimizing 21118    Time: 0:02:37 ( 7.45 ms/it)
+  objective:  13328.321410691162
+Minimizing 21132    Time: 0:02:37 ( 7.45 ms/it)
+  objective:  13328.318806762189
+Minimizing 21146    Time: 0:02:37 ( 7.45 ms/it)
+  objective:  13328.308190752672
+Minimizing 21160    Time: 0:02:37 ( 7.45 ms/it)
+  objective:  13328.302914141444
+Minimizing 21174    Time: 0:02:37 ( 7.45 ms/it)
+  objective:  13328.275662781438
+Minimizing 21188    Time: 0:02:37 ( 7.45 ms/it)
+  objective:  13328.267332250558
+Minimizing 21202    Time: 0:02:37 ( 7.45 ms/it)
+  objective:  13328.267030443574
+Minimizing 21216    Time: 0:02:38 ( 7.45 ms/it)
+  objective:  13328.26497987088
+Minimizing 21230    Time: 0:02:38 ( 7.45 ms/it)
+  objective:  13328.262967928327
+Minimizing 21244    Time: 0:02:38 ( 7.45 ms/it)
+  objective:  13328.258266324825
+Minimizing 21258    Time: 0:02:38 ( 7.45 ms/it)
+  objective:  13328.237147518405
+Minimizing 21272    Time: 0:02:38 ( 7.45 ms/it)
+  objective:  13328.231564114918
+Minimizing 21286    Time: 0:02:38 ( 7.45 ms/it)
+  objective:  13328.230363839379
+Minimizing 21300    Time: 0:02:38 ( 7.45 ms/it)
+  objective:  13328.219418272914
+Minimizing 21314    Time: 0:02:38 ( 7.45 ms/it)
+  objective:  13328.214686666295
+Minimizing 21328    Time: 0:02:38 ( 7.45 ms/it)
+  objective:  13328.329554605742
+Minimizing 21342    Time: 0:02:39 ( 7.45 ms/it)
+  objective:  13328.129434561284
+Minimizing 21356    Time: 0:02:39 ( 7.45 ms/it)
+  objective:  13328.058548558576
+Minimizing 21370    Time: 0:02:39 ( 7.45 ms/it)
+  objective:  13328.030552163902
+Minimizing 21384    Time: 0:02:39 ( 7.45 ms/it)
+  objective:  13328.030064496554
+Minimizing 21398    Time: 0:02:39 ( 7.45 ms/it)
+  objective:  13328.019692689646
+Minimizing 21413    Time: 0:02:39 ( 7.45 ms/it)
+  objective:  13328.016799902507
+Minimizing 21428    Time: 0:02:39 ( 7.45 ms/it)
+  objective:  13328.011096218768
+Minimizing 21442    Time: 0:02:39 ( 7.45 ms/it)
+  objective:  13328.004205592631
+Minimizing 21457    Time: 0:02:39 ( 7.45 ms/it)
+  objective:  13327.990796425882
+Minimizing 21472    Time: 0:02:39 ( 7.45 ms/it)
+  objective:  13327.923040778485
+Minimizing 21487    Time: 0:02:40 ( 7.45 ms/it)
+  objective:  13327.802714455902
+Minimizing 21502    Time: 0:02:40 ( 7.45 ms/it)
+  objective:  13327.80199397778
+Minimizing 21517    Time: 0:02:40 ( 7.45 ms/it)
+  objective:  13327.801556733619
+Minimizing 21532    Time: 0:02:40 ( 7.45 ms/it)
+  objective:  13327.799584313805
+Minimizing 21547    Time: 0:02:40 ( 7.45 ms/it)
+  objective:  13327.796538176422
+Minimizing 21563    Time: 0:02:40 ( 7.45 ms/it)
+  objective:  13327.792956428872
+Minimizing 21578    Time: 0:02:40 ( 7.45 ms/it)
+  objective:  13327.782168812788
+Minimizing 21593    Time: 0:02:40 ( 7.45 ms/it)
+  objective:  13327.76420606087
+Minimizing 21608    Time: 0:02:40 ( 7.45 ms/it)
+  objective:  13327.756326261602
+Minimizing 21623    Time: 0:02:41 ( 7.45 ms/it)
+  objective:  13327.742099629606
+Minimizing 21638    Time: 0:02:41 ( 7.45 ms/it)
+  objective:  13327.731609365132
+Minimizing 21653    Time: 0:02:41 ( 7.45 ms/it)
+  objective:  13327.691137984082
+Minimizing 21668    Time: 0:02:41 ( 7.44 ms/it)
+  objective:  13327.68437747578
+Minimizing 21683    Time: 0:02:41 ( 7.44 ms/it)
+  objective:  13327.683860725629
+Minimizing 21698    Time: 0:02:41 ( 7.44 ms/it)
+  objective:  13327.681427554002
+Minimizing 21712    Time: 0:02:41 ( 7.44 ms/it)
+  objective:  13327.676566439797
+Minimizing 21726    Time: 0:02:41 ( 7.44 ms/it)
+  objective:  13327.671769973153
+Minimizing 21740    Time: 0:02:41 ( 7.44 ms/it)
+  objective:  13327.668825924484
+Minimizing 21753    Time: 0:02:41 ( 7.44 ms/it)
+  objective:  13327.634667787177
+Minimizing 21768    Time: 0:02:42 ( 7.44 ms/it)
+  objective:  13327.555445089332
+Minimizing 21783    Time: 0:02:42 ( 7.44 ms/it)
+  objective:  13327.488345760896
+Minimizing 21798    Time: 0:02:42 ( 7.44 ms/it)
+  objective:  13327.483523630784
+Minimizing 21813    Time: 0:02:42 ( 7.44 ms/it)
+  objective:  13327.47910430107
+Minimizing 21828    Time: 0:02:42 ( 7.44 ms/it)
+  objective:  13327.47512143894
+Minimizing 21843    Time: 0:02:42 ( 7.44 ms/it)
+  objective:  13327.463279945252
+Minimizing 21858    Time: 0:02:42 ( 7.44 ms/it)
+  objective:  13327.45415418852
+Minimizing 21873    Time: 0:02:42 ( 7.44 ms/it)
+  objective:  13327.447017111721
+Minimizing 21888    Time: 0:02:42 ( 7.44 ms/it)
+  objective:  13327.446229432302
+Minimizing 21903    Time: 0:02:42 ( 7.44 ms/it)
+  objective:  13327.438987079368
+Minimizing 21918    Time: 0:02:43 ( 7.44 ms/it)
+  objective:  13327.429711103207
+Minimizing 21933    Time: 0:02:43 ( 7.44 ms/it)
+  objective:  13327.42328607491
+Minimizing 21948    Time: 0:02:43 ( 7.44 ms/it)
+  objective:  13327.420143696254
+Minimizing 21963    Time: 0:02:43 ( 7.44 ms/it)
+  objective:  13327.400751692949
+Minimizing 21978    Time: 0:02:43 ( 7.44 ms/it)
+  objective:  13327.362244055315
+Minimizing 21993    Time: 0:02:43 ( 7.44 ms/it)
+  objective:  13327.31157680729
+Minimizing 22008    Time: 0:02:43 ( 7.44 ms/it)
+  objective:  13327.25999445426
+Minimizing 22023    Time: 0:02:43 ( 7.44 ms/it)
+  objective:  13327.25922332704
+Minimizing 22038    Time: 0:02:43 ( 7.44 ms/it)
+  objective:  13327.2462541735
+Minimizing 22053    Time: 0:02:44 ( 7.44 ms/it)
+  objective:  13327.234894800225
+Minimizing 22068    Time: 0:02:44 ( 7.44 ms/it)
+  objective:  13327.224112585216
+Minimizing 22083    Time: 0:02:44 ( 7.44 ms/it)
+  objective:  13327.223133419866
+Minimizing 22098    Time: 0:02:44 ( 7.44 ms/it)
+  objective:  13327.210676335919
+Minimizing 22113    Time: 0:02:44 ( 7.44 ms/it)
+  objective:  13327.20305388247
+Minimizing 22128    Time: 0:02:44 ( 7.44 ms/it)
+  objective:  13327.19294880733
+Minimizing 22143    Time: 0:02:44 ( 7.44 ms/it)
+  objective:  13327.178508058394
+Minimizing 22158    Time: 0:02:44 ( 7.44 ms/it)
+  objective:  13327.06178661278
+Minimizing 22173    Time: 0:02:44 ( 7.44 ms/it)
+  objective:  13326.98340218766
+Minimizing 22188    Time: 0:02:44 ( 7.44 ms/it)
+  objective:  13326.981887870861
+Minimizing 22203    Time: 0:02:45 ( 7.43 ms/it)
+  objective:  13326.978497942364
+Minimizing 22218    Time: 0:02:45 ( 7.43 ms/it)
+  objective:  13326.974235244918
+Minimizing 22233    Time: 0:02:45 ( 7.43 ms/it)
+  objective:  13326.973542317428
+Minimizing 22248    Time: 0:02:45 ( 7.43 ms/it)
+  objective:  13326.969457978688
+Minimizing 22263    Time: 0:02:45 ( 7.43 ms/it)
+  objective:  13326.962927732857
+Minimizing 22278    Time: 0:02:45 ( 7.43 ms/it)
+  objective:  13326.960001248939
+Minimizing 22293    Time: 0:02:45 ( 7.43 ms/it)
+  objective:  13326.946917483125
+Minimizing 22308    Time: 0:02:45 ( 7.43 ms/it)
+  objective:  13326.824894673482
+Minimizing 22323    Time: 0:02:45 ( 7.43 ms/it)
+  objective:  13326.806908089908
+Minimizing 22338    Time: 0:02:46 ( 7.43 ms/it)
+  objective:  13326.802424988018
+Minimizing 22353    Time: 0:02:46 ( 7.43 ms/it)
+  objective:  13326.79990728727
+Minimizing 22368    Time: 0:02:46 ( 7.43 ms/it)
+  objective:  13326.789041377328
+Minimizing 22383    Time: 0:02:46 ( 7.43 ms/it)
+  objective:  13326.78609542313
+Minimizing 22398    Time: 0:02:46 ( 7.43 ms/it)
+  objective:  13326.785530430388
+Minimizing 22413    Time: 0:02:46 ( 7.43 ms/it)
+  objective:  13326.7803489203
+Minimizing 22428    Time: 0:02:46 ( 7.43 ms/it)
+  objective:  13326.767347653018
+Minimizing 22443    Time: 0:02:46 ( 7.43 ms/it)
+  objective:  13326.765994263114
+Minimizing 22457    Time: 0:02:46 ( 7.43 ms/it)
+  objective:  13326.714850444558
+Minimizing 22472    Time: 0:02:46 ( 7.43 ms/it)
+  objective:  13326.691845574547
+Minimizing 22487    Time: 0:02:47 ( 7.43 ms/it)
+  objective:  13326.691346704683
+Minimizing 22502    Time: 0:02:47 ( 7.43 ms/it)
+  objective:  13326.689862591105
+Minimizing 22517    Time: 0:02:47 ( 7.43 ms/it)
+  objective:  13326.688520467971
+Minimizing 22532    Time: 0:02:47 ( 7.43 ms/it)
+  objective:  13326.68485120796
+Minimizing 22547    Time: 0:02:47 ( 7.43 ms/it)
+  objective:  13326.679527234563
+Minimizing 22562    Time: 0:02:47 ( 7.43 ms/it)
+  objective:  13326.65707452214
+Minimizing 22577    Time: 0:02:47 ( 7.43 ms/it)
+  objective:  13326.629409959925
+Minimizing 22592    Time: 0:02:47 ( 7.43 ms/it)
+  objective:  13326.628672901024
+Minimizing 22607    Time: 0:02:47 ( 7.43 ms/it)
+  objective:  13326.616007351273
+Minimizing 22622    Time: 0:02:48 ( 7.43 ms/it)
+  objective:  13326.605576266433
+Minimizing 22637    Time: 0:02:48 ( 7.43 ms/it)
+  objective:  13326.603204174622
+Minimizing 22652    Time: 0:02:48 ( 7.43 ms/it)
+  objective:  13326.58620426726
+Minimizing 22667    Time: 0:02:48 ( 7.43 ms/it)
+  objective:  13326.585748073667
+Minimizing 22682    Time: 0:02:48 ( 7.42 ms/it)
+  objective:  13326.579198923508
+Minimizing 22697    Time: 0:02:48 ( 7.42 ms/it)
+  objective:  13326.57213118158
+Minimizing 22713    Time: 0:02:48 ( 7.42 ms/it)
+  objective:  13326.556858142758
+Minimizing 22728    Time: 0:02:48 ( 7.42 ms/it)
+  objective:  13326.501577811192
+Minimizing 22743    Time: 0:02:48 ( 7.42 ms/it)
+  objective:  13326.493334192353
+Minimizing 22758    Time: 0:02:48 ( 7.42 ms/it)
+  objective:  13326.485613655852
+Minimizing 22773    Time: 0:02:49 ( 7.42 ms/it)
+  objective:  13326.47581872304
+Minimizing 22788    Time: 0:02:49 ( 7.42 ms/it)
+  objective:  13326.473547372152
+Minimizing 22803    Time: 0:02:49 ( 7.42 ms/it)
+  objective:  13326.469559979545
+Minimizing 22818    Time: 0:02:49 ( 7.42 ms/it)
+  objective:  13326.414714141778
+Minimizing 22832    Time: 0:02:49 ( 7.42 ms/it)
+  objective:  13326.389299300594
+Minimizing 22846    Time: 0:02:49 ( 7.42 ms/it)
+  objective:  13326.388618226614
+Minimizing 22860    Time: 0:02:49 ( 7.42 ms/it)
+  objective:  13326.384338204836
+Minimizing 22874    Time: 0:02:49 ( 7.42 ms/it)
+  objective:  13326.370478796729
+Minimizing 22888    Time: 0:02:49 ( 7.42 ms/it)
+  objective:  13326.314260219879
+Minimizing 22903    Time: 0:02:49 ( 7.42 ms/it)
+  objective:  13326.205822746946
+Minimizing 22918    Time: 0:02:50 ( 7.42 ms/it)
+  objective:  13326.20143877827
+Minimizing 22933    Time: 0:02:50 ( 7.42 ms/it)
+  objective:  13326.196282373894
+Minimizing 22948    Time: 0:02:50 ( 7.42 ms/it)
+  objective:  13326.18270598077
+Minimizing 22963    Time: 0:02:50 ( 7.42 ms/it)
+  objective:  13326.160977791122
+Minimizing 22978    Time: 0:02:50 ( 7.42 ms/it)
+  objective:  13326.135123221378
+Minimizing 22993    Time: 0:02:50 ( 7.42 ms/it)
+  objective:  13325.980681415851
+Minimizing 23008    Time: 0:02:50 ( 7.42 ms/it)
+  objective:  13325.975564473425
+Minimizing 23023    Time: 0:02:50 ( 7.42 ms/it)
+  objective:  13325.973374115936
+Minimizing 23038    Time: 0:02:50 ( 7.42 ms/it)
+  objective:  13325.967804808824
+Minimizing 23052    Time: 0:02:50 ( 7.42 ms/it)
+  objective:  13325.964185826357
+Minimizing 23067    Time: 0:02:51 ( 7.42 ms/it)
+  objective:  13325.955189713743
+Minimizing 23082    Time: 0:02:51 ( 7.42 ms/it)
+  objective:  13325.943982802688
+Minimizing 23097    Time: 0:02:51 ( 7.42 ms/it)
+  objective:  13325.941626119842
+Minimizing 23111    Time: 0:02:51 ( 7.42 ms/it)
+  objective:  13325.92404661802
+Minimizing 23126    Time: 0:02:51 ( 7.42 ms/it)
+  objective:  13325.92209266215
+Minimizing 23141    Time: 0:02:51 ( 7.42 ms/it)
+  objective:  13325.8994712145
+Minimizing 23156    Time: 0:02:51 ( 7.42 ms/it)
+  objective:  13325.897759617335
+Minimizing 23171    Time: 0:02:51 ( 7.42 ms/it)
+  objective:  13325.894248749399
+Minimizing 23186    Time: 0:02:51 ( 7.42 ms/it)
+  objective:  13325.890072594048
+Minimizing 23201    Time: 0:02:52 ( 7.42 ms/it)
+  objective:  13325.888626762615
+Minimizing 23216    Time: 0:02:52 ( 7.41 ms/it)
+  objective:  13325.877981894148
+Minimizing 23231    Time: 0:02:52 ( 7.41 ms/it)
+  objective:  13325.860686741304
+Minimizing 23246    Time: 0:02:52 ( 7.41 ms/it)
+  objective:  13325.843581273628
+Minimizing 23261    Time: 0:02:52 ( 7.41 ms/it)
+  objective:  13325.841605235531
+Minimizing 23276    Time: 0:02:52 ( 7.41 ms/it)
+  objective:  13325.816536874867
+Minimizing 23291    Time: 0:02:52 ( 7.41 ms/it)
+  objective:  13325.815848166232
+Minimizing 23306    Time: 0:02:52 ( 7.41 ms/it)
+  objective:  13325.80247579953
+Minimizing 23321    Time: 0:02:52 ( 7.41 ms/it)
+  objective:  13325.782456923072
+Minimizing 23336    Time: 0:02:52 ( 7.41 ms/it)
+  objective:  13325.720044661815
+Minimizing 23351    Time: 0:02:53 ( 7.41 ms/it)
+  objective:  13325.710170793784
+Minimizing 23366    Time: 0:02:53 ( 7.41 ms/it)
+  objective:  13325.709577160946
+Minimizing 23381    Time: 0:02:53 ( 7.41 ms/it)
+  objective:  13325.707372138211
+Minimizing 23396    Time: 0:02:53 ( 7.41 ms/it)
+  objective:  13325.703062869128
+Minimizing 23410    Time: 0:02:53 ( 7.41 ms/it)
+  objective:  13325.701521256255
+Minimizing 23424    Time: 0:02:53 ( 7.41 ms/it)
+  objective:  13325.693897443103
+Minimizing 23438    Time: 0:02:53 ( 7.41 ms/it)
+  objective:  13325.692140432984
+Minimizing 23452    Time: 0:02:53 ( 7.41 ms/it)
+  objective:  13325.674455428962
+Minimizing 23466    Time: 0:02:53 ( 7.41 ms/it)
+  objective:  13325.667299645087
+Minimizing 23480    Time: 0:02:54 ( 7.41 ms/it)
+  objective:  13325.658870860054
+Minimizing 23495    Time: 0:02:54 ( 7.41 ms/it)
+  objective:  13325.6497698409
+Minimizing 23510    Time: 0:02:54 ( 7.41 ms/it)
+  objective:  13325.641894808949
+Minimizing 23525    Time: 0:02:54 ( 7.41 ms/it)
+  objective:  13325.617560442712
+Minimizing 23540    Time: 0:02:54 ( 7.41 ms/it)
+  objective:  13325.616688481437
+Minimizing 23555    Time: 0:02:54 ( 7.41 ms/it)
+  objective:  13325.610835894855
+Minimizing 23570    Time: 0:02:54 ( 7.41 ms/it)
+  objective:  13325.60775627155
+Minimizing 23585    Time: 0:02:54 ( 7.41 ms/it)
+  objective:  13325.595576413907
+Minimizing 23600    Time: 0:02:54 ( 7.41 ms/it)
+  objective:  13325.591879340587
+Minimizing 23614    Time: 0:02:54 ( 7.41 ms/it)
+  objective:  13325.574613286524
+Minimizing 23628    Time: 0:02:55 ( 7.41 ms/it)
+  objective:  13325.569888214362
+Minimizing 23642    Time: 0:02:55 ( 7.41 ms/it)
+  objective:  13325.556328988969
+Minimizing 23656    Time: 0:02:55 ( 7.41 ms/it)
+  objective:  13325.531548377927
+Minimizing 23671    Time: 0:02:55 ( 7.41 ms/it)
+  objective:  13325.513682343044
+Minimizing 23686    Time: 0:02:55 ( 7.41 ms/it)
+  objective:  13325.379046877395
+Minimizing 23701    Time: 0:02:55 ( 7.41 ms/it)
+  objective:  13325.374777401761
+Minimizing 23716    Time: 0:02:55 ( 7.41 ms/it)
+  objective:  13325.37213175586
+Minimizing 23731    Time: 0:02:55 ( 7.41 ms/it)
+  objective:  13325.365311146343
+Minimizing 23746    Time: 0:02:55 ( 7.41 ms/it)
+  objective:  13325.352803217691
+Minimizing 23757    Time: 0:02:56 ( 7.41 ms/it)
+  objective:  13325.348705264383
+Minimizing 23772    Time: 0:02:56 ( 7.41 ms/it)
+  objective:  13325.345511018873
+Minimizing 23787    Time: 0:02:56 ( 7.41 ms/it)
+  objective:  13325.333673063098
+Minimizing 23802    Time: 0:02:56 ( 7.41 ms/it)
+  objective:  13325.326626249283
+Minimizing 23817    Time: 0:02:56 ( 7.41 ms/it)
+  objective:  13325.280241778695
+Minimizing 23832    Time: 0:02:56 ( 7.41 ms/it)
+  objective:  13325.183053750574
+Minimizing 23847    Time: 0:02:56 ( 7.41 ms/it)
+  objective:  13325.171392884993
+Minimizing 23862    Time: 0:02:56 ( 7.41 ms/it)
+  objective:  13325.164722867601
+Minimizing 23877    Time: 0:02:56 ( 7.41 ms/it)
+  objective:  13325.14522989998
+Minimizing 23892    Time: 0:02:56 ( 7.41 ms/it)
+  objective:  13325.108478127833
+Minimizing 23907    Time: 0:02:57 ( 7.41 ms/it)
+  objective:  13325.096405844102
+Minimizing 23922    Time: 0:02:57 ( 7.41 ms/it)
+  objective:  13325.095868863602
+Minimizing 23937    Time: 0:02:57 ( 7.41 ms/it)
+  objective:  13325.094868936678
+Minimizing 23952    Time: 0:02:57 ( 7.41 ms/it)
+  objective:  13325.094432796497
+Minimizing 23967    Time: 0:02:57 ( 7.40 ms/it)
+  objective:  13325.090544140287
+Minimizing 23982    Time: 0:02:57 ( 7.40 ms/it)
+  objective:  13325.088217323995
+Minimizing 23997    Time: 0:02:57 ( 7.40 ms/it)
+  objective:  13325.07502238761
+Minimizing 24012    Time: 0:02:57 ( 7.40 ms/it)
+  objective:  13325.072326211899
+Minimizing 24027    Time: 0:02:57 ( 7.40 ms/it)
+  objective:  13325.07069916978
+Minimizing 24042    Time: 0:02:57 ( 7.40 ms/it)
+  objective:  13325.068001290172
+Minimizing 24057    Time: 0:02:58 ( 7.40 ms/it)
+  objective:  13325.061703774802
+Minimizing 24072    Time: 0:02:58 ( 7.40 ms/it)
+  objective:  13325.008824948964
+Minimizing 24087    Time: 0:02:58 ( 7.40 ms/it)
+  objective:  13324.904550778054
+Minimizing 24102    Time: 0:02:58 ( 7.40 ms/it)
+  objective:  13324.90324775927
+Minimizing 24117    Time: 0:02:58 ( 7.40 ms/it)
+  objective:  13324.90288263165
+Minimizing 24132    Time: 0:02:58 ( 7.40 ms/it)
+  objective:  13324.902287951147
+Minimizing 24147    Time: 0:02:58 ( 7.40 ms/it)
+  objective:  13324.902033610633
+Minimizing 24162    Time: 0:02:58 ( 7.40 ms/it)
+  objective:  13324.900693161544
+Minimizing 24177    Time: 0:02:58 ( 7.40 ms/it)
+  objective:  13324.900147081731
+Minimizing 24192    Time: 0:02:59 ( 7.40 ms/it)
+  objective:  13324.89009152702
+Minimizing 24207    Time: 0:02:59 ( 7.40 ms/it)
+  objective:  13324.887641981564
+Minimizing 24222    Time: 0:02:59 ( 7.40 ms/it)
+  objective:  13324.887168602057
+Minimizing 24237    Time: 0:02:59 ( 7.40 ms/it)
+  objective:  13324.885237294933
+Minimizing 24252    Time: 0:02:59 ( 7.40 ms/it)
+  objective:  13324.883043830414
+Minimizing 24267    Time: 0:02:59 ( 7.40 ms/it)
+  objective:  13324.877744016267
+Minimizing 24282    Time: 0:02:59 ( 7.40 ms/it)
+  objective:  13324.872394857492
+Minimizing 24297    Time: 0:02:59 ( 7.40 ms/it)
+  objective:  13324.869976458896
+Minimizing 24312    Time: 0:02:59 ( 7.40 ms/it)
+  objective:  13324.856658037606
+Minimizing 24327    Time: 0:02:59 ( 7.40 ms/it)
+  objective:  13324.842168211311
+Minimizing 24342    Time: 0:03:00 ( 7.40 ms/it)
+  objective:  13324.841681454971
+Minimizing 24357    Time: 0:03:00 ( 7.40 ms/it)
+  objective:  13324.83994435682
+Minimizing 24372    Time: 0:03:00 ( 7.40 ms/it)
+  objective:  13324.838790802693
+Minimizing 24387    Time: 0:03:00 ( 7.40 ms/it)
+  objective:  13324.830820353658
+Minimizing 24402    Time: 0:03:00 ( 7.40 ms/it)
+  objective:  13324.829414578591
+Minimizing 24417    Time: 0:03:00 ( 7.40 ms/it)
+  objective:  13324.824027084105
+Minimizing 24432    Time: 0:03:00 ( 7.40 ms/it)
+  objective:  13324.81773927102
+Minimizing 24447    Time: 0:03:00 ( 7.40 ms/it)
+  objective:  13324.81669238946
+Minimizing 24462    Time: 0:03:00 ( 7.40 ms/it)
+  objective:  13324.803258516695
+Minimizing 24477    Time: 0:03:01 ( 7.40 ms/it)
+  objective:  13324.802185734923
+Minimizing 24492    Time: 0:03:01 ( 7.40 ms/it)
+  objective:  13324.798339415298
+Minimizing 24507    Time: 0:03:01 ( 7.40 ms/it)
+  objective:  13324.791289974397
+Minimizing 24522    Time: 0:03:01 ( 7.40 ms/it)
+  objective:  13324.77044996353
+Minimizing 24537    Time: 0:03:01 ( 7.40 ms/it)
+  objective:  13324.764464140477
+Minimizing 24552    Time: 0:03:01 ( 7.39 ms/it)
+  objective:  13324.730256418421
+Minimizing 24567    Time: 0:03:01 ( 7.39 ms/it)
+  objective:  13326.022418661785
+Minimizing 24582    Time: 0:03:01 ( 7.39 ms/it)
+  objective:  13324.153400086827
+Minimizing 24597    Time: 0:03:01 ( 7.39 ms/it)
+  objective:  13324.150538522881
+Minimizing 24612    Time: 0:03:01 ( 7.39 ms/it)
+  objective:  13324.146447345862
+Minimizing 24627    Time: 0:03:02 ( 7.39 ms/it)
+  objective:  13324.14104121388
+Minimizing 24642    Time: 0:03:02 ( 7.39 ms/it)
+  objective:  13324.12903281604
+Minimizing 24657    Time: 0:03:02 ( 7.39 ms/it)
+  objective:  13324.12710478298
+Minimizing 24672    Time: 0:03:02 ( 7.39 ms/it)
+  objective:  13324.104210816498
+Minimizing 24687    Time: 0:03:02 ( 7.39 ms/it)
+  objective:  13324.102294374534
+Minimizing 24702    Time: 0:03:02 ( 7.39 ms/it)
+  objective:  13324.013521233443
+Minimizing 24717    Time: 0:03:02 ( 7.39 ms/it)
+  objective:  13324.012767639535
+Minimizing 24732    Time: 0:03:02 ( 7.39 ms/it)
+  objective:  13323.999600773197
+Minimizing 24747    Time: 0:03:02 ( 7.39 ms/it)
+  objective:  13323.973975600384
+Minimizing 24762    Time: 0:03:02 ( 7.39 ms/it)
+  objective:  13323.953986663531
+Minimizing 24777    Time: 0:03:03 ( 7.39 ms/it)
+  objective:  13323.951145742278
+Minimizing 24792    Time: 0:03:03 ( 7.39 ms/it)
+  objective:  13323.933557569748
+Minimizing 24807    Time: 0:03:03 ( 7.39 ms/it)
+  objective:  13323.926519979286
+Minimizing 24822    Time: 0:03:03 ( 7.39 ms/it)
+  objective:  13323.909402831603
+Minimizing 24837    Time: 0:03:03 ( 7.39 ms/it)
+  objective:  13323.908270431682
+Minimizing 24852    Time: 0:03:03 ( 7.39 ms/it)
+  objective:  13323.87601469834
+Minimizing 24867    Time: 0:03:03 ( 7.39 ms/it)
+  objective:  13323.84950467461
+Minimizing 24882    Time: 0:03:03 ( 7.39 ms/it)
+  objective:  13323.847874241706
+Minimizing 24897    Time: 0:03:03 ( 7.39 ms/it)
+  objective:  13323.837617182857
+Minimizing 24912    Time: 0:03:04 ( 7.39 ms/it)
+  objective:  13323.818408620544
+Minimizing 24927    Time: 0:03:04 ( 7.39 ms/it)
+  objective:  13323.815859885333
+Minimizing 24942    Time: 0:03:04 ( 7.39 ms/it)
+  objective:  13323.795033596762
+Minimizing 24957    Time: 0:03:04 ( 7.39 ms/it)
+  objective:  13323.793107847785
+Minimizing 24972    Time: 0:03:04 ( 7.39 ms/it)
+  objective:  13323.786818467648
+Minimizing 24987    Time: 0:03:04 ( 7.38 ms/it)
+  objective:  13323.767473702552
+Minimizing 25002    Time: 0:03:04 ( 7.38 ms/it)
+  objective:  13323.754195074507
+Minimizing 25017    Time: 0:03:04 ( 7.38 ms/it)
+  objective:  13323.74774496138
+Minimizing 25032    Time: 0:03:04 ( 7.38 ms/it)
+  objective:  13323.745692061973
+Minimizing 25047    Time: 0:03:04 ( 7.38 ms/it)
+  objective:  13323.714089051457
+Minimizing 25062    Time: 0:03:05 ( 7.38 ms/it)
+  objective:  13323.713284750862
+Minimizing 25077    Time: 0:03:05 ( 7.38 ms/it)
+  objective:  13323.70995258943
+Minimizing 25092    Time: 0:03:05 ( 7.38 ms/it)
+  objective:  13323.707830206738
+Minimizing 25107    Time: 0:03:05 ( 7.38 ms/it)
+  objective:  13323.695364852902
+Minimizing 25122    Time: 0:03:05 ( 7.38 ms/it)
+  objective:  13323.69076560215
+Minimizing 25137    Time: 0:03:05 ( 7.38 ms/it)
+  objective:  13323.666001855192
+Minimizing 25152    Time: 0:03:05 ( 7.38 ms/it)
+  objective:  13323.659685598323
+Minimizing 25167    Time: 0:03:05 ( 7.38 ms/it)
+  objective:  13323.657266785522
+Minimizing 25182    Time: 0:03:05 ( 7.38 ms/it)
+  objective:  13323.641496908589
+Minimizing 25197    Time: 0:03:05 ( 7.38 ms/it)
+  objective:  13323.638214224382
+Minimizing 25212    Time: 0:03:06 ( 7.38 ms/it)
+  objective:  13323.628734145575
+Minimizing 25227    Time: 0:03:06 ( 7.38 ms/it)
+  objective:  13323.600730227598
+Minimizing 25242    Time: 0:03:06 ( 7.38 ms/it)
+  objective:  13323.588046129662
+Minimizing 25257    Time: 0:03:06 ( 7.38 ms/it)
+  objective:  13323.579173321472
+Minimizing 25272    Time: 0:03:06 ( 7.38 ms/it)
+  objective:  13323.452811563227
+Minimizing 25287    Time: 0:03:06 ( 7.38 ms/it)
+  objective:  13323.433882145197
+Minimizing 25302    Time: 0:03:06 ( 7.38 ms/it)
+  objective:  13323.431114574167
+Minimizing 25317    Time: 0:03:06 ( 7.38 ms/it)
+  objective:  13323.422939608135
+Minimizing 25332    Time: 0:03:06 ( 7.38 ms/it)
+  objective:  13323.415516952664
+Minimizing 25347    Time: 0:03:07 ( 7.38 ms/it)
+  objective:  13323.408141641325
+Minimizing 25362    Time: 0:03:07 ( 7.38 ms/it)
+  objective:  13323.407468803576
+Minimizing 25377    Time: 0:03:07 ( 7.38 ms/it)
+  objective:  13323.406963321642
+Minimizing 25392    Time: 0:03:07 ( 7.38 ms/it)
+  objective:  13323.405562594504
+Minimizing 25407    Time: 0:03:07 ( 7.38 ms/it)
+  objective:  13323.39598296152
+Minimizing 25422    Time: 0:03:07 ( 7.38 ms/it)
+  objective:  13323.393840426506
+Minimizing 25437    Time: 0:03:07 ( 7.38 ms/it)
+  objective:  13323.39183965289
+Minimizing 25452    Time: 0:03:07 ( 7.38 ms/it)
+  objective:  13323.385370975171
+Minimizing 25467    Time: 0:03:07 ( 7.38 ms/it)
+  objective:  13323.373981211902
+Minimizing 25481    Time: 0:03:07 ( 7.38 ms/it)
+  objective:  13323.36608427098
+Minimizing 25496    Time: 0:03:08 ( 7.38 ms/it)
+  objective:  13323.332715147466
+Minimizing 25511    Time: 0:03:08 ( 7.38 ms/it)
+  objective:  13323.331194170503
+Minimizing 25526    Time: 0:03:08 ( 7.38 ms/it)
+  objective:  13323.321211081711
+Minimizing 25541    Time: 0:03:08 ( 7.38 ms/it)
+  objective:  13323.305323059656
+Minimizing 25556    Time: 0:03:08 ( 7.38 ms/it)
+  objective:  13323.30428709458
+Minimizing 25571    Time: 0:03:08 ( 7.38 ms/it)
+  objective:  13323.295611087058
+Minimizing 25586    Time: 0:03:08 ( 7.38 ms/it)
+  objective:  13323.280049537338
+Minimizing 25601    Time: 0:03:08 ( 7.38 ms/it)
+  objective:  13323.254074576238
+Minimizing 25615    Time: 0:03:08 ( 7.38 ms/it)
+  objective:  13323.253141506138
+Minimizing 25629    Time: 0:03:09 ( 7.38 ms/it)
+  objective:  13323.252279394059
+Minimizing 25644    Time: 0:03:09 ( 7.38 ms/it)
+  objective:  13323.245402390792
+Minimizing 25659    Time: 0:03:09 ( 7.38 ms/it)
+  objective:  13323.235658150064
+Minimizing 25674    Time: 0:03:09 ( 7.37 ms/it)
+  objective:  13323.220348235525
+Minimizing 25689    Time: 0:03:09 ( 7.37 ms/it)
+  objective:  13323.207788501983
+Minimizing 25704    Time: 0:03:09 ( 7.37 ms/it)
+  objective:  13323.201768893297
+Minimizing 25719    Time: 0:03:09 ( 7.37 ms/it)
+  objective:  13323.17476754119
+Minimizing 25734    Time: 0:03:09 ( 7.37 ms/it)
+  objective:  13323.092335446432
+Minimizing 25748    Time: 0:03:09 ( 7.37 ms/it)
+  objective:  13323.074852658014
+Minimizing 25762    Time: 0:03:09 ( 7.37 ms/it)
+  objective:  13323.07417329667
+Minimizing 25776    Time: 0:03:10 ( 7.37 ms/it)
+  objective:  13323.066033492622
+Minimizing 25790    Time: 0:03:10 ( 7.37 ms/it)
+  objective:  13323.057276168867
+Minimizing 25804    Time: 0:03:10 ( 7.37 ms/it)
+  objective:  13323.045625411774
+Minimizing 25818    Time: 0:03:10 ( 7.37 ms/it)
+  objective:  13323.037920157803
+Minimizing 25832    Time: 0:03:10 ( 7.37 ms/it)
+  objective:  13323.025756669085
+Minimizing 25846    Time: 0:03:10 ( 7.37 ms/it)
+  objective:  13323.003811378163
+Minimizing 25860    Time: 0:03:10 ( 7.37 ms/it)
+  objective:  13323.00200276084
+Minimizing 25874    Time: 0:03:10 ( 7.37 ms/it)
+  objective:  13322.980983352521
+Minimizing 25888    Time: 0:03:10 ( 7.37 ms/it)
+  objective:  13322.964120379023
+Minimizing 25902    Time: 0:03:11 ( 7.37 ms/it)
+  objective:  13322.938493517868
+Minimizing 25916    Time: 0:03:11 ( 7.37 ms/it)
+  objective:  13322.937282661922
+Minimizing 25930    Time: 0:03:11 ( 7.37 ms/it)
+  objective:  13322.925165767592
+Minimizing 25944    Time: 0:03:11 ( 7.37 ms/it)
+  objective:  13322.914264750201
+Minimizing 25958    Time: 0:03:11 ( 7.37 ms/it)
+  objective:  13322.893767596834
+Minimizing 25972    Time: 0:03:11 ( 7.37 ms/it)
+  objective:  13322.887569413055
+Minimizing 25986    Time: 0:03:11 ( 7.37 ms/it)
+  objective:  13322.856335111152
+Minimizing 26000    Time: 0:03:11 ( 7.37 ms/it)
+  objective:  13322.830045455325
+Minimizing 26014    Time: 0:03:11 ( 7.37 ms/it)
+  objective:  13322.829677535032
+Minimizing 26028    Time: 0:03:11 ( 7.37 ms/it)
+  objective:  13322.828527920763
+Minimizing 26042    Time: 0:03:12 ( 7.37 ms/it)
+  objective:  13322.823237409917
+Minimizing 26056    Time: 0:03:12 ( 7.37 ms/it)
+  objective:  13322.818195578366
+Minimizing 26070    Time: 0:03:12 ( 7.37 ms/it)
+  objective:  13322.80875493215
+Minimizing 26084    Time: 0:03:12 ( 7.37 ms/it)
+  objective:  13322.80598625989
+Minimizing 26098    Time: 0:03:12 ( 7.37 ms/it)
+  objective:  13322.793072501387
+Minimizing 26112    Time: 0:03:12 ( 7.37 ms/it)
+  objective:  13322.711680029097
+Minimizing 26126    Time: 0:03:12 ( 7.37 ms/it)
+  objective:  13322.622893114225
+Minimizing 26140    Time: 0:03:12 ( 7.37 ms/it)
+  objective:  13322.609232675371
+Minimizing 26154    Time: 0:03:12 ( 7.37 ms/it)
+  objective:  13322.60323604694
+Minimizing 26168    Time: 0:03:12 ( 7.37 ms/it)
+  objective:  13322.587258746164
+Minimizing 26182    Time: 0:03:13 ( 7.37 ms/it)
+  objective:  13322.585943856888
+Minimizing 26197    Time: 0:03:13 ( 7.37 ms/it)
+  objective:  13322.578621595632
+Minimizing 26212    Time: 0:03:13 ( 7.37 ms/it)
+  objective:  13322.564742343791
+Minimizing 26227    Time: 0:03:13 ( 7.37 ms/it)
+  objective:  13322.550643448398
+Minimizing 26242    Time: 0:03:13 ( 7.37 ms/it)
+  objective:  13322.397103768744
+Minimizing 26257    Time: 0:03:13 ( 7.37 ms/it)
+  objective:  13322.395288103522
+Minimizing 26272    Time: 0:03:13 ( 7.37 ms/it)
+  objective:  13322.392920041893
+Minimizing 26287    Time: 0:03:13 ( 7.37 ms/it)
+  objective:  13322.384667976541
+Minimizing 26302    Time: 0:03:13 ( 7.37 ms/it)
+  objective:  13322.367273587166
+Minimizing 26317    Time: 0:03:13 ( 7.37 ms/it)
+  objective:  13322.342793149131
+Minimizing 26332    Time: 0:03:14 ( 7.37 ms/it)
+  objective:  13322.333035099044
+Minimizing 26347    Time: 0:03:14 ( 7.37 ms/it)
+  objective:  13322.332014789368
+Minimizing 26362    Time: 0:03:14 ( 7.37 ms/it)
+  objective:  13322.331486568233
+Minimizing 26377    Time: 0:03:14 ( 7.37 ms/it)
+  objective:  13322.328950214986
+Minimizing 26392    Time: 0:03:14 ( 7.37 ms/it)
+  objective:  13322.321527398293
+Minimizing 26407    Time: 0:03:14 ( 7.37 ms/it)
+  objective:  13322.31607248084
+Minimizing 26422    Time: 0:03:14 ( 7.37 ms/it)
+  objective:  13322.301771162834
+Minimizing 26437    Time: 0:03:14 ( 7.37 ms/it)
+  objective:  13322.300793330141
+Minimizing 26452    Time: 0:03:14 ( 7.37 ms/it)
+  objective:  13322.29982655574
+Minimizing 26467    Time: 0:03:15 ( 7.37 ms/it)
+  objective:  13322.296720845508
+Minimizing 26482    Time: 0:03:15 ( 7.37 ms/it)
+  objective:  13322.292709230489
+Minimizing 26497    Time: 0:03:15 ( 7.37 ms/it)
+  objective:  13322.286665776453
+Minimizing 26512    Time: 0:03:15 ( 7.37 ms/it)
+  objective:  13322.285265880768
+Minimizing 26527    Time: 0:03:15 ( 7.37 ms/it)
+  objective:  13322.280546433743
+Minimizing 26542    Time: 0:03:15 ( 7.37 ms/it)
+  objective:  13322.251513132229
+Minimizing 26557    Time: 0:03:15 ( 7.37 ms/it)
+  objective:  13322.452238480808
+Minimizing 26572    Time: 0:03:15 ( 7.37 ms/it)
+  objective:  13322.208876201577
+Minimizing 26587    Time: 0:03:15 ( 7.37 ms/it)
+  objective:  13322.205290361206
+Minimizing 26602    Time: 0:03:15 ( 7.37 ms/it)
+  objective:  13322.202571761634
+Minimizing 26617    Time: 0:03:16 ( 7.37 ms/it)
+  objective:  13322.196502808802
+Minimizing 26632    Time: 0:03:16 ( 7.37 ms/it)
+  objective:  13322.194362599257
+Minimizing 26647    Time: 0:03:16 ( 7.37 ms/it)
+  objective:  13322.181679792324
+Minimizing 26662    Time: 0:03:16 ( 7.37 ms/it)
+  objective:  13322.179746557638
+Minimizing 26677    Time: 0:03:16 ( 7.37 ms/it)
+  objective:  13322.171555529203
+Minimizing 26691    Time: 0:03:16 ( 7.37 ms/it)
+  objective:  13322.15271407082
+Minimizing 26706    Time: 0:03:16 ( 7.37 ms/it)
+  objective:  13322.141665783522
+Minimizing 26721    Time: 0:03:16 ( 7.36 ms/it)
+  objective:  13322.141002141958
+Minimizing 26736    Time: 0:03:16 ( 7.36 ms/it)
+  objective:  13322.140011221854
+Minimizing 26751    Time: 0:03:17 ( 7.36 ms/it)
+  objective:  13322.138679229465
+Minimizing 26766    Time: 0:03:17 ( 7.36 ms/it)
+  objective:  13322.13798806598
+Minimizing 26781    Time: 0:03:17 ( 7.36 ms/it)
+  objective:  13322.131744615035
+Minimizing 26796    Time: 0:03:17 ( 7.36 ms/it)
+  objective:  13322.11933034641
+Minimizing 26811    Time: 0:03:17 ( 7.36 ms/it)
+  objective:  13322.11386716702
+Minimizing 26826    Time: 0:03:17 ( 7.36 ms/it)
+  objective:  13322.113270087648
+Minimizing 26841    Time: 0:03:17 ( 7.36 ms/it)
+  objective:  13322.109717672225
+Minimizing 26856    Time: 0:03:17 ( 7.36 ms/it)
+  objective:  13322.098246238951
+Minimizing 26871    Time: 0:03:17 ( 7.36 ms/it)
+  objective:  13322.086853060508
+Minimizing 26886    Time: 0:03:17 ( 7.36 ms/it)
+  objective:  13322.057586892319
+Minimizing 26901    Time: 0:03:18 ( 7.36 ms/it)
+  objective:  13322.006503733268
+Minimizing 26916    Time: 0:03:18 ( 7.36 ms/it)
+  objective:  13322.000420133685
+Minimizing 26931    Time: 0:03:18 ( 7.36 ms/it)
+  objective:  13321.996076839307
+Minimizing 26946    Time: 0:03:18 ( 7.36 ms/it)
+  objective:  13321.989806150494
+Minimizing 26961    Time: 0:03:18 ( 7.36 ms/it)
+  objective:  13321.983023296969
+Minimizing 26976    Time: 0:03:18 ( 7.36 ms/it)
+  objective:  13321.982359693138
+Minimizing 26991    Time: 0:03:18 ( 7.36 ms/it)
+  objective:  13321.971005187384
+Minimizing 27006    Time: 0:03:18 ( 7.36 ms/it)
+  objective:  13321.93727157425
+Minimizing 27021    Time: 0:03:18 ( 7.36 ms/it)
+  objective:  13321.775091008894
+Minimizing 27035    Time: 0:03:19 ( 7.36 ms/it)
+  objective:  13320.355020921808
+Minimizing 27050    Time: 0:03:19 ( 7.36 ms/it)
+  objective:  13320.268693353137
+Minimizing 27063    Time: 0:03:19 ( 7.36 ms/it)
+  objective:  13320.255492047756
+Minimizing 27077    Time: 0:03:19 ( 7.36 ms/it)
+  objective:  13320.24996317181
+Minimizing 27090    Time: 0:03:19 ( 7.36 ms/it)
+  objective:  13320.24833526052
+Minimizing 27104    Time: 0:03:19 ( 7.36 ms/it)
+  objective:  13320.247088204822
+Minimizing 27118    Time: 0:03:19 ( 7.36 ms/it)
+  objective:  13320.235097612196
+Minimizing 27131    Time: 0:03:19 ( 7.36 ms/it)
+  objective:  13320.216857939973
+Minimizing 27145    Time: 0:03:19 ( 7.36 ms/it)
+  objective:  13320.182347763417
+Minimizing 27158    Time: 0:03:19 ( 7.36 ms/it)
+  objective:  13320.135860622016
+Minimizing 27171    Time: 0:03:20 ( 7.36 ms/it)
+  objective:  13320.029316663145
+Minimizing 27185    Time: 0:03:20 ( 7.36 ms/it)
+  objective:  13320.02211984538
+Minimizing 27199    Time: 0:03:20 ( 7.36 ms/it)
+  objective:  13320.021497332142
+Minimizing 27213    Time: 0:03:20 ( 7.36 ms/it)
+  objective:  13320.017043822838
+Minimizing 27226    Time: 0:03:20 ( 7.36 ms/it)
+  objective:  13320.013329172769
+Minimizing 27240    Time: 0:03:20 ( 7.36 ms/it)
+  objective:  13320.008658601844
+Minimizing 27253    Time: 0:03:20 ( 7.36 ms/it)
+  objective:  13320.003166060706
+Minimizing 27266    Time: 0:03:20 ( 7.36 ms/it)
+  objective:  13319.996948459258
+Minimizing 27280    Time: 0:03:20 ( 7.36 ms/it)
+  objective:  13319.99377048327
+Minimizing 27293    Time: 0:03:20 ( 7.36 ms/it)
+  objective:  13319.998264845897
+Minimizing 27306    Time: 0:03:21 ( 7.36 ms/it)
+  objective:  13319.971517492624
+Minimizing 27319    Time: 0:03:21 ( 7.36 ms/it)
+  objective:  13319.953655871344
+Minimizing 27333    Time: 0:03:21 ( 7.36 ms/it)
+  objective:  13319.947685920648
+Minimizing 27347    Time: 0:03:21 ( 7.37 ms/it)
+  objective:  13319.941252851218
+Minimizing 27361    Time: 0:03:21 ( 7.37 ms/it)
+  objective:  13319.9323335433
+Minimizing 27374    Time: 0:03:21 ( 7.37 ms/it)
+  objective:  13319.913367418427
+Minimizing 27387    Time: 0:03:21 ( 7.37 ms/it)
+  objective:  13319.905690780812
+Minimizing 27401    Time: 0:03:21 ( 7.37 ms/it)
+  objective:  13319.875624269189
+Minimizing 27415    Time: 0:03:21 ( 7.37 ms/it)
+  objective:  13319.84675169368
+Minimizing 27428    Time: 0:03:22 ( 7.37 ms/it)
+  objective:  13319.846303897197
+Minimizing 27442    Time: 0:03:22 ( 7.37 ms/it)
+  objective:  13319.845749282773
+Minimizing 27456    Time: 0:03:22 ( 7.37 ms/it)
+  objective:  13319.845291399251
+Minimizing 27470    Time: 0:03:22 ( 7.37 ms/it)
+  objective:  13319.845051824494
+Minimizing 27484    Time: 0:03:22 ( 7.37 ms/it)
+  objective:  13319.844095454348
+Minimizing 27498    Time: 0:03:22 ( 7.37 ms/it)
+  objective:  13319.840598749506
+Minimizing 27512    Time: 0:03:22 ( 7.37 ms/it)
+  objective:  13319.837857473962
+Minimizing 27526    Time: 0:03:22 ( 7.37 ms/it)
+  objective:  13319.79612609424
+Minimizing 27540    Time: 0:03:22 ( 7.37 ms/it)
+  objective:  13319.789676178218
+Minimizing 27554    Time: 0:03:22 ( 7.37 ms/it)
+  objective:  13319.789506520086
+Minimizing 27568    Time: 0:03:23 ( 7.37 ms/it)
+  objective:  13319.789278508862
+Minimizing 27582    Time: 0:03:23 ( 7.37 ms/it)
+  objective:  13319.78813816674
+Minimizing 27596    Time: 0:03:23 ( 7.37 ms/it)
+  objective:  13319.787031867294
+Minimizing 27610    Time: 0:03:23 ( 7.37 ms/it)
+  objective:  13319.785944017975
+Minimizing 27624    Time: 0:03:23 ( 7.37 ms/it)
+  objective:  13319.781280139534
+Minimizing 27638    Time: 0:03:23 ( 7.37 ms/it)
+  objective:  13319.772868442597
+Minimizing 27652    Time: 0:03:23 ( 7.37 ms/it)
+  objective:  13319.772606038416
+Minimizing 27666    Time: 0:03:23 ( 7.37 ms/it)
+  objective:  13319.771758236166
+Minimizing 27680    Time: 0:03:23 ( 7.37 ms/it)
+  objective:  13319.769941226856
+Minimizing 27694    Time: 0:03:24 ( 7.37 ms/it)
+  objective:  13319.7686089208
+Minimizing 27708    Time: 0:03:24 ( 7.37 ms/it)
+  objective:  13319.768114253413
+Minimizing 27722    Time: 0:03:24 ( 7.37 ms/it)
+  objective:  13319.766768285743
+Minimizing 27736    Time: 0:03:24 ( 7.37 ms/it)
+  objective:  13319.76396185653
+Minimizing 27750    Time: 0:03:24 ( 7.37 ms/it)
+  objective:  13319.749242424266
+Minimizing 27764    Time: 0:03:24 ( 7.37 ms/it)
+  objective:  13319.73570178308
+Minimizing 27778    Time: 0:03:24 ( 7.37 ms/it)
+  objective:  13319.73507797248
+Minimizing 27792    Time: 0:03:24 ( 7.37 ms/it)
+  objective:  13319.732811269496
+Minimizing 27807    Time: 0:03:24 ( 7.37 ms/it)
+  objective:  13319.728503624225
+Minimizing 27821    Time: 0:03:24 ( 7.37 ms/it)
+  objective:  13319.72802864568
+Minimizing 27835    Time: 0:03:25 ( 7.37 ms/it)
+  objective:  13319.72743548028
+Minimizing 27849    Time: 0:03:25 ( 7.37 ms/it)
+  objective:  13319.727014199423
+Minimizing 27864    Time: 0:03:25 ( 7.37 ms/it)
+  objective:  13319.724385208174
+Minimizing 27879    Time: 0:03:25 ( 7.37 ms/it)
+  objective:  13319.71931295967
+Minimizing 27894    Time: 0:03:25 ( 7.37 ms/it)
+  objective:  13319.716267866985
+Minimizing 27909    Time: 0:03:25 ( 7.37 ms/it)
+  objective:  13319.710757417226
+Minimizing 27924    Time: 0:03:25 ( 7.37 ms/it)
+  objective:  13319.709103234767
+Minimizing 27939    Time: 0:03:25 ( 7.37 ms/it)
+  objective:  13319.70489650403
+Minimizing 27954    Time: 0:03:25 ( 7.37 ms/it)
+  objective:  13319.704015958108
+Minimizing 27968    Time: 0:03:26 ( 7.37 ms/it)
+  objective:  13319.701313402664
+Minimizing 27983    Time: 0:03:26 ( 7.37 ms/it)
+  objective:  13319.70070770607
+Minimizing 27998    Time: 0:03:26 ( 7.37 ms/it)
+  objective:  13319.699261675909
+Minimizing 28013    Time: 0:03:26 ( 7.37 ms/it)
+  objective:  13319.698619921619
+Minimizing 28028    Time: 0:03:26 ( 7.37 ms/it)
+  objective:  13319.694911960716
+Minimizing 28043    Time: 0:03:26 ( 7.37 ms/it)
+  objective:  13319.6916300117
+Minimizing 28058    Time: 0:03:26 ( 7.37 ms/it)
+  objective:  13319.690280105366
+Minimizing 28072    Time: 0:03:26 ( 7.37 ms/it)
+  objective:  13319.678269729804
+Minimizing 28087    Time: 0:03:26 ( 7.37 ms/it)
+  objective:  13319.677907532605
+Minimizing 28102    Time: 0:03:26 ( 7.37 ms/it)
+  objective:  13319.677675337283
+Minimizing 28117    Time: 0:03:27 ( 7.37 ms/it)
+  objective:  13319.67733594947
+Minimizing 28132    Time: 0:03:27 ( 7.36 ms/it)
+  objective:  13319.677177272402
+Minimizing 28147    Time: 0:03:27 ( 7.36 ms/it)
+  objective:  13319.675933531544
+Minimizing 28162    Time: 0:03:27 ( 7.36 ms/it)
+  objective:  13319.66441256176
+Minimizing 28177    Time: 0:03:27 ( 7.36 ms/it)
+  objective:  13319.657249074691
+Minimizing 28192    Time: 0:03:27 ( 7.36 ms/it)
+  objective:  13319.657064692568
+Minimizing 28207    Time: 0:03:27 ( 7.36 ms/it)
+  objective:  13319.655611829701
+Minimizing 28222    Time: 0:03:27 ( 7.36 ms/it)
+  objective:  13319.654655720806
+Minimizing 28237    Time: 0:03:27 ( 7.36 ms/it)
+  objective:  13319.654321062539
+Minimizing 28251    Time: 0:03:28 ( 7.36 ms/it)
+  objective:  13319.653769916084
+Minimizing 28266    Time: 0:03:28 ( 7.36 ms/it)
+  objective:  13319.653167271143
+Minimizing 28281    Time: 0:03:28 ( 7.36 ms/it)
+  objective:  13319.643104551564
+Minimizing 28296    Time: 0:03:28 ( 7.36 ms/it)
+  objective:  13319.642104129307
+Minimizing 28311    Time: 0:03:28 ( 7.36 ms/it)
+  objective:  13319.64105514645
+Minimizing 28326    Time: 0:03:28 ( 7.36 ms/it)
+  objective:  13319.640492291219
+Minimizing 28341    Time: 0:03:28 ( 7.36 ms/it)
+  objective:  13319.637611923332
+Minimizing 28356    Time: 0:03:28 ( 7.36 ms/it)
+  objective:  13319.637148376743
+Minimizing 28371    Time: 0:03:28 ( 7.36 ms/it)
+  objective:  13319.630214671299
+Minimizing 28385    Time: 0:03:28 ( 7.36 ms/it)
+  objective:  13319.62879007963
+Minimizing 28400    Time: 0:03:29 ( 7.36 ms/it)
+  objective:  13319.627111768408
+Minimizing 28415    Time: 0:03:29 ( 7.36 ms/it)
+  objective:  13319.623462026808
+Minimizing 28430    Time: 0:03:29 ( 7.36 ms/it)
+  objective:  13319.62319760515
+Minimizing 28445    Time: 0:03:29 ( 7.36 ms/it)
+  objective:  13319.622131633892
+Minimizing 28460    Time: 0:03:29 ( 7.36 ms/it)
+  objective:  13319.618381701745
+Minimizing 28475    Time: 0:03:29 ( 7.36 ms/it)
+  objective:  13319.617745313531
+Minimizing 28490    Time: 0:03:29 ( 7.36 ms/it)
+  objective:  13319.612934042554
+Minimizing 28505    Time: 0:03:29 ( 7.36 ms/it)
+  objective:  13319.60968476854
+Minimizing 28520    Time: 0:03:29 ( 7.36 ms/it)
+  objective:  13319.608559428685
+Minimizing 28535    Time: 0:03:30 ( 7.36 ms/it)
+  objective:  13319.603150398267
+Minimizing 28550    Time: 0:03:30 ( 7.36 ms/it)
+  objective:  13319.601729654212
+Minimizing 28565    Time: 0:03:30 ( 7.36 ms/it)
+  objective:  13319.59146858484
+Minimizing 28580    Time: 0:03:30 ( 7.36 ms/it)
+  objective:  13319.591104162624
+Minimizing 28595    Time: 0:03:30 ( 7.36 ms/it)
+  objective:  13319.585510293706
+Minimizing 28610    Time: 0:03:30 ( 7.36 ms/it)
+  objective:  13319.577368768718
+Minimizing 28625    Time: 0:03:30 ( 7.36 ms/it)
+  objective:  13319.577057157352
+Minimizing 28640    Time: 0:03:30 ( 7.36 ms/it)
+  objective:  13319.573558962365
+Minimizing 28655    Time: 0:03:30 ( 7.36 ms/it)
+  objective:  13319.573188172333
+Minimizing 28670    Time: 0:03:30 ( 7.36 ms/it)
+  objective:  13319.569496985481
+Minimizing 28685    Time: 0:03:31 ( 7.36 ms/it)
+  objective:  13319.541556845405
+Minimizing 28700    Time: 0:03:31 ( 7.36 ms/it)
+  objective:  13319.541100967792
+Minimizing 28715    Time: 0:03:31 ( 7.36 ms/it)
+  objective:  13319.539460200045
+Minimizing 28730    Time: 0:03:31 ( 7.36 ms/it)
+  objective:  13319.534000743093
+Minimizing 28745    Time: 0:03:31 ( 7.36 ms/it)
+  objective:  13319.533223673512
+Minimizing 28760    Time: 0:03:31 ( 7.36 ms/it)
+  objective:  13319.517272859826
+Minimizing 28775    Time: 0:03:31 ( 7.36 ms/it)
+  objective:  13319.506597837986
+Minimizing 28790    Time: 0:03:31 ( 7.36 ms/it)
+  objective:  13319.50530365578
+Minimizing 28805    Time: 0:03:31 ( 7.36 ms/it)
+  objective:  13319.50127408265
+Minimizing 28820    Time: 0:03:31 ( 7.35 ms/it)
+  objective:  13319.495029655955
+Minimizing 28835    Time: 0:03:32 ( 7.35 ms/it)
+  objective:  13319.493436708246
+Minimizing 28850    Time: 0:03:32 ( 7.35 ms/it)
+  objective:  13319.490607815882
+Minimizing 28865    Time: 0:03:32 ( 7.35 ms/it)
+  objective:  13319.483190313069
+Minimizing 28880    Time: 0:03:32 ( 7.35 ms/it)
+  objective:  13319.478792590773
+Minimizing 28895    Time: 0:03:32 ( 7.35 ms/it)
+  objective:  13319.476323713883
+Minimizing 28910    Time: 0:03:32 ( 7.35 ms/it)
+  objective:  13319.462226738498
+Minimizing 28925    Time: 0:03:32 ( 7.35 ms/it)
+  objective:  13319.46134095687
+Minimizing 28940    Time: 0:03:32 ( 7.35 ms/it)
+  objective:  13319.459095027385
+Minimizing 28955    Time: 0:03:32 ( 7.35 ms/it)
+  objective:  13319.457224911472
+Minimizing 28970    Time: 0:03:33 ( 7.35 ms/it)
+  objective:  13319.456653249435
+Minimizing 28985    Time: 0:03:33 ( 7.35 ms/it)
+  objective:  13319.447232531107
+Minimizing 29000    Time: 0:03:33 ( 7.35 ms/it)
+  objective:  13319.445363997816
+Minimizing 29015    Time: 0:03:33 ( 7.35 ms/it)
+  objective:  13319.441663757869
+Minimizing 29030    Time: 0:03:33 ( 7.35 ms/it)
+  objective:  13319.420885132626
+Minimizing 29045    Time: 0:03:33 ( 7.35 ms/it)
+  objective:  13319.414901625438
+Minimizing 29060    Time: 0:03:33 ( 7.35 ms/it)
+  objective:  13319.414382389965
+Minimizing 29075    Time: 0:03:33 ( 7.35 ms/it)
+  objective:  13319.412799412326
+Minimizing 29090    Time: 0:03:33 ( 7.35 ms/it)
+  objective:  13319.410662610084
+Minimizing 29105    Time: 0:03:33 ( 7.35 ms/it)
+  objective:  13319.392904156717
+Minimizing 29120    Time: 0:03:34 ( 7.35 ms/it)
+  objective:  13319.39287942015
+Minimizing 29135    Time: 0:03:34 ( 7.35 ms/it)
+  objective:  13319.392607632923
+Minimizing 29150    Time: 0:03:34 ( 7.35 ms/it)
+  objective:  13319.392009263844
+Minimizing 29165    Time: 0:03:34 ( 7.35 ms/it)
+  objective:  13319.391336786895
+Minimizing 29180    Time: 0:03:34 ( 7.35 ms/it)
+  objective:  13319.390675376315
+Minimizing 29195    Time: 0:03:34 ( 7.35 ms/it)
+  objective:  13319.383947805865
+Minimizing 29210    Time: 0:03:34 ( 7.35 ms/it)
+  objective:  13319.38274871571
+Minimizing 29225    Time: 0:03:34 ( 7.35 ms/it)
+  objective:  13319.38227535208
+Minimizing 29240    Time: 0:03:34 ( 7.35 ms/it)
+  objective:  13319.381353344506
+Minimizing 29254    Time: 0:03:35 ( 7.35 ms/it)
+  objective:  13319.380384236909
+Minimizing 29268    Time: 0:03:35 ( 7.35 ms/it)
+  objective:  13319.380001343772
+Minimizing 29283    Time: 0:03:35 ( 7.35 ms/it)
+  objective:  13319.375219436624
+Minimizing 29298    Time: 0:03:35 ( 7.35 ms/it)
+  objective:  13319.360260937756
+Minimizing 29313    Time: 0:03:35 ( 7.35 ms/it)
+  objective:  13319.360185300844
+Minimizing 29328    Time: 0:03:35 ( 7.35 ms/it)
+  objective:  13319.359162387744
+Minimizing 29343    Time: 0:03:35 ( 7.35 ms/it)
+  objective:  13319.357486142922
+Minimizing 29358    Time: 0:03:35 ( 7.35 ms/it)
+  objective:  13319.35687510611
+Minimizing 29373    Time: 0:03:35 ( 7.35 ms/it)
+  objective:  13319.350726837045
+Minimizing 29388    Time: 0:03:35 ( 7.35 ms/it)
+  objective:  13319.331905344297
+Minimizing 29403    Time: 0:03:36 ( 7.35 ms/it)
+  objective:  13319.331704520562
+Minimizing 29418    Time: 0:03:36 ( 7.35 ms/it)
+  objective:  13319.330348258285
+Minimizing 29433    Time: 0:03:36 ( 7.35 ms/it)
+  objective:  13319.325553988165
+Minimizing 29448    Time: 0:03:36 ( 7.35 ms/it)
+  objective:  13319.325371520972
+Minimizing 29463    Time: 0:03:36 ( 7.35 ms/it)
+  objective:  13319.322879812316
+Minimizing 29478    Time: 0:03:36 ( 7.35 ms/it)
+  objective:  13319.32194799732
+Minimizing 29493    Time: 0:03:36 ( 7.35 ms/it)
+  objective:  13319.321219206904
+Minimizing 29508    Time: 0:03:36 ( 7.35 ms/it)
+  objective:  13319.320024299479
+Minimizing 29523    Time: 0:03:36 ( 7.35 ms/it)
+  objective:  13319.317654963248
+Minimizing 29538    Time: 0:03:36 ( 7.35 ms/it)
+  objective:  13319.305325047899
+Minimizing 29553    Time: 0:03:37 ( 7.35 ms/it)
+  objective:  13319.300509318608
+Minimizing 29568    Time: 0:03:37 ( 7.35 ms/it)
+  objective:  13319.300171874464
+Minimizing 29583    Time: 0:03:37 ( 7.35 ms/it)
+  objective:  13319.299932244016
+Minimizing 29598    Time: 0:03:37 ( 7.35 ms/it)
+  objective:  13319.299345807565
+Minimizing 29613    Time: 0:03:37 ( 7.35 ms/it)
+  objective:  13319.298664570248
+Minimizing 29628    Time: 0:03:37 ( 7.35 ms/it)
+  objective:  13319.295059581244
+Minimizing 29643    Time: 0:03:37 ( 7.34 ms/it)
+  objective:  13319.280243673638
+Minimizing 29658    Time: 0:03:37 ( 7.34 ms/it)
+  objective:  13319.280134142595
+Minimizing 29673    Time: 0:03:37 ( 7.34 ms/it)
+  objective:  13319.279147828332
+Minimizing 29688    Time: 0:03:38 ( 7.34 ms/it)
+  objective:  13319.277990260598
+Minimizing 29703    Time: 0:03:38 ( 7.34 ms/it)
+  objective:  13319.27665077828
+Minimizing 29718    Time: 0:03:38 ( 7.34 ms/it)
+  objective:  13319.275810941253
+Minimizing 29733    Time: 0:03:38 ( 7.34 ms/it)
+  objective:  13319.267151519714
+Minimizing 29748    Time: 0:03:38 ( 7.34 ms/it)
+  objective:  13319.266617043555
+Minimizing 29763    Time: 0:03:38 ( 7.34 ms/it)
+  objective:  13319.265311843526
+Minimizing 29778    Time: 0:03:38 ( 7.34 ms/it)
+  objective:  13319.261467029908
+Minimizing 29793    Time: 0:03:38 ( 7.34 ms/it)
+  objective:  13319.252957024117
+Minimizing 29808    Time: 0:03:38 ( 7.34 ms/it)
+  objective:  13319.252680339341
+Minimizing 29823    Time: 0:03:38 ( 7.34 ms/it)
+  objective:  13319.252363490014
+Minimizing 29838    Time: 0:03:39 ( 7.34 ms/it)
+  objective:  13319.252082797291
+Minimizing 29853    Time: 0:03:39 ( 7.34 ms/it)
+  objective:  13319.249658341476
+Minimizing 29868    Time: 0:03:39 ( 7.34 ms/it)
+  objective:  13319.24663186555
+Minimizing 29883    Time: 0:03:39 ( 7.34 ms/it)
+  objective:  13319.245967011084
+Minimizing 29898    Time: 0:03:39 ( 7.34 ms/it)
+  objective:  13319.240411905295
+Minimizing 29913    Time: 0:03:39 ( 7.34 ms/it)
+  objective:  13319.238758139138
+Minimizing 29928    Time: 0:03:39 ( 7.34 ms/it)
+  objective:  13319.238407833996
+Minimizing 29943    Time: 0:03:39 ( 7.34 ms/it)
+  objective:  13319.237981691462
+Minimizing 29958    Time: 0:03:39 ( 7.34 ms/it)
+  objective:  13319.237674684482
+Minimizing 29973    Time: 0:03:40 ( 7.34 ms/it)
+  objective:  13319.235851250225
+Minimizing 29988    Time: 0:03:40 ( 7.34 ms/it)
+  objective:  13319.228648968856
+Minimizing 30003    Time: 0:03:40 ( 7.34 ms/it)
+  objective:  13319.228628546422
+Minimizing 30018    Time: 0:03:40 ( 7.34 ms/it)
+  objective:  13319.22806362201
+Minimizing 30033    Time: 0:03:40 ( 7.34 ms/it)
+  objective:  13319.22728526601
+Minimizing 30048    Time: 0:03:40 ( 7.34 ms/it)
+  objective:  13319.225981076728
+Minimizing 30063    Time: 0:03:40 ( 7.34 ms/it)
+  objective:  13319.22500575472
+Minimizing 30078    Time: 0:03:40 ( 7.34 ms/it)
+  objective:  13319.224119420294
+Minimizing 30093    Time: 0:03:40 ( 7.34 ms/it)
+  objective:  13319.22388071465
+Minimizing 30108    Time: 0:03:40 ( 7.34 ms/it)
+  objective:  13319.206780291599
+Minimizing 30123    Time: 0:03:41 ( 7.34 ms/it)
+  objective:  13319.17522597342
+Minimizing 30138    Time: 0:03:41 ( 7.34 ms/it)
+  objective:  13319.174902652987
+Minimizing 30153    Time: 0:03:41 ( 7.34 ms/it)
+  objective:  13319.173807306739
+Minimizing 30168    Time: 0:03:41 ( 7.34 ms/it)
+  objective:  13319.172805273483
+Minimizing 30183    Time: 0:03:41 ( 7.34 ms/it)
+  objective:  13319.17138143997
+Minimizing 30198    Time: 0:03:41 ( 7.34 ms/it)
+  objective:  13319.166776528771
+Minimizing 30213    Time: 0:03:41 ( 7.34 ms/it)
+  objective:  13319.160868803054
+Minimizing 30228    Time: 0:03:41 ( 7.34 ms/it)
+  objective:  13319.147296470823
+Minimizing 30243    Time: 0:03:41 ( 7.34 ms/it)
+  objective:  13319.12987647175
+Minimizing 30258    Time: 0:03:41 ( 7.34 ms/it)
+  objective:  13319.087285741334
+Minimizing 30273    Time: 0:03:42 ( 7.34 ms/it)
+  objective:  13319.086711761018
+Minimizing 30288    Time: 0:03:42 ( 7.34 ms/it)
+  objective:  13319.08369201218
+Minimizing 30303    Time: 0:03:42 ( 7.34 ms/it)
+  objective:  13319.08128603507
+Minimizing 30318    Time: 0:03:42 ( 7.34 ms/it)
+  objective:  13319.080915554194
+Minimizing 30333    Time: 0:03:42 ( 7.34 ms/it)
+  objective:  13319.072859063032
+Minimizing 30348    Time: 0:03:42 ( 7.34 ms/it)
+  objective:  13319.059522382813
+Minimizing 30362    Time: 0:03:42 ( 7.34 ms/it)
+  objective:  13318.984527460343
+Minimizing 30377    Time: 0:03:42 ( 7.34 ms/it)
+  objective:  13318.951953823882
+Minimizing 30392    Time: 0:03:42 ( 7.34 ms/it)
+  objective:  13318.951673342744
+Minimizing 30407    Time: 0:03:43 ( 7.33 ms/it)
+  objective:  13318.950388999918
+Minimizing 30422    Time: 0:03:43 ( 7.33 ms/it)
+  objective:  13318.949722401056
+Minimizing 30437    Time: 0:03:43 ( 7.33 ms/it)
+  objective:  13318.949379876518
+Minimizing 30452    Time: 0:03:43 ( 7.33 ms/it)
+  objective:  13318.948498774116
+Minimizing 30467    Time: 0:03:43 ( 7.33 ms/it)
+  objective:  13318.94424841879
+Minimizing 30482    Time: 0:03:43 ( 7.33 ms/it)
+  objective:  13318.938211981294
+Minimizing 30497    Time: 0:03:43 ( 7.33 ms/it)
+  objective:  13318.937600586985
+Minimizing 30512    Time: 0:03:43 ( 7.33 ms/it)
+  objective:  13318.926512712482
+Minimizing 30528    Time: 0:03:43 ( 7.33 ms/it)
+  objective:  13318.925488212335
+Minimizing 30543    Time: 0:03:43 ( 7.33 ms/it)
+  objective:  13318.921620370485
+Minimizing 30558    Time: 0:03:44 ( 7.33 ms/it)
+  objective:  13318.920561537787
+Minimizing 30573    Time: 0:03:44 ( 7.33 ms/it)
+  objective:  13318.919127931265
+Minimizing 30588    Time: 0:03:44 ( 7.33 ms/it)
+  objective:  13318.913355053723
+Minimizing 30603    Time: 0:03:44 ( 7.33 ms/it)
+  objective:  13318.894152780296
+Minimizing 30616    Time: 0:03:44 ( 7.33 ms/it)
+  objective:  13318.892957417469
+Minimizing 30631    Time: 0:03:44 ( 7.33 ms/it)
+  objective:  13318.892085047803
+Minimizing 30646    Time: 0:03:44 ( 7.33 ms/it)
+  objective:  13318.89120573389
+Minimizing 30661    Time: 0:03:44 ( 7.33 ms/it)
+  objective:  13318.890321445535
+Minimizing 30676    Time: 0:03:44 ( 7.33 ms/it)
+  objective:  13318.887567563754
+Minimizing 30691    Time: 0:03:45 ( 7.33 ms/it)
+  objective:  13318.88701855982
+Minimizing 30705    Time: 0:03:45 ( 7.33 ms/it)
+  objective:  13318.8825619954
+Minimizing 30719    Time: 0:03:45 ( 7.33 ms/it)
+  objective:  13318.877883625493
+Minimizing 30734    Time: 0:03:45 ( 7.33 ms/it)
+  objective:  13318.875786700592
+Minimizing 30749    Time: 0:03:45 ( 7.33 ms/it)
+  objective:  13318.85820491056
+Minimizing 30765    Time: 0:03:45 ( 7.33 ms/it)
+  objective:  13318.857843792983
+Minimizing 30781    Time: 0:03:45 ( 7.33 ms/it)
+  objective:  13318.85461594892
+Minimizing 30797    Time: 0:03:45 ( 7.33 ms/it)
+  objective:  13318.851330874706
+Minimizing 30812    Time: 0:03:45 ( 7.33 ms/it)
+  objective:  13318.828961918189
+Minimizing 30827    Time: 0:03:45 ( 7.33 ms/it)
+  objective:  13318.823229199974
+Minimizing 30842    Time: 0:03:46 ( 7.33 ms/it)
+  objective:  13318.822322328968
+Minimizing 30857    Time: 0:03:46 ( 7.33 ms/it)
+  objective:  13318.817577720503
+Minimizing 30872    Time: 0:03:46 ( 7.33 ms/it)
+  objective:  13318.81217702775
+Minimizing 30887    Time: 0:03:46 ( 7.33 ms/it)
+  objective:  13318.809610448792
+Minimizing 30902    Time: 0:03:46 ( 7.33 ms/it)
+  objective:  13318.809001793401
+Minimizing 30917    Time: 0:03:46 ( 7.33 ms/it)
+  objective:  13318.807110029331
+Minimizing 30932    Time: 0:03:46 ( 7.33 ms/it)
+  objective:  13318.799004679458
+Minimizing 30947    Time: 0:03:46 ( 7.33 ms/it)
+  objective:  13318.797360402808
+Minimizing 30962    Time: 0:03:46 ( 7.33 ms/it)
+  objective:  13318.796594720654
+Minimizing 30977    Time: 0:03:47 ( 7.33 ms/it)
+  objective:  13318.796140296763
+Minimizing 30992    Time: 0:03:47 ( 7.33 ms/it)
+  objective:  13318.795310017842
+Minimizing 31007    Time: 0:03:47 ( 7.33 ms/it)
+  objective:  13318.794545561163
+Minimizing 31022    Time: 0:03:47 ( 7.33 ms/it)
+  objective:  13318.785633063919
+Minimizing 31037    Time: 0:03:47 ( 7.33 ms/it)
+  objective:  13318.768674989027
+Minimizing 31052    Time: 0:03:47 ( 7.33 ms/it)
+  objective:  13318.767123796817
+Minimizing 31067    Time: 0:03:47 ( 7.33 ms/it)
+  objective:  13318.76304193525
+Minimizing 31082    Time: 0:03:47 ( 7.33 ms/it)
+  objective:  13318.761402975302
+Minimizing 31097    Time: 0:03:47 ( 7.33 ms/it)
+  objective:  13318.758187083324
+Minimizing 31112    Time: 0:03:47 ( 7.33 ms/it)
+  objective:  13318.746061366153
+Minimizing 31127    Time: 0:03:48 ( 7.33 ms/it)
+  objective:  13318.736358005015
+Minimizing 31142    Time: 0:03:48 ( 7.33 ms/it)
+  objective:  13318.736107113873
+Minimizing 31157    Time: 0:03:48 ( 7.33 ms/it)
+  objective:  13318.73301025531
+Minimizing 31172    Time: 0:03:48 ( 7.33 ms/it)
+  objective:  13318.730416240898
+Minimizing 31187    Time: 0:03:48 ( 7.33 ms/it)
+  objective:  13318.718362650325
+Minimizing 31202    Time: 0:03:48 ( 7.33 ms/it)
+  objective:  13318.71647735151
+Minimizing 31217    Time: 0:03:48 ( 7.33 ms/it)
+  objective:  13318.710605837085
+Minimizing 31232    Time: 0:03:48 ( 7.33 ms/it)
+  objective:  13318.696013906883
+Minimizing 31247    Time: 0:03:48 ( 7.32 ms/it)
+  objective:  13318.591494342429
+Minimizing 31262    Time: 0:03:48 ( 7.32 ms/it)
+  objective:  13318.555966758504
+Minimizing 31277    Time: 0:03:49 ( 7.32 ms/it)
+  objective:  13318.555690192545
+Minimizing 31292    Time: 0:03:49 ( 7.32 ms/it)
+  objective:  13318.555215810848
+Minimizing 31306    Time: 0:03:49 ( 7.32 ms/it)
+  objective:  13318.555073483702
+Minimizing 31320    Time: 0:03:49 ( 7.32 ms/it)
+  objective:  13318.553965392071
+Minimizing 31335    Time: 0:03:49 ( 7.32 ms/it)
+  objective:  13318.552727107133
+Minimizing 31349    Time: 0:03:49 ( 7.32 ms/it)
+  objective:  13318.552189863258
+Minimizing 31363    Time: 0:03:49 ( 7.32 ms/it)
+  objective:  13318.549193947183
+Minimizing 31378    Time: 0:03:49 ( 7.32 ms/it)
+  objective:  13318.511521852575
+Minimizing 31392    Time: 0:03:49 ( 7.32 ms/it)
+  objective:  13318.504667198911
+Minimizing 31406    Time: 0:03:50 ( 7.32 ms/it)
+  objective:  13318.504362174921
+Minimizing 31420    Time: 0:03:50 ( 7.32 ms/it)
+  objective:  13318.50402450448
+Minimizing 31434    Time: 0:03:50 ( 7.32 ms/it)
+  objective:  13318.503791047871
+Minimizing 31448    Time: 0:03:50 ( 7.32 ms/it)
+  objective:  13318.50336429839
+Minimizing 31463    Time: 0:03:50 ( 7.32 ms/it)
+  objective:  13318.503278658653
+Minimizing 31477    Time: 0:03:50 ( 7.32 ms/it)
+  objective:  13318.500972039416
+Minimizing 31492    Time: 0:03:50 ( 7.32 ms/it)
+  objective:  13318.496195650063
+Minimizing 31507    Time: 0:03:50 ( 7.32 ms/it)
+  objective:  13318.49520516509
+Minimizing 31522    Time: 0:03:50 ( 7.32 ms/it)
+  objective:  13318.490938155577
+Minimizing 31536    Time: 0:03:50 ( 7.32 ms/it)
+  objective:  13318.490180608147
+Minimizing 31550    Time: 0:03:51 ( 7.32 ms/it)
+  objective:  13318.489932952696
+Minimizing 31564    Time: 0:03:51 ( 7.32 ms/it)
+  objective:  13318.489744292747
+Minimizing 31577    Time: 0:03:51 ( 7.32 ms/it)
+  objective:  13318.488708573757
+Minimizing 31590    Time: 0:03:51 ( 7.32 ms/it)
+  objective:  13318.480740670144
+Minimizing 31604    Time: 0:03:51 ( 7.32 ms/it)
+  objective:  13318.48009399086
+Minimizing 31618    Time: 0:03:51 ( 7.32 ms/it)
+  objective:  13318.47979039482
+Minimizing 31633    Time: 0:03:51 ( 7.32 ms/it)
+  objective:  13318.479098601936
+Minimizing 31648    Time: 0:03:51 ( 7.32 ms/it)
+  objective:  13318.478548612882
+Minimizing 31663    Time: 0:03:51 ( 7.32 ms/it)
+  objective:  13318.468912793163
+Minimizing 31678    Time: 0:03:52 ( 7.32 ms/it)
+  objective:  13318.467061815405
+Minimizing 31693    Time: 0:03:52 ( 7.32 ms/it)
+  objective:  13318.466138965552
+Minimizing 31708    Time: 0:03:52 ( 7.32 ms/it)
+  objective:  13318.465292640598
+Minimizing 31723    Time: 0:03:52 ( 7.32 ms/it)
+  objective:  13318.461608042853
+Minimizing 31738    Time: 0:03:52 ( 7.32 ms/it)
+  objective:  13318.46111039797
+Minimizing 31753    Time: 0:03:52 ( 7.32 ms/it)
+  objective:  13318.46022822574
+Minimizing 31768    Time: 0:03:52 ( 7.32 ms/it)
+  objective:  13318.456600592312
+Minimizing 31783    Time: 0:03:52 ( 7.32 ms/it)
+  objective:  13318.453648369948
+Minimizing 31798    Time: 0:03:52 ( 7.32 ms/it)
+  objective:  13318.442545409358
+Minimizing 31813    Time: 0:03:52 ( 7.32 ms/it)
+  objective:  13318.440104803143
+Minimizing 31828    Time: 0:03:53 ( 7.32 ms/it)
+  objective:  13318.437657174305
+Minimizing 31843    Time: 0:03:53 ( 7.32 ms/it)
+  objective:  13318.436943231121
+Minimizing 31858    Time: 0:03:53 ( 7.32 ms/it)
+  objective:  13318.43442024423
+Minimizing 31873    Time: 0:03:53 ( 7.32 ms/it)
+  objective:  13318.427245381637
+Minimizing 31888    Time: 0:03:53 ( 7.32 ms/it)
+  objective:  13318.426920206577
+Minimizing 31903    Time: 0:03:53 ( 7.32 ms/it)
+  objective:  13318.426926414817
+Minimizing 31918    Time: 0:03:53 ( 7.32 ms/it)
+  objective:  13318.426870108146
+Minimizing 31932    Time: 0:03:53 ( 7.32 ms/it)
+  objective:  13318.42685512139
+Minimizing 31946    Time: 0:03:53 ( 7.32 ms/it)
+  objective:  13318.42687163921
+Minimizing 31960    Time: 0:03:53 ( 7.32 ms/it)
+  objective:  13318.426863760804
+Minimizing 31975    Time: 0:03:54 ( 7.32 ms/it)
+  objective:  13318.426859661398
+Minimizing 31990    Time: 0:03:54 ( 7.32 ms/it)
+  objective:  13318.426850211632
+Minimizing 32005    Time: 0:03:54 ( 7.32 ms/it)
+  objective:  13318.426848752657
+Minimizing 32020    Time: 0:03:54 ( 7.32 ms/it)
+  objective:  13318.426842984656
+Minimizing 32035    Time: 0:03:54 ( 7.32 ms/it)
+  objective:  13318.42683950116
+Minimizing 32050    Time: 0:03:54 ( 7.32 ms/it)
+  objective:  13318.426846553382
+Minimizing 32065    Time: 0:03:54 ( 7.32 ms/it)
+  objective:  13318.426840526998
+Minimizing 32081    Time: 0:03:54 ( 7.32 ms/it)
+  objective:  13318.426838949803
+Minimizing 32096    Time: 0:03:54 ( 7.32 ms/it)
+  objective:  13318.426848268617
+Minimizing 32100    Time: 0:03:54 ( 7.32 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01830.0139-1.320.1876
Test: Star-Run-0.01420.0438-0.320.7465
Test: S20-Star-0.01110.0409-0.270.7862
Test: SLJ-S200.00990.04420.220.8236
Test: BPT-SLJ-0.02310.0422-0.550.5836
a10.31670.04766.65<1e-10
Sex: Boys0.20480.013914.73<1e-48
Test: Star-Run & a10.29860.15051.980.0472
Test: S20-Star & a10.04940.14190.350.7275
Test: SLJ-S20 & a1-0.06580.1520-0.430.6649
Test: BPT-SLJ & a10.31510.14162.230.0261
Test: Star-Run & Sex: Boys-0.12130.0438-2.770.0056
Test: S20-Star & Sex: Boys0.06640.04091.630.1041
Test: SLJ-S20 & Sex: Boys0.04660.04421.050.2919
Test: BPT-SLJ & Sex: Boys0.16340.04223.870.0001
a1 & Sex: Boys0.05020.04761.060.2912
Test: Star-Run & a1 & Sex: Boys-0.01140.1505-0.080.9395
Test: S20-Star & a1 & Sex: Boys-0.02880.1419-0.200.8390
Test: SLJ-S20 & a1 & Sex: Boys-0.06030.1520-0.400.6916
Test: BPT-SLJ & a1 & Sex: Boys0.14150.14161.000.3178
Test: BPT0.9054
Test: SLJ0.9717
Test: Star_r0.9914
Test: Run0.9717
Test: S20_r0.9786
Residual0.0003
+
+
+
+
m_cpx_0_SeqDiff = let
+  f_cpx_0 = @formula(
+    zScore ~ 1 + Test * a1 * Sex + (0 + Test | Child)
+  )
+  fit(MixedModel, f_cpx_0, dat; contrasts=contr1b)
+end
+
+
Minimizing 15    Time: 0:00:00 ( 6.84 ms/it)
+  objective:  13919.774893920156
+Minimizing 30    Time: 0:00:00 ( 6.78 ms/it)
+  objective:  13934.06374034128
+Minimizing 45    Time: 0:00:00 ( 6.76 ms/it)
+  objective:  13808.794217331397
+Minimizing 60    Time: 0:00:00 ( 6.79 ms/it)
+  objective:  13807.163430095352
+Minimizing 75    Time: 0:00:00 ( 6.78 ms/it)
+  objective:  13805.380843030245
+Minimizing 90    Time: 0:00:00 ( 6.77 ms/it)
+  objective:  13796.147954658398
+Minimizing 105    Time: 0:00:00 ( 6.78 ms/it)
+  objective:  13789.167221291498
+Minimizing 120    Time: 0:00:00 ( 6.78 ms/it)
+  objective:  13785.10690625065
+Minimizing 135    Time: 0:00:00 ( 6.78 ms/it)
+  objective:  13770.622002111955
+Minimizing 150    Time: 0:00:01 ( 6.78 ms/it)
+  objective:  13756.957204824645
+Minimizing 165    Time: 0:00:01 ( 6.78 ms/it)
+  objective:  13749.738906704768
+Minimizing 180    Time: 0:00:01 ( 6.77 ms/it)
+  objective:  13749.022661135381
+Minimizing 195    Time: 0:00:01 ( 6.77 ms/it)
+  objective:  13743.450583952044
+Minimizing 210    Time: 0:00:01 ( 6.77 ms/it)
+  objective:  13737.371538731917
+Minimizing 225    Time: 0:00:01 ( 6.77 ms/it)
+  objective:  13734.843831763124
+Minimizing 240    Time: 0:00:01 ( 6.76 ms/it)
+  objective:  13726.12535518125
+Minimizing 255    Time: 0:00:01 ( 6.76 ms/it)
+  objective:  13718.079823864355
+Minimizing 270    Time: 0:00:01 ( 6.76 ms/it)
+  objective:  13713.7998795954
+Minimizing 285    Time: 0:00:01 ( 6.76 ms/it)
+  objective:  13708.429833563376
+Minimizing 300    Time: 0:00:02 ( 6.76 ms/it)
+  objective:  13706.19462994505
+Minimizing 315    Time: 0:00:02 ( 6.76 ms/it)
+  objective:  13702.526901215406
+Minimizing 330    Time: 0:00:02 ( 6.75 ms/it)
+  objective:  13701.407982621175
+Minimizing 345    Time: 0:00:02 ( 6.75 ms/it)
+  objective:  13697.216752981687
+Minimizing 360    Time: 0:00:02 ( 6.76 ms/it)
+  objective:  13687.964778485086
+Minimizing 375    Time: 0:00:02 ( 6.76 ms/it)
+  objective:  13680.159437377395
+Minimizing 390    Time: 0:00:02 ( 6.76 ms/it)
+  objective:  13673.285851322282
+Minimizing 405    Time: 0:00:02 ( 6.75 ms/it)
+  objective:  13667.51095094907
+Minimizing 420    Time: 0:00:02 ( 6.75 ms/it)
+  objective:  13676.351853648895
+Minimizing 435    Time: 0:00:02 ( 6.75 ms/it)
+  objective:  13648.233972394144
+Minimizing 450    Time: 0:00:03 ( 6.75 ms/it)
+  objective:  13646.289861123538
+Minimizing 465    Time: 0:00:03 ( 6.75 ms/it)
+  objective:  13639.024506617558
+Minimizing 480    Time: 0:00:03 ( 6.75 ms/it)
+  objective:  13634.782904938736
+Minimizing 495    Time: 0:00:03 ( 6.75 ms/it)
+  objective:  13630.423725385735
+Minimizing 510    Time: 0:00:03 ( 6.75 ms/it)
+  objective:  13618.510459066481
+Minimizing 525    Time: 0:00:03 ( 6.75 ms/it)
+  objective:  13617.232596673148
+Minimizing 540    Time: 0:00:03 ( 6.75 ms/it)
+  objective:  13615.641902185882
+Minimizing 555    Time: 0:00:03 ( 6.75 ms/it)
+  objective:  13611.868963963523
+Minimizing 570    Time: 0:00:03 ( 6.75 ms/it)
+  objective:  13609.090749091069
+Minimizing 585    Time: 0:00:03 ( 6.75 ms/it)
+  objective:  13601.752460463606
+Minimizing 600    Time: 0:00:04 ( 6.75 ms/it)
+  objective:  13600.331198118234
+Minimizing 615    Time: 0:00:04 ( 6.75 ms/it)
+  objective:  13599.201972133687
+Minimizing 630    Time: 0:00:04 ( 6.75 ms/it)
+  objective:  13597.07032570483
+Minimizing 645    Time: 0:00:04 ( 6.75 ms/it)
+  objective:  13586.41672012295
+Minimizing 660    Time: 0:00:04 ( 6.75 ms/it)
+  objective:  13585.84425906483
+Minimizing 675    Time: 0:00:04 ( 6.75 ms/it)
+  objective:  13584.30181207391
+Minimizing 690    Time: 0:00:04 ( 6.75 ms/it)
+  objective:  13583.156506241612
+Minimizing 705    Time: 0:00:04 ( 6.75 ms/it)
+  objective:  13582.366929028874
+Minimizing 720    Time: 0:00:04 ( 6.75 ms/it)
+  objective:  13577.470939937491
+Minimizing 735    Time: 0:00:04 ( 6.75 ms/it)
+  objective:  13575.814437644036
+Minimizing 750    Time: 0:00:05 ( 6.75 ms/it)
+  objective:  13572.849954089357
+Minimizing 765    Time: 0:00:05 ( 6.75 ms/it)
+  objective:  13568.097652526605
+Minimizing 780    Time: 0:00:05 ( 6.75 ms/it)
+  objective:  13563.483166512051
+Minimizing 795    Time: 0:00:05 ( 6.75 ms/it)
+  objective:  13560.757080004918
+Minimizing 810    Time: 0:00:05 ( 6.75 ms/it)
+  objective:  13553.878850962861
+Minimizing 825    Time: 0:00:05 ( 6.75 ms/it)
+  objective:  13549.3931844344
+Minimizing 840    Time: 0:00:05 ( 6.75 ms/it)
+  objective:  13546.2450776345
+Minimizing 855    Time: 0:00:05 ( 6.75 ms/it)
+  objective:  13540.393957147226
+Minimizing 870    Time: 0:00:05 ( 6.75 ms/it)
+  objective:  13536.78564915241
+Minimizing 885    Time: 0:00:05 ( 6.75 ms/it)
+  objective:  13517.485461952754
+Minimizing 900    Time: 0:00:06 ( 6.75 ms/it)
+  objective:  13516.743165448926
+Minimizing 915    Time: 0:00:06 ( 6.75 ms/it)
+  objective:  13513.522331677901
+Minimizing 930    Time: 0:00:06 ( 6.75 ms/it)
+  objective:  13509.079679671442
+Minimizing 945    Time: 0:00:06 ( 6.75 ms/it)
+  objective:  13508.197049533424
+Minimizing 960    Time: 0:00:06 ( 6.75 ms/it)
+  objective:  13507.527489509594
+Minimizing 975    Time: 0:00:06 ( 6.74 ms/it)
+  objective:  13506.268276394374
+Minimizing 990    Time: 0:00:06 ( 6.74 ms/it)
+  objective:  13505.98630900788
+Minimizing 1005    Time: 0:00:06 ( 6.74 ms/it)
+  objective:  13503.66746683239
+Minimizing 1020    Time: 0:00:06 ( 6.74 ms/it)
+  objective:  13503.33522657401
+Minimizing 1035    Time: 0:00:06 ( 6.74 ms/it)
+  objective:  13500.303539770604
+Minimizing 1050    Time: 0:00:07 ( 6.75 ms/it)
+  objective:  13498.361545832799
+Minimizing 1065    Time: 0:00:07 ( 6.75 ms/it)
+  objective:  13495.652365547001
+Minimizing 1080    Time: 0:00:07 ( 6.75 ms/it)
+  objective:  13494.420022849823
+Minimizing 1095    Time: 0:00:07 ( 6.75 ms/it)
+  objective:  13493.987817923211
+Minimizing 1111    Time: 0:00:07 ( 6.76 ms/it)
+  objective:  13492.931869125729
+Minimizing 1127    Time: 0:00:07 ( 6.76 ms/it)
+  objective:  13492.158798502358
+Minimizing 1142    Time: 0:00:07 ( 6.76 ms/it)
+  objective:  13491.954239806008
+Minimizing 1157    Time: 0:00:07 ( 6.76 ms/it)
+  objective:  13489.119521278852
+Minimizing 1172    Time: 0:00:07 ( 6.76 ms/it)
+  objective:  13487.798822978679
+Minimizing 1187    Time: 0:00:08 ( 6.75 ms/it)
+  objective:  13485.250482008072
+Minimizing 1202    Time: 0:00:08 ( 6.76 ms/it)
+  objective:  13484.736374020526
+Minimizing 1217    Time: 0:00:08 ( 6.75 ms/it)
+  objective:  13484.422177126231
+Minimizing 1232    Time: 0:00:08 ( 6.75 ms/it)
+  objective:  13484.079088188613
+Minimizing 1247    Time: 0:00:08 ( 6.75 ms/it)
+  objective:  13483.848685171259
+Minimizing 1262    Time: 0:00:08 ( 6.75 ms/it)
+  objective:  13483.042934731639
+Minimizing 1277    Time: 0:00:08 ( 6.75 ms/it)
+  objective:  13481.598443091214
+Minimizing 1292    Time: 0:00:08 ( 6.75 ms/it)
+  objective:  13480.879115890944
+Minimizing 1307    Time: 0:00:08 ( 6.75 ms/it)
+  objective:  13479.402071573073
+Minimizing 1322    Time: 0:00:08 ( 6.75 ms/it)
+  objective:  13478.46694483886
+Minimizing 1337    Time: 0:00:09 ( 6.75 ms/it)
+  objective:  13477.76733882117
+Minimizing 1352    Time: 0:00:09 ( 6.75 ms/it)
+  objective:  13477.356396287898
+Minimizing 1367    Time: 0:00:09 ( 6.75 ms/it)
+  objective:  13476.73797488567
+Minimizing 1382    Time: 0:00:09 ( 6.75 ms/it)
+  objective:  13476.608043450935
+Minimizing 1397    Time: 0:00:09 ( 6.75 ms/it)
+  objective:  13475.094820783583
+Minimizing 1412    Time: 0:00:09 ( 6.75 ms/it)
+  objective:  13473.918005475272
+Minimizing 1427    Time: 0:00:09 ( 6.75 ms/it)
+  objective:  13473.409871329422
+Minimizing 1442    Time: 0:00:09 ( 6.75 ms/it)
+  objective:  13471.7466789237
+Minimizing 1457    Time: 0:00:09 ( 6.75 ms/it)
+  objective:  13471.105716562059
+Minimizing 1472    Time: 0:00:09 ( 6.75 ms/it)
+  objective:  13470.17220789985
+Minimizing 1487    Time: 0:00:10 ( 6.75 ms/it)
+  objective:  13470.034715340327
+Minimizing 1502    Time: 0:00:10 ( 6.75 ms/it)
+  objective:  13469.476169595357
+Minimizing 1517    Time: 0:00:10 ( 6.75 ms/it)
+  objective:  13465.799769624056
+Minimizing 1532    Time: 0:00:10 ( 6.75 ms/it)
+  objective:  13464.496539541913
+Minimizing 1547    Time: 0:00:10 ( 6.75 ms/it)
+  objective:  13464.064225560396
+Minimizing 1562    Time: 0:00:10 ( 6.75 ms/it)
+  objective:  13463.637813806818
+Minimizing 1577    Time: 0:00:10 ( 6.75 ms/it)
+  objective:  13463.169545751429
+Minimizing 1592    Time: 0:00:10 ( 6.75 ms/it)
+  objective:  13462.968788733539
+Minimizing 1607    Time: 0:00:10 ( 6.75 ms/it)
+  objective:  13462.24264395712
+Minimizing 1622    Time: 0:00:10 ( 6.75 ms/it)
+  objective:  13461.954158376597
+Minimizing 1637    Time: 0:00:11 ( 6.75 ms/it)
+  objective:  13461.349282803356
+Minimizing 1652    Time: 0:00:11 ( 6.75 ms/it)
+  objective:  13459.714563331981
+Minimizing 1667    Time: 0:00:11 ( 6.75 ms/it)
+  objective:  13458.610303038193
+Minimizing 1682    Time: 0:00:11 ( 6.75 ms/it)
+  objective:  13458.228239980752
+Minimizing 1697    Time: 0:00:11 ( 6.75 ms/it)
+  objective:  13457.721304545666
+Minimizing 1712    Time: 0:00:11 ( 6.75 ms/it)
+  objective:  13457.037324119745
+Minimizing 1727    Time: 0:00:11 ( 6.75 ms/it)
+  objective:  13456.408258280819
+Minimizing 1742    Time: 0:00:11 ( 6.75 ms/it)
+  objective:  13455.766895393572
+Minimizing 1757    Time: 0:00:11 ( 6.75 ms/it)
+  objective:  13454.371704913545
+Minimizing 1772    Time: 0:00:11 ( 6.75 ms/it)
+  objective:  13450.595171898414
+Minimizing 1787    Time: 0:00:12 ( 6.75 ms/it)
+  objective:  13450.523300232227
+Minimizing 1802    Time: 0:00:12 ( 6.75 ms/it)
+  objective:  13450.339610162999
+Minimizing 1817    Time: 0:00:12 ( 6.75 ms/it)
+  objective:  13450.016868499246
+Minimizing 1832    Time: 0:00:12 ( 6.75 ms/it)
+  objective:  13449.94140149498
+Minimizing 1847    Time: 0:00:12 ( 6.76 ms/it)
+  objective:  13449.816967594066
+Minimizing 1862    Time: 0:00:12 ( 6.76 ms/it)
+  objective:  13449.319770789785
+Minimizing 1877    Time: 0:00:12 ( 6.76 ms/it)
+  objective:  13449.110196556918
+Minimizing 1892    Time: 0:00:12 ( 6.76 ms/it)
+  objective:  13448.795751941441
+Minimizing 1907    Time: 0:00:12 ( 6.76 ms/it)
+  objective:  13449.074992096692
+Minimizing 1922    Time: 0:00:12 ( 6.76 ms/it)
+  objective:  13447.39077325366
+Minimizing 1937    Time: 0:00:13 ( 6.76 ms/it)
+  objective:  13445.09898862688
+Minimizing 1952    Time: 0:00:13 ( 6.76 ms/it)
+  objective:  13444.932383720035
+Minimizing 1967    Time: 0:00:13 ( 6.77 ms/it)
+  objective:  13445.604980674863
+Minimizing 1982    Time: 0:00:13 ( 6.77 ms/it)
+  objective:  13442.275084992973
+Minimizing 1997    Time: 0:00:13 ( 6.77 ms/it)
+  objective:  13442.149408422498
+Minimizing 2011    Time: 0:00:13 ( 6.77 ms/it)
+  objective:  13441.947494954904
+Minimizing 2025    Time: 0:00:13 ( 6.77 ms/it)
+  objective:  13441.802660236346
+Minimizing 2039    Time: 0:00:13 ( 6.78 ms/it)
+  objective:  13441.735454740257
+Minimizing 2053    Time: 0:00:13 ( 6.78 ms/it)
+  objective:  13441.283110526892
+Minimizing 2067    Time: 0:00:14 ( 6.79 ms/it)
+  objective:  13441.215844876555
+Minimizing 2081    Time: 0:00:14 ( 6.79 ms/it)
+  objective:  13440.91066331276
+Minimizing 2095    Time: 0:00:14 ( 6.79 ms/it)
+  objective:  13440.737892112076
+Minimizing 2110    Time: 0:00:14 ( 6.79 ms/it)
+  objective:  13440.314860408005
+Minimizing 2125    Time: 0:00:14 ( 6.79 ms/it)
+  objective:  13440.131066346934
+Minimizing 2140    Time: 0:00:14 ( 6.79 ms/it)
+  objective:  13439.823098394561
+Minimizing 2155    Time: 0:00:14 ( 6.79 ms/it)
+  objective:  13437.977472611477
+Minimizing 2170    Time: 0:00:14 ( 6.79 ms/it)
+  objective:  13436.813123262982
+Minimizing 2185    Time: 0:00:14 ( 6.79 ms/it)
+  objective:  13436.027968008246
+Minimizing 2200    Time: 0:00:14 ( 6.80 ms/it)
+  objective:  13435.469328884566
+Minimizing 2215    Time: 0:00:15 ( 6.80 ms/it)
+  objective:  13435.015305421868
+Minimizing 2230    Time: 0:00:15 ( 6.80 ms/it)
+  objective:  13434.804641040762
+Minimizing 2244    Time: 0:00:15 ( 6.80 ms/it)
+  objective:  13434.639119015439
+Minimizing 2258    Time: 0:00:15 ( 6.80 ms/it)
+  objective:  13434.345101108986
+Minimizing 2272    Time: 0:00:15 ( 6.81 ms/it)
+  objective:  13434.085744287622
+Minimizing 2286    Time: 0:00:15 ( 6.81 ms/it)
+  objective:  13433.73017433774
+Minimizing 2301    Time: 0:00:15 ( 6.81 ms/it)
+  objective:  13433.444091123158
+Minimizing 2316    Time: 0:00:15 ( 6.81 ms/it)
+  objective:  13432.535262732941
+Minimizing 2331    Time: 0:00:15 ( 6.81 ms/it)
+  objective:  13432.028432277402
+Minimizing 2346    Time: 0:00:15 ( 6.81 ms/it)
+  objective:  13431.749397252977
+Minimizing 2361    Time: 0:00:16 ( 6.81 ms/it)
+  objective:  13431.02924160735
+Minimizing 2376    Time: 0:00:16 ( 6.81 ms/it)
+  objective:  13430.340936352091
+Minimizing 2391    Time: 0:00:16 ( 6.81 ms/it)
+  objective:  13430.250705790255
+Minimizing 2406    Time: 0:00:16 ( 6.81 ms/it)
+  objective:  13428.7299317184
+Minimizing 2421    Time: 0:00:16 ( 6.81 ms/it)
+  objective:  13428.600177746244
+Minimizing 2436    Time: 0:00:16 ( 6.81 ms/it)
+  objective:  13428.35456343239
+Minimizing 2450    Time: 0:00:16 ( 6.82 ms/it)
+  objective:  13428.2381590402
+Minimizing 2464    Time: 0:00:16 ( 6.82 ms/it)
+  objective:  13427.619953700036
+Minimizing 2478    Time: 0:00:16 ( 6.82 ms/it)
+  objective:  13427.031311257415
+Minimizing 2493    Time: 0:00:17 ( 6.83 ms/it)
+  objective:  13426.884080365933
+Minimizing 2508    Time: 0:00:17 ( 6.83 ms/it)
+  objective:  13426.658416826038
+Minimizing 2523    Time: 0:00:17 ( 6.83 ms/it)
+  objective:  13426.43264934884
+Minimizing 2538    Time: 0:00:17 ( 6.83 ms/it)
+  objective:  13425.969080593051
+Minimizing 2553    Time: 0:00:17 ( 6.83 ms/it)
+  objective:  13425.901331297908
+Minimizing 2568    Time: 0:00:17 ( 6.83 ms/it)
+  objective:  13425.53287567669
+Minimizing 2583    Time: 0:00:17 ( 6.83 ms/it)
+  objective:  13425.311139249592
+Minimizing 2598    Time: 0:00:17 ( 6.83 ms/it)
+  objective:  13425.09968612664
+Minimizing 2613    Time: 0:00:17 ( 6.83 ms/it)
+  objective:  13424.68932567427
+Minimizing 2628    Time: 0:00:17 ( 6.83 ms/it)
+  objective:  13424.200897974464
+Minimizing 2643    Time: 0:00:18 ( 6.83 ms/it)
+  objective:  13423.5384676123
+Minimizing 2658    Time: 0:00:18 ( 6.83 ms/it)
+  objective:  13421.729598823855
+Minimizing 2673    Time: 0:00:18 ( 6.83 ms/it)
+  objective:  13421.398866678508
+Minimizing 2688    Time: 0:00:18 ( 6.83 ms/it)
+  objective:  13420.907891894625
+Minimizing 2703    Time: 0:00:18 ( 6.83 ms/it)
+  objective:  13420.67928072399
+Minimizing 2718    Time: 0:00:18 ( 6.83 ms/it)
+  objective:  13420.383694176075
+Minimizing 2733    Time: 0:00:18 ( 6.83 ms/it)
+  objective:  13420.288588365496
+Minimizing 2748    Time: 0:00:18 ( 6.83 ms/it)
+  objective:  13420.079692674131
+Minimizing 2763    Time: 0:00:18 ( 6.83 ms/it)
+  objective:  13418.959660698398
+Minimizing 2778    Time: 0:00:18 ( 6.83 ms/it)
+  objective:  13417.069232054215
+Minimizing 2793    Time: 0:00:19 ( 6.83 ms/it)
+  objective:  13416.976691040429
+Minimizing 2808    Time: 0:00:19 ( 6.83 ms/it)
+  objective:  13416.388248267795
+Minimizing 2823    Time: 0:00:19 ( 6.83 ms/it)
+  objective:  13415.979897269957
+Minimizing 2838    Time: 0:00:19 ( 6.83 ms/it)
+  objective:  13415.476616105261
+Minimizing 2855    Time: 0:00:19 ( 6.83 ms/it)
+  objective:  13415.205429758818
+Minimizing 2870    Time: 0:00:19 ( 6.83 ms/it)
+  objective:  13414.602298967635
+Minimizing 2885    Time: 0:00:19 ( 6.83 ms/it)
+  objective:  13414.430555188512
+Minimizing 2900    Time: 0:00:19 ( 6.83 ms/it)
+  objective:  13413.691742222043
+Minimizing 2915    Time: 0:00:19 ( 6.83 ms/it)
+  objective:  13411.808598661191
+Minimizing 2930    Time: 0:00:20 ( 6.83 ms/it)
+  objective:  13411.001015575603
+Minimizing 2945    Time: 0:00:20 ( 6.83 ms/it)
+  objective:  13410.683756899736
+Minimizing 2960    Time: 0:00:20 ( 6.83 ms/it)
+  objective:  13410.392192634768
+Minimizing 2975    Time: 0:00:20 ( 6.83 ms/it)
+  objective:  13410.297924170125
+Minimizing 2990    Time: 0:00:20 ( 6.83 ms/it)
+  objective:  13410.246482225055
+Minimizing 3005    Time: 0:00:20 ( 6.83 ms/it)
+  objective:  13410.182515826004
+Minimizing 3020    Time: 0:00:20 ( 6.83 ms/it)
+  objective:  13410.126147990966
+Minimizing 3035    Time: 0:00:20 ( 6.83 ms/it)
+  objective:  13409.887593711617
+Minimizing 3050    Time: 0:00:20 ( 6.83 ms/it)
+  objective:  13409.737112608782
+Minimizing 3065    Time: 0:00:20 ( 6.83 ms/it)
+  objective:  13409.542533619075
+Minimizing 3080    Time: 0:00:21 ( 6.83 ms/it)
+  objective:  13410.713853535854
+Minimizing 3095    Time: 0:00:21 ( 6.83 ms/it)
+  objective:  13408.999170151452
+Minimizing 3110    Time: 0:00:21 ( 6.83 ms/it)
+  objective:  13408.846049140207
+Minimizing 3125    Time: 0:00:21 ( 6.83 ms/it)
+  objective:  13408.81458514851
+Minimizing 3140    Time: 0:00:21 ( 6.83 ms/it)
+  objective:  13408.628653805099
+Minimizing 3155    Time: 0:00:21 ( 6.83 ms/it)
+  objective:  13408.458549214956
+Minimizing 3170    Time: 0:00:21 ( 6.83 ms/it)
+  objective:  13408.082805525133
+Minimizing 3186    Time: 0:00:21 ( 6.83 ms/it)
+  objective:  13407.901308307417
+Minimizing 3201    Time: 0:00:21 ( 6.83 ms/it)
+  objective:  13407.508925649294
+Minimizing 3216    Time: 0:00:21 ( 6.83 ms/it)
+  objective:  13407.4424865896
+Minimizing 3231    Time: 0:00:22 ( 6.83 ms/it)
+  objective:  13407.325513060896
+Minimizing 3246    Time: 0:00:22 ( 6.83 ms/it)
+  objective:  13407.269127944564
+Minimizing 3262    Time: 0:00:22 ( 6.83 ms/it)
+  objective:  13407.121673193426
+Minimizing 3277    Time: 0:00:22 ( 6.83 ms/it)
+  objective:  13406.938982763691
+Minimizing 3292    Time: 0:00:22 ( 6.83 ms/it)
+  objective:  13406.739485440572
+Minimizing 3307    Time: 0:00:22 ( 6.83 ms/it)
+  objective:  13406.386341910991
+Minimizing 3322    Time: 0:00:22 ( 6.83 ms/it)
+  objective:  13406.33130509931
+Minimizing 3337    Time: 0:00:22 ( 6.83 ms/it)
+  objective:  13406.250773542939
+Minimizing 3352    Time: 0:00:22 ( 6.83 ms/it)
+  objective:  13406.178296293969
+Minimizing 3367    Time: 0:00:22 ( 6.83 ms/it)
+  objective:  13405.990703369345
+Minimizing 3382    Time: 0:00:23 ( 6.83 ms/it)
+  objective:  13405.963545486426
+Minimizing 3397    Time: 0:00:23 ( 6.83 ms/it)
+  objective:  13405.643476446741
+Minimizing 3412    Time: 0:00:23 ( 6.83 ms/it)
+  objective:  13405.063527673585
+Minimizing 3427    Time: 0:00:23 ( 6.83 ms/it)
+  objective:  13405.02946117951
+Minimizing 3442    Time: 0:00:23 ( 6.83 ms/it)
+  objective:  13404.856109886401
+Minimizing 3457    Time: 0:00:23 ( 6.83 ms/it)
+  objective:  13404.64640477307
+Minimizing 3472    Time: 0:00:23 ( 6.83 ms/it)
+  objective:  13404.525980073835
+Minimizing 3487    Time: 0:00:23 ( 6.83 ms/it)
+  objective:  13404.262173605995
+Minimizing 3500    Time: 0:00:23 ( 6.84 ms/it)
+  objective:  13404.127115545998
+Minimizing 3513    Time: 0:00:24 ( 6.84 ms/it)
+  objective:  13403.94978845527
+Minimizing 3527    Time: 0:00:24 ( 6.85 ms/it)
+  objective:  13403.706382205128
+Minimizing 3541    Time: 0:00:24 ( 6.85 ms/it)
+  objective:  13403.639038920504
+Minimizing 3556    Time: 0:00:24 ( 6.85 ms/it)
+  objective:  13403.25089431525
+Minimizing 3571    Time: 0:00:24 ( 6.85 ms/it)
+  objective:  13402.97013170414
+Minimizing 3586    Time: 0:00:24 ( 6.85 ms/it)
+  objective:  13402.734308727733
+Minimizing 3601    Time: 0:00:24 ( 6.85 ms/it)
+  objective:  13402.567589494363
+Minimizing 3615    Time: 0:00:24 ( 6.85 ms/it)
+  objective:  13402.068284457127
+Minimizing 3629    Time: 0:00:24 ( 6.85 ms/it)
+  objective:  13401.89108302733
+Minimizing 3644    Time: 0:00:24 ( 6.86 ms/it)
+  objective:  13401.434992449445
+Minimizing 3659    Time: 0:00:25 ( 6.86 ms/it)
+  objective:  13401.339816215353
+Minimizing 3674    Time: 0:00:25 ( 6.86 ms/it)
+  objective:  13401.011237968727
+Minimizing 3689    Time: 0:00:25 ( 6.86 ms/it)
+  objective:  13400.899273724535
+Minimizing 3704    Time: 0:00:25 ( 6.86 ms/it)
+  objective:  13400.631030999117
+Minimizing 3718    Time: 0:00:25 ( 6.86 ms/it)
+  objective:  13400.615890685847
+Minimizing 3733    Time: 0:00:25 ( 6.86 ms/it)
+  objective:  13400.506604714843
+Minimizing 3748    Time: 0:00:25 ( 6.86 ms/it)
+  objective:  13400.043648063256
+Minimizing 3763    Time: 0:00:25 ( 6.86 ms/it)
+  objective:  13400.009337708929
+Minimizing 3778    Time: 0:00:25 ( 6.86 ms/it)
+  objective:  13399.856856851271
+Minimizing 3793    Time: 0:00:26 ( 6.86 ms/it)
+  objective:  13399.587067141008
+Minimizing 3808    Time: 0:00:26 ( 6.86 ms/it)
+  objective:  13399.033220450634
+Minimizing 3823    Time: 0:00:26 ( 6.86 ms/it)
+  objective:  13399.005206893125
+Minimizing 3838    Time: 0:00:26 ( 6.86 ms/it)
+  objective:  13398.836297056536
+Minimizing 3853    Time: 0:00:26 ( 6.86 ms/it)
+  objective:  13398.758431093047
+Minimizing 3868    Time: 0:00:26 ( 6.86 ms/it)
+  objective:  13398.346197751132
+Minimizing 3883    Time: 0:00:26 ( 6.86 ms/it)
+  objective:  13398.290112862276
+Minimizing 3897    Time: 0:00:26 ( 6.86 ms/it)
+  objective:  13397.998509687663
+Minimizing 3911    Time: 0:00:26 ( 6.88 ms/it)
+  objective:  13397.885113499557
+Minimizing 3922    Time: 0:00:27 ( 6.89 ms/it)
+  objective:  13397.802749459792
+Minimizing 3937    Time: 0:00:27 ( 6.89 ms/it)
+  objective:  13397.628794346805
+Minimizing 3952    Time: 0:00:27 ( 6.89 ms/it)
+  objective:  13397.438395749821
+Minimizing 3967    Time: 0:00:27 ( 6.89 ms/it)
+  objective:  13396.853688099436
+Minimizing 3982    Time: 0:00:27 ( 6.89 ms/it)
+  objective:  13396.742808832169
+Minimizing 3997    Time: 0:00:27 ( 6.89 ms/it)
+  objective:  13396.691638782577
+Minimizing 4012    Time: 0:00:27 ( 6.89 ms/it)
+  objective:  13396.607579192801
+Minimizing 4027    Time: 0:00:27 ( 6.89 ms/it)
+  objective:  13396.434557549088
+Minimizing 4042    Time: 0:00:27 ( 6.89 ms/it)
+  objective:  13396.2596687127
+Minimizing 4057    Time: 0:00:27 ( 6.89 ms/it)
+  objective:  13396.050723842935
+Minimizing 4072    Time: 0:00:28 ( 6.89 ms/it)
+  objective:  13395.983879727624
+Minimizing 4087    Time: 0:00:28 ( 6.89 ms/it)
+  objective:  13395.615054056798
+Minimizing 4102    Time: 0:00:28 ( 6.89 ms/it)
+  objective:  13395.489220327814
+Minimizing 4117    Time: 0:00:28 ( 6.89 ms/it)
+  objective:  13395.208256389757
+Minimizing 4132    Time: 0:00:28 ( 6.89 ms/it)
+  objective:  13394.964858138323
+Minimizing 4147    Time: 0:00:28 ( 6.89 ms/it)
+  objective:  13394.68285716961
+Minimizing 4162    Time: 0:00:28 ( 6.89 ms/it)
+  objective:  13394.539461565291
+Minimizing 4177    Time: 0:00:28 ( 6.89 ms/it)
+  objective:  13394.443118022282
+Minimizing 4192    Time: 0:00:28 ( 6.89 ms/it)
+  objective:  13394.209761507474
+Minimizing 4207    Time: 0:00:28 ( 6.89 ms/it)
+  objective:  13393.935845590357
+Minimizing 4222    Time: 0:00:29 ( 6.88 ms/it)
+  objective:  13393.666365170902
+Minimizing 4237    Time: 0:00:29 ( 6.88 ms/it)
+  objective:  13393.213668260985
+Minimizing 4252    Time: 0:00:29 ( 6.88 ms/it)
+  objective:  13392.56600051163
+Minimizing 4267    Time: 0:00:29 ( 6.88 ms/it)
+  objective:  13392.499011912507
+Minimizing 4283    Time: 0:00:29 ( 6.88 ms/it)
+  objective:  13392.431023451369
+Minimizing 4298    Time: 0:00:29 ( 6.88 ms/it)
+  objective:  13392.377736224109
+Minimizing 4314    Time: 0:00:29 ( 6.88 ms/it)
+  objective:  13392.264877478905
+Minimizing 4329    Time: 0:00:29 ( 6.88 ms/it)
+  objective:  13392.168621039098
+Minimizing 4344    Time: 0:00:29 ( 6.88 ms/it)
+  objective:  13391.880897793235
+Minimizing 4360    Time: 0:00:29 ( 6.88 ms/it)
+  objective:  13391.8287680043
+Minimizing 4375    Time: 0:00:30 ( 6.88 ms/it)
+  objective:  13391.529423234104
+Minimizing 4390    Time: 0:00:30 ( 6.88 ms/it)
+  objective:  13391.242169677382
+Minimizing 4405    Time: 0:00:30 ( 6.88 ms/it)
+  objective:  13391.056664459808
+Minimizing 4420    Time: 0:00:30 ( 6.88 ms/it)
+  objective:  13391.017208363635
+Minimizing 4435    Time: 0:00:30 ( 6.88 ms/it)
+  objective:  13390.991640826513
+Minimizing 4451    Time: 0:00:30 ( 6.88 ms/it)
+  objective:  13390.90667081951
+Minimizing 4467    Time: 0:00:30 ( 6.88 ms/it)
+  objective:  13390.799363491387
+Minimizing 4482    Time: 0:00:30 ( 6.88 ms/it)
+  objective:  13390.760200463643
+Minimizing 4497    Time: 0:00:30 ( 6.88 ms/it)
+  objective:  13390.714811117723
+Minimizing 4512    Time: 0:00:31 ( 6.88 ms/it)
+  objective:  13390.518470292845
+Minimizing 4527    Time: 0:00:31 ( 6.88 ms/it)
+  objective:  13390.329435973516
+Minimizing 4542    Time: 0:00:31 ( 6.88 ms/it)
+  objective:  13390.078339683248
+Minimizing 4557    Time: 0:00:31 ( 6.88 ms/it)
+  objective:  13390.031498748845
+Minimizing 4572    Time: 0:00:31 ( 6.88 ms/it)
+  objective:  13389.949992722948
+Minimizing 4587    Time: 0:00:31 ( 6.87 ms/it)
+  objective:  13389.763644823739
+Minimizing 4602    Time: 0:00:31 ( 6.87 ms/it)
+  objective:  13389.62784133396
+Minimizing 4617    Time: 0:00:31 ( 6.87 ms/it)
+  objective:  13389.460760133683
+Minimizing 4632    Time: 0:00:31 ( 6.87 ms/it)
+  objective:  13389.016476854988
+Minimizing 4647    Time: 0:00:31 ( 6.87 ms/it)
+  objective:  13388.959273912442
+Minimizing 4662    Time: 0:00:32 ( 6.87 ms/it)
+  objective:  13388.891763682856
+Minimizing 4677    Time: 0:00:32 ( 6.87 ms/it)
+  objective:  13388.820401609235
+Minimizing 4692    Time: 0:00:32 ( 6.87 ms/it)
+  objective:  13388.751192318137
+Minimizing 4707    Time: 0:00:32 ( 6.87 ms/it)
+  objective:  13388.739910386357
+Minimizing 4722    Time: 0:00:32 ( 6.87 ms/it)
+  objective:  13388.503706668635
+Minimizing 4737    Time: 0:00:32 ( 6.87 ms/it)
+  objective:  13388.459867990445
+Minimizing 4752    Time: 0:00:32 ( 6.87 ms/it)
+  objective:  13388.279306965618
+Minimizing 4767    Time: 0:00:32 ( 6.87 ms/it)
+  objective:  13388.189210024313
+Minimizing 4782    Time: 0:00:32 ( 6.87 ms/it)
+  objective:  13387.92470124815
+Minimizing 4797    Time: 0:00:32 ( 6.87 ms/it)
+  objective:  13387.89868886891
+Minimizing 4812    Time: 0:00:33 ( 6.87 ms/it)
+  objective:  13387.377972867136
+Minimizing 4827    Time: 0:00:33 ( 6.87 ms/it)
+  objective:  13387.34641313323
+Minimizing 4842    Time: 0:00:33 ( 6.87 ms/it)
+  objective:  13387.2694558531
+Minimizing 4857    Time: 0:00:33 ( 6.87 ms/it)
+  objective:  13387.127526261815
+Minimizing 4872    Time: 0:00:33 ( 6.87 ms/it)
+  objective:  13386.991657510138
+Minimizing 4887    Time: 0:00:33 ( 6.87 ms/it)
+  objective:  13386.90283250594
+Minimizing 4902    Time: 0:00:33 ( 6.87 ms/it)
+  objective:  13386.67754627928
+Minimizing 4917    Time: 0:00:33 ( 6.87 ms/it)
+  objective:  13386.321305879152
+Minimizing 4932    Time: 0:00:33 ( 6.87 ms/it)
+  objective:  13386.26595318252
+Minimizing 4947    Time: 0:00:33 ( 6.87 ms/it)
+  objective:  13386.099825550475
+Minimizing 4962    Time: 0:00:34 ( 6.87 ms/it)
+  objective:  13385.919946994349
+Minimizing 4977    Time: 0:00:34 ( 6.87 ms/it)
+  objective:  13385.149191722252
+Minimizing 4992    Time: 0:00:34 ( 6.87 ms/it)
+  objective:  13384.964423516161
+Minimizing 5007    Time: 0:00:34 ( 6.87 ms/it)
+  objective:  13384.904525178492
+Minimizing 5022    Time: 0:00:34 ( 6.87 ms/it)
+  objective:  13384.757298589007
+Minimizing 5038    Time: 0:00:34 ( 6.87 ms/it)
+  objective:  13384.660289430067
+Minimizing 5054    Time: 0:00:34 ( 6.87 ms/it)
+  objective:  13384.340072467508
+Minimizing 5069    Time: 0:00:34 ( 6.87 ms/it)
+  objective:  13384.120576951216
+Minimizing 5084    Time: 0:00:34 ( 6.87 ms/it)
+  objective:  13384.066943781087
+Minimizing 5099    Time: 0:00:35 ( 6.87 ms/it)
+  objective:  13383.882201841698
+Minimizing 5115    Time: 0:00:35 ( 6.87 ms/it)
+  objective:  13383.86097009945
+Minimizing 5130    Time: 0:00:35 ( 6.87 ms/it)
+  objective:  13383.76770953826
+Minimizing 5145    Time: 0:00:35 ( 6.87 ms/it)
+  objective:  13383.843311364195
+Minimizing 5160    Time: 0:00:35 ( 6.87 ms/it)
+  objective:  13383.547071372173
+Minimizing 5175    Time: 0:00:35 ( 6.87 ms/it)
+  objective:  13383.361992406477
+Minimizing 5190    Time: 0:00:35 ( 6.86 ms/it)
+  objective:  13383.265792056372
+Minimizing 5205    Time: 0:00:35 ( 6.86 ms/it)
+  objective:  13383.237556206703
+Minimizing 5220    Time: 0:00:35 ( 6.86 ms/it)
+  objective:  13383.122292311135
+Minimizing 5235    Time: 0:00:35 ( 6.86 ms/it)
+  objective:  13383.091164666985
+Minimizing 5250    Time: 0:00:36 ( 6.86 ms/it)
+  objective:  13382.82807980945
+Minimizing 5265    Time: 0:00:36 ( 6.86 ms/it)
+  objective:  13382.771836568187
+Minimizing 5280    Time: 0:00:36 ( 6.86 ms/it)
+  objective:  13382.442727021204
+Minimizing 5295    Time: 0:00:36 ( 6.86 ms/it)
+  objective:  13382.337126181345
+Minimizing 5310    Time: 0:00:36 ( 6.86 ms/it)
+  objective:  13382.279183836028
+Minimizing 5325    Time: 0:00:36 ( 6.86 ms/it)
+  objective:  13382.159433179433
+Minimizing 5340    Time: 0:00:36 ( 6.86 ms/it)
+  objective:  13381.88664285669
+Minimizing 5355    Time: 0:00:36 ( 6.86 ms/it)
+  objective:  13381.856052897834
+Minimizing 5370    Time: 0:00:36 ( 6.86 ms/it)
+  objective:  13381.749828866079
+Minimizing 5385    Time: 0:00:36 ( 6.86 ms/it)
+  objective:  13381.710138183313
+Minimizing 5400    Time: 0:00:37 ( 6.86 ms/it)
+  objective:  13381.588205927212
+Minimizing 5414    Time: 0:00:37 ( 6.86 ms/it)
+  objective:  13381.42252826711
+Minimizing 5429    Time: 0:00:37 ( 6.86 ms/it)
+  objective:  13380.861420636233
+Minimizing 5444    Time: 0:00:37 ( 6.87 ms/it)
+  objective:  13380.836075844505
+Minimizing 5459    Time: 0:00:37 ( 6.87 ms/it)
+  objective:  13380.768408174998
+Minimizing 5474    Time: 0:00:37 ( 6.87 ms/it)
+  objective:  13380.680055026627
+Minimizing 5489    Time: 0:00:37 ( 6.87 ms/it)
+  objective:  13380.506751959038
+Minimizing 5504    Time: 0:00:37 ( 6.87 ms/it)
+  objective:  13380.435753297963
+Minimizing 5519    Time: 0:00:37 ( 6.87 ms/it)
+  objective:  13380.185214291741
+Minimizing 5533    Time: 0:00:38 ( 6.87 ms/it)
+  objective:  13380.180568083852
+Minimizing 5548    Time: 0:00:38 ( 6.87 ms/it)
+  objective:  13380.070313659075
+Minimizing 5563    Time: 0:00:38 ( 6.87 ms/it)
+  objective:  13379.94142494828
+Minimizing 5578    Time: 0:00:38 ( 6.87 ms/it)
+  objective:  13379.905747871795
+Minimizing 5593    Time: 0:00:38 ( 6.87 ms/it)
+  objective:  13379.830238168375
+Minimizing 5608    Time: 0:00:38 ( 6.87 ms/it)
+  objective:  13379.456550914256
+Minimizing 5623    Time: 0:00:38 ( 6.87 ms/it)
+  objective:  13379.233924313361
+Minimizing 5638    Time: 0:00:38 ( 6.87 ms/it)
+  objective:  13378.129228646212
+Minimizing 5653    Time: 0:00:38 ( 6.87 ms/it)
+  objective:  13377.79493796228
+Minimizing 5668    Time: 0:00:38 ( 6.87 ms/it)
+  objective:  13377.67468089141
+Minimizing 5683    Time: 0:00:39 ( 6.87 ms/it)
+  objective:  13377.654639531625
+Minimizing 5698    Time: 0:00:39 ( 6.87 ms/it)
+  objective:  13377.628983467628
+Minimizing 5713    Time: 0:00:39 ( 6.87 ms/it)
+  objective:  13377.620588554608
+Minimizing 5728    Time: 0:00:39 ( 6.87 ms/it)
+  objective:  13377.553407903593
+Minimizing 5743    Time: 0:00:39 ( 6.87 ms/it)
+  objective:  13377.493974512232
+Minimizing 5758    Time: 0:00:39 ( 6.87 ms/it)
+  objective:  13377.463183533386
+Minimizing 5773    Time: 0:00:39 ( 6.87 ms/it)
+  objective:  13377.426853135068
+Minimizing 5788    Time: 0:00:39 ( 6.87 ms/it)
+  objective:  13377.415010865836
+Minimizing 5803    Time: 0:00:39 ( 6.87 ms/it)
+  objective:  13377.365693036067
+Minimizing 5818    Time: 0:00:39 ( 6.87 ms/it)
+  objective:  13377.35246267174
+Minimizing 5833    Time: 0:00:40 ( 6.87 ms/it)
+  objective:  13377.348444642368
+Minimizing 5848    Time: 0:00:40 ( 6.87 ms/it)
+  objective:  13377.342267280903
+Minimizing 5864    Time: 0:00:40 ( 6.87 ms/it)
+  objective:  13377.298746704757
+Minimizing 5879    Time: 0:00:40 ( 6.87 ms/it)
+  objective:  13377.28087785084
+Minimizing 5894    Time: 0:00:40 ( 6.87 ms/it)
+  objective:  13377.2489520995
+Minimizing 5909    Time: 0:00:40 ( 6.87 ms/it)
+  objective:  13377.228498033066
+Minimizing 5924    Time: 0:00:40 ( 6.87 ms/it)
+  objective:  13377.183408165278
+Minimizing 5939    Time: 0:00:40 ( 6.87 ms/it)
+  objective:  13377.137680763175
+Minimizing 5954    Time: 0:00:40 ( 6.87 ms/it)
+  objective:  13377.08602648355
+Minimizing 5969    Time: 0:00:41 ( 6.87 ms/it)
+  objective:  13377.067422838132
+Minimizing 5984    Time: 0:00:41 ( 6.87 ms/it)
+  objective:  13376.928430755637
+Minimizing 5999    Time: 0:00:41 ( 6.87 ms/it)
+  objective:  13376.79084285604
+Minimizing 6014    Time: 0:00:41 ( 6.87 ms/it)
+  objective:  13376.744470016034
+Minimizing 6029    Time: 0:00:41 ( 6.87 ms/it)
+  objective:  13376.660883232995
+Minimizing 6042    Time: 0:00:41 ( 6.87 ms/it)
+  objective:  13376.652006979835
+Minimizing 6056    Time: 0:00:41 ( 6.87 ms/it)
+  objective:  13376.579470635406
+Minimizing 6071    Time: 0:00:41 ( 6.87 ms/it)
+  objective:  13376.531513897833
+Minimizing 6086    Time: 0:00:41 ( 6.87 ms/it)
+  objective:  13376.455603861272
+Minimizing 6100    Time: 0:00:41 ( 6.88 ms/it)
+  objective:  13376.441846797563
+Minimizing 6115    Time: 0:00:42 ( 6.88 ms/it)
+  objective:  13376.345929743504
+Minimizing 6130    Time: 0:00:42 ( 6.88 ms/it)
+  objective:  13376.331609797831
+Minimizing 6145    Time: 0:00:42 ( 6.88 ms/it)
+  objective:  13376.13132879656
+Minimizing 6160    Time: 0:00:42 ( 6.88 ms/it)
+  objective:  13376.12029260354
+Minimizing 6175    Time: 0:00:42 ( 6.88 ms/it)
+  objective:  13375.990249421564
+Minimizing 6191    Time: 0:00:42 ( 6.87 ms/it)
+  objective:  13375.953330678916
+Minimizing 6207    Time: 0:00:42 ( 6.87 ms/it)
+  objective:  13375.688983930486
+Minimizing 6223    Time: 0:00:42 ( 6.87 ms/it)
+  objective:  13375.642318482853
+Minimizing 6238    Time: 0:00:42 ( 6.87 ms/it)
+  objective:  13375.62789762528
+Minimizing 6253    Time: 0:00:42 ( 6.87 ms/it)
+  objective:  13375.575289033928
+Minimizing 6268    Time: 0:00:43 ( 6.87 ms/it)
+  objective:  13375.542532701766
+Minimizing 6283    Time: 0:00:43 ( 6.87 ms/it)
+  objective:  13375.474428153415
+Minimizing 6298    Time: 0:00:43 ( 6.87 ms/it)
+  objective:  13375.448782051913
+Minimizing 6313    Time: 0:00:43 ( 6.87 ms/it)
+  objective:  13375.42897508297
+Minimizing 6328    Time: 0:00:43 ( 6.87 ms/it)
+  objective:  13375.383805562073
+Minimizing 6343    Time: 0:00:43 ( 6.87 ms/it)
+  objective:  13375.351178518737
+Minimizing 6358    Time: 0:00:43 ( 6.87 ms/it)
+  objective:  13375.246187327422
+Minimizing 6373    Time: 0:00:43 ( 6.87 ms/it)
+  objective:  13375.234876103808
+Minimizing 6388    Time: 0:00:43 ( 6.87 ms/it)
+  objective:  13375.202533653195
+Minimizing 6403    Time: 0:00:44 ( 6.87 ms/it)
+  objective:  13375.142522374634
+Minimizing 6418    Time: 0:00:44 ( 6.87 ms/it)
+  objective:  13375.090950316771
+Minimizing 6433    Time: 0:00:44 ( 6.87 ms/it)
+  objective:  13375.000291092656
+Minimizing 6448    Time: 0:00:44 ( 6.87 ms/it)
+  objective:  13374.897247572851
+Minimizing 6463    Time: 0:00:44 ( 6.87 ms/it)
+  objective:  13374.66669194882
+Minimizing 6478    Time: 0:00:44 ( 6.87 ms/it)
+  objective:  13374.604318647675
+Minimizing 6493    Time: 0:00:44 ( 6.87 ms/it)
+  objective:  13374.532537273699
+Minimizing 6508    Time: 0:00:44 ( 6.87 ms/it)
+  objective:  13374.450593598376
+Minimizing 6523    Time: 0:00:44 ( 6.87 ms/it)
+  objective:  13374.410235455143
+Minimizing 6538    Time: 0:00:44 ( 6.87 ms/it)
+  objective:  13374.335032133211
+Minimizing 6553    Time: 0:00:45 ( 6.87 ms/it)
+  objective:  13374.311289281868
+Minimizing 6568    Time: 0:00:45 ( 6.87 ms/it)
+  objective:  13374.180349728704
+Minimizing 6583    Time: 0:00:45 ( 6.87 ms/it)
+  objective:  13373.9969534653
+Minimizing 6598    Time: 0:00:45 ( 6.87 ms/it)
+  objective:  13373.870840727373
+Minimizing 6613    Time: 0:00:45 ( 6.87 ms/it)
+  objective:  13373.816123240897
+Minimizing 6628    Time: 0:00:45 ( 6.87 ms/it)
+  objective:  13373.754243562558
+Minimizing 6643    Time: 0:00:45 ( 6.87 ms/it)
+  objective:  13373.713547662323
+Minimizing 6656    Time: 0:00:45 ( 6.88 ms/it)
+  objective:  13373.691049970112
+Minimizing 6671    Time: 0:00:45 ( 6.88 ms/it)
+  objective:  13373.606736935537
+Minimizing 6686    Time: 0:00:45 ( 6.88 ms/it)
+  objective:  13373.544961816653
+Minimizing 6701    Time: 0:00:46 ( 6.88 ms/it)
+  objective:  13373.454028481472
+Minimizing 6716    Time: 0:00:46 ( 6.88 ms/it)
+  objective:  13373.407356424868
+Minimizing 6731    Time: 0:00:46 ( 6.88 ms/it)
+  objective:  13373.266439249994
+Minimizing 6746    Time: 0:00:46 ( 6.88 ms/it)
+  objective:  13373.08355878386
+Minimizing 6761    Time: 0:00:46 ( 6.88 ms/it)
+  objective:  13372.700572223679
+Minimizing 6776    Time: 0:00:46 ( 6.88 ms/it)
+  objective:  13371.965579579111
+Minimizing 6791    Time: 0:00:46 ( 6.88 ms/it)
+  objective:  13371.926754086373
+Minimizing 6806    Time: 0:00:46 ( 6.88 ms/it)
+  objective:  13371.890815737628
+Minimizing 6821    Time: 0:00:46 ( 6.88 ms/it)
+  objective:  13371.825391745828
+Minimizing 6836    Time: 0:00:47 ( 6.88 ms/it)
+  objective:  13371.783398776293
+Minimizing 6851    Time: 0:00:47 ( 6.88 ms/it)
+  objective:  13371.692600466078
+Minimizing 6866    Time: 0:00:47 ( 6.88 ms/it)
+  objective:  13371.562734198014
+Minimizing 6881    Time: 0:00:47 ( 6.88 ms/it)
+  objective:  13371.441694604473
+Minimizing 6896    Time: 0:00:47 ( 6.88 ms/it)
+  objective:  13371.37050822581
+Minimizing 6911    Time: 0:00:47 ( 6.88 ms/it)
+  objective:  13371.317465892302
+Minimizing 6927    Time: 0:00:47 ( 6.88 ms/it)
+  objective:  13370.770915597685
+Minimizing 6943    Time: 0:00:47 ( 6.87 ms/it)
+  objective:  13370.61077787495
+Minimizing 6958    Time: 0:00:47 ( 6.87 ms/it)
+  objective:  13370.590431240962
+Minimizing 6973    Time: 0:00:47 ( 6.87 ms/it)
+  objective:  13370.578297688822
+Minimizing 6988    Time: 0:00:48 ( 6.87 ms/it)
+  objective:  13370.557051298892
+Minimizing 7003    Time: 0:00:48 ( 6.87 ms/it)
+  objective:  13370.491426723544
+Minimizing 7018    Time: 0:00:48 ( 6.87 ms/it)
+  objective:  13370.45833799184
+Minimizing 7033    Time: 0:00:48 ( 6.87 ms/it)
+  objective:  13370.303519399778
+Minimizing 7048    Time: 0:00:48 ( 6.88 ms/it)
+  objective:  13370.130961288618
+Minimizing 7063    Time: 0:00:48 ( 6.87 ms/it)
+  objective:  13370.114242299387
+Minimizing 7078    Time: 0:00:48 ( 6.87 ms/it)
+  objective:  13370.104257503328
+Minimizing 7093    Time: 0:00:48 ( 6.87 ms/it)
+  objective:  13370.079336794166
+Minimizing 7108    Time: 0:00:48 ( 6.87 ms/it)
+  objective:  13370.054328481412
+Minimizing 7123    Time: 0:00:48 ( 6.87 ms/it)
+  objective:  13370.016732800585
+Minimizing 7138    Time: 0:00:49 ( 6.87 ms/it)
+  objective:  13369.981639876583
+Minimizing 7153    Time: 0:00:49 ( 6.87 ms/it)
+  objective:  13369.964754558721
+Minimizing 7168    Time: 0:00:49 ( 6.87 ms/it)
+  objective:  13369.912558451513
+Minimizing 7183    Time: 0:00:49 ( 6.87 ms/it)
+  objective:  13369.879356093967
+Minimizing 7198    Time: 0:00:49 ( 6.87 ms/it)
+  objective:  13369.755105152108
+Minimizing 7213    Time: 0:00:49 ( 6.87 ms/it)
+  objective:  13369.750238435037
+Minimizing 7228    Time: 0:00:49 ( 6.87 ms/it)
+  objective:  13369.711429808987
+Minimizing 7243    Time: 0:00:49 ( 6.87 ms/it)
+  objective:  13369.671289654492
+Minimizing 7258    Time: 0:00:49 ( 6.87 ms/it)
+  objective:  13369.589318805323
+Minimizing 7273    Time: 0:00:49 ( 6.87 ms/it)
+  objective:  13369.569123039473
+Minimizing 7288    Time: 0:00:50 ( 6.87 ms/it)
+  objective:  13369.546600320944
+Minimizing 7303    Time: 0:00:50 ( 6.87 ms/it)
+  objective:  13369.506954672463
+Minimizing 7318    Time: 0:00:50 ( 6.87 ms/it)
+  objective:  13369.490255486591
+Minimizing 7333    Time: 0:00:50 ( 6.87 ms/it)
+  objective:  13369.475027861248
+Minimizing 7348    Time: 0:00:50 ( 6.87 ms/it)
+  objective:  13369.41372521386
+Minimizing 7363    Time: 0:00:50 ( 6.87 ms/it)
+  objective:  13369.384289339381
+Minimizing 7378    Time: 0:00:50 ( 6.87 ms/it)
+  objective:  13369.31353933843
+Minimizing 7393    Time: 0:00:50 ( 6.87 ms/it)
+  objective:  13369.240696109307
+Minimizing 7408    Time: 0:00:50 ( 6.87 ms/it)
+  objective:  13369.214018439532
+Minimizing 7423    Time: 0:00:51 ( 6.87 ms/it)
+  objective:  13369.021392362374
+Minimizing 7437    Time: 0:00:51 ( 6.87 ms/it)
+  objective:  13369.016678510023
+Minimizing 7451    Time: 0:00:51 ( 6.88 ms/it)
+  objective:  13368.96123365561
+Minimizing 7465    Time: 0:00:51 ( 6.88 ms/it)
+  objective:  13368.904309428835
+Minimizing 7479    Time: 0:00:51 ( 6.88 ms/it)
+  objective:  13368.860184313882
+Minimizing 7493    Time: 0:00:51 ( 6.88 ms/it)
+  objective:  13368.8439067129
+Minimizing 7507    Time: 0:00:51 ( 6.88 ms/it)
+  objective:  13368.717814799435
+Minimizing 7522    Time: 0:00:51 ( 6.88 ms/it)
+  objective:  13368.343124411134
+Minimizing 7537    Time: 0:00:51 ( 6.88 ms/it)
+  objective:  13368.311637371691
+Minimizing 7552    Time: 0:00:51 ( 6.88 ms/it)
+  objective:  13368.280821540306
+Minimizing 7567    Time: 0:00:52 ( 6.88 ms/it)
+  objective:  13368.138206312658
+Minimizing 7582    Time: 0:00:52 ( 6.88 ms/it)
+  objective:  13368.042496879469
+Minimizing 7597    Time: 0:00:52 ( 6.88 ms/it)
+  objective:  13367.98447241549
+Minimizing 7612    Time: 0:00:52 ( 6.88 ms/it)
+  objective:  13367.969536239238
+Minimizing 7627    Time: 0:00:52 ( 6.88 ms/it)
+  objective:  13367.92077512548
+Minimizing 7642    Time: 0:00:52 ( 6.88 ms/it)
+  objective:  13367.882886299041
+Minimizing 7657    Time: 0:00:52 ( 6.88 ms/it)
+  objective:  13367.750793392719
+Minimizing 7672    Time: 0:00:52 ( 6.88 ms/it)
+  objective:  13367.627396631491
+Minimizing 7687    Time: 0:00:52 ( 6.88 ms/it)
+  objective:  13367.451115931457
+Minimizing 7702    Time: 0:00:52 ( 6.88 ms/it)
+  objective:  13367.317819530603
+Minimizing 7717    Time: 0:00:53 ( 6.88 ms/it)
+  objective:  13367.210093133785
+Minimizing 7732    Time: 0:00:53 ( 6.88 ms/it)
+  objective:  13367.199462880133
+Minimizing 7747    Time: 0:00:53 ( 6.88 ms/it)
+  objective:  13367.186724468564
+Minimizing 7762    Time: 0:00:53 ( 6.88 ms/it)
+  objective:  13367.145439082844
+Minimizing 7777    Time: 0:00:53 ( 6.88 ms/it)
+  objective:  13367.069368550663
+Minimizing 7792    Time: 0:00:53 ( 6.88 ms/it)
+  objective:  13367.058887317871
+Minimizing 7807    Time: 0:00:53 ( 6.88 ms/it)
+  objective:  13367.049009146838
+Minimizing 7822    Time: 0:00:53 ( 6.88 ms/it)
+  objective:  13367.012231147062
+Minimizing 7837    Time: 0:00:53 ( 6.88 ms/it)
+  objective:  13366.973243621316
+Minimizing 7852    Time: 0:00:53 ( 6.88 ms/it)
+  objective:  13366.878363601681
+Minimizing 7867    Time: 0:00:54 ( 6.88 ms/it)
+  objective:  13366.874336539528
+Minimizing 7882    Time: 0:00:54 ( 6.88 ms/it)
+  objective:  13366.835411262255
+Minimizing 7897    Time: 0:00:54 ( 6.88 ms/it)
+  objective:  13366.810178074462
+Minimizing 7912    Time: 0:00:54 ( 6.88 ms/it)
+  objective:  13366.789897525887
+Minimizing 7927    Time: 0:00:54 ( 6.88 ms/it)
+  objective:  13366.74776966164
+Minimizing 7942    Time: 0:00:54 ( 6.88 ms/it)
+  objective:  13366.709351786172
+Minimizing 7957    Time: 0:00:54 ( 6.88 ms/it)
+  objective:  13366.694821793048
+Minimizing 7972    Time: 0:00:54 ( 6.88 ms/it)
+  objective:  13366.57840019682
+Minimizing 7987    Time: 0:00:54 ( 6.88 ms/it)
+  objective:  13366.470667255584
+Minimizing 8002    Time: 0:00:55 ( 6.88 ms/it)
+  objective:  13366.437363007673
+Minimizing 8017    Time: 0:00:55 ( 6.88 ms/it)
+  objective:  13366.408919171648
+Minimizing 8032    Time: 0:00:55 ( 6.88 ms/it)
+  objective:  13366.377921794956
+Minimizing 8047    Time: 0:00:55 ( 6.88 ms/it)
+  objective:  13366.092883387093
+Minimizing 8062    Time: 0:00:55 ( 6.88 ms/it)
+  objective:  13366.073007818377
+Minimizing 8077    Time: 0:00:55 ( 6.88 ms/it)
+  objective:  13365.751529599023
+Minimizing 8092    Time: 0:00:55 ( 6.88 ms/it)
+  objective:  13365.746638929042
+Minimizing 8108    Time: 0:00:55 ( 6.87 ms/it)
+  objective:  13365.709298701142
+Minimizing 8123    Time: 0:00:55 ( 6.87 ms/it)
+  objective:  13365.667712559138
+Minimizing 8138    Time: 0:00:55 ( 6.87 ms/it)
+  objective:  13365.549607618836
+Minimizing 8153    Time: 0:00:56 ( 6.87 ms/it)
+  objective:  13365.538019369007
+Minimizing 8168    Time: 0:00:56 ( 6.87 ms/it)
+  objective:  13365.482405331248
+Minimizing 8183    Time: 0:00:56 ( 6.87 ms/it)
+  objective:  13365.329488810814
+Minimizing 8198    Time: 0:00:56 ( 6.87 ms/it)
+  objective:  13365.304353701904
+Minimizing 8214    Time: 0:00:56 ( 6.87 ms/it)
+  objective:  13365.251173428536
+Minimizing 8230    Time: 0:00:56 ( 6.87 ms/it)
+  objective:  13365.20377052974
+Minimizing 8245    Time: 0:00:56 ( 6.87 ms/it)
+  objective:  13365.147682533097
+Minimizing 8260    Time: 0:00:56 ( 6.87 ms/it)
+  objective:  13364.990440609196
+Minimizing 8275    Time: 0:00:56 ( 6.87 ms/it)
+  objective:  13364.914183577748
+Minimizing 8290    Time: 0:00:56 ( 6.87 ms/it)
+  objective:  13364.85599994762
+Minimizing 8305    Time: 0:00:57 ( 6.87 ms/it)
+  objective:  13364.803023507819
+Minimizing 8320    Time: 0:00:57 ( 6.87 ms/it)
+  objective:  13364.792236169873
+Minimizing 8335    Time: 0:00:57 ( 6.87 ms/it)
+  objective:  13364.744402544049
+Minimizing 8350    Time: 0:00:57 ( 6.87 ms/it)
+  objective:  13364.65439851064
+Minimizing 8365    Time: 0:00:57 ( 6.87 ms/it)
+  objective:  13364.492123087126
+Minimizing 8380    Time: 0:00:57 ( 6.87 ms/it)
+  objective:  13364.453285737145
+Minimizing 8395    Time: 0:00:57 ( 6.87 ms/it)
+  objective:  13364.443538133251
+Minimizing 8410    Time: 0:00:57 ( 6.87 ms/it)
+  objective:  13364.395906754144
+Minimizing 8425    Time: 0:00:57 ( 6.87 ms/it)
+  objective:  13364.340842878904
+Minimizing 8440    Time: 0:00:58 ( 6.87 ms/it)
+  objective:  13364.31954021076
+Minimizing 8454    Time: 0:00:58 ( 6.87 ms/it)
+  objective:  13364.15082419027
+Minimizing 8469    Time: 0:00:58 ( 6.87 ms/it)
+  objective:  13363.946133674239
+Minimizing 8484    Time: 0:00:58 ( 6.87 ms/it)
+  objective:  13363.877118735014
+Minimizing 8499    Time: 0:00:58 ( 6.87 ms/it)
+  objective:  13363.84631438554
+Minimizing 8514    Time: 0:00:58 ( 6.87 ms/it)
+  objective:  13363.798975420104
+Minimizing 8529    Time: 0:00:58 ( 6.87 ms/it)
+  objective:  13363.780800568034
+Minimizing 8544    Time: 0:00:58 ( 6.87 ms/it)
+  objective:  13363.698922377065
+Minimizing 8559    Time: 0:00:58 ( 6.87 ms/it)
+  objective:  13363.570915435797
+Minimizing 8574    Time: 0:00:58 ( 6.87 ms/it)
+  objective:  13363.552328472368
+Minimizing 8589    Time: 0:00:59 ( 6.87 ms/it)
+  objective:  13363.501190475516
+Minimizing 8604    Time: 0:00:59 ( 6.87 ms/it)
+  objective:  13363.336162788073
+Minimizing 8619    Time: 0:00:59 ( 6.87 ms/it)
+  objective:  13362.974966944355
+Minimizing 8634    Time: 0:00:59 ( 6.87 ms/it)
+  objective:  13362.968368875212
+Minimizing 8649    Time: 0:00:59 ( 6.87 ms/it)
+  objective:  13362.866489196058
+Minimizing 8664    Time: 0:00:59 ( 6.87 ms/it)
+  objective:  13362.838600096045
+Minimizing 8679    Time: 0:00:59 ( 6.87 ms/it)
+  objective:  13362.808891039924
+Minimizing 8694    Time: 0:00:59 ( 6.87 ms/it)
+  objective:  13362.527900107394
+Minimizing 8709    Time: 0:00:59 ( 6.87 ms/it)
+  objective:  13362.508007631404
+Minimizing 8724    Time: 0:00:59 ( 6.87 ms/it)
+  objective:  13362.48485447964
+Minimizing 8740    Time: 0:01:00 ( 6.87 ms/it)
+  objective:  13362.410317078422
+Minimizing 8755    Time: 0:01:00 ( 6.87 ms/it)
+  objective:  13362.333785017021
+Minimizing 8770    Time: 0:01:00 ( 6.87 ms/it)
+  objective:  13362.27268205415
+Minimizing 8785    Time: 0:01:00 ( 6.87 ms/it)
+  objective:  13362.048657450992
+Minimizing 8800    Time: 0:01:00 ( 6.87 ms/it)
+  objective:  13361.86490765214
+Minimizing 8815    Time: 0:01:00 ( 6.87 ms/it)
+  objective:  13361.845050337884
+Minimizing 8830    Time: 0:01:00 ( 6.87 ms/it)
+  objective:  13361.821267377585
+Minimizing 8845    Time: 0:01:00 ( 6.87 ms/it)
+  objective:  13361.770521018887
+Minimizing 8860    Time: 0:01:00 ( 6.87 ms/it)
+  objective:  13361.694090222045
+Minimizing 8875    Time: 0:01:00 ( 6.87 ms/it)
+  objective:  13361.640573441233
+Minimizing 8890    Time: 0:01:01 ( 6.87 ms/it)
+  objective:  13361.50920443482
+Minimizing 8905    Time: 0:01:01 ( 6.87 ms/it)
+  objective:  13361.43440077248
+Minimizing 8920    Time: 0:01:01 ( 6.87 ms/it)
+  objective:  13361.119453477986
+Minimizing 8935    Time: 0:01:01 ( 6.87 ms/it)
+  objective:  13361.095143978411
+Minimizing 8950    Time: 0:01:01 ( 6.87 ms/it)
+  objective:  13361.059780435193
+Minimizing 8965    Time: 0:01:01 ( 6.87 ms/it)
+  objective:  13361.026289798669
+Minimizing 8980    Time: 0:01:01 ( 6.87 ms/it)
+  objective:  13360.975520260952
+Minimizing 8995    Time: 0:01:01 ( 6.87 ms/it)
+  objective:  13360.867847210837
+Minimizing 9010    Time: 0:01:01 ( 6.87 ms/it)
+  objective:  13360.859843643819
+Minimizing 9025    Time: 0:01:01 ( 6.87 ms/it)
+  objective:  13360.79965364631
+Minimizing 9040    Time: 0:01:02 ( 6.87 ms/it)
+  objective:  13360.78628051134
+Minimizing 9055    Time: 0:01:02 ( 6.87 ms/it)
+  objective:  13360.75043879226
+Minimizing 9070    Time: 0:01:02 ( 6.87 ms/it)
+  objective:  13360.64209006112
+Minimizing 9085    Time: 0:01:02 ( 6.87 ms/it)
+  objective:  13360.62469935129
+Minimizing 9100    Time: 0:01:02 ( 6.87 ms/it)
+  objective:  13360.548142264728
+Minimizing 9115    Time: 0:01:02 ( 6.87 ms/it)
+  objective:  13360.518455400656
+Minimizing 9130    Time: 0:01:02 ( 6.87 ms/it)
+  objective:  13360.302842979363
+Minimizing 9145    Time: 0:01:02 ( 6.87 ms/it)
+  objective:  13360.242952398636
+Minimizing 9160    Time: 0:01:02 ( 6.87 ms/it)
+  objective:  13360.169740957208
+Minimizing 9175    Time: 0:01:03 ( 6.87 ms/it)
+  objective:  13359.947214352076
+Minimizing 9190    Time: 0:01:03 ( 6.87 ms/it)
+  objective:  13359.868346662057
+Minimizing 9205    Time: 0:01:03 ( 6.87 ms/it)
+  objective:  13359.807298778796
+Minimizing 9220    Time: 0:01:03 ( 6.87 ms/it)
+  objective:  13359.733491990533
+Minimizing 9235    Time: 0:01:03 ( 6.87 ms/it)
+  objective:  13359.646021967448
+Minimizing 9250    Time: 0:01:03 ( 6.87 ms/it)
+  objective:  13359.422486817035
+Minimizing 9265    Time: 0:01:03 ( 6.87 ms/it)
+  objective:  13359.335114335146
+Minimizing 9280    Time: 0:01:03 ( 6.87 ms/it)
+  objective:  13359.21155468587
+Minimizing 9295    Time: 0:01:03 ( 6.87 ms/it)
+  objective:  13359.166843952218
+Minimizing 9310    Time: 0:01:03 ( 6.87 ms/it)
+  objective:  13358.747913751838
+Minimizing 9325    Time: 0:01:04 ( 6.87 ms/it)
+  objective:  13358.696437464743
+Minimizing 9340    Time: 0:01:04 ( 6.87 ms/it)
+  objective:  13358.65018815524
+Minimizing 9355    Time: 0:01:04 ( 6.87 ms/it)
+  objective:  13358.641094259729
+Minimizing 9370    Time: 0:01:04 ( 6.87 ms/it)
+  objective:  13358.559805156925
+Minimizing 9385    Time: 0:01:04 ( 6.87 ms/it)
+  objective:  13358.547125297358
+Minimizing 9400    Time: 0:01:04 ( 6.87 ms/it)
+  objective:  13358.512463110703
+Minimizing 9415    Time: 0:01:04 ( 6.87 ms/it)
+  objective:  13358.48364217821
+Minimizing 9430    Time: 0:01:04 ( 6.87 ms/it)
+  objective:  13358.428020901782
+Minimizing 9445    Time: 0:01:04 ( 6.87 ms/it)
+  objective:  13358.200210589042
+Minimizing 9460    Time: 0:01:04 ( 6.87 ms/it)
+  objective:  13358.1787328727
+Minimizing 9475    Time: 0:01:05 ( 6.87 ms/it)
+  objective:  13358.16922394975
+Minimizing 9490    Time: 0:01:05 ( 6.87 ms/it)
+  objective:  13358.162856104063
+Minimizing 9505    Time: 0:01:05 ( 6.87 ms/it)
+  objective:  13358.146613102224
+Minimizing 9520    Time: 0:01:05 ( 6.87 ms/it)
+  objective:  13358.132933572335
+Minimizing 9535    Time: 0:01:05 ( 6.87 ms/it)
+  objective:  13358.02146463866
+Minimizing 9550    Time: 0:01:05 ( 6.87 ms/it)
+  objective:  13358.008704936423
+Minimizing 9565    Time: 0:01:05 ( 6.87 ms/it)
+  objective:  13357.996215790721
+Minimizing 9580    Time: 0:01:05 ( 6.87 ms/it)
+  objective:  13357.965784736298
+Minimizing 9595    Time: 0:01:05 ( 6.87 ms/it)
+  objective:  13357.927778730067
+Minimizing 9610    Time: 0:01:06 ( 6.87 ms/it)
+  objective:  13357.891812179056
+Minimizing 9625    Time: 0:01:06 ( 6.87 ms/it)
+  objective:  13357.850452371327
+Minimizing 9640    Time: 0:01:06 ( 6.87 ms/it)
+  objective:  13357.749120784145
+Minimizing 9655    Time: 0:01:06 ( 6.87 ms/it)
+  objective:  13357.73899789466
+Minimizing 9669    Time: 0:01:06 ( 6.87 ms/it)
+  objective:  13357.67472151856
+Minimizing 9684    Time: 0:01:06 ( 6.87 ms/it)
+  objective:  13357.639983173358
+Minimizing 9699    Time: 0:01:06 ( 6.87 ms/it)
+  objective:  13357.5706609611
+Minimizing 9714    Time: 0:01:06 ( 6.87 ms/it)
+  objective:  13357.545856598612
+Minimizing 9729    Time: 0:01:06 ( 6.87 ms/it)
+  objective:  13357.470692066163
+Minimizing 9744    Time: 0:01:06 ( 6.87 ms/it)
+  objective:  13357.391749440772
+Minimizing 9759    Time: 0:01:07 ( 6.87 ms/it)
+  objective:  13357.379048838782
+Minimizing 9774    Time: 0:01:07 ( 6.87 ms/it)
+  objective:  13357.29391176656
+Minimizing 9789    Time: 0:01:07 ( 6.87 ms/it)
+  objective:  13357.273432366761
+Minimizing 9804    Time: 0:01:07 ( 6.87 ms/it)
+  objective:  13357.108316384503
+Minimizing 9819    Time: 0:01:07 ( 6.87 ms/it)
+  objective:  13356.833747665492
+Minimizing 9834    Time: 0:01:07 ( 6.87 ms/it)
+  objective:  13356.825744483125
+Minimizing 9849    Time: 0:01:07 ( 6.87 ms/it)
+  objective:  13356.796599988353
+Minimizing 9864    Time: 0:01:07 ( 6.87 ms/it)
+  objective:  13356.781355628991
+Minimizing 9879    Time: 0:01:07 ( 6.87 ms/it)
+  objective:  13356.765395320064
+Minimizing 9894    Time: 0:01:07 ( 6.87 ms/it)
+  objective:  13356.761655913862
+Minimizing 9909    Time: 0:01:08 ( 6.87 ms/it)
+  objective:  13356.689726603901
+Minimizing 9924    Time: 0:01:08 ( 6.87 ms/it)
+  objective:  13356.647753417856
+Minimizing 9939    Time: 0:01:08 ( 6.87 ms/it)
+  objective:  13356.510383463341
+Minimizing 9954    Time: 0:01:08 ( 6.87 ms/it)
+  objective:  13356.501531919435
+Minimizing 9969    Time: 0:01:08 ( 6.87 ms/it)
+  objective:  13356.469089095459
+Minimizing 9984    Time: 0:01:08 ( 6.87 ms/it)
+  objective:  13356.450332585562
+Minimizing 9999    Time: 0:01:08 ( 6.87 ms/it)
+  objective:  13356.403033817878
+Minimizing 10014    Time: 0:01:08 ( 6.87 ms/it)
+  objective:  13356.38004150764
+Minimizing 10029    Time: 0:01:08 ( 6.87 ms/it)
+  objective:  13356.33580595671
+Minimizing 10044    Time: 0:01:09 ( 6.87 ms/it)
+  objective:  13356.321089908
+Minimizing 10059    Time: 0:01:09 ( 6.87 ms/it)
+  objective:  13356.28983799689
+Minimizing 10074    Time: 0:01:09 ( 6.87 ms/it)
+  objective:  13356.253746486553
+Minimizing 10089    Time: 0:01:09 ( 6.87 ms/it)
+  objective:  13356.233201505558
+Minimizing 10104    Time: 0:01:09 ( 6.87 ms/it)
+  objective:  13356.071788036905
+Minimizing 10119    Time: 0:01:09 ( 6.87 ms/it)
+  objective:  13356.058297816235
+Minimizing 10134    Time: 0:01:09 ( 6.87 ms/it)
+  objective:  13355.967407573859
+Minimizing 10149    Time: 0:01:09 ( 6.87 ms/it)
+  objective:  13355.832206948042
+Minimizing 10164    Time: 0:01:09 ( 6.87 ms/it)
+  objective:  13355.824421166646
+Minimizing 10179    Time: 0:01:09 ( 6.87 ms/it)
+  objective:  13355.799631444053
+Minimizing 10194    Time: 0:01:10 ( 6.87 ms/it)
+  objective:  13355.777966823232
+Minimizing 10209    Time: 0:01:10 ( 6.87 ms/it)
+  objective:  13355.732665592957
+Minimizing 10224    Time: 0:01:10 ( 6.87 ms/it)
+  objective:  13355.70132113299
+Minimizing 10239    Time: 0:01:10 ( 6.87 ms/it)
+  objective:  13355.639947712865
+Minimizing 10254    Time: 0:01:10 ( 6.87 ms/it)
+  objective:  13355.612745383703
+Minimizing 10269    Time: 0:01:10 ( 6.87 ms/it)
+  objective:  13355.547027447261
+Minimizing 10284    Time: 0:01:10 ( 6.87 ms/it)
+  objective:  13355.504593239217
+Minimizing 10299    Time: 0:01:10 ( 6.87 ms/it)
+  objective:  13355.479096091447
+Minimizing 10314    Time: 0:01:10 ( 6.87 ms/it)
+  objective:  13355.30346653847
+Minimizing 10329    Time: 0:01:10 ( 6.87 ms/it)
+  objective:  13355.160378422326
+Minimizing 10344    Time: 0:01:11 ( 6.87 ms/it)
+  objective:  13355.140047027344
+Minimizing 10359    Time: 0:01:11 ( 6.87 ms/it)
+  objective:  13355.087764095952
+Minimizing 10374    Time: 0:01:11 ( 6.87 ms/it)
+  objective:  13355.037673298008
+Minimizing 10388    Time: 0:01:11 ( 6.87 ms/it)
+  objective:  13355.019166930098
+Minimizing 10403    Time: 0:01:11 ( 6.87 ms/it)
+  objective:  13354.947949785586
+Minimizing 10418    Time: 0:01:11 ( 6.87 ms/it)
+  objective:  13354.842603864556
+Minimizing 10433    Time: 0:01:11 ( 6.87 ms/it)
+  objective:  13354.821451744581
+Minimizing 10448    Time: 0:01:11 ( 6.87 ms/it)
+  objective:  13354.695303323715
+Minimizing 10463    Time: 0:01:11 ( 6.87 ms/it)
+  objective:  13354.681614634705
+Minimizing 10478    Time: 0:01:12 ( 6.87 ms/it)
+  objective:  13354.632806485191
+Minimizing 10493    Time: 0:01:12 ( 6.87 ms/it)
+  objective:  13354.610220616662
+Minimizing 10508    Time: 0:01:12 ( 6.87 ms/it)
+  objective:  13354.55800435182
+Minimizing 10523    Time: 0:01:12 ( 6.87 ms/it)
+  objective:  13354.497687714254
+Minimizing 10538    Time: 0:01:12 ( 6.87 ms/it)
+  objective:  13354.478728011163
+Minimizing 10553    Time: 0:01:12 ( 6.87 ms/it)
+  objective:  13354.442410703945
+Minimizing 10568    Time: 0:01:12 ( 6.87 ms/it)
+  objective:  13354.425110274613
+Minimizing 10583    Time: 0:01:12 ( 6.87 ms/it)
+  objective:  13354.413514680775
+Minimizing 10598    Time: 0:01:12 ( 6.87 ms/it)
+  objective:  13354.360065265944
+Minimizing 10613    Time: 0:01:12 ( 6.88 ms/it)
+  objective:  13354.22378888051
+Minimizing 10627    Time: 0:01:13 ( 6.88 ms/it)
+  objective:  13354.218188786632
+Minimizing 10641    Time: 0:01:13 ( 6.88 ms/it)
+  objective:  13354.206655216753
+Minimizing 10656    Time: 0:01:13 ( 6.88 ms/it)
+  objective:  13354.14143136362
+Minimizing 10671    Time: 0:01:13 ( 6.88 ms/it)
+  objective:  13354.135715560813
+Minimizing 10685    Time: 0:01:13 ( 6.88 ms/it)
+  objective:  13354.079150386264
+Minimizing 10700    Time: 0:01:13 ( 6.88 ms/it)
+  objective:  13354.038270700228
+Minimizing 10715    Time: 0:01:13 ( 6.88 ms/it)
+  objective:  13354.019891930562
+Minimizing 10729    Time: 0:01:13 ( 6.88 ms/it)
+  objective:  13353.97626718069
+Minimizing 10744    Time: 0:01:13 ( 6.88 ms/it)
+  objective:  13353.857207170382
+Minimizing 10759    Time: 0:01:13 ( 6.88 ms/it)
+  objective:  13353.726796034593
+Minimizing 10774    Time: 0:01:14 ( 6.88 ms/it)
+  objective:  13353.716947599278
+Minimizing 10788    Time: 0:01:14 ( 6.88 ms/it)
+  objective:  13353.658712991411
+Minimizing 10803    Time: 0:01:14 ( 6.88 ms/it)
+  objective:  13353.511173341547
+Minimizing 10818    Time: 0:01:14 ( 6.88 ms/it)
+  objective:  13353.390063043851
+Minimizing 10833    Time: 0:01:14 ( 6.88 ms/it)
+  objective:  13353.021567532029
+Minimizing 10848    Time: 0:01:14 ( 6.88 ms/it)
+  objective:  13352.897244007203
+Minimizing 10863    Time: 0:01:14 ( 6.88 ms/it)
+  objective:  13352.846662323944
+Minimizing 10878    Time: 0:01:14 ( 6.88 ms/it)
+  objective:  13352.796194022776
+Minimizing 10893    Time: 0:01:14 ( 6.88 ms/it)
+  objective:  13352.723458942033
+Minimizing 10908    Time: 0:01:15 ( 6.88 ms/it)
+  objective:  13352.68034056737
+Minimizing 10923    Time: 0:01:15 ( 6.88 ms/it)
+  objective:  13352.591796056404
+Minimizing 10938    Time: 0:01:15 ( 6.88 ms/it)
+  objective:  13352.572507063749
+Minimizing 10953    Time: 0:01:15 ( 6.88 ms/it)
+  objective:  13352.526107956568
+Minimizing 10968    Time: 0:01:15 ( 6.88 ms/it)
+  objective:  13352.493794555921
+Minimizing 10983    Time: 0:01:15 ( 6.88 ms/it)
+  objective:  13352.548909451558
+Minimizing 10998    Time: 0:01:15 ( 6.88 ms/it)
+  objective:  13352.39552116918
+Minimizing 11013    Time: 0:01:15 ( 6.88 ms/it)
+  objective:  13352.387975489793
+Minimizing 11028    Time: 0:01:15 ( 6.88 ms/it)
+  objective:  13352.369685569334
+Minimizing 11043    Time: 0:01:15 ( 6.88 ms/it)
+  objective:  13352.326268345656
+Minimizing 11058    Time: 0:01:16 ( 6.88 ms/it)
+  objective:  13352.292750105575
+Minimizing 11073    Time: 0:01:16 ( 6.88 ms/it)
+  objective:  13352.229121718381
+Minimizing 11088    Time: 0:01:16 ( 6.88 ms/it)
+  objective:  13352.20230627391
+Minimizing 11103    Time: 0:01:16 ( 6.88 ms/it)
+  objective:  13352.193106757222
+Minimizing 11118    Time: 0:01:16 ( 6.88 ms/it)
+  objective:  13352.171193355884
+Minimizing 11133    Time: 0:01:16 ( 6.88 ms/it)
+  objective:  13352.11501445964
+Minimizing 11148    Time: 0:01:16 ( 6.88 ms/it)
+  objective:  13352.061546726996
+Minimizing 11163    Time: 0:01:16 ( 6.88 ms/it)
+  objective:  13352.004294380182
+Minimizing 11178    Time: 0:01:16 ( 6.88 ms/it)
+  objective:  13351.850530314987
+Minimizing 11193    Time: 0:01:16 ( 6.88 ms/it)
+  objective:  13352.021791293439
+Minimizing 11208    Time: 0:01:17 ( 6.88 ms/it)
+  objective:  13351.855297271475
+Minimizing 11223    Time: 0:01:17 ( 6.88 ms/it)
+  objective:  13351.434472677873
+Minimizing 11238    Time: 0:01:17 ( 6.88 ms/it)
+  objective:  13351.330530201827
+Minimizing 11253    Time: 0:01:17 ( 6.88 ms/it)
+  objective:  13351.241411714283
+Minimizing 11268    Time: 0:01:17 ( 6.88 ms/it)
+  objective:  13351.196807353474
+Minimizing 11283    Time: 0:01:17 ( 6.88 ms/it)
+  objective:  13350.978595137974
+Minimizing 11298    Time: 0:01:17 ( 6.88 ms/it)
+  objective:  13350.953308095632
+Minimizing 11313    Time: 0:01:17 ( 6.88 ms/it)
+  objective:  13350.758985716144
+Minimizing 11328    Time: 0:01:17 ( 6.88 ms/it)
+  objective:  13350.488864257655
+Minimizing 11343    Time: 0:01:18 ( 6.88 ms/it)
+  objective:  13350.480755878918
+Minimizing 11358    Time: 0:01:18 ( 6.88 ms/it)
+  objective:  13350.436838050795
+Minimizing 11373    Time: 0:01:18 ( 6.88 ms/it)
+  objective:  13350.410163312066
+Minimizing 11388    Time: 0:01:18 ( 6.88 ms/it)
+  objective:  13350.304128227202
+Minimizing 11403    Time: 0:01:18 ( 6.88 ms/it)
+  objective:  13350.291904704667
+Minimizing 11418    Time: 0:01:18 ( 6.88 ms/it)
+  objective:  13350.234178762905
+Minimizing 11433    Time: 0:01:18 ( 6.88 ms/it)
+  objective:  13350.564948266867
+Minimizing 11448    Time: 0:01:18 ( 6.88 ms/it)
+  objective:  13349.670339506287
+Minimizing 11463    Time: 0:01:18 ( 6.88 ms/it)
+  objective:  13349.405120001582
+Minimizing 11478    Time: 0:01:18 ( 6.88 ms/it)
+  objective:  13349.40036385908
+Minimizing 11493    Time: 0:01:19 ( 6.88 ms/it)
+  objective:  13349.392075374664
+Minimizing 11508    Time: 0:01:19 ( 6.88 ms/it)
+  objective:  13349.388374022368
+Minimizing 11523    Time: 0:01:19 ( 6.88 ms/it)
+  objective:  13349.32275659265
+Minimizing 11538    Time: 0:01:19 ( 6.88 ms/it)
+  objective:  13349.306908622631
+Minimizing 11553    Time: 0:01:19 ( 6.88 ms/it)
+  objective:  13349.266337743844
+Minimizing 11568    Time: 0:01:19 ( 6.88 ms/it)
+  objective:  13349.253371557563
+Minimizing 11583    Time: 0:01:19 ( 6.88 ms/it)
+  objective:  13349.215085777294
+Minimizing 11598    Time: 0:01:19 ( 6.88 ms/it)
+  objective:  13349.15306443839
+Minimizing 11613    Time: 0:01:19 ( 6.88 ms/it)
+  objective:  13349.142172054228
+Minimizing 11628    Time: 0:01:20 ( 6.88 ms/it)
+  objective:  13349.058044554607
+Minimizing 11643    Time: 0:01:20 ( 6.88 ms/it)
+  objective:  13349.093074445256
+Minimizing 11658    Time: 0:01:20 ( 6.88 ms/it)
+  objective:  13348.750111515088
+Minimizing 11673    Time: 0:01:20 ( 6.88 ms/it)
+  objective:  13348.734737960767
+Minimizing 11688    Time: 0:01:20 ( 6.88 ms/it)
+  objective:  13348.706401411218
+Minimizing 11703    Time: 0:01:20 ( 6.88 ms/it)
+  objective:  13348.632721919224
+Minimizing 11718    Time: 0:01:20 ( 6.88 ms/it)
+  objective:  13348.626477856982
+Minimizing 11733    Time: 0:01:20 ( 6.88 ms/it)
+  objective:  13348.593688355424
+Minimizing 11748    Time: 0:01:20 ( 6.88 ms/it)
+  objective:  13348.546381084569
+Minimizing 11763    Time: 0:01:20 ( 6.88 ms/it)
+  objective:  13348.529474397888
+Minimizing 11778    Time: 0:01:21 ( 6.88 ms/it)
+  objective:  13348.498395327479
+Minimizing 11793    Time: 0:01:21 ( 6.88 ms/it)
+  objective:  13348.485240423724
+Minimizing 11808    Time: 0:01:21 ( 6.88 ms/it)
+  objective:  13348.441812635116
+Minimizing 11823    Time: 0:01:21 ( 6.88 ms/it)
+  objective:  13348.32903826562
+Minimizing 11838    Time: 0:01:21 ( 6.88 ms/it)
+  objective:  13348.221673585009
+Minimizing 11853    Time: 0:01:21 ( 6.88 ms/it)
+  objective:  13348.214896810481
+Minimizing 11868    Time: 0:01:21 ( 6.88 ms/it)
+  objective:  13348.199077259924
+Minimizing 11883    Time: 0:01:21 ( 6.88 ms/it)
+  objective:  13348.178457346374
+Minimizing 11898    Time: 0:01:21 ( 6.88 ms/it)
+  objective:  13348.128041487078
+Minimizing 11913    Time: 0:01:22 ( 6.88 ms/it)
+  objective:  13348.121589186667
+Minimizing 11928    Time: 0:01:22 ( 6.88 ms/it)
+  objective:  13348.08802215854
+Minimizing 11943    Time: 0:01:22 ( 6.88 ms/it)
+  objective:  13348.005637507522
+Minimizing 11958    Time: 0:01:22 ( 6.88 ms/it)
+  objective:  13347.893594041052
+Minimizing 11973    Time: 0:01:22 ( 6.89 ms/it)
+  objective:  13347.884248236129
+Minimizing 11988    Time: 0:01:22 ( 6.89 ms/it)
+  objective:  13347.86313546617
+Minimizing 12003    Time: 0:01:22 ( 6.89 ms/it)
+  objective:  13347.845103490552
+Minimizing 12018    Time: 0:01:22 ( 6.89 ms/it)
+  objective:  13347.766117480576
+Minimizing 12033    Time: 0:01:22 ( 6.89 ms/it)
+  objective:  13347.579171432684
+Minimizing 12048    Time: 0:01:22 ( 6.89 ms/it)
+  objective:  13347.570580253945
+Minimizing 12063    Time: 0:01:23 ( 6.89 ms/it)
+  objective:  13347.560911486748
+Minimizing 12078    Time: 0:01:23 ( 6.89 ms/it)
+  objective:  13347.512697220773
+Minimizing 12093    Time: 0:01:23 ( 6.89 ms/it)
+  objective:  13347.489100522747
+Minimizing 12108    Time: 0:01:23 ( 6.89 ms/it)
+  objective:  13347.483349469083
+Minimizing 12123    Time: 0:01:23 ( 6.89 ms/it)
+  objective:  13347.418547751557
+Minimizing 12138    Time: 0:01:23 ( 6.89 ms/it)
+  objective:  13347.401719889378
+Minimizing 12153    Time: 0:01:23 ( 6.89 ms/it)
+  objective:  13347.375341032988
+Minimizing 12168    Time: 0:01:23 ( 6.89 ms/it)
+  objective:  13347.244254993107
+Minimizing 12183    Time: 0:01:23 ( 6.89 ms/it)
+  objective:  13347.201986245804
+Minimizing 12198    Time: 0:01:24 ( 6.89 ms/it)
+  objective:  13347.183600776145
+Minimizing 12213    Time: 0:01:24 ( 6.89 ms/it)
+  objective:  13347.16438171349
+Minimizing 12228    Time: 0:01:24 ( 6.89 ms/it)
+  objective:  13347.138739760412
+Minimizing 12243    Time: 0:01:24 ( 6.89 ms/it)
+  objective:  13347.124679966422
+Minimizing 12258    Time: 0:01:24 ( 6.89 ms/it)
+  objective:  13347.098915820774
+Minimizing 12273    Time: 0:01:24 ( 6.89 ms/it)
+  objective:  13347.032805347706
+Minimizing 12288    Time: 0:01:24 ( 6.89 ms/it)
+  objective:  13347.026001475497
+Minimizing 12303    Time: 0:01:24 ( 6.89 ms/it)
+  objective:  13346.994198280372
+Minimizing 12318    Time: 0:01:24 ( 6.89 ms/it)
+  objective:  13346.962589302864
+Minimizing 12333    Time: 0:01:24 ( 6.89 ms/it)
+  objective:  13346.995920389236
+Minimizing 12348    Time: 0:01:25 ( 6.89 ms/it)
+  objective:  13346.917171199217
+Minimizing 12363    Time: 0:01:25 ( 6.89 ms/it)
+  objective:  13346.867482292975
+Minimizing 12378    Time: 0:01:25 ( 6.89 ms/it)
+  objective:  13346.849120384446
+Minimizing 12393    Time: 0:01:25 ( 6.89 ms/it)
+  objective:  13346.799488106466
+Minimizing 12408    Time: 0:01:25 ( 6.89 ms/it)
+  objective:  13346.686944626352
+Minimizing 12423    Time: 0:01:25 ( 6.89 ms/it)
+  objective:  13346.48642662655
+Minimizing 12438    Time: 0:01:25 ( 6.89 ms/it)
+  objective:  13346.92594848002
+Minimizing 12453    Time: 0:01:25 ( 6.89 ms/it)
+  objective:  13345.624771598246
+Minimizing 12468    Time: 0:01:25 ( 6.89 ms/it)
+  objective:  13345.622492908384
+Minimizing 12483    Time: 0:01:25 ( 6.89 ms/it)
+  objective:  13345.617883374114
+Minimizing 12498    Time: 0:01:26 ( 6.89 ms/it)
+  objective:  13345.614110157265
+Minimizing 12513    Time: 0:01:26 ( 6.89 ms/it)
+  objective:  13345.605564614998
+Minimizing 12528    Time: 0:01:26 ( 6.89 ms/it)
+  objective:  13345.554699084685
+Minimizing 12543    Time: 0:01:26 ( 6.89 ms/it)
+  objective:  13345.544308497454
+Minimizing 12558    Time: 0:01:26 ( 6.89 ms/it)
+  objective:  13345.491858643989
+Minimizing 12573    Time: 0:01:26 ( 6.89 ms/it)
+  objective:  13345.48424000253
+Minimizing 12588    Time: 0:01:26 ( 6.89 ms/it)
+  objective:  13345.416693552645
+Minimizing 12603    Time: 0:01:26 ( 6.89 ms/it)
+  objective:  13345.409384650491
+Minimizing 12618    Time: 0:01:26 ( 6.89 ms/it)
+  objective:  13345.365771790457
+Minimizing 12633    Time: 0:01:27 ( 6.89 ms/it)
+  objective:  13345.323263412756
+Minimizing 12648    Time: 0:01:27 ( 6.89 ms/it)
+  objective:  13345.315997259175
+Minimizing 12663    Time: 0:01:27 ( 6.89 ms/it)
+  objective:  13345.27816786291
+Minimizing 12678    Time: 0:01:27 ( 6.89 ms/it)
+  objective:  13345.265947161184
+Minimizing 12693    Time: 0:01:27 ( 6.89 ms/it)
+  objective:  13345.241204354403
+Minimizing 12708    Time: 0:01:27 ( 6.89 ms/it)
+  objective:  13345.151973099411
+Minimizing 12723    Time: 0:01:27 ( 6.89 ms/it)
+  objective:  13345.148730351953
+Minimizing 12738    Time: 0:01:27 ( 6.89 ms/it)
+  objective:  13345.143100795212
+Minimizing 12753    Time: 0:01:27 ( 6.89 ms/it)
+  objective:  13345.137366424366
+Minimizing 12768    Time: 0:01:27 ( 6.89 ms/it)
+  objective:  13345.124378438704
+Minimizing 12783    Time: 0:01:28 ( 6.89 ms/it)
+  objective:  13345.111920154908
+Minimizing 12798    Time: 0:01:28 ( 6.89 ms/it)
+  objective:  13345.102961771212
+Minimizing 12813    Time: 0:01:28 ( 6.89 ms/it)
+  objective:  13345.085259086474
+Minimizing 12828    Time: 0:01:28 ( 6.89 ms/it)
+  objective:  13345.07752774781
+Minimizing 12843    Time: 0:01:28 ( 6.89 ms/it)
+  objective:  13345.039197739126
+Minimizing 12858    Time: 0:01:28 ( 6.89 ms/it)
+  objective:  13345.037761111911
+Minimizing 12873    Time: 0:01:28 ( 6.89 ms/it)
+  objective:  13344.945741753778
+Minimizing 12888    Time: 0:01:28 ( 6.89 ms/it)
+  objective:  13344.928693851587
+Minimizing 12903    Time: 0:01:28 ( 6.89 ms/it)
+  objective:  13344.918280682403
+Minimizing 12918    Time: 0:01:29 ( 6.89 ms/it)
+  objective:  13344.895861285731
+Minimizing 12933    Time: 0:01:29 ( 6.89 ms/it)
+  objective:  13344.889098035783
+Minimizing 12948    Time: 0:01:29 ( 6.89 ms/it)
+  objective:  13344.884181286761
+Minimizing 12963    Time: 0:01:29 ( 6.89 ms/it)
+  objective:  13344.869898359102
+Minimizing 12978    Time: 0:01:29 ( 6.89 ms/it)
+  objective:  13344.857608767015
+Minimizing 12993    Time: 0:01:29 ( 6.89 ms/it)
+  objective:  13344.769712908826
+Minimizing 13008    Time: 0:01:29 ( 6.89 ms/it)
+  objective:  13344.763956469105
+Minimizing 13023    Time: 0:01:29 ( 6.89 ms/it)
+  objective:  13344.754110087357
+Minimizing 13038    Time: 0:01:29 ( 6.89 ms/it)
+  objective:  13344.736951753956
+Minimizing 13053    Time: 0:01:29 ( 6.89 ms/it)
+  objective:  13344.723479470667
+Minimizing 13068    Time: 0:01:30 ( 6.89 ms/it)
+  objective:  13344.656055844891
+Minimizing 13083    Time: 0:01:30 ( 6.89 ms/it)
+  objective:  13344.579038085307
+Minimizing 13098    Time: 0:01:30 ( 6.89 ms/it)
+  objective:  13344.572954430514
+Minimizing 13113    Time: 0:01:30 ( 6.89 ms/it)
+  objective:  13344.531565643221
+Minimizing 13128    Time: 0:01:30 ( 6.89 ms/it)
+  objective:  13344.481509258061
+Minimizing 13143    Time: 0:01:30 ( 6.89 ms/it)
+  objective:  13344.474669982046
+Minimizing 13158    Time: 0:01:30 ( 6.89 ms/it)
+  objective:  13344.421327486154
+Minimizing 13173    Time: 0:01:30 ( 6.89 ms/it)
+  objective:  13344.392066209948
+Minimizing 13188    Time: 0:01:30 ( 6.89 ms/it)
+  objective:  13344.358304086294
+Minimizing 13203    Time: 0:01:30 ( 6.89 ms/it)
+  objective:  13344.334720780731
+Minimizing 13218    Time: 0:01:31 ( 6.89 ms/it)
+  objective:  13344.308567977416
+Minimizing 13233    Time: 0:01:31 ( 6.89 ms/it)
+  objective:  13344.23706030583
+Minimizing 13248    Time: 0:01:31 ( 6.89 ms/it)
+  objective:  13344.041097178415
+Minimizing 13263    Time: 0:01:31 ( 6.89 ms/it)
+  objective:  13344.007011748909
+Minimizing 13278    Time: 0:01:31 ( 6.89 ms/it)
+  objective:  13343.98747079874
+Minimizing 13293    Time: 0:01:31 ( 6.89 ms/it)
+  objective:  13343.960268394076
+Minimizing 13308    Time: 0:01:31 ( 6.89 ms/it)
+  objective:  13343.930557065454
+Minimizing 13323    Time: 0:01:31 ( 6.89 ms/it)
+  objective:  13343.893869436302
+Minimizing 13338    Time: 0:01:31 ( 6.89 ms/it)
+  objective:  13343.872125136259
+Minimizing 13353    Time: 0:01:32 ( 6.89 ms/it)
+  objective:  13343.8251938412
+Minimizing 13368    Time: 0:01:32 ( 6.89 ms/it)
+  objective:  13343.80767860065
+Minimizing 13383    Time: 0:01:32 ( 6.89 ms/it)
+  objective:  13343.742183542854
+Minimizing 13398    Time: 0:01:32 ( 6.89 ms/it)
+  objective:  13343.524706619333
+Minimizing 13413    Time: 0:01:32 ( 6.89 ms/it)
+  objective:  13343.519191973894
+Minimizing 13428    Time: 0:01:32 ( 6.89 ms/it)
+  objective:  13343.513145236284
+Minimizing 13443    Time: 0:01:32 ( 6.89 ms/it)
+  objective:  13343.503985637704
+Minimizing 13458    Time: 0:01:32 ( 6.89 ms/it)
+  objective:  13343.500045022993
+Minimizing 13473    Time: 0:01:32 ( 6.89 ms/it)
+  objective:  13343.47800719097
+Minimizing 13488    Time: 0:01:32 ( 6.89 ms/it)
+  objective:  13343.4472022357
+Minimizing 13503    Time: 0:01:33 ( 6.89 ms/it)
+  objective:  13343.435520621017
+Minimizing 13518    Time: 0:01:33 ( 6.89 ms/it)
+  objective:  13343.388298944199
+Minimizing 13533    Time: 0:01:33 ( 6.89 ms/it)
+  objective:  13343.385343308371
+Minimizing 13548    Time: 0:01:33 ( 6.89 ms/it)
+  objective:  13343.340063787291
+Minimizing 13563    Time: 0:01:33 ( 6.89 ms/it)
+  objective:  13343.324650076785
+Minimizing 13578    Time: 0:01:33 ( 6.89 ms/it)
+  objective:  13344.13969906688
+Minimizing 13593    Time: 0:01:33 ( 6.89 ms/it)
+  objective:  13342.765057540884
+Minimizing 13608    Time: 0:01:33 ( 6.89 ms/it)
+  objective:  13342.751842431957
+Minimizing 13623    Time: 0:01:33 ( 6.89 ms/it)
+  objective:  13342.750640718186
+Minimizing 13638    Time: 0:01:34 ( 6.89 ms/it)
+  objective:  13342.748811103957
+Minimizing 13653    Time: 0:01:34 ( 6.89 ms/it)
+  objective:  13342.746436606496
+Minimizing 13668    Time: 0:01:34 ( 6.89 ms/it)
+  objective:  13342.731080381374
+Minimizing 13683    Time: 0:01:34 ( 6.89 ms/it)
+  objective:  13342.633762626407
+Minimizing 13698    Time: 0:01:34 ( 6.89 ms/it)
+  objective:  13342.629647834343
+Minimizing 13713    Time: 0:01:34 ( 6.89 ms/it)
+  objective:  13342.626952746563
+Minimizing 13728    Time: 0:01:34 ( 6.89 ms/it)
+  objective:  13342.612588184544
+Minimizing 13743    Time: 0:01:34 ( 6.89 ms/it)
+  objective:  13342.608858129817
+Minimizing 13758    Time: 0:01:34 ( 6.89 ms/it)
+  objective:  13342.60751029876
+Minimizing 13773    Time: 0:01:34 ( 6.89 ms/it)
+  objective:  13342.598050464214
+Minimizing 13788    Time: 0:01:35 ( 6.89 ms/it)
+  objective:  13342.569516726078
+Minimizing 13803    Time: 0:01:35 ( 6.89 ms/it)
+  objective:  13342.558288428969
+Minimizing 13818    Time: 0:01:35 ( 6.89 ms/it)
+  objective:  13342.54944619767
+Minimizing 13833    Time: 0:01:35 ( 6.89 ms/it)
+  objective:  13342.546611815378
+Minimizing 13848    Time: 0:01:35 ( 6.89 ms/it)
+  objective:  13342.542115552831
+Minimizing 13863    Time: 0:01:35 ( 6.89 ms/it)
+  objective:  13342.497274557572
+Minimizing 13878    Time: 0:01:35 ( 6.89 ms/it)
+  objective:  13342.42747276464
+Minimizing 13893    Time: 0:01:35 ( 6.89 ms/it)
+  objective:  13342.388931177571
+Minimizing 13908    Time: 0:01:35 ( 6.89 ms/it)
+  objective:  13342.381274291023
+Minimizing 13923    Time: 0:01:35 ( 6.89 ms/it)
+  objective:  13342.358395179981
+Minimizing 13938    Time: 0:01:36 ( 6.89 ms/it)
+  objective:  13342.343988913286
+Minimizing 13953    Time: 0:01:36 ( 6.89 ms/it)
+  objective:  13342.340213374962
+Minimizing 13968    Time: 0:01:36 ( 6.89 ms/it)
+  objective:  13342.31791719931
+Minimizing 13983    Time: 0:01:36 ( 6.89 ms/it)
+  objective:  13342.308906502047
+Minimizing 13998    Time: 0:01:36 ( 6.89 ms/it)
+  objective:  13342.254002878639
+Minimizing 14013    Time: 0:01:36 ( 6.89 ms/it)
+  objective:  13342.24522380287
+Minimizing 14028    Time: 0:01:36 ( 6.89 ms/it)
+  objective:  13342.226891926119
+Minimizing 14043    Time: 0:01:36 ( 6.89 ms/it)
+  objective:  13342.219861720085
+Minimizing 14058    Time: 0:01:36 ( 6.89 ms/it)
+  objective:  13342.186885459821
+Minimizing 14073    Time: 0:01:37 ( 6.89 ms/it)
+  objective:  13342.06056912605
+Minimizing 14088    Time: 0:01:37 ( 6.89 ms/it)
+  objective:  13342.050330003163
+Minimizing 14103    Time: 0:01:37 ( 6.89 ms/it)
+  objective:  13342.02049463983
+Minimizing 14118    Time: 0:01:37 ( 6.89 ms/it)
+  objective:  13342.00188136968
+Minimizing 14133    Time: 0:01:37 ( 6.89 ms/it)
+  objective:  13342.000270805038
+Minimizing 14148    Time: 0:01:37 ( 6.89 ms/it)
+  objective:  13341.959773801522
+Minimizing 14163    Time: 0:01:37 ( 6.89 ms/it)
+  objective:  13341.958891604772
+Minimizing 14178    Time: 0:01:37 ( 6.89 ms/it)
+  objective:  13341.938013339626
+Minimizing 14193    Time: 0:01:37 ( 6.89 ms/it)
+  objective:  13341.908900629838
+Minimizing 14208    Time: 0:01:37 ( 6.89 ms/it)
+  objective:  13341.87365635595
+Minimizing 14223    Time: 0:01:38 ( 6.89 ms/it)
+  objective:  13341.86515817271
+Minimizing 14238    Time: 0:01:38 ( 6.89 ms/it)
+  objective:  13341.82831220379
+Minimizing 14253    Time: 0:01:38 ( 6.89 ms/it)
+  objective:  13341.825611467793
+Minimizing 14268    Time: 0:01:38 ( 6.89 ms/it)
+  objective:  13341.779450060836
+Minimizing 14283    Time: 0:01:38 ( 6.89 ms/it)
+  objective:  13341.778552667434
+Minimizing 14298    Time: 0:01:38 ( 6.89 ms/it)
+  objective:  13341.736163047404
+Minimizing 14313    Time: 0:01:38 ( 6.89 ms/it)
+  objective:  13341.728549369509
+Minimizing 14328    Time: 0:01:38 ( 6.89 ms/it)
+  objective:  13341.703864172676
+Minimizing 14343    Time: 0:01:38 ( 6.89 ms/it)
+  objective:  13341.69038413126
+Minimizing 14358    Time: 0:01:38 ( 6.89 ms/it)
+  objective:  13341.671579814429
+Minimizing 14373    Time: 0:01:39 ( 6.89 ms/it)
+  objective:  13341.634711431245
+Minimizing 14388    Time: 0:01:39 ( 6.89 ms/it)
+  objective:  13341.60397035969
+Minimizing 14403    Time: 0:01:39 ( 6.89 ms/it)
+  objective:  13341.445012514996
+Minimizing 14418    Time: 0:01:39 ( 6.89 ms/it)
+  objective:  13341.365024654224
+Minimizing 14433    Time: 0:01:39 ( 6.89 ms/it)
+  objective:  13341.36092733089
+Minimizing 14448    Time: 0:01:39 ( 6.90 ms/it)
+  objective:  13341.326760094555
+Minimizing 14463    Time: 0:01:39 ( 6.90 ms/it)
+  objective:  13341.313083627363
+Minimizing 14478    Time: 0:01:39 ( 6.90 ms/it)
+  objective:  13341.281522976933
+Minimizing 14493    Time: 0:01:39 ( 6.90 ms/it)
+  objective:  13341.130622068958
+Minimizing 14508    Time: 0:01:40 ( 6.90 ms/it)
+  objective:  13341.125170117783
+Minimizing 14523    Time: 0:01:40 ( 6.90 ms/it)
+  objective:  13341.123832796904
+Minimizing 14538    Time: 0:01:40 ( 6.90 ms/it)
+  objective:  13341.115333240581
+Minimizing 14553    Time: 0:01:40 ( 6.90 ms/it)
+  objective:  13341.105340322218
+Minimizing 14568    Time: 0:01:40 ( 6.90 ms/it)
+  objective:  13341.10004119383
+Minimizing 14583    Time: 0:01:40 ( 6.90 ms/it)
+  objective:  13341.04061021009
+Minimizing 14598    Time: 0:01:40 ( 6.90 ms/it)
+  objective:  13340.998292570279
+Minimizing 14613    Time: 0:01:40 ( 6.90 ms/it)
+  objective:  13340.980129442374
+Minimizing 14628    Time: 0:01:40 ( 6.90 ms/it)
+  objective:  13340.887975051883
+Minimizing 14643    Time: 0:01:40 ( 6.90 ms/it)
+  objective:  13340.850648001644
+Minimizing 14657    Time: 0:01:41 ( 6.90 ms/it)
+  objective:  13340.775113465228
+Minimizing 14672    Time: 0:01:41 ( 6.90 ms/it)
+  objective:  13340.618883324612
+Minimizing 14687    Time: 0:01:41 ( 6.90 ms/it)
+  objective:  13339.93038837218
+Minimizing 14702    Time: 0:01:41 ( 6.90 ms/it)
+  objective:  13339.802194371841
+Minimizing 14717    Time: 0:01:41 ( 6.90 ms/it)
+  objective:  13339.79417384282
+Minimizing 14732    Time: 0:01:41 ( 6.90 ms/it)
+  objective:  13339.784141392709
+Minimizing 14747    Time: 0:01:41 ( 6.90 ms/it)
+  objective:  13339.778731724022
+Minimizing 14762    Time: 0:01:41 ( 6.90 ms/it)
+  objective:  13339.736090131417
+Minimizing 14777    Time: 0:01:41 ( 6.90 ms/it)
+  objective:  13339.716177164373
+Minimizing 14792    Time: 0:01:42 ( 6.90 ms/it)
+  objective:  13339.634403322947
+Minimizing 14807    Time: 0:01:42 ( 6.90 ms/it)
+  objective:  13340.087119650794
+Minimizing 14822    Time: 0:01:42 ( 6.90 ms/it)
+  objective:  13339.414491383934
+Minimizing 14837    Time: 0:01:42 ( 6.90 ms/it)
+  objective:  13339.411070155846
+Minimizing 14852    Time: 0:01:42 ( 6.90 ms/it)
+  objective:  13339.39178276476
+Minimizing 14867    Time: 0:01:42 ( 6.90 ms/it)
+  objective:  13339.378136657353
+Minimizing 14882    Time: 0:01:42 ( 6.90 ms/it)
+  objective:  13339.365104074852
+Minimizing 14897    Time: 0:01:42 ( 6.90 ms/it)
+  objective:  13339.344841596583
+Minimizing 14912    Time: 0:01:42 ( 6.90 ms/it)
+  objective:  13339.291933889188
+Minimizing 14927    Time: 0:01:42 ( 6.90 ms/it)
+  objective:  13339.098997031077
+Minimizing 14942    Time: 0:01:43 ( 6.90 ms/it)
+  objective:  13339.093388413305
+Minimizing 14957    Time: 0:01:43 ( 6.90 ms/it)
+  objective:  13339.06705239074
+Minimizing 14972    Time: 0:01:43 ( 6.90 ms/it)
+  objective:  13339.04894409424
+Minimizing 14987    Time: 0:01:43 ( 6.90 ms/it)
+  objective:  13339.038613203244
+Minimizing 15002    Time: 0:01:43 ( 6.90 ms/it)
+  objective:  13338.997378755244
+Minimizing 15017    Time: 0:01:43 ( 6.90 ms/it)
+  objective:  13338.932883187394
+Minimizing 15032    Time: 0:01:43 ( 6.90 ms/it)
+  objective:  13338.84944278629
+Minimizing 15047    Time: 0:01:43 ( 6.90 ms/it)
+  objective:  13338.621033794087
+Minimizing 15062    Time: 0:01:43 ( 6.90 ms/it)
+  objective:  13338.614366147114
+Minimizing 15077    Time: 0:01:44 ( 6.90 ms/it)
+  objective:  13338.59184768518
+Minimizing 15092    Time: 0:01:44 ( 6.90 ms/it)
+  objective:  13338.579031379988
+Minimizing 15107    Time: 0:01:44 ( 6.90 ms/it)
+  objective:  13338.558751188066
+Minimizing 15122    Time: 0:01:44 ( 6.90 ms/it)
+  objective:  13338.549335210111
+Minimizing 15137    Time: 0:01:44 ( 6.90 ms/it)
+  objective:  13338.452531246578
+Minimizing 15152    Time: 0:01:44 ( 6.90 ms/it)
+  objective:  13338.433048698353
+Minimizing 15167    Time: 0:01:44 ( 6.90 ms/it)
+  objective:  13338.433354479355
+Minimizing 15182    Time: 0:01:44 ( 6.90 ms/it)
+  objective:  13338.404106597074
+Minimizing 15197    Time: 0:01:44 ( 6.90 ms/it)
+  objective:  13338.382898093128
+Minimizing 15212    Time: 0:01:44 ( 6.90 ms/it)
+  objective:  13338.380698813948
+Minimizing 15227    Time: 0:01:45 ( 6.90 ms/it)
+  objective:  13338.376206276116
+Minimizing 15242    Time: 0:01:45 ( 6.90 ms/it)
+  objective:  13338.37009423472
+Minimizing 15257    Time: 0:01:45 ( 6.90 ms/it)
+  objective:  13338.332375120226
+Minimizing 15272    Time: 0:01:45 ( 6.90 ms/it)
+  objective:  13338.315422826745
+Minimizing 15287    Time: 0:01:45 ( 6.90 ms/it)
+  objective:  13338.30016207126
+Minimizing 15302    Time: 0:01:45 ( 6.90 ms/it)
+  objective:  13338.28237848018
+Minimizing 15317    Time: 0:01:45 ( 6.90 ms/it)
+  objective:  13338.058349866842
+Minimizing 15332    Time: 0:01:45 ( 6.90 ms/it)
+  objective:  13337.995714274839
+Minimizing 15347    Time: 0:01:45 ( 6.90 ms/it)
+  objective:  13337.991177958575
+Minimizing 15362    Time: 0:01:46 ( 6.90 ms/it)
+  objective:  13337.977711082582
+Minimizing 15377    Time: 0:01:46 ( 6.90 ms/it)
+  objective:  13337.964303749359
+Minimizing 15392    Time: 0:01:46 ( 6.90 ms/it)
+  objective:  13337.950417868175
+Minimizing 15407    Time: 0:01:46 ( 6.90 ms/it)
+  objective:  13337.924826576214
+Minimizing 15422    Time: 0:01:46 ( 6.90 ms/it)
+  objective:  13337.922166207369
+Minimizing 15437    Time: 0:01:46 ( 6.90 ms/it)
+  objective:  13337.915342303437
+Minimizing 15452    Time: 0:01:46 ( 6.90 ms/it)
+  objective:  13337.892925831431
+Minimizing 15467    Time: 0:01:46 ( 6.90 ms/it)
+  objective:  13337.870776943266
+Minimizing 15482    Time: 0:01:46 ( 6.90 ms/it)
+  objective:  13337.82368052403
+Minimizing 15497    Time: 0:01:46 ( 6.90 ms/it)
+  objective:  13337.802456079604
+Minimizing 15512    Time: 0:01:47 ( 6.90 ms/it)
+  objective:  13337.778319405399
+Minimizing 15527    Time: 0:01:47 ( 6.90 ms/it)
+  objective:  13337.723085633355
+Minimizing 15542    Time: 0:01:47 ( 6.90 ms/it)
+  objective:  13337.681336951027
+Minimizing 15557    Time: 0:01:47 ( 6.90 ms/it)
+  objective:  13337.623714578207
+Minimizing 15572    Time: 0:01:47 ( 6.90 ms/it)
+  objective:  13337.619912843751
+Minimizing 15587    Time: 0:01:47 ( 6.90 ms/it)
+  objective:  13337.592407365271
+Minimizing 15602    Time: 0:01:47 ( 6.90 ms/it)
+  objective:  13337.524977112553
+Minimizing 15617    Time: 0:01:47 ( 6.90 ms/it)
+  objective:  13337.501384933617
+Minimizing 15632    Time: 0:01:47 ( 6.90 ms/it)
+  objective:  13337.372896101675
+Minimizing 15647    Time: 0:01:47 ( 6.90 ms/it)
+  objective:  13337.244769924691
+Minimizing 15662    Time: 0:01:48 ( 6.90 ms/it)
+  objective:  13337.219267041073
+Minimizing 15677    Time: 0:01:48 ( 6.90 ms/it)
+  objective:  13337.209281208532
+Minimizing 15692    Time: 0:01:48 ( 6.90 ms/it)
+  objective:  13337.194651381185
+Minimizing 15707    Time: 0:01:48 ( 6.90 ms/it)
+  objective:  13337.163609343253
+Minimizing 15722    Time: 0:01:48 ( 6.90 ms/it)
+  objective:  13337.367427068624
+Minimizing 15737    Time: 0:01:48 ( 6.90 ms/it)
+  objective:  13337.000329675291
+Minimizing 15752    Time: 0:01:48 ( 6.90 ms/it)
+  objective:  13336.977122713302
+Minimizing 15767    Time: 0:01:48 ( 6.90 ms/it)
+  objective:  13336.94370140139
+Minimizing 15782    Time: 0:01:48 ( 6.90 ms/it)
+  objective:  13336.92256336382
+Minimizing 15797    Time: 0:01:49 ( 6.90 ms/it)
+  objective:  13336.912565204024
+Minimizing 15812    Time: 0:01:49 ( 6.90 ms/it)
+  objective:  13336.846513392375
+Minimizing 15827    Time: 0:01:49 ( 6.90 ms/it)
+  objective:  13336.756989461734
+Minimizing 15842    Time: 0:01:49 ( 6.90 ms/it)
+  objective:  13336.615976333116
+Minimizing 15857    Time: 0:01:49 ( 6.90 ms/it)
+  objective:  13336.269223868083
+Minimizing 15872    Time: 0:01:49 ( 6.90 ms/it)
+  objective:  13336.261366299914
+Minimizing 15887    Time: 0:01:49 ( 6.90 ms/it)
+  objective:  13336.25777220614
+Minimizing 15902    Time: 0:01:49 ( 6.90 ms/it)
+  objective:  13336.256172290574
+Minimizing 15917    Time: 0:01:49 ( 6.90 ms/it)
+  objective:  13336.251656851491
+Minimizing 15931    Time: 0:01:49 ( 6.90 ms/it)
+  objective:  13336.245449235874
+Minimizing 15945    Time: 0:01:50 ( 6.90 ms/it)
+  objective:  13336.232147575625
+Minimizing 15958    Time: 0:01:50 ( 6.90 ms/it)
+  objective:  13336.21083419981
+Minimizing 15972    Time: 0:01:50 ( 6.90 ms/it)
+  objective:  13336.182484429686
+Minimizing 15986    Time: 0:01:50 ( 6.91 ms/it)
+  objective:  13336.175686567054
+Minimizing 16000    Time: 0:01:50 ( 6.91 ms/it)
+  objective:  13336.14994236223
+Minimizing 16014    Time: 0:01:50 ( 6.91 ms/it)
+  objective:  13336.103789755027
+Minimizing 16028    Time: 0:01:50 ( 6.91 ms/it)
+  objective:  13335.925686136121
+Minimizing 16042    Time: 0:01:50 ( 6.91 ms/it)
+  objective:  13335.920391065847
+Minimizing 16056    Time: 0:01:50 ( 6.91 ms/it)
+  objective:  13335.896999862474
+Minimizing 16070    Time: 0:01:51 ( 6.91 ms/it)
+  objective:  13335.861006309897
+Minimizing 16084    Time: 0:01:51 ( 6.91 ms/it)
+  objective:  13335.777256011628
+Minimizing 16098    Time: 0:01:51 ( 6.91 ms/it)
+  objective:  13335.745853048706
+Minimizing 16112    Time: 0:01:51 ( 6.91 ms/it)
+  objective:  13335.74129549932
+Minimizing 16126    Time: 0:01:51 ( 6.91 ms/it)
+  objective:  13335.801324112384
+Minimizing 16140    Time: 0:01:51 ( 6.91 ms/it)
+  objective:  13335.624783455154
+Minimizing 16154    Time: 0:01:51 ( 6.91 ms/it)
+  objective:  13335.579189488424
+Minimizing 16168    Time: 0:01:51 ( 6.91 ms/it)
+  objective:  13335.415252530795
+Minimizing 16182    Time: 0:01:51 ( 6.91 ms/it)
+  objective:  13335.223253945842
+Minimizing 16196    Time: 0:01:51 ( 6.91 ms/it)
+  objective:  13335.221178535947
+Minimizing 16210    Time: 0:01:52 ( 6.91 ms/it)
+  objective:  13335.219536509227
+Minimizing 16224    Time: 0:01:52 ( 6.91 ms/it)
+  objective:  13335.21841234529
+Minimizing 16238    Time: 0:01:52 ( 6.91 ms/it)
+  objective:  13335.217925056699
+Minimizing 16252    Time: 0:01:52 ( 6.91 ms/it)
+  objective:  13335.210506432384
+Minimizing 16266    Time: 0:01:52 ( 6.91 ms/it)
+  objective:  13335.183660413655
+Minimizing 16280    Time: 0:01:52 ( 6.91 ms/it)
+  objective:  13335.133617477237
+Minimizing 16294    Time: 0:01:52 ( 6.91 ms/it)
+  objective:  13335.122020143863
+Minimizing 16308    Time: 0:01:52 ( 6.91 ms/it)
+  objective:  13335.118897177003
+Minimizing 16322    Time: 0:01:52 ( 6.91 ms/it)
+  objective:  13335.108725729217
+Minimizing 16336    Time: 0:01:52 ( 6.91 ms/it)
+  objective:  13335.105212219372
+Minimizing 16350    Time: 0:01:53 ( 6.91 ms/it)
+  objective:  13335.095815820729
+Minimizing 16364    Time: 0:01:53 ( 6.91 ms/it)
+  objective:  13335.06571175185
+Minimizing 16378    Time: 0:01:53 ( 6.91 ms/it)
+  objective:  13335.061220820906
+Minimizing 16392    Time: 0:01:53 ( 6.92 ms/it)
+  objective:  13335.008390296942
+Minimizing 16406    Time: 0:01:53 ( 6.92 ms/it)
+  objective:  13334.99009610057
+Minimizing 16420    Time: 0:01:53 ( 6.92 ms/it)
+  objective:  13334.916519054612
+Minimizing 16435    Time: 0:01:53 ( 6.92 ms/it)
+  objective:  13334.845595854706
+Minimizing 16449    Time: 0:01:53 ( 6.92 ms/it)
+  objective:  13334.843490636966
+Minimizing 16463    Time: 0:01:53 ( 6.92 ms/it)
+  objective:  13334.836694603458
+Minimizing 16477    Time: 0:01:53 ( 6.92 ms/it)
+  objective:  13334.82669134588
+Minimizing 16491    Time: 0:01:54 ( 6.92 ms/it)
+  objective:  13334.806544282648
+Minimizing 16505    Time: 0:01:54 ( 6.92 ms/it)
+  objective:  13334.801103618047
+Minimizing 16519    Time: 0:01:54 ( 6.92 ms/it)
+  objective:  13334.759274538636
+Minimizing 16533    Time: 0:01:54 ( 6.92 ms/it)
+  objective:  13334.751347941929
+Minimizing 16547    Time: 0:01:54 ( 6.92 ms/it)
+  objective:  13334.728748197238
+Minimizing 16561    Time: 0:01:54 ( 6.92 ms/it)
+  objective:  13334.64750024394
+Minimizing 16575    Time: 0:01:54 ( 6.92 ms/it)
+  objective:  13334.583685550962
+Minimizing 16589    Time: 0:01:54 ( 6.92 ms/it)
+  objective:  13334.583280504725
+Minimizing 16603    Time: 0:01:54 ( 6.92 ms/it)
+  objective:  13334.569952924663
+Minimizing 16618    Time: 0:01:54 ( 6.92 ms/it)
+  objective:  13334.567988298215
+Minimizing 16633    Time: 0:01:55 ( 6.92 ms/it)
+  objective:  13334.546262842334
+Minimizing 16648    Time: 0:01:55 ( 6.92 ms/it)
+  objective:  13334.544138925514
+Minimizing 16663    Time: 0:01:55 ( 6.92 ms/it)
+  objective:  13334.510710211216
+Minimizing 16678    Time: 0:01:55 ( 6.92 ms/it)
+  objective:  13334.476889563892
+Minimizing 16693    Time: 0:01:55 ( 6.92 ms/it)
+  objective:  13334.454626424107
+Minimizing 16708    Time: 0:01:55 ( 6.92 ms/it)
+  objective:  13334.362062850785
+Minimizing 16723    Time: 0:01:55 ( 6.92 ms/it)
+  objective:  13334.360619089632
+Minimizing 16738    Time: 0:01:55 ( 6.92 ms/it)
+  objective:  13334.341550746001
+Minimizing 16753    Time: 0:01:55 ( 6.92 ms/it)
+  objective:  13334.320402974976
+Minimizing 16768    Time: 0:01:56 ( 6.92 ms/it)
+  objective:  13334.307457812945
+Minimizing 16783    Time: 0:01:56 ( 6.92 ms/it)
+  objective:  13334.224614883999
+Minimizing 16798    Time: 0:01:56 ( 6.92 ms/it)
+  objective:  13334.095644262517
+Minimizing 16813    Time: 0:01:56 ( 6.92 ms/it)
+  objective:  13334.054532773436
+Minimizing 16828    Time: 0:01:56 ( 6.92 ms/it)
+  objective:  13334.052562382632
+Minimizing 16843    Time: 0:01:56 ( 6.92 ms/it)
+  objective:  13334.02038898351
+Minimizing 16858    Time: 0:01:56 ( 6.92 ms/it)
+  objective:  13334.014248053682
+Minimizing 16873    Time: 0:01:56 ( 6.92 ms/it)
+  objective:  13333.988855241048
+Minimizing 16888    Time: 0:01:56 ( 6.92 ms/it)
+  objective:  13333.867898937198
+Minimizing 16903    Time: 0:01:56 ( 6.92 ms/it)
+  objective:  13333.859219855774
+Minimizing 16918    Time: 0:01:57 ( 6.92 ms/it)
+  objective:  13333.846383981363
+Minimizing 16933    Time: 0:01:57 ( 6.92 ms/it)
+  objective:  13333.84357290766
+Minimizing 16948    Time: 0:01:57 ( 6.92 ms/it)
+  objective:  13333.84133947506
+Minimizing 16963    Time: 0:01:57 ( 6.92 ms/it)
+  objective:  13333.837289346578
+Minimizing 16978    Time: 0:01:57 ( 6.92 ms/it)
+  objective:  13333.819624881588
+Minimizing 16993    Time: 0:01:57 ( 6.92 ms/it)
+  objective:  13333.75923950121
+Minimizing 17008    Time: 0:01:57 ( 6.92 ms/it)
+  objective:  13333.758536256828
+Minimizing 17023    Time: 0:01:57 ( 6.92 ms/it)
+  objective:  13333.746101364486
+Minimizing 17038    Time: 0:01:57 ( 6.92 ms/it)
+  objective:  13333.741782106052
+Minimizing 17053    Time: 0:01:58 ( 6.92 ms/it)
+  objective:  13333.720342352091
+Minimizing 17068    Time: 0:01:58 ( 6.92 ms/it)
+  objective:  13333.688469015011
+Minimizing 17083    Time: 0:01:58 ( 6.92 ms/it)
+  objective:  13333.609223999862
+Minimizing 17098    Time: 0:01:58 ( 6.92 ms/it)
+  objective:  13333.608730716616
+Minimizing 17112    Time: 0:01:58 ( 6.92 ms/it)
+  objective:  13333.606889065406
+Minimizing 17125    Time: 0:01:58 ( 6.92 ms/it)
+  objective:  13333.600911253809
+Minimizing 17139    Time: 0:01:58 ( 6.92 ms/it)
+  objective:  13333.599414144795
+Minimizing 17153    Time: 0:01:58 ( 6.92 ms/it)
+  objective:  13333.58434479226
+Minimizing 17166    Time: 0:01:58 ( 6.93 ms/it)
+  objective:  13333.561687261295
+Minimizing 17180    Time: 0:01:58 ( 6.93 ms/it)
+  objective:  13333.559435556308
+Minimizing 17194    Time: 0:01:59 ( 6.93 ms/it)
+  objective:  13333.552408369571
+Minimizing 17209    Time: 0:01:59 ( 6.93 ms/it)
+  objective:  13333.54321048825
+Minimizing 17224    Time: 0:01:59 ( 6.93 ms/it)
+  objective:  13333.515494220796
+Minimizing 17239    Time: 0:01:59 ( 6.93 ms/it)
+  objective:  13333.499174522709
+Minimizing 17254    Time: 0:01:59 ( 6.93 ms/it)
+  objective:  13333.493274650835
+Minimizing 17269    Time: 0:01:59 ( 6.93 ms/it)
+  objective:  13333.455186557658
+Minimizing 17284    Time: 0:01:59 ( 6.93 ms/it)
+  objective:  13333.437993298452
+Minimizing 17299    Time: 0:01:59 ( 6.93 ms/it)
+  objective:  13333.320915752374
+Minimizing 17314    Time: 0:01:59 ( 6.93 ms/it)
+  objective:  13333.290149521541
+Minimizing 17329    Time: 0:02:00 ( 6.93 ms/it)
+  objective:  13333.289035316557
+Minimizing 17344    Time: 0:02:00 ( 6.93 ms/it)
+  objective:  13333.279764576284
+Minimizing 17359    Time: 0:02:00 ( 6.93 ms/it)
+  objective:  13333.262703755448
+Minimizing 17374    Time: 0:02:00 ( 6.93 ms/it)
+  objective:  13333.231376314456
+Minimizing 17389    Time: 0:02:00 ( 6.93 ms/it)
+  objective:  13333.22627246497
+Minimizing 17404    Time: 0:02:00 ( 6.93 ms/it)
+  objective:  13333.216820631176
+Minimizing 17419    Time: 0:02:00 ( 6.93 ms/it)
+  objective:  13333.180143574398
+Minimizing 17434    Time: 0:02:00 ( 6.93 ms/it)
+  objective:  13333.158613650587
+Minimizing 17449    Time: 0:02:00 ( 6.93 ms/it)
+  objective:  13333.14023486711
+Minimizing 17464    Time: 0:02:00 ( 6.93 ms/it)
+  objective:  13333.126857057105
+Minimizing 17479    Time: 0:02:01 ( 6.93 ms/it)
+  objective:  13333.106941937003
+Minimizing 17494    Time: 0:02:01 ( 6.93 ms/it)
+  objective:  13333.056970260404
+Minimizing 17509    Time: 0:02:01 ( 6.93 ms/it)
+  objective:  13333.050433719633
+Minimizing 17524    Time: 0:02:01 ( 6.93 ms/it)
+  objective:  13333.04520694173
+Minimizing 17539    Time: 0:02:01 ( 6.93 ms/it)
+  objective:  13333.02308609712
+Minimizing 17554    Time: 0:02:01 ( 6.93 ms/it)
+  objective:  13333.013994719513
+Minimizing 17569    Time: 0:02:01 ( 6.93 ms/it)
+  objective:  13332.977936083349
+Minimizing 17584    Time: 0:02:01 ( 6.93 ms/it)
+  objective:  13332.975511619945
+Minimizing 17599    Time: 0:02:01 ( 6.93 ms/it)
+  objective:  13332.958474803265
+Minimizing 17614    Time: 0:02:01 ( 6.93 ms/it)
+  objective:  13332.952854860698
+Minimizing 17629    Time: 0:02:02 ( 6.93 ms/it)
+  objective:  13332.919815430047
+Minimizing 17644    Time: 0:02:02 ( 6.93 ms/it)
+  objective:  13332.882694401924
+Minimizing 17659    Time: 0:02:02 ( 6.93 ms/it)
+  objective:  13332.878704391602
+Minimizing 17674    Time: 0:02:02 ( 6.93 ms/it)
+  objective:  13332.851167182474
+Minimizing 17689    Time: 0:02:02 ( 6.93 ms/it)
+  objective:  13332.788209513761
+Minimizing 17704    Time: 0:02:02 ( 6.93 ms/it)
+  objective:  13332.703097644699
+Minimizing 17719    Time: 0:02:02 ( 6.93 ms/it)
+  objective:  13332.701187844163
+Minimizing 17734    Time: 0:02:02 ( 6.93 ms/it)
+  objective:  13332.697815801825
+Minimizing 17749    Time: 0:02:02 ( 6.93 ms/it)
+  objective:  13332.690323827534
+Minimizing 17764    Time: 0:02:03 ( 6.93 ms/it)
+  objective:  13332.687247730108
+Minimizing 17779    Time: 0:02:03 ( 6.93 ms/it)
+  objective:  13332.673238247422
+Minimizing 17794    Time: 0:02:03 ( 6.93 ms/it)
+  objective:  13332.660994644226
+Minimizing 17809    Time: 0:02:03 ( 6.93 ms/it)
+  objective:  13332.64622725836
+Minimizing 17824    Time: 0:02:03 ( 6.93 ms/it)
+  objective:  13332.645241794671
+Minimizing 17839    Time: 0:02:03 ( 6.93 ms/it)
+  objective:  13332.640238329994
+Minimizing 17854    Time: 0:02:03 ( 6.93 ms/it)
+  objective:  13332.631622952686
+Minimizing 17869    Time: 0:02:03 ( 6.93 ms/it)
+  objective:  13332.612804732627
+Minimizing 17884    Time: 0:02:03 ( 6.93 ms/it)
+  objective:  13332.607820559344
+Minimizing 17899    Time: 0:02:03 ( 6.92 ms/it)
+  objective:  13332.600145817894
+Minimizing 17914    Time: 0:02:04 ( 6.92 ms/it)
+  objective:  13332.529633013582
+Minimizing 17929    Time: 0:02:04 ( 6.92 ms/it)
+  objective:  13332.501587057173
+Minimizing 17944    Time: 0:02:04 ( 6.92 ms/it)
+  objective:  13332.499290583983
+Minimizing 17959    Time: 0:02:04 ( 6.92 ms/it)
+  objective:  13332.489295514795
+Minimizing 17974    Time: 0:02:04 ( 6.92 ms/it)
+  objective:  13332.484992574231
+Minimizing 17989    Time: 0:02:04 ( 6.92 ms/it)
+  objective:  13332.454313224043
+Minimizing 18004    Time: 0:02:04 ( 6.92 ms/it)
+  objective:  13332.453012897939
+Minimizing 18019    Time: 0:02:04 ( 6.92 ms/it)
+  objective:  13332.444465408546
+Minimizing 18034    Time: 0:02:04 ( 6.92 ms/it)
+  objective:  13332.43640637002
+Minimizing 18049    Time: 0:02:04 ( 6.92 ms/it)
+  objective:  13332.415250480924
+Minimizing 18064    Time: 0:02:05 ( 6.92 ms/it)
+  objective:  13332.38002569967
+Minimizing 18079    Time: 0:02:05 ( 6.92 ms/it)
+  objective:  13332.343783860095
+Minimizing 18094    Time: 0:02:05 ( 6.92 ms/it)
+  objective:  13332.342943183561
+Minimizing 18109    Time: 0:02:05 ( 6.92 ms/it)
+  objective:  13332.328013336613
+Minimizing 18124    Time: 0:02:05 ( 6.92 ms/it)
+  objective:  13332.304945329277
+Minimizing 18139    Time: 0:02:05 ( 6.92 ms/it)
+  objective:  13332.288060278857
+Minimizing 18154    Time: 0:02:05 ( 6.92 ms/it)
+  objective:  13332.252287369141
+Minimizing 18169    Time: 0:02:05 ( 6.92 ms/it)
+  objective:  13332.194609537379
+Minimizing 18184    Time: 0:02:05 ( 6.92 ms/it)
+  objective:  13332.183633060944
+Minimizing 18199    Time: 0:02:05 ( 6.92 ms/it)
+  objective:  13332.144665969412
+Minimizing 18214    Time: 0:02:06 ( 6.92 ms/it)
+  objective:  13332.135833717242
+Minimizing 18229    Time: 0:02:06 ( 6.92 ms/it)
+  objective:  13332.13156465999
+Minimizing 18244    Time: 0:02:06 ( 6.92 ms/it)
+  objective:  13332.11959062986
+Minimizing 18259    Time: 0:02:06 ( 6.92 ms/it)
+  objective:  13332.110825140153
+Minimizing 18274    Time: 0:02:06 ( 6.92 ms/it)
+  objective:  13332.09751962519
+Minimizing 18289    Time: 0:02:06 ( 6.92 ms/it)
+  objective:  13332.079641709526
+Minimizing 18304    Time: 0:02:06 ( 6.92 ms/it)
+  objective:  13332.079195920887
+Minimizing 18319    Time: 0:02:06 ( 6.92 ms/it)
+  objective:  13332.05560646786
+Minimizing 18334    Time: 0:02:06 ( 6.92 ms/it)
+  objective:  13332.033062607668
+Minimizing 18349    Time: 0:02:07 ( 6.92 ms/it)
+  objective:  13332.017118449927
+Minimizing 18364    Time: 0:02:07 ( 6.92 ms/it)
+  objective:  13331.834283003831
+Minimizing 18379    Time: 0:02:07 ( 6.92 ms/it)
+  objective:  13331.829523296801
+Minimizing 18394    Time: 0:02:07 ( 6.92 ms/it)
+  objective:  13331.82185353677
+Minimizing 18409    Time: 0:02:07 ( 6.92 ms/it)
+  objective:  13331.811126325367
+Minimizing 18424    Time: 0:02:07 ( 6.92 ms/it)
+  objective:  13331.796843724907
+Minimizing 18439    Time: 0:02:07 ( 6.92 ms/it)
+  objective:  13331.773058564875
+Minimizing 18454    Time: 0:02:07 ( 6.92 ms/it)
+  objective:  13331.75078281766
+Minimizing 18469    Time: 0:02:07 ( 6.92 ms/it)
+  objective:  13331.740070893735
+Minimizing 18484    Time: 0:02:07 ( 6.92 ms/it)
+  objective:  13331.729061820188
+Minimizing 18499    Time: 0:02:08 ( 6.92 ms/it)
+  objective:  13331.727070406356
+Minimizing 18514    Time: 0:02:08 ( 6.92 ms/it)
+  objective:  13331.698226752102
+Minimizing 18529    Time: 0:02:08 ( 6.92 ms/it)
+  objective:  13331.662677190616
+Minimizing 18544    Time: 0:02:08 ( 6.92 ms/it)
+  objective:  13331.435033106842
+Minimizing 18559    Time: 0:02:08 ( 6.92 ms/it)
+  objective:  13331.418241083811
+Minimizing 18574    Time: 0:02:08 ( 6.92 ms/it)
+  objective:  13331.412412329562
+Minimizing 18589    Time: 0:02:08 ( 6.92 ms/it)
+  objective:  13331.406819941993
+Minimizing 18604    Time: 0:02:08 ( 6.92 ms/it)
+  objective:  13331.403233328943
+Minimizing 18619    Time: 0:02:08 ( 6.92 ms/it)
+  objective:  13331.398361885542
+Minimizing 18634    Time: 0:02:08 ( 6.92 ms/it)
+  objective:  13331.387101725872
+Minimizing 18649    Time: 0:02:09 ( 6.92 ms/it)
+  objective:  13331.38005361161
+Minimizing 18664    Time: 0:02:09 ( 6.92 ms/it)
+  objective:  13331.326018027437
+Minimizing 18679    Time: 0:02:09 ( 6.92 ms/it)
+  objective:  13331.04874836732
+Minimizing 18694    Time: 0:02:09 ( 6.92 ms/it)
+  objective:  13331.016200164442
+Minimizing 18709    Time: 0:02:09 ( 6.92 ms/it)
+  objective:  13331.014856694055
+Minimizing 18724    Time: 0:02:09 ( 6.92 ms/it)
+  objective:  13331.000092066279
+Minimizing 18739    Time: 0:02:09 ( 6.92 ms/it)
+  objective:  13330.98174820219
+Minimizing 18754    Time: 0:02:09 ( 6.92 ms/it)
+  objective:  13330.923246875565
+Minimizing 18769    Time: 0:02:09 ( 6.92 ms/it)
+  objective:  13331.057292039288
+Minimizing 18784    Time: 0:02:09 ( 6.92 ms/it)
+  objective:  13330.820249156663
+Minimizing 18799    Time: 0:02:10 ( 6.92 ms/it)
+  objective:  13330.819560777869
+Minimizing 18814    Time: 0:02:10 ( 6.92 ms/it)
+  objective:  13330.81742589306
+Minimizing 18829    Time: 0:02:10 ( 6.92 ms/it)
+  objective:  13330.813268829435
+Minimizing 18844    Time: 0:02:10 ( 6.92 ms/it)
+  objective:  13330.809176184055
+Minimizing 18859    Time: 0:02:10 ( 6.92 ms/it)
+  objective:  13330.780886687935
+Minimizing 18874    Time: 0:02:10 ( 6.92 ms/it)
+  objective:  13330.735498194335
+Minimizing 18889    Time: 0:02:10 ( 6.92 ms/it)
+  objective:  13330.717466289127
+Minimizing 18904    Time: 0:02:10 ( 6.92 ms/it)
+  objective:  13330.671095346988
+Minimizing 18919    Time: 0:02:10 ( 6.92 ms/it)
+  objective:  13330.658830652581
+Minimizing 18934    Time: 0:02:10 ( 6.92 ms/it)
+  objective:  13330.646016764695
+Minimizing 18949    Time: 0:02:11 ( 6.92 ms/it)
+  objective:  13330.629924788125
+Minimizing 18964    Time: 0:02:11 ( 6.92 ms/it)
+  objective:  13330.575998203167
+Minimizing 18979    Time: 0:02:11 ( 6.92 ms/it)
+  objective:  13330.5184689651
+Minimizing 18994    Time: 0:02:11 ( 6.92 ms/it)
+  objective:  13330.51788424806
+Minimizing 19009    Time: 0:02:11 ( 6.92 ms/it)
+  objective:  13330.51448597193
+Minimizing 19024    Time: 0:02:11 ( 6.92 ms/it)
+  objective:  13330.509704314725
+Minimizing 19039    Time: 0:02:11 ( 6.92 ms/it)
+  objective:  13330.503051528925
+Minimizing 19054    Time: 0:02:11 ( 6.92 ms/it)
+  objective:  13330.479735019435
+Minimizing 19069    Time: 0:02:11 ( 6.92 ms/it)
+  objective:  13330.46355300842
+Minimizing 19084    Time: 0:02:12 ( 6.92 ms/it)
+  objective:  13330.432544731171
+Minimizing 19099    Time: 0:02:12 ( 6.92 ms/it)
+  objective:  13330.422039944147
+Minimizing 19114    Time: 0:02:12 ( 6.92 ms/it)
+  objective:  13330.358435354035
+Minimizing 19129    Time: 0:02:12 ( 6.92 ms/it)
+  objective:  13330.335687806124
+Minimizing 19144    Time: 0:02:12 ( 6.92 ms/it)
+  objective:  13330.334375663617
+Minimizing 19159    Time: 0:02:12 ( 6.92 ms/it)
+  objective:  13330.322470435196
+Minimizing 19174    Time: 0:02:12 ( 6.92 ms/it)
+  objective:  13330.277306498145
+Minimizing 19189    Time: 0:02:12 ( 6.92 ms/it)
+  objective:  13330.270544234161
+Minimizing 19204    Time: 0:02:12 ( 6.92 ms/it)
+  objective:  13330.209340675552
+Minimizing 19219    Time: 0:02:12 ( 6.92 ms/it)
+  objective:  13330.205449058165
+Minimizing 19234    Time: 0:02:13 ( 6.92 ms/it)
+  objective:  13330.199347486465
+Minimizing 19249    Time: 0:02:13 ( 6.92 ms/it)
+  objective:  13330.186147581022
+Minimizing 19264    Time: 0:02:13 ( 6.92 ms/it)
+  objective:  13330.15832003655
+Minimizing 19279    Time: 0:02:13 ( 6.92 ms/it)
+  objective:  13330.074843961767
+Minimizing 19294    Time: 0:02:13 ( 6.92 ms/it)
+  objective:  13330.02981807918
+Minimizing 19309    Time: 0:02:13 ( 6.92 ms/it)
+  objective:  13330.02634915684
+Minimizing 19324    Time: 0:02:13 ( 6.92 ms/it)
+  objective:  13329.999798483856
+Minimizing 19339    Time: 0:02:13 ( 6.92 ms/it)
+  objective:  13329.969057325463
+Minimizing 19354    Time: 0:02:13 ( 6.92 ms/it)
+  objective:  13329.963734783938
+Minimizing 19369    Time: 0:02:13 ( 6.92 ms/it)
+  objective:  13329.95293151198
+Minimizing 19384    Time: 0:02:14 ( 6.92 ms/it)
+  objective:  13329.933692490544
+Minimizing 19399    Time: 0:02:14 ( 6.92 ms/it)
+  objective:  13329.91544806432
+Minimizing 19414    Time: 0:02:14 ( 6.92 ms/it)
+  objective:  13329.897226474255
+Minimizing 19429    Time: 0:02:14 ( 6.92 ms/it)
+  objective:  13329.833230712793
+Minimizing 19444    Time: 0:02:14 ( 6.92 ms/it)
+  objective:  13329.83249528308
+Minimizing 19459    Time: 0:02:14 ( 6.92 ms/it)
+  objective:  13329.828962245141
+Minimizing 19474    Time: 0:02:14 ( 6.91 ms/it)
+  objective:  13329.825982439135
+Minimizing 19489    Time: 0:02:14 ( 6.91 ms/it)
+  objective:  13329.821616552857
+Minimizing 19504    Time: 0:02:14 ( 6.91 ms/it)
+  objective:  13329.814144659802
+Minimizing 19519    Time: 0:02:14 ( 6.91 ms/it)
+  objective:  13329.79126935944
+Minimizing 19534    Time: 0:02:15 ( 6.91 ms/it)
+  objective:  13329.789456750936
+Minimizing 19549    Time: 0:02:15 ( 6.91 ms/it)
+  objective:  13329.755567503045
+Minimizing 19564    Time: 0:02:15 ( 6.91 ms/it)
+  objective:  13329.75065324807
+Minimizing 19579    Time: 0:02:15 ( 6.91 ms/it)
+  objective:  13329.74617634451
+Minimizing 19594    Time: 0:02:15 ( 6.91 ms/it)
+  objective:  13329.729968831707
+Minimizing 19609    Time: 0:02:15 ( 6.91 ms/it)
+  objective:  13329.718980641403
+Minimizing 19624    Time: 0:02:15 ( 6.91 ms/it)
+  objective:  13329.652253531873
+Minimizing 19639    Time: 0:02:15 ( 6.91 ms/it)
+  objective:  13329.650557134548
+Minimizing 19654    Time: 0:02:15 ( 6.91 ms/it)
+  objective:  13329.647233110918
+Minimizing 19669    Time: 0:02:15 ( 6.91 ms/it)
+  objective:  13329.637596124892
+Minimizing 19684    Time: 0:02:16 ( 6.91 ms/it)
+  objective:  13329.629141556885
+Minimizing 19699    Time: 0:02:16 ( 6.91 ms/it)
+  objective:  13329.597380038285
+Minimizing 19714    Time: 0:02:16 ( 6.91 ms/it)
+  objective:  13329.590077110573
+Minimizing 19729    Time: 0:02:16 ( 6.91 ms/it)
+  objective:  13329.550919230649
+Minimizing 19744    Time: 0:02:16 ( 6.91 ms/it)
+  objective:  13329.53728489844
+Minimizing 19759    Time: 0:02:16 ( 6.91 ms/it)
+  objective:  13329.528018004312
+Minimizing 19774    Time: 0:02:16 ( 6.91 ms/it)
+  objective:  13329.492447316414
+Minimizing 19789    Time: 0:02:16 ( 6.91 ms/it)
+  objective:  13329.468808185818
+Minimizing 19804    Time: 0:02:16 ( 6.91 ms/it)
+  objective:  13329.46771172135
+Minimizing 19819    Time: 0:02:17 ( 6.91 ms/it)
+  objective:  13329.465938575828
+Minimizing 19834    Time: 0:02:17 ( 6.91 ms/it)
+  objective:  13329.459824371843
+Minimizing 19849    Time: 0:02:17 ( 6.91 ms/it)
+  objective:  13329.455256816233
+Minimizing 19864    Time: 0:02:17 ( 6.91 ms/it)
+  objective:  13329.448280938886
+Minimizing 19879    Time: 0:02:17 ( 6.91 ms/it)
+  objective:  13329.4346785796
+Minimizing 19894    Time: 0:02:17 ( 6.91 ms/it)
+  objective:  13329.425251753863
+Minimizing 19909    Time: 0:02:17 ( 6.91 ms/it)
+  objective:  13329.351055840372
+Minimizing 19924    Time: 0:02:17 ( 6.91 ms/it)
+  objective:  13329.347318027976
+Minimizing 19939    Time: 0:02:17 ( 6.91 ms/it)
+  objective:  13329.346121603194
+Minimizing 19954    Time: 0:02:17 ( 6.91 ms/it)
+  objective:  13329.342177374921
+Minimizing 19969    Time: 0:02:18 ( 6.91 ms/it)
+  objective:  13329.33260046925
+Minimizing 19984    Time: 0:02:18 ( 6.91 ms/it)
+  objective:  13329.32460494536
+Minimizing 19999    Time: 0:02:18 ( 6.91 ms/it)
+  objective:  13329.289010897519
+Minimizing 20014    Time: 0:02:18 ( 6.91 ms/it)
+  objective:  13329.263827779709
+Minimizing 20029    Time: 0:02:18 ( 6.91 ms/it)
+  objective:  13329.262276728165
+Minimizing 20044    Time: 0:02:18 ( 6.91 ms/it)
+  objective:  13329.250618800848
+Minimizing 20059    Time: 0:02:18 ( 6.91 ms/it)
+  objective:  13329.247834330512
+Minimizing 20074    Time: 0:02:18 ( 6.91 ms/it)
+  objective:  13329.204974296998
+Minimizing 20089    Time: 0:02:18 ( 6.91 ms/it)
+  objective:  13329.16436753214
+Minimizing 20104    Time: 0:02:18 ( 6.91 ms/it)
+  objective:  13329.1626961406
+Minimizing 20119    Time: 0:02:19 ( 6.91 ms/it)
+  objective:  13329.154209735047
+Minimizing 20134    Time: 0:02:19 ( 6.91 ms/it)
+  objective:  13329.143880958145
+Minimizing 20149    Time: 0:02:19 ( 6.91 ms/it)
+  objective:  13329.117962488679
+Minimizing 20164    Time: 0:02:19 ( 6.91 ms/it)
+  objective:  13329.10457216049
+Minimizing 20179    Time: 0:02:19 ( 6.91 ms/it)
+  objective:  13329.099038291482
+Minimizing 20194    Time: 0:02:19 ( 6.91 ms/it)
+  objective:  13329.072185730416
+Minimizing 20209    Time: 0:02:19 ( 6.91 ms/it)
+  objective:  13329.071166924543
+Minimizing 20224    Time: 0:02:19 ( 6.91 ms/it)
+  objective:  13329.055917316553
+Minimizing 20239    Time: 0:02:19 ( 6.91 ms/it)
+  objective:  13329.04550733111
+Minimizing 20254    Time: 0:02:19 ( 6.91 ms/it)
+  objective:  13328.962482216775
+Minimizing 20269    Time: 0:02:20 ( 6.91 ms/it)
+  objective:  13328.927748833332
+Minimizing 20284    Time: 0:02:20 ( 6.91 ms/it)
+  objective:  13328.926812721613
+Minimizing 20299    Time: 0:02:20 ( 6.91 ms/it)
+  objective:  13328.919377647391
+Minimizing 20314    Time: 0:02:20 ( 6.91 ms/it)
+  objective:  13328.913483455151
+Minimizing 20329    Time: 0:02:20 ( 6.91 ms/it)
+  objective:  13328.909083503539
+Minimizing 20344    Time: 0:02:20 ( 6.91 ms/it)
+  objective:  13328.90125764248
+Minimizing 20359    Time: 0:02:20 ( 6.91 ms/it)
+  objective:  13328.891911836923
+Minimizing 20374    Time: 0:02:20 ( 6.91 ms/it)
+  objective:  13328.882679207527
+Minimizing 20389    Time: 0:02:20 ( 6.91 ms/it)
+  objective:  13328.831126148958
+Minimizing 20404    Time: 0:02:21 ( 6.91 ms/it)
+  objective:  13328.816507312069
+Minimizing 20419    Time: 0:02:21 ( 6.91 ms/it)
+  objective:  13328.815969157455
+Minimizing 20434    Time: 0:02:21 ( 6.91 ms/it)
+  objective:  13328.815103480607
+Minimizing 20449    Time: 0:02:21 ( 6.91 ms/it)
+  objective:  13328.807968008216
+Minimizing 20464    Time: 0:02:21 ( 6.91 ms/it)
+  objective:  13328.79801758843
+Minimizing 20479    Time: 0:02:21 ( 6.91 ms/it)
+  objective:  13328.79601185149
+Minimizing 20494    Time: 0:02:21 ( 6.91 ms/it)
+  objective:  13328.788488880979
+Minimizing 20509    Time: 0:02:21 ( 6.91 ms/it)
+  objective:  13328.782002255859
+Minimizing 20524    Time: 0:02:21 ( 6.91 ms/it)
+  objective:  13328.751616276677
+Minimizing 20539    Time: 0:02:21 ( 6.91 ms/it)
+  objective:  13328.745351114805
+Minimizing 20554    Time: 0:02:22 ( 6.91 ms/it)
+  objective:  13328.743074135215
+Minimizing 20569    Time: 0:02:22 ( 6.91 ms/it)
+  objective:  13328.723902436861
+Minimizing 20584    Time: 0:02:22 ( 6.91 ms/it)
+  objective:  13328.719389493199
+Minimizing 20598    Time: 0:02:22 ( 6.91 ms/it)
+  objective:  13328.702692766936
+Minimizing 20611    Time: 0:02:22 ( 6.91 ms/it)
+  objective:  13328.69678127634
+Minimizing 20624    Time: 0:02:22 ( 6.91 ms/it)
+  objective:  13328.661492028768
+Minimizing 20637    Time: 0:02:22 ( 6.91 ms/it)
+  objective:  13328.645504165099
+Minimizing 20651    Time: 0:02:22 ( 6.91 ms/it)
+  objective:  13328.6131297545
+Minimizing 20665    Time: 0:02:22 ( 6.91 ms/it)
+  objective:  13328.59599418984
+Minimizing 20679    Time: 0:02:22 ( 6.91 ms/it)
+  objective:  13328.563284711126
+Minimizing 20693    Time: 0:02:23 ( 6.91 ms/it)
+  objective:  13328.563004483498
+Minimizing 20709    Time: 0:02:23 ( 6.91 ms/it)
+  objective:  13328.561069517382
+Minimizing 20725    Time: 0:02:23 ( 6.91 ms/it)
+  objective:  13328.555234422049
+Minimizing 20741    Time: 0:02:23 ( 6.91 ms/it)
+  objective:  13328.549043014864
+Minimizing 20757    Time: 0:02:23 ( 6.91 ms/it)
+  objective:  13328.540497406568
+Minimizing 20773    Time: 0:02:23 ( 6.91 ms/it)
+  objective:  13328.540005980147
+Minimizing 20788    Time: 0:02:23 ( 6.91 ms/it)
+  objective:  13328.536864956477
+Minimizing 20804    Time: 0:02:23 ( 6.91 ms/it)
+  objective:  13328.531331802318
+Minimizing 20820    Time: 0:02:23 ( 6.91 ms/it)
+  objective:  13328.513318566002
+Minimizing 20836    Time: 0:02:24 ( 6.91 ms/it)
+  objective:  13328.50008276588
+Minimizing 20852    Time: 0:02:24 ( 6.91 ms/it)
+  objective:  13328.496851011798
+Minimizing 20868    Time: 0:02:24 ( 6.91 ms/it)
+  objective:  13328.484168365874
+Minimizing 20884    Time: 0:02:24 ( 6.91 ms/it)
+  objective:  13328.48124605757
+Minimizing 20900    Time: 0:02:24 ( 6.91 ms/it)
+  objective:  13328.46839925462
+Minimizing 20916    Time: 0:02:24 ( 6.91 ms/it)
+  objective:  13328.467590104665
+Minimizing 20931    Time: 0:02:24 ( 6.91 ms/it)
+  objective:  13328.461319288479
+Minimizing 20947    Time: 0:02:24 ( 6.91 ms/it)
+  objective:  13328.453933112985
+Minimizing 20963    Time: 0:02:24 ( 6.91 ms/it)
+  objective:  13328.43419432555
+Minimizing 20979    Time: 0:02:24 ( 6.91 ms/it)
+  objective:  13328.432500284369
+Minimizing 20995    Time: 0:02:25 ( 6.91 ms/it)
+  objective:  13328.421113762604
+Minimizing 21011    Time: 0:02:25 ( 6.91 ms/it)
+  objective:  13328.38416793786
+Minimizing 21027    Time: 0:02:25 ( 6.91 ms/it)
+  objective:  13328.379851898833
+Minimizing 21043    Time: 0:02:25 ( 6.91 ms/it)
+  objective:  13328.376751316799
+Minimizing 21059    Time: 0:02:25 ( 6.91 ms/it)
+  objective:  13328.369806187402
+Minimizing 21075    Time: 0:02:25 ( 6.91 ms/it)
+  objective:  13328.362042835848
+Minimizing 21090    Time: 0:02:25 ( 6.91 ms/it)
+  objective:  13328.343381699531
+Minimizing 21105    Time: 0:02:25 ( 6.91 ms/it)
+  objective:  13328.337823394271
+Minimizing 21120    Time: 0:02:25 ( 6.91 ms/it)
+  objective:  13328.321451358046
+Minimizing 21136    Time: 0:02:26 ( 6.91 ms/it)
+  objective:  13328.312059720367
+Minimizing 21152    Time: 0:02:26 ( 6.91 ms/it)
+  objective:  13328.30609299626
+Minimizing 21168    Time: 0:02:26 ( 6.91 ms/it)
+  objective:  13328.28875842451
+Minimizing 21184    Time: 0:02:26 ( 6.91 ms/it)
+  objective:  13328.267972218964
+Minimizing 21200    Time: 0:02:26 ( 6.91 ms/it)
+  objective:  13328.267036784477
+Minimizing 21216    Time: 0:02:26 ( 6.91 ms/it)
+  objective:  13328.26497987088
+Minimizing 21231    Time: 0:02:26 ( 6.91 ms/it)
+  objective:  13328.262630383411
+Minimizing 21247    Time: 0:02:26 ( 6.91 ms/it)
+  objective:  13328.25573451454
+Minimizing 21263    Time: 0:02:26 ( 6.91 ms/it)
+  objective:  13328.23286972697
+Minimizing 21279    Time: 0:02:26 ( 6.91 ms/it)
+  objective:  13328.231315208788
+Minimizing 21295    Time: 0:02:27 ( 6.91 ms/it)
+  objective:  13328.224954260731
+Minimizing 21311    Time: 0:02:27 ( 6.91 ms/it)
+  objective:  13328.217138625194
+Minimizing 21327    Time: 0:02:27 ( 6.91 ms/it)
+  objective:  13328.187494608996
+Minimizing 21343    Time: 0:02:27 ( 6.91 ms/it)
+  objective:  13328.12351370039
+Minimizing 21359    Time: 0:02:27 ( 6.91 ms/it)
+  objective:  13328.055052723168
+Minimizing 21375    Time: 0:02:27 ( 6.90 ms/it)
+  objective:  13328.030370397268
+Minimizing 21390    Time: 0:02:27 ( 6.90 ms/it)
+  objective:  13328.026212190089
+Minimizing 21406    Time: 0:02:27 ( 6.90 ms/it)
+  objective:  13328.018676974956
+Minimizing 21422    Time: 0:02:27 ( 6.90 ms/it)
+  objective:  13328.015866306698
+Minimizing 21438    Time: 0:02:28 ( 6.90 ms/it)
+  objective:  13328.006198655297
+Minimizing 21454    Time: 0:02:28 ( 6.90 ms/it)
+  objective:  13327.99892970141
+Minimizing 21470    Time: 0:02:28 ( 6.90 ms/it)
+  objective:  13327.928966481551
+Minimizing 21486    Time: 0:02:28 ( 6.90 ms/it)
+  objective:  13328.02470796887
+Minimizing 21501    Time: 0:02:28 ( 6.90 ms/it)
+  objective:  13327.801979061704
+Minimizing 21517    Time: 0:02:28 ( 6.90 ms/it)
+  objective:  13327.801556733619
+Minimizing 21532    Time: 0:02:28 ( 6.90 ms/it)
+  objective:  13327.799584313805
+Minimizing 21547    Time: 0:02:28 ( 6.90 ms/it)
+  objective:  13327.796538176422
+Minimizing 21562    Time: 0:02:28 ( 6.90 ms/it)
+  objective:  13327.793278801822
+Minimizing 21577    Time: 0:02:28 ( 6.90 ms/it)
+  objective:  13327.782332322291
+Minimizing 21592    Time: 0:02:29 ( 6.90 ms/it)
+  objective:  13327.764839668736
+Minimizing 21608    Time: 0:02:29 ( 6.90 ms/it)
+  objective:  13327.756326261602
+Minimizing 21624    Time: 0:02:29 ( 6.90 ms/it)
+  objective:  13327.741892600308
+Minimizing 21639    Time: 0:02:29 ( 6.90 ms/it)
+  objective:  13327.729415401547
+Minimizing 21655    Time: 0:02:29 ( 6.90 ms/it)
+  objective:  13327.690183491577
+Minimizing 21669    Time: 0:02:29 ( 6.90 ms/it)
+  objective:  13327.684329612544
+Minimizing 21684    Time: 0:02:29 ( 6.90 ms/it)
+  objective:  13327.683795853838
+Minimizing 21699    Time: 0:02:29 ( 6.90 ms/it)
+  objective:  13327.680709614477
+Minimizing 21714    Time: 0:02:29 ( 6.90 ms/it)
+  objective:  13327.676415991162
+Minimizing 21729    Time: 0:02:29 ( 6.90 ms/it)
+  objective:  13327.67130588476
+Minimizing 21744    Time: 0:02:30 ( 6.90 ms/it)
+  objective:  13327.662230510286
+Minimizing 21759    Time: 0:02:30 ( 6.90 ms/it)
+  objective:  13327.603528149026
+Minimizing 21774    Time: 0:02:30 ( 6.90 ms/it)
+  objective:  13329.111544503598
+Minimizing 21789    Time: 0:02:30 ( 6.90 ms/it)
+  objective:  13327.484090964128
+Minimizing 21804    Time: 0:02:30 ( 6.90 ms/it)
+  objective:  13327.482741733402
+Minimizing 21818    Time: 0:02:30 ( 6.90 ms/it)
+  objective:  13327.47717121819
+Minimizing 21833    Time: 0:02:30 ( 6.90 ms/it)
+  objective:  13327.467571572735
+Minimizing 21848    Time: 0:02:30 ( 6.90 ms/it)
+  objective:  13327.4628520002
+Minimizing 21863    Time: 0:02:30 ( 6.90 ms/it)
+  objective:  13327.452499485793
+Minimizing 21878    Time: 0:02:31 ( 6.90 ms/it)
+  objective:  13327.446835663191
+Minimizing 21893    Time: 0:02:31 ( 6.90 ms/it)
+  objective:  13327.445871786542
+Minimizing 21908    Time: 0:02:31 ( 6.90 ms/it)
+  objective:  13327.434015690771
+Minimizing 21924    Time: 0:02:31 ( 6.90 ms/it)
+  objective:  13327.443386763123
+Minimizing 21939    Time: 0:02:31 ( 6.90 ms/it)
+  objective:  13327.423161290528
+Minimizing 21954    Time: 0:02:31 ( 6.90 ms/it)
+  objective:  13327.414213814496
+Minimizing 21969    Time: 0:02:31 ( 6.90 ms/it)
+  objective:  13327.38938667193
+Minimizing 21984    Time: 0:02:31 ( 6.90 ms/it)
+  objective:  13327.32583775944
+Minimizing 21999    Time: 0:02:31 ( 6.90 ms/it)
+  objective:  13327.262107619375
+Minimizing 22014    Time: 0:02:31 ( 6.90 ms/it)
+  objective:  13327.259904104038
+Minimizing 22029    Time: 0:02:32 ( 6.90 ms/it)
+  objective:  13327.25594936443
+Minimizing 22044    Time: 0:02:32 ( 6.90 ms/it)
+  objective:  13327.23970314509
+Minimizing 22059    Time: 0:02:32 ( 6.90 ms/it)
+  objective:  13327.232849535932
+Minimizing 22074    Time: 0:02:32 ( 6.90 ms/it)
+  objective:  13327.223380963369
+Minimizing 22089    Time: 0:02:32 ( 6.90 ms/it)
+  objective:  13327.222790139938
+Minimizing 22104    Time: 0:02:32 ( 6.90 ms/it)
+  objective:  13327.205473040536
+Minimizing 22119    Time: 0:02:32 ( 6.90 ms/it)
+  objective:  13327.199450099346
+Minimizing 22134    Time: 0:02:32 ( 6.90 ms/it)
+  objective:  13327.190917415996
+Minimizing 22150    Time: 0:02:32 ( 6.90 ms/it)
+  objective:  13327.122476330384
+Minimizing 22165    Time: 0:02:32 ( 6.90 ms/it)
+  objective:  13328.560754599028
+Minimizing 22180    Time: 0:02:33 ( 6.90 ms/it)
+  objective:  13326.982377612716
+Minimizing 22195    Time: 0:02:33 ( 6.90 ms/it)
+  objective:  13326.980520266443
+Minimizing 22210    Time: 0:02:33 ( 6.90 ms/it)
+  objective:  13326.976321790004
+Minimizing 22226    Time: 0:02:33 ( 6.90 ms/it)
+  objective:  13326.973612000358
+Minimizing 22241    Time: 0:02:33 ( 6.90 ms/it)
+  objective:  13326.973099495845
+Minimizing 22256    Time: 0:02:33 ( 6.90 ms/it)
+  objective:  13326.966781852469
+Minimizing 22271    Time: 0:02:33 ( 6.90 ms/it)
+  objective:  13326.960174615451
+Minimizing 22286    Time: 0:02:33 ( 6.90 ms/it)
+  objective:  13326.957446704444
+Minimizing 22301    Time: 0:02:33 ( 6.90 ms/it)
+  objective:  13326.903577389181
+Minimizing 22317    Time: 0:02:34 ( 6.90 ms/it)
+  objective:  13326.81188356712
+Minimizing 22332    Time: 0:02:34 ( 6.90 ms/it)
+  objective:  13326.80533651678
+Minimizing 22347    Time: 0:02:34 ( 6.90 ms/it)
+  objective:  13326.80114071528
+Minimizing 22362    Time: 0:02:34 ( 6.90 ms/it)
+  objective:  13326.79290784846
+Minimizing 22378    Time: 0:02:34 ( 6.90 ms/it)
+  objective:  13326.786212197694
+Minimizing 22393    Time: 0:02:34 ( 6.90 ms/it)
+  objective:  13326.78560582918
+Minimizing 22408    Time: 0:02:34 ( 6.90 ms/it)
+  objective:  13326.78352692166
+Minimizing 22423    Time: 0:02:34 ( 6.90 ms/it)
+  objective:  13326.771890382966
+Minimizing 22439    Time: 0:02:34 ( 6.90 ms/it)
+  objective:  13326.766326823505
+Minimizing 22455    Time: 0:02:34 ( 6.90 ms/it)
+  objective:  13326.72886697483
+Minimizing 22470    Time: 0:02:35 ( 6.90 ms/it)
+  objective:  13326.691905235355
+Minimizing 22486    Time: 0:02:35 ( 6.90 ms/it)
+  objective:  13326.691417169044
+Minimizing 22501    Time: 0:02:35 ( 6.90 ms/it)
+  objective:  13326.689990353269
+Minimizing 22516    Time: 0:02:35 ( 6.90 ms/it)
+  objective:  13326.688596868997
+Minimizing 22531    Time: 0:02:35 ( 6.90 ms/it)
+  objective:  13326.68589525792
+Minimizing 22546    Time: 0:02:35 ( 6.90 ms/it)
+  objective:  13326.679548932058
+Minimizing 22561    Time: 0:02:35 ( 6.90 ms/it)
+  objective:  13326.665983564177
+Minimizing 22576    Time: 0:02:35 ( 6.90 ms/it)
+  objective:  13326.629279067965
+Minimizing 22592    Time: 0:02:35 ( 6.90 ms/it)
+  objective:  13326.628672901024
+Minimizing 22608    Time: 0:02:35 ( 6.90 ms/it)
+  objective:  13326.615305702268
+Minimizing 22624    Time: 0:02:36 ( 6.90 ms/it)
+  objective:  13326.605019161463
+Minimizing 22640    Time: 0:02:36 ( 6.90 ms/it)
+  objective:  13326.598885612599
+Minimizing 22656    Time: 0:02:36 ( 6.90 ms/it)
+  objective:  13326.586264002006
+Minimizing 22672    Time: 0:02:36 ( 6.90 ms/it)
+  objective:  13326.585417005328
+Minimizing 22688    Time: 0:02:36 ( 6.90 ms/it)
+  objective:  13326.5773219443
+Minimizing 22704    Time: 0:02:36 ( 6.90 ms/it)
+  objective:  13326.564808780757
+Minimizing 22720    Time: 0:02:36 ( 6.90 ms/it)
+  objective:  13326.539768572613
+Minimizing 22736    Time: 0:02:36 ( 6.90 ms/it)
+  objective:  13326.494433528118
+Minimizing 22752    Time: 0:02:36 ( 6.90 ms/it)
+  objective:  13326.49289796767
+Minimizing 22768    Time: 0:02:37 ( 6.90 ms/it)
+  objective:  13326.479554432073
+Minimizing 22784    Time: 0:02:37 ( 6.90 ms/it)
+  objective:  13326.473930317916
+Minimizing 22800    Time: 0:02:37 ( 6.90 ms/it)
+  objective:  13326.47062744174
+Minimizing 22816    Time: 0:02:37 ( 6.90 ms/it)
+  objective:  13326.429113892009
+Minimizing 22832    Time: 0:02:37 ( 6.89 ms/it)
+  objective:  13326.389299300594
+Minimizing 22848    Time: 0:02:37 ( 6.89 ms/it)
+  objective:  13326.388486062191
+Minimizing 22864    Time: 0:02:37 ( 6.89 ms/it)
+  objective:  13326.382510892407
+Minimizing 22880    Time: 0:02:37 ( 6.89 ms/it)
+  objective:  13326.360334032463
+Minimizing 22896    Time: 0:02:37 ( 6.89 ms/it)
+  objective:  13326.251126782692
+Minimizing 22912    Time: 0:02:37 ( 6.89 ms/it)
+  objective:  13326.201596900799
+Minimizing 22928    Time: 0:02:38 ( 6.89 ms/it)
+  objective:  13326.200756263825
+Minimizing 22944    Time: 0:02:38 ( 6.89 ms/it)
+  objective:  13326.186645147784
+Minimizing 22960    Time: 0:02:38 ( 6.89 ms/it)
+  objective:  13326.16572093666
+Minimizing 22976    Time: 0:02:38 ( 6.89 ms/it)
+  objective:  13326.13902118634
+Minimizing 22992    Time: 0:02:38 ( 6.89 ms/it)
+  objective:  13325.999315058587
+Minimizing 23008    Time: 0:02:38 ( 6.89 ms/it)
+  objective:  13325.975564473425
+Minimizing 23024    Time: 0:02:38 ( 6.89 ms/it)
+  objective:  13325.972831541738
+Minimizing 23040    Time: 0:02:38 ( 6.89 ms/it)
+  objective:  13325.967454374375
+Minimizing 23056    Time: 0:02:38 ( 6.89 ms/it)
+  objective:  13325.963359604662
+Minimizing 23072    Time: 0:02:38 ( 6.89 ms/it)
+  objective:  13325.95130167601
+Minimizing 23088    Time: 0:02:39 ( 6.89 ms/it)
+  objective:  13325.943465455792
+Minimizing 23104    Time: 0:02:39 ( 6.89 ms/it)
+  objective:  13325.933078365735
+Minimizing 23120    Time: 0:02:39 ( 6.89 ms/it)
+  objective:  13325.919712834038
+Minimizing 23136    Time: 0:02:39 ( 6.89 ms/it)
+  objective:  13325.899783913053
+Minimizing 23152    Time: 0:02:39 ( 6.89 ms/it)
+  objective:  13325.898596879335
+Minimizing 23167    Time: 0:02:39 ( 6.89 ms/it)
+  objective:  13325.894608529423
+Minimizing 23181    Time: 0:02:39 ( 6.89 ms/it)
+  objective:  13325.891204203785
+Minimizing 23195    Time: 0:02:39 ( 6.89 ms/it)
+  objective:  13325.889488856781
+Minimizing 23209    Time: 0:02:39 ( 6.89 ms/it)
+  objective:  13325.883675640216
+Minimizing 23224    Time: 0:02:40 ( 6.89 ms/it)
+  objective:  13325.866046063544
+Minimizing 23238    Time: 0:02:40 ( 6.89 ms/it)
+  objective:  13325.85696938257
+Minimizing 23252    Time: 0:02:40 ( 6.89 ms/it)
+  objective:  13325.84344341918
+Minimizing 23266    Time: 0:02:40 ( 6.89 ms/it)
+  objective:  13325.834944784154
+Minimizing 23280    Time: 0:02:40 ( 6.89 ms/it)
+  objective:  13325.816188973069
+Minimizing 23294    Time: 0:02:40 ( 6.89 ms/it)
+  objective:  13325.815294583466
+Minimizing 23308    Time: 0:02:40 ( 6.89 ms/it)
+  objective:  13325.801841052642
+Minimizing 23322    Time: 0:02:40 ( 6.89 ms/it)
+  objective:  13325.778420452581
+Minimizing 23336    Time: 0:02:40 ( 6.89 ms/it)
+  objective:  13325.720044661815
+Minimizing 23350    Time: 0:02:40 ( 6.89 ms/it)
+  objective:  13325.710355581548
+Minimizing 23364    Time: 0:02:41 ( 6.89 ms/it)
+  objective:  13325.709689893927
+Minimizing 23379    Time: 0:02:41 ( 6.89 ms/it)
+  objective:  13325.708151664323
+Minimizing 23394    Time: 0:02:41 ( 6.89 ms/it)
+  objective:  13325.703164592858
+Minimizing 23409    Time: 0:02:41 ( 6.89 ms/it)
+  objective:  13325.701900485277
+Minimizing 23425    Time: 0:02:41 ( 6.89 ms/it)
+  objective:  13325.693808845848
+Minimizing 23441    Time: 0:02:41 ( 6.89 ms/it)
+  objective:  13325.68981808627
+Minimizing 23457    Time: 0:02:41 ( 6.89 ms/it)
+  objective:  13325.671538165429
+Minimizing 23473    Time: 0:02:41 ( 6.89 ms/it)
+  objective:  13325.664247808396
+Minimizing 23489    Time: 0:02:41 ( 6.89 ms/it)
+  objective:  13325.651794773388
+Minimizing 23505    Time: 0:02:41 ( 6.89 ms/it)
+  objective:  13325.642637510537
+Minimizing 23521    Time: 0:02:42 ( 6.89 ms/it)
+  objective:  13325.619599476951
+Minimizing 23537    Time: 0:02:42 ( 6.89 ms/it)
+  objective:  13325.616775359442
+Minimizing 23552    Time: 0:02:42 ( 6.89 ms/it)
+  objective:  13325.612685194785
+Minimizing 23568    Time: 0:02:42 ( 6.89 ms/it)
+  objective:  13325.608067382876
+Minimizing 23584    Time: 0:02:42 ( 6.89 ms/it)
+  objective:  13325.595928968884
+Minimizing 23600    Time: 0:02:42 ( 6.89 ms/it)
+  objective:  13325.591879340587
+Minimizing 23616    Time: 0:02:42 ( 6.89 ms/it)
+  objective:  13325.573347906087
+Minimizing 23632    Time: 0:02:42 ( 6.89 ms/it)
+  objective:  13325.569694765603
+Minimizing 23648    Time: 0:02:42 ( 6.89 ms/it)
+  objective:  13325.538788847232
+Minimizing 23664    Time: 0:02:43 ( 6.89 ms/it)
+  objective:  13325.52459089207
+Minimizing 23680    Time: 0:02:43 ( 6.89 ms/it)
+  objective:  13326.438067200346
+Minimizing 23696    Time: 0:02:43 ( 6.89 ms/it)
+  objective:  13325.374923427014
+Minimizing 23712    Time: 0:02:43 ( 6.89 ms/it)
+  objective:  13325.373184006072
+Minimizing 23728    Time: 0:02:43 ( 6.89 ms/it)
+  objective:  13325.368453568684
+Minimizing 23744    Time: 0:02:43 ( 6.89 ms/it)
+  objective:  13325.353329633705
+Minimizing 23760    Time: 0:02:43 ( 6.89 ms/it)
+  objective:  13325.347739125566
+Minimizing 23776    Time: 0:02:43 ( 6.89 ms/it)
+  objective:  13325.347940115164
+Minimizing 23791    Time: 0:02:43 ( 6.89 ms/it)
+  objective:  13325.32932654817
+Minimizing 23807    Time: 0:02:43 ( 6.89 ms/it)
+  objective:  13325.325268700886
+Minimizing 23823    Time: 0:02:44 ( 6.89 ms/it)
+  objective:  13325.242215933562
+Minimizing 23839    Time: 0:02:44 ( 6.89 ms/it)
+  objective:  13325.172006022054
+Minimizing 23854    Time: 0:02:44 ( 6.89 ms/it)
+  objective:  13325.171226626553
+Minimizing 23868    Time: 0:02:44 ( 6.89 ms/it)
+  objective:  13325.156228543521
+Minimizing 23882    Time: 0:02:44 ( 6.89 ms/it)
+  objective:  13325.128125648043
+Minimizing 23896    Time: 0:02:44 ( 6.89 ms/it)
+  objective:  13325.10150062821
+Minimizing 23910    Time: 0:02:44 ( 6.89 ms/it)
+  objective:  13325.096286907108
+Minimizing 23924    Time: 0:02:44 ( 6.89 ms/it)
+  objective:  13325.095722644866
+Minimizing 23938    Time: 0:02:44 ( 6.89 ms/it)
+  objective:  13325.09481461077
+Minimizing 23954    Time: 0:02:44 ( 6.89 ms/it)
+  objective:  13325.094347071587
+Minimizing 23970    Time: 0:02:45 ( 6.89 ms/it)
+  objective:  13325.089907856367
+Minimizing 23986    Time: 0:02:45 ( 6.89 ms/it)
+  objective:  13325.087706015067
+Minimizing 24002    Time: 0:02:45 ( 6.89 ms/it)
+  objective:  13325.073322889613
+Minimizing 24018    Time: 0:02:45 ( 6.89 ms/it)
+  objective:  13325.071698878019
+Minimizing 24034    Time: 0:02:45 ( 6.89 ms/it)
+  objective:  13325.069605354234
+Minimizing 24050    Time: 0:02:45 ( 6.89 ms/it)
+  objective:  13325.06731050594
+Minimizing 24066    Time: 0:02:45 ( 6.89 ms/it)
+  objective:  13325.031029600403
+Minimizing 24082    Time: 0:02:45 ( 6.89 ms/it)
+  objective:  13324.928921728992
+Minimizing 24098    Time: 0:02:45 ( 6.89 ms/it)
+  objective:  13324.903449190679
+Minimizing 24114    Time: 0:02:46 ( 6.88 ms/it)
+  objective:  13324.902986358444
+Minimizing 24130    Time: 0:02:46 ( 6.88 ms/it)
+  objective:  13324.902310571691
+Minimizing 24146    Time: 0:02:46 ( 6.88 ms/it)
+  objective:  13324.9020104558
+Minimizing 24162    Time: 0:02:46 ( 6.88 ms/it)
+  objective:  13324.900693161544
+Minimizing 24178    Time: 0:02:46 ( 6.88 ms/it)
+  objective:  13324.900124771419
+Minimizing 24194    Time: 0:02:46 ( 6.88 ms/it)
+  objective:  13324.888367541382
+Minimizing 24210    Time: 0:02:46 ( 6.88 ms/it)
+  objective:  13324.88766143292
+Minimizing 24226    Time: 0:02:46 ( 6.88 ms/it)
+  objective:  13324.886939615433
+Minimizing 24241    Time: 0:02:46 ( 6.88 ms/it)
+  objective:  13324.884405500881
+Minimizing 24255    Time: 0:02:46 ( 6.88 ms/it)
+  objective:  13324.882491166427
+Minimizing 24271    Time: 0:02:47 ( 6.88 ms/it)
+  objective:  13324.873052820709
+Minimizing 24287    Time: 0:02:47 ( 6.88 ms/it)
+  objective:  13324.872226935026
+Minimizing 24303    Time: 0:02:47 ( 6.88 ms/it)
+  objective:  13324.86797736722
+Minimizing 24319    Time: 0:02:47 ( 6.88 ms/it)
+  objective:  13324.84477803143
+Minimizing 24335    Time: 0:02:47 ( 6.88 ms/it)
+  objective:  13324.841977485383
+Minimizing 24351    Time: 0:02:47 ( 6.88 ms/it)
+  objective:  13324.841054861448
+Minimizing 24367    Time: 0:02:47 ( 6.88 ms/it)
+  objective:  13324.839558815904
+Minimizing 24382    Time: 0:02:47 ( 6.88 ms/it)
+  objective:  13324.832945818256
+Minimizing 24398    Time: 0:02:47 ( 6.88 ms/it)
+  objective:  13324.829589407134
+Minimizing 24414    Time: 0:02:48 ( 6.88 ms/it)
+  objective:  13324.825581579164
+Minimizing 24430    Time: 0:02:48 ( 6.88 ms/it)
+  objective:  13324.817998542305
+Minimizing 24446    Time: 0:02:48 ( 6.88 ms/it)
+  objective:  13324.816962923098
+Minimizing 24462    Time: 0:02:48 ( 6.88 ms/it)
+  objective:  13324.803258516695
+Minimizing 24478    Time: 0:02:48 ( 6.88 ms/it)
+  objective:  13324.80214209207
+Minimizing 24494    Time: 0:02:48 ( 6.88 ms/it)
+  objective:  13324.797411330379
+Minimizing 24510    Time: 0:02:48 ( 6.88 ms/it)
+  objective:  13324.790132120514
+Minimizing 24526    Time: 0:02:48 ( 6.88 ms/it)
+  objective:  13324.766449680392
+Minimizing 24542    Time: 0:02:48 ( 6.88 ms/it)
+  objective:  13324.76292447331
+Minimizing 24558    Time: 0:02:48 ( 6.88 ms/it)
+  objective:  13324.678182131509
+Minimizing 24574    Time: 0:02:49 ( 6.88 ms/it)
+  objective:  13324.16256173722
+Minimizing 24590    Time: 0:02:49 ( 6.88 ms/it)
+  objective:  13324.151142159579
+Minimizing 24606    Time: 0:02:49 ( 6.88 ms/it)
+  objective:  13324.147462384019
+Minimizing 24622    Time: 0:02:49 ( 6.88 ms/it)
+  objective:  13324.143824523184
+Minimizing 24638    Time: 0:02:49 ( 6.88 ms/it)
+  objective:  13324.13157006503
+Minimizing 24654    Time: 0:02:49 ( 6.88 ms/it)
+  objective:  13324.127718992924
+Minimizing 24670    Time: 0:02:49 ( 6.88 ms/it)
+  objective:  13324.108382862512
+Minimizing 24686    Time: 0:02:49 ( 6.88 ms/it)
+  objective:  13324.03378389457
+Minimizing 24702    Time: 0:02:49 ( 6.88 ms/it)
+  objective:  13324.013521233443
+Minimizing 24718    Time: 0:02:49 ( 6.88 ms/it)
+  objective:  13324.012666784824
+Minimizing 24733    Time: 0:02:50 ( 6.88 ms/it)
+  objective:  13324.000040468163
+Minimizing 24749    Time: 0:02:50 ( 6.88 ms/it)
+  objective:  13323.968423096347
+Minimizing 24765    Time: 0:02:50 ( 6.88 ms/it)
+  objective:  13323.953850897597
+Minimizing 24781    Time: 0:02:50 ( 6.88 ms/it)
+  objective:  13323.94573838498
+Minimizing 24797    Time: 0:02:50 ( 6.88 ms/it)
+  objective:  13323.931988113167
+Minimizing 24813    Time: 0:02:50 ( 6.88 ms/it)
+  objective:  13323.9140349089
+Minimizing 24829    Time: 0:02:50 ( 6.88 ms/it)
+  objective:  13323.908955066028
+Minimizing 24845    Time: 0:02:50 ( 6.88 ms/it)
+  objective:  13323.89419766872
+Minimizing 24861    Time: 0:02:50 ( 6.88 ms/it)
+  objective:  13323.852324141655
+Minimizing 24877    Time: 0:02:51 ( 6.88 ms/it)
+  objective:  13323.848371638538
+Minimizing 24893    Time: 0:02:51 ( 6.87 ms/it)
+  objective:  13323.840699705455
+Minimizing 24909    Time: 0:02:51 ( 6.87 ms/it)
+  objective:  13323.81877841182
+Minimizing 24924    Time: 0:02:51 ( 6.87 ms/it)
+  objective:  13323.817457392302
+Minimizing 24940    Time: 0:02:51 ( 6.87 ms/it)
+  objective:  13323.837986796425
+Minimizing 24956    Time: 0:02:51 ( 6.87 ms/it)
+  objective:  13323.79314238258
+Minimizing 24972    Time: 0:02:51 ( 6.87 ms/it)
+  objective:  13323.786818467648
+Minimizing 24988    Time: 0:02:51 ( 6.87 ms/it)
+  objective:  13323.766236201918
+Minimizing 25004    Time: 0:02:51 ( 6.87 ms/it)
+  objective:  13323.750464486337
+Minimizing 25020    Time: 0:02:51 ( 6.87 ms/it)
+  objective:  13323.747747567686
+Minimizing 25036    Time: 0:02:52 ( 6.87 ms/it)
+  objective:  13323.743292526895
+Minimizing 25052    Time: 0:02:52 ( 6.87 ms/it)
+  objective:  13323.713804753104
+Minimizing 25068    Time: 0:02:52 ( 6.87 ms/it)
+  objective:  13323.712822744434
+Minimizing 25084    Time: 0:02:52 ( 6.87 ms/it)
+  objective:  13323.70937624514
+Minimizing 25100    Time: 0:02:52 ( 6.87 ms/it)
+  objective:  13323.704561479113
+Minimizing 25116    Time: 0:02:52 ( 6.87 ms/it)
+  objective:  13323.691608221881
+Minimizing 25132    Time: 0:02:52 ( 6.87 ms/it)
+  objective:  13323.67334602181
+Minimizing 25148    Time: 0:02:52 ( 6.87 ms/it)
+  objective:  13323.659798414825
+Minimizing 25164    Time: 0:02:52 ( 6.87 ms/it)
+  objective:  13323.65746454343
+Minimizing 25180    Time: 0:02:53 ( 6.87 ms/it)
+  objective:  13323.640948931832
+Minimizing 25196    Time: 0:02:53 ( 6.87 ms/it)
+  objective:  13323.638226269642
+Minimizing 25212    Time: 0:02:53 ( 6.87 ms/it)
+  objective:  13323.628734145575
+Minimizing 25228    Time: 0:02:53 ( 6.87 ms/it)
+  objective:  13323.597906984185
+Minimizing 25244    Time: 0:02:53 ( 6.87 ms/it)
+  objective:  13323.587541658359
+Minimizing 25260    Time: 0:02:53 ( 6.87 ms/it)
+  objective:  13323.554590172716
+Minimizing 25276    Time: 0:02:53 ( 6.87 ms/it)
+  objective:  13323.434175322996
+Minimizing 25292    Time: 0:02:53 ( 6.87 ms/it)
+  objective:  13323.433494755533
+Minimizing 25307    Time: 0:02:53 ( 6.87 ms/it)
+  objective:  13323.428571804805
+Minimizing 25323    Time: 0:02:53 ( 6.87 ms/it)
+  objective:  13323.422066582134
+Minimizing 25339    Time: 0:02:54 ( 6.87 ms/it)
+  objective:  13323.413393725787
+Minimizing 25354    Time: 0:02:54 ( 6.87 ms/it)
+  objective:  13323.407697837014
+Minimizing 25369    Time: 0:02:54 ( 6.87 ms/it)
+  objective:  13323.407231690624
+Minimizing 25384    Time: 0:02:54 ( 6.87 ms/it)
+  objective:  13323.4066585603
+Minimizing 25399    Time: 0:02:54 ( 6.87 ms/it)
+  objective:  13323.40371533185
+Minimizing 25414    Time: 0:02:54 ( 6.87 ms/it)
+  objective:  13323.39458685054
+Minimizing 25429    Time: 0:02:54 ( 6.87 ms/it)
+  objective:  13323.393447367955
+Minimizing 25444    Time: 0:02:54 ( 6.87 ms/it)
+  objective:  13323.38865229694
+Minimizing 25459    Time: 0:02:54 ( 6.87 ms/it)
+  objective:  13323.377278377913
+Minimizing 25474    Time: 0:02:54 ( 6.87 ms/it)
+  objective:  13323.372870266161
+Minimizing 25489    Time: 0:02:55 ( 6.87 ms/it)
+  objective:  13323.338608801394
+Minimizing 25504    Time: 0:02:55 ( 6.87 ms/it)
+  objective:  13323.331808491392
+Minimizing 25519    Time: 0:02:55 ( 6.87 ms/it)
+  objective:  13323.32543553997
+Minimizing 25534    Time: 0:02:55 ( 6.87 ms/it)
+  objective:  13323.308513585755
+Minimizing 25549    Time: 0:02:55 ( 6.87 ms/it)
+  objective:  13323.304795191798
+Minimizing 25564    Time: 0:02:55 ( 6.87 ms/it)
+  objective:  13323.301294638586
+Minimizing 25579    Time: 0:02:55 ( 6.87 ms/it)
+  objective:  13323.290065557652
+Minimizing 25594    Time: 0:02:55 ( 6.87 ms/it)
+  objective:  13323.256049413176
+Minimizing 25609    Time: 0:02:55 ( 6.87 ms/it)
+  objective:  13323.253375282322
+Minimizing 25624    Time: 0:02:55 ( 6.87 ms/it)
+  objective:  13323.252761078387
+Minimizing 25639    Time: 0:02:56 ( 6.87 ms/it)
+  objective:  13323.2490163912
+Minimizing 25654    Time: 0:02:56 ( 6.87 ms/it)
+  objective:  13323.238773807418
+Minimizing 25669    Time: 0:02:56 ( 6.87 ms/it)
+  objective:  13323.226849554223
+Minimizing 25684    Time: 0:02:56 ( 6.87 ms/it)
+  objective:  13323.20930974609
+Minimizing 25699    Time: 0:02:56 ( 6.87 ms/it)
+  objective:  13323.2056473704
+Minimizing 25714    Time: 0:02:56 ( 6.87 ms/it)
+  objective:  13323.178228027202
+Minimizing 25729    Time: 0:02:56 ( 6.87 ms/it)
+  objective:  13323.13293205484
+Minimizing 25744    Time: 0:02:56 ( 6.87 ms/it)
+  objective:  13323.075127984703
+Minimizing 25759    Time: 0:02:56 ( 6.87 ms/it)
+  objective:  13323.074458831194
+Minimizing 25774    Time: 0:02:57 ( 6.87 ms/it)
+  objective:  13323.068142148768
+Minimizing 25789    Time: 0:02:57 ( 6.87 ms/it)
+  objective:  13323.057414702387
+Minimizing 25804    Time: 0:02:57 ( 6.87 ms/it)
+  objective:  13323.045625411774
+Minimizing 25819    Time: 0:02:57 ( 6.87 ms/it)
+  objective:  13323.03783585751
+Minimizing 25834    Time: 0:02:57 ( 6.87 ms/it)
+  objective:  13323.018657660461
+Minimizing 25849    Time: 0:02:57 ( 6.87 ms/it)
+  objective:  13323.003403881725
+Minimizing 25864    Time: 0:02:57 ( 6.87 ms/it)
+  objective:  13322.99810934886
+Minimizing 25879    Time: 0:02:57 ( 6.87 ms/it)
+  objective:  13322.9729513809
+Minimizing 25894    Time: 0:02:57 ( 6.87 ms/it)
+  objective:  13322.938968131013
+Minimizing 25909    Time: 0:02:57 ( 6.87 ms/it)
+  objective:  13322.93806398478
+Minimizing 25924    Time: 0:02:58 ( 6.87 ms/it)
+  objective:  13322.929529983026
+Minimizing 25939    Time: 0:02:58 ( 6.87 ms/it)
+  objective:  13322.918040594188
+Minimizing 25954    Time: 0:02:58 ( 6.87 ms/it)
+  objective:  13322.898878752792
+Minimizing 25969    Time: 0:02:58 ( 6.87 ms/it)
+  objective:  13322.888402011275
+Minimizing 25984    Time: 0:02:58 ( 6.87 ms/it)
+  objective:  13322.872896282599
+Minimizing 25999    Time: 0:02:58 ( 6.87 ms/it)
+  objective:  13322.830113556178
+Minimizing 26014    Time: 0:02:58 ( 6.87 ms/it)
+  objective:  13322.829677535032
+Minimizing 26029    Time: 0:02:58 ( 6.87 ms/it)
+  objective:  13322.828304435447
+Minimizing 26044    Time: 0:02:58 ( 6.87 ms/it)
+  objective:  13322.822400007892
+Minimizing 26059    Time: 0:02:58 ( 6.87 ms/it)
+  objective:  13322.817119775675
+Minimizing 26074    Time: 0:02:59 ( 6.87 ms/it)
+  objective:  13322.807389417154
+Minimizing 26089    Time: 0:02:59 ( 6.87 ms/it)
+  objective:  13322.803204736949
+Minimizing 26104    Time: 0:02:59 ( 6.87 ms/it)
+  objective:  13322.785861942204
+Minimizing 26119    Time: 0:02:59 ( 6.87 ms/it)
+  objective:  13322.692722944514
+Minimizing 26134    Time: 0:02:59 ( 6.87 ms/it)
+  objective:  13322.609752601988
+Minimizing 26149    Time: 0:02:59 ( 6.87 ms/it)
+  objective:  13322.603516203802
+Minimizing 26164    Time: 0:02:59 ( 6.87 ms/it)
+  objective:  13322.591255335486
+Minimizing 26179    Time: 0:02:59 ( 6.87 ms/it)
+  objective:  13322.586202882914
+Minimizing 26194    Time: 0:02:59 ( 6.87 ms/it)
+  objective:  13322.594188662246
+Minimizing 26209    Time: 0:02:59 ( 6.87 ms/it)
+  objective:  13322.565544025274
+Minimizing 26224    Time: 0:03:00 ( 6.87 ms/it)
+  objective:  13322.55949812493
+Minimizing 26239    Time: 0:03:00 ( 6.87 ms/it)
+  objective:  13322.406509473818
+Minimizing 26254    Time: 0:03:00 ( 6.87 ms/it)
+  objective:  13322.395590189743
+Minimizing 26269    Time: 0:03:00 ( 6.87 ms/it)
+  objective:  13322.393859894291
+Minimizing 26284    Time: 0:03:00 ( 6.87 ms/it)
+  objective:  13322.385943464134
+Minimizing 26299    Time: 0:03:00 ( 6.87 ms/it)
+  objective:  13322.369477663582
+Minimizing 26314    Time: 0:03:00 ( 6.87 ms/it)
+  objective:  13322.34905602473
+Minimizing 26329    Time: 0:03:00 ( 6.87 ms/it)
+  objective:  13322.333796735838
+Minimizing 26344    Time: 0:03:00 ( 6.87 ms/it)
+  objective:  13322.332139578954
+Minimizing 26359    Time: 0:03:01 ( 6.87 ms/it)
+  objective:  13322.331663799996
+Minimizing 26374    Time: 0:03:01 ( 6.87 ms/it)
+  objective:  13322.329607107575
+Minimizing 26389    Time: 0:03:01 ( 6.87 ms/it)
+  objective:  13322.32342997234
+Minimizing 26404    Time: 0:03:01 ( 6.87 ms/it)
+  objective:  13322.319395621336
+Minimizing 26419    Time: 0:03:01 ( 6.87 ms/it)
+  objective:  13322.301532050958
+Minimizing 26434    Time: 0:03:01 ( 6.87 ms/it)
+  objective:  13322.30090986214
+Minimizing 26449    Time: 0:03:01 ( 6.87 ms/it)
+  objective:  13322.300128561488
+Minimizing 26464    Time: 0:03:01 ( 6.87 ms/it)
+  objective:  13322.298473328701
+Minimizing 26479    Time: 0:03:01 ( 6.87 ms/it)
+  objective:  13322.293163996146
+Minimizing 26494    Time: 0:03:01 ( 6.87 ms/it)
+  objective:  13322.290279489287
+Minimizing 26509    Time: 0:03:02 ( 6.87 ms/it)
+  objective:  13322.285388918506
+Minimizing 26524    Time: 0:03:02 ( 6.87 ms/it)
+  objective:  13322.282342203762
+Minimizing 26539    Time: 0:03:02 ( 6.87 ms/it)
+  objective:  13322.255137888918
+Minimizing 26554    Time: 0:03:02 ( 6.87 ms/it)
+  objective:  13322.233192796892
+Minimizing 26569    Time: 0:03:02 ( 6.87 ms/it)
+  objective:  13322.209166855464
+Minimizing 26584    Time: 0:03:02 ( 6.87 ms/it)
+  objective:  13322.207201735087
+Minimizing 26599    Time: 0:03:02 ( 6.87 ms/it)
+  objective:  13322.203106752553
+Minimizing 26614    Time: 0:03:02 ( 6.87 ms/it)
+  objective:  13322.196700367946
+Minimizing 26629    Time: 0:03:02 ( 6.87 ms/it)
+  objective:  13322.195682090081
+Minimizing 26644    Time: 0:03:02 ( 6.87 ms/it)
+  objective:  13322.181781145337
+Minimizing 26659    Time: 0:03:03 ( 6.87 ms/it)
+  objective:  13322.17987567914
+Minimizing 26674    Time: 0:03:03 ( 6.87 ms/it)
+  objective:  13322.173563138465
+Minimizing 26689    Time: 0:03:03 ( 6.87 ms/it)
+  objective:  13322.147485170193
+Minimizing 26704    Time: 0:03:03 ( 6.87 ms/it)
+  objective:  13322.141737244005
+Minimizing 26719    Time: 0:03:03 ( 6.87 ms/it)
+  objective:  13322.141230367415
+Minimizing 26734    Time: 0:03:03 ( 6.87 ms/it)
+  objective:  13322.140126766579
+Minimizing 26749    Time: 0:03:03 ( 6.87 ms/it)
+  objective:  13322.138709305771
+Minimizing 26764    Time: 0:03:03 ( 6.87 ms/it)
+  objective:  13322.138082907855
+Minimizing 26779    Time: 0:03:03 ( 6.87 ms/it)
+  objective:  13322.132185935756
+Minimizing 26794    Time: 0:03:03 ( 6.87 ms/it)
+  objective:  13322.120911144331
+Minimizing 26809    Time: 0:03:04 ( 6.87 ms/it)
+  objective:  13322.113991364298
+Minimizing 26824    Time: 0:03:04 ( 6.87 ms/it)
+  objective:  13322.113387134857
+Minimizing 26839    Time: 0:03:04 ( 6.87 ms/it)
+  objective:  13322.11046375033
+Minimizing 26854    Time: 0:03:04 ( 6.87 ms/it)
+  objective:  13322.099336957108
+Minimizing 26869    Time: 0:03:04 ( 6.87 ms/it)
+  objective:  13322.084524258113
+Minimizing 26884    Time: 0:03:04 ( 6.87 ms/it)
+  objective:  13322.063839652139
+Minimizing 26899    Time: 0:03:04 ( 6.87 ms/it)
+  objective:  13322.0155499369
+Minimizing 26914    Time: 0:03:04 ( 6.87 ms/it)
+  objective:  13322.000558433385
+Minimizing 26929    Time: 0:03:04 ( 6.87 ms/it)
+  objective:  13321.997768479632
+Minimizing 26944    Time: 0:03:04 ( 6.87 ms/it)
+  objective:  13321.989875511877
+Minimizing 26959    Time: 0:03:05 ( 6.87 ms/it)
+  objective:  13321.984484698885
+Minimizing 26974    Time: 0:03:05 ( 6.87 ms/it)
+  objective:  13321.982594697605
+Minimizing 26989    Time: 0:03:05 ( 6.87 ms/it)
+  objective:  13321.974760900397
+Minimizing 27004    Time: 0:03:05 ( 6.87 ms/it)
+  objective:  13321.941009857852
+Minimizing 27019    Time: 0:03:05 ( 6.87 ms/it)
+  objective:  13321.806900048294
+Minimizing 27034    Time: 0:03:05 ( 6.87 ms/it)
+  objective:  13321.422455843116
+Minimizing 27049    Time: 0:03:05 ( 6.87 ms/it)
+  objective:  13320.268549029715
+Minimizing 27064    Time: 0:03:05 ( 6.87 ms/it)
+  objective:  13320.255488383424
+Minimizing 27079    Time: 0:03:05 ( 6.87 ms/it)
+  objective:  13320.250728623287
+Minimizing 27094    Time: 0:03:06 ( 6.87 ms/it)
+  objective:  13320.24796584739
+Minimizing 27109    Time: 0:03:06 ( 6.87 ms/it)
+  objective:  13320.2452907572
+Minimizing 27124    Time: 0:03:06 ( 6.87 ms/it)
+  objective:  13320.223813572185
+Minimizing 27139    Time: 0:03:06 ( 6.87 ms/it)
+  objective:  13320.208108329185
+Minimizing 27154    Time: 0:03:06 ( 6.87 ms/it)
+  objective:  13320.148346046684
+Minimizing 27169    Time: 0:03:06 ( 6.87 ms/it)
+  objective:  13320.052495868484
+Minimizing 27184    Time: 0:03:06 ( 6.87 ms/it)
+  objective:  13320.021999924502
+Minimizing 27199    Time: 0:03:06 ( 6.87 ms/it)
+  objective:  13320.021497332142
+Minimizing 27214    Time: 0:03:06 ( 6.87 ms/it)
+  objective:  13320.016425152353
+Minimizing 27229    Time: 0:03:06 ( 6.87 ms/it)
+  objective:  13320.012925402028
+Minimizing 27244    Time: 0:03:07 ( 6.87 ms/it)
+  objective:  13320.007580268633
+Minimizing 27259    Time: 0:03:07 ( 6.87 ms/it)
+  objective:  13319.99838856989
+Minimizing 27274    Time: 0:03:07 ( 6.87 ms/it)
+  objective:  13319.994593361116
+Minimizing 27289    Time: 0:03:07 ( 6.87 ms/it)
+  objective:  13319.98299152295
+Minimizing 27304    Time: 0:03:07 ( 6.87 ms/it)
+  objective:  13319.971892012793
+Minimizing 27319    Time: 0:03:07 ( 6.87 ms/it)
+  objective:  13319.953655871344
+Minimizing 27334    Time: 0:03:07 ( 6.87 ms/it)
+  objective:  13319.947623654865
+Minimizing 27349    Time: 0:03:07 ( 6.87 ms/it)
+  objective:  13319.938840763323
+Minimizing 27364    Time: 0:03:07 ( 6.87 ms/it)
+  objective:  13319.931863788632
+Minimizing 27379    Time: 0:03:07 ( 6.87 ms/it)
+  objective:  13319.907144755925
+Minimizing 27394    Time: 0:03:08 ( 6.87 ms/it)
+  objective:  13319.904510039138
+Minimizing 27409    Time: 0:03:08 ( 6.87 ms/it)
+  objective:  13319.848195948405
+Minimizing 27424    Time: 0:03:08 ( 6.87 ms/it)
+  objective:  13319.846388926875
+Minimizing 27439    Time: 0:03:08 ( 6.86 ms/it)
+  objective:  13319.845897955325
+Minimizing 27454    Time: 0:03:08 ( 6.86 ms/it)
+  objective:  13319.845384788816
+Minimizing 27469    Time: 0:03:08 ( 6.86 ms/it)
+  objective:  13319.845057090832
+Minimizing 27484    Time: 0:03:08 ( 6.86 ms/it)
+  objective:  13319.844095454348
+Minimizing 27499    Time: 0:03:08 ( 6.86 ms/it)
+  objective:  13319.840902387266
+Minimizing 27514    Time: 0:03:08 ( 6.86 ms/it)
+  objective:  13319.834684381596
+Minimizing 27529    Time: 0:03:08 ( 6.86 ms/it)
+  objective:  13319.82131133374
+Minimizing 27544    Time: 0:03:09 ( 6.86 ms/it)
+  objective:  13319.789587220163
+Minimizing 27559    Time: 0:03:09 ( 6.86 ms/it)
+  objective:  13319.789443854082
+Minimizing 27574    Time: 0:03:09 ( 6.86 ms/it)
+  objective:  13319.789026905608
+Minimizing 27589    Time: 0:03:09 ( 6.86 ms/it)
+  objective:  13319.787153598852
+Minimizing 27604    Time: 0:03:09 ( 6.86 ms/it)
+  objective:  13319.78665749525
+Minimizing 27619    Time: 0:03:09 ( 6.86 ms/it)
+  objective:  13319.783850553227
+Minimizing 27634    Time: 0:03:09 ( 6.86 ms/it)
+  objective:  13319.773207589213
+Minimizing 27649    Time: 0:03:09 ( 6.86 ms/it)
+  objective:  13319.772689421778
+Minimizing 27664    Time: 0:03:09 ( 6.87 ms/it)
+  objective:  13319.77213598913
+Minimizing 27679    Time: 0:03:10 ( 6.87 ms/it)
+  objective:  13319.77019615729
+Minimizing 27694    Time: 0:03:10 ( 6.87 ms/it)
+  objective:  13319.7686089208
+Minimizing 27709    Time: 0:03:10 ( 6.87 ms/it)
+  objective:  13319.768075620057
+Minimizing 27724    Time: 0:03:10 ( 6.87 ms/it)
+  objective:  13319.766322738826
+Minimizing 27739    Time: 0:03:10 ( 6.87 ms/it)
+  objective:  13319.763331976617
+Minimizing 27754    Time: 0:03:10 ( 6.87 ms/it)
+  objective:  13319.735881350993
+Minimizing 27769    Time: 0:03:10 ( 6.87 ms/it)
+  objective:  13319.735561155161
+Minimizing 27784    Time: 0:03:10 ( 6.87 ms/it)
+  objective:  13319.734453965473
+Minimizing 27799    Time: 0:03:10 ( 6.87 ms/it)
+  objective:  13319.730423380228
+Minimizing 27814    Time: 0:03:10 ( 6.87 ms/it)
+  objective:  13319.728180403981
+Minimizing 27829    Time: 0:03:11 ( 6.86 ms/it)
+  objective:  13319.72781844296
+Minimizing 27844    Time: 0:03:11 ( 6.86 ms/it)
+  objective:  13319.727311298688
+Minimizing 27859    Time: 0:03:11 ( 6.86 ms/it)
+  objective:  13319.726190035217
+Minimizing 27874    Time: 0:03:11 ( 6.86 ms/it)
+  objective:  13319.71980743183
+Minimizing 27889    Time: 0:03:11 ( 6.86 ms/it)
+  objective:  13319.718782030264
+Minimizing 27904    Time: 0:03:11 ( 6.86 ms/it)
+  objective:  13319.710864137538
+Minimizing 27919    Time: 0:03:11 ( 6.86 ms/it)
+  objective:  13319.710562760447
+Minimizing 27934    Time: 0:03:11 ( 6.86 ms/it)
+  objective:  13319.705356261344
+Minimizing 27949    Time: 0:03:11 ( 6.86 ms/it)
+  objective:  13319.70447774035
+Minimizing 27964    Time: 0:03:11 ( 6.86 ms/it)
+  objective:  13319.701828833466
+Minimizing 27979    Time: 0:03:12 ( 6.86 ms/it)
+  objective:  13319.700727715084
+Minimizing 27994    Time: 0:03:12 ( 6.86 ms/it)
+  objective:  13319.699621296662
+Minimizing 28009    Time: 0:03:12 ( 6.86 ms/it)
+  objective:  13319.69867906229
+Minimizing 28024    Time: 0:03:12 ( 6.86 ms/it)
+  objective:  13319.696766442576
+Minimizing 28039    Time: 0:03:12 ( 6.86 ms/it)
+  objective:  13319.691624293875
+Minimizing 28054    Time: 0:03:12 ( 6.86 ms/it)
+  objective:  13319.690739939979
+Minimizing 28069    Time: 0:03:12 ( 6.86 ms/it)
+  objective:  13319.680722921854
+Minimizing 28084    Time: 0:03:12 ( 6.86 ms/it)
+  objective:  13319.677997205537
+Minimizing 28099    Time: 0:03:12 ( 6.86 ms/it)
+  objective:  13319.677811397996
+Minimizing 28114    Time: 0:03:12 ( 6.86 ms/it)
+  objective:  13319.67738327969
+Minimizing 28129    Time: 0:03:13 ( 6.86 ms/it)
+  objective:  13319.677226634798
+Minimizing 28144    Time: 0:03:13 ( 6.86 ms/it)
+  objective:  13319.676019415681
+Minimizing 28159    Time: 0:03:13 ( 6.86 ms/it)
+  objective:  13319.669088710667
+Minimizing 28174    Time: 0:03:13 ( 6.86 ms/it)
+  objective:  13319.65728714707
+Minimizing 28189    Time: 0:03:13 ( 6.86 ms/it)
+  objective:  13319.657030676506
+Minimizing 28204    Time: 0:03:13 ( 6.86 ms/it)
+  objective:  13319.655993959444
+Minimizing 28219    Time: 0:03:13 ( 6.86 ms/it)
+  objective:  13319.65466138946
+Minimizing 28234    Time: 0:03:13 ( 6.86 ms/it)
+  objective:  13319.654402434448
+Minimizing 28249    Time: 0:03:13 ( 6.86 ms/it)
+  objective:  13319.653831912918
+Minimizing 28264    Time: 0:03:14 ( 6.86 ms/it)
+  objective:  13319.653259204628
+Minimizing 28279    Time: 0:03:14 ( 6.86 ms/it)
+  objective:  13319.644774496875
+Minimizing 28294    Time: 0:03:14 ( 6.86 ms/it)
+  objective:  13319.642204099
+Minimizing 28309    Time: 0:03:14 ( 6.86 ms/it)
+  objective:  13319.641135271682
+Minimizing 28324    Time: 0:03:14 ( 6.86 ms/it)
+  objective:  13319.640681949008
+Minimizing 28339    Time: 0:03:14 ( 6.86 ms/it)
+  objective:  13319.638560322928
+Minimizing 28354    Time: 0:03:14 ( 6.86 ms/it)
+  objective:  13319.637176410644
+Minimizing 28369    Time: 0:03:14 ( 6.86 ms/it)
+  objective:  13319.63155167397
+Minimizing 28384    Time: 0:03:14 ( 6.86 ms/it)
+  objective:  13319.628818496829
+Minimizing 28399    Time: 0:03:14 ( 6.86 ms/it)
+  objective:  13319.627807899102
+Minimizing 28414    Time: 0:03:15 ( 6.86 ms/it)
+  objective:  13319.623492199447
+Minimizing 28429    Time: 0:03:15 ( 6.86 ms/it)
+  objective:  13319.623167782906
+Minimizing 28444    Time: 0:03:15 ( 6.86 ms/it)
+  objective:  13319.622348744932
+Minimizing 28459    Time: 0:03:15 ( 6.86 ms/it)
+  objective:  13319.618558170128
+Minimizing 28474    Time: 0:03:15 ( 6.86 ms/it)
+  objective:  13319.6178068476
+Minimizing 28489    Time: 0:03:15 ( 6.86 ms/it)
+  objective:  13319.614513968307
+Minimizing 28504    Time: 0:03:15 ( 6.86 ms/it)
+  objective:  13319.609692625687
+Minimizing 28519    Time: 0:03:15 ( 6.86 ms/it)
+  objective:  13319.608866740964
+Minimizing 28534    Time: 0:03:15 ( 6.86 ms/it)
+  objective:  13319.603108909461
+Minimizing 28549    Time: 0:03:15 ( 6.86 ms/it)
+  objective:  13319.601996628408
+Minimizing 28564    Time: 0:03:16 ( 6.86 ms/it)
+  objective:  13319.591364269916
+Minimizing 28579    Time: 0:03:16 ( 6.86 ms/it)
+  objective:  13319.59113128393
+Minimizing 28594    Time: 0:03:16 ( 6.86 ms/it)
+  objective:  13319.586932560574
+Minimizing 28609    Time: 0:03:16 ( 6.86 ms/it)
+  objective:  13319.577102816227
+Minimizing 28624    Time: 0:03:16 ( 6.86 ms/it)
+  objective:  13319.57699748849
+Minimizing 28639    Time: 0:03:16 ( 6.86 ms/it)
+  objective:  13319.573669211997
+Minimizing 28654    Time: 0:03:16 ( 6.86 ms/it)
+  objective:  13319.573296881368
+Minimizing 28669    Time: 0:03:16 ( 6.86 ms/it)
+  objective:  13319.569988628704
+Minimizing 28684    Time: 0:03:16 ( 6.86 ms/it)
+  objective:  13319.554054480686
+Minimizing 28699    Time: 0:03:16 ( 6.86 ms/it)
+  objective:  13319.541212009935
+Minimizing 28714    Time: 0:03:17 ( 6.86 ms/it)
+  objective:  13319.539904969075
+Minimizing 28729    Time: 0:03:17 ( 6.86 ms/it)
+  objective:  13319.534360820122
+Minimizing 28744    Time: 0:03:17 ( 6.86 ms/it)
+  objective:  13319.533288538762
+Minimizing 28759    Time: 0:03:17 ( 6.86 ms/it)
+  objective:  13319.517475092274
+Minimizing 28774    Time: 0:03:17 ( 6.86 ms/it)
+  objective:  13319.506605894669
+Minimizing 28789    Time: 0:03:17 ( 6.86 ms/it)
+  objective:  13319.505671887411
+Minimizing 28804    Time: 0:03:17 ( 6.86 ms/it)
+  objective:  13319.501485319357
+Minimizing 28819    Time: 0:03:17 ( 6.86 ms/it)
+  objective:  13319.495530651257
+Minimizing 28834    Time: 0:03:17 ( 6.86 ms/it)
+  objective:  13319.493524352743
+Minimizing 28849    Time: 0:03:17 ( 6.86 ms/it)
+  objective:  13319.4907841393
+Minimizing 28864    Time: 0:03:18 ( 6.86 ms/it)
+  objective:  13319.484216522906
+Minimizing 28879    Time: 0:03:18 ( 6.86 ms/it)
+  objective:  13319.478785074141
+Minimizing 28894    Time: 0:03:18 ( 6.86 ms/it)
+  objective:  13319.476925632305
+Minimizing 28909    Time: 0:03:18 ( 6.86 ms/it)
+  objective:  13319.461900662136
+Minimizing 28924    Time: 0:03:18 ( 6.86 ms/it)
+  objective:  13319.4614213553
+Minimizing 28939    Time: 0:03:18 ( 6.86 ms/it)
+  objective:  13319.459380531785
+Minimizing 28954    Time: 0:03:18 ( 6.86 ms/it)
+  objective:  13319.457299720787
+Minimizing 28969    Time: 0:03:18 ( 6.86 ms/it)
+  objective:  13319.456737670174
+Minimizing 28984    Time: 0:03:18 ( 6.86 ms/it)
+  objective:  13319.446059792332
+Minimizing 28999    Time: 0:03:18 ( 6.86 ms/it)
+  objective:  13319.445503213952
+Minimizing 29014    Time: 0:03:19 ( 6.86 ms/it)
+  objective:  13319.442146683781
+Minimizing 29029    Time: 0:03:19 ( 6.86 ms/it)
+  objective:  13319.415166991312
+Minimizing 29044    Time: 0:03:19 ( 6.86 ms/it)
+  objective:  13319.41487045007
+Minimizing 29059    Time: 0:03:19 ( 6.86 ms/it)
+  objective:  13319.414466591668
+Minimizing 29074    Time: 0:03:19 ( 6.86 ms/it)
+  objective:  13319.41299531865
+Minimizing 29089    Time: 0:03:19 ( 6.86 ms/it)
+  objective:  13319.410858698539
+Minimizing 29104    Time: 0:03:19 ( 6.86 ms/it)
+  objective:  13319.394128321961
+Minimizing 29119    Time: 0:03:19 ( 6.86 ms/it)
+  objective:  13319.392837391308
+Minimizing 29134    Time: 0:03:19 ( 6.86 ms/it)
+  objective:  13319.392651456335
+Minimizing 29149    Time: 0:03:20 ( 6.86 ms/it)
+  objective:  13319.392090191497
+Minimizing 29164    Time: 0:03:20 ( 6.86 ms/it)
+  objective:  13319.391447792324
+Minimizing 29179    Time: 0:03:20 ( 6.86 ms/it)
+  objective:  13319.390749635233
+Minimizing 29194    Time: 0:03:20 ( 6.86 ms/it)
+  objective:  13319.383641825247
+Minimizing 29209    Time: 0:03:20 ( 6.86 ms/it)
+  objective:  13319.382706084856
+Minimizing 29224    Time: 0:03:20 ( 6.86 ms/it)
+  objective:  13319.38233220455
+Minimizing 29239    Time: 0:03:20 ( 6.86 ms/it)
+  objective:  13319.381453434966
+Minimizing 29254    Time: 0:03:20 ( 6.86 ms/it)
+  objective:  13319.380384236909
+Minimizing 29269    Time: 0:03:20 ( 6.86 ms/it)
+  objective:  13319.37983905754
+Minimizing 29284    Time: 0:03:20 ( 6.86 ms/it)
+  objective:  13319.37318632497
+Minimizing 29299    Time: 0:03:21 ( 6.86 ms/it)
+  objective:  13319.360341063817
+Minimizing 29314    Time: 0:03:21 ( 6.86 ms/it)
+  objective:  13319.360157723713
+Minimizing 29329    Time: 0:03:21 ( 6.86 ms/it)
+  objective:  13319.359211586256
+Minimizing 29344    Time: 0:03:21 ( 6.86 ms/it)
+  objective:  13319.357355823944
+Minimizing 29359    Time: 0:03:21 ( 6.86 ms/it)
+  objective:  13319.356791910817
+Minimizing 29374    Time: 0:03:21 ( 6.86 ms/it)
+  objective:  13319.349634279424
+Minimizing 29389    Time: 0:03:21 ( 6.86 ms/it)
+  objective:  13319.33225182908
+Minimizing 29404    Time: 0:03:21 ( 6.86 ms/it)
+  objective:  13319.331679539988
+Minimizing 29419    Time: 0:03:21 ( 6.86 ms/it)
+  objective:  13319.330117062593
+Minimizing 29434    Time: 0:03:21 ( 6.86 ms/it)
+  objective:  13319.325558373574
+Minimizing 29449    Time: 0:03:22 ( 6.86 ms/it)
+  objective:  13319.32533681416
+Minimizing 29464    Time: 0:03:22 ( 6.86 ms/it)
+  objective:  13319.322487801095
+Minimizing 29479    Time: 0:03:22 ( 6.86 ms/it)
+  objective:  13319.32188202138
+Minimizing 29494    Time: 0:03:22 ( 6.86 ms/it)
+  objective:  13319.32115749891
+Minimizing 29509    Time: 0:03:22 ( 6.86 ms/it)
+  objective:  13319.319919370362
+Minimizing 29524    Time: 0:03:22 ( 6.86 ms/it)
+  objective:  13319.317433819335
+Minimizing 29539    Time: 0:03:22 ( 6.86 ms/it)
+  objective:  13319.303214779968
+Minimizing 29554    Time: 0:03:22 ( 6.86 ms/it)
+  objective:  13319.30049935331
+Minimizing 29569    Time: 0:03:22 ( 6.86 ms/it)
+  objective:  13319.300145254194
+Minimizing 29584    Time: 0:03:22 ( 6.86 ms/it)
+  objective:  13319.299859899693
+Minimizing 29599    Time: 0:03:23 ( 6.86 ms/it)
+  objective:  13319.299305651613
+Minimizing 29614    Time: 0:03:23 ( 6.86 ms/it)
+  objective:  13319.298650861601
+Minimizing 29629    Time: 0:03:23 ( 6.86 ms/it)
+  objective:  13319.307562207527
+Minimizing 29644    Time: 0:03:23 ( 6.86 ms/it)
+  objective:  13319.280197813845
+Minimizing 29659    Time: 0:03:23 ( 6.86 ms/it)
+  objective:  13319.280041465448
+Minimizing 29674    Time: 0:03:23 ( 6.86 ms/it)
+  objective:  13319.279121145533
+Minimizing 29689    Time: 0:03:23 ( 6.86 ms/it)
+  objective:  13319.277906575444
+Minimizing 29704    Time: 0:03:23 ( 6.86 ms/it)
+  objective:  13319.27648061463
+Minimizing 29719    Time: 0:03:23 ( 6.86 ms/it)
+  objective:  13319.27572350124
+Minimizing 29734    Time: 0:03:24 ( 6.86 ms/it)
+  objective:  13319.266880042254
+Minimizing 29749    Time: 0:03:24 ( 6.86 ms/it)
+  objective:  13319.266561918645
+Minimizing 29764    Time: 0:03:24 ( 6.86 ms/it)
+  objective:  13319.265186968245
+Minimizing 29779    Time: 0:03:24 ( 6.86 ms/it)
+  objective:  13319.25982279767
+Minimizing 29794    Time: 0:03:24 ( 6.86 ms/it)
+  objective:  13319.252946444889
+Minimizing 29809    Time: 0:03:24 ( 6.86 ms/it)
+  objective:  13319.25265336885
+Minimizing 29824    Time: 0:03:24 ( 6.86 ms/it)
+  objective:  13319.25234013448
+Minimizing 29839    Time: 0:03:24 ( 6.86 ms/it)
+  objective:  13319.25199416552
+Minimizing 29854    Time: 0:03:24 ( 6.86 ms/it)
+  objective:  13319.248944443927
+Minimizing 29869    Time: 0:03:24 ( 6.86 ms/it)
+  objective:  13319.246634433468
+Minimizing 29884    Time: 0:03:25 ( 6.86 ms/it)
+  objective:  13319.245844556339
+Minimizing 29899    Time: 0:03:25 ( 6.86 ms/it)
+  objective:  13319.24041514429
+Minimizing 29914    Time: 0:03:25 ( 6.86 ms/it)
+  objective:  13319.238714900013
+Minimizing 29929    Time: 0:03:25 ( 6.86 ms/it)
+  objective:  13319.238333718167
+Minimizing 29944    Time: 0:03:25 ( 6.86 ms/it)
+  objective:  13319.237901613335
+Minimizing 29959    Time: 0:03:25 ( 6.86 ms/it)
+  objective:  13319.237569940946
+Minimizing 29974    Time: 0:03:25 ( 6.86 ms/it)
+  objective:  13319.235634601806
+Minimizing 29989    Time: 0:03:25 ( 6.86 ms/it)
+  objective:  13319.228652360864
+Minimizing 30004    Time: 0:03:25 ( 6.86 ms/it)
+  objective:  13319.228561906508
+Minimizing 30019    Time: 0:03:25 ( 6.86 ms/it)
+  objective:  13319.228046899982
+Minimizing 30034    Time: 0:03:26 ( 6.86 ms/it)
+  objective:  13319.22710673805
+Minimizing 30049    Time: 0:03:26 ( 6.86 ms/it)
+  objective:  13319.225931763169
+Minimizing 30064    Time: 0:03:26 ( 6.86 ms/it)
+  objective:  13319.224867787387
+Minimizing 30079    Time: 0:03:26 ( 6.86 ms/it)
+  objective:  13319.22414478629
+Minimizing 30094    Time: 0:03:26 ( 6.86 ms/it)
+  objective:  13319.220894244238
+Minimizing 30109    Time: 0:03:26 ( 6.86 ms/it)
+  objective:  13319.223722022303
+Minimizing 30124    Time: 0:03:26 ( 6.86 ms/it)
+  objective:  13319.175184773034
+Minimizing 30139    Time: 0:03:26 ( 6.86 ms/it)
+  objective:  13319.174862641274
+Minimizing 30154    Time: 0:03:26 ( 6.86 ms/it)
+  objective:  13319.173213871778
+Minimizing 30169    Time: 0:03:26 ( 6.86 ms/it)
+  objective:  13319.17276937126
+Minimizing 30184    Time: 0:03:27 ( 6.86 ms/it)
+  objective:  13319.171252959379
+Minimizing 30199    Time: 0:03:27 ( 6.86 ms/it)
+  objective:  13319.166585182014
+Minimizing 30214    Time: 0:03:27 ( 6.86 ms/it)
+  objective:  13319.158402624322
+Minimizing 30229    Time: 0:03:27 ( 6.86 ms/it)
+  objective:  13319.147387968056
+Minimizing 30244    Time: 0:03:27 ( 6.86 ms/it)
+  objective:  13319.113353657434
+Minimizing 30259    Time: 0:03:27 ( 6.86 ms/it)
+  objective:  13319.087326319117
+Minimizing 30274    Time: 0:03:27 ( 6.86 ms/it)
+  objective:  13319.086572416621
+Minimizing 30289    Time: 0:03:27 ( 6.86 ms/it)
+  objective:  13319.083536452148
+Minimizing 30304    Time: 0:03:27 ( 6.86 ms/it)
+  objective:  13319.08128510078
+Minimizing 30319    Time: 0:03:27 ( 6.86 ms/it)
+  objective:  13319.080896167245
+Minimizing 30334    Time: 0:03:28 ( 6.86 ms/it)
+  objective:  13319.071622916323
+Minimizing 30349    Time: 0:03:28 ( 6.86 ms/it)
+  objective:  13319.058759014864
+Minimizing 30364    Time: 0:03:28 ( 6.86 ms/it)
+  objective:  13318.967923063654
+Minimizing 30379    Time: 0:03:28 ( 6.86 ms/it)
+  objective:  13318.95189589198
+Minimizing 30394    Time: 0:03:28 ( 6.86 ms/it)
+  objective:  13318.951559366978
+Minimizing 30409    Time: 0:03:28 ( 6.86 ms/it)
+  objective:  13318.95016177863
+Minimizing 30424    Time: 0:03:28 ( 6.86 ms/it)
+  objective:  13318.949680432095
+Minimizing 30439    Time: 0:03:28 ( 6.86 ms/it)
+  objective:  13318.949289823096
+Minimizing 30454    Time: 0:03:28 ( 6.86 ms/it)
+  objective:  13318.948242705825
+Minimizing 30469    Time: 0:03:29 ( 6.86 ms/it)
+  objective:  13318.942267297578
+Minimizing 30484    Time: 0:03:29 ( 6.86 ms/it)
+  objective:  13318.938201766781
+Minimizing 30499    Time: 0:03:29 ( 6.86 ms/it)
+  objective:  13318.936820964635
+Minimizing 30514    Time: 0:03:29 ( 6.86 ms/it)
+  objective:  13318.925886723679
+Minimizing 30529    Time: 0:03:29 ( 6.86 ms/it)
+  objective:  13318.925435335113
+Minimizing 30544    Time: 0:03:29 ( 6.86 ms/it)
+  objective:  13318.920956352958
+Minimizing 30559    Time: 0:03:29 ( 6.86 ms/it)
+  objective:  13318.920537958053
+Minimizing 30574    Time: 0:03:29 ( 6.86 ms/it)
+  objective:  13318.918995824351
+Minimizing 30589    Time: 0:03:29 ( 6.86 ms/it)
+  objective:  13318.922051668735
+Minimizing 30604    Time: 0:03:29 ( 6.86 ms/it)
+  objective:  13318.89733898679
+Minimizing 30619    Time: 0:03:30 ( 6.86 ms/it)
+  objective:  13318.892939684898
+Minimizing 30634    Time: 0:03:30 ( 6.86 ms/it)
+  objective:  13318.891872379463
+Minimizing 30649    Time: 0:03:30 ( 6.86 ms/it)
+  objective:  13318.891054441905
+Minimizing 30664    Time: 0:03:30 ( 6.86 ms/it)
+  objective:  13318.889973928206
+Minimizing 30679    Time: 0:03:30 ( 6.86 ms/it)
+  objective:  13318.88753616187
+Minimizing 30694    Time: 0:03:30 ( 6.86 ms/it)
+  objective:  13318.886504811424
+Minimizing 30709    Time: 0:03:30 ( 6.86 ms/it)
+  objective:  13318.881112211311
+Minimizing 30724    Time: 0:03:30 ( 6.86 ms/it)
+  objective:  13318.877419668235
+Minimizing 30739    Time: 0:03:30 ( 6.86 ms/it)
+  objective:  13318.868394685516
+Minimizing 30754    Time: 0:03:30 ( 6.86 ms/it)
+  objective:  13318.8581605988
+Minimizing 30769    Time: 0:03:31 ( 6.86 ms/it)
+  objective:  13318.85713138587
+Minimizing 30784    Time: 0:03:31 ( 6.86 ms/it)
+  objective:  13318.854234413273
+Minimizing 30799    Time: 0:03:31 ( 6.86 ms/it)
+  objective:  13318.850390373613
+Minimizing 30814    Time: 0:03:31 ( 6.86 ms/it)
+  objective:  13318.837969755055
+Minimizing 30829    Time: 0:03:31 ( 6.86 ms/it)
+  objective:  13318.823183799788
+Minimizing 30844    Time: 0:03:31 ( 6.86 ms/it)
+  objective:  13318.821716557752
+Minimizing 30859    Time: 0:03:31 ( 6.86 ms/it)
+  objective:  13318.816919184566
+Minimizing 30874    Time: 0:03:31 ( 6.86 ms/it)
+  objective:  13318.811515493187
+Minimizing 30889    Time: 0:03:31 ( 6.86 ms/it)
+  objective:  13318.809563593051
+Minimizing 30904    Time: 0:03:31 ( 6.86 ms/it)
+  objective:  13318.808869929693
+Minimizing 30919    Time: 0:03:32 ( 6.86 ms/it)
+  objective:  13318.80621255796
+Minimizing 30934    Time: 0:03:32 ( 6.86 ms/it)
+  objective:  13318.798838254312
+Minimizing 30949    Time: 0:03:32 ( 6.86 ms/it)
+  objective:  13318.797319051882
+Minimizing 30964    Time: 0:03:32 ( 6.86 ms/it)
+  objective:  13318.796533040295
+Minimizing 30979    Time: 0:03:32 ( 6.86 ms/it)
+  objective:  13318.795993607971
+Minimizing 30994    Time: 0:03:32 ( 6.86 ms/it)
+  objective:  13318.79496820386
+Minimizing 31009    Time: 0:03:32 ( 6.86 ms/it)
+  objective:  13318.794424001258
+Minimizing 31024    Time: 0:03:32 ( 6.86 ms/it)
+  objective:  13318.77881645714
+Minimizing 31039    Time: 0:03:32 ( 6.86 ms/it)
+  objective:  13318.768578322823
+Minimizing 31054    Time: 0:03:32 ( 6.86 ms/it)
+  objective:  13318.766335496519
+Minimizing 31069    Time: 0:03:33 ( 6.86 ms/it)
+  objective:  13318.762424403569
+Minimizing 31084    Time: 0:03:33 ( 6.86 ms/it)
+  objective:  13318.76133113807
+Minimizing 31099    Time: 0:03:33 ( 6.86 ms/it)
+  objective:  13318.757385488585
+Minimizing 31114    Time: 0:03:33 ( 6.86 ms/it)
+  objective:  13318.742703805518
+Minimizing 31129    Time: 0:03:33 ( 6.86 ms/it)
+  objective:  13318.736372023443
+Minimizing 31144    Time: 0:03:33 ( 6.86 ms/it)
+  objective:  13318.73592477999
+Minimizing 31159    Time: 0:03:33 ( 6.86 ms/it)
+  objective:  13318.733248780263
+Minimizing 31174    Time: 0:03:33 ( 6.86 ms/it)
+  objective:  13318.72997828864
+Minimizing 31189    Time: 0:03:33 ( 6.86 ms/it)
+  objective:  13318.71687801517
+Minimizing 31204    Time: 0:03:34 ( 6.86 ms/it)
+  objective:  13318.716341016494
+Minimizing 31219    Time: 0:03:34 ( 6.86 ms/it)
+  objective:  13318.73195776678
+Minimizing 31234    Time: 0:03:34 ( 6.86 ms/it)
+  objective:  13318.69436955129
+Minimizing 31249    Time: 0:03:34 ( 6.86 ms/it)
+  objective:  13318.567047572127
+Minimizing 31264    Time: 0:03:34 ( 6.86 ms/it)
+  objective:  13318.555847161522
+Minimizing 31279    Time: 0:03:34 ( 6.86 ms/it)
+  objective:  13318.555585473441
+Minimizing 31294    Time: 0:03:34 ( 6.86 ms/it)
+  objective:  13318.555218169116
+Minimizing 31309    Time: 0:03:34 ( 6.86 ms/it)
+  objective:  13318.555014153942
+Minimizing 31324    Time: 0:03:34 ( 6.86 ms/it)
+  objective:  13318.553477215974
+Minimizing 31339    Time: 0:03:34 ( 6.86 ms/it)
+  objective:  13318.552698611835
+Minimizing 31354    Time: 0:03:35 ( 6.86 ms/it)
+  objective:  13318.551583247012
+Minimizing 31369    Time: 0:03:35 ( 6.86 ms/it)
+  objective:  13318.525152111979
+Minimizing 31384    Time: 0:03:35 ( 6.86 ms/it)
+  objective:  13318.50466107596
+Minimizing 31399    Time: 0:03:35 ( 6.86 ms/it)
+  objective:  13318.504483145924
+Minimizing 31414    Time: 0:03:35 ( 6.86 ms/it)
+  objective:  13318.504148584718
+Minimizing 31429    Time: 0:03:35 ( 6.86 ms/it)
+  objective:  13318.503907172766
+Minimizing 31444    Time: 0:03:35 ( 6.86 ms/it)
+  objective:  13318.503521238177
+Minimizing 31459    Time: 0:03:35 ( 6.86 ms/it)
+  objective:  13318.503289076325
+Minimizing 31474    Time: 0:03:35 ( 6.86 ms/it)
+  objective:  13318.502459596319
+Minimizing 31489    Time: 0:03:35 ( 6.86 ms/it)
+  objective:  13318.49627119019
+Minimizing 31504    Time: 0:03:36 ( 6.86 ms/it)
+  objective:  13318.495306164172
+Minimizing 31519    Time: 0:03:36 ( 6.86 ms/it)
+  objective:  13318.492374510402
+Minimizing 31534    Time: 0:03:36 ( 6.86 ms/it)
+  objective:  13318.490290971793
+Minimizing 31549    Time: 0:03:36 ( 6.86 ms/it)
+  objective:  13318.48985730666
+Minimizing 31564    Time: 0:03:36 ( 6.86 ms/it)
+  objective:  13318.489744292747
+Minimizing 31579    Time: 0:03:36 ( 6.86 ms/it)
+  objective:  13318.486726732954
+Minimizing 31593    Time: 0:03:36 ( 6.86 ms/it)
+  objective:  13318.48022669289
+Minimizing 31608    Time: 0:03:36 ( 6.86 ms/it)
+  objective:  13318.480097212654
+Minimizing 31623    Time: 0:03:36 ( 6.86 ms/it)
+  objective:  13318.479483446848
+Minimizing 31638    Time: 0:03:36 ( 6.86 ms/it)
+  objective:  13318.47910676575
+Minimizing 31652    Time: 0:03:37 ( 6.86 ms/it)
+  objective:  13318.477661737634
+Minimizing 31666    Time: 0:03:37 ( 6.86 ms/it)
+  objective:  13318.467330348692
+Minimizing 31678    Time: 0:03:37 ( 6.86 ms/it)
+  objective:  13318.467061815405
+Minimizing 31692    Time: 0:03:37 ( 6.86 ms/it)
+  objective:  13318.4662447788
+Minimizing 31706    Time: 0:03:37 ( 6.86 ms/it)
+  objective:  13318.465420935856
+Minimizing 31720    Time: 0:03:37 ( 6.86 ms/it)
+  objective:  13318.461528615968
+Minimizing 31735    Time: 0:03:37 ( 6.86 ms/it)
+  objective:  13318.461141356514
+Minimizing 31749    Time: 0:03:37 ( 6.86 ms/it)
+  objective:  13318.46025494275
+Minimizing 31763    Time: 0:03:37 ( 6.86 ms/it)
+  objective:  13318.458797042593
+Minimizing 31777    Time: 0:03:38 ( 6.86 ms/it)
+  objective:  13318.454866237691
+Minimizing 31792    Time: 0:03:38 ( 6.86 ms/it)
+  objective:  13318.448094710475
+Minimizing 31806    Time: 0:03:38 ( 6.86 ms/it)
+  objective:  13318.442435281817
+Minimizing 31820    Time: 0:03:38 ( 6.86 ms/it)
+  objective:  13318.438825138
+Minimizing 31834    Time: 0:03:38 ( 6.86 ms/it)
+  objective:  13318.437178064822
+Minimizing 31847    Time: 0:03:38 ( 6.86 ms/it)
+  objective:  13318.436841250936
+Minimizing 31861    Time: 0:03:38 ( 6.86 ms/it)
+  objective:  13318.434138978802
+Minimizing 31875    Time: 0:03:38 ( 6.86 ms/it)
+  objective:  13318.427068281642
+Minimizing 31889    Time: 0:03:38 ( 6.86 ms/it)
+  objective:  13318.426858292281
+Minimizing 31903    Time: 0:03:38 ( 6.86 ms/it)
+  objective:  13318.426926414817
+Minimizing 31918    Time: 0:03:39 ( 6.86 ms/it)
+  objective:  13318.426870108146
+Minimizing 31933    Time: 0:03:39 ( 6.86 ms/it)
+  objective:  13318.426866771188
+Minimizing 31948    Time: 0:03:39 ( 6.86 ms/it)
+  objective:  13318.42686730239
+Minimizing 31963    Time: 0:03:39 ( 6.86 ms/it)
+  objective:  13318.426851992088
+Minimizing 31978    Time: 0:03:39 ( 6.86 ms/it)
+  objective:  13318.42686271423
+Minimizing 31993    Time: 0:03:39 ( 6.86 ms/it)
+  objective:  13318.42684531596
+Minimizing 32008    Time: 0:03:39 ( 6.86 ms/it)
+  objective:  13318.426845584909
+Minimizing 32023    Time: 0:03:39 ( 6.86 ms/it)
+  objective:  13318.426848744784
+Minimizing 32038    Time: 0:03:39 ( 6.86 ms/it)
+  objective:  13318.426841513778
+Minimizing 32053    Time: 0:03:40 ( 6.86 ms/it)
+  objective:  13318.426840389177
+Minimizing 32068    Time: 0:03:40 ( 6.86 ms/it)
+  objective:  13318.426845112364
+Minimizing 32083    Time: 0:03:40 ( 6.86 ms/it)
+  objective:  13318.426842506873
+Minimizing 32098    Time: 0:03:40 ( 6.86 ms/it)
+  objective:  13318.42684245354
+Minimizing 32100    Time: 0:03:40 ( 6.86 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01830.0139-1.320.1876
Test: Star-Run-0.01420.0438-0.320.7465
Test: S20-Star-0.01110.0409-0.270.7862
Test: SLJ-S200.00990.04420.220.8236
Test: BPT-SLJ-0.02310.0422-0.550.5836
a10.31670.04766.65<1e-10
Sex: Boys0.20480.013914.73<1e-48
Test: Star-Run & a10.29860.15051.980.0472
Test: S20-Star & a10.04940.14190.350.7275
Test: SLJ-S20 & a1-0.06580.1520-0.430.6649
Test: BPT-SLJ & a10.31510.14162.230.0261
Test: Star-Run & Sex: Boys-0.12130.0438-2.770.0056
Test: S20-Star & Sex: Boys0.06640.04091.630.1041
Test: SLJ-S20 & Sex: Boys0.04660.04421.050.2919
Test: BPT-SLJ & Sex: Boys0.16340.04223.870.0001
a1 & Sex: Boys0.05020.04761.060.2912
Test: Star-Run & a1 & Sex: Boys-0.01140.1505-0.080.9395
Test: S20-Star & a1 & Sex: Boys-0.02880.1419-0.200.8390
Test: SLJ-S20 & a1 & Sex: Boys-0.06030.1520-0.400.6916
Test: BPT-SLJ & a1 & Sex: Boys0.14150.14161.000.3178
Test: BPT0.9054
Test: SLJ0.9717
Test: Star_r0.9914
Test: Run0.9717
Test: S20_r0.9786
Residual0.0003
+
+
+
+
VarCorr(m_cpx_0_SeqDiff)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
ChildTest: Run0.9442062220.971702743
Test: Star_r0.9828374160.991381569+0.64
Test: S20_r0.9575933630.978566995+0.22+0.78
Test: SLJ0.9441360950.971666659+0.51+0.12-0.54
Test: BPT0.8197840020.905419241+0.15+0.42+0.11+0.38
Residual0.0000000960.000309326
+
+
+
+
m_cpx_0_SeqDiff.PCA
+
+
(Child = 
+Principal components based on correlation matrix
+ Test: Run      1.0     .      .      .      .
+ Test: Star_r   0.64   1.0     .      .      .
+ Test: S20_r    0.22   0.78   1.0     .      .
+ Test: SLJ      0.51   0.12  -0.54   1.0     .
+ Test: BPT      0.15   0.42   0.11   0.38   1.0
+
+Normalized cumulative variances:
+[0.4586, 0.803, 0.9738, 1.0, 1.0]
+
+Component loadings
+                 PC1    PC2    PC3    PC4    PC5
+ Test: Run     -0.5    0.24  -0.56   0.61  -0.0
+ Test: Star_r  -0.64  -0.16  -0.02  -0.48  -0.58
+ Test: S20_r   -0.42  -0.59   0.03  -0.09   0.69
+ Test: SLJ     -0.19   0.71  -0.07  -0.51   0.44
+ Test: BPT     -0.36   0.25   0.82   0.36  -0.0,)
+
+
+
+
f_cpx_1 = @formula(
+  zScore ~ 1 + Test * a1 * Sex + (1 + Test | Child)
+)
+m_cpx_1_SeqDiff =
+fit(MixedModel, f_cpx_1, dat; contrasts=contr1b)
+
+
Minimizing 15    Time: 0:00:00 ( 6.75 ms/it)
+  objective:  13992.236627340779
+Minimizing 30    Time: 0:00:00 ( 6.78 ms/it)
+  objective:  13841.827333118908
+Minimizing 46    Time: 0:00:00 ( 6.75 ms/it)
+  objective:  13777.493789325295
+Minimizing 62    Time: 0:00:00 ( 6.73 ms/it)
+  objective:  13772.872983178971
+Minimizing 77    Time: 0:00:00 ( 6.72 ms/it)
+  objective:  13760.898044900383
+Minimizing 92    Time: 0:00:00 ( 6.72 ms/it)
+  objective:  13757.94612351193
+Minimizing 107    Time: 0:00:00 ( 6.72 ms/it)
+  objective:  13748.357981786798
+Minimizing 122    Time: 0:00:00 ( 6.72 ms/it)
+  objective:  13744.757616729485
+Minimizing 137    Time: 0:00:00 ( 6.72 ms/it)
+  objective:  13739.966470629659
+Minimizing 152    Time: 0:00:01 ( 6.72 ms/it)
+  objective:  13716.545957536473
+Minimizing 167    Time: 0:00:01 ( 6.72 ms/it)
+  objective:  13710.064283256732
+Minimizing 182    Time: 0:00:01 ( 6.73 ms/it)
+  objective:  13696.051134898527
+Minimizing 197    Time: 0:00:01 ( 6.73 ms/it)
+  objective:  13692.114899055618
+Minimizing 212    Time: 0:00:01 ( 6.73 ms/it)
+  objective:  13685.070924484331
+Minimizing 227    Time: 0:00:01 ( 6.73 ms/it)
+  objective:  13672.89278358193
+Minimizing 242    Time: 0:00:01 ( 6.73 ms/it)
+  objective:  13661.980753634167
+Minimizing 257    Time: 0:00:01 ( 6.73 ms/it)
+  objective:  13652.330741699696
+Minimizing 273    Time: 0:00:01 ( 6.72 ms/it)
+  objective:  13643.691435874693
+Minimizing 289    Time: 0:00:01 ( 6.72 ms/it)
+  objective:  13642.458938908409
+Minimizing 305    Time: 0:00:02 ( 6.72 ms/it)
+  objective:  13639.567256285494
+Minimizing 321    Time: 0:00:02 ( 6.72 ms/it)
+  objective:  13635.270641935873
+Minimizing 336    Time: 0:00:02 ( 6.72 ms/it)
+  objective:  13632.134726806788
+Minimizing 351    Time: 0:00:02 ( 6.72 ms/it)
+  objective:  13628.761272475658
+Minimizing 366    Time: 0:00:02 ( 6.73 ms/it)
+  objective:  13624.189653118381
+Minimizing 381    Time: 0:00:02 ( 6.73 ms/it)
+  objective:  13620.312662915818
+Minimizing 396    Time: 0:00:02 ( 6.73 ms/it)
+  objective:  13618.417326557159
+Minimizing 411    Time: 0:00:02 ( 6.74 ms/it)
+  objective:  13609.36978720955
+Minimizing 426    Time: 0:00:02 ( 6.74 ms/it)
+  objective:  13607.234474665878
+Minimizing 441    Time: 0:00:02 ( 6.75 ms/it)
+  objective:  13604.504124525956
+Minimizing 456    Time: 0:00:03 ( 6.76 ms/it)
+  objective:  13602.045160800197
+Minimizing 471    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13600.801804760627
+Minimizing 486    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13598.14423866852
+Minimizing 501    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13597.34932065606
+Minimizing 516    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13595.704406463956
+Minimizing 531    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13594.245308691123
+Minimizing 546    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13593.789234561955
+Minimizing 562    Time: 0:00:03 ( 6.76 ms/it)
+  objective:  13592.585058519155
+Minimizing 577    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13588.988369878443
+Minimizing 592    Time: 0:00:04 ( 6.76 ms/it)
+  objective:  13588.288747463237
+Minimizing 607    Time: 0:00:04 ( 6.76 ms/it)
+  objective:  13587.355794477331
+Minimizing 623    Time: 0:00:04 ( 6.76 ms/it)
+  objective:  13586.7483568133
+Minimizing 638    Time: 0:00:04 ( 6.76 ms/it)
+  objective:  13583.043956016008
+Minimizing 653    Time: 0:00:04 ( 6.77 ms/it)
+  objective:  13582.832811241413
+Minimizing 668    Time: 0:00:04 ( 6.77 ms/it)
+  objective:  13574.479359698835
+Minimizing 683    Time: 0:00:04 ( 6.77 ms/it)
+  objective:  13568.874482939911
+Minimizing 698    Time: 0:00:04 ( 6.77 ms/it)
+  objective:  13566.419593540963
+Minimizing 713    Time: 0:00:04 ( 6.77 ms/it)
+  objective:  13560.72496586899
+Minimizing 728    Time: 0:00:04 ( 6.77 ms/it)
+  objective:  13554.750402167076
+Minimizing 743    Time: 0:00:05 ( 6.77 ms/it)
+  objective:  13541.189813718764
+Minimizing 758    Time: 0:00:05 ( 6.76 ms/it)
+  objective:  13494.999215824471
+Minimizing 773    Time: 0:00:05 ( 6.78 ms/it)
+  objective:  13492.648850050697
+Minimizing 786    Time: 0:00:05 ( 6.80 ms/it)
+  objective:  13488.098061394303
+Minimizing 801    Time: 0:00:05 ( 6.80 ms/it)
+  objective:  13487.32443868601
+Minimizing 816    Time: 0:00:05 ( 6.80 ms/it)
+  objective:  13485.804641650837
+Minimizing 831    Time: 0:00:05 ( 6.80 ms/it)
+  objective:  13484.972100949992
+Minimizing 846    Time: 0:00:05 ( 6.80 ms/it)
+  objective:  13482.206646936727
+Minimizing 861    Time: 0:00:05 ( 6.80 ms/it)
+  objective:  13481.957981714644
+Minimizing 876    Time: 0:00:05 ( 6.81 ms/it)
+  objective:  13480.668897363736
+Minimizing 890    Time: 0:00:06 ( 6.81 ms/it)
+  objective:  13477.848993872809
+Minimizing 905    Time: 0:00:06 ( 6.82 ms/it)
+  objective:  13477.61909756872
+Minimizing 920    Time: 0:00:06 ( 6.82 ms/it)
+  objective:  13477.08038639943
+Minimizing 935    Time: 0:00:06 ( 6.82 ms/it)
+  objective:  13476.438883850664
+Minimizing 950    Time: 0:00:06 ( 6.82 ms/it)
+  objective:  13475.129701195961
+Minimizing 965    Time: 0:00:06 ( 6.81 ms/it)
+  objective:  13473.015547506504
+Minimizing 980    Time: 0:00:06 ( 6.81 ms/it)
+  objective:  13468.897406319942
+Minimizing 996    Time: 0:00:06 ( 6.81 ms/it)
+  objective:  13468.531470264665
+Minimizing 1011    Time: 0:00:06 ( 6.81 ms/it)
+  objective:  13462.405574791788
+Minimizing 1026    Time: 0:00:06 ( 6.81 ms/it)
+  objective:  13453.976129003247
+Minimizing 1041    Time: 0:00:07 ( 6.81 ms/it)
+  objective:  13433.67933769874
+Minimizing 1056    Time: 0:00:07 ( 6.81 ms/it)
+  objective:  13421.061688833033
+Minimizing 1071    Time: 0:00:07 ( 6.81 ms/it)
+  objective:  13419.311719043399
+Minimizing 1086    Time: 0:00:07 ( 6.82 ms/it)
+  objective:  13416.643668426506
+Minimizing 1101    Time: 0:00:07 ( 6.82 ms/it)
+  objective:  13412.486559248086
+Minimizing 1116    Time: 0:00:07 ( 6.82 ms/it)
+  objective:  13408.443196157146
+Minimizing 1131    Time: 0:00:07 ( 6.81 ms/it)
+  objective:  13403.757208356918
+Minimizing 1147    Time: 0:00:07 ( 6.81 ms/it)
+  objective:  13403.131604382761
+Minimizing 1163    Time: 0:00:07 ( 6.81 ms/it)
+  objective:  13398.354195700864
+Minimizing 1178    Time: 0:00:08 ( 6.81 ms/it)
+  objective:  13395.26226109868
+Minimizing 1193    Time: 0:00:08 ( 6.81 ms/it)
+  objective:  13391.908013950517
+Minimizing 1208    Time: 0:00:08 ( 6.81 ms/it)
+  objective:  13391.74473191856
+Minimizing 1223    Time: 0:00:08 ( 6.81 ms/it)
+  objective:  13389.736211543415
+Minimizing 1238    Time: 0:00:08 ( 6.80 ms/it)
+  objective:  13388.349268089878
+Minimizing 1253    Time: 0:00:08 ( 6.80 ms/it)
+  objective:  13384.994787373966
+Minimizing 1268    Time: 0:00:08 ( 6.80 ms/it)
+  objective:  13383.408577161244
+Minimizing 1283    Time: 0:00:08 ( 6.80 ms/it)
+  objective:  13379.77084859628
+Minimizing 1298    Time: 0:00:08 ( 6.80 ms/it)
+  objective:  13378.356226256132
+Minimizing 1313    Time: 0:00:08 ( 6.80 ms/it)
+  objective:  13371.204840040773
+Minimizing 1328    Time: 0:00:09 ( 6.80 ms/it)
+  objective:  13368.552162300708
+Minimizing 1343    Time: 0:00:09 ( 6.80 ms/it)
+  objective:  13360.956304682499
+Minimizing 1358    Time: 0:00:09 ( 6.80 ms/it)
+  objective:  13351.888021394683
+Minimizing 1373    Time: 0:00:09 ( 6.80 ms/it)
+  objective:  13344.62582498425
+Minimizing 1388    Time: 0:00:09 ( 6.80 ms/it)
+  objective:  13336.39649426828
+Minimizing 1403    Time: 0:00:09 ( 6.80 ms/it)
+  objective:  13324.383979126607
+Minimizing 1418    Time: 0:00:09 ( 6.80 ms/it)
+  objective:  13322.61377139554
+Minimizing 1433    Time: 0:00:09 ( 6.80 ms/it)
+  objective:  13320.57700703424
+Minimizing 1448    Time: 0:00:09 ( 6.79 ms/it)
+  objective:  13318.4552940086
+Minimizing 1463    Time: 0:00:09 ( 6.79 ms/it)
+  objective:  13316.762270926934
+Minimizing 1478    Time: 0:00:10 ( 6.79 ms/it)
+  objective:  13314.408444318266
+Minimizing 1493    Time: 0:00:10 ( 6.79 ms/it)
+  objective:  13313.767416919145
+Minimizing 1509    Time: 0:00:10 ( 6.79 ms/it)
+  objective:  13312.854553939309
+Minimizing 1524    Time: 0:00:10 ( 6.79 ms/it)
+  objective:  13311.419052878919
+Minimizing 1539    Time: 0:00:10 ( 6.79 ms/it)
+  objective:  13310.612248396574
+Minimizing 1554    Time: 0:00:10 ( 6.79 ms/it)
+  objective:  13309.0929672551
+Minimizing 1569    Time: 0:00:10 ( 6.79 ms/it)
+  objective:  13308.660574705282
+Minimizing 1584    Time: 0:00:10 ( 6.79 ms/it)
+  objective:  13305.877100199563
+Minimizing 1600    Time: 0:00:10 ( 6.79 ms/it)
+  objective:  13302.793196995844
+Minimizing 1615    Time: 0:00:10 ( 6.79 ms/it)
+  objective:  13286.848302695595
+Minimizing 1630    Time: 0:00:11 ( 6.79 ms/it)
+  objective:  13281.248852850098
+Minimizing 1645    Time: 0:00:11 ( 6.79 ms/it)
+  objective:  13273.201509256221
+Minimizing 1660    Time: 0:00:11 ( 6.79 ms/it)
+  objective:  13269.96410347159
+Minimizing 1676    Time: 0:00:11 ( 6.79 ms/it)
+  objective:  13265.32337813532
+Minimizing 1691    Time: 0:00:11 ( 6.79 ms/it)
+  objective:  13258.242799779066
+Minimizing 1706    Time: 0:00:11 ( 6.79 ms/it)
+  objective:  13224.157705357982
+Minimizing 1722    Time: 0:00:11 ( 6.79 ms/it)
+  objective:  13218.836364054543
+Minimizing 1737    Time: 0:00:11 ( 6.79 ms/it)
+  objective:  13172.04866272863
+Minimizing 1752    Time: 0:00:11 ( 6.79 ms/it)
+  objective:  13170.738575918338
+Minimizing 1768    Time: 0:00:11 ( 6.79 ms/it)
+  objective:  13167.32485531435
+Minimizing 1783    Time: 0:00:12 ( 6.79 ms/it)
+  objective:  13165.006059175896
+Minimizing 1799    Time: 0:00:12 ( 6.79 ms/it)
+  objective:  13163.136445673794
+Minimizing 1814    Time: 0:00:12 ( 6.79 ms/it)
+  objective:  13160.840416817664
+Minimizing 1830    Time: 0:00:12 ( 6.79 ms/it)
+  objective:  13158.762434916745
+Minimizing 1845    Time: 0:00:12 ( 6.79 ms/it)
+  objective:  13156.638211053461
+Minimizing 1860    Time: 0:00:12 ( 6.79 ms/it)
+  objective:  13150.721932839908
+Minimizing 1875    Time: 0:00:12 ( 6.78 ms/it)
+  objective:  13145.595892946672
+Minimizing 1890    Time: 0:00:12 ( 6.78 ms/it)
+  objective:  13144.827830569018
+Minimizing 1905    Time: 0:00:12 ( 6.78 ms/it)
+  objective:  13144.730932205304
+Minimizing 1921    Time: 0:00:13 ( 6.78 ms/it)
+  objective:  13144.71833535924
+Minimizing 1936    Time: 0:00:13 ( 6.78 ms/it)
+  objective:  13144.718762883233
+Minimizing 1953    Time: 0:00:13 ( 6.79 ms/it)
+  objective:  13144.709759888443
+Minimizing 1970    Time: 0:00:13 ( 6.79 ms/it)
+  objective:  13144.709424346802
+Minimizing 1985    Time: 0:00:13 ( 6.79 ms/it)
+  objective:  13144.715370203223
+Minimizing 2000    Time: 0:00:13 ( 6.79 ms/it)
+  objective:  13144.710256644277
+Minimizing 2016    Time: 0:00:13 ( 6.79 ms/it)
+  objective:  13144.721392364707
+Minimizing 2032    Time: 0:00:13 ( 6.79 ms/it)
+  objective:  13144.704198067135
+Minimizing 2047    Time: 0:00:13 ( 6.79 ms/it)
+  objective:  13144.705960082254
+Minimizing 2063    Time: 0:00:13 ( 6.78 ms/it)
+  objective:  13144.72094398216
+Minimizing 2078    Time: 0:00:14 ( 6.78 ms/it)
+  objective:  13144.706341214944
+Minimizing 2093    Time: 0:00:14 ( 6.78 ms/it)
+  objective:  13144.708741448092
+Minimizing 2108    Time: 0:00:14 ( 6.78 ms/it)
+  objective:  13144.706141769188
+Minimizing 2123    Time: 0:00:14 ( 6.78 ms/it)
+  objective:  13144.705232628257
+Minimizing 2138    Time: 0:00:14 ( 6.78 ms/it)
+  objective:  13144.705378644489
+Minimizing 2153    Time: 0:00:14 ( 6.78 ms/it)
+  objective:  13144.709370245488
+Minimizing 2167    Time: 0:00:14 ( 6.78 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01910.0140-1.360.17270.5852
Test: Star-Run-0.01300.0445-0.290.76980.8496
Test: S20-Star-0.00970.0446-0.220.82820.6785
Test: SLJ-S200.00930.04410.210.83310.8706
Test: BPT-SLJ-0.01970.0424-0.460.64220.9671
a10.32010.04796.69<1e-10
Sex: Boys0.20530.014014.69<1e-48
Test: Star-Run & a10.29580.15261.940.0526
Test: S20-Star & a10.06310.15430.410.6824
Test: SLJ-S20 & a1-0.09020.1521-0.590.5532
Test: BPT-SLJ & a10.33250.14212.340.0193
Test: Star-Run & Sex: Boys-0.12630.0445-2.840.0046
Test: S20-Star & Sex: Boys0.07150.04461.610.1084
Test: SLJ-S20 & Sex: Boys0.04260.04410.970.3335
Test: BPT-SLJ & Sex: Boys0.16690.04243.94<1e-04
a1 & Sex: Boys0.04440.04790.930.3536
Test: Star-Run & a1 & Sex: Boys-0.01420.1526-0.090.9259
Test: S20-Star & a1 & Sex: Boys-0.02260.1543-0.150.8836
Test: SLJ-S20 & a1 & Sex: Boys-0.05060.1521-0.330.7395
Test: BPT-SLJ & a1 & Sex: Boys0.13970.14210.980.3255
Residual0.0000
+
+
+
+
m_cpx_1_SeqDiff.PCA
+
+
(Child = 
+Principal components based on correlation matrix
+ (Intercept)      1.0     .      .      .      .
+ Test: Star-Run   0.75   1.0     .      .      .
+ Test: S20-Star   0.04   0.67   1.0     .      .
+ Test: SLJ-S20   -0.44  -0.19   0.42   1.0     .
+ Test: BPT-SLJ   -0.07  -0.24  -0.37  -0.32   1.0
+
+Normalized cumulative variances:
+[0.4315, 0.7875, 0.9292, 1.0, 1.0]
+
+Component loadings
+                   PC1    PC2    PC3    PC4    PC5
+ (Intercept)     -0.47   0.46  -0.17  -0.59   0.43
+ Test: Star-Run  -0.66   0.14   0.2    0.06  -0.71
+ Test: S20-Star  -0.49  -0.41   0.45   0.33   0.53
+ Test: SLJ-S20    0.03  -0.67   0.14  -0.7   -0.17
+ Test: BPT-SLJ    0.31   0.39   0.84  -0.19   0.01,)
+
+
+
+
+

2.4 PCA-based HypothesisCoding: contr4

+

The fourth set of contrasts uses HypothesisCoding to specify the set of contrasts implementing the loadings of the four principle components of the published LMM based on test scores, not test effects (contrasts) - coarse-grained, that is roughly according to their signs. This is actually a very interesting and plausible solution nobody had proposed a priori.

+
    +
  • PC1: BPT - Run_r
    • +
  • PC2: (Star_r + S20_r + SLJ) - (BPT + Run_r)
    • +
  • PC3: Star_r - (S20_r + SLJ)
    • +
  • PC4: S20_r - SLJ
    • +
+

PC1 contrasts the worst and the best indicator of physical health; PC2 contrasts these two against the core indicators of physical fitness; PC3 contrasts the cognitive and the physical tests within the narrow set of physical fitness components; and PC4, finally, contrasts two types of lower muscular fitness differing in speed and power.

+
+
contr4 = Dict(
+  :School => Grouping(),
+  :Child => Grouping(),
+  :Cohort => Grouping(),
+  :Sex => EffectsCoding(; levels=["Girls", "Boys"]),
+  :Test => HypothesisCoding(
+    [
+      -1 0 0 0 +1
+      -3 +2 +2 +2 -3
+      0 +2 -1 -1 0
+      0 0 +1 -1 0
+    ];
+    levels=["Run", "Star_r", "S20_r", "SLJ", "BPT"],
+    labels=["c5.1", "c234.15", "c2.34", "c3.4"],
+  ),
+);
+
+
+
m_cpx_1_PC = fit(MixedModel, f_cpx_1, dat; contrasts=contr4)
+
+
Minimizing 15    Time: 0:00:00 ( 6.88 ms/it)
+  objective:  13816.332602330232
+Minimizing 30    Time: 0:00:00 ( 6.82 ms/it)
+  objective:  13819.656571653324
+Minimizing 45    Time: 0:00:00 ( 6.79 ms/it)
+  objective:  13791.585455961931
+Minimizing 60    Time: 0:00:00 ( 6.77 ms/it)
+  objective:  13780.67571280646
+Minimizing 75    Time: 0:00:00 ( 6.78 ms/it)
+  objective:  13773.874746283942
+Minimizing 90    Time: 0:00:00 ( 6.78 ms/it)
+  objective:  13772.086478567438
+Minimizing 105    Time: 0:00:00 ( 6.80 ms/it)
+  objective:  13764.66194285501
+Minimizing 120    Time: 0:00:00 ( 6.81 ms/it)
+  objective:  13758.522570130917
+Minimizing 135    Time: 0:00:00 ( 6.81 ms/it)
+  objective:  13744.732814488541
+Minimizing 150    Time: 0:00:01 ( 6.81 ms/it)
+  objective:  13733.668211491575
+Minimizing 165    Time: 0:00:01 ( 6.81 ms/it)
+  objective:  13732.36340006553
+Minimizing 180    Time: 0:00:01 ( 6.82 ms/it)
+  objective:  13728.264261401562
+Minimizing 195    Time: 0:00:01 ( 6.81 ms/it)
+  objective:  13723.675896663455
+Minimizing 210    Time: 0:00:01 ( 6.81 ms/it)
+  objective:  13708.984335873063
+Minimizing 226    Time: 0:00:01 ( 6.80 ms/it)
+  objective:  13706.154390410527
+Minimizing 241    Time: 0:00:01 ( 6.79 ms/it)
+  objective:  13695.813163832274
+Minimizing 256    Time: 0:00:01 ( 6.79 ms/it)
+  objective:  13687.171487634814
+Minimizing 271    Time: 0:00:01 ( 6.79 ms/it)
+  objective:  13681.930135484894
+Minimizing 287    Time: 0:00:01 ( 6.78 ms/it)
+  objective:  13667.212654789928
+Minimizing 303    Time: 0:00:02 ( 6.78 ms/it)
+  objective:  13661.137116128333
+Minimizing 318    Time: 0:00:02 ( 6.78 ms/it)
+  objective:  13648.711313588408
+Minimizing 333    Time: 0:00:02 ( 6.78 ms/it)
+  objective:  13640.226573704029
+Minimizing 348    Time: 0:00:02 ( 6.79 ms/it)
+  objective:  13617.082625055522
+Minimizing 363    Time: 0:00:02 ( 6.78 ms/it)
+  objective:  13615.026428660676
+Minimizing 378    Time: 0:00:02 ( 6.78 ms/it)
+  objective:  13610.071531565009
+Minimizing 393    Time: 0:00:02 ( 6.78 ms/it)
+  objective:  13606.914276340576
+Minimizing 409    Time: 0:00:02 ( 6.78 ms/it)
+  objective:  13601.226598413992
+Minimizing 424    Time: 0:00:02 ( 6.77 ms/it)
+  objective:  13592.804836582302
+Minimizing 439    Time: 0:00:02 ( 6.77 ms/it)
+  objective:  13568.28721410715
+Minimizing 454    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13534.056522645864
+Minimizing 469    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13529.915286408752
+Minimizing 485    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13514.271096737903
+Minimizing 500    Time: 0:00:03 ( 6.76 ms/it)
+  objective:  13509.704795192432
+Minimizing 515    Time: 0:00:03 ( 6.76 ms/it)
+  objective:  13502.135581336523
+Minimizing 530    Time: 0:00:03 ( 6.76 ms/it)
+  objective:  13506.742422807103
+Minimizing 545    Time: 0:00:03 ( 6.76 ms/it)
+  objective:  13463.385223559373
+Minimizing 560    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13462.810806506597
+Minimizing 575    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13456.78806683152
+Minimizing 590    Time: 0:00:03 ( 6.77 ms/it)
+  objective:  13454.98161661686
+Minimizing 605    Time: 0:00:04 ( 6.77 ms/it)
+  objective:  13452.632935365473
+Minimizing 620    Time: 0:00:04 ( 6.77 ms/it)
+  objective:  13449.7338857047
+Minimizing 635    Time: 0:00:04 ( 6.78 ms/it)
+  objective:  13446.226209510103
+Minimizing 649    Time: 0:00:04 ( 6.79 ms/it)
+  objective:  13443.323829727764
+Minimizing 664    Time: 0:00:04 ( 6.80 ms/it)
+  objective:  13440.292480622637
+Minimizing 678    Time: 0:00:04 ( 6.81 ms/it)
+  objective:  13430.352151089013
+Minimizing 693    Time: 0:00:04 ( 6.81 ms/it)
+  objective:  13426.67388857832
+Minimizing 708    Time: 0:00:04 ( 6.82 ms/it)
+  objective:  13424.94699773231
+Minimizing 723    Time: 0:00:04 ( 6.83 ms/it)
+  objective:  13424.095826991637
+Minimizing 738    Time: 0:00:05 ( 6.83 ms/it)
+  objective:  13423.12578855658
+Minimizing 753    Time: 0:00:05 ( 6.82 ms/it)
+  objective:  13422.366024823343
+Minimizing 768    Time: 0:00:05 ( 6.82 ms/it)
+  objective:  13418.14667611393
+Minimizing 783    Time: 0:00:05 ( 6.82 ms/it)
+  objective:  13416.68757398241
+Minimizing 798    Time: 0:00:05 ( 6.82 ms/it)
+  objective:  13410.005246821544
+Minimizing 813    Time: 0:00:05 ( 6.82 ms/it)
+  objective:  13397.903233037963
+Minimizing 828    Time: 0:00:05 ( 6.82 ms/it)
+  objective:  13382.120781459795
+Minimizing 843    Time: 0:00:05 ( 6.82 ms/it)
+  objective:  13366.505614135458
+Minimizing 858    Time: 0:00:05 ( 6.82 ms/it)
+  objective:  13364.025405890228
+Minimizing 873    Time: 0:00:05 ( 6.82 ms/it)
+  objective:  13344.389366420699
+Minimizing 888    Time: 0:00:06 ( 6.83 ms/it)
+  objective:  13344.033429230505
+Minimizing 903    Time: 0:00:06 ( 6.83 ms/it)
+  objective:  13340.611014517257
+Minimizing 918    Time: 0:00:06 ( 6.83 ms/it)
+  objective:  13338.890144004705
+Minimizing 933    Time: 0:00:06 ( 6.83 ms/it)
+  objective:  13336.292964622058
+Minimizing 948    Time: 0:00:06 ( 6.84 ms/it)
+  objective:  13334.410375687352
+Minimizing 963    Time: 0:00:06 ( 6.84 ms/it)
+  objective:  13330.7964364582
+Minimizing 977    Time: 0:00:06 ( 6.85 ms/it)
+  objective:  13327.077580748039
+Minimizing 992    Time: 0:00:06 ( 6.85 ms/it)
+  objective:  13325.67516979968
+Minimizing 1006    Time: 0:00:06 ( 6.86 ms/it)
+  objective:  13323.546768697706
+Minimizing 1020    Time: 0:00:07 ( 6.86 ms/it)
+  objective:  13320.731071181115
+Minimizing 1034    Time: 0:00:07 ( 6.87 ms/it)
+  objective:  13317.376753146702
+Minimizing 1048    Time: 0:00:07 ( 6.88 ms/it)
+  objective:  13316.511893543866
+Minimizing 1062    Time: 0:00:07 ( 6.89 ms/it)
+  objective:  13316.011429793944
+Minimizing 1075    Time: 0:00:07 ( 6.90 ms/it)
+  objective:  13312.653803260066
+Minimizing 1089    Time: 0:00:07 ( 6.92 ms/it)
+  objective:  13311.798952526675
+Minimizing 1103    Time: 0:00:07 ( 6.92 ms/it)
+  objective:  13311.153731812388
+Minimizing 1117    Time: 0:00:07 ( 6.93 ms/it)
+  objective:  13309.648125015112
+Minimizing 1131    Time: 0:00:07 ( 6.94 ms/it)
+  objective:  13309.426799283116
+Minimizing 1144    Time: 0:00:07 ( 6.95 ms/it)
+  objective:  13307.433029087246
+Minimizing 1159    Time: 0:00:08 ( 6.95 ms/it)
+  objective:  13301.755871152942
+Minimizing 1174    Time: 0:00:08 ( 6.95 ms/it)
+  objective:  13294.200399732013
+Minimizing 1189    Time: 0:00:08 ( 6.95 ms/it)
+  objective:  13290.387146921057
+Minimizing 1204    Time: 0:00:08 ( 6.95 ms/it)
+  objective:  13287.385817066097
+Minimizing 1219    Time: 0:00:08 ( 6.94 ms/it)
+  objective:  13279.505459288368
+Minimizing 1234    Time: 0:00:08 ( 6.94 ms/it)
+  objective:  13273.29319327083
+Minimizing 1249    Time: 0:00:08 ( 6.94 ms/it)
+  objective:  13260.459047355165
+Minimizing 1264    Time: 0:00:08 ( 6.94 ms/it)
+  objective:  13239.77648263349
+Minimizing 1279    Time: 0:00:08 ( 6.94 ms/it)
+  objective:  13232.372454307391
+Minimizing 1294    Time: 0:00:08 ( 6.94 ms/it)
+  objective:  13219.20806476663
+Minimizing 1309    Time: 0:00:09 ( 6.94 ms/it)
+  objective:  13207.397301005229
+Minimizing 1324    Time: 0:00:09 ( 6.94 ms/it)
+  objective:  13207.04314413754
+Minimizing 1339    Time: 0:00:09 ( 6.94 ms/it)
+  objective:  13199.883679693623
+Minimizing 1355    Time: 0:00:09 ( 6.95 ms/it)
+  objective:  13196.533831483379
+Minimizing 1371    Time: 0:00:09 ( 6.95 ms/it)
+  objective:  13190.483032705422
+Minimizing 1385    Time: 0:00:09 ( 6.95 ms/it)
+  objective:  13183.917344881687
+Minimizing 1400    Time: 0:00:09 ( 6.96 ms/it)
+  objective:  13180.88049893771
+Minimizing 1415    Time: 0:00:09 ( 6.96 ms/it)
+  objective:  13172.490587439534
+Minimizing 1430    Time: 0:00:09 ( 6.95 ms/it)
+  objective:  13158.24100738352
+Minimizing 1445    Time: 0:00:10 ( 6.95 ms/it)
+  objective:  13157.580295724736
+Minimizing 1460    Time: 0:00:10 ( 6.95 ms/it)
+  objective:  13156.772916139555
+Minimizing 1475    Time: 0:00:10 ( 6.95 ms/it)
+  objective:  13155.054826345557
+Minimizing 1490    Time: 0:00:10 ( 6.95 ms/it)
+  objective:  13150.94076273813
+Minimizing 1505    Time: 0:00:10 ( 6.95 ms/it)
+  objective:  13147.509740719834
+Minimizing 1520    Time: 0:00:10 ( 6.95 ms/it)
+  objective:  13140.819616088833
+Minimizing 1535    Time: 0:00:10 ( 6.95 ms/it)
+  objective:  13137.350813194833
+Minimizing 1550    Time: 0:00:10 ( 6.95 ms/it)
+  objective:  13136.919053304591
+Minimizing 1565    Time: 0:00:10 ( 6.95 ms/it)
+  objective:  13134.483498100337
+Minimizing 1580    Time: 0:00:10 ( 6.95 ms/it)
+  objective:  13134.447373534378
+Minimizing 1595    Time: 0:00:11 ( 6.95 ms/it)
+  objective:  13134.431338372917
+Minimizing 1610    Time: 0:00:11 ( 6.95 ms/it)
+  objective:  13134.429919027709
+Minimizing 1625    Time: 0:00:11 ( 6.95 ms/it)
+  objective:  13134.432391109323
+Minimizing 1640    Time: 0:00:11 ( 6.95 ms/it)
+  objective:  13134.434131062284
+Minimizing 1655    Time: 0:00:11 ( 6.95 ms/it)
+  objective:  13134.43136893095
+Minimizing 1670    Time: 0:00:11 ( 6.95 ms/it)
+  objective:  13134.434588496297
+Minimizing 1685    Time: 0:00:11 ( 6.95 ms/it)
+  objective:  13134.432497580332
+Minimizing 1700    Time: 0:00:11 ( 6.95 ms/it)
+  objective:  13134.42282214512
+Minimizing 1715    Time: 0:00:11 ( 6.95 ms/it)
+  objective:  13134.435660578849
+Minimizing 1730    Time: 0:00:12 ( 6.95 ms/it)
+  objective:  13134.431615684036
+Minimizing 1743    Time: 0:00:12 ( 6.97 ms/it)
+  objective:  13134.437980213028
+Minimizing 1757    Time: 0:00:12 ( 6.97 ms/it)
+  objective:  13134.43474466115
+Minimizing 1772    Time: 0:00:12 ( 6.97 ms/it)
+  objective:  13134.43327911022
+Minimizing 1788    Time: 0:00:12 ( 6.96 ms/it)
+  objective:  13134.427159230705
+Minimizing 1799    Time: 0:00:12 ( 6.96 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01820.0140-1.300.19420.6870
Test: c5.1-0.04030.0429-0.940.34811.3839
Test: c234.150.01620.16780.100.92310.6312
Test: c2.340.00990.07580.130.89580.9852
Test: c3.4-0.01370.0445-0.310.75841.5296
a10.32070.04806.67<1e-10
Sex: Boys0.20520.014014.63<1e-47
Test: c5.1 & a10.61600.14694.19<1e-04
Test: c234.15 & a10.06410.57280.110.9108
Test: c2.34 & a1-0.06290.2584-0.240.8077
Test: c3.4 & a10.07420.15330.480.6284
Test: c5.1 & Sex: Boys0.15730.04293.670.0002
Test: c234.15 & Sex: Boys-0.82020.1678-4.89<1e-05
Test: c2.34 & Sex: Boys-0.18370.0758-2.420.0154
Test: c3.4 & Sex: Boys-0.04650.0445-1.040.2964
a1 & Sex: Boys0.05080.04801.060.2908
Test: c5.1 & a1 & Sex: Boys0.03840.14690.260.7937
Test: c234.15 & a1 & Sex: Boys-0.35680.5728-0.620.5333
Test: c2.34 & a1 & Sex: Boys0.14480.25840.560.5752
Test: c3.4 & a1 & Sex: Boys0.06080.15330.400.6916
Residual0.0000
+
+
+
+
VarCorr(m_cpx_1_PC)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Child(Intercept)0.471981870.68700937
Test: c5.11.915179891.38390025-0.11
Test: c234.150.398403700.63119228+0.83-0.48
Test: c2.340.970575840.98517807+0.69-0.15+0.88
Test: c3.42.339775401.52963244+0.12+0.08+0.40+0.63
Residual0.000000000.00001040
+
+
+
+
m_cpx_1_PC.PCA
+
+
(Child = 
+Principal components based on correlation matrix
+ (Intercept)     1.0     .      .      .      .
+ Test: c5.1     -0.11   1.0     .      .      .
+ Test: c234.15   0.83  -0.48   1.0     .      .
+ Test: c2.34     0.69  -0.15   0.88   1.0     .
+ Test: c3.4      0.12   0.08   0.4    0.63   1.0
+
+Normalized cumulative variances:
+[0.5849, 0.8176, 0.9767, 1.0, 1.0]
+
+Component loadings
+                  PC1    PC2    PC3    PC4    PC5
+ (Intercept)    -0.48  -0.12   0.61   0.5   -0.38
+ Test: c5.1      0.2    0.73   0.57  -0.1    0.3
+ Test: c234.15  -0.57  -0.18   0.02  -0.0    0.8
+ Test: c2.34    -0.55   0.2   -0.01  -0.73  -0.35
+ Test: c3.4     -0.32   0.61  -0.55   0.46  -0.08,)
+
+
+

There is a numerical interaction with a z-value > 2.0 for the first PCA (i.e., BPT - Run_r). This interaction would really need to be replicated to be taken seriously. It is probably due to larger “unfitness” gains in boys than girls (i.e., in BPT) relative to the slightly larger health-related “fitness” gains of girls than boys (i.e., in Run_r).

+
+
contr4b = merge(
+  Dict(nm => Grouping() for nm in (:School, :Child, :Cohort)),
+  Dict(
+    :Sex => EffectsCoding(; levels=["Girls", "Boys"]),
+    :Test => HypothesisCoding(
+      [
+        0.49 -0.04 0.20 0.03 -0.85
+        0.70 -0.56 -0.21 -0.13 0.37
+        0.31 0.68 -0.56 -0.35 0.00
+        0.04 0.08 0.61 -0.78 0.13
+      ];
+      levels=["Run", "Star_r", "S20_r", "SLJ", "BPT"],
+      labels=["c5.1", "c234.15", "c12.34", "c3.4"],
+    ),
+  ),
+);
+
+
+
m_cpx_1_PC_2 = fit(MixedModel, f_cpx_1, dat; contrasts=contr4b)
+
+
Minimizing 15    Time: 0:00:00 ( 6.95 ms/it)
+  objective:  13850.24151055307
+Minimizing 30    Time: 0:00:00 ( 6.95 ms/it)
+  objective:  13838.10560061899
+Minimizing 45    Time: 0:00:00 ( 6.94 ms/it)
+  objective:  13780.927582361033
+Minimizing 60    Time: 0:00:00 ( 6.94 ms/it)
+  objective:  13772.847096673908
+Minimizing 75    Time: 0:00:00 ( 6.92 ms/it)
+  objective:  13761.023906063292
+Minimizing 90    Time: 0:00:00 ( 6.92 ms/it)
+  objective:  13712.810888035232
+Minimizing 105    Time: 0:00:00 ( 6.94 ms/it)
+  objective:  13703.180631551688
+Minimizing 120    Time: 0:00:00 ( 6.95 ms/it)
+  objective:  13692.380747939407
+Minimizing 135    Time: 0:00:00 ( 6.95 ms/it)
+  objective:  13684.220472205616
+Minimizing 150    Time: 0:00:01 ( 6.94 ms/it)
+  objective:  13677.886588077183
+Minimizing 165    Time: 0:00:01 ( 6.92 ms/it)
+  objective:  13671.394614708945
+Minimizing 180    Time: 0:00:01 ( 6.91 ms/it)
+  objective:  13660.661746517948
+Minimizing 195    Time: 0:00:01 ( 6.91 ms/it)
+  objective:  13636.166198863193
+Minimizing 210    Time: 0:00:01 ( 6.91 ms/it)
+  objective:  13630.469487943494
+Minimizing 225    Time: 0:00:01 ( 6.91 ms/it)
+  objective:  13627.278645578153
+Minimizing 240    Time: 0:00:01 ( 6.90 ms/it)
+  objective:  13620.449884102767
+Minimizing 255    Time: 0:00:01 ( 6.90 ms/it)
+  objective:  13610.459682344223
+Minimizing 270    Time: 0:00:01 ( 6.89 ms/it)
+  objective:  13598.363790982221
+Minimizing 285    Time: 0:00:01 ( 6.89 ms/it)
+  objective:  13596.620097651725
+Minimizing 300    Time: 0:00:02 ( 6.89 ms/it)
+  objective:  13591.281476002816
+Minimizing 315    Time: 0:00:02 ( 6.88 ms/it)
+  objective:  13584.031821209064
+Minimizing 330    Time: 0:00:02 ( 6.88 ms/it)
+  objective:  13574.017253943832
+Minimizing 345    Time: 0:00:02 ( 6.88 ms/it)
+  objective:  13555.855304774563
+Minimizing 360    Time: 0:00:02 ( 6.88 ms/it)
+  objective:  13552.806626147372
+Minimizing 375    Time: 0:00:02 ( 6.88 ms/it)
+  objective:  13547.092330697484
+Minimizing 390    Time: 0:00:02 ( 6.88 ms/it)
+  objective:  13532.911687696724
+Minimizing 405    Time: 0:00:02 ( 6.88 ms/it)
+  objective:  13520.872533328955
+Minimizing 420    Time: 0:00:02 ( 6.89 ms/it)
+  objective:  13518.365460546862
+Minimizing 435    Time: 0:00:02 ( 6.89 ms/it)
+  objective:  13515.797058975942
+Minimizing 450    Time: 0:00:03 ( 6.89 ms/it)
+  objective:  13514.065460224185
+Minimizing 465    Time: 0:00:03 ( 6.89 ms/it)
+  objective:  13502.387613892985
+Minimizing 480    Time: 0:00:03 ( 6.89 ms/it)
+  objective:  13501.371486198099
+Minimizing 495    Time: 0:00:03 ( 6.89 ms/it)
+  objective:  13495.650745079183
+Minimizing 510    Time: 0:00:03 ( 6.89 ms/it)
+  objective:  13493.574736526003
+Minimizing 525    Time: 0:00:03 ( 6.89 ms/it)
+  objective:  13487.24491238776
+Minimizing 540    Time: 0:00:03 ( 6.89 ms/it)
+  objective:  13479.970415348675
+Minimizing 555    Time: 0:00:03 ( 6.89 ms/it)
+  objective:  13467.40530566145
+Minimizing 570    Time: 0:00:03 ( 6.89 ms/it)
+  objective:  13466.37840979021
+Minimizing 585    Time: 0:00:04 ( 6.88 ms/it)
+  objective:  13444.808014244067
+Minimizing 600    Time: 0:00:04 ( 6.88 ms/it)
+  objective:  13416.153199135573
+Minimizing 615    Time: 0:00:04 ( 6.88 ms/it)
+  objective:  13412.752158354255
+Minimizing 630    Time: 0:00:04 ( 6.88 ms/it)
+  objective:  13403.054644450516
+Minimizing 645    Time: 0:00:04 ( 6.88 ms/it)
+  objective:  13401.161340292405
+Minimizing 660    Time: 0:00:04 ( 6.88 ms/it)
+  objective:  13399.522092879924
+Minimizing 675    Time: 0:00:04 ( 6.88 ms/it)
+  objective:  13395.327109975005
+Minimizing 690    Time: 0:00:04 ( 6.88 ms/it)
+  objective:  13391.840157092141
+Minimizing 705    Time: 0:00:04 ( 6.88 ms/it)
+  objective:  13387.823883136785
+Minimizing 720    Time: 0:00:04 ( 6.88 ms/it)
+  objective:  13384.034588419112
+Minimizing 737    Time: 0:00:05 ( 6.90 ms/it)
+  objective:  13376.996328404828
+Minimizing 754    Time: 0:00:05 ( 6.89 ms/it)
+  objective:  13366.136746271266
+Minimizing 769    Time: 0:00:05 ( 6.89 ms/it)
+  objective:  13331.19146440213
+Minimizing 784    Time: 0:00:05 ( 6.89 ms/it)
+  objective:  13322.826223245342
+Minimizing 799    Time: 0:00:05 ( 6.89 ms/it)
+  objective:  13294.31975397146
+Minimizing 814    Time: 0:00:05 ( 6.89 ms/it)
+  objective:  13291.596260652514
+Minimizing 829    Time: 0:00:05 ( 6.88 ms/it)
+  objective:  13279.664025394144
+Minimizing 844    Time: 0:00:05 ( 6.88 ms/it)
+  objective:  13278.088882631753
+Minimizing 859    Time: 0:00:05 ( 6.88 ms/it)
+  objective:  13276.429779799728
+Minimizing 874    Time: 0:00:06 ( 6.88 ms/it)
+  objective:  13275.05837348754
+Minimizing 889    Time: 0:00:06 ( 6.88 ms/it)
+  objective:  13268.619728254474
+Minimizing 904    Time: 0:00:06 ( 6.88 ms/it)
+  objective:  13267.842004174803
+Minimizing 919    Time: 0:00:06 ( 6.88 ms/it)
+  objective:  13262.838248273736
+Minimizing 934    Time: 0:00:06 ( 6.88 ms/it)
+  objective:  13260.90469525801
+Minimizing 949    Time: 0:00:06 ( 6.88 ms/it)
+  objective:  13260.588538399563
+Minimizing 964    Time: 0:00:06 ( 6.88 ms/it)
+  objective:  13260.523629822099
+Minimizing 979    Time: 0:00:06 ( 6.88 ms/it)
+  objective:  13259.208036389507
+Minimizing 993    Time: 0:00:06 ( 6.89 ms/it)
+  objective:  13258.201803771197
+Minimizing 1007    Time: 0:00:06 ( 6.90 ms/it)
+  objective:  13256.83947944238
+Minimizing 1021    Time: 0:00:07 ( 6.91 ms/it)
+  objective:  13256.705809842271
+Minimizing 1035    Time: 0:00:07 ( 6.91 ms/it)
+  objective:  13255.863057219874
+Minimizing 1049    Time: 0:00:07 ( 6.92 ms/it)
+  objective:  13255.736158535015
+Minimizing 1063    Time: 0:00:07 ( 6.92 ms/it)
+  objective:  13255.63587310417
+Minimizing 1077    Time: 0:00:07 ( 6.93 ms/it)
+  objective:  13255.519203594042
+Minimizing 1091    Time: 0:00:07 ( 6.93 ms/it)
+  objective:  13255.389298437833
+Minimizing 1105    Time: 0:00:07 ( 6.93 ms/it)
+  objective:  13255.118205318227
+Minimizing 1119    Time: 0:00:07 ( 6.94 ms/it)
+  objective:  13255.088864998266
+Minimizing 1133    Time: 0:00:07 ( 6.94 ms/it)
+  objective:  13255.062787202129
+Minimizing 1147    Time: 0:00:07 ( 6.95 ms/it)
+  objective:  13255.061423943407
+Minimizing 1161    Time: 0:00:08 ( 6.95 ms/it)
+  objective:  13255.056097710563
+Minimizing 1175    Time: 0:00:08 ( 6.95 ms/it)
+  objective:  13255.056234650416
+Minimizing 1190    Time: 0:00:08 ( 6.96 ms/it)
+  objective:  13255.055002824854
+Minimizing 1204    Time: 0:00:08 ( 6.96 ms/it)
+  objective:  13255.054838904995
+Minimizing 1218    Time: 0:00:08 ( 6.97 ms/it)
+  objective:  13255.05491801395
+Minimizing 1232    Time: 0:00:08 ( 6.98 ms/it)
+  objective:  13255.054799779056
+Minimizing 1246    Time: 0:00:08 ( 6.98 ms/it)
+  objective:  13255.054791186776
+Minimizing 1259    Time: 0:00:08 ( 6.99 ms/it)
+  objective:  13255.054784246706
+Minimizing 1273    Time: 0:00:08 ( 7.00 ms/it)
+  objective:  13255.054746274604
+Minimizing 1287    Time: 0:00:09 ( 7.00 ms/it)
+  objective:  13255.054704167065
+Minimizing 1301    Time: 0:00:09 ( 7.01 ms/it)
+  objective:  13255.054710118318
+Minimizing 1315    Time: 0:00:09 ( 7.02 ms/it)
+  objective:  13255.054771622381
+Minimizing 1329    Time: 0:00:09 ( 7.02 ms/it)
+  objective:  13255.05471708742
+Minimizing 1343    Time: 0:00:09 ( 7.02 ms/it)
+  objective:  13255.054834318915
+Minimizing 1357    Time: 0:00:09 ( 7.03 ms/it)
+  objective:  13255.054715139035
+Minimizing 1371    Time: 0:00:09 ( 7.03 ms/it)
+  objective:  13255.054669005374
+Minimizing 1385    Time: 0:00:09 ( 7.04 ms/it)
+  objective:  13255.054763583583
+Minimizing 1399    Time: 0:00:09 ( 7.04 ms/it)
+  objective:  13255.054697954416
+Minimizing 1403    Time: 0:00:09 ( 7.04 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01590.0141-1.130.25940.6435
Test: c5.10.02450.03010.810.41600.9323
Test: c234.150.00590.03120.190.84880.5608
Test: c12.340.00130.02850.040.96430.4152
Test: c3.4-0.00600.0313-0.190.84821.1259
a10.31250.04826.49<1e-10
Sex: Boys0.19730.014114.03<1e-43
Test: c5.1 & a1-0.38060.1023-3.720.0002
Test: c234.15 & a1-0.11200.1065-1.050.2927
Test: c12.34 & a1-0.15090.0985-1.530.1256
Test: c3.4 & a10.06590.10710.620.5385
Test: c5.1 & Sex: Boys-0.14150.0301-4.70<1e-05
Test: c234.15 & Sex: Boys0.13750.03124.41<1e-04
Test: c12.34 & Sex: Boys-0.05720.0285-2.010.0448
Test: c3.4 & Sex: Boys-0.01710.0313-0.540.5863
a1 & Sex: Boys0.04910.04821.020.3085
Test: c5.1 & a1 & Sex: Boys-0.04980.1023-0.490.6263
Test: c234.15 & a1 & Sex: Boys0.05100.10650.480.6317
Test: c12.34 & a1 & Sex: Boys0.02350.09850.240.8118
Test: c3.4 & a1 & Sex: Boys0.07160.10710.670.5040
Residual0.0001
+
+
+
+
VarCorr(m_cpx_1_PC_2)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Child(Intercept)0.4141513100.643545888
Test: c5.10.8691735710.932294788+0.26
Test: c234.150.3144512820.560759558-0.08-0.41
Test: c12.340.1724273190.415243686+0.93+0.54-0.04
Test: c3.41.2676087831.125881336-0.07+0.26-0.97-0.15
Residual0.0000000100.000101193
+
+
+
+
m_cpx_1_PC_2.PCA
+
+
(Child = 
+Principal components based on correlation matrix
+ (Intercept)     1.0     .      .      .      .
+ Test: c5.1      0.26   1.0     .      .      .
+ Test: c234.15  -0.08  -0.41   1.0     .      .
+ Test: c12.34    0.93   0.54  -0.04   1.0     .
+ Test: c3.4     -0.07   0.26  -0.97  -0.15   1.0
+
+Normalized cumulative variances:
+[0.4658, 0.8678, 0.9987, 1.0, 1.0]
+
+Component loadings
+                  PC1    PC2    PC3    PC4    PC5
+ (Intercept)    -0.45   0.43   0.49   0.03   0.61
+ Test: c5.1     -0.49   0.01  -0.82  -0.05   0.29
+ Test: c234.15   0.45   0.51  -0.17  -0.7    0.14
+ Test: c12.34   -0.49   0.47   0.05  -0.13  -0.72
+ Test: c3.4     -0.34  -0.58   0.24  -0.7   -0.0,)
+
+
+
+
f_zcp_1 = @formula(zScore ~ 1 + Test*a1*Sex + zerocorr(1 + Test | Child))
+m_zcp_1_PC_2 = fit(MixedModel, f_zcp_1, dat; contrasts=contr4b)
+
+
Minimizing 21    Time: 0:00:00 ( 4.89 ms/it)
+  objective:  13783.233343890173
+Minimizing 42    Time: 0:00:00 ( 4.91 ms/it)
+  objective:  13681.464929881493
+Minimizing 62    Time: 0:00:00 ( 4.96 ms/it)
+  objective:  13668.548336113578
+Minimizing 83    Time: 0:00:00 ( 4.95 ms/it)
+  objective:  13661.29491583141
+Minimizing 104    Time: 0:00:00 ( 4.95 ms/it)
+  objective:  13639.038750316176
+Minimizing 125    Time: 0:00:00 ( 4.94 ms/it)
+  objective:  13625.056668479876
+Minimizing 145    Time: 0:00:00 ( 4.95 ms/it)
+  objective:  13621.198908831146
+Minimizing 166    Time: 0:00:00 ( 4.95 ms/it)
+  objective:  13611.319217932381
+Minimizing 187    Time: 0:00:00 ( 4.95 ms/it)
+  objective:  13608.36643724149
+Minimizing 208    Time: 0:00:01 ( 4.95 ms/it)
+  objective:  13597.737356090172
+Minimizing 229    Time: 0:00:01 ( 4.95 ms/it)
+  objective:  13546.149390347673
+Minimizing 250    Time: 0:00:01 ( 4.95 ms/it)
+  objective:  13508.091954147378
+Minimizing 271    Time: 0:00:01 ( 4.94 ms/it)
+  objective:  13448.258506957267
+Minimizing 292    Time: 0:00:01 ( 4.93 ms/it)
+  objective:  13441.56761418299
+Minimizing 313    Time: 0:00:01 ( 4.93 ms/it)
+  objective:  13438.392783886804
+Minimizing 334    Time: 0:00:01 ( 4.93 ms/it)
+  objective:  13432.59101495325
+Minimizing 355    Time: 0:00:01 ( 4.93 ms/it)
+  objective:  13423.460288299873
+Minimizing 377    Time: 0:00:01 ( 4.92 ms/it)
+  objective:  13354.965375328546
+Minimizing 398    Time: 0:00:01 ( 4.92 ms/it)
+  objective:  13266.235515146866
+Minimizing 419    Time: 0:00:02 ( 4.92 ms/it)
+  objective:  13225.004241497227
+Minimizing 440    Time: 0:00:02 ( 4.92 ms/it)
+  objective:  13143.428498894005
+Minimizing 461    Time: 0:00:02 ( 4.92 ms/it)
+  objective:  13091.28040350812
+Minimizing 482    Time: 0:00:02 ( 4.91 ms/it)
+  objective:  13058.26570571358
+Minimizing 505    Time: 0:00:02 ( 4.94 ms/it)
+  objective:  13005.383286547469
+Minimizing 526    Time: 0:00:02 ( 4.94 ms/it)
+  objective:  13005.287061735376
+Minimizing 547    Time: 0:00:02 ( 4.93 ms/it)
+  objective:  13007.14799122524
+Minimizing 568    Time: 0:00:02 ( 4.93 ms/it)
+  objective:  13006.541079391318
+Minimizing 589    Time: 0:00:02 ( 4.93 ms/it)
+  objective:  13004.64820935615
+Minimizing 607    Time: 0:00:02 ( 4.93 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01720.0142-1.210.22480.7313
Test: c5.10.02260.03050.740.45900.6944
Test: c234.150.00300.03040.100.92190.3699
Test: c12.340.01030.03180.330.74450.9437
Test: c3.4-0.00840.0317-0.260.79160.7067
a10.31290.04886.42<1e-09
Sex: Boys0.19660.014213.84<1e-42
Test: c5.1 & a1-0.38170.1032-3.700.0002
Test: c234.15 & a1-0.10790.1038-1.040.2989
Test: c12.34 & a1-0.13520.1094-1.240.2168
Test: c3.4 & a10.08580.10850.790.4290
Test: c5.1 & Sex: Boys-0.14580.0305-4.78<1e-05
Test: c234.15 & Sex: Boys0.13590.03044.47<1e-05
Test: c12.34 & Sex: Boys-0.03680.0318-1.160.2470
Test: c3.4 & Sex: Boys-0.01880.0317-0.590.5545
a1 & Sex: Boys0.04530.04880.930.3530
Test: c5.1 & a1 & Sex: Boys-0.04760.1032-0.460.6448
Test: c234.15 & a1 & Sex: Boys0.03870.10380.370.7095
Test: c12.34 & a1 & Sex: Boys0.05340.10940.490.6259
Test: c3.4 & a1 & Sex: Boys0.06900.10850.640.5250
Residual0.0000
+
+
+
+
VarCorr(m_zcp_1_PC_2)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Child(Intercept)0.534755230.73126960
Test: c5.10.482257290.69444747.
Test: c234.150.136840260.36991926..
Test: c12.340.890573740.94370214...
Test: c3.40.499463930.70672762....
Residual0.000000000.00000085
+
+
+
+
MixedModels.likelihoodratiotest(m_zcp_1_PC_2, m_cpx_1_PC_2)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Child)2613004
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 + Test | Child)3613255-25110NaN
+
+
+
+
+
+

3 Other topics

+
+

3.1 Contrasts are re-parameterizations of the same model

+

The choice of contrast does not affect the model objective, in other words, they all yield the same goodness of fit. It does not matter whether a contrast is orthogonal or not.

+
+
[
+  objective(m_ovi_SeqDiff),
+  objective(m_ovi_Helmert),
+  objective(m_ovi_Hypo),
+]
+
+
3-element Vector{Float64}:
+ 13810.00650113982
+ 13810.006501139793
+ 13810.006501139791
+
+
+
+
+

3.2 VCs and CPs depend on contrast coding

+

Trivially, the meaning of a contrast depends on its definition. Consequently, the contrast specification has a big effect on the random-effect structure. As an illustration, we refit the LMMs with variance components (VCs) and correlation parameters (CPs) for Child-related contrasts of Test. Unfortunately, it is not easy, actually rather quite difficult, to grasp the meaning of correlations of contrast-based effects; they represent two-way interactions.

+
+
begin
+  f_Child = @formula zScore ~
+    1 + Test * a1 * Sex + (1 + Test | Child)
+  m_Child_SDC = fit(MixedModel, f_Child, dat; contrasts=contr1)
+  m_Child_HeC = fit(MixedModel, f_Child, dat; contrasts=contr2)
+  m_Child_HyC = fit(MixedModel, f_Child, dat; contrasts=contr3)
+  m_Child_PCA = fit(MixedModel, f_Child, dat; contrasts=contr4)
+end
+
+
Minimizing 14    Time: 0:00:00 ( 7.38 ms/it)
+  objective:  13864.841025986228
+Minimizing 28    Time: 0:00:00 ( 7.40 ms/it)
+  objective:  13845.893578754876
+Minimizing 42    Time: 0:00:00 ( 7.36 ms/it)
+  objective:  13779.642766649653
+Minimizing 56    Time: 0:00:00 ( 7.43 ms/it)
+  objective:  13774.420150810312
+Minimizing 70    Time: 0:00:00 ( 7.42 ms/it)
+  objective:  13763.255128405614
+Minimizing 84    Time: 0:00:00 ( 7.46 ms/it)
+  objective:  13758.347505903877
+Minimizing 99    Time: 0:00:00 ( 7.41 ms/it)
+  objective:  13756.880079794735
+Minimizing 114    Time: 0:00:00 ( 7.38 ms/it)
+  objective:  13748.769476413026
+Minimizing 128    Time: 0:00:00 ( 7.36 ms/it)
+  objective:  13745.93833595192
+Minimizing 142    Time: 0:00:01 ( 7.36 ms/it)
+  objective:  13719.337037060759
+Minimizing 156    Time: 0:00:01 ( 7.35 ms/it)
+  objective:  13713.02271569964
+Minimizing 171    Time: 0:00:01 ( 7.33 ms/it)
+  objective:  13710.233015094
+Minimizing 185    Time: 0:00:01 ( 7.33 ms/it)
+  objective:  13705.921017381152
+Minimizing 199    Time: 0:00:01 ( 7.32 ms/it)
+  objective:  13703.845507302389
+Minimizing 213    Time: 0:00:01 ( 7.32 ms/it)
+  objective:  13701.708647031715
+Minimizing 227    Time: 0:00:01 ( 7.31 ms/it)
+  objective:  13699.71787171471
+Minimizing 241    Time: 0:00:01 ( 7.31 ms/it)
+  objective:  13690.619306364151
+Minimizing 255    Time: 0:00:01 ( 7.31 ms/it)
+  objective:  13690.29357377909
+Minimizing 269    Time: 0:00:01 ( 7.31 ms/it)
+  objective:  13687.474526688906
+Minimizing 284    Time: 0:00:02 ( 7.30 ms/it)
+  objective:  13687.04984767899
+Minimizing 298    Time: 0:00:02 ( 7.30 ms/it)
+  objective:  13686.375572547107
+Minimizing 312    Time: 0:00:02 ( 7.30 ms/it)
+  objective:  13685.048629360046
+Minimizing 326    Time: 0:00:02 ( 7.31 ms/it)
+  objective:  13683.45090568854
+Minimizing 340    Time: 0:00:02 ( 7.33 ms/it)
+  objective:  13670.28900574682
+Minimizing 354    Time: 0:00:02 ( 7.33 ms/it)
+  objective:  13666.130497207325
+Minimizing 368    Time: 0:00:02 ( 7.34 ms/it)
+  objective:  13663.176077165926
+Minimizing 382    Time: 0:00:02 ( 7.33 ms/it)
+  objective:  13657.89791477644
+Minimizing 396    Time: 0:00:02 ( 7.33 ms/it)
+  objective:  13653.787699088523
+Minimizing 410    Time: 0:00:03 ( 7.34 ms/it)
+  objective:  13651.978565149546
+Minimizing 424    Time: 0:00:03 ( 7.34 ms/it)
+  objective:  13650.257119077532
+Minimizing 438    Time: 0:00:03 ( 7.35 ms/it)
+  objective:  13596.718990553814
+Minimizing 452    Time: 0:00:03 ( 7.35 ms/it)
+  objective:  13595.870964499867
+Minimizing 466    Time: 0:00:03 ( 7.35 ms/it)
+  objective:  13587.115499796888
+Minimizing 480    Time: 0:00:03 ( 7.36 ms/it)
+  objective:  13583.500285415314
+Minimizing 494    Time: 0:00:03 ( 7.36 ms/it)
+  objective:  13578.21380550712
+Minimizing 508    Time: 0:00:03 ( 7.36 ms/it)
+  objective:  13573.672392208908
+Minimizing 522    Time: 0:00:03 ( 7.36 ms/it)
+  objective:  13561.744968132942
+Minimizing 536    Time: 0:00:03 ( 7.36 ms/it)
+  objective:  13556.121370537869
+Minimizing 550    Time: 0:00:04 ( 7.36 ms/it)
+  objective:  13536.968713369119
+Minimizing 564    Time: 0:00:04 ( 7.35 ms/it)
+  objective:  13525.032867750167
+Minimizing 578    Time: 0:00:04 ( 7.36 ms/it)
+  objective:  13524.904133885815
+Minimizing 592    Time: 0:00:04 ( 7.35 ms/it)
+  objective:  13523.363880581528
+Minimizing 606    Time: 0:00:04 ( 7.35 ms/it)
+  objective:  13521.632787209572
+Minimizing 621    Time: 0:00:04 ( 7.34 ms/it)
+  objective:  13520.78772943154
+Minimizing 635    Time: 0:00:04 ( 7.34 ms/it)
+  objective:  13518.695643430212
+Minimizing 649    Time: 0:00:04 ( 7.35 ms/it)
+  objective:  13517.036591524637
+Minimizing 663    Time: 0:00:04 ( 7.35 ms/it)
+  objective:  13510.7797885588
+Minimizing 677    Time: 0:00:04 ( 7.35 ms/it)
+  objective:  13510.176477523295
+Minimizing 691    Time: 0:00:05 ( 7.35 ms/it)
+  objective:  13506.246601124869
+Minimizing 705    Time: 0:00:05 ( 7.35 ms/it)
+  objective:  13503.56922521325
+Minimizing 719    Time: 0:00:05 ( 7.36 ms/it)
+  objective:  13502.614113315482
+Minimizing 732    Time: 0:00:05 ( 7.37 ms/it)
+  objective:  13497.016552788686
+Minimizing 745    Time: 0:00:05 ( 7.38 ms/it)
+  objective:  13489.782425035293
+Minimizing 757    Time: 0:00:05 ( 7.40 ms/it)
+  objective:  13488.94185046242
+Minimizing 770    Time: 0:00:05 ( 7.41 ms/it)
+  objective:  13488.627314452053
+Minimizing 783    Time: 0:00:05 ( 7.42 ms/it)
+  objective:  13487.64272385076
+Minimizing 797    Time: 0:00:05 ( 7.42 ms/it)
+  objective:  13485.080584581614
+Minimizing 811    Time: 0:00:06 ( 7.42 ms/it)
+  objective:  13484.637993051896
+Minimizing 825    Time: 0:00:06 ( 7.42 ms/it)
+  objective:  13482.7367410328
+Minimizing 839    Time: 0:00:06 ( 7.42 ms/it)
+  objective:  13482.481133163245
+Minimizing 853    Time: 0:00:06 ( 7.42 ms/it)
+  objective:  13480.919770174864
+Minimizing 867    Time: 0:00:06 ( 7.42 ms/it)
+  objective:  13479.338020027433
+Minimizing 881    Time: 0:00:06 ( 7.42 ms/it)
+  objective:  13478.268180507002
+Minimizing 895    Time: 0:00:06 ( 7.42 ms/it)
+  objective:  13474.89802321393
+Minimizing 910    Time: 0:00:06 ( 7.42 ms/it)
+  objective:  13474.818139830619
+Minimizing 924    Time: 0:00:06 ( 7.42 ms/it)
+  objective:  13473.899599287739
+Minimizing 938    Time: 0:00:06 ( 7.42 ms/it)
+  objective:  13472.991812527842
+Minimizing 951    Time: 0:00:07 ( 7.44 ms/it)
+  objective:  13472.554082196628
+Minimizing 964    Time: 0:00:07 ( 7.45 ms/it)
+  objective:  13468.538171214794
+Minimizing 977    Time: 0:00:07 ( 7.45 ms/it)
+  objective:  13467.639716883765
+Minimizing 992    Time: 0:00:07 ( 7.45 ms/it)
+  objective:  13467.578593657796
+Minimizing 1007    Time: 0:00:07 ( 7.44 ms/it)
+  objective:  13466.23114230048
+Minimizing 1022    Time: 0:00:07 ( 7.43 ms/it)
+  objective:  13464.516615018845
+Minimizing 1036    Time: 0:00:07 ( 7.43 ms/it)
+  objective:  13463.980091173093
+Minimizing 1050    Time: 0:00:07 ( 7.43 ms/it)
+  objective:  13462.773479719443
+Minimizing 1065    Time: 0:00:07 ( 7.43 ms/it)
+  objective:  13462.216706877145
+Minimizing 1080    Time: 0:00:08 ( 7.42 ms/it)
+  objective:  13461.358939771475
+Minimizing 1095    Time: 0:00:08 ( 7.41 ms/it)
+  objective:  13458.882016874835
+Minimizing 1110    Time: 0:00:08 ( 7.41 ms/it)
+  objective:  13458.617667955717
+Minimizing 1125    Time: 0:00:08 ( 7.40 ms/it)
+  objective:  13457.543642033306
+Minimizing 1140    Time: 0:00:08 ( 7.39 ms/it)
+  objective:  13456.274499247207
+Minimizing 1155    Time: 0:00:08 ( 7.39 ms/it)
+  objective:  13454.019270400633
+Minimizing 1170    Time: 0:00:08 ( 7.39 ms/it)
+  objective:  13453.999404741633
+Minimizing 1185    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13453.081996300418
+Minimizing 1200    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13451.84992165603
+Minimizing 1215    Time: 0:00:08 ( 7.37 ms/it)
+  objective:  13451.461555001071
+Minimizing 1230    Time: 0:00:09 ( 7.36 ms/it)
+  objective:  13450.057680870392
+Minimizing 1245    Time: 0:00:09 ( 7.36 ms/it)
+  objective:  13448.822191192878
+Minimizing 1260    Time: 0:00:09 ( 7.35 ms/it)
+  objective:  13445.389702814973
+Minimizing 1275    Time: 0:00:09 ( 7.35 ms/it)
+  objective:  13428.298074435195
+Minimizing 1290    Time: 0:00:09 ( 7.35 ms/it)
+  objective:  13419.14473015425
+Minimizing 1305    Time: 0:00:09 ( 7.34 ms/it)
+  objective:  13414.257278873745
+Minimizing 1320    Time: 0:00:09 ( 7.34 ms/it)
+  objective:  13403.679148710464
+Minimizing 1335    Time: 0:00:09 ( 7.33 ms/it)
+  objective:  13396.232708204952
+Minimizing 1350    Time: 0:00:09 ( 7.33 ms/it)
+  objective:  13350.927349250676
+Minimizing 1365    Time: 0:00:09 ( 7.32 ms/it)
+  objective:  13339.274454825776
+Minimizing 1380    Time: 0:00:10 ( 7.32 ms/it)
+  objective:  13335.342672513361
+Minimizing 1395    Time: 0:00:10 ( 7.32 ms/it)
+  objective:  13329.844363219323
+Minimizing 1410    Time: 0:00:10 ( 7.32 ms/it)
+  objective:  13318.587893036965
+Minimizing 1425    Time: 0:00:10 ( 7.31 ms/it)
+  objective:  13318.483025400827
+Minimizing 1440    Time: 0:00:10 ( 7.31 ms/it)
+  objective:  13317.603252419402
+Minimizing 1455    Time: 0:00:10 ( 7.31 ms/it)
+  objective:  13316.97518029595
+Minimizing 1470    Time: 0:00:10 ( 7.30 ms/it)
+  objective:  13316.072121185236
+Minimizing 1485    Time: 0:00:10 ( 7.30 ms/it)
+  objective:  13315.359104246527
+Minimizing 1500    Time: 0:00:10 ( 7.30 ms/it)
+  objective:  13313.924189835161
+Minimizing 1515    Time: 0:00:11 ( 7.29 ms/it)
+  objective:  13313.797955305534
+Minimizing 1530    Time: 0:00:11 ( 7.29 ms/it)
+  objective:  13311.316728242207
+Minimizing 1545    Time: 0:00:11 ( 7.29 ms/it)
+  objective:  13311.254955979413
+Minimizing 1560    Time: 0:00:11 ( 7.29 ms/it)
+  objective:  13310.752448072686
+Minimizing 1575    Time: 0:00:11 ( 7.28 ms/it)
+  objective:  13310.263050807902
+Minimizing 1590    Time: 0:00:11 ( 7.28 ms/it)
+  objective:  13309.18452747412
+Minimizing 1605    Time: 0:00:11 ( 7.28 ms/it)
+  objective:  13307.424914636562
+Minimizing 1620    Time: 0:00:11 ( 7.27 ms/it)
+  objective:  13306.590932389605
+Minimizing 1635    Time: 0:00:11 ( 7.27 ms/it)
+  objective:  13304.340360901668
+Minimizing 1649    Time: 0:00:11 ( 7.27 ms/it)
+  objective:  13304.231132097222
+Minimizing 1664    Time: 0:00:12 ( 7.27 ms/it)
+  objective:  13304.038819103182
+Minimizing 1679    Time: 0:00:12 ( 7.27 ms/it)
+  objective:  13303.680926864385
+Minimizing 1694    Time: 0:00:12 ( 7.26 ms/it)
+  objective:  13303.419516689974
+Minimizing 1709    Time: 0:00:12 ( 7.26 ms/it)
+  objective:  13302.681011149776
+Minimizing 1724    Time: 0:00:12 ( 7.26 ms/it)
+  objective:  13302.011843218294
+Minimizing 1739    Time: 0:00:12 ( 7.26 ms/it)
+  objective:  13301.443772030732
+Minimizing 1754    Time: 0:00:12 ( 7.25 ms/it)
+  objective:  13300.149896853356
+Minimizing 1769    Time: 0:00:12 ( 7.25 ms/it)
+  objective:  13299.8611757061
+Minimizing 1784    Time: 0:00:12 ( 7.25 ms/it)
+  objective:  13299.58302719945
+Minimizing 1798    Time: 0:00:13 ( 7.25 ms/it)
+  objective:  13299.122255901151
+Minimizing 1813    Time: 0:00:13 ( 7.25 ms/it)
+  objective:  13298.51099639821
+Minimizing 1828    Time: 0:00:13 ( 7.25 ms/it)
+  objective:  13297.229667072825
+Minimizing 1843    Time: 0:00:13 ( 7.25 ms/it)
+  objective:  13296.744068662185
+Minimizing 1858    Time: 0:00:13 ( 7.25 ms/it)
+  objective:  13295.195782863346
+Minimizing 1873    Time: 0:00:13 ( 7.24 ms/it)
+  objective:  13294.97360063158
+Minimizing 1888    Time: 0:00:13 ( 7.24 ms/it)
+  objective:  13294.294746616506
+Minimizing 1903    Time: 0:00:13 ( 7.24 ms/it)
+  objective:  13292.943878194666
+Minimizing 1918    Time: 0:00:13 ( 7.24 ms/it)
+  objective:  13292.491278275134
+Minimizing 1933    Time: 0:00:13 ( 7.23 ms/it)
+  objective:  13292.397969650163
+Minimizing 1947    Time: 0:00:14 ( 7.23 ms/it)
+  objective:  13291.894899531384
+Minimizing 1962    Time: 0:00:14 ( 7.23 ms/it)
+  objective:  13291.356819490858
+Minimizing 1977    Time: 0:00:14 ( 7.23 ms/it)
+  objective:  13290.752636017758
+Minimizing 1992    Time: 0:00:14 ( 7.23 ms/it)
+  objective:  13289.566208013915
+Minimizing 2007    Time: 0:00:14 ( 7.23 ms/it)
+  objective:  13288.537436593717
+Minimizing 2022    Time: 0:00:14 ( 7.23 ms/it)
+  objective:  13288.037102696675
+Minimizing 2037    Time: 0:00:14 ( 7.22 ms/it)
+  objective:  13287.095374361612
+Minimizing 2052    Time: 0:00:14 ( 7.22 ms/it)
+  objective:  13286.236736689549
+Minimizing 2067    Time: 0:00:14 ( 7.22 ms/it)
+  objective:  13284.47508208947
+Minimizing 2082    Time: 0:00:15 ( 7.22 ms/it)
+  objective:  13282.954953712266
+Minimizing 2097    Time: 0:00:15 ( 7.22 ms/it)
+  objective:  13279.51512109107
+Minimizing 2112    Time: 0:00:15 ( 7.22 ms/it)
+  objective:  13275.962618584832
+Minimizing 2126    Time: 0:00:15 ( 7.22 ms/it)
+  objective:  13265.934409907422
+Minimizing 2140    Time: 0:00:15 ( 7.22 ms/it)
+  objective:  13254.338813870418
+Minimizing 2155    Time: 0:00:15 ( 7.21 ms/it)
+  objective:  13252.290857140033
+Minimizing 2170    Time: 0:00:15 ( 7.21 ms/it)
+  objective:  13250.185037324627
+Minimizing 2185    Time: 0:00:15 ( 7.21 ms/it)
+  objective:  13244.551926471962
+Minimizing 2200    Time: 0:00:15 ( 7.21 ms/it)
+  objective:  13238.150409325623
+Minimizing 2217    Time: 0:00:15 ( 7.21 ms/it)
+  objective:  13232.850150665225
+Minimizing 2233    Time: 0:00:16 ( 7.21 ms/it)
+  objective:  13226.338353252722
+Minimizing 2248    Time: 0:00:16 ( 7.21 ms/it)
+  objective:  13223.438841768424
+Minimizing 2262    Time: 0:00:16 ( 7.21 ms/it)
+  objective:  13218.113302238213
+Minimizing 2276    Time: 0:00:16 ( 7.21 ms/it)
+  objective:  13217.731166378071
+Minimizing 2291    Time: 0:00:16 ( 7.21 ms/it)
+  objective:  13214.768506554843
+Minimizing 2306    Time: 0:00:16 ( 7.21 ms/it)
+  objective:  13211.085951421177
+Minimizing 2321    Time: 0:00:16 ( 7.21 ms/it)
+  objective:  13201.440930079378
+Minimizing 2336    Time: 0:00:16 ( 7.21 ms/it)
+  objective:  13196.688185621882
+Minimizing 2351    Time: 0:00:16 ( 7.21 ms/it)
+  objective:  13188.13659060486
+Minimizing 2366    Time: 0:00:17 ( 7.21 ms/it)
+  objective:  13180.288196244132
+Minimizing 2381    Time: 0:00:17 ( 7.20 ms/it)
+  objective:  13164.871640060272
+Minimizing 2396    Time: 0:00:17 ( 7.20 ms/it)
+  objective:  13160.4412493243
+Minimizing 2411    Time: 0:00:17 ( 7.20 ms/it)
+  objective:  13146.791395771419
+Minimizing 2425    Time: 0:00:17 ( 7.20 ms/it)
+  objective:  13144.898178043746
+Minimizing 2440    Time: 0:00:17 ( 7.20 ms/it)
+  objective:  13136.092751673175
+Minimizing 2455    Time: 0:00:17 ( 7.20 ms/it)
+  objective:  13112.94757467517
+Minimizing 2470    Time: 0:00:17 ( 7.20 ms/it)
+  objective:  13109.07036788974
+Minimizing 2485    Time: 0:00:17 ( 7.20 ms/it)
+  objective:  13109.041576801654
+Minimizing 2500    Time: 0:00:18 ( 7.20 ms/it)
+  objective:  13109.001231908522
+Minimizing 2515    Time: 0:00:18 ( 7.20 ms/it)
+  objective:  13109.015337759716
+Minimizing 2530    Time: 0:00:18 ( 7.20 ms/it)
+  objective:  13109.034734655841
+Minimizing 2545    Time: 0:00:18 ( 7.20 ms/it)
+  objective:  13109.038868225558
+Minimizing 2560    Time: 0:00:18 ( 7.20 ms/it)
+  objective:  13109.061958145263
+Minimizing 2575    Time: 0:00:18 ( 7.20 ms/it)
+  objective:  13109.01477411139
+Minimizing 2589    Time: 0:00:18 ( 7.20 ms/it)
+  objective:  13109.003292939946
+Minimizing 2604    Time: 0:00:18 ( 7.20 ms/it)
+  objective:  13109.042902489833
+Minimizing 2618    Time: 0:00:18 ( 7.20 ms/it)
+  objective:  13109.006932114848
+Minimizing 2633    Time: 0:00:18 ( 7.20 ms/it)
+  objective:  13109.006471395347
+Minimizing 2648    Time: 0:00:19 ( 7.20 ms/it)
+  objective:  13109.009539637394
+Minimizing 2663    Time: 0:00:19 ( 7.20 ms/it)
+  objective:  13109.003172373152
+Minimizing 2678    Time: 0:00:19 ( 7.19 ms/it)
+  objective:  13109.006089056216
+Minimizing 2690    Time: 0:00:19 ( 7.19 ms/it)
+Minimizing 15    Time: 0:00:00 ( 7.03 ms/it)
+  objective:  14484.818769769932
+Minimizing 30    Time: 0:00:00 ( 7.01 ms/it)
+  objective:  15149.245792406973
+Minimizing 45    Time: 0:00:00 ( 7.04 ms/it)
+  objective:  13769.912489896697
+Minimizing 59    Time: 0:00:00 ( 7.15 ms/it)
+  objective:  13761.74191805981
+Minimizing 73    Time: 0:00:00 ( 7.17 ms/it)
+  objective:  13758.810515322486
+Minimizing 87    Time: 0:00:00 ( 7.17 ms/it)
+  objective:  13747.89144631826
+Minimizing 102    Time: 0:00:00 ( 7.15 ms/it)
+  objective:  13738.014532122419
+Minimizing 117    Time: 0:00:00 ( 7.13 ms/it)
+  objective:  13731.266747706104
+Minimizing 132    Time: 0:00:00 ( 7.12 ms/it)
+  objective:  13724.618309431458
+Minimizing 147    Time: 0:00:01 ( 7.12 ms/it)
+  objective:  13654.800970509314
+Minimizing 162    Time: 0:00:01 ( 7.10 ms/it)
+  objective:  13653.23607155866
+Minimizing 177    Time: 0:00:01 ( 7.09 ms/it)
+  objective:  13647.684656378384
+Minimizing 192    Time: 0:00:01 ( 7.09 ms/it)
+  objective:  13644.340551829799
+Minimizing 207    Time: 0:00:01 ( 7.08 ms/it)
+  objective:  13639.157628992602
+Minimizing 222    Time: 0:00:01 ( 7.08 ms/it)
+  objective:  13637.212982312521
+Minimizing 236    Time: 0:00:01 ( 7.09 ms/it)
+  objective:  13631.420008504178
+Minimizing 250    Time: 0:00:01 ( 7.12 ms/it)
+  objective:  13627.76036240306
+Minimizing 264    Time: 0:00:01 ( 7.13 ms/it)
+  objective:  13618.13712888892
+Minimizing 278    Time: 0:00:01 ( 7.15 ms/it)
+  objective:  13602.718359392646
+Minimizing 292    Time: 0:00:02 ( 7.17 ms/it)
+  objective:  13594.795656013492
+Minimizing 306    Time: 0:00:02 ( 7.17 ms/it)
+  objective:  13592.296898878354
+Minimizing 320    Time: 0:00:02 ( 7.18 ms/it)
+  objective:  13589.80172428917
+Minimizing 334    Time: 0:00:02 ( 7.19 ms/it)
+  objective:  13587.694269563737
+Minimizing 348    Time: 0:00:02 ( 7.21 ms/it)
+  objective:  13586.165199506271
+Minimizing 362    Time: 0:00:02 ( 7.21 ms/it)
+  objective:  13583.971722038295
+Minimizing 376    Time: 0:00:02 ( 7.22 ms/it)
+  objective:  13579.163713327973
+Minimizing 390    Time: 0:00:02 ( 7.23 ms/it)
+  objective:  13572.566887500128
+Minimizing 404    Time: 0:00:02 ( 7.24 ms/it)
+  objective:  13571.997138661252
+Minimizing 418    Time: 0:00:03 ( 7.24 ms/it)
+  objective:  13569.486490841591
+Minimizing 431    Time: 0:00:03 ( 7.26 ms/it)
+  objective:  13568.964181688567
+Minimizing 445    Time: 0:00:03 ( 7.27 ms/it)
+  objective:  13567.440966488095
+Minimizing 459    Time: 0:00:03 ( 7.27 ms/it)
+  objective:  13566.783662910664
+Minimizing 473    Time: 0:00:03 ( 7.27 ms/it)
+  objective:  13565.013170825518
+Minimizing 487    Time: 0:00:03 ( 7.27 ms/it)
+  objective:  13562.508977192181
+Minimizing 501    Time: 0:00:03 ( 7.28 ms/it)
+  objective:  13554.218956510995
+Minimizing 515    Time: 0:00:03 ( 7.28 ms/it)
+  objective:  13553.000339553691
+Minimizing 529    Time: 0:00:03 ( 7.28 ms/it)
+  objective:  13549.8952039791
+Minimizing 543    Time: 0:00:03 ( 7.29 ms/it)
+  objective:  13546.692917706656
+Minimizing 557    Time: 0:00:04 ( 7.29 ms/it)
+  objective:  13545.77669800381
+Minimizing 571    Time: 0:00:04 ( 7.29 ms/it)
+  objective:  13541.315990069146
+Minimizing 585    Time: 0:00:04 ( 7.29 ms/it)
+  objective:  13537.015846784452
+Minimizing 599    Time: 0:00:04 ( 7.29 ms/it)
+  objective:  13537.941379972595
+Minimizing 613    Time: 0:00:04 ( 7.29 ms/it)
+  objective:  13526.733567803945
+Minimizing 627    Time: 0:00:04 ( 7.30 ms/it)
+  objective:  13517.545489108263
+Minimizing 641    Time: 0:00:04 ( 7.31 ms/it)
+  objective:  13512.95366684183
+Minimizing 655    Time: 0:00:04 ( 7.31 ms/it)
+  objective:  13508.438718048426
+Minimizing 669    Time: 0:00:04 ( 7.31 ms/it)
+  objective:  13507.36919131755
+Minimizing 683    Time: 0:00:04 ( 7.31 ms/it)
+  objective:  13503.892437927789
+Minimizing 697    Time: 0:00:05 ( 7.32 ms/it)
+  objective:  13500.878851006266
+Minimizing 711    Time: 0:00:05 ( 7.32 ms/it)
+  objective:  13500.540697427685
+Minimizing 725    Time: 0:00:05 ( 7.32 ms/it)
+  objective:  13499.647851582413
+Minimizing 739    Time: 0:00:05 ( 7.32 ms/it)
+  objective:  13497.31636822351
+Minimizing 753    Time: 0:00:05 ( 7.32 ms/it)
+  objective:  13496.74235689282
+Minimizing 767    Time: 0:00:05 ( 7.32 ms/it)
+  objective:  13495.236254695941
+Minimizing 781    Time: 0:00:05 ( 7.33 ms/it)
+  objective:  13494.460546471433
+Minimizing 794    Time: 0:00:05 ( 7.34 ms/it)
+  objective:  13493.116948996358
+Minimizing 808    Time: 0:00:05 ( 7.34 ms/it)
+  objective:  13491.55222243048
+Minimizing 821    Time: 0:00:06 ( 7.35 ms/it)
+  objective:  13485.298372171179
+Minimizing 834    Time: 0:00:06 ( 7.36 ms/it)
+  objective:  13484.846928479092
+Minimizing 848    Time: 0:00:06 ( 7.36 ms/it)
+  objective:  13481.798310757702
+Minimizing 862    Time: 0:00:06 ( 7.36 ms/it)
+  objective:  13481.54557346375
+Minimizing 876    Time: 0:00:06 ( 7.36 ms/it)
+  objective:  13479.242262358726
+Minimizing 890    Time: 0:00:06 ( 7.36 ms/it)
+  objective:  13477.776281089456
+Minimizing 904    Time: 0:00:06 ( 7.36 ms/it)
+  objective:  13471.293038345124
+Minimizing 918    Time: 0:00:06 ( 7.36 ms/it)
+  objective:  13470.261187016251
+Minimizing 932    Time: 0:00:06 ( 7.36 ms/it)
+  objective:  13462.889959603483
+Minimizing 946    Time: 0:00:06 ( 7.36 ms/it)
+  objective:  13458.245608738893
+Minimizing 960    Time: 0:00:07 ( 7.37 ms/it)
+  objective:  13448.82679411889
+Minimizing 974    Time: 0:00:07 ( 7.37 ms/it)
+  objective:  13446.907386952858
+Minimizing 988    Time: 0:00:07 ( 7.37 ms/it)
+  objective:  13445.662653153297
+Minimizing 1002    Time: 0:00:07 ( 7.37 ms/it)
+  objective:  13440.418761966415
+Minimizing 1016    Time: 0:00:07 ( 7.37 ms/it)
+  objective:  13436.471265100285
+Minimizing 1030    Time: 0:00:07 ( 7.37 ms/it)
+  objective:  13435.588412697325
+Minimizing 1044    Time: 0:00:07 ( 7.37 ms/it)
+  objective:  13431.482927769925
+Minimizing 1057    Time: 0:00:07 ( 7.38 ms/it)
+  objective:  13422.652037011016
+Minimizing 1071    Time: 0:00:07 ( 7.38 ms/it)
+  objective:  13403.243319638073
+Minimizing 1085    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13400.098319343466
+Minimizing 1099    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13392.128289924643
+Minimizing 1113    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13390.187741748465
+Minimizing 1127    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13388.262281691314
+Minimizing 1141    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13385.439643098529
+Minimizing 1155    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13380.929415013677
+Minimizing 1169    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13366.210191090278
+Minimizing 1183    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13353.708329809568
+Minimizing 1197    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13352.423984292298
+Minimizing 1211    Time: 0:00:08 ( 7.38 ms/it)
+  objective:  13350.44711648039
+Minimizing 1225    Time: 0:00:09 ( 7.38 ms/it)
+  objective:  13347.356347077643
+Minimizing 1239    Time: 0:00:09 ( 7.38 ms/it)
+  objective:  13345.241849598242
+Minimizing 1252    Time: 0:00:09 ( 7.38 ms/it)
+  objective:  13342.73638115912
+Minimizing 1266    Time: 0:00:09 ( 7.38 ms/it)
+  objective:  13340.971598195058
+Minimizing 1280    Time: 0:00:09 ( 7.39 ms/it)
+  objective:  13337.339740364754
+Minimizing 1294    Time: 0:00:09 ( 7.38 ms/it)
+  objective:  13336.211162891705
+Minimizing 1308    Time: 0:00:09 ( 7.38 ms/it)
+  objective:  13328.602816670158
+Minimizing 1322    Time: 0:00:09 ( 7.38 ms/it)
+  objective:  13328.241141181046
+Minimizing 1336    Time: 0:00:09 ( 7.38 ms/it)
+  objective:  13326.212511028396
+Minimizing 1351    Time: 0:00:09 ( 7.39 ms/it)
+  objective:  13325.046683382912
+Minimizing 1366    Time: 0:00:10 ( 7.39 ms/it)
+  objective:  13324.624948563855
+Minimizing 1380    Time: 0:00:10 ( 7.39 ms/it)
+  objective:  13323.218171707762
+Minimizing 1394    Time: 0:00:10 ( 7.39 ms/it)
+  objective:  13321.328471264103
+Minimizing 1408    Time: 0:00:10 ( 7.39 ms/it)
+  objective:  13320.5146919408
+Minimizing 1422    Time: 0:00:10 ( 7.39 ms/it)
+  objective:  13319.747185641492
+Minimizing 1436    Time: 0:00:10 ( 7.39 ms/it)
+  objective:  13317.807118322351
+Minimizing 1450    Time: 0:00:10 ( 7.39 ms/it)
+  objective:  13316.583428790953
+Minimizing 1463    Time: 0:00:10 ( 7.39 ms/it)
+  objective:  13316.369017908946
+Minimizing 1476    Time: 0:00:10 ( 7.40 ms/it)
+  objective:  13314.720825314682
+Minimizing 1490    Time: 0:00:11 ( 7.40 ms/it)
+  objective:  13313.942277059949
+Minimizing 1504    Time: 0:00:11 ( 7.40 ms/it)
+  objective:  13310.55964418828
+Minimizing 1518    Time: 0:00:11 ( 7.40 ms/it)
+  objective:  13308.084344920353
+Minimizing 1532    Time: 0:00:11 ( 7.40 ms/it)
+  objective:  13306.527777598953
+Minimizing 1546    Time: 0:00:11 ( 7.41 ms/it)
+  objective:  13305.292557824127
+Minimizing 1560    Time: 0:00:11 ( 7.41 ms/it)
+  objective:  13304.99530357393
+Minimizing 1574    Time: 0:00:11 ( 7.41 ms/it)
+  objective:  13304.21735900284
+Minimizing 1588    Time: 0:00:11 ( 7.41 ms/it)
+  objective:  13303.882313797236
+Minimizing 1602    Time: 0:00:11 ( 7.41 ms/it)
+  objective:  13300.523697426834
+Minimizing 1616    Time: 0:00:11 ( 7.41 ms/it)
+  objective:  13300.326373889926
+Minimizing 1630    Time: 0:00:12 ( 7.41 ms/it)
+  objective:  13300.099524684862
+Minimizing 1644    Time: 0:00:12 ( 7.41 ms/it)
+  objective:  13300.039316488837
+Minimizing 1658    Time: 0:00:12 ( 7.41 ms/it)
+  objective:  13300.016441446089
+Minimizing 1672    Time: 0:00:12 ( 7.41 ms/it)
+  objective:  13299.992881394806
+Minimizing 1686    Time: 0:00:12 ( 7.41 ms/it)
+  objective:  13299.986797741367
+Minimizing 1699    Time: 0:00:12 ( 7.42 ms/it)
+  objective:  13299.985735700582
+Minimizing 1713    Time: 0:00:12 ( 7.42 ms/it)
+  objective:  13299.985038568862
+Minimizing 1727    Time: 0:00:12 ( 7.42 ms/it)
+  objective:  13299.98481352735
+Minimizing 1741    Time: 0:00:12 ( 7.42 ms/it)
+  objective:  13299.983782423922
+Minimizing 1755    Time: 0:00:13 ( 7.42 ms/it)
+  objective:  13299.97619429884
+Minimizing 1769    Time: 0:00:13 ( 7.42 ms/it)
+  objective:  13299.975995964516
+Minimizing 1783    Time: 0:00:13 ( 7.41 ms/it)
+  objective:  13299.9729981102
+Minimizing 1797    Time: 0:00:13 ( 7.41 ms/it)
+  objective:  13299.96839557476
+Minimizing 1811    Time: 0:00:13 ( 7.41 ms/it)
+  objective:  13299.967537347824
+Minimizing 1825    Time: 0:00:13 ( 7.41 ms/it)
+  objective:  13299.961759416285
+Minimizing 1839    Time: 0:00:13 ( 7.41 ms/it)
+  objective:  13299.960301776562
+Minimizing 1853    Time: 0:00:13 ( 7.41 ms/it)
+  objective:  13299.959559300827
+Minimizing 1867    Time: 0:00:13 ( 7.41 ms/it)
+  objective:  13299.958113731467
+Minimizing 1881    Time: 0:00:13 ( 7.41 ms/it)
+  objective:  13299.954657530849
+Minimizing 1895    Time: 0:00:14 ( 7.41 ms/it)
+  objective:  13299.918198116953
+Minimizing 1909    Time: 0:00:14 ( 7.40 ms/it)
+  objective:  13299.91660130379
+Minimizing 1923    Time: 0:00:14 ( 7.40 ms/it)
+  objective:  13299.916022577483
+Minimizing 1937    Time: 0:00:14 ( 7.41 ms/it)
+  objective:  13299.915282904272
+Minimizing 1951    Time: 0:00:14 ( 7.41 ms/it)
+  objective:  13299.91275801108
+Minimizing 1965    Time: 0:00:14 ( 7.41 ms/it)
+  objective:  13299.911371354538
+Minimizing 1979    Time: 0:00:14 ( 7.40 ms/it)
+  objective:  13299.907843613837
+Minimizing 1994    Time: 0:00:14 ( 7.40 ms/it)
+  objective:  13299.887608155754
+Minimizing 2008    Time: 0:00:14 ( 7.40 ms/it)
+  objective:  13299.886695265886
+Minimizing 2022    Time: 0:00:14 ( 7.40 ms/it)
+  objective:  13299.883886907977
+Minimizing 2037    Time: 0:00:15 ( 7.40 ms/it)
+  objective:  13299.867251779331
+Minimizing 2051    Time: 0:00:15 ( 7.40 ms/it)
+  objective:  13299.780878727397
+Minimizing 2065    Time: 0:00:15 ( 7.40 ms/it)
+  objective:  13299.716876275386
+Minimizing 2079    Time: 0:00:15 ( 7.39 ms/it)
+  objective:  13299.704992282015
+Minimizing 2093    Time: 0:00:15 ( 7.39 ms/it)
+  objective:  13299.7023027518
+Minimizing 2107    Time: 0:00:15 ( 7.39 ms/it)
+  objective:  13299.702110452912
+Minimizing 2121    Time: 0:00:15 ( 7.39 ms/it)
+  objective:  13299.699853481201
+Minimizing 2136    Time: 0:00:15 ( 7.39 ms/it)
+  objective:  13299.695840824585
+Minimizing 2151    Time: 0:00:15 ( 7.39 ms/it)
+  objective:  13299.689033835588
+Minimizing 2166    Time: 0:00:15 ( 7.39 ms/it)
+  objective:  13299.686354496705
+Minimizing 2180    Time: 0:00:16 ( 7.38 ms/it)
+  objective:  13299.684333503523
+Minimizing 2195    Time: 0:00:16 ( 7.38 ms/it)
+  objective:  13299.67685816715
+Minimizing 2210    Time: 0:00:16 ( 7.38 ms/it)
+  objective:  13299.671855180815
+Minimizing 2225    Time: 0:00:16 ( 7.38 ms/it)
+  objective:  13299.669831754843
+Minimizing 2240    Time: 0:00:16 ( 7.37 ms/it)
+  objective:  13299.664821198458
+Minimizing 2255    Time: 0:00:16 ( 7.37 ms/it)
+  objective:  13299.66390414616
+Minimizing 2270    Time: 0:00:16 ( 7.37 ms/it)
+  objective:  13299.663356312478
+Minimizing 2285    Time: 0:00:16 ( 7.36 ms/it)
+  objective:  13299.66278192561
+Minimizing 2300    Time: 0:00:16 ( 7.36 ms/it)
+  objective:  13299.661119690383
+Minimizing 2315    Time: 0:00:17 ( 7.36 ms/it)
+  objective:  13299.638889618582
+Minimizing 2330    Time: 0:00:17 ( 7.35 ms/it)
+  objective:  13299.635406443267
+Minimizing 2345    Time: 0:00:17 ( 7.35 ms/it)
+  objective:  13299.634563153246
+Minimizing 2360    Time: 0:00:17 ( 7.35 ms/it)
+  objective:  13299.631799048322
+Minimizing 2375    Time: 0:00:17 ( 7.34 ms/it)
+  objective:  13299.630767917333
+Minimizing 2390    Time: 0:00:17 ( 7.34 ms/it)
+  objective:  13299.622092789927
+Minimizing 2405    Time: 0:00:17 ( 7.34 ms/it)
+  objective:  13299.62150079933
+Minimizing 2420    Time: 0:00:17 ( 7.34 ms/it)
+  objective:  13299.619576690864
+Minimizing 2434    Time: 0:00:17 ( 7.34 ms/it)
+  objective:  13299.618630976212
+Minimizing 2449    Time: 0:00:17 ( 7.34 ms/it)
+  objective:  13299.590949282428
+Minimizing 2464    Time: 0:00:18 ( 7.33 ms/it)
+  objective:  13299.587271440381
+Minimizing 2479    Time: 0:00:18 ( 7.33 ms/it)
+  objective:  13299.584474582123
+Minimizing 2494    Time: 0:00:18 ( 7.33 ms/it)
+  objective:  13299.582613714403
+Minimizing 2509    Time: 0:00:18 ( 7.33 ms/it)
+  objective:  13299.582530809057
+Minimizing 2524    Time: 0:00:18 ( 7.32 ms/it)
+  objective:  13299.581651118206
+Minimizing 2539    Time: 0:00:18 ( 7.32 ms/it)
+  objective:  13299.580380826883
+Minimizing 2554    Time: 0:00:18 ( 7.32 ms/it)
+  objective:  13299.575468890718
+Minimizing 2569    Time: 0:00:18 ( 7.32 ms/it)
+  objective:  13299.574941438987
+Minimizing 2584    Time: 0:00:18 ( 7.31 ms/it)
+  objective:  13299.57473547748
+Minimizing 2599    Time: 0:00:19 ( 7.31 ms/it)
+  objective:  13299.574675681244
+Minimizing 2614    Time: 0:00:19 ( 7.31 ms/it)
+  objective:  13299.574759437543
+Minimizing 2629    Time: 0:00:19 ( 7.31 ms/it)
+  objective:  13299.57465236247
+Minimizing 2644    Time: 0:00:19 ( 7.31 ms/it)
+  objective:  13299.574693544753
+Minimizing 2659    Time: 0:00:19 ( 7.31 ms/it)
+  objective:  13299.574655547316
+Minimizing 2673    Time: 0:00:19 ( 7.31 ms/it)
+  objective:  13299.574664291795
+Minimizing 2687    Time: 0:00:19 ( 7.31 ms/it)
+  objective:  13299.574650705501
+Minimizing 2702    Time: 0:00:19 ( 7.30 ms/it)
+  objective:  13299.574662029554
+Minimizing 2717    Time: 0:00:19 ( 7.30 ms/it)
+  objective:  13299.57465825256
+Minimizing 2732    Time: 0:00:19 ( 7.30 ms/it)
+  objective:  13299.574659096776
+Minimizing 2747    Time: 0:00:20 ( 7.30 ms/it)
+  objective:  13299.574653715434
+Minimizing 2754    Time: 0:00:20 ( 7.30 ms/it)
+Minimizing 15    Time: 0:00:00 ( 6.88 ms/it)
+  objective:  13919.888179726298
+Minimizing 30    Time: 0:00:00 ( 6.87 ms/it)
+  objective:  13815.883938504396
+Minimizing 45    Time: 0:00:00 ( 6.89 ms/it)
+  objective:  13786.33833284796
+Minimizing 60    Time: 0:00:00 ( 6.89 ms/it)
+  objective:  13784.43109247562
+Minimizing 75    Time: 0:00:00 ( 6.89 ms/it)
+  objective:  13780.643883138247
+Minimizing 90    Time: 0:00:00 ( 6.91 ms/it)
+  objective:  13778.159421052334
+Minimizing 105    Time: 0:00:00 ( 6.92 ms/it)
+  objective:  13748.771484139796
+Minimizing 120    Time: 0:00:00 ( 6.93 ms/it)
+  objective:  13742.232397522686
+Minimizing 135    Time: 0:00:00 ( 6.93 ms/it)
+  objective:  13735.058257565554
+Minimizing 150    Time: 0:00:01 ( 6.93 ms/it)
+  objective:  13728.752825850144
+Minimizing 165    Time: 0:00:01 ( 6.93 ms/it)
+  objective:  13722.711479649493
+Minimizing 180    Time: 0:00:01 ( 6.94 ms/it)
+  objective:  13721.51956269921
+Minimizing 194    Time: 0:00:01 ( 6.97 ms/it)
+  objective:  13718.383898739745
+Minimizing 209    Time: 0:00:01 ( 6.98 ms/it)
+  objective:  13714.643657575401
+Minimizing 224    Time: 0:00:01 ( 6.97 ms/it)
+  objective:  13714.081910390509
+Minimizing 239    Time: 0:00:01 ( 6.97 ms/it)
+  objective:  13710.919152905153
+Minimizing 254    Time: 0:00:01 ( 6.97 ms/it)
+  objective:  13708.377044422363
+Minimizing 269    Time: 0:00:01 ( 6.97 ms/it)
+  objective:  13707.531207325808
+Minimizing 284    Time: 0:00:01 ( 6.97 ms/it)
+  objective:  13704.781395081307
+Minimizing 299    Time: 0:00:02 ( 6.97 ms/it)
+  objective:  13703.318775764576
+Minimizing 314    Time: 0:00:02 ( 6.97 ms/it)
+  objective:  13700.864623521473
+Minimizing 329    Time: 0:00:02 ( 6.97 ms/it)
+  objective:  13696.941601516359
+Minimizing 344    Time: 0:00:02 ( 6.97 ms/it)
+  objective:  13688.26463997189
+Minimizing 359    Time: 0:00:02 ( 6.97 ms/it)
+  objective:  13684.813655986602
+Minimizing 374    Time: 0:00:02 ( 6.97 ms/it)
+  objective:  13676.32159524804
+Minimizing 389    Time: 0:00:02 ( 6.97 ms/it)
+  objective:  13662.599234298144
+Minimizing 404    Time: 0:00:02 ( 6.97 ms/it)
+  objective:  13642.145530001137
+Minimizing 419    Time: 0:00:02 ( 6.97 ms/it)
+  objective:  13632.808208863455
+Minimizing 434    Time: 0:00:03 ( 6.97 ms/it)
+  objective:  13627.330102655375
+Minimizing 449    Time: 0:00:03 ( 6.96 ms/it)
+  objective:  13605.865467742173
+Minimizing 464    Time: 0:00:03 ( 6.97 ms/it)
+  objective:  13603.213778256239
+Minimizing 478    Time: 0:00:03 ( 6.98 ms/it)
+  objective:  13599.3686192823
+Minimizing 492    Time: 0:00:03 ( 6.99 ms/it)
+  objective:  13587.380071440111
+Minimizing 507    Time: 0:00:03 ( 6.99 ms/it)
+  objective:  13564.684557248402
+Minimizing 522    Time: 0:00:03 ( 6.99 ms/it)
+  objective:  13560.285636726672
+Minimizing 536    Time: 0:00:03 ( 6.99 ms/it)
+  objective:  13553.942275228554
+Minimizing 551    Time: 0:00:03 ( 6.99 ms/it)
+  objective:  13551.432660416802
+Minimizing 566    Time: 0:00:03 ( 6.99 ms/it)
+  objective:  13546.68115545698
+Minimizing 581    Time: 0:00:04 ( 6.99 ms/it)
+  objective:  13542.457680659856
+Minimizing 596    Time: 0:00:04 ( 6.99 ms/it)
+  objective:  13531.282504736759
+Minimizing 611    Time: 0:00:04 ( 6.99 ms/it)
+  objective:  13507.686555040127
+Minimizing 626    Time: 0:00:04 ( 6.99 ms/it)
+  objective:  13501.795122595562
+Minimizing 642    Time: 0:00:04 ( 7.00 ms/it)
+  objective:  13488.233557449079
+Minimizing 658    Time: 0:00:04 ( 7.00 ms/it)
+  objective:  13469.388268558534
+Minimizing 673    Time: 0:00:04 ( 6.99 ms/it)
+  objective:  13464.469901100732
+Minimizing 688    Time: 0:00:04 ( 6.99 ms/it)
+  objective:  13463.11122495486
+Minimizing 703    Time: 0:00:04 ( 6.99 ms/it)
+  objective:  13462.282211793492
+Minimizing 718    Time: 0:00:05 ( 6.99 ms/it)
+  objective:  13460.87544580958
+Minimizing 733    Time: 0:00:05 ( 6.98 ms/it)
+  objective:  13458.552168422742
+Minimizing 748    Time: 0:00:05 ( 6.98 ms/it)
+  objective:  13458.34861197463
+Minimizing 763    Time: 0:00:05 ( 6.98 ms/it)
+  objective:  13455.94230646456
+Minimizing 778    Time: 0:00:05 ( 6.98 ms/it)
+  objective:  13454.57298623756
+Minimizing 793    Time: 0:00:05 ( 6.98 ms/it)
+  objective:  13448.645169117874
+Minimizing 808    Time: 0:00:05 ( 6.98 ms/it)
+  objective:  13448.61643171533
+Minimizing 823    Time: 0:00:05 ( 6.98 ms/it)
+  objective:  13447.566058433054
+Minimizing 838    Time: 0:00:05 ( 6.98 ms/it)
+  objective:  13446.591300746077
+Minimizing 853    Time: 0:00:05 ( 6.98 ms/it)
+  objective:  13445.207405540794
+Minimizing 868    Time: 0:00:06 ( 6.97 ms/it)
+  objective:  13443.570739741306
+Minimizing 883    Time: 0:00:06 ( 6.97 ms/it)
+  objective:  13439.295178485554
+Minimizing 898    Time: 0:00:06 ( 6.97 ms/it)
+  objective:  13438.37428408036
+Minimizing 913    Time: 0:00:06 ( 6.97 ms/it)
+  objective:  13436.702654517001
+Minimizing 928    Time: 0:00:06 ( 6.97 ms/it)
+  objective:  13434.755561863487
+Minimizing 943    Time: 0:00:06 ( 6.97 ms/it)
+  objective:  13432.637122840591
+Minimizing 958    Time: 0:00:06 ( 6.97 ms/it)
+  objective:  13424.008808011087
+Minimizing 973    Time: 0:00:06 ( 6.97 ms/it)
+  objective:  13423.410638560272
+Minimizing 988    Time: 0:00:06 ( 6.97 ms/it)
+  objective:  13415.272322237295
+Minimizing 1003    Time: 0:00:06 ( 6.97 ms/it)
+  objective:  13412.38322487795
+Minimizing 1018    Time: 0:00:07 ( 6.97 ms/it)
+  objective:  13401.797763866794
+Minimizing 1033    Time: 0:00:07 ( 6.97 ms/it)
+  objective:  13384.65012255188
+Minimizing 1048    Time: 0:00:07 ( 6.97 ms/it)
+  objective:  13355.424734630244
+Minimizing 1063    Time: 0:00:07 ( 6.97 ms/it)
+  objective:  13350.87330040599
+Minimizing 1078    Time: 0:00:07 ( 6.97 ms/it)
+  objective:  13349.023877738146
+Minimizing 1093    Time: 0:00:07 ( 6.97 ms/it)
+  objective:  13343.796538168943
+Minimizing 1108    Time: 0:00:07 ( 6.96 ms/it)
+  objective:  13329.905809950593
+Minimizing 1123    Time: 0:00:07 ( 6.96 ms/it)
+  objective:  13318.125331359799
+Minimizing 1138    Time: 0:00:07 ( 6.96 ms/it)
+  objective:  13315.558830251946
+Minimizing 1153    Time: 0:00:08 ( 6.96 ms/it)
+  objective:  13306.260512878187
+Minimizing 1168    Time: 0:00:08 ( 6.96 ms/it)
+  objective:  13304.46073174938
+Minimizing 1183    Time: 0:00:08 ( 6.96 ms/it)
+  objective:  13294.11897909896
+Minimizing 1198    Time: 0:00:08 ( 6.96 ms/it)
+  objective:  13293.423984179884
+Minimizing 1213    Time: 0:00:08 ( 6.96 ms/it)
+  objective:  13288.038433850394
+Minimizing 1228    Time: 0:00:08 ( 6.96 ms/it)
+  objective:  13279.134590073561
+Minimizing 1243    Time: 0:00:08 ( 6.96 ms/it)
+  objective:  13277.59210004643
+Minimizing 1257    Time: 0:00:08 ( 6.97 ms/it)
+  objective:  13276.592823444138
+Minimizing 1271    Time: 0:00:08 ( 6.98 ms/it)
+  objective:  13275.14079082162
+Minimizing 1285    Time: 0:00:08 ( 6.98 ms/it)
+  objective:  13273.95376134415
+Minimizing 1299    Time: 0:00:09 ( 6.98 ms/it)
+  objective:  13271.896365992914
+Minimizing 1313    Time: 0:00:09 ( 6.99 ms/it)
+  objective:  13268.022286465668
+Minimizing 1327    Time: 0:00:09 ( 6.99 ms/it)
+  objective:  13266.960992555294
+Minimizing 1341    Time: 0:00:09 ( 6.99 ms/it)
+  objective:  13264.20269177662
+Minimizing 1355    Time: 0:00:09 ( 7.00 ms/it)
+  objective:  13264.056451031138
+Minimizing 1369    Time: 0:00:09 ( 7.00 ms/it)
+  objective:  13262.014099073072
+Minimizing 1383    Time: 0:00:09 ( 7.01 ms/it)
+  objective:  13261.39514845531
+Minimizing 1397    Time: 0:00:09 ( 7.01 ms/it)
+  objective:  13258.973968907303
+Minimizing 1411    Time: 0:00:09 ( 7.02 ms/it)
+  objective:  13252.131282131086
+Minimizing 1425    Time: 0:00:10 ( 7.02 ms/it)
+  objective:  13242.019296540384
+Minimizing 1439    Time: 0:00:10 ( 7.03 ms/it)
+  objective:  13240.161053406526
+Minimizing 1453    Time: 0:00:10 ( 7.03 ms/it)
+  objective:  13232.076313444995
+Minimizing 1467    Time: 0:00:10 ( 7.03 ms/it)
+  objective:  13230.218340676787
+Minimizing 1481    Time: 0:00:10 ( 7.04 ms/it)
+  objective:  13225.47226706834
+Minimizing 1495    Time: 0:00:10 ( 7.04 ms/it)
+  objective:  13221.133445492596
+Minimizing 1508    Time: 0:00:10 ( 7.05 ms/it)
+  objective:  13213.376585905382
+Minimizing 1522    Time: 0:00:10 ( 7.05 ms/it)
+  objective:  13196.480471017174
+Minimizing 1536    Time: 0:00:10 ( 7.06 ms/it)
+  objective:  13176.58414446337
+Minimizing 1550    Time: 0:00:10 ( 7.06 ms/it)
+  objective:  13176.124813405811
+Minimizing 1564    Time: 0:00:11 ( 7.06 ms/it)
+  objective:  13176.122626118784
+Minimizing 1578    Time: 0:00:11 ( 7.06 ms/it)
+  objective:  13176.122256583723
+Minimizing 1592    Time: 0:00:11 ( 7.07 ms/it)
+  objective:  13176.122205140986
+Minimizing 1606    Time: 0:00:11 ( 7.07 ms/it)
+  objective:  13176.12100398689
+Minimizing 1620    Time: 0:00:11 ( 7.07 ms/it)
+  objective:  13176.11691388562
+Minimizing 1634    Time: 0:00:11 ( 7.08 ms/it)
+  objective:  13176.120781069796
+Minimizing 1648    Time: 0:00:11 ( 7.08 ms/it)
+  objective:  13176.121256129176
+Minimizing 1662    Time: 0:00:11 ( 7.09 ms/it)
+  objective:  13176.121651204958
+Minimizing 1676    Time: 0:00:11 ( 7.09 ms/it)
+  objective:  13176.121153868342
+Minimizing 1690    Time: 0:00:11 ( 7.09 ms/it)
+  objective:  13176.120604542695
+Minimizing 1704    Time: 0:00:12 ( 7.09 ms/it)
+  objective:  13176.121718868628
+Minimizing 1718    Time: 0:00:12 ( 7.09 ms/it)
+  objective:  13176.12001931647
+Minimizing 1732    Time: 0:00:12 ( 7.09 ms/it)
+  objective:  13176.12056770247
+Minimizing 1746    Time: 0:00:12 ( 7.10 ms/it)
+  objective:  13176.12362314183
+Minimizing 1760    Time: 0:00:12 ( 7.10 ms/it)
+  objective:  13176.122318892172
+Minimizing 1774    Time: 0:00:12 ( 7.10 ms/it)
+  objective:  13176.119855460041
+Minimizing 1774    Time: 0:00:12 ( 7.10 ms/it)
+Minimizing 14    Time: 0:00:00 ( 7.34 ms/it)
+  objective:  13799.690289755828
+Minimizing 28    Time: 0:00:00 ( 7.40 ms/it)
+  objective:  13814.98771423492
+Minimizing 42    Time: 0:00:00 ( 7.40 ms/it)
+  objective:  13791.53328914472
+Minimizing 56    Time: 0:00:00 ( 7.39 ms/it)
+  objective:  13784.223194780694
+Minimizing 70    Time: 0:00:00 ( 7.39 ms/it)
+  objective:  13776.902801456363
+Minimizing 84    Time: 0:00:00 ( 7.43 ms/it)
+  objective:  13773.23052871109
+Minimizing 98    Time: 0:00:00 ( 7.44 ms/it)
+  objective:  13767.8482604022
+Minimizing 112    Time: 0:00:00 ( 7.43 ms/it)
+  objective:  13761.47186604549
+Minimizing 126    Time: 0:00:00 ( 7.41 ms/it)
+  objective:  13756.858539470975
+Minimizing 140    Time: 0:00:01 ( 7.41 ms/it)
+  objective:  13737.366960877882
+Minimizing 154    Time: 0:00:01 ( 7.42 ms/it)
+  objective:  13733.454930466967
+Minimizing 168    Time: 0:00:01 ( 7.44 ms/it)
+  objective:  13731.63266226235
+Minimizing 182    Time: 0:00:01 ( 7.46 ms/it)
+  objective:  13727.74135783894
+Minimizing 196    Time: 0:00:01 ( 7.46 ms/it)
+  objective:  13723.11371396379
+Minimizing 210    Time: 0:00:01 ( 7.47 ms/it)
+  objective:  13708.984335873063
+Minimizing 224    Time: 0:00:01 ( 7.47 ms/it)
+  objective:  13706.349013616393
+Minimizing 238    Time: 0:00:01 ( 7.47 ms/it)
+  objective:  13701.587847825613
+Minimizing 252    Time: 0:00:01 ( 7.46 ms/it)
+  objective:  13687.990772626152
+Minimizing 266    Time: 0:00:01 ( 7.45 ms/it)
+  objective:  13685.677862680417
+Minimizing 280    Time: 0:00:02 ( 7.44 ms/it)
+  objective:  13676.107302250028
+Minimizing 294    Time: 0:00:02 ( 7.45 ms/it)
+  objective:  13666.898255152362
+Minimizing 308    Time: 0:00:02 ( 7.46 ms/it)
+  objective:  13650.318569806805
+Minimizing 322    Time: 0:00:02 ( 7.46 ms/it)
+  objective:  13648.261527641524
+Minimizing 336    Time: 0:00:02 ( 7.46 ms/it)
+  objective:  13637.863939047435
+Minimizing 350    Time: 0:00:02 ( 7.46 ms/it)
+  objective:  13630.575572279373
+Minimizing 364    Time: 0:00:02 ( 7.47 ms/it)
+  objective:  13614.88660370166
+Minimizing 378    Time: 0:00:02 ( 7.47 ms/it)
+  objective:  13610.071531565009
+Minimizing 392    Time: 0:00:02 ( 7.46 ms/it)
+  objective:  13607.239932400262
+Minimizing 406    Time: 0:00:03 ( 7.46 ms/it)
+  objective:  13602.74249920054
+Minimizing 420    Time: 0:00:03 ( 7.46 ms/it)
+  objective:  13598.932864674047
+Minimizing 434    Time: 0:00:03 ( 7.46 ms/it)
+  objective:  13577.700711742516
+Minimizing 448    Time: 0:00:03 ( 7.46 ms/it)
+  objective:  13540.98406420846
+Minimizing 462    Time: 0:00:03 ( 7.46 ms/it)
+  objective:  13532.706236090708
+Minimizing 476    Time: 0:00:03 ( 7.46 ms/it)
+  objective:  13520.379128555374
+Minimizing 490    Time: 0:00:03 ( 7.46 ms/it)
+  objective:  13513.302198592966
+Minimizing 504    Time: 0:00:03 ( 7.46 ms/it)
+  objective:  13508.088088117613
+Minimizing 518    Time: 0:00:03 ( 7.46 ms/it)
+  objective:  13500.338741727523
+Minimizing 532    Time: 0:00:03 ( 7.45 ms/it)
+  objective:  13492.837756229885
+Minimizing 546    Time: 0:00:04 ( 7.46 ms/it)
+  objective:  13465.566937206575
+Minimizing 560    Time: 0:00:04 ( 7.46 ms/it)
+  objective:  13462.810806506597
+Minimizing 574    Time: 0:00:04 ( 7.46 ms/it)
+  objective:  13456.77155015275
+Minimizing 588    Time: 0:00:04 ( 7.46 ms/it)
+  objective:  13455.020788706599
+Minimizing 602    Time: 0:00:04 ( 7.46 ms/it)
+  objective:  13453.754699343932
+Minimizing 616    Time: 0:00:04 ( 7.46 ms/it)
+  objective:  13450.692854851492
+Minimizing 630    Time: 0:00:04 ( 7.46 ms/it)
+  objective:  13448.528232286815
+Minimizing 644    Time: 0:00:04 ( 7.46 ms/it)
+  objective:  13443.455904456314
+Minimizing 658    Time: 0:00:04 ( 7.46 ms/it)
+  objective:  13442.133160955593
+Minimizing 672    Time: 0:00:05 ( 7.45 ms/it)
+  objective:  13437.000825261872
+Minimizing 686    Time: 0:00:05 ( 7.45 ms/it)
+  objective:  13426.941025741398
+Minimizing 700    Time: 0:00:05 ( 7.45 ms/it)
+  objective:  13426.602804233313
+Minimizing 714    Time: 0:00:05 ( 7.45 ms/it)
+  objective:  13424.676702505378
+Minimizing 727    Time: 0:00:05 ( 7.46 ms/it)
+  objective:  13423.693517994077
+Minimizing 740    Time: 0:00:05 ( 7.47 ms/it)
+  objective:  13422.970916159189
+Minimizing 753    Time: 0:00:05 ( 7.48 ms/it)
+  objective:  13422.366024823343
+Minimizing 766    Time: 0:00:05 ( 7.49 ms/it)
+  objective:  13417.735615639387
+Minimizing 779    Time: 0:00:05 ( 7.50 ms/it)
+  objective:  13417.103077060594
+Minimizing 793    Time: 0:00:05 ( 7.50 ms/it)
+  objective:  13412.083820849708
+Minimizing 807    Time: 0:00:06 ( 7.49 ms/it)
+  objective:  13402.651882951526
+Minimizing 821    Time: 0:00:06 ( 7.49 ms/it)
+  objective:  13394.434166434541
+Minimizing 835    Time: 0:00:06 ( 7.49 ms/it)
+  objective:  13375.566997549904
+Minimizing 849    Time: 0:00:06 ( 7.49 ms/it)
+  objective:  13366.072629510745
+Minimizing 863    Time: 0:00:06 ( 7.49 ms/it)
+  objective:  13356.16795565781
+Minimizing 877    Time: 0:00:06 ( 7.48 ms/it)
+  objective:  13344.647086932076
+Minimizing 891    Time: 0:00:06 ( 7.48 ms/it)
+  objective:  13343.92449473987
+Minimizing 905    Time: 0:00:06 ( 7.48 ms/it)
+  objective:  13340.338483102038
+Minimizing 919    Time: 0:00:06 ( 7.49 ms/it)
+  objective:  13338.464328736067
+Minimizing 933    Time: 0:00:06 ( 7.48 ms/it)
+  objective:  13336.292964622058
+Minimizing 947    Time: 0:00:07 ( 7.48 ms/it)
+  objective:  13334.4332020742
+Minimizing 961    Time: 0:00:07 ( 7.49 ms/it)
+  objective:  13331.56700019956
+Minimizing 975    Time: 0:00:07 ( 7.49 ms/it)
+  objective:  13328.17346719306
+Minimizing 989    Time: 0:00:07 ( 7.48 ms/it)
+  objective:  13325.702470719727
+Minimizing 1003    Time: 0:00:07 ( 7.48 ms/it)
+  objective:  13324.023172127942
+Minimizing 1017    Time: 0:00:07 ( 7.48 ms/it)
+  objective:  13320.680490710627
+Minimizing 1031    Time: 0:00:07 ( 7.48 ms/it)
+  objective:  13318.093795479333
+Minimizing 1045    Time: 0:00:07 ( 7.48 ms/it)
+  objective:  13316.52141299202
+Minimizing 1059    Time: 0:00:07 ( 7.47 ms/it)
+  objective:  13316.400689379763
+Minimizing 1073    Time: 0:00:08 ( 7.48 ms/it)
+  objective:  13313.380829079018
+Minimizing 1087    Time: 0:00:08 ( 7.48 ms/it)
+  objective:  13311.809365524692
+Minimizing 1100    Time: 0:00:08 ( 7.48 ms/it)
+  objective:  13311.647481701904
+Minimizing 1114    Time: 0:00:08 ( 7.48 ms/it)
+  objective:  13309.703775092261
+Minimizing 1128    Time: 0:00:08 ( 7.48 ms/it)
+  objective:  13309.600251360622
+Minimizing 1142    Time: 0:00:08 ( 7.47 ms/it)
+  objective:  13307.588390650577
+Minimizing 1156    Time: 0:00:08 ( 7.47 ms/it)
+  objective:  13302.61931799214
+Minimizing 1170    Time: 0:00:08 ( 7.47 ms/it)
+  objective:  13296.309421655256
+Minimizing 1184    Time: 0:00:08 ( 7.47 ms/it)
+  objective:  13292.065207682012
+Minimizing 1198    Time: 0:00:08 ( 7.47 ms/it)
+  objective:  13288.750471524778
+Minimizing 1212    Time: 0:00:09 ( 7.46 ms/it)
+  objective:  13284.53302863962
+Minimizing 1226    Time: 0:00:09 ( 7.46 ms/it)
+  objective:  13278.725620617566
+Minimizing 1240    Time: 0:00:09 ( 7.47 ms/it)
+  objective:  13272.254949326918
+Minimizing 1254    Time: 0:00:09 ( 7.46 ms/it)
+  objective:  13247.357388104036
+Minimizing 1268    Time: 0:00:09 ( 7.46 ms/it)
+  objective:  13235.296743677274
+Minimizing 1282    Time: 0:00:09 ( 7.46 ms/it)
+  objective:  13231.563146665052
+Minimizing 1296    Time: 0:00:09 ( 7.46 ms/it)
+  objective:  13216.005783504253
+Minimizing 1310    Time: 0:00:09 ( 7.46 ms/it)
+  objective:  13207.247931517064
+Minimizing 1324    Time: 0:00:09 ( 7.46 ms/it)
+  objective:  13207.04314413754
+Minimizing 1338    Time: 0:00:09 ( 7.45 ms/it)
+  objective:  13200.474647403462
+Minimizing 1352    Time: 0:00:10 ( 7.45 ms/it)
+  objective:  13197.289365754332
+Minimizing 1366    Time: 0:00:10 ( 7.45 ms/it)
+  objective:  13192.429658603141
+Minimizing 1380    Time: 0:00:10 ( 7.45 ms/it)
+  objective:  13187.930481763717
+Minimizing 1394    Time: 0:00:10 ( 7.45 ms/it)
+  objective:  13182.061168171422
+Minimizing 1408    Time: 0:00:10 ( 7.45 ms/it)
+  objective:  13178.252241522467
+Minimizing 1422    Time: 0:00:10 ( 7.45 ms/it)
+  objective:  13165.31283846611
+Minimizing 1436    Time: 0:00:10 ( 7.45 ms/it)
+  objective:  13157.735247360528
+Minimizing 1450    Time: 0:00:10 ( 7.45 ms/it)
+  objective:  13157.199392885057
+Minimizing 1464    Time: 0:00:10 ( 7.45 ms/it)
+  objective:  13156.50308670917
+Minimizing 1478    Time: 0:00:11 ( 7.45 ms/it)
+  objective:  13154.813020916175
+Minimizing 1492    Time: 0:00:11 ( 7.45 ms/it)
+  objective:  13150.31367202608
+Minimizing 1506    Time: 0:00:11 ( 7.45 ms/it)
+  objective:  13147.247403814355
+Minimizing 1520    Time: 0:00:11 ( 7.45 ms/it)
+  objective:  13140.819616088833
+Minimizing 1534    Time: 0:00:11 ( 7.44 ms/it)
+  objective:  13137.304527376531
+Minimizing 1548    Time: 0:00:11 ( 7.44 ms/it)
+  objective:  13137.00180729988
+Minimizing 1562    Time: 0:00:11 ( 7.44 ms/it)
+  objective:  13134.553269421682
+Minimizing 1576    Time: 0:00:11 ( 7.44 ms/it)
+  objective:  13134.482516308737
+Minimizing 1590    Time: 0:00:11 ( 7.44 ms/it)
+  objective:  13134.427148029063
+Minimizing 1604    Time: 0:00:11 ( 7.44 ms/it)
+  objective:  13134.433973989173
+Minimizing 1618    Time: 0:00:12 ( 7.44 ms/it)
+  objective:  13134.433782834356
+Minimizing 1632    Time: 0:00:12 ( 7.44 ms/it)
+  objective:  13134.424998009024
+Minimizing 1646    Time: 0:00:12 ( 7.44 ms/it)
+  objective:  13134.432448809108
+Minimizing 1660    Time: 0:00:12 ( 7.44 ms/it)
+  objective:  13134.439745825148
+Minimizing 1674    Time: 0:00:12 ( 7.46 ms/it)
+  objective:  13134.435048403218
+Minimizing 1688    Time: 0:00:12 ( 7.46 ms/it)
+  objective:  13134.438355184306
+Minimizing 1702    Time: 0:00:12 ( 7.46 ms/it)
+  objective:  13134.434327619849
+Minimizing 1716    Time: 0:00:12 ( 7.46 ms/it)
+  objective:  13134.436954488774
+Minimizing 1730    Time: 0:00:12 ( 7.45 ms/it)
+  objective:  13134.431615684036
+Minimizing 1744    Time: 0:00:12 ( 7.45 ms/it)
+  objective:  13134.423937564177
+Minimizing 1758    Time: 0:00:13 ( 7.45 ms/it)
+  objective:  13134.433076300586
+Minimizing 1772    Time: 0:00:13 ( 7.46 ms/it)
+  objective:  13134.43327911022
+Minimizing 1786    Time: 0:00:13 ( 7.46 ms/it)
+  objective:  13134.4345713199
+Minimizing 1799    Time: 0:00:13 ( 7.46 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01820.0140-1.300.19420.6870
Test: c5.1-0.04030.0429-0.940.34811.3839
Test: c234.150.01620.16780.100.92310.6312
Test: c2.340.00990.07580.130.89580.9852
Test: c3.4-0.01370.0445-0.310.75841.5296
a10.32070.04806.67<1e-10
Sex: Boys0.20520.014014.63<1e-47
Test: c5.1 & a10.61600.14694.19<1e-04
Test: c234.15 & a10.06410.57280.110.9108
Test: c2.34 & a1-0.06290.2584-0.240.8077
Test: c3.4 & a10.07420.15330.480.6284
Test: c5.1 & Sex: Boys0.15730.04293.670.0002
Test: c234.15 & Sex: Boys-0.82020.1678-4.89<1e-05
Test: c2.34 & Sex: Boys-0.18370.0758-2.420.0154
Test: c3.4 & Sex: Boys-0.04650.0445-1.040.2964
a1 & Sex: Boys0.05080.04801.060.2908
Test: c5.1 & a1 & Sex: Boys0.03840.14690.260.7937
Test: c234.15 & a1 & Sex: Boys-0.35680.5728-0.620.5333
Test: c2.34 & a1 & Sex: Boys0.14480.25840.560.5752
Test: c3.4 & a1 & Sex: Boys0.06080.15330.400.6916
Residual0.0000
+
+
+
+
VarCorr(m_Child_SDC)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Child(Intercept)0.456783150.67585735
Test: Star_r0.536151830.73222389+0.75
Test: S20_r0.466256020.68282942+0.04+0.68
Test: SLJ0.219290720.46828487-0.51+0.18+0.83
Test: BPT0.795425200.89186613-0.17-0.37-0.35-0.24
Residual0.000000000.00000691
+
+
+
+
VarCorr(m_Child_HeC)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Child(Intercept)0.3615227530.601267621
Test: Star_r0.1797151190.423928200+0.11
Test: S20_r0.0574519690.239691403-0.40+0.35
Test: SLJ0.1193728180.345503717-0.32+0.04-0.60
Test: BPT0.0352727080.187810298-0.12-0.26-0.19+0.39
Residual0.0000000490.000221448
+
+
+
+
VarCorr(m_Child_HyC)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Child(Intercept)0.462867340.68034355
Test: BPT-other1.276804061.12995755+0.88
Test: Star-End0.950806340.97509299+0.46-0.01
Test: S20-Star1.263805821.12419118-0.10-0.15+0.17
Test: SLJ-S200.293666560.54191011-0.11+0.22-0.67+0.18
Residual0.000000000.00001522
+
+
+
+
VarCorr(m_Child_PCA)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Child(Intercept)0.471981870.68700937
Test: c5.11.915179891.38390025-0.11
Test: c234.150.398403700.63119228+0.83-0.48
Test: c2.340.970575840.98517807+0.69-0.15+0.88
Test: c3.42.339775401.52963244+0.12+0.08+0.40+0.63
Residual0.000000000.00001040
+
+
+

The CPs for the various contrasts are in line with expectations. For the SDC we observe substantial negative CPs between neighboring contrasts. For the orthogonal HeC, all CPs are small; they are uncorrelated. HyC contains some of the SDC contrasts and we observe again the negative CPs. The (roughly) PCA-based contrasts are small with one exception; there is a sizeable CP of +.41 between GM and the core of adjusted physical fitness (c234.15).

+

Do these differences in CPs imply that we can move to zcpLMMs when we have orthogonal contrasts? We pursue this question with by refitting the four LMMs with zerocorr() and compare the goodness of fit.

+
+
begin
+  f_Child0 = @formula zScore ~
+    1 + Test * a1 * Sex + zerocorr(1 + Test | Child)
+  m_Child_SDC0 = fit(MixedModel, f_Child0, dat; contrasts=contr1)
+  m_Child_HeC0 = fit(MixedModel, f_Child0, dat; contrasts=contr2)
+  m_Child_HyC0 = fit(MixedModel, f_Child0, dat; contrasts=contr3)
+  m_Child_PCA0 = fit(MixedModel, f_Child0, dat; contrasts=contr4)
+end
+
+
Minimizing 20    Time: 0:00:00 ( 5.16 ms/it)
+  objective:  13801.840082943849
+Minimizing 41    Time: 0:00:00 ( 5.08 ms/it)
+  objective:  13798.088511524951
+Minimizing 62    Time: 0:00:00 ( 5.06 ms/it)
+  objective:  13796.729778581182
+Minimizing 82    Time: 0:00:00 ( 5.05 ms/it)
+  objective:  13796.212345608132
+Minimizing 102    Time: 0:00:00 ( 5.07 ms/it)
+  objective:  13796.20566915371
+Minimizing 121    Time: 0:00:00 ( 5.12 ms/it)
+  objective:  13796.201088517759
+Minimizing 141    Time: 0:00:00 ( 5.10 ms/it)
+  objective:  13796.20062481871
+Minimizing 161    Time: 0:00:00 ( 5.10 ms/it)
+  objective:  13796.200603124427
+Minimizing 166    Time: 0:00:00 ( 5.09 ms/it)
+Minimizing 20    Time: 0:00:00 ( 5.02 ms/it)
+  objective:  13777.86936071649
+Minimizing 40    Time: 0:00:00 ( 5.05 ms/it)
+  objective:  13751.435654438266
+Minimizing 60    Time: 0:00:00 ( 5.10 ms/it)
+  objective:  13744.158578651432
+Minimizing 79    Time: 0:00:00 ( 5.21 ms/it)
+  objective:  13722.102325285614
+Minimizing 99    Time: 0:00:00 ( 5.19 ms/it)
+  objective:  13698.864914585596
+Minimizing 119    Time: 0:00:00 ( 5.17 ms/it)
+  objective:  13675.037724363621
+Minimizing 139    Time: 0:00:00 ( 5.15 ms/it)
+  objective:  13669.500265406032
+Minimizing 159    Time: 0:00:00 ( 5.19 ms/it)
+  objective:  13666.601484438732
+Minimizing 178    Time: 0:00:00 ( 5.20 ms/it)
+  objective:  13664.62857516346
+Minimizing 198    Time: 0:00:01 ( 5.19 ms/it)
+  objective:  13658.994251554324
+Minimizing 218    Time: 0:00:01 ( 5.18 ms/it)
+  objective:  13651.869567634305
+Minimizing 238    Time: 0:00:01 ( 5.20 ms/it)
+  objective:  13647.157274289502
+Minimizing 258    Time: 0:00:01 ( 5.19 ms/it)
+  objective:  13643.99421763185
+Minimizing 278    Time: 0:00:01 ( 5.20 ms/it)
+  objective:  13634.071910279778
+Minimizing 298    Time: 0:00:01 ( 5.20 ms/it)
+  objective:  13611.253260503185
+Minimizing 318    Time: 0:00:01 ( 5.20 ms/it)
+  objective:  13572.593989700017
+Minimizing 338    Time: 0:00:01 ( 5.19 ms/it)
+  objective:  13553.878759422732
+Minimizing 358    Time: 0:00:01 ( 5.19 ms/it)
+  objective:  13547.856240491867
+Minimizing 378    Time: 0:00:01 ( 5.18 ms/it)
+  objective:  13502.19686150935
+Minimizing 398    Time: 0:00:02 ( 5.18 ms/it)
+  objective:  13474.224490691908
+Minimizing 418    Time: 0:00:02 ( 5.17 ms/it)
+  objective:  13425.90477785928
+Minimizing 438    Time: 0:00:02 ( 5.16 ms/it)
+  objective:  13412.427282714802
+Minimizing 458    Time: 0:00:02 ( 5.18 ms/it)
+  objective:  13403.801051929535
+Minimizing 478    Time: 0:00:02 ( 5.18 ms/it)
+  objective:  13388.673016598339
+Minimizing 498    Time: 0:00:02 ( 5.18 ms/it)
+  objective:  13386.288818941037
+Minimizing 518    Time: 0:00:02 ( 5.18 ms/it)
+  objective:  13377.441475318126
+Minimizing 538    Time: 0:00:02 ( 5.17 ms/it)
+  objective:  13370.259104194884
+Minimizing 558    Time: 0:00:02 ( 5.17 ms/it)
+  objective:  13367.369029424292
+Minimizing 578    Time: 0:00:02 ( 5.17 ms/it)
+  objective:  13365.101712476157
+Minimizing 599    Time: 0:00:03 ( 5.17 ms/it)
+  objective:  13361.308738745414
+Minimizing 619    Time: 0:00:03 ( 5.16 ms/it)
+  objective:  13357.450535290518
+Minimizing 640    Time: 0:00:03 ( 5.16 ms/it)
+  objective:  13350.20610579828
+Minimizing 660    Time: 0:00:03 ( 5.16 ms/it)
+  objective:  13334.739287137127
+Minimizing 680    Time: 0:00:03 ( 5.15 ms/it)
+  objective:  13328.795718790963
+Minimizing 700    Time: 0:00:03 ( 5.15 ms/it)
+  objective:  13283.718608480907
+Minimizing 720    Time: 0:00:03 ( 5.15 ms/it)
+  objective:  13267.202844101223
+Minimizing 740    Time: 0:00:03 ( 5.15 ms/it)
+  objective:  13193.519004086498
+Minimizing 760    Time: 0:00:03 ( 5.16 ms/it)
+  objective:  13183.016734053206
+Minimizing 779    Time: 0:00:04 ( 5.16 ms/it)
+  objective:  13178.856704928665
+Minimizing 799    Time: 0:00:04 ( 5.16 ms/it)
+  objective:  13167.353935667721
+Minimizing 819    Time: 0:00:04 ( 5.16 ms/it)
+  objective:  13159.669089089628
+Minimizing 839    Time: 0:00:04 ( 5.16 ms/it)
+  objective:  13145.049961976925
+Minimizing 859    Time: 0:00:04 ( 5.16 ms/it)
+  objective:  13103.90558579212
+Minimizing 880    Time: 0:00:04 ( 5.15 ms/it)
+  objective:  13068.957720565246
+Minimizing 900    Time: 0:00:04 ( 5.15 ms/it)
+  objective:  13067.831647235886
+Minimizing 920    Time: 0:00:04 ( 5.15 ms/it)
+  objective:  13067.78037017677
+Minimizing 941    Time: 0:00:04 ( 5.15 ms/it)
+  objective:  13067.78825339333
+Minimizing 962    Time: 0:00:04 ( 5.14 ms/it)
+  objective:  13067.795915521361
+Minimizing 984    Time: 0:00:05 ( 5.13 ms/it)
+  objective:  13067.718710527042
+Minimizing 988    Time: 0:00:05 ( 5.13 ms/it)
+Minimizing 22    Time: 0:00:00 ( 4.69 ms/it)
+  objective:  13790.073907703707
+Minimizing 44    Time: 0:00:00 ( 4.69 ms/it)
+  objective:  13771.670965021149
+Minimizing 66    Time: 0:00:00 ( 4.71 ms/it)
+  objective:  13755.664061426653
+Minimizing 87    Time: 0:00:00 ( 4.76 ms/it)
+  objective:  13747.465550982335
+Minimizing 108    Time: 0:00:00 ( 4.82 ms/it)
+  objective:  13730.301500466248
+Minimizing 130    Time: 0:00:00 ( 4.80 ms/it)
+  objective:  13688.67645685529
+Minimizing 151    Time: 0:00:00 ( 4.80 ms/it)
+  objective:  13608.293077886097
+Minimizing 172    Time: 0:00:00 ( 4.86 ms/it)
+  objective:  13599.473504072888
+Minimizing 194    Time: 0:00:00 ( 4.85 ms/it)
+  objective:  13591.277410855968
+Minimizing 216    Time: 0:00:01 ( 4.84 ms/it)
+  objective:  13581.084125263322
+Minimizing 238    Time: 0:00:01 ( 4.83 ms/it)
+  objective:  13547.159759337155
+Minimizing 260    Time: 0:00:01 ( 4.82 ms/it)
+  objective:  13539.059128529203
+Minimizing 282    Time: 0:00:01 ( 4.81 ms/it)
+  objective:  13536.754523006224
+Minimizing 304    Time: 0:00:01 ( 4.81 ms/it)
+  objective:  13534.084209389424
+Minimizing 326    Time: 0:00:01 ( 4.80 ms/it)
+  objective:  13528.59897845464
+Minimizing 347    Time: 0:00:01 ( 4.80 ms/it)
+  objective:  13528.056359095957
+Minimizing 369    Time: 0:00:01 ( 4.80 ms/it)
+  objective:  13519.60772759054
+Minimizing 391    Time: 0:00:01 ( 4.80 ms/it)
+  objective:  13492.50350522164
+Minimizing 412    Time: 0:00:01 ( 4.80 ms/it)
+  objective:  13409.822939818652
+Minimizing 433    Time: 0:00:02 ( 4.82 ms/it)
+  objective:  13373.494186909578
+Minimizing 455    Time: 0:00:02 ( 4.82 ms/it)
+  objective:  13353.467999600267
+Minimizing 477    Time: 0:00:02 ( 4.81 ms/it)
+  objective:  13307.067361501671
+Minimizing 498    Time: 0:00:02 ( 4.82 ms/it)
+  objective:  13301.080914859456
+Minimizing 520    Time: 0:00:02 ( 4.81 ms/it)
+  objective:  13262.430734973896
+Minimizing 542    Time: 0:00:02 ( 4.81 ms/it)
+  objective:  13220.083469635225
+Minimizing 564    Time: 0:00:02 ( 4.81 ms/it)
+  objective:  13143.58809747886
+Minimizing 585    Time: 0:00:02 ( 4.81 ms/it)
+  objective:  13109.042536250374
+Minimizing 606    Time: 0:00:02 ( 4.81 ms/it)
+  objective:  13067.524690211401
+Minimizing 628    Time: 0:00:03 ( 4.81 ms/it)
+  objective:  13065.942942702968
+Minimizing 650    Time: 0:00:03 ( 4.80 ms/it)
+  objective:  13066.532255472805
+Minimizing 672    Time: 0:00:03 ( 4.80 ms/it)
+  objective:  13066.566853468365
+Minimizing 694    Time: 0:00:03 ( 4.80 ms/it)
+  objective:  13066.880043387835
+Minimizing 707    Time: 0:00:03 ( 4.79 ms/it)
+Minimizing 22    Time: 0:00:00 ( 4.68 ms/it)
+  objective:  13794.154003683772
+Minimizing 43    Time: 0:00:00 ( 4.76 ms/it)
+  objective:  13756.627746982058
+Minimizing 64    Time: 0:00:00 ( 4.89 ms/it)
+  objective:  13705.382967517973
+Minimizing 85    Time: 0:00:00 ( 4.86 ms/it)
+  objective:  13685.52768416985
+Minimizing 107    Time: 0:00:00 ( 4.84 ms/it)
+  objective:  13671.312378892324
+Minimizing 129    Time: 0:00:00 ( 4.82 ms/it)
+  objective:  13661.69728603626
+Minimizing 151    Time: 0:00:00 ( 4.81 ms/it)
+  objective:  13604.925277998052
+Minimizing 172    Time: 0:00:00 ( 4.83 ms/it)
+  objective:  13596.698452328332
+Minimizing 193    Time: 0:00:00 ( 4.84 ms/it)
+  objective:  13574.922962707264
+Minimizing 215    Time: 0:00:01 ( 4.83 ms/it)
+  objective:  13508.247427506576
+Minimizing 237    Time: 0:00:01 ( 4.82 ms/it)
+  objective:  13497.034128687104
+Minimizing 259    Time: 0:00:01 ( 4.82 ms/it)
+  objective:  13456.883701287552
+Minimizing 280    Time: 0:00:01 ( 4.82 ms/it)
+  objective:  13439.835180333714
+Minimizing 302    Time: 0:00:01 ( 4.81 ms/it)
+  objective:  13437.083616760072
+Minimizing 323    Time: 0:00:01 ( 4.81 ms/it)
+  objective:  13419.10168892931
+Minimizing 344    Time: 0:00:01 ( 4.81 ms/it)
+  objective:  13357.418124709591
+Minimizing 366    Time: 0:00:01 ( 4.81 ms/it)
+  objective:  13343.382880526886
+Minimizing 387    Time: 0:00:01 ( 4.82 ms/it)
+  objective:  13331.687094072098
+Minimizing 407    Time: 0:00:01 ( 4.84 ms/it)
+  objective:  13320.712612247444
+Minimizing 428    Time: 0:00:02 ( 4.84 ms/it)
+  objective:  13238.815642997404
+Minimizing 449    Time: 0:00:02 ( 4.84 ms/it)
+  objective:  13173.645232293493
+Minimizing 471    Time: 0:00:02 ( 4.83 ms/it)
+  objective:  13152.210993701869
+Minimizing 492    Time: 0:00:02 ( 4.84 ms/it)
+  objective:  13131.427762340492
+Minimizing 513    Time: 0:00:02 ( 4.84 ms/it)
+  objective:  13106.628157707397
+Minimizing 535    Time: 0:00:02 ( 4.83 ms/it)
+  objective:  13053.284148149643
+Minimizing 557    Time: 0:00:02 ( 4.83 ms/it)
+  objective:  13050.517145347476
+Minimizing 579    Time: 0:00:02 ( 4.83 ms/it)
+  objective:  13050.065897029577
+Minimizing 601    Time: 0:00:02 ( 4.82 ms/it)
+  objective:  13049.660644023228
+Minimizing 622    Time: 0:00:03 ( 4.83 ms/it)
+  objective:  13050.174055599113
+Minimizing 644    Time: 0:00:03 ( 4.82 ms/it)
+  objective:  13050.670904996397
+Minimizing 658    Time: 0:00:03 ( 4.82 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Child
(Intercept)-0.01800.0141-1.280.20090.7670
Test: c5.1-0.03900.0447-0.870.38281.2320
Test: c234.15-0.00730.1700-0.040.96600.0698
Test: c2.340.01050.07290.140.88591.4253
Test: c3.4-0.01200.0447-0.270.78761.1523
a10.32100.04836.64<1e-10
Sex: Boys0.20510.014114.54<1e-47
Test: c5.1 & a10.62300.15304.07<1e-04
Test: c234.15 & a10.07630.58090.130.8955
Test: c2.34 & a1-0.07990.2480-0.320.7473
Test: c3.4 & a10.09010.15380.590.5581
Test: c5.1 & Sex: Boys0.15810.04473.540.0004
Test: c234.15 & Sex: Boys-0.82950.1700-4.88<1e-05
Test: c2.34 & Sex: Boys-0.15870.0729-2.180.0296
Test: c3.4 & Sex: Boys-0.05430.0447-1.210.2244
a1 & Sex: Boys0.04940.04831.020.3064
Test: c5.1 & a1 & Sex: Boys0.02780.15300.180.8559
Test: c234.15 & a1 & Sex: Boys-0.41230.5809-0.710.4778
Test: c2.34 & a1 & Sex: Boys0.16700.24800.670.5006
Test: c3.4 & a1 & Sex: Boys0.08880.15380.580.5639
Residual0.0000
+
+
+
+
MixedModels.likelihoodratiotest(m_Child_SDC0, m_Child_SDC)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Child)2613796
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 + Test | Child)361310968710<1e-99
+
+
+
+
MixedModels.likelihoodratiotest(m_Child_HeC0, m_Child_HeC)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Child)2613068
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 + Test | Child)3613300-23210NaN
+
+
+
+
MixedModels.likelihoodratiotest(m_Child_HyC0, m_Child_HyC)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Child)2613066
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 + Test | Child)3613176-11010NaN
+
+
+
+
MixedModels.likelihoodratiotest(m_Child_PCA0, m_Child_PCA)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Child)2613049
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 + Test | Child)3613134-8510NaN
+
+
+

Obviously, we can not drop CPs from any of the LMMs. The full LMMs all have the same objective, but we can compare the goodness-of-fit statistics of zcpLMMs more directly.

+
+
begin
+  zcpLMM = ["SDC0", "HeC0", "HyC0", "PCA0"]
+  mods = [m_Child_SDC0, m_Child_HeC0, m_Child_HyC0, m_Child_PCA0]
+  gof_summary = sort!(
+    DataFrame(;
+      zcpLMM=zcpLMM,
+      dof=dof.(mods),
+      deviance=deviance.(mods),
+      AIC=aic.(mods),
+      BIC=bic.(mods),
+    ),
+    :deviance,
+  )
+end
+
+
4×5 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowzcpLMMdofdevianceAICBIC
StringInt64Float64Float64Float64
1PCA02613049.413101.413270.8
2HyC02613065.813117.813287.2
3HeC02613067.713119.713289.1
4SDC02613796.213848.214017.6
+
+
+
+

The best fit was obtained for the PCA-based zcpLMM. Somewhat surprisingly the second best fit was obtained for the SDC. The relatively poor performance of HeC-based zcpLMM is puzzling to me. I thought it might be related to imbalance in design in the present data, but this does not appear to be the case. The same comparison of SequentialDifferenceCoding and Helmert Coding also showed a worse fit for the zcp-HeC LMM than the zcp-SDC LMM.

+
+
+

3.3 VCs and CPs depend on random factor

+

VCs and CPs resulting from a set of test contrasts can also be estimated for the random factor School. Of course, these VCs and CPs may look different from the ones we just estimated for Child.

+

The effect of age (i.e., developmental gain) varies within School. Therefore, we also include its VCs and CPs in this model; the school-related VC for Sex was not significant.

+
+
f_School = @formula zScore ~
+  1 + Test * a1 * Sex + (1 + Test + a1 | School);
+m_School_SeqDiff = fit(MixedModel, f_School, dat; contrasts=contr1);
+m_School_Helmert = fit(MixedModel, f_School, dat; contrasts=contr2);
+m_School_Hypo = fit(MixedModel, f_School, dat; contrasts=contr3);
+m_School_PCA = fit(MixedModel, f_School, dat; contrasts=contr4);
+
+
Minimizing 70    Time: 0:00:00 ( 1.44 ms/it)
+  objective:  13811.64529664444
+Minimizing 205    Time: 0:00:00 ( 0.98 ms/it)
+  objective:  13774.773213004657
+Minimizing 341    Time: 0:00:00 ( 0.89 ms/it)
+  objective:  13773.325252364972
+Minimizing 477    Time: 0:00:00 ( 0.85 ms/it)
+  objective:  13772.694297112774
+Minimizing 611    Time: 0:00:00 ( 0.83 ms/it)
+  objective:  13772.671154527705
+Minimizing 745    Time: 0:00:00 ( 0.82 ms/it)
+  objective:  13772.670793562127
+Minimizing 792    Time: 0:00:00 ( 0.81 ms/it)
+
+Minimizing 793    Time: 0:00:00 ( 0.81 ms/it)
+  objective:  13772.670780690043
+Minimizing 135    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13840.443004276474
+Minimizing 271    Time: 0:00:00 ( 0.74 ms/it)
+  objective:  13805.146355123181
+Minimizing 405    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13801.599266262609
+Minimizing 540    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13801.36258054134
+Minimizing 675    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13801.3294428708
+Minimizing 810    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13801.318010056366
+Minimizing 944    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13801.315003946416
+Minimizing 1080    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13801.316409231427
+Minimizing 1214    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13801.31406667234
+Minimizing 1271    Time: 0:00:00 ( 0.75 ms/it)
+Minimizing 137    Time: 0:00:00 ( 0.73 ms/it)
+  objective:  13780.812348816995
+Minimizing 273    Time: 0:00:00 ( 0.74 ms/it)
+  objective:  13773.116746328047
+Minimizing 409    Time: 0:00:00 ( 0.74 ms/it)
+  objective:  13772.734800127791
+Minimizing 544    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13772.674846291791
+Minimizing 675    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13772.671079144462
+Minimizing 810    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13772.670785512491
+Minimizing 830    Time: 0:00:00 ( 0.75 ms/it)
+
+Minimizing 831    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13772.67078220487
+Minimizing 137    Time: 0:00:00 ( 0.73 ms/it)
+  objective:  13776.235548768575
+Minimizing 273    Time: 0:00:00 ( 0.74 ms/it)
+  objective:  13772.972354322641
+Minimizing 408    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13772.724964654355
+Minimizing 542    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13772.675673083704
+Minimizing 677    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  13772.671035917809
+Minimizing 772    Time: 0:00:00 ( 0.75 ms/it)
+
+
+
+
VarCorr(m_School_SeqDiff)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
School(Intercept)0.0279860.167291
Test: Star_r0.1939480.440395+0.01
Test: S20_r0.1305050.361255+0.12-0.77
Test: SLJ0.1854940.430690-0.15+0.51-0.67
Test: BPT0.0865210.294144-0.35-0.63+0.45-0.66
a10.1526530.390708+0.23+0.04+0.34-0.24+0.29
Residual0.8375620.915184
+
+
+
+
VarCorr(m_School_Helmert)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
School(Intercept)0.00000000.0000000
Test: Star_r0.05096010.2257435+NaN
Test: S20_r0.00614100.0783648+NaN-0.17
Test: SLJ0.00882940.0939650+NaN+0.47-0.29
Test: BPT0.00106550.0326421+NaN-0.35-0.13+0.47
a10.17822160.4221630+NaN+0.08+0.62-0.06+0.35
Residual0.86596800.9305740
+
+
+
+
VarCorr(m_School_Hypo)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
School(Intercept)0.0279880.167297
Test: BPT-other0.7214570.849386-0.59
Test: Star-End0.1939710.440421+0.01-0.23
Test: S20-Star0.1304710.361208+0.12+0.06-0.77
Test: SLJ-S200.1854610.430652-0.15+0.30+0.51-0.67
a10.1526630.390721+0.23+0.34+0.04+0.34-0.24
Residual0.8375640.915185
+
+
+
+
VarCorr(m_School_PCA)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
School(Intercept)0.0279890.167300
Test: c5.10.1018290.319106-0.36
Test: c234.152.4050341.550817+0.28+0.42
Test: c2.340.2885110.537132-0.04+0.02+0.37
Test: c3.40.1853850.430564+0.15-0.68-0.37-0.10
a10.1526250.390673+0.23+0.38+0.02-0.26+0.24
Residual0.8375750.915191
+
+
+

We compare again how much of the fit resides in the CPs.

+
+
begin
+  f_School0 = @formula zScore ~
+    1 + Test * a1 * Sex + zerocorr(1 + Test + a1 | School)
+  m_School_SDC0 = fit(MixedModel, f_School0, dat; contrasts=contr1)
+  m_School_HeC0 = fit(MixedModel, f_School0, dat; contrasts=contr2)
+  m_School_HyC0 = fit(MixedModel, f_School0, dat; contrasts=contr3)
+  m_School_PCA0 = fit(MixedModel, f_School0, dat; contrasts=contr4)
+  #
+  zcpLMM2 = ["SDC0", "HeC0", "HyC0", "PCA0"]
+  mods2 = [
+    m_School_SDC0, m_School_HeC0, m_School_HyC0, m_School_PCA0
+  ]
+  gof_summary2 = sort!(
+    DataFrame(;
+      zcpLMM=zcpLMM2,
+      dof=dof.(mods2),
+      deviance=deviance.(mods2),
+      AIC=aic.(mods2),
+      BIC=bic.(mods2),
+    ),
+    :deviance,
+  )
+end
+
+
4×5 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowzcpLMMdofdevianceAICBIC
StringInt64Float64Float64Float64
1HeC02713789.513843.514019.5
2PCA02713793.713847.714023.6
3HyC02713797.113851.114027.0
4SDC02713801.213855.214031.2
+
+
+
+

For the random factor School the Helmert contrast, followed by PCA-based contrasts have least information in the CPs; SDC has the largest contribution from CPs. Interesting.

+
+
+
+

4 That’s it

+

That’s it for this tutorial. It is time to try your own contrast coding. You can use these data; there are many alternatives to set up hypotheses for the five tests. Of course and even better, code up some contrasts for data of your own.

+

Have fun!

+
+
+Fühner, T., Granacher, U., Golle, K., & Kliegl, R. (2021). Age and sex effects in physical fitness components of 108,295 third graders including 515 primary schools and 9 cohorts. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-97000-4 +
+
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/contrasts_kwdyz11.html b/contrasts_kwdyz11.html new file mode 100644 index 0000000..b3808e8 --- /dev/null +++ b/contrasts_kwdyz11.html @@ -0,0 +1,2058 @@ + + + + + + + + + + + +Contrast Coding of Visual Attention Effects – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Contrast Coding of Visual Attention Effects

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +
+
+Code +
using Chain
+using DataFrames
+using MixedModels
+using SMLP2024: dataset
+using StatsBase
+using StatsModels
+
+import ProgressMeter
+ProgressMeter.ijulia_behavior(:clear);
+
+
+
+

1 A word of caution

+

For a (quasi-)experimental set of data, there is (or should be) a clear a priori theoretical commitment to specific hypotheses about differences between factor levels and how these differences enter in interactions with other factors. This specification should be used in the first LMM and reported, irrespective of the outcome. If alternative theories lead to alternative a priori contrast specifications, both analyses are justified. If the observed means render the specification completely irrelevant, the comparisons originally planned could still be reported in a Supplement).

+

In this script, we are working through a large number of different contrasts for the same data. The purpose is to introduce both the preprogrammed (“canned”) and the general options to specify hypotheses about main effects and interactions. Obviously, we do not endorse generating a plot of the means and specifying the contrasts accordingly. This is known as the Texas sharpshooter fallacy. The link leads to an illustration and brief historical account by Wagenmakers (2018).

+

Irrespective of how results turn out, there is nothing wrong with specifying a set of post-hoc contrasts to gain a better understanding of what the data are trying to tell us. Of course, in an article or report about the study, the a priori and post-hoc nature of contrast specifications must be made clear. Some kind of alpha-level adjustment (e.g., Bonferroni) may be called for, too. And, of course, there are grey zones.

+

There is quite a bit of statistical literature on contrasts. Two “local” references are Brehm & Alday (2022) and Schad et al. (2020).

+

For further readings see “Further Readings” in Schad et al. (2020).

+
+
+

2 Example data

+

We take the KWDYZ dataset from Kliegl et al. (2011). This is an experiment looking at three effects of visual cueing under four different cue-target relations (CTRs). Two horizontal rectangles are displayed above and below a central fixation point or they displayed in vertical orientation to the left and right of the fixation point. Subjects react to the onset of a small visual target occurring at one of the four ends of the two rectangles. The target is cued validly on 70% of trials by a brief flash of the corner of the rectangle at which it appears; it is cued invalidly at the three other locations 10% of the trials each.

+

We specify three contrasts for the four-level factor CTR that are derived from spatial, object-based, and attractor-like features of attention. They map onto sequential differences between appropriately ordered factor levels.

+

We also have a dataset from a replication and extension of this study Kliegl et al. (2015) Both data sets are available in R-package RePsychLing

+
+
+

3 Preprocessing

+
+
dat1 = DataFrame(dataset(:kwdyz11))
+cellmeans = combine(
+  groupby(dat1, [:CTR]),
+  :rt => mean,
+  :rt => std,
+  :rt => length,
+  :rt => (x -> std(x) / sqrt(length(x))) => :rt_semean,
+)
+
+
4×5 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowCTRrt_meanrt_stdrt_lengthrt_semean
StringFloat32Float32Int64Float64
1val358.03283.4579201410.588067
2sod391.26792.66228631.73177
3dos405.14692.689228431.73836
4dod402.395.391328631.78278
+
+
+
+
+
+

4 Julia contrast options

+

We use the same formula for all analyses

+
+
form = @formula rt ~ 1 + CTR + (1 + CTR | Subj)
+
+

This is the default order of factor levels.

+
+
show(StatsModels.levels(dat1.CTR))
+
+
["val", "sod", "dos", "dod"]
+
+
+

Controlling the ordering of levels for contrasts:

+
    +
  1. kwarg levels to order the levels
    2. +
  2. The first level is set as the baseline; with kwarg base a different level can be specified.
    3. +
+
+

4.1 SeqDiffCoding

+

The SeqDiffCoding contrast corresponds to MASS::contr.sdif() in R. The assignment of random factors such as Subj to Grouping() is necessary when the sample size is very large. We recommend to include it always, but in this tutorial we do so only in the first example.

+
+
m1 = let levels = ["val", "sod", "dos", "dod"]
+  contrasts = Dict(
+    :CTR => SeqDiffCoding(; levels),
+    :Subj => Grouping()
+  )
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)389.73367.090654.97<1e-9955.1938
CTR: sod33.78173.286910.28<1e-2423.2439
CTR: dos13.98522.30516.07<1e-0810.7456
CTR: dod-2.74692.2140-1.240.21479.5064
Residual69.8349
+
+
+
+
+

4.2 HypothesisCoding

+

HypothesisCoding is the most general option available. We can implement all “canned” contrasts ourselves. The next example reproduces the test statistcs from SeqDiffCoding - with a minor modification illustrating the flexibility of going beyond the default version.

+
+
m1b = let levels = ["val", "sod", "dos", "dod"]
+  contrasts = Dict(
+    :CTR => HypothesisCoding(
+      [
+        -1  1 0  0
+         0 -1 1  0
+         0  0 1 -1
+      ];
+      levels,
+      labels=["spt", "obj", "grv"],
+    ),
+  )
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)389.73367.091054.96<1e-9955.1969
CTR: spt33.78173.287510.28<1e-2423.2490
CTR: obj13.98522.30546.07<1e-0810.7491
CTR: grv2.74702.21391.240.21479.5049
Residual69.8349
+
+
+

The difference to the preprogrammed SeqDiffCoding is that for the third contrast we changed the direction of the contrast such that the sign of the effect is positive when the result is in agreement with theoretical expectation, that is we subtract the fourth level from the third, not the third level from the fourth.

+
+
+

4.3 DummyCoding

+

This contrast corresponds to contr.treatment() in R

+
+
m2 = let
+  contrasts = Dict(:CTR => DummyCoding(; base="val"))
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)358.09146.154058.19<1e-9947.9101
CTR: dod45.02004.363410.32<1e-2432.2899
CTR: dos47.76693.556513.43<1e-4025.5361
CTR: sod33.78173.287410.28<1e-2423.2483
Residual69.8348
+
+
+

The DummyCoding contrast has the disadvantage that the intercept returns the mean of the level specified as base, default is the first level, not the GM.

+
+
+

4.4 YchycaeitCoding

+

The contrasts returned by DummyCoding may be exactly what we want. Can’t we have them, but also have the intercept estimate the GM, rather than the mean of the base level? Yes, we can! We call this “You can have your cake and it eat, too”-Coding (YchycaeitCoding). And we use HypothesisCoding to achieve this outcome.

+
+
m2b = let levels = ["val", "sod", "dos", "dod"]
+  contrasts = Dict(
+    :CTR => HypothesisCoding(
+      [
+        -1 1 0 0
+        -1 0 1 0
+        -1 0 0 1
+      ];
+      levels,
+      labels=levels[2:end],
+    )
+  )
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)389.73367.091154.96<1e-9955.1983
CTR: sod33.78173.287410.28<1e-2423.2487
CTR: dos47.76693.556813.43<1e-4025.5386
CTR: dod45.02004.363510.32<1e-2432.2906
Residual69.8348
+
+
+

We can simply relevel the factor or move the column with -1s for a different base.

+
+
m2c = let levels = ["val", "sod", "dos", "dod"]
+  contrasts = Dict(
+    :CTR => HypothesisCoding(
+      [
+        -1/2 1/2   0   0
+        -1/2   0 1/2   0
+        -1/2   0   0  1/2
+      ];
+      levels,
+      labels=levels[2:end],
+    )
+  )
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)389.73367.093054.95<1e-9955.2127
CTR: sod16.89091.643710.28<1e-2411.6245
CTR: dos23.88351.778113.43<1e-4012.7664
CTR: dod22.51002.181810.32<1e-2416.1459
Residual69.8349
+
+
+

We can simply relevel the factor or move the column with -1s for a different base.

+
+
+

4.5 EffectsCoding

+

This contrast corresponds almost to contr.sum() in R.

+
+
m3 = let
+  contrasts = Dict(:CTR => EffectsCoding(; base="dod"))
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)389.73367.090654.96<1e-9955.1943
CTR: dos16.12481.440311.20<1e-287.3306
CTR: sod2.13961.33371.600.10876.0065
CTR: val-31.64222.6421-11.98<1e-3219.9491
Residual69.8349
+
+
+

The “almost” qualification refers to the fact that contr.sum() uses the last factor levels as default base level; EffectsCoding uses the first level.

+
+
m3b = let levels = [ "dod", "val", "sod", "dos"]
+  contrasts = Dict(
+    :CTR => HypothesisCoding(
+      [
+         -1/4   3/4 -1/4  -1/4
+         -1/4  -1/4  3/4  -1/4
+         -1/4  -1/4 -1/4   3/4
+      ];
+      levels,
+      labels=levels[2:end],
+    )
+  )
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)389.73367.091054.96<1e-9955.1975
CTR: val-31.64222.6421-11.98<1e-3219.9493
CTR: sod2.13961.33381.600.10876.0076
CTR: dos16.12481.440411.19<1e-287.3317
Residual69.8348
+
+
+
+
m3c = let levels = [ "dod", "val", "sod", "dos"]
+  contrasts = Dict(
+    :CTR => HypothesisCoding(
+      [
+         -1/2   3/2 -1/2  -1/2
+         -1/2  -1/2  3/2  -1/2
+         -1/2  -1/2 -1/2   3/2
+      ];
+      levels,
+      labels=levels[2:end],
+    )
+  )
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)389.73367.090854.96<1e-9955.1959
CTR: val-63.28435.2842-11.98<1e-3239.8982
CTR: sod4.27912.66761.600.108712.0152
CTR: dos32.24952.880811.19<1e-2814.6631
Residual69.8349
+
+
+
+
+

4.6 HelmertCoding

+

HelmertCoding codes each level as the difference from the average of the lower levels. With the default order of CTR levels we get the following test statistics. These contrasts are othogonal.

+
+
m4 = let
+  contrasts = Dict(:CTR => HelmertCoding())
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)389.73367.094054.94<1e-9955.2207
CTR: dos1.37351.10721.240.21484.7558
CTR: sod-4.20390.6843-6.14<1e-093.3493
CTR: val-10.54740.8807-11.98<1e-326.6500
Residual69.8348
+
+
+
+ HeC1: (2 - 1)/2           # (391 - 358)/2
++ HeC2: (3 - (2+1)/2)/3     # (405 - (391 + 358)/2)/3
++ HeC3: (4 - (3+2+1)/3)/4   # (402 - (405 + 391 + 358)/3)/4
+
+
+

4.7 Reverse HelmertCoding

+

Reverse HelmertCoding codes each level as the difference from the average of the higher levels. To estimate these effects we simply reverse the order of factor levels. Of course, the contrasts are also orthogonal.

+
+
m4b = let levels = reverse(StatsModels.levels(dat1.CTR))
+  contrasts = Dict(:CTR => HelmertCoding(; levels))
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)389.73367.094054.94<1e-9955.2207
CTR: dos1.37351.10721.240.21484.7558
CTR: sod-4.20390.6843-6.14<1e-093.3493
CTR: val-10.54740.8807-11.98<1e-326.6500
Residual69.8348
+
+
+
+ HeC1:(3 - 4)/2            # (405 - 402)/2
++ HeC2:(2 - (3+4)/2)/3      # (391 - (405 + 402)/2)/3
++ HeC3:(1 - (2+3+4)/3/4     # (356  -(391 + 405 + 402)/3)/4
+
+
+

4.8 Anova Coding

+

Factorial designs (i.e., lab experiments) are traditionally analyzed with analysis of variance. The test statistics of main effects and interactions are based on an orthogonal set of contrasts. We specify them with HypothesisCoding.

+
+

4.8.1 A(2) x B(2)

+

An A(2) x B(2) design can be recast as an F(4) design with the levels (A1-B1, A1-B2, A2-B1, A2-B2). The following contrast specification returns estimates for the main effect of A, the main effect of B, and the interaction of A and B. In a figure With A on the x-axis and the levels of B shown as two lines, the interaction tests the null hypothesis that the two lines are parallel. A positive coefficient implies overadditivity (diverging lines toward the right) and a negative coefficient underadditivity (converging lines).

+
+
m5 = let levels = ["val", "sod", "dos", "dod"]
+  contrasts = Dict(
+    :CTR => HypothesisCoding(
+      [
+        -1 -1 +1 +1          # A
+        -1 +1 -1 +1          # B
+        +1 -1 -1 +1          # A x B
+      ];
+      levels,
+      labels=["A", "B", "AxB"],
+    ),
+  )
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)389.73367.092454.95<1e-9955.2085
CTR: A59.00525.181811.39<1e-2936.2007
CTR: B31.03484.67516.64<1e-1031.7147
CTR: AxB-36.52863.0925-11.81<1e-3116.0032
Residual69.8348
+
+
+

It is also helpful to see the corresponding layout of the four means for the interaction of A and B (i.e., the third contrast)

+
        B1     B2
+   A1   +1     -1
+   A2   -1     +1
+

Thus, interaction tests whether the difference between main diagonal and minor diagonal is different from zero.

+
+
+

4.8.2 A(2) x B(2) x C(2)

+

Going beyond the four level factor; it is also helpful to see the corresponding layout of the eight means for the interaction of A and B and C.

+
          C1              C2
+      B1     B2        B1     B2
+ A1   +1     -1   A1   -1     +1
+ A2   -1     +1   A2   +1     -1
+
+
+

4.8.3 A(2) x B(2) x C(3)

+

TO BE DONE

+
+
+
+

4.9 Nested coding

+

Nested contrasts are often specified as follow up as post-hoc tests for ANOVA interactions. They are orthogonal. We specify them with HypothesisCoding.

+

An A(2) x B(2) design can be recast as an F(4) design with the levels (A1-B1, A1-B2, A2-B1, A2-B2). The following contrast specification returns an estimate for the main effect of A and the effects of B nested in the two levels of A. In a figure With A on the x-axis and the levels of B shown as two lines, the second contrast tests whether A1-B1 is different from A1-B2 and the third contrast tests whether A2-B1 is different from A2-B2.

+
+
m8 = let levels = ["val", "sod", "dos", "dod"]
+  contrasts = Dict(
+    :CTR => HypothesisCoding(
+      [
+        -1 -1 +1 +1
+        -1 +1  0  0
+         0  0 +1 -1
+      ];
+      levels,
+      labels=["do_so", "spt", "grv"],
+    ),
+  )
+  fit(MixedModel, form, dat1; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)389.73367.091354.96<1e-9955.1998
CTR: do_so59.00525.182611.39<1e-2936.2082
CTR: spt33.78173.287410.28<1e-2423.2481
CTR: grv2.74702.21421.240.21489.5101
Residual69.8348
+
+
+

The three contrasts for one main effect and two nested contrasts are orthogonal. There is no test of the interaction (parallelism).

+
+
+
+

5 Other orthogonal contrasts

+

For factors with more than four levels there are many options for specifying orthogonal contrasts as long as one proceeds in a top-down strictly hiearchical fashion.

+

Suppose you have a factor with seven levels and let’s ignore shifting columns. In this case, you have six options for the first contrast, that is 6 vs. 1, 5 vs.2 , 4 vs. 3, 3 vs. 4, 2 vs. 5, and 1 vs. 6 levels. Then, you specify orthogonal contrasts for partitions with more than 2 elements and so on. That is, you don’t specify a contrast that crosses an earlier partition line.

+

In the following example, after an initial 4 vs 3 partitioning of levels, we specify AnovaCoding for the left and HelmertCoding for the right partition.

+
+
contrasts = Dict(
+  :CTR => HypothesisCoding(
+    [
+      -1/4 -1/4 -1/4 -1/4 +1/3 +1/3 +1/3
+      -1/2 -1/2 +1/2 +1/2    0    0    0
+      -1/2 +1/2 -1/2 +1/2    0    0    0
+      +1/2 -1/2 -1/2 +1/2    0    0    0
+         0    0    0    0   -1   +1    0
+         0    0    0    0 -1/2 -1/2    1
+    ];
+    levels=["A1", "A2", "A3", "A4", "A5", "A6", "A7"],
+    labels=["c567.1234", "B", "C", "BxC", "c6.5", "c6.56"],
+  ),
+);
+
+

There are two rules that hold for all orthogonal contrasts:

+
    +
  1. The weights within rows sum to zero.
    2. +
  2. For all pairs of rows, the sum of the products of weights in the same columns sums to zero.
    3. +
+
+
+

6 Appendix: Summary (Dave Kleinschmidt)

+

StatsModels

+

StatsModels.jl provides a few commonly used contrast coding schemes, some less-commonly used schemes, and structs that allow you to manually specify your own, custom schemes.

+
+

6.1 Standard contrasts

+

The most commonly used contrasts are DummyCoding and EffectsCoding (which are similar to contr.treatment() and contr.sum() in R, respectively).

+
+
+

6.2 “Exotic” contrasts (rk: well …)

+

We also provide HelmertCoding and SeqDiffCoding (corresponding to base R’s contr.helmert() and MASS::contr.sdif()).

+
+
+

6.3 Manual contrasts

+

ContrastsCoding()

+

There are two ways to manually specify contrasts. First, you can specify them directly via ContrastsCoding. If you do, it’s good practice to specify the levels corresponding to the rows of the matrix, although they can be omitted in which case they’ll be inferred from the data.

+

HypothesisCoding()

+

A better way to specify manual contrasts is via HypothesisCoding, where each row of the matrix corresponds to the weights given to the cell means of the levels corresponding to each column (see Schad et al. (2020) for more information).

+ + + +
+
+ + Back to top

References

+
+Brehm, L., & Alday, P. M. (2022). Contrast coding choices in a decade of mixed models. Journal of Memory and Language, 125, 104334. https://doi.org/10.1016/j.jml.2022.104334 +
+
+Kliegl, R., Kushela, J., & Laubrock, J. (2015). Object orientation and target size modulate the speed of visual attention. Department of Psychology, University of Potsdam. +
+
+Kliegl, R., Wei, P., Dambacher, M., Yan, M., & Zhou, X. (2011). Experimental effects and individual differences in linear mixed models: Estimating the relationship between spatial, object, and attraction effects in visual attention. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2010.00238 +
+
+Schad, D. J., Vasishth, S., Hohenstein, S., & Kliegl, R. (2020). How to capitalize on a priori contrasts in linear (mixed) models: A tutorial. Journal of Memory and Language, 110, 104038. https://doi.org/10.1016/j.jml.2019.104038 +
+
+ + +
+ + + + + \ No newline at end of file diff --git a/fggk21.html b/fggk21.html new file mode 100644 index 0000000..f3b6f5b --- /dev/null +++ b/fggk21.html @@ -0,0 +1,8567 @@ + + + + + + + + + + + +Basics with Emotikon Project – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Basics with Emotikon Project

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

This script uses a subset of data reported in Fühner et al. (2021). To circumvent delays associated with model fitting we work with models that are less complex than those in the reference publication. All the data to reproduce the models in the publication are used here, too; the script requires only a few changes to specify the more complex models in the article.

+

The script is structured in four main sections:

+
    +
  1. Setup with reading and examining the data, plotting the main results, and specifying the contrasts for the fixed factor Test
    2. +
  2. a demonstration of model complexification to determine a parsimonious random-effect structure appropriate for and supported by the data, including also a quite elaborate demonstration of principle component analyses (PCAs) of levels (scores) and effects,
    3. +
  3. specification of nested fixed effects or interactions in the levels of another, superordinate factors,
    4. +
  4. a Glossary of MixedModels.jl commands to inspect the information generated for a fitted model object.
    5. +
+
+

1 Packages and functions

+
+
+Code +
using AlgebraOfGraphics
+using AlgebraOfGraphics: linear
+using Arrow
+using CairoMakie
+using CategoricalArrays
+using Chain
+using DataFrameMacros
+using DataFrames
+using MixedModels
+using MixedModelsMakie
+using MixedModelsMakie: simplelinreg
+using ProgressMeter
+using Random
+using Statistics
+using StatsBase
+using SMLP2024: dataset
+
+ProgressMeter.ijulia_behavior(:clear)
+CairoMakie.activate!(; type="svg")
+
+
+
+
+

2 Readme

+

Number of scores: 525126 in dataset(:fggk21)

+
    +

  1. Cohort: 9 levels; 2011-2019

    2. +

  2. School: 515 levels

    3. +

  3. Child: 108295 levels; all children are between 8.0 and 8.99 years old

    4. +

  4. Sex: “Girls” (n=55,086), “Boys” (n= 53,209)

    5. +

  5. age: testdate - middle of month of birthdate

    6. +

  6. Test: 5 levels

    +
      +
    • Endurance (Run): 6 minute endurance run [m]; to nearest 9m in 9x18m field
      • +
    • Coordination (Star_r): star coordination run [m/s]; 9x9m field, 4 x diagonal = 50.912 m
      • +
    • Speed(S20_r): 20-meters sprint [m/s]
      • +
    • Muscle power low (SLJ): standing long jump [cm]
      • +
    • Muscle power up (BPT): 1-kg medicine ball push test [m]
      • +
    7. +

  7. score - see units

    8. +
+
+
+

3 Preprocessing

+
+

3.1 Read data

+
+
df = DataFrame(dataset(:fggk21))
+transform!(df,
+    :age => (x -> x .- 8.5) => :a1,
+    :Sex => categorical => :Sex,
+    :Test => categorical => :Test,
+  )
+levels!(df.Sex, ["male", "female"])
+recode!(df.Sex, "male" => "Boys", "female" => "Girls")
+levels!(df.Test, ["Run", "Star_r", "S20_r", "SLJ", "BPT"])
+recode!(
+  df.Test,
+  "Run" => "Endurance",
+  "Star_r" => "Coordination",
+  "S20_r" => "Speed",
+  "SLJ" => "PowerLOW",
+  "BPT" => "PowerUP",
+)
+describe(df)
+
+
8×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1Cohort201120190String
2SchoolS100043S8002000String
3ChildC002352C1179660String
4SexBoysGirls0CategoricalValue{String, UInt32}
5age8.560737.994528.558529.106090Float64
6TestEndurancePowerUP0CategoricalValue{String, UInt32}
7score226.1411.141524.651161530.00Float64
8a10.0607297-0.5054760.05852160.6060920Float64
+
+
+
+
+

3.1.1 Transformations

+

We center age at 8.5 years and compute z-scores for each Test. With these variables the data frame df contains all variables used for the final model in the original publication.

+
+
select!(groupby(df, :Test),  Not(:score), :score => zscore => :zScore)
+
+
525126×8 DataFrame
525101 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowTestCohortSchoolChildSexagea1zScore
Cat…StringStringStringCat…Float64Float64Float64
1Speed2013S100067C002352Boys7.99452-0.5054761.7913
2PowerUP2013S100067C002352Boys7.99452-0.505476-0.0622317
3PowerLOW2013S100067C002352Boys7.99452-0.505476-0.0336567
4Coordination2013S100067C002352Boys7.99452-0.5054761.46874
5Endurance2013S100067C002352Boys7.99452-0.5054760.331058
6Speed2013S100067C002353Boys7.99452-0.5054761.15471
7PowerUP2013S100067C002353Boys7.99452-0.5054760.498354
8PowerLOW2013S100067C002353Boys7.99452-0.505476-0.498822
9Coordination2013S100067C002353Boys7.99452-0.505476-0.9773
10Endurance2013S100067C002353Boys7.99452-0.5054760.574056
11Speed2013S100067C002354Boys7.99452-0.5054760.0551481
12PowerUP2013S100067C002354Boys7.99452-0.5054760.218061
13PowerLOW2013S100067C002354Boys7.99452-0.505476-0.757248
⋮⋮⋮⋮⋮⋮⋮⋮⋮
525115Coordination2018S401470C117964Boys9.106090.606092-1.43175
525116Endurance2018S401470C117964Boys9.106090.606092-0.944681
525117Speed2018S401470C117965Girls9.106090.6060920.31086
525118PowerUP2018S401470C117965Girls9.106090.6060920.0779146
525119PowerLOW2018S401470C117965Girls9.106090.606092-0.137027
525120Coordination2018S401470C117965Girls9.106090.606092-1.8077
525121Endurance2018S401470C117965Girls9.106090.6060920.513306
525122Speed2018S800200C117966Boys9.106090.6060920.0551481
525123PowerUP2018S800200C117966Boys9.106090.6060920.0779146
525124PowerLOW2018S800200C117966Boys9.106090.606092-1.32578
525125Coordination2018S800200C117966Boys9.106090.6060920.473217
525126Endurance2018S800200C117966Boys9.106090.606092-0.0941883
+
+
+
+
+
+
+

3.2 Extract a stratified subsample

+

For the prupose of the tutorial, we extract a random sample of 1000 boys and 1000 girls. Child, School, and Cohort are grouping variables. Traditionally, they are called random factors because the units (levels) of the factor are assumed to be a random sample from the population of their units (levels).

+

Cohort has only nine “groups” and could have been included as a set of polynomical fixed-effect contrasts rather than a random factor. This choice warrants a short excursion: The secular trends are very different for different tests and require the inclusion of interaction terms with Test contrasts (see Figure 4 in (Fühner et al., 2021). The authors opted to absorb these effects in cohort-related variance components for the Test contrasts and plan to address the details of secular changes in a separate analysis.

+

For complex designs, when they are in the theoretical focus of an article, factors and covariates should be specified as part of the fixed effects. If they are not in the theoretical focus, but serve as statistical control variables, they could be put in the RES - if supported by the data.

+

Stratified sampling: We generate a Child table with information about children. MersenneTwister(42) specifies 42 as the seed for the random number generator to ensure reproducibility of the stratification. For a different pattern of results choose, for example, 84. We randomly sample 1000 boys and 1000 girls from this table; they are stored in samp. Then, we extract the corresponding subset of these children’s test scores from df and store them dat.

+
+
Child = unique(select(df, :Cohort, :School, :Child, :Sex, :age))
+sample = let
+  rng = MersenneTwister(42)
+  combine(
+    groupby(Child, :Sex), x -> x[rand(rng, 1:nrow(x), 1000), :]
+  )
+end
+insamp(x) = x ∈ sample.Child
+dat = @subset(df, insamp(:Child))
+
+
9663×8 DataFrame
9638 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowTestCohortSchoolChildSexagea1zScore
Cat…StringStringStringCat…Float64Float64Float64
1Speed2013S101540C002482Girls7.99452-0.5054760.0551481
2PowerUP2013S101540C002482Girls7.99452-0.505476-0.342524
3PowerLOW2013S101540C002482Girls7.99452-0.5054760.74162
4Coordination2013S101540C002482Girls7.99452-0.505476-0.209186
5Endurance2013S101540C002482Girls7.99452-0.505476-0.127938
6Speed2013S102090C002513Girls7.99452-0.505476-0.422921
7PowerUP2013S102090C002513Girls7.99452-0.505476-0.762963
8PowerLOW2013S102090C002513Girls7.99452-0.505476-0.188712
9Coordination2013S102090C002513Girls7.99452-0.5054760.81382
10Endurance2013S102090C002513Girls7.99452-0.5054760.452557
11Speed2013S103299C002621Girls7.99452-0.5054760.859704
12PowerUP2013S103299C002621Girls7.99452-0.5054760.218061
13PowerLOW2013S103299C002621Girls7.99452-0.5054760.74162
⋮⋮⋮⋮⋮⋮⋮⋮⋮
9652Coordination2018S111508C117805Girls9.106090.606092-0.95589
9653Endurance2018S111508C117805Girls9.106090.606092-1.67367
9654Speed2018S111648C117827Boys9.106090.6060920.859704
9655PowerUP2018S111648C117827Boys9.106090.6060920.6385
9656PowerLOW2018S111648C117827Boys9.106090.6060920.483194
9657Coordination2018S111648C117827Boys9.106090.6060920.708474
9658Endurance2018S111648C117827Boys9.106090.606092-0.337186
9659Speed2018S112197C117868Boys9.106090.6060920.578748
9660PowerUP2018S112197C117868Boys9.106090.6060920.498354
9661PowerLOW2018S112197C117868Boys9.106090.606092-0.550508
9662Coordination2018S112197C117868Boys9.106090.6060920.538978
9663Endurance2018S112197C117868Boys9.106090.6060920.938553
+
+
+
+

Due to missing scores for some tests we have about 2% less than 10,000 observtions.

+
+
+

3.3 No evidence for age x Sex x Test interaction

+

The main results are captured in the figure constructed in this section. We build it both for the full data and the stratified subset.

+
+
df2 = combine(
+  groupby(
+    select(df, :, :age => ByRow(x -> round(x; digits=1)) => :age),
+    [:Sex, :Test, :age],
+  ),
+  :zScore => mean => :zScore,
+  :zScore => length => :n,
+)
+
+
120×5 DataFrame
95 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowSexTestagezScoren
Cat…Cat…Float64Float64Int64
1BoysSpeed8.0-0.02651381223
2BoysPowerUP8.00.0269731227
3BoysPowerLOW8.00.1216091227
4BoysCoordination8.0-0.05717261186
5BoysEndurance8.00.2926951210
6GirlsSpeed8.0-0.351641411
7GirlsPowerUP8.0-0.6103551417
8GirlsPowerLOW8.0-0.2798721418
9GirlsCoordination8.0-0.2682211381
10GirlsEndurance8.0-0.2455731387
11BoysSpeed8.10.06083973042
12BoysPowerUP8.10.09554133069
13BoysPowerLOW8.10.1230993069
⋮⋮⋮⋮⋮⋮
109BoysCoordination9.00.2549734049
110BoysEndurance9.00.2580824034
111GirlsSpeed9.1-0.02861721154
112GirlsPowerUP9.1-0.07523011186
113GirlsPowerLOW9.1-0.0945871174
114GirlsCoordination9.10.002762521162
115GirlsEndurance9.1-0.2355911150
116BoysSpeed9.10.3257451303
117BoysPowerUP9.10.6164161320
118BoysPowerLOW9.10.2675771310
119BoysCoordination9.10.2543421297
120BoysEndurance9.10.2510451294
+
+
+
+
+

3.3.1 Figure(s) of interaction

+

The core results of the article are reported in Figure 2 of Fühner et al. (2021). In summary:

+
    +
  • Main effects of age and Sex: There are developmental gains in the ninth year of life; boys outperform girls. There is no main effect of Test because of z-scoring.
    • +
  • Interactions of Test and age: Tests differ in how much children improve during the year (i.e., the magnitude of developmental gain), that is slopes depend on Test.
    • +
  • Interactions of Test and Sex: The sex difference is test dependent, that is the difference between the slopes depends on Test.
    • +
  • The most distinctive result is the absence of evidence for an age x Sex x Test interaction, that is the slopes for boys and girls are statistically parallel for each of the five tests.
    • +
+
+
+Code +
let
+  design1 = mapping(:age, :zScore; color=:Sex, col=:Test)
+  lines1 = design1 * linear()
+  means1 = design1 * visual(Scatter; markersize=5)
+  draw(data(df2) * means1 + data(df) * lines1;)
+end
+
+
+
+
+
+ +
+
+Figure 1: Age trends by sex for each Test for the full data set +
+
+
+
+
+

Figure 1 shows performance differences for the full set of data between 8.0 and 9.2 years by sex in the five physical fitness tests presented as z-transformed data computed separately for each test.

+
    +
  • Endurance = cardiorespiratory endurance (i.e., 6-min-run test),
    • +
  • Coordination = star-run test,
    • +
  • Speed = 20-m linear sprint test,
    • +
  • PowerLOW = power of lower limbs (i.e., standing long jump test),
    • +
  • PowerUP = power of upper limbs (i.e., ball push test),
    • +
  • SD = standard deviation. Points are binned observed child means; lines are simple regression fits to the observations.
    • +
+

What do the results look like for the stratified subsample? Here the parallelism is much less clear. In the final LMM we test whether the two regression lines in each of the five panels are statistically parallel for this subset of data. That is, we test the interaction of Sex and age as nested within the levels of Test. Most people want to know the signficance of these five Sex x age interactions.

+

The theoretical focus of the article, however, is on comparisons between tests displayed next to each other. We ask whether the degree of parallelism is statistically the same for Endurance and Coordination (H1), Coordination and Speed (H2), Speed and PowerLOW (H3), and PowerLow and PowerUP (H4). Hypotheses H1 to H4 require Sequential Difference contrasts c1 to c4 for Test; they are tested as fixed effects for`H1 x age x Sex, H2 x age x Sex, H3 x age x Sex, and H4 x age x Sex.

+
+
+Code +
dat2 = combine(
+  groupby(
+    select(dat, :, :age => ByRow(x -> round(x; digits=1)) => :age),
+    [:Sex, :Test, :age],
+  ),
+  :zScore => mean => :zScore,
+  :zScore => length => :n,
+)
+
+
+
120×5 DataFrame
95 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowSexTestagezScoren
Cat…Cat…Float64Float64Int64
1GirlsSpeed8.0-0.32311428
2GirlsPowerUP8.0-0.59047626
3GirlsPowerLOW8.00.067799227
4GirlsCoordination8.00.027331825
5GirlsEndurance8.0-0.1733726
6BoysSpeed8.00.39463419
7BoysPowerUP8.00.32870319
8BoysPowerLOW8.00.10507719
9BoysCoordination8.0-0.17001819
10BoysEndurance8.00.40708419
11GirlsSpeed8.1-0.20593454
12GirlsPowerUP8.1-0.65396154
13GirlsPowerLOW8.1-0.15712754
⋮⋮⋮⋮⋮⋮
109GirlsCoordination9.0-0.00018884268
110GirlsEndurance9.0-0.23444868
111GirlsSpeed9.1-0.28504725
112GirlsPowerUP9.1-0.11268425
113GirlsPowerLOW9.1-0.11764524
114GirlsCoordination9.1-0.12784425
115GirlsEndurance9.1-0.58749624
116BoysSpeed9.10.37980817
117BoysPowerUP9.10.52308517
118BoysPowerLOW9.10.29469617
119BoysCoordination9.10.20930916
120BoysEndurance9.10.26651216
+
+
+
+
+
+Code +
let
+  design2 = mapping(:age, :zScore; color=:Sex, col=:Test)
+  lines2 = design2 * linear()
+  means2 = design2 * visual(Scatter; markersize=5)
+  draw(data(dat2) * means2 + data(dat) * lines2;)
+end
+
+
+
+
+
+ +
+
+Figure 2: Age trends by sex for each Test for the stratified sample +
+
+
+
+
+

Figure 2 Performance differences for subset of data between 8.0 and 9.2 years by sex in the five physical fitness tests presented as z-transformed data computed separately for each test.

+
    +
  • Endurance = cardiorespiratory endurance (i.e., 6-min-run test),
    • +
  • Coordination = star-run test,
    • +
  • Speed = 20-m linear sprint test,
    • +
  • PowerLOW = power of lower limbs (i.e., standing long jump test),
    • +
  • PowerUP = power of upper limbs (i.e., ball push test),
    • +
  • SD = standard deviation. Points are binned observed child means; lines are simple regression fits to the observations.
    • +
+
+
+

3.3.2 Regression on age by Sex for each Test

+

Another set of relevant statistics are the slopes for the regression of performance on age for boys and girls in each of the five tests. The lines in Figures 1 and 2, however, are computed directly from the raw data with the linear() command.

+
+
combine(
+  groupby(df, [:Sex, :Test]),
+  [:age, :zScore] => simplelinreg => :coef,
+)
+
+
10×3 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowSexTestcoef
Cat…Cat…Tuple…
1BoysEndurance(0.00256718, 0.0291899)
2BoysCoordination(-2.47279, 0.302819)
3BoysSpeed(-2.12689, 0.267153)
4BoysPowerLOW(-1.4307, 0.189659)
5BoysPowerUP(-4.35864, 0.549005)
6GirlsEndurance(-0.692022, 0.0523217)
7GirlsCoordination(-2.50524, 0.279119)
8GirlsSpeed(-2.34431, 0.255687)
9GirlsPowerLOW(-1.87241, 0.196917)
10GirlsPowerUP(-4.82271, 0.524799)
+
+
+
+
+
combine(
+  groupby(dat, [:Sex, :Test]),
+  [:age, :zScore] => simplelinreg => :coef,
+)
+
+
10×3 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowSexTestcoef
Cat…Cat…Tuple…
1BoysEndurance(0.39203, -0.0150694)
2BoysCoordination(-3.33518, 0.401051)
3BoysSpeed(-1.75685, 0.228662)
4BoysPowerLOW(-1.06646, 0.151546)
5BoysPowerUP(-4.15536, 0.5245)
6GirlsEndurance(0.941712, -0.141158)
7GirlsCoordination(-0.681898, 0.0714891)
8GirlsSpeed(-0.786382, 0.0725931)
9GirlsPowerLOW(-0.208472, 0.00150731)
10GirlsPowerUP(-5.23593, 0.570806)
+
+
+
+
+
+
+

3.4 SeqDiffCoding of Test

+

SeqDiffCoding was used in the publication. This specification tests pairwise differences between the five neighboring levels of Test, that is:

+
    +
  • H1: Star_r - Run (2-1)
    • +
  • H2: S20_r - Star_r (3-2)
    • +
  • H3: SLJ - S20_r (4-3)
    • +
  • H4: BPT - SLJ (5-4)
    • +
+

The levels were sorted such that these contrasts map onto four a priori hypotheses; in other words, they are theoretically motivated pairwise comparisons. The motivation also encompasses theoretically motivated interactions with Sex. The order of levels can also be explicitly specified during contrast construction. This is very useful if levels are in a different order in the dataframe.

+

Note that random factors Child, School, and Cohort are declared as Grouping variables. Technically, this specification is required for variables with a very large number of levels (e.g., 100K+ children). We recommend the explicit specification for all random factors as a general coding style.

+

The first command recodes names indicating the physical fitness components used in the above figures and tables back to the shorter actual test names. This reduces clutter in LMM outputs.

+
+
recode!(
+  dat.Test,
+  "Endurance" => "Run",
+  "Coordination" => "Star_r",
+  "Speed" => "S20_r",
+  "PowerLOW" => "SLJ",
+  "PowerUP" => "BMT",
+)
+contrasts = merge(
+  Dict(nm => SeqDiffCoding() for nm in (:Test, :Sex)),
+  Dict(nm => Grouping() for nm in (:Child, :School, :Cohort)),
+);
+
+

The statistical disadvantage of SeqDiffCoding is that the contrasts are not orthogonal, that is the contrasts are correlated. This is obvious from the fact that levels 2, 3, and 4 are all used in two contrasts. One consequence of this is that correlation parameters estimated between neighboring contrasts (e.g., 2-1 and 3-2) are difficult to interpret. Usually, they will be negative because assuming some practical limitations on the overall range (e.g., between levels 1 and 3), a small “2-1” effect “correlates” negatively with a larger “3-2” effect for mathematical reasons.

+

Obviously, the tradeoff between theoretical motivation and statistical purity is something that must be considered carefully when planning the analysis.

+

Various options for contrast coding are the topic of the MixedModelsTutorial_contrasts_emotikon.jl and MixedModelsTutorial_contrasts_kwdyz.jl notebooks.

+
+
+
+

4 Model complexification

+

We fit and compare three LMMs with the same fixed-effect structure but increasing complexity of the random-effect structure for School. We ignore the other two random factors Child and Cohort to avoid undue delays when fitting the models.

+
    +
  1. LMM m_ovi: allowing only varying intercepts (“Grand Means”);
    2. +
  2. LMM m_zcp: adding variance components (VCs) for the four Test contrasts, Sex, and age to LMM m_ovi, yielding the zero-correlation parameters LMM;
    3. +
  3. LMM m_cpx: adding correlation parameters (CPs) to LMM m_zcp; yielding a complex LMM.
    4. +
+

In a final part illustrate how to check whether the complex model is supported by the data, rather than leading to a singular fit and, if supported by the data, whether there is an increase in goodness of fit associated with the model complexification.

+
+

4.1 LMM m_ovi

+

In its random-effect structure (RES) we only vary intercepts (i.e., Grand Means) for School (LMM m_ovi), that is we allow that the schools differ in the average fitness of its children, average over the five tests.

+

It is well known that such a simple RES is likely to be anti-conservative with respect to fixed-effect test statistics.

+
+
m_ovi = let
+  f = @formula zScore ~ 1 + Test * Sex * a1 + (1 | School)
+  fit(MixedModel, f, dat; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_School
(Intercept)-0.01740.0198-0.880.37870.3417
Test: Star_r-0.00260.0303-0.090.9304
Test: S20_r0.00970.03020.320.7469
Test: SLJ0.00330.03000.110.9119
Test: BMT-0.05390.0299-1.800.0711
Sex: Girls-0.43720.0205-21.35<1e-99
a10.17610.03544.97<1e-06
Test: Star_r & Sex: Girls0.38020.06056.28<1e-09
Test: S20_r & Sex: Girls-0.20380.0604-3.380.0007
Test: SLJ & Sex: Girls-0.06690.0600-1.110.2651
Test: BMT & Sex: Girls-0.27300.0598-4.57<1e-05
Test: Star_r & a10.31460.10383.030.0024
Test: S20_r & a1-0.07670.1034-0.740.4584
Test: SLJ & a1-0.07450.1027-0.730.4683
Test: BMT & a10.47240.10254.61<1e-05
Sex: Girls & a1-0.10440.0709-1.470.1405
Test: Star_r & Sex: Girls & a1-0.19120.2076-0.920.3570
Test: S20_r & Sex: Girls & a10.17420.20680.840.3997
Test: SLJ & Sex: Girls & a10.00840.20540.040.9672
Test: BMT & Sex: Girls & a10.19230.20500.940.3482
Residual0.9152
+
+
+

Is the model singular (overparameterized, degenerate)? In other words: Is the model not supported by the data?

+
+
issingular(m_ovi)
+
+
false
+
+
+

Models varying only in intercepts are almost always supported by the data.

+
+
+

4.2 LMM m_zcp

+

In this LMM we allow that schools differ not only in GM, but also in the size of the four contrasts defined for Test, in the difference between boys and girls (Sex) and the developmental gain children achieve within the third grade (age).

+

We assume that there is covariance associated with these CPs beyond residual noise, that is we assume that there is no detectable evidence in the data that the CPs are different from zero.

+
+
m_zcp = let
+  f = @formula(
+    zScore ~
+      1 + Test * Sex * a1 + zerocorr(1 + Test + Sex + a1 | School)
+  )
+  fit(MixedModel, f, dat; contrasts)
+end
+
+
Minimizing 92    Time: 0:00:00 ( 1.09 ms/it)
+  objective:  25923.4801198341
+Minimizing 151    Time: 0:00:00 ( 5.35 ms/it)
+
+Minimizing 152    Time: 0:00:00 ( 5.43 ms/it)
+  objective:  25923.46046970646
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_School
(Intercept)-0.02470.0198-1.240.21360.3257
Test: Star_r0.00290.03110.090.92540.2208
Test: S20_r0.01020.02860.360.72090.0503
Test: SLJ0.00490.02830.170.86340.0000
Test: BMT-0.04600.0309-1.490.13680.2286
Sex: Girls-0.43990.0322-13.64<1e-410.4503
a10.18400.05733.210.00130.7780
Test: Star_r & Sex: Girls0.37840.05776.55<1e-10
Test: S20_r & Sex: Girls-0.20190.0570-3.540.0004
Test: SLJ & Sex: Girls-0.06660.0566-1.180.2392
Test: BMT & Sex: Girls-0.27800.0570-4.87<1e-05
Test: Star_r & a10.31010.09923.130.0018
Test: S20_r & a1-0.07280.0976-0.750.4558
Test: SLJ & a1-0.07840.0968-0.810.4180
Test: BMT & a10.47410.09794.84<1e-05
Sex: Girls & a1-0.14530.0791-1.840.0662
Test: Star_r & Sex: Girls & a1-0.15700.1982-0.790.4284
Test: S20_r & Sex: Girls & a10.17990.19520.920.3566
Test: SLJ & Sex: Girls & a10.00610.19360.030.9749
Test: BMT & Sex: Girls & a10.16570.19580.850.3974
Residual0.8623
+
+
+

Depending on sampling, this model estimating variance components for School may or may not be supported by the data.

+
+
issingular(m_zcp)
+
+
true
+
+
+
+
+

4.3 LMM m_cpx

+

In the complex LMM investigated in this sequence we give up the assumption of zero-correlation between VCs.

+
+
m_cpx = let
+  f = @formula(
+    zScore ~ 1 + Test * Sex * a1 + (1 + Test + Sex + a1 | School)
+  )
+  fit(MixedModel, f, dat; contrasts)
+end
+
+
Minimizing 120    Time: 0:00:00 ( 0.84 ms/it)
+  objective:  25913.7359953439
+Minimizing 239    Time: 0:00:00 ( 0.85 ms/it)
+  objective:  25874.84433102616
+Minimizing 356    Time: 0:00:00 ( 0.85 ms/it)
+  objective:  25872.397822536375
+Minimizing 474    Time: 0:00:00 ( 0.85 ms/it)
+  objective:  25870.171971430293
+Minimizing 626    Time: 0:00:00 ( 0.89 ms/it)
+  objective:  25865.73386935914
+Minimizing 769    Time: 0:00:00 ( 0.88 ms/it)
+  objective:  25864.63197654432
+Minimizing 887    Time: 0:00:00 ( 0.88 ms/it)
+  objective:  25864.549175037082
+Minimizing 1005    Time: 0:00:00 ( 0.88 ms/it)
+  objective:  25864.512781331156
+Minimizing 1124    Time: 0:00:00 ( 0.87 ms/it)
+  objective:  25864.43678434264
+Minimizing 1243    Time: 0:00:01 ( 0.87 ms/it)
+  objective:  25864.406774217532
+Minimizing 1362    Time: 0:00:01 ( 0.87 ms/it)
+  objective:  25864.40183533508
+Minimizing 1482    Time: 0:00:01 ( 0.87 ms/it)
+  objective:  25864.394286069102
+Minimizing 1600    Time: 0:00:01 ( 0.87 ms/it)
+  objective:  25864.390976008268
+Minimizing 1718    Time: 0:00:01 ( 0.86 ms/it)
+  objective:  25864.3897916446
+Minimizing 1830    Time: 0:00:01 ( 0.86 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_School
(Intercept)-0.02680.0198-1.350.17570.3264
Test: Star_r0.00550.03100.180.85780.2203
Test: S20_r0.01040.03010.350.72910.1757
Test: SLJ0.00240.02940.080.93570.1478
Test: BMT-0.04010.0305-1.320.18820.2245
Sex: Girls-0.43560.0320-13.61<1e-410.4483
a10.18770.05663.310.00090.7688
Test: Star_r & Sex: Girls0.37590.05756.53<1e-10
Test: S20_r & Sex: Girls-0.19690.0572-3.440.0006
Test: SLJ & Sex: Girls-0.06730.0567-1.190.2349
Test: BMT & Sex: Girls-0.27170.0566-4.80<1e-05
Test: Star_r & a10.31120.09883.150.0016
Test: S20_r & a1-0.07070.0981-0.720.4709
Test: SLJ & a1-0.06420.0971-0.660.5086
Test: BMT & a10.46780.09714.82<1e-05
Sex: Girls & a1-0.14040.0783-1.790.0728
Test: Star_r & Sex: Girls & a1-0.16320.1974-0.830.4086
Test: S20_r & Sex: Girls & a10.18620.19610.950.3424
Test: SLJ & Sex: Girls & a10.00110.19410.010.9954
Test: BMT & Sex: Girls & a10.16380.19420.840.3990
Residual0.8598
+
+
+

We also need to see the VCs and CPs of the random-effect structure (RES).

+
+
VarCorr(m_cpx)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
School(Intercept)0.1065410.326406
Test: Star_r0.0485370.220312+0.10
Test: S20_r0.0308800.175726-0.19-0.22
Test: SLJ0.0218530.147828-0.10+0.07-0.83
Test: BMT0.0503890.224474-0.75+0.43-0.13+0.16
Sex: Girls0.2009600.448286-0.04+0.25+0.20-0.27-0.10
a10.5910580.768803+0.02-0.13+0.03-0.49+0.17-0.06
Residual0.7392520.859798
+
+
+
+
issingular(m_cpx)
+
+
false
+
+
+

The complex model may or may not be supported by the data.

+
+
+

4.4 Model comparisons

+

The checks of model singularity indicate that the three models are supported by the data. Does model complexification also increase the goodness of fit or are we only fitting noise?

+
+

4.4.1 LRT and goodness-of-fit statistics

+

As the thee models are strictly hierarchically nested, we compare them with a likelihood-ratio tests (LRT) and AIC and BIC goodness-of-fit statistics derived from them.

+
+
MixedModels.likelihoodratiotest(m_ovi, m_zcp, m_cpx)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
zScore ~ 1 + Test + Sex + a1 + Test & Sex + Test & a1 + Sex & a1 + Test & Sex & a1 + (1 | School)2226273
zScore ~ 1 + Test + Sex + a1 + Test & Sex + Test & a1 + Sex & a1 + Test & Sex & a1 + zerocorr(1 + Test + Sex + a1 | School)28259233496<1e-71
zScore ~ 1 + Test + Sex + a1 + Test & Sex + Test & a1 + Sex & a1 + Test & Sex & a1 + (1 + Test + Sex + a1 | School)49258645921<1e-04
+
+
+
+
+Code +
gof_summary = let
+  nms = [:m_ovi, :m_zcp, :m_cpx]
+  mods = eval.(nms)
+  DataFrame(;
+    name=nms,
+    dof=dof.(mods),
+    deviance=deviance.(mods),
+    AIC=aic.(mods),
+    AICc=aicc.(mods),
+    BIC=bic.(mods),
+  )
+end
+
+
+
3×6 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RownamedofdevianceAICAICcBIC
SymbolInt64Float64Float64Float64Float64
1m_ovi2226272.926316.926317.026474.8
2m_zcp2825923.525979.525979.626180.4
3m_cpx4925864.425962.425962.926314.0
+
+
+
+

These statistics will depend on sampling. In general, smaller deviance, AIC, and BIC indicate an improvement in goodness of fit. Usually, χ² should be larger than the associated degrees of freedom; for AIC and BIC the decrease should amount to more than 5, according to some literature. Severity of meeting these criteria increases from deviance to AIC to BIC. Therefore, it is not always the case that the criteria are unanimous in their verdict. Basically, the more confirmatory the analysis, the more one may go with deviance and AIC; for exploratory analyses the BIC is probably a better guide. There are grey zones here.

+
+
+

4.4.2 Comparing fixed effects of m_ovi, m_zcp, and m_cpx

+

We check whether enriching the RES changed the significance of fixed effects in the final model.

+
+
+Code +
m_ovi_fe = DataFrame(coeftable(m_ovi));
+m_zcp_fe = DataFrame(coeftable(m_zcp));
+m_cpx_fe = DataFrame(coeftable(m_cpx));
+m_all = hcat(
+  m_ovi_fe[:, [1, 2, 4]],
+  leftjoin(
+    m_zcp_fe[:, [1, 2, 4]],
+    m_cpx_fe[:, [1, 2, 4]];
+    on=:Name,
+    makeunique=true,
+  );
+  makeunique=true,
+)
+rename!(
+  m_all,
+  "Coef." => "b_ovi",
+  "Coef._2" => "b_zcp",
+  "Coef._1" => "b_cpx",
+  "z" => "z_ovi",
+  "z_2" => "z_zcp",
+  "z_1" => "z_cpx",
+)
+m_all2 =
+  round.(
+    m_all[:, [:b_ovi, :b_zcp, :b_cpx, :z_ovi, :z_zcp, :z_cpx]],
+    digits=2,
+  )
+m_all3 = hcat(m_all.Name, m_all2)
+
+
+
20×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowx1b_ovib_zcpb_cpxz_oviz_zcpz_cpx
StringFloat64Float64Float64Float64Float64Float64
1(Intercept)-0.02-0.02-0.03-0.88-1.24-1.35
2Test: Star_r-0.00.00.01-0.090.090.18
3Test: S20_r0.010.010.010.320.360.35
4Test: SLJ0.00.00.00.110.170.08
5Test: BMT-0.05-0.05-0.04-1.8-1.49-1.32
6Sex: Girls-0.44-0.44-0.44-21.35-13.64-13.61
7a10.180.180.194.973.213.31
8Test: Star_r & Sex: Girls0.380.380.386.286.556.53
9Test: S20_r & Sex: Girls-0.2-0.2-0.2-3.38-3.54-3.44
10Test: SLJ & Sex: Girls-0.07-0.07-0.07-1.11-1.18-1.19
11Test: BMT & Sex: Girls-0.27-0.28-0.27-4.57-4.87-4.8
12Test: Star_r & a10.310.310.313.033.133.15
13Test: S20_r & a1-0.08-0.07-0.07-0.74-0.75-0.72
14Test: SLJ & a1-0.07-0.08-0.06-0.73-0.81-0.66
15Test: BMT & a10.470.470.474.614.844.82
16Sex: Girls & a1-0.1-0.15-0.14-1.47-1.84-1.79
17Test: Star_r & Sex: Girls & a1-0.19-0.16-0.16-0.92-0.79-0.83
18Test: S20_r & Sex: Girls & a10.170.180.190.840.920.95
19Test: SLJ & Sex: Girls & a10.010.010.00.040.030.01
20Test: BMT & Sex: Girls & a10.190.170.160.940.850.84
+
+
+
+

The three models usually do not differ in fixed-effect estimates. For main effects of age and Sex, z-values decrease strongly with the complexity of the model (i.e., standard errors are larger). For other coefficients, the changes are not very large and not consistent.

+

In general, dropping significant variance components and/or correlation parameters may lead to anti-conservative estimates of fixed effects (e.g., Schielzeth & Forstmeier, 2008). Basically, some of the variance allocated to age and Sex in LMM m_ovi could also be due to differences between schools. This ambiguity increased the uncertainty of the respective fixed effects in the other two LMMs.

+
+
+
+

4.5 Fitting an overparameterized LMM

+

The complex LMM was not overparameterized with respect to School, because there are over 400 schools in the data. When the number of units (levels) of a grouping factor is small relative to the number of parameters we are trying to estimate, we often end up with an overparameterized / degenerate random-effect structure.

+

As an illustration, we fit a full CP matrix for the Cohort. As there are only nine cohorts in the data, we may be asking too much to estimate 5*6/2 = 15 VC/CP parameters.

+
+
m_cpxCohort = let
+  f = @formula zScore ~ 1 + Test * a1 * Sex + (1 + Test | Cohort)
+  fit(MixedModel, f, dat; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Cohort
(Intercept)-0.00090.0161-0.060.95480.0378
Test: Star_r-0.00730.0380-0.190.84820.0614
Test: S20_r0.00560.03830.150.88450.0636
Test: SLJ0.01010.04510.220.82260.0959
Test: BMT-0.05560.0335-1.660.09680.0330
a10.20550.03495.90<1e-08
Sex: Girls-0.42380.0201-21.07<1e-97
Test: Star_r & a10.28490.11012.590.0097
Test: S20_r & a1-0.11720.1096-1.070.2851
Test: SLJ & a1-0.02700.1094-0.250.8049
Test: BMT & a10.45550.10844.20<1e-04
Test: Star_r & Sex: Girls0.37000.06405.78<1e-08
Test: S20_r & Sex: Girls-0.21160.0638-3.320.0009
Test: SLJ & Sex: Girls-0.05520.0634-0.870.3844
Test: BMT & Sex: Girls-0.27180.0632-4.30<1e-04
a1 & Sex: Girls-0.13680.0690-1.980.0473
Test: Star_r & a1 & Sex: Girls-0.20990.2194-0.960.3387
Test: S20_r & a1 & Sex: Girls0.16090.21860.740.4616
Test: SLJ & a1 & Sex: Girls0.02250.21720.100.9174
Test: BMT & a1 & Sex: Girls0.19010.21660.880.3801
Residual0.9671
+
+
+
+
VarCorr(m_cpxCohort)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Cohort(Intercept)0.00142740.0377808
Test: Star_r0.00376630.0613705-0.90
Test: S20_r0.00404380.0635905-1.00+0.87
Test: SLJ0.00918970.0958627+0.98-0.97-0.97
Test: BMT0.00108880.0329966-1.00+0.91+0.99-0.99
Residual0.93531390.9671163
+
+
+
+
issingular(m_cpxCohort)
+
+
true
+
+
+

The model is overparameterized with several CPs estimated between |.98| and |1.00|. How about the zero-correlation parameter (zcp) version of this LMM?

+
+
m_zcpCohort = let
+  f = @formula(
+    zScore ~ 1 + Test * a1 * Sex + zerocorr(1 + Test | Cohort)
+  )
+  fit(MixedModel, f, dat; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Cohort
(Intercept)-0.00220.0152-0.150.88370.0341
Test: Star_r-0.00420.0339-0.120.90230.0331
Test: S20_r0.00880.03190.270.78370.0000
Test: SLJ0.00450.03170.140.88760.0000
Test: BMT-0.05360.0316-1.690.09030.0000
a10.19990.03515.70<1e-07
Sex: Girls-0.42450.0201-21.08<1e-97
Test: Star_r & a10.30780.11012.800.0052
Test: S20_r & a1-0.08490.1093-0.780.4377
Test: SLJ & a1-0.07480.1086-0.690.4911
Test: BMT & a10.47170.10844.35<1e-04
Test: Star_r & Sex: Girls0.37290.06405.82<1e-08
Test: S20_r & Sex: Girls-0.20790.0638-3.260.0011
Test: SLJ & Sex: Girls-0.06110.0634-0.960.3354
Test: BMT & Sex: Girls-0.26970.0632-4.27<1e-04
a1 & Sex: Girls-0.13720.0691-1.990.0470
Test: Star_r & a1 & Sex: Girls-0.20410.2195-0.930.3525
Test: S20_r & a1 & Sex: Girls0.17330.21870.790.4280
Test: SLJ & a1 & Sex: Girls0.00510.21720.020.9811
Test: BMT & a1 & Sex: Girls0.19620.21680.900.3655
Residual0.9680
+
+
+
+
issingular(m_zcpCohort)
+
+
true
+
+
+

This zcpLMM is also singular. Three of the five VCs are estimated as zero. This raises the possibility that LMM m_oviCohort might fit as well as LMM m_zcpCohort.

+
+
m_oviCohort = let
+  f = @formula zScore ~ 1 + Test * a1 * Sex + (1 | Cohort)
+  fit(MixedModel, f, dat; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Cohort
(Intercept)-0.00220.0152-0.150.88350.0340
Test: Star_r-0.00320.0320-0.100.9196
Test: S20_r0.00870.03190.270.7847
Test: SLJ0.00450.03170.140.8869
Test: BMT-0.05360.0316-1.690.0903
a10.19990.03515.69<1e-07
Sex: Girls-0.42450.0201-21.08<1e-97
Test: Star_r & a10.31350.10972.860.0043
Test: S20_r & a1-0.08480.1093-0.780.4378
Test: SLJ & a1-0.07480.1086-0.690.4910
Test: BMT & a10.47180.10844.35<1e-04
Test: Star_r & Sex: Girls0.37360.06405.84<1e-08
Test: S20_r & Sex: Girls-0.20790.0638-3.260.0011
Test: SLJ & Sex: Girls-0.06110.0634-0.960.3356
Test: BMT & Sex: Girls-0.26970.0632-4.27<1e-04
a1 & Sex: Girls-0.13710.0691-1.990.0471
Test: Star_r & a1 & Sex: Girls-0.20220.2195-0.920.3568
Test: S20_r & a1 & Sex: Girls0.17330.21870.790.4281
Test: SLJ & a1 & Sex: Girls0.00510.21720.020.9811
Test: BMT & a1 & Sex: Girls0.19610.21680.900.3657
Residual0.9681
+
+
+
+
issingular(m_oviCohort)
+
+
false
+
+
+

This solves the problem with singularity, but does LMM m_zcpCohort fit noise relative to the LMM m_oviCohort?

+
+
MixedModels.likelihoodratiotest(m_oviCohort, m_zcpCohort)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 | Cohort)2226803
zScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Cohort)2626803040.9968
+
+
+
+
gof_summary2 = let
+  mods = [m_oviCohort, m_zcpCohort, m_cpxCohort]
+  DataFrame(;
+    dof=dof.(mods),
+    deviance=deviance.(mods),
+    AIC=aic.(mods),
+    AICc=aicc.(mods),
+    BIC=bic.(mods),
+  )
+end
+
+
3×5 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowdofdevianceAICAICcBIC
Int64Float64Float64Float64Float64
12226802.926846.926847.027004.7
22626802.726854.726854.827041.3
33626790.526862.526862.727120.8
+
+
+
+

Indeed, adding VCs is fitting noise. Again, the goodness of fit statistics unanimously favor the selection of the LMM m_oviCohort.

+

Not shown here, but the Cohort-related VCs for the Test contrasts could be estimated reliably for the full data. Thus, the small number of cohorts does not necessarily prevent the determination of reliable differences between tests across cohorts. What if we include VCs and CPs related to random factors Child and School?

+
+
+

4.6 Fitting the published LMM m1 to the reduced data

+
+
+
+ +
+
+Warning +
+
+
+

The following LMMs m1, m2, etc. take a bit longer (e.g., close to 6 minutes in the Pluto notebook, close to 3 minutes in the REPL on a MacBook Pro).

+
+
+

LMM m1 reported in Fühner et al. (2021) included random factors for School, Child, and Cohort. The RES for School was specified like in LMM m_cpx. The RES for Child included VCs and CPs for Test, but not for linear developmental gain in the ninth year of life a1 or Sex; they are between-Child effects.

+

The RES for Cohort included only VCs, no CPs for Test. The parsimony was due to the small number of nine levels for this grouping factor.

+

Here we fit this LMM m1 for the reduced data. For a different subset of similar size on MacBook Pro [13 | 15 | 16] this took [303 | 250 | 244 ] s; for LMM m1a (i.e., dropping 1 school-relate VC for Sex), times are [212 | 165 | 160] s. The corresponding lme4 times for LMM m1 are [397 | 348 | 195].

+

Finally, times for fitting the full set of data –not in this script–, for LMM m1are [60 | 62 | 85] minutes (!); for LMM m1a the times were [46 | 48 | 34] minutes. It was not possible to fit the full set of data with lme4; after about 13 to 18 minutes the program stopped with: Error in eval_f(x, ...) : Downdated VtV is not positive definite.

+
+
m1 = let
+  f = @formula(
+    zScore ~
+      1 +
+      Test * a1 * Sex +
+      (1 + Test + a1 + Sex | School) +
+      (1 + Test | Child) +
+      zerocorr(1 + Test | Cohort)
+  )
+  fit(MixedModel, f, dat; contrasts)
+end
+
+
Minimizing 2    Time: 0:00:01 ( 0.61  s/it)
+  objective:  26263.798630532
+Minimizing 10    Time: 0:00:01 ( 0.13  s/it)
+  objective:  26032.802373659586
+Minimizing 18    Time: 0:00:01 (79.12 ms/it)
+  objective:  25909.53939555904
+Minimizing 26    Time: 0:00:01 (58.76 ms/it)
+  objective:  25986.562881083046
+Minimizing 34    Time: 0:00:01 (48.00 ms/it)
+  objective:  25898.37560027311
+Minimizing 42    Time: 0:00:01 (41.44 ms/it)
+  objective:  25880.23790580926
+Minimizing 50    Time: 0:00:01 (36.87 ms/it)
+  objective:  27685.247320995768
+Minimizing 58    Time: 0:00:01 (33.60 ms/it)
+  objective:  26001.805014197045
+Minimizing 66    Time: 0:00:02 (31.12 ms/it)
+  objective:  25912.198496703462
+Minimizing 74    Time: 0:00:02 (29.18 ms/it)
+  objective:  25949.27949843974
+Minimizing 82    Time: 0:00:02 (27.63 ms/it)
+  objective:  25921.90653011039
+Minimizing 90    Time: 0:00:02 (26.37 ms/it)
+  objective:  25836.707585481112
+Minimizing 98    Time: 0:00:02 (25.28 ms/it)
+  objective:  25218.359672227205
+Minimizing 106    Time: 0:00:02 (24.35 ms/it)
+  objective:  25054.4576240143
+Minimizing 114    Time: 0:00:02 (23.55 ms/it)
+  objective:  25052.320501610564
+Minimizing 122    Time: 0:00:02 (22.84 ms/it)
+  objective:  24999.384497783954
+Minimizing 130    Time: 0:00:02 (22.27 ms/it)
+  objective:  24956.41343582866
+Minimizing 138    Time: 0:00:03 (21.75 ms/it)
+  objective:  24914.946584626545
+Minimizing 146    Time: 0:00:03 (21.26 ms/it)
+  objective:  24893.847196663002
+Minimizing 154    Time: 0:00:03 (20.83 ms/it)
+  objective:  24865.46703494803
+Minimizing 162    Time: 0:00:03 (20.43 ms/it)
+  objective:  24852.889649488112
+Minimizing 170    Time: 0:00:03 (20.08 ms/it)
+  objective:  24833.636378199408
+Minimizing 178    Time: 0:00:03 (19.76 ms/it)
+  objective:  24821.186037541243
+Minimizing 186    Time: 0:00:03 (19.46 ms/it)
+  objective:  24791.370242087396
+Minimizing 194    Time: 0:00:03 (19.23 ms/it)
+  objective:  24786.669733863615
+Minimizing 202    Time: 0:00:03 (19.02 ms/it)
+  objective:  24772.413565464158
+Minimizing 210    Time: 0:00:03 (18.83 ms/it)
+  objective:  24766.46535676397
+Minimizing 218    Time: 0:00:04 (18.62 ms/it)
+  objective:  24762.134758352295
+Minimizing 226    Time: 0:00:04 (18.44 ms/it)
+  objective:  24757.76410159139
+Minimizing 234    Time: 0:00:04 (18.25 ms/it)
+  objective:  24753.58928112908
+Minimizing 242    Time: 0:00:04 (18.08 ms/it)
+  objective:  24747.304757821454
+Minimizing 250    Time: 0:00:04 (17.94 ms/it)
+  objective:  24742.824950872637
+Minimizing 257    Time: 0:00:04 (17.84 ms/it)
+  objective:  24744.29596899636
+Minimizing 265    Time: 0:00:04 (17.72 ms/it)
+  objective:  24739.72217648851
+Minimizing 273    Time: 0:00:04 (17.60 ms/it)
+  objective:  24736.572876439044
+Minimizing 281    Time: 0:00:04 (17.47 ms/it)
+  objective:  24731.59897556257
+Minimizing 289    Time: 0:00:05 (17.34 ms/it)
+  objective:  24725.792527037745
+Minimizing 297    Time: 0:00:05 (17.24 ms/it)
+  objective:  24723.377321071024
+Minimizing 305    Time: 0:00:05 (17.15 ms/it)
+  objective:  24718.708289643593
+Minimizing 313    Time: 0:00:05 (17.05 ms/it)
+  objective:  24714.227854491233
+Minimizing 321    Time: 0:00:05 (16.95 ms/it)
+  objective:  24716.813508738473
+Minimizing 329    Time: 0:00:05 (16.88 ms/it)
+  objective:  24715.458991594605
+Minimizing 337    Time: 0:00:05 (16.81 ms/it)
+  objective:  24712.949077361896
+Minimizing 345    Time: 0:00:05 (16.75 ms/it)
+  objective:  24708.166068056376
+Minimizing 353    Time: 0:00:05 (16.68 ms/it)
+  objective:  24706.765806621424
+Minimizing 361    Time: 0:00:06 (16.63 ms/it)
+  objective:  24707.913284010036
+Minimizing 369    Time: 0:00:06 (16.55 ms/it)
+  objective:  24705.05082837271
+Minimizing 377    Time: 0:00:06 (16.48 ms/it)
+  objective:  24703.819443003686
+Minimizing 385    Time: 0:00:06 (16.41 ms/it)
+  objective:  24703.677592919787
+Minimizing 393    Time: 0:00:06 (16.34 ms/it)
+  objective:  24703.18196364243
+Minimizing 401    Time: 0:00:06 (16.27 ms/it)
+  objective:  24702.363946121135
+Minimizing 409    Time: 0:00:06 (16.21 ms/it)
+  objective:  24701.63190102658
+Minimizing 417    Time: 0:00:06 (16.14 ms/it)
+  objective:  24701.09678875273
+Minimizing 425    Time: 0:00:06 (16.09 ms/it)
+  objective:  24700.896820937352
+Minimizing 433    Time: 0:00:06 (16.03 ms/it)
+  objective:  24700.61501064425
+Minimizing 441    Time: 0:00:07 (15.98 ms/it)
+  objective:  24700.15273724909
+Minimizing 449    Time: 0:00:07 (15.92 ms/it)
+  objective:  24699.86376118735
+Minimizing 457    Time: 0:00:07 (15.87 ms/it)
+  objective:  24699.362939096965
+Minimizing 465    Time: 0:00:07 (15.82 ms/it)
+  objective:  24699.246571817333
+Minimizing 473    Time: 0:00:07 (15.84 ms/it)
+  objective:  24699.151605194136
+Minimizing 481    Time: 0:00:07 (15.80 ms/it)
+  objective:  24699.115862227438
+Minimizing 489    Time: 0:00:07 (15.76 ms/it)
+  objective:  24698.8833308756
+Minimizing 497    Time: 0:00:07 (15.72 ms/it)
+  objective:  24698.1518893173
+Minimizing 505    Time: 0:00:07 (15.68 ms/it)
+  objective:  24698.060517805414
+Minimizing 513    Time: 0:00:08 (15.65 ms/it)
+  objective:  24697.917889344237
+Minimizing 521    Time: 0:00:08 (15.61 ms/it)
+  objective:  24697.383587125103
+Minimizing 529    Time: 0:00:08 (15.57 ms/it)
+  objective:  24696.53884071827
+Minimizing 537    Time: 0:00:08 (15.53 ms/it)
+  objective:  24695.698401968824
+Minimizing 545    Time: 0:00:08 (15.49 ms/it)
+  objective:  24695.55574667539
+Minimizing 553    Time: 0:00:08 (15.50 ms/it)
+  objective:  24695.12738701081
+Minimizing 560    Time: 0:00:08 (15.51 ms/it)
+  objective:  24693.13130315552
+Minimizing 567    Time: 0:00:08 (15.53 ms/it)
+  objective:  24692.038142708494
+Minimizing 574    Time: 0:00:08 (15.54 ms/it)
+  objective:  24678.58646800586
+Minimizing 581    Time: 0:00:09 (15.55 ms/it)
+  objective:  24678.818908833953
+Minimizing 587    Time: 0:00:09 (15.57 ms/it)
+  objective:  24676.389341414582
+Minimizing 593    Time: 0:00:09 (15.58 ms/it)
+  objective:  24671.56373576434
+Minimizing 599    Time: 0:00:09 (15.59 ms/it)
+  objective:  24667.705378898834
+Minimizing 606    Time: 0:00:09 (15.61 ms/it)
+  objective:  24667.11381795389
+Minimizing 614    Time: 0:00:09 (15.57 ms/it)
+  objective:  24666.42084029146
+Minimizing 622    Time: 0:00:09 (15.54 ms/it)
+  objective:  24661.542615706596
+Minimizing 630    Time: 0:00:09 (15.52 ms/it)
+  objective:  24660.649476650564
+Minimizing 638    Time: 0:00:09 (15.49 ms/it)
+  objective:  24659.11680999573
+Minimizing 646    Time: 0:00:09 (15.46 ms/it)
+  objective:  24658.428278522435
+Minimizing 654    Time: 0:00:10 (15.43 ms/it)
+  objective:  24657.25952023676
+Minimizing 662    Time: 0:00:10 (15.40 ms/it)
+  objective:  24656.446859476826
+Minimizing 670    Time: 0:00:10 (15.37 ms/it)
+  objective:  24655.168140020396
+Minimizing 678    Time: 0:00:10 (15.34 ms/it)
+  objective:  24654.28041030011
+Minimizing 686    Time: 0:00:10 (15.31 ms/it)
+  objective:  24654.06780229167
+Minimizing 694    Time: 0:00:10 (15.28 ms/it)
+  objective:  24653.68974304936
+Minimizing 702    Time: 0:00:10 (15.25 ms/it)
+  objective:  24653.619827101793
+Minimizing 710    Time: 0:00:10 (15.22 ms/it)
+  objective:  24653.302664698494
+Minimizing 718    Time: 0:00:10 (15.19 ms/it)
+  objective:  24652.929449183524
+Minimizing 726    Time: 0:00:11 (15.17 ms/it)
+  objective:  24652.857108753622
+Minimizing 734    Time: 0:00:11 (15.15 ms/it)
+  objective:  24652.625565747767
+Minimizing 742    Time: 0:00:11 (15.13 ms/it)
+  objective:  24652.411127345717
+Minimizing 750    Time: 0:00:11 (15.10 ms/it)
+  objective:  24652.463586626553
+Minimizing 758    Time: 0:00:11 (15.08 ms/it)
+  objective:  24652.239417018252
+Minimizing 766    Time: 0:00:11 (15.05 ms/it)
+  objective:  24652.160141388435
+Minimizing 774    Time: 0:00:11 (15.03 ms/it)
+  objective:  24652.03116855609
+Minimizing 782    Time: 0:00:11 (15.02 ms/it)
+  objective:  24651.847046943094
+Minimizing 790    Time: 0:00:11 (14.99 ms/it)
+  objective:  24651.784127586812
+Minimizing 798    Time: 0:00:11 (14.97 ms/it)
+  objective:  24651.69274006296
+Minimizing 806    Time: 0:00:12 (14.95 ms/it)
+  objective:  24651.687996332294
+Minimizing 814    Time: 0:00:12 (14.94 ms/it)
+  objective:  24651.635508109248
+Minimizing 822    Time: 0:00:12 (14.92 ms/it)
+  objective:  24651.56875905724
+Minimizing 830    Time: 0:00:12 (14.90 ms/it)
+  objective:  24651.496126553197
+Minimizing 838    Time: 0:00:12 (14.88 ms/it)
+  objective:  24651.40481671076
+Minimizing 846    Time: 0:00:12 (14.86 ms/it)
+  objective:  24651.407354575
+Minimizing 854    Time: 0:00:12 (14.84 ms/it)
+  objective:  24651.323434626982
+Minimizing 862    Time: 0:00:12 (14.83 ms/it)
+  objective:  24651.29504631603
+Minimizing 870    Time: 0:00:12 (14.81 ms/it)
+  objective:  24651.311300901056
+Minimizing 878    Time: 0:00:12 (14.79 ms/it)
+  objective:  24651.298276056354
+Minimizing 886    Time: 0:00:13 (14.77 ms/it)
+  objective:  24651.31025509899
+Minimizing 894    Time: 0:00:13 (14.76 ms/it)
+  objective:  24651.274571547197
+Minimizing 902    Time: 0:00:13 (14.75 ms/it)
+  objective:  24651.26716024734
+Minimizing 910    Time: 0:00:13 (14.74 ms/it)
+  objective:  24651.248490126316
+Minimizing 918    Time: 0:00:13 (14.73 ms/it)
+  objective:  24651.22649740443
+Minimizing 926    Time: 0:00:13 (14.71 ms/it)
+  objective:  24651.210418520248
+Minimizing 934    Time: 0:00:13 (14.70 ms/it)
+  objective:  24651.19685424194
+Minimizing 942    Time: 0:00:13 (14.69 ms/it)
+  objective:  24651.18964335481
+Minimizing 950    Time: 0:00:13 (14.68 ms/it)
+  objective:  24651.17759926623
+Minimizing 958    Time: 0:00:14 (14.66 ms/it)
+  objective:  24651.149153464376
+Minimizing 966    Time: 0:00:14 (14.65 ms/it)
+  objective:  24651.129084109743
+Minimizing 974    Time: 0:00:14 (14.64 ms/it)
+  objective:  24651.11081631579
+Minimizing 982    Time: 0:00:14 (14.63 ms/it)
+  objective:  24651.10298571695
+Minimizing 990    Time: 0:00:14 (14.61 ms/it)
+  objective:  24651.098979115603
+Minimizing 998    Time: 0:00:14 (14.60 ms/it)
+  objective:  24651.093761686083
+Minimizing 1006    Time: 0:00:14 (14.59 ms/it)
+  objective:  24651.09213271858
+Minimizing 1014    Time: 0:00:14 (14.57 ms/it)
+  objective:  24651.08392533141
+Minimizing 1022    Time: 0:00:14 (14.56 ms/it)
+  objective:  24651.067859962568
+Minimizing 1030    Time: 0:00:14 (14.55 ms/it)
+  objective:  24651.06520194812
+Minimizing 1038    Time: 0:00:15 (14.54 ms/it)
+  objective:  24651.060810670366
+Minimizing 1046    Time: 0:00:15 (14.53 ms/it)
+  objective:  24651.05670350074
+Minimizing 1054    Time: 0:00:15 (14.51 ms/it)
+  objective:  24651.054605778412
+Minimizing 1062    Time: 0:00:15 (14.50 ms/it)
+  objective:  24651.049284248373
+Minimizing 1070    Time: 0:00:15 (14.49 ms/it)
+  objective:  24651.0464201929
+Minimizing 1078    Time: 0:00:15 (14.48 ms/it)
+  objective:  24651.045596860844
+Minimizing 1086    Time: 0:00:15 (14.47 ms/it)
+  objective:  24651.040420920246
+Minimizing 1094    Time: 0:00:15 (14.46 ms/it)
+  objective:  24651.039182749253
+Minimizing 1102    Time: 0:00:15 (14.45 ms/it)
+  objective:  24651.037359140148
+Minimizing 1110    Time: 0:00:16 (14.44 ms/it)
+  objective:  24651.03330462238
+Minimizing 1118    Time: 0:00:16 (14.43 ms/it)
+  objective:  24651.03151596733
+Minimizing 1126    Time: 0:00:16 (14.42 ms/it)
+  objective:  24651.030760071368
+Minimizing 1134    Time: 0:00:16 (14.41 ms/it)
+  objective:  24651.027194386923
+Minimizing 1142    Time: 0:00:16 (14.40 ms/it)
+  objective:  24651.025631575212
+Minimizing 1150    Time: 0:00:16 (14.38 ms/it)
+  objective:  24651.024596174822
+Minimizing 1158    Time: 0:00:16 (14.37 ms/it)
+  objective:  24651.02241015472
+Minimizing 1166    Time: 0:00:16 (14.36 ms/it)
+  objective:  24651.021905047914
+Minimizing 1174    Time: 0:00:16 (14.35 ms/it)
+  objective:  24651.020751495194
+Minimizing 1182    Time: 0:00:16 (14.34 ms/it)
+  objective:  24651.01996893064
+Minimizing 1190    Time: 0:00:17 (14.33 ms/it)
+  objective:  24651.019319883733
+Minimizing 1198    Time: 0:00:17 (14.33 ms/it)
+  objective:  24651.019552708614
+Minimizing 1206    Time: 0:00:17 (14.33 ms/it)
+  objective:  24651.018277036655
+Minimizing 1214    Time: 0:00:17 (14.33 ms/it)
+  objective:  24651.018391170022
+Minimizing 1222    Time: 0:00:17 (14.32 ms/it)
+  objective:  24651.01799501553
+Minimizing 1230    Time: 0:00:17 (14.31 ms/it)
+  objective:  24651.017306673806
+Minimizing 1238    Time: 0:00:17 (14.31 ms/it)
+  objective:  24651.01716018239
+Minimizing 1246    Time: 0:00:17 (14.30 ms/it)
+  objective:  24651.01700720565
+Minimizing 1254    Time: 0:00:17 (14.29 ms/it)
+  objective:  24651.017156225666
+Minimizing 1262    Time: 0:00:18 (14.28 ms/it)
+  objective:  24651.017078825065
+Minimizing 1270    Time: 0:00:18 (14.28 ms/it)
+  objective:  24651.017081048733
+Minimizing 1278    Time: 0:00:18 (14.27 ms/it)
+  objective:  24651.017027630012
+Minimizing 1286    Time: 0:00:18 (14.27 ms/it)
+  objective:  24651.0165249109
+Minimizing 1294    Time: 0:00:18 (14.26 ms/it)
+  objective:  24651.016514311843
+Minimizing 1302    Time: 0:00:18 (14.26 ms/it)
+  objective:  24651.01643433857
+Minimizing 1310    Time: 0:00:18 (14.25 ms/it)
+  objective:  24651.016395205435
+Minimizing 1318    Time: 0:00:18 (14.25 ms/it)
+  objective:  24651.016339884165
+Minimizing 1326    Time: 0:00:18 (14.24 ms/it)
+  objective:  24651.016310890016
+Minimizing 1334    Time: 0:00:18 (14.24 ms/it)
+  objective:  24651.016295792695
+Minimizing 1342    Time: 0:00:19 (14.24 ms/it)
+  objective:  24651.016276136255
+Minimizing 1350    Time: 0:00:19 (14.23 ms/it)
+  objective:  24651.016203243234
+Minimizing 1358    Time: 0:00:19 (14.23 ms/it)
+  objective:  24651.0161912915
+Minimizing 1366    Time: 0:00:19 (14.22 ms/it)
+  objective:  24651.01610875644
+Minimizing 1374    Time: 0:00:19 (14.21 ms/it)
+  objective:  24651.015989577405
+Minimizing 1382    Time: 0:00:19 (14.21 ms/it)
+  objective:  24651.01595999897
+Minimizing 1390    Time: 0:00:19 (14.20 ms/it)
+  objective:  24651.015947854627
+Minimizing 1398    Time: 0:00:19 (14.20 ms/it)
+  objective:  24651.01595044735
+Minimizing 1406    Time: 0:00:19 (14.19 ms/it)
+  objective:  24651.015917806668
+Minimizing 1414    Time: 0:00:20 (14.19 ms/it)
+  objective:  24651.015892329753
+Minimizing 1422    Time: 0:00:20 (14.18 ms/it)
+  objective:  24651.015874911966
+Minimizing 1430    Time: 0:00:20 (14.18 ms/it)
+  objective:  24651.015796210548
+Minimizing 1438    Time: 0:00:20 (14.18 ms/it)
+  objective:  24651.01577879441
+Minimizing 1446    Time: 0:00:20 (14.17 ms/it)
+  objective:  24651.015705768066
+Minimizing 1454    Time: 0:00:20 (14.17 ms/it)
+  objective:  24651.01572673725
+Minimizing 1462    Time: 0:00:20 (14.16 ms/it)
+  objective:  24651.01561362497
+Minimizing 1470    Time: 0:00:20 (14.16 ms/it)
+  objective:  24651.01560258209
+Minimizing 1478    Time: 0:00:20 (14.15 ms/it)
+  objective:  24651.01557792974
+Minimizing 1486    Time: 0:00:21 (14.15 ms/it)
+  objective:  24651.015571319556
+Minimizing 1494    Time: 0:00:21 (14.14 ms/it)
+  objective:  24651.01549903939
+Minimizing 1502    Time: 0:00:21 (14.14 ms/it)
+  objective:  24651.01547620207
+Minimizing 1510    Time: 0:00:21 (14.13 ms/it)
+  objective:  24651.01541885895
+Minimizing 1518    Time: 0:00:21 (14.13 ms/it)
+  objective:  24651.015409991058
+Minimizing 1539    Time: 0:00:21 (14.16 ms/it)
+  objective:  24651.01534260885
+Minimizing 1549    Time: 0:00:21 (14.15 ms/it)
+  objective:  24651.015331627394
+Minimizing 1557    Time: 0:00:22 (14.15 ms/it)
+  objective:  24651.015299824558
+Minimizing 1565    Time: 0:00:22 (14.14 ms/it)
+  objective:  24651.015234156173
+Minimizing 1573    Time: 0:00:22 (14.14 ms/it)
+  objective:  24651.015191282593
+Minimizing 1581    Time: 0:00:22 (14.13 ms/it)
+  objective:  24651.015172503638
+Minimizing 1589    Time: 0:00:22 (14.13 ms/it)
+  objective:  24651.01514874262
+Minimizing 1597    Time: 0:00:22 (14.13 ms/it)
+  objective:  24651.01511623719
+Minimizing 1605    Time: 0:00:22 (14.12 ms/it)
+  objective:  24651.015053709496
+Minimizing 1613    Time: 0:00:22 (14.12 ms/it)
+  objective:  24651.01502497716
+Minimizing 1621    Time: 0:00:22 (14.12 ms/it)
+  objective:  24651.01497446181
+Minimizing 1629    Time: 0:00:22 (14.11 ms/it)
+  objective:  24651.014941877227
+Minimizing 1637    Time: 0:00:23 (14.11 ms/it)
+  objective:  24651.01494574121
+Minimizing 1645    Time: 0:00:23 (14.10 ms/it)
+  objective:  24651.01487685147
+Minimizing 1653    Time: 0:00:23 (14.10 ms/it)
+  objective:  24651.014834119243
+Minimizing 1661    Time: 0:00:23 (14.10 ms/it)
+  objective:  24651.014801303565
+Minimizing 1669    Time: 0:00:23 (14.09 ms/it)
+  objective:  24651.014780782607
+Minimizing 1677    Time: 0:00:23 (14.09 ms/it)
+  objective:  24651.014723134558
+Minimizing 1684    Time: 0:00:23 (14.09 ms/it)
+  objective:  24651.01470064149
+Minimizing 1692    Time: 0:00:23 (14.09 ms/it)
+  objective:  24651.014673650465
+Minimizing 1700    Time: 0:00:23 (14.08 ms/it)
+  objective:  24651.014635317995
+Minimizing 1708    Time: 0:00:24 (14.08 ms/it)
+  objective:  24651.01459270252
+Minimizing 1716    Time: 0:00:24 (14.08 ms/it)
+  objective:  24651.01455564439
+Minimizing 1724    Time: 0:00:24 (14.08 ms/it)
+  objective:  24651.014519669363
+Minimizing 1732    Time: 0:00:24 (14.07 ms/it)
+  objective:  24651.014505054405
+Minimizing 1740    Time: 0:00:24 (14.07 ms/it)
+  objective:  24651.01449206109
+Minimizing 1748    Time: 0:00:24 (14.06 ms/it)
+  objective:  24651.01445794005
+Minimizing 1756    Time: 0:00:24 (14.06 ms/it)
+  objective:  24651.0144058898
+Minimizing 1764    Time: 0:00:24 (14.05 ms/it)
+  objective:  24651.014378787884
+Minimizing 1772    Time: 0:00:24 (14.05 ms/it)
+  objective:  24651.01434324346
+Minimizing 1780    Time: 0:00:24 (14.04 ms/it)
+  objective:  24651.014334378262
+Minimizing 1788    Time: 0:00:25 (14.04 ms/it)
+  objective:  24651.014273782956
+Minimizing 1796    Time: 0:00:25 (14.03 ms/it)
+  objective:  24651.014245677798
+Minimizing 1804    Time: 0:00:25 (14.02 ms/it)
+  objective:  24651.01423229063
+Minimizing 1812    Time: 0:00:25 (14.02 ms/it)
+  objective:  24651.014222059363
+Minimizing 1820    Time: 0:00:25 (14.01 ms/it)
+  objective:  24651.01418156778
+Minimizing 1828    Time: 0:00:25 (14.01 ms/it)
+  objective:  24651.014162000316
+Minimizing 1836    Time: 0:00:25 (14.01 ms/it)
+  objective:  24651.014132598506
+Minimizing 1844    Time: 0:00:25 (14.01 ms/it)
+  objective:  24651.0141144742
+Minimizing 1852    Time: 0:00:25 (14.01 ms/it)
+  objective:  24651.01409509868
+Minimizing 1860    Time: 0:00:26 (14.01 ms/it)
+  objective:  24651.0140637523
+Minimizing 1868    Time: 0:00:26 (14.00 ms/it)
+  objective:  24651.01405627704
+Minimizing 1876    Time: 0:00:26 (14.00 ms/it)
+  objective:  24651.014037037705
+Minimizing 1877    Time: 0:00:26 (14.00 ms/it)
+
+Minimizing 1878    Time: 0:00:26 (14.00 ms/it)
+  objective:  24651.014036789165
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Childσ_Schoolσ_Cohort
(Intercept)-0.01330.0192-0.690.48770.58730.21380.0132
Test: Star_r0.00880.03750.240.81350.74340.33280.0638
Test: S20_r0.00920.02930.310.75460.63410.32800.0076
Test: SLJ0.00340.03000.110.91020.54210.32120.0339
Test: BMT-0.03830.0296-1.290.19540.72130.31300.0000
a10.19560.05453.590.00030.2825
Sex: Girls-0.42870.0312-13.75<1e-420.1435
Test: Star_r & a10.28980.08873.270.0011
Test: S20_r & a1-0.07840.0817-0.960.3370
Test: SLJ & a1-0.06310.0769-0.820.4118
Test: BMT & a10.47130.08525.54<1e-07
Test: Star_r & Sex: Girls0.37560.05117.35<1e-12
Test: S20_r & Sex: Girls-0.18670.0475-3.93<1e-04
Test: SLJ & Sex: Girls-0.06950.0445-1.560.1184
Test: BMT & Sex: Girls-0.28120.0495-5.68<1e-07
a1 & Sex: Girls-0.12670.1048-1.210.2269
Test: Star_r & a1 & Sex: Girls-0.13820.1756-0.790.4312
Test: S20_r & a1 & Sex: Girls0.17530.16331.070.2832
Test: SLJ & a1 & Sex: Girls-0.01540.1530-0.100.9196
Test: BMT & a1 & Sex: Girls0.15480.17020.910.3630
Residual0.5331
+
+
+
+
VarCorr(m1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Child(Intercept)0.344952710.58732675
Test: Star_r0.552697630.74343636+0.14
Test: S20_r0.402125110.63413336+0.00-0.52
Test: SLJ0.293855970.54208484+0.05-0.04-0.35
Test: BMT0.520321980.72133347-0.35+0.14-0.18-0.22
School(Intercept)0.045707020.21379201
Test: Star_r0.110768690.33281931-0.06
Test: S20_r0.107610180.32803990-0.08-0.39
Test: SLJ0.103188430.32122956-0.19+0.21-0.81
Test: BMT0.097951810.31297254-0.33-0.02+0.13-0.38
a10.079825650.28253433+0.62-0.20+0.11-0.61+0.45
Sex: Girls0.020591800.14349843-0.45+0.45+0.31-0.27-0.15-0.37
Cohort(Intercept)0.000174410.01320652
Test: Star_r0.004065490.06376120.
Test: S20_r0.000057020.00755108..
Test: SLJ0.001149590.03390556...
Test: BMT0.000000000.00000000....
Residual0.284225220.53312777
+
+
+
+
issingular(m1)
+
+
true
+
+
+

Depending on the random number for stratified samplign, LMM m1 may or may not be supported by the data.

+

We also fit an alternative parameterization, estimating VCs and CPs for Test scores rather than Test effects by replacing the 1 + ... in the RE terms with 0 + ....

+
+
m2 = let
+  f = @formula(
+    zScore ~
+      1 +
+      Test * a1 * Sex +
+      (0 + Test + a1 + Sex | School) +
+      (0 + Test | Child) +
+      zerocorr(0 + Test | Cohort)
+  )
+  fit(MixedModel, f, dat; contrasts)
+end
+
+
Minimizing 8    Time: 0:00:00 (13.13 ms/it)
+  objective:  26540.628428689935
+Minimizing 16    Time: 0:00:00 (12.98 ms/it)
+  objective:  27013.20266803618
+Minimizing 24    Time: 0:00:00 (13.30 ms/it)
+  objective:  26938.094323301608
+Minimizing 32    Time: 0:00:00 (13.37 ms/it)
+  objective:  26816.3549662712
+Minimizing 40    Time: 0:00:00 (13.36 ms/it)
+  objective:  26764.50301863997
+Minimizing 48    Time: 0:00:00 (13.37 ms/it)
+  objective:  26748.52532251481
+Minimizing 56    Time: 0:00:00 (13.35 ms/it)
+  objective:  27607.74073471175
+Minimizing 64    Time: 0:00:00 (13.34 ms/it)
+  objective:  27035.518325217672
+Minimizing 72    Time: 0:00:00 (13.26 ms/it)
+  objective:  26743.04860536076
+Minimizing 80    Time: 0:00:01 (13.20 ms/it)
+  objective:  26914.37946666574
+Minimizing 88    Time: 0:00:01 (13.15 ms/it)
+  objective:  26750.53281469535
+Minimizing 96    Time: 0:00:01 (13.13 ms/it)
+  objective:  26718.60071956578
+Minimizing 104    Time: 0:00:01 (13.14 ms/it)
+  objective:  25121.980475037257
+Minimizing 112    Time: 0:00:01 (13.13 ms/it)
+  objective:  25011.13301114371
+Minimizing 120    Time: 0:00:01 (13.13 ms/it)
+  objective:  24997.24925764957
+Minimizing 128    Time: 0:00:01 (13.12 ms/it)
+  objective:  24953.65373230762
+Minimizing 136    Time: 0:00:01 (13.11 ms/it)
+  objective:  24958.120022161565
+Minimizing 144    Time: 0:00:01 (13.10 ms/it)
+  objective:  24900.668392425818
+Minimizing 152    Time: 0:00:01 (13.10 ms/it)
+  objective:  24893.96205392398
+Minimizing 160    Time: 0:00:02 (13.10 ms/it)
+  objective:  24873.636974246172
+Minimizing 168    Time: 0:00:02 (13.23 ms/it)
+  objective:  24859.799925935025
+Minimizing 174    Time: 0:00:02 (13.37 ms/it)
+  objective:  24831.117936158902
+Minimizing 181    Time: 0:00:02 (13.49 ms/it)
+  objective:  24808.0196415159
+Minimizing 188    Time: 0:00:02 (13.61 ms/it)
+  objective:  24810.622141936405
+Minimizing 195    Time: 0:00:02 (13.71 ms/it)
+  objective:  24764.74836706018
+Minimizing 201    Time: 0:00:02 (13.81 ms/it)
+  objective:  24750.864655709753
+Minimizing 207    Time: 0:00:02 (13.90 ms/it)
+  objective:  24728.7014244585
+Minimizing 213    Time: 0:00:02 (13.98 ms/it)
+  objective:  24721.802789535843
+Minimizing 220    Time: 0:00:03 (14.05 ms/it)
+  objective:  24724.978892060455
+Minimizing 228    Time: 0:00:03 (14.03 ms/it)
+  objective:  24714.559421284797
+Minimizing 236    Time: 0:00:03 (13.99 ms/it)
+  objective:  24713.749570295942
+Minimizing 244    Time: 0:00:03 (13.95 ms/it)
+  objective:  24703.141786459488
+Minimizing 252    Time: 0:00:03 (13.93 ms/it)
+  objective:  24706.525178828168
+Minimizing 260    Time: 0:00:03 (13.94 ms/it)
+  objective:  24696.13455573986
+Minimizing 268    Time: 0:00:03 (13.92 ms/it)
+  objective:  24691.657571354426
+Minimizing 277    Time: 0:00:03 (13.88 ms/it)
+  objective:  24685.654228629654
+Minimizing 286    Time: 0:00:03 (13.83 ms/it)
+  objective:  24682.51124245504
+Minimizing 295    Time: 0:00:04 (13.79 ms/it)
+  objective:  24681.05913327908
+Minimizing 304    Time: 0:00:04 (13.75 ms/it)
+  objective:  24676.468113148057
+Minimizing 312    Time: 0:00:04 (13.72 ms/it)
+  objective:  24678.73775724656
+Minimizing 321    Time: 0:00:04 (13.68 ms/it)
+  objective:  24673.49816406841
+Minimizing 330    Time: 0:00:04 (13.65 ms/it)
+  objective:  24671.722762394973
+Minimizing 338    Time: 0:00:04 (13.66 ms/it)
+  objective:  24666.188917121326
+Minimizing 345    Time: 0:00:04 (13.68 ms/it)
+  objective:  24664.77883152221
+Minimizing 353    Time: 0:00:04 (13.66 ms/it)
+  objective:  24659.4656714057
+Minimizing 361    Time: 0:00:04 (13.64 ms/it)
+  objective:  24661.552652523183
+Minimizing 369    Time: 0:00:05 (13.62 ms/it)
+  objective:  24657.54743509639
+Minimizing 377    Time: 0:00:05 (13.60 ms/it)
+  objective:  24657.906277135386
+Minimizing 385    Time: 0:00:05 (13.58 ms/it)
+  objective:  24657.22098196359
+Minimizing 393    Time: 0:00:05 (13.58 ms/it)
+  objective:  24654.084092197903
+Minimizing 401    Time: 0:00:05 (13.56 ms/it)
+  objective:  24654.660376842156
+Minimizing 410    Time: 0:00:05 (13.53 ms/it)
+  objective:  24653.05933991388
+Minimizing 419    Time: 0:00:05 (13.51 ms/it)
+  objective:  24652.690785417355
+Minimizing 428    Time: 0:00:05 (13.48 ms/it)
+  objective:  24651.576869207296
+Minimizing 437    Time: 0:00:05 (13.45 ms/it)
+  objective:  24651.345249844635
+Minimizing 446    Time: 0:00:05 (13.42 ms/it)
+  objective:  24650.935321003628
+Minimizing 455    Time: 0:00:06 (13.39 ms/it)
+  objective:  24650.460659691147
+Minimizing 464    Time: 0:00:06 (13.37 ms/it)
+  objective:  24650.033417605235
+Minimizing 473    Time: 0:00:06 (13.35 ms/it)
+  objective:  24649.881648448623
+Minimizing 482    Time: 0:00:06 (13.33 ms/it)
+  objective:  24649.617387416143
+Minimizing 491    Time: 0:00:06 (13.31 ms/it)
+  objective:  24649.54221767746
+Minimizing 500    Time: 0:00:06 (13.29 ms/it)
+  objective:  24649.053690507884
+Minimizing 509    Time: 0:00:06 (13.27 ms/it)
+  objective:  24648.905068218475
+Minimizing 518    Time: 0:00:06 (13.25 ms/it)
+  objective:  24648.68854571072
+Minimizing 527    Time: 0:00:06 (13.23 ms/it)
+  objective:  24648.58020652706
+Minimizing 536    Time: 0:00:07 (13.21 ms/it)
+  objective:  24648.4299572324
+Minimizing 544    Time: 0:00:07 (13.20 ms/it)
+  objective:  24648.359137537387
+Minimizing 553    Time: 0:00:07 (13.19 ms/it)
+  objective:  24648.121508962126
+Minimizing 562    Time: 0:00:07 (13.17 ms/it)
+  objective:  24648.099993862208
+Minimizing 570    Time: 0:00:07 (13.17 ms/it)
+  objective:  24648.006992328505
+Minimizing 578    Time: 0:00:07 (13.17 ms/it)
+  objective:  24647.966554206134
+Minimizing 586    Time: 0:00:07 (13.16 ms/it)
+  objective:  24647.839547408417
+Minimizing 594    Time: 0:00:07 (13.16 ms/it)
+  objective:  24647.748757753267
+Minimizing 602    Time: 0:00:07 (13.16 ms/it)
+  objective:  24647.703056529623
+Minimizing 610    Time: 0:00:08 (13.16 ms/it)
+  objective:  24647.65634691922
+Minimizing 618    Time: 0:00:08 (13.16 ms/it)
+  objective:  24647.56439257673
+Minimizing 626    Time: 0:00:08 (13.17 ms/it)
+  objective:  24647.50067749583
+Minimizing 634    Time: 0:00:08 (13.18 ms/it)
+  objective:  24647.459579058945
+Minimizing 642    Time: 0:00:08 (13.18 ms/it)
+  objective:  24647.47873341732
+Minimizing 651    Time: 0:00:08 (13.16 ms/it)
+  objective:  24647.343086450237
+Minimizing 660    Time: 0:00:08 (13.15 ms/it)
+  objective:  24647.34116898825
+Minimizing 669    Time: 0:00:08 (13.14 ms/it)
+  objective:  24647.348823056633
+Minimizing 678    Time: 0:00:08 (13.14 ms/it)
+  objective:  24647.27411314765
+Minimizing 686    Time: 0:00:09 (13.13 ms/it)
+  objective:  24647.250608569215
+Minimizing 695    Time: 0:00:09 (13.12 ms/it)
+  objective:  24647.22520161305
+Minimizing 704    Time: 0:00:09 (13.11 ms/it)
+  objective:  24647.23198462193
+Minimizing 713    Time: 0:00:09 (13.10 ms/it)
+  objective:  24647.200084946257
+Minimizing 722    Time: 0:00:09 (13.08 ms/it)
+  objective:  24647.209350541543
+Minimizing 731    Time: 0:00:09 (13.07 ms/it)
+  objective:  24647.150885420226
+Minimizing 740    Time: 0:00:09 (13.06 ms/it)
+  objective:  24647.129446947456
+Minimizing 749    Time: 0:00:09 (13.06 ms/it)
+  objective:  24647.125530564546
+Minimizing 758    Time: 0:00:09 (13.05 ms/it)
+  objective:  24647.13683106716
+Minimizing 766    Time: 0:00:09 (13.05 ms/it)
+  objective:  24647.091680123256
+Minimizing 775    Time: 0:00:10 (13.04 ms/it)
+  objective:  24647.098428585
+Minimizing 783    Time: 0:00:10 (13.04 ms/it)
+  objective:  24647.094389500824
+Minimizing 791    Time: 0:00:10 (13.04 ms/it)
+  objective:  24647.05192467993
+Minimizing 799    Time: 0:00:10 (13.04 ms/it)
+  objective:  24647.069433509034
+Minimizing 807    Time: 0:00:10 (13.04 ms/it)
+  objective:  24647.038241950948
+Minimizing 815    Time: 0:00:10 (13.05 ms/it)
+  objective:  24647.03569193548
+Minimizing 823    Time: 0:00:10 (13.05 ms/it)
+  objective:  24647.02367144438
+Minimizing 831    Time: 0:00:10 (13.05 ms/it)
+  objective:  24647.014044510084
+Minimizing 839    Time: 0:00:10 (13.05 ms/it)
+  objective:  24647.00941961279
+Minimizing 847    Time: 0:00:11 (13.05 ms/it)
+  objective:  24646.98532430834
+Minimizing 856    Time: 0:00:11 (13.04 ms/it)
+  objective:  24646.965197407808
+Minimizing 864    Time: 0:00:11 (13.04 ms/it)
+  objective:  24646.97000154094
+Minimizing 872    Time: 0:00:11 (13.05 ms/it)
+  objective:  24646.98178371837
+Minimizing 880    Time: 0:00:11 (13.05 ms/it)
+  objective:  24646.983776073634
+Minimizing 889    Time: 0:00:11 (13.04 ms/it)
+  objective:  24646.9319241572
+Minimizing 897    Time: 0:00:11 (13.04 ms/it)
+  objective:  24646.931978735825
+Minimizing 906    Time: 0:00:11 (13.03 ms/it)
+  objective:  24646.928020882224
+Minimizing 915    Time: 0:00:11 (13.03 ms/it)
+  objective:  24646.946012804347
+Minimizing 923    Time: 0:00:12 (13.03 ms/it)
+  objective:  24646.92061262158
+Minimizing 931    Time: 0:00:12 (13.03 ms/it)
+  objective:  24646.919364692974
+Minimizing 940    Time: 0:00:12 (13.04 ms/it)
+  objective:  24646.952962425272
+Minimizing 948    Time: 0:00:12 (13.04 ms/it)
+  objective:  24646.910649209145
+Minimizing 956    Time: 0:00:12 (13.04 ms/it)
+  objective:  24646.908075010928
+Minimizing 964    Time: 0:00:12 (13.04 ms/it)
+  objective:  24646.9254264908
+Minimizing 973    Time: 0:00:12 (13.03 ms/it)
+  objective:  24646.93194511519
+Minimizing 982    Time: 0:00:12 (13.02 ms/it)
+  objective:  24646.895063711723
+Minimizing 991    Time: 0:00:12 (13.01 ms/it)
+  objective:  24646.922305142616
+Minimizing 1000    Time: 0:00:13 (13.00 ms/it)
+  objective:  24646.891929413974
+Minimizing 1009    Time: 0:00:13 (13.00 ms/it)
+  objective:  24646.891099685534
+Minimizing 1018    Time: 0:00:13 (12.99 ms/it)
+  objective:  24646.889709339343
+Minimizing 1027    Time: 0:00:13 (12.98 ms/it)
+  objective:  24646.88810835629
+Minimizing 1036    Time: 0:00:13 (12.97 ms/it)
+  objective:  24646.887297493624
+Minimizing 1045    Time: 0:00:13 (12.96 ms/it)
+  objective:  24646.885784030797
+Minimizing 1054    Time: 0:00:13 (12.96 ms/it)
+  objective:  24646.885450896836
+Minimizing 1063    Time: 0:00:13 (12.95 ms/it)
+  objective:  24646.88351475452
+Minimizing 1072    Time: 0:00:13 (12.94 ms/it)
+  objective:  24646.88199866625
+Minimizing 1081    Time: 0:00:13 (12.93 ms/it)
+  objective:  24646.88095125399
+Minimizing 1090    Time: 0:00:14 (12.92 ms/it)
+  objective:  24646.878879374555
+Minimizing 1099    Time: 0:00:14 (12.92 ms/it)
+  objective:  24646.877898696042
+Minimizing 1108    Time: 0:00:14 (12.91 ms/it)
+  objective:  24646.87745016371
+Minimizing 1117    Time: 0:00:14 (12.90 ms/it)
+  objective:  24646.877039090396
+Minimizing 1126    Time: 0:00:14 (12.90 ms/it)
+  objective:  24646.87678163265
+Minimizing 1134    Time: 0:00:14 (12.90 ms/it)
+  objective:  24646.87620262606
+Minimizing 1142    Time: 0:00:14 (12.90 ms/it)
+  objective:  24646.875795879896
+Minimizing 1150    Time: 0:00:14 (12.90 ms/it)
+  objective:  24646.87474805087
+Minimizing 1158    Time: 0:00:14 (12.91 ms/it)
+  objective:  24646.874187797846
+Minimizing 1166    Time: 0:00:15 (12.91 ms/it)
+  objective:  24646.874226968095
+Minimizing 1174    Time: 0:00:15 (12.91 ms/it)
+  objective:  24646.873636962468
+Minimizing 1182    Time: 0:00:15 (12.91 ms/it)
+  objective:  24646.873134800753
+Minimizing 1190    Time: 0:00:15 (12.91 ms/it)
+  objective:  24646.873438121496
+Minimizing 1199    Time: 0:00:15 (12.91 ms/it)
+  objective:  24646.872948485317
+Minimizing 1207    Time: 0:00:15 (12.91 ms/it)
+  objective:  24646.8732539235
+Minimizing 1215    Time: 0:00:15 (12.91 ms/it)
+  objective:  24646.87300746512
+Minimizing 1223    Time: 0:00:15 (12.91 ms/it)
+  objective:  24646.872786788605
+Minimizing 1231    Time: 0:00:15 (12.91 ms/it)
+  objective:  24646.872757429453
+Minimizing 1239    Time: 0:00:15 (12.91 ms/it)
+  objective:  24646.872603076466
+Minimizing 1247    Time: 0:00:16 (12.91 ms/it)
+  objective:  24646.87241628748
+Minimizing 1255    Time: 0:00:16 (12.91 ms/it)
+  objective:  24646.872497083332
+Minimizing 1264    Time: 0:00:16 (12.91 ms/it)
+  objective:  24646.872561572494
+Minimizing 1273    Time: 0:00:16 (12.91 ms/it)
+  objective:  24646.87224029468
+Minimizing 1281    Time: 0:00:16 (12.91 ms/it)
+  objective:  24646.87251841221
+Minimizing 1289    Time: 0:00:16 (12.91 ms/it)
+  objective:  24646.872198901816
+Minimizing 1297    Time: 0:00:16 (12.91 ms/it)
+  objective:  24646.872169110415
+Minimizing 1306    Time: 0:00:16 (12.91 ms/it)
+  objective:  24646.872090365337
+Minimizing 1314    Time: 0:00:16 (12.91 ms/it)
+  objective:  24646.872070330082
+Minimizing 1322    Time: 0:00:17 (12.92 ms/it)
+  objective:  24646.872060506095
+Minimizing 1330    Time: 0:00:17 (12.92 ms/it)
+  objective:  24646.87204513132
+Minimizing 1336    Time: 0:00:17 (12.95 ms/it)
+  objective:  24646.87204792878
+Minimizing 1343    Time: 0:00:17 (12.95 ms/it)
+  objective:  24646.872012693733
+Minimizing 1350    Time: 0:00:17 (12.97 ms/it)
+  objective:  24646.871999364565
+Minimizing 1356    Time: 0:00:17 (12.99 ms/it)
+  objective:  24646.87199265028
+Minimizing 1363    Time: 0:00:17 (13.00 ms/it)
+  objective:  24646.871988545125
+Minimizing 1370    Time: 0:00:17 (13.02 ms/it)
+  objective:  24646.871980256794
+Minimizing 1376    Time: 0:00:17 (13.04 ms/it)
+  objective:  24646.871978006355
+Minimizing 1384    Time: 0:00:18 (13.04 ms/it)
+  objective:  24646.871966648752
+Minimizing 1391    Time: 0:00:18 (13.06 ms/it)
+  objective:  24646.87196321425
+Minimizing 1398    Time: 0:00:18 (13.07 ms/it)
+  objective:  24646.87195640274
+Minimizing 1406    Time: 0:00:18 (13.07 ms/it)
+  objective:  24646.87195198112
+Minimizing 1414    Time: 0:00:18 (13.08 ms/it)
+  objective:  24646.871947813554
+Minimizing 1422    Time: 0:00:18 (13.08 ms/it)
+  objective:  24646.871944653773
+Minimizing 1429    Time: 0:00:18 (13.08 ms/it)
+  objective:  24646.8719410536
+Minimizing 1437    Time: 0:00:18 (13.09 ms/it)
+  objective:  24646.871936481075
+Minimizing 1444    Time: 0:00:18 (13.10 ms/it)
+  objective:  24646.871933621453
+Minimizing 1451    Time: 0:00:19 (13.10 ms/it)
+  objective:  24646.871927196844
+Minimizing 1459    Time: 0:00:19 (13.11 ms/it)
+  objective:  24646.871925448948
+Minimizing 1467    Time: 0:00:19 (13.12 ms/it)
+  objective:  24646.871922590137
+Minimizing 1475    Time: 0:00:19 (13.12 ms/it)
+  objective:  24646.871922920815
+Minimizing 1483    Time: 0:00:19 (13.12 ms/it)
+  objective:  24646.871921975307
+Minimizing 1491    Time: 0:00:19 (13.13 ms/it)
+  objective:  24646.871918160934
+Minimizing 1499    Time: 0:00:19 (13.13 ms/it)
+  objective:  24646.87191685501
+Minimizing 1507    Time: 0:00:19 (13.14 ms/it)
+  objective:  24646.87191685694
+Minimizing 1515    Time: 0:00:19 (13.14 ms/it)
+  objective:  24646.871916002106
+Minimizing 1523    Time: 0:00:20 (13.14 ms/it)
+  objective:  24646.871913093604
+Minimizing 1530    Time: 0:00:20 (13.15 ms/it)
+
+Minimizing 1531    Time: 0:00:20 (13.15 ms/it)
+  objective:  24646.871911983937
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Childσ_Schoolσ_Cohort
(Intercept)-0.01360.0195-0.700.4844
Test: Star_r0.00930.03650.260.79860.79690.31030.0208
Test: S20_r0.00600.03540.170.86530.76750.33050.0563
Test: SLJ0.00520.03390.150.87870.78030.24780.0156
Test: BMT-0.03880.0300-1.290.19670.69670.19690.0000
a10.19350.05453.550.00040.2833
Sex: Girls-0.42900.0312-13.75<1e-420.1434
Test: Star_r & a10.29320.08873.310.0009
Test: S20_r & a1-0.10070.0826-1.220.2228
Test: SLJ & a1-0.04560.0773-0.590.5549
Test: BMT & a10.46850.08535.50<1e-07
Test: Star_r & Sex: Girls0.37590.05117.36<1e-12
Test: S20_r & Sex: Girls-0.18920.0475-3.98<1e-04
Test: SLJ & Sex: Girls-0.06800.0445-1.530.1262
Test: BMT & Sex: Girls-0.28150.0495-5.68<1e-07
a1 & Sex: Girls-0.12620.1049-1.200.2290
Test: Star_r & a1 & Sex: Girls-0.13710.1756-0.780.4348
Test: S20_r & a1 & Sex: Girls0.17200.16341.050.2926
Test: SLJ & a1 & Sex: Girls-0.01250.1529-0.080.9348
Test: BMT & a1 & Sex: Girls0.15420.17020.910.3648
Test: Run0.74980.37020.0548
Residual0.5122
+
+
+
+
issingular(m2)
+
+
true
+
+
+

Depending on the random number generator seed, the model may or may not be supported in the alternative parameterization of scores. The fixed-effects profile is not affected (see 2.8 below).

+
+
+
+ +
+
+Caution +
+
+
+

RK: The order of RE terms is critical. In formula f2 the zerocorr() term must be placed last as shown. If it is placed first, School-related and Child-related CPs are estimated/reported (?) as zero. This was not the case for formula m1. Thus, it appears to be related to the 0-intercepts in School and Child terms. Need a reprex.

+
+
+
+
VarCorr(m2)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
ChildTest: Run0.56222900.7498193
Test: Star_r0.63508990.7969253+0.50
Test: S20_r0.58907960.7675152+0.56+0.64
Test: SLJ0.60893460.7803426+0.55+0.60+0.72
Test: BMT0.48543980.6967351+0.19+0.40+0.37+0.49
SchoolTest: Run0.13704550.3701965
Test: Star_r0.09627830.3102874+0.53
Test: S20_r0.10923830.3305122+0.47+0.48
Test: SLJ0.06138380.2477575+0.47+0.76+0.41
Test: BMT0.03875410.1968606+0.15+0.38+0.18+0.02
a10.08023360.2832554+0.58+0.47+0.56-0.05+0.65
Sex: Girls0.02055460.1433686-0.63-0.27+0.06-0.28-0.58-0.37
CohortTest: Run0.00300660.0548320
Test: Star_r0.00043440.0208416.
Test: S20_r0.00317160.0563170..
Test: SLJ0.00024480.0156459...
Test: BMT0.00000000.0000000....
Residual0.26231270.5121647
+
+
+
+
+

4.7 Principle Component Analysis of Random Effect Structure (rePCA)

+

The ìssingular() command is sort of a shortcut for a quick inspection of the principle components (PCs) of the variance-covariance matrix of the RES. With the MixedModels.PCA() command, we also obtain information about the amount of cumulative variance accounted for as we add PCs.

+

The output also provides PC loadings which may facilitate interpretation of the CP matrices (if estimated). This topic will be picked uo in a separate vignette. See also Fühner et al. (2021) for an application.

+
+
+

4.8 Effects in RES

+

For every random factor, MixedModels.PCA() extracts as many PCs as there are VCs. Therefore, the cumulation of variance across PCs within a random factor will always add up to 100% – at the latest with the last VC, but, in the case of overparameterized LMMs, the ceiling will be reached earlier. The final PCs are usually quite small.

+

PCs are extracted in the order of the amount of unique variance they account for. The first PC accounts for the largest and the final PC for the least amount of variance. The number the PCs with percent variance above a certain threshold indicates the number of weighted composites needed and reflects the dimensionality of the orthogonal space within which (almost) all the variance can be accounted for. The weights for forming composite scores are the listed loadings. For ease of interpretation it is often useful to change the sign of some composite scores.

+

The PCA for LMM m1 shows that each of the five PCs for Child accounts for a non-zero percent of unique variance.

+

For School fewer than seven PCs have unique variance. The exact number depends on sampling. The overparameterization of School might be resolved when the CPs for Sex are dropped from the LMM.

+

Cohort was estimated with CPs forced to zero. Therefore, the VCs were forced to be orthogonal; they already represent the PCA solution. However, depending on sampling, not all PCs may be identified for this random factor either.

+

Importantly, again depending on sampling, a non-singular fit does not imply that unique variance is associated with all PCs (i.e., not for last PC for School). Embrace uncertainty!

+
+
MixedModels.PCA(m1)
+
+
(Child = 
+Principal components based on correlation matrix
+ (Intercept)    1.0     .      .      .      .
+ Test: Star_r   0.14   1.0     .      .      .
+ Test: S20_r    0.0   -0.52   1.0     .      .
+ Test: SLJ      0.05  -0.04  -0.35   1.0     .
+ Test: BMT     -0.35   0.14  -0.18  -0.22   1.0
+
+Normalized cumulative variances:
+[0.3288, 0.616, 0.8285, 0.9371, 1.0]
+
+Component loadings
+                 PC1    PC2    PC3    PC4    PC5
+ (Intercept)   -0.04   0.58   0.55  -0.6   -0.03
+ Test: Star_r  -0.61  -0.04   0.45   0.4    0.51
+ Test: S20_r    0.71  -0.02   0.15   0.03   0.69
+ Test: SLJ     -0.28   0.44  -0.69  -0.21   0.46
+ Test: BMT     -0.21  -0.69  -0.01  -0.66   0.22, School = 
+Principal components based on correlation matrix
+ (Intercept)    1.0     .      .      .      .      .      .
+ Test: Star_r  -0.06   1.0     .      .      .      .      .
+ Test: S20_r   -0.08  -0.39   1.0     .      .      .      .
+ Test: SLJ     -0.19   0.21  -0.81   1.0     .      .      .
+ Test: BMT     -0.33  -0.02   0.13  -0.38   1.0     .      .
+ a1             0.62  -0.2    0.11  -0.61   0.45   1.0     .
+ Sex: Girls    -0.45   0.45   0.31  -0.27  -0.15  -0.37   1.0
+
+Normalized cumulative variances:
+[0.3567, 0.6348, 0.8056, 0.9684, 0.9999, 1.0, 1.0]
+
+Component loadings
+                 PC1    PC2    PC3    PC4    PC5    PC6    PC7
+ (Intercept)   -0.25  -0.49  -0.41  -0.37   0.19   0.52   0.27
+ Test: Star_r   0.31   0.12   0.18  -0.75   0.47  -0.25  -0.09
+ Test: S20_r   -0.4    0.44  -0.32   0.2    0.52  -0.28   0.4
+ Test: SLJ      0.55  -0.31   0.1    0.2    0.02  -0.19   0.72
+ Test: BMT     -0.29   0.14   0.79   0.01   0.16   0.42   0.27
+ a1            -0.52  -0.27   0.16  -0.31  -0.42  -0.56   0.21
+ Sex: Girls     0.14   0.6   -0.19  -0.34  -0.53   0.24   0.35, Cohort = 
+Principal components based on correlation matrix
+ (Intercept)   1.0  .    .    .    .
+ Test: Star_r  0.0  1.0  .    .    .
+ Test: S20_r   0.0  0.0  1.0  .    .
+ Test: SLJ     0.0  0.0  0.0  1.0  .
+ Test: BMT     0.0  0.0  0.0  0.0  0.0
+
+Normalized cumulative variances:
+[0.25, 0.5, 0.75, 1.0, 1.0]
+
+Component loadings
+                PC1   PC2   PC3   PC4     PC5
+ (Intercept)   1.0   0.0   0.0   0.0     0.0
+ Test: Star_r  0.0   1.0   0.0   0.0     0.0
+ Test: S20_r   0.0   0.0   1.0   0.0     0.0
+ Test: SLJ     0.0   0.0   0.0   1.0     0.0
+ Test: BMT     0.0   0.0   0.0   0.0   NaN)
+
+
+
+

4.8.1 Scores in RES

+

Now lets looks at the PCA results for the alternative parameterization of LMM m2. It is important to note that the reparameterization to base estimates of VCs and CPs on scores rather than effects applies only to the Test factor (i.e., the first factor in the formula); VCs for Sex and age refer to the associated effects.

+

Depending on sampling, the difference between LMM m1 and LMM m2 may show that overparameterization according to PCs may depend on the specification chosen for the other the random-effect structure.

+
+
+
+ +
+
+Note +
+
+
+

For the complete data, all PCs had unique variance associated with them.

+
+
+
+
MixedModels.PCA(m2)
+
+
(Child = 
+Principal components based on correlation matrix
+ Test: Run     1.0    .     .     .     .
+ Test: Star_r  0.5   1.0    .     .     .
+ Test: S20_r   0.56  0.64  1.0    .     .
+ Test: SLJ     0.55  0.6   0.72  1.0    .
+ Test: BMT     0.19  0.4   0.37  0.49  1.0
+
+Normalized cumulative variances:
+[0.6117, 0.7769, 0.8677, 0.9485, 1.0]
+
+Component loadings
+                 PC1    PC2    PC3    PC4    PC5
+ Test: Run     -0.42   0.53   0.63   0.37   0.08
+ Test: Star_r  -0.47   0.03  -0.65   0.58  -0.16
+ Test: S20_r   -0.49   0.15  -0.24  -0.49   0.66
+ Test: SLJ     -0.5   -0.05   0.09  -0.5   -0.7
+ Test: BMT     -0.34  -0.83   0.33   0.2    0.2, School = 
+Principal components based on correlation matrix
+ Test: Run      1.0     .      .      .      .      .      .
+ Test: Star_r   0.53   1.0     .      .      .      .      .
+ Test: S20_r    0.47   0.48   1.0     .      .      .      .
+ Test: SLJ      0.47   0.76   0.41   1.0     .      .      .
+ Test: BMT      0.15   0.38   0.18   0.02   1.0     .      .
+ a1             0.58   0.47   0.56  -0.05   0.65   1.0     .
+ Sex: Girls    -0.63  -0.27   0.06  -0.28  -0.58  -0.37   1.0
+
+Normalized cumulative variances:
+[0.4822, 0.6938, 0.8508, 0.9553, 1.0, 1.0, 1.0]
+
+Component loadings
+                 PC1    PC2    PC3    PC4    PC5    PC6    PC7
+ Test: Run     -0.44   0.08   0.15  -0.64  -0.16   0.41   0.42
+ Test: Star_r  -0.44   0.28   0.02   0.41  -0.56   0.29  -0.4
+ Test: S20_r   -0.34   0.27  -0.58  -0.1    0.6    0.15  -0.28
+ Test: SLJ     -0.32   0.56   0.33   0.25   0.22  -0.48   0.36
+ Test: BMT     -0.32  -0.53  -0.02   0.55   0.25   0.3    0.41
+ a1            -0.41  -0.35  -0.42  -0.14  -0.33  -0.63   0.05
+ Sex: Girls     0.34   0.35  -0.59   0.18  -0.29   0.14   0.53, Cohort = 
+Principal components based on correlation matrix
+ Test: Run     1.0  .    .    .    .
+ Test: Star_r  0.0  1.0  .    .    .
+ Test: S20_r   0.0  0.0  1.0  .    .
+ Test: SLJ     0.0  0.0  0.0  1.0  .
+ Test: BMT     0.0  0.0  0.0  0.0  0.0
+
+Normalized cumulative variances:
+[0.25, 0.5, 0.75, 1.0, 1.0]
+
+Component loadings
+                PC1   PC2   PC3   PC4     PC5
+ Test: Run     0.0   0.0   1.0   0.0     0.0
+ Test: Star_r  1.0   0.0   0.0   0.0     0.0
+ Test: S20_r   0.0   1.0   0.0   0.0     0.0
+ Test: SLJ     0.0   0.0   0.0   1.0     0.0
+ Test: BMT     0.0   0.0   0.0   0.0   NaN)
+
+
+
+
+
+

4.9 Summary of results for stratified subset of data

+

Returning to the theoretical focus of the article, the significant main effects of age and Sex, the interactions between age and c1 and c4 contrasts and the interactions between Sex and three test contrasts (c1, c2, c4) are replicated. Obviously, the subset of data is much noisier than the full set.

+
+
+
+

5 Age x Sex nested in levels of Test

+

In this final LMM, we test post-hoc five age x Sex interactions by nesting the interaction in the levels of Test. As this LMM m2_nested is a reparameterization of LMM m2.

+
+
m2_nested = let
+  f = @formula(
+    zScore ~
+      1 +
+      Test +
+      Test & (a1 * Sex) +
+      (0 + Test + a1 + Sex | School) +
+      (0 + Test | Child) +
+      zerocorr(0 + Test | Cohort)
+  )
+  fit(MixedModel, f, dat; contrasts)
+end
+
+
Minimizing 8    Time: 0:00:00 (13.70 ms/it)
+  objective:  26540.628428689935
+Minimizing 16    Time: 0:00:00 (13.64 ms/it)
+  objective:  27013.20266803618
+Minimizing 24    Time: 0:00:00 (13.30 ms/it)
+  objective:  26938.094323301608
+Minimizing 32    Time: 0:00:00 (13.17 ms/it)
+  objective:  26816.3549662712
+Minimizing 40    Time: 0:00:00 (13.11 ms/it)
+  objective:  26764.50301863997
+Minimizing 48    Time: 0:00:00 (13.04 ms/it)
+  objective:  26748.52532251481
+Minimizing 56    Time: 0:00:00 (13.00 ms/it)
+  objective:  27607.74073471175
+Minimizing 64    Time: 0:00:00 (12.99 ms/it)
+  objective:  27035.518325217672
+Minimizing 72    Time: 0:00:00 (12.99 ms/it)
+  objective:  26743.04860536076
+Minimizing 80    Time: 0:00:01 (13.00 ms/it)
+  objective:  26914.37946666574
+Minimizing 88    Time: 0:00:01 (13.00 ms/it)
+  objective:  26750.53281469535
+Minimizing 96    Time: 0:00:01 (13.01 ms/it)
+  objective:  26718.60071956578
+Minimizing 104    Time: 0:00:01 (13.02 ms/it)
+  objective:  25121.98047503751
+Minimizing 112    Time: 0:00:01 (13.03 ms/it)
+  objective:  25011.13301114361
+Minimizing 120    Time: 0:00:01 (13.04 ms/it)
+  objective:  24997.24925764848
+Minimizing 128    Time: 0:00:01 (13.04 ms/it)
+  objective:  24953.65373233215
+Minimizing 136    Time: 0:00:01 (13.04 ms/it)
+  objective:  24958.120022841867
+Minimizing 144    Time: 0:00:01 (13.03 ms/it)
+  objective:  24900.66839637635
+Minimizing 152    Time: 0:00:01 (13.03 ms/it)
+  objective:  24893.96206104777
+Minimizing 160    Time: 0:00:02 (13.03 ms/it)
+  objective:  24873.63692651054
+Minimizing 168    Time: 0:00:02 (13.03 ms/it)
+  objective:  24859.80007224154
+Minimizing 176    Time: 0:00:02 (13.03 ms/it)
+  objective:  24821.00416416074
+Minimizing 184    Time: 0:00:02 (13.02 ms/it)
+  objective:  24798.88989693187
+Minimizing 192    Time: 0:00:02 (13.03 ms/it)
+  objective:  24766.536560173186
+Minimizing 200    Time: 0:00:02 (13.09 ms/it)
+  objective:  24761.12632180778
+Minimizing 208    Time: 0:00:02 (13.08 ms/it)
+  objective:  24724.779616583975
+Minimizing 216    Time: 0:00:02 (13.08 ms/it)
+  objective:  24724.49577912816
+Minimizing 224    Time: 0:00:02 (13.08 ms/it)
+  objective:  24715.333118629766
+Minimizing 232    Time: 0:00:03 (13.08 ms/it)
+  objective:  24719.605256362156
+Minimizing 240    Time: 0:00:03 (13.07 ms/it)
+  objective:  24711.563183426377
+Minimizing 248    Time: 0:00:03 (13.06 ms/it)
+  objective:  24704.355886249505
+Minimizing 256    Time: 0:00:03 (13.04 ms/it)
+  objective:  24698.8869058769
+Minimizing 264    Time: 0:00:03 (13.03 ms/it)
+  objective:  24691.48588894165
+Minimizing 272    Time: 0:00:03 (13.02 ms/it)
+  objective:  24687.055840617446
+Minimizing 280    Time: 0:00:03 (13.01 ms/it)
+  objective:  24686.841979619934
+Minimizing 288    Time: 0:00:03 (13.00 ms/it)
+  objective:  24689.10334812927
+Minimizing 296    Time: 0:00:03 (13.00 ms/it)
+  objective:  24686.91689329679
+Minimizing 304    Time: 0:00:03 (12.99 ms/it)
+  objective:  24679.277504378595
+Minimizing 312    Time: 0:00:04 (12.99 ms/it)
+  objective:  24674.37908345073
+Minimizing 320    Time: 0:00:04 (13.00 ms/it)
+  objective:  24673.037400233883
+Minimizing 328    Time: 0:00:04 (13.00 ms/it)
+  objective:  24665.151187408235
+Minimizing 336    Time: 0:00:04 (12.99 ms/it)
+  objective:  24665.56628719779
+Minimizing 344    Time: 0:00:04 (12.98 ms/it)
+  objective:  24663.666240292307
+Minimizing 352    Time: 0:00:04 (12.98 ms/it)
+  objective:  24660.144044891324
+Minimizing 360    Time: 0:00:04 (12.97 ms/it)
+  objective:  24656.98099205523
+Minimizing 368    Time: 0:00:04 (12.96 ms/it)
+  objective:  24657.25658119654
+Minimizing 376    Time: 0:00:04 (12.96 ms/it)
+  objective:  24655.730669618533
+Minimizing 384    Time: 0:00:04 (12.97 ms/it)
+  objective:  24655.69430533625
+Minimizing 392    Time: 0:00:05 (12.98 ms/it)
+  objective:  24654.44310560729
+Minimizing 400    Time: 0:00:05 (12.99 ms/it)
+  objective:  24653.219229949507
+Minimizing 408    Time: 0:00:05 (13.00 ms/it)
+  objective:  24652.796781257577
+Minimizing 416    Time: 0:00:05 (13.01 ms/it)
+  objective:  24652.338978753978
+Minimizing 424    Time: 0:00:05 (13.02 ms/it)
+  objective:  24651.634776007268
+Minimizing 432    Time: 0:00:05 (13.02 ms/it)
+  objective:  24651.28876234426
+Minimizing 440    Time: 0:00:05 (13.04 ms/it)
+  objective:  24651.142181131778
+Minimizing 448    Time: 0:00:05 (13.04 ms/it)
+  objective:  24650.645188828414
+Minimizing 456    Time: 0:00:05 (13.06 ms/it)
+  objective:  24649.9503876729
+Minimizing 464    Time: 0:00:06 (13.07 ms/it)
+  objective:  24649.431921117473
+Minimizing 472    Time: 0:00:06 (13.08 ms/it)
+  objective:  24649.263203381008
+Minimizing 480    Time: 0:00:06 (13.08 ms/it)
+  objective:  24649.182140293356
+Minimizing 488    Time: 0:00:06 (13.08 ms/it)
+  objective:  24649.076840932066
+Minimizing 496    Time: 0:00:06 (13.08 ms/it)
+  objective:  24648.86837610833
+Minimizing 504    Time: 0:00:06 (13.08 ms/it)
+  objective:  24648.66301915268
+Minimizing 512    Time: 0:00:06 (13.09 ms/it)
+  objective:  24648.46906034212
+Minimizing 520    Time: 0:00:06 (13.10 ms/it)
+  objective:  24648.40844853464
+Minimizing 527    Time: 0:00:06 (13.13 ms/it)
+  objective:  24648.324979985635
+Minimizing 534    Time: 0:00:07 (13.14 ms/it)
+  objective:  24648.227736776775
+Minimizing 542    Time: 0:00:07 (13.14 ms/it)
+  objective:  24648.0698415218
+Minimizing 551    Time: 0:00:07 (13.13 ms/it)
+  objective:  24647.96157394349
+Minimizing 560    Time: 0:00:07 (13.12 ms/it)
+  objective:  24647.81291490624
+Minimizing 569    Time: 0:00:07 (13.10 ms/it)
+  objective:  24647.821741951735
+Minimizing 578    Time: 0:00:07 (13.09 ms/it)
+  objective:  24647.70521181008
+Minimizing 586    Time: 0:00:07 (13.09 ms/it)
+  objective:  24647.621294995508
+Minimizing 594    Time: 0:00:07 (13.08 ms/it)
+  objective:  24647.523956625257
+Minimizing 602    Time: 0:00:07 (13.08 ms/it)
+  objective:  24647.432574352344
+Minimizing 610    Time: 0:00:07 (13.09 ms/it)
+  objective:  24647.417512320426
+Minimizing 617    Time: 0:00:08 (13.10 ms/it)
+  objective:  24647.341430766
+Minimizing 624    Time: 0:00:08 (13.12 ms/it)
+  objective:  24647.28424499031
+Minimizing 631    Time: 0:00:08 (13.14 ms/it)
+  objective:  24647.319589671955
+Minimizing 638    Time: 0:00:08 (13.17 ms/it)
+  objective:  24647.294944454985
+Minimizing 645    Time: 0:00:08 (13.18 ms/it)
+  objective:  24647.20468054565
+Minimizing 653    Time: 0:00:08 (13.20 ms/it)
+  objective:  24647.136963121862
+Minimizing 659    Time: 0:00:08 (13.27 ms/it)
+  objective:  24647.111462800207
+Minimizing 666    Time: 0:00:08 (13.31 ms/it)
+  objective:  24647.087755084372
+Minimizing 673    Time: 0:00:08 (13.33 ms/it)
+  objective:  24647.09819673813
+Minimizing 680    Time: 0:00:09 (13.36 ms/it)
+  objective:  24647.07095995461
+Minimizing 687    Time: 0:00:09 (13.38 ms/it)
+  objective:  24647.05393722359
+Minimizing 694    Time: 0:00:09 (13.41 ms/it)
+  objective:  24647.03777228371
+Minimizing 702    Time: 0:00:09 (13.41 ms/it)
+  objective:  24647.024951772564
+Minimizing 710    Time: 0:00:09 (13.41 ms/it)
+  objective:  24647.003427057458
+Minimizing 718    Time: 0:00:09 (13.41 ms/it)
+  objective:  24646.99498896721
+Minimizing 726    Time: 0:00:09 (13.40 ms/it)
+  objective:  24646.990730607544
+Minimizing 734    Time: 0:00:09 (13.40 ms/it)
+  objective:  24646.97906190287
+Minimizing 742    Time: 0:00:09 (13.40 ms/it)
+  objective:  24646.975885698288
+Minimizing 749    Time: 0:00:10 (13.43 ms/it)
+  objective:  24646.972542464464
+Minimizing 756    Time: 0:00:10 (13.44 ms/it)
+  objective:  24646.968502912674
+Minimizing 764    Time: 0:00:10 (13.43 ms/it)
+  objective:  24646.962831158467
+Minimizing 772    Time: 0:00:10 (13.43 ms/it)
+  objective:  24646.958674438207
+Minimizing 780    Time: 0:00:10 (13.43 ms/it)
+  objective:  24646.955826234305
+Minimizing 788    Time: 0:00:10 (13.43 ms/it)
+  objective:  24646.953060155472
+Minimizing 796    Time: 0:00:10 (13.42 ms/it)
+  objective:  24646.9521054153
+Minimizing 804    Time: 0:00:10 (13.42 ms/it)
+  objective:  24646.948478094586
+Minimizing 812    Time: 0:00:10 (13.42 ms/it)
+  objective:  24646.944019135743
+Minimizing 820    Time: 0:00:11 (13.42 ms/it)
+  objective:  24646.938953934987
+Minimizing 828    Time: 0:00:11 (13.43 ms/it)
+  objective:  24646.93645979881
+Minimizing 836    Time: 0:00:11 (13.43 ms/it)
+  objective:  24646.933609880332
+Minimizing 843    Time: 0:00:11 (13.47 ms/it)
+  objective:  24646.932514518587
+Minimizing 849    Time: 0:00:11 (13.52 ms/it)
+  objective:  24646.929970979138
+Minimizing 855    Time: 0:00:11 (13.55 ms/it)
+  objective:  24646.92748526191
+Minimizing 863    Time: 0:00:11 (13.55 ms/it)
+  objective:  24646.925214377407
+Minimizing 871    Time: 0:00:11 (13.55 ms/it)
+  objective:  24646.920939489395
+Minimizing 878    Time: 0:00:11 (13.57 ms/it)
+  objective:  24646.918463218877
+Minimizing 885    Time: 0:00:12 (13.58 ms/it)
+  objective:  24646.916193014382
+Minimizing 892    Time: 0:00:12 (13.60 ms/it)
+  objective:  24646.913466439146
+Minimizing 900    Time: 0:00:12 (13.60 ms/it)
+  objective:  24646.911044916513
+Minimizing 909    Time: 0:00:12 (13.59 ms/it)
+  objective:  24646.907532175603
+Minimizing 917    Time: 0:00:12 (13.58 ms/it)
+  objective:  24646.90240499966
+Minimizing 925    Time: 0:00:12 (13.57 ms/it)
+  objective:  24646.900343604117
+Minimizing 934    Time: 0:00:12 (13.56 ms/it)
+  objective:  24646.898298083204
+Minimizing 942    Time: 0:00:12 (13.56 ms/it)
+  objective:  24646.899863259383
+Minimizing 950    Time: 0:00:12 (13.55 ms/it)
+  objective:  24646.894943659114
+Minimizing 958    Time: 0:00:12 (13.55 ms/it)
+  objective:  24646.89168722892
+Minimizing 966    Time: 0:00:13 (13.55 ms/it)
+  objective:  24646.891871020714
+Minimizing 974    Time: 0:00:13 (13.56 ms/it)
+  objective:  24646.888629902307
+Minimizing 982    Time: 0:00:13 (13.56 ms/it)
+  objective:  24646.88626510772
+Minimizing 990    Time: 0:00:13 (13.56 ms/it)
+  objective:  24646.88553929728
+Minimizing 998    Time: 0:00:13 (13.57 ms/it)
+  objective:  24646.88299185763
+Minimizing 1006    Time: 0:00:13 (13.56 ms/it)
+  objective:  24646.88143965367
+Minimizing 1014    Time: 0:00:13 (13.57 ms/it)
+  objective:  24646.88048005483
+Minimizing 1022    Time: 0:00:13 (13.57 ms/it)
+  objective:  24646.87982841355
+Minimizing 1030    Time: 0:00:13 (13.57 ms/it)
+  objective:  24646.879231505678
+Minimizing 1038    Time: 0:00:14 (13.57 ms/it)
+  objective:  24646.87890762983
+Minimizing 1046    Time: 0:00:14 (13.57 ms/it)
+  objective:  24646.877528120545
+Minimizing 1054    Time: 0:00:14 (13.57 ms/it)
+  objective:  24646.876792163104
+Minimizing 1062    Time: 0:00:14 (13.57 ms/it)
+  objective:  24646.876315545996
+Minimizing 1070    Time: 0:00:14 (13.58 ms/it)
+  objective:  24646.87562600356
+Minimizing 1078    Time: 0:00:14 (13.58 ms/it)
+  objective:  24646.875434427
+Minimizing 1086    Time: 0:00:14 (13.58 ms/it)
+  objective:  24646.87526486469
+Minimizing 1088    Time: 0:00:14 (13.58 ms/it)
+
+Minimizing 1089    Time: 0:00:14 (13.58 ms/it)
+  objective:  24646.875245743373
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Childσ_Schoolσ_Cohort
(Intercept)-0.01360.0195-0.700.4840
Test: Star_r0.00930.03660.260.79840.79770.31030.0213
Test: S20_r0.00600.03540.170.86570.76830.33070.0563
Test: SLJ0.00520.03390.150.87810.78110.24790.0157
Test: BMT-0.03880.0300-1.290.19700.69780.19670.0000
Test: Run & a1-0.05600.0783-0.720.4745
Test: Star_r & a10.23720.08012.960.0031
Test: S20_r & a10.13650.07861.740.0823
Test: SLJ & a10.09080.07741.170.2405
Test: BMT & a10.55930.07167.81<1e-14
Test: Run & Sex: Girls-0.53270.0448-11.90<1e-31
Test: Star_r & Sex: Girls-0.15690.0461-3.400.0007
Test: S20_r & Sex: Girls-0.34600.0449-7.70<1e-13
Test: SLJ & Sex: Girls-0.41400.0447-9.26<1e-19
Test: BMT & Sex: Girls-0.69550.0414-16.80<1e-62
Test: Run & a1 & Sex: Girls-0.14560.1523-0.960.3393
Test: Star_r & a1 & Sex: Girls-0.28260.1572-1.800.0723
Test: S20_r & a1 & Sex: Girls-0.11060.1530-0.720.4696
Test: SLJ & a1 & Sex: Girls-0.12310.1516-0.810.4168
Test: BMT & a1 & Sex: Girls0.03110.14040.220.8246
a10.2847
Sex: Girls0.1431
Test: Run0.75060.37030.0548
Residual0.5109
+
+
+

The results show that none of the interactions in the panels of Figure 2 is significant. The size and direction of interaction effects correspond with what is shown in Figure 2.

+
+

5.0.1 CONSTRUCTION SITE: More model comparisons

+
+
+Code +
gof_summary3 = let
+  nms = [:m1, :m2, :m2_nested]
+  mods = eval.(nms)
+  DataFrame(;
+    name=nms,
+    dof=dof.(mods),
+    deviance=deviance.(mods),
+    AIC=aic.(mods),
+    AICc=aicc.(mods),
+    BIC=bic.(mods),
+  )
+end
+
+
+
3×6 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RownamedofdevianceAICAICcBIC
SymbolInt64Float64Float64Float64Float64
1m16924651.024789.024790.025284.2
2m26924646.924784.924785.925280.0
3m2_nested6924646.924784.924785.925280.0
+
+
+
+
+
n, p, q, k = size(m1)  # nobs, fe params, VCs+CPs, re terms
+
+
(9663, 20, 13076, 3)
+
+
+

In principle, the models should yield the save deviance. When models are not supported by the data, that is for singular models, there may be small differences between deviances for these reparameterizations. During optimization such models search for the absolute minimum in a very shallow surface and may end up in a local minimum instead.

+
+
+

5.0.2 Geometric degrees of freedom

+

From MixedModels documentation: “The sum of the leverage values is the rank of the model matrix and n - sum(leverage(m)) is the degrees of freedom for residuals. The sum of the leverage values is also the trace of the so-called”hat” matrixH.”

+

New term: geometric degrees of freedom.

+
+
m1_geomdf = sum(leverage(m1))  # geom_dof
+
+
5442.453779672519
+
+
+
+
sum(leverage(m2))
+
+
5764.6241405763985
+
+
+
+
sum(leverage(m2_nested))
+
+
5784.56392159439
+
+
+
+
n - m1_geomdf
+
+
4220.546220327481
+
+
+
+
m1.feterm.rank
+
+
20
+
+
+
+
dof(m1)
+
+
69
+
+
+
+
+
+

6 Glossary of MixedModels.jl commands

+

Here we introduce most of the commands available in the MixedModels.jl package that allow the immediate inspection and analysis of results returned in a fitted linear mixed-effect model.

+

Postprocessing related to conditional modes will be dealt with in a different tutorial.

+
+

6.1 Overall summary statistics

+
+ julia> m1.optsum         # MixedModels.OptSummary:  gets all info
++ julia> loglikelihood(m1) # StatsBase.loglikelihood: return loglikelihood
+                             of the model
++ julia> deviance(m1)      # StatsBase.deviance: negative twice the log-likelihood
+                             relative to saturated model
++ julia> objective(m1)     # MixedModels.objective: saturated model not clear:
+                             negative twice the log-likelihood
++ julia> nobs(m1)          # n of observations; they are not independent
++ julia> dof(m1)           # n of degrees of freedom is number of model parameters
++ julia> aic(m1)           # objective(m1) + 2*dof(m1)
++ julia> bic(m1)           # objective(m1) + dof(m1)*log(nobs(m1))
+
+
m1.optsum            # MixedModels.OptSummary:  gets all info
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Initialization
Initial parameter vector[1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
Initial objective value25840.653755409803
Optimizer settings
Optimizer (from NLopt)LN_BOBYQA
Lower bounds[0.0, -Inf, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, 0.0, -Inf, 0.0, 0.0, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, 0.0, -Inf, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
ftol_rel1.0e-12
ftol_abs1.0e-8
xtol_rel0.0
xtol_abs[1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10]
initial_step[0.75, 1.0, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 0.75, 1.0, 0.75, 0.75, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 0.75, 1.0, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75]
maxfeval-1
maxtime-1.0
Result
Function evaluations1877
Final parameter vector[1.1017, 0.1998, 0.0008, 0.0492, -0.4673, 1.3801, -0.627, -0.0493, 0.2525, 1.0108, -0.4543, -0.1363, 0.907, -0.3692, 1.1806, 0.401, -0.0372, -0.0463, -0.1124, -0.1955, 0.3284, -0.1207, 0.6232, -0.2454, 0.121, -0.0246, -0.0876, 0.1131, 0.5624, -0.4873, 0.0578, 0.051, 0.1313, 0.3136, -0.404, -0.3872, -0.0241, 0.3732, 0.1119, -0.165, 0.0177, -0.0054, 0.0, 0.0248, 0.1196, 0.0142, 0.0636, 0.0]
Final objective value24651.014
Return codeFTOL_REACHED
+
+
+
+
loglikelihood(m1) # StatsBase.loglikelihood: return loglikelihood of the model
+
+
-12325.507018394583
+
+
+
+
deviance(m1)      # StatsBase.deviance: negative twice the log-likelihood relative to saturated mode`
+
+
24651.014036789165
+
+
+
+
objective(m1)    # MixedModels.objective: saturated model not clear: negative twice the log-likelihood
+
+
24651.014036789165
+
+
+
+
nobs(m1) # n of observations; they are not independent
+
+
9663
+
+
+
+
n_, p_, q_, k_ = size(m1)
+
+
(9663, 20, 13076, 3)
+
+
+
+
dof(m1)  # n of degrees of freedom is number of model parameters
+
+
69
+
+
+
+
geom_df = sum(leverage(m1)) # trace of hat / rank of model matrix / geom dof
+
+
5442.453779672519
+
+
+
+
resid_df = nobs(m1) - geom_df  # eff. residual degrees of freedom
+
+
4220.546220327481
+
+
+
+
aic(m1)  # objective(m1) + 2*dof(m1)
+
+
24789.014036789165
+
+
+
+
bic(m1)  # objective(m1) + dof(m1)*log(nobs(m1))
+
+
25284.162138011117
+
+
+
+
+

6.2 Fixed-effect statistics

+
+ julia> coeftable(m1)     # StatsBase.coeftable: fixed-effects statiscs;
+                             default level=0.95
++ julia> Arrow.write("./data/m_cpx_fe.arrow", DataFrame(coeftable(m1)));
++ julia> coef(m1)          # StatsBase.coef - parts of the table
++ julia> fixef(m1)         # MixedModels.fixef: not the same as coef()
+                             for rank-deficient case
++ julia> m1.beta           # alternative extractor
++ julia> fixefnames(m1)    # works also for coefnames(m1)
++ julia> vcov(m1)          # StatsBase.vcov: var-cov matrix of fixed-effects coef.
++ julia> stderror(m1)      # StatsBase.stderror: SE for fixed-effects coefficients
++ julia> propertynames(m1) # names of available extractors
+
+
coeftable(m1) # StatsBase.coeftable: fixed-effects statiscs; default level=0.95
+
+ +++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Coef.Std. ErrorzPr(>
(Intercept)-0.01332310.0191965-0.690.4877
Test: Star_r0.008838610.03747070.240.8135
Test: S20_r0.00915020.02927330.310.7546
Test: SLJ0.003378670.02995120.110.9102
Test: BMT-0.03829410.0295742-1.290.1954
a10.1955880.05445373.590.0003
Sex: Girls-0.4287430.0311829-13.75<1e-42
Test: Star_r & a10.2897780.08874713.270.0011
Test: S20_r & a1-0.07844860.0817091-0.960.3370
Test: SLJ & a1-0.06309660.0768863-0.820.4118
Test: BMT & a10.4713130.08515045.54<1e-07
Test: Star_r & Sex: Girls0.3755950.05108317.35<1e-12
Test: S20_r & Sex: Girls-0.1867260.0475081-3.93<1e-04
Test: SLJ & Sex: Girls-0.06950770.0445161-1.560.1184
Test: BMT & Sex: Girls-0.2812370.0495209-5.68<1e-07
a1 & Sex: Girls-0.1266580.104828-1.210.2269
Test: Star_r & a1 & Sex: Girls-0.1382020.17558-0.790.4312
Test: S20_r & a1 & Sex: Girls0.1752870.1633351.070.2832
Test: SLJ & a1 & Sex: Girls-0.0154490.152981-0.100.9196
Test: BMT & a1 & Sex: Girls0.1547820.1701580.910.3630
+
+
+
+
#Arrow.write("./data/m_cpx_fe.arrow", DataFrame(coeftable(m1)));
+
+
+
coef(m1)              # StatsBase.coef; parts of the table
+
+
20-element Vector{Float64}:
+ -0.013323098599731003
+  0.008838610043037353
+  0.009150200653972363
+  0.0033786745702838548
+ -0.038294149701005455
+  0.19558778043000458
+ -0.4287431844207055
+  0.2897784524160024
+ -0.07844856274209384
+ -0.06309657343028045
+  0.4713127757906535
+  0.37559520558051235
+ -0.18672567514518862
+ -0.06950765283326864
+ -0.28123708084784266
+ -0.12665848284693404
+ -0.1382024418769914
+  0.17528665552673206
+ -0.015448951451204404
+  0.15478177087438777
+
+
+
+
fixef(m1)    # MixedModels.fixef: not the same as coef() for rank-deficient case
+
+
20-element Vector{Float64}:
+ -0.013323098599731003
+  0.008838610043037353
+  0.009150200653972363
+  0.0033786745702838548
+ -0.038294149701005455
+  0.19558778043000458
+ -0.4287431844207055
+  0.2897784524160024
+ -0.07844856274209384
+ -0.06309657343028045
+  0.4713127757906535
+  0.37559520558051235
+ -0.18672567514518862
+ -0.06950765283326864
+ -0.28123708084784266
+ -0.12665848284693404
+ -0.1382024418769914
+  0.17528665552673206
+ -0.015448951451204404
+  0.15478177087438777
+
+
+
+
m1.β                  # alternative extractor
+
+
20-element Vector{Float64}:
+ -0.013323098599731003
+  0.008838610043037353
+  0.009150200653972363
+  0.0033786745702838548
+ -0.038294149701005455
+  0.19558778043000458
+ -0.4287431844207055
+  0.2897784524160024
+ -0.07844856274209384
+ -0.06309657343028045
+  0.4713127757906535
+  0.37559520558051235
+ -0.18672567514518862
+ -0.06950765283326864
+ -0.28123708084784266
+ -0.12665848284693404
+ -0.1382024418769914
+  0.17528665552673206
+ -0.015448951451204404
+  0.15478177087438777
+
+
+
+
fixefnames(m1)        # works also for coefnames(m1)
+
+
20-element Vector{String}:
+ "(Intercept)"
+ "Test: Star_r"
+ "Test: S20_r"
+ "Test: SLJ"
+ "Test: BMT"
+ "a1"
+ "Sex: Girls"
+ "Test: Star_r & a1"
+ "Test: S20_r & a1"
+ "Test: SLJ & a1"
+ "Test: BMT & a1"
+ "Test: Star_r & Sex: Girls"
+ "Test: S20_r & Sex: Girls"
+ "Test: SLJ & Sex: Girls"
+ "Test: BMT & Sex: Girls"
+ "a1 & Sex: Girls"
+ "Test: Star_r & a1 & Sex: Girls"
+ "Test: S20_r & a1 & Sex: Girls"
+ "Test: SLJ & a1 & Sex: Girls"
+ "Test: BMT & a1 & Sex: Girls"
+
+
+
+
vcov(m1)   # StatsBase.vcov: var-cov matrix of fixed-effects coefficients
+
+
20×20 Matrix{Float64}:
+  0.000368507   2.44342e-5   -1.70167e-5   …   3.88654e-6    2.86561e-6
+  2.44342e-5    0.00140405   -0.000421338     -9.36638e-7   -1.02894e-7
+ -1.70167e-5   -0.000421338   0.000856925      2.05905e-5   -6.93885e-7
+ -2.70244e-5    5.45269e-5   -0.000468523     -2.44479e-5    1.37177e-6
+ -0.000143181   3.24318e-5   -7.84036e-6       1.42252e-6   -1.19388e-5
+ -4.45756e-5   -7.41319e-5    2.71349e-5   …  -6.00773e-5   -5.10063e-5
+ -4.42921e-5    6.31848e-5    4.36702e-5      -7.38387e-5    0.000282261
+ -2.72845e-5   -0.000442502   0.000212823      8.7135e-6     2.60536e-6
+  3.37114e-6    0.000213207  -0.000395339     -0.000104421   8.19859e-6
+ -2.00104e-5    9.27317e-8    0.000175292      0.000158974  -6.11699e-5
+  7.30239e-5   -2.67159e-5    2.85901e-5   …  -5.78764e-5    8.27668e-5
+  6.87754e-6    8.83986e-7   -2.07244e-6       8.74402e-6   -0.000108167
+  5.13701e-6   -1.97254e-6    7.70952e-6       0.000673632   0.000118037
+ -5.21248e-6   -1.56374e-6   -4.65457e-6      -0.00136386    0.000576685
+ -1.1505e-6     6.37318e-7   -9.09835e-7       0.000577351  -0.00171455
+ -4.53729e-6   -1.20613e-5    3.47115e-7   …   0.000152123  -0.00391934
+ -1.27935e-5   -2.83726e-5    2.85735e-5      -0.000140172   0.00179133
+  4.7094e-7     2.86411e-5   -4.71121e-5      -0.0115791    -0.00195509
+  3.88654e-6   -9.36638e-7    2.05905e-5       0.0234033    -0.00998405
+  2.86561e-6   -1.02894e-7   -6.93885e-7      -0.00998405    0.0289536
+
+
+
+
vcov(m1; corr=true) # StatsBase.vcov: correlation matrix of fixed-effects coefficients
+
+
20×20 Matrix{Float64}:
+  1.0           0.0339691    -0.0302817    …   0.00132343    0.000877288
+  0.0339691     1.0          -0.384121        -0.000163396  -1.61379e-5
+ -0.0302817    -0.384121      1.0              0.00459787   -0.000139304
+ -0.0470023     0.0485853    -0.534375        -0.00533568    0.000269164
+ -0.252202      0.0292662    -0.00905631       0.000314419  -0.00237244
+ -0.0426429    -0.0363317     0.0170228    …  -0.00721181   -0.00550484
+ -0.0739925     0.054076      0.0478407       -0.0154785     0.0531965
+ -0.0160155    -0.133067      0.0819207        0.000641801   0.000172529
+  0.00214924    0.0696369    -0.165283        -0.00835373    0.000589681
+ -0.0135576     3.21875e-5    0.0778828        0.0135157    -0.0046756
+  0.0446741    -0.00837319    0.0114699    …  -0.004443      0.0057124
+  0.00701347    0.000461824  -0.0013859        0.00111891   -0.0124442
+  0.00563274   -0.00110807    0.00554355       0.0926864     0.0146016
+ -0.00609965   -0.000937468  -0.00357183      -0.200269      0.0761325
+ -0.00121025    0.00034346   -0.000627628      0.0762102    -0.203475
+ -0.00225475   -0.00307062    0.000113116  …   0.00948593   -0.219728
+ -0.00379571   -0.00431254    0.00555926      -0.00521853    0.0599583
+  0.000150198   0.00467971   -0.00985332      -0.463402     -0.0703454
+  0.00132343   -0.000163396   0.00459787       1.0          -0.383546
+  0.000877288  -1.61379e-5   -0.000139304     -0.383546      1.0
+
+
+
+
stderror(m1)       # StatsBase.stderror: SE for fixed-effects coefficients
+
+
20-element Vector{Float64}:
+ 0.01919652732574372
+ 0.03747068322761317
+ 0.029273285330210397
+ 0.02995116114500642
+ 0.029574196102221305
+ 0.05445374107486756
+ 0.031182881518992782
+ 0.08874709312691241
+ 0.08170907406598975
+ 0.0768863280893603
+ 0.08515035859926307
+ 0.051083135211708375
+ 0.047508147450545
+ 0.04451607577961671
+ 0.049520893473334576
+ 0.10482774123584619
+ 0.17557988961760165
+ 0.16333492470289673
+ 0.15298128937140495
+ 0.1701576435993861
+
+
+
+
propertynames(m1)  # names of available extractors
+
+
(:formula, :reterms, :Xymat, :feterm, :sqrtwts, :parmap, :dims, :A, :L, :optsum, :θ, :theta, :β, :beta, :βs, :betas, :λ, :lambda, :stderror, :σ, :sigma, :σs, :sigmas, :σρs, :sigmarhos, :b, :u, :lowerbd, :X, :y, :corr, :vcov, :PCA, :rePCA, :objective, :pvalues)
+
+
+
+
+

6.3 Covariance parameter estimates

+

These commands inform us about the model parameters associated with the RES.

+
+ julia> issingular(m1)        # Test singularity for param. vector m1.theta
++ julia> VarCorr(m1)           # MixedModels.VarCorr: est. of RES
++ julia> propertynames(m1)
++ julia> m1.σ                  # residual; or: m1.sigma
++ julia> m1.σs                 # VCs; m1.sigmas
++ julia> m1.θ                  # Parameter vector for RES (w/o residual); m1.theta
++ julia> MixedModels.sdest(m1) #  prsqrt(MixedModels.varest(m1))
++ julia> BlockDescription(m1)  #  Description of blocks of A and L in an LMM
+
+
issingular(m1) # Test if model is singular for parameter vector m1.theta (default)
+
+
true
+
+
+
+
issingular(m2)
+
+
true
+
+
+
+
VarCorr(m1) # MixedModels.VarCorr: estimates of random-effect structure (RES)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Child(Intercept)0.344952710.58732675
Test: Star_r0.552697630.74343636+0.14
Test: S20_r0.402125110.63413336+0.00-0.52
Test: SLJ0.293855970.54208484+0.05-0.04-0.35
Test: BMT0.520321980.72133347-0.35+0.14-0.18-0.22
School(Intercept)0.045707020.21379201
Test: Star_r0.110768690.33281931-0.06
Test: S20_r0.107610180.32803990-0.08-0.39
Test: SLJ0.103188430.32122956-0.19+0.21-0.81
Test: BMT0.097951810.31297254-0.33-0.02+0.13-0.38
a10.079825650.28253433+0.62-0.20+0.11-0.61+0.45
Sex: Girls0.020591800.14349843-0.45+0.45+0.31-0.27-0.15-0.37
Cohort(Intercept)0.000174410.01320652
Test: Star_r0.004065490.06376120.
Test: S20_r0.000057020.00755108..
Test: SLJ0.001149590.03390556...
Test: BMT0.000000000.00000000....
Residual0.284225220.53312777
+
+
+
+
VarCorr(m2)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
ChildTest: Run0.56222900.7498193
Test: Star_r0.63508990.7969253+0.50
Test: S20_r0.58907960.7675152+0.56+0.64
Test: SLJ0.60893460.7803426+0.55+0.60+0.72
Test: BMT0.48543980.6967351+0.19+0.40+0.37+0.49
SchoolTest: Run0.13704550.3701965
Test: Star_r0.09627830.3102874+0.53
Test: S20_r0.10923830.3305122+0.47+0.48
Test: SLJ0.06138380.2477575+0.47+0.76+0.41
Test: BMT0.03875410.1968606+0.15+0.38+0.18+0.02
a10.08023360.2832554+0.58+0.47+0.56-0.05+0.65
Sex: Girls0.02055460.1433686-0.63-0.27+0.06-0.28-0.58-0.37
CohortTest: Run0.00300660.0548320
Test: Star_r0.00043440.0208416.
Test: S20_r0.00317160.0563170..
Test: SLJ0.00024480.0156459...
Test: BMT0.00000000.0000000....
Residual0.26231270.5121647
+
+
+
+
m1.σs      # VCs; m1.sigmas
+
+
(Child = (var"(Intercept)" = 0.58732674709639, var"Test: Star_r" = 0.7434363644610155, var"Test: S20_r" = 0.6341333569851855, var"Test: SLJ" = 0.542084839663831, var"Test: BMT" = 0.7213334730280074), School = (var"(Intercept)" = 0.21379201285323374, var"Test: Star_r" = 0.3328193108027132, var"Test: S20_r" = 0.3280398999015345, var"Test: SLJ" = 0.32122955990402635, var"Test: BMT" = 0.31297254325848156, a1 = 0.2825343257018283, var"Sex: Girls" = 0.14349842884369796), Cohort = (var"(Intercept)" = 0.013206524133973831, var"Test: Star_r" = 0.06376119875100951, var"Test: S20_r" = 0.0075510776232103555, var"Test: SLJ" = 0.033905564967234667, var"Test: BMT" = 0.0))
+
+
+
+
m1.θ       # Parameter vector for RES (w/o residual); m1.theta
+
+
48-element Vector{Float64}:
+  1.1016622708905508
+  0.199835310203799
+  0.0007687193071826845
+  0.04921136459080601
+ -0.4672660515500969
+  1.3800877425160862
+ -0.6269901567906759
+ -0.04925944187798209
+  0.25246320095240976
+  1.0107889422013527
+  ⋮
+ -0.1650040625391254
+  0.017653103520679392
+ -0.005432524642045863
+  0.0
+  0.024771780682443064
+  0.11959834514264477
+  0.014163729752105945
+  0.06359744704679811
+  0.0
+
+
+
+
BlockDescription(m1) #  Description of blocks of A and L in a LinearMixedModel
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
rowsChildSchoolCohortfixed
9930BlkDiag
3101SparseBlkDiag
45DenseDenseBlkDiag/Dense
21DenseDenseDenseDense
+
+
+
+
m2.θ
+
+
48-element Vector{Float64}:
+  1.464019911728255
+  0.7822363274753275
+  0.842409314371142
+  0.8433757455902053
+  0.26413446683350555
+  1.3450740553826162
+  0.6154158471181771
+  0.5681284803530833
+  0.48038263080030397
+  1.075790626516376
+  ⋮
+ -0.18622672591150657
+  0.0013289707983532247
+ -6.60220078341385e-5
+  0.0
+  0.1070593783172049
+  0.040693237170298845
+  0.1099588509456823
+  0.030548645240017568
+  0.0
+
+
+
+
BlockDescription(m2)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
rowsChildSchoolCohortfixed
9930BlkDiag
3101SparseBlkDiag
45DenseDenseBlkDiag/Dense
21DenseDenseDenseDense
+
+
+
+
+

6.4 Model “predictions”

+

These commands inform us about extracion of conditional modes/means and (co-)variances, that using the model parameters to improve the predictions for units (levels) of the grouping (random) factors. We need this information, e.g., for partial-effect response profiles (e.g., facet plot) or effect profiles (e.g., caterpillar plot), or visualizing the borrowing-strength effect for correlation parameters (e.g., shrinkage plots). We are using the fit of LMM m2.

+
julia> condVar(m2)
+

Some plotting functions are currently available from the MixedModelsMakie package or via custom functions.

+
+ julia> caterpillar!(m2)
++ julia> shrinkage!(m2)
+
+

6.4.1 Conditional covariances

+
+
condVar(m1)
+
+
3-element Vector{Array{Float64, 3}}:
+ [0.058517014101499745 0.009125061044062778 … 0.003278708654799139 -0.02203512152066885; 0.009125061044062778 0.2881484213953011 … -0.00858828480580812 0.02637014650433069; … ; 0.003278708654799139 -0.00858828480580812 … 0.1948955299646874 -0.07616988873567451; -0.02203512152066885 0.02637014650433069 … -0.07616988873567451 0.25747094954024247;;; 0.06479408146453051 0.011134998138892753 … 0.003908367629588989 -0.028871049730723873; 0.011134998138892753 0.29633356972928204 … -0.007311382249300798 0.027042641223018752; … ; 0.003908367629588989 -0.007311382249300798 … 0.19801398892464123 -0.07674628588169728; -0.028871049730723873 0.027042641223018752 … -0.07674628588169728 0.2663833521833813;;; 0.06199490961166209 0.010378470648666283 … 0.0033211887933385473 -0.025799744230828064; 0.010378470648666283 0.2919637022019177 … -0.008099599279587298 0.02645587001375013; … ; 0.0033211887933385473 -0.008099599279587298 … 0.19709146464205915 -0.07626677986337765; -0.025799744230828064 0.02645587001375013 … -0.07626677986337765 0.26226810420768637;;; … ;;; 0.09251275858088985 0.011430464054488698 … -0.004372759710309024 -0.025512877879956514; 0.011430464054488698 0.29433912821382646 … -0.008626529501006942 0.02760443787517547; … ; -0.004372759710309024 -0.008626529501006942 … 0.196293768010921 -0.07428957822771329; -0.025512877879956514 0.02760443787517547 … -0.07428957822771329 0.26236636750253356;;; 0.09874036143268264 0.009185552998062238 … -0.0001573075613284164 -0.02462263824544809; 0.009185552998062238 0.2911679711936218 … -0.008776482510154664 0.027936646374801047; … ; -0.0001573075613284164 -0.008776482510154664 … 0.19584727323067816 -0.07501977375670275; -0.02462263824544809 0.027936646374801047 … -0.07501977375670275 0.2596118385454262;;; 0.08569687177833775 0.008946686416428581 … 0.0006592569895041175 -0.02086375290989933; 0.008946686416428581 0.28863042124474664 … -0.008936779278627875 0.02745669612294799; … ; 0.0006592569895041175 -0.008936779278627875 … 0.1948682013379163 -0.07525256402491896; -0.02086375290989933 0.02745669612294799 … -0.07525256402491896 0.2567755880517776]
+ [0.03738662222206967 -0.0018006453467180224 … 0.029918884081614444 -0.01241043132709623; -0.0018006453467180224 0.09043180671160218 … -0.012288226712182382 0.017349422342826458; … ; 0.029918884081614444 -0.012288226712182382 … 0.06552240047084834 -0.012696720591960402; -0.01241043132709623 0.017349422342826458 … -0.012696720591960402 0.018266200494610186;;; 0.032175235508378135 0.0002020231146900986 … 0.026239700886540457 -0.010631094016241854; 0.0002020231146900986 0.07633547772742386 … -0.0076489885496848755 0.014171038510892612; … ; 0.026239700886540457 -0.0076489885496848755 … 0.057371204435359965 -0.010702426636433508; -0.010631094016241854 0.014171038510892612 … -0.010702426636433508 0.015840463462564926;;; 0.0293211217872406 -0.0031722468514524776 … 0.021508439538618022 -0.010442275909060876; -0.0031722468514524776 0.08771293893576042 … -0.013169827546768145 0.01806934628580798; … ; 0.021508439538618022 -0.013169827546768145 … 0.051822149216928896 -0.010311386841756282; -0.010442275909060876 0.01806934628580798 … -0.010311386841756282 0.016719819496053525;;; … ;;; 0.0267612826186117 0.002887196156566762 … 0.019017122735360242 -0.006845576372521417; 0.002887196156566762 0.07608301622502864 … -0.005537174830550229 0.012875566802524871; … ; 0.019017122735360242 -0.005537174830550229 … 0.050902196361920206 -0.006078576026170587; -0.006845576372521417 0.012875566802524871 … -0.006078576026170587 0.013661101965124827;;; 0.027228645015369164 0.003500454962860299 … 0.020300439683147174 -0.006150803022415798; 0.003500454962860299 0.07600122149129579 … -0.005010775855234807 0.012574439977365934; … ; 0.020300439683147174 -0.005010775855234807 … 0.05302863567353013 -0.00626554725377429; -0.006150803022415798 0.012574439977365934 … -0.00626554725377429 0.013684452210043274;;; 0.043339821193180326 -0.0030818917690474392 … 0.03637959916087954 -0.01363338241537408; -0.0030818917690474392 0.0993577864113983 … -0.015052524931708725 0.019104775094131096; … ; 0.03637959916087954 -0.015052524931708725 … 0.07506039488224463 -0.014302079998948012; -0.01363338241537408 0.019104775094131096 … -0.014302079998948012 0.019440414377459254]
+ [0.00016380655132630802 6.905263271054442e-6 … -4.776505132880116e-6 0.0; 6.905263271054442e-6 0.002357685591177613 … -0.0001535595839979168 0.0; … ; -4.776505132880116e-6 -0.0001535595839979168 … 0.0008635382997135062 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00016040198449826642 8.762829702597266e-6 … -6.068571712892409e-6 0.0; 8.762829702597266e-6 0.002044995779766137 … -0.0001672018085446938 0.0; … ; -6.068571712892409e-6 -0.0001672018085446938 … 0.000795065708027814 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00016236819064567072 7.626721927893261e-6 … -5.190387582391803e-6 0.0; 7.626721927893261e-6 0.002242186063452188 … -0.00015894848127273218 0.0; … ; -5.190387582391803e-6 -0.00015894848127273218 … 0.0008353101094087038 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00016175490481618921 8.411278134019113e-6 … -5.514796437067044e-6 0.0; 8.411278134019113e-6 0.0021643858461830056 … -0.00016336647221245931 0.0; … ; -5.514796437067044e-6 -0.00016336647221245931 … 0.0008232493551568546 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00015933306329354854 8.683518670335156e-6 … -6.949857071192867e-6 0.0; 8.683518670335156e-6 0.001962786328235615 … -0.0001688729211888127 0.0; … ; -6.949857071192867e-6 -0.0001688729211888127 … 0.0007737698549017665 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0001585591247886362 7.442199159884006e-6 … -7.387837364196275e-6 0.0; 7.442199159884006e-6 0.0019342658473775333 … -0.0001675218103327855 0.0; … ; -7.387837364196275e-6 -0.0001675218103327855 … 0.0007566342723566994 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00016065497306304335 8.005529319817548e-6 … -6.461140807188436e-6 0.0; 8.005529319817548e-6 0.0020775998050236873 … -0.00016680712988184607 0.0; … ; -6.461140807188436e-6 -0.00016680712988184607 … 0.0007993999850391486 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00015831595367890705 8.46712980408871e-6 … -7.964470854652428e-6 0.0; 8.46712980408871e-6 0.0018894476583808555 … -0.00017251367903727097 0.0; … ; -7.964470854652428e-6 -0.00017251367903727097 … 0.0007531486757886536 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00015649962194566806 9.755584275305604e-6 … -7.749920208393922e-6 0.0; 9.755584275305604e-6 0.0017892434008092122 … -0.00017175668964200161 0.0; … ; -7.749920208393922e-6 -0.00017175668964200161 … 0.0007269191158251669 0.0; 0.0 0.0 … 0.0 0.0]
+
+
+
+
condVar(m2)
+
+
3-element Vector{Array{Float64, 3}}:
+ [0.17121835522819978 0.02755529331418949 … 0.03416706723888015 -0.00724754030926216; 0.02755529331418949 0.17389268509524453 … 0.03753020675650032 0.0188623774335696; … ; 0.03416706723888015 0.03753020675650032 … 0.16031637538660184 0.028820239120887613; -0.00724754030926216 0.0188623774335696 … 0.028820239120887613 0.160864500973238;;; 0.18242691671642797 0.0343250931186835 … 0.04147060082369551 -0.009143227675242742; 0.0343250931186835 0.18567343287578988 … 0.04866689163492084 0.021405410618718965; … ; 0.04147060082369551 0.04866689163492084 … 0.17266718224767022 0.03123733241019097; -0.009143227675242742 0.021405410618718965 … 0.03123733241019097 0.16331432606425472;;; 0.1762667317205972 0.0311279231692659 … 0.038186553972383325 -0.007994491834495502; 0.0311279231692659 0.18039096830847456 … 0.04354026417410434 0.02014451692076348; … ; 0.038186553972383325 0.04354026417410434 … 0.1670059987629768 0.030077380180769695; -0.007994491834495502 0.02014451692076348 … 0.030077380180769695 0.16204289027390376;;; … ;;; 0.21741247314653891 0.07093054334800777 … 0.06747150120313818 0.02145098713166718; 0.07093054334800777 0.22154962297484282 … 0.07284423050626627 0.05096352617049405; … ; 0.06747150120313818 0.07284423050626627 … 0.18375387349238714 0.05065449550765889; 0.02145098713166718 0.05096352617049405 … 0.05065449550765889 0.18636447571570958;;; 0.22425843774021273 0.07771036649190435 … 0.07692945052170154 0.032146495017884365; 0.07771036649190435 0.22464983667288474 … 0.07893678447646196 0.058693580990441856; … ; 0.07692945052170154 0.07893678447646196 … 0.192417812551263 0.06096244826047057; 0.032146495017884365 0.058693580990441856 … 0.06096244826047057 0.1952055411652199;;; 0.20613389930796294 0.0614937415744954 … 0.062270378347748845 0.02206740872792677; 0.0614937415744954 0.20740253938205164 … 0.06405704423364168 0.047854049528350144; … ; 0.062270378347748845 0.06405704423364168 … 0.179269076031725 0.0517136334805597; 0.02206740872792677 0.047854049528350144 … 0.0517136334805597 0.18687984024155918]
+ [0.10768671208086496 0.04839952818075811 … 0.04766653444192149 -0.027767847991630235; 0.04839952818075811 0.07949753046909964 … 0.03493825637829439 -0.01030835164217005; … ; 0.04766653444192149 0.03493825637829439 … 0.06574810342307015 -0.01275585556178704; -0.027767847991630235 -0.01030835164217005 … -0.01275585556178704 0.018227082481554094;;; 0.08790407612332059 0.040393261282076404 … 0.0400599319562759 -0.02269489800607808; 0.040393261282076404 0.06914721694875275 … 0.03206702198915689 -0.008425353467929443; … ; 0.0400599319562759 0.03206702198915689 … 0.05749755077922079 -0.010761924798630933; -0.02269489800607808 -0.008425353467929443 … -0.010761924798630933 0.015807162176876503;;; 0.09636492785218488 0.035884742431221237 … 0.038333680468635785 -0.025416695484362636; 0.035884742431221237 0.06308321756031697 … 0.02476724723597186 -0.007242081284198682; … ; 0.038333680468635785 0.02476724723597186 … 0.05198552853377316 -0.010391082211543702; -0.025416695484362636 -0.007242081284198682 … -0.010391082211543702 0.01668905433983563;;; … ;;; 0.07441056179847254 0.0305275873520764 … 0.029374260939504657 -0.017029674955578736; 0.0305275873520764 0.06267297540942841 … 0.02349135736015709 -0.004060003964855782; … ; 0.029374260939504657 0.02349135736015709 … 0.05102700756299053 -0.006132825015515446; -0.017029674955578736 -0.004060003964855782 … -0.006132825015515446 0.013633872415800339;;; 0.07356358901088007 0.030536524186054008 … 0.0301082996724446 -0.016067883424430905; 0.030536524186054008 0.06346336276661714 … 0.024768348896849156 -0.0033951944729095373; … ; 0.0301082996724446 0.024768348896849156 … 0.053147386583431176 -0.006319089569004934; -0.016067883424430905 -0.0033951944729095373 … -0.006319089569004934 0.013656675423188653;;; 0.12473526100613286 0.05796391233780463 … 0.05756242970998203 -0.031000768758675107; 0.05796391233780463 0.09050849232950609 … 0.042023372901318 -0.011779510213246015; … ; 0.05756242970998203 0.042023372901318 … 0.07536073523942259 -0.014362415085092188; -0.031000768758675107 -0.011779510213246015 … -0.014362415085092188 0.01940248264853897]
+ [0.0018738580389070167 3.097002198457398e-5 … 2.2076784503534793e-5 0.0; 3.097002198457398e-5 0.00039802187403120584 … 5.648497477795947e-6 0.0; … ; 2.2076784503534793e-5 5.648497477795947e-6 … 0.00023124160140848118 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0016450448619856558 3.729737750101121e-5 … 2.7027760485222434e-5 0.0; 3.729737750101121e-5 0.0003867402285022758 … 7.734284697291955e-6 0.0; … ; 2.7027760485222434e-5 7.734284697291955e-6 … 0.00022686693600509704 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0017882787826426782 3.243821501424111e-5 … 2.4047614324889707e-5 0.0; 3.243821501424111e-5 0.00039393409516233735 … 6.455070883862393e-6 0.0; … ; 2.4047614324889707e-5 6.455070883862393e-6 … 0.00022946203827014857 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0017313946330208606 3.4368248113981814e-5 … 2.50561873272665e-5 0.0; 3.4368248113981814e-5 0.0003918409974798727 … 6.670472162105535e-6 0.0; … ; 2.50561873272665e-5 6.670472162105535e-6 … 0.0002288103664780839 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0015868478954532295 3.8887650632968376e-5 … 2.7809025410647586e-5 0.0; 3.8887650632968376e-5 0.00038367187497630265 … 8.087941724439915e-6 0.0; … ; 2.7809025410647586e-5 8.087941724439915e-6 … 0.00022546408538126053 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0015688425920921166 4.000704168244891e-5 … 2.8473059101798132e-5 0.0; 4.000704168244891e-5 0.00038022839783514767 … 8.897931443372262e-6 0.0; … ; 2.8473059101798132e-5 8.897931443372262e-6 … 0.00022425219715871693 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0016705183562560773 3.6934723422390205e-5 … 2.6047147905636665e-5 0.0; 3.6934723422390205e-5 0.0003871521092119287 … 7.488540995030283e-6 0.0; … ; 2.6047147905636665e-5 7.488540995030283e-6 … 0.00022711057986501076 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0015336542451690207 4.090147164649518e-5 … 2.9024635466401744e-5 0.0; 4.090147164649518e-5 0.00038008695637156323 … 8.800302599711073e-6 0.0; … ; 2.9024635466401744e-5 8.800302599711073e-6 … 0.00022397611621172725 0.0; 0.0 0.0 … 0.0 0.0;;; 0.001453691844155124 4.285465800881136e-5 … 3.0624467338295236e-5 0.0; 4.285465800881136e-5 0.00037403984354553936 … 9.967957192788682e-6 0.0; … ; 3.0624467338295236e-5 9.967957192788682e-6 … 0.00022206826601886315 0.0; 0.0 0.0 … 0.0 0.0]
+
+
+

They are hard to look at. Let’s take pictures.

+
+
+

6.4.2 Caterpillar plots

+
+
+Code +
caterpillar!(
+  Figure(; resolution=(800, 400)), ranefinfo(m1, :Cohort)
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 3: Prediction intervals of the random effects for Cohort in model m1 +
+
+
+
+
+
+
+

6.4.3 Shrinkage plots

+

These are just teasers. We will pick this up in a separate tutorial. Enjoy!

+
+
+Code +
shrinkageplot!(Figure(; resolution=(800, 800)), m1, :Cohort)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 4: Shrinkage plot of the random effects for Cohort in model m1 +
+
+
+
+
+
+
+Code +
shrinkageplot!(Figure(; resolution=(800, 800)), m2, :Cohort)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 5: Shrinkage plot of the random effects for Cohort in model m2 +
+
+
+
+
+
+
+Fühner, T., Granacher, U., Golle, K., & Kliegl, R. (2021). Age and sex effects in physical fitness components of 108,295 third graders including 515 primary schools and 9 cohorts. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-97000-4 +
+
+Schielzeth, H., & Forstmeier, W. (2008). Conclusions beyond support: Overconfident estimates in mixed models. Behavioral Ecology, 20(2), 416–420. https://doi.org/10.1093/beheco/arn145 +
+
+ + +
+
+
+ + Back to top
+ +
+ + + + + \ No newline at end of file diff --git a/fggk21_files/figure-html/fig-agetrends-output-1.svg b/fggk21_files/figure-html/fig-agetrends-output-1.svg new file mode 100644 index 0000000..8c5a93f --- /dev/null +++ b/fggk21_files/figure-html/fig-agetrends-output-1.svg @@ -0,0 +1,825 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/fggk21_files/figure-html/fig-agetrendssamp-output-1.svg b/fggk21_files/figure-html/fig-agetrendssamp-output-1.svg new file mode 100644 index 0000000..95ca71c --- /dev/null +++ b/fggk21_files/figure-html/fig-agetrendssamp-output-1.svg @@ -0,0 +1,816 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/fggk21_files/figure-html/fig-caterpillarcohort-output-2.svg b/fggk21_files/figure-html/fig-caterpillarcohort-output-2.svg new file mode 100644 index 0000000..b1e05b7 --- /dev/null +++ b/fggk21_files/figure-html/fig-caterpillarcohort-output-2.svg @@ -0,0 +1,1029 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/fggk21_files/figure-html/fig-m1shrinkagecohort-output-2.svg b/fggk21_files/figure-html/fig-m1shrinkagecohort-output-2.svg new file mode 100644 index 0000000..56383f2 --- /dev/null +++ b/fggk21_files/figure-html/fig-m1shrinkagecohort-output-2.svg @@ -0,0 +1,1433 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/fggk21_files/figure-html/fig-m2shrinkagecohort-output-2.svg b/fggk21_files/figure-html/fig-m2shrinkagecohort-output-2.svg new file mode 100644 index 0000000..9f42997 --- /dev/null +++ b/fggk21_files/figure-html/fig-m2shrinkagecohort-output-2.svg @@ -0,0 +1,1486 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/glmm.html b/glmm.html new file mode 100644 index 0000000..a98f2ec --- /dev/null +++ b/glmm.html @@ -0,0 +1,2603 @@ + + + + + + + + + + + +Generalized linear mixed models – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Generalized linear mixed models

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

Load the packages to be used

+
+
+Code +
using AlgebraOfGraphics
+using CairoMakie
+using DataFrameMacros
+using DataFrames
+using MixedModels
+using MixedModelsMakie
+using SMLP2024: dataset
+
+CairoMakie.activate!(; type="svg")
+
+import ProgressMeter
+ProgressMeter.ijulia_behavior(:clear)
+
+
+ +
+

1 Matrix notation for the sleepstudy model

+
+
sleepstudy = DataFrame(dataset(:sleepstudy))
+
+
180×3 DataFrame
155 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowsubjdaysreaction
StringInt8Float64
1S3080249.56
2S3081258.705
3S3082250.801
4S3083321.44
5S3084356.852
6S3085414.69
7S3086382.204
8S3087290.149
9S3088430.585
10S3089466.353
11S3090222.734
12S3091205.266
13S3092202.978
⋮⋮⋮⋮
169S3718350.781
170S3719369.469
171S3720269.412
172S3721273.474
173S3722297.597
174S3723310.632
175S3724287.173
176S3725329.608
177S3726334.482
178S3727343.22
179S3728369.142
180S3729364.124
+
+
+
+
+
contrasts = Dict(:subj => Grouping())
+m1 = let f = @formula reaction ~ 1 + days + (1 + days | subj)
+  fit(MixedModel, f, sleepstudy; contrasts)
+end
+println(m1)
+
+
Linear mixed model fit by maximum likelihood
+ reaction ~ 1 + days + (1 + days | subj)
+   logLik   -2 logLik     AIC       AICc        BIC    
+  -875.9697  1751.9393  1763.9393  1764.4249  1783.0971
+
+Variance components:
+            Column    Variance Std.Dev.   Corr.
+subj     (Intercept)  565.51067 23.78047
+         days          32.68212  5.71683 +0.08
+Residual              654.94145 25.59182
+ Number of obs: 180; levels of grouping factors: 18
+
+  Fixed-effects parameters:
+──────────────────────────────────────────────────
+                Coef.  Std. Error      z  Pr(>|z|)
+──────────────────────────────────────────────────
+(Intercept)  251.405      6.63226  37.91    <1e-99
+days          10.4673     1.50224   6.97    <1e-11
+──────────────────────────────────────────────────
+
+
+

The response vector, y, has 180 elements. The fixed-effects coefficient vector, β, has 2 elements and the fixed-effects model matrix, X, is of size 180 × 2.

+
+
m1.y
+
+
180-element view(::Matrix{Float64}, :, 3) with eltype Float64:
+ 249.56
+ 258.7047
+ 250.8006
+ 321.4398
+ 356.8519
+ 414.6901
+ 382.2038
+ 290.1486
+ 430.5853
+ 466.3535
+   ⋮
+ 273.474
+ 297.5968
+ 310.6316
+ 287.1726
+ 329.6076
+ 334.4818
+ 343.2199
+ 369.1417
+ 364.1236
+
+
+
+
m1.β
+
+
2-element Vector{Float64}:
+ 251.4051048484851
+  10.4672859595964
+
+
+
+
m1.X
+
+
180×2 Matrix{Float64}:
+ 1.0  0.0
+ 1.0  1.0
+ 1.0  2.0
+ 1.0  3.0
+ 1.0  4.0
+ 1.0  5.0
+ 1.0  6.0
+ 1.0  7.0
+ 1.0  8.0
+ 1.0  9.0
+ ⋮    
+ 1.0  1.0
+ 1.0  2.0
+ 1.0  3.0
+ 1.0  4.0
+ 1.0  5.0
+ 1.0  6.0
+ 1.0  7.0
+ 1.0  8.0
+ 1.0  9.0
+
+
+

The second column of X is just the days vector and the first column is all 1’s.

+

There are 36 random effects, 2 for each of the 18 levels of subj. The “estimates” (technically, the conditional means or conditional modes) are returned as a vector of matrices, one matrix for each grouping factor. In this case there is only one grouping factor for the random effects so there is one one matrix which contains 18 intercept random effects and 18 slope random effects.

+
+
m1.b
+
+
1-element Vector{Matrix{Float64}}:
+ [2.815819244491056 -40.04844174471745 … 0.723262023911267 12.118907794830154; 9.075511691265579 -8.644079439241482 … -0.9710526311820062 1.3106980688400298]
+
+
+
+
only(m1.b)   # only one grouping factor
+
+
2×18 Matrix{Float64}:
+ 2.81582  -40.0484   -38.4331  22.8321   …  -24.7101   0.723262  12.1189
+ 9.07551   -8.64408   -5.5134  -4.65872       4.6597  -0.971053   1.3107
+
+
+

There is a model matrix, Z, for the random effects. In general it has one chunk of columns for the first grouping factor, a chunk of columns for the second grouping factor, etc.

+

In this case there is only one grouping factor.

+
+
Int.(first(m1.reterms))
+
+
180×36 Matrix{Int64}:
+ 1  0  0  0  0  0  0  0  0  0  0  0  0  …  0  0  0  0  0  0  0  0  0  0  0  0
+ 1  1  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0
+ 1  2  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0
+ 1  3  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0
+ 1  4  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0
+ 1  5  0  0  0  0  0  0  0  0  0  0  0  …  0  0  0  0  0  0  0  0  0  0  0  0
+ 1  6  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0
+ 1  7  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0
+ 1  8  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0
+ 1  9  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  0  0
+ ⋮              ⋮              ⋮        ⋱     ⋮              ⋮              ⋮
+ 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  1  1
+ 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  1  2
+ 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  1  3
+ 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  1  4
+ 0  0  0  0  0  0  0  0  0  0  0  0  0  …  0  0  0  0  0  0  0  0  0  0  1  5
+ 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  1  6
+ 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  1  7
+ 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  1  8
+ 0  0  0  0  0  0  0  0  0  0  0  0  0     0  0  0  0  0  0  0  0  0  0  1  9
+
+
+

The defining property of a linear model or linear mixed model is that the fitted values are linear combinations of the fixed-effects parameters and the random effects. We can write the fitted values as

+
+
m1.X * m1.β + only(m1.reterms) * vec(only(m1.b))
+
+
180-element Vector{Float64}:
+ 254.22092409297616
+ 273.7637217438381
+ 293.30651939470016
+ 312.8493170455621
+ 332.3921146964241
+ 351.93491234728606
+ 371.47770999814804
+ 391.02050764901
+ 410.563305299872
+ 430.106102950734
+   ⋮
+ 275.3019966717517
+ 287.0799807001881
+ 298.85796472862455
+ 310.635948757061
+ 322.4139327854974
+ 334.19191681393386
+ 345.96990084237024
+ 357.74788487080673
+ 369.52586889924316
+
+
+
+
fitted(m1)   # just to check that these are indeed the same as calculated above
+
+
180-element Vector{Float64}:
+ 254.22092409297616
+ 273.7637217438382
+ 293.30651939470016
+ 312.84931704556215
+ 332.3921146964241
+ 351.9349123472861
+ 371.47770999814804
+ 391.02050764901
+ 410.563305299872
+ 430.106102950734
+   ⋮
+ 275.3019966717517
+ 287.0799807001881
+ 298.85796472862455
+ 310.63594875706093
+ 322.4139327854974
+ 334.1919168139338
+ 345.96990084237024
+ 357.74788487080673
+ 369.5258688992431
+
+
+

In symbols we would write the linear predictor expression as \[ +\boldsymbol{\eta} = \mathbf{X}\boldsymbol{\beta} +\mathbf{Z b} +\] where \(\boldsymbol{\eta}\) has 180 elements, \(\boldsymbol{\beta}\) has 2 elements, \(\bf b\) has 36 elements, \(\bf X\) is of size 180 × 2 and \(\bf Z\) is of size 180 × 36.

+

For a linear model or linear mixed model the linear predictor is the mean response, \(\boldsymbol\mu\). That is, we can write the probability model in terms of a 180-dimensional random variable, \(\mathcal Y\), for the response and a 36-dimensional random variable, \(\mathcal B\), for the random effects as \[ +\begin{aligned} +(\mathcal{Y} | \mathcal{B}=\bf{b}) &\sim\mathcal{N}(\bf{ X\boldsymbol\beta + Z b},\sigma^2\bf{I})\\\\ +\mathcal{B}&\sim\mathcal{N}(\bf{0},\boldsymbol{\Sigma}_{\boldsymbol\theta}) . +\end{aligned} +\] where \(\boldsymbol{\Sigma}_\boldsymbol{\theta}\) is a 36 × 36 symmetric covariance matrix that has a special form - it consists of 18 diagonal blocks, each of size 2 × 2 and all the same.

+

Recall that this symmetric matrix can be constructed from the parameters \(\boldsymbol\theta\), which generate the lower triangular matrix \(\boldsymbol\lambda\), and the estimate \(\widehat{\sigma^2}\).

+
+
m1.θ
+
+
3-element Vector{Float64}:
+ 0.9292213162629417
+ 0.018168381188609965
+ 0.22264487194620683
+
+
+
+
λ = only(m1.λ)  # with multiple grouping factors there will be multiple λ's
+
+
2×2 LinearAlgebra.LowerTriangular{Float64, Matrix{Float64}}:
+ 0.929221    ⋅ 
+ 0.0181684  0.222645
+
+
+
+
Σ = varest(m1) * (λ * λ')
+
+
2×2 Matrix{Float64}:
+ 565.511  11.057
+  11.057  32.6821
+
+
+

Compare the diagonal elements to the Variance column of

+
+
VarCorr(m1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
subj(Intercept)565.5106723.78047
days32.682125.71683+0.08
Residual654.9414525.59182
+
+
+
+
+

2 Linear predictors in LMMs and GLMMs

+

Writing the model for \(\mathcal Y\) as \[ +(\mathcal{Y} | \mathcal{B}=\bf{b})\sim\mathcal{N}(\bf{ X\boldsymbol\beta + Z b},\sigma^2\bf{I}) +\] may seem like over-mathematization (or “overkill”, if you prefer) relative to expressions like \[ +y_i = \beta_1 x_{i,1} + \beta_2 x_{i,2}+ b_1 z_{i,1} +\dots+b_{36} z_{i,36}+\epsilon_i +\] but this more abstract form is necessary for generalizations.

+

The way that I read the first form is

+
+
+
+ +
+
+

The conditional distribution of the response vector, \(\mathcal Y\), given that the random effects vector, \(\mathcal B =\bf b\), is a multivariate normal (or Gaussian) distribution whose mean, \(\boldsymbol\mu\), is the linear predictor, \(\boldsymbol\eta=\bf{X\boldsymbol\beta+Zb}\), and whose covariance matrix is \(\sigma^2\bf I\). That is, conditional on \(\bf b\), the elements of \(\mathcal Y\) are independent normal random variables with constant variance, \(\sigma^2\), and means of the form \(\boldsymbol\mu = \boldsymbol\eta = \bf{X\boldsymbol\beta+Zb}\).

+
+
+
+

So the only things that differ in the distributions of the \(y_i\)’s are the means and they are determined by this linear predictor, \(\boldsymbol\eta = \bf{X\boldsymbol\beta+Zb}\).

+
+
+

3 Generalized Linear Mixed Models

+

Consider first a GLMM for a vector, \(\bf y\), of binary (i.e. yes/no) responses. The probability model for the conditional distribution \(\mathcal Y|\mathcal B=\bf b\) consists of independent Bernoulli distributions where the mean, \(\mu_i\), for the i’th response is again determined by the i’th element of a linear predictor, \(\boldsymbol\eta = \mathbf{X}\boldsymbol\beta+\mathbf{Z b}\).

+

However, in this case we will run into trouble if we try to make \(\boldsymbol\mu=\boldsymbol\eta\) because \(\mu_i\) is the probability of “success” for the i’th response and must be between 0 and 1. We can’t guarantee that the i’th component of \(\boldsymbol\eta\) will be between 0 and 1. To get around this problem we apply a transformation to take \(\eta_i\) to \(\mu_i\). For historical reasons this transformation is called the inverse link, written \(g^{-1}\), and the opposite transformation - from the probability scale to an unbounded scale - is called the link, g.

+

Each probability distribution in the exponential family (which is most of the important ones), has a canonical link which comes from the form of the distribution itself. The details aren’t as important as recognizing that the distribution itself determines a preferred link function.

+

For the Bernoulli distribution, the canonical link is the logit or log-odds function, \[ +\eta = g(\mu) = \log\left(\frac{\mu}{1-\mu}\right), +\] (it’s called log-odds because it is the logarithm of the odds ratio, \(p/(1-p)\)) and the canonical inverse link is the logistic \[ +\mu=g^{-1}(\eta)=\frac{1}{1+\exp(-\eta)}. +\] This is why fitting a binary response is sometimes called logistic regression.

+

For later use we define a Julia logistic function. See this presentation for more information than you could possibly want to know on how Julia converts code like this to run on the processor.

+
+
increment(x) = x + one(x)
+logistic(η) = inv(increment(exp(-η)))
+
+
logistic (generic function with 1 method)
+
+
+

To reiterate, the probability model for a Generalized Linear Mixed Model (GLMM) is \[ +\begin{aligned} +(\mathcal{Y} | \mathcal{B}=\bf{b}) &\sim\mathcal{D}(\bf{g^{-1}(X\boldsymbol\beta + Z b)},\phi)\\\\ +\mathcal{B}&\sim\mathcal{N}(\bf{0},\Sigma_{\boldsymbol\theta}) . +\end{aligned} +\] where \(\mathcal{D}\) is the distribution family (such as Bernoulli or Poisson), \(g^{-1}\) is the inverse link and \(\phi\) is a scale parameter for \(\mathcal{D}\) if it has one. The important cases of the Bernoulli and Poisson distributions don’t have a scale parameter - once you know the mean you know everything you need to know about the distribution. (For those following the presentation, this poem by John Keats is the one with the couplet “Beauty is truth, truth beauty - that is all ye know on earth and all ye need to know.”)

+
+

3.1 An example of a Bernoulli GLMM

+

The contra dataset in the MixedModels package is from a survey on the use of artificial contraception by women in Bangladesh.

+
+
contra = DataFrame(dataset(:contra))
+
+
1934×5 DataFrame
1909 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowdisturbanlivchageuse
StringStringStringFloat64String
1D01Y3+18.44N
2D01Y0-5.56N
3D01Y21.44N
4D01Y3+8.44N
5D01Y0-13.56N
6D01Y0-11.56N
7D01Y3+18.44N
8D01Y3+-3.56N
9D01Y1-5.56N
10D01Y3+1.44N
11D01Y0-11.56Y
12D01Y0-2.56N
13D01Y1-4.56N
⋮⋮⋮⋮⋮⋮
1923D61N0-11.56Y
1924D61N3+1.44N
1925D61N1-5.56N
1926D61N3+14.44N
1927D61N3+19.44N
1928D61N2-9.56Y
1929D61N2-2.56N
1930D61N3+14.44N
1931D61N2-4.56N
1932D61N3+14.44N
1933D61N0-13.56N
1934D61N3+10.44N
+
+
+
+
+
combine(groupby(contra, :dist), nrow)
+
+
60×2 DataFrame
35 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowdistnrow
StringInt64
1D01117
2D0220
3D032
4D0430
5D0539
6D0665
7D0718
8D0837
9D0923
10D1013
11D1121
12D1229
13D1324
⋮⋮⋮
49D494
50D5019
51D5137
52D5261
53D5319
54D556
55D5645
56D5727
57D5833
58D5910
59D6032
60D6142
+
+
+
+

The information recorded included woman’s age, the number of live children she has, whether she lives in an urban or rural setting, and the political district in which she lives.

+

The age was centered. Unfortunately, the version of the data to which I had access did not record what the centering value was.

+

A data plot, Figure 1, shows that the probability of contraception use is not linear in age - it is low for younger women, higher for women in the middle of the range (assumed to be women in late 20’s to early 30’s) and low again for older women (late 30’s to early 40’s in this survey).

+

If we fit a model with only the age term in the fixed effects, that term will not be significant. This doesn’t mean that there is no “age effect”, it only means that there is no significant linear effect for age.

+
+
+Code +
draw(
+  data(
+    @transform(
+      contra,
+      :numuse = Int(:use == "Y"),
+      :urb = ifelse(:urban == "Y", "Urban", "Rural")
+    )
+  ) *
+  mapping(
+    :age => "Centered age (yr)",
+    :numuse => "Frequency of contraception use";
+    col=:urb,
+    color=:livch,
+  ) *
+  smooth();
+  figure=(; resolution=(800, 450)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 1: Smoothed relative frequency of contraception use versus centered age for women in the 1989 Bangladesh Fertility Survey +
+
+
+
+
+
+
contrasts = Dict(
+  :dist => Grouping(),
+  :urban => HelmertCoding(),
+  :livch => DummyCoding(), # default, but no harm in being explicit
+)
+nAGQ = 9
+dist = Bernoulli()
+gm1 = let
+  form = @formula(
+    use ~ 1 + age + abs2(age) + urban + livch + (1 | dist)
+  )
+  fit(MixedModel, form, contra, dist; nAGQ, contrasts)
+end
+
+
Minimizing 2    Time: 0:00:00 (91.92 ms/it)
+  objective:  2396.263956678254
+Minimizing 208    Time: 0:00:01 ( 5.00 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_dist
(Intercept)-0.68710.1686-4.08<1e-040.4786
age0.00350.00920.380.7021
abs2(age)-0.00460.0007-6.29<1e-09
urban: Y0.34830.06005.81<1e-08
livch: 10.81520.16225.03<1e-06
livch: 20.91650.18514.95<1e-06
livch: 3+0.91530.18584.93<1e-06
+
+
+

Notice that the linear term for age is not significant but the quadratic term for age is highly significant. We usually retain the lower order term, even if it is not significant, if the higher order term is significant.

+

Notice also that the parameter estimates for the treatment contrasts for livch are similar. Thus the distinction of 1, 2, or 3+ children is not as important as the contrast between having any children and not having any. Those women who already have children are more likely to use artificial contraception.

+

Furthermore, the women without children have a different probability vs age profile than the women with children. To allow for this we define a binary children factor and incorporate an age&children interaction.

+
+
VarCorr(gm1)
+
+ + + + + + + + + + + + + + + +
ColumnVarianceStd.Dev
dist(Intercept)0.229070.47861
+
+
+

Notice that there is no “residual” variance being estimated. This is because the Bernoulli distribution doesn’t have a scale parameter.

+
+
+

3.2 Convert livch to a binary factor

+
+
@transform!(contra, :children = :livch ≠ "0")
+# add the associated contrast specifier
+contrasts[:children] = EffectsCoding()
+
+
EffectsCoding(nothing, nothing)
+
+
+
+
gm2 = let
+  form = @formula(
+    use ~
+      1 +
+      age * children +
+      abs2(age) +
+      children +
+      urban +
+      (1 | dist)
+  )
+  fit(MixedModel, form, contra, dist; nAGQ, contrasts)
+end
+
+
Minimizing 95    Time: 0:00:00 ( 1.06 ms/it)
+  objective:  2364.92044533042
+Minimizing 140    Time: 0:00:00 ( 1.06 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_dist
(Intercept)-0.36150.1275-2.840.00460.4757
age-0.01310.0110-1.190.2351
children: true0.60550.10355.85<1e-08
abs2(age)-0.00580.0008-6.89<1e-11
urban: Y0.35670.06025.93<1e-08
age & children: true0.03420.01272.690.0072
+
+
+
+
+Code +
let
+  mods = [gm2, gm1]
+  DataFrame(;
+    model=[:gm2, :gm1],
+    npar=dof.(mods),
+    deviance=deviance.(mods),
+    AIC=aic.(mods),
+    BIC=bic.(mods),
+    AICc=aicc.(mods),
+  )
+end
+
+
+
2×6 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowmodelnpardevianceAICBICAICc
SymbolInt64Float64Float64Float64Float64
1gm272364.922379.182418.152379.24
2gm182372.462388.732433.272388.81
+
+
+
+

Because these models are not nested, we cannot do a likelihood ratio test. Nevertheless we see that the deviance is much lower in the model with age & children even though the 3 levels of livch have been collapsed into a single level of children. There is a substantial decrease in the deviance even though there are fewer parameters in model gm2 than in gm1. This decrease is because the flexibility of the model - its ability to model the behavior of the response - is being put to better use in gm2 than in gm1.

+

At present the calculation of the geomdof as sum(influence(m)) is not correctly defined in our code for a GLMM so we need to do some more work before we can examine those values.

+
+
+

3.3 Using urban&dist as a grouping factor

+

It turns out that there can be more difference between urban and rural settings within the same political district than there is between districts. To model this difference we build a model with urban&dist as a grouping factor.

+
+
gm3 = let
+  form = @formula(
+    use ~
+      1 +
+      age * children +
+      abs2(age) +
+      children +
+      urban +
+      (1 | urban & dist)
+  )
+  fit(MixedModel, form, contra, dist; nAGQ, contrasts)
+end
+
+
Minimizing 90    Time: 0:00:00 ( 1.12 ms/it)
+  objective:  2353.829069935298
+Minimizing 144    Time: 0:00:00 ( 1.11 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_urban & dist
(Intercept)-0.34150.1269-2.690.00710.5761
age-0.01290.0112-1.160.2472
children: true0.60640.10455.80<1e-08
abs2(age)-0.00560.0008-6.66<1e-10
urban: Y0.39360.08594.58<1e-05
age & children: true0.03320.01282.590.0096
+
+
+
+
+Code +
let
+  mods = [gm3, gm2, gm1]
+  DataFrame(;
+    model=[:gm3, :gm2, :gm1],
+    npar=dof.(mods),
+    deviance=deviance.(mods),
+    AIC=aic.(mods),
+    BIC=bic.(mods),
+    AICc=aicc.(mods),
+  )
+end
+
+
+
3×6 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowmodelnpardevianceAICBICAICc
SymbolInt64Float64Float64Float64Float64
1gm372353.822368.482407.462368.54
2gm272364.922379.182418.152379.24
3gm182372.462388.732433.272388.81
+
+
+
+

Notice that the parameter count in gm3 is the same as that of gm2 - the thing that has changed is the number of levels of the grouping factor- resulting in a much lower deviance for gm3. This reinforces the idea that a simple count of the number of parameters to be estimated does not always reflect the complexity of the model.

+
+
gm2
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_dist
(Intercept)-0.36150.1275-2.840.00460.4757
age-0.01310.0110-1.190.2351
children: true0.60550.10355.85<1e-08
abs2(age)-0.00580.0008-6.89<1e-11
urban: Y0.35670.06025.93<1e-08
age & children: true0.03420.01272.690.0072
+
+
+
+
gm3
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_urban & dist
(Intercept)-0.34150.1269-2.690.00710.5761
age-0.01290.0112-1.160.2472
children: true0.60640.10455.80<1e-08
abs2(age)-0.00560.0008-6.66<1e-10
urban: Y0.39360.08594.58<1e-05
age & children: true0.03320.01282.590.0096
+
+
+

The coefficient for age may be regarded as insignificant but we retain it for two reasons: we have a term of age² (written abs2(age)) in the model and we have a significant interaction age & children in the model.

+
+
+

3.4 Predictions for some subgroups

+

For a “typical” district (random effect near zero) the predictions on the linear predictor scale for a woman whose age is near the centering value (i.e. centered age of zero) are:

+
+
using Effects
+design = Dict(
+  :children => [true, false], :urban => ["Y", "N"], :age => [0.0]
+)
+preds = effects(design, gm3; invlink=AutoInvLink())
+
+
4×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowchildrenageurbanuse: Yerrlowerupper
BoolFloat64StringFloat64Float64Float64Float64
1true0.0Y0.658940.033830.625110.69277
2false0.0Y0.3648680.05341020.3114580.418278
3true0.0N0.467890.02813790.4397520.496028
4false0.0N0.2072650.03640590.170860.243671
+
+
+
+
+
+
+

4 Summarizing the results

+
    +
  • From the data plot we can see a quadratic trend in the probability by age.
    • +
  • The patterns for women with children are similar and we do not need to distinguish between 1, 2, and 3+ children.
    • +
  • We do distinguish between those women who do not have children and those with children. This shows up in a significant age & children interaction term.
    • +
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/glmm_files/figure-html/fig-contradata-output-2.svg b/glmm_files/figure-html/fig-contradata-output-2.svg new file mode 100644 index 0000000..019421e --- /dev/null +++ b/glmm_files/figure-html/fig-contradata-output-2.svg @@ -0,0 +1,595 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/index.html b/index.html new file mode 100644 index 0000000..908e133 --- /dev/null +++ b/index.html @@ -0,0 +1,994 @@ + + + + + + + + + + + +Seventh Summer School on Statistical Methods for Linguistics and Psychology – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Seventh Summer School on Statistical Methods for Linguistics and Psychology

+

Advanced methods in frequentist statistics with Julia

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Reinhold Kliegl and Douglas Bates

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

This site provides materials for the Advanced frequentist methods stream of the Summer School on Statistical Methods to be held at the University of Potsdam, 11-15 September, 2024.

+
+

1 Installation of tools prior to the course

+

This section of the summer school will use the Julia programming language and related tools. Because most students in the course will have more experience with R and the RStudio integrated development environment (IDE), it is necessary to install and configure several software systems prior to the course.

+
+

1.1 git

+

We will assume that you have git installed and are able to clone a repository from github. If not, Happy Git with R is a good place to learn about git for data science.

+

This website is built using quarto, described below, from the repository. Clone this repository with, e.g.

+
git clone https://github.com/RePsychLing/SMLP2024
+
+
+

1.2 Julia Programming Language

+

We will use Julia v1.9 in the summer school. We recommend using Juliaup to install and manage Julia versions. Juliaup makes it trivial to upgrade to new Julia releases or even use old ones. Alternatively, you can download the version appropriate for your setup from here: Julia Programming Language

+
+
+

1.3 Visual Studio Code (VS Code)

+

We will use VS Code IDE, that is Julia : VS Code ~ R : RStudio. You can download the version appropriate for your setup from here: VS Code

+
+
+

1.4 Quarto

+

The web site and other documents for this course are rendered using a knitr-like system called Quarto. You can download the version appropriate for your setup from here: quarto

+
+
+
+

2 Summary

+

We assume everybody will succeed with the installations. If not, please get in touch!

+

You may also want to give it a try to get these tools to play nicely with each other – but don’t spend too much time on this.

+ + +
+ + Back to top
+ +
+ + + + + \ No newline at end of file diff --git a/kb07.html b/kb07.html new file mode 100644 index 0000000..8fb8d69 --- /dev/null +++ b/kb07.html @@ -0,0 +1,1676 @@ + + + + + + + + + + + +Bootstrapping a fitted model – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Bootstrapping a fitted model

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

Begin by loading the packages to be used.

+
+
+Code +
using AlgebraOfGraphics
+using CairoMakie
+using DataFrames
+using MixedModels
+using ProgressMeter
+using Random
+using SMLP2024: dataset
+
+CairoMakie.activate!(; type="svg")
+
+ProgressMeter.ijulia_behavior(:clear)
+
+
+
+

1 Data set and model

+

The kb07 data (Kronmüller & Barr, 2007) are one of the datasets provided by the MixedModels package.

+
+
kb07 = dataset(:kb07)
+
+
Arrow.Table with 1789 rows, 7 columns, and schema:
+ :subj      String
+ :item      String
+ :spkr      String
+ :prec      String
+ :load      String
+ :rt_trunc  Int16
+ :rt_raw    Int16
+
+
+

Convert the table to a DataFrame for summary.

+
+
describe(DataFrame(kb07))
+
+
7×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1subjS030S1030String
2itemI01I320String
3spkrnewold0String
4precbreakmaintain0String
5loadnoyes0String
6rt_trunc2182.25791940.051710Int16
7rt_raw2226.245791940.0159230Int16
+
+
+
+

The experimental factors; spkr, prec, and load, are two-level factors. The EffectsCoding contrast is used with these to create a \(\pm1\) encoding. Furthermore, Grouping contrasts are assigned to the subj and item factors. This is not a contrast per-se but an indication that these factors will be used as grouping factors for random effects and, therefore, there is no need to create a contrast matrix. For large numbers of levels in a grouping factor, an attempt to create a contrast matrix may cause memory overflow.

+

It is not important in these cases but a good practice in any case.

+
+
contrasts = merge(
+  Dict(nm => EffectsCoding() for nm in (:spkr, :prec, :load)),
+  Dict(nm => Grouping() for nm in (:subj, :item)),
+)
+
+

The display of an initial model fit

+
+
kbm01 = let
+  form = @formula(
+    rt_trunc ~
+      1 +
+      spkr * prec * load +
+      (1 + spkr + prec + load | subj) +
+      (1 + spkr + prec + load | item)
+  )
+  fit(MixedModel, form, kb07; contrasts)
+end
+
+
Minimizing 2    Time: 0:00:00 ( 0.43  s/it)
+  objective:  29353.753287212847
+Minimizing 614    Time: 0:00:01 ( 2.88 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_subjσ_item
(Intercept)2181.675077.302928.22<1e-99301.8414362.1701
spkr: old67.749018.28243.710.000243.039640.5288
prec: maintain-333.920647.1621-7.08<1e-1162.1035246.9365
load: yes78.768119.52474.03<1e-0465.147342.1755
spkr: old & prec: maintain-21.963415.8062-1.390.1647
spkr: old & load: yes18.383815.80621.160.2448
prec: maintain & load: yes4.533315.80620.290.7743
spkr: old & prec: maintain & load: yes23.605215.80621.490.1353
Residual668.5061
+
+
+

does not include the estimated correlations of the random effects.

+

The VarCorr extractor displays these.

+
+
VarCorr(kbm01)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
subj(Intercept)91108.2428301.8414
spkr: old1852.409843.0396+0.78
prec: maintain3856.840862.1035-0.59+0.02
load: yes4244.168965.1473+0.36+0.82+0.53
item(Intercept)131167.1572362.1701
spkr: old1642.582540.5288+0.44
prec: maintain60977.6107246.9365-0.69+0.35
load: yes1778.776342.1755+0.32+0.23-0.15
Residual446900.4651668.5061
+
+
+

None of the two-factor or three-factor interaction terms in the fixed-effects are significant. In the random-effects terms only the scalar random effects and the prec random effect for item appear to be warranted, leading to the reduced formula

+
+
kbm02 = let
+  form = @formula(
+    rt_trunc ~
+      1 + spkr + prec + load + (1 | subj) + (1 + prec | item)
+  )
+  fit(MixedModel, form, kb07; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)2181.852677.468128.16<1e-99364.7125298.0259
spkr: old67.879016.07854.22<1e-04
prec: maintain-333.790647.4472-7.03<1e-11252.5212
load: yes78.590416.07854.89<1e-05
Residual680.0319
+
+
+
+
VarCorr(kbm02)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)133015.241364.713
prec: maintain63766.936252.521-0.70
subj(Intercept)88819.437298.026
Residual462443.388680.032
+
+
+

These two models are nested and can be compared with a likelihood-ratio test.

+
+
MixedModels.likelihoodratiotest(kbm02, kbm01)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
rt_trunc ~ 1 + spkr + prec + load + (1 | subj) + (1 + prec | item)928664
rt_trunc ~ 1 + spkr + prec + load + spkr & prec + spkr & load + prec & load + spkr & prec & load + (1 + spkr + prec + load | subj) + (1 + spkr + prec + load | item)292863727200.1436
+
+
+

The p-value of approximately 17% leads us to prefer the simpler model, kbm02, to the more complex, kbm01.

+
+
+

2 A bootstrap sample

+

Create a bootstrap sample of a few thousand parameter estimates from the reduced model. The pseudo-random number generator is initialized to a fixed value for reproducibility.

+
+
Random.seed!(1234321)
+kbm02samp = parametricbootstrap(2000, kbm02)
+kbm02tbl = kbm02samp.tbl
+
+
Table with 14 columns and 2000 rows:
+      obj      β1       β2       β3        β4       σ        σ1       σ2       ⋯
+    ┌───────────────────────────────────────────────────────────────────────────
+ 1  │ 28769.9  2152.69  60.3921  -359.134  54.1231  702.318  421.687  212.916  ⋯
+ 2  │ 28775.3  2221.81  61.7318  -349.417  71.1992  697.028  442.809  278.93   ⋯
+ 3  │ 28573.6  2325.31  53.9668  -404.225  81.3407  657.746  364.354  274.392  ⋯
+ 4  │ 28682.9  2274.65  52.9438  -384.66   69.6788  682.801  398.9    319.726  ⋯
+ 5  │ 28616.1  2191.34  54.7785  -357.919  111.829  673.878  253.244  235.909  ⋯
+ 6  │ 28634.1  2176.73  95.9793  -378.383  82.1559  671.685  510.996  309.876  ⋯
+ 7  │ 28620.4  2128.07  57.6928  -267.747  60.4621  666.986  396.878  295.008  ⋯
+ 8  │ 28665.9  2152.59  55.8208  -274.382  76.5128  684.746  369.091  274.446  ⋯
+ 9  │ 28651.2  2088.3   76.4636  -296.14   95.6124  680.282  265.684  186.041  ⋯
+ 10 │ 28679.9  2375.23  72.2422  -423.922  94.0833  680.729  425.72   287.686  ⋯
+ 11 │ 28686.8  2034.11  59.9149  -224.859  59.5448  682.433  308.676  272.183  ⋯
+ 12 │ 28702.2  2130.98  83.3953  -271.61   79.974   689.679  401.972  264.255  ⋯
+ 13 │ 28625.0  2055.35  55.3526  -363.937  67.7128  676.818  309.148  265.792  ⋯
+ 14 │ 28570.3  2160.73  81.5575  -321.913  108.979  661.524  426.871  259.827  ⋯
+ 15 │ 28613.6  2237.73  36.9996  -362.443  82.1812  673.873  312.863  241.595  ⋯
+ 16 │ 28681.4  2306.64  43.8801  -429.983  103.082  686.437  360.536  204.514  ⋯
+ 17 │ 28577.2  2143.78  66.2959  -373.598  93.157   668.15   321.47   183.119  ⋯
+ ⋮  │    ⋮        ⋮        ⋮        ⋮         ⋮        ⋮        ⋮        ⋮     ⋱
+
+
+

One of the uses of such a sample is to form “confidence intervals” on the parameters by obtaining the shortest interval that covers a given proportion (95%, by default) of the sample.

+
+
confint(kbm02samp)
+
+
DictTable with 2 columns and 9 rows:
+ par   lower     upper
+ ────┬────────────────────
+ β1  │ 2028.01   2337.92
+ β2  │ 38.431    99.5944
+ β3  │ -439.321  -245.864
+ β4  │ 46.0262   107.511
+ ρ1  │ -0.89799  -0.445596
+ σ   │ 655.249   701.497
+ σ1  │ 261.196   448.509
+ σ2  │ 175.489   312.043
+ σ3  │ 228.099   357.789
+
+
+

A sample like this can be used for more than just creating an interval because it approximates the distribution of the estimator. For the fixed-effects parameters the estimators are close to being normally distributed, Figure 1.

+
+
+Code +
draw(
+  data(kbm02samp.β) * mapping(:β; color=:coefname) * AlgebraOfGraphics.density();
+  figure=(; resolution=(800, 450)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 1: Comparative densities of the fixed-effects coefficients in kbm02samp +
+
+
+
+
+
+
+Code +
let pars = ["σ1", "σ2", "σ3"]
+  draw(
+    data(kbm02tbl) *
+    mapping(pars .=> "σ"; color=dims(1) => renamer(pars)) *
+    AlgebraOfGraphics.density();
+    figure=(; resolution=(800, 450)),
+  )
+end
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 2: Density plot of bootstrap samples standard deviation of random effects +
+
+
+
+
+
+
+Code +
draw(
+  data(kbm02tbl) *
+  mapping(:ρ1 => "Correlation") *
+  AlgebraOfGraphics.density();
+  figure=(; resolution=(800, 450)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 3: Density plot of correlation parameters in bootstrap sample from model kbm02 +
+
+
+
+
+
+
+

3 References

+
+
+Kronmüller, E., & Barr, D. J. (2007). Perspective-free pragmatics: Broken precedents and the recovery-from-preemption hypothesis. Journal of Memory and Language, 56(3), 436–455. https://doi.org/10.1016/j.jml.2006.05.002 +
+
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/kb07_files/figure-html/fig-kbm02fedens-output-2.svg b/kb07_files/figure-html/fig-kbm02fedens-output-2.svg new file mode 100644 index 0000000..df523a6 --- /dev/null +++ b/kb07_files/figure-html/fig-kbm02fedens-output-2.svg @@ -0,0 +1,459 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kb07_files/figure-html/fig-kbm02sampcorrdens-output-2.svg b/kb07_files/figure-html/fig-kbm02sampcorrdens-output-2.svg new file mode 100644 index 0000000..0d91560 --- /dev/null +++ b/kb07_files/figure-html/fig-kbm02sampcorrdens-output-2.svg @@ -0,0 +1,252 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kb07_files/figure-html/fig-kbm02sampsigmadens-output-2.svg b/kb07_files/figure-html/fig-kbm02sampsigmadens-output-2.svg new file mode 100644 index 0000000..4f8c841 --- /dev/null +++ b/kb07_files/figure-html/fig-kbm02sampsigmadens-output-2.svg @@ -0,0 +1,226 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kkl15.html b/kkl15.html new file mode 100644 index 0000000..c08fab0 --- /dev/null +++ b/kkl15.html @@ -0,0 +1,5148 @@ + + + + + + + + + + + +RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +
+

1 Background

+

Kliegl et al. (2015) is a follow-up to Kliegl et al. (2011) (see also script kwdyz11.qmd) from an experiment looking at a variety of effects of visual cueing under four different cue-target relations (CTRs). In this experiment two rectangles are displayed (1) in horizontal orientation , (2) in vertical orientation, (3) in left diagonal orientation, or in (4) right diagonal orientation relative to a central fixation point. Subjects react to the onset of a small or a large visual target occurring at one of the four ends of the two rectangles. The target is cued validly on 70% of trials by a brief flash of the corner of the rectangle at which it appears; it is cued invalidly at the three other locations 10% of the trials each. This implies a latent imbalance in design that is not visible in the repeated-measures ANOVA, but we will show its effect in the random-effect structure and conditional modes.

+

There are a couple of differences between the first and this follow-up experiment, rendering it more a conceptual than a direct replication. First, the original experiment was carried out at Peking University and this follow-up at Potsdam University. Second, diagonal orientations of rectangles and large target sizes were not part of the design of Kliegl et al. (2011).

+

We specify three contrasts for the four-level factor CTR that are derived from spatial, object-based, and attractor-like features of attention. They map onto sequential differences between appropriately ordered factor levels. Replicating Kliegl et al. (2011), the attraction effect was not significant as a fixed effect, but yielded a highly reliable variance component (VC; i.e., reliable individual differences in positive and negative attraction effects cancel the fixed effect). Moreover, these individual differences in the attraction effect were negatively correlated with those in the spatial effect.

+

This comparison is of interest because a few years after the publication of Kliegl et al. (2011), the theoretically critical correlation parameter (CP) between the spatial effect and the attraction effect was determined as the source of a non-singular LMM in that paper. The present study served the purpose to estimate this parameter with a larger sample and a wider variety of experimental conditions.

+

Here we also include two additional experimental manipulations of target size and orientation of cue rectangle. A similar analysis was reported in the parsimonious mixed-model paper (Bates et al., 2015); it was also used in a paper of GAMMs (Baayen et al., 2017). Data and R scripts of those analyses are also available in R-package RePsychLing.

+

The analysis is based on log-transformed reaction times lrt, indicated by a boxcox() check of model residuals.

+

In this vignette we focus on the reduction of model complexity. And we start with a quote:

+

“Neither the [maximal] nor the [minimal] linear mixed models are appropriate for most repeated measures analysis. Using the [maximal] model is generally wasteful and costly in terms of statiscal power for testing hypotheses. On the other hand, the [minimal] model fails to account for nontrivial correlation among repeated measurements. This results in inflated [T]ype I error rates when non-negligible correlation does in fact exist. We can usually find middle ground, a covariance model that adequately accounts for correlation but is more parsimonious than the [maximal] model. Doing so allows us full control over [T]ype I error rates without needlessly sacrificing power.”

+

Stroup, W. W. (2012, p. 185). Generalized linear mixed models: Modern concepts, methods and applica?ons. CRC Press, Boca Raton.

+
+
+

2 Packages

+
+
+Code +
using Arrow
+using AlgebraOfGraphics
+using AlgebraOfGraphics: density
+using BoxCox
+using CairoMakie
+using CategoricalArrays
+using Chain
+using DataFrameMacros
+using DataFrames
+using MixedModels
+using MixedModelsMakie
+using ProgressMeter
+using Random
+using SMLP2024: dataset
+using StatsBase
+
+ProgressMeter.ijulia_behavior(:clear)
+CairoMakie.activate!(; type="svg")
+
+
+
+
+

3 Read data, compute and plot means

+
+
dat = DataFrame(dataset(:kkl15))
+describe(dat)
+
+
[ Info: Downloading kkl15 dataset
+
+
+
5×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1SubjS001S1470String
2CTRdodval0String
3rt293.147150.22276.594749.4810Float32
4cardinalcardinaldiagonal0String
5sizebigsmall0String
+
+
+
+
+
dat_subj = combine(
+  groupby(dat, [:Subj, :CTR]),
+  nrow => :n,
+  :rt => mean => :rt_m,
+  :rt => (c -> mean(log, c)) => :lrt_m,
+)
+dat_subj.CTR = categorical(dat_subj.CTR, levels=levels(dat.CTR))
+describe(dat_subj)
+
+
5×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1SubjS001S1470String
2CTRvaldod0CategoricalValue{String, UInt32}
3n156.2944964.04480Int64
4rt_m308.223208.194304.862584.710Float32
5lrt_m5.69085.332265.698486.361410Float32
+
+
+
+
+
+Code +
boxplot(
+  dat_subj.CTR.refs,
+  dat_subj.lrt_m;
+  orientation=:horizontal,
+  show_notch=true,
+  axis=(;
+    yticks=(
+      1:4,
+      [
+        "valid cue",
+        "same obj/diff pos",
+        "diff obj/same pos",
+        "diff obj/diff pos",
+      ]
+    )
+  ),
+  figure=(; resolution=(800, 300)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 1: Comparative boxplots of mean log response time by subject under different conditions +
+
+
+
+
+

Mean of log reaction times for four cue-target relations. Targets appeared at (a) the cued position (valid) in a rectangle, (b) in the same rectangle cue, but at its other end, (c) on the second rectangle, but at a corresponding horizontal/vertical physical distance, or (d) at the other end of the second rectangle, that is \(\sqrt{2}\) of horizontal/vertical distance diagonally across from the cue, that is also at larger physical distance compared to (c).

+

We remove the outlier subject and replot, but we model the data points in dat and check whether this subject appears as an outlier in the caterpillar plot of conditional modes.

+
+
+Code +
let dat_subj = filter(r -> r.rt_m < 510, dat_subj)
+  boxplot(
+    dat_subj.CTR.refs,
+    dat_subj.lrt_m;
+    orientation=:horizontal,
+    show_notch=true,
+    axis=(;
+      yticks=(
+        1:4,
+        [
+          "valid cue",
+          "same obj/diff pos",
+          "diff obj/same pos",
+          "diff obj/diff pos",
+        ]
+      )
+    ),
+    figure=(; resolution=(800, 300)),
+  )
+end
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 2: Comparative boxplots of mean log response time by subject under different conditions without outlier +
+
+
+
+
+
+
+

4 Setup of linear mixed model

+
+

4.1 Contrasts

+
+
contrasts = Dict(
+  :Subj => Grouping(),
+  :CTR => SeqDiffCoding(; levels=["val", "sod", "dos", "dod"]),
+  :cardinal => EffectsCoding(; levels=["cardinal", "diagonal"]),
+  :size => EffectsCoding(; levels=["big", "small"])
+)
+
+
Dict{Symbol, StatsModels.AbstractContrasts} with 4 entries:
+  :CTR      => SeqDiffCoding(["val", "sod", "dos", "dod"])
+  :size     => EffectsCoding(nothing, ["big", "small"])
+  :Subj     => Grouping()
+  :cardinal => EffectsCoding(nothing, ["cardinal", "diagonal"])
+
+
+
+
m_max_rt = let
+  form = @formula rt ~ 1 + CTR * size * cardinal +
+                           (1 + CTR * size * cardinal | Subj)
+  fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 2    Time: 0:00:00 (53.17 ms/it)
+  objective:  601241.5133290171
+Minimizing 1002    Time: 0:00:01 ( 1.79 ms/it)
+  objective:  599333.7272274668
+Minimizing 1408    Time: 0:00:02 ( 1.66 ms/it)
+  objective:  599308.5351230928
+Minimizing 1489    Time: 0:00:02 ( 1.64 ms/it)
+  objective:  599306.5889190858
+Minimizing 1570    Time: 0:00:02 ( 1.62 ms/it)
+  objective:  599304.9501835523
+Minimizing 1649    Time: 0:00:02 ( 1.61 ms/it)
+  objective:  599303.8187455853
+Minimizing 1727    Time: 0:00:02 ( 1.61 ms/it)
+  objective:  599301.3769616229
+Minimizing 1799    Time: 0:00:02 ( 1.60 ms/it)
+  objective:  599300.3700002148
+Minimizing 1872    Time: 0:00:02 ( 1.60 ms/it)
+  objective:  599299.4958301473
+Minimizing 1943    Time: 0:00:03 ( 1.59 ms/it)
+  objective:  599299.0448406646
+Minimizing 2010    Time: 0:00:03 ( 1.59 ms/it)
+  objective:  599298.6887895833
+Minimizing 2087    Time: 0:00:03 ( 1.58 ms/it)
+  objective:  599298.5089541689
+Minimizing 2162    Time: 0:00:03 ( 1.58 ms/it)
+  objective:  599298.3506449385
+Minimizing 2233    Time: 0:00:03 ( 1.57 ms/it)
+  objective:  599298.2531667919
+Minimizing 2307    Time: 0:00:03 ( 1.57 ms/it)
+  objective:  599298.1823067392
+Minimizing 2379    Time: 0:00:03 ( 1.56 ms/it)
+  objective:  599298.1003545397
+Minimizing 2449    Time: 0:00:03 ( 1.56 ms/it)
+  objective:  599298.0592483006
+Minimizing 2519    Time: 0:00:03 ( 1.57 ms/it)
+  objective:  599298.016369703
+Minimizing 2585    Time: 0:00:04 ( 1.57 ms/it)
+  objective:  599297.9868262913
+Minimizing 2651    Time: 0:00:04 ( 1.57 ms/it)
+  objective:  599297.9623177209
+Minimizing 2717    Time: 0:00:04 ( 1.57 ms/it)
+  objective:  599297.9362295533
+Minimizing 2783    Time: 0:00:04 ( 1.57 ms/it)
+  objective:  599297.9158610143
+Minimizing 2847    Time: 0:00:04 ( 1.57 ms/it)
+  objective:  599297.8907719204
+Minimizing 2913    Time: 0:00:04 ( 1.57 ms/it)
+  objective:  599297.8655307669
+Minimizing 2977    Time: 0:00:04 ( 1.57 ms/it)
+  objective:  599297.838000219
+Minimizing 3044    Time: 0:00:04 ( 1.57 ms/it)
+  objective:  599297.8130767004
+Minimizing 3109    Time: 0:00:04 ( 1.57 ms/it)
+  objective:  599297.7940609268
+Minimizing 3177    Time: 0:00:04 ( 1.57 ms/it)
+  objective:  599297.7789314522
+Minimizing 3243    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  599297.7661259501
+Minimizing 3309    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  599297.7549728255
+Minimizing 3375    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  599297.7460114947
+Minimizing 3440    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  599297.73789173
+Minimizing 3507    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  599297.7294974972
+Minimizing 3574    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  599297.7197337943
+Minimizing 3639    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  599297.71324412
+Minimizing 3704    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  599297.7098021851
+Minimizing 3770    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  599297.7007812862
+Minimizing 3838    Time: 0:00:06 ( 1.57 ms/it)
+  objective:  599297.6951566301
+Minimizing 3904    Time: 0:00:06 ( 1.57 ms/it)
+  objective:  599297.6904034589
+Minimizing 3971    Time: 0:00:06 ( 1.57 ms/it)
+  objective:  599297.682983504
+Minimizing 4038    Time: 0:00:06 ( 1.57 ms/it)
+  objective:  599297.6760234906
+Minimizing 4112    Time: 0:00:06 ( 1.57 ms/it)
+  objective:  599297.6708892871
+Minimizing 4180    Time: 0:00:06 ( 1.57 ms/it)
+  objective:  599297.6661708716
+Minimizing 4246    Time: 0:00:06 ( 1.57 ms/it)
+  objective:  599297.6605394017
+Minimizing 4311    Time: 0:00:06 ( 1.57 ms/it)
+  objective:  599297.6580177272
+Minimizing 4385    Time: 0:00:06 ( 1.57 ms/it)
+  objective:  599297.6559655914
+Minimizing 4457    Time: 0:00:06 ( 1.56 ms/it)
+  objective:  599297.6544785632
+Minimizing 4528    Time: 0:00:07 ( 1.56 ms/it)
+  objective:  599297.654285845
+Minimizing 4595    Time: 0:00:07 ( 1.56 ms/it)
+  objective:  599297.6534127444
+Minimizing 4667    Time: 0:00:07 ( 1.57 ms/it)
+
+Minimizing 4668    Time: 0:00:07 ( 1.57 ms/it)
+  objective:  599297.6521322867
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)308.40785.838252.83<1e-990.0000
CTR: sod23.26852.56699.06<1e-185.9416
CTR: dos13.08011.54348.47<1e-164.0401
CTR: dod2.77242.09181.330.185013.6316
size: small26.41305.83824.52<1e-0554.0253
cardinal: diagonal6.67491.78823.730.000211.4760
CTR: sod & size: small8.79252.56693.430.000621.4147
CTR: dos & size: small-0.70651.5434-0.460.64727.8354
CTR: dod & size: small7.49832.09183.580.00037.9016
CTR: sod & cardinal: diagonal3.63181.10243.290.00103.5394
CTR: dos & cardinal: diagonal1.33401.24481.070.28391.8675
CTR: dod & cardinal: diagonal-0.22451.3222-0.170.86524.1664
size: small & cardinal: diagonal2.05021.78821.150.251611.4335
CTR: sod & size: small & cardinal: diagonal-0.76381.1024-0.690.48844.3866
CTR: dos & size: small & cardinal: diagonal-0.15551.2448-0.120.90061.6228
CTR: dod & size: small & cardinal: diagonal4.17671.32223.160.00162.2306
Residual63.0091
+
+
+
+
m_cpx_rt = let
+  form = @formula rt ~ 1 + CTR * size * cardinal +
+                           (1 + CTR + size + cardinal | Subj)
+  fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 302    Time: 0:00:00 ( 0.33 ms/it)
+  objective:  599405.9108670889
+Minimizing 915    Time: 0:00:00 ( 0.22 ms/it)
+  objective:  599403.6000686976
+Minimizing 924    Time: 0:00:00 ( 0.22 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)308.40756.063650.86<1e-9940.4009
CTR: sod23.27232.47679.40<1e-2021.3210
CTR: dos13.10981.46248.96<1e-187.5108
CTR: dod2.71421.91781.420.157013.7084
size: small26.38656.06364.35<1e-0438.9502
cardinal: diagonal6.65051.73193.840.000115.6643
CTR: sod & size: small8.79562.47673.550.0004
CTR: dos & size: small-0.72301.4624-0.490.6210
CTR: dod & size: small7.41881.91783.870.0001
CTR: sod & cardinal: diagonal3.64130.92103.95<1e-04
CTR: dos & cardinal: diagonal1.32781.21761.090.2755
CTR: dod & cardinal: diagonal-0.31451.2217-0.260.7968
size: small & cardinal: diagonal2.04711.73191.180.2372
CTR: sod & size: small & cardinal: diagonal-0.76100.9210-0.830.4086
CTR: dos & size: small & cardinal: diagonal-0.13351.2176-0.110.9127
CTR: dod & size: small & cardinal: diagonal4.11171.22173.370.0008
Residual63.1010
+
+
+
+
+

4.2 Box-Cox

+
+
bc1 = fit(BoxCoxTransformation, m_max_rt)
+
+
+
bc2 = fit(BoxCoxTransformation, m_cpx_rt)
+
+
Minimizing 669    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -24614.57221183604
+Minimizing 1134    Time: 0:00:00 ( 0.16 ms/it)
+Minimizing 666    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -7521.599763635946
+Minimizing 894    Time: 0:00:00 ( 0.16 ms/it)
+Minimizing 668    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27394.902843102427
+Minimizing 1173    Time: 0:00:00 ( 0.16 ms/it)
+
+Minimizing 1174    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27395.25159135472
+Minimizing 670    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27948.912130319124
+Minimizing 1293    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27948.924023849817
+Minimizing 1293    Time: 0:00:00 ( 0.16 ms/it)
+Minimizing 670    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27948.757616138104
+Minimizing 1292    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27949.950395857046
+Minimizing 1496    Time: 0:00:00 ( 0.16 ms/it)
+
+Minimizing 1497    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27949.950563489412
+Minimizing 670    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27949.955411736297
+Minimizing 1295    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27949.95958768511
+Minimizing 1899    Time: 0:00:00 ( 0.17 ms/it)
+  objective:  -27949.961513601214
+Minimizing 2357    Time: 0:00:00 ( 0.17 ms/it)
+Minimizing 661    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27949.66778994192
+Minimizing 1100    Time: 0:00:00 ( 0.16 ms/it)
+Minimizing 661    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27948.215871258206
+Minimizing 1278    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27948.380519227445
+Minimizing 1499    Time: 0:00:00 ( 0.16 ms/it)
+Minimizing 667    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27948.211640368994
+Minimizing 1285    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27949.93485052913
+Minimizing 1312    Time: 0:00:00 ( 0.16 ms/it)
+Minimizing 661    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27949.922438171878
+Minimizing 1216    Time: 0:00:00 ( 0.16 ms/it)
+Minimizing 663    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27947.646299135915
+Minimizing 1189    Time: 0:00:00 ( 0.16 ms/it)
+Minimizing 668    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27948.357951161892
+Minimizing 948    Time: 0:00:00 ( 0.16 ms/it)
+Minimizing 669    Time: 0:00:00 ( 0.16 ms/it)
+  objective:  -27948.135260234907
+Minimizing 1288    Time: 0:00:00 ( 0.17 ms/it)
+  objective:  -27948.35069788193
+Minimizing 1394    Time: 0:00:00 ( 0.17 ms/it)
+Minimizing 571    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27948.556266149943
+Minimizing 1107    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27949.943733548123
+Minimizing 1577    Time: 0:00:00 ( 0.19 ms/it)
+Minimizing 571    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27947.752620644653
+Minimizing 1105    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27948.35024549874
+Minimizing 1211    Time: 0:00:00 ( 0.19 ms/it)
+Minimizing 571    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27946.898268664925
+Minimizing 1106    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27948.352563028613
+Minimizing 1442    Time: 0:00:00 ( 0.19 ms/it)
+Minimizing 571    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27945.4392929145
+Minimizing 1107    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27949.894231694314
+Minimizing 1632    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27949.952141882208
+Minimizing 1690    Time: 0:00:00 ( 0.19 ms/it)
+Minimizing 571    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27948.002337963364
+Minimizing 1106    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27949.916584711973
+Minimizing 1764    Time: 0:00:00 ( 0.22 ms/it)
+Minimizing 572    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27949.87281263115
+Minimizing 920    Time: 0:00:00 ( 0.19 ms/it)
+Minimizing 571    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27946.451386707562
+Minimizing 1107    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27948.382683844105
+Minimizing 1632    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27948.40667584678
+Minimizing 2156    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27949.946197014546
+Minimizing 2301    Time: 0:00:00 ( 0.19 ms/it)
+Minimizing 572    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27947.144567634066
+Minimizing 1108    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27949.94662606359
+Minimizing 1629    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27949.954183490412
+Minimizing 1683    Time: 0:00:00 ( 0.19 ms/it)
+Minimizing 573    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27946.396120890593
+Minimizing 1109    Time: 0:00:00 ( 0.19 ms/it)
+  objective:  -27949.86351095891
+Minimizing 1621    Time: 0:00:00 ( 0.19 ms/it)
+
+
+
Box-Cox transformation
+
+estimated λ: -0.7011
+resultant transformation:
+
+ y^-0.7 - 1
+------------
+    -0.7
+
+
+
+
boxcoxplot(bc2; conf_level=0.95)
+
+

Clear evidence for skew. Traditionally, we used log transforms for reaction times. even stronger than log. We stay with log for now. Could try 1/sqrt(rt).

+
+
+
+

5 Maximum LMM

+

This is the maximum LMM for the design.

+
+
m_max = let
+  form = @formula log(rt) ~ 1 + CTR * size * cardinal +
+                           (1 + CTR * size * cardinal | Subj)
+  fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 280    Time: 0:00:00 ( 0.39 ms/it)
+  objective:  -24344.49369586553
+Minimizing 537    Time: 0:00:00 ( 0.91 ms/it)
+  objective:  -24517.6628400372
+Minimizing 605    Time: 0:00:00 ( 0.98 ms/it)
+  objective:  -24555.16903341203
+Minimizing 670    Time: 0:00:00 ( 1.04 ms/it)
+  objective:  -24581.85603137942
+Minimizing 732    Time: 0:00:00 ( 1.11 ms/it)
+  objective:  -24612.51016961235
+Minimizing 797    Time: 0:00:00 ( 1.15 ms/it)
+  objective:  -24628.64490700058
+Minimizing 863    Time: 0:00:01 ( 1.18 ms/it)
+  objective:  -24643.657937553337
+Minimizing 927    Time: 0:00:01 ( 1.22 ms/it)
+  objective:  -24653.370421164207
+Minimizing 992    Time: 0:00:01 ( 1.24 ms/it)
+  objective:  -24660.203959272734
+Minimizing 1057    Time: 0:00:01 ( 1.26 ms/it)
+  objective:  -24667.287317070408
+Minimizing 1122    Time: 0:00:01 ( 1.28 ms/it)
+  objective:  -24672.258492913992
+Minimizing 1185    Time: 0:00:01 ( 1.29 ms/it)
+  objective:  -24677.09650168264
+Minimizing 1247    Time: 0:00:01 ( 1.32 ms/it)
+  objective:  -24679.41683775858
+Minimizing 1308    Time: 0:00:01 ( 1.33 ms/it)
+  objective:  -24680.821080844416
+Minimizing 1369    Time: 0:00:01 ( 1.35 ms/it)
+  objective:  -24682.871185270804
+Minimizing 1429    Time: 0:00:01 ( 1.37 ms/it)
+  objective:  -24683.68700158556
+Minimizing 1485    Time: 0:00:02 ( 1.39 ms/it)
+  objective:  -24684.38180890402
+Minimizing 1543    Time: 0:00:02 ( 1.40 ms/it)
+  objective:  -24685.08096270136
+Minimizing 1606    Time: 0:00:02 ( 1.41 ms/it)
+  objective:  -24685.58717231793
+Minimizing 1668    Time: 0:00:02 ( 1.43 ms/it)
+  objective:  -24686.327771869914
+Minimizing 1724    Time: 0:00:02 ( 1.44 ms/it)
+  objective:  -24686.765732314692
+Minimizing 1786    Time: 0:00:02 ( 1.45 ms/it)
+  objective:  -24687.208423667387
+Minimizing 1851    Time: 0:00:02 ( 1.45 ms/it)
+  objective:  -24687.46422440867
+Minimizing 1914    Time: 0:00:02 ( 1.47 ms/it)
+  objective:  -24687.77353305462
+Minimizing 1977    Time: 0:00:02 ( 1.47 ms/it)
+  objective:  -24688.055476200585
+Minimizing 2040    Time: 0:00:03 ( 1.48 ms/it)
+  objective:  -24688.263986482827
+Minimizing 2104    Time: 0:00:03 ( 1.48 ms/it)
+  objective:  -24688.41085390256
+Minimizing 2168    Time: 0:00:03 ( 1.49 ms/it)
+  objective:  -24688.510091456636
+Minimizing 2230    Time: 0:00:03 ( 1.50 ms/it)
+  objective:  -24688.626815100517
+Minimizing 2293    Time: 0:00:03 ( 1.50 ms/it)
+  objective:  -24688.786150506727
+Minimizing 2354    Time: 0:00:03 ( 1.51 ms/it)
+  objective:  -24688.921403604832
+Minimizing 2415    Time: 0:00:03 ( 1.51 ms/it)
+  objective:  -24689.04558789387
+Minimizing 2476    Time: 0:00:03 ( 1.52 ms/it)
+  objective:  -24689.138732700238
+Minimizing 2540    Time: 0:00:03 ( 1.52 ms/it)
+  objective:  -24689.25634717969
+Minimizing 2603    Time: 0:00:03 ( 1.53 ms/it)
+  objective:  -24689.32178687373
+Minimizing 2666    Time: 0:00:04 ( 1.53 ms/it)
+  objective:  -24689.383129021808
+Minimizing 2727    Time: 0:00:04 ( 1.54 ms/it)
+  objective:  -24689.445498747224
+Minimizing 2787    Time: 0:00:04 ( 1.54 ms/it)
+  objective:  -24689.492366772312
+Minimizing 2849    Time: 0:00:04 ( 1.54 ms/it)
+  objective:  -24689.549276651924
+Minimizing 2911    Time: 0:00:04 ( 1.55 ms/it)
+  objective:  -24689.579660419473
+Minimizing 2972    Time: 0:00:04 ( 1.55 ms/it)
+  objective:  -24689.623103626887
+Minimizing 3034    Time: 0:00:04 ( 1.55 ms/it)
+  objective:  -24689.64817250226
+Minimizing 3095    Time: 0:00:04 ( 1.56 ms/it)
+  objective:  -24689.67756382241
+Minimizing 3155    Time: 0:00:04 ( 1.56 ms/it)
+  objective:  -24689.70155394707
+Minimizing 3217    Time: 0:00:05 ( 1.56 ms/it)
+  objective:  -24689.711604502423
+Minimizing 3279    Time: 0:00:05 ( 1.56 ms/it)
+  objective:  -24689.741803671848
+Minimizing 3340    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  -24689.756486392092
+Minimizing 3397    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  -24689.782681178272
+Minimizing 3458    Time: 0:00:05 ( 1.57 ms/it)
+  objective:  -24689.79904827584
+Minimizing 3519    Time: 0:00:05 ( 1.58 ms/it)
+  objective:  -24689.811664155077
+Minimizing 3576    Time: 0:00:05 ( 1.58 ms/it)
+  objective:  -24689.832272791067
+Minimizing 3631    Time: 0:00:05 ( 1.59 ms/it)
+  objective:  -24689.846943347868
+Minimizing 3684    Time: 0:00:05 ( 1.59 ms/it)
+  objective:  -24689.859783560656
+Minimizing 3738    Time: 0:00:05 ( 1.60 ms/it)
+  objective:  -24689.87040968405
+Minimizing 3793    Time: 0:00:06 ( 1.60 ms/it)
+  objective:  -24689.88248782755
+Minimizing 3847    Time: 0:00:06 ( 1.61 ms/it)
+  objective:  -24689.891976152267
+Minimizing 3898    Time: 0:00:06 ( 1.61 ms/it)
+  objective:  -24689.90856011295
+Minimizing 3952    Time: 0:00:06 ( 1.62 ms/it)
+  objective:  -24689.921611031605
+Minimizing 4009    Time: 0:00:06 ( 1.62 ms/it)
+  objective:  -24689.93332821091
+Minimizing 4064    Time: 0:00:06 ( 1.63 ms/it)
+  objective:  -24689.947150683132
+Minimizing 4120    Time: 0:00:06 ( 1.63 ms/it)
+  objective:  -24689.959586570294
+Minimizing 4174    Time: 0:00:06 ( 1.64 ms/it)
+  objective:  -24689.97492259156
+Minimizing 4230    Time: 0:00:06 ( 1.64 ms/it)
+  objective:  -24689.997709516756
+Minimizing 4286    Time: 0:00:07 ( 1.64 ms/it)
+  objective:  -24690.01688062803
+Minimizing 4340    Time: 0:00:07 ( 1.65 ms/it)
+  objective:  -24690.03944919549
+Minimizing 4394    Time: 0:00:07 ( 1.65 ms/it)
+  objective:  -24690.070831240482
+Minimizing 4449    Time: 0:00:07 ( 1.65 ms/it)
+  objective:  -24690.099384243964
+Minimizing 4506    Time: 0:00:07 ( 1.65 ms/it)
+  objective:  -24690.11916440891
+Minimizing 4561    Time: 0:00:07 ( 1.66 ms/it)
+  objective:  -24690.14990084028
+Minimizing 4615    Time: 0:00:07 ( 1.66 ms/it)
+  objective:  -24690.17554253544
+Minimizing 4671    Time: 0:00:07 ( 1.67 ms/it)
+  objective:  -24690.205742037153
+Minimizing 4726    Time: 0:00:07 ( 1.67 ms/it)
+  objective:  -24690.230630456765
+Minimizing 4781    Time: 0:00:08 ( 1.68 ms/it)
+  objective:  -24690.256914950685
+Minimizing 4838    Time: 0:00:08 ( 1.68 ms/it)
+  objective:  -24690.27813074185
+Minimizing 4894    Time: 0:00:08 ( 1.68 ms/it)
+  objective:  -24690.30541057862
+Minimizing 4950    Time: 0:00:08 ( 1.68 ms/it)
+  objective:  -24690.327397927475
+Minimizing 5004    Time: 0:00:08 ( 1.68 ms/it)
+  objective:  -24690.346203909896
+Minimizing 5053    Time: 0:00:08 ( 1.69 ms/it)
+  objective:  -24690.38598318767
+Minimizing 5107    Time: 0:00:08 ( 1.69 ms/it)
+  objective:  -24690.410694586277
+Minimizing 5161    Time: 0:00:08 ( 1.69 ms/it)
+  objective:  -24690.431893818473
+Minimizing 5223    Time: 0:00:08 ( 1.69 ms/it)
+  objective:  -24690.46066885148
+Minimizing 5279    Time: 0:00:08 ( 1.70 ms/it)
+  objective:  -24690.48975152527
+Minimizing 5336    Time: 0:00:09 ( 1.70 ms/it)
+  objective:  -24690.521938111433
+Minimizing 5391    Time: 0:00:09 ( 1.70 ms/it)
+  objective:  -24690.56059169984
+Minimizing 5439    Time: 0:00:09 ( 1.70 ms/it)
+  objective:  -24690.58632941207
+Minimizing 5502    Time: 0:00:09 ( 1.70 ms/it)
+  objective:  -24690.629416433098
+Minimizing 5564    Time: 0:00:09 ( 1.70 ms/it)
+  objective:  -24690.651217042803
+Minimizing 5622    Time: 0:00:09 ( 1.71 ms/it)
+  objective:  -24690.679891578282
+Minimizing 5677    Time: 0:00:09 ( 1.71 ms/it)
+  objective:  -24690.71192270383
+Minimizing 5733    Time: 0:00:09 ( 1.71 ms/it)
+  objective:  -24690.73981743516
+Minimizing 5789    Time: 0:00:09 ( 1.71 ms/it)
+  objective:  -24690.768262826554
+Minimizing 5845    Time: 0:00:10 ( 1.71 ms/it)
+  objective:  -24690.785410578115
+Minimizing 5900    Time: 0:00:10 ( 1.71 ms/it)
+  objective:  -24690.798420149837
+Minimizing 5954    Time: 0:00:10 ( 1.71 ms/it)
+  objective:  -24690.8118344737
+Minimizing 6007    Time: 0:00:10 ( 1.72 ms/it)
+  objective:  -24690.816844156172
+Minimizing 6061    Time: 0:00:10 ( 1.72 ms/it)
+  objective:  -24690.82537653554
+Minimizing 6115    Time: 0:00:10 ( 1.72 ms/it)
+  objective:  -24690.833901967286
+Minimizing 6169    Time: 0:00:10 ( 1.72 ms/it)
+  objective:  -24690.84461807572
+Minimizing 6224    Time: 0:00:10 ( 1.72 ms/it)
+  objective:  -24690.856062801773
+Minimizing 6282    Time: 0:00:10 ( 1.72 ms/it)
+  objective:  -24690.868349267726
+Minimizing 6340    Time: 0:00:10 ( 1.73 ms/it)
+  objective:  -24690.885528806994
+Minimizing 6394    Time: 0:00:11 ( 1.73 ms/it)
+  objective:  -24690.901368388113
+Minimizing 6447    Time: 0:00:11 ( 1.73 ms/it)
+  objective:  -24690.914195467732
+Minimizing 6499    Time: 0:00:11 ( 1.73 ms/it)
+  objective:  -24690.928099101333
+Minimizing 6555    Time: 0:00:11 ( 1.73 ms/it)
+  objective:  -24690.940258972973
+Minimizing 6609    Time: 0:00:11 ( 1.73 ms/it)
+  objective:  -24690.956272128416
+Minimizing 6667    Time: 0:00:11 ( 1.73 ms/it)
+  objective:  -24690.963139498734
+Minimizing 6725    Time: 0:00:11 ( 1.73 ms/it)
+  objective:  -24690.98317827665
+Minimizing 6781    Time: 0:00:11 ( 1.73 ms/it)
+  objective:  -24690.99926445535
+Minimizing 6836    Time: 0:00:11 ( 1.74 ms/it)
+  objective:  -24691.01126955079
+Minimizing 6890    Time: 0:00:11 ( 1.74 ms/it)
+  objective:  -24691.026189958673
+Minimizing 6944    Time: 0:00:12 ( 1.74 ms/it)
+  objective:  -24691.034047865738
+Minimizing 7007    Time: 0:00:12 ( 1.74 ms/it)
+  objective:  -24691.049093556325
+Minimizing 7069    Time: 0:00:12 ( 1.74 ms/it)
+  objective:  -24691.060075480636
+Minimizing 7126    Time: 0:00:12 ( 1.74 ms/it)
+  objective:  -24691.069658743167
+Minimizing 7180    Time: 0:00:12 ( 1.74 ms/it)
+  objective:  -24691.078796865077
+Minimizing 7239    Time: 0:00:12 ( 1.74 ms/it)
+  objective:  -24691.08441883025
+Minimizing 7296    Time: 0:00:12 ( 1.74 ms/it)
+  objective:  -24691.092176569346
+Minimizing 7351    Time: 0:00:12 ( 1.74 ms/it)
+  objective:  -24691.097122659132
+Minimizing 7408    Time: 0:00:12 ( 1.74 ms/it)
+  objective:  -24691.10542221075
+Minimizing 7466    Time: 0:00:13 ( 1.74 ms/it)
+  objective:  -24691.112863763305
+Minimizing 7522    Time: 0:00:13 ( 1.75 ms/it)
+  objective:  -24691.12079736585
+Minimizing 7579    Time: 0:00:13 ( 1.75 ms/it)
+  objective:  -24691.125546861418
+Minimizing 7636    Time: 0:00:13 ( 1.75 ms/it)
+  objective:  -24691.13162587659
+Minimizing 7692    Time: 0:00:13 ( 1.75 ms/it)
+  objective:  -24691.13719776697
+Minimizing 7742    Time: 0:00:13 ( 1.75 ms/it)
+  objective:  -24691.14183026872
+Minimizing 7790    Time: 0:00:13 ( 1.75 ms/it)
+  objective:  -24691.142521577545
+Minimizing 7848    Time: 0:00:13 ( 1.75 ms/it)
+  objective:  -24691.153918324657
+Minimizing 7904    Time: 0:00:13 ( 1.75 ms/it)
+  objective:  -24691.160053764655
+Minimizing 7959    Time: 0:00:13 ( 1.76 ms/it)
+  objective:  -24691.165499715382
+Minimizing 8016    Time: 0:00:14 ( 1.76 ms/it)
+  objective:  -24691.17032315898
+Minimizing 8073    Time: 0:00:14 ( 1.76 ms/it)
+  objective:  -24691.17483361258
+Minimizing 8129    Time: 0:00:14 ( 1.76 ms/it)
+  objective:  -24691.17912672143
+Minimizing 8194    Time: 0:00:14 ( 1.76 ms/it)
+  objective:  -24691.18270206745
+Minimizing 8257    Time: 0:00:14 ( 1.76 ms/it)
+  objective:  -24691.187248426395
+Minimizing 8313    Time: 0:00:14 ( 1.76 ms/it)
+  objective:  -24691.192242587123
+Minimizing 8369    Time: 0:00:14 ( 1.76 ms/it)
+  objective:  -24691.195999495085
+Minimizing 8423    Time: 0:00:14 ( 1.76 ms/it)
+  objective:  -24691.198777845544
+Minimizing 8472    Time: 0:00:14 ( 1.76 ms/it)
+  objective:  -24691.20122908061
+Minimizing 8528    Time: 0:00:15 ( 1.76 ms/it)
+  objective:  -24691.20248893931
+Minimizing 8590    Time: 0:00:15 ( 1.76 ms/it)
+  objective:  -24691.20624882134
+Minimizing 8653    Time: 0:00:15 ( 1.76 ms/it)
+  objective:  -24691.207865920725
+Minimizing 8716    Time: 0:00:15 ( 1.76 ms/it)
+  objective:  -24691.20954336344
+Minimizing 8773    Time: 0:00:15 ( 1.77 ms/it)
+  objective:  -24691.21153070078
+Minimizing 8833    Time: 0:00:15 ( 1.77 ms/it)
+  objective:  -24691.21287213654
+Minimizing 8894    Time: 0:00:15 ( 1.77 ms/it)
+  objective:  -24691.214056561486
+Minimizing 8954    Time: 0:00:15 ( 1.77 ms/it)
+  objective:  -24691.21583574853
+Minimizing 9015    Time: 0:00:15 ( 1.77 ms/it)
+  objective:  -24691.21746472066
+Minimizing 9075    Time: 0:00:16 ( 1.77 ms/it)
+  objective:  -24691.21902381971
+Minimizing 9133    Time: 0:00:16 ( 1.77 ms/it)
+  objective:  -24691.220335733426
+Minimizing 9191    Time: 0:00:16 ( 1.77 ms/it)
+  objective:  -24691.221662041
+Minimizing 9247    Time: 0:00:16 ( 1.77 ms/it)
+  objective:  -24691.223278771227
+Minimizing 9304    Time: 0:00:16 ( 1.77 ms/it)
+  objective:  -24691.224470682424
+Minimizing 9359    Time: 0:00:16 ( 1.77 ms/it)
+  objective:  -24691.226363492195
+Minimizing 9413    Time: 0:00:16 ( 1.77 ms/it)
+  objective:  -24691.227852349064
+Minimizing 9474    Time: 0:00:16 ( 1.77 ms/it)
+  objective:  -24691.2289116233
+Minimizing 9529    Time: 0:00:16 ( 1.77 ms/it)
+  objective:  -24691.230563164798
+Minimizing 9584    Time: 0:00:16 ( 1.77 ms/it)
+  objective:  -24691.23201952899
+Minimizing 9647    Time: 0:00:17 ( 1.77 ms/it)
+  objective:  -24691.232898533795
+Minimizing 9692    Time: 0:00:17 ( 1.77 ms/it)
+
+Minimizing 9693    Time: 0:00:17 ( 1.77 ms/it)
+  objective:  -24691.233857897616
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)5.69110.0174327.71<1e-990.0000
CTR: sod0.07440.00789.52<1e-200.0098
CTR: dos0.04080.00478.75<1e-170.0176
CTR: dod0.00180.00560.310.75530.0275
size: small0.09210.01745.30<1e-060.1607
cardinal: diagonal0.02050.00523.91<1e-040.0297
CTR: sod & size: small0.02440.00783.120.00180.0670
CTR: dos & size: small-0.00540.0047-1.150.24910.0201
CTR: dod & size: small0.01830.00563.250.00110.0284
CTR: sod & cardinal: diagonal0.01010.00323.120.00180.0123
CTR: dos & cardinal: diagonal0.00460.00371.240.21500.0046
CTR: dod & cardinal: diagonal-0.00540.0039-1.390.16490.0096
size: small & cardinal: diagonal0.00420.00520.800.42640.0369
CTR: sod & size: small & cardinal: diagonal-0.00320.0032-0.980.32870.0092
CTR: dos & size: small & cardinal: diagonal-0.00050.0037-0.140.89250.0022
CTR: dod & size: small & cardinal: diagonal0.01110.00392.850.00440.0065
Residual0.1902
+
+
+
+
issingular(m_max)
+
+
true
+
+
+
+
only(MixedModels.PCA(m_max))
+
+

+Principal components based on correlation matrix
+ (Intercept)                                  …    .      .      .      .      .
+ CTR: sod                                          .      .      .      .      .
+ CTR: dos                                          .      .      .      .      .
+ CTR: dod                                          .      .      .      .      .
+ size: small                                       .      .      .      .      .
+ CTR: sod & size: small                       …    .      .      .      .      .
+ CTR: dos & size: small                            .      .      .      .      .
+ CTR: dod & size: small                            .      .      .      .      .
+ cardinal: diagonal                                .      .      .      .      .
+ CTR: sod & cardinal: diagonal                     .      .      .      .      .
+ CTR: dos & cardinal: diagonal                …    .      .      .      .      .
+ CTR: dod & cardinal: diagonal                    1.0     .      .      .      .
+ size: small & cardinal: diagonal                -0.33   1.0     .      .      .
+ CTR: sod & size: small & cardinal: diagonal     -0.02  -0.19   1.0     .      .
+ CTR: dos & size: small & cardinal: diagonal      0.72   0.05  -0.13   1.0     .
+ CTR: dod & size: small & cardinal: diagonal  …   0.3   -0.17   0.15  -0.09   1.0
+
+Normalized cumulative variances:
+[0.2908, 0.5455, 0.6945, 0.8011, 0.8909, 0.9407, 0.9821, 0.993, 0.9996, 0.9999, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
+
+Component loadings
+                                              …    PC13    PC14    PC15    PC16
+ (Intercept)                                     -0.0    -0.0    -0.0    -1.0
+ CTR: sod                                        -0.17   -0.15    0.11   -0.0
+ CTR: dos                                         0.31   -0.04   -0.15    0.0
+ CTR: dod                                         0.17    0.12    0.44   -0.0
+ size: small                                  …  -0.08    0.47   -0.45    0.0
+ CTR: sod & size: small                          -0.1    -0.31   -0.29    0.0
+ CTR: dos & size: small                           0.26   -0.25   -0.05    0.0
+ CTR: dod & size: small                           0.4     0.08    0.35   -0.0
+ cardinal: diagonal                              -0.01   -0.36    0.03   -0.0
+ CTR: sod & cardinal: diagonal                …  -0.05    0.11    0.2    -0.0
+ CTR: dos & cardinal: diagonal                    0.09    0.33   -0.18    0.0
+ CTR: dod & cardinal: diagonal                    0.32   -0.17   -0.53    0.0
+ size: small & cardinal: diagonal                -0.02   -0.39    0.0     0.0
+ CTR: sod & size: small & cardinal: diagonal      0.24    0.11   -0.01    0.0
+ CTR: dos & size: small & cardinal: diagonal  …  -0.3     0.33    0.0     0.0
+ CTR: dod & size: small & cardinal: diagonal     -0.58   -0.17    0.01    0.0
+
+
+
+
VarCorr(m_max)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Subj(Intercept)0.000000000.00000000
CTR: sod0.000096210.00980884+NaN
CTR: dos0.000310710.01762702+NaN-0.01
CTR: dod0.000758130.02753408+NaN+0.36+0.01
size: small0.025821990.16069223+NaN+0.89+0.14+0.57
CTR: sod & size: small0.004493140.06703088+NaN+0.11+0.24+0.45+0.50
CTR: dos & size: small0.000403500.02008728+NaN+0.31-0.65+0.64+0.21-0.11
CTR: dod & size: small0.000807500.02841649+NaN+0.56+0.38+0.47+0.77+0.80+0.00
cardinal: diagonal0.000882860.02971301+NaN-0.13+0.35+0.05-0.06-0.23-0.18-0.22
CTR: sod & cardinal: diagonal0.000151080.01229140+NaN+0.02+0.15+0.31-0.01-0.25+0.19-0.15+0.29
CTR: dos & cardinal: diagonal0.000021100.00459384+NaN-0.66+0.65-0.07-0.41+0.34-0.54+0.08+0.06-0.09
CTR: dod & cardinal: diagonal0.000092540.00961973+NaN+0.26-0.36+0.59+0.12-0.31+0.82-0.11+0.11+0.67-0.48
size: small & cardinal: diagonal0.001361590.03689977+NaN+0.13+0.33-0.13+0.07-0.04-0.21+0.11-0.46-0.18+0.32-0.33
CTR: sod & size: small & cardinal: diagonal0.000085300.00923559+NaN+0.36-0.33+0.09+0.39+0.08+0.13+0.00+0.13-0.12-0.46-0.02-0.19
CTR: dos & size: small & cardinal: diagonal0.000004980.00223067+NaN+0.12-0.28+0.34-0.09-0.53+0.72-0.28+0.20+0.27-0.28+0.72+0.05-0.13
CTR: dod & size: small & cardinal: diagonal0.000041920.00647421+NaN+0.31+0.33+0.80+0.59+0.70+0.24+0.73+0.04+0.29+0.16+0.30-0.17+0.15-0.09
Residual0.036175850.19019951
+
+
+
+
+

6 Reduction strategy 1

+
+

6.1 Zero-correlation parameter LMM (1)

+

Force CPs to zero.

+
+
m_zcp1 = let
+  form = @formula log(rt) ~ 1 + CTR * size * cardinal +
+                   zerocorr(1 + CTR * size * cardinal | Subj)
+  fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 359    Time: 0:00:00 ( 0.28 ms/it)
+  objective:  -24555.63858432091
+Minimizing 622    Time: 0:00:00 ( 0.27 ms/it)
+
+Minimizing 623    Time: 0:00:00 ( 0.27 ms/it)
+  objective:  -24588.607470697283
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)5.69100.0173329.51<1e-990.0024
CTR: sod0.07440.00779.66<1e-210.0369
CTR: dos0.04090.00459.09<1e-190.0003
CTR: dod0.00150.00530.290.77490.0012
size: small0.09210.01735.33<1e-070.1598
cardinal: diagonal0.02040.00523.92<1e-040.0159
CTR: sod & size: small0.02440.00773.170.00150.0555
CTR: dos & size: small-0.00530.0045-1.190.23440.0240
CTR: dod & size: small0.01810.00533.390.00070.0357
CTR: sod & cardinal: diagonal0.01010.00333.030.00250.0151
CTR: dos & cardinal: diagonal0.00460.00371.260.20920.0000
CTR: dod & cardinal: diagonal-0.00550.0037-1.490.13580.0016
size: small & cardinal: diagonal0.00410.00520.790.42910.0444
CTR: sod & size: small & cardinal: diagonal-0.00320.0033-0.950.34270.0082
CTR: dos & size: small & cardinal: diagonal-0.00050.0037-0.130.89450.0000
CTR: dod & size: small & cardinal: diagonal0.01090.00372.940.00330.0031
Residual0.1903
+
+
+
+
issingular(m_zcp1)
+
+
true
+
+
+
+
only(MixedModels.PCA(m_zcp1))
+
+

+Principal components based on correlation matrix
+ (Intercept)                                  …  .    .    .    .    .    .    .
+ CTR: sod                                        .    .    .    .    .    .    .
+ CTR: dos                                        .    .    .    .    .    .    .
+ CTR: dod                                        .    .    .    .    .    .    .
+ size: small                                     .    .    .    .    .    .    .
+ CTR: sod & size: small                       …  .    .    .    .    .    .    .
+ CTR: dos & size: small                          .    .    .    .    .    .    .
+ CTR: dod & size: small                          .    .    .    .    .    .    .
+ cardinal: diagonal                              .    .    .    .    .    .    .
+ CTR: sod & cardinal: diagonal                   1.0  .    .    .    .    .    .
+ CTR: dos & cardinal: diagonal                …  0.0  0.0  .    .    .    .    .
+ CTR: dod & cardinal: diagonal                   0.0  0.0  1.0  .    .    .    .
+ size: small & cardinal: diagonal                0.0  0.0  0.0  1.0  .    .    .
+ CTR: sod & size: small & cardinal: diagonal     0.0  0.0  0.0  0.0  1.0  .    .
+ CTR: dos & size: small & cardinal: diagonal     0.0  0.0  0.0  0.0  0.0  0.0  .
+ CTR: dod & size: small & cardinal: diagonal  …  0.0  0.0  0.0  0.0  0.0  0.0  1.0
+
+Normalized cumulative variances:
+[0.0714, 0.1429, 0.2143, 0.2857, 0.3571, 0.4286, 0.5, 0.5714, 0.6429, 0.7143, 0.7857, 0.8571, 0.9286, 1.0, 1.0, 1.0]
+
+Component loadings
+                                              …   PC13   PC14     PC15     PC16
+ (Intercept)                                     0.0    0.0      0.0      0.0
+ CTR: sod                                        0.0    0.0      0.0      0.0
+ CTR: dos                                        0.0    0.0      0.0      0.0
+ CTR: dod                                        1.0    0.0      0.0      0.0
+ size: small                                  …  0.0    0.0      0.0      0.0
+ CTR: sod & size: small                          0.0    1.0      0.0      0.0
+ CTR: dos & size: small                          0.0    0.0      0.0      0.0
+ CTR: dod & size: small                          0.0    0.0      0.0      0.0
+ cardinal: diagonal                              0.0    0.0      0.0      0.0
+ CTR: sod & cardinal: diagonal                …  0.0    0.0      0.0      0.0
+ CTR: dos & cardinal: diagonal                   0.0    0.0    NaN        0.0
+ CTR: dod & cardinal: diagonal                   0.0    0.0      0.0      0.0
+ size: small & cardinal: diagonal                0.0    0.0      0.0      0.0
+ CTR: sod & size: small & cardinal: diagonal     0.0    0.0      0.0      0.0
+ CTR: dos & size: small & cardinal: diagonal  …  0.0    0.0      0.0    NaN
+ CTR: dod & size: small & cardinal: diagonal     0.0    0.0      0.0      0.0
+
+
+
+
VarCorr(m_zcp1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Subj(Intercept)0.0000057180.002391288
CTR: sod0.0013598140.036875659.
CTR: dos0.0000000640.000252674..
CTR: dod0.0000014260.001193943...
size: small0.0255330200.159790549....
CTR: sod & size: small0.0030752110.055454589.....
CTR: dos & size: small0.0005781160.024044053......
CTR: dod & size: small0.0012758370.035718859.......
cardinal: diagonal0.0002522440.015882183........
CTR: sod & cardinal: diagonal0.0002277320.015090802.........
CTR: dos & cardinal: diagonal0.0000000000.000000000..........
CTR: dod & cardinal: diagonal0.0000025860.001607978...........
size: small & cardinal: diagonal0.0019739660.044429339............
CTR: sod & size: small & cardinal: diagonal0.0000674120.008210458.............
CTR: dos & size: small & cardinal: diagonal0.0000000000.000000000..............
CTR: dod & size: small & cardinal: diagonal0.0000094230.003069670...............
Residual0.0361966380.190254141
+
+
+
+
+

6.2 Reduced zcp LMM

+

Take out VC for interactions.

+
+
m_zcp1_rdc = let
+  form = @formula log(rt) ~ 1 + CTR * size * cardinal +
+                   zerocorr(1 + CTR + size + cardinal | Subj)
+  fit(MixedModel, form, dat; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)5.69110.0173329.28<1e-990.1572
CTR: sod0.07450.00779.66<1e-210.0666
CTR: dos0.04090.00459.09<1e-190.0240
CTR: dod0.00150.00530.290.77510.0357
size: small0.09200.01735.33<1e-060.0295
cardinal: diagonal0.02040.00533.880.00010.0476
CTR: sod & size: small0.02440.00773.170.0015
CTR: dos & size: small-0.00540.0045-1.200.2317
CTR: dod & size: small0.01810.00533.390.0007
CTR: sod & cardinal: diagonal0.01010.00283.640.0003
CTR: dos & cardinal: diagonal0.00460.00371.250.2125
CTR: dod & cardinal: diagonal-0.00560.0037-1.510.1317
size: small & cardinal: diagonal0.00420.00530.790.4286
CTR: sod & size: small & cardinal: diagonal-0.00310.0028-1.120.2610
CTR: dos & size: small & cardinal: diagonal-0.00050.0037-0.120.9023
CTR: dod & size: small & cardinal: diagonal0.01090.00372.950.0032
Residual0.1904
+
+
+
+
issingular(m_zcp1_rdc)
+
+
false
+
+
+
+
only(MixedModels.PCA(m_zcp1_rdc))
+
+

+Principal components based on correlation matrix
+ (Intercept)         1.0  .    .    .    .    .
+ CTR: sod            0.0  1.0  .    .    .    .
+ CTR: dos            0.0  0.0  1.0  .    .    .
+ CTR: dod            0.0  0.0  0.0  1.0  .    .
+ size: small         0.0  0.0  0.0  0.0  1.0  .
+ cardinal: diagonal  0.0  0.0  0.0  0.0  0.0  1.0
+
+Normalized cumulative variances:
+[0.1667, 0.3333, 0.5, 0.6667, 0.8333, 1.0]
+
+Component loadings
+                      PC1   PC2   PC3   PC4   PC5   PC6
+ (Intercept)         1.0   0.0   0.0   0.0   0.0   0.0
+ CTR: sod            0.0   1.0   0.0   0.0   0.0   0.0
+ CTR: dos            0.0   0.0   1.0   0.0   0.0   0.0
+ CTR: dod            0.0   0.0   0.0   1.0   0.0   0.0
+ size: small         0.0   0.0   0.0   0.0   1.0   0.0
+ cardinal: diagonal  0.0   0.0   0.0   0.0   0.0   1.0
+
+
+
+
VarCorr(m_zcp1_rdc)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Subj(Intercept)0.02470210.1571690
CTR: sod0.00444090.0666397.
CTR: dos0.00057630.0240059..
CTR: dod0.00127520.0357104...
size: small0.00087220.0295334....
cardinal: diagonal0.00226780.0476216.....
Residual0.03626010.1904207
+
+
+
+
+

6.3 Model comparison 1

+

Let’s compare the three models.

+
+
gof_summary = let
+  nms = [:m_zcp1_rdc, :m_zcp1, :m_max]
+  mods = eval.(nms)
+  lrt = MixedModels.likelihoodratiotest(m_zcp1_rdc, m_zcp1, m_max)
+  DataFrame(;
+    name = nms,
+    dof=dof.(mods),
+    deviance=round.(deviance.(mods), digits=0),
+    AIC=round.(aic.(mods),digits=0),
+    AICc=round.(aicc.(mods),digits=0),
+     BIC=round.(bic.(mods),digits=0),
+    χ²=vcat(:., round.(lrt.tests.deviancediff, digits=0)),
+    χ²_dof=vcat(:., round.(lrt.tests.dofdiff, digits=0)),
+    pvalue=vcat(:., round.(lrt.tests.pvalues, digits=3))
+  )
+end
+
+
3×9 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RownamedofdevianceAICAICcBICχ²χ²_dofpvalue
SymbolInt64Float64Float64Float64Float64AnyAnyAny
1m_zcp1_rdc23-24558.0-24512.0-24512.0-24308.0...
2m_zcp133-24589.0-24523.0-24523.0-24229.030.010.00.001
3m_max153-24691.0-24385.0-24384.0-23025.0103.0120.00.872
+
+
+
+
+
+

6.4 Parsimonious LMM (1)

+

Extend zcp-reduced LMM with CPs

+
+
m_prm1 = let
+  form = @formula log(rt) ~ 1 + CTR * size * cardinal +
+                           (1 + CTR + size + cardinal | Subj)
+  fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 756    Time: 0:00:00 ( 0.14 ms/it)
+  objective:  -24615.345521636216
+Minimizing 882    Time: 0:00:00 ( 0.15 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)5.69110.0176323.58<1e-990.1117
CTR: sod0.07450.00769.75<1e-210.0660
CTR: dos0.04080.00439.42<1e-200.0213
CTR: dod0.00170.00510.340.73740.0326
size: small0.09210.01765.23<1e-060.1184
cardinal: diagonal0.02040.00533.880.00010.0476
CTR: sod & size: small0.02440.00763.190.0014
CTR: dos & size: small-0.00540.0043-1.240.2156
CTR: dod & size: small0.01810.00513.550.0004
CTR: sod & cardinal: diagonal0.01010.00283.640.0003
CTR: dos & cardinal: diagonal0.00460.00371.250.2118
CTR: dod & cardinal: diagonal-0.00560.0037-1.510.1301
size: small & cardinal: diagonal0.00420.00530.790.4272
CTR: sod & size: small & cardinal: diagonal-0.00310.0028-1.130.2597
CTR: dos & size: small & cardinal: diagonal-0.00050.0037-0.130.8997
CTR: dod & size: small & cardinal: diagonal0.01100.00372.980.0029
Residual0.1904
+
+
+
+
issingular(m_prm1)
+
+
false
+
+
+
+
only(MixedModels.PCA(m_prm1))
+
+

+Principal components based on correlation matrix
+ (Intercept)          1.0     .      .      .      .      .
+ CTR: sod             0.68   1.0     .      .      .      .
+ CTR: dos             0.17  -0.08   1.0     .      .      .
+ CTR: dod             0.83   0.6    0.28   1.0     .      .
+ size: small         -0.42  -0.15  -0.11  -0.03   1.0     .
+ cardinal: diagonal   0.05  -0.05  -0.02   0.05   0.07   1.0
+
+Normalized cumulative variances:
+[0.4224, 0.6045, 0.7798, 0.9344, 0.9891, 1.0]
+
+Component loadings
+                       PC1    PC2    PC3    PC4    PC5    PC6
+ (Intercept)         -0.6   -0.0    0.02   0.11  -0.31   0.73
+ CTR: sod            -0.5    0.27   0.31  -0.11   0.75  -0.09
+ CTR: dos            -0.16  -0.54  -0.71  -0.23   0.35   0.07
+ CTR: dod            -0.55   0.16  -0.2   -0.26  -0.45  -0.6
+ size: small          0.24   0.58  -0.25  -0.67  -0.0    0.31
+ cardinal: diagonal  -0.01   0.52  -0.55   0.64   0.12  -0.03
+
+
+
+
VarCorr(m_prm1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Subj(Intercept)0.01247600.1116961
CTR: sod0.00435570.0659974+0.68
CTR: dos0.00045360.0212982+0.17-0.08
CTR: dod0.00106430.0326233+0.83+0.60+0.28
size: small0.01401200.1183725-0.42-0.15-0.11-0.03
cardinal: diagonal0.00226870.0476312+0.05-0.05-0.02+0.05+0.07
Residual0.03626610.1904365
+
+
+

We note that the critical correlation parameter between spatial (sod) and attraction (dod) is now estimated at .60 – not that close to the 1.0 boundary that caused singularity in Kliegl et al. (2011).

+
+
+

6.5 Model comparison 2

+
+
gof_summary = let
+  nms = [:m_zcp1_rdc, :m_prm1, :m_max]
+  mods = eval.(nms)
+  lrt = MixedModels.likelihoodratiotest(m_prm1, m_zcp1, m_max)
+  DataFrame(;
+    name = nms,
+    dof=dof.(mods),
+    deviance=round.(deviance.(mods), digits=0),
+    AIC=round.(aic.(mods),digits=0),
+    AICc=round.(aicc.(mods),digits=0),
+     BIC=round.(bic.(mods),digits=0),
+    χ²=vcat(:., round.(lrt.tests.deviancediff, digits=0)),
+    χ²_dof=vcat(:., round.(lrt.tests.dofdiff, digits=0)),
+    pvalue=vcat(:., round.(lrt.tests.pvalues, digits=3))
+  )
+end
+
+
3×9 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RownamedofdevianceAICAICcBICχ²χ²_dofpvalue
SymbolInt64Float64Float64Float64Float64AnyAnyAny
1m_zcp1_rdc23-24558.0-24512.0-24512.0-24308.0...
2m_prm138-24615.0-24539.0-24539.0-24201.027.05.00.0
3m_max153-24691.0-24385.0-24384.0-23025.076.0115.00.998
+
+
+
+
+
+
+

7 Reduction strategy 2

+
+

7.1 Complex LMM

+

Take out interaction VCs.

+
+
m_cpx = let
+  form = @formula log(rt) ~ 1 + CTR * size * cardinal +
+                           (1 + CTR + size + cardinal | Subj)
+  fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 747    Time: 0:00:00 ( 0.14 ms/it)
+  objective:  -24615.344285958156
+Minimizing 882    Time: 0:00:00 ( 0.14 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)5.69110.0176323.58<1e-990.1117
CTR: sod0.07450.00769.75<1e-210.0660
CTR: dos0.04080.00439.42<1e-200.0213
CTR: dod0.00170.00510.340.73740.0326
size: small0.09210.01765.23<1e-060.1184
cardinal: diagonal0.02040.00533.880.00010.0476
CTR: sod & size: small0.02440.00763.190.0014
CTR: dos & size: small-0.00540.0043-1.240.2156
CTR: dod & size: small0.01810.00513.550.0004
CTR: sod & cardinal: diagonal0.01010.00283.640.0003
CTR: dos & cardinal: diagonal0.00460.00371.250.2118
CTR: dod & cardinal: diagonal-0.00560.0037-1.510.1301
size: small & cardinal: diagonal0.00420.00530.790.4272
CTR: sod & size: small & cardinal: diagonal-0.00310.0028-1.130.2597
CTR: dos & size: small & cardinal: diagonal-0.00050.0037-0.130.8997
CTR: dod & size: small & cardinal: diagonal0.01100.00372.980.0029
Residual0.1904
+
+
+
+
+

7.2 Zero-correlation parameter LMM (2)

+

Take out interaction VCs.

+
+
m_zcp2 = let
+  form = @formula log(rt) ~ 1 + CTR * size * cardinal +
+                   zerocorr(1 + CTR + size + cardinal | Subj)
+  fit(MixedModel, form, dat; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)5.69110.0173329.28<1e-990.1572
CTR: sod0.07450.00779.66<1e-210.0666
CTR: dos0.04090.00459.09<1e-190.0240
CTR: dod0.00150.00530.290.77510.0357
size: small0.09200.01735.33<1e-060.0295
cardinal: diagonal0.02040.00533.880.00010.0476
CTR: sod & size: small0.02440.00773.170.0015
CTR: dos & size: small-0.00540.0045-1.200.2317
CTR: dod & size: small0.01810.00533.390.0007
CTR: sod & cardinal: diagonal0.01010.00283.640.0003
CTR: dos & cardinal: diagonal0.00460.00371.250.2125
CTR: dod & cardinal: diagonal-0.00560.0037-1.510.1317
size: small & cardinal: diagonal0.00420.00530.790.4286
CTR: sod & size: small & cardinal: diagonal-0.00310.0028-1.120.2610
CTR: dos & size: small & cardinal: diagonal-0.00050.0037-0.120.9023
CTR: dod & size: small & cardinal: diagonal0.01090.00372.950.0032
Residual0.1904
+
+
+
+
+

7.3 Model comparison 3

+
+
gof_summary = let
+  nms = [:m_zcp2, :m_cpx, :m_max]
+  mods = eval.(nms)
+  lrt = MixedModels.likelihoodratiotest(m_zcp2, m_cpx, m_max)
+  DataFrame(;
+    name = nms,
+    dof=dof.(mods),
+    deviance=round.(deviance.(mods), digits=0),
+    AIC=round.(aic.(mods),digits=0),
+    AICc=round.(aicc.(mods),digits=0),
+     BIC=round.(bic.(mods),digits=0),
+    χ²=vcat(:., round.(lrt.tests.deviancediff, digits=0)),
+    χ²_dof=vcat(:., round.(lrt.tests.dofdiff, digits=0)),
+    pvalue=vcat(:., round.(lrt.tests.pvalues, digits=3))
+  )
+end
+
+
3×9 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RownamedofdevianceAICAICcBICχ²χ²_dofpvalue
SymbolInt64Float64Float64Float64Float64AnyAnyAny
1m_zcp223-24558.0-24512.0-24512.0-24308.0...
2m_cpx38-24615.0-24539.0-24539.0-24201.057.015.00.0
3m_max153-24691.0-24385.0-24384.0-23025.076.0115.00.998
+
+
+
+
+
+
+

8 Check LMM for untransformed reaction times

+

We see the model is singular.

+
+
issingular(m_cpx_rt)
+
+
false
+
+
+
+
MixedModels.PCA(m_cpx_rt)
+
+
(Subj = 
+Principal components based on correlation matrix
+ (Intercept)          1.0     .      .      .      .     .
+ CTR: sod             0.85   1.0     .      .      .     .
+ CTR: dos             0.57   0.19   1.0     .      .     .
+ CTR: dod             0.69   0.54   0.3    1.0     .     .
+ size: small         -0.45  -0.24  -0.2   -0.14   1.0    .
+ cardinal: diagonal  -0.07  -0.05  -0.08   0.12   0.2   1.0
+
+Normalized cumulative variances:
+[0.4663, 0.6647, 0.8027, 0.9256, 0.9972, 1.0]
+
+Component loadings
+                       PC1    PC2    PC3    PC4    PC5    PC6
+ (Intercept)         -0.59  -0.01   0.02  -0.02  -0.17  -0.79
+ CTR: sod            -0.49  -0.12   0.43  -0.22  -0.51   0.5
+ CTR: dos            -0.35   0.12  -0.87  -0.05  -0.15   0.27
+ CTR: dod            -0.45  -0.36   0.05  -0.13   0.79   0.17
+ size: small          0.29  -0.5   -0.17  -0.77  -0.14  -0.17
+ cardinal: diagonal   0.05  -0.77  -0.12   0.58  -0.21   0.01,)
+
+
+
+
+

9 Other checks

+
+
m_prm1.θ
+m_prm1.lowerbd
+m_prm1.λ
+
+
1-element Vector{LinearAlgebra.LowerTriangular{Float64, Matrix{Float64}}}:
+ [0.5865267168508366 0.0 … 0.0 0.0; 0.2340167850124924 0.25561457444461316 … 0.0 0.0; … ; -0.26413464224340244 0.11690271242753608 … 0.41140541039774686 0.0; 0.012095155354957103 -0.027858557605851694 … 0.034863559031980046 0.24512741354973927]
+
+
+
+
+

10 Diagnostic plots of LMM residuals

+

Do model residuals meet LMM assumptions? Classic plots are

+
    +
  • Residual over fitted
    • +
  • Quantiles of model residuals over theoretical quantiles of normal distribution
    • +
+
+

10.1 Residual-over-fitted plot

+

The slant in residuals show a lower and upper boundary of reaction times, that is we have have too few short and too few long residuals. Not ideal, but at least width of the residual band looks similar across the fitted values, that is there is no evidence for heteroskedasticity.

+
+
+Code +
CairoMakie.activate!(; type="png")
+scatter(fitted(m_prm1), residuals(m_prm1); alpha=0.3)
+
+
+
+
+ +
+
+Figure 3: Residuals versus fitted values for model m1 +
+
+
+
+

With many observations the scatterplot is not that informative. Contour plots or heatmaps may be an alternative.

+
+
+Code +
set_aog_theme!()
+draw(
+  data((; f=fitted(m_prm1), r=residuals(m_prm1))) *
+  mapping(
+    :f => "Fitted values from m1", :r => "Residuals from m1"
+  ) *
+  density();
+)
+
+
+
+
+ +
+
+Figure 4: Heatmap of residuals versus fitted values for model m1 +
+
+
+
+
+
+

10.2 Q-Q plot

+

The plot of quantiles of model residuals over corresponding quantiles of the normal distribution should yield a straight line along the main diagonal.

+
+
+Code +
CairoMakie.activate!(; type="png")
+qqnorm(
+  residuals(m_prm1);
+  qqline=:none,
+  axis=(;
+    xlabel="Standard normal quantiles",
+    ylabel="Quantiles of the residuals from model m1",
+  ),
+)
+
+
+
+
+ +
+
+Figure 5: Quantile-quantile plot of the residuals for model m1 versus a standard normal +
+
+
+
+
+
+

10.3 Observed and theoretical normal distribution

+

******We****** can see this in this plot. Overall, it does not look too bad.

+
+
+Code +
CairoMakie.activate!(; type="svg")
+let
+  n = nrow(dat)
+  dat_rz = (;
+    value=vcat(residuals(m_prm1) ./ std(residuals(m_prm1)), randn(n)),
+    curve=repeat(["residual", "normal"]; inner=n),
+  )
+  draw(
+    data(dat_rz) *
+    mapping(:value; color=:curve) *
+    density(; bandwidth=0.1);
+  )
+end
+
+
+
+
+
+ +
+
+Figure 6: Kernel density plot of the standardized residuals for model m1 versus a standard normal +
+
+
+
+
+
+
+
+

11 Conditional modes

+
+

11.1 Caterpillar plot

+
+
+Code +
cm1 = only(ranefinfo(m_prm1))
+caterpillar!(Figure(; resolution=(800, 1200)), cm1; orderby=2)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 7: Prediction intervals of the subject random effects in model m1 +
+
+
+
+
+
+
+

11.2 Shrinkage plot

+
+

11.2.1 Log-transformed reaction times (LMM m_prm1)

+
+
+Code +
shrinkageplot!(Figure(; resolution=(1000, 1200)), m_prm1)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 8: Shrinkage plots of the subject random effects in model m1L +
+
+
+
+
+
+
+
+
+

12 Parametric bootstrap

+

Here we

+
    +
  • generate a bootstrap sample
    • +
  • compute shortest covergage intervals for the LMM parameters
    • +
  • plot densities of bootstrapped parameter estimates for residual, fixed effects, variance components, and correlation parameters
    • +
+
+

12.1 Generate a bootstrap sample

+

We generate 2500 samples for the 15 model parameters (4 fixed effect, 7 VCs, 15 CPs, and 1 residual).

+
+
samp = parametricbootstrap(MersenneTwister(1234321), 2500, m_prm1;
+                           optsum_overrides=(; ftol_rel=1e-8));
+
+
+
tbl = samp.tbl
+
+
Table with 60 columns and 2500 rows:
+      obj       β01      β02        β03        β04           β05        ⋯
+    ┌────────────────────────────────────────────────────────────────────
+ 1  │ -25134.3  5.68049  0.0689586  0.0393162  -0.00115769   0.0780895  ⋯
+ 2  │ -24466.7  5.66099  0.0666992  0.0419148  -0.00303919   0.101403   ⋯
+ 3  │ -24448.9  5.69085  0.0782739  0.0370164  0.00841092    0.0970059  ⋯
+ 4  │ -24614.9  5.6972   0.0762013  0.0462014  -0.00324924   0.0631257  ⋯
+ 5  │ -25168.8  5.68336  0.068479   0.0386334  0.00257696    0.117804   ⋯
+ 6  │ -24729.2  5.69257  0.0843594  0.0442014  0.00172628    0.0942692  ⋯
+ 7  │ -24499.0  5.67623  0.0705523  0.0418438  -0.00636468   0.0768522  ⋯
+ 8  │ -24804.4  5.69366  0.0765241  0.0457697  0.00292816    0.0869247  ⋯
+ 9  │ -24396.7  5.68366  0.0814744  0.0495446  -0.00164492   0.101197   ⋯
+ 10 │ -24577.8  5.6861   0.0794278  0.0344818  0.00620175    0.0818131  ⋯
+ 11 │ -25086.6  5.6658   0.0662387  0.0445268  -0.000810468  0.0947995  ⋯
+ 12 │ -24364.5  5.67832  0.0697527  0.0446342  -0.000982085  0.102116   ⋯
+ 13 │ -24972.8  5.6935   0.0729605  0.0421222  -0.00304358   0.0842434  ⋯
+ 14 │ -24616.6  5.70284  0.0766981  0.0432196  -0.00502166   0.0714031  ⋯
+ 15 │ -24772.4  5.68248  0.0825491  0.0371784  0.00389937    0.088786   ⋯
+ 16 │ -24632.9  5.69897  0.0718701  0.0408715  0.000512973   0.0851757  ⋯
+ 17 │ -24397.0  5.70738  0.0970143  0.0363716  0.00149944    0.0856153  ⋯
+ ⋮  │    ⋮         ⋮         ⋮          ⋮           ⋮            ⋮      ⋱
+
+
+
+
+

12.2 Shortest coverage interval

+
+
confint(samp)
+
+
DictTable with 2 columns and 38 rows:
+ par   lower        upper
+ ────┬────────────────────────
+ β01 │ 5.65513      5.72404
+ β02 │ 0.06039      0.0904927
+ β03 │ 0.0325719    0.0489381
+ β04 │ -0.00815893  0.0115762
+ β05 │ 0.0580068    0.124695
+ β06 │ 0.0104524    0.0312924
+ β07 │ 0.00972538   0.0388313
+ β08 │ -0.0138485   0.00313372
+ β09 │ 0.00790021   0.0275058
+ β10 │ 0.00485266   0.0157063
+ β11 │ -0.00250178  0.0117922
+ β12 │ -0.0124778   0.00200615
+ β13 │ -0.00597154  0.0135989
+ β14 │ -0.00828454  0.0022754
+ β15 │ -0.00784482  0.00657597
+ β16 │ 0.00368085   0.0180065
+ ρ01 │ 0.38489      1.0
+  ⋮  │      ⋮           ⋮
+
+
+

We can also visualize the shortest coverage intervals for fixed effects with the ridgeplot() command:

+
+
+Code +
ridgeplot(samp; show_intercept=false)
+
+
+
+
+
+ +
+
+Figure 9: Ridge plot of fixed-effects bootstrap samples from model m1L +
+
+
+
+
+
+
+

12.3 Comparative density plots of bootstrapped parameter estimates

+
+

12.3.1 Residual

+
+
+Code +
draw(
+  data(tbl) *
+  mapping(:σ => "Residual") *
+  density();
+  figure=(; resolution=(800, 400)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 10: Kernel density estimate from bootstrap samples of the residual standard deviation for model m_prm1 +
+
+
+
+
+
+
+

12.3.2 Fixed effects and associated variance components (w/o GM)

+

The shortest coverage interval for the GM ranges from x to x ms and the associate variance component from .x to .x. To keep the plot range small we do not include their densities here.

+
+
+Code +
rn = renamer([
+  "(Intercept)" => "GM",
+  "CTR: sod" => "spatial effect",
+  "CTR: dos" => "object effect",
+  "CTR: dod" => "attraction effect",
+  "(Intercept), CTR: sod" => "GM, spatial",
+  "(Intercept), CTR: dos" => "GM, object",
+  "CTR: sod, CTR: dos" => "spatial, object",
+  "(Intercept), CTR: dod" => "GM, attraction",
+  "CTR: sod, CTR: dod" => "spatial, attraction",
+  "CTR: dos, CTR: dod" => "object, attraction",
+])
+draw(
+  data(tbl) *
+  mapping(
+    [:β02, :β03, :β04] .=> "Experimental effect size [ms]";
+    color=dims(1) =>
+    renamer(["spatial effect", "object effect", "attraction effect"]) =>
+    "Experimental effects",
+  ) *
+  density();
+  figure=(; resolution=(800, 350)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 11: Kernel density estimate from bootstrap samples of the fixed effects for model m_prm1 +
+
+
+
+
+

The densitiies correspond nicely with the shortest coverage intervals.

+
+
+Code +
draw(
+  data(tbl) *
+  mapping(
+    [:σ2, :σ3, :σ4] .=> "Standard deviations [ms]";
+    color=dims(1) =>
+    renamer(["spatial effect", "object effect", "attraction effect"]) =>
+    "Variance components",
+  ) *
+  density();
+  figure=(; resolution=(800, 350)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 12: Kernel density estimate from bootstrap samples of the standard deviations for model m1L (excluding Grand Mean) +
+
+
+
+
+

The VC are all very nicely defined.

+
+
+

12.3.3 Correlation parameters (CPs)

+
+
+Code +
draw(
+  data(tbl) *
+  mapping(
+    [:ρ01, :ρ02, :ρ03, :ρ04, :ρ05, :ρ06] .=> "Correlation";
+    color=dims(1) =>
+    renamer(["GM, spatial", "GM, object", "spatial, object",
+    "GM, attraction", "spatial, attraction", "object, attraction"]) =>
+    "Correlation parameters",
+  ) *
+  density();
+  figure=(; resolution=(800, 350)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 13: Kernel density estimate from bootstrap samples of the standard deviations for model m1L +
+
+
+
+
+

Three CPs stand out positively, the correlation between GM and the spatial effect, GM and attraction effect, and the correlation between spatial and attraction effects. The second CP was positive, but not significant in the first study. The third CP replicates a CP that was judged questionable in script kwdyz11.jl.

+

The three remaining CPs are not well defined for log-transformed reaction times; they only fit noise and should be removed. It is also possible that fitting the complex experimental design (including target size and rectangle orientation) will lead to more acceptable estimates. The corresponding plot based on LMM m1_rt for raw reaction times still shows them with very wide distributions, but acceptable.

+
+
+
+
+

13 References

+
+
+Baayen, H., Vasishth, S., Kliegl, R., & Bates, D. (2017). The cave of shadows: Addressing the human factor with generalized additive mixed models. Journal of Memory and Language, 94, 206–234. https://doi.org/10.1016/j.jml.2016.11.006 +
+
+Bates, D., Kliegl, R., Vasishth, S., & Baayen, H. (2015). Parsimonious mixed models. arXiv. https://doi.org/10.48550/ARXIV.1506.04967 +
+
+Kliegl, R., Kushela, J., & Laubrock, J. (2015). Object orientation and target size modulate the speed of visual attention. Department of Psychology, University of Potsdam. +
+
+Kliegl, R., Wei, P., Dambacher, M., Yan, M., & Zhou, X. (2011). Experimental effects and individual differences in linear mixed models: Estimating the relationship between spatial, object, and attraction effects in visual attention. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2010.00238 +
+
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/kkl15_files/figure-html/fig-betadensitym1-output-2.svg b/kkl15_files/figure-html/fig-betadensitym1-output-2.svg new file mode 100644 index 0000000..2d533be --- /dev/null +++ b/kkl15_files/figure-html/fig-betadensitym1-output-2.svg @@ -0,0 +1,643 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kkl15_files/figure-html/fig-bsridgem1-output-1.svg b/kkl15_files/figure-html/fig-bsridgem1-output-1.svg new file mode 100644 index 0000000..9567ca3 --- /dev/null +++ b/kkl15_files/figure-html/fig-bsridgem1-output-1.svg @@ -0,0 +1,2199 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kkl15_files/figure-html/fig-bxpltsubjcond-output-2.svg b/kkl15_files/figure-html/fig-bxpltsubjcond-output-2.svg new file mode 100644 index 0000000..2d1b87f --- /dev/null +++ b/kkl15_files/figure-html/fig-bxpltsubjcond-output-2.svg @@ -0,0 +1,617 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kkl15_files/figure-html/fig-bxpltsubjcond2-output-2.svg b/kkl15_files/figure-html/fig-bxpltsubjcond2-output-2.svg new file mode 100644 index 0000000..877b29b --- /dev/null +++ b/kkl15_files/figure-html/fig-bxpltsubjcond2-output-2.svg @@ -0,0 +1,607 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kkl15_files/figure-html/fig-caterpillarm1-output-2.svg b/kkl15_files/figure-html/fig-caterpillarm1-output-2.svg new file mode 100644 index 0000000..957b4b7 --- /dev/null +++ b/kkl15_files/figure-html/fig-caterpillarm1-output-2.svg @@ -0,0 +1,4399 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kkl15_files/figure-html/fig-caterpillarm1l-output-2.svg b/kkl15_files/figure-html/fig-caterpillarm1l-output-2.svg new file mode 100644 index 0000000..c470758 --- /dev/null +++ b/kkl15_files/figure-html/fig-caterpillarm1l-output-2.svg @@ -0,0 +1,6401 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kkl15_files/figure-html/fig-corrdensitym1-output-2.svg b/kkl15_files/figure-html/fig-corrdensitym1-output-2.svg new file mode 100644 index 0000000..90c475c --- /dev/null +++ b/kkl15_files/figure-html/fig-corrdensitym1-output-2.svg @@ -0,0 +1,785 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kkl15_files/figure-html/fig-sigmadensitym1-output-2.svg b/kkl15_files/figure-html/fig-sigmadensitym1-output-2.svg new file mode 100644 index 0000000..ffda9ab --- /dev/null +++ b/kkl15_files/figure-html/fig-sigmadensitym1-output-2.svg @@ -0,0 +1,236 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kkl15_files/figure-html/fig-sigmasdensitym1-output-2.svg b/kkl15_files/figure-html/fig-sigmasdensitym1-output-2.svg new file mode 100644 index 0000000..ecbb366 --- /dev/null +++ b/kkl15_files/figure-html/fig-sigmasdensitym1-output-2.svg @@ -0,0 +1,658 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kkl15_files/figure-html/fig-stdresidm1dens-output-1.svg b/kkl15_files/figure-html/fig-stdresidm1dens-output-1.svg new file mode 100644 index 0000000..75a67d6 --- /dev/null +++ b/kkl15_files/figure-html/fig-stdresidm1dens-output-1.svg @@ -0,0 +1,262 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kwdyz11.html b/kwdyz11.html new file mode 100644 index 0000000..6f1527a --- /dev/null +++ b/kwdyz11.html @@ -0,0 +1,2030 @@ + + + + + + + + + + + +RePsychLing Kliegl et al. (2011) – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

RePsychLing Kliegl et al. (2011)

+
+ + + +
+ +
+
Author
+
+

Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +
+

1 Background

+

We take the kwdyz11 dataset (Kliegl et al., 2011) from an experiment looking at three effects of visual cueing under four different cue-target relations (CTRs). Two horizontal rectangles are displayed above and below a central fixation point or they displayed in vertical orientation to the left and right of the fixation point. Subjects react to the onset of a small visual target occurring at one of the four ends of the two rectangles. The target is cued validly on 70% of trials by a brief flash of the corner of the rectangle at which it appears; it is cued invalidly at the three other locations 10% of the trials each.

+

We specify three contrasts for the four-level factor CTR that are derived from spatial, object-based, and attractor-like features of attention. They map onto sequential differences between appropriately ordered factor levels. At the level of fixed effects, there is the noteworthy result, that the attraction effect was estimated at 2 ms, that is clearly not significant. Nevertheless, there was a highly reliable variance component (VC) estimated for this effect. Moreover, the reliable individual differences in the attraction effect were negatively correlated with those in the spatial effect.

+

Unfortunately, a few years after the publication, we determined that the reported LMM is actually singular and that the singularity is linked to a theoretically critical correlation parameter (CP) between the spatial effect and the attraction effect. Fortunately, there is also a larger dataset kkl15 from a replication and extension of this study (Kliegl et al., 2015), analyzed with kkl15.jl notebook. The critical CP (along with other fixed effects and CPs) was replicated in this study.

+

A more comprehensive analysis was reported in the parsimonious mixed-model paper (Bates et al., 2015). Data and R scripts are also available in R-package RePsychLing. In this and the complementary kkl15.jl scripts, we provide some corresponding analyses with MixedModels.jl.

+
+
+

2 Packages

+
+
+Code +
using AlgebraOfGraphics
+using AlgebraOfGraphics: density
+using CairoMakie
+using CategoricalArrays
+using Chain
+using DataFrames
+using DataFrameMacros
+using Distributions
+using MixedModels
+using MixedModelsMakie
+using Random
+using SMLP2024: dataset
+using StatsBase
+
+CairoMakie.activate!(; type="svg")
+
+import ProgressMeter
+ProgressMeter.ijulia_behavior(:clear)
+
+
+
+
+

3 Read data, compute and plot densities and means

+
+
dat = DataFrame(dataset(:kwdyz11))
+describe(dat)
+
+
5×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1ItemI002I6030String
2SubjS01S610String
3CTRdodval0String
4dirhorver0String
5rt370.426150.1358.6705.70Float32
+
+
+
+

We recommend to code the levels/units of random factor / grouping variable not as a number, but as a string starting with a letter and of the same length for all levels/units.

+

We also recommend to sort levels of factors into a meaningful order, that is overwrite the default alphabetic ordering. This is also a good place to choose alternative names for variables in the context of the present analysis.

+

The LMM analysis is based on log-transformed reaction times lrt, indicated by a boxcox() check of model residuals. With the exception of diagnostic plots of model residuals, the analysis of untransformed reaction times did not lead to different results and exhibited the same problems of model identification (see Kliegl et al., 2011).

+

Comparative density plots of all response times by cue-target relation, Figure 1, show the times for valid cues to be faster than for the other conditions.

+
+
+Code +
draw(
+  data(dat) *
+  mapping(
+    :rt => log => "log(Reaction time [ms])";
+    color=:CTR =>
+      renamer("val" => "valid cue", "sod" => "some obj/diff pos", "dos" => "diff obj/same pos", "dod" => "diff obj/diff pos") => "Cue-target relation",
+  ) *
+  density(),
+)
+
+
+
+
+
+ +
+
+Figure 1: Comparative density plots of log reaction time for different cue-target relations. +
+
+
+
+
+

An alternative visualization without overlap of the conditions can be accomplished with ridge plots.

+

To be done

+

For the next set of plots we average subjects’ data within the four experimental conditions. This table could be used as input for a repeated-measures ANOVA.

+
+
dat_subj = combine(
+  groupby(dat, [:Subj, :CTR]),
+  :rt => length => :n,
+  :rt => mean => :rt_m,
+  :rt => (r -> mean(log, r)) => :lrt_m,
+)
+dat_subj.CTR = categorical(dat_subj.CTR, levels=levels(dat.CTR))
+dat_subj
+
+
244×5 DataFrame
219 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowSubjCTRnrt_mlrt_m
StringCat…Int64Float32Float32
1S01val330413.3326.01272
2S01sod48437.7216.0682
3S01dos47443.3666.08269
4S01dod45434.3166.06079
5S02val333365.8995.87511
6S02sod47396.1495.95916
7S02dos46439.4876.06949
8S02dod48441.0426.06883
9S03val336371.4465.90489
10S03sod46446.8546.09464
11S03dos48471.3026.14287
12S03dod47476.5326.15706
13S04val336403.1815.99124
⋮⋮⋮⋮⋮⋮
233S59val330313.0325.72433
234S59sod46304.8525.70367
235S59dos48342.6125.80927
236S59dod47308.1915.71206
237S60val330372.8355.90832
238S60sod48382.0875.93183
239S60dos44395.1095.96918
240S60dod47387.6895.95208
241S61val320312.3745.72542
242S61sod45383.0025.92798
243S61dos45357.9985.85563
244S61dod46391.0875.95344
+
+
+
+
+
+Code +
boxplot(
+  dat_subj.CTR.refs,
+  dat_subj.lrt_m;
+  orientation=:horizontal,
+  show_notch=true,
+  axis=(;
+    yticks=(
+      1:4,
+      [
+        "valid cue",
+        "same obj/diff pos",
+        "diff obj/same pos",
+        "diff obj/diff pos",
+      ],
+    ),
+  ),
+  figure=(; resolution=(800, 300)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 2: Comparative boxplots of log response time by cue-target relation. +
+
+
+
+
+

Mean of log reaction times for four cue-target relations. Targets appeared at (a) the cued position (valid) in a rectangle, (b) in the same rectangle cue, but at its other end, (c) on the second rectangle, but at a corresponding horizontal/vertical physical distance, or (d) at the other end of the second rectangle, that is \(\sqrt{2}\) of horizontal/vertical distance diagonally across from the cue, that is also at larger physical distance compared to (c).

+

A better alternative to the boxplot is a dotplot. It also displays subjects’ condition means.

+

To be done

+
+
+

4 Linear mixed model

+
+
contrasts = Dict(
+  :CTR => SeqDiffCoding(; levels=["val", "sod", "dos", "dod"]),
+  :Subj => Grouping(),
+)
+m1 = let
+  form = @formula(log(rt) ~ 1 + CTR + (1 + CTR | Subj))
+  fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 91    Time: 0:00:00 ( 1.10 ms/it)
+  objective:  -12777.832370442246
+Minimizing 234    Time: 0:00:00 ( 3.59 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)5.93580.0185320.53<1e-990.1441
CTR: sod0.08780.008410.48<1e-240.0582
CTR: dos0.03660.00625.92<1e-080.0274
CTR: dod-0.00860.0060-1.430.15150.0249
Residual0.1920
+
+
+
+
VarCorr(m1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
Subj(Intercept)0.02076520.1441013
CTR: sod0.00338510.0581813+0.48
CTR: dos0.00075290.0274393-0.24-0.15
CTR: dod0.00062220.0249430+0.30+0.93-0.43
Residual0.03685430.1919748
+
+
+
+
issingular(m1)
+
+
true
+
+
+

LMM m1 is not fully supported by the data; it is overparameterized. This is also visible in the PCA: only three, not four PCS are needed to account for all the variance and covariance in the random-effect structure. The problem is the +.93 CP for spatial sod and attraction dod effects.

+
+
first(MixedModels.PCA(m1))
+
+

+Principal components based on correlation matrix
+ (Intercept)   1.0     .      .      .
+ CTR: sod      0.48   1.0     .      .
+ CTR: dos     -0.24  -0.15   1.0     .
+ CTR: dod      0.3    0.93  -0.43   1.0
+
+Normalized cumulative variances:
+[0.5887, 0.8095, 1.0, 1.0]
+
+Component loadings
+                PC1   PC2    PC3    PC4
+ (Intercept)  -0.4   0.04   0.9    0.17
+ CTR: sod     -0.6   0.4   -0.16  -0.68
+ CTR: dos      0.33  0.91   0.06   0.23
+ CTR: dod     -0.61  0.08  -0.41   0.68
+
+
+
+
+

5 Diagnostic plots of LMM residuals

+

Do model residuals meet LMM assumptions? Classic plots are

+
    +
  • Residual over fitted
    • +
  • Quantiles of model residuals over theoretical quantiles of normal distribution
    • +
+
+

5.1 Residual-over-fitted plot

+

The slant in residuals show a lower and upper boundary of reaction times, that is we have have too few short and too few long residuals. Not ideal, but at least width of the residual band looks similar across the fitted values, that is there is no evidence for heteroskedasticity.

+
+
+Code +
CairoMakie.activate!(; type="png")
+set_aog_theme!()
+draw(
+  data((; f=fitted(m1), r=residuals(m1))) *
+  mapping(:f => "Fitted values", :r => "Residual from model m1") *
+  visual(Scatter);
+)
+
+
+
+
+ +
+
+Figure 3: Residuals versus the fitted values for model m1 of the log response time. +
+
+
+
+

With many observations the scatterplot is not that informative. Contour plots or heatmaps may be an alternative.

+
+
+Code +
CairoMakie.activate!(; type="png")
+draw(
+  data((; f=fitted(m1), r=residuals(m1))) *
+  mapping(
+    :f => "Fitted log response time", :r => "Residual from model m1"
+  ) *
+  density();
+)
+
+
+
+
+ +
+
+Figure 4: Heatmap of residuals versus fitted values for model m1 +
+
+
+
+
+
+

5.2 Q-Q plot

+

The plot of quantiles of model residuals over corresponding quantiles of the normal distribution should yield a straight line along the main diagonal.

+
+
qqnorm(residuals(m1); qqline=:none)
+
+ +
+
+
+
+

5.3 Observed and theoretical normal distribution

+

The violation of expectation is again due to the fact that the distribution of residuals is much narrower than expected from a normal distribution, as shown in Figure 5. Overall, it does not look too bad.

+
+
+Code +
let
+  n = nrow(dat)
+  dat_rz = DataFrame(;
+    value=vcat(residuals(m1) ./ std(residuals(m1)), randn(n)),
+    curve=vcat(fill.("residual", n), fill.("normal", n)),
+  )
+  draw(
+    data(dat_rz) *
+    mapping(:value => "Standardized residuals"; color=:curve) *
+    density(; bandwidth=0.1);
+  )
+end
+
+
+
+
+ +
+
+Figure 5: Kernel density plot of the standardized residuals from model m1 compared to a Gaussian +
+
+
+
+
+
+
+

6 Conditional modes

+

Now we move on to visualizations that are based on model parameters and subjects’ data, that is “predictions” of the LMM for subject’s GM and experimental effects. Three important plots are

+
    +
  • Overlay
    • +
  • Caterpillar
    • +
  • Shrinkage
    • +
+
+

6.1 Overlay

+

The first plot overlays shrinkage-corrected conditional modes of the random effects with within-subject-based and pooled GMs and experimental effects.

+

To be done

+
+
+

6.2 Caterpillar plot

+

The caterpillar plot, Figure 6, also reveals the high correlation between spatial sod and attraction dod effects.

+
+
+Code +
caterpillar!(
+  Figure(; resolution=(800, 1000)), ranefinfo(m1, :Subj); orderby=2
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+ +
+
+Figure 6: Prediction intervals on the random effects for Subj in model m1 +
+
+
+
+
+
+

6.3 Shrinkage plot

+

Figure 7 provides more evidence for a problem with the visualization of the spatial sod and attraction dod CP. The corresponding panel illustrates an implosion of conditional modes.

+
+
+Code +
shrinkageplot!(Figure(; resolution=(1000, 1000)), m1)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+ +
+
+Figure 7: Shrinkage plot of the conditional means of the random effects for model m1 +
+
+
+
+
+
+
+

7 Parametric bootstrap

+

Here we

+
    +
  • generate a bootstrap sample
    • +
  • compute shortest covergage intervals for the LMM parameters
    • +
  • plot densities of bootstrapped parameter estimates for residual, fixed effects, variance components, and correlation parameters
    • +
+
+

7.1 Generate a bootstrap sample

+

We generate 2500 samples for the 15 model parameters (4 fixed effect, 4 VCs, 6 CPs, and 1 residual).

+
+
+Code +
Random.seed!(1234321)
+samp = parametricbootstrap(2500, m1)
+tbl = samp.tbl
+
+
+
Table with 26 columns and 2500 rows:
+      obj       β1       β2         β3         β4            σ         ⋯
+    ┌───────────────────────────────────────────────────────────────────
+ 1  │ -12802.3  5.93392  0.0864217  0.0488879  -0.0121845    0.192002  ⋯
+ 2  │ -12794.3  5.95776  0.0989845  0.0318746  -0.00666059   0.191977  ⋯
+ 3  │ -12930.6  5.93669  0.0885609  0.0209795  -0.000656525  0.191496  ⋯
+ 4  │ -12946.5  5.93165  0.0767126  0.0343558  -0.00499489   0.191438  ⋯
+ 5  │ -12677.5  5.91938  0.098459   0.0272958  -0.0055567    0.19247   ⋯
+ 6  │ -12869.7  5.96313  0.104458   0.0321628  -0.00719279   0.191751  ⋯
+ 7  │ -12478.1  5.9328   0.0947867  0.0409709  -0.00461593   0.193076  ⋯
+ 8  │ -12463.7  5.94439  0.0977167  0.0321098  -0.00625436   0.193115  ⋯
+ 9  │ -12836.2  5.9728   0.0971358  0.0416612  6.09081e-5    0.191843  ⋯
+ 10 │ -12932.3  5.94709  0.0742454  0.0505267  -0.0131469    0.191583  ⋯
+ 11 │ -12736.2  5.93625  0.0728171  0.0479423  -0.0183166    0.192193  ⋯
+ 12 │ -13048.0  5.89421  0.0680593  0.0481657  -0.0214161    0.191101  ⋯
+ 13 │ -13313.8  5.92608  0.0892317  0.0473894  -0.0141683    0.190142  ⋯
+ 14 │ -12575.0  5.89235  0.0846161  0.0341537  -0.00613088   0.192728  ⋯
+ 15 │ -12833.1  5.94217  0.0774875  0.0422339  -0.00279008   0.191772  ⋯
+ 16 │ -12504.5  5.94385  0.0898021  0.0429785  -0.00705666   0.192833  ⋯
+ 17 │ -12875.5  5.96624  0.0976104  0.0370803  -0.00071879   0.191577  ⋯
+ ⋮  │    ⋮         ⋮         ⋮          ⋮           ⋮           ⋮      ⋱
+
+
+
+
+

7.2 Shortest coverage interval

+

The upper limit of the interval for the critical CP CTR: sod, CTR: dod is hitting the upper wall of a perfect correlation. This is evidence of singularity. The other intervals do not exhibit such pathologies; they appear to be ok.

+
+
+Code +
confint(samp)
+
+
+
DictTable with 2 columns and 15 rows:
+ par   lower       upper
+ ────┬───────────────────────
+ β1  │ 5.89917     5.97256
+ β2  │ 0.0719299   0.104589
+ β3  │ 0.0251177   0.0491665
+ β4  │ -0.0207178  0.00268306
+ ρ1  │ 0.243773    0.713001
+ ρ2  │ -0.967759   0.195359
+ ρ3  │ -0.643591   0.552114
+ ρ4  │ -0.132618   0.739806
+ ρ5  │ 0.573728    0.999995
+ ρ6  │ -0.89138    0.436214
+ σ   │ 0.190559    0.193644
+ σ1  │ 0.116566    0.168565
+ σ2  │ 0.0445663   0.0698869
+ σ3  │ 0.00946878  0.0409758
+ σ4  │ 0.0133055   0.0376021
+
+
+
+
+

7.3 Comparative density plots of bootstrapped parameter estimates

+
+
+

7.4 Residual

+
+
+Code +
draw(
+  data(tbl) *
+  mapping(:σ => "Residual standard deviation") *
+  density();
+)
+
+
+
+
+ +
+
+Figure 8 +
+
+
+
+
+
+

7.5 Fixed effects (w/o GM)

+

The shortest coverage interval for the GM ranges from 376 to 404 ms. To keep the plot range small we do not include its density here.

+
+
+Code +
labels = [
+  "CTR: sod" => "spatial effect",
+  "CTR: dos" => "object effect",
+  "CTR: dod" => "attraction effect",
+  "(Intercept)" => "grand mean",
+]
+draw(
+  data(tbl) *
+  mapping(
+    [:β2, :β3, :β4] .=> "Experimental effect size [ms]";
+    color=dims(1) => renamer(["spatial", "object", "attraction"] .* " effect") =>
+    "Experimental effects",
+  ) *
+  density();
+)
+
+
+
+
+ +
+
+Figure 9: Comparative density plots of the fixed-effects parameters for model m1 +
+
+
+
+

The densitiies correspond nicely with the shortest coverage intervals.

+
+
+

7.6 Variance components (VCs)

+
+
+Code +
draw(
+  data(tbl) *
+  mapping(
+    [:σ1, :σ2, :σ3, :σ4] .=> "Standard deviations [ms]";
+    color=dims(1) =>
+    renamer(append!(["Grand mean"],["spatial", "object", "attraction"] .* " effect")) =>
+    "Variance components",
+  ) *
+  density();
+)
+
+
+
+
+ +
+
+Figure 10: Comparative density plots of the variance components for model m1 +
+
+
+
+

The VC are all very nicely defined.

+
+
+

7.7 Correlation parameters (CPs)

+
+
+Code +
let
+  labels = [
+    "(Intercept), CTR: sod" => "GM, spatial",
+    "(Intercept), CTR: dos" => "GM, object",
+    "CTR: sod, CTR: dos" => "spatial, object",
+    "(Intercept), CTR: dod" => "GM, attraction",
+    "CTR: sod, CTR: dod" => "spatial, attraction",
+    "CTR: dos, CTR: dod" => "object, attraction",
+  ]
+  draw(
+    data(tbl) *
+    mapping(
+      [:ρ1, :ρ2, :ρ3, :ρ4, :ρ5, :ρ6] .=> "Correlation";
+      color=dims(1) => renamer(last.(labels)) => "Correlation parameters",
+    ) *
+    density();
+  )
+end
+
+
+
+
+ +
+
+Figure 11: Comparative density plots of the correlation parameters for model m1 +
+
+
+
+

Two of the CPs stand out positively. First, the correlation between GM and the spatial effect is well defined. Second, as discussed throughout this script, the CP between spatial and attraction effect is close to the 1.0 border and clearly not well defined. Therefore, this CP will be replicated with a larger sample in script kkl15.jl (Kliegl et al., 2015).

+
+
+
+

8 References

+
+
+Bates, D., Kliegl, R., Vasishth, S., & Baayen, H. (2015). Parsimonious mixed models. arXiv. https://doi.org/10.48550/ARXIV.1506.04967 +
+
+Kliegl, R., Kushela, J., & Laubrock, J. (2015). Object orientation and target size modulate the speed of visual attention. Department of Psychology, University of Potsdam. +
+
+Kliegl, R., Wei, P., Dambacher, M., Yan, M., & Zhou, X. (2011). Experimental effects and individual differences in linear mixed models: Estimating the relationship between spatial, object, and attraction effects in visual attention. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2010.00238 +
+
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/kwdyz11_files/figure-html/fig-comparativedensity-output-1.svg b/kwdyz11_files/figure-html/fig-comparativedensity-output-1.svg new file mode 100644 index 0000000..2a9d520 --- /dev/null +++ b/kwdyz11_files/figure-html/fig-comparativedensity-output-1.svg @@ -0,0 +1,672 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/kwdyz11_files/figure-html/fig-logrtboxplots-output-2.svg b/kwdyz11_files/figure-html/fig-logrtboxplots-output-2.svg new file mode 100644 index 0000000..22d5957 --- /dev/null +++ b/kwdyz11_files/figure-html/fig-logrtboxplots-output-2.svg @@ -0,0 +1,582 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned.html b/largescaledesigned.html new file mode 100644 index 0000000..2513edb --- /dev/null +++ b/largescaledesigned.html @@ -0,0 +1,4430 @@ + + + + + + + + + + + +largescaledesigned – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +
+

1 A large-scale designed experiment

+

Load the packages to be used.

+
+
+Code +
using AlgebraOfGraphics
+using Arrow
+using CairoMakie
+using Chain
+using DataFrameMacros
+using DataFrames
+using Effects
+using MixedModels
+using MixedModelsMakie
+using SMLP2024: dataset
+using StandardizedPredictors
+using StatsBase
+
+CairoMakie.activate!(; type="svg")
+import ProgressMeter
+ProgressMeter.ijulia_behavior(:clear)
+
+
+

The English Lexicon Project (Balota et al., 2007) was a large-scale multicenter study to examine properties of English words. It incorporated both a lexical decision task and a word recognition task. Different groups of subjects participated in the different tasks.

+
+
+

2 Extracting data tables from the raw data

+

The raw data are available as an OSF project as Zip files for each of the tasks. These Zip files contain one data file for each participant, which has a mixture of demographic data, responses on some pre-tests, and the actual trial runs.

+

Parsing these data files is not fun – see this repository for some of the code used to untangle the data. (This repository is an unregistered Julia package.)

+

Some lessons from this:

+
    +
  • When an identifier is described as a “unique subject id”, it probably isn’t.
    • +
  • In a multi-center trial, the coordinating center should assign the range of id’s for each satellite site. Failure of a satellite site to stay within its range should result in banishment to a Siberian work camp.
    • +
  • As with all data cleaning, the prevailing attitude should be “trust, but verify”. Just because you are told that the file is in a certain format, doesn’t mean it is. Just because you are told that the identifiers are unique doesn’t mean they are, etc.
    • +
  • It works best if each file has a well-defined, preferably simple, structure. These data files had two different formats mushed together.
    • +
  • This is the idea of “tidy data” - each file contains only one type of record along with well-defined rules of how you relate one file to another.
    • +
  • If one of the fields is a date, declare the only acceptable form of writing a date, preferably yyyy-mm-dd. Anyone getting creative about the format of the dates will be required to write the software to parse that form (and that is usually not an easy task).
    • +
  • As you make changes in a file, document them. If you look at the EnglishLexicon.jl repository you will see that it is in the form of scripts that take the original Zip files and produce the Arrow files. That way, if necessary, the changes can be undone or modified.
    • +
  • Remember, the data are only as useful as their provenance. If you invested a lot of time and money in gathering the data you should treat it as a valued resource and exercise great care with it.
    • +
  • The Arrow.jl package allows you to add metadata as key/value pairs, called a Dict (or dictionary). Use this capability. The name of the file is not a suitable location for metadata.
    • +
+
+

2.1 Trial-level data from the LDT

+

In the lexical decision task the study participant is shown a character string, under carefully controlled conditions, and responds according to whether they identify the string as a word or not. Two responses are recorded: whether the choice of word/non-word is correct and the time that elapsed between exposure to the string and registering a decision.

+

Several covariates, some relating to the subject and some relating to the target, were recorded. Initially we consider only the trial-level data.

+
+
ldttrial = dataset(:ELP_ldt_trial)
+
+
[ Info: Downloading ELP_ldt_trial dataset
+
+
+
Arrow.Table with 2745952 rows, 5 columns, and schema:
+ :subj  String
+ :seq   Int16
+ :acc   Union{Missing, Bool}
+ :rt    Int16
+ :item  String
+
+with metadata given by a Base.ImmutableDict{String, String} with 3 entries:
+  "source"    => "https://osf.io/n63s2"
+  "reference" => "Balota et al. (2007), The English Lexicon Project, Behavior R…
+  "title"     => "Trial-level data from Lexical Discrimination Task in the Engl…
+
+
+

The two response variables are acc - the accuracy of the response - and rt, the response time in milliseconds. There is one trial-level covariate, seq, the sequence number of the trial within subj. Each subject participated in two sessions on different days, with 2000 trials recorded on the first day.

+

Notice the metadata with a citation and a URL for the OSF project.

+

We convert to a DataFrame and add a Boolean column s2 which is true for trials in the second session.

+
+
ldttrial = @transform(DataFrame(ldttrial), :s2 = :seq > 2000)
+describe(ldttrial)
+
+
6×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64Type
1subjS001S8160String
2seq1687.2111687.033740Int16
3acc0.85604false1.0true1370Union{Missing, Bool}
4rt846.325-16160732.0320610Int16
5itemAarodzuss0String
6s20.407128false0.0true0Bool
+
+
+
+
+
+

2.2 Initial data exploration

+

From the basic summary of ldttrial we can see that there are some questionable response times — negative values and values over 32 seconds.

+

Because of obvious outliers we will use the median response time, which is not strongly influenced by outliers, rather than the mean response time when summarizing by item or by subject.

+

Also, there are missing values of the accuracy. We should check if these are associated with particular subjects or particular items.

+
+

2.2.1 Summaries by item

+

To summarize by item we group the trials by item and use combine to produce the various summary statistics. As we will create similar summaries by subject, we incorporate an ‘i’ in the names of these summaries (and an ‘s’ in the name of the summaries by subject) to be able to identify the grouping used.

+
+
byitem = @chain ldttrial begin
+  groupby(:item)
+  @combine(
+    :ni = length(:acc),               # no. of obs
+    :imiss = count(ismissing, :acc),  # no. of missing acc
+    :iacc = count(skipmissing(:acc)), # no. of accurate
+    :imedianrt = median(:rt),
+  )
+  @transform!(
+    :wrdlen = Int8(length(:item)),
+    :ipropacc = :iacc / :ni
+  )
+end
+
+
80962×7 DataFrame
80937 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowitemniimissiaccimedianrtwrdlenipropacc
StringInt64Int64Int64Float64Int8Float64
1a35026743.010.742857
2e35019824.010.542857
3aah34021770.530.617647
4aal34032702.530.941176
5Aaron33031625.050.939394
6Aarod33023810.050.69697
7aback34015710.050.441176
8ahack34034662.051.0
9abacus34017671.560.5
10alacus34029640.060.852941
11abandon34032641.070.941176
12acandon34033725.570.970588
13abandoned34031667.590.911765
⋮⋮⋮⋮⋮⋮⋮⋮
80951zoology33032623.070.969697
80952poology33032757.070.969697
80953zoom35034548.040.971429
80954zool35030633.040.857143
80955zooming33029617.070.878788
80956sooming33030721.070.909091
80957zooms33030598.050.909091
80958cooms33031660.050.939394
80959zucchini34029781.580.852941
80960hucchini34032727.580.941176
80961Zurich34021731.560.617647
80962Zurach34026811.060.764706
+
+
+
+

It can be seen that the items occur in word/nonword pairs and the pairs are sorted alphabetically by the word in the pair (ignoring case). We can add the word/nonword status for the items as

+
+
byitem.isword = isodd.(eachindex(byitem.item))
+describe(byitem)
+
+
8×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1itemAarodzuss0String
2ni33.91663034.0370Int64
3imiss0.016921500.020Int64
4iacc29.0194031.0370Int64
5imedianrt753.069458.0737.51691.00Float64
6wrdlen7.998818.0210Int8
7ipropacc0.8556160.00.9117651.00Float64
8isword0.5false0.5true0Bool
+
+
+
+

This table shows that some of the items were never identified correctly. These are

+
+
filter(:iacc => iszero, byitem)
+
+
9×8 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowitemniimissiaccimedianrtwrdlenipropaccisword
StringInt64Int64Int64Float64Int8Float64Bool
1baobab3400616.560.0true
2haulage3400708.570.0true
3leitmotif3500688.090.0true
4miasmal3500774.070.0true
5peahen3400684.060.0true
6plosive3400663.070.0true
7plugugly3300709.080.0true
8poshest3400740.070.0true
9servo3300697.050.0true
+
+
+
+

Notice that these are all words but somewhat obscure words such that none of the subjects exposed to the word identified it correctly.

+

We can incorporate characteristics like wrdlen and isword back into the original trial table with a “left join”. This operation joins two tables by values in a common column. It is called a left join because the left (or first) table takes precedence, in the sense that every row in the left table is present in the result. If there is no matching row in the second table then missing values are inserted for the columns from the right table in the result.

+
+
describe(
+  leftjoin!(
+    ldttrial,
+    select(byitem, :item, :wrdlen, :isword);
+    on=:item,
+  ),
+)
+
+
8×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64Type
1subjS001S8160String
2seq1687.2111687.033740Int16
3acc0.85604false1.0true1370Union{Missing, Bool}
4rt846.325-16160732.0320610Int16
5itemAarodzuss0String
6s20.407128false0.0true0Bool
7wrdlen7.9983518.0210Union{Missing, Int8}
8isword0.499995false0.0true0Union{Missing, Bool}
+
+
+
+

Notice that the wrdlen and isword variables in this table allow for missing values, because they are derived from the second argument, but there are no missing values for these variables. If there is no need to allow for missing values, there is a slight advantage in disallowing them in the element type, because the code to check for and handle missing values is not needed.

+

This could be done separately for each column or for the whole data frame, as in

+
+
describe(disallowmissing!(ldttrial; error=false))
+
+
8×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64Type
1subjS001S8160String
2seq1687.2111687.033740Int16
3acc0.85604false1.0true1370Union{Missing, Bool}
4rt846.325-16160732.0320610Int16
5itemAarodzuss0String
6s20.407128false0.0true0Bool
7wrdlen7.9983518.0210Int8
8isword0.499995false0.0true0Bool
+
+
+
+
+ +
+
+

The named argument error=false is required because there is one column, acc, that does incorporate missing values. If error=false were not given then the error thrown when trying to disallowmissing on the acc column would be propagated and the top-level call would fail.

+
+
+
+

A barchart of the word length counts, Figure 1, shows that the majority of the items are between 3 and 14 characters.

+
+
+Code +
let
+  wlen = 1:21
+  draw(
+    data((; wrdlen=wlen, count=counts(byitem.wrdlen, wlen))) *
+    mapping(:wrdlen => "Length of word", :count) *
+    visual(BarPlot),
+  )
+end
+
+
+
+
+
+ +
+
+Figure 1: Barchart of word lengths in the items used in the lexical decision task. +
+
+
+
+
+

To examine trends in accuracy by word length we plot the proportion accurate versus word-length separately for words and non-words with the area of each marker proportional to the number of observations for that combination (Figure 2).

+
+
+Code +
let
+  itemsummry = combine(
+    groupby(byitem, [:wrdlen, :isword]),
+    :ni => sum,
+    :imiss => sum,
+    :iacc => sum,
+  )
+  @transform!(
+    itemsummry,
+    :iacc_mean = :iacc_sum / (:ni_sum - :imiss_sum)
+  )
+  @transform!(itemsummry, :msz = sqrt((:ni_sum - :imiss_sum) / 800))
+  draw(
+    data(itemsummry) * mapping(
+      :wrdlen => "Word length",
+      :iacc_mean => "Proportion accurate";
+      color=:isword,
+      markersize=:msz,
+    );
+    figure=(; resolution=(800, 450)),
+  )
+end
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 2: Proportion of accurate trials in the LDT versus word length separately for words and non-words. The area of the marker is proportional to the number of observations represented. +
+
+
+
+
+

The pattern in the range of word lengths with non-negligible counts (there are points in the plot down to word lengths of 1 and up to word lengths of 21 but these points are very small) is that the accuracy for words is nearly constant at about 84% and the accuracy for nonwords is slightly higher until lengths of 13, at which point it falls off a bit.

+
+
+

2.2.2 Summaries by subject

+

A summary of accuracy and median response time by subject

+
+
bysubj = @chain ldttrial begin
+  groupby(:subj)
+  @combine(
+    :ns = length(:acc),               # no. of obs
+    :smiss = count(ismissing, :acc),  # no. of missing acc
+    :sacc = count(skipmissing(:acc)), # no. of accurate
+    :smedianrt = median(:rt),
+  )
+  @transform!(:spropacc = :sacc / :ns)
+end
+
+
814×6 DataFrame
789 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowsubjnssmisssaccsmedianrtspropacc
StringInt64Int64Int64Float64Float64
1S001337403158554.00.935981
2S002337213031960.00.898873
3S003337233006813.00.891459
4S004337413062619.00.907528
5S005337402574677.00.762893
6S006337402927855.00.867516
7S007337442877918.50.852697
8S0083372127311310.00.809905
9S0093374132669657.00.791049
10S010337402722757.00.806758
11S011337402894632.00.857736
12S012337442979692.00.882928
13S0133374229801114.00.883225
⋮⋮⋮⋮⋮⋮⋮
803S805337452881534.00.853883
804S806337413097841.50.917902
805S807337432994704.00.887374
806S808337422751630.50.815353
807S809337242603627.00.771945
808S810337413242603.50.960877
809S811337422861827.00.847955
810S812337263012471.00.893238
811S813337242932823.00.869514
812S814337413070773.00.909899
813S815337413024602.00.896266
814S816337402950733.00.874333
+
+
+
+

shows some anomalies

+
+
describe(bysubj)
+
+
6×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1subjS001S8160String
2ns3373.4133703374.033740Int64
3smiss1.6830501.0220Int64
4sacc2886.3317272928.032860Int64
5smedianrt760.992205.0735.01804.00Float64
6spropacc0.8556130.5118550.8680310.9739180Float64
+
+
+
+

First, some subjects are accurate on only about half of their trials, which is the proportion that would be expected from random guessing. A plot of the median response time versus proportion accurate, Figure 3, shows that the subjects with lower accuracy are some of the fastest responders, further indicating that these subjects are sacrificing accuracy for speed.

+
+
+Code +
draw(
+  data(bysubj) *
+  mapping(
+    :spropacc => "Proportion accurate",
+    :smedianrt => "Median response time (ms)",
+  ) *
+  (visual(Scatter) + smooth())
+)
+
+
+
+
+
+ +
+
+Figure 3: Median response time versus proportion accurate by subject in the LDT. +
+
+
+
+
+

As described in Balota et al. (2007), the participants performed the trials in blocks of 250 followed by a short break. During the break they were given feedback concerning accuracy and response latency in the previous block of trials. If the accuracy was less than 80% the participant was encouraged to improve their accuracy. Similarly, if the mean response latency was greater than 1000 ms, the participant was encouraged to decrease their response time. During the trials immediate feedback was given if the response was incorrect.

+

Nevertheless, approximately 15% of the subjects were unable to maintain 80% accuracy on their trials

+
+
count(<(0.8), bysubj.spropacc) / nrow(bysubj)
+
+
0.15233415233415235
+
+
+

and there is some association of faster response times with low accuracy. The majority of the subjects whose median response time is less than 500 ms. are accurate on less than 75% of their trials. Another way of characterizing the relationship is that none of the subjects with 90% accuracy or greater had a median response time less than 500 ms.

+
+
minimum(@subset(bysubj, :spropacc > 0.9).smedianrt)
+
+
505.0
+
+
+

It is common in analyses of response latency in a lexical discrimination task to consider only the latencies on correct identifications and to trim outliers. In Balota et al. (2007) a two-stage outlier removal strategy was used; first removing responses less than 200 ms or greater than 3000 ms then removing responses more than three standard deviations from the participant’s mean response.

+

As described in Section 2.2.3 we will analyze these data on a speed scale (the inverse of response time) using only the first-stage outlier removal of response latencies less than 200 ms or greater than 3000 ms. On the speed scale the limits are 0.333 per second up to 5 per second.

+

To examine the effects of the fast but inaccurate responders we will fit models to the data from all the participants and to the data from the 85% of participants who maintained an overall accuracy of 80% or greater.

+
+
pruned = @chain ldttrial begin
+  @subset(!ismissing(:acc), 200 ≤ :rt ≤ 3000,)
+  leftjoin!(select(bysubj, :subj, :spropacc); on=:subj)
+  dropmissing!
+end
+size(pruned)
+
+
(2714311, 9)
+
+
+
+
describe(pruned)
+
+
9×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1subjS001S8160String
2seq1684.5611684.033740Int16
3acc0.859884false1.0true0Bool
4rt838.712200733.030000Int16
5itemAarodzuss0String
6s20.40663false0.0true0Bool
7wrdlen7.9924418.0210Int8
8isword0.500126false1.0true0Bool
9spropacc0.8571690.5118550.8692950.9739180Float64
+
+
+
+
+
+

2.2.3 Choice of response scale

+

As we have indicated, generally the response times are analyzed for the correct identifications only. Furthermore, unrealistically large or small response times are eliminated. For this example we only use the responses between 200 and 3000 ms.

+

A density plot of the pruned response times, Figure 4, shows they are skewed to the right.

+
+
+Code +
draw(
+  data(pruned) *
+  mapping(:rt => "Response time (ms.) for correct responses") *
+  AlgebraOfGraphics.density();
+  figure=(; resolution=(800, 450)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 4: Kernel density plot of the pruned response times (ms.) in the LDT. +
+
+
+
+
+

In such cases it is common to transform the response to a scale such as the logarithm of the response time or to the speed of the response, which is the inverse of the response time.

+

The density of the response speed, in responses per second, is shown in Figure 5.

+
+
+Code +
draw(
+  data(pruned) *
+  mapping(
+    :rt => (x -> 1000 / x) => "Response speed (s⁻¹) for correct responses") *
+  AlgebraOfGraphics.density();
+  figure=(; resolution=(800, 450)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 5: Kernel density plot of the pruned response speed in the LDT. +
+
+
+
+
+

Figure 4 and Figure 5 indicate that it may be more reasonable to establish a lower bound of 1/3 second (333 ms) on the response latency, corresponding to an upper bound of 3 per second on the response speed. However, only about one half of one percent of the correct responses have latencies in the range of 200 ms. to 333 ms.

+
+
count(
+  r -> !ismissing(r.acc) && 200 < r.rt < 333,
+  eachrow(ldttrial),
+) / count(!ismissing, ldttrial.acc)
+
+
0.005867195806137328
+
+
+

so the exact position of the lower cut-off point on the response latencies is unlikely to be very important.

+
+ +
+
+

If you examine the code for (fit-elpldtspeeddens?), you will see that the conversion from rt to speed is done inline rather than creating and storing a new variable in the DataFrame.

+

I prefer to keep the DataFrame simple with the integer variables (e.g. :rt) if possible.

+

I recommend using the StandardizedPredictors.jl capabilities to center numeric variables or convert to zscores.

+
+
+
+
+
+

2.2.4 Transformation of response and the form of the model

+

As noted in Box & Cox (1964), a transformation of the response that produces a more Gaussian distribution often will also produce a simpler model structure. For example, Figure 6 shows the smoothed relationship between word length and response time for words and non-words separately,

+
+
+Code +
draw(
+  data(pruned) *
+  mapping(
+    :wrdlen => "Word length",
+    :rt => "Response time (ms)";
+    :color => :isword,
+  ) * smooth()
+)
+
+
+
+
+
+ +
+
+Figure 6: Scatterplot smooths of response time versus word length in the LDT. +
+
+
+
+
+

and Figure 7 shows the similar relationships for speed

+
+
+Code +
draw(
+  data(pruned) *
+  mapping(
+    :wrdlen => "Word length",
+    :rt => (x -> 1000/x) => "Speed of response (s⁻¹)";
+    :color => :isword,
+  ) * smooth()
+)
+
+
+
+
+
+ +
+
+Figure 7: Scatterplot smooths of response speed versus word length in the LDT. +
+
+
+
+
+

For the most part the smoother lines in Figure 7 are reasonably straight. The small amount of curvature is associated with short word lengths, say less than 4 characters, of which there are comparatively few in the study.

+

Figure 8 shows a “violin plot” - the empirical density of the response speed by word length separately for words and nonwords. The lines on the plot are fit by linear regression.

+
+
+Code +
let
+  plt = data(@subset(pruned, :wrdlen > 3, :wrdlen < 14))
+  plt *= mapping(
+    :wrdlen => "Word length",
+    :rt => (x -> 1000/x) => "Speed of response (s⁻¹)",
+    color=:isword,
+    side=:isword,
+  )
+  plt *= visual(Violin)
+  draw(plt, axis=(; limits=(nothing, (0.0, 2.8))))
+end
+
+
+
+
+
+ +
+
+Figure 8: Empirical density of response speed versus word length by word/non-word status. +
+
+
+
+
+
+
+
+

2.3 Models with scalar random effects

+

A major purpose of the English Lexicon Project is to characterize the items (words or nonwords) according to the observed accuracy of identification and to response latency, taking into account subject-to-subject variability, and to relate these to lexical characteristics of the items.

+

In Balota et al. (2007) the item response latency is characterized by the average response latency from the correct trials after outlier removal.

+

Mixed-effects models allow us greater flexibility and, we hope, precision in characterizing the items by controlling for subject-to-subject variability and for item characteristics such as word/nonword and item length.

+

We begin with a model that has scalar random effects for item and for subject and incorporates fixed-effects for word/nonword and for item length and for the interaction of these terms.

+
+

2.3.1 Establish the contrasts

+

Because there are a large number of items in the data set it is important to assign a Grouping() contrast to item (and, less importantly, to subj). For the isword factor we will use an EffectsCoding contrast with the base level as false. The non-words are assigned -1 in this contrast and the words are assigned +1. The wrdlen covariate is on its original scale but centered at 8 characters.

+

Thus the (Intercept) coefficient is the predicted speed of response for a typical subject and typical item (without regard to word/non-word status) of 8 characters.

+

Set these contrasts

+
+
contrasts = Dict(
+  :subj => Grouping(),
+  :item => Grouping(),
+  :isword => EffectsCoding(; base=false),
+  :wrdlen => Center(8),
+)
+
+
Dict{Symbol, Any} with 4 entries:
+  :item   => Grouping()
+  :wrdlen => Center(8)
+  :isword => EffectsCoding(false, nothing)
+  :subj   => Grouping()
+
+
+

and fit a first model with simple, scalar, random effects for subj and item.

+
+
elm01 = let
+  form = @formula(
+    1000 / rt ~ 1 + isword * wrdlen + (1 | item) + (1 | subj)
+  )
+  fit(MixedModel, form, pruned; contrasts)
+end
+
+
Minimizing 2    Time: 0:00:00 ( 0.33  s/it)
+  objective:  2.7266989738341523e6
+Minimizing 10    Time: 0:00:02 ( 0.22  s/it)
+  objective:  2.54791986993588e6
+Minimizing 11    Time: 0:00:02 ( 0.21  s/it)
+  objective:  2.5477518552180286e6
+Minimizing 12    Time: 0:00:02 ( 0.20  s/it)
+  objective:  2.5477597344292034e6
+Minimizing 13    Time: 0:00:02 ( 0.19  s/it)
+  objective:  2.5478041435803813e6
+Minimizing 14    Time: 0:00:02 ( 0.19  s/it)
+  objective:  2.547734626944677e6
+Minimizing 15    Time: 0:00:02 ( 0.18  s/it)
+  objective:  2.5477332006914383e6
+Minimizing 16    Time: 0:00:02 ( 0.18  s/it)
+  objective:  2.5477314166843807e6
+Minimizing 17    Time: 0:00:02 ( 0.17  s/it)
+  objective:  2.5477281107397955e6
+Minimizing 18    Time: 0:00:03 ( 0.17  s/it)
+  objective:  2.5477271061153305e6
+Minimizing 19    Time: 0:00:03 ( 0.17  s/it)
+  objective:  2.5477267354581975e6
+Minimizing 20    Time: 0:00:03 ( 0.16  s/it)
+  objective:  2.5477250237434045e6
+Minimizing 21    Time: 0:00:03 ( 0.16  s/it)
+  objective:  2.547721761413731e6
+Minimizing 22    Time: 0:00:03 ( 0.16  s/it)
+  objective:  2.5477152538446435e6
+Minimizing 23    Time: 0:00:03 ( 0.16  s/it)
+  objective:  2.5477038026323644e6
+Minimizing 24    Time: 0:00:03 ( 0.15  s/it)
+  objective:  2.547687283736832e6
+Minimizing 25    Time: 0:00:03 ( 0.15  s/it)
+  objective:  2.5476642897852818e6
+Minimizing 26    Time: 0:00:03 ( 0.15  s/it)
+  objective:  2.5476747912287745e6
+Minimizing 27    Time: 0:00:04 ( 0.15  s/it)
+  objective:  2.5478612597469087e6
+Minimizing 28    Time: 0:00:04 ( 0.15  s/it)
+  objective:  2.5476016701920675e6
+Minimizing 29    Time: 0:00:04 ( 0.15  s/it)
+  objective:  2.547533486707639e6
+Minimizing 30    Time: 0:00:04 ( 0.15  s/it)
+  objective:  2.5474570471784873e6
+Minimizing 31    Time: 0:00:04 ( 0.14  s/it)
+  objective:  2.5475489231080017e6
+Minimizing 32    Time: 0:00:04 ( 0.14  s/it)
+  objective:  2.5492153915398596e6
+Minimizing 33    Time: 0:00:04 ( 0.14  s/it)
+  objective:  2.5474903052397575e6
+Minimizing 34    Time: 0:00:04 ( 0.14  s/it)
+  objective:  2.547841784697444e6
+Minimizing 35    Time: 0:00:04 ( 0.14  s/it)
+  objective:  2.5474595570143443e6
+Minimizing 36    Time: 0:00:05 ( 0.14  s/it)
+  objective:  2.547457139701927e6
+Minimizing 37    Time: 0:00:05 ( 0.14  s/it)
+  objective:  2.547458181007241e6
+Minimizing 38    Time: 0:00:05 ( 0.14  s/it)
+  objective:  2.547476739788983e6
+Minimizing 39    Time: 0:00:05 ( 0.14  s/it)
+  objective:  2.5474566821279465e6
+Minimizing 40    Time: 0:00:05 ( 0.14  s/it)
+  objective:  2.54745660988993e6
+Minimizing 41    Time: 0:00:05 ( 0.14  s/it)
+  objective:  2.547456628379439e6
+Minimizing 42    Time: 0:00:05 ( 0.14  s/it)
+  objective:  2.547457340940068e6
+Minimizing 43    Time: 0:00:05 ( 0.13  s/it)
+  objective:  2.547456569392055e6
+Minimizing 44    Time: 0:00:05 ( 0.13  s/it)
+  objective:  2.547456539806895e6
+Minimizing 45    Time: 0:00:05 ( 0.13  s/it)
+  objective:  2.5474565263292952e6
+Minimizing 46    Time: 0:00:06 ( 0.13  s/it)
+  objective:  2.5474565253121504e6
+Minimizing 47    Time: 0:00:06 ( 0.13  s/it)
+  objective:  2.547456524841241e6
+Minimizing 48    Time: 0:00:06 ( 0.13  s/it)
+  objective:  2.5474565248725615e6
+Minimizing 49    Time: 0:00:06 ( 0.13  s/it)
+  objective:  2.5474565286854426e6
+Minimizing 50    Time: 0:00:06 ( 0.13  s/it)
+  objective:  2.547456524731621e6
+Minimizing 51    Time: 0:00:06 ( 0.13  s/it)
+  objective:  2.547456524831969e6
+Minimizing 52    Time: 0:00:06 ( 0.13  s/it)
+  objective:  2.547456527911539e6
+Minimizing 53    Time: 0:00:06 ( 0.13  s/it)
+  objective:  2.547456524730126e6
+Minimizing 53    Time: 0:00:06 ( 0.13  s/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)1.37580.0090153.69<1e-990.11850.2550
isword: true0.06250.0005131.35<1e-99
wrdlen(centered: 8)-0.04360.0002-225.38<1e-99
isword: true & wrdlen(centered: 8)-0.00560.0002-28.83<1e-99
Residual0.3781
+
+
+

The predicted response speed by word length and word/nonword status can be summarized as

+
+
effects(Dict(:isword => [false, true], :wrdlen => 4:2:12), elm01)
+
+
10×6 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowwrdlenisword1000 / rterrlowerupper
Int64BoolFloat64Float64Float64Float64
14false1.465550.009031111.456521.47458
26false1.389470.008981241.380491.39845
38false1.313380.008964591.304421.32235
410false1.23730.008981341.228321.24628
512false1.161210.009031291.152181.17025
64true1.63510.00903111.626071.64413
76true1.53670.008981241.527721.54569
88true1.438310.008964591.429341.44727
910true1.339910.008981331.330921.34889
1012true1.241510.009031281.232481.25054
+
+
+
+

If we restrict to only those subjects with 80% accuracy or greater the model becomes

+
+
elm02 = let
+  form = @formula(
+    1000 / rt ~ 1 + isword * wrdlen + (1 | item) + (1 | subj)
+  )
+  dat = @subset(pruned, :spropacc > 0.8)
+  fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 2    Time: 0:00:00 (90.05 ms/it)
+  objective:  1.785784454054488e6
+Minimizing 3    Time: 0:00:00 (93.74 ms/it)
+  objective:  1.706955807895325e6
+Minimizing 5    Time: 0:00:00 (90.62 ms/it)
+  objective:  1.7096084055650986e6
+Minimizing 7    Time: 0:00:00 (89.21 ms/it)
+  objective:  1.632475326661393e6
+Minimizing 9    Time: 0:00:00 (88.94 ms/it)
+  objective:  1.6308713786709716e6
+Minimizing 11    Time: 0:00:00 (88.39 ms/it)
+  objective:  1.6306926636436544e6
+Minimizing 13    Time: 0:00:01 (87.60 ms/it)
+  objective:  1.63051455649771e6
+Minimizing 15    Time: 0:00:01 (86.70 ms/it)
+  objective:  1.6305061481623596e6
+Minimizing 17    Time: 0:00:01 (86.18 ms/it)
+  objective:  1.6304874273651254e6
+Minimizing 19    Time: 0:00:01 (85.37 ms/it)
+  objective:  1.630472047621346e6
+Minimizing 21    Time: 0:00:01 (85.47 ms/it)
+  objective:  1.6304442820480603e6
+Minimizing 23    Time: 0:00:01 (85.87 ms/it)
+  objective:  1.6306054243878725e6
+Minimizing 25    Time: 0:00:02 (85.63 ms/it)
+  objective:  1.6304138602695977e6
+Minimizing 27    Time: 0:00:02 (85.09 ms/it)
+  objective:  1.6303847542605218e6
+Minimizing 29    Time: 0:00:02 (84.79 ms/it)
+  objective:  1.630379610046544e6
+Minimizing 31    Time: 0:00:02 (84.64 ms/it)
+  objective:  1.6303774101036177e6
+Minimizing 33    Time: 0:00:02 (84.34 ms/it)
+  objective:  1.6303723086549612e6
+Minimizing 35    Time: 0:00:02 (84.01 ms/it)
+  objective:  1.6303668436558153e6
+Minimizing 37    Time: 0:00:03 (84.87 ms/it)
+  objective:  1.630366182470287e6
+Minimizing 39    Time: 0:00:03 (84.98 ms/it)
+  objective:  1.6303660445637105e6
+Minimizing 41    Time: 0:00:03 (84.94 ms/it)
+  objective:  1.630365959547105e6
+Minimizing 43    Time: 0:00:03 (84.93 ms/it)
+  objective:  1.6303659564438118e6
+Minimizing 45    Time: 0:00:03 (84.86 ms/it)
+  objective:  1.6303659561360003e6
+Minimizing 47    Time: 0:00:03 (84.96 ms/it)
+  objective:  1.6303659561178428e6
+Minimizing 49    Time: 0:00:04 (84.83 ms/it)
+  objective:  1.630365956261281e6
+Minimizing 51    Time: 0:00:04 (84.56 ms/it)
+  objective:  1.6303659560896743e6
+Minimizing 53    Time: 0:00:04 (84.44 ms/it)
+  objective:  1.6303659560892093e6
+Minimizing 53    Time: 0:00:04 (84.45 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)1.36110.0088153.98<1e-990.12470.2318
isword: true0.06560.0005133.73<1e-99
wrdlen(centered: 8)-0.04440.0002-222.65<1e-99
isword: true & wrdlen(centered: 8)-0.00570.0002-28.73<1e-99
Residual0.3342
+
+
+
+
effects(Dict(:isword => [false, true], :wrdlen => 4:2:12), elm02)
+
+
10×6 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowwrdlenisword1000 / rterrlowerupper
Int64BoolFloat64Float64Float64Float64
14false1.450360.008924671.441441.45929
26false1.372970.008871011.36411.38184
38false1.295570.008853091.286721.30443
410false1.218180.008871111.209311.22705
512false1.140780.008924871.131861.14971
64true1.627350.008924661.618421.63627
76true1.527020.008871011.518151.53589
88true1.42670.008853091.417841.43555
910true1.326370.00887111.31751.33524
1012true1.226050.008924841.217121.23497
+
+
+
+

The differences in the fixed-effects parameter estimates between a model fit to the full data set and one fit to the data from accurate responders only, are small.

+

However, the random effects for the item, while highly correlated, are not perfectly correlated.

+
+
+Code +
CairoMakie.activate!(; type="png")
+disallowmissing!(
+  leftjoin!(
+    byitem,
+    leftjoin!(
+      rename!(DataFrame(raneftables(elm01)[:item]), [:item, :elm01]),
+      rename!(DataFrame(raneftables(elm02)[:item]), [:item, :elm02]);
+      on=:item,
+    ),
+    on=:item,
+  ),
+)
+disallowmissing!(
+  leftjoin!(
+    bysubj,
+    leftjoin!(
+      rename!(DataFrame(raneftables(elm01)[:subj]), [:subj, :elm01]),
+      rename!(DataFrame(raneftables(elm02)[:subj]), [:subj, :elm02]);
+      on=:subj,
+    ),
+    on=:subj,
+  ); error=false,
+)
+draw(
+  data(byitem) * mapping(
+    :elm01 => "Conditional means of item random effects for model elm01",
+    :elm02 => "Conditional means of item random effects for model elm02";
+    color=:isword,
+  ) * visual(Scatter; alpha=0.2);
+  axis=(; width=600, height=600),
+)
+
+
+
+
+ +
+
+Figure 9: Conditional means of scalar random effects for item in model elm01, fit to the pruned data, versus those for model elm02, fit to the pruned data with inaccurate subjects removed. +
+
+
+
+
+
+
+ +
+
+Note +
+
+
+

Adjust the alpha on Figure 9.

+
+
+

Figure 9 is exactly of the form that would be expected in a sample from a correlated multivariate Gaussian distribution. The correlation of the two sets of conditional means is about 96%.

+
+
cor(Matrix(select(byitem, :elm01, :elm02)))
+
+
2×2 Matrix{Float64}:
+ 1.0       0.958655
+ 0.958655  1.0
+
+
+

These models take only a few seconds to fit on a modern laptop computer, which is quite remarkable given the size of the data set and the number of random effects.

+

The amount of time to fit more complex models will be much greater so we may want to move those fits to more powerful server computers. We can split the tasks of fitting and analyzing a model between computers by saving the optimization summary after the model fit and later creating the MixedModel object followed by restoring the optsum object.

+
+
saveoptsum("./fits/elm01.json", elm01);
+
+
+
elm01a = restoreoptsum!(
+  let
+    form = @formula(
+      1000 / rt ~ 1 + isword * wrdlen + (1 | item) + (1 | subj)
+    )
+    MixedModel(form, pruned; contrasts)
+  end,
+  "./fits/elm01.json",
+)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)1.37580.0090153.69<1e-990.11850.2550
isword: true0.06250.0005131.35<1e-99
wrdlen(centered: 8)-0.04360.0002-225.38<1e-99
isword: true & wrdlen(centered: 8)-0.00560.0002-28.83<1e-99
Residual0.3781
+
+
+

Other covariates associated with the item are available as

+
+
elpldtitem = DataFrame(dataset("ELP_ldt_item"))
+describe(elpldtitem)
+
+
[ Info: Downloading ELP_ldt_item dataset
+
+
+
9×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64Type
1itemAarodzuss0String
2Ortho_N1.5330901.0250Int8
3BG_Sum13938.41113026.059803177Union{Missing, Int32}
4BG_Mean1921.255.51907.06910.0177Union{Missing, Float32}
5BG_Freq_By_Pos2043.0801928.069854Union{Missing, Int16}
6itemno40481.5140481.5809620Int32
7isword0.5false0.5true0Bool
8wrdlen7.998818.0210Int8
9pairno20241.0120241.0404810Int32
+
+
+
+

and those associated with the subject are

+
+
elpldtsubj = DataFrame(dataset("ELP_ldt_subj"))
+describe(elpldtsubj)
+
+
[ Info: Downloading ELP_ldt_subj dataset
+
+
+
20×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyAnyAnyInt64Type
1subjS001S8160String
2univKansasWayne State0String
3sexfm8Union{Missing, String}
4DOB1938-06-071984-11-140Date
5MEQ44.493219.044.075.08Union{Missing, Float32}
6vision5.5116906.071Union{Missing, Int8}
7hearing5.8610106.071Union{Missing, Int8}
8educatn8.89681112.0280Int8
9ncorrct29.8505530.04018Union{Missing, Int8}
10rawscor31.99251332.04018Union{Missing, Int8}
11vocabAge17.812310.317.821.019Union{Missing, Float32}
12shipTime3.086103.091Union{Missing, Int8}
13readTime2.502150.02.015.01Union{Missing, Float32}
14preshlth5.4870806.071Union{Missing, Int8}
15pasthlth4.9298905.071Union{Missing, Int8}
16S1start2001-03-16T13:49:272001-10-16T11:38:28.5002003-07-29T18:48:440DateTime
17S2start2001-03-19T10:00:352001-10-19T14:24:19.5002003-07-30T13:07:450DateTime
18MEQstrt2001-03-22T18:32:002001-10-23T11:26:132003-07-30T14:30:497Union{Missing, DateTime}
19filename101DATA.LDTData998.LDT0String
20frstLangEnglishother8Union{Missing, String}
+
+
+
+

For the simple model elm01 the estimated standard deviation of the random effects for subject is greater than that of the random effects for item, a common occurrence. A caterpillar plot, Figure 10,

+
+
+Code +
qqcaterpillar!(
+  Figure(resolution=(800, 650)),
+  ranefinfo(elm01, :subj),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 10: Conditional means and 95% prediction intervals for subject random effects in elm01. +
+
+
+
+
+

shows definite distinctions between subjects because the widths of the prediction intervals are small compared to the range of the conditional modes. Also, there is at least one outlier with a conditional mode over 1.0.

+

Figure 11 is the corresponding caterpillar plot for model elm02 fit to the data with inaccurate responders eliminated.

+
+
+Code +
qqcaterpillar!(
+  Figure(resolution=(800, 650)),
+  ranefinfo(elm02, :subj),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 11: Conditional means and 95% prediction intervals for subject random effects in elm02. +
+
+
+
+
+
+
+ +
+ + Back to top

References

+
+Balota, D. A., Yap, M. J., Hutchison, K. A., Cortese, M. J., Kessler, B., Loftis, B., Neely, J. H., Nelson, D. L., Simpson, G. B., & Treiman, R. (2007). The english lexicon project. Behavior Research Methods, 39(3), 445–459. https://doi.org/10.3758/bf03193014 +
+
+Box, G. E. P., & Cox, D. R. (1964). An analysis of transformations. Journal of the Royal Statistical Society: Series B (Methodological), 26(2), 211–243. https://doi.org/10.1111/j.2517-6161.1964.tb00553.x +
+
+ + +
+ + + + + \ No newline at end of file diff --git a/largescaledesigned_files/figure-html/fig-elm01caterpillarsubj-output-2.svg b/largescaledesigned_files/figure-html/fig-elm01caterpillarsubj-output-2.svg new file mode 100644 index 0000000..2fb67a6 --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-elm01caterpillarsubj-output-2.svg @@ -0,0 +1,3433 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-elm01univ-output-1.svg b/largescaledesigned_files/figure-html/fig-elm01univ-output-1.svg new file mode 100644 index 0000000..7b04070 --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-elm01univ-output-1.svg @@ -0,0 +1,680 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-elm01vocabage-output-1.svg b/largescaledesigned_files/figure-html/fig-elm01vocabage-output-1.svg new file mode 100644 index 0000000..17fbd61 --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-elm01vocabage-output-1.svg @@ -0,0 +1,1346 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-elm02caterpillarsubj-output-2.svg b/largescaledesigned_files/figure-html/fig-elm02caterpillarsubj-output-2.svg new file mode 100644 index 0000000..3268bd8 --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-elm02caterpillarsubj-output-2.svg @@ -0,0 +1,2932 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-elm02univ-output-1.svg b/largescaledesigned_files/figure-html/fig-elm02univ-output-1.svg new file mode 100644 index 0000000..67f52af --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-elm02univ-output-1.svg @@ -0,0 +1,797 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-elpldtrtdens-output-2.svg b/largescaledesigned_files/figure-html/fig-elpldtrtdens-output-2.svg new file mode 100644 index 0000000..a202d1a --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-elpldtrtdens-output-2.svg @@ -0,0 +1,454 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-elpldtspeeddens-output-2.svg b/largescaledesigned_files/figure-html/fig-elpldtspeeddens-output-2.svg new file mode 100644 index 0000000..415c39b --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-elpldtspeeddens-output-2.svg @@ -0,0 +1,357 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-ldtmedianrtvspropacc-output-1.svg b/largescaledesigned_files/figure-html/fig-ldtmedianrtvspropacc-output-1.svg new file mode 100644 index 0000000..6ee6446 --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-ldtmedianrtvspropacc-output-1.svg @@ -0,0 +1,1202 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-ldtrtvswrdlen-output-1.svg b/largescaledesigned_files/figure-html/fig-ldtrtvswrdlen-output-1.svg new file mode 100644 index 0000000..9bc4cf0 --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-ldtrtvswrdlen-output-1.svg @@ -0,0 +1,397 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-ldtspeedvswrdlen-output-1.svg b/largescaledesigned_files/figure-html/fig-ldtspeedvswrdlen-output-1.svg new file mode 100644 index 0000000..58d6d27 --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-ldtspeedvswrdlen-output-1.svg @@ -0,0 +1,427 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-ldtwrdlenhist-output-1.svg b/largescaledesigned_files/figure-html/fig-ldtwrdlenhist-output-1.svg new file mode 100644 index 0000000..924766d --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-ldtwrdlenhist-output-1.svg @@ -0,0 +1,244 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-propvswrdlen-output-2.svg b/largescaledesigned_files/figure-html/fig-propvswrdlen-output-2.svg new file mode 100644 index 0000000..5729d41 --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-propvswrdlen-output-2.svg @@ -0,0 +1,455 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/largescaledesigned_files/figure-html/fig-speedviolin-output-1.svg b/largescaledesigned_files/figure-html/fig-speedviolin-output-1.svg new file mode 100644 index 0000000..a2913e4 --- /dev/null +++ b/largescaledesigned_files/figure-html/fig-speedviolin-output-1.svg @@ -0,0 +1,484 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/mrk17.html b/mrk17.html new file mode 100644 index 0000000..d43a511 --- /dev/null +++ b/mrk17.html @@ -0,0 +1,4002 @@ + + + + + + + + + + + +RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +
+

1 Setup

+

Packages we (might) use.

+
+
using CategoricalArrays
+using DataFrames
+using MixedModels
+using MixedModelsMakie
+using SMLP2024: dataset
+using Statistics: mean, std
+
+
+
dat = DataFrame(dataset(:mrk17_exp1))
+describe(dat)
+
+
9×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1subjS01S730String
2itemACHEYEAR0String
3trial239.9582239.04800Int16
4FHFLF0String
5Prelunr0String
6Qclrdeg0String
7lQclrdeg0String
8lTNWWD0String
9rt647.173301601.029940Int16
+
+
+
+
+
+

2 Specification

+

This section covers the general terminology and advice for model specification.

+
+

2.1 Response, covariates, and factors

+

Linear mixed models (LMMs), like many other types of statistical models, describe a relationship between a response variable and covariates that have been measured or observed along with the response. The statistical model assumes that the residuals of the fitted response (i.e., not the responses) are normally – also identically and independently – distributed. This is the first assumption of normality in the LMM. It is standard practice that model residuals are inspected and, if serious skew is indicated, that the response is Box-Cox transformed (unless not justified for theoretical reasons) to fulfill this model assumption.

+

In the following we distinguish between categorical covariates and numerical covariates. Categorical covariates are factors. The important characteristic of a factor is that, for each observed value of the response, the factor takes on the value of one of a set of discrete levels. The levels can be unordered (nominal) or ordered (ordinal). We use the term covariate when we refer to numerical covariates, that is to continuous measures with some distribution. In principle, statistical models are not constrained by the distribution of observations across levels of factors and covariates, but the distribution may lead to problems of model identification and it does implications for statistical power.

+

Statistical power, especially for the detection of interactions, is best when observations are uniformly distributed across levels of factors or uniform across the values of covariates. In experimental designs, uniform distributions may be achieved by balanced assignment of subjects (or other carriers of responses) to the levels of factors or combinations of factor levels. In observational contexts, we achieve uniform distributions by stratification (e..g., on age, gender, or IQ scores). Statistical power is worse for skewed than normal distributions (I think …). Therefore, although it is not required to meet an assumption of the statistical model, it may be useful to consider Box-Cox transformations of covariates.

+
+
+

2.2 Nested and crossed random (grouping) factors

+

In LMMs the levels of at least one of the factors represents units in the data set that are assumed to be sampled, ideally randomly, from a population that is normally distributed with respect to the response. This is the second assumption of normal distribution in LMMs. In psychology and linguistics the observational units are often the subjects or items (e..g., texts, sentences, words, pictures) in the study. We may use numbers, such as subject identifiers, to designate the particular levels that we observed; we recommend to prepend these numbers with “S” or “I” to avoid confusion with numeric variables.

+

Random sampling is the basis of generalization from the sample to the population. The core statistics we will estimate in this context are variances and correlations of grand means and (quasi-)experimental effects. These terms will be explained below. What we want to stress here is that the estimation of (co-)variances / correlations requires a larger number of units (levels) than the estimation of means. Therefore, from a practical perspective, it is important that random factors are represented with many units.

+

When there is more than one random factor, we must be clear about their relation. The two prototypical cases are that the factors are nested or crossed. In multilevel models, a special case of mixed models, the levels of the random factors are strictly nested. For example, at a given time, every student attends a specific class in a specific school. Students, classes, and schools could be three random factors. As soon as we look at this scenario across several school years, the nesting quickly falls apart because students may move between classes and between schools.

+

In psychology and linguistics, random factors are often crossed, for example, when every subject reads every word in every sentence in a word-by-word self-paced reading experiment (or alternatively: when every word in every sentence elicits a response from every subject). However, in an eye-movement experiment (for example), the perfect crossing on a measure like fixation duration is not attainable because of blinks or skipping of words.

+

In summary, the typical situation in experimental and observational studies with more than one random factor is partial crossing or partial nesting of levels of the random factors. Linear mixed models handle these situations very well.

+
+
+

2.3 Experimental and quasi-experimental fixed factors / covariates

+

Fixed experimental factor or covariate. In experiments the units (or levels) of the random factor(s) are assigned to manipulations implemented in their design. The researcher controls the assignment of units of the random factor(s) (e.g., subjects, items) to experimental manipulations. These manipulations are represented as factors with a fixed and discrete set of levels (e.g., training vs. control group) or as covariates associated with continuous numeric values (e.g., presentation times).

+

Fixed quasi-experimental factor or covariate. In observational studies (which can also be experiments) the units (or levels) of random factors may “bring along” characteristics that represent the levels of quasi-experimental factors or covariates beyond the control of the researcher. Whether a a subject is female, male, or diverse or whether a word is a noun, a verb, or an adjective are examples of quasi-experimental factors of gender or word type, respectively. Subject-related covariates are body height, body mass, and IQ scores; word-related covariates are their lengths, frequency, and cloze predictability.

+
+
+

2.4 Between-unit and within-unit factors / covariates

+

The distinction between between-unit and within-unit factors is always relative to a random (grouping) factor of an experimental design. A between-unit factor / covariate is a factor for which every unit of the random factor is assigned to or characterized by only one level of the factor. A within-unit factor is a factor for which units of the random factor appear at every level of the factor.

+

For the typical random factor, say Subject, there is little ambiguity because we are used to the between-within distinction from ANOVAs, more specifically the F1-ANOVA. In psycholinguistics, there is the tradition to test effects also for the second random factor Item in an F2-ANOVA. Importantly, for a given fixed factor all four combinations are possible. For example, Gender is a fixed quasi-experimental between-subject / within-item factor; word frequency is a fixed quasi-experimental within-subject / between-item covariate; Prime-target relation is a fixed experimental within-subject / within-item factor (assuming that targets are presented both in a primed and in an unprimed situation); and when a training manipulation is defined by the items used in the training, then in a training-control group design, the fixed factor Group is a fixed experimental between-subject / between-item factor.

+

These distinctions are critical for setting up LMMs because variance components for (quasi-)experimental effects can only be specified for within-unit effects. Note also that loss of data (within limits), counterbalancing or blocking of items are irrelevant for these definitions.

+
+ +
+

2.6 Contrast- and trend-based fixed-effect model parameters

+

Fixed factors and covariates are expected to have effects on the response. Fixed-effect model parameters estimate the hypothesized main and interaction effects of the study. The estimates of factors are based on contrasts; the estimates of covariates are based on trends. Conceptually, they correspond to unstandardized regression coefficients in multiple regression.

+

The intercept is a special regression coefficient; it estimates the value of the dependent variable when all fixed effects associated with factors and trends associated with covariates are zero. In experimental designs with higher-order interactions there is an advantage of specifying the LMM in such a way that the intercept estimates the grand mean (GM; mean of the means of design cells). This happens if (a) contrasts for factors are chosen such that the intercept estimates the GM (positive: EffectsCoding, SeqDifferenceCoding, or HelmertCoding contrasts; negative: DummyCoding), (b) orthogonal polynomial trends are used (Helmert, anova-based), and (c) covariates are centered on their mean before inclusion in the model. As always, there may be good theoretical reasons to depart from the default recommendation.

+

The specification of contrasts / trends does not depend on the status of the fixed factor / covariate. It does not matter whether a factor varies between or within the units of a random factor or whether it is an experimental or quasi-experimental factor. Contrasts are not specified for random (grouping) factors.

+
+
+

2.7 Variance components (VCs) and correlation parameters (CPs)

+

Variance components (VCs) and correlation parameters (CPs) are within-group model parameters; they correspond to (some of the) within-unit (quasi-)experimental fixed-effect model parameters. Thus, we may be able to estimate a subject-related VC for word frequency. If we included a linear trend for word frequency, the VC estimates the subject-related variance in these slopes. We cannot estimate an item-related VC for the word-frequency slopes because there is only one frequency associated with words. Analogously, we may able to estimate an item-related VC for the effect of Group (training vs. control), but we cannot estimate a subject-related VC for this effect.

+

The within-between characteristics of fixed factors and covariates relative to the random factor(s) are features of the design of the experiment or observational study. They fundamentally constrain the specification of the LMM. That’s why it is of upmost importance to be absolutely clear about their status.

+
+
+

2.8 Conditional modes of random effects

+

In this outline of the dimensions underlying the specification of an LMM, we have said nothing so far about the conditional modes of random effects (i.e., the results shown in caterpillar and shrinkage plots). They are not needed for model specification or model selection.

+

The VC is the prior variance of the random effects, whereas var(ranef(model)) is the variance of the posterior means/modes of the random effects. See Kliegl et al. (2010, VisualCognition); Rizopoulos (2019, stackexchange.

+
+
+
+

3 Background for data

+
+

3.1 Abstract

+

This semantic-priming experiment was reported in Masson, Rabe, & Kliegl (2017, Exp. 1, Memory & Cognition). It is a direct replication of an experiment reported in Masson & Kliegl (2013, Exp. 1, JEPLMC). Following a prime word a related or unrelated high- or low-frequency target word or a nonword was presented in clear or dim font. The subject’s task was to decide as quickly as possible whether the target was a word or a nonword, that is subjects performed a lexical decision task (LDT). The reaction time and the accuracy of the response were recorded. Only correct reaction times to words are included. After filtering there were 16,409 observations recorded from 73 subjects and 240 items.

+
+
+

3.2 Codebook

+

The data (variables and observations) used by Masson et al. (2017) are available in file MRK17_Exp1.RDS

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
VariableDescription
SubjSubject identifier
ItemTarget (non-)word
trialTrial number
FTarget frequency is high or low
PPrime is related or unrelated to target
QTarget quality is clear or degraded
lQLast-trial target quality is clear or degraded
lTLast-trail target requires word or nonword response
rtReaction time [ms]
+

lagQlty and lagTrgt refer to experimental conditions in the last trial.

+

Corresponding indicator variables (-1/+1):

+
+
+
+

4 Setup

+

Packages we (might) use.

+
+
using CategoricalArrays
+using DataFrames
+using MixedModels
+using MixedModelsMakie
+using SMLP2024: dataset
+using Statistics: mean, std
+
+
+
dat = DataFrame(dataset(:mrk17_exp1))
+describe(dat)
+
+
9×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1subjS01S730String
2itemACHEYEAR0String
3trial239.9582239.04800Int16
4FHFLF0String
5Prelunr0String
6Qclrdeg0String
7lQclrdeg0String
8lTNWWD0String
9rt647.173301601.029940Int16
+
+
+
+
+
cells = combine(
+  groupby(dat, [:F, :P, :Q, :lQ, :lT]),
+  nrow => :n,
+  :rt => mean => :rt_m,
+  :rt => std => :rt_sd
+ # :rt => (c -> mean(log, c)) => :lrt_m,
+)
+#dat_subj.CTR = categorical(dat_subj.CTR, levels=levels(dat.CTR))
+cells
+
+
32×8 DataFrame
7 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowFPQlQlTnrt_mrt_sd
StringStringStringStringStringInt64Float64Float64
1LFunrdegdegNW491683.831197.074
2LFunrdegdegWD488676.848198.409
3LFunrdegclrNW499685.265186.73
4LFunrdegclrWD519676.711167.794
5LFunrclrdegNW528658.225213.144
6LFunrclrdegWD494653.03196.71
7LFunrclrclrNW506648.057209.625
8LFunrclrclrWD499641.669183.413
9LFreldegdegNW492667.872175.869
10LFreldegdegWD498653.43180.58
11LFreldegclrNW539650.16167.51
12LFreldegclrWD512678.119217.209
13LFrelclrdegNW499635.076192.738
⋮⋮⋮⋮⋮⋮⋮⋮⋮
21HFunrclrdegNW553644.6203.261
22HFunrclrdegWD495633.046172.604
23HFunrclrclrNW495619.626176.55
24HFunrclrclrWD532620.838192.689
25HFreldegdegNW508659.266192.637
26HFreldegdegWD546637.793177.496
27HFreldegclrNW507645.493179.962
28HFreldegclrWD517667.685231.145
29HFrelclrdegNW527615.647176.826
30HFrelclrdegWD504620.569173.764
31HFrelclrclrNW546635.319205.19
32HFrelclrclrWD499613.094192.13
+
+
+
+
+
+

5 Complex LMM

+

The following LMM is not the maximal factorial LMM because we do not include interaction terms and associated correlation parameters in the RE structure.

+
+

5.1 Model fit

+
+
contrasts =
+    Dict( :F => EffectsCoding(; levels=["LF", "HF"]) ,
+          :P => EffectsCoding(; levels=["unr", "rel"]),
+          :Q => EffectsCoding(; levels=["deg", "clr"]),
+          :lQ =>EffectsCoding(; levels=["deg", "clr"]),
+          :lT =>EffectsCoding(; levels=["NW", "WD"])
+          );
+
+m_cpx = let
+    form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +
+                               (1+F+P+Q+lQ+lT | subj) +
+                               (1  +P+Q+lQ+lT | item);
+    fit(MixedModel, form, dat; contrasts)
+end
+
+VarCorr(m_cpx)
+
+
Minimizing 2    Time: 0:00:00 ( 0.47  s/it)
+  objective:  12236.319236009644
+Minimizing 94    Time: 0:00:02 (28.47 ms/it)
+  objective:  7249.120182300852
+Minimizing 118    Time: 0:00:02 (25.01 ms/it)
+  objective:  7291.763575891249
+Minimizing 127    Time: 0:00:03 (24.04 ms/it)
+  objective:  7260.015819733898
+Minimizing 136    Time: 0:00:03 (23.33 ms/it)
+  objective:  7265.705633144413
+Minimizing 144    Time: 0:00:03 (22.77 ms/it)
+  objective:  7250.4488876317155
+Minimizing 153    Time: 0:00:03 (22.14 ms/it)
+  objective:  7225.734296682795
+Minimizing 162    Time: 0:00:03 (21.57 ms/it)
+  objective:  7282.113857963857
+Minimizing 171    Time: 0:00:03 (21.07 ms/it)
+  objective:  7194.502401177749
+Minimizing 180    Time: 0:00:03 (20.63 ms/it)
+  objective:  7191.653078945403
+Minimizing 188    Time: 0:00:03 (20.30 ms/it)
+  objective:  7189.129797213707
+Minimizing 197    Time: 0:00:03 (19.91 ms/it)
+  objective:  7188.046411656474
+Minimizing 206    Time: 0:00:04 (19.54 ms/it)
+  objective:  7187.413963547601
+Minimizing 215    Time: 0:00:04 (19.20 ms/it)
+  objective:  7185.895171538268
+Minimizing 224    Time: 0:00:04 (18.89 ms/it)
+  objective:  7185.392720281486
+Minimizing 233    Time: 0:00:04 (18.60 ms/it)
+  objective:  7184.568988840304
+Minimizing 242    Time: 0:00:04 (18.34 ms/it)
+  objective:  7182.692960774513
+Minimizing 251    Time: 0:00:04 (18.11 ms/it)
+  objective:  7182.522314101793
+Minimizing 260    Time: 0:00:04 (17.88 ms/it)
+  objective:  7182.29089946668
+Minimizing 269    Time: 0:00:04 (17.68 ms/it)
+  objective:  7181.754508324871
+Minimizing 278    Time: 0:00:04 (17.47 ms/it)
+  objective:  7180.945383345449
+Minimizing 287    Time: 0:00:04 (17.29 ms/it)
+  objective:  7180.373542670413
+Minimizing 296    Time: 0:00:05 (17.12 ms/it)
+  objective:  7179.226769182374
+Minimizing 305    Time: 0:00:05 (16.96 ms/it)
+  objective:  7178.784230042712
+Minimizing 314    Time: 0:00:05 (16.81 ms/it)
+  objective:  7177.594055525869
+Minimizing 323    Time: 0:00:05 (16.67 ms/it)
+  objective:  7177.577846900931
+Minimizing 332    Time: 0:00:05 (16.53 ms/it)
+  objective:  7176.851940538352
+Minimizing 341    Time: 0:00:05 (16.40 ms/it)
+  objective:  7176.3684658895745
+Minimizing 350    Time: 0:00:05 (16.28 ms/it)
+  objective:  7175.803517624161
+Minimizing 359    Time: 0:00:05 (16.16 ms/it)
+  objective:  7174.947370512575
+Minimizing 368    Time: 0:00:05 (16.06 ms/it)
+  objective:  7173.799157161966
+Minimizing 377    Time: 0:00:06 (15.95 ms/it)
+  objective:  7172.044003247749
+Minimizing 386    Time: 0:00:06 (15.86 ms/it)
+  objective:  7170.623702565538
+Minimizing 395    Time: 0:00:06 (15.76 ms/it)
+  objective:  7167.498474823836
+Minimizing 405    Time: 0:00:06 (15.65 ms/it)
+  objective:  7166.234236515456
+Minimizing 414    Time: 0:00:06 (15.56 ms/it)
+  objective:  7163.279053940776
+Minimizing 423    Time: 0:00:06 (15.47 ms/it)
+  objective:  7166.484821169435
+Minimizing 432    Time: 0:00:06 (15.39 ms/it)
+  objective:  7157.994744801014
+Minimizing 441    Time: 0:00:06 (15.32 ms/it)
+  objective:  7156.9352798284035
+Minimizing 449    Time: 0:00:06 (15.28 ms/it)
+  objective:  7153.823112245293
+Minimizing 458    Time: 0:00:06 (15.22 ms/it)
+  objective:  7150.3162245249505
+Minimizing 467    Time: 0:00:07 (15.16 ms/it)
+  objective:  7149.935835371041
+Minimizing 476    Time: 0:00:07 (15.09 ms/it)
+  objective:  7149.555923262043
+Minimizing 485    Time: 0:00:07 (15.02 ms/it)
+  objective:  7149.399721863012
+Minimizing 494    Time: 0:00:07 (15.04 ms/it)
+  objective:  7149.087433332951
+Minimizing 502    Time: 0:00:07 (15.02 ms/it)
+  objective:  7148.656317523455
+Minimizing 510    Time: 0:00:07 (15.00 ms/it)
+  objective:  7148.592065632587
+Minimizing 518    Time: 0:00:07 (14.99 ms/it)
+  objective:  7148.531024188875
+Minimizing 525    Time: 0:00:07 (14.99 ms/it)
+  objective:  7148.461544048104
+Minimizing 533    Time: 0:00:07 (14.96 ms/it)
+  objective:  7148.3530388399
+Minimizing 542    Time: 0:00:08 (14.90 ms/it)
+  objective:  7148.2535865238815
+Minimizing 552    Time: 0:00:08 (14.83 ms/it)
+  objective:  7148.096511008455
+Minimizing 560    Time: 0:00:08 (14.83 ms/it)
+  objective:  7148.087202572238
+Minimizing 567    Time: 0:00:08 (14.83 ms/it)
+  objective:  7147.968226099975
+Minimizing 575    Time: 0:00:08 (14.80 ms/it)
+  objective:  7147.907102954974
+Minimizing 583    Time: 0:00:08 (14.78 ms/it)
+  objective:  7147.8802277912
+Minimizing 591    Time: 0:00:08 (14.76 ms/it)
+  objective:  7147.8572204082575
+Minimizing 599    Time: 0:00:08 (14.76 ms/it)
+  objective:  7147.843379811635
+Minimizing 608    Time: 0:00:08 (14.72 ms/it)
+  objective:  7147.814327096406
+Minimizing 617    Time: 0:00:09 (14.67 ms/it)
+  objective:  7147.786617758407
+Minimizing 626    Time: 0:00:09 (14.63 ms/it)
+  objective:  7147.746216287041
+Minimizing 635    Time: 0:00:09 (14.58 ms/it)
+  objective:  7147.72352665989
+Minimizing 645    Time: 0:00:09 (14.53 ms/it)
+  objective:  7147.706120026539
+Minimizing 655    Time: 0:00:09 (14.47 ms/it)
+  objective:  7147.685331593435
+Minimizing 665    Time: 0:00:09 (14.42 ms/it)
+  objective:  7147.667208014207
+Minimizing 674    Time: 0:00:09 (14.38 ms/it)
+  objective:  7147.655838866062
+Minimizing 684    Time: 0:00:09 (14.33 ms/it)
+  objective:  7147.648682816166
+Minimizing 694    Time: 0:00:09 (14.29 ms/it)
+  objective:  7147.643906748403
+Minimizing 704    Time: 0:00:10 (14.25 ms/it)
+  objective:  7147.640036536273
+Minimizing 713    Time: 0:00:10 (14.22 ms/it)
+  objective:  7147.649157243677
+Minimizing 723    Time: 0:00:10 (14.17 ms/it)
+  objective:  7147.641059189774
+Minimizing 733    Time: 0:00:10 (14.13 ms/it)
+  objective:  7147.632414838774
+Minimizing 743    Time: 0:00:10 (14.09 ms/it)
+  objective:  7147.630874786184
+Minimizing 752    Time: 0:00:10 (14.05 ms/it)
+  objective:  7147.629970859568
+Minimizing 762    Time: 0:00:10 (14.01 ms/it)
+  objective:  7147.629159600505
+Minimizing 772    Time: 0:00:10 (13.97 ms/it)
+  objective:  7147.62765579575
+Minimizing 782    Time: 0:00:10 (13.93 ms/it)
+  objective:  7147.626155045783
+Minimizing 792    Time: 0:00:11 (13.90 ms/it)
+  objective:  7147.625473967293
+Minimizing 801    Time: 0:00:11 (13.87 ms/it)
+  objective:  7147.624735383692
+Minimizing 811    Time: 0:00:11 (13.83 ms/it)
+  objective:  7147.623986570402
+Minimizing 821    Time: 0:00:11 (13.80 ms/it)
+  objective:  7147.623752366062
+Minimizing 831    Time: 0:00:11 (13.76 ms/it)
+  objective:  7147.623424582311
+Minimizing 841    Time: 0:00:11 (13.73 ms/it)
+  objective:  7147.623085873498
+Minimizing 851    Time: 0:00:11 (13.70 ms/it)
+  objective:  7147.622338565887
+Minimizing 861    Time: 0:00:11 (13.66 ms/it)
+  objective:  7147.621927586488
+Minimizing 871    Time: 0:00:11 (13.63 ms/it)
+  objective:  7147.621667937003
+Minimizing 881    Time: 0:00:11 (13.60 ms/it)
+  objective:  7147.621486368276
+Minimizing 890    Time: 0:00:12 (13.58 ms/it)
+  objective:  7147.621245138782
+Minimizing 900    Time: 0:00:12 (13.56 ms/it)
+  objective:  7147.62111766812
+Minimizing 909    Time: 0:00:12 (13.54 ms/it)
+  objective:  7147.6208043017605
+Minimizing 918    Time: 0:00:12 (13.52 ms/it)
+  objective:  7147.620751938777
+Minimizing 927    Time: 0:00:12 (13.52 ms/it)
+  objective:  7147.62059723712
+Minimizing 935    Time: 0:00:12 (13.53 ms/it)
+  objective:  7147.620403036227
+Minimizing 944    Time: 0:00:12 (13.51 ms/it)
+  objective:  7147.620600623897
+Minimizing 953    Time: 0:00:12 (13.49 ms/it)
+  objective:  7147.619235369456
+Minimizing 962    Time: 0:00:12 (13.46 ms/it)
+  objective:  7147.619017957963
+Minimizing 971    Time: 0:00:13 (13.44 ms/it)
+  objective:  7147.617286513275
+Minimizing 981    Time: 0:00:13 (13.42 ms/it)
+  objective:  7147.617946449571
+Minimizing 991    Time: 0:00:13 (13.39 ms/it)
+  objective:  7147.614530798965
+Minimizing 1001    Time: 0:00:13 (13.36 ms/it)
+  objective:  7147.612787960818
+Minimizing 1011    Time: 0:00:13 (13.34 ms/it)
+  objective:  7147.606365572075
+Minimizing 1021    Time: 0:00:13 (13.32 ms/it)
+  objective:  7147.605624336107
+Minimizing 1031    Time: 0:00:13 (13.29 ms/it)
+  objective:  7147.598997387338
+Minimizing 1040    Time: 0:00:13 (13.28 ms/it)
+  objective:  7147.594605025298
+Minimizing 1049    Time: 0:00:13 (13.26 ms/it)
+  objective:  7147.589251057229
+Minimizing 1058    Time: 0:00:14 (13.24 ms/it)
+  objective:  7147.587092246818
+Minimizing 1067    Time: 0:00:14 (13.23 ms/it)
+  objective:  7147.583649293367
+Minimizing 1077    Time: 0:00:14 (13.20 ms/it)
+  objective:  7147.579369436586
+Minimizing 1087    Time: 0:00:14 (13.18 ms/it)
+  objective:  7147.5754409231195
+Minimizing 1097    Time: 0:00:14 (13.16 ms/it)
+  objective:  7147.572885912015
+Minimizing 1107    Time: 0:00:14 (13.14 ms/it)
+  objective:  7147.569409032434
+Minimizing 1117    Time: 0:00:14 (13.12 ms/it)
+  objective:  7147.567171456462
+Minimizing 1127    Time: 0:00:14 (13.10 ms/it)
+  objective:  7147.566020002094
+Minimizing 1137    Time: 0:00:14 (13.08 ms/it)
+  objective:  7147.564517396884
+Minimizing 1147    Time: 0:00:14 (13.07 ms/it)
+  objective:  7147.563380941506
+Minimizing 1156    Time: 0:00:15 (13.05 ms/it)
+  objective:  7147.562783241351
+Minimizing 1165    Time: 0:00:15 (13.04 ms/it)
+  objective:  7147.561684794379
+Minimizing 1174    Time: 0:00:15 (13.02 ms/it)
+  objective:  7147.560752567861
+Minimizing 1183    Time: 0:00:15 (13.01 ms/it)
+  objective:  7147.560016035949
+Minimizing 1192    Time: 0:00:15 (13.00 ms/it)
+  objective:  7147.559163946607
+Minimizing 1201    Time: 0:00:15 (12.98 ms/it)
+  objective:  7147.559037397831
+Minimizing 1210    Time: 0:00:15 (12.97 ms/it)
+  objective:  7147.558254661318
+Minimizing 1219    Time: 0:00:15 (12.96 ms/it)
+  objective:  7147.5581615072715
+Minimizing 1229    Time: 0:00:15 (12.94 ms/it)
+  objective:  7147.557388884788
+Minimizing 1239    Time: 0:00:16 (12.93 ms/it)
+  objective:  7147.556740704144
+Minimizing 1249    Time: 0:00:16 (12.91 ms/it)
+  objective:  7147.556070237347
+Minimizing 1259    Time: 0:00:16 (12.90 ms/it)
+  objective:  7147.5569175676865
+Minimizing 1269    Time: 0:00:16 (12.88 ms/it)
+  objective:  7147.555004775328
+Minimizing 1279    Time: 0:00:16 (12.87 ms/it)
+  objective:  7147.554706891277
+Minimizing 1289    Time: 0:00:16 (12.85 ms/it)
+  objective:  7147.554257919288
+Minimizing 1299    Time: 0:00:16 (12.84 ms/it)
+  objective:  7147.553951891185
+Minimizing 1309    Time: 0:00:16 (12.82 ms/it)
+  objective:  7147.553362379362
+Minimizing 1319    Time: 0:00:16 (12.81 ms/it)
+  objective:  7147.553160958638
+Minimizing 1328    Time: 0:00:16 (12.80 ms/it)
+  objective:  7147.55287535691
+Minimizing 1338    Time: 0:00:17 (12.78 ms/it)
+  objective:  7147.552613773639
+Minimizing 1348    Time: 0:00:17 (12.77 ms/it)
+  objective:  7147.552311581183
+Minimizing 1358    Time: 0:00:17 (12.75 ms/it)
+  objective:  7147.551816636063
+Minimizing 1367    Time: 0:00:17 (12.75 ms/it)
+  objective:  7147.551755227561
+Minimizing 1377    Time: 0:00:17 (12.73 ms/it)
+  objective:  7147.551631565507
+Minimizing 1387    Time: 0:00:17 (12.72 ms/it)
+  objective:  7147.551611470475
+Minimizing 1397    Time: 0:00:17 (12.71 ms/it)
+  objective:  7147.5513987918075
+Minimizing 1407    Time: 0:00:17 (12.69 ms/it)
+  objective:  7147.551219481732
+Minimizing 1417    Time: 0:00:17 (12.68 ms/it)
+  objective:  7147.551171980433
+Minimizing 1427    Time: 0:00:18 (12.67 ms/it)
+  objective:  7147.551061215947
+Minimizing 1437    Time: 0:00:18 (12.66 ms/it)
+  objective:  7147.551117184621
+Minimizing 1447    Time: 0:00:18 (12.65 ms/it)
+  objective:  7147.55091185511
+Minimizing 1457    Time: 0:00:18 (12.63 ms/it)
+  objective:  7147.550822764177
+Minimizing 1467    Time: 0:00:18 (12.62 ms/it)
+  objective:  7147.5507422044675
+Minimizing 1477    Time: 0:00:18 (12.61 ms/it)
+  objective:  7147.550911690253
+Minimizing 1487    Time: 0:00:18 (12.60 ms/it)
+  objective:  7147.550803823427
+Minimizing 1497    Time: 0:00:18 (12.58 ms/it)
+  objective:  7147.550676876308
+Minimizing 1507    Time: 0:00:18 (12.57 ms/it)
+  objective:  7147.55061104588
+Minimizing 1517    Time: 0:00:19 (12.56 ms/it)
+  objective:  7147.5506926822745
+Minimizing 1527    Time: 0:00:19 (12.55 ms/it)
+  objective:  7147.550577280057
+Minimizing 1537    Time: 0:00:19 (12.54 ms/it)
+  objective:  7147.550555210411
+Minimizing 1547    Time: 0:00:19 (12.53 ms/it)
+  objective:  7147.5505321694445
+Minimizing 1557    Time: 0:00:19 (12.52 ms/it)
+  objective:  7147.550605309394
+Minimizing 1567    Time: 0:00:19 (12.51 ms/it)
+  objective:  7147.550497050396
+Minimizing 1577    Time: 0:00:19 (12.50 ms/it)
+  objective:  7147.550527636891
+Minimizing 1587    Time: 0:00:19 (12.49 ms/it)
+  objective:  7147.550474180846
+Minimizing 1597    Time: 0:00:19 (12.48 ms/it)
+  objective:  7147.550447320714
+Minimizing 1607    Time: 0:00:20 (12.47 ms/it)
+  objective:  7147.550437475353
+Minimizing 1617    Time: 0:00:20 (12.46 ms/it)
+  objective:  7147.55043172267
+Minimizing 1625    Time: 0:00:20 (12.46 ms/it)
+  objective:  7147.550417227425
+Minimizing 1635    Time: 0:00:20 (12.45 ms/it)
+  objective:  7147.550407555644
+Minimizing 1645    Time: 0:00:20 (12.44 ms/it)
+  objective:  7147.5504031649125
+Minimizing 1654    Time: 0:00:20 (12.43 ms/it)
+  objective:  7147.550395744943
+Minimizing 1664    Time: 0:00:20 (12.43 ms/it)
+  objective:  7147.550391444152
+Minimizing 1674    Time: 0:00:20 (12.42 ms/it)
+  objective:  7147.5503806632105
+Minimizing 1684    Time: 0:00:20 (12.41 ms/it)
+  objective:  7147.550385842741
+Minimizing 1694    Time: 0:00:21 (12.40 ms/it)
+  objective:  7147.550371837158
+Minimizing 1703    Time: 0:00:21 (12.40 ms/it)
+  objective:  7147.550364911636
+Minimizing 1712    Time: 0:00:21 (12.39 ms/it)
+  objective:  7147.550363162931
+Minimizing 1717    Time: 0:00:21 (12.41 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)0.003202240.05658830
P: rel0.000128920.01135418+0.05
Q: clr0.000159990.01264876+0.36+0.38
lQ: clr0.000034760.00589548+0.38+0.03+0.03
lT: WD0.000157500.01254988-0.11-0.87-0.01-0.35
subj(Intercept)0.030617830.17497951
F: HF0.000044400.00666334+0.36
P: rel0.000127340.01128461+0.35+0.89
Q: clr0.000789850.02810434+0.41+0.45+0.73
lQ: clr0.000116150.01077740+0.06+0.17+0.18+0.58
lT: WD0.001045310.03233128+0.26-0.10-0.02+0.37+0.51
Residual0.085699550.29274486
+
+
+
+
issingular(m_cpx)
+
+
true
+
+
+
+
MixedModels.PCA(m_cpx)
+
+
(item = 
+Principal components based on correlation matrix
+ (Intercept)   1.0     .      .      .      .
+ P: rel        0.05   1.0     .      .      .
+ Q: clr        0.36   0.38   1.0     .      .
+ lQ: clr       0.38   0.03   0.03   1.0     .
+ lT: WD       -0.11  -0.87  -0.01  -0.35   1.0
+
+Normalized cumulative variances:
+[0.4223, 0.6865, 0.906, 1.0, 1.0]
+
+Component loadings
+                PC1    PC2    PC3    PC4    PC5
+ (Intercept)  -0.3   -0.67   0.02  -0.67   0.07
+ P: rel       -0.6    0.38   0.22  -0.05   0.67
+ Q: clr       -0.31  -0.34   0.7    0.46  -0.28
+ lQ: clr      -0.31  -0.41  -0.62   0.55   0.19
+ lT: WD        0.6   -0.34   0.27   0.15   0.66, subj = 
+Principal components based on correlation matrix
+ (Intercept)   1.0     .      .      .      .      .
+ F: HF         0.36   1.0     .      .      .      .
+ P: rel        0.35   0.89   1.0     .      .      .
+ Q: clr        0.41   0.45   0.73   1.0     .      .
+ lQ: clr       0.06   0.17   0.18   0.58   1.0     .
+ lT: WD        0.26  -0.1   -0.02   0.37   0.51   1.0
+
+Normalized cumulative variances:
+[0.4768, 0.7358, 0.8812, 0.9431, 1.0, 1.0]
+
+Component loadings
+                PC1    PC2    PC3    PC4    PC5    PC6
+ (Intercept)  -0.34   0.02   0.84   0.31  -0.27  -0.11
+ F: HF        -0.45   0.42  -0.12  -0.41  -0.44   0.5
+ P: rel       -0.51   0.35  -0.16  -0.17   0.27  -0.7
+ Q: clr       -0.52  -0.15  -0.11   0.41   0.57   0.45
+ lQ: clr      -0.32  -0.51  -0.43   0.3   -0.56  -0.21
+ lT: WD       -0.21  -0.65   0.24  -0.67   0.16  -0.0)
+
+
+

Variance-covariance matrix of random-effect structure suggests overparameterization for both subject-related and item-related components.

+

We don’t look at fixed effects before model selection.

+
+
+

5.2 VCs and CPs

+

We can also look separately at item- and subj-related VCs and CPs for subjects and items.

+
+
first(m_cpx.λ)
+
+
5×5 LinearAlgebra.LowerTriangular{Float64, Matrix{Float64}}:
+  0.193302      ⋅             ⋅           ⋅         ⋅ 
+  0.00210882   0.0387279      ⋅           ⋅         ⋅ 
+  0.0155967    0.0156512     0.0371304    ⋅         ⋅ 
+  0.00757715   0.000160552  -0.002579    0.018479   ⋅ 
+ -0.00483918  -0.037088      0.0174059  -0.011658  0.0
+
+
+

VP is zero for last diagonal entry; not supported by data.

+
+
last(m_cpx.λ)
+
+
6×6 LinearAlgebra.LowerTriangular{Float64, Matrix{Float64}}:
+ 0.59772       ⋅          ⋅           ⋅          ⋅          ⋅ 
+ 0.00814528   0.0212543   ⋅           ⋅          ⋅          ⋅ 
+ 0.0134425    0.0317765  0.0171892    ⋅          ⋅          ⋅ 
+ 0.0392834    0.0309145  0.0686994   0.0446995   ⋅          ⋅ 
+ 0.00205584   0.0057383  0.00227977  0.0362352  0.0         ⋅ 
+ 0.0283307   -0.0222477  0.0131949   0.0578621  0.0731409  0.0450343
+
+
+

VP is zero for fourth diagonal entry; not supported by data.

+
+
+
+

6 Zero-correlation parameter LMM

+
+

6.1 Model fit

+

We take out correlation parameters.

+
+
m_zcp = let
+    form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +
+                       zerocorr(1+F+P+Q+lQ+lT | subj) +
+                       zerocorr(1  +P+Q+lQ+lT | item);
+    fit(MixedModel, form, dat; contrasts)
+end
+
+VarCorr(m_zcp)
+issingular(m_zcp)
+MixedModels.PCA(m_zcp)
+
+
Minimizing 10    Time: 0:00:00 (10.81 ms/it)
+  objective:  12062.171283470983
+Minimizing 21    Time: 0:00:00 (10.21 ms/it)
+  objective:  11804.406038310086
+Minimizing 32    Time: 0:00:00 (10.05 ms/it)
+  objective:  7418.254529819783
+Minimizing 43    Time: 0:00:00 (10.01 ms/it)
+  objective:  7303.449354651699
+Minimizing 54    Time: 0:00:00 ( 9.99 ms/it)
+  objective:  7248.369118002118
+Minimizing 65    Time: 0:00:00 ( 9.97 ms/it)
+  objective:  7228.106822778762
+Minimizing 76    Time: 0:00:00 ( 9.94 ms/it)
+  objective:  7221.239180885949
+Minimizing 87    Time: 0:00:00 ( 9.91 ms/it)
+  objective:  7216.145839563162
+Minimizing 98    Time: 0:00:00 ( 9.92 ms/it)
+  objective:  7210.857335837595
+Minimizing 109    Time: 0:00:01 ( 9.90 ms/it)
+  objective:  7203.337839287926
+Minimizing 120    Time: 0:00:01 ( 9.87 ms/it)
+  objective:  7200.461288500643
+Minimizing 131    Time: 0:00:01 ( 9.85 ms/it)
+  objective:  7194.188203194858
+Minimizing 142    Time: 0:00:01 ( 9.84 ms/it)
+  objective:  7188.62093968041
+Minimizing 153    Time: 0:00:01 ( 9.83 ms/it)
+  objective:  7188.552220867151
+Minimizing 164    Time: 0:00:01 ( 9.84 ms/it)
+  objective:  7188.538545300813
+Minimizing 174    Time: 0:00:01 ( 9.85 ms/it)
+  objective:  7188.5159700931645
+Minimizing 184    Time: 0:00:01 ( 9.94 ms/it)
+  objective:  7188.504641846532
+Minimizing 194    Time: 0:00:01 ( 9.99 ms/it)
+  objective:  7188.502722500439
+Minimizing 203    Time: 0:00:02 (10.05 ms/it)
+  objective:  7188.500891905364
+Minimizing 213    Time: 0:00:02 (10.08 ms/it)
+  objective:  7188.500141354659
+Minimizing 223    Time: 0:00:02 (10.11 ms/it)
+  objective:  7188.491875002939
+Minimizing 233    Time: 0:00:02 (10.13 ms/it)
+  objective:  7188.49129998534
+Minimizing 243    Time: 0:00:02 (10.15 ms/it)
+  objective:  7188.489739323119
+Minimizing 253    Time: 0:00:02 (10.18 ms/it)
+  objective:  7188.489571414967
+Minimizing 263    Time: 0:00:02 (10.19 ms/it)
+  objective:  7188.489439972358
+Minimizing 273    Time: 0:00:02 (10.21 ms/it)
+  objective:  7188.489442655698
+Minimizing 282    Time: 0:00:02 (10.24 ms/it)
+  objective:  7188.489422058375
+Minimizing 291    Time: 0:00:02 (10.28 ms/it)
+  objective:  7188.489416642211
+Minimizing 291    Time: 0:00:02 (10.28 ms/it)
+
+
+
(item = 
+Principal components based on correlation matrix
+ (Intercept)  1.0  .    .    .    .
+ P: rel       0.0  1.0  .    .    .
+ Q: clr       0.0  0.0  1.0  .    .
+ lQ: clr      0.0  0.0  0.0  1.0  .
+ lT: WD       0.0  0.0  0.0  0.0  1.0
+
+Normalized cumulative variances:
+[0.2, 0.4, 0.6, 0.8, 1.0]
+
+Component loadings
+               PC1   PC2   PC3   PC4   PC5
+ (Intercept)  1.0   0.0   0.0   0.0   0.0
+ P: rel       0.0   1.0   0.0   0.0   0.0
+ Q: clr       0.0   0.0   1.0   0.0   0.0
+ lQ: clr      0.0   0.0   0.0   1.0   0.0
+ lT: WD       0.0   0.0   0.0   0.0   1.0, subj = 
+Principal components based on correlation matrix
+ (Intercept)  1.0  .    .    .    .    .
+ F: HF        0.0  1.0  .    .    .    .
+ P: rel       0.0  0.0  1.0  .    .    .
+ Q: clr       0.0  0.0  0.0  1.0  .    .
+ lQ: clr      0.0  0.0  0.0  0.0  1.0  .
+ lT: WD       0.0  0.0  0.0  0.0  0.0  1.0
+
+Normalized cumulative variances:
+[0.1667, 0.3333, 0.5, 0.6667, 0.8333, 1.0]
+
+Component loadings
+               PC1   PC2   PC3   PC4   PC5   PC6
+ (Intercept)  1.0   0.0   0.0   0.0   0.0   0.0
+ F: HF        0.0   1.0   0.0   0.0   0.0   0.0
+ P: rel       0.0   0.0   1.0   0.0   0.0   0.0
+ Q: clr       0.0   0.0   0.0   0.0   1.0   0.0
+ lQ: clr      0.0   0.0   0.0   1.0   0.0   0.0
+ lT: WD       0.0   0.0   0.0   0.0   0.0   1.0)
+
+
+
+
MixedModels.likelihoodratiotest(m_zcp, m_cpx)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + F + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lQ + lT | item)447188
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + F + P + Q + lQ + lT | subj) + (1 + P + Q + lQ + lT | item)69714841250.0233
+
+
+

Looks ok. It might be a good idea to prune the LMM by removing small VCs.

+
+
m_zcp2 = let
+    form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +
+                       zerocorr(1  +P+Q+lQ+lT | subj) +
+                       zerocorr(1  +P+Q   +lT | item);
+    fit(MixedModel, form, dat; contrasts)
+end
+VarCorr(m_zcp2)
+
+
Minimizing 2    Time: 0:00:00 (56.46 ms/it)
+  objective:  11209.21592702474
+Minimizing 20    Time: 0:00:00 (10.86 ms/it)
+  objective:  9509.729918166682
+Minimizing 38    Time: 0:00:00 ( 8.38 ms/it)
+  objective:  7311.387549009839
+Minimizing 56    Time: 0:00:00 ( 7.51 ms/it)
+  objective:  7247.174405763663
+Minimizing 74    Time: 0:00:00 ( 7.06 ms/it)
+  objective:  7240.910845804518
+Minimizing 93    Time: 0:00:00 ( 6.75 ms/it)
+  objective:  7236.328263902658
+Minimizing 111    Time: 0:00:00 ( 6.58 ms/it)
+  objective:  7227.556398242914
+Minimizing 129    Time: 0:00:00 ( 6.46 ms/it)
+  objective:  7195.345776753049
+Minimizing 148    Time: 0:00:00 ( 6.33 ms/it)
+  objective:  7190.682639021281
+Minimizing 167    Time: 0:00:01 ( 6.24 ms/it)
+  objective:  7188.96412272415
+Minimizing 186    Time: 0:00:01 ( 6.17 ms/it)
+  objective:  7188.6956407151665
+Minimizing 205    Time: 0:00:01 ( 6.12 ms/it)
+  objective:  7188.575330252084
+Minimizing 224    Time: 0:00:01 ( 6.07 ms/it)
+  objective:  7188.567615979038
+Minimizing 243    Time: 0:00:01 ( 6.02 ms/it)
+  objective:  7188.564560144665
+Minimizing 262    Time: 0:00:01 ( 5.99 ms/it)
+  objective:  7188.563992529358
+Minimizing 279    Time: 0:00:01 ( 6.00 ms/it)
+  objective:  7188.563962432219
+Minimizing 298    Time: 0:00:01 ( 5.98 ms/it)
+  objective:  7188.563961681864
+Minimizing 300    Time: 0:00:01 ( 5.97 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)0.003202030.05658650
P: rel0.000071390.00844898.
Q: clr0.000147200.01213252..
lT: WD0.000117080.01082057...
subj(Intercept)0.030610990.17495996
P: rel0.000099210.00996029.
Q: clr0.000773050.02780380..
lQ: clr0.000118890.01090373...
lT: WD0.001061470.03258023....
Residual0.085915850.29311406
+
+
+
+
MixedModels.likelihoodratiotest(m_zcp2, m_zcp, m_cpx)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lT | item)427189
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + F + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lQ + lT | item)447188020.9634
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + F + P + Q + lQ + lT | subj) + (1 + P + Q + lQ + lT | item)69714841250.0233
+
+
+

We can perhaps remove some more.

+
+
m_zcp3 = let
+    form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +
+                       zerocorr(1    +Q   +lT | subj) + (1 | item);
+    fit(MixedModel, form, dat; contrasts)
+end
+VarCorr(m_zcp3)
+
+
Minimizing 36    Time: 0:00:00 ( 2.79 ms/it)
+  objective:  7204.514658399277
+Minimizing 97    Time: 0:00:00 ( 1.51 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)0.00320490.0566117
subj(Intercept)0.03061690.1749768
Q: clr0.00076290.0276198.
lT: WD0.00106450.0326267..
Residual0.08647520.2940667
+
+
+
+
MixedModels.likelihoodratiotest(m_zcp3, m_zcp2, m_zcp, m_cpx)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + Q + lT | subj) + (1 | item)377196
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lT | item)427189850.1781
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + F + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lQ + lT | item)447188020.9634
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + F + P + Q + lQ + lT | subj) + (1 + P + Q + lQ + lT | item)69714841250.0233
+
+
+

And another iteration.

+
+
m_zcp4 = let
+    form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +
+                       zerocorr(1         +lT | subj) + (1 | item);
+    fit(MixedModel, form, dat; contrasts)
+end
+VarCorr(m_zcp4)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)0.00320670.0566278
subj(Intercept)0.03066080.1751022
lT: WD0.00108960.0330097.
Residual0.08723300.2953523
+
+
+
+
MixedModels.likelihoodratiotest(m_zcp4, m_zcp3, m_zcp2, m_zcp, m_cpx)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + lT | subj) + (1 | item)367259
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + Q + lT | subj) + (1 | item)377196631<1e-14
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lT | item)427189850.1781
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + F + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lQ + lT | item)447188020.9634
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + F + P + Q + lQ + lT | subj) + (1 + P + Q + lQ + lT | item)69714841250.0233
+
+
+

Too much removed. Stay with m_zcp3, but extend with CPs.

+
+
m_prm = let
+    form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +
+                               (1+    Q+lT | subj) + (1 | item);
+    fit(MixedModel, form, dat; contrasts)
+end
+VarCorr(m_prm)
+
+
Minimizing 132    Time: 0:00:00 ( 0.76 ms/it)
+  objective:  7199.899053847559
+Minimizing 244    Time: 0:00:00 ( 0.77 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)0.00320100.0565773
subj(Intercept)0.03062160.1749903
Q: clr0.00076260.0276159+0.42
lT: WD0.00106210.0325898+0.26+0.38
Residual0.08647690.2940695
+
+
+
+

6.1.1 post-hoc LMM

+
+
m_prm = let
+    form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +
+                               (1+    Q+lT | subj) + (1 | item);
+    fit(MixedModel, form, dat; contrasts)
+end
+VarCorr(m_prm)
+
+
Minimizing 133    Time: 0:00:00 ( 0.75 ms/it)
+  objective:  7199.569489268595
+Minimizing 244    Time: 0:00:00 ( 0.76 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)0.00320100.0565773
subj(Intercept)0.03062160.1749903
Q: clr0.00076260.0276159+0.42
lT: WD0.00106210.0325898+0.26+0.38
Residual0.08647690.2940695
+
+
+
+
+
+

6.2 VCs and CPs

+
+
MixedModels.likelihoodratiotest(m_zcp3, m_prm, m_cpx)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + Q + lT | subj) + (1 | item)377196
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + Q + lT | subj) + (1 | item)4071801630.0011
:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + F + P + Q + lQ + lT | subj) + (1 + P + Q + lQ + lT | item)69714833290.2916
+
+
+

The LRT favors the complex LMM, but not that χ² < 2*(χ²-dof); AIC and BIC suggest against selection.

+
+
gof_summary = let
+  nms = [:m_zcp3, :m_prm, :m_cpx]
+  mods = eval.(nms)
+  lrt = MixedModels.likelihoodratiotest(m_zcp3, m_prm, m_cpx)
+  DataFrame(;
+    name = nms,
+    dof=dof.(mods),
+    deviance=round.(deviance.(mods), digits=0),
+    AIC=round.(aic.(mods),digits=0),
+    AICc=round.(aicc.(mods),digits=0),
+    BIC=round.(bic.(mods),digits=0),
+    χ²=vcat(:., round.(lrt.tests.deviancediff, digits=0)),
+    χ²_dof=vcat(:., round.(lrt.tests.dofdiff, digits=0)),
+    pvalue=vcat(:., round.(lrt.tests.pvalues, digits=3))
+  )
+end
+
+
3×9 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RownamedofdevianceAICAICcBICχ²χ²_dofpvalue
SymbolInt64Float64Float64Float64Float64AnyAnyAny
1m_zcp3377196.07270.07270.07555.0...
2m_prm407180.07260.07260.07568.016.03.00.001
3m_cpx697148.07286.07286.07817.033.029.00.292
+
+
+
+
+
+
+

7 Parsimonious LMM - replication of MRK17 LMM

+

The LMM is not nested in the previous sequence.

+
+

7.1 Crossed fixed effects

+
+
m_mrk17_crossed =let
+   form = @formula (1000/rt) ~ 1 + F*P*Q*lQ*lT +
+        (1+Q | subj) + zerocorr(0+lT | subj) + zerocorr(1 + P | item) ;
+    fit(MixedModel, form, dat; contrasts)
+end
+
+VarCorr(m_prm)
+
+
Minimizing 67    Time: 0:00:00 ( 1.49 ms/it)
+  objective:  7232.27363287469
+Minimizing 135    Time: 0:00:00 ( 1.50 ms/it)
+  objective:  7186.823792113832
+Minimizing 167    Time: 0:00:00 ( 1.50 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)0.00320100.0565773
subj(Intercept)0.03062160.1749903
Q: clr0.00076260.0276159+0.42
lT: WD0.00106210.0325898+0.26+0.38
Residual0.08647690.2940695
+
+
+
+
show(m_mrk17_crossed)
+
+
Linear mixed model fit by maximum likelihood
+ :(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + Q | subj) + zerocorr(0 + lT | subj) + zerocorr(1 + P | item)
+   logLik   -2 logLik     AIC       AICc        BIC    
+ -3593.4086  7186.8171  7264.8171  7265.0077  7565.3349
+
+Variance components:
+            Column    Variance   Std.Dev.    Corr.
+item     (Intercept)  0.00320570 0.05661895
+         P: rel       0.00006692 0.00818053   .  
+subj     (Intercept)  0.03061792 0.17497978
+         Q: clr       0.00076255 0.02761426 +0.42
+         lT: WD       0.00106214 0.03259043   .     .  
+Residual              0.08640796 0.29395231
+ Number of obs: 16409; levels of grouping factors: 240, 73
+
+  Fixed-effects parameters:
+─────────────────────────────────────────────────────────────────────────────────────
+                                                   Coef.  Std. Error      z  Pr(>|z|)
+─────────────────────────────────────────────────────────────────────────────────────
+(Intercept)                                  1.63743      0.0209305   78.23    <1e-99
+F: HF                                        0.019366     0.00431956   4.48    <1e-05
+P: rel                                       0.0188081    0.00236112   7.97    <1e-14
+Q: clr                                       0.042901     0.00396839  10.81    <1e-26
+lQ: clr                                      0.00174652   0.00231531   0.75    0.4506
+lT: WD                                       0.00836024   0.00446189   1.87    0.0610
+F: HF & P: rel                              -0.00711249   0.00236171  -3.01    0.0026
+F: HF & Q: clr                              -0.00141259   0.00230067  -0.61    0.5392
+P: rel & Q: clr                              0.00134141   0.00230239   0.58    0.5602
+F: HF & lQ: clr                             -0.00105227   0.00232139  -0.45    0.6503
+P: rel & lQ: clr                            -0.00240742   0.00232055  -1.04    0.2995
+Q: clr & lQ: clr                             0.00759804   0.00231922   3.28    0.0011
+F: HF & lT: WD                               0.000536559  0.00231365   0.23    0.8166
+P: rel & lT: WD                              2.93106e-5   0.00231751   0.01    0.9899
+Q: clr & lT: WD                              0.0018553    0.00231773   0.80    0.4234
+lQ: clr & lT: WD                            -0.00524633   0.00231607  -2.27    0.0235
+F: HF & P: rel & Q: clr                     -0.00036224   0.00230272  -0.16    0.8750
+F: HF & P: rel & lQ: clr                    -0.00117753   0.00232025  -0.51    0.6118
+F: HF & Q: clr & lQ: clr                     0.00273697   0.00232283   1.18    0.2387
+P: rel & Q: clr & lQ: clr                   -0.0039323    0.0023222   -1.69    0.0904
+F: HF & P: rel & lT: WD                      0.0019864    0.00231879   0.86    0.3916
+F: HF & Q: clr & lT: WD                     -0.00112586   0.00231657  -0.49    0.6270
+P: rel & Q: clr & lT: WD                     0.000261834  0.00231924   0.11    0.9101
+F: HF & lQ: clr & lT: WD                     0.00150677   0.00232198   0.65    0.5164
+P: rel & lQ: clr & lT: WD                    8.67915e-5   0.00232177   0.04    0.9702
+Q: clr & lQ: clr & lT: WD                    0.00916752   0.00231827   3.95    <1e-04
+F: HF & P: rel & Q: clr & lQ: clr           -0.00234443   0.00231978  -1.01    0.3122
+F: HF & P: rel & Q: clr & lT: WD            -0.00148723   0.00232057  -0.64    0.5216
+F: HF & P: rel & lQ: clr & lT: WD            0.00308764   0.00232145   1.33    0.1835
+F: HF & Q: clr & lQ: clr & lT: WD            0.00393597   0.00232128   1.70    0.0900
+P: rel & Q: clr & lQ: clr & lT: WD           0.00202623   0.00232125   0.87    0.3827
+F: HF & P: rel & Q: clr & lQ: clr & lT: WD   0.00144888   0.00231838   0.62    0.5320
+─────────────────────────────────────────────────────────────────────────────────────
+
+
+

Finally, a look at the fixed effects. The four-factor interaction reported in Masson & Kliegl (2013) was not replicated.

+
+
+

7.2 Nested fixed effects

+
+
m_mrk17_nested =let
+   form = @formula (1000/rt) ~ 1 + Q/(F/P) +
+        (1+Q | subj) + zerocorr(0+lT | subj) + zerocorr(1 + P | item) ;
+    fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 74    Time: 0:00:00 ( 1.35 ms/it)
+  objective:  7270.272343414747
+Minimizing 147    Time: 0:00:00 ( 1.37 ms/it)
+  objective:  7237.687119961029
+Minimizing 172    Time: 0:00:00 ( 1.37 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)1.63740.020978.20<1e-990.05650.1751
Q: clr0.04280.004010.74<1e-260.0278
Q: deg & F: HF0.02090.00494.26<1e-04
Q: clr & F: HF0.01790.00493.650.0003
Q: deg & F: LF & P: rel0.02440.00475.18<1e-06
Q: clr & F: LF & P: rel0.02780.00475.94<1e-08
Q: deg & F: HF & P: rel0.01100.00472.350.0186
Q: clr & F: HF & P: rel0.01260.00462.710.0067
P: rel0.0086
lT: WD0.0337
Residual0.2944
+
+
+
+
+
+

8 Questions from the Discussion forum

+
+

8.1 Nesting within products of factors

+

Include parenthesis

+
+
m_mrk17_nested =let
+   form = @formula (1000/rt) ~ 1 + Q/(F/P) +
+        (1+Q | subj) + zerocorr(0+lT | subj) + zerocorr(1 + P | item) ;
+    fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 73    Time: 0:00:00 ( 1.38 ms/it)
+  objective:  7271.786376361126
+Minimizing 147    Time: 0:00:00 ( 1.38 ms/it)
+  objective:  7237.687119961029
+Minimizing 172    Time: 0:00:00 ( 1.37 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)1.63740.020978.20<1e-990.05650.1751
Q: clr0.04280.004010.74<1e-260.0278
Q: deg & F: HF0.02090.00494.26<1e-04
Q: clr & F: HF0.01790.00493.650.0003
Q: deg & F: LF & P: rel0.02440.00475.18<1e-06
Q: clr & F: LF & P: rel0.02780.00475.94<1e-08
Q: deg & F: HF & P: rel0.01100.00472.350.0186
Q: clr & F: HF & P: rel0.01260.00462.710.0067
P: rel0.0086
lT: WD0.0337
Residual0.2944
+
+
+
+
+

8.2 Selection in fixed effects

+
+
using RegressionFormulae
+# m_prm_5 is equivalent to m_prm
+m_prm_5 = let
+    form = @formula (1000/rt) ~ 1+(F+P+Q+lQ+lT)^5 + (1+Q+lT | subj) + (1 | item);
+    fit(MixedModel, form, dat; contrasts)
+end;
+
+m_prm_4 = let
+    form = @formula (1000/rt) ~ 1+(F+P+Q+lQ+lT)^4 + (1+Q+lT | subj) + (1 | item);
+    fit(MixedModel, form, dat; contrasts)
+end;
+
+m_prm_3 = let
+    form = @formula (1000/rt) ~ 1+(F+P+Q+lQ+lT)^3 + (1+Q+lT | subj) + (1 | item);
+    fit(MixedModel, form, dat; contrasts)
+end;
+
+m_prm_2 = let
+    form = @formula (1000/rt) ~ 1+(F+P+Q+lQ+lT)^2 + (1+Q+lT | subj) + (1 | item);
+    fit(MixedModel, form, dat; contrasts)
+end;
+
+m_prm_1 = let
+    form = @formula (1000/rt) ~ 1+ F+P+Q+lQ+lT +   (1+Q+lT | subj) + (1 | item);
+    fit(MixedModel, form, dat; contrasts)
+end;
+
+# Compare the fits
+gof_summary = let
+  nms = [:m_prm_1, :m_prm_2, :m_prm_3, :m_prm_4, :m_prm_5]
+  mods = eval.(nms)
+  lrt = MixedModels.likelihoodratiotest(m_prm_1, m_prm_2, m_prm_3, m_prm_4, m_prm_5)
+  DataFrame(;
+    name = nms,
+    dof=dof.(mods),
+    deviance=deviance.(mods),
+    AIC=aic.(mods),
+    AICc=aicc.(mods),
+    BIC=bic.(mods),
+    χ²=vcat(:.,lrt.tests.deviancediff),
+    χ²_dof=vcat(:.,lrt.tests.dofdiff),
+    pvalue=vcat(:., round.(lrt.tests.pvalues, digits=3))
+  )
+end
+
+
Minimizing 133    Time: 0:00:00 ( 0.76 ms/it)
+  objective:  7199.569489268595
+Minimizing 244    Time: 0:00:00 ( 0.76 ms/it)
+Minimizing 136    Time: 0:00:00 ( 0.74 ms/it)
+  objective:  7378.040963546835
+Minimizing 273    Time: 0:00:00 ( 0.74 ms/it)
+  objective:  7180.613748762729
+Minimizing 321    Time: 0:00:00 ( 0.74 ms/it)
+Minimizing 133    Time: 0:00:00 ( 0.76 ms/it)
+  objective:  7210.014080210343
+Minimizing 257    Time: 0:00:00 ( 0.76 ms/it)
+Minimizing 147    Time: 0:00:00 ( 0.68 ms/it)
+  objective:  7209.592926189283
+Minimizing 224    Time: 0:00:00 ( 0.68 ms/it)
+Minimizing 155    Time: 0:00:00 ( 0.65 ms/it)
+  objective:  7237.619383975656
+Minimizing 241    Time: 0:00:00 ( 0.65 ms/it)
+
+
+
5×9 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RownamedofdevianceAICAICcBICχ²χ²_dofpvalue
SymbolInt64Float64Float64Float64Float64AnyAnyAny
1m_prm_1147237.397265.397265.417373.27...
2m_prm_2247209.27257.27257.277442.1328.1868100.002
3m_prm_3347187.577255.577255.717517.5621.6317100.017
4m_prm_4397180.617258.617258.87559.136.9586350.224
5m_prm_5407180.217260.217260.417568.440.39859110.528
+
+
+
+

Depending on the level of precision of your hypothesis. You could stay with main effect (BIC), include 2-factor interactions (AIC; also called simple interactions) or include 3-factor interactions [χ² < 2*(χ²-dof); also called 2-way interactions].

+
+
+

8.3 Posthoc LMM

+

We are using only three factors for the illustruation.

+
+
m_prm3 = let
+    form = @formula (1000/rt) ~ 1 + lT*lQ*Q +
+                               (1+    Q+lT | subj) + (1 | item);
+    fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 155    Time: 0:00:00 ( 0.65 ms/it)
+  objective:  7483.769438170831
+Minimizing 310    Time: 0:00:00 ( 0.65 ms/it)
+  objective:  7291.145532339682
+Minimizing 343    Time: 0:00:00 ( 0.65 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)1.63760.021078.10<1e-990.05960.1750
lT: WD0.00870.00451.930.05300.0328
lQ: clr0.00160.00230.700.4818
Q: clr0.04280.004010.78<1e-260.0276
lT: WD & lQ: clr-0.00560.0023-2.390.0167
lT: WD & Q: clr0.00200.00230.870.3862
lQ: clr & Q: clr0.00760.00233.260.0011
lT: WD & lQ: clr & Q: clr0.00920.00233.94<1e-04
Residual0.2949
+
+
+

The lT & lQ & Q interactions is significant. Let’s follow it up with a post-hoc LMM, that is looking at the lQ & Q interaction in the two levels of whether the last word was a target or not.

+
+
m_prm3_posthoc = let
+    form = @formula (1000/rt) ~ 1 + lT/(lQ*Q) +
+                               (1+    Q+lT | subj) + (1 | item);
+    fit(MixedModel, form, dat; contrasts)
+end
+
+
Minimizing 154    Time: 0:00:00 ( 0.65 ms/it)
+  objective:  7331.749225564165
+Minimizing 297    Time: 0:00:00 ( 0.65 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)1.63760.021078.10<1e-990.05960.1750
lT: WD0.00870.00451.930.05300.0328
lT: NW & lQ: clr0.00720.00332.190.0285
lT: WD & lQ: clr-0.00390.0033-1.190.2326
lT: NW & Q: clr0.04080.00468.87<1e-18
lT: WD & Q: clr0.04480.00469.73<1e-21
lT: NW & lQ: clr & Q: clr-0.00160.0033-0.490.6275
lT: WD & lQ: clr & Q: clr0.01670.00335.08<1e-06
Q: clr0.0276
Residual0.2949
+
+
+

The source of the interaction are trials where the last trial was a word target; there is no evidence for the interaction when the last trial was a nonword target.

+

The original and post-hoc LMM have the same goodness of fit.

+
+
[objective(m_prm3), objective(m_prm3_posthoc)]
+
+
2-element Vector{Float64}:
+ 7291.145437442645
+ 7291.145437447412
+
+
+
+
+

8.4 Info

+
+
versioninfo()
+
+
Julia Version 1.10.4
+Commit 48d4fd48430 (2024-06-04 10:41 UTC)
+Build Info:
+  Official https://julialang.org/ release
+Platform Info:
+  OS: Linux (x86_64-linux-gnu)
+  CPU: 16 × Intel(R) Xeon(R) E-2288G CPU @ 3.70GHz
+  WORD_SIZE: 64
+  LIBM: libopenlibm
+  LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
+Threads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)
+Environment:
+  JULIA_LOAD_PATH = @:@stdlib
+
+
+ + +
+
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/partial_within.html b/partial_within.html new file mode 100644 index 0000000..9f1619e --- /dev/null +++ b/partial_within.html @@ -0,0 +1,1699 @@ + + + + + + + + + + + +Partially-within subjects designs – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Partially-within subjects designs

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

Begin by loading the packages to be used.

+
+
+Code +
using AlgebraOfGraphics
+using CairoMakie
+using DataFrames
+using MixedModels
+using MixedModelsMakie
+using MixedModelsSim
+using ProgressMeter
+using Random
+
+CairoMakie.activate!(; type="svg")
+
+ProgressMeter.ijulia_behavior(:clear)
+
+
+
+
+Code +
n_subj = 40
+n_item = 3
+# things are expressed as "between", so "within subjects" is "between items"
+item_btwn = Dict(:frequency => ["high", "medium", "low"])
+design = simdat_crossed(MersenneTwister(42), n_subj, n_item;
+                        item_btwn = item_btwn)
+design = DataFrame(design)
+
+
+
120×4 DataFrame
95 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowsubjitemfrequencydv
StringStringStringFloat64
1S01I1high-0.556027
2S02I1high-0.444383
3S03I1high0.0271553
4S04I1high-0.299484
5S05I1high1.77786
6S06I1high-1.1449
7S07I1high-0.468606
8S08I1high0.156143
9S09I1high-2.64199
10S10I1high1.00331
11S11I1high1.08238
12S12I1high0.187028
13S13I1high0.518149
⋮⋮⋮⋮⋮
109S29I3low-0.338763
110S30I3low-0.0953675
111S31I3low0.768972
112S32I3low1.44244
113S33I3low-0.275032
114S34I3low-0.379637
115S35I3low-0.722696
116S36I3low0.139661
117S37I3low-1.40934
118S38I3low1.05546
119S39I3low-2.23782
120S40I3low1.15915
+
+
+
+
+
+Code +
unique!(select(design, :item, :frequency))
+
+
+
3×2 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowitemfrequency
StringString
1I1high
2I2medium
3I3low
+
+
+
+
+
+Code +
m0 = let contrasts, form
+    contrasts = Dict(:frequency => HelmertCoding(base="high"))
+    form = @formula(dv ~ 1 + frequency +
+                    (1 + frequency | subj))
+    fit(MixedModel, form, design; contrasts)
+end
+
+
+
Minimizing 4    Time: 0:00:00 (27.12 ms/it)
+  objective:  394.74620416420464
+Minimizing 73    Time: 0:00:00 (13.13 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_subj
(Intercept)-0.08300.0983-0.840.39830.5440
frequency: low0.04110.12580.330.74380.7051
frequency: medium-0.00600.0572-0.110.91600.2928
Residual0.5204
+
+
+
+
+Code +
corrmat = [ 1    0.1 -0.2
+            0.1  1    0.1
+           -0.2  0.1  1 ]
+re_subj = create_re(1.2, 1.5, 1.5; corrmat)
+
+
+
3×3 LinearAlgebra.LowerTriangular{Float64, Matrix{Float64}}:
+  1.2    ⋅         ⋅ 
+  0.15  1.49248    ⋅ 
+ -0.3   0.180907  1.45852
+
+
+
+
+Code +
θ = createθ(m0; subj=re_subj)
+
+
+
6-element Vector{Float64}:
+  1.2
+  0.15000000000000002
+ -0.30000000000000004
+  1.49248115565993
+  0.18090680674665818
+  1.4585173044131932
+
+
+
+
+Code +
σ = 1;
+β = [1.0, -3, -2];
+
+
+
+
+Code +
fit!(simulate!(m0; θ, β, σ))
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_subj
(Intercept)1.00240.23084.34<1e-041.2898
frequency: low-2.95110.2350-12.56<1e-351.2281
frequency: medium-2.16890.2723-7.97<1e-141.6527
Residual1.1845
+
+
+
+
+Code +
shrinkageplot(m0)
+
+
+
+
+

+
+
+
+
+
+
+Code +
caterpillar(m0; orderby=nothing, vline_at_zero=true)
+
+
+
+
+

+
+
+
+
+
+
+Code +
design[!, :dv] .= response(m0)
+
+
+
120-element Vector{Float64}:
+ 15.057970446500043
+  5.1204354589939385
+  7.721170724963708
+  2.8917642652835136
+  3.008174534901129
+  8.242532408788222
+  4.95234926256407
+  8.26008013912291
+  5.169516496084133
+  6.988490665208847
+  ⋮
+  1.8891725952397342
+  2.908433059510399
+  2.9404207880287423
+  0.7216674213195389
+ -2.949507828847464
+  5.145924368618845
+  0.8248733146211791
+ -2.4920339370472924
+ -3.6917852507767233
+
+
+
+
+Code +
design_partial = filter(design) do row
+    subj = parse(Int, row.subj[2:end])
+    item = parse(Int, row.item[2:end])
+    # for even-numbered subjects, we keep all conditions
+    # for odd-numbered subjects, we keep only the two "odd" items,
+    # i.e. the first and last conditions
+    return iseven(subj) || isodd(item)
+end
+sort!(unique!(select(design_partial, :subj, :frequency)), :subj)
+
+
+
100×2 DataFrame
75 rows omitted
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowsubjfrequency
StringString
1S01high
2S01low
3S02high
4S02medium
5S02low
6S03high
7S03low
8S04high
9S04medium
10S04low
11S05high
12S05low
13S06high
⋮⋮⋮
89S36medium
90S36low
91S37high
92S37low
93S38high
94S38medium
95S38low
96S39high
97S39low
98S40high
99S40medium
100S40low
+
+
+
+
+
+Code +
m1 = let contrasts, form
+    contrasts = Dict(:frequency => HelmertCoding(base="high"))
+    form = @formula(dv ~ 1 + frequency +
+                    (1 + frequency | subj))
+    fit(MixedModel, form, design_partial; contrasts)
+end
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_subj
(Intercept)1.26380.28254.47<1e-051.3155
frequency: low-2.95110.2351-12.56<1e-351.2017
frequency: medium-1.90750.3034-6.29<1e-091.5740
Residual1.2377
+
+
+
+
+Code +
shrinkageplot(m1)
+
+
+
+
+

+
+
+
+
+
+
+Code +
caterpillar(m1; orderby=nothing, vline_at_zero=true)
+
+
+
+
+

+
+
+
+
+ + + + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/partial_within_files/figure-html/cell-10-output-1.svg b/partial_within_files/figure-html/cell-10-output-1.svg new file mode 100644 index 0000000..a7eb852 --- /dev/null +++ b/partial_within_files/figure-html/cell-10-output-1.svg @@ -0,0 +1,963 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/partial_within_files/figure-html/cell-11-output-1.svg b/partial_within_files/figure-html/cell-11-output-1.svg new file mode 100644 index 0000000..a775948 --- /dev/null +++ b/partial_within_files/figure-html/cell-11-output-1.svg @@ -0,0 +1,1470 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/partial_within_files/figure-html/cell-15-output-1.svg b/partial_within_files/figure-html/cell-15-output-1.svg new file mode 100644 index 0000000..7c36f26 --- /dev/null +++ b/partial_within_files/figure-html/cell-15-output-1.svg @@ -0,0 +1,918 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/partial_within_files/figure-html/cell-16-output-1.svg b/partial_within_files/figure-html/cell-16-output-1.svg new file mode 100644 index 0000000..378e34f --- /dev/null +++ b/partial_within_files/figure-html/cell-16-output-1.svg @@ -0,0 +1,1534 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/pkg.html b/pkg.html new file mode 100644 index 0000000..faf4888 --- /dev/null +++ b/pkg.html @@ -0,0 +1,981 @@ + + + + + + + + + + + +Package management and reproducible environments – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Package management and reproducible environments

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

Julius Krumbiegel also has a great blog post with more details on Julia environments.

+

Julia packages can be configured (in a file called Project.toml) on a per-project basis. The packaged sources and compiled versions are stored in a central location, e.g. ~/.julia/packages and ~/.julia/compiled on Linux systems, but the configuration of packages to be used can be local to a project. The Pkg package is used to modify the local project’s configuration. (An alternative is “package mode” in the read-eval-print-loop or REPL, which we will show at the summer school.) Start julia in the directory of the cloned SMLP2024 repository

+
+
using Pkg        # there's a package called 'Pkg' to manipulate package configs
+Pkg.activate(".")# activate the current directory as the project
+
+

If you’ve received an environment from someone/somewhere else – such as this course repository – then you’ll need to first “instantiate” it (i.e., install all the dependencies).

+
+
Pkg.instantiate()# only needed the first time you work in a project
+Pkg.update()     # get the latest package versions compatible with the project
+
+
+
Pkg.status()
+
+

Occasionally the Pkg.status function call will give info about new versions being available but blocked by requirements of other packages. This is to be expected - the package system is large and the web of dependencies are complex. Generally the Julia package system is very good at resolving dependencies.

+ + + + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/profiling.html b/profiling.html new file mode 100644 index 0000000..b1e2671 --- /dev/null +++ b/profiling.html @@ -0,0 +1,1390 @@ + + + + + + + + + + + +Confidence intervals from profiled objective – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Confidence intervals from profiled objective

+
+ + + +
+ +
+
Author
+
+

Douglas Bates

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +
+

1 Assessing the variability of parameter estimates

+

Statistical methods that are based on probability models can be used to provide us with a “best guess” of the value of parameters, such as the effect of a particular experimental treatment, in the form of a parameter estimate. In addition, the probability model can be used to assess the uncertainty in the estimate.

+

Often the information about the uncertainty is reduced to a single number, a p-value for a test of a null hypothesis, such as the effect being zero, versus the alternative of a non-zero effect. But quoting a single number from a model fit to experimental data, which may have required considerable effort and expense to obtain, will often mean discarding a considerable amount of the information in the data. In the days when computing was expensive and labor-intensive this may have been unavoidable. However, modern computing hardware and software systems provide us with the opportunity of much more intensive evaluation of the uncertainty. At a minimum, instead of focussing solely on the question of whether a coefficient could reasonably be zero, we can formulate confidence intervals on individual parameter estimates or confidence regions on groups of parameters.

+

We have seen the used of a parametric bootstrap to create a sample from the distribution of the estimators of the parameters, and how such samples can be used to create coverage intervals. The bootstrap is based on simulating response vectors from the model that has been fit to the observed data and refitting the same model to these simulated responses.

+

In this section we explore another approach based on refitting the model, keeping the same responses but holding one of the parameters fixed at a specified value.

+
+

1.1 Profiling a model for the kb07 data

+

Load the packages to be used

+
+
+Code +
using AlgebraOfGraphics
+using CairoMakie
+using MixedModels
+using MixedModelsMakie
+using Random
+using SMLP2024: dataset
+
+CairoMakie.activate!(; type="svg")
+import ProgressMeter
+ProgressMeter.ijulia_behavior(:clear)
+
+
+

Load the data and define the contrasts so that the coefficients for each of the experimental variables, load, spkr and prec, are positive.

+
+
contrasts = Dict( # base levels so estimates for speed are positive
+  :load => EffectsCoding(; base="yes"),
+  :prec => EffectsCoding(; base="break"),
+  :spkr => EffectsCoding(; base="old"),
+)
+kb07 = Table(dataset(:kb07))
+
+
Table with 7 columns and 1789 rows:
+      subj  item  spkr  prec      load  rt_trunc  rt_raw
+    ┌───────────────────────────────────────────────────
+ 1  │ S030  I01   new   break     yes   2267      2267
+ 2  │ S030  I02   old   maintain  no    3856      3856
+ 3  │ S030  I03   old   break     no    1567      1567
+ 4  │ S030  I04   new   maintain  no    1732      1732
+ 5  │ S030  I05   new   break     no    2660      2660
+ 6  │ S030  I06   old   maintain  yes   2763      2763
+ 7  │ S030  I07   old   break     yes   3528      3528
+ 8  │ S030  I08   new   maintain  yes   1741      1741
+ 9  │ S030  I09   new   break     yes   3692      3692
+ 10 │ S030  I10   old   maintain  no    1949      1949
+ 11 │ S030  I11   old   break     no    2189      2189
+ 12 │ S030  I12   new   maintain  no    2207      2207
+ 13 │ S030  I13   new   break     no    2078      2078
+ 14 │ S030  I14   old   maintain  yes   1901      1901
+ 15 │ S030  I15   old   break     yes   4015      4015
+ 16 │ S030  I16   new   maintain  yes   1880      1880
+ 17 │ S030  I17   new   break     yes   1444      1444
+ ⋮  │  ⋮     ⋮     ⋮       ⋮       ⋮       ⋮        ⋮
+
+
+

Now we fit and profile a model. The response is defined as 1000 / rt_raw where rt_raw is measured in milliseconds. Thus the response being modeled is the speed measured in responses per second.

+
+
pr01 = let f = @formula 1000 / rt_raw ~
+    1 + load + spkr + prec + (1 + prec | item) + (1 | subj)
+  profile(fit(MixedModel, f, kb07; contrasts))
+end
+println(pr01.m) # model is a property of the profile object
+
+
Minimizing 2    Time: 0:00:00 ( 0.39  s/it)
+  objective:  -1585.184301073052
+Minimizing 67    Time: 0:00:01 (22.29 ms/it)
+Linear mixed model fit by maximum likelihood
+ :(1000 / rt_raw) ~ 1 + load + spkr + prec + (1 + prec | item) + (1 | subj)
+   logLik   -2 logLik     AIC       AICc        BIC    
+   846.2869 -1692.5738 -1674.5738 -1674.4726 -1625.1691
+
+Variance components:
+             Column      Variance  Std.Dev.   Corr.
+item     (Intercept)     0.0061053 0.0781364
+         prec: maintain  0.0020476 0.0452502 -0.21
+subj     (Intercept)     0.0054186 0.0736113
+Residual                 0.0194484 0.1394577
+ Number of obs: 1789; levels of grouping factors: 32, 56
+
+  Fixed-effects parameters:
+──────────────────────────────────────────────────────
+                    Coef.  Std. Error      z  Pr(>|z|)
+──────────────────────────────────────────────────────
+(Intercept)     0.531523   0.0172749   30.77    <1e-99
+load: no        0.0212959  0.00329731   6.46    <1e-09
+spkr: new       0.011218   0.00329732   3.40    0.0007
+prec: maintain  0.0698293  0.00865212   8.07    <1e-15
+──────────────────────────────────────────────────────
+
+
+

Evaluation of pr01 is similar to other model fits in these notes except that the call to fit is wrapped in a call to profile. Because the object returned from profile includes the original model fit as its m property, it is not necessary to save the original model fit separately.

+
+
+

1.2 Fixing values of parameters

+

The information from the profile is encapsulated in a table.

+
+
pr01.tbl
+
+
Table with 15 columns and 249 rows:
+      p  ζ          β1        β2         β3         β4         σ         ⋯
+    ┌─────────────────────────────────────────────────────────────────────
+ 1  │ σ  -4.11624   0.531525  0.021298   0.0112154  0.0698319  0.130088  ⋯
+ 2  │ σ  -3.59106   0.531525  0.0212977  0.0112157  0.0698316  0.131224  ⋯
+ 3  │ σ  -3.06898   0.531524  0.0212975  0.011216   0.0698313  0.13237   ⋯
+ 4  │ σ  -2.54996   0.531524  0.0212972  0.0112164  0.069831   0.133526  ⋯
+ 5  │ σ  -2.03399   0.531524  0.021297   0.0112167  0.0698307  0.134692  ⋯
+ 6  │ σ  -1.52104   0.531524  0.0212967  0.011217   0.0698303  0.135868  ⋯
+ 7  │ σ  -1.01107   0.531523  0.0212964  0.0112173  0.06983    0.137054  ⋯
+ 8  │ σ  -0.504067  0.531523  0.0212961  0.0112177  0.0698297  0.138251  ⋯
+ 9  │ σ  0.0        0.531523  0.0212959  0.011218   0.0698293  0.139458  ⋯
+ 10 │ σ  0.501151   0.531523  0.0212956  0.0112184  0.069829   0.140675  ⋯
+ 11 │ σ  0.999416   0.531522  0.0212953  0.0112187  0.0698286  0.141904  ⋯
+ 12 │ σ  1.49482    0.531522  0.021295   0.0112191  0.0698282  0.143143  ⋯
+ 13 │ σ  1.98739    0.531522  0.0212947  0.0112195  0.0698279  0.144392  ⋯
+ 14 │ σ  2.47714    0.531521  0.0212944  0.0112199  0.0698275  0.145653  ⋯
+ 15 │ σ  2.9641     0.531521  0.021294   0.0112202  0.0698271  0.146925  ⋯
+ 16 │ σ  3.4483     0.531521  0.0212937  0.0112206  0.0698267  0.148208  ⋯
+ 17 │ σ  3.92976    0.53152   0.0212934  0.011221   0.0698263  0.149502  ⋯
+ ⋮  │ ⋮      ⋮         ⋮          ⋮          ⋮          ⋮         ⋮      ⋱
+
+
+

Each row of the table summarizes a fit of the original model to the original data but with one of the parameters held fixed. For the first 18 rows of the table, the parameter being held fixed is \(\sigma\), as shown in the p column. In the next set of rows the parameter being held fixed will be \(\beta_1\), the intercept.

+

There are blocks of rows for the fixed-effects (\(\boldsymbol{\beta}\)) parameters, the variance components (on the scale of a standard deviation), and the \(\boldsymbol{\theta}\) parameters that generate the covariance factor \(\boldsymbol{\Lambda}_{\boldsymbol{\theta}}\). (At present the correlation parameters are not profiled - we may add them later but that computation is rather awkward.)

+
+
show(unique(pr01.tbl.p))
+
+
[:σ, :β1, :β2, :β3, :β4, :θ1, :θ2, :θ3, :θ4, :σ1, :σ2, :σ3]
+
+
+

To reiterate, the first row contains the parameter estimates for this model fit to the original response values with the constraint that \(\sigma=0.130088\), instead of the global estimate \(\hat{\sigma}=0.139458\) in the row for which \(\zeta=0.0\).

+

The global estimates are included in every block at the row for which \(\zeta=0.0\).

+
+
filter(r -> iszero(r.ζ), pr01.tbl)
+
+
Table with 15 columns and 12 rows:
+      p   ζ    β1        β2         β3        β4         σ         σ1         ⋯
+    ┌──────────────────────────────────────────────────────────────────────────
+ 1  │ σ   0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+ 2  │ β1  0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+ 3  │ β2  0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+ 4  │ β3  0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+ 5  │ β4  0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+ 6  │ θ1  0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+ 7  │ θ2  0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+ 8  │ θ3  0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+ 9  │ θ4  0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+ 10 │ σ1  0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+ 11 │ σ2  0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+ 12 │ σ3  0.0  0.531523  0.0212959  0.011218  0.0698293  0.139458  0.0781364  ⋯
+
+
+

The \(\zeta\) column in this table is a measure of the quality of the fit from the parameters in each row, relative to the global parameter estimates, as measured by the change in the objective (negative twice the log-likelihood).

+

The minimum value for the objective is that at the global parameter estimates. The change in the objective when we constrain one parameter to a particular value has approximately a \(\chi^2\) distribution on 1 degree of freedom, which is the square of a standard normal distribution, \(\mathcal{Z}^2\). We can convert this change in the quality of the fit to the scale of the standard normal distribution by taking the signed square root, which is the square root of the change in the objective with the sign of \(\psi-\hat{\psi}\) where \(\psi\) represents the parameter being profiled. This is the value labelled \(\zeta\) in the table.

+

To review:

+
    +

  • Each row in the table is the result of re-fitting the original model with the parameter in the p column held fixed at a particular value, as shown in the column for that parameter.

    • +

  • The \(\zeta\) column is the signed square root of the change in the objective from the global parameter estimates.

    • +

  • Thus in the block of rows where \(\sigma\) is held fixed, the \(\zeta\) values in rows for which \(\sigma<\hat\sigma\) are negative and those for which \(\sigma > \hat\sigma\) have positive values of \(\zeta\).

    • +

  • Rows in which \(\zeta=0.0\) are the global parameter estimates.

    • +
+
+
+

1.3 Profile zeta plots

+

Figure 1 shows, for each of the fixed effects parameters, \(\zeta\) versus the parameter value.

+
+
+Code +
zetaplot!(Figure(; resolution=(1200, 350)), pr01; ptyp='β')
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 1: ζ versus the value of the coefficient for the fixed-effects parameters in a model of response speed for the kb07 data. +
+
+
+
+
+

The lines on these panels are read like normal probability plots, i.e. QQ plots against a standard normal distribution. Those on the \(\beta_2\) and \(\beta_3\) panels are, to the resolution of the plot, straight lines which indicates that the estimators of those parameters are normally distributed over the region of interest.

+

The points in the \(\beta_1\) and \(\beta_4\) panels are slightly over-dispersed relative to the straight line, which means that the estimators of these parameters are distributed like a T-distribution with a moderate number of degrees of freedom.

+

The profile-\(\zeta\) function can be used to generate confidence intervals on the parameters

+
+
confint(pr01)
+
+
DictTable with 3 columns and 8 rows:
+ par   estimate   lower       upper
+ ────┬─────────────────────────────────
+ β1  │ 0.531523   0.497103    0.565942
+ β2  │ 0.0212959  0.0148295   0.0277621
+ β3  │ 0.011218   0.00475174  0.0176844
+ β4  │ 0.0698293  0.0523046   0.0873562
+ σ   │ 0.139458   0.13486     0.144322
+ σ1  │ 0.0781364  0.0612443   0.103257
+ σ2  │ 0.0452502  0.0338521   0.0618819
+ σ3  │ 0.0736113  0.0600845   0.0916852
+
+
+

as shown in Figure 2, which shows the absolute value of \(\zeta\), which is simply the square root of the difference in the objective, versus the parameter being profiled.

+
+
+Code +
zetaplot!(Figure(; resolution=(1200, 330)), pr01; ptyp='β', absv=true)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 2: Absolute value of ζ versus value of the coefficient for the fixed-effects parameters in a model of response speed for the kb07 data. The horizontal lines are confidence intervals with nominal 50%, 80%, 90%, 95% and 99% confidence. +
+
+
+
+
+

The 95% confidence intervals are the second horizontal lines from the top in each panel, at 1.96 on the vertical scale.

+
+
+Code +
zetaplot!(Figure(; resolution=(1200, 330)), pr01; ptyp='σ', absv=true)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 3: Absolute value of ζ versus value of the coefficient for the variance component parameters in a model of response speed for the kb07 data. The horizontal lines are confidence intervals with nominal 50%, 80%, 90%, 95% and 99% confidence. +
+
+
+
+
+

Figure 3 shows similar confidence intervals on the parameters representing standard deviations as does Figure 4 for the \(\theta\) parameters.

+
+
+Code +
zetaplot!(Figure(; resolution=(1200, 330)), pr01; ptyp='θ', absv=true)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 4: Absolute value of ζ versus parameter value for the θ parameters in a model of response speed for the kb07 data. The horizontal lines are confidence intervals with nominal 50%, 80%, 90%, 95% and 99% confidence. +
+
+
+
+
+
+
+
+

2 Comparisons with the parametric bootstrap

+

With two methods of assessing the variability in the parameter estimates — the parametric bootstrap and profiling the objective function — we should compare and contrast these approaches.

+

Profiling the objective has two main advantages:

+
    +

  • Profiling is deterministic whereas the parametric bootstrap is stochastic because it is based on a random sample from the model at the estimated parameter values. Repeating the evaluation of the bootstrap intervals will result in slightly different end points for the coverage intervals. The variability in the end points is a function of the size of the bootstrap sample.

    • +

  • Generally profiling is faster than the bootstrap. In this example the profiling required fitting a reduced model to the data 237 times (249 rows in the table but 12 of these rows are repetitions of the global estimates). To obtain reasonable precision in a bootstrap usually requires thousands of samples.

    • +
+

The main advantage of the bootstrap is that any parameter or any function of the parameters, such as the predicted response at some setting of the experimental factors, can be evaluated and assessed from the sample. When doing so, however, it is necessary to work with large samples so as to avoid undesirable levels of sample re-use.

+

Because profiling different parameters requires customized code for each parameter type, it is more difficult to generalize this approach to different types of parameters or predictions. We have already mentioned that the correlation parameters are not profiled in the current version of the code.

+

For comparison with the profiling results, we create a table of 2500 bootstrap samples

+
+
Random.seed!(8765678)
+samp01 = parametricbootstrap(2500, pr01.m; optsum_overrides=(; ftol_rel=1e-8))
+confint(samp01)
+
+
DictTable with 2 columns and 9 rows:
+ par   lower       upper
+ ────┬──────────────────────
+ β1  │ 0.497384    0.562267
+ β2  │ 0.0147172   0.0274258
+ β3  │ 0.00481004  0.0169282
+ β4  │ 0.0524409   0.0857708
+ ρ1  │ -0.577499   0.183778
+ σ   │ 0.134747    0.144354
+ σ1  │ 0.0587605   0.0986544
+ σ2  │ 0.0316108   0.0571327
+ σ3  │ 0.0572288   0.087862
+
+
+

Comparing these intervals with those from the profile results shows that the intervals on the fixed-effects parameters from the two methods are quite similar. The intervals on \(\sigma\) from the two methods are quite similar but the bootstrap intervals on the other variance components are shifted to the left relative to those from the profiling results. The reason for this is because the bootstrap intervals are chosen to be the shortest intervals with the desired coverage. In a density that is skewed to the right, as these are, the shortest interval will be to the left of an interval with equal tail coverage, which is how the profile-based intervals are constructed.

+

The profile-\(\zeta\) function can be transformed to an equivalent density function which we plot in Figure 5, showing the skewness of the variance component parameters other than \(\sigma\).

+
+
+Code +
profiledensity!(Figure(; resolution=(1200, 300)), pr01; share_y_scale=false)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 5: Density functions of the marginal distribution of the estimators of variance component parameters in a model of response speed in the kb07 data. +
+
+
+
+
+

Alternatively we can see the skewness in the plots of \(\zeta\)

+
+
+Code +
zetaplot!(Figure(; resolution=(1100, 330)), pr01; ptyp='σ')
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 6: Plot of ζ scores for the variance component parameters in a model of response speed for the kb07 data +
+
+
+
+
+

The skewness is less obvious in Figure 7 because of the stochastic nature of the bootstrap sample and the kernel density estimators.

+
+
+Code +
let pars=["σ", "σ1", "σ2", "σ3"]
+  draw(
+    data(samp01.tbl) *
+    mapping(
+      pars .=> "Variance component parameters";
+      color=dims(1) => renamer(pars),
+    ) *
+    AlgebraOfGraphics.density()
+  )
+end
+
+
+
+
+
+ +
+
+Figure 7: Kernel density estimator plots from bootstrap samples of the variance component parameters in a model of response speed for the kb07 data +
+
+
+
+
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/profiling_files/figure-html/fig-kb07abszetabeta-output-2.svg b/profiling_files/figure-html/fig-kb07abszetabeta-output-2.svg new file mode 100644 index 0000000..9602888 --- /dev/null +++ b/profiling_files/figure-html/fig-kb07abszetabeta-output-2.svg @@ -0,0 +1,506 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/profiling_files/figure-html/fig-kb07abszetasigma-output-2.svg b/profiling_files/figure-html/fig-kb07abszetasigma-output-2.svg new file mode 100644 index 0000000..d1058aa --- /dev/null +++ b/profiling_files/figure-html/fig-kb07abszetasigma-output-2.svg @@ -0,0 +1,465 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/profiling_files/figure-html/fig-kb07abszetatheta-output-2.svg b/profiling_files/figure-html/fig-kb07abszetatheta-output-2.svg new file mode 100644 index 0000000..945db98 --- /dev/null +++ b/profiling_files/figure-html/fig-kb07abszetatheta-output-2.svg @@ -0,0 +1,413 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/profiling_files/figure-html/fig-kb07bskdesigma-output-1.svg b/profiling_files/figure-html/fig-kb07bskdesigma-output-1.svg new file mode 100644 index 0000000..fb736ac --- /dev/null +++ b/profiling_files/figure-html/fig-kb07bskdesigma-output-1.svg @@ -0,0 +1,357 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/profiling_files/figure-html/fig-kb07densigma-output-2.svg b/profiling_files/figure-html/fig-kb07densigma-output-2.svg new file mode 100644 index 0000000..bc7890e --- /dev/null +++ b/profiling_files/figure-html/fig-kb07densigma-output-2.svg @@ -0,0 +1,752 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/profiling_files/figure-html/fig-kb07zetabeta-output-2.svg b/profiling_files/figure-html/fig-kb07zetabeta-output-2.svg new file mode 100644 index 0000000..d24718e --- /dev/null +++ b/profiling_files/figure-html/fig-kb07zetabeta-output-2.svg @@ -0,0 +1,576 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/profiling_files/figure-html/fig-kb07zetasigma-output-2.svg b/profiling_files/figure-html/fig-kb07zetasigma-output-2.svg new file mode 100644 index 0000000..530f321 --- /dev/null +++ b/profiling_files/figure-html/fig-kb07zetasigma-output-2.svg @@ -0,0 +1,525 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/robots.txt b/robots.txt new file mode 100644 index 0000000..fcb223d --- /dev/null +++ b/robots.txt @@ -0,0 +1 @@ +Sitemap: https://RePsychLing.github.io/SMLP2024/sitemap.xml diff --git a/search.json b/search.json new file mode 100644 index 0000000..5dc4d3d --- /dev/null +++ b/search.json @@ -0,0 +1,1619 @@ +[ + { + "objectID": "fggk21.html", + "href": "fggk21.html", + "title": "Basics with Emotikon Project", + "section": "", + "text": "This script uses a subset of data reported in Fühner et al. (2021). To circumvent delays associated with model fitting we work with models that are less complex than those in the reference publication. All the data to reproduce the models in the publication are used here, too; the script requires only a few changes to specify the more complex models in the article.\nThe script is structured in four main sections:" + }, + { + "objectID": "fggk21.html#read-data", + "href": "fggk21.html#read-data", + "title": "Basics with Emotikon Project", + "section": "3.1 Read data", + "text": "3.1 Read data\n\ndf = DataFrame(dataset(:fggk21))\ntransform!(df,\n :age => (x -> x .- 8.5) => :a1,\n :Sex => categorical => :Sex,\n :Test => categorical => :Test,\n )\nlevels!(df.Sex, [\"male\", \"female\"])\nrecode!(df.Sex, \"male\" => \"Boys\", \"female\" => \"Girls\")\nlevels!(df.Test, [\"Run\", \"Star_r\", \"S20_r\", \"SLJ\", \"BPT\"])\nrecode!(\n df.Test,\n \"Run\" => \"Endurance\",\n \"Star_r\" => \"Coordination\",\n \"S20_r\" => \"Speed\",\n \"SLJ\" => \"PowerLOW\",\n \"BPT\" => \"PowerUP\",\n)\ndescribe(df)\n\n8×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nDataType\n\n\n\n\n1\nCohort\n\n2011\n\n2019\n0\nString\n\n\n2\nSchool\n\nS100043\n\nS800200\n0\nString\n\n\n3\nChild\n\nC002352\n\nC117966\n0\nString\n\n\n4\nSex\n\nBoys\n\nGirls\n0\nCategoricalValue{String, UInt32}\n\n\n5\nage\n8.56073\n7.99452\n8.55852\n9.10609\n0\nFloat64\n\n\n6\nTest\n\nEndurance\n\nPowerUP\n0\nCategoricalValue{String, UInt32}\n\n\n7\nscore\n226.141\n1.14152\n4.65116\n1530.0\n0\nFloat64\n\n\n8\na1\n0.0607297\n-0.505476\n0.0585216\n0.606092\n0\nFloat64\n\n\n\n\n\n\n\n3.1.1 Transformations\nWe center age at 8.5 years and compute z-scores for each Test. With these variables the data frame df contains all variables used for the final model in the original publication.\n\nselect!(groupby(df, :Test), Not(:score), :score => zscore => :zScore)\n\n525126×8 DataFrame525101 rows omitted\n\n\n\nRow\nTest\nCohort\nSchool\nChild\nSex\nage\na1\nzScore\n\n\n\nCat…\nString\nString\nString\nCat…\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\nSpeed\n2013\nS100067\nC002352\nBoys\n7.99452\n-0.505476\n1.7913\n\n\n2\nPowerUP\n2013\nS100067\nC002352\nBoys\n7.99452\n-0.505476\n-0.0622317\n\n\n3\nPowerLOW\n2013\nS100067\nC002352\nBoys\n7.99452\n-0.505476\n-0.0336567\n\n\n4\nCoordination\n2013\nS100067\nC002352\nBoys\n7.99452\n-0.505476\n1.46874\n\n\n5\nEndurance\n2013\nS100067\nC002352\nBoys\n7.99452\n-0.505476\n0.331058\n\n\n6\nSpeed\n2013\nS100067\nC002353\nBoys\n7.99452\n-0.505476\n1.15471\n\n\n7\nPowerUP\n2013\nS100067\nC002353\nBoys\n7.99452\n-0.505476\n0.498354\n\n\n8\nPowerLOW\n2013\nS100067\nC002353\nBoys\n7.99452\n-0.505476\n-0.498822\n\n\n9\nCoordination\n2013\nS100067\nC002353\nBoys\n7.99452\n-0.505476\n-0.9773\n\n\n10\nEndurance\n2013\nS100067\nC002353\nBoys\n7.99452\n-0.505476\n0.574056\n\n\n11\nSpeed\n2013\nS100067\nC002354\nBoys\n7.99452\n-0.505476\n0.0551481\n\n\n12\nPowerUP\n2013\nS100067\nC002354\nBoys\n7.99452\n-0.505476\n0.218061\n\n\n13\nPowerLOW\n2013\nS100067\nC002354\nBoys\n7.99452\n-0.505476\n-0.757248\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n525115\nCoordination\n2018\nS401470\nC117964\nBoys\n9.10609\n0.606092\n-1.43175\n\n\n525116\nEndurance\n2018\nS401470\nC117964\nBoys\n9.10609\n0.606092\n-0.944681\n\n\n525117\nSpeed\n2018\nS401470\nC117965\nGirls\n9.10609\n0.606092\n0.31086\n\n\n525118\nPowerUP\n2018\nS401470\nC117965\nGirls\n9.10609\n0.606092\n0.0779146\n\n\n525119\nPowerLOW\n2018\nS401470\nC117965\nGirls\n9.10609\n0.606092\n-0.137027\n\n\n525120\nCoordination\n2018\nS401470\nC117965\nGirls\n9.10609\n0.606092\n-1.8077\n\n\n525121\nEndurance\n2018\nS401470\nC117965\nGirls\n9.10609\n0.606092\n0.513306\n\n\n525122\nSpeed\n2018\nS800200\nC117966\nBoys\n9.10609\n0.606092\n0.0551481\n\n\n525123\nPowerUP\n2018\nS800200\nC117966\nBoys\n9.10609\n0.606092\n0.0779146\n\n\n525124\nPowerLOW\n2018\nS800200\nC117966\nBoys\n9.10609\n0.606092\n-1.32578\n\n\n525125\nCoordination\n2018\nS800200\nC117966\nBoys\n9.10609\n0.606092\n0.473217\n\n\n525126\nEndurance\n2018\nS800200\nC117966\nBoys\n9.10609\n0.606092\n-0.0941883" + }, + { + "objectID": "fggk21.html#extract-a-stratified-subsample", + "href": "fggk21.html#extract-a-stratified-subsample", + "title": "Basics with Emotikon Project", + "section": "3.2 Extract a stratified subsample", + "text": "3.2 Extract a stratified subsample\nFor the prupose of the tutorial, we extract a random sample of 1000 boys and 1000 girls. Child, School, and Cohort are grouping variables. Traditionally, they are called random factors because the units (levels) of the factor are assumed to be a random sample from the population of their units (levels).\nCohort has only nine “groups” and could have been included as a set of polynomical fixed-effect contrasts rather than a random factor. This choice warrants a short excursion: The secular trends are very different for different tests and require the inclusion of interaction terms with Test contrasts (see Figure 4 in (Fühner et al., 2021). The authors opted to absorb these effects in cohort-related variance components for the Test contrasts and plan to address the details of secular changes in a separate analysis.\nFor complex designs, when they are in the theoretical focus of an article, factors and covariates should be specified as part of the fixed effects. If they are not in the theoretical focus, but serve as statistical control variables, they could be put in the RES - if supported by the data.\nStratified sampling: We generate a Child table with information about children. MersenneTwister(42) specifies 42 as the seed for the random number generator to ensure reproducibility of the stratification. For a different pattern of results choose, for example, 84. We randomly sample 1000 boys and 1000 girls from this table; they are stored in samp. Then, we extract the corresponding subset of these children’s test scores from df and store them dat.\n\nChild = unique(select(df, :Cohort, :School, :Child, :Sex, :age))\nsample = let\n rng = MersenneTwister(42)\n combine(\n groupby(Child, :Sex), x -> x[rand(rng, 1:nrow(x), 1000), :]\n )\nend\ninsamp(x) = x ∈ sample.Child\ndat = @subset(df, insamp(:Child))\n\n9663×8 DataFrame9638 rows omitted\n\n\n\nRow\nTest\nCohort\nSchool\nChild\nSex\nage\na1\nzScore\n\n\n\nCat…\nString\nString\nString\nCat…\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\nSpeed\n2013\nS101540\nC002482\nGirls\n7.99452\n-0.505476\n0.0551481\n\n\n2\nPowerUP\n2013\nS101540\nC002482\nGirls\n7.99452\n-0.505476\n-0.342524\n\n\n3\nPowerLOW\n2013\nS101540\nC002482\nGirls\n7.99452\n-0.505476\n0.74162\n\n\n4\nCoordination\n2013\nS101540\nC002482\nGirls\n7.99452\n-0.505476\n-0.209186\n\n\n5\nEndurance\n2013\nS101540\nC002482\nGirls\n7.99452\n-0.505476\n-0.127938\n\n\n6\nSpeed\n2013\nS102090\nC002513\nGirls\n7.99452\n-0.505476\n-0.422921\n\n\n7\nPowerUP\n2013\nS102090\nC002513\nGirls\n7.99452\n-0.505476\n-0.762963\n\n\n8\nPowerLOW\n2013\nS102090\nC002513\nGirls\n7.99452\n-0.505476\n-0.188712\n\n\n9\nCoordination\n2013\nS102090\nC002513\nGirls\n7.99452\n-0.505476\n0.81382\n\n\n10\nEndurance\n2013\nS102090\nC002513\nGirls\n7.99452\n-0.505476\n0.452557\n\n\n11\nSpeed\n2013\nS103299\nC002621\nGirls\n7.99452\n-0.505476\n0.859704\n\n\n12\nPowerUP\n2013\nS103299\nC002621\nGirls\n7.99452\n-0.505476\n0.218061\n\n\n13\nPowerLOW\n2013\nS103299\nC002621\nGirls\n7.99452\n-0.505476\n0.74162\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n9652\nCoordination\n2018\nS111508\nC117805\nGirls\n9.10609\n0.606092\n-0.95589\n\n\n9653\nEndurance\n2018\nS111508\nC117805\nGirls\n9.10609\n0.606092\n-1.67367\n\n\n9654\nSpeed\n2018\nS111648\nC117827\nBoys\n9.10609\n0.606092\n0.859704\n\n\n9655\nPowerUP\n2018\nS111648\nC117827\nBoys\n9.10609\n0.606092\n0.6385\n\n\n9656\nPowerLOW\n2018\nS111648\nC117827\nBoys\n9.10609\n0.606092\n0.483194\n\n\n9657\nCoordination\n2018\nS111648\nC117827\nBoys\n9.10609\n0.606092\n0.708474\n\n\n9658\nEndurance\n2018\nS111648\nC117827\nBoys\n9.10609\n0.606092\n-0.337186\n\n\n9659\nSpeed\n2018\nS112197\nC117868\nBoys\n9.10609\n0.606092\n0.578748\n\n\n9660\nPowerUP\n2018\nS112197\nC117868\nBoys\n9.10609\n0.606092\n0.498354\n\n\n9661\nPowerLOW\n2018\nS112197\nC117868\nBoys\n9.10609\n0.606092\n-0.550508\n\n\n9662\nCoordination\n2018\nS112197\nC117868\nBoys\n9.10609\n0.606092\n0.538978\n\n\n9663\nEndurance\n2018\nS112197\nC117868\nBoys\n9.10609\n0.606092\n0.938553\n\n\n\n\n\n\nDue to missing scores for some tests we have about 2% less than 10,000 observtions." + }, + { + "objectID": "fggk21.html#no-evidence-for-age-x-sex-x-test-interaction", + "href": "fggk21.html#no-evidence-for-age-x-sex-x-test-interaction", + "title": "Basics with Emotikon Project", + "section": "3.3 No evidence for age x Sex x Test interaction", + "text": "3.3 No evidence for age x Sex x Test interaction\nThe main results are captured in the figure constructed in this section. We build it both for the full data and the stratified subset.\n\ndf2 = combine(\n groupby(\n select(df, :, :age => ByRow(x -> round(x; digits=1)) => :age),\n [:Sex, :Test, :age],\n ),\n :zScore => mean => :zScore,\n :zScore => length => :n,\n)\n\n120×5 DataFrame95 rows omitted\n\n\n\nRow\nSex\nTest\nage\nzScore\nn\n\n\n\nCat…\nCat…\nFloat64\nFloat64\nInt64\n\n\n\n\n1\nBoys\nSpeed\n8.0\n-0.0265138\n1223\n\n\n2\nBoys\nPowerUP\n8.0\n0.026973\n1227\n\n\n3\nBoys\nPowerLOW\n8.0\n0.121609\n1227\n\n\n4\nBoys\nCoordination\n8.0\n-0.0571726\n1186\n\n\n5\nBoys\nEndurance\n8.0\n0.292695\n1210\n\n\n6\nGirls\nSpeed\n8.0\n-0.35164\n1411\n\n\n7\nGirls\nPowerUP\n8.0\n-0.610355\n1417\n\n\n8\nGirls\nPowerLOW\n8.0\n-0.279872\n1418\n\n\n9\nGirls\nCoordination\n8.0\n-0.268221\n1381\n\n\n10\nGirls\nEndurance\n8.0\n-0.245573\n1387\n\n\n11\nBoys\nSpeed\n8.1\n0.0608397\n3042\n\n\n12\nBoys\nPowerUP\n8.1\n0.0955413\n3069\n\n\n13\nBoys\nPowerLOW\n8.1\n0.123099\n3069\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n109\nBoys\nCoordination\n9.0\n0.254973\n4049\n\n\n110\nBoys\nEndurance\n9.0\n0.258082\n4034\n\n\n111\nGirls\nSpeed\n9.1\n-0.0286172\n1154\n\n\n112\nGirls\nPowerUP\n9.1\n-0.0752301\n1186\n\n\n113\nGirls\nPowerLOW\n9.1\n-0.094587\n1174\n\n\n114\nGirls\nCoordination\n9.1\n0.00276252\n1162\n\n\n115\nGirls\nEndurance\n9.1\n-0.235591\n1150\n\n\n116\nBoys\nSpeed\n9.1\n0.325745\n1303\n\n\n117\nBoys\nPowerUP\n9.1\n0.616416\n1320\n\n\n118\nBoys\nPowerLOW\n9.1\n0.267577\n1310\n\n\n119\nBoys\nCoordination\n9.1\n0.254342\n1297\n\n\n120\nBoys\nEndurance\n9.1\n0.251045\n1294\n\n\n\n\n\n\n\n3.3.1 Figure(s) of interaction\nThe core results of the article are reported in Figure 2 of Fühner et al. (2021). In summary:\n\nMain effects of age and Sex: There are developmental gains in the ninth year of life; boys outperform girls. There is no main effect of Test because of z-scoring.\nInteractions of Test and age: Tests differ in how much children improve during the year (i.e., the magnitude of developmental gain), that is slopes depend on Test.\nInteractions of Test and Sex: The sex difference is test dependent, that is the difference between the slopes depends on Test.\nThe most distinctive result is the absence of evidence for an age x Sex x Test interaction, that is the slopes for boys and girls are statistically parallel for each of the five tests.\n\n\n\nCode\nlet\n design1 = mapping(:age, :zScore; color=:Sex, col=:Test)\n lines1 = design1 * linear()\n means1 = design1 * visual(Scatter; markersize=5)\n draw(data(df2) * means1 + data(df) * lines1;)\nend\n\n\n\n\n\n\n\n\nFigure 1: Age trends by sex for each Test for the full data set\n\n\n\n\n\nFigure 1 shows performance differences for the full set of data between 8.0 and 9.2 years by sex in the five physical fitness tests presented as z-transformed data computed separately for each test.\n\nEndurance = cardiorespiratory endurance (i.e., 6-min-run test),\nCoordination = star-run test,\nSpeed = 20-m linear sprint test,\nPowerLOW = power of lower limbs (i.e., standing long jump test),\nPowerUP = power of upper limbs (i.e., ball push test),\nSD = standard deviation. Points are binned observed child means; lines are simple regression fits to the observations.\n\nWhat do the results look like for the stratified subsample? Here the parallelism is much less clear. In the final LMM we test whether the two regression lines in each of the five panels are statistically parallel for this subset of data. That is, we test the interaction of Sex and age as nested within the levels of Test. Most people want to know the signficance of these five Sex x age interactions.\nThe theoretical focus of the article, however, is on comparisons between tests displayed next to each other. We ask whether the degree of parallelism is statistically the same for Endurance and Coordination (H1), Coordination and Speed (H2), Speed and PowerLOW (H3), and PowerLow and PowerUP (H4). Hypotheses H1 to H4 require Sequential Difference contrasts c1 to c4 for Test; they are tested as fixed effects for`H1 x age x Sex, H2 x age x Sex, H3 x age x Sex, and H4 x age x Sex.\n\n\nCode\ndat2 = combine(\n groupby(\n select(dat, :, :age => ByRow(x -> round(x; digits=1)) => :age),\n [:Sex, :Test, :age],\n ),\n :zScore => mean => :zScore,\n :zScore => length => :n,\n)\n\n\n120×5 DataFrame95 rows omitted\n\n\n\nRow\nSex\nTest\nage\nzScore\nn\n\n\n\nCat…\nCat…\nFloat64\nFloat64\nInt64\n\n\n\n\n1\nGirls\nSpeed\n8.0\n-0.323114\n28\n\n\n2\nGirls\nPowerUP\n8.0\n-0.590476\n26\n\n\n3\nGirls\nPowerLOW\n8.0\n0.0677992\n27\n\n\n4\nGirls\nCoordination\n8.0\n0.0273318\n25\n\n\n5\nGirls\nEndurance\n8.0\n-0.17337\n26\n\n\n6\nBoys\nSpeed\n8.0\n0.394634\n19\n\n\n7\nBoys\nPowerUP\n8.0\n0.328703\n19\n\n\n8\nBoys\nPowerLOW\n8.0\n0.105077\n19\n\n\n9\nBoys\nCoordination\n8.0\n-0.170018\n19\n\n\n10\nBoys\nEndurance\n8.0\n0.407084\n19\n\n\n11\nGirls\nSpeed\n8.1\n-0.205934\n54\n\n\n12\nGirls\nPowerUP\n8.1\n-0.653961\n54\n\n\n13\nGirls\nPowerLOW\n8.1\n-0.157127\n54\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n109\nGirls\nCoordination\n9.0\n-0.000188842\n68\n\n\n110\nGirls\nEndurance\n9.0\n-0.234448\n68\n\n\n111\nGirls\nSpeed\n9.1\n-0.285047\n25\n\n\n112\nGirls\nPowerUP\n9.1\n-0.112684\n25\n\n\n113\nGirls\nPowerLOW\n9.1\n-0.117645\n24\n\n\n114\nGirls\nCoordination\n9.1\n-0.127844\n25\n\n\n115\nGirls\nEndurance\n9.1\n-0.587496\n24\n\n\n116\nBoys\nSpeed\n9.1\n0.379808\n17\n\n\n117\nBoys\nPowerUP\n9.1\n0.523085\n17\n\n\n118\nBoys\nPowerLOW\n9.1\n0.294696\n17\n\n\n119\nBoys\nCoordination\n9.1\n0.209309\n16\n\n\n120\nBoys\nEndurance\n9.1\n0.266512\n16\n\n\n\n\n\n\n\n\nCode\nlet\n design2 = mapping(:age, :zScore; color=:Sex, col=:Test)\n lines2 = design2 * linear()\n means2 = design2 * visual(Scatter; markersize=5)\n draw(data(dat2) * means2 + data(dat) * lines2;)\nend\n\n\n\n\n\n\n\n\nFigure 2: Age trends by sex for each Test for the stratified sample\n\n\n\n\n\nFigure 2 Performance differences for subset of data between 8.0 and 9.2 years by sex in the five physical fitness tests presented as z-transformed data computed separately for each test.\n\nEndurance = cardiorespiratory endurance (i.e., 6-min-run test),\nCoordination = star-run test,\nSpeed = 20-m linear sprint test,\nPowerLOW = power of lower limbs (i.e., standing long jump test),\nPowerUP = power of upper limbs (i.e., ball push test),\nSD = standard deviation. Points are binned observed child means; lines are simple regression fits to the observations.\n\n\n\n3.3.2 Regression on age by Sex for each Test\nAnother set of relevant statistics are the slopes for the regression of performance on age for boys and girls in each of the five tests. The lines in Figures 1 and 2, however, are computed directly from the raw data with the linear() command.\n\ncombine(\n groupby(df, [:Sex, :Test]),\n [:age, :zScore] => simplelinreg => :coef,\n)\n\n10×3 DataFrame\n\n\n\nRow\nSex\nTest\ncoef\n\n\n\nCat…\nCat…\nTuple…\n\n\n\n\n1\nBoys\nEndurance\n(0.00256718, 0.0291899)\n\n\n2\nBoys\nCoordination\n(-2.47279, 0.302819)\n\n\n3\nBoys\nSpeed\n(-2.12689, 0.267153)\n\n\n4\nBoys\nPowerLOW\n(-1.4307, 0.189659)\n\n\n5\nBoys\nPowerUP\n(-4.35864, 0.549005)\n\n\n6\nGirls\nEndurance\n(-0.692022, 0.0523217)\n\n\n7\nGirls\nCoordination\n(-2.50524, 0.279119)\n\n\n8\nGirls\nSpeed\n(-2.34431, 0.255687)\n\n\n9\nGirls\nPowerLOW\n(-1.87241, 0.196917)\n\n\n10\nGirls\nPowerUP\n(-4.82271, 0.524799)\n\n\n\n\n\n\n\ncombine(\n groupby(dat, [:Sex, :Test]),\n [:age, :zScore] => simplelinreg => :coef,\n)\n\n10×3 DataFrame\n\n\n\nRow\nSex\nTest\ncoef\n\n\n\nCat…\nCat…\nTuple…\n\n\n\n\n1\nBoys\nEndurance\n(0.39203, -0.0150694)\n\n\n2\nBoys\nCoordination\n(-3.33518, 0.401051)\n\n\n3\nBoys\nSpeed\n(-1.75685, 0.228662)\n\n\n4\nBoys\nPowerLOW\n(-1.06646, 0.151546)\n\n\n5\nBoys\nPowerUP\n(-4.15536, 0.5245)\n\n\n6\nGirls\nEndurance\n(0.941712, -0.141158)\n\n\n7\nGirls\nCoordination\n(-0.681898, 0.0714891)\n\n\n8\nGirls\nSpeed\n(-0.786382, 0.0725931)\n\n\n9\nGirls\nPowerLOW\n(-0.208472, 0.00150731)\n\n\n10\nGirls\nPowerUP\n(-5.23593, 0.570806)" + }, + { + "objectID": "fggk21.html#seqdiffcoding-of-test", + "href": "fggk21.html#seqdiffcoding-of-test", + "title": "Basics with Emotikon Project", + "section": "3.4 SeqDiffCoding of Test", + "text": "3.4 SeqDiffCoding of Test\nSeqDiffCoding was used in the publication. This specification tests pairwise differences between the five neighboring levels of Test, that is:\n\nH1: Star_r - Run (2-1)\nH2: S20_r - Star_r (3-2)\nH3: SLJ - S20_r (4-3)\nH4: BPT - SLJ (5-4)\n\nThe levels were sorted such that these contrasts map onto four a priori hypotheses; in other words, they are theoretically motivated pairwise comparisons. The motivation also encompasses theoretically motivated interactions with Sex. The order of levels can also be explicitly specified during contrast construction. This is very useful if levels are in a different order in the dataframe.\nNote that random factors Child, School, and Cohort are declared as Grouping variables. Technically, this specification is required for variables with a very large number of levels (e.g., 100K+ children). We recommend the explicit specification for all random factors as a general coding style.\nThe first command recodes names indicating the physical fitness components used in the above figures and tables back to the shorter actual test names. This reduces clutter in LMM outputs.\n\nrecode!(\n dat.Test,\n \"Endurance\" => \"Run\",\n \"Coordination\" => \"Star_r\",\n \"Speed\" => \"S20_r\",\n \"PowerLOW\" => \"SLJ\",\n \"PowerUP\" => \"BMT\",\n)\ncontrasts = merge(\n Dict(nm => SeqDiffCoding() for nm in (:Test, :Sex)),\n Dict(nm => Grouping() for nm in (:Child, :School, :Cohort)),\n);\n\nThe statistical disadvantage of SeqDiffCoding is that the contrasts are not orthogonal, that is the contrasts are correlated. This is obvious from the fact that levels 2, 3, and 4 are all used in two contrasts. One consequence of this is that correlation parameters estimated between neighboring contrasts (e.g., 2-1 and 3-2) are difficult to interpret. Usually, they will be negative because assuming some practical limitations on the overall range (e.g., between levels 1 and 3), a small “2-1” effect “correlates” negatively with a larger “3-2” effect for mathematical reasons.\nObviously, the tradeoff between theoretical motivation and statistical purity is something that must be considered carefully when planning the analysis.\nVarious options for contrast coding are the topic of the MixedModelsTutorial_contrasts_emotikon.jl and MixedModelsTutorial_contrasts_kwdyz.jl notebooks." + }, + { + "objectID": "fggk21.html#lmm-m_ovi", + "href": "fggk21.html#lmm-m_ovi", + "title": "Basics with Emotikon Project", + "section": "4.1 LMM m_ovi", + "text": "4.1 LMM m_ovi\nIn its random-effect structure (RES) we only vary intercepts (i.e., Grand Means) for School (LMM m_ovi), that is we allow that the schools differ in the average fitness of its children, average over the five tests.\nIt is well known that such a simple RES is likely to be anti-conservative with respect to fixed-effect test statistics.\n\nm_ovi = let\n f = @formula zScore ~ 1 + Test * Sex * a1 + (1 | School)\n fit(MixedModel, f, dat; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_School\n\n\n(Intercept)\n-0.0174\n0.0198\n-0.88\n0.3787\n0.3417\n\n\nTest: Star_r\n-0.0026\n0.0303\n-0.09\n0.9304\n\n\n\nTest: S20_r\n0.0097\n0.0302\n0.32\n0.7469\n\n\n\nTest: SLJ\n0.0033\n0.0300\n0.11\n0.9119\n\n\n\nTest: BMT\n-0.0539\n0.0299\n-1.80\n0.0711\n\n\n\nSex: Girls\n-0.4372\n0.0205\n-21.35\n<1e-99\n\n\n\na1\n0.1761\n0.0354\n4.97\n<1e-06\n\n\n\nTest: Star_r & Sex: Girls\n0.3802\n0.0605\n6.28\n<1e-09\n\n\n\nTest: S20_r & Sex: Girls\n-0.2038\n0.0604\n-3.38\n0.0007\n\n\n\nTest: SLJ & Sex: Girls\n-0.0669\n0.0600\n-1.11\n0.2651\n\n\n\nTest: BMT & Sex: Girls\n-0.2730\n0.0598\n-4.57\n<1e-05\n\n\n\nTest: Star_r & a1\n0.3146\n0.1038\n3.03\n0.0024\n\n\n\nTest: S20_r & a1\n-0.0767\n0.1034\n-0.74\n0.4584\n\n\n\nTest: SLJ & a1\n-0.0745\n0.1027\n-0.73\n0.4683\n\n\n\nTest: BMT & a1\n0.4724\n0.1025\n4.61\n<1e-05\n\n\n\nSex: Girls & a1\n-0.1044\n0.0709\n-1.47\n0.1405\n\n\n\nTest: Star_r & Sex: Girls & a1\n-0.1912\n0.2076\n-0.92\n0.3570\n\n\n\nTest: S20_r & Sex: Girls & a1\n0.1742\n0.2068\n0.84\n0.3997\n\n\n\nTest: SLJ & Sex: Girls & a1\n0.0084\n0.2054\n0.04\n0.9672\n\n\n\nTest: BMT & Sex: Girls & a1\n0.1923\n0.2050\n0.94\n0.3482\n\n\n\nResidual\n0.9152\n\n\n\n\n\n\n\n\n\nIs the model singular (overparameterized, degenerate)? In other words: Is the model not supported by the data?\n\nissingular(m_ovi)\n\nfalse\n\n\nModels varying only in intercepts are almost always supported by the data." + }, + { + "objectID": "fggk21.html#lmm-m_zcp", + "href": "fggk21.html#lmm-m_zcp", + "title": "Basics with Emotikon Project", + "section": "4.2 LMM m_zcp", + "text": "4.2 LMM m_zcp\nIn this LMM we allow that schools differ not only in GM, but also in the size of the four contrasts defined for Test, in the difference between boys and girls (Sex) and the developmental gain children achieve within the third grade (age).\nWe assume that there is covariance associated with these CPs beyond residual noise, that is we assume that there is no detectable evidence in the data that the CPs are different from zero.\n\nm_zcp = let\n f = @formula(\n zScore ~\n 1 + Test * Sex * a1 + zerocorr(1 + Test + Sex + a1 | School)\n )\n fit(MixedModel, f, dat; contrasts)\nend\n\nMinimizing 92 Time: 0:00:00 ( 1.09 ms/it)\n objective: 25923.4801198341\nMinimizing 151 Time: 0:00:00 ( 5.35 ms/it)\n\nMinimizing 152 Time: 0:00:00 ( 5.43 ms/it)\n objective: 25923.46046970646\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_School\n\n\n(Intercept)\n-0.0247\n0.0198\n-1.24\n0.2136\n0.3257\n\n\nTest: Star_r\n0.0029\n0.0311\n0.09\n0.9254\n0.2208\n\n\nTest: S20_r\n0.0102\n0.0286\n0.36\n0.7209\n0.0503\n\n\nTest: SLJ\n0.0049\n0.0283\n0.17\n0.8634\n0.0000\n\n\nTest: BMT\n-0.0460\n0.0309\n-1.49\n0.1368\n0.2286\n\n\nSex: Girls\n-0.4399\n0.0322\n-13.64\n<1e-41\n0.4503\n\n\na1\n0.1840\n0.0573\n3.21\n0.0013\n0.7780\n\n\nTest: Star_r & Sex: Girls\n0.3784\n0.0577\n6.55\n<1e-10\n\n\n\nTest: S20_r & Sex: Girls\n-0.2019\n0.0570\n-3.54\n0.0004\n\n\n\nTest: SLJ & Sex: Girls\n-0.0666\n0.0566\n-1.18\n0.2392\n\n\n\nTest: BMT & Sex: Girls\n-0.2780\n0.0570\n-4.87\n<1e-05\n\n\n\nTest: Star_r & a1\n0.3101\n0.0992\n3.13\n0.0018\n\n\n\nTest: S20_r & a1\n-0.0728\n0.0976\n-0.75\n0.4558\n\n\n\nTest: SLJ & a1\n-0.0784\n0.0968\n-0.81\n0.4180\n\n\n\nTest: BMT & a1\n0.4741\n0.0979\n4.84\n<1e-05\n\n\n\nSex: Girls & a1\n-0.1453\n0.0791\n-1.84\n0.0662\n\n\n\nTest: Star_r & Sex: Girls & a1\n-0.1570\n0.1982\n-0.79\n0.4284\n\n\n\nTest: S20_r & Sex: Girls & a1\n0.1799\n0.1952\n0.92\n0.3566\n\n\n\nTest: SLJ & Sex: Girls & a1\n0.0061\n0.1936\n0.03\n0.9749\n\n\n\nTest: BMT & Sex: Girls & a1\n0.1657\n0.1958\n0.85\n0.3974\n\n\n\nResidual\n0.8623\n\n\n\n\n\n\n\n\n\nDepending on sampling, this model estimating variance components for School may or may not be supported by the data.\n\nissingular(m_zcp)\n\ntrue" + }, + { + "objectID": "fggk21.html#lmm-m_cpx", + "href": "fggk21.html#lmm-m_cpx", + "title": "Basics with Emotikon Project", + "section": "4.3 LMM m_cpx", + "text": "4.3 LMM m_cpx\nIn the complex LMM investigated in this sequence we give up the assumption of zero-correlation between VCs.\n\nm_cpx = let\n f = @formula(\n zScore ~ 1 + Test * Sex * a1 + (1 + Test + Sex + a1 | School)\n )\n fit(MixedModel, f, dat; contrasts)\nend\n\nMinimizing 120 Time: 0:00:00 ( 0.84 ms/it)\n objective: 25913.7359953439\nMinimizing 239 Time: 0:00:00 ( 0.85 ms/it)\n objective: 25874.84433102616\nMinimizing 356 Time: 0:00:00 ( 0.85 ms/it)\n objective: 25872.397822536375\nMinimizing 474 Time: 0:00:00 ( 0.85 ms/it)\n objective: 25870.171971430293\nMinimizing 626 Time: 0:00:00 ( 0.89 ms/it)\n objective: 25865.73386935914\nMinimizing 769 Time: 0:00:00 ( 0.88 ms/it)\n objective: 25864.63197654432\nMinimizing 887 Time: 0:00:00 ( 0.88 ms/it)\n objective: 25864.549175037082\nMinimizing 1005 Time: 0:00:00 ( 0.88 ms/it)\n objective: 25864.512781331156\nMinimizing 1124 Time: 0:00:00 ( 0.87 ms/it)\n objective: 25864.43678434264\nMinimizing 1243 Time: 0:00:01 ( 0.87 ms/it)\n objective: 25864.406774217532\nMinimizing 1362 Time: 0:00:01 ( 0.87 ms/it)\n objective: 25864.40183533508\nMinimizing 1482 Time: 0:00:01 ( 0.87 ms/it)\n objective: 25864.394286069102\nMinimizing 1600 Time: 0:00:01 ( 0.87 ms/it)\n objective: 25864.390976008268\nMinimizing 1718 Time: 0:00:01 ( 0.86 ms/it)\n objective: 25864.3897916446\nMinimizing 1830 Time: 0:00:01 ( 0.86 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_School\n\n\n(Intercept)\n-0.0268\n0.0198\n-1.35\n0.1757\n0.3264\n\n\nTest: Star_r\n0.0055\n0.0310\n0.18\n0.8578\n0.2203\n\n\nTest: S20_r\n0.0104\n0.0301\n0.35\n0.7291\n0.1757\n\n\nTest: SLJ\n0.0024\n0.0294\n0.08\n0.9357\n0.1478\n\n\nTest: BMT\n-0.0401\n0.0305\n-1.32\n0.1882\n0.2245\n\n\nSex: Girls\n-0.4356\n0.0320\n-13.61\n<1e-41\n0.4483\n\n\na1\n0.1877\n0.0566\n3.31\n0.0009\n0.7688\n\n\nTest: Star_r & Sex: Girls\n0.3759\n0.0575\n6.53\n<1e-10\n\n\n\nTest: S20_r & Sex: Girls\n-0.1969\n0.0572\n-3.44\n0.0006\n\n\n\nTest: SLJ & Sex: Girls\n-0.0673\n0.0567\n-1.19\n0.2349\n\n\n\nTest: BMT & Sex: Girls\n-0.2717\n0.0566\n-4.80\n<1e-05\n\n\n\nTest: Star_r & a1\n0.3112\n0.0988\n3.15\n0.0016\n\n\n\nTest: S20_r & a1\n-0.0707\n0.0981\n-0.72\n0.4709\n\n\n\nTest: SLJ & a1\n-0.0642\n0.0971\n-0.66\n0.5086\n\n\n\nTest: BMT & a1\n0.4678\n0.0971\n4.82\n<1e-05\n\n\n\nSex: Girls & a1\n-0.1404\n0.0783\n-1.79\n0.0728\n\n\n\nTest: Star_r & Sex: Girls & a1\n-0.1632\n0.1974\n-0.83\n0.4086\n\n\n\nTest: S20_r & Sex: Girls & a1\n0.1862\n0.1961\n0.95\n0.3424\n\n\n\nTest: SLJ & Sex: Girls & a1\n0.0011\n0.1941\n0.01\n0.9954\n\n\n\nTest: BMT & Sex: Girls & a1\n0.1638\n0.1942\n0.84\n0.3990\n\n\n\nResidual\n0.8598\n\n\n\n\n\n\n\n\n\nWe also need to see the VCs and CPs of the random-effect structure (RES).\n\nVarCorr(m_cpx)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\n\nSchool\n(Intercept)\n0.106541\n0.326406\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.048537\n0.220312\n+0.10\n\n\n\n\n\n\n\n\nTest: S20_r\n0.030880\n0.175726\n-0.19\n-0.22\n\n\n\n\n\n\n\nTest: SLJ\n0.021853\n0.147828\n-0.10\n+0.07\n-0.83\n\n\n\n\n\n\nTest: BMT\n0.050389\n0.224474\n-0.75\n+0.43\n-0.13\n+0.16\n\n\n\n\n\nSex: Girls\n0.200960\n0.448286\n-0.04\n+0.25\n+0.20\n-0.27\n-0.10\n\n\n\n\na1\n0.591058\n0.768803\n+0.02\n-0.13\n+0.03\n-0.49\n+0.17\n-0.06\n\n\nResidual\n\n0.739252\n0.859798\n\n\n\n\n\n\n\n\n\n\n\n\nissingular(m_cpx)\n\nfalse\n\n\nThe complex model may or may not be supported by the data." + }, + { + "objectID": "fggk21.html#model-comparisons", + "href": "fggk21.html#model-comparisons", + "title": "Basics with Emotikon Project", + "section": "4.4 Model comparisons", + "text": "4.4 Model comparisons\nThe checks of model singularity indicate that the three models are supported by the data. Does model complexification also increase the goodness of fit or are we only fitting noise?\n\n4.4.1 LRT and goodness-of-fit statistics\nAs the thee models are strictly hierarchically nested, we compare them with a likelihood-ratio tests (LRT) and AIC and BIC goodness-of-fit statistics derived from them.\n\nMixedModels.likelihoodratiotest(m_ovi, m_zcp, m_cpx)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\nzScore ~ 1 + Test + Sex + a1 + Test & Sex + Test & a1 + Sex & a1 + Test & Sex & a1 + (1 | School)\n22\n26273\n\n\n\n\n\nzScore ~ 1 + Test + Sex + a1 + Test & Sex + Test & a1 + Sex & a1 + Test & Sex & a1 + zerocorr(1 + Test + Sex + a1 | School)\n28\n25923\n349\n6\n<1e-71\n\n\nzScore ~ 1 + Test + Sex + a1 + Test & Sex + Test & a1 + Sex & a1 + Test & Sex & a1 + (1 + Test + Sex + a1 | School)\n49\n25864\n59\n21\n<1e-04\n\n\n\n\n\n\n\nCode\ngof_summary = let\n nms = [:m_ovi, :m_zcp, :m_cpx]\n mods = eval.(nms)\n DataFrame(;\n name=nms,\n dof=dof.(mods),\n deviance=deviance.(mods),\n AIC=aic.(mods),\n AICc=aicc.(mods),\n BIC=bic.(mods),\n )\nend\n\n\n3×6 DataFrame\n\n\n\nRow\nname\ndof\ndeviance\nAIC\nAICc\nBIC\n\n\n\nSymbol\nInt64\nFloat64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\nm_ovi\n22\n26272.9\n26316.9\n26317.0\n26474.8\n\n\n2\nm_zcp\n28\n25923.5\n25979.5\n25979.6\n26180.4\n\n\n3\nm_cpx\n49\n25864.4\n25962.4\n25962.9\n26314.0\n\n\n\n\n\n\nThese statistics will depend on sampling. In general, smaller deviance, AIC, and BIC indicate an improvement in goodness of fit. Usually, χ² should be larger than the associated degrees of freedom; for AIC and BIC the decrease should amount to more than 5, according to some literature. Severity of meeting these criteria increases from deviance to AIC to BIC. Therefore, it is not always the case that the criteria are unanimous in their verdict. Basically, the more confirmatory the analysis, the more one may go with deviance and AIC; for exploratory analyses the BIC is probably a better guide. There are grey zones here.\n\n\n4.4.2 Comparing fixed effects of m_ovi, m_zcp, and m_cpx\nWe check whether enriching the RES changed the significance of fixed effects in the final model.\n\n\nCode\nm_ovi_fe = DataFrame(coeftable(m_ovi));\nm_zcp_fe = DataFrame(coeftable(m_zcp));\nm_cpx_fe = DataFrame(coeftable(m_cpx));\nm_all = hcat(\n m_ovi_fe[:, [1, 2, 4]],\n leftjoin(\n m_zcp_fe[:, [1, 2, 4]],\n m_cpx_fe[:, [1, 2, 4]];\n on=:Name,\n makeunique=true,\n );\n makeunique=true,\n)\nrename!(\n m_all,\n \"Coef.\" => \"b_ovi\",\n \"Coef._2\" => \"b_zcp\",\n \"Coef._1\" => \"b_cpx\",\n \"z\" => \"z_ovi\",\n \"z_2\" => \"z_zcp\",\n \"z_1\" => \"z_cpx\",\n)\nm_all2 =\n round.(\n m_all[:, [:b_ovi, :b_zcp, :b_cpx, :z_ovi, :z_zcp, :z_cpx]],\n digits=2,\n )\nm_all3 = hcat(m_all.Name, m_all2)\n\n\n20×7 DataFrame\n\n\n\nRow\nx1\nb_ovi\nb_zcp\nb_cpx\nz_ovi\nz_zcp\nz_cpx\n\n\n\nString\nFloat64\nFloat64\nFloat64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\n(Intercept)\n-0.02\n-0.02\n-0.03\n-0.88\n-1.24\n-1.35\n\n\n2\nTest: Star_r\n-0.0\n0.0\n0.01\n-0.09\n0.09\n0.18\n\n\n3\nTest: S20_r\n0.01\n0.01\n0.01\n0.32\n0.36\n0.35\n\n\n4\nTest: SLJ\n0.0\n0.0\n0.0\n0.11\n0.17\n0.08\n\n\n5\nTest: BMT\n-0.05\n-0.05\n-0.04\n-1.8\n-1.49\n-1.32\n\n\n6\nSex: Girls\n-0.44\n-0.44\n-0.44\n-21.35\n-13.64\n-13.61\n\n\n7\na1\n0.18\n0.18\n0.19\n4.97\n3.21\n3.31\n\n\n8\nTest: Star_r & Sex: Girls\n0.38\n0.38\n0.38\n6.28\n6.55\n6.53\n\n\n9\nTest: S20_r & Sex: Girls\n-0.2\n-0.2\n-0.2\n-3.38\n-3.54\n-3.44\n\n\n10\nTest: SLJ & Sex: Girls\n-0.07\n-0.07\n-0.07\n-1.11\n-1.18\n-1.19\n\n\n11\nTest: BMT & Sex: Girls\n-0.27\n-0.28\n-0.27\n-4.57\n-4.87\n-4.8\n\n\n12\nTest: Star_r & a1\n0.31\n0.31\n0.31\n3.03\n3.13\n3.15\n\n\n13\nTest: S20_r & a1\n-0.08\n-0.07\n-0.07\n-0.74\n-0.75\n-0.72\n\n\n14\nTest: SLJ & a1\n-0.07\n-0.08\n-0.06\n-0.73\n-0.81\n-0.66\n\n\n15\nTest: BMT & a1\n0.47\n0.47\n0.47\n4.61\n4.84\n4.82\n\n\n16\nSex: Girls & a1\n-0.1\n-0.15\n-0.14\n-1.47\n-1.84\n-1.79\n\n\n17\nTest: Star_r & Sex: Girls & a1\n-0.19\n-0.16\n-0.16\n-0.92\n-0.79\n-0.83\n\n\n18\nTest: S20_r & Sex: Girls & a1\n0.17\n0.18\n0.19\n0.84\n0.92\n0.95\n\n\n19\nTest: SLJ & Sex: Girls & a1\n0.01\n0.01\n0.0\n0.04\n0.03\n0.01\n\n\n20\nTest: BMT & Sex: Girls & a1\n0.19\n0.17\n0.16\n0.94\n0.85\n0.84\n\n\n\n\n\n\nThe three models usually do not differ in fixed-effect estimates. For main effects of age and Sex, z-values decrease strongly with the complexity of the model (i.e., standard errors are larger). For other coefficients, the changes are not very large and not consistent.\nIn general, dropping significant variance components and/or correlation parameters may lead to anti-conservative estimates of fixed effects (e.g., Schielzeth & Forstmeier, 2008). Basically, some of the variance allocated to age and Sex in LMM m_ovi could also be due to differences between schools. This ambiguity increased the uncertainty of the respective fixed effects in the other two LMMs." + }, + { + "objectID": "fggk21.html#fitting-an-overparameterized-lmm", + "href": "fggk21.html#fitting-an-overparameterized-lmm", + "title": "Basics with Emotikon Project", + "section": "4.5 Fitting an overparameterized LMM", + "text": "4.5 Fitting an overparameterized LMM\nThe complex LMM was not overparameterized with respect to School, because there are over 400 schools in the data. When the number of units (levels) of a grouping factor is small relative to the number of parameters we are trying to estimate, we often end up with an overparameterized / degenerate random-effect structure.\nAs an illustration, we fit a full CP matrix for the Cohort. As there are only nine cohorts in the data, we may be asking too much to estimate 5*6/2 = 15 VC/CP parameters.\n\nm_cpxCohort = let\n f = @formula zScore ~ 1 + Test * a1 * Sex + (1 + Test | Cohort)\n fit(MixedModel, f, dat; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Cohort\n\n\n(Intercept)\n-0.0009\n0.0161\n-0.06\n0.9548\n0.0378\n\n\nTest: Star_r\n-0.0073\n0.0380\n-0.19\n0.8482\n0.0614\n\n\nTest: S20_r\n0.0056\n0.0383\n0.15\n0.8845\n0.0636\n\n\nTest: SLJ\n0.0101\n0.0451\n0.22\n0.8226\n0.0959\n\n\nTest: BMT\n-0.0556\n0.0335\n-1.66\n0.0968\n0.0330\n\n\na1\n0.2055\n0.0349\n5.90\n<1e-08\n\n\n\nSex: Girls\n-0.4238\n0.0201\n-21.07\n<1e-97\n\n\n\nTest: Star_r & a1\n0.2849\n0.1101\n2.59\n0.0097\n\n\n\nTest: S20_r & a1\n-0.1172\n0.1096\n-1.07\n0.2851\n\n\n\nTest: SLJ & a1\n-0.0270\n0.1094\n-0.25\n0.8049\n\n\n\nTest: BMT & a1\n0.4555\n0.1084\n4.20\n<1e-04\n\n\n\nTest: Star_r & Sex: Girls\n0.3700\n0.0640\n5.78\n<1e-08\n\n\n\nTest: S20_r & Sex: Girls\n-0.2116\n0.0638\n-3.32\n0.0009\n\n\n\nTest: SLJ & Sex: Girls\n-0.0552\n0.0634\n-0.87\n0.3844\n\n\n\nTest: BMT & Sex: Girls\n-0.2718\n0.0632\n-4.30\n<1e-04\n\n\n\na1 & Sex: Girls\n-0.1368\n0.0690\n-1.98\n0.0473\n\n\n\nTest: Star_r & a1 & Sex: Girls\n-0.2099\n0.2194\n-0.96\n0.3387\n\n\n\nTest: S20_r & a1 & Sex: Girls\n0.1609\n0.2186\n0.74\n0.4616\n\n\n\nTest: SLJ & a1 & Sex: Girls\n0.0225\n0.2172\n0.10\n0.9174\n\n\n\nTest: BMT & a1 & Sex: Girls\n0.1901\n0.2166\n0.88\n0.3801\n\n\n\nResidual\n0.9671\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_cpxCohort)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\nCohort\n(Intercept)\n0.0014274\n0.0377808\n\n\n\n\n\n\n\nTest: Star_r\n0.0037663\n0.0613705\n-0.90\n\n\n\n\n\n\nTest: S20_r\n0.0040438\n0.0635905\n-1.00\n+0.87\n\n\n\n\n\nTest: SLJ\n0.0091897\n0.0958627\n+0.98\n-0.97\n-0.97\n\n\n\n\nTest: BMT\n0.0010888\n0.0329966\n-1.00\n+0.91\n+0.99\n-0.99\n\n\nResidual\n\n0.9353139\n0.9671163\n\n\n\n\n\n\n\n\n\n\nissingular(m_cpxCohort)\n\ntrue\n\n\nThe model is overparameterized with several CPs estimated between |.98| and |1.00|. How about the zero-correlation parameter (zcp) version of this LMM?\n\nm_zcpCohort = let\n f = @formula(\n zScore ~ 1 + Test * a1 * Sex + zerocorr(1 + Test | Cohort)\n )\n fit(MixedModel, f, dat; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Cohort\n\n\n(Intercept)\n-0.0022\n0.0152\n-0.15\n0.8837\n0.0341\n\n\nTest: Star_r\n-0.0042\n0.0339\n-0.12\n0.9023\n0.0331\n\n\nTest: S20_r\n0.0088\n0.0319\n0.27\n0.7837\n0.0000\n\n\nTest: SLJ\n0.0045\n0.0317\n0.14\n0.8876\n0.0000\n\n\nTest: BMT\n-0.0536\n0.0316\n-1.69\n0.0903\n0.0000\n\n\na1\n0.1999\n0.0351\n5.70\n<1e-07\n\n\n\nSex: Girls\n-0.4245\n0.0201\n-21.08\n<1e-97\n\n\n\nTest: Star_r & a1\n0.3078\n0.1101\n2.80\n0.0052\n\n\n\nTest: S20_r & a1\n-0.0849\n0.1093\n-0.78\n0.4377\n\n\n\nTest: SLJ & a1\n-0.0748\n0.1086\n-0.69\n0.4911\n\n\n\nTest: BMT & a1\n0.4717\n0.1084\n4.35\n<1e-04\n\n\n\nTest: Star_r & Sex: Girls\n0.3729\n0.0640\n5.82\n<1e-08\n\n\n\nTest: S20_r & Sex: Girls\n-0.2079\n0.0638\n-3.26\n0.0011\n\n\n\nTest: SLJ & Sex: Girls\n-0.0611\n0.0634\n-0.96\n0.3354\n\n\n\nTest: BMT & Sex: Girls\n-0.2697\n0.0632\n-4.27\n<1e-04\n\n\n\na1 & Sex: Girls\n-0.1372\n0.0691\n-1.99\n0.0470\n\n\n\nTest: Star_r & a1 & Sex: Girls\n-0.2041\n0.2195\n-0.93\n0.3525\n\n\n\nTest: S20_r & a1 & Sex: Girls\n0.1733\n0.2187\n0.79\n0.4280\n\n\n\nTest: SLJ & a1 & Sex: Girls\n0.0051\n0.2172\n0.02\n0.9811\n\n\n\nTest: BMT & a1 & Sex: Girls\n0.1962\n0.2168\n0.90\n0.3655\n\n\n\nResidual\n0.9680\n\n\n\n\n\n\n\n\n\n\nissingular(m_zcpCohort)\n\ntrue\n\n\nThis zcpLMM is also singular. Three of the five VCs are estimated as zero. This raises the possibility that LMM m_oviCohort might fit as well as LMM m_zcpCohort.\n\nm_oviCohort = let\n f = @formula zScore ~ 1 + Test * a1 * Sex + (1 | Cohort)\n fit(MixedModel, f, dat; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Cohort\n\n\n(Intercept)\n-0.0022\n0.0152\n-0.15\n0.8835\n0.0340\n\n\nTest: Star_r\n-0.0032\n0.0320\n-0.10\n0.9196\n\n\n\nTest: S20_r\n0.0087\n0.0319\n0.27\n0.7847\n\n\n\nTest: SLJ\n0.0045\n0.0317\n0.14\n0.8869\n\n\n\nTest: BMT\n-0.0536\n0.0316\n-1.69\n0.0903\n\n\n\na1\n0.1999\n0.0351\n5.69\n<1e-07\n\n\n\nSex: Girls\n-0.4245\n0.0201\n-21.08\n<1e-97\n\n\n\nTest: Star_r & a1\n0.3135\n0.1097\n2.86\n0.0043\n\n\n\nTest: S20_r & a1\n-0.0848\n0.1093\n-0.78\n0.4378\n\n\n\nTest: SLJ & a1\n-0.0748\n0.1086\n-0.69\n0.4910\n\n\n\nTest: BMT & a1\n0.4718\n0.1084\n4.35\n<1e-04\n\n\n\nTest: Star_r & Sex: Girls\n0.3736\n0.0640\n5.84\n<1e-08\n\n\n\nTest: S20_r & Sex: Girls\n-0.2079\n0.0638\n-3.26\n0.0011\n\n\n\nTest: SLJ & Sex: Girls\n-0.0611\n0.0634\n-0.96\n0.3356\n\n\n\nTest: BMT & Sex: Girls\n-0.2697\n0.0632\n-4.27\n<1e-04\n\n\n\na1 & Sex: Girls\n-0.1371\n0.0691\n-1.99\n0.0471\n\n\n\nTest: Star_r & a1 & Sex: Girls\n-0.2022\n0.2195\n-0.92\n0.3568\n\n\n\nTest: S20_r & a1 & Sex: Girls\n0.1733\n0.2187\n0.79\n0.4281\n\n\n\nTest: SLJ & a1 & Sex: Girls\n0.0051\n0.2172\n0.02\n0.9811\n\n\n\nTest: BMT & a1 & Sex: Girls\n0.1961\n0.2168\n0.90\n0.3657\n\n\n\nResidual\n0.9681\n\n\n\n\n\n\n\n\n\n\nissingular(m_oviCohort)\n\nfalse\n\n\nThis solves the problem with singularity, but does LMM m_zcpCohort fit noise relative to the LMM m_oviCohort?\n\nMixedModels.likelihoodratiotest(m_oviCohort, m_zcpCohort)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 | Cohort)\n22\n26803\n\n\n\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Cohort)\n26\n26803\n0\n4\n0.9968\n\n\n\n\n\n\ngof_summary2 = let\n mods = [m_oviCohort, m_zcpCohort, m_cpxCohort]\n DataFrame(;\n dof=dof.(mods),\n deviance=deviance.(mods),\n AIC=aic.(mods),\n AICc=aicc.(mods),\n BIC=bic.(mods),\n )\nend\n\n3×5 DataFrame\n\n\n\nRow\ndof\ndeviance\nAIC\nAICc\nBIC\n\n\n\nInt64\nFloat64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\n22\n26802.9\n26846.9\n26847.0\n27004.7\n\n\n2\n26\n26802.7\n26854.7\n26854.8\n27041.3\n\n\n3\n36\n26790.5\n26862.5\n26862.7\n27120.8\n\n\n\n\n\n\nIndeed, adding VCs is fitting noise. Again, the goodness of fit statistics unanimously favor the selection of the LMM m_oviCohort.\nNot shown here, but the Cohort-related VCs for the Test contrasts could be estimated reliably for the full data. Thus, the small number of cohorts does not necessarily prevent the determination of reliable differences between tests across cohorts. What if we include VCs and CPs related to random factors Child and School?" + }, + { + "objectID": "fggk21.html#fitting-the-published-lmm-m1-to-the-reduced-data", + "href": "fggk21.html#fitting-the-published-lmm-m1-to-the-reduced-data", + "title": "Basics with Emotikon Project", + "section": "4.6 Fitting the published LMM m1 to the reduced data", + "text": "4.6 Fitting the published LMM m1 to the reduced data\n\n\n\n\n\n\nWarning\n\n\n\nThe following LMMs m1, m2, etc. take a bit longer (e.g., close to 6 minutes in the Pluto notebook, close to 3 minutes in the REPL on a MacBook Pro).\n\n\nLMM m1 reported in Fühner et al. (2021) included random factors for School, Child, and Cohort. The RES for School was specified like in LMM m_cpx. The RES for Child included VCs and CPs for Test, but not for linear developmental gain in the ninth year of life a1 or Sex; they are between-Child effects.\nThe RES for Cohort included only VCs, no CPs for Test. The parsimony was due to the small number of nine levels for this grouping factor.\nHere we fit this LMM m1 for the reduced data. For a different subset of similar size on MacBook Pro [13 | 15 | 16] this took [303 | 250 | 244 ] s; for LMM m1a (i.e., dropping 1 school-relate VC for Sex), times are [212 | 165 | 160] s. The corresponding lme4 times for LMM m1 are [397 | 348 | 195].\nFinally, times for fitting the full set of data –not in this script–, for LMM m1are [60 | 62 | 85] minutes (!); for LMM m1a the times were [46 | 48 | 34] minutes. It was not possible to fit the full set of data with lme4; after about 13 to 18 minutes the program stopped with: Error in eval_f(x, ...) : Downdated VtV is not positive definite.\n\nm1 = let\n f = @formula(\n zScore ~\n 1 +\n Test * a1 * Sex +\n (1 + Test + a1 + Sex | School) +\n (1 + Test | Child) +\n zerocorr(1 + Test | Cohort)\n )\n fit(MixedModel, f, dat; contrasts)\nend\n\nMinimizing 2 Time: 0:00:01 ( 0.61 s/it)\n objective: 26263.798630532\nMinimizing 10 Time: 0:00:01 ( 0.13 s/it)\n objective: 26032.802373659586\nMinimizing 18 Time: 0:00:01 (79.12 ms/it)\n objective: 25909.53939555904\nMinimizing 26 Time: 0:00:01 (58.76 ms/it)\n objective: 25986.562881083046\nMinimizing 34 Time: 0:00:01 (48.00 ms/it)\n objective: 25898.37560027311\nMinimizing 42 Time: 0:00:01 (41.44 ms/it)\n objective: 25880.23790580926\nMinimizing 50 Time: 0:00:01 (36.87 ms/it)\n objective: 27685.247320995768\nMinimizing 58 Time: 0:00:01 (33.60 ms/it)\n objective: 26001.805014197045\nMinimizing 66 Time: 0:00:02 (31.12 ms/it)\n objective: 25912.198496703462\nMinimizing 74 Time: 0:00:02 (29.18 ms/it)\n objective: 25949.27949843974\nMinimizing 82 Time: 0:00:02 (27.63 ms/it)\n objective: 25921.90653011039\nMinimizing 90 Time: 0:00:02 (26.37 ms/it)\n objective: 25836.707585481112\nMinimizing 98 Time: 0:00:02 (25.28 ms/it)\n objective: 25218.359672227205\nMinimizing 106 Time: 0:00:02 (24.35 ms/it)\n objective: 25054.4576240143\nMinimizing 114 Time: 0:00:02 (23.55 ms/it)\n objective: 25052.320501610564\nMinimizing 122 Time: 0:00:02 (22.84 ms/it)\n objective: 24999.384497783954\nMinimizing 130 Time: 0:00:02 (22.27 ms/it)\n objective: 24956.41343582866\nMinimizing 138 Time: 0:00:03 (21.75 ms/it)\n objective: 24914.946584626545\nMinimizing 146 Time: 0:00:03 (21.26 ms/it)\n objective: 24893.847196663002\nMinimizing 154 Time: 0:00:03 (20.83 ms/it)\n objective: 24865.46703494803\nMinimizing 162 Time: 0:00:03 (20.43 ms/it)\n objective: 24852.889649488112\nMinimizing 170 Time: 0:00:03 (20.08 ms/it)\n objective: 24833.636378199408\nMinimizing 178 Time: 0:00:03 (19.76 ms/it)\n objective: 24821.186037541243\nMinimizing 186 Time: 0:00:03 (19.46 ms/it)\n objective: 24791.370242087396\nMinimizing 194 Time: 0:00:03 (19.23 ms/it)\n objective: 24786.669733863615\nMinimizing 202 Time: 0:00:03 (19.02 ms/it)\n objective: 24772.413565464158\nMinimizing 210 Time: 0:00:03 (18.83 ms/it)\n objective: 24766.46535676397\nMinimizing 218 Time: 0:00:04 (18.62 ms/it)\n objective: 24762.134758352295\nMinimizing 226 Time: 0:00:04 (18.44 ms/it)\n objective: 24757.76410159139\nMinimizing 234 Time: 0:00:04 (18.25 ms/it)\n objective: 24753.58928112908\nMinimizing 242 Time: 0:00:04 (18.08 ms/it)\n objective: 24747.304757821454\nMinimizing 250 Time: 0:00:04 (17.94 ms/it)\n objective: 24742.824950872637\nMinimizing 257 Time: 0:00:04 (17.84 ms/it)\n objective: 24744.29596899636\nMinimizing 265 Time: 0:00:04 (17.72 ms/it)\n objective: 24739.72217648851\nMinimizing 273 Time: 0:00:04 (17.60 ms/it)\n objective: 24736.572876439044\nMinimizing 281 Time: 0:00:04 (17.47 ms/it)\n objective: 24731.59897556257\nMinimizing 289 Time: 0:00:05 (17.34 ms/it)\n objective: 24725.792527037745\nMinimizing 297 Time: 0:00:05 (17.24 ms/it)\n objective: 24723.377321071024\nMinimizing 305 Time: 0:00:05 (17.15 ms/it)\n objective: 24718.708289643593\nMinimizing 313 Time: 0:00:05 (17.05 ms/it)\n objective: 24714.227854491233\nMinimizing 321 Time: 0:00:05 (16.95 ms/it)\n objective: 24716.813508738473\nMinimizing 329 Time: 0:00:05 (16.88 ms/it)\n objective: 24715.458991594605\nMinimizing 337 Time: 0:00:05 (16.81 ms/it)\n objective: 24712.949077361896\nMinimizing 345 Time: 0:00:05 (16.75 ms/it)\n objective: 24708.166068056376\nMinimizing 353 Time: 0:00:05 (16.68 ms/it)\n objective: 24706.765806621424\nMinimizing 361 Time: 0:00:06 (16.63 ms/it)\n objective: 24707.913284010036\nMinimizing 369 Time: 0:00:06 (16.55 ms/it)\n objective: 24705.05082837271\nMinimizing 377 Time: 0:00:06 (16.48 ms/it)\n objective: 24703.819443003686\nMinimizing 385 Time: 0:00:06 (16.41 ms/it)\n objective: 24703.677592919787\nMinimizing 393 Time: 0:00:06 (16.34 ms/it)\n objective: 24703.18196364243\nMinimizing 401 Time: 0:00:06 (16.27 ms/it)\n objective: 24702.363946121135\nMinimizing 409 Time: 0:00:06 (16.21 ms/it)\n objective: 24701.63190102658\nMinimizing 417 Time: 0:00:06 (16.14 ms/it)\n objective: 24701.09678875273\nMinimizing 425 Time: 0:00:06 (16.09 ms/it)\n objective: 24700.896820937352\nMinimizing 433 Time: 0:00:06 (16.03 ms/it)\n objective: 24700.61501064425\nMinimizing 441 Time: 0:00:07 (15.98 ms/it)\n objective: 24700.15273724909\nMinimizing 449 Time: 0:00:07 (15.92 ms/it)\n objective: 24699.86376118735\nMinimizing 457 Time: 0:00:07 (15.87 ms/it)\n objective: 24699.362939096965\nMinimizing 465 Time: 0:00:07 (15.82 ms/it)\n objective: 24699.246571817333\nMinimizing 473 Time: 0:00:07 (15.84 ms/it)\n objective: 24699.151605194136\nMinimizing 481 Time: 0:00:07 (15.80 ms/it)\n objective: 24699.115862227438\nMinimizing 489 Time: 0:00:07 (15.76 ms/it)\n objective: 24698.8833308756\nMinimizing 497 Time: 0:00:07 (15.72 ms/it)\n objective: 24698.1518893173\nMinimizing 505 Time: 0:00:07 (15.68 ms/it)\n objective: 24698.060517805414\nMinimizing 513 Time: 0:00:08 (15.65 ms/it)\n objective: 24697.917889344237\nMinimizing 521 Time: 0:00:08 (15.61 ms/it)\n objective: 24697.383587125103\nMinimizing 529 Time: 0:00:08 (15.57 ms/it)\n objective: 24696.53884071827\nMinimizing 537 Time: 0:00:08 (15.53 ms/it)\n objective: 24695.698401968824\nMinimizing 545 Time: 0:00:08 (15.49 ms/it)\n objective: 24695.55574667539\nMinimizing 553 Time: 0:00:08 (15.50 ms/it)\n objective: 24695.12738701081\nMinimizing 560 Time: 0:00:08 (15.51 ms/it)\n objective: 24693.13130315552\nMinimizing 567 Time: 0:00:08 (15.53 ms/it)\n objective: 24692.038142708494\nMinimizing 574 Time: 0:00:08 (15.54 ms/it)\n objective: 24678.58646800586\nMinimizing 581 Time: 0:00:09 (15.55 ms/it)\n objective: 24678.818908833953\nMinimizing 587 Time: 0:00:09 (15.57 ms/it)\n objective: 24676.389341414582\nMinimizing 593 Time: 0:00:09 (15.58 ms/it)\n objective: 24671.56373576434\nMinimizing 599 Time: 0:00:09 (15.59 ms/it)\n objective: 24667.705378898834\nMinimizing 606 Time: 0:00:09 (15.61 ms/it)\n objective: 24667.11381795389\nMinimizing 614 Time: 0:00:09 (15.57 ms/it)\n objective: 24666.42084029146\nMinimizing 622 Time: 0:00:09 (15.54 ms/it)\n objective: 24661.542615706596\nMinimizing 630 Time: 0:00:09 (15.52 ms/it)\n objective: 24660.649476650564\nMinimizing 638 Time: 0:00:09 (15.49 ms/it)\n objective: 24659.11680999573\nMinimizing 646 Time: 0:00:09 (15.46 ms/it)\n objective: 24658.428278522435\nMinimizing 654 Time: 0:00:10 (15.43 ms/it)\n objective: 24657.25952023676\nMinimizing 662 Time: 0:00:10 (15.40 ms/it)\n objective: 24656.446859476826\nMinimizing 670 Time: 0:00:10 (15.37 ms/it)\n objective: 24655.168140020396\nMinimizing 678 Time: 0:00:10 (15.34 ms/it)\n objective: 24654.28041030011\nMinimizing 686 Time: 0:00:10 (15.31 ms/it)\n objective: 24654.06780229167\nMinimizing 694 Time: 0:00:10 (15.28 ms/it)\n objective: 24653.68974304936\nMinimizing 702 Time: 0:00:10 (15.25 ms/it)\n objective: 24653.619827101793\nMinimizing 710 Time: 0:00:10 (15.22 ms/it)\n objective: 24653.302664698494\nMinimizing 718 Time: 0:00:10 (15.19 ms/it)\n objective: 24652.929449183524\nMinimizing 726 Time: 0:00:11 (15.17 ms/it)\n objective: 24652.857108753622\nMinimizing 734 Time: 0:00:11 (15.15 ms/it)\n objective: 24652.625565747767\nMinimizing 742 Time: 0:00:11 (15.13 ms/it)\n objective: 24652.411127345717\nMinimizing 750 Time: 0:00:11 (15.10 ms/it)\n objective: 24652.463586626553\nMinimizing 758 Time: 0:00:11 (15.08 ms/it)\n objective: 24652.239417018252\nMinimizing 766 Time: 0:00:11 (15.05 ms/it)\n objective: 24652.160141388435\nMinimizing 774 Time: 0:00:11 (15.03 ms/it)\n objective: 24652.03116855609\nMinimizing 782 Time: 0:00:11 (15.02 ms/it)\n objective: 24651.847046943094\nMinimizing 790 Time: 0:00:11 (14.99 ms/it)\n objective: 24651.784127586812\nMinimizing 798 Time: 0:00:11 (14.97 ms/it)\n objective: 24651.69274006296\nMinimizing 806 Time: 0:00:12 (14.95 ms/it)\n objective: 24651.687996332294\nMinimizing 814 Time: 0:00:12 (14.94 ms/it)\n objective: 24651.635508109248\nMinimizing 822 Time: 0:00:12 (14.92 ms/it)\n objective: 24651.56875905724\nMinimizing 830 Time: 0:00:12 (14.90 ms/it)\n objective: 24651.496126553197\nMinimizing 838 Time: 0:00:12 (14.88 ms/it)\n objective: 24651.40481671076\nMinimizing 846 Time: 0:00:12 (14.86 ms/it)\n objective: 24651.407354575\nMinimizing 854 Time: 0:00:12 (14.84 ms/it)\n objective: 24651.323434626982\nMinimizing 862 Time: 0:00:12 (14.83 ms/it)\n objective: 24651.29504631603\nMinimizing 870 Time: 0:00:12 (14.81 ms/it)\n objective: 24651.311300901056\nMinimizing 878 Time: 0:00:12 (14.79 ms/it)\n objective: 24651.298276056354\nMinimizing 886 Time: 0:00:13 (14.77 ms/it)\n objective: 24651.31025509899\nMinimizing 894 Time: 0:00:13 (14.76 ms/it)\n objective: 24651.274571547197\nMinimizing 902 Time: 0:00:13 (14.75 ms/it)\n objective: 24651.26716024734\nMinimizing 910 Time: 0:00:13 (14.74 ms/it)\n objective: 24651.248490126316\nMinimizing 918 Time: 0:00:13 (14.73 ms/it)\n objective: 24651.22649740443\nMinimizing 926 Time: 0:00:13 (14.71 ms/it)\n objective: 24651.210418520248\nMinimizing 934 Time: 0:00:13 (14.70 ms/it)\n objective: 24651.19685424194\nMinimizing 942 Time: 0:00:13 (14.69 ms/it)\n objective: 24651.18964335481\nMinimizing 950 Time: 0:00:13 (14.68 ms/it)\n objective: 24651.17759926623\nMinimizing 958 Time: 0:00:14 (14.66 ms/it)\n objective: 24651.149153464376\nMinimizing 966 Time: 0:00:14 (14.65 ms/it)\n objective: 24651.129084109743\nMinimizing 974 Time: 0:00:14 (14.64 ms/it)\n objective: 24651.11081631579\nMinimizing 982 Time: 0:00:14 (14.63 ms/it)\n objective: 24651.10298571695\nMinimizing 990 Time: 0:00:14 (14.61 ms/it)\n objective: 24651.098979115603\nMinimizing 998 Time: 0:00:14 (14.60 ms/it)\n objective: 24651.093761686083\nMinimizing 1006 Time: 0:00:14 (14.59 ms/it)\n objective: 24651.09213271858\nMinimizing 1014 Time: 0:00:14 (14.57 ms/it)\n objective: 24651.08392533141\nMinimizing 1022 Time: 0:00:14 (14.56 ms/it)\n objective: 24651.067859962568\nMinimizing 1030 Time: 0:00:14 (14.55 ms/it)\n objective: 24651.06520194812\nMinimizing 1038 Time: 0:00:15 (14.54 ms/it)\n objective: 24651.060810670366\nMinimizing 1046 Time: 0:00:15 (14.53 ms/it)\n objective: 24651.05670350074\nMinimizing 1054 Time: 0:00:15 (14.51 ms/it)\n objective: 24651.054605778412\nMinimizing 1062 Time: 0:00:15 (14.50 ms/it)\n objective: 24651.049284248373\nMinimizing 1070 Time: 0:00:15 (14.49 ms/it)\n objective: 24651.0464201929\nMinimizing 1078 Time: 0:00:15 (14.48 ms/it)\n objective: 24651.045596860844\nMinimizing 1086 Time: 0:00:15 (14.47 ms/it)\n objective: 24651.040420920246\nMinimizing 1094 Time: 0:00:15 (14.46 ms/it)\n objective: 24651.039182749253\nMinimizing 1102 Time: 0:00:15 (14.45 ms/it)\n objective: 24651.037359140148\nMinimizing 1110 Time: 0:00:16 (14.44 ms/it)\n objective: 24651.03330462238\nMinimizing 1118 Time: 0:00:16 (14.43 ms/it)\n objective: 24651.03151596733\nMinimizing 1126 Time: 0:00:16 (14.42 ms/it)\n objective: 24651.030760071368\nMinimizing 1134 Time: 0:00:16 (14.41 ms/it)\n objective: 24651.027194386923\nMinimizing 1142 Time: 0:00:16 (14.40 ms/it)\n objective: 24651.025631575212\nMinimizing 1150 Time: 0:00:16 (14.38 ms/it)\n objective: 24651.024596174822\nMinimizing 1158 Time: 0:00:16 (14.37 ms/it)\n objective: 24651.02241015472\nMinimizing 1166 Time: 0:00:16 (14.36 ms/it)\n objective: 24651.021905047914\nMinimizing 1174 Time: 0:00:16 (14.35 ms/it)\n objective: 24651.020751495194\nMinimizing 1182 Time: 0:00:16 (14.34 ms/it)\n objective: 24651.01996893064\nMinimizing 1190 Time: 0:00:17 (14.33 ms/it)\n objective: 24651.019319883733\nMinimizing 1198 Time: 0:00:17 (14.33 ms/it)\n objective: 24651.019552708614\nMinimizing 1206 Time: 0:00:17 (14.33 ms/it)\n objective: 24651.018277036655\nMinimizing 1214 Time: 0:00:17 (14.33 ms/it)\n objective: 24651.018391170022\nMinimizing 1222 Time: 0:00:17 (14.32 ms/it)\n objective: 24651.01799501553\nMinimizing 1230 Time: 0:00:17 (14.31 ms/it)\n objective: 24651.017306673806\nMinimizing 1238 Time: 0:00:17 (14.31 ms/it)\n objective: 24651.01716018239\nMinimizing 1246 Time: 0:00:17 (14.30 ms/it)\n objective: 24651.01700720565\nMinimizing 1254 Time: 0:00:17 (14.29 ms/it)\n objective: 24651.017156225666\nMinimizing 1262 Time: 0:00:18 (14.28 ms/it)\n objective: 24651.017078825065\nMinimizing 1270 Time: 0:00:18 (14.28 ms/it)\n objective: 24651.017081048733\nMinimizing 1278 Time: 0:00:18 (14.27 ms/it)\n objective: 24651.017027630012\nMinimizing 1286 Time: 0:00:18 (14.27 ms/it)\n objective: 24651.0165249109\nMinimizing 1294 Time: 0:00:18 (14.26 ms/it)\n objective: 24651.016514311843\nMinimizing 1302 Time: 0:00:18 (14.26 ms/it)\n objective: 24651.01643433857\nMinimizing 1310 Time: 0:00:18 (14.25 ms/it)\n objective: 24651.016395205435\nMinimizing 1318 Time: 0:00:18 (14.25 ms/it)\n objective: 24651.016339884165\nMinimizing 1326 Time: 0:00:18 (14.24 ms/it)\n objective: 24651.016310890016\nMinimizing 1334 Time: 0:00:18 (14.24 ms/it)\n objective: 24651.016295792695\nMinimizing 1342 Time: 0:00:19 (14.24 ms/it)\n objective: 24651.016276136255\nMinimizing 1350 Time: 0:00:19 (14.23 ms/it)\n objective: 24651.016203243234\nMinimizing 1358 Time: 0:00:19 (14.23 ms/it)\n objective: 24651.0161912915\nMinimizing 1366 Time: 0:00:19 (14.22 ms/it)\n objective: 24651.01610875644\nMinimizing 1374 Time: 0:00:19 (14.21 ms/it)\n objective: 24651.015989577405\nMinimizing 1382 Time: 0:00:19 (14.21 ms/it)\n objective: 24651.01595999897\nMinimizing 1390 Time: 0:00:19 (14.20 ms/it)\n objective: 24651.015947854627\nMinimizing 1398 Time: 0:00:19 (14.20 ms/it)\n objective: 24651.01595044735\nMinimizing 1406 Time: 0:00:19 (14.19 ms/it)\n objective: 24651.015917806668\nMinimizing 1414 Time: 0:00:20 (14.19 ms/it)\n objective: 24651.015892329753\nMinimizing 1422 Time: 0:00:20 (14.18 ms/it)\n objective: 24651.015874911966\nMinimizing 1430 Time: 0:00:20 (14.18 ms/it)\n objective: 24651.015796210548\nMinimizing 1438 Time: 0:00:20 (14.18 ms/it)\n objective: 24651.01577879441\nMinimizing 1446 Time: 0:00:20 (14.17 ms/it)\n objective: 24651.015705768066\nMinimizing 1454 Time: 0:00:20 (14.17 ms/it)\n objective: 24651.01572673725\nMinimizing 1462 Time: 0:00:20 (14.16 ms/it)\n objective: 24651.01561362497\nMinimizing 1470 Time: 0:00:20 (14.16 ms/it)\n objective: 24651.01560258209\nMinimizing 1478 Time: 0:00:20 (14.15 ms/it)\n objective: 24651.01557792974\nMinimizing 1486 Time: 0:00:21 (14.15 ms/it)\n objective: 24651.015571319556\nMinimizing 1494 Time: 0:00:21 (14.14 ms/it)\n objective: 24651.01549903939\nMinimizing 1502 Time: 0:00:21 (14.14 ms/it)\n objective: 24651.01547620207\nMinimizing 1510 Time: 0:00:21 (14.13 ms/it)\n objective: 24651.01541885895\nMinimizing 1518 Time: 0:00:21 (14.13 ms/it)\n objective: 24651.015409991058\nMinimizing 1539 Time: 0:00:21 (14.16 ms/it)\n objective: 24651.01534260885\nMinimizing 1549 Time: 0:00:21 (14.15 ms/it)\n objective: 24651.015331627394\nMinimizing 1557 Time: 0:00:22 (14.15 ms/it)\n objective: 24651.015299824558\nMinimizing 1565 Time: 0:00:22 (14.14 ms/it)\n objective: 24651.015234156173\nMinimizing 1573 Time: 0:00:22 (14.14 ms/it)\n objective: 24651.015191282593\nMinimizing 1581 Time: 0:00:22 (14.13 ms/it)\n objective: 24651.015172503638\nMinimizing 1589 Time: 0:00:22 (14.13 ms/it)\n objective: 24651.01514874262\nMinimizing 1597 Time: 0:00:22 (14.13 ms/it)\n objective: 24651.01511623719\nMinimizing 1605 Time: 0:00:22 (14.12 ms/it)\n objective: 24651.015053709496\nMinimizing 1613 Time: 0:00:22 (14.12 ms/it)\n objective: 24651.01502497716\nMinimizing 1621 Time: 0:00:22 (14.12 ms/it)\n objective: 24651.01497446181\nMinimizing 1629 Time: 0:00:22 (14.11 ms/it)\n objective: 24651.014941877227\nMinimizing 1637 Time: 0:00:23 (14.11 ms/it)\n objective: 24651.01494574121\nMinimizing 1645 Time: 0:00:23 (14.10 ms/it)\n objective: 24651.01487685147\nMinimizing 1653 Time: 0:00:23 (14.10 ms/it)\n objective: 24651.014834119243\nMinimizing 1661 Time: 0:00:23 (14.10 ms/it)\n objective: 24651.014801303565\nMinimizing 1669 Time: 0:00:23 (14.09 ms/it)\n objective: 24651.014780782607\nMinimizing 1677 Time: 0:00:23 (14.09 ms/it)\n objective: 24651.014723134558\nMinimizing 1684 Time: 0:00:23 (14.09 ms/it)\n objective: 24651.01470064149\nMinimizing 1692 Time: 0:00:23 (14.09 ms/it)\n objective: 24651.014673650465\nMinimizing 1700 Time: 0:00:23 (14.08 ms/it)\n objective: 24651.014635317995\nMinimizing 1708 Time: 0:00:24 (14.08 ms/it)\n objective: 24651.01459270252\nMinimizing 1716 Time: 0:00:24 (14.08 ms/it)\n objective: 24651.01455564439\nMinimizing 1724 Time: 0:00:24 (14.08 ms/it)\n objective: 24651.014519669363\nMinimizing 1732 Time: 0:00:24 (14.07 ms/it)\n objective: 24651.014505054405\nMinimizing 1740 Time: 0:00:24 (14.07 ms/it)\n objective: 24651.01449206109\nMinimizing 1748 Time: 0:00:24 (14.06 ms/it)\n objective: 24651.01445794005\nMinimizing 1756 Time: 0:00:24 (14.06 ms/it)\n objective: 24651.0144058898\nMinimizing 1764 Time: 0:00:24 (14.05 ms/it)\n objective: 24651.014378787884\nMinimizing 1772 Time: 0:00:24 (14.05 ms/it)\n objective: 24651.01434324346\nMinimizing 1780 Time: 0:00:24 (14.04 ms/it)\n objective: 24651.014334378262\nMinimizing 1788 Time: 0:00:25 (14.04 ms/it)\n objective: 24651.014273782956\nMinimizing 1796 Time: 0:00:25 (14.03 ms/it)\n objective: 24651.014245677798\nMinimizing 1804 Time: 0:00:25 (14.02 ms/it)\n objective: 24651.01423229063\nMinimizing 1812 Time: 0:00:25 (14.02 ms/it)\n objective: 24651.014222059363\nMinimizing 1820 Time: 0:00:25 (14.01 ms/it)\n objective: 24651.01418156778\nMinimizing 1828 Time: 0:00:25 (14.01 ms/it)\n objective: 24651.014162000316\nMinimizing 1836 Time: 0:00:25 (14.01 ms/it)\n objective: 24651.014132598506\nMinimizing 1844 Time: 0:00:25 (14.01 ms/it)\n objective: 24651.0141144742\nMinimizing 1852 Time: 0:00:25 (14.01 ms/it)\n objective: 24651.01409509868\nMinimizing 1860 Time: 0:00:26 (14.01 ms/it)\n objective: 24651.0140637523\nMinimizing 1868 Time: 0:00:26 (14.00 ms/it)\n objective: 24651.01405627704\nMinimizing 1876 Time: 0:00:26 (14.00 ms/it)\n objective: 24651.014037037705\nMinimizing 1877 Time: 0:00:26 (14.00 ms/it)\n\nMinimizing 1878 Time: 0:00:26 (14.00 ms/it)\n objective: 24651.014036789165\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\nσ_School\nσ_Cohort\n\n\n(Intercept)\n-0.0133\n0.0192\n-0.69\n0.4877\n0.5873\n0.2138\n0.0132\n\n\nTest: Star_r\n0.0088\n0.0375\n0.24\n0.8135\n0.7434\n0.3328\n0.0638\n\n\nTest: S20_r\n0.0092\n0.0293\n0.31\n0.7546\n0.6341\n0.3280\n0.0076\n\n\nTest: SLJ\n0.0034\n0.0300\n0.11\n0.9102\n0.5421\n0.3212\n0.0339\n\n\nTest: BMT\n-0.0383\n0.0296\n-1.29\n0.1954\n0.7213\n0.3130\n0.0000\n\n\na1\n0.1956\n0.0545\n3.59\n0.0003\n\n0.2825\n\n\n\nSex: Girls\n-0.4287\n0.0312\n-13.75\n<1e-42\n\n0.1435\n\n\n\nTest: Star_r & a1\n0.2898\n0.0887\n3.27\n0.0011\n\n\n\n\n\nTest: S20_r & a1\n-0.0784\n0.0817\n-0.96\n0.3370\n\n\n\n\n\nTest: SLJ & a1\n-0.0631\n0.0769\n-0.82\n0.4118\n\n\n\n\n\nTest: BMT & a1\n0.4713\n0.0852\n5.54\n<1e-07\n\n\n\n\n\nTest: Star_r & Sex: Girls\n0.3756\n0.0511\n7.35\n<1e-12\n\n\n\n\n\nTest: S20_r & Sex: Girls\n-0.1867\n0.0475\n-3.93\n<1e-04\n\n\n\n\n\nTest: SLJ & Sex: Girls\n-0.0695\n0.0445\n-1.56\n0.1184\n\n\n\n\n\nTest: BMT & Sex: Girls\n-0.2812\n0.0495\n-5.68\n<1e-07\n\n\n\n\n\na1 & Sex: Girls\n-0.1267\n0.1048\n-1.21\n0.2269\n\n\n\n\n\nTest: Star_r & a1 & Sex: Girls\n-0.1382\n0.1756\n-0.79\n0.4312\n\n\n\n\n\nTest: S20_r & a1 & Sex: Girls\n0.1753\n0.1633\n1.07\n0.2832\n\n\n\n\n\nTest: SLJ & a1 & Sex: Girls\n-0.0154\n0.1530\n-0.10\n0.9196\n\n\n\n\n\nTest: BMT & a1 & Sex: Girls\n0.1548\n0.1702\n0.91\n0.3630\n\n\n\n\n\nResidual\n0.5331\n\n\n\n\n\n\n\n\n\n\n\n\nVarCorr(m1)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\n\nChild\n(Intercept)\n0.34495271\n0.58732675\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.55269763\n0.74343636\n+0.14\n\n\n\n\n\n\n\n\nTest: S20_r\n0.40212511\n0.63413336\n+0.00\n-0.52\n\n\n\n\n\n\n\nTest: SLJ\n0.29385597\n0.54208484\n+0.05\n-0.04\n-0.35\n\n\n\n\n\n\nTest: BMT\n0.52032198\n0.72133347\n-0.35\n+0.14\n-0.18\n-0.22\n\n\n\n\nSchool\n(Intercept)\n0.04570702\n0.21379201\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.11076869\n0.33281931\n-0.06\n\n\n\n\n\n\n\n\nTest: S20_r\n0.10761018\n0.32803990\n-0.08\n-0.39\n\n\n\n\n\n\n\nTest: SLJ\n0.10318843\n0.32122956\n-0.19\n+0.21\n-0.81\n\n\n\n\n\n\nTest: BMT\n0.09795181\n0.31297254\n-0.33\n-0.02\n+0.13\n-0.38\n\n\n\n\n\na1\n0.07982565\n0.28253433\n+0.62\n-0.20\n+0.11\n-0.61\n+0.45\n\n\n\n\nSex: Girls\n0.02059180\n0.14349843\n-0.45\n+0.45\n+0.31\n-0.27\n-0.15\n-0.37\n\n\nCohort\n(Intercept)\n0.00017441\n0.01320652\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.00406549\n0.06376120\n.\n\n\n\n\n\n\n\n\nTest: S20_r\n0.00005702\n0.00755108\n.\n.\n\n\n\n\n\n\n\nTest: SLJ\n0.00114959\n0.03390556\n.\n.\n.\n\n\n\n\n\n\nTest: BMT\n0.00000000\n0.00000000\n.\n.\n.\n.\n\n\n\n\nResidual\n\n0.28422522\n0.53312777\n\n\n\n\n\n\n\n\n\n\n\n\nissingular(m1)\n\ntrue\n\n\nDepending on the random number for stratified samplign, LMM m1 may or may not be supported by the data.\nWe also fit an alternative parameterization, estimating VCs and CPs for Test scores rather than Test effects by replacing the 1 + ... in the RE terms with 0 + ....\n\nm2 = let\n f = @formula(\n zScore ~\n 1 +\n Test * a1 * Sex +\n (0 + Test + a1 + Sex | School) +\n (0 + Test | Child) +\n zerocorr(0 + Test | Cohort)\n )\n fit(MixedModel, f, dat; contrasts)\nend\n\nMinimizing 8 Time: 0:00:00 (13.13 ms/it)\n objective: 26540.628428689935\nMinimizing 16 Time: 0:00:00 (12.98 ms/it)\n objective: 27013.20266803618\nMinimizing 24 Time: 0:00:00 (13.30 ms/it)\n objective: 26938.094323301608\nMinimizing 32 Time: 0:00:00 (13.37 ms/it)\n objective: 26816.3549662712\nMinimizing 40 Time: 0:00:00 (13.36 ms/it)\n objective: 26764.50301863997\nMinimizing 48 Time: 0:00:00 (13.37 ms/it)\n objective: 26748.52532251481\nMinimizing 56 Time: 0:00:00 (13.35 ms/it)\n objective: 27607.74073471175\nMinimizing 64 Time: 0:00:00 (13.34 ms/it)\n objective: 27035.518325217672\nMinimizing 72 Time: 0:00:00 (13.26 ms/it)\n objective: 26743.04860536076\nMinimizing 80 Time: 0:00:01 (13.20 ms/it)\n objective: 26914.37946666574\nMinimizing 88 Time: 0:00:01 (13.15 ms/it)\n objective: 26750.53281469535\nMinimizing 96 Time: 0:00:01 (13.13 ms/it)\n objective: 26718.60071956578\nMinimizing 104 Time: 0:00:01 (13.14 ms/it)\n objective: 25121.980475037257\nMinimizing 112 Time: 0:00:01 (13.13 ms/it)\n objective: 25011.13301114371\nMinimizing 120 Time: 0:00:01 (13.13 ms/it)\n objective: 24997.24925764957\nMinimizing 128 Time: 0:00:01 (13.12 ms/it)\n objective: 24953.65373230762\nMinimizing 136 Time: 0:00:01 (13.11 ms/it)\n objective: 24958.120022161565\nMinimizing 144 Time: 0:00:01 (13.10 ms/it)\n objective: 24900.668392425818\nMinimizing 152 Time: 0:00:01 (13.10 ms/it)\n objective: 24893.96205392398\nMinimizing 160 Time: 0:00:02 (13.10 ms/it)\n objective: 24873.636974246172\nMinimizing 168 Time: 0:00:02 (13.23 ms/it)\n objective: 24859.799925935025\nMinimizing 174 Time: 0:00:02 (13.37 ms/it)\n objective: 24831.117936158902\nMinimizing 181 Time: 0:00:02 (13.49 ms/it)\n objective: 24808.0196415159\nMinimizing 188 Time: 0:00:02 (13.61 ms/it)\n objective: 24810.622141936405\nMinimizing 195 Time: 0:00:02 (13.71 ms/it)\n objective: 24764.74836706018\nMinimizing 201 Time: 0:00:02 (13.81 ms/it)\n objective: 24750.864655709753\nMinimizing 207 Time: 0:00:02 (13.90 ms/it)\n objective: 24728.7014244585\nMinimizing 213 Time: 0:00:02 (13.98 ms/it)\n objective: 24721.802789535843\nMinimizing 220 Time: 0:00:03 (14.05 ms/it)\n objective: 24724.978892060455\nMinimizing 228 Time: 0:00:03 (14.03 ms/it)\n objective: 24714.559421284797\nMinimizing 236 Time: 0:00:03 (13.99 ms/it)\n objective: 24713.749570295942\nMinimizing 244 Time: 0:00:03 (13.95 ms/it)\n objective: 24703.141786459488\nMinimizing 252 Time: 0:00:03 (13.93 ms/it)\n objective: 24706.525178828168\nMinimizing 260 Time: 0:00:03 (13.94 ms/it)\n objective: 24696.13455573986\nMinimizing 268 Time: 0:00:03 (13.92 ms/it)\n objective: 24691.657571354426\nMinimizing 277 Time: 0:00:03 (13.88 ms/it)\n objective: 24685.654228629654\nMinimizing 286 Time: 0:00:03 (13.83 ms/it)\n objective: 24682.51124245504\nMinimizing 295 Time: 0:00:04 (13.79 ms/it)\n objective: 24681.05913327908\nMinimizing 304 Time: 0:00:04 (13.75 ms/it)\n objective: 24676.468113148057\nMinimizing 312 Time: 0:00:04 (13.72 ms/it)\n objective: 24678.73775724656\nMinimizing 321 Time: 0:00:04 (13.68 ms/it)\n objective: 24673.49816406841\nMinimizing 330 Time: 0:00:04 (13.65 ms/it)\n objective: 24671.722762394973\nMinimizing 338 Time: 0:00:04 (13.66 ms/it)\n objective: 24666.188917121326\nMinimizing 345 Time: 0:00:04 (13.68 ms/it)\n objective: 24664.77883152221\nMinimizing 353 Time: 0:00:04 (13.66 ms/it)\n objective: 24659.4656714057\nMinimizing 361 Time: 0:00:04 (13.64 ms/it)\n objective: 24661.552652523183\nMinimizing 369 Time: 0:00:05 (13.62 ms/it)\n objective: 24657.54743509639\nMinimizing 377 Time: 0:00:05 (13.60 ms/it)\n objective: 24657.906277135386\nMinimizing 385 Time: 0:00:05 (13.58 ms/it)\n objective: 24657.22098196359\nMinimizing 393 Time: 0:00:05 (13.58 ms/it)\n objective: 24654.084092197903\nMinimizing 401 Time: 0:00:05 (13.56 ms/it)\n objective: 24654.660376842156\nMinimizing 410 Time: 0:00:05 (13.53 ms/it)\n objective: 24653.05933991388\nMinimizing 419 Time: 0:00:05 (13.51 ms/it)\n objective: 24652.690785417355\nMinimizing 428 Time: 0:00:05 (13.48 ms/it)\n objective: 24651.576869207296\nMinimizing 437 Time: 0:00:05 (13.45 ms/it)\n objective: 24651.345249844635\nMinimizing 446 Time: 0:00:05 (13.42 ms/it)\n objective: 24650.935321003628\nMinimizing 455 Time: 0:00:06 (13.39 ms/it)\n objective: 24650.460659691147\nMinimizing 464 Time: 0:00:06 (13.37 ms/it)\n objective: 24650.033417605235\nMinimizing 473 Time: 0:00:06 (13.35 ms/it)\n objective: 24649.881648448623\nMinimizing 482 Time: 0:00:06 (13.33 ms/it)\n objective: 24649.617387416143\nMinimizing 491 Time: 0:00:06 (13.31 ms/it)\n objective: 24649.54221767746\nMinimizing 500 Time: 0:00:06 (13.29 ms/it)\n objective: 24649.053690507884\nMinimizing 509 Time: 0:00:06 (13.27 ms/it)\n objective: 24648.905068218475\nMinimizing 518 Time: 0:00:06 (13.25 ms/it)\n objective: 24648.68854571072\nMinimizing 527 Time: 0:00:06 (13.23 ms/it)\n objective: 24648.58020652706\nMinimizing 536 Time: 0:00:07 (13.21 ms/it)\n objective: 24648.4299572324\nMinimizing 544 Time: 0:00:07 (13.20 ms/it)\n objective: 24648.359137537387\nMinimizing 553 Time: 0:00:07 (13.19 ms/it)\n objective: 24648.121508962126\nMinimizing 562 Time: 0:00:07 (13.17 ms/it)\n objective: 24648.099993862208\nMinimizing 570 Time: 0:00:07 (13.17 ms/it)\n objective: 24648.006992328505\nMinimizing 578 Time: 0:00:07 (13.17 ms/it)\n objective: 24647.966554206134\nMinimizing 586 Time: 0:00:07 (13.16 ms/it)\n objective: 24647.839547408417\nMinimizing 594 Time: 0:00:07 (13.16 ms/it)\n objective: 24647.748757753267\nMinimizing 602 Time: 0:00:07 (13.16 ms/it)\n objective: 24647.703056529623\nMinimizing 610 Time: 0:00:08 (13.16 ms/it)\n objective: 24647.65634691922\nMinimizing 618 Time: 0:00:08 (13.16 ms/it)\n objective: 24647.56439257673\nMinimizing 626 Time: 0:00:08 (13.17 ms/it)\n objective: 24647.50067749583\nMinimizing 634 Time: 0:00:08 (13.18 ms/it)\n objective: 24647.459579058945\nMinimizing 642 Time: 0:00:08 (13.18 ms/it)\n objective: 24647.47873341732\nMinimizing 651 Time: 0:00:08 (13.16 ms/it)\n objective: 24647.343086450237\nMinimizing 660 Time: 0:00:08 (13.15 ms/it)\n objective: 24647.34116898825\nMinimizing 669 Time: 0:00:08 (13.14 ms/it)\n objective: 24647.348823056633\nMinimizing 678 Time: 0:00:08 (13.14 ms/it)\n objective: 24647.27411314765\nMinimizing 686 Time: 0:00:09 (13.13 ms/it)\n objective: 24647.250608569215\nMinimizing 695 Time: 0:00:09 (13.12 ms/it)\n objective: 24647.22520161305\nMinimizing 704 Time: 0:00:09 (13.11 ms/it)\n objective: 24647.23198462193\nMinimizing 713 Time: 0:00:09 (13.10 ms/it)\n objective: 24647.200084946257\nMinimizing 722 Time: 0:00:09 (13.08 ms/it)\n objective: 24647.209350541543\nMinimizing 731 Time: 0:00:09 (13.07 ms/it)\n objective: 24647.150885420226\nMinimizing 740 Time: 0:00:09 (13.06 ms/it)\n objective: 24647.129446947456\nMinimizing 749 Time: 0:00:09 (13.06 ms/it)\n objective: 24647.125530564546\nMinimizing 758 Time: 0:00:09 (13.05 ms/it)\n objective: 24647.13683106716\nMinimizing 766 Time: 0:00:09 (13.05 ms/it)\n objective: 24647.091680123256\nMinimizing 775 Time: 0:00:10 (13.04 ms/it)\n objective: 24647.098428585\nMinimizing 783 Time: 0:00:10 (13.04 ms/it)\n objective: 24647.094389500824\nMinimizing 791 Time: 0:00:10 (13.04 ms/it)\n objective: 24647.05192467993\nMinimizing 799 Time: 0:00:10 (13.04 ms/it)\n objective: 24647.069433509034\nMinimizing 807 Time: 0:00:10 (13.04 ms/it)\n objective: 24647.038241950948\nMinimizing 815 Time: 0:00:10 (13.05 ms/it)\n objective: 24647.03569193548\nMinimizing 823 Time: 0:00:10 (13.05 ms/it)\n objective: 24647.02367144438\nMinimizing 831 Time: 0:00:10 (13.05 ms/it)\n objective: 24647.014044510084\nMinimizing 839 Time: 0:00:10 (13.05 ms/it)\n objective: 24647.00941961279\nMinimizing 847 Time: 0:00:11 (13.05 ms/it)\n objective: 24646.98532430834\nMinimizing 856 Time: 0:00:11 (13.04 ms/it)\n objective: 24646.965197407808\nMinimizing 864 Time: 0:00:11 (13.04 ms/it)\n objective: 24646.97000154094\nMinimizing 872 Time: 0:00:11 (13.05 ms/it)\n objective: 24646.98178371837\nMinimizing 880 Time: 0:00:11 (13.05 ms/it)\n objective: 24646.983776073634\nMinimizing 889 Time: 0:00:11 (13.04 ms/it)\n objective: 24646.9319241572\nMinimizing 897 Time: 0:00:11 (13.04 ms/it)\n objective: 24646.931978735825\nMinimizing 906 Time: 0:00:11 (13.03 ms/it)\n objective: 24646.928020882224\nMinimizing 915 Time: 0:00:11 (13.03 ms/it)\n objective: 24646.946012804347\nMinimizing 923 Time: 0:00:12 (13.03 ms/it)\n objective: 24646.92061262158\nMinimizing 931 Time: 0:00:12 (13.03 ms/it)\n objective: 24646.919364692974\nMinimizing 940 Time: 0:00:12 (13.04 ms/it)\n objective: 24646.952962425272\nMinimizing 948 Time: 0:00:12 (13.04 ms/it)\n objective: 24646.910649209145\nMinimizing 956 Time: 0:00:12 (13.04 ms/it)\n objective: 24646.908075010928\nMinimizing 964 Time: 0:00:12 (13.04 ms/it)\n objective: 24646.9254264908\nMinimizing 973 Time: 0:00:12 (13.03 ms/it)\n objective: 24646.93194511519\nMinimizing 982 Time: 0:00:12 (13.02 ms/it)\n objective: 24646.895063711723\nMinimizing 991 Time: 0:00:12 (13.01 ms/it)\n objective: 24646.922305142616\nMinimizing 1000 Time: 0:00:13 (13.00 ms/it)\n objective: 24646.891929413974\nMinimizing 1009 Time: 0:00:13 (13.00 ms/it)\n objective: 24646.891099685534\nMinimizing 1018 Time: 0:00:13 (12.99 ms/it)\n objective: 24646.889709339343\nMinimizing 1027 Time: 0:00:13 (12.98 ms/it)\n objective: 24646.88810835629\nMinimizing 1036 Time: 0:00:13 (12.97 ms/it)\n objective: 24646.887297493624\nMinimizing 1045 Time: 0:00:13 (12.96 ms/it)\n objective: 24646.885784030797\nMinimizing 1054 Time: 0:00:13 (12.96 ms/it)\n objective: 24646.885450896836\nMinimizing 1063 Time: 0:00:13 (12.95 ms/it)\n objective: 24646.88351475452\nMinimizing 1072 Time: 0:00:13 (12.94 ms/it)\n objective: 24646.88199866625\nMinimizing 1081 Time: 0:00:13 (12.93 ms/it)\n objective: 24646.88095125399\nMinimizing 1090 Time: 0:00:14 (12.92 ms/it)\n objective: 24646.878879374555\nMinimizing 1099 Time: 0:00:14 (12.92 ms/it)\n objective: 24646.877898696042\nMinimizing 1108 Time: 0:00:14 (12.91 ms/it)\n objective: 24646.87745016371\nMinimizing 1117 Time: 0:00:14 (12.90 ms/it)\n objective: 24646.877039090396\nMinimizing 1126 Time: 0:00:14 (12.90 ms/it)\n objective: 24646.87678163265\nMinimizing 1134 Time: 0:00:14 (12.90 ms/it)\n objective: 24646.87620262606\nMinimizing 1142 Time: 0:00:14 (12.90 ms/it)\n objective: 24646.875795879896\nMinimizing 1150 Time: 0:00:14 (12.90 ms/it)\n objective: 24646.87474805087\nMinimizing 1158 Time: 0:00:14 (12.91 ms/it)\n objective: 24646.874187797846\nMinimizing 1166 Time: 0:00:15 (12.91 ms/it)\n objective: 24646.874226968095\nMinimizing 1174 Time: 0:00:15 (12.91 ms/it)\n objective: 24646.873636962468\nMinimizing 1182 Time: 0:00:15 (12.91 ms/it)\n objective: 24646.873134800753\nMinimizing 1190 Time: 0:00:15 (12.91 ms/it)\n objective: 24646.873438121496\nMinimizing 1199 Time: 0:00:15 (12.91 ms/it)\n objective: 24646.872948485317\nMinimizing 1207 Time: 0:00:15 (12.91 ms/it)\n objective: 24646.8732539235\nMinimizing 1215 Time: 0:00:15 (12.91 ms/it)\n objective: 24646.87300746512\nMinimizing 1223 Time: 0:00:15 (12.91 ms/it)\n objective: 24646.872786788605\nMinimizing 1231 Time: 0:00:15 (12.91 ms/it)\n objective: 24646.872757429453\nMinimizing 1239 Time: 0:00:15 (12.91 ms/it)\n objective: 24646.872603076466\nMinimizing 1247 Time: 0:00:16 (12.91 ms/it)\n objective: 24646.87241628748\nMinimizing 1255 Time: 0:00:16 (12.91 ms/it)\n objective: 24646.872497083332\nMinimizing 1264 Time: 0:00:16 (12.91 ms/it)\n objective: 24646.872561572494\nMinimizing 1273 Time: 0:00:16 (12.91 ms/it)\n objective: 24646.87224029468\nMinimizing 1281 Time: 0:00:16 (12.91 ms/it)\n objective: 24646.87251841221\nMinimizing 1289 Time: 0:00:16 (12.91 ms/it)\n objective: 24646.872198901816\nMinimizing 1297 Time: 0:00:16 (12.91 ms/it)\n objective: 24646.872169110415\nMinimizing 1306 Time: 0:00:16 (12.91 ms/it)\n objective: 24646.872090365337\nMinimizing 1314 Time: 0:00:16 (12.91 ms/it)\n objective: 24646.872070330082\nMinimizing 1322 Time: 0:00:17 (12.92 ms/it)\n objective: 24646.872060506095\nMinimizing 1330 Time: 0:00:17 (12.92 ms/it)\n objective: 24646.87204513132\nMinimizing 1336 Time: 0:00:17 (12.95 ms/it)\n objective: 24646.87204792878\nMinimizing 1343 Time: 0:00:17 (12.95 ms/it)\n objective: 24646.872012693733\nMinimizing 1350 Time: 0:00:17 (12.97 ms/it)\n objective: 24646.871999364565\nMinimizing 1356 Time: 0:00:17 (12.99 ms/it)\n objective: 24646.87199265028\nMinimizing 1363 Time: 0:00:17 (13.00 ms/it)\n objective: 24646.871988545125\nMinimizing 1370 Time: 0:00:17 (13.02 ms/it)\n objective: 24646.871980256794\nMinimizing 1376 Time: 0:00:17 (13.04 ms/it)\n objective: 24646.871978006355\nMinimizing 1384 Time: 0:00:18 (13.04 ms/it)\n objective: 24646.871966648752\nMinimizing 1391 Time: 0:00:18 (13.06 ms/it)\n objective: 24646.87196321425\nMinimizing 1398 Time: 0:00:18 (13.07 ms/it)\n objective: 24646.87195640274\nMinimizing 1406 Time: 0:00:18 (13.07 ms/it)\n objective: 24646.87195198112\nMinimizing 1414 Time: 0:00:18 (13.08 ms/it)\n objective: 24646.871947813554\nMinimizing 1422 Time: 0:00:18 (13.08 ms/it)\n objective: 24646.871944653773\nMinimizing 1429 Time: 0:00:18 (13.08 ms/it)\n objective: 24646.8719410536\nMinimizing 1437 Time: 0:00:18 (13.09 ms/it)\n objective: 24646.871936481075\nMinimizing 1444 Time: 0:00:18 (13.10 ms/it)\n objective: 24646.871933621453\nMinimizing 1451 Time: 0:00:19 (13.10 ms/it)\n objective: 24646.871927196844\nMinimizing 1459 Time: 0:00:19 (13.11 ms/it)\n objective: 24646.871925448948\nMinimizing 1467 Time: 0:00:19 (13.12 ms/it)\n objective: 24646.871922590137\nMinimizing 1475 Time: 0:00:19 (13.12 ms/it)\n objective: 24646.871922920815\nMinimizing 1483 Time: 0:00:19 (13.12 ms/it)\n objective: 24646.871921975307\nMinimizing 1491 Time: 0:00:19 (13.13 ms/it)\n objective: 24646.871918160934\nMinimizing 1499 Time: 0:00:19 (13.13 ms/it)\n objective: 24646.87191685501\nMinimizing 1507 Time: 0:00:19 (13.14 ms/it)\n objective: 24646.87191685694\nMinimizing 1515 Time: 0:00:19 (13.14 ms/it)\n objective: 24646.871916002106\nMinimizing 1523 Time: 0:00:20 (13.14 ms/it)\n objective: 24646.871913093604\nMinimizing 1530 Time: 0:00:20 (13.15 ms/it)\n\nMinimizing 1531 Time: 0:00:20 (13.15 ms/it)\n objective: 24646.871911983937\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\nσ_School\nσ_Cohort\n\n\n(Intercept)\n-0.0136\n0.0195\n-0.70\n0.4844\n\n\n\n\n\nTest: Star_r\n0.0093\n0.0365\n0.26\n0.7986\n0.7969\n0.3103\n0.0208\n\n\nTest: S20_r\n0.0060\n0.0354\n0.17\n0.8653\n0.7675\n0.3305\n0.0563\n\n\nTest: SLJ\n0.0052\n0.0339\n0.15\n0.8787\n0.7803\n0.2478\n0.0156\n\n\nTest: BMT\n-0.0388\n0.0300\n-1.29\n0.1967\n0.6967\n0.1969\n0.0000\n\n\na1\n0.1935\n0.0545\n3.55\n0.0004\n\n0.2833\n\n\n\nSex: Girls\n-0.4290\n0.0312\n-13.75\n<1e-42\n\n0.1434\n\n\n\nTest: Star_r & a1\n0.2932\n0.0887\n3.31\n0.0009\n\n\n\n\n\nTest: S20_r & a1\n-0.1007\n0.0826\n-1.22\n0.2228\n\n\n\n\n\nTest: SLJ & a1\n-0.0456\n0.0773\n-0.59\n0.5549\n\n\n\n\n\nTest: BMT & a1\n0.4685\n0.0853\n5.50\n<1e-07\n\n\n\n\n\nTest: Star_r & Sex: Girls\n0.3759\n0.0511\n7.36\n<1e-12\n\n\n\n\n\nTest: S20_r & Sex: Girls\n-0.1892\n0.0475\n-3.98\n<1e-04\n\n\n\n\n\nTest: SLJ & Sex: Girls\n-0.0680\n0.0445\n-1.53\n0.1262\n\n\n\n\n\nTest: BMT & Sex: Girls\n-0.2815\n0.0495\n-5.68\n<1e-07\n\n\n\n\n\na1 & Sex: Girls\n-0.1262\n0.1049\n-1.20\n0.2290\n\n\n\n\n\nTest: Star_r & a1 & Sex: Girls\n-0.1371\n0.1756\n-0.78\n0.4348\n\n\n\n\n\nTest: S20_r & a1 & Sex: Girls\n0.1720\n0.1634\n1.05\n0.2926\n\n\n\n\n\nTest: SLJ & a1 & Sex: Girls\n-0.0125\n0.1529\n-0.08\n0.9348\n\n\n\n\n\nTest: BMT & a1 & Sex: Girls\n0.1542\n0.1702\n0.91\n0.3648\n\n\n\n\n\nTest: Run\n\n\n\n\n0.7498\n0.3702\n0.0548\n\n\nResidual\n0.5122\n\n\n\n\n\n\n\n\n\n\n\n\nissingular(m2)\n\ntrue\n\n\nDepending on the random number generator seed, the model may or may not be supported in the alternative parameterization of scores. The fixed-effects profile is not affected (see 2.8 below).\n\n\n\n\n\n\nCaution\n\n\n\nRK: The order of RE terms is critical. In formula f2 the zerocorr() term must be placed last as shown. If it is placed first, School-related and Child-related CPs are estimated/reported (?) as zero. This was not the case for formula m1. Thus, it appears to be related to the 0-intercepts in School and Child terms. Need a reprex.\n\n\n\nVarCorr(m2)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\n\nChild\nTest: Run\n0.5622290\n0.7498193\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.6350899\n0.7969253\n+0.50\n\n\n\n\n\n\n\n\nTest: S20_r\n0.5890796\n0.7675152\n+0.56\n+0.64\n\n\n\n\n\n\n\nTest: SLJ\n0.6089346\n0.7803426\n+0.55\n+0.60\n+0.72\n\n\n\n\n\n\nTest: BMT\n0.4854398\n0.6967351\n+0.19\n+0.40\n+0.37\n+0.49\n\n\n\n\nSchool\nTest: Run\n0.1370455\n0.3701965\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.0962783\n0.3102874\n+0.53\n\n\n\n\n\n\n\n\nTest: S20_r\n0.1092383\n0.3305122\n+0.47\n+0.48\n\n\n\n\n\n\n\nTest: SLJ\n0.0613838\n0.2477575\n+0.47\n+0.76\n+0.41\n\n\n\n\n\n\nTest: BMT\n0.0387541\n0.1968606\n+0.15\n+0.38\n+0.18\n+0.02\n\n\n\n\n\na1\n0.0802336\n0.2832554\n+0.58\n+0.47\n+0.56\n-0.05\n+0.65\n\n\n\n\nSex: Girls\n0.0205546\n0.1433686\n-0.63\n-0.27\n+0.06\n-0.28\n-0.58\n-0.37\n\n\nCohort\nTest: Run\n0.0030066\n0.0548320\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.0004344\n0.0208416\n.\n\n\n\n\n\n\n\n\nTest: S20_r\n0.0031716\n0.0563170\n.\n.\n\n\n\n\n\n\n\nTest: SLJ\n0.0002448\n0.0156459\n.\n.\n.\n\n\n\n\n\n\nTest: BMT\n0.0000000\n0.0000000\n.\n.\n.\n.\n\n\n\n\nResidual\n\n0.2623127\n0.5121647" + }, + { + "objectID": "fggk21.html#principle-component-analysis-of-random-effect-structure-repca", + "href": "fggk21.html#principle-component-analysis-of-random-effect-structure-repca", + "title": "Basics with Emotikon Project", + "section": "4.7 Principle Component Analysis of Random Effect Structure (rePCA)", + "text": "4.7 Principle Component Analysis of Random Effect Structure (rePCA)\nThe ìssingular() command is sort of a shortcut for a quick inspection of the principle components (PCs) of the variance-covariance matrix of the RES. With the MixedModels.PCA() command, we also obtain information about the amount of cumulative variance accounted for as we add PCs.\nThe output also provides PC loadings which may facilitate interpretation of the CP matrices (if estimated). This topic will be picked uo in a separate vignette. See also Fühner et al. (2021) for an application." + }, + { + "objectID": "fggk21.html#effects-in-res", + "href": "fggk21.html#effects-in-res", + "title": "Basics with Emotikon Project", + "section": "4.8 Effects in RES", + "text": "4.8 Effects in RES\nFor every random factor, MixedModels.PCA() extracts as many PCs as there are VCs. Therefore, the cumulation of variance across PCs within a random factor will always add up to 100% – at the latest with the last VC, but, in the case of overparameterized LMMs, the ceiling will be reached earlier. The final PCs are usually quite small.\nPCs are extracted in the order of the amount of unique variance they account for. The first PC accounts for the largest and the final PC for the least amount of variance. The number the PCs with percent variance above a certain threshold indicates the number of weighted composites needed and reflects the dimensionality of the orthogonal space within which (almost) all the variance can be accounted for. The weights for forming composite scores are the listed loadings. For ease of interpretation it is often useful to change the sign of some composite scores.\nThe PCA for LMM m1 shows that each of the five PCs for Child accounts for a non-zero percent of unique variance.\nFor School fewer than seven PCs have unique variance. The exact number depends on sampling. The overparameterization of School might be resolved when the CPs for Sex are dropped from the LMM.\nCohort was estimated with CPs forced to zero. Therefore, the VCs were forced to be orthogonal; they already represent the PCA solution. However, depending on sampling, not all PCs may be identified for this random factor either.\nImportantly, again depending on sampling, a non-singular fit does not imply that unique variance is associated with all PCs (i.e., not for last PC for School). Embrace uncertainty!\n\nMixedModels.PCA(m1)\n\n(Child = \nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . .\n Test: Star_r 0.14 1.0 . . .\n Test: S20_r 0.0 -0.52 1.0 . .\n Test: SLJ 0.05 -0.04 -0.35 1.0 .\n Test: BMT -0.35 0.14 -0.18 -0.22 1.0\n\nNormalized cumulative variances:\n[0.3288, 0.616, 0.8285, 0.9371, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5\n (Intercept) -0.04 0.58 0.55 -0.6 -0.03\n Test: Star_r -0.61 -0.04 0.45 0.4 0.51\n Test: S20_r 0.71 -0.02 0.15 0.03 0.69\n Test: SLJ -0.28 0.44 -0.69 -0.21 0.46\n Test: BMT -0.21 -0.69 -0.01 -0.66 0.22, School = \nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . . . .\n Test: Star_r -0.06 1.0 . . . . .\n Test: S20_r -0.08 -0.39 1.0 . . . .\n Test: SLJ -0.19 0.21 -0.81 1.0 . . .\n Test: BMT -0.33 -0.02 0.13 -0.38 1.0 . .\n a1 0.62 -0.2 0.11 -0.61 0.45 1.0 .\n Sex: Girls -0.45 0.45 0.31 -0.27 -0.15 -0.37 1.0\n\nNormalized cumulative variances:\n[0.3567, 0.6348, 0.8056, 0.9684, 0.9999, 1.0, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5 PC6 PC7\n (Intercept) -0.25 -0.49 -0.41 -0.37 0.19 0.52 0.27\n Test: Star_r 0.31 0.12 0.18 -0.75 0.47 -0.25 -0.09\n Test: S20_r -0.4 0.44 -0.32 0.2 0.52 -0.28 0.4\n Test: SLJ 0.55 -0.31 0.1 0.2 0.02 -0.19 0.72\n Test: BMT -0.29 0.14 0.79 0.01 0.16 0.42 0.27\n a1 -0.52 -0.27 0.16 -0.31 -0.42 -0.56 0.21\n Sex: Girls 0.14 0.6 -0.19 -0.34 -0.53 0.24 0.35, Cohort = \nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . .\n Test: Star_r 0.0 1.0 . . .\n Test: S20_r 0.0 0.0 1.0 . .\n Test: SLJ 0.0 0.0 0.0 1.0 .\n Test: BMT 0.0 0.0 0.0 0.0 0.0\n\nNormalized cumulative variances:\n[0.25, 0.5, 0.75, 1.0, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5\n (Intercept) 1.0 0.0 0.0 0.0 0.0\n Test: Star_r 0.0 1.0 0.0 0.0 0.0\n Test: S20_r 0.0 0.0 1.0 0.0 0.0\n Test: SLJ 0.0 0.0 0.0 1.0 0.0\n Test: BMT 0.0 0.0 0.0 0.0 NaN)\n\n\n\n4.8.1 Scores in RES\nNow lets looks at the PCA results for the alternative parameterization of LMM m2. It is important to note that the reparameterization to base estimates of VCs and CPs on scores rather than effects applies only to the Test factor (i.e., the first factor in the formula); VCs for Sex and age refer to the associated effects.\nDepending on sampling, the difference between LMM m1 and LMM m2 may show that overparameterization according to PCs may depend on the specification chosen for the other the random-effect structure.\n\n\n\n\n\n\nNote\n\n\n\nFor the complete data, all PCs had unique variance associated with them.\n\n\n\nMixedModels.PCA(m2)\n\n(Child = \nPrincipal components based on correlation matrix\n Test: Run 1.0 . . . .\n Test: Star_r 0.5 1.0 . . .\n Test: S20_r 0.56 0.64 1.0 . .\n Test: SLJ 0.55 0.6 0.72 1.0 .\n Test: BMT 0.19 0.4 0.37 0.49 1.0\n\nNormalized cumulative variances:\n[0.6117, 0.7769, 0.8677, 0.9485, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5\n Test: Run -0.42 0.53 0.63 0.37 0.08\n Test: Star_r -0.47 0.03 -0.65 0.58 -0.16\n Test: S20_r -0.49 0.15 -0.24 -0.49 0.66\n Test: SLJ -0.5 -0.05 0.09 -0.5 -0.7\n Test: BMT -0.34 -0.83 0.33 0.2 0.2, School = \nPrincipal components based on correlation matrix\n Test: Run 1.0 . . . . . .\n Test: Star_r 0.53 1.0 . . . . .\n Test: S20_r 0.47 0.48 1.0 . . . .\n Test: SLJ 0.47 0.76 0.41 1.0 . . .\n Test: BMT 0.15 0.38 0.18 0.02 1.0 . .\n a1 0.58 0.47 0.56 -0.05 0.65 1.0 .\n Sex: Girls -0.63 -0.27 0.06 -0.28 -0.58 -0.37 1.0\n\nNormalized cumulative variances:\n[0.4822, 0.6938, 0.8508, 0.9553, 1.0, 1.0, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5 PC6 PC7\n Test: Run -0.44 0.08 0.15 -0.64 -0.16 0.41 0.42\n Test: Star_r -0.44 0.28 0.02 0.41 -0.56 0.29 -0.4\n Test: S20_r -0.34 0.27 -0.58 -0.1 0.6 0.15 -0.28\n Test: SLJ -0.32 0.56 0.33 0.25 0.22 -0.48 0.36\n Test: BMT -0.32 -0.53 -0.02 0.55 0.25 0.3 0.41\n a1 -0.41 -0.35 -0.42 -0.14 -0.33 -0.63 0.05\n Sex: Girls 0.34 0.35 -0.59 0.18 -0.29 0.14 0.53, Cohort = \nPrincipal components based on correlation matrix\n Test: Run 1.0 . . . .\n Test: Star_r 0.0 1.0 . . .\n Test: S20_r 0.0 0.0 1.0 . .\n Test: SLJ 0.0 0.0 0.0 1.0 .\n Test: BMT 0.0 0.0 0.0 0.0 0.0\n\nNormalized cumulative variances:\n[0.25, 0.5, 0.75, 1.0, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5\n Test: Run 0.0 0.0 1.0 0.0 0.0\n Test: Star_r 1.0 0.0 0.0 0.0 0.0\n Test: S20_r 0.0 1.0 0.0 0.0 0.0\n Test: SLJ 0.0 0.0 0.0 1.0 0.0\n Test: BMT 0.0 0.0 0.0 0.0 NaN)" + }, + { + "objectID": "fggk21.html#summary-of-results-for-stratified-subset-of-data", + "href": "fggk21.html#summary-of-results-for-stratified-subset-of-data", + "title": "Basics with Emotikon Project", + "section": "4.9 Summary of results for stratified subset of data", + "text": "4.9 Summary of results for stratified subset of data\nReturning to the theoretical focus of the article, the significant main effects of age and Sex, the interactions between age and c1 and c4 contrasts and the interactions between Sex and three test contrasts (c1, c2, c4) are replicated. Obviously, the subset of data is much noisier than the full set." + }, + { + "objectID": "fggk21.html#overall-summary-statistics", + "href": "fggk21.html#overall-summary-statistics", + "title": "Basics with Emotikon Project", + "section": "6.1 Overall summary statistics", + "text": "6.1 Overall summary statistics\n+ julia> m1.optsum # MixedModels.OptSummary: gets all info\n+ julia> loglikelihood(m1) # StatsBase.loglikelihood: return loglikelihood\n of the model\n+ julia> deviance(m1) # StatsBase.deviance: negative twice the log-likelihood\n relative to saturated model\n+ julia> objective(m1) # MixedModels.objective: saturated model not clear:\n negative twice the log-likelihood\n+ julia> nobs(m1) # n of observations; they are not independent\n+ julia> dof(m1) # n of degrees of freedom is number of model parameters\n+ julia> aic(m1) # objective(m1) + 2*dof(m1)\n+ julia> bic(m1) # objective(m1) + dof(m1)*log(nobs(m1))\n\nm1.optsum # MixedModels.OptSummary: gets all info\n\n\n\n\n\n\n\n\nInitialization\n\n\n\nInitial parameter vector\n[1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]\n\n\nInitial objective value\n25840.653755409803\n\n\nOptimizer settings\n\n\n\nOptimizer (from NLopt)\nLN_BOBYQA\n\n\nLower bounds\n[0.0, -Inf, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, 0.0, -Inf, 0.0, 0.0, -Inf, -Inf, -Inf, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, -Inf, 0.0, -Inf, -Inf, 0.0, -Inf, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]\n\n\nftol_rel\n1.0e-12\n\n\nftol_abs\n1.0e-8\n\n\nxtol_rel\n0.0\n\n\nxtol_abs\n[1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10, 1.0e-10]\n\n\ninitial_step\n[0.75, 1.0, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 0.75, 1.0, 0.75, 0.75, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 1.0, 0.75, 1.0, 1.0, 0.75, 1.0, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75]\n\n\nmaxfeval\n-1\n\n\nmaxtime\n-1.0\n\n\nResult\n\n\n\nFunction evaluations\n1877\n\n\nFinal parameter vector\n[1.1017, 0.1998, 0.0008, 0.0492, -0.4673, 1.3801, -0.627, -0.0493, 0.2525, 1.0108, -0.4543, -0.1363, 0.907, -0.3692, 1.1806, 0.401, -0.0372, -0.0463, -0.1124, -0.1955, 0.3284, -0.1207, 0.6232, -0.2454, 0.121, -0.0246, -0.0876, 0.1131, 0.5624, -0.4873, 0.0578, 0.051, 0.1313, 0.3136, -0.404, -0.3872, -0.0241, 0.3732, 0.1119, -0.165, 0.0177, -0.0054, 0.0, 0.0248, 0.1196, 0.0142, 0.0636, 0.0]\n\n\nFinal objective value\n24651.014\n\n\nReturn code\nFTOL_REACHED\n\n\n\n\n\n\nloglikelihood(m1) # StatsBase.loglikelihood: return loglikelihood of the model\n\n-12325.507018394583\n\n\n\ndeviance(m1) # StatsBase.deviance: negative twice the log-likelihood relative to saturated mode`\n\n24651.014036789165\n\n\n\nobjective(m1) # MixedModels.objective: saturated model not clear: negative twice the log-likelihood\n\n24651.014036789165\n\n\n\nnobs(m1) # n of observations; they are not independent\n\n9663\n\n\n\nn_, p_, q_, k_ = size(m1)\n\n(9663, 20, 13076, 3)\n\n\n\ndof(m1) # n of degrees of freedom is number of model parameters\n\n69\n\n\n\ngeom_df = sum(leverage(m1)) # trace of hat / rank of model matrix / geom dof\n\n5442.453779672519\n\n\n\nresid_df = nobs(m1) - geom_df # eff. residual degrees of freedom\n\n4220.546220327481\n\n\n\naic(m1) # objective(m1) + 2*dof(m1)\n\n24789.014036789165\n\n\n\nbic(m1) # objective(m1) + dof(m1)*log(nobs(m1))\n\n25284.162138011117" + }, + { + "objectID": "fggk21.html#fixed-effect-statistics", + "href": "fggk21.html#fixed-effect-statistics", + "title": "Basics with Emotikon Project", + "section": "6.2 Fixed-effect statistics", + "text": "6.2 Fixed-effect statistics\n+ julia> coeftable(m1) # StatsBase.coeftable: fixed-effects statiscs;\n default level=0.95\n+ julia> Arrow.write(\"./data/m_cpx_fe.arrow\", DataFrame(coeftable(m1)));\n+ julia> coef(m1) # StatsBase.coef - parts of the table\n+ julia> fixef(m1) # MixedModels.fixef: not the same as coef()\n for rank-deficient case\n+ julia> m1.beta # alternative extractor\n+ julia> fixefnames(m1) # works also for coefnames(m1)\n+ julia> vcov(m1) # StatsBase.vcov: var-cov matrix of fixed-effects coef.\n+ julia> stderror(m1) # StatsBase.stderror: SE for fixed-effects coefficients\n+ julia> propertynames(m1) # names of available extractors\n\ncoeftable(m1) # StatsBase.coeftable: fixed-effects statiscs; default level=0.95\n\n\n\n\n\n\n\n\n\n\n\n\nCoef.\nStd. Error\nz\nPr(>\n\n\n\n\n(Intercept)\n-0.0133231\n0.0191965\n-0.69\n0.4877\n\n\nTest: Star_r\n0.00883861\n0.0374707\n0.24\n0.8135\n\n\nTest: S20_r\n0.0091502\n0.0292733\n0.31\n0.7546\n\n\nTest: SLJ\n0.00337867\n0.0299512\n0.11\n0.9102\n\n\nTest: BMT\n-0.0382941\n0.0295742\n-1.29\n0.1954\n\n\na1\n0.195588\n0.0544537\n3.59\n0.0003\n\n\nSex: Girls\n-0.428743\n0.0311829\n-13.75\n<1e-42\n\n\nTest: Star_r & a1\n0.289778\n0.0887471\n3.27\n0.0011\n\n\nTest: S20_r & a1\n-0.0784486\n0.0817091\n-0.96\n0.3370\n\n\nTest: SLJ & a1\n-0.0630966\n0.0768863\n-0.82\n0.4118\n\n\nTest: BMT & a1\n0.471313\n0.0851504\n5.54\n<1e-07\n\n\nTest: Star_r & Sex: Girls\n0.375595\n0.0510831\n7.35\n<1e-12\n\n\nTest: S20_r & Sex: Girls\n-0.186726\n0.0475081\n-3.93\n<1e-04\n\n\nTest: SLJ & Sex: Girls\n-0.0695077\n0.0445161\n-1.56\n0.1184\n\n\nTest: BMT & Sex: Girls\n-0.281237\n0.0495209\n-5.68\n<1e-07\n\n\na1 & Sex: Girls\n-0.126658\n0.104828\n-1.21\n0.2269\n\n\nTest: Star_r & a1 & Sex: Girls\n-0.138202\n0.17558\n-0.79\n0.4312\n\n\nTest: S20_r & a1 & Sex: Girls\n0.175287\n0.163335\n1.07\n0.2832\n\n\nTest: SLJ & a1 & Sex: Girls\n-0.015449\n0.152981\n-0.10\n0.9196\n\n\nTest: BMT & a1 & Sex: Girls\n0.154782\n0.170158\n0.91\n0.3630\n\n\n\n\n\n\n#Arrow.write(\"./data/m_cpx_fe.arrow\", DataFrame(coeftable(m1)));\n\n\ncoef(m1) # StatsBase.coef; parts of the table\n\n20-element Vector{Float64}:\n -0.013323098599731003\n 0.008838610043037353\n 0.009150200653972363\n 0.0033786745702838548\n -0.038294149701005455\n 0.19558778043000458\n -0.4287431844207055\n 0.2897784524160024\n -0.07844856274209384\n -0.06309657343028045\n 0.4713127757906535\n 0.37559520558051235\n -0.18672567514518862\n -0.06950765283326864\n -0.28123708084784266\n -0.12665848284693404\n -0.1382024418769914\n 0.17528665552673206\n -0.015448951451204404\n 0.15478177087438777\n\n\n\nfixef(m1) # MixedModels.fixef: not the same as coef() for rank-deficient case\n\n20-element Vector{Float64}:\n -0.013323098599731003\n 0.008838610043037353\n 0.009150200653972363\n 0.0033786745702838548\n -0.038294149701005455\n 0.19558778043000458\n -0.4287431844207055\n 0.2897784524160024\n -0.07844856274209384\n -0.06309657343028045\n 0.4713127757906535\n 0.37559520558051235\n -0.18672567514518862\n -0.06950765283326864\n -0.28123708084784266\n -0.12665848284693404\n -0.1382024418769914\n 0.17528665552673206\n -0.015448951451204404\n 0.15478177087438777\n\n\n\nm1.β # alternative extractor\n\n20-element Vector{Float64}:\n -0.013323098599731003\n 0.008838610043037353\n 0.009150200653972363\n 0.0033786745702838548\n -0.038294149701005455\n 0.19558778043000458\n -0.4287431844207055\n 0.2897784524160024\n -0.07844856274209384\n -0.06309657343028045\n 0.4713127757906535\n 0.37559520558051235\n -0.18672567514518862\n -0.06950765283326864\n -0.28123708084784266\n -0.12665848284693404\n -0.1382024418769914\n 0.17528665552673206\n -0.015448951451204404\n 0.15478177087438777\n\n\n\nfixefnames(m1) # works also for coefnames(m1)\n\n20-element Vector{String}:\n \"(Intercept)\"\n \"Test: Star_r\"\n \"Test: S20_r\"\n \"Test: SLJ\"\n \"Test: BMT\"\n \"a1\"\n \"Sex: Girls\"\n \"Test: Star_r & a1\"\n \"Test: S20_r & a1\"\n \"Test: SLJ & a1\"\n \"Test: BMT & a1\"\n \"Test: Star_r & Sex: Girls\"\n \"Test: S20_r & Sex: Girls\"\n \"Test: SLJ & Sex: Girls\"\n \"Test: BMT & Sex: Girls\"\n \"a1 & Sex: Girls\"\n \"Test: Star_r & a1 & Sex: Girls\"\n \"Test: S20_r & a1 & Sex: Girls\"\n \"Test: SLJ & a1 & Sex: Girls\"\n \"Test: BMT & a1 & Sex: Girls\"\n\n\n\nvcov(m1) # StatsBase.vcov: var-cov matrix of fixed-effects coefficients\n\n20×20 Matrix{Float64}:\n 0.000368507 2.44342e-5 -1.70167e-5 … 3.88654e-6 2.86561e-6\n 2.44342e-5 0.00140405 -0.000421338 -9.36638e-7 -1.02894e-7\n -1.70167e-5 -0.000421338 0.000856925 2.05905e-5 -6.93885e-7\n -2.70244e-5 5.45269e-5 -0.000468523 -2.44479e-5 1.37177e-6\n -0.000143181 3.24318e-5 -7.84036e-6 1.42252e-6 -1.19388e-5\n -4.45756e-5 -7.41319e-5 2.71349e-5 … -6.00773e-5 -5.10063e-5\n -4.42921e-5 6.31848e-5 4.36702e-5 -7.38387e-5 0.000282261\n -2.72845e-5 -0.000442502 0.000212823 8.7135e-6 2.60536e-6\n 3.37114e-6 0.000213207 -0.000395339 -0.000104421 8.19859e-6\n -2.00104e-5 9.27317e-8 0.000175292 0.000158974 -6.11699e-5\n 7.30239e-5 -2.67159e-5 2.85901e-5 … -5.78764e-5 8.27668e-5\n 6.87754e-6 8.83986e-7 -2.07244e-6 8.74402e-6 -0.000108167\n 5.13701e-6 -1.97254e-6 7.70952e-6 0.000673632 0.000118037\n -5.21248e-6 -1.56374e-6 -4.65457e-6 -0.00136386 0.000576685\n -1.1505e-6 6.37318e-7 -9.09835e-7 0.000577351 -0.00171455\n -4.53729e-6 -1.20613e-5 3.47115e-7 … 0.000152123 -0.00391934\n -1.27935e-5 -2.83726e-5 2.85735e-5 -0.000140172 0.00179133\n 4.7094e-7 2.86411e-5 -4.71121e-5 -0.0115791 -0.00195509\n 3.88654e-6 -9.36638e-7 2.05905e-5 0.0234033 -0.00998405\n 2.86561e-6 -1.02894e-7 -6.93885e-7 -0.00998405 0.0289536\n\n\n\nvcov(m1; corr=true) # StatsBase.vcov: correlation matrix of fixed-effects coefficients\n\n20×20 Matrix{Float64}:\n 1.0 0.0339691 -0.0302817 … 0.00132343 0.000877288\n 0.0339691 1.0 -0.384121 -0.000163396 -1.61379e-5\n -0.0302817 -0.384121 1.0 0.00459787 -0.000139304\n -0.0470023 0.0485853 -0.534375 -0.00533568 0.000269164\n -0.252202 0.0292662 -0.00905631 0.000314419 -0.00237244\n -0.0426429 -0.0363317 0.0170228 … -0.00721181 -0.00550484\n -0.0739925 0.054076 0.0478407 -0.0154785 0.0531965\n -0.0160155 -0.133067 0.0819207 0.000641801 0.000172529\n 0.00214924 0.0696369 -0.165283 -0.00835373 0.000589681\n -0.0135576 3.21875e-5 0.0778828 0.0135157 -0.0046756\n 0.0446741 -0.00837319 0.0114699 … -0.004443 0.0057124\n 0.00701347 0.000461824 -0.0013859 0.00111891 -0.0124442\n 0.00563274 -0.00110807 0.00554355 0.0926864 0.0146016\n -0.00609965 -0.000937468 -0.00357183 -0.200269 0.0761325\n -0.00121025 0.00034346 -0.000627628 0.0762102 -0.203475\n -0.00225475 -0.00307062 0.000113116 … 0.00948593 -0.219728\n -0.00379571 -0.00431254 0.00555926 -0.00521853 0.0599583\n 0.000150198 0.00467971 -0.00985332 -0.463402 -0.0703454\n 0.00132343 -0.000163396 0.00459787 1.0 -0.383546\n 0.000877288 -1.61379e-5 -0.000139304 -0.383546 1.0\n\n\n\nstderror(m1) # StatsBase.stderror: SE for fixed-effects coefficients\n\n20-element Vector{Float64}:\n 0.01919652732574372\n 0.03747068322761317\n 0.029273285330210397\n 0.02995116114500642\n 0.029574196102221305\n 0.05445374107486756\n 0.031182881518992782\n 0.08874709312691241\n 0.08170907406598975\n 0.0768863280893603\n 0.08515035859926307\n 0.051083135211708375\n 0.047508147450545\n 0.04451607577961671\n 0.049520893473334576\n 0.10482774123584619\n 0.17557988961760165\n 0.16333492470289673\n 0.15298128937140495\n 0.1701576435993861\n\n\n\npropertynames(m1) # names of available extractors\n\n(:formula, :reterms, :Xymat, :feterm, :sqrtwts, :parmap, :dims, :A, :L, :optsum, :θ, :theta, :β, :beta, :βs, :betas, :λ, :lambda, :stderror, :σ, :sigma, :σs, :sigmas, :σρs, :sigmarhos, :b, :u, :lowerbd, :X, :y, :corr, :vcov, :PCA, :rePCA, :objective, :pvalues)" + }, + { + "objectID": "fggk21.html#covariance-parameter-estimates", + "href": "fggk21.html#covariance-parameter-estimates", + "title": "Basics with Emotikon Project", + "section": "6.3 Covariance parameter estimates", + "text": "6.3 Covariance parameter estimates\nThese commands inform us about the model parameters associated with the RES.\n+ julia> issingular(m1) # Test singularity for param. vector m1.theta\n+ julia> VarCorr(m1) # MixedModels.VarCorr: est. of RES\n+ julia> propertynames(m1)\n+ julia> m1.σ # residual; or: m1.sigma\n+ julia> m1.σs # VCs; m1.sigmas\n+ julia> m1.θ # Parameter vector for RES (w/o residual); m1.theta\n+ julia> MixedModels.sdest(m1) # prsqrt(MixedModels.varest(m1))\n+ julia> BlockDescription(m1) # Description of blocks of A and L in an LMM\n\nissingular(m1) # Test if model is singular for parameter vector m1.theta (default)\n\ntrue\n\n\n\nissingular(m2)\n\ntrue\n\n\n\nVarCorr(m1) # MixedModels.VarCorr: estimates of random-effect structure (RES)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\n\nChild\n(Intercept)\n0.34495271\n0.58732675\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.55269763\n0.74343636\n+0.14\n\n\n\n\n\n\n\n\nTest: S20_r\n0.40212511\n0.63413336\n+0.00\n-0.52\n\n\n\n\n\n\n\nTest: SLJ\n0.29385597\n0.54208484\n+0.05\n-0.04\n-0.35\n\n\n\n\n\n\nTest: BMT\n0.52032198\n0.72133347\n-0.35\n+0.14\n-0.18\n-0.22\n\n\n\n\nSchool\n(Intercept)\n0.04570702\n0.21379201\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.11076869\n0.33281931\n-0.06\n\n\n\n\n\n\n\n\nTest: S20_r\n0.10761018\n0.32803990\n-0.08\n-0.39\n\n\n\n\n\n\n\nTest: SLJ\n0.10318843\n0.32122956\n-0.19\n+0.21\n-0.81\n\n\n\n\n\n\nTest: BMT\n0.09795181\n0.31297254\n-0.33\n-0.02\n+0.13\n-0.38\n\n\n\n\n\na1\n0.07982565\n0.28253433\n+0.62\n-0.20\n+0.11\n-0.61\n+0.45\n\n\n\n\nSex: Girls\n0.02059180\n0.14349843\n-0.45\n+0.45\n+0.31\n-0.27\n-0.15\n-0.37\n\n\nCohort\n(Intercept)\n0.00017441\n0.01320652\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.00406549\n0.06376120\n.\n\n\n\n\n\n\n\n\nTest: S20_r\n0.00005702\n0.00755108\n.\n.\n\n\n\n\n\n\n\nTest: SLJ\n0.00114959\n0.03390556\n.\n.\n.\n\n\n\n\n\n\nTest: BMT\n0.00000000\n0.00000000\n.\n.\n.\n.\n\n\n\n\nResidual\n\n0.28422522\n0.53312777\n\n\n\n\n\n\n\n\n\n\n\n\nVarCorr(m2)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\n\nChild\nTest: Run\n0.5622290\n0.7498193\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.6350899\n0.7969253\n+0.50\n\n\n\n\n\n\n\n\nTest: S20_r\n0.5890796\n0.7675152\n+0.56\n+0.64\n\n\n\n\n\n\n\nTest: SLJ\n0.6089346\n0.7803426\n+0.55\n+0.60\n+0.72\n\n\n\n\n\n\nTest: BMT\n0.4854398\n0.6967351\n+0.19\n+0.40\n+0.37\n+0.49\n\n\n\n\nSchool\nTest: Run\n0.1370455\n0.3701965\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.0962783\n0.3102874\n+0.53\n\n\n\n\n\n\n\n\nTest: S20_r\n0.1092383\n0.3305122\n+0.47\n+0.48\n\n\n\n\n\n\n\nTest: SLJ\n0.0613838\n0.2477575\n+0.47\n+0.76\n+0.41\n\n\n\n\n\n\nTest: BMT\n0.0387541\n0.1968606\n+0.15\n+0.38\n+0.18\n+0.02\n\n\n\n\n\na1\n0.0802336\n0.2832554\n+0.58\n+0.47\n+0.56\n-0.05\n+0.65\n\n\n\n\nSex: Girls\n0.0205546\n0.1433686\n-0.63\n-0.27\n+0.06\n-0.28\n-0.58\n-0.37\n\n\nCohort\nTest: Run\n0.0030066\n0.0548320\n\n\n\n\n\n\n\n\n\nTest: Star_r\n0.0004344\n0.0208416\n.\n\n\n\n\n\n\n\n\nTest: S20_r\n0.0031716\n0.0563170\n.\n.\n\n\n\n\n\n\n\nTest: SLJ\n0.0002448\n0.0156459\n.\n.\n.\n\n\n\n\n\n\nTest: BMT\n0.0000000\n0.0000000\n.\n.\n.\n.\n\n\n\n\nResidual\n\n0.2623127\n0.5121647\n\n\n\n\n\n\n\n\n\n\n\n\nm1.σs # VCs; m1.sigmas\n\n(Child = (var\"(Intercept)\" = 0.58732674709639, var\"Test: Star_r\" = 0.7434363644610155, var\"Test: S20_r\" = 0.6341333569851855, var\"Test: SLJ\" = 0.542084839663831, var\"Test: BMT\" = 0.7213334730280074), School = (var\"(Intercept)\" = 0.21379201285323374, var\"Test: Star_r\" = 0.3328193108027132, var\"Test: S20_r\" = 0.3280398999015345, var\"Test: SLJ\" = 0.32122955990402635, var\"Test: BMT\" = 0.31297254325848156, a1 = 0.2825343257018283, var\"Sex: Girls\" = 0.14349842884369796), Cohort = (var\"(Intercept)\" = 0.013206524133973831, var\"Test: Star_r\" = 0.06376119875100951, var\"Test: S20_r\" = 0.0075510776232103555, var\"Test: SLJ\" = 0.033905564967234667, var\"Test: BMT\" = 0.0))\n\n\n\nm1.θ # Parameter vector for RES (w/o residual); m1.theta\n\n48-element Vector{Float64}:\n 1.1016622708905508\n 0.199835310203799\n 0.0007687193071826845\n 0.04921136459080601\n -0.4672660515500969\n 1.3800877425160862\n -0.6269901567906759\n -0.04925944187798209\n 0.25246320095240976\n 1.0107889422013527\n ⋮\n -0.1650040625391254\n 0.017653103520679392\n -0.005432524642045863\n 0.0\n 0.024771780682443064\n 0.11959834514264477\n 0.014163729752105945\n 0.06359744704679811\n 0.0\n\n\n\nBlockDescription(m1) # Description of blocks of A and L in a LinearMixedModel\n\n\n\n\nrows\nChild\nSchool\nCohort\nfixed\n\n\n9930\nBlkDiag\n\n\n\n\n\n3101\nSparse\nBlkDiag\n\n\n\n\n45\nDense\nDense\nBlkDiag/Dense\n\n\n\n21\nDense\nDense\nDense\nDense\n\n\n\n\n\n\nm2.θ\n\n48-element Vector{Float64}:\n 1.464019911728255\n 0.7822363274753275\n 0.842409314371142\n 0.8433757455902053\n 0.26413446683350555\n 1.3450740553826162\n 0.6154158471181771\n 0.5681284803530833\n 0.48038263080030397\n 1.075790626516376\n ⋮\n -0.18622672591150657\n 0.0013289707983532247\n -6.60220078341385e-5\n 0.0\n 0.1070593783172049\n 0.040693237170298845\n 0.1099588509456823\n 0.030548645240017568\n 0.0\n\n\n\nBlockDescription(m2)\n\n\n\n\nrows\nChild\nSchool\nCohort\nfixed\n\n\n9930\nBlkDiag\n\n\n\n\n\n3101\nSparse\nBlkDiag\n\n\n\n\n45\nDense\nDense\nBlkDiag/Dense\n\n\n\n21\nDense\nDense\nDense\nDense" + }, + { + "objectID": "fggk21.html#model-predictions", + "href": "fggk21.html#model-predictions", + "title": "Basics with Emotikon Project", + "section": "6.4 Model “predictions”", + "text": "6.4 Model “predictions”\nThese commands inform us about extracion of conditional modes/means and (co-)variances, that using the model parameters to improve the predictions for units (levels) of the grouping (random) factors. We need this information, e.g., for partial-effect response profiles (e.g., facet plot) or effect profiles (e.g., caterpillar plot), or visualizing the borrowing-strength effect for correlation parameters (e.g., shrinkage plots). We are using the fit of LMM m2.\njulia> condVar(m2)\nSome plotting functions are currently available from the MixedModelsMakie package or via custom functions.\n+ julia> caterpillar!(m2)\n+ julia> shrinkage!(m2)\n\n6.4.1 Conditional covariances\n\ncondVar(m1)\n\n3-element Vector{Array{Float64, 3}}:\n [0.058517014101499745 0.009125061044062778 … 0.003278708654799139 -0.02203512152066885; 0.009125061044062778 0.2881484213953011 … -0.00858828480580812 0.02637014650433069; … ; 0.003278708654799139 -0.00858828480580812 … 0.1948955299646874 -0.07616988873567451; -0.02203512152066885 0.02637014650433069 … -0.07616988873567451 0.25747094954024247;;; 0.06479408146453051 0.011134998138892753 … 0.003908367629588989 -0.028871049730723873; 0.011134998138892753 0.29633356972928204 … -0.007311382249300798 0.027042641223018752; … ; 0.003908367629588989 -0.007311382249300798 … 0.19801398892464123 -0.07674628588169728; -0.028871049730723873 0.027042641223018752 … -0.07674628588169728 0.2663833521833813;;; 0.06199490961166209 0.010378470648666283 … 0.0033211887933385473 -0.025799744230828064; 0.010378470648666283 0.2919637022019177 … -0.008099599279587298 0.02645587001375013; … ; 0.0033211887933385473 -0.008099599279587298 … 0.19709146464205915 -0.07626677986337765; -0.025799744230828064 0.02645587001375013 … -0.07626677986337765 0.26226810420768637;;; … ;;; 0.09251275858088985 0.011430464054488698 … -0.004372759710309024 -0.025512877879956514; 0.011430464054488698 0.29433912821382646 … -0.008626529501006942 0.02760443787517547; … ; -0.004372759710309024 -0.008626529501006942 … 0.196293768010921 -0.07428957822771329; -0.025512877879956514 0.02760443787517547 … -0.07428957822771329 0.26236636750253356;;; 0.09874036143268264 0.009185552998062238 … -0.0001573075613284164 -0.02462263824544809; 0.009185552998062238 0.2911679711936218 … -0.008776482510154664 0.027936646374801047; … ; -0.0001573075613284164 -0.008776482510154664 … 0.19584727323067816 -0.07501977375670275; -0.02462263824544809 0.027936646374801047 … -0.07501977375670275 0.2596118385454262;;; 0.08569687177833775 0.008946686416428581 … 0.0006592569895041175 -0.02086375290989933; 0.008946686416428581 0.28863042124474664 … -0.008936779278627875 0.02745669612294799; … ; 0.0006592569895041175 -0.008936779278627875 … 0.1948682013379163 -0.07525256402491896; -0.02086375290989933 0.02745669612294799 … -0.07525256402491896 0.2567755880517776]\n [0.03738662222206967 -0.0018006453467180224 … 0.029918884081614444 -0.01241043132709623; -0.0018006453467180224 0.09043180671160218 … -0.012288226712182382 0.017349422342826458; … ; 0.029918884081614444 -0.012288226712182382 … 0.06552240047084834 -0.012696720591960402; -0.01241043132709623 0.017349422342826458 … -0.012696720591960402 0.018266200494610186;;; 0.032175235508378135 0.0002020231146900986 … 0.026239700886540457 -0.010631094016241854; 0.0002020231146900986 0.07633547772742386 … -0.0076489885496848755 0.014171038510892612; … ; 0.026239700886540457 -0.0076489885496848755 … 0.057371204435359965 -0.010702426636433508; -0.010631094016241854 0.014171038510892612 … -0.010702426636433508 0.015840463462564926;;; 0.0293211217872406 -0.0031722468514524776 … 0.021508439538618022 -0.010442275909060876; -0.0031722468514524776 0.08771293893576042 … -0.013169827546768145 0.01806934628580798; … ; 0.021508439538618022 -0.013169827546768145 … 0.051822149216928896 -0.010311386841756282; -0.010442275909060876 0.01806934628580798 … -0.010311386841756282 0.016719819496053525;;; … ;;; 0.0267612826186117 0.002887196156566762 … 0.019017122735360242 -0.006845576372521417; 0.002887196156566762 0.07608301622502864 … -0.005537174830550229 0.012875566802524871; … ; 0.019017122735360242 -0.005537174830550229 … 0.050902196361920206 -0.006078576026170587; -0.006845576372521417 0.012875566802524871 … -0.006078576026170587 0.013661101965124827;;; 0.027228645015369164 0.003500454962860299 … 0.020300439683147174 -0.006150803022415798; 0.003500454962860299 0.07600122149129579 … -0.005010775855234807 0.012574439977365934; … ; 0.020300439683147174 -0.005010775855234807 … 0.05302863567353013 -0.00626554725377429; -0.006150803022415798 0.012574439977365934 … -0.00626554725377429 0.013684452210043274;;; 0.043339821193180326 -0.0030818917690474392 … 0.03637959916087954 -0.01363338241537408; -0.0030818917690474392 0.0993577864113983 … -0.015052524931708725 0.019104775094131096; … ; 0.03637959916087954 -0.015052524931708725 … 0.07506039488224463 -0.014302079998948012; -0.01363338241537408 0.019104775094131096 … -0.014302079998948012 0.019440414377459254]\n [0.00016380655132630802 6.905263271054442e-6 … -4.776505132880116e-6 0.0; 6.905263271054442e-6 0.002357685591177613 … -0.0001535595839979168 0.0; … ; -4.776505132880116e-6 -0.0001535595839979168 … 0.0008635382997135062 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00016040198449826642 8.762829702597266e-6 … -6.068571712892409e-6 0.0; 8.762829702597266e-6 0.002044995779766137 … -0.0001672018085446938 0.0; … ; -6.068571712892409e-6 -0.0001672018085446938 … 0.000795065708027814 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00016236819064567072 7.626721927893261e-6 … -5.190387582391803e-6 0.0; 7.626721927893261e-6 0.002242186063452188 … -0.00015894848127273218 0.0; … ; -5.190387582391803e-6 -0.00015894848127273218 … 0.0008353101094087038 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00016175490481618921 8.411278134019113e-6 … -5.514796437067044e-6 0.0; 8.411278134019113e-6 0.0021643858461830056 … -0.00016336647221245931 0.0; … ; -5.514796437067044e-6 -0.00016336647221245931 … 0.0008232493551568546 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00015933306329354854 8.683518670335156e-6 … -6.949857071192867e-6 0.0; 8.683518670335156e-6 0.001962786328235615 … -0.0001688729211888127 0.0; … ; -6.949857071192867e-6 -0.0001688729211888127 … 0.0007737698549017665 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0001585591247886362 7.442199159884006e-6 … -7.387837364196275e-6 0.0; 7.442199159884006e-6 0.0019342658473775333 … -0.0001675218103327855 0.0; … ; -7.387837364196275e-6 -0.0001675218103327855 … 0.0007566342723566994 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00016065497306304335 8.005529319817548e-6 … -6.461140807188436e-6 0.0; 8.005529319817548e-6 0.0020775998050236873 … -0.00016680712988184607 0.0; … ; -6.461140807188436e-6 -0.00016680712988184607 … 0.0007993999850391486 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00015831595367890705 8.46712980408871e-6 … -7.964470854652428e-6 0.0; 8.46712980408871e-6 0.0018894476583808555 … -0.00017251367903727097 0.0; … ; -7.964470854652428e-6 -0.00017251367903727097 … 0.0007531486757886536 0.0; 0.0 0.0 … 0.0 0.0;;; 0.00015649962194566806 9.755584275305604e-6 … -7.749920208393922e-6 0.0; 9.755584275305604e-6 0.0017892434008092122 … -0.00017175668964200161 0.0; … ; -7.749920208393922e-6 -0.00017175668964200161 … 0.0007269191158251669 0.0; 0.0 0.0 … 0.0 0.0]\n\n\n\ncondVar(m2)\n\n3-element Vector{Array{Float64, 3}}:\n [0.17121835522819978 0.02755529331418949 … 0.03416706723888015 -0.00724754030926216; 0.02755529331418949 0.17389268509524453 … 0.03753020675650032 0.0188623774335696; … ; 0.03416706723888015 0.03753020675650032 … 0.16031637538660184 0.028820239120887613; -0.00724754030926216 0.0188623774335696 … 0.028820239120887613 0.160864500973238;;; 0.18242691671642797 0.0343250931186835 … 0.04147060082369551 -0.009143227675242742; 0.0343250931186835 0.18567343287578988 … 0.04866689163492084 0.021405410618718965; … ; 0.04147060082369551 0.04866689163492084 … 0.17266718224767022 0.03123733241019097; -0.009143227675242742 0.021405410618718965 … 0.03123733241019097 0.16331432606425472;;; 0.1762667317205972 0.0311279231692659 … 0.038186553972383325 -0.007994491834495502; 0.0311279231692659 0.18039096830847456 … 0.04354026417410434 0.02014451692076348; … ; 0.038186553972383325 0.04354026417410434 … 0.1670059987629768 0.030077380180769695; -0.007994491834495502 0.02014451692076348 … 0.030077380180769695 0.16204289027390376;;; … ;;; 0.21741247314653891 0.07093054334800777 … 0.06747150120313818 0.02145098713166718; 0.07093054334800777 0.22154962297484282 … 0.07284423050626627 0.05096352617049405; … ; 0.06747150120313818 0.07284423050626627 … 0.18375387349238714 0.05065449550765889; 0.02145098713166718 0.05096352617049405 … 0.05065449550765889 0.18636447571570958;;; 0.22425843774021273 0.07771036649190435 … 0.07692945052170154 0.032146495017884365; 0.07771036649190435 0.22464983667288474 … 0.07893678447646196 0.058693580990441856; … ; 0.07692945052170154 0.07893678447646196 … 0.192417812551263 0.06096244826047057; 0.032146495017884365 0.058693580990441856 … 0.06096244826047057 0.1952055411652199;;; 0.20613389930796294 0.0614937415744954 … 0.062270378347748845 0.02206740872792677; 0.0614937415744954 0.20740253938205164 … 0.06405704423364168 0.047854049528350144; … ; 0.062270378347748845 0.06405704423364168 … 0.179269076031725 0.0517136334805597; 0.02206740872792677 0.047854049528350144 … 0.0517136334805597 0.18687984024155918]\n [0.10768671208086496 0.04839952818075811 … 0.04766653444192149 -0.027767847991630235; 0.04839952818075811 0.07949753046909964 … 0.03493825637829439 -0.01030835164217005; … ; 0.04766653444192149 0.03493825637829439 … 0.06574810342307015 -0.01275585556178704; -0.027767847991630235 -0.01030835164217005 … -0.01275585556178704 0.018227082481554094;;; 0.08790407612332059 0.040393261282076404 … 0.0400599319562759 -0.02269489800607808; 0.040393261282076404 0.06914721694875275 … 0.03206702198915689 -0.008425353467929443; … ; 0.0400599319562759 0.03206702198915689 … 0.05749755077922079 -0.010761924798630933; -0.02269489800607808 -0.008425353467929443 … -0.010761924798630933 0.015807162176876503;;; 0.09636492785218488 0.035884742431221237 … 0.038333680468635785 -0.025416695484362636; 0.035884742431221237 0.06308321756031697 … 0.02476724723597186 -0.007242081284198682; … ; 0.038333680468635785 0.02476724723597186 … 0.05198552853377316 -0.010391082211543702; -0.025416695484362636 -0.007242081284198682 … -0.010391082211543702 0.01668905433983563;;; … ;;; 0.07441056179847254 0.0305275873520764 … 0.029374260939504657 -0.017029674955578736; 0.0305275873520764 0.06267297540942841 … 0.02349135736015709 -0.004060003964855782; … ; 0.029374260939504657 0.02349135736015709 … 0.05102700756299053 -0.006132825015515446; -0.017029674955578736 -0.004060003964855782 … -0.006132825015515446 0.013633872415800339;;; 0.07356358901088007 0.030536524186054008 … 0.0301082996724446 -0.016067883424430905; 0.030536524186054008 0.06346336276661714 … 0.024768348896849156 -0.0033951944729095373; … ; 0.0301082996724446 0.024768348896849156 … 0.053147386583431176 -0.006319089569004934; -0.016067883424430905 -0.0033951944729095373 … -0.006319089569004934 0.013656675423188653;;; 0.12473526100613286 0.05796391233780463 … 0.05756242970998203 -0.031000768758675107; 0.05796391233780463 0.09050849232950609 … 0.042023372901318 -0.011779510213246015; … ; 0.05756242970998203 0.042023372901318 … 0.07536073523942259 -0.014362415085092188; -0.031000768758675107 -0.011779510213246015 … -0.014362415085092188 0.01940248264853897]\n [0.0018738580389070167 3.097002198457398e-5 … 2.2076784503534793e-5 0.0; 3.097002198457398e-5 0.00039802187403120584 … 5.648497477795947e-6 0.0; … ; 2.2076784503534793e-5 5.648497477795947e-6 … 0.00023124160140848118 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0016450448619856558 3.729737750101121e-5 … 2.7027760485222434e-5 0.0; 3.729737750101121e-5 0.0003867402285022758 … 7.734284697291955e-6 0.0; … ; 2.7027760485222434e-5 7.734284697291955e-6 … 0.00022686693600509704 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0017882787826426782 3.243821501424111e-5 … 2.4047614324889707e-5 0.0; 3.243821501424111e-5 0.00039393409516233735 … 6.455070883862393e-6 0.0; … ; 2.4047614324889707e-5 6.455070883862393e-6 … 0.00022946203827014857 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0017313946330208606 3.4368248113981814e-5 … 2.50561873272665e-5 0.0; 3.4368248113981814e-5 0.0003918409974798727 … 6.670472162105535e-6 0.0; … ; 2.50561873272665e-5 6.670472162105535e-6 … 0.0002288103664780839 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0015868478954532295 3.8887650632968376e-5 … 2.7809025410647586e-5 0.0; 3.8887650632968376e-5 0.00038367187497630265 … 8.087941724439915e-6 0.0; … ; 2.7809025410647586e-5 8.087941724439915e-6 … 0.00022546408538126053 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0015688425920921166 4.000704168244891e-5 … 2.8473059101798132e-5 0.0; 4.000704168244891e-5 0.00038022839783514767 … 8.897931443372262e-6 0.0; … ; 2.8473059101798132e-5 8.897931443372262e-6 … 0.00022425219715871693 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0016705183562560773 3.6934723422390205e-5 … 2.6047147905636665e-5 0.0; 3.6934723422390205e-5 0.0003871521092119287 … 7.488540995030283e-6 0.0; … ; 2.6047147905636665e-5 7.488540995030283e-6 … 0.00022711057986501076 0.0; 0.0 0.0 … 0.0 0.0;;; 0.0015336542451690207 4.090147164649518e-5 … 2.9024635466401744e-5 0.0; 4.090147164649518e-5 0.00038008695637156323 … 8.800302599711073e-6 0.0; … ; 2.9024635466401744e-5 8.800302599711073e-6 … 0.00022397611621172725 0.0; 0.0 0.0 … 0.0 0.0;;; 0.001453691844155124 4.285465800881136e-5 … 3.0624467338295236e-5 0.0; 4.285465800881136e-5 0.00037403984354553936 … 9.967957192788682e-6 0.0; … ; 3.0624467338295236e-5 9.967957192788682e-6 … 0.00022206826601886315 0.0; 0.0 0.0 … 0.0 0.0]\n\n\nThey are hard to look at. Let’s take pictures.\n\n\n6.4.2 Caterpillar plots\n\n\nCode\ncaterpillar!(\n Figure(; resolution=(800, 400)), ranefinfo(m1, :Cohort)\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 3: Prediction intervals of the random effects for Cohort in model m1\n\n\n\n\n\n\n\n6.4.3 Shrinkage plots\nThese are just teasers. We will pick this up in a separate tutorial. Enjoy!\n\n\nCode\nshrinkageplot!(Figure(; resolution=(800, 800)), m1, :Cohort)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 4: Shrinkage plot of the random effects for Cohort in model m1\n\n\n\n\n\n\n\nCode\nshrinkageplot!(Figure(; resolution=(800, 800)), m2, :Cohort)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 5: Shrinkage plot of the random effects for Cohort in model m2\n\n\n\n\n\n\n\nFühner, T., Granacher, U., Golle, K., & Kliegl, R. (2021). Age and sex effects in physical fitness components of 108,295 third graders including 515 primary schools and 9 cohorts. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-97000-4\n\n\nSchielzeth, H., & Forstmeier, W. (2008). Conclusions beyond support: Overconfident estimates in mixed models. Behavioral Ecology, 20(2), 416–420. https://doi.org/10.1093/beheco/arn145" + }, + { + "objectID": "largescaledesigned.html", + "href": "largescaledesigned.html", + "title": "1 A large-scale designed experiment", + "section": "", + "text": "Load the packages to be used.\n\n\nCode\nusing AlgebraOfGraphics\nusing Arrow\nusing CairoMakie\nusing Chain\nusing DataFrameMacros\nusing DataFrames\nusing Effects\nusing MixedModels\nusing MixedModelsMakie\nusing SMLP2024: dataset\nusing StandardizedPredictors\nusing StatsBase\n\nCairoMakie.activate!(; type=\"svg\")\nimport ProgressMeter\nProgressMeter.ijulia_behavior(:clear)\n\n\nThe English Lexicon Project (Balota et al., 2007) was a large-scale multicenter study to examine properties of English words. It incorporated both a lexical decision task and a word recognition task. Different groups of subjects participated in the different tasks.", + "crumbs": [ + "Worked examples", + "A large-scale designed experiment" + ] + }, + { + "objectID": "largescaledesigned.html#trial-level-data-from-the-ldt", + "href": "largescaledesigned.html#trial-level-data-from-the-ldt", + "title": "1 A large-scale designed experiment", + "section": "2.1 Trial-level data from the LDT", + "text": "2.1 Trial-level data from the LDT\nIn the lexical decision task the study participant is shown a character string, under carefully controlled conditions, and responds according to whether they identify the string as a word or not. Two responses are recorded: whether the choice of word/non-word is correct and the time that elapsed between exposure to the string and registering a decision.\nSeveral covariates, some relating to the subject and some relating to the target, were recorded. Initially we consider only the trial-level data.\n\nldttrial = dataset(:ELP_ldt_trial)\n\n[ Info: Downloading ELP_ldt_trial dataset\n\n\nArrow.Table with 2745952 rows, 5 columns, and schema:\n :subj String\n :seq Int16\n :acc Union{Missing, Bool}\n :rt Int16\n :item String\n\nwith metadata given by a Base.ImmutableDict{String, String} with 3 entries:\n \"source\" => \"https://osf.io/n63s2\"\n \"reference\" => \"Balota et al. (2007), The English Lexicon Project, Behavior R…\n \"title\" => \"Trial-level data from Lexical Discrimination Task in the Engl…\n\n\nThe two response variables are acc - the accuracy of the response - and rt, the response time in milliseconds. There is one trial-level covariate, seq, the sequence number of the trial within subj. Each subject participated in two sessions on different days, with 2000 trials recorded on the first day.\nNotice the metadata with a citation and a URL for the OSF project.\nWe convert to a DataFrame and add a Boolean column s2 which is true for trials in the second session.\n\nldttrial = @transform(DataFrame(ldttrial), :s2 = :seq > 2000)\ndescribe(ldttrial)\n\n6×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nType\n\n\n\n\n1\nsubj\n\nS001\n\nS816\n0\nString\n\n\n2\nseq\n1687.21\n1\n1687.0\n3374\n0\nInt16\n\n\n3\nacc\n0.85604\nfalse\n1.0\ntrue\n1370\nUnion{Missing, Bool}\n\n\n4\nrt\n846.325\n-16160\n732.0\n32061\n0\nInt16\n\n\n5\nitem\n\nAarod\n\nzuss\n0\nString\n\n\n6\ns2\n0.407128\nfalse\n0.0\ntrue\n0\nBool", + "crumbs": [ + "Worked examples", + "A large-scale designed experiment" + ] + }, + { + "objectID": "largescaledesigned.html#sec-ldtinitialexplore", + "href": "largescaledesigned.html#sec-ldtinitialexplore", + "title": "1 A large-scale designed experiment", + "section": "2.2 Initial data exploration", + "text": "2.2 Initial data exploration\nFrom the basic summary of ldttrial we can see that there are some questionable response times — negative values and values over 32 seconds.\nBecause of obvious outliers we will use the median response time, which is not strongly influenced by outliers, rather than the mean response time when summarizing by item or by subject.\nAlso, there are missing values of the accuracy. We should check if these are associated with particular subjects or particular items.\n\n2.2.1 Summaries by item\nTo summarize by item we group the trials by item and use combine to produce the various summary statistics. As we will create similar summaries by subject, we incorporate an ‘i’ in the names of these summaries (and an ‘s’ in the name of the summaries by subject) to be able to identify the grouping used.\n\nbyitem = @chain ldttrial begin\n groupby(:item)\n @combine(\n :ni = length(:acc), # no. of obs\n :imiss = count(ismissing, :acc), # no. of missing acc\n :iacc = count(skipmissing(:acc)), # no. of accurate\n :imedianrt = median(:rt),\n )\n @transform!(\n :wrdlen = Int8(length(:item)),\n :ipropacc = :iacc / :ni\n )\nend\n\n80962×7 DataFrame80937 rows omitted\n\n\n\nRow\nitem\nni\nimiss\niacc\nimedianrt\nwrdlen\nipropacc\n\n\n\nString\nInt64\nInt64\nInt64\nFloat64\nInt8\nFloat64\n\n\n\n\n1\na\n35\n0\n26\n743.0\n1\n0.742857\n\n\n2\ne\n35\n0\n19\n824.0\n1\n0.542857\n\n\n3\naah\n34\n0\n21\n770.5\n3\n0.617647\n\n\n4\naal\n34\n0\n32\n702.5\n3\n0.941176\n\n\n5\nAaron\n33\n0\n31\n625.0\n5\n0.939394\n\n\n6\nAarod\n33\n0\n23\n810.0\n5\n0.69697\n\n\n7\naback\n34\n0\n15\n710.0\n5\n0.441176\n\n\n8\nahack\n34\n0\n34\n662.0\n5\n1.0\n\n\n9\nabacus\n34\n0\n17\n671.5\n6\n0.5\n\n\n10\nalacus\n34\n0\n29\n640.0\n6\n0.852941\n\n\n11\nabandon\n34\n0\n32\n641.0\n7\n0.941176\n\n\n12\nacandon\n34\n0\n33\n725.5\n7\n0.970588\n\n\n13\nabandoned\n34\n0\n31\n667.5\n9\n0.911765\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n80951\nzoology\n33\n0\n32\n623.0\n7\n0.969697\n\n\n80952\npoology\n33\n0\n32\n757.0\n7\n0.969697\n\n\n80953\nzoom\n35\n0\n34\n548.0\n4\n0.971429\n\n\n80954\nzool\n35\n0\n30\n633.0\n4\n0.857143\n\n\n80955\nzooming\n33\n0\n29\n617.0\n7\n0.878788\n\n\n80956\nsooming\n33\n0\n30\n721.0\n7\n0.909091\n\n\n80957\nzooms\n33\n0\n30\n598.0\n5\n0.909091\n\n\n80958\ncooms\n33\n0\n31\n660.0\n5\n0.939394\n\n\n80959\nzucchini\n34\n0\n29\n781.5\n8\n0.852941\n\n\n80960\nhucchini\n34\n0\n32\n727.5\n8\n0.941176\n\n\n80961\nZurich\n34\n0\n21\n731.5\n6\n0.617647\n\n\n80962\nZurach\n34\n0\n26\n811.0\n6\n0.764706\n\n\n\n\n\n\nIt can be seen that the items occur in word/nonword pairs and the pairs are sorted alphabetically by the word in the pair (ignoring case). We can add the word/nonword status for the items as\n\nbyitem.isword = isodd.(eachindex(byitem.item))\ndescribe(byitem)\n\n8×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nDataType\n\n\n\n\n1\nitem\n\nAarod\n\nzuss\n0\nString\n\n\n2\nni\n33.9166\n30\n34.0\n37\n0\nInt64\n\n\n3\nimiss\n0.0169215\n0\n0.0\n2\n0\nInt64\n\n\n4\niacc\n29.0194\n0\n31.0\n37\n0\nInt64\n\n\n5\nimedianrt\n753.069\n458.0\n737.5\n1691.0\n0\nFloat64\n\n\n6\nwrdlen\n7.9988\n1\n8.0\n21\n0\nInt8\n\n\n7\nipropacc\n0.855616\n0.0\n0.911765\n1.0\n0\nFloat64\n\n\n8\nisword\n0.5\nfalse\n0.5\ntrue\n0\nBool\n\n\n\n\n\n\nThis table shows that some of the items were never identified correctly. These are\n\nfilter(:iacc => iszero, byitem)\n\n9×8 DataFrame\n\n\n\nRow\nitem\nni\nimiss\niacc\nimedianrt\nwrdlen\nipropacc\nisword\n\n\n\nString\nInt64\nInt64\nInt64\nFloat64\nInt8\nFloat64\nBool\n\n\n\n\n1\nbaobab\n34\n0\n0\n616.5\n6\n0.0\ntrue\n\n\n2\nhaulage\n34\n0\n0\n708.5\n7\n0.0\ntrue\n\n\n3\nleitmotif\n35\n0\n0\n688.0\n9\n0.0\ntrue\n\n\n4\nmiasmal\n35\n0\n0\n774.0\n7\n0.0\ntrue\n\n\n5\npeahen\n34\n0\n0\n684.0\n6\n0.0\ntrue\n\n\n6\nplosive\n34\n0\n0\n663.0\n7\n0.0\ntrue\n\n\n7\nplugugly\n33\n0\n0\n709.0\n8\n0.0\ntrue\n\n\n8\nposhest\n34\n0\n0\n740.0\n7\n0.0\ntrue\n\n\n9\nservo\n33\n0\n0\n697.0\n5\n0.0\ntrue\n\n\n\n\n\n\nNotice that these are all words but somewhat obscure words such that none of the subjects exposed to the word identified it correctly.\nWe can incorporate characteristics like wrdlen and isword back into the original trial table with a “left join”. This operation joins two tables by values in a common column. It is called a left join because the left (or first) table takes precedence, in the sense that every row in the left table is present in the result. If there is no matching row in the second table then missing values are inserted for the columns from the right table in the result.\n\ndescribe(\n leftjoin!(\n ldttrial,\n select(byitem, :item, :wrdlen, :isword);\n on=:item,\n ),\n)\n\n8×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nType\n\n\n\n\n1\nsubj\n\nS001\n\nS816\n0\nString\n\n\n2\nseq\n1687.21\n1\n1687.0\n3374\n0\nInt16\n\n\n3\nacc\n0.85604\nfalse\n1.0\ntrue\n1370\nUnion{Missing, Bool}\n\n\n4\nrt\n846.325\n-16160\n732.0\n32061\n0\nInt16\n\n\n5\nitem\n\nAarod\n\nzuss\n0\nString\n\n\n6\ns2\n0.407128\nfalse\n0.0\ntrue\n0\nBool\n\n\n7\nwrdlen\n7.99835\n1\n8.0\n21\n0\nUnion{Missing, Int8}\n\n\n8\nisword\n0.499995\nfalse\n0.0\ntrue\n0\nUnion{Missing, Bool}\n\n\n\n\n\n\nNotice that the wrdlen and isword variables in this table allow for missing values, because they are derived from the second argument, but there are no missing values for these variables. If there is no need to allow for missing values, there is a slight advantage in disallowing them in the element type, because the code to check for and handle missing values is not needed.\nThis could be done separately for each column or for the whole data frame, as in\n\ndescribe(disallowmissing!(ldttrial; error=false))\n\n8×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nType\n\n\n\n\n1\nsubj\n\nS001\n\nS816\n0\nString\n\n\n2\nseq\n1687.21\n1\n1687.0\n3374\n0\nInt16\n\n\n3\nacc\n0.85604\nfalse\n1.0\ntrue\n1370\nUnion{Missing, Bool}\n\n\n4\nrt\n846.325\n-16160\n732.0\n32061\n0\nInt16\n\n\n5\nitem\n\nAarod\n\nzuss\n0\nString\n\n\n6\ns2\n0.407128\nfalse\n0.0\ntrue\n0\nBool\n\n\n7\nwrdlen\n7.99835\n1\n8.0\n21\n0\nInt8\n\n\n8\nisword\n0.499995\nfalse\n0.0\ntrue\n0\nBool\n\n\n\n\n\n\n\n\n\n\n\n\nNamed argument “error”\n\n\n\n\n\nThe named argument error=false is required because there is one column, acc, that does incorporate missing values. If error=false were not given then the error thrown when trying to disallowmissing on the acc column would be propagated and the top-level call would fail.\n\n\n\nA barchart of the word length counts, Figure 1, shows that the majority of the items are between 3 and 14 characters.\n\n\nCode\nlet\n wlen = 1:21\n draw(\n data((; wrdlen=wlen, count=counts(byitem.wrdlen, wlen))) *\n mapping(:wrdlen => \"Length of word\", :count) *\n visual(BarPlot),\n )\nend\n\n\n\n\n\n\n\n\nFigure 1: Barchart of word lengths in the items used in the lexical decision task.\n\n\n\n\n\nTo examine trends in accuracy by word length we plot the proportion accurate versus word-length separately for words and non-words with the area of each marker proportional to the number of observations for that combination (Figure 2).\n\n\nCode\nlet\n itemsummry = combine(\n groupby(byitem, [:wrdlen, :isword]),\n :ni => sum,\n :imiss => sum,\n :iacc => sum,\n )\n @transform!(\n itemsummry,\n :iacc_mean = :iacc_sum / (:ni_sum - :imiss_sum)\n )\n @transform!(itemsummry, :msz = sqrt((:ni_sum - :imiss_sum) / 800))\n draw(\n data(itemsummry) * mapping(\n :wrdlen => \"Word length\",\n :iacc_mean => \"Proportion accurate\";\n color=:isword,\n markersize=:msz,\n );\n figure=(; resolution=(800, 450)),\n )\nend\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 2: Proportion of accurate trials in the LDT versus word length separately for words and non-words. The area of the marker is proportional to the number of observations represented.\n\n\n\n\n\nThe pattern in the range of word lengths with non-negligible counts (there are points in the plot down to word lengths of 1 and up to word lengths of 21 but these points are very small) is that the accuracy for words is nearly constant at about 84% and the accuracy for nonwords is slightly higher until lengths of 13, at which point it falls off a bit.\n\n\n2.2.2 Summaries by subject\nA summary of accuracy and median response time by subject\n\nbysubj = @chain ldttrial begin\n groupby(:subj)\n @combine(\n :ns = length(:acc), # no. of obs\n :smiss = count(ismissing, :acc), # no. of missing acc\n :sacc = count(skipmissing(:acc)), # no. of accurate\n :smedianrt = median(:rt),\n )\n @transform!(:spropacc = :sacc / :ns)\nend\n\n814×6 DataFrame789 rows omitted\n\n\n\nRow\nsubj\nns\nsmiss\nsacc\nsmedianrt\nspropacc\n\n\n\nString\nInt64\nInt64\nInt64\nFloat64\nFloat64\n\n\n\n\n1\nS001\n3374\n0\n3158\n554.0\n0.935981\n\n\n2\nS002\n3372\n1\n3031\n960.0\n0.898873\n\n\n3\nS003\n3372\n3\n3006\n813.0\n0.891459\n\n\n4\nS004\n3374\n1\n3062\n619.0\n0.907528\n\n\n5\nS005\n3374\n0\n2574\n677.0\n0.762893\n\n\n6\nS006\n3374\n0\n2927\n855.0\n0.867516\n\n\n7\nS007\n3374\n4\n2877\n918.5\n0.852697\n\n\n8\nS008\n3372\n1\n2731\n1310.0\n0.809905\n\n\n9\nS009\n3374\n13\n2669\n657.0\n0.791049\n\n\n10\nS010\n3374\n0\n2722\n757.0\n0.806758\n\n\n11\nS011\n3374\n0\n2894\n632.0\n0.857736\n\n\n12\nS012\n3374\n4\n2979\n692.0\n0.882928\n\n\n13\nS013\n3374\n2\n2980\n1114.0\n0.883225\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n803\nS805\n3374\n5\n2881\n534.0\n0.853883\n\n\n804\nS806\n3374\n1\n3097\n841.5\n0.917902\n\n\n805\nS807\n3374\n3\n2994\n704.0\n0.887374\n\n\n806\nS808\n3374\n2\n2751\n630.5\n0.815353\n\n\n807\nS809\n3372\n4\n2603\n627.0\n0.771945\n\n\n808\nS810\n3374\n1\n3242\n603.5\n0.960877\n\n\n809\nS811\n3374\n2\n2861\n827.0\n0.847955\n\n\n810\nS812\n3372\n6\n3012\n471.0\n0.893238\n\n\n811\nS813\n3372\n4\n2932\n823.0\n0.869514\n\n\n812\nS814\n3374\n1\n3070\n773.0\n0.909899\n\n\n813\nS815\n3374\n1\n3024\n602.0\n0.896266\n\n\n814\nS816\n3374\n0\n2950\n733.0\n0.874333\n\n\n\n\n\n\nshows some anomalies\n\ndescribe(bysubj)\n\n6×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nDataType\n\n\n\n\n1\nsubj\n\nS001\n\nS816\n0\nString\n\n\n2\nns\n3373.41\n3370\n3374.0\n3374\n0\nInt64\n\n\n3\nsmiss\n1.68305\n0\n1.0\n22\n0\nInt64\n\n\n4\nsacc\n2886.33\n1727\n2928.0\n3286\n0\nInt64\n\n\n5\nsmedianrt\n760.992\n205.0\n735.0\n1804.0\n0\nFloat64\n\n\n6\nspropacc\n0.855613\n0.511855\n0.868031\n0.973918\n0\nFloat64\n\n\n\n\n\n\nFirst, some subjects are accurate on only about half of their trials, which is the proportion that would be expected from random guessing. A plot of the median response time versus proportion accurate, Figure 3, shows that the subjects with lower accuracy are some of the fastest responders, further indicating that these subjects are sacrificing accuracy for speed.\n\n\nCode\ndraw(\n data(bysubj) *\n mapping(\n :spropacc => \"Proportion accurate\",\n :smedianrt => \"Median response time (ms)\",\n ) *\n (visual(Scatter) + smooth())\n)\n\n\n\n\n\n\n\n\nFigure 3: Median response time versus proportion accurate by subject in the LDT.\n\n\n\n\n\nAs described in Balota et al. (2007), the participants performed the trials in blocks of 250 followed by a short break. During the break they were given feedback concerning accuracy and response latency in the previous block of trials. If the accuracy was less than 80% the participant was encouraged to improve their accuracy. Similarly, if the mean response latency was greater than 1000 ms, the participant was encouraged to decrease their response time. During the trials immediate feedback was given if the response was incorrect.\nNevertheless, approximately 15% of the subjects were unable to maintain 80% accuracy on their trials\n\ncount(<(0.8), bysubj.spropacc) / nrow(bysubj)\n\n0.15233415233415235\n\n\nand there is some association of faster response times with low accuracy. The majority of the subjects whose median response time is less than 500 ms. are accurate on less than 75% of their trials. Another way of characterizing the relationship is that none of the subjects with 90% accuracy or greater had a median response time less than 500 ms.\n\nminimum(@subset(bysubj, :spropacc > 0.9).smedianrt)\n\n505.0\n\n\nIt is common in analyses of response latency in a lexical discrimination task to consider only the latencies on correct identifications and to trim outliers. In Balota et al. (2007) a two-stage outlier removal strategy was used; first removing responses less than 200 ms or greater than 3000 ms then removing responses more than three standard deviations from the participant’s mean response.\nAs described in Section 2.2.3 we will analyze these data on a speed scale (the inverse of response time) using only the first-stage outlier removal of response latencies less than 200 ms or greater than 3000 ms. On the speed scale the limits are 0.333 per second up to 5 per second.\nTo examine the effects of the fast but inaccurate responders we will fit models to the data from all the participants and to the data from the 85% of participants who maintained an overall accuracy of 80% or greater.\n\npruned = @chain ldttrial begin\n @subset(!ismissing(:acc), 200 ≤ :rt ≤ 3000,)\n leftjoin!(select(bysubj, :subj, :spropacc); on=:subj)\n dropmissing!\nend\nsize(pruned)\n\n(2714311, 9)\n\n\n\ndescribe(pruned)\n\n9×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nDataType\n\n\n\n\n1\nsubj\n\nS001\n\nS816\n0\nString\n\n\n2\nseq\n1684.56\n1\n1684.0\n3374\n0\nInt16\n\n\n3\nacc\n0.859884\nfalse\n1.0\ntrue\n0\nBool\n\n\n4\nrt\n838.712\n200\n733.0\n3000\n0\nInt16\n\n\n5\nitem\n\nAarod\n\nzuss\n0\nString\n\n\n6\ns2\n0.40663\nfalse\n0.0\ntrue\n0\nBool\n\n\n7\nwrdlen\n7.99244\n1\n8.0\n21\n0\nInt8\n\n\n8\nisword\n0.500126\nfalse\n1.0\ntrue\n0\nBool\n\n\n9\nspropacc\n0.857169\n0.511855\n0.869295\n0.973918\n0\nFloat64\n\n\n\n\n\n\n\n\n2.2.3 Choice of response scale\nAs we have indicated, generally the response times are analyzed for the correct identifications only. Furthermore, unrealistically large or small response times are eliminated. For this example we only use the responses between 200 and 3000 ms.\nA density plot of the pruned response times, Figure 4, shows they are skewed to the right.\n\n\nCode\ndraw(\n data(pruned) *\n mapping(:rt => \"Response time (ms.) for correct responses\") *\n AlgebraOfGraphics.density();\n figure=(; resolution=(800, 450)),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 4: Kernel density plot of the pruned response times (ms.) in the LDT.\n\n\n\n\n\nIn such cases it is common to transform the response to a scale such as the logarithm of the response time or to the speed of the response, which is the inverse of the response time.\nThe density of the response speed, in responses per second, is shown in Figure 5.\n\n\nCode\ndraw(\n data(pruned) *\n mapping(\n :rt => (x -> 1000 / x) => \"Response speed (s⁻¹) for correct responses\") *\n AlgebraOfGraphics.density();\n figure=(; resolution=(800, 450)),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 5: Kernel density plot of the pruned response speed in the LDT.\n\n\n\n\n\nFigure 4 and Figure 5 indicate that it may be more reasonable to establish a lower bound of 1/3 second (333 ms) on the response latency, corresponding to an upper bound of 3 per second on the response speed. However, only about one half of one percent of the correct responses have latencies in the range of 200 ms. to 333 ms.\n\ncount(\n r -> !ismissing(r.acc) && 200 < r.rt < 333,\n eachrow(ldttrial),\n) / count(!ismissing, ldttrial.acc)\n\n0.005867195806137328\n\n\nso the exact position of the lower cut-off point on the response latencies is unlikely to be very important.\n\n\n\n\n\n\nUsing inline transformations vs defining new columns\n\n\n\n\n\nIf you examine the code for (fit-elpldtspeeddens?), you will see that the conversion from rt to speed is done inline rather than creating and storing a new variable in the DataFrame.\nI prefer to keep the DataFrame simple with the integer variables (e.g. :rt) if possible.\nI recommend using the StandardizedPredictors.jl capabilities to center numeric variables or convert to zscores.\n\n\n\n\n\n2.2.4 Transformation of response and the form of the model\nAs noted in Box & Cox (1964), a transformation of the response that produces a more Gaussian distribution often will also produce a simpler model structure. For example, Figure 6 shows the smoothed relationship between word length and response time for words and non-words separately,\n\n\nCode\ndraw(\n data(pruned) *\n mapping(\n :wrdlen => \"Word length\",\n :rt => \"Response time (ms)\";\n :color => :isword,\n ) * smooth()\n)\n\n\n\n\n\n\n\n\nFigure 6: Scatterplot smooths of response time versus word length in the LDT.\n\n\n\n\n\nand Figure 7 shows the similar relationships for speed\n\n\nCode\ndraw(\n data(pruned) *\n mapping(\n :wrdlen => \"Word length\",\n :rt => (x -> 1000/x) => \"Speed of response (s⁻¹)\";\n :color => :isword,\n ) * smooth()\n)\n\n\n\n\n\n\n\n\nFigure 7: Scatterplot smooths of response speed versus word length in the LDT.\n\n\n\n\n\nFor the most part the smoother lines in Figure 7 are reasonably straight. The small amount of curvature is associated with short word lengths, say less than 4 characters, of which there are comparatively few in the study.\nFigure 8 shows a “violin plot” - the empirical density of the response speed by word length separately for words and nonwords. The lines on the plot are fit by linear regression.\n\n\nCode\nlet\n plt = data(@subset(pruned, :wrdlen > 3, :wrdlen < 14))\n plt *= mapping(\n :wrdlen => \"Word length\",\n :rt => (x -> 1000/x) => \"Speed of response (s⁻¹)\",\n color=:isword,\n side=:isword,\n )\n plt *= visual(Violin)\n draw(plt, axis=(; limits=(nothing, (0.0, 2.8))))\nend\n\n\n\n\n\n\n\n\nFigure 8: Empirical density of response speed versus word length by word/non-word status.", + "crumbs": [ + "Worked examples", + "A large-scale designed experiment" + ] + }, + { + "objectID": "largescaledesigned.html#sec-ldtinitialmodel", + "href": "largescaledesigned.html#sec-ldtinitialmodel", + "title": "1 A large-scale designed experiment", + "section": "2.3 Models with scalar random effects", + "text": "2.3 Models with scalar random effects\nA major purpose of the English Lexicon Project is to characterize the items (words or nonwords) according to the observed accuracy of identification and to response latency, taking into account subject-to-subject variability, and to relate these to lexical characteristics of the items.\nIn Balota et al. (2007) the item response latency is characterized by the average response latency from the correct trials after outlier removal.\nMixed-effects models allow us greater flexibility and, we hope, precision in characterizing the items by controlling for subject-to-subject variability and for item characteristics such as word/nonword and item length.\nWe begin with a model that has scalar random effects for item and for subject and incorporates fixed-effects for word/nonword and for item length and for the interaction of these terms.\n\n2.3.1 Establish the contrasts\nBecause there are a large number of items in the data set it is important to assign a Grouping() contrast to item (and, less importantly, to subj). For the isword factor we will use an EffectsCoding contrast with the base level as false. The non-words are assigned -1 in this contrast and the words are assigned +1. The wrdlen covariate is on its original scale but centered at 8 characters.\nThus the (Intercept) coefficient is the predicted speed of response for a typical subject and typical item (without regard to word/non-word status) of 8 characters.\nSet these contrasts\n\ncontrasts = Dict(\n :subj => Grouping(),\n :item => Grouping(),\n :isword => EffectsCoding(; base=false),\n :wrdlen => Center(8),\n)\n\nDict{Symbol, Any} with 4 entries:\n :item => Grouping()\n :wrdlen => Center(8)\n :isword => EffectsCoding(false, nothing)\n :subj => Grouping()\n\n\nand fit a first model with simple, scalar, random effects for subj and item.\n\nelm01 = let\n form = @formula(\n 1000 / rt ~ 1 + isword * wrdlen + (1 | item) + (1 | subj)\n )\n fit(MixedModel, form, pruned; contrasts)\nend\n\nMinimizing 2 Time: 0:00:00 ( 0.33 s/it)\n objective: 2.7266989738341523e6\nMinimizing 10 Time: 0:00:02 ( 0.22 s/it)\n objective: 2.54791986993588e6\nMinimizing 11 Time: 0:00:02 ( 0.21 s/it)\n objective: 2.5477518552180286e6\nMinimizing 12 Time: 0:00:02 ( 0.20 s/it)\n objective: 2.5477597344292034e6\nMinimizing 13 Time: 0:00:02 ( 0.19 s/it)\n objective: 2.5478041435803813e6\nMinimizing 14 Time: 0:00:02 ( 0.19 s/it)\n objective: 2.547734626944677e6\nMinimizing 15 Time: 0:00:02 ( 0.18 s/it)\n objective: 2.5477332006914383e6\nMinimizing 16 Time: 0:00:02 ( 0.18 s/it)\n objective: 2.5477314166843807e6\nMinimizing 17 Time: 0:00:02 ( 0.17 s/it)\n objective: 2.5477281107397955e6\nMinimizing 18 Time: 0:00:03 ( 0.17 s/it)\n objective: 2.5477271061153305e6\nMinimizing 19 Time: 0:00:03 ( 0.17 s/it)\n objective: 2.5477267354581975e6\nMinimizing 20 Time: 0:00:03 ( 0.16 s/it)\n objective: 2.5477250237434045e6\nMinimizing 21 Time: 0:00:03 ( 0.16 s/it)\n objective: 2.547721761413731e6\nMinimizing 22 Time: 0:00:03 ( 0.16 s/it)\n objective: 2.5477152538446435e6\nMinimizing 23 Time: 0:00:03 ( 0.16 s/it)\n objective: 2.5477038026323644e6\nMinimizing 24 Time: 0:00:03 ( 0.15 s/it)\n objective: 2.547687283736832e6\nMinimizing 25 Time: 0:00:03 ( 0.15 s/it)\n objective: 2.5476642897852818e6\nMinimizing 26 Time: 0:00:03 ( 0.15 s/it)\n objective: 2.5476747912287745e6\nMinimizing 27 Time: 0:00:04 ( 0.15 s/it)\n objective: 2.5478612597469087e6\nMinimizing 28 Time: 0:00:04 ( 0.15 s/it)\n objective: 2.5476016701920675e6\nMinimizing 29 Time: 0:00:04 ( 0.15 s/it)\n objective: 2.547533486707639e6\nMinimizing 30 Time: 0:00:04 ( 0.15 s/it)\n objective: 2.5474570471784873e6\nMinimizing 31 Time: 0:00:04 ( 0.14 s/it)\n objective: 2.5475489231080017e6\nMinimizing 32 Time: 0:00:04 ( 0.14 s/it)\n objective: 2.5492153915398596e6\nMinimizing 33 Time: 0:00:04 ( 0.14 s/it)\n objective: 2.5474903052397575e6\nMinimizing 34 Time: 0:00:04 ( 0.14 s/it)\n objective: 2.547841784697444e6\nMinimizing 35 Time: 0:00:04 ( 0.14 s/it)\n objective: 2.5474595570143443e6\nMinimizing 36 Time: 0:00:05 ( 0.14 s/it)\n objective: 2.547457139701927e6\nMinimizing 37 Time: 0:00:05 ( 0.14 s/it)\n objective: 2.547458181007241e6\nMinimizing 38 Time: 0:00:05 ( 0.14 s/it)\n objective: 2.547476739788983e6\nMinimizing 39 Time: 0:00:05 ( 0.14 s/it)\n objective: 2.5474566821279465e6\nMinimizing 40 Time: 0:00:05 ( 0.14 s/it)\n objective: 2.54745660988993e6\nMinimizing 41 Time: 0:00:05 ( 0.14 s/it)\n objective: 2.547456628379439e6\nMinimizing 42 Time: 0:00:05 ( 0.14 s/it)\n objective: 2.547457340940068e6\nMinimizing 43 Time: 0:00:05 ( 0.13 s/it)\n objective: 2.547456569392055e6\nMinimizing 44 Time: 0:00:05 ( 0.13 s/it)\n objective: 2.547456539806895e6\nMinimizing 45 Time: 0:00:05 ( 0.13 s/it)\n objective: 2.5474565263292952e6\nMinimizing 46 Time: 0:00:06 ( 0.13 s/it)\n objective: 2.5474565253121504e6\nMinimizing 47 Time: 0:00:06 ( 0.13 s/it)\n objective: 2.547456524841241e6\nMinimizing 48 Time: 0:00:06 ( 0.13 s/it)\n objective: 2.5474565248725615e6\nMinimizing 49 Time: 0:00:06 ( 0.13 s/it)\n objective: 2.5474565286854426e6\nMinimizing 50 Time: 0:00:06 ( 0.13 s/it)\n objective: 2.547456524731621e6\nMinimizing 51 Time: 0:00:06 ( 0.13 s/it)\n objective: 2.547456524831969e6\nMinimizing 52 Time: 0:00:06 ( 0.13 s/it)\n objective: 2.547456527911539e6\nMinimizing 53 Time: 0:00:06 ( 0.13 s/it)\n objective: 2.547456524730126e6\nMinimizing 53 Time: 0:00:06 ( 0.13 s/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n1.3758\n0.0090\n153.69\n<1e-99\n0.1185\n0.2550\n\n\nisword: true\n0.0625\n0.0005\n131.35\n<1e-99\n\n\n\n\nwrdlen(centered: 8)\n-0.0436\n0.0002\n-225.38\n<1e-99\n\n\n\n\nisword: true & wrdlen(centered: 8)\n-0.0056\n0.0002\n-28.83\n<1e-99\n\n\n\n\nResidual\n0.3781\n\n\n\n\n\n\n\n\n\n\nThe predicted response speed by word length and word/nonword status can be summarized as\n\neffects(Dict(:isword => [false, true], :wrdlen => 4:2:12), elm01)\n\n10×6 DataFrame\n\n\n\nRow\nwrdlen\nisword\n1000 / rt\nerr\nlower\nupper\n\n\n\nInt64\nBool\nFloat64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\n4\nfalse\n1.46555\n0.00903111\n1.45652\n1.47458\n\n\n2\n6\nfalse\n1.38947\n0.00898124\n1.38049\n1.39845\n\n\n3\n8\nfalse\n1.31338\n0.00896459\n1.30442\n1.32235\n\n\n4\n10\nfalse\n1.2373\n0.00898134\n1.22832\n1.24628\n\n\n5\n12\nfalse\n1.16121\n0.00903129\n1.15218\n1.17025\n\n\n6\n4\ntrue\n1.6351\n0.0090311\n1.62607\n1.64413\n\n\n7\n6\ntrue\n1.5367\n0.00898124\n1.52772\n1.54569\n\n\n8\n8\ntrue\n1.43831\n0.00896459\n1.42934\n1.44727\n\n\n9\n10\ntrue\n1.33991\n0.00898133\n1.33092\n1.34889\n\n\n10\n12\ntrue\n1.24151\n0.00903128\n1.23248\n1.25054\n\n\n\n\n\n\nIf we restrict to only those subjects with 80% accuracy or greater the model becomes\n\nelm02 = let\n form = @formula(\n 1000 / rt ~ 1 + isword * wrdlen + (1 | item) + (1 | subj)\n )\n dat = @subset(pruned, :spropacc > 0.8)\n fit(MixedModel, form, dat; contrasts)\nend\n\nMinimizing 2 Time: 0:00:00 (90.05 ms/it)\n objective: 1.785784454054488e6\nMinimizing 3 Time: 0:00:00 (93.74 ms/it)\n objective: 1.706955807895325e6\nMinimizing 5 Time: 0:00:00 (90.62 ms/it)\n objective: 1.7096084055650986e6\nMinimizing 7 Time: 0:00:00 (89.21 ms/it)\n objective: 1.632475326661393e6\nMinimizing 9 Time: 0:00:00 (88.94 ms/it)\n objective: 1.6308713786709716e6\nMinimizing 11 Time: 0:00:00 (88.39 ms/it)\n objective: 1.6306926636436544e6\nMinimizing 13 Time: 0:00:01 (87.60 ms/it)\n objective: 1.63051455649771e6\nMinimizing 15 Time: 0:00:01 (86.70 ms/it)\n objective: 1.6305061481623596e6\nMinimizing 17 Time: 0:00:01 (86.18 ms/it)\n objective: 1.6304874273651254e6\nMinimizing 19 Time: 0:00:01 (85.37 ms/it)\n objective: 1.630472047621346e6\nMinimizing 21 Time: 0:00:01 (85.47 ms/it)\n objective: 1.6304442820480603e6\nMinimizing 23 Time: 0:00:01 (85.87 ms/it)\n objective: 1.6306054243878725e6\nMinimizing 25 Time: 0:00:02 (85.63 ms/it)\n objective: 1.6304138602695977e6\nMinimizing 27 Time: 0:00:02 (85.09 ms/it)\n objective: 1.6303847542605218e6\nMinimizing 29 Time: 0:00:02 (84.79 ms/it)\n objective: 1.630379610046544e6\nMinimizing 31 Time: 0:00:02 (84.64 ms/it)\n objective: 1.6303774101036177e6\nMinimizing 33 Time: 0:00:02 (84.34 ms/it)\n objective: 1.6303723086549612e6\nMinimizing 35 Time: 0:00:02 (84.01 ms/it)\n objective: 1.6303668436558153e6\nMinimizing 37 Time: 0:00:03 (84.87 ms/it)\n objective: 1.630366182470287e6\nMinimizing 39 Time: 0:00:03 (84.98 ms/it)\n objective: 1.6303660445637105e6\nMinimizing 41 Time: 0:00:03 (84.94 ms/it)\n objective: 1.630365959547105e6\nMinimizing 43 Time: 0:00:03 (84.93 ms/it)\n objective: 1.6303659564438118e6\nMinimizing 45 Time: 0:00:03 (84.86 ms/it)\n objective: 1.6303659561360003e6\nMinimizing 47 Time: 0:00:03 (84.96 ms/it)\n objective: 1.6303659561178428e6\nMinimizing 49 Time: 0:00:04 (84.83 ms/it)\n objective: 1.630365956261281e6\nMinimizing 51 Time: 0:00:04 (84.56 ms/it)\n objective: 1.6303659560896743e6\nMinimizing 53 Time: 0:00:04 (84.44 ms/it)\n objective: 1.6303659560892093e6\nMinimizing 53 Time: 0:00:04 (84.45 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n1.3611\n0.0088\n153.98\n<1e-99\n0.1247\n0.2318\n\n\nisword: true\n0.0656\n0.0005\n133.73\n<1e-99\n\n\n\n\nwrdlen(centered: 8)\n-0.0444\n0.0002\n-222.65\n<1e-99\n\n\n\n\nisword: true & wrdlen(centered: 8)\n-0.0057\n0.0002\n-28.73\n<1e-99\n\n\n\n\nResidual\n0.3342\n\n\n\n\n\n\n\n\n\n\n\neffects(Dict(:isword => [false, true], :wrdlen => 4:2:12), elm02)\n\n10×6 DataFrame\n\n\n\nRow\nwrdlen\nisword\n1000 / rt\nerr\nlower\nupper\n\n\n\nInt64\nBool\nFloat64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\n4\nfalse\n1.45036\n0.00892467\n1.44144\n1.45929\n\n\n2\n6\nfalse\n1.37297\n0.00887101\n1.3641\n1.38184\n\n\n3\n8\nfalse\n1.29557\n0.00885309\n1.28672\n1.30443\n\n\n4\n10\nfalse\n1.21818\n0.00887111\n1.20931\n1.22705\n\n\n5\n12\nfalse\n1.14078\n0.00892487\n1.13186\n1.14971\n\n\n6\n4\ntrue\n1.62735\n0.00892466\n1.61842\n1.63627\n\n\n7\n6\ntrue\n1.52702\n0.00887101\n1.51815\n1.53589\n\n\n8\n8\ntrue\n1.4267\n0.00885309\n1.41784\n1.43555\n\n\n9\n10\ntrue\n1.32637\n0.0088711\n1.3175\n1.33524\n\n\n10\n12\ntrue\n1.22605\n0.00892484\n1.21712\n1.23497\n\n\n\n\n\n\nThe differences in the fixed-effects parameter estimates between a model fit to the full data set and one fit to the data from accurate responders only, are small.\nHowever, the random effects for the item, while highly correlated, are not perfectly correlated.\n\n\nCode\nCairoMakie.activate!(; type=\"png\")\ndisallowmissing!(\n leftjoin!(\n byitem,\n leftjoin!(\n rename!(DataFrame(raneftables(elm01)[:item]), [:item, :elm01]),\n rename!(DataFrame(raneftables(elm02)[:item]), [:item, :elm02]);\n on=:item,\n ),\n on=:item,\n ),\n)\ndisallowmissing!(\n leftjoin!(\n bysubj,\n leftjoin!(\n rename!(DataFrame(raneftables(elm01)[:subj]), [:subj, :elm01]),\n rename!(DataFrame(raneftables(elm02)[:subj]), [:subj, :elm02]);\n on=:subj,\n ),\n on=:subj,\n ); error=false,\n)\ndraw(\n data(byitem) * mapping(\n :elm01 => \"Conditional means of item random effects for model elm01\",\n :elm02 => \"Conditional means of item random effects for model elm02\";\n color=:isword,\n ) * visual(Scatter; alpha=0.2);\n axis=(; width=600, height=600),\n)\n\n\n\n\n\n\n\nFigure 9: Conditional means of scalar random effects for item in model elm01, fit to the pruned data, versus those for model elm02, fit to the pruned data with inaccurate subjects removed.\n\n\n\n\n\n\n\n\n\n\nNote\n\n\n\nAdjust the alpha on Figure 9.\n\n\nFigure 9 is exactly of the form that would be expected in a sample from a correlated multivariate Gaussian distribution. The correlation of the two sets of conditional means is about 96%.\n\ncor(Matrix(select(byitem, :elm01, :elm02)))\n\n2×2 Matrix{Float64}:\n 1.0 0.958655\n 0.958655 1.0\n\n\nThese models take only a few seconds to fit on a modern laptop computer, which is quite remarkable given the size of the data set and the number of random effects.\nThe amount of time to fit more complex models will be much greater so we may want to move those fits to more powerful server computers. We can split the tasks of fitting and analyzing a model between computers by saving the optimization summary after the model fit and later creating the MixedModel object followed by restoring the optsum object.\n\nsaveoptsum(\"./fits/elm01.json\", elm01);\n\n\nelm01a = restoreoptsum!(\n let\n form = @formula(\n 1000 / rt ~ 1 + isword * wrdlen + (1 | item) + (1 | subj)\n )\n MixedModel(form, pruned; contrasts)\n end,\n \"./fits/elm01.json\",\n)\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n1.3758\n0.0090\n153.69\n<1e-99\n0.1185\n0.2550\n\n\nisword: true\n0.0625\n0.0005\n131.35\n<1e-99\n\n\n\n\nwrdlen(centered: 8)\n-0.0436\n0.0002\n-225.38\n<1e-99\n\n\n\n\nisword: true & wrdlen(centered: 8)\n-0.0056\n0.0002\n-28.83\n<1e-99\n\n\n\n\nResidual\n0.3781\n\n\n\n\n\n\n\n\n\n\nOther covariates associated with the item are available as\n\nelpldtitem = DataFrame(dataset(\"ELP_ldt_item\"))\ndescribe(elpldtitem)\n\n[ Info: Downloading ELP_ldt_item dataset\n\n\n9×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nType\n\n\n\n\n1\nitem\n\nAarod\n\nzuss\n0\nString\n\n\n2\nOrtho_N\n1.53309\n0\n1.0\n25\n0\nInt8\n\n\n3\nBG_Sum\n13938.4\n11\n13026.0\n59803\n177\nUnion{Missing, Int32}\n\n\n4\nBG_Mean\n1921.25\n5.5\n1907.0\n6910.0\n177\nUnion{Missing, Float32}\n\n\n5\nBG_Freq_By_Pos\n2043.08\n0\n1928.0\n6985\n4\nUnion{Missing, Int16}\n\n\n6\nitemno\n40481.5\n1\n40481.5\n80962\n0\nInt32\n\n\n7\nisword\n0.5\nfalse\n0.5\ntrue\n0\nBool\n\n\n8\nwrdlen\n7.9988\n1\n8.0\n21\n0\nInt8\n\n\n9\npairno\n20241.0\n1\n20241.0\n40481\n0\nInt32\n\n\n\n\n\n\nand those associated with the subject are\n\nelpldtsubj = DataFrame(dataset(\"ELP_ldt_subj\"))\ndescribe(elpldtsubj)\n\n[ Info: Downloading ELP_ldt_subj dataset\n\n\n20×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nAny\nAny\nInt64\nType\n\n\n\n\n1\nsubj\n\nS001\n\nS816\n0\nString\n\n\n2\nuniv\n\nKansas\n\nWayne State\n0\nString\n\n\n3\nsex\n\nf\n\nm\n8\nUnion{Missing, String}\n\n\n4\nDOB\n\n1938-06-07\n\n1984-11-14\n0\nDate\n\n\n5\nMEQ\n44.4932\n19.0\n44.0\n75.0\n8\nUnion{Missing, Float32}\n\n\n6\nvision\n5.51169\n0\n6.0\n7\n1\nUnion{Missing, Int8}\n\n\n7\nhearing\n5.86101\n0\n6.0\n7\n1\nUnion{Missing, Int8}\n\n\n8\neducatn\n8.89681\n1\n12.0\n28\n0\nInt8\n\n\n9\nncorrct\n29.8505\n5\n30.0\n40\n18\nUnion{Missing, Int8}\n\n\n10\nrawscor\n31.9925\n13\n32.0\n40\n18\nUnion{Missing, Int8}\n\n\n11\nvocabAge\n17.8123\n10.3\n17.8\n21.0\n19\nUnion{Missing, Float32}\n\n\n12\nshipTime\n3.0861\n0\n3.0\n9\n1\nUnion{Missing, Int8}\n\n\n13\nreadTime\n2.50215\n0.0\n2.0\n15.0\n1\nUnion{Missing, Float32}\n\n\n14\npreshlth\n5.48708\n0\n6.0\n7\n1\nUnion{Missing, Int8}\n\n\n15\npasthlth\n4.92989\n0\n5.0\n7\n1\nUnion{Missing, Int8}\n\n\n16\nS1start\n\n2001-03-16T13:49:27\n2001-10-16T11:38:28.500\n2003-07-29T18:48:44\n0\nDateTime\n\n\n17\nS2start\n\n2001-03-19T10:00:35\n2001-10-19T14:24:19.500\n2003-07-30T13:07:45\n0\nDateTime\n\n\n18\nMEQstrt\n\n2001-03-22T18:32:00\n2001-10-23T11:26:13\n2003-07-30T14:30:49\n7\nUnion{Missing, DateTime}\n\n\n19\nfilename\n\n101DATA.LDT\n\nData998.LDT\n0\nString\n\n\n20\nfrstLang\n\nEnglish\n\nother\n8\nUnion{Missing, String}\n\n\n\n\n\n\nFor the simple model elm01 the estimated standard deviation of the random effects for subject is greater than that of the random effects for item, a common occurrence. A caterpillar plot, Figure 10,\n\n\nCode\nqqcaterpillar!(\n Figure(resolution=(800, 650)),\n ranefinfo(elm01, :subj),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 10: Conditional means and 95% prediction intervals for subject random effects in elm01.\n\n\n\n\n\nshows definite distinctions between subjects because the widths of the prediction intervals are small compared to the range of the conditional modes. Also, there is at least one outlier with a conditional mode over 1.0.\nFigure 11 is the corresponding caterpillar plot for model elm02 fit to the data with inaccurate responders eliminated.\n\n\nCode\nqqcaterpillar!(\n Figure(resolution=(800, 650)),\n ranefinfo(elm02, :subj),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 11: Conditional means and 95% prediction intervals for subject random effects in elm02.", + "crumbs": [ + "Worked examples", + "A large-scale designed experiment" + ] + }, + { + "objectID": "largescaledesigned.html#random-effects-from-the-simple-model-related-to-covariates", + "href": "largescaledesigned.html#random-effects-from-the-simple-model-related-to-covariates", + "title": "1 A large-scale designed experiment", + "section": "2.4 Random effects from the simple model related to covariates", + "text": "2.4 Random effects from the simple model related to covariates\nThe random effects “estimates” (technically they are “conditional means”) from the simple model elm01 provide a measure of how much the item or subject differs from the population. (We use elm01 because the main difference between elm01 and elm02 are that some subjects were dropped before fitting elm02.)\nFor the item its length and word/non-word status have already been incorporated in the model. At this point the subjects are just being treated as a homogeneous population.\nThe random effects conditional means have been extracted and incorporated in the byitem and bysubj tables. Now add selected demographic and item-specific measures.\n\nitemextended = leftjoin(\n byitem,\n select(elpldtitem, 1:5);\n on = :item,\n)\nsubjextended = leftjoin(\n bysubj,\n select(elpldtsubj, 1:3, :vocabAge);\n on=:subj,\n)\n\n814×11 DataFrame789 rows omitted\n\n\n\nRow\nsubj\nns\nsmiss\nsacc\nsmedianrt\nspropacc\nelm01\nelm02\nuniv\nsex\nvocabAge\n\n\n\nString\nInt64\nInt64\nInt64\nFloat64\nFloat64\nFloat64\nFloat64?\nString?\nString?\nFloat32?\n\n\n\n\n1\nS001\n3374\n0\n3158\n554.0\n0.935981\n0.411459\n0.426624\nMorehead\nm\n19.8\n\n\n2\nS002\n3372\n1\n3031\n960.0\n0.898873\n-0.30907\n-0.293732\nMorehead\nf\n17.8\n\n\n3\nS003\n3372\n3\n3006\n813.0\n0.891459\n-0.153078\n-0.139436\nMorehead\nf\n18.2\n\n\n4\nS004\n3374\n1\n3062\n619.0\n0.907528\n0.213047\n0.22754\nMorehead\nf\n18.6\n\n\n5\nS005\n3374\n0\n2574\n677.0\n0.762893\n0.0850349\nmissing\nMorehead\nf\n16.2\n\n\n6\nS006\n3374\n0\n2927\n855.0\n0.867516\n-0.207356\n-0.192651\nMorehead\nf\n17.8\n\n\n7\nS007\n3374\n4\n2877\n918.5\n0.852697\n-0.182201\n-0.166357\nMorehead\nf\n17.4\n\n\n8\nS008\n3372\n1\n2731\n1310.0\n0.809905\n-0.541434\n-0.526828\nMorehead\nm\n16.2\n\n\n9\nS009\n3374\n13\n2669\n657.0\n0.791049\n0.154926\nmissing\nMorehead\nf\n16.6\n\n\n10\nS010\n3374\n0\n2722\n757.0\n0.806758\n-0.0541104\n-0.0403266\nMorehead\nf\n17.0\n\n\n11\nS011\n3374\n0\n2894\n632.0\n0.857736\n0.217734\n0.231618\nMorehead\nf\n17.4\n\n\n12\nS012\n3374\n4\n2979\n692.0\n0.882928\n0.062351\n0.0770981\nMorehead\nm\n18.2\n\n\n13\nS013\n3374\n2\n2980\n1114.0\n0.883225\n-0.409761\n-0.3956\nMorehead\nf\n18.2\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n803\nS805\n3374\n5\n2881\n534.0\n0.853883\n0.480461\n0.495683\nWash. Univ\nm\n19.0\n\n\n804\nS806\n3374\n1\n3097\n841.5\n0.917902\n-0.1888\n-0.173376\nWash. Univ\nm\n19.8\n\n\n805\nS807\n3374\n3\n2994\n704.0\n0.887374\n0.01919\n0.0338241\nWash. Univ\nm\n17.4\n\n\n806\nS808\n3374\n2\n2751\n630.5\n0.815353\n0.199416\n0.214299\nWash. Univ\nf\n18.6\n\n\n807\nS809\n3372\n4\n2603\n627.0\n0.771945\n0.2277\nmissing\nWash. Univ\nm\n15.1\n\n\n808\nS810\n3374\n1\n3242\n603.5\n0.960877\n0.252522\n0.266822\nWash. Univ\nm\n19.8\n\n\n809\nS811\n3374\n2\n2861\n827.0\n0.847955\n-0.158097\n-0.143568\nWash. Univ\nf\n16.2\n\n\n810\nS812\n3372\n6\n3012\n471.0\n0.893238\n0.748427\n0.76354\nWash. Univ\nf\n19.8\n\n\n811\nS813\n3372\n4\n2932\n823.0\n0.869514\n-0.167166\n-0.153846\nWash. Univ\nm\n17.4\n\n\n812\nS814\n3374\n1\n3070\n773.0\n0.909899\n-0.0753662\n-0.0606956\nWash. Univ\nf\n18.6\n\n\n813\nS815\n3374\n1\n3024\n602.0\n0.896266\n0.249134\n0.2643\nWash. Univ\nf\n18.6\n\n\n814\nS816\n3374\n0\n2950\n733.0\n0.874333\n-0.0364596\n-0.0222916\nWash. Univ\nf\n17.8\n\n\n\n\n\n\nAs shown in Figure 12, there does not seem to be a strong relationship between vocabulary age and speed of response by subject.\n\n\nCode\ndraw(\n data(dropmissing(select(subjextended, :elm01, :vocabAge, :sex))) *\n mapping(\n :vocabAge => \"Vocabulary age (yr) of subject\",\n :elm01 => \"Random effect in model elm01\";\n color=:sex,\n ) * visual(Scatter; alpha=0.6)\n)\n\n\n\n\n\n\n\n\nFigure 12: Random effect for subject in model elm01 versus vocabulary age\n\n\n\n\n\n\n\nCode\ndraw(\n data(dropmissing(select(subjextended, :elm01, :univ))) *\n mapping(\n :elm01 => \"Random effect in model elm01\";\n color=:univ => \"University\",\n ) * AlgebraOfGraphics.density()\n)\n\n\n\n\n\n\n\n\nFigure 13: Estimated density of random effects for subject in model elm01 by university\n\n\n\n\n\n\n\nCode\ndraw(\n data(dropmissing(select(subjextended, :elm02, :univ))) *\n mapping(\n :elm02 => \"Random effect in model elm02 (accurate responders only)\";\n color=:univ => \"University\",\n ) * AlgebraOfGraphics.density()\n)\n\n\n\n\n\n\n\n\nFigure 14: Estimated density of random effects for subject in model elm02, fit to accurate responders only, by university\n\n\n\n\n\n\n\nCode\nCairoMakie.activate!(; type=\"png\")\ndraw(\n data(dropmissing(select(itemextended, :elm01, :BG_Mean, :isword))) *\n mapping(\n :BG_Mean => \"Mean bigram frequency\",\n :elm01 => \"Random effect in model elm01\";\n color=:isword,\n ) * visual(Scatter; alpha=0.2)\n)\n\n\n\n\n\n\n\nFigure 15: Random effect in model elm01 versus mean bigram frequency, by word/nonword status", + "crumbs": [ + "Worked examples", + "A large-scale designed experiment" + ] + }, + { + "objectID": "shrinkageplot.html", + "href": "shrinkageplot.html", + "title": "More on shrinkage plots", + "section": "", + "text": "I have stated that the likelihood criterion used to fit linear mixed-effects can be considered as balancing fidelity to the data (i.e. fits the observed data well) versus model complexity.\nThis is similar to some of the criterion used in Machine Learning (ML), except that the criterion for LMMs has a rigorous mathematical basis.\nIn the shrinkage plot we consider the values of the random-effects coefficients for the fitted values of the model versus those from a model in which there is no penalty for model complexity.\nIf there is strong subject-to-subject variation then the model fit will tend to values of the random effects similar to those without a penalty on complexity.\nIf the random effects term is not contributing much (i.e. it is “inert”) then the random effects will be shrunk considerably towards zero in some directions.\nCode\nusing CairoMakie\nusing DataFrames\nusing LinearAlgebra\nusing MixedModels\nusing MixedModelsMakie\nusing Random\nusing ProgressMeter\n\nProgressMeter.ijulia_behavior(:clear);\nLoad the kb07 data set (don’t tell Reinhold that I used these data).\nkb07 = MixedModels.dataset(:kb07)\n\nArrow.Table with 1789 rows, 7 columns, and schema:\n :subj String\n :item String\n :spkr String\n :prec String\n :load String\n :rt_trunc Int16\n :rt_raw Int16\ncontrasts = Dict(\n :subj => Grouping(),\n :item => Grouping(),\n :spkr => HelmertCoding(),\n :prec => HelmertCoding(),\n :load => HelmertCoding(),\n)\nm1 = let\n form = @formula(\n rt_trunc ~\n 1 +\n spkr * prec * load +\n (1 + spkr + prec + load | subj) +\n (1 + spkr + prec + load | item)\n )\n fit(MixedModel, form, kb07; contrasts)\nend\n\nMinimizing 2 Time: 0:00:00 ( 0.46 s/it)\n objective: 29353.753287212847\nMinimizing 614 Time: 0:00:01 ( 3.05 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_subj\nσ_item\n\n\n(Intercept)\n2181.6750\n77.3029\n28.22\n<1e-99\n301.8414\n362.1701\n\n\nspkr: old\n67.7490\n18.2824\n3.71\n0.0002\n43.0396\n40.5288\n\n\nprec: maintain\n-333.9206\n47.1621\n-7.08\n<1e-11\n62.1035\n246.9365\n\n\nload: yes\n78.7681\n19.5247\n4.03\n<1e-04\n65.1473\n42.1755\n\n\nspkr: old & prec: maintain\n-21.9634\n15.8062\n-1.39\n0.1647\n\n\n\n\nspkr: old & load: yes\n18.3838\n15.8062\n1.16\n0.2448\n\n\n\n\nprec: maintain & load: yes\n4.5333\n15.8062\n0.29\n0.7743\n\n\n\n\nspkr: old & prec: maintain & load: yes\n23.6052\n15.8062\n1.49\n0.1353\n\n\n\n\nResidual\n668.5061\nVarCorr(m1)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\nsubj\n(Intercept)\n91108.2428\n301.8414\n\n\n\n\n\n\nspkr: old\n1852.4098\n43.0396\n+0.78\n\n\n\n\n\nprec: maintain\n3856.8408\n62.1035\n-0.59\n+0.02\n\n\n\n\nload: yes\n4244.1689\n65.1473\n+0.36\n+0.82\n+0.53\n\n\nitem\n(Intercept)\n131167.1572\n362.1701\n\n\n\n\n\n\nspkr: old\n1642.5825\n40.5288\n+0.44\n\n\n\n\n\nprec: maintain\n60977.6107\n246.9365\n-0.69\n+0.35\n\n\n\n\nload: yes\n1778.7763\n42.1755\n+0.32\n+0.23\n-0.15\n\n\nResidual\n\n446900.4651\n668.5061\nissingular(m1)\n\ntrue\nprint(m1)\n\nLinear mixed model fit by maximum likelihood\n rt_trunc ~ 1 + spkr + prec + load + spkr & prec + spkr & load + prec & load + spkr & prec & load + (1 + spkr + prec + load | subj) + (1 + spkr + prec + load | item)\n logLik -2 logLik AIC AICc BIC \n -14318.5697 28637.1394 28695.1394 28696.1286 28854.3323\n\nVariance components:\n Column Variance Std.Dev. Corr.\nsubj (Intercept) 91108.2428 301.8414\n spkr: old 1852.4098 43.0396 +0.78\n prec: maintain 3856.8408 62.1035 -0.59 +0.02\n load: yes 4244.1689 65.1473 +0.36 +0.82 +0.53\nitem (Intercept) 131167.1572 362.1701\n spkr: old 1642.5825 40.5288 +0.44\n prec: maintain 60977.6107 246.9365 -0.69 +0.35\n load: yes 1778.7763 42.1755 +0.32 +0.23 -0.15\nResidual 446900.4651 668.5061\n Number of obs: 1789; levels of grouping factors: 56, 32\n\n Fixed-effects parameters:\n───────────────────────────────────────────────────────────────────────────────\n Coef. Std. Error z Pr(>|z|)\n───────────────────────────────────────────────────────────────────────────────\n(Intercept) 2181.67 77.3029 28.22 <1e-99\nspkr: old 67.749 18.2824 3.71 0.0002\nprec: maintain -333.921 47.1621 -7.08 <1e-11\nload: yes 78.7681 19.5247 4.03 <1e-04\nspkr: old & prec: maintain -21.9634 15.8062 -1.39 0.1647\nspkr: old & load: yes 18.3838 15.8062 1.16 0.2448\nprec: maintain & load: yes 4.53334 15.8062 0.29 0.7743\nspkr: old & prec: maintain & load: yes 23.6052 15.8062 1.49 0.1353\n───────────────────────────────────────────────────────────────────────────────", + "crumbs": [ + "Visualizations and diagnostics", + "More on shrinkage plots" + ] + }, + { + "objectID": "shrinkageplot.html#expressing-the-covariance-of-random-effects", + "href": "shrinkageplot.html#expressing-the-covariance-of-random-effects", + "title": "More on shrinkage plots", + "section": "1 Expressing the covariance of random effects", + "text": "1 Expressing the covariance of random effects\nEarlier today we mentioned that the parameters being optimized are from a “matrix square root” of the covariance matrix for the random effects. There is one such lower triangular matrix for each grouping factor.\n\nl1 = first(m1.λ) # Cholesky factor of relative covariance for subj\n\n4×4 LowerTriangular{Float64, Matrix{Float64}}:\n 0.451516 ⋅ ⋅ ⋅ \n 0.0502699 0.0402238 ⋅ ⋅ \n -0.0551878 0.072667 0.0174361 ⋅ \n 0.0351047 0.084743 0.0329117 0.0\n\n\nNotice the zero on the diagonal. A triangular matrix with zeros on the diagonal is singular.\n\nl2 = last(m1.λ) # this one is also singular\n\n4×4 LowerTriangular{Float64, Matrix{Float64}}:\n 0.54176 ⋅ ⋅ ⋅ \n 0.0267807 0.0543902 ⋅ ⋅ \n -0.253141 0.269008 0.0 ⋅ \n 0.0200425 0.00611238 -0.044718 0.0392618\n\n\nTo regenerate the covariance matrix we need to know that the covariance is not the square of l1, it is l1 * l1' (so that the result is symmetric) and multiplied by σ̂²\n\nΣ₁ = varest(m1) .* (l1 * l1')\n\n4×4 Matrix{Float64}:\n 91108.2 10143.6 -11136.0 7083.52\n 10143.6 1852.41 66.4352 2311.99\n -11136.0 66.4352 3856.84 2142.67\n 7083.52 2311.99 2142.67 4244.17\n\n\n\ndiag(Σ₁) # compare to the variance column in the VarCorr output\n\n4-element Vector{Float64}:\n 91108.24276565886\n 1852.4098101273987\n 3856.8407541614683\n 4244.168853804923\n\n\n\nsqrt.(diag(Σ₁))\n\n4-element Vector{Float64}:\n 301.8414198973674\n 43.03963069227475\n 62.10346813312014\n 65.1472858514069", + "crumbs": [ + "Visualizations and diagnostics", + "More on shrinkage plots" + ] + }, + { + "objectID": "shrinkageplot.html#shrinkage-plots", + "href": "shrinkageplot.html#shrinkage-plots", + "title": "More on shrinkage plots", + "section": "2 Shrinkage plots", + "text": "2 Shrinkage plots\n\n\nCode\nshrinkageplot(m1)\n\n\n\n\n\n\n\n\nFigure 1: Shrinkage plot of model m1\n\n\n\n\n\nThe upper left panel shows the perfect negative correlation for those two components of the random effects.\n\nshrinkageplot(m1, :item)\n\n\n\n\n\n\n\n\n\nX1 = Int.(m1.X')\n\n8×1789 Matrix{Int64}:\n 1 1 1 1 1 1 1 1 1 1 … 1 1 1 1 1 1 1 1 1\n -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1\n -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1\n 1 -1 -1 -1 -1 1 1 1 1 -1 1 1 1 -1 -1 -1 -1 1 1\n 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1\n -1 -1 -1 1 1 1 1 -1 -1 -1 … 1 -1 -1 -1 -1 1 1 1 1\n -1 -1 1 -1 1 1 -1 1 -1 -1 -1 1 -1 -1 1 -1 1 1 -1\n 1 -1 1 1 -1 1 -1 -1 1 -1 -1 -1 1 -1 1 1 -1 1 -1\n\n\n\nX1 * X1'\n\n8×8 Matrix{Int64}:\n 1789 -1 -1 3 -3 1 1 3\n -1 1789 -3 1 -1 3 3 1\n -1 -3 1789 1 -1 3 3 1\n 3 1 1 1789 3 -1 -1 -3\n -3 -1 -1 3 1789 1 1 3\n 1 3 3 -1 1 1789 -3 -1\n 1 3 3 -1 1 -3 1789 -1\n 3 1 1 -3 3 -1 -1 1789", + "crumbs": [ + "Visualizations and diagnostics", + "More on shrinkage plots" + ] + }, + { + "objectID": "shrinkageplot.html#how-to-interpret-a-shrinkage-plot", + "href": "shrinkageplot.html#how-to-interpret-a-shrinkage-plot", + "title": "More on shrinkage plots", + "section": "3 How to interpret a shrinkage plot", + "text": "3 How to interpret a shrinkage plot\n\nExtreme shrinkage (shrunk to a line or to a point) is easy to interpret - the term is not providing benefit and can be removed.\nWhen the range of the blue dots (shrunk values) is comparable to those of the red dots (unshrunk) it indicates that the term after shrinkage is about as strong as without shrinkage.\nBy itself, this doesn’t mean that the term is important. In some ways you need to get a feeling for the absolute magnitude of the random effects in addition to the relative magnitude.\nSmall magnitude and small relative magnitude indicate you can drop that term", + "crumbs": [ + "Visualizations and diagnostics", + "More on shrinkage plots" + ] + }, + { + "objectID": "shrinkageplot.html#conclusions-from-these-plots", + "href": "shrinkageplot.html#conclusions-from-these-plots", + "title": "More on shrinkage plots", + "section": "4 Conclusions from these plots", + "text": "4 Conclusions from these plots\n\nOnly the intercept for the subj appears to be contributing explanatory power\nFor the item both the intercept and the spkr appear to be contributing\n\n\nm2 = let\n form = @formula(\n rt_trunc ~\n 1 + prec * spkr * load + (1 | subj) + (1 + prec | item)\n )\n fit(MixedModel, form, kb07; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n2181.7582\n77.4709\n28.16\n<1e-99\n364.7286\n298.1109\n\n\nprec: maintain\n-333.8582\n47.4629\n-7.03\n<1e-11\n252.6687\n\n\n\nspkr: old\n67.8114\n16.0526\n4.22\n<1e-04\n\n\n\n\nload: yes\n78.6849\n16.0525\n4.90\n<1e-06\n\n\n\n\nprec: maintain & spkr: old\n-21.8802\n16.0525\n-1.36\n0.1729\n\n\n\n\nprec: maintain & load: yes\n4.4710\n16.0526\n0.28\n0.7806\n\n\n\n\nspkr: old & load: yes\n18.3214\n16.0526\n1.14\n0.2537\n\n\n\n\nprec: maintain & spkr: old & load: yes\n23.5219\n16.0525\n1.47\n0.1428\n\n\n\n\nResidual\n678.9318\n\n\n\n\n\n\n\n\n\n\n\nVarCorr(m2)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\nitem\n(Intercept)\n133026.916\n364.729\n\n\n\n\nprec: maintain\n63841.497\n252.669\n-0.70\n\n\nsubj\n(Intercept)\n88870.081\n298.111\n\n\n\nResidual\n\n460948.432\n678.932\n\n\n\n\n\n\n\n\nCode\nshrinkageplot(m2)\n\n\n\n\n\n\n\n\nFigure 2: Shrinkage plot of model m2\n\n\n\n\n\n\nm3 = let\n form = @formula(\n rt_trunc ~\n 1 + prec + spkr + load + (1 | subj) + (1 + prec | item)\n )\n fit(MixedModel, form, kb07; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n2181.8526\n77.4681\n28.16\n<1e-99\n364.7125\n298.0259\n\n\nprec: maintain\n-333.7906\n47.4472\n-7.03\n<1e-11\n252.5212\n\n\n\nspkr: old\n67.8790\n16.0785\n4.22\n<1e-04\n\n\n\n\nload: yes\n78.5904\n16.0785\n4.89\n<1e-05\n\n\n\n\nResidual\n680.0319\n\n\n\n\n\n\n\n\n\n\n\nVarCorr(m3)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\nitem\n(Intercept)\n133015.243\n364.713\n\n\n\n\nprec: maintain\n63766.936\n252.521\n-0.70\n\n\nsubj\n(Intercept)\n88819.436\n298.026\n\n\n\nResidual\n\n462443.388\n680.032\n\n\n\n\n\n\n\nrng = Random.seed!(1234321);\n\n\nm3btstrp = parametricbootstrap(rng, 2000, m3);\n\n\nDataFrame(shortestcovint(m3btstrp))\n\n9×5 DataFrame\n\n\n\nRow\ntype\ngroup\nnames\nlower\nupper\n\n\n\nString\nString?\nString?\nFloat64\nFloat64\n\n\n\n\n1\nβ\nmissing\n(Intercept)\n2030.64\n2335.35\n\n\n2\nβ\nmissing\nprec: maintain\n-418.691\n-237.661\n\n\n3\nβ\nmissing\nspkr: old\n36.0823\n97.0289\n\n\n4\nβ\nmissing\nload: yes\n47.7298\n109.975\n\n\n5\nσ\nitem\n(Intercept)\n264.8\n450.381\n\n\n6\nσ\nitem\nprec: maintain\n170.059\n310.399\n\n\n7\nρ\nitem\n(Intercept), prec: maintain\n-0.909187\n-0.475621\n\n\n8\nσ\nsubj\n(Intercept)\n229.638\n356.51\n\n\n9\nσ\nresidual\nmissing\n657.775\n701.589\n\n\n\n\n\n\n\nridgeplot(m3btstrp)\n\n\n\n\n\n\n\nFigure 3: Ridge plot of the fixed-effects coefficients from the bootstrap sample\n\n\n\n\n\n\nridgeplot(m3btstrp; show_intercept=false)\n\n\n\n\n\n\n\nFigure 4: Ridge plot of the fixed-effects coefficients from the bootstrap sample (with the intercept)\n\n\n\n\n\n\nm4 = let\n form = @formula(\n rt_trunc ~\n 1 + prec + spkr + load + (1 + prec | item) + (1 | subj)\n )\n fit(MixedModel, form, kb07; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n2181.8526\n77.4681\n28.16\n<1e-99\n364.7125\n298.0259\n\n\nprec: maintain\n-333.7906\n47.4472\n-7.03\n<1e-11\n252.5212\n\n\n\nspkr: old\n67.8790\n16.0785\n4.22\n<1e-04\n\n\n\n\nload: yes\n78.5904\n16.0785\n4.89\n<1e-05\n\n\n\n\nResidual\n680.0319\n\n\n\n\n\n\n\n\n\n\n\nm4bstrp = parametricbootstrap(rng, 2000, m4);\n\n\nridgeplot(m4bstrp; show_intercept=false)\n\n\n\n\n\n\n\n\n\nDataFrame(shortestcovint(m4bstrp))\n\n9×5 DataFrame\n\n\n\nRow\ntype\ngroup\nnames\nlower\nupper\n\n\n\nString\nString?\nString?\nFloat64\nFloat64\n\n\n\n\n1\nβ\nmissing\n(Intercept)\n2035.63\n2335.48\n\n\n2\nβ\nmissing\nprec: maintain\n-425.693\n-241.478\n\n\n3\nβ\nmissing\nspkr: old\n37.7216\n100.267\n\n\n4\nβ\nmissing\nload: yes\n50.2233\n112.426\n\n\n5\nσ\nitem\n(Intercept)\n263.272\n452.286\n\n\n6\nσ\nitem\nprec: maintain\n171.223\n313.796\n\n\n7\nρ\nitem\n(Intercept), prec: maintain\n-0.917884\n-0.4704\n\n\n8\nσ\nsubj\n(Intercept)\n228.256\n357.775\n\n\n9\nσ\nresidual\nmissing\n656.872\n701.941\n\n\n\n\n\n\n\nVarCorr(m4)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\nitem\n(Intercept)\n133015.243\n364.713\n\n\n\n\nprec: maintain\n63766.936\n252.521\n-0.70\n\n\nsubj\n(Intercept)\n88819.436\n298.026\n\n\n\nResidual\n\n462443.388\n680.032\n\n\n\n\n\n\n\n\nCode\nlet mods = [m1, m2, m4]\n DataFrame(;\n geomdof=(sum ∘ leverage).(mods),\n npar=dof.(mods),\n deviance=deviance.(mods),\n AIC=aic.(mods),\n BIC=bic.(mods),\n AICc=aicc.(mods),\n )\nend\n\n\n3×6 DataFrame\n\n\n\nRow\ngeomdof\nnpar\ndeviance\nAIC\nBIC\nAICc\n\n\n\nFloat64\nInt64\nFloat64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\n131.551\n29\n28637.1\n28695.1\n28854.3\n28696.1\n\n\n2\n107.543\n13\n28658.5\n28684.5\n28755.8\n28684.7\n\n\n3\n103.478\n9\n28663.9\n28681.9\n28731.3\n28682.0\n\n\n\n\n\n\n\nscatter(fitted(m4), residuals(m4))\n\n\n\n\n\n\n\nFigure 5: Residuals versus fitted values for model m4", + "crumbs": [ + "Visualizations and diagnostics", + "More on shrinkage plots" + ] + }, + { + "objectID": "glmm.html", + "href": "glmm.html", + "title": "Generalized linear mixed models", + "section": "", + "text": "Load the packages to be used\nCode\nusing AlgebraOfGraphics\nusing CairoMakie\nusing DataFrameMacros\nusing DataFrames\nusing MixedModels\nusing MixedModelsMakie\nusing SMLP2024: dataset\n\nCairoMakie.activate!(; type=\"svg\")\n\nimport ProgressMeter\nProgressMeter.ijulia_behavior(:clear)", + "crumbs": [ + "Generalized linear mixed models" + ] + }, + { + "objectID": "glmm.html#matrix-notation-for-the-sleepstudy-model", + "href": "glmm.html#matrix-notation-for-the-sleepstudy-model", + "title": "Generalized linear mixed models", + "section": "1 Matrix notation for the sleepstudy model", + "text": "1 Matrix notation for the sleepstudy model\n\nsleepstudy = DataFrame(dataset(:sleepstudy))\n\n180×3 DataFrame155 rows omitted\n\n\n\nRow\nsubj\ndays\nreaction\n\n\n\nString\nInt8\nFloat64\n\n\n\n\n1\nS308\n0\n249.56\n\n\n2\nS308\n1\n258.705\n\n\n3\nS308\n2\n250.801\n\n\n4\nS308\n3\n321.44\n\n\n5\nS308\n4\n356.852\n\n\n6\nS308\n5\n414.69\n\n\n7\nS308\n6\n382.204\n\n\n8\nS308\n7\n290.149\n\n\n9\nS308\n8\n430.585\n\n\n10\nS308\n9\n466.353\n\n\n11\nS309\n0\n222.734\n\n\n12\nS309\n1\n205.266\n\n\n13\nS309\n2\n202.978\n\n\n⋮\n⋮\n⋮\n⋮\n\n\n169\nS371\n8\n350.781\n\n\n170\nS371\n9\n369.469\n\n\n171\nS372\n0\n269.412\n\n\n172\nS372\n1\n273.474\n\n\n173\nS372\n2\n297.597\n\n\n174\nS372\n3\n310.632\n\n\n175\nS372\n4\n287.173\n\n\n176\nS372\n5\n329.608\n\n\n177\nS372\n6\n334.482\n\n\n178\nS372\n7\n343.22\n\n\n179\nS372\n8\n369.142\n\n\n180\nS372\n9\n364.124\n\n\n\n\n\n\n\ncontrasts = Dict(:subj => Grouping())\nm1 = let f = @formula reaction ~ 1 + days + (1 + days | subj)\n fit(MixedModel, f, sleepstudy; contrasts)\nend\nprintln(m1)\n\nLinear mixed model fit by maximum likelihood\n reaction ~ 1 + days + (1 + days | subj)\n logLik -2 logLik AIC AICc BIC \n -875.9697 1751.9393 1763.9393 1764.4249 1783.0971\n\nVariance components:\n Column Variance Std.Dev. Corr.\nsubj (Intercept) 565.51067 23.78047\n days 32.68212 5.71683 +0.08\nResidual 654.94145 25.59182\n Number of obs: 180; levels of grouping factors: 18\n\n Fixed-effects parameters:\n──────────────────────────────────────────────────\n Coef. Std. Error z Pr(>|z|)\n──────────────────────────────────────────────────\n(Intercept) 251.405 6.63226 37.91 <1e-99\ndays 10.4673 1.50224 6.97 <1e-11\n──────────────────────────────────────────────────\n\n\nThe response vector, y, has 180 elements. The fixed-effects coefficient vector, β, has 2 elements and the fixed-effects model matrix, X, is of size 180 × 2.\n\nm1.y\n\n180-element view(::Matrix{Float64}, :, 3) with eltype Float64:\n 249.56\n 258.7047\n 250.8006\n 321.4398\n 356.8519\n 414.6901\n 382.2038\n 290.1486\n 430.5853\n 466.3535\n ⋮\n 273.474\n 297.5968\n 310.6316\n 287.1726\n 329.6076\n 334.4818\n 343.2199\n 369.1417\n 364.1236\n\n\n\nm1.β\n\n2-element Vector{Float64}:\n 251.4051048484851\n 10.4672859595964\n\n\n\nm1.X\n\n180×2 Matrix{Float64}:\n 1.0 0.0\n 1.0 1.0\n 1.0 2.0\n 1.0 3.0\n 1.0 4.0\n 1.0 5.0\n 1.0 6.0\n 1.0 7.0\n 1.0 8.0\n 1.0 9.0\n ⋮ \n 1.0 1.0\n 1.0 2.0\n 1.0 3.0\n 1.0 4.0\n 1.0 5.0\n 1.0 6.0\n 1.0 7.0\n 1.0 8.0\n 1.0 9.0\n\n\nThe second column of X is just the days vector and the first column is all 1’s.\nThere are 36 random effects, 2 for each of the 18 levels of subj. The “estimates” (technically, the conditional means or conditional modes) are returned as a vector of matrices, one matrix for each grouping factor. In this case there is only one grouping factor for the random effects so there is one one matrix which contains 18 intercept random effects and 18 slope random effects.\n\nm1.b\n\n1-element Vector{Matrix{Float64}}:\n [2.815819244491056 -40.04844174471745 … 0.723262023911267 12.118907794830154; 9.075511691265579 -8.644079439241482 … -0.9710526311820062 1.3106980688400298]\n\n\n\nonly(m1.b) # only one grouping factor\n\n2×18 Matrix{Float64}:\n 2.81582 -40.0484 -38.4331 22.8321 … -24.7101 0.723262 12.1189\n 9.07551 -8.64408 -5.5134 -4.65872 4.6597 -0.971053 1.3107\n\n\nThere is a model matrix, Z, for the random effects. In general it has one chunk of columns for the first grouping factor, a chunk of columns for the second grouping factor, etc.\nIn this case there is only one grouping factor.\n\nInt.(first(m1.reterms))\n\n180×36 Matrix{Int64}:\n 1 0 0 0 0 0 0 0 0 0 0 0 0 … 0 0 0 0 0 0 0 0 0 0 0 0\n 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 1 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 1 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 1 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 1 5 0 0 0 0 0 0 0 0 0 0 0 … 0 0 0 0 0 0 0 0 0 0 0 0\n 1 6 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 1 7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 1 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 1 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n ⋮ ⋮ ⋮ ⋱ ⋮ ⋮ ⋮\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 4\n 0 0 0 0 0 0 0 0 0 0 0 0 0 … 0 0 0 0 0 0 0 0 0 0 1 5\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 6\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 7\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 8\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 9\n\n\nThe defining property of a linear model or linear mixed model is that the fitted values are linear combinations of the fixed-effects parameters and the random effects. We can write the fitted values as\n\nm1.X * m1.β + only(m1.reterms) * vec(only(m1.b))\n\n180-element Vector{Float64}:\n 254.22092409297616\n 273.7637217438381\n 293.30651939470016\n 312.8493170455621\n 332.3921146964241\n 351.93491234728606\n 371.47770999814804\n 391.02050764901\n 410.563305299872\n 430.106102950734\n ⋮\n 275.3019966717517\n 287.0799807001881\n 298.85796472862455\n 310.635948757061\n 322.4139327854974\n 334.19191681393386\n 345.96990084237024\n 357.74788487080673\n 369.52586889924316\n\n\n\nfitted(m1) # just to check that these are indeed the same as calculated above\n\n180-element Vector{Float64}:\n 254.22092409297616\n 273.7637217438382\n 293.30651939470016\n 312.84931704556215\n 332.3921146964241\n 351.9349123472861\n 371.47770999814804\n 391.02050764901\n 410.563305299872\n 430.106102950734\n ⋮\n 275.3019966717517\n 287.0799807001881\n 298.85796472862455\n 310.63594875706093\n 322.4139327854974\n 334.1919168139338\n 345.96990084237024\n 357.74788487080673\n 369.5258688992431\n\n\nIn symbols we would write the linear predictor expression as \\[\n\\boldsymbol{\\eta} = \\mathbf{X}\\boldsymbol{\\beta} +\\mathbf{Z b}\n\\] where \\(\\boldsymbol{\\eta}\\) has 180 elements, \\(\\boldsymbol{\\beta}\\) has 2 elements, \\(\\bf b\\) has 36 elements, \\(\\bf X\\) is of size 180 × 2 and \\(\\bf Z\\) is of size 180 × 36.\nFor a linear model or linear mixed model the linear predictor is the mean response, \\(\\boldsymbol\\mu\\). That is, we can write the probability model in terms of a 180-dimensional random variable, \\(\\mathcal Y\\), for the response and a 36-dimensional random variable, \\(\\mathcal B\\), for the random effects as \\[\n\\begin{aligned}\n(\\mathcal{Y} | \\mathcal{B}=\\bf{b}) &\\sim\\mathcal{N}(\\bf{ X\\boldsymbol\\beta + Z b},\\sigma^2\\bf{I})\\\\\\\\\n\\mathcal{B}&\\sim\\mathcal{N}(\\bf{0},\\boldsymbol{\\Sigma}_{\\boldsymbol\\theta}) .\n\\end{aligned}\n\\] where \\(\\boldsymbol{\\Sigma}_\\boldsymbol{\\theta}\\) is a 36 × 36 symmetric covariance matrix that has a special form - it consists of 18 diagonal blocks, each of size 2 × 2 and all the same.\nRecall that this symmetric matrix can be constructed from the parameters \\(\\boldsymbol\\theta\\), which generate the lower triangular matrix \\(\\boldsymbol\\lambda\\), and the estimate \\(\\widehat{\\sigma^2}\\).\n\nm1.θ\n\n3-element Vector{Float64}:\n 0.9292213162629417\n 0.018168381188609965\n 0.22264487194620683\n\n\n\nλ = only(m1.λ) # with multiple grouping factors there will be multiple λ's\n\n2×2 LinearAlgebra.LowerTriangular{Float64, Matrix{Float64}}:\n 0.929221 ⋅ \n 0.0181684 0.222645\n\n\n\nΣ = varest(m1) * (λ * λ')\n\n2×2 Matrix{Float64}:\n 565.511 11.057\n 11.057 32.6821\n\n\nCompare the diagonal elements to the Variance column of\n\nVarCorr(m1)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\nsubj\n(Intercept)\n565.51067\n23.78047\n\n\n\n\ndays\n32.68212\n5.71683\n+0.08\n\n\nResidual\n\n654.94145\n25.59182", + "crumbs": [ + "Generalized linear mixed models" + ] + }, + { + "objectID": "glmm.html#linear-predictors-in-lmms-and-glmms", + "href": "glmm.html#linear-predictors-in-lmms-and-glmms", + "title": "Generalized linear mixed models", + "section": "2 Linear predictors in LMMs and GLMMs", + "text": "2 Linear predictors in LMMs and GLMMs\nWriting the model for \\(\\mathcal Y\\) as \\[\n(\\mathcal{Y} | \\mathcal{B}=\\bf{b})\\sim\\mathcal{N}(\\bf{ X\\boldsymbol\\beta + Z b},\\sigma^2\\bf{I})\n\\] may seem like over-mathematization (or “overkill”, if you prefer) relative to expressions like \\[\ny_i = \\beta_1 x_{i,1} + \\beta_2 x_{i,2}+ b_1 z_{i,1} +\\dots+b_{36} z_{i,36}+\\epsilon_i\n\\] but this more abstract form is necessary for generalizations.\nThe way that I read the first form is\n\n\n\n\n\n\nThe conditional distribution of the response vector, \\(\\mathcal Y\\), given that the random effects vector, \\(\\mathcal B =\\bf b\\), is a multivariate normal (or Gaussian) distribution whose mean, \\(\\boldsymbol\\mu\\), is the linear predictor, \\(\\boldsymbol\\eta=\\bf{X\\boldsymbol\\beta+Zb}\\), and whose covariance matrix is \\(\\sigma^2\\bf I\\). That is, conditional on \\(\\bf b\\), the elements of \\(\\mathcal Y\\) are independent normal random variables with constant variance, \\(\\sigma^2\\), and means of the form \\(\\boldsymbol\\mu = \\boldsymbol\\eta = \\bf{X\\boldsymbol\\beta+Zb}\\).\n\n\n\nSo the only things that differ in the distributions of the \\(y_i\\)’s are the means and they are determined by this linear predictor, \\(\\boldsymbol\\eta = \\bf{X\\boldsymbol\\beta+Zb}\\).", + "crumbs": [ + "Generalized linear mixed models" + ] + }, + { + "objectID": "glmm.html#generalized-linear-mixed-models", + "href": "glmm.html#generalized-linear-mixed-models", + "title": "Generalized linear mixed models", + "section": "3 Generalized Linear Mixed Models", + "text": "3 Generalized Linear Mixed Models\nConsider first a GLMM for a vector, \\(\\bf y\\), of binary (i.e. yes/no) responses. The probability model for the conditional distribution \\(\\mathcal Y|\\mathcal B=\\bf b\\) consists of independent Bernoulli distributions where the mean, \\(\\mu_i\\), for the i’th response is again determined by the i’th element of a linear predictor, \\(\\boldsymbol\\eta = \\mathbf{X}\\boldsymbol\\beta+\\mathbf{Z b}\\).\nHowever, in this case we will run into trouble if we try to make \\(\\boldsymbol\\mu=\\boldsymbol\\eta\\) because \\(\\mu_i\\) is the probability of “success” for the i’th response and must be between 0 and 1. We can’t guarantee that the i’th component of \\(\\boldsymbol\\eta\\) will be between 0 and 1. To get around this problem we apply a transformation to take \\(\\eta_i\\) to \\(\\mu_i\\). For historical reasons this transformation is called the inverse link, written \\(g^{-1}\\), and the opposite transformation - from the probability scale to an unbounded scale - is called the link, g.\nEach probability distribution in the exponential family (which is most of the important ones), has a canonical link which comes from the form of the distribution itself. The details aren’t as important as recognizing that the distribution itself determines a preferred link function.\nFor the Bernoulli distribution, the canonical link is the logit or log-odds function, \\[\n\\eta = g(\\mu) = \\log\\left(\\frac{\\mu}{1-\\mu}\\right),\n\\] (it’s called log-odds because it is the logarithm of the odds ratio, \\(p/(1-p)\\)) and the canonical inverse link is the logistic \\[\n\\mu=g^{-1}(\\eta)=\\frac{1}{1+\\exp(-\\eta)}.\n\\] This is why fitting a binary response is sometimes called logistic regression.\nFor later use we define a Julia logistic function. See this presentation for more information than you could possibly want to know on how Julia converts code like this to run on the processor.\n\nincrement(x) = x + one(x)\nlogistic(η) = inv(increment(exp(-η)))\n\nlogistic (generic function with 1 method)\n\n\nTo reiterate, the probability model for a Generalized Linear Mixed Model (GLMM) is \\[\n\\begin{aligned}\n(\\mathcal{Y} | \\mathcal{B}=\\bf{b}) &\\sim\\mathcal{D}(\\bf{g^{-1}(X\\boldsymbol\\beta + Z b)},\\phi)\\\\\\\\\n\\mathcal{B}&\\sim\\mathcal{N}(\\bf{0},\\Sigma_{\\boldsymbol\\theta}) .\n\\end{aligned}\n\\] where \\(\\mathcal{D}\\) is the distribution family (such as Bernoulli or Poisson), \\(g^{-1}\\) is the inverse link and \\(\\phi\\) is a scale parameter for \\(\\mathcal{D}\\) if it has one. The important cases of the Bernoulli and Poisson distributions don’t have a scale parameter - once you know the mean you know everything you need to know about the distribution. (For those following the presentation, this poem by John Keats is the one with the couplet “Beauty is truth, truth beauty - that is all ye know on earth and all ye need to know.”)\n\n3.1 An example of a Bernoulli GLMM\nThe contra dataset in the MixedModels package is from a survey on the use of artificial contraception by women in Bangladesh.\n\ncontra = DataFrame(dataset(:contra))\n\n1934×5 DataFrame1909 rows omitted\n\n\n\nRow\ndist\nurban\nlivch\nage\nuse\n\n\n\nString\nString\nString\nFloat64\nString\n\n\n\n\n1\nD01\nY\n3+\n18.44\nN\n\n\n2\nD01\nY\n0\n-5.56\nN\n\n\n3\nD01\nY\n2\n1.44\nN\n\n\n4\nD01\nY\n3+\n8.44\nN\n\n\n5\nD01\nY\n0\n-13.56\nN\n\n\n6\nD01\nY\n0\n-11.56\nN\n\n\n7\nD01\nY\n3+\n18.44\nN\n\n\n8\nD01\nY\n3+\n-3.56\nN\n\n\n9\nD01\nY\n1\n-5.56\nN\n\n\n10\nD01\nY\n3+\n1.44\nN\n\n\n11\nD01\nY\n0\n-11.56\nY\n\n\n12\nD01\nY\n0\n-2.56\nN\n\n\n13\nD01\nY\n1\n-4.56\nN\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n1923\nD61\nN\n0\n-11.56\nY\n\n\n1924\nD61\nN\n3+\n1.44\nN\n\n\n1925\nD61\nN\n1\n-5.56\nN\n\n\n1926\nD61\nN\n3+\n14.44\nN\n\n\n1927\nD61\nN\n3+\n19.44\nN\n\n\n1928\nD61\nN\n2\n-9.56\nY\n\n\n1929\nD61\nN\n2\n-2.56\nN\n\n\n1930\nD61\nN\n3+\n14.44\nN\n\n\n1931\nD61\nN\n2\n-4.56\nN\n\n\n1932\nD61\nN\n3+\n14.44\nN\n\n\n1933\nD61\nN\n0\n-13.56\nN\n\n\n1934\nD61\nN\n3+\n10.44\nN\n\n\n\n\n\n\n\ncombine(groupby(contra, :dist), nrow)\n\n60×2 DataFrame35 rows omitted\n\n\n\nRow\ndist\nnrow\n\n\n\nString\nInt64\n\n\n\n\n1\nD01\n117\n\n\n2\nD02\n20\n\n\n3\nD03\n2\n\n\n4\nD04\n30\n\n\n5\nD05\n39\n\n\n6\nD06\n65\n\n\n7\nD07\n18\n\n\n8\nD08\n37\n\n\n9\nD09\n23\n\n\n10\nD10\n13\n\n\n11\nD11\n21\n\n\n12\nD12\n29\n\n\n13\nD13\n24\n\n\n⋮\n⋮\n⋮\n\n\n49\nD49\n4\n\n\n50\nD50\n19\n\n\n51\nD51\n37\n\n\n52\nD52\n61\n\n\n53\nD53\n19\n\n\n54\nD55\n6\n\n\n55\nD56\n45\n\n\n56\nD57\n27\n\n\n57\nD58\n33\n\n\n58\nD59\n10\n\n\n59\nD60\n32\n\n\n60\nD61\n42\n\n\n\n\n\n\nThe information recorded included woman’s age, the number of live children she has, whether she lives in an urban or rural setting, and the political district in which she lives.\nThe age was centered. Unfortunately, the version of the data to which I had access did not record what the centering value was.\nA data plot, Figure 1, shows that the probability of contraception use is not linear in age - it is low for younger women, higher for women in the middle of the range (assumed to be women in late 20’s to early 30’s) and low again for older women (late 30’s to early 40’s in this survey).\nIf we fit a model with only the age term in the fixed effects, that term will not be significant. This doesn’t mean that there is no “age effect”, it only means that there is no significant linear effect for age.\n\n\nCode\ndraw(\n data(\n @transform(\n contra,\n :numuse = Int(:use == \"Y\"),\n :urb = ifelse(:urban == \"Y\", \"Urban\", \"Rural\")\n )\n ) *\n mapping(\n :age => \"Centered age (yr)\",\n :numuse => \"Frequency of contraception use\";\n col=:urb,\n color=:livch,\n ) *\n smooth();\n figure=(; resolution=(800, 450)),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 1: Smoothed relative frequency of contraception use versus centered age for women in the 1989 Bangladesh Fertility Survey\n\n\n\n\n\n\ncontrasts = Dict(\n :dist => Grouping(),\n :urban => HelmertCoding(),\n :livch => DummyCoding(), # default, but no harm in being explicit\n)\nnAGQ = 9\ndist = Bernoulli()\ngm1 = let\n form = @formula(\n use ~ 1 + age + abs2(age) + urban + livch + (1 | dist)\n )\n fit(MixedModel, form, contra, dist; nAGQ, contrasts)\nend\n\nMinimizing 2 Time: 0:00:00 (91.92 ms/it)\n objective: 2396.263956678254\nMinimizing 208 Time: 0:00:01 ( 5.00 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_dist\n\n\n(Intercept)\n-0.6871\n0.1686\n-4.08\n<1e-04\n0.4786\n\n\nage\n0.0035\n0.0092\n0.38\n0.7021\n\n\n\nabs2(age)\n-0.0046\n0.0007\n-6.29\n<1e-09\n\n\n\nurban: Y\n0.3483\n0.0600\n5.81\n<1e-08\n\n\n\nlivch: 1\n0.8152\n0.1622\n5.03\n<1e-06\n\n\n\nlivch: 2\n0.9165\n0.1851\n4.95\n<1e-06\n\n\n\nlivch: 3+\n0.9153\n0.1858\n4.93\n<1e-06\n\n\n\n\n\n\nNotice that the linear term for age is not significant but the quadratic term for age is highly significant. We usually retain the lower order term, even if it is not significant, if the higher order term is significant.\nNotice also that the parameter estimates for the treatment contrasts for livch are similar. Thus the distinction of 1, 2, or 3+ children is not as important as the contrast between having any children and not having any. Those women who already have children are more likely to use artificial contraception.\nFurthermore, the women without children have a different probability vs age profile than the women with children. To allow for this we define a binary children factor and incorporate an age&children interaction.\n\nVarCorr(gm1)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\n\n\ndist\n(Intercept)\n0.22907\n0.47861\n\n\n\n\n\nNotice that there is no “residual” variance being estimated. This is because the Bernoulli distribution doesn’t have a scale parameter.\n\n\n3.2 Convert livch to a binary factor\n\n@transform!(contra, :children = :livch ≠ \"0\")\n# add the associated contrast specifier\ncontrasts[:children] = EffectsCoding()\n\nEffectsCoding(nothing, nothing)\n\n\n\ngm2 = let\n form = @formula(\n use ~\n 1 +\n age * children +\n abs2(age) +\n children +\n urban +\n (1 | dist)\n )\n fit(MixedModel, form, contra, dist; nAGQ, contrasts)\nend\n\nMinimizing 95 Time: 0:00:00 ( 1.06 ms/it)\n objective: 2364.92044533042\nMinimizing 140 Time: 0:00:00 ( 1.06 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_dist\n\n\n(Intercept)\n-0.3615\n0.1275\n-2.84\n0.0046\n0.4757\n\n\nage\n-0.0131\n0.0110\n-1.19\n0.2351\n\n\n\nchildren: true\n0.6055\n0.1035\n5.85\n<1e-08\n\n\n\nabs2(age)\n-0.0058\n0.0008\n-6.89\n<1e-11\n\n\n\nurban: Y\n0.3567\n0.0602\n5.93\n<1e-08\n\n\n\nage & children: true\n0.0342\n0.0127\n2.69\n0.0072\n\n\n\n\n\n\n\n\nCode\nlet\n mods = [gm2, gm1]\n DataFrame(;\n model=[:gm2, :gm1],\n npar=dof.(mods),\n deviance=deviance.(mods),\n AIC=aic.(mods),\n BIC=bic.(mods),\n AICc=aicc.(mods),\n )\nend\n\n\n2×6 DataFrame\n\n\n\nRow\nmodel\nnpar\ndeviance\nAIC\nBIC\nAICc\n\n\n\nSymbol\nInt64\nFloat64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\ngm2\n7\n2364.92\n2379.18\n2418.15\n2379.24\n\n\n2\ngm1\n8\n2372.46\n2388.73\n2433.27\n2388.81\n\n\n\n\n\n\nBecause these models are not nested, we cannot do a likelihood ratio test. Nevertheless we see that the deviance is much lower in the model with age & children even though the 3 levels of livch have been collapsed into a single level of children. There is a substantial decrease in the deviance even though there are fewer parameters in model gm2 than in gm1. This decrease is because the flexibility of the model - its ability to model the behavior of the response - is being put to better use in gm2 than in gm1.\nAt present the calculation of the geomdof as sum(influence(m)) is not correctly defined in our code for a GLMM so we need to do some more work before we can examine those values.\n\n\n3.3 Using urban&dist as a grouping factor\nIt turns out that there can be more difference between urban and rural settings within the same political district than there is between districts. To model this difference we build a model with urban&dist as a grouping factor.\n\ngm3 = let\n form = @formula(\n use ~\n 1 +\n age * children +\n abs2(age) +\n children +\n urban +\n (1 | urban & dist)\n )\n fit(MixedModel, form, contra, dist; nAGQ, contrasts)\nend\n\nMinimizing 90 Time: 0:00:00 ( 1.12 ms/it)\n objective: 2353.829069935298\nMinimizing 144 Time: 0:00:00 ( 1.11 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_urban & dist\n\n\n(Intercept)\n-0.3415\n0.1269\n-2.69\n0.0071\n0.5761\n\n\nage\n-0.0129\n0.0112\n-1.16\n0.2472\n\n\n\nchildren: true\n0.6064\n0.1045\n5.80\n<1e-08\n\n\n\nabs2(age)\n-0.0056\n0.0008\n-6.66\n<1e-10\n\n\n\nurban: Y\n0.3936\n0.0859\n4.58\n<1e-05\n\n\n\nage & children: true\n0.0332\n0.0128\n2.59\n0.0096\n\n\n\n\n\n\n\n\nCode\nlet\n mods = [gm3, gm2, gm1]\n DataFrame(;\n model=[:gm3, :gm2, :gm1],\n npar=dof.(mods),\n deviance=deviance.(mods),\n AIC=aic.(mods),\n BIC=bic.(mods),\n AICc=aicc.(mods),\n )\nend\n\n\n3×6 DataFrame\n\n\n\nRow\nmodel\nnpar\ndeviance\nAIC\nBIC\nAICc\n\n\n\nSymbol\nInt64\nFloat64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\ngm3\n7\n2353.82\n2368.48\n2407.46\n2368.54\n\n\n2\ngm2\n7\n2364.92\n2379.18\n2418.15\n2379.24\n\n\n3\ngm1\n8\n2372.46\n2388.73\n2433.27\n2388.81\n\n\n\n\n\n\nNotice that the parameter count in gm3 is the same as that of gm2 - the thing that has changed is the number of levels of the grouping factor- resulting in a much lower deviance for gm3. This reinforces the idea that a simple count of the number of parameters to be estimated does not always reflect the complexity of the model.\n\ngm2\n\n\n\n\n\nEst.\nSE\nz\np\nσ_dist\n\n\n(Intercept)\n-0.3615\n0.1275\n-2.84\n0.0046\n0.4757\n\n\nage\n-0.0131\n0.0110\n-1.19\n0.2351\n\n\n\nchildren: true\n0.6055\n0.1035\n5.85\n<1e-08\n\n\n\nabs2(age)\n-0.0058\n0.0008\n-6.89\n<1e-11\n\n\n\nurban: Y\n0.3567\n0.0602\n5.93\n<1e-08\n\n\n\nage & children: true\n0.0342\n0.0127\n2.69\n0.0072\n\n\n\n\n\n\n\ngm3\n\n\n\n\n\nEst.\nSE\nz\np\nσ_urban & dist\n\n\n(Intercept)\n-0.3415\n0.1269\n-2.69\n0.0071\n0.5761\n\n\nage\n-0.0129\n0.0112\n-1.16\n0.2472\n\n\n\nchildren: true\n0.6064\n0.1045\n5.80\n<1e-08\n\n\n\nabs2(age)\n-0.0056\n0.0008\n-6.66\n<1e-10\n\n\n\nurban: Y\n0.3936\n0.0859\n4.58\n<1e-05\n\n\n\nage & children: true\n0.0332\n0.0128\n2.59\n0.0096\n\n\n\n\n\n\nThe coefficient for age may be regarded as insignificant but we retain it for two reasons: we have a term of age² (written abs2(age)) in the model and we have a significant interaction age & children in the model.\n\n\n3.4 Predictions for some subgroups\nFor a “typical” district (random effect near zero) the predictions on the linear predictor scale for a woman whose age is near the centering value (i.e. centered age of zero) are:\n\nusing Effects\ndesign = Dict(\n :children => [true, false], :urban => [\"Y\", \"N\"], :age => [0.0]\n)\npreds = effects(design, gm3; invlink=AutoInvLink())\n\n4×7 DataFrame\n\n\n\nRow\nchildren\nage\nurban\nuse: Y\nerr\nlower\nupper\n\n\n\nBool\nFloat64\nString\nFloat64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\ntrue\n0.0\nY\n0.65894\n0.03383\n0.62511\n0.69277\n\n\n2\nfalse\n0.0\nY\n0.364868\n0.0534102\n0.311458\n0.418278\n\n\n3\ntrue\n0.0\nN\n0.46789\n0.0281379\n0.439752\n0.496028\n\n\n4\nfalse\n0.0\nN\n0.207265\n0.0364059\n0.17086\n0.243671", + "crumbs": [ + "Generalized linear mixed models" + ] + }, + { + "objectID": "glmm.html#summarizing-the-results", + "href": "glmm.html#summarizing-the-results", + "title": "Generalized linear mixed models", + "section": "4 Summarizing the results", + "text": "4 Summarizing the results\n\nFrom the data plot we can see a quadratic trend in the probability by age.\nThe patterns for women with children are similar and we do not need to distinguish between 1, 2, and 3+ children.\nWe do distinguish between those women who do not have children and those with children. This shows up in a significant age & children interaction term.", + "crumbs": [ + "Generalized linear mixed models" + ] + }, + { + "objectID": "bootstrap.html", + "href": "bootstrap.html", + "title": "Parametric bootstrap for mixed-effects models", + "section": "", + "text": "The speed of MixedModels.jl relative to its predecessors makes the parametric bootstrap much more computationally tractable. This is valuable because the parametric bootstrap can be used to produce more accurate confidence intervals than methods based on standard errors or profiling of the likelihood surface.\nThis page is adapted from the MixedModels.jl docs\n\n1 The parametric bootstrap\nBootstrapping is a family of procedures for generating sample values of a statistic, allowing for visualization of the distribution of the statistic or for inference from this sample of values. Bootstrapping also belongs to a larger family of procedures called resampling, which are based on creating new samples of data from an existing one, then computing statistics on the new samples, in order to examine the distribution of the relevant statistics.\nA parametric bootstrap is used with a parametric model, m, that has been fit to data. The procedure is to simulate n response vectors from m using the estimated parameter values and refit m to these responses in turn, accumulating the statistics of interest at each iteration.\nThe parameters of a LinearMixedModel object are the fixed-effects parameters, β, the standard deviation, σ, of the per-observation noise, and the covariance parameter, θ, that defines the variance-covariance matrices of the random effects. A technical description of the covariance parameter can be found in the MixedModels.jl docs. Lisa Schwetlick and Daniel Backhaus have provided a more beginner-friendly description of the covariance parameter in the documentation for MixedModelsSim.jl. For today’s purposes – looking at the uncertainty in the estimates from a fitted model – we can simply use values from the fitted model, but we will revisit the parametric bootstrap as a convenient way to simulate new data, potentially with different parameter values, for power analysis.\nAttach the packages to be used\n\n\nCode\nusing AlgebraOfGraphics\nusing CairoMakie\nusing DataFrames\nusing MixedModels\nusing MixedModelsMakie\nusing Random\nusing SMLP2024: dataset\n\nusing AlgebraOfGraphics: AlgebraOfGraphics as AoG\nCairoMakie.activate!(; type=\"svg\") # use SVG (other options include PNG)\n\nimport ProgressMeter\nProgressMeter.ijulia_behavior(:clear);\n\n\nNote that the precise stream of random numbers generated for a given seed can change between Julia versions. For exact reproducibility, you either need to have the exact same Julia version or use the StableRNGs package.\n\n\n2 A model of moderate complexity\nThe kb07 data (Kronmüller & Barr, 2007) are one of the datasets provided by the MixedModels package.\n\nkb07 = dataset(:kb07)\n\nArrow.Table with 1789 rows, 7 columns, and schema:\n :subj String\n :item String\n :spkr String\n :prec String\n :load String\n :rt_trunc Int16\n :rt_raw Int16\n\n\nConvert the table to a DataFrame for summary.\n\nkb07 = DataFrame(kb07)\ndescribe(kb07)\n\n7×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nDataType\n\n\n\n\n1\nsubj\n\nS030\n\nS103\n0\nString\n\n\n2\nitem\n\nI01\n\nI32\n0\nString\n\n\n3\nspkr\n\nnew\n\nold\n0\nString\n\n\n4\nprec\n\nbreak\n\nmaintain\n0\nString\n\n\n5\nload\n\nno\n\nyes\n0\nString\n\n\n6\nrt_trunc\n2182.2\n579\n1940.0\n5171\n0\nInt16\n\n\n7\nrt_raw\n2226.24\n579\n1940.0\n15923\n0\nInt16\n\n\n\n\n\n\nThe experimental factors; spkr, prec, and load, are two-level factors.\n\ncontrasts = Dict(:spkr => EffectsCoding(),\n :prec => EffectsCoding(),\n :load => EffectsCoding(),\n :subj => Grouping(),\n :item => Grouping())\n\nThe EffectsCoding contrast is used with these to create a ±1 encoding. Furthermore, Grouping contrasts are assigned to the subj and item factors. This is not a contrast per-se but an indication that these factors will be used as grouping factors for random effects and, therefore, there is no need to create a contrast matrix. For large numbers of levels in a grouping factor, an attempt to create a contrast matrix may cause memory overflow.\nIt is not important in these cases but a good practice in any case.\nWe can look at an initial fit of moderate complexity:\n\nform = @formula(rt_trunc ~ 1 + spkr * prec * load +\n (1 + spkr + prec + load | subj) +\n (1 + spkr + prec + load | item))\nm0 = fit(MixedModel, form, kb07; contrasts)\n\nMinimizing 2 Time: 0:00:00 ( 0.42 s/it)\n objective: 29353.753287212847\nMinimizing 614 Time: 0:00:01 ( 2.82 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_subj\nσ_item\n\n\n(Intercept)\n2181.6750\n77.3029\n28.22\n<1e-99\n301.8414\n362.1701\n\n\nspkr: old\n67.7490\n18.2824\n3.71\n0.0002\n43.0396\n40.5288\n\n\nprec: maintain\n-333.9206\n47.1621\n-7.08\n<1e-11\n62.1035\n246.9365\n\n\nload: yes\n78.7681\n19.5247\n4.03\n<1e-04\n65.1473\n42.1755\n\n\nspkr: old & prec: maintain\n-21.9634\n15.8062\n-1.39\n0.1647\n\n\n\n\nspkr: old & load: yes\n18.3838\n15.8062\n1.16\n0.2448\n\n\n\n\nprec: maintain & load: yes\n4.5333\n15.8062\n0.29\n0.7743\n\n\n\n\nspkr: old & prec: maintain & load: yes\n23.6052\n15.8062\n1.49\n0.1353\n\n\n\n\nResidual\n668.5061\n\n\n\n\n\n\n\n\n\n\nThe default display in Quarto uses the pretty MIME show method for the model and omits the estimated correlations of the random effects.\nThe VarCorr extractor displays these.\n\nVarCorr(m0)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\nsubj\n(Intercept)\n91108.2428\n301.8414\n\n\n\n\n\n\nspkr: old\n1852.4098\n43.0396\n+0.78\n\n\n\n\n\nprec: maintain\n3856.8408\n62.1035\n-0.59\n+0.02\n\n\n\n\nload: yes\n4244.1689\n65.1473\n+0.36\n+0.82\n+0.53\n\n\nitem\n(Intercept)\n131167.1572\n362.1701\n\n\n\n\n\n\nspkr: old\n1642.5825\n40.5288\n+0.44\n\n\n\n\n\nprec: maintain\n60977.6107\n246.9365\n-0.69\n+0.35\n\n\n\n\nload: yes\n1778.7763\n42.1755\n+0.32\n+0.23\n-0.15\n\n\nResidual\n\n446900.4651\n668.5061\n\n\n\n\n\n\n\n\nNone of the two-factor or three-factor interaction terms in the fixed-effects are significant. In the random-effects terms only the scalar random effects and the prec random effect for item appear to be warranted, leading to the reduced formula\n\n# formula f4 from https://doi.org/10.33016/nextjournal.100002\nform = @formula(rt_trunc ~ 1 + spkr * prec * load + (1 | subj) + (1 + prec | item))\n\nm1 = fit(MixedModel, form, kb07; contrasts)\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n2181.7582\n77.4709\n28.16\n<1e-99\n364.7286\n298.1109\n\n\nspkr: old\n67.8114\n16.0526\n4.22\n<1e-04\n\n\n\n\nprec: maintain\n-333.8582\n47.4629\n-7.03\n<1e-11\n252.6687\n\n\n\nload: yes\n78.6849\n16.0525\n4.90\n<1e-06\n\n\n\n\nspkr: old & prec: maintain\n-21.8802\n16.0525\n-1.36\n0.1729\n\n\n\n\nspkr: old & load: yes\n18.3214\n16.0526\n1.14\n0.2537\n\n\n\n\nprec: maintain & load: yes\n4.4710\n16.0526\n0.28\n0.7806\n\n\n\n\nspkr: old & prec: maintain & load: yes\n23.5219\n16.0525\n1.47\n0.1428\n\n\n\n\nResidual\n678.9318\n\n\n\n\n\n\n\n\n\n\n\nVarCorr(m1)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\nitem\n(Intercept)\n133026.916\n364.729\n\n\n\n\nprec: maintain\n63841.497\n252.669\n-0.70\n\n\nsubj\n(Intercept)\n88870.081\n298.111\n\n\n\nResidual\n\n460948.432\n678.932\n\n\n\n\n\n\nThese two models are nested and can be compared with a likelihood-ratio test.\n\nMixedModels.likelihoodratiotest(m0, m1)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\nrt_trunc ~ 1 + spkr + prec + load + spkr & prec + spkr & load + prec & load + spkr & prec & load + (1 | subj) + (1 + prec | item)\n13\n28658\n\n\n\n\n\nrt_trunc ~ 1 + spkr + prec + load + spkr & prec + spkr & load + prec & load + spkr & prec & load + (1 + spkr + prec + load | subj) + (1 + spkr + prec + load | item)\n29\n28637\n21\n16\n0.1655\n\n\n\n\n\nThe p-value of approximately 20% leads us to prefer the simpler model, m1, to the more complex, m0.\n\n\n3 Bootstrap basics\nTo bootstrap the model parameters, first initialize a random number generator then create a bootstrap sample and extract the table of parameter estimates from it.\n\nconst RNG = MersenneTwister(42)\nsamp = parametricbootstrap(RNG, 5_000, m1)\ntbl = samp.tbl\n\nTable with 18 columns and 5000 rows:\n obj β1 β2 β3 β4 β5 β6 ⋯\n ┌────────────────────────────────────────────────────────────────────\n 1 │ 28691.9 2049.88 71.6398 -268.333 75.1566 -17.8459 -1.90968 ⋯\n 2 │ 28642.6 2197.6 73.0832 -313.409 68.018 -18.062 30.2431 ⋯\n 3 │ 28748.1 2175.11 85.4784 -311.786 48.8342 -27.3483 4.97874 ⋯\n 4 │ 28683.7 2158.48 86.0498 -301.013 71.0117 -45.3329 27.3747 ⋯\n 5 │ 28692.6 2342.6 81.9722 -305.764 78.9518 -37.0137 -5.59876 ⋯\n 6 │ 28691.5 2198.64 90.4229 -387.855 93.4767 -37.2244 3.13351 ⋯\n 7 │ 28687.7 2239.49 74.5619 -364.151 93.9154 -43.441 19.1572 ⋯\n 8 │ 28673.4 2217.89 76.4077 -409.228 52.5047 -11.5777 26.4348 ⋯\n 9 │ 28688.6 2184.89 81.3252 -298.693 89.8482 -5.99902 21.3162 ⋯\n 10 │ 28597.6 2227.36 57.3436 -332.635 102.565 -3.85194 7.23222 ⋯\n 11 │ 28700.8 2131.61 58.1253 -363.018 88.0576 -24.1592 2.26268 ⋯\n 12 │ 28644.8 2145.05 94.7216 -242.292 103.158 -27.2788 27.0664 ⋯\n 13 │ 28678.8 2113.56 55.4491 -382.425 89.8252 -12.9669 -2.74384 ⋯\n 14 │ 28773.5 2123.38 117.383 -258.883 78.8998 -63.4576 26.0778 ⋯\n 15 │ 28566.3 2168.89 52.8287 -289.968 71.5994 -14.7045 23.0165 ⋯\n 16 │ 28764.1 2344.66 54.5199 -356.183 54.3082 -36.4798 32.3478 ⋯\n 17 │ 28547.7 2230.3 56.2337 -329.448 88.8918 -27.7721 23.5031 ⋯\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱\n\n\nAn empirical density plot of the estimates of the residual standard deviation is obtained as\n\nplt = data(tbl) * mapping(:σ) * AoG.density()\ndraw(plt; axis=(;title=\"Parametric bootstrap estimates of σ\"))\n\n\n\n\n\n\n\n\nA density plot of the estimates of the standard deviation of the random effects is obtained as\n\nplt = data(tbl) * mapping(\n [:σ1, :σ2, :σ3] .=> \"Bootstrap replicates of standard deviations\";\n color=dims(1) => renamer([\"Item intercept\", \"Item speaker\", \"Subj\"])\n) * AoG.density()\ndraw(plt; figure=(;supertitle=\"Parametric bootstrap estimates of variance components\"))\n\n\n\n\n\n\n\n\nThe bootstrap sample can be used to generate intervals that cover a certain percentage of the bootstrapped values. We refer to these as “coverage intervals”, similar to a confidence interval. The shortest such intervals, obtained with the shortestcovint extractor, correspond to a highest posterior density interval in Bayesian inference.\nWe generate these for all random and fixed effects:\n\nconfint(samp)\n\nDictTable with 2 columns and 13 rows:\n par lower upper\n ────┬─────────────────────\n β1 │ 2032.48 2340.42\n β2 │ 34.3982 97.8079\n β3 │ -428.09 -244.474\n β4 │ 48.642 110.179\n β5 │ -53.4336 8.42791\n β6 │ -11.3574 51.5299\n β7 │ -27.1799 36.3332\n β8 │ -9.28905 53.5917\n ρ1 │ -0.912786 -0.471123\n σ │ 654.965 700.928\n σ1 │ 265.276 456.967\n σ2 │ 179.402 321.013\n σ3 │ 232.096 359.745\n\n\n\ndraw(\n data(samp.β) * mapping(:β; color=:coefname) * AoG.density();\n figure=(; resolution=(800, 450)),\n)\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\n\nFor the fixed effects, MixedModelsMakie provides a convenience interface to plot the combined coverage intervals and density plots\n\nridgeplot(samp)\n\n\n\n\n\n\n\n\nOften the intercept will be on a different scale and potentially less interesting, so we can stop it from being included in the plot:\n\nridgeplot(samp; show_intercept=false)\n\n\n\n\n\n\n\n\n\n\n4 Singularity\nLet’s consider the classic dysetuff dataset:\n\ndyestuff = dataset(:dyestuff)\nmdye = fit(MixedModel, @formula(yield ~ 1 + (1 | batch)), dyestuff)\n\n\n\n\n\nEst.\nSE\nz\np\nσ_batch\n\n\n(Intercept)\n1527.5000\n17.6946\n86.33\n<1e-99\n37.2603\n\n\nResidual\n49.5101\n\n\n\n\n\n\n\n\n\n\nsampdye = parametricbootstrap(MersenneTwister(1234321), 10_000, mdye)\ntbldye = sampdye.tbl\n\n┌ Warning: NLopt was roundoff limited\n└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/optsummary.jl:160\n┌ Warning: NLopt was roundoff limited\n└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/optsummary.jl:160\n┌ Warning: NLopt was roundoff limited\n└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/optsummary.jl:160\n┌ Warning: NLopt was roundoff limited\n└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/optsummary.jl:160\n┌ Warning: NLopt was roundoff limited\n└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/optsummary.jl:160\n\n\nTable with 5 columns and 10000 rows:\n obj β1 σ σ1 θ1\n ┌────────────────────────────────────────────────\n 1 │ 339.022 1509.13 67.4315 14.312 0.212245\n 2 │ 322.689 1538.08 47.9831 25.5673 0.53284\n 3 │ 324.002 1508.02 50.1346 21.7622 0.434076\n 4 │ 331.887 1538.47 53.2238 41.0559 0.771383\n 5 │ 317.771 1520.62 45.2975 19.1802 0.423428\n 6 │ 315.181 1536.94 36.7556 49.1832 1.33812\n 7 │ 333.641 1519.88 53.8161 46.712 0.867993\n 8 │ 325.729 1528.43 47.8989 37.6367 0.785752\n 9 │ 311.601 1497.46 41.4 15.1257 0.365355\n 10 │ 335.244 1532.65 64.616 0.0 0.0\n 11 │ 327.935 1552.54 57.2036 0.485275 0.00848329\n 12 │ 323.861 1519.28 49.355 24.3703 0.493776\n 13 │ 332.736 1509.04 59.6272 18.2905 0.306747\n 14 │ 328.243 1531.7 51.5431 32.4743 0.630042\n 15 │ 336.186 1536.17 64.0205 15.243 0.238096\n 16 │ 329.468 1526.42 58.6856 0.0 0.0\n 17 │ 320.086 1517.67 43.218 35.9663 0.832207\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮\n\n\n\nplt = data(tbldye) * mapping(:σ1) * AoG.density()\ndraw(plt; axis=(;title=\"Parametric bootstrap estimates of σ_batch\"))\n\n\n\n\n\n\n\n\nNotice that this density plot has a spike, or mode, at zero. Although this mode appears to be diffuse, this is an artifact of the way that density plots are created. In fact, it is a pulse, as can be seen from a histogram.\n\nplt = data(tbldye) * mapping(:σ1) * AoG.histogram(;bins=100)\ndraw(plt; axis=(;title=\"Parametric bootstrap estimates of σ_batch\"))\n\n\n\n\n\n\n\n\nA value of zero for the standard deviation of the random effects is an example of a singular covariance. It is easy to detect the singularity in the case of a scalar random-effects term. However, it is not as straightforward to detect singularity in vector-valued random-effects terms.\nFor example, if we bootstrap a model fit to the sleepstudy data\n\nsleepstudy = dataset(:sleepstudy)\nmsleep = fit(MixedModel, @formula(reaction ~ 1 + days + (1 + days | subj)),\n sleepstudy)\n\n\n\n\n\nEst.\nSE\nz\np\nσ_subj\n\n\n(Intercept)\n251.4051\n6.6323\n37.91\n<1e-99\n23.7805\n\n\ndays\n10.4673\n1.5022\n6.97\n<1e-11\n5.7168\n\n\nResidual\n25.5918\n\n\n\n\n\n\n\n\n\n\nsampsleep = parametricbootstrap(MersenneTwister(666), 10_000, msleep)\ntblsleep = sampsleep.tbl\n\nTable with 10 columns and 10000 rows:\n obj β1 β2 σ σ1 σ2 ρ1 ⋯\n ┌───────────────────────────────────────────────────────────────────\n 1 │ 1721.95 252.488 11.0328 22.4544 29.6185 6.33343 0.233383 ⋯\n 2 │ 1760.85 260.763 8.55352 27.3833 20.8074 4.32877 0.914412 ⋯\n 3 │ 1750.88 246.709 12.4613 25.9951 15.8702 6.33404 0.200358 ⋯\n 4 │ 1777.33 247.683 12.9824 27.7966 27.5413 4.9878 0.121411 ⋯\n 5 │ 1738.05 245.649 10.5792 25.3596 21.5208 4.26131 0.052677 ⋯\n 6 │ 1751.25 255.669 10.1984 26.1432 22.5389 4.58209 0.225968 ⋯\n 7 │ 1727.51 248.986 7.62095 24.6451 19.0858 4.34881 0.212916 ⋯\n 8 │ 1754.18 246.075 11.0469 26.9407 19.8341 4.55961 -0.202146 ⋯\n 9 │ 1757.47 245.407 13.7475 25.8265 20.0014 7.7647 -0.266385 ⋯\n 10 │ 1752.8 253.911 11.4977 25.7077 20.6409 6.27298 0.171494 ⋯\n 11 │ 1707.8 248.887 10.1608 23.9684 10.5923 4.32044 1.0 ⋯\n 12 │ 1773.69 252.542 10.7379 26.8795 27.7956 6.20553 0.156472 ⋯\n 13 │ 1761.27 254.712 11.0373 25.7998 23.2005 7.30831 0.368175 ⋯\n 14 │ 1737.0 260.299 10.5659 24.6504 29.0113 4.26877 -0.078572 ⋯\n 15 │ 1760.12 258.949 10.1464 27.2088 8.03014 7.01905 0.72704 ⋯\n 16 │ 1723.7 249.204 11.7868 24.9861 18.6887 3.08433 0.633218 ⋯\n 17 │ 1734.14 262.586 8.96611 24.001 26.7969 5.376 0.297087 ⋯\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱\n\n\nthe singularity can be exhibited as a standard deviation of zero or as a correlation of ±1.\n\nconfint(sampsleep)\n\nDictTable with 2 columns and 6 rows:\n par lower upper\n ────┬───────────────────\n β1 │ 237.905 264.231\n β2 │ 7.44545 13.3516\n ρ1 │ -0.409926 1.0\n σ │ 22.6345 28.5125\n σ1 │ 10.6755 33.3567\n σ2 │ 3.07053 7.82205\n\n\nA histogram of the estimated correlations from the bootstrap sample has a spike at +1.\n\nplt = data(tblsleep) * mapping(:ρ1) * AoG.histogram(;bins=100)\ndraw(plt; axis=(;title=\"Parametric bootstrap samples of correlation of random effects\"))\n\n\n\n\n\n\n\n\nor, as a count,\n\ncount(tblsleep.ρ1 .≈ 1)\n\n292\n\n\nClose examination of the histogram shows a few values of -1.\n\ncount(tblsleep.ρ1 .≈ -1)\n\n2\n\n\nFurthermore there are even a few cases where the estimate of the standard deviation of the random effect for the intercept is zero.\n\ncount(tblsleep.σ1 .≈ 0)\n\n6\n\n\nThere is a general condition to check for singularity of an estimated covariance matrix or matrices in a bootstrap sample. The parameter optimized in the estimation is θ, the relative covariance parameter. Some of the elements of this parameter vector must be non-negative and, when one of these components is approximately zero, one of the covariance matrices will be singular.\nThe issingular method for a MixedModel object that tests if a parameter vector θ corresponds to a boundary or singular fit.\nThis operation is encapsulated in a method for the issingular function that works on MixedModelBootstrap objects.\n\ncount(issingular(sampsleep))\n\n300\n\n\n\n\n5 References\n\n\nKronmüller, E., & Barr, D. J. (2007). Perspective-free pragmatics: Broken precedents and the recovery-from-preemption hypothesis. Journal of Memory and Language, 56(3), 436–455. https://doi.org/10.1016/j.jml.2006.05.002\n\n\n\n\n\n\n Back to top", + "crumbs": [ + "Bootstrap and profiling", + "Parametric bootstrap for mixed-effects models" + ] + }, + { + "objectID": "sleepstudy.html", + "href": "sleepstudy.html", + "title": "Analysis of the sleepstudy data", + "section": "", + "text": "The sleepstudy data are from a study of the effects of sleep deprivation on response time reported in Balkin et al. (2000) and in Belenky et al. (2003). Eighteen subjects were allowed only 3 hours of time to sleep each night for 9 successive nights. Their reaction time was measured each day, starting the day before the first night of sleep deprivation, when the subjects were on their regular sleep schedule.\n\n\n\n\n\n\nNote\n\n\n\nThis description is inaccurate. In fact the first two days were acclimatization, the third was a baseline and sleep deprivation was only enforced after day 2. To allow for comparison with earlier analyses of these data we retain the old data description for this notebook only.\n\n\n\n1 Loading the data\nFirst attach the MixedModels package and other packages for plotting. The CairoMakie package allows the Makie graphics system (Danisch & Krumbiegel, 2021) to generate high quality static images. Activate that package with the SVG (Scalable Vector Graphics) backend.\n\n\nCode\nusing CairoMakie # graphics back-end\nusing DataFrames\nusing KernelDensity # density estimation\nusing MixedModels\nusing MixedModelsMakie # diagnostic plots\nusing ProgressMeter\nusing Random # random number generators\nusing RCall # call R from Julia\nusing SMLP2024\nusing SMLP2024: dataset\n\nProgressMeter.ijulia_behavior(:clear)\nCairoMakie.activate!(; type=\"svg\")\n\n\nThe sleepstudy data are one of the datasets available with the MixedModels package. It is re-exported by the SMLP2024 package’s dataset function.\n\nsleepstudy = dataset(\"sleepstudy\")\n\nArrow.Table with 180 rows, 3 columns, and schema:\n :subj String\n :days Int8\n :reaction Float64\n\n\nFigure 1 displays the data in a multi-panel plot created with the lattice package in R (Sarkar, 2008), using RCall.jl.\nprint(lattice::xyplot(reaction ~ days | subj,\n $(DataFrame(sleepstudy)),\n aspect=\"xy\",\n layout=c(9,2),\n type=c(\"g\", \"p\", \"r\"),\n index.cond=function(x,y) coef(lm(y ~ x))[1],\n xlab = \"Days of sleep deprivation\",\n ylab = \"Average reaction time (ms)\"\n))\nNULL\n\n\n\n\n\n\n\n\nFigure 1: Average response time versus days of sleep deprivation by subject\n\n\n\n\n\nEach panel shows the data from one subject and a line fit by least squares to that subject’s data. Starting at the lower left panel and proceeding across rows, the panels are ordered by increasing intercept of the least squares line.\nThere are some deviations from linearity within the panels but the deviations are neither substantial nor systematic.\n\n\n2 Fitting an initial model\n\ncontrasts = Dict{Symbol,Any}(:subj => Grouping())\nm1 = let f = @formula(reaction ~ 1 + days + (1 + days | subj))\n fit(MixedModel, f, sleepstudy; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_subj\n\n\n(Intercept)\n251.4051\n6.6323\n37.91\n<1e-99\n23.7805\n\n\ndays\n10.4673\n1.5022\n6.97\n<1e-11\n5.7168\n\n\nResidual\n25.5918\n\n\n\n\n\n\n\n\n\nThis model includes fixed effects for the intercept, representing the typical reaction time at the beginning of the experiment with zero days of sleep deprivation, and the slope w.r.t. days of sleep deprivation. The parameter estimates are about 250 ms. typical reaction time without deprivation and a typical increase of 10.5 ms. per day of sleep deprivation.\nThe random effects represent shifts from the typical behavior for each subject. The shift in the intercept has a standard deviation of about 24 ms. which would suggest a range of about 200 ms. to 300 ms. in the intercepts. Similarly within-subject slopes would be expected to have a range of about 0 ms./day up to 20 ms./day.\nThe random effects for the slope and for the intercept are allowed to be correlated within subject. The estimated correlation, 0.08, is small. This estimate is not shown in the default display above but is shown in the output from VarCorr (variance components and correlations).\n\nVarCorr(m1)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\nsubj\n(Intercept)\n565.51067\n23.78047\n\n\n\n\ndays\n32.68212\n5.71683\n+0.08\n\n\nResidual\n\n654.94145\n25.59182\n\n\n\n\n\n\nTechnically, the random effects for each subject are unobserved random variables and are not “parameters” in the model per se. Hence we do not report standard errors or confidence intervals for these deviations. However, we can produce prediction intervals on the random effects for each subject. Because the experimental design is balanced, these intervals will have the same width for all subjects.\nA plot of the prediction intervals versus the level of the grouping factor (subj, in this case) is sometimes called a caterpillar plot because it can look like a fuzzy caterpillar if there are many levels of the grouping factor. By default, the levels of the grouping factor are sorted by increasing value of the first random effect.\n\n\nCode\ncaterpillar(m1; vline_at_zero=true)\n\n\n\n\n\n\n\n\nFigure 2: Prediction intervals on random effects for model m1\n\n\n\n\n\nFigure 2 reinforces the conclusion that there is little correlation between the random effect for intercept and the random effect for slope.\n\n\n3 A model with uncorrelated random effects\nThe zerocorr function applied to a random-effects term creates uncorrelated vector-valued per-subject random effects.\n\nm2 = let f = @formula reaction ~ 1 + days + zerocorr(1 + days | subj)\n fit(MixedModel, f, sleepstudy; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_subj\n\n\n(Intercept)\n251.4051\n6.7077\n37.48\n<1e-99\n24.1714\n\n\ndays\n10.4673\n1.5193\n6.89\n<1e-11\n5.7994\n\n\nResidual\n25.5561\n\n\n\n\n\n\n\n\n\nAgain, the default display doesn’t show that there is no correlation parameter to be estimated in this model, but the VarCorr display does.\n\nVarCorr(m2)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\nsubj\n(Intercept)\n584.25897\n24.17145\n\n\n\n\ndays\n33.63281\n5.79938\n.\n\n\nResidual\n\n653.11578\n25.55613\n\n\n\n\n\n\nThis model has a slightly lower log-likelihood than does m1 and one fewer parameter than m1. A likelihood-ratio test can be used to compare these nested models.\n\nMixedModels.likelihoodratiotest(m2, m1)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\nreaction ~ 1 + days + zerocorr(1 + days | subj)\n5\n1752\n\n\n\n\n\nreaction ~ 1 + days + (1 + days | subj)\n6\n1752\n0\n1\n0.8004\n\n\n\n\n\nAlternatively, the AIC or BIC values can be compared.\n\n\nCode\nlet mods = [m2, m1]\n Table(;\n model=[:m2, :m1],\n pars=dof.(mods),\n geomdof=(sum ∘ leverage).(mods),\n AIC=aic.(mods),\n BIC=bic.(mods),\n AICc=aicc.(mods),\n )\nend\n\n\nTable with 6 columns and 2 rows:\n model pars geomdof AIC BIC AICc\n ┌────────────────────────────────────────────────\n 1 │ m2 5 29.045 1762.0 1777.97 1762.35\n 2 │ m1 6 28.6115 1763.94 1783.1 1764.42\n\n\nThe goodness of fit measures: AIC, BIC, and AICc, are all on a “smaller is better” scale and, hence, they all prefer m2.\nThe pars column, which is the same as the model-dof column in the likelihood ratio test output, is simply a count of the number of parameters to be estimated when fitting the model. For example, in m2 there are two fixed-effects parameters and three variance components (including the residual variance).\nAn alternative, more geometrically inspired definition of “degrees of freedom”, is the sum of the leverage values, called geomdof in this table.\nInterestingly, the model with fewer parameters, m2, has a greater sum of the leverage values than the model with more parameters, m1. We’re not sure what to make of that.\nIn both cases the sum of the leverage values is toward the upper end of the range of possible values, which is the rank of the fixed-effects model matrix (2) up to the rank of the fixed-effects plus the random effects model matrix (2 + 36 = 38).\n\n\n\n\n\n\nNote\n\n\n\nI think that the upper bound may be 36, not 38, because the two columns of X lie in the column span of Z\n\n\nThis comparison does show, however, that a simple count of the parameters in a mixed-effects model can underestimate, sometimes drastically, the model complexity. This is because a single variance component or multiple components can add many dimensions to the linear predictor.\n\n\n4 Some diagnostic plots\nIn mixed-effects models the linear predictor expression incorporates fixed-effects parameters, which summarize trends for the population or certain well-defined subpopulations, and random effects which represent deviations associated with the experimental units or observational units - individual subjects, in this case. The random effects are modeled as unobserved random variables.\nThe conditional means of these random variables, sometimes called the BLUPs or Best Linear Unbiased Predictors, are not simply the least squares estimates. They are attenuated or shrunk towards zero to reflect the fact that the individuals are assumed to come from a population. A shrinkage plot, Figure 3, shows the BLUPs from the model fit compared to the values without any shrinkage. If the BLUPs are similar to the unshrunk values then the more complicated model accounting for individual differences is supported. If the BLUPs are strongly shrunk towards zero then the additional complexity in the model to account for individual differences is not providing sufficient increase in fidelity to the data to warrant inclusion.\n\n\nCode\nshrinkageplot!(Figure(; resolution=(500, 500)), m1)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 3: Shrinkage plot of means of the random effects in model m1\n\n\n\n\n\n\n\n\n\n\n\nNote\n\n\n\nThis plot could be drawn as shrinkageplot(m1). The reason for explicitly creating a Figure to be modified by shrinkageplot! is to control the resolution.\n\n\nThis plot shows an intermediate pattern. The random effects are somewhat shrunk toward the origin, a model simplification trend, but not completely shrunk - indicating that fidelity to the data is enhanced with these additional coefficients in the linear predictor.\nIf the shrinkage were primarily in one direction - for example, if the arrows from the unshrunk values to the shrunk values were mostly in the vertical direction - then we would get an indication that we could drop the random effect for slope and revert to a simpler model. This is not the case here.\nAs would be expected, the unshrunk values that are further from the origin tend to be shrunk more toward the origin. That is, the arrows that originate furthest from the origin are longer. However, that is not always the case. The arrow in the upper right corner, from S337, is relatively short. Examination of the panel for S337 in the data plot shows a strong linear trend, even though both the intercept and the slope are unusually large. The neighboring panels in the data plot, S330 and S331, have more variability around the least squares line and are subject to a greater amount of shrinkage in the model. (They correspond to the two arrows on the right hand side of the figure around -5 on the vertical scale.)\n\n\n5 Assessing variability by bootstrapping\nThe speed of fitting linear mixed-effects models using MixedModels.jl allows for using simulation-based approaches to inference instead of relying on approximate standard errors. A parametric bootstrap sample for model m is a collection of models of the same form as m fit to data values simulated from m. That is, we pretend that m and its parameter values are the true parameter values, simulate data from these values, and estimate parameters from the simulated data.\nSimulating and fitting a substantial number of model fits, 5000 in this case, takes only a few seconds, following which we extract a data frame of the parameter estimates and plot densities of some of these estimates.\n\nrng = Random.seed!(42) # initialize a random number generator\nm1bstp = parametricbootstrap(rng, 5000, m1)\ntbl = m1bstp.tbl\n\nTable with 10 columns and 5000 rows:\n obj β1 β2 σ σ1 σ2 ρ1 ⋯\n ┌────────────────────────────────────────────────────────────────────\n 1 │ 1717.29 260.712 9.84975 23.4092 15.3317 6.40296 -0.025958 ⋯\n 2 │ 1744.06 262.253 12.3008 25.7047 16.3183 5.54688 0.552607 ⋯\n 3 │ 1714.16 253.149 12.879 22.2753 25.4787 6.1444 0.0691545 ⋯\n 4 │ 1711.54 263.376 11.5798 23.3128 18.8039 4.6557 0.103361 ⋯\n 5 │ 1741.66 248.429 9.39444 25.4355 20.141 5.27357 -0.163599 ⋯\n 6 │ 1754.81 256.794 8.024 26.5087 10.6781 7.14162 0.335471 ⋯\n 7 │ 1777.73 253.388 8.83556 27.8623 17.8325 7.17386 0.00378979 ⋯\n 8 │ 1768.59 254.441 11.4479 27.4033 16.2489 6.67042 0.725394 ⋯\n 9 │ 1753.56 244.906 11.3423 25.6046 25.3607 5.98654 -0.171821 ⋯\n 10 │ 1722.61 257.088 9.18397 23.3386 24.9274 5.18012 0.181143 ⋯\n 11 │ 1738.16 251.262 11.6568 25.7823 17.6663 4.07218 0.25814 ⋯\n 12 │ 1747.76 258.302 12.8015 26.1085 19.2398 5.06066 0.879712 ⋯\n 13 │ 1745.91 254.57 11.8062 24.8863 24.2513 6.14642 0.0126996 ⋯\n 14 │ 1738.8 251.179 10.3226 24.2672 23.7195 6.32645 0.368592 ⋯\n 15 │ 1724.76 238.603 11.5045 25.23 19.0263 3.64038 -0.34657 ⋯\n 16 │ 1777.7 254.133 8.26398 26.9846 26.3715 7.8283 -0.288773 ⋯\n 17 │ 1748.33 251.571 9.5294 26.2927 21.9611 4.31316 -0.150104 ⋯\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱\n\n\nAn empirical density plot of the estimates for the fixed-effects coefficients, Figure 4, shows the normal distribution, “bell-curve”, shape as we might expect.\n\n\nCode\nbegin\n f1 = Figure(; resolution=(1000, 400))\n CairoMakie.density!(\n Axis(f1[1, 1]; xlabel=\"Intercept [ms]\"), tbl.β1\n )\n CairoMakie.density!(\n Axis(f1[1, 2]; xlabel=\"Coefficient of days [ms/day]\"),\n tbl.β2\n )\n f1\nend\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 4: Empirical density plots of bootstrap replications of fixed-effects parameter estimates\n\n\n\n\n\nIt is also possible to create interval estimates of the parameters from the bootstrap replicates. We define the 1-α shortestcovint to be the shortest interval that contains a proportion 1-α (defaults to 95%) of the bootstrap estimates of the parameter.\n\nTable(shortestcovint(m1bstp))\n\nTable with 5 columns and 6 rows:\n type group names lower upper\n ┌──────────────────────────────────────────────────────\n 1 │ β missing (Intercept) 239.64 265.228\n 2 │ β missing days 7.42347 13.1607\n 3 │ σ subj (Intercept) 10.1722 33.0876\n 4 │ σ subj days 2.9948 7.66116\n 5 │ ρ subj (Intercept), days -0.401353 1.0\n 6 │ σ residual missing 22.701 28.5016\n\n\nThe intervals look reasonable except that the upper end point of the interval for ρ1, the correlation coefficient, is 1.0 . It turns out that the estimates of ρ have a great deal of variability.\nBecause there are several values on the boundary (ρ = 1.0) and a pulse like this is not handled well by a density plot, we plot this sample as a histogram, Figure 5.\n\n\nCode\nhist(\n tbl.ρ1;\n bins=40,\n axis=(; xlabel=\"Estimated correlation of the random effects\"),\n figure=(; resolution=(500, 500)),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 5: Histogram of bootstrap replications of the within-subject correlation parameter\n\n\n\n\n\nFinally, density plots for the variance components (but on the scale of the standard deviation), Figure 6, show reasonable symmetry.\n\n\nCode\nbegin\n f2 = Figure(; resolution=(1000, 300))\n CairoMakie.density!(\n Axis(f2[1, 1]; xlabel=\"Residual σ\"),\n tbl.σ,\n )\n CairoMakie.density!(\n Axis(f2[1, 2]; xlabel=\"subj-Intercept σ\"),\n tbl.σ1,\n )\n CairoMakie.density!(\n Axis(f2[1, 3]; xlabel=\"subj-slope σ\"),\n tbl.σ2,\n )\n f2\nend\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 6: Empirical density plots of bootstrap replicates of standard deviation estimates\n\n\n\n\n\nThe estimates of the coefficients, β₁ and β₂, are not highly correlated as shown in a scatterplot of the bootstrap estimates, Figure 7 .\n\nvcov(m1; corr=true) # correlation estimate from the model\n\n2×2 Matrix{Float64}:\n 1.0 -0.137545\n -0.137545 1.0\n\n\n\n\nCode\nlet\n scatter(\n tbl.β1, tbl.β2,\n color=(:blue, 0.20),\n axis=(; xlabel=\"Intercept\", ylabel=\"Coefficient of days\"),\n figure=(; resolution=(500, 500)),\n )\n contour!(kde((tbl.β1, tbl.β2)))\n current_figure()\nend\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 7: Scatter-plot of bootstrap replicates of fixed-effects estimates with contours\n\n\n\n\n\n\n\n6 References\n\n\nBalkin, T., Thome, D., Sing, H., Thomas, M., Redmond, D., Wesensten, N., Williams, J., Hall, S., & Belenky, G. (2000). Effects of sleep schedules on commercial motor vehicle driver performance (DOT-MC-00-133). Federal Motor Carrier Safety Administration. https://doi.org/10.21949/1503015.\n\n\nBelenky, G., Wesensten, N. J., Thorne, D. R., Thomas, M. L., Sing, H. C., Redmond, D. P., Russo, M. B., & Balkin, T. J. (2003). Patterns of performance degradation and restoration during sleep restriction and subsequent recovery: A sleep dose-response study. Journal of Sleep Research, 12(1), 1–12. https://doi.org/10.1046/j.1365-2869.2003.00337.x\n\n\nDanisch, S., & Krumbiegel, J. (2021). Makie.jl: Flexible high-performance data visualization for julia. Journal of Open Source Software, 6(65), 3349. https://doi.org/10.21105/joss.03349\n\n\nSarkar, D. (2008). Lattice: Mutivariate data visualization with r. Springer-Verlag GmbH. https://www.ebook.de/de/product/11429038/deepayan_sarkar_lattice.html\n\n\n\n\n\n\n Back to top", + "crumbs": [ + "Worked examples", + "Analysis of the sleepstudy data" + ] + }, + { + "objectID": "profiling.html", + "href": "profiling.html", + "title": "Confidence intervals from profiled objective", + "section": "", + "text": "Statistical methods that are based on probability models can be used to provide us with a “best guess” of the value of parameters, such as the effect of a particular experimental treatment, in the form of a parameter estimate. In addition, the probability model can be used to assess the uncertainty in the estimate.\nOften the information about the uncertainty is reduced to a single number, a p-value for a test of a null hypothesis, such as the effect being zero, versus the alternative of a non-zero effect. But quoting a single number from a model fit to experimental data, which may have required considerable effort and expense to obtain, will often mean discarding a considerable amount of the information in the data. In the days when computing was expensive and labor-intensive this may have been unavoidable. However, modern computing hardware and software systems provide us with the opportunity of much more intensive evaluation of the uncertainty. At a minimum, instead of focussing solely on the question of whether a coefficient could reasonably be zero, we can formulate confidence intervals on individual parameter estimates or confidence regions on groups of parameters.\nWe have seen the used of a parametric bootstrap to create a sample from the distribution of the estimators of the parameters, and how such samples can be used to create coverage intervals. The bootstrap is based on simulating response vectors from the model that has been fit to the observed data and refitting the same model to these simulated responses.\nIn this section we explore another approach based on refitting the model, keeping the same responses but holding one of the parameters fixed at a specified value.\n\n\nLoad the packages to be used\n\n\nCode\nusing AlgebraOfGraphics\nusing CairoMakie\nusing MixedModels\nusing MixedModelsMakie\nusing Random\nusing SMLP2024: dataset\n\nCairoMakie.activate!(; type=\"svg\")\nimport ProgressMeter\nProgressMeter.ijulia_behavior(:clear)\n\n\nLoad the data and define the contrasts so that the coefficients for each of the experimental variables, load, spkr and prec, are positive.\n\ncontrasts = Dict( # base levels so estimates for speed are positive\n :load => EffectsCoding(; base=\"yes\"),\n :prec => EffectsCoding(; base=\"break\"),\n :spkr => EffectsCoding(; base=\"old\"),\n)\nkb07 = Table(dataset(:kb07))\n\nTable with 7 columns and 1789 rows:\n subj item spkr prec load rt_trunc rt_raw\n ┌───────────────────────────────────────────────────\n 1 │ S030 I01 new break yes 2267 2267\n 2 │ S030 I02 old maintain no 3856 3856\n 3 │ S030 I03 old break no 1567 1567\n 4 │ S030 I04 new maintain no 1732 1732\n 5 │ S030 I05 new break no 2660 2660\n 6 │ S030 I06 old maintain yes 2763 2763\n 7 │ S030 I07 old break yes 3528 3528\n 8 │ S030 I08 new maintain yes 1741 1741\n 9 │ S030 I09 new break yes 3692 3692\n 10 │ S030 I10 old maintain no 1949 1949\n 11 │ S030 I11 old break no 2189 2189\n 12 │ S030 I12 new maintain no 2207 2207\n 13 │ S030 I13 new break no 2078 2078\n 14 │ S030 I14 old maintain yes 1901 1901\n 15 │ S030 I15 old break yes 4015 4015\n 16 │ S030 I16 new maintain yes 1880 1880\n 17 │ S030 I17 new break yes 1444 1444\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮\n\n\nNow we fit and profile a model. The response is defined as 1000 / rt_raw where rt_raw is measured in milliseconds. Thus the response being modeled is the speed measured in responses per second.\n\npr01 = let f = @formula 1000 / rt_raw ~\n 1 + load + spkr + prec + (1 + prec | item) + (1 | subj)\n profile(fit(MixedModel, f, kb07; contrasts))\nend\nprintln(pr01.m) # model is a property of the profile object\n\nMinimizing 2 Time: 0:00:00 ( 0.39 s/it)\n objective: -1585.184301073052\nMinimizing 67 Time: 0:00:01 (22.29 ms/it)\nLinear mixed model fit by maximum likelihood\n :(1000 / rt_raw) ~ 1 + load + spkr + prec + (1 + prec | item) + (1 | subj)\n logLik -2 logLik AIC AICc BIC \n 846.2869 -1692.5738 -1674.5738 -1674.4726 -1625.1691\n\nVariance components:\n Column Variance Std.Dev. Corr.\nitem (Intercept) 0.0061053 0.0781364\n prec: maintain 0.0020476 0.0452502 -0.21\nsubj (Intercept) 0.0054186 0.0736113\nResidual 0.0194484 0.1394577\n Number of obs: 1789; levels of grouping factors: 32, 56\n\n Fixed-effects parameters:\n──────────────────────────────────────────────────────\n Coef. Std. Error z Pr(>|z|)\n──────────────────────────────────────────────────────\n(Intercept) 0.531523 0.0172749 30.77 <1e-99\nload: no 0.0212959 0.00329731 6.46 <1e-09\nspkr: new 0.011218 0.00329732 3.40 0.0007\nprec: maintain 0.0698293 0.00865212 8.07 <1e-15\n──────────────────────────────────────────────────────\n\n\nEvaluation of pr01 is similar to other model fits in these notes except that the call to fit is wrapped in a call to profile. Because the object returned from profile includes the original model fit as its m property, it is not necessary to save the original model fit separately.\n\n\n\nThe information from the profile is encapsulated in a table.\n\npr01.tbl\n\nTable with 15 columns and 249 rows:\n p ζ β1 β2 β3 β4 σ ⋯\n ┌─────────────────────────────────────────────────────────────────────\n 1 │ σ -4.11624 0.531525 0.021298 0.0112154 0.0698319 0.130088 ⋯\n 2 │ σ -3.59106 0.531525 0.0212977 0.0112157 0.0698316 0.131224 ⋯\n 3 │ σ -3.06898 0.531524 0.0212975 0.011216 0.0698313 0.13237 ⋯\n 4 │ σ -2.54996 0.531524 0.0212972 0.0112164 0.069831 0.133526 ⋯\n 5 │ σ -2.03399 0.531524 0.021297 0.0112167 0.0698307 0.134692 ⋯\n 6 │ σ -1.52104 0.531524 0.0212967 0.011217 0.0698303 0.135868 ⋯\n 7 │ σ -1.01107 0.531523 0.0212964 0.0112173 0.06983 0.137054 ⋯\n 8 │ σ -0.504067 0.531523 0.0212961 0.0112177 0.0698297 0.138251 ⋯\n 9 │ σ 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 ⋯\n 10 │ σ 0.501151 0.531523 0.0212956 0.0112184 0.069829 0.140675 ⋯\n 11 │ σ 0.999416 0.531522 0.0212953 0.0112187 0.0698286 0.141904 ⋯\n 12 │ σ 1.49482 0.531522 0.021295 0.0112191 0.0698282 0.143143 ⋯\n 13 │ σ 1.98739 0.531522 0.0212947 0.0112195 0.0698279 0.144392 ⋯\n 14 │ σ 2.47714 0.531521 0.0212944 0.0112199 0.0698275 0.145653 ⋯\n 15 │ σ 2.9641 0.531521 0.021294 0.0112202 0.0698271 0.146925 ⋯\n 16 │ σ 3.4483 0.531521 0.0212937 0.0112206 0.0698267 0.148208 ⋯\n 17 │ σ 3.92976 0.53152 0.0212934 0.011221 0.0698263 0.149502 ⋯\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱\n\n\nEach row of the table summarizes a fit of the original model to the original data but with one of the parameters held fixed. For the first 18 rows of the table, the parameter being held fixed is \\(\\sigma\\), as shown in the p column. In the next set of rows the parameter being held fixed will be \\(\\beta_1\\), the intercept.\nThere are blocks of rows for the fixed-effects (\\(\\boldsymbol{\\beta}\\)) parameters, the variance components (on the scale of a standard deviation), and the \\(\\boldsymbol{\\theta}\\) parameters that generate the covariance factor \\(\\boldsymbol{\\Lambda}_{\\boldsymbol{\\theta}}\\). (At present the correlation parameters are not profiled - we may add them later but that computation is rather awkward.)\n\nshow(unique(pr01.tbl.p))\n\n[:σ, :β1, :β2, :β3, :β4, :θ1, :θ2, :θ3, :θ4, :σ1, :σ2, :σ3]\n\n\nTo reiterate, the first row contains the parameter estimates for this model fit to the original response values with the constraint that \\(\\sigma=0.130088\\), instead of the global estimate \\(\\hat{\\sigma}=0.139458\\) in the row for which \\(\\zeta=0.0\\).\nThe global estimates are included in every block at the row for which \\(\\zeta=0.0\\).\n\nfilter(r -> iszero(r.ζ), pr01.tbl)\n\nTable with 15 columns and 12 rows:\n p ζ β1 β2 β3 β4 σ σ1 ⋯\n ┌──────────────────────────────────────────────────────────────────────────\n 1 │ σ 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 2 │ β1 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 3 │ β2 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 4 │ β3 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 5 │ β4 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 6 │ θ1 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 7 │ θ2 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 8 │ θ3 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 9 │ θ4 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 10 │ σ1 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 11 │ σ2 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 12 │ σ3 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n\n\nThe \\(\\zeta\\) column in this table is a measure of the quality of the fit from the parameters in each row, relative to the global parameter estimates, as measured by the change in the objective (negative twice the log-likelihood).\nThe minimum value for the objective is that at the global parameter estimates. The change in the objective when we constrain one parameter to a particular value has approximately a \\(\\chi^2\\) distribution on 1 degree of freedom, which is the square of a standard normal distribution, \\(\\mathcal{Z}^2\\). We can convert this change in the quality of the fit to the scale of the standard normal distribution by taking the signed square root, which is the square root of the change in the objective with the sign of \\(\\psi-\\hat{\\psi}\\) where \\(\\psi\\) represents the parameter being profiled. This is the value labelled \\(\\zeta\\) in the table.\nTo review:\n\nEach row in the table is the result of re-fitting the original model with the parameter in the p column held fixed at a particular value, as shown in the column for that parameter.\nThe \\(\\zeta\\) column is the signed square root of the change in the objective from the global parameter estimates.\nThus in the block of rows where \\(\\sigma\\) is held fixed, the \\(\\zeta\\) values in rows for which \\(\\sigma<\\hat\\sigma\\) are negative and those for which \\(\\sigma > \\hat\\sigma\\) have positive values of \\(\\zeta\\).\nRows in which \\(\\zeta=0.0\\) are the global parameter estimates.\n\n\n\n\nFigure 1 shows, for each of the fixed effects parameters, \\(\\zeta\\) versus the parameter value.\n\n\nCode\nzetaplot!(Figure(; resolution=(1200, 350)), pr01; ptyp='β')\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 1: ζ versus the value of the coefficient for the fixed-effects parameters in a model of response speed for the kb07 data.\n\n\n\n\n\nThe lines on these panels are read like normal probability plots, i.e. QQ plots against a standard normal distribution. Those on the \\(\\beta_2\\) and \\(\\beta_3\\) panels are, to the resolution of the plot, straight lines which indicates that the estimators of those parameters are normally distributed over the region of interest.\nThe points in the \\(\\beta_1\\) and \\(\\beta_4\\) panels are slightly over-dispersed relative to the straight line, which means that the estimators of these parameters are distributed like a T-distribution with a moderate number of degrees of freedom.\nThe profile-\\(\\zeta\\) function can be used to generate confidence intervals on the parameters\n\nconfint(pr01)\n\nDictTable with 3 columns and 8 rows:\n par estimate lower upper\n ────┬─────────────────────────────────\n β1 │ 0.531523 0.497103 0.565942\n β2 │ 0.0212959 0.0148295 0.0277621\n β3 │ 0.011218 0.00475174 0.0176844\n β4 │ 0.0698293 0.0523046 0.0873562\n σ │ 0.139458 0.13486 0.144322\n σ1 │ 0.0781364 0.0612443 0.103257\n σ2 │ 0.0452502 0.0338521 0.0618819\n σ3 │ 0.0736113 0.0600845 0.0916852\n\n\nas shown in Figure 2, which shows the absolute value of \\(\\zeta\\), which is simply the square root of the difference in the objective, versus the parameter being profiled.\n\n\nCode\nzetaplot!(Figure(; resolution=(1200, 330)), pr01; ptyp='β', absv=true)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 2: Absolute value of ζ versus value of the coefficient for the fixed-effects parameters in a model of response speed for the kb07 data. The horizontal lines are confidence intervals with nominal 50%, 80%, 90%, 95% and 99% confidence.\n\n\n\n\n\nThe 95% confidence intervals are the second horizontal lines from the top in each panel, at 1.96 on the vertical scale.\n\n\nCode\nzetaplot!(Figure(; resolution=(1200, 330)), pr01; ptyp='σ', absv=true)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 3: Absolute value of ζ versus value of the coefficient for the variance component parameters in a model of response speed for the kb07 data. The horizontal lines are confidence intervals with nominal 50%, 80%, 90%, 95% and 99% confidence.\n\n\n\n\n\nFigure 3 shows similar confidence intervals on the parameters representing standard deviations as does Figure 4 for the \\(\\theta\\) parameters.\n\n\nCode\nzetaplot!(Figure(; resolution=(1200, 330)), pr01; ptyp='θ', absv=true)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 4: Absolute value of ζ versus parameter value for the θ parameters in a model of response speed for the kb07 data. The horizontal lines are confidence intervals with nominal 50%, 80%, 90%, 95% and 99% confidence.", + "crumbs": [ + "Bootstrap and profiling", + "Confidence intervals from profiled objective" + ] + }, + { + "objectID": "profiling.html#profiling-a-model-for-the-kb07-data", + "href": "profiling.html#profiling-a-model-for-the-kb07-data", + "title": "Confidence intervals from profiled objective", + "section": "", + "text": "Load the packages to be used\n\n\nCode\nusing AlgebraOfGraphics\nusing CairoMakie\nusing MixedModels\nusing MixedModelsMakie\nusing Random\nusing SMLP2024: dataset\n\nCairoMakie.activate!(; type=\"svg\")\nimport ProgressMeter\nProgressMeter.ijulia_behavior(:clear)\n\n\nLoad the data and define the contrasts so that the coefficients for each of the experimental variables, load, spkr and prec, are positive.\n\ncontrasts = Dict( # base levels so estimates for speed are positive\n :load => EffectsCoding(; base=\"yes\"),\n :prec => EffectsCoding(; base=\"break\"),\n :spkr => EffectsCoding(; base=\"old\"),\n)\nkb07 = Table(dataset(:kb07))\n\nTable with 7 columns and 1789 rows:\n subj item spkr prec load rt_trunc rt_raw\n ┌───────────────────────────────────────────────────\n 1 │ S030 I01 new break yes 2267 2267\n 2 │ S030 I02 old maintain no 3856 3856\n 3 │ S030 I03 old break no 1567 1567\n 4 │ S030 I04 new maintain no 1732 1732\n 5 │ S030 I05 new break no 2660 2660\n 6 │ S030 I06 old maintain yes 2763 2763\n 7 │ S030 I07 old break yes 3528 3528\n 8 │ S030 I08 new maintain yes 1741 1741\n 9 │ S030 I09 new break yes 3692 3692\n 10 │ S030 I10 old maintain no 1949 1949\n 11 │ S030 I11 old break no 2189 2189\n 12 │ S030 I12 new maintain no 2207 2207\n 13 │ S030 I13 new break no 2078 2078\n 14 │ S030 I14 old maintain yes 1901 1901\n 15 │ S030 I15 old break yes 4015 4015\n 16 │ S030 I16 new maintain yes 1880 1880\n 17 │ S030 I17 new break yes 1444 1444\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮\n\n\nNow we fit and profile a model. The response is defined as 1000 / rt_raw where rt_raw is measured in milliseconds. Thus the response being modeled is the speed measured in responses per second.\n\npr01 = let f = @formula 1000 / rt_raw ~\n 1 + load + spkr + prec + (1 + prec | item) + (1 | subj)\n profile(fit(MixedModel, f, kb07; contrasts))\nend\nprintln(pr01.m) # model is a property of the profile object\n\nMinimizing 2 Time: 0:00:00 ( 0.39 s/it)\n objective: -1585.184301073052\nMinimizing 67 Time: 0:00:01 (22.29 ms/it)\nLinear mixed model fit by maximum likelihood\n :(1000 / rt_raw) ~ 1 + load + spkr + prec + (1 + prec | item) + (1 | subj)\n logLik -2 logLik AIC AICc BIC \n 846.2869 -1692.5738 -1674.5738 -1674.4726 -1625.1691\n\nVariance components:\n Column Variance Std.Dev. Corr.\nitem (Intercept) 0.0061053 0.0781364\n prec: maintain 0.0020476 0.0452502 -0.21\nsubj (Intercept) 0.0054186 0.0736113\nResidual 0.0194484 0.1394577\n Number of obs: 1789; levels of grouping factors: 32, 56\n\n Fixed-effects parameters:\n──────────────────────────────────────────────────────\n Coef. Std. Error z Pr(>|z|)\n──────────────────────────────────────────────────────\n(Intercept) 0.531523 0.0172749 30.77 <1e-99\nload: no 0.0212959 0.00329731 6.46 <1e-09\nspkr: new 0.011218 0.00329732 3.40 0.0007\nprec: maintain 0.0698293 0.00865212 8.07 <1e-15\n──────────────────────────────────────────────────────\n\n\nEvaluation of pr01 is similar to other model fits in these notes except that the call to fit is wrapped in a call to profile. Because the object returned from profile includes the original model fit as its m property, it is not necessary to save the original model fit separately.", + "crumbs": [ + "Bootstrap and profiling", + "Confidence intervals from profiled objective" + ] + }, + { + "objectID": "profiling.html#fixing-values-of-parameters", + "href": "profiling.html#fixing-values-of-parameters", + "title": "Confidence intervals from profiled objective", + "section": "", + "text": "The information from the profile is encapsulated in a table.\n\npr01.tbl\n\nTable with 15 columns and 249 rows:\n p ζ β1 β2 β3 β4 σ ⋯\n ┌─────────────────────────────────────────────────────────────────────\n 1 │ σ -4.11624 0.531525 0.021298 0.0112154 0.0698319 0.130088 ⋯\n 2 │ σ -3.59106 0.531525 0.0212977 0.0112157 0.0698316 0.131224 ⋯\n 3 │ σ -3.06898 0.531524 0.0212975 0.011216 0.0698313 0.13237 ⋯\n 4 │ σ -2.54996 0.531524 0.0212972 0.0112164 0.069831 0.133526 ⋯\n 5 │ σ -2.03399 0.531524 0.021297 0.0112167 0.0698307 0.134692 ⋯\n 6 │ σ -1.52104 0.531524 0.0212967 0.011217 0.0698303 0.135868 ⋯\n 7 │ σ -1.01107 0.531523 0.0212964 0.0112173 0.06983 0.137054 ⋯\n 8 │ σ -0.504067 0.531523 0.0212961 0.0112177 0.0698297 0.138251 ⋯\n 9 │ σ 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 ⋯\n 10 │ σ 0.501151 0.531523 0.0212956 0.0112184 0.069829 0.140675 ⋯\n 11 │ σ 0.999416 0.531522 0.0212953 0.0112187 0.0698286 0.141904 ⋯\n 12 │ σ 1.49482 0.531522 0.021295 0.0112191 0.0698282 0.143143 ⋯\n 13 │ σ 1.98739 0.531522 0.0212947 0.0112195 0.0698279 0.144392 ⋯\n 14 │ σ 2.47714 0.531521 0.0212944 0.0112199 0.0698275 0.145653 ⋯\n 15 │ σ 2.9641 0.531521 0.021294 0.0112202 0.0698271 0.146925 ⋯\n 16 │ σ 3.4483 0.531521 0.0212937 0.0112206 0.0698267 0.148208 ⋯\n 17 │ σ 3.92976 0.53152 0.0212934 0.011221 0.0698263 0.149502 ⋯\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱\n\n\nEach row of the table summarizes a fit of the original model to the original data but with one of the parameters held fixed. For the first 18 rows of the table, the parameter being held fixed is \\(\\sigma\\), as shown in the p column. In the next set of rows the parameter being held fixed will be \\(\\beta_1\\), the intercept.\nThere are blocks of rows for the fixed-effects (\\(\\boldsymbol{\\beta}\\)) parameters, the variance components (on the scale of a standard deviation), and the \\(\\boldsymbol{\\theta}\\) parameters that generate the covariance factor \\(\\boldsymbol{\\Lambda}_{\\boldsymbol{\\theta}}\\). (At present the correlation parameters are not profiled - we may add them later but that computation is rather awkward.)\n\nshow(unique(pr01.tbl.p))\n\n[:σ, :β1, :β2, :β3, :β4, :θ1, :θ2, :θ3, :θ4, :σ1, :σ2, :σ3]\n\n\nTo reiterate, the first row contains the parameter estimates for this model fit to the original response values with the constraint that \\(\\sigma=0.130088\\), instead of the global estimate \\(\\hat{\\sigma}=0.139458\\) in the row for which \\(\\zeta=0.0\\).\nThe global estimates are included in every block at the row for which \\(\\zeta=0.0\\).\n\nfilter(r -> iszero(r.ζ), pr01.tbl)\n\nTable with 15 columns and 12 rows:\n p ζ β1 β2 β3 β4 σ σ1 ⋯\n ┌──────────────────────────────────────────────────────────────────────────\n 1 │ σ 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 2 │ β1 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 3 │ β2 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 4 │ β3 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 5 │ β4 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 6 │ θ1 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 7 │ θ2 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 8 │ θ3 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 9 │ θ4 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 10 │ σ1 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 11 │ σ2 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n 12 │ σ3 0.0 0.531523 0.0212959 0.011218 0.0698293 0.139458 0.0781364 ⋯\n\n\nThe \\(\\zeta\\) column in this table is a measure of the quality of the fit from the parameters in each row, relative to the global parameter estimates, as measured by the change in the objective (negative twice the log-likelihood).\nThe minimum value for the objective is that at the global parameter estimates. The change in the objective when we constrain one parameter to a particular value has approximately a \\(\\chi^2\\) distribution on 1 degree of freedom, which is the square of a standard normal distribution, \\(\\mathcal{Z}^2\\). We can convert this change in the quality of the fit to the scale of the standard normal distribution by taking the signed square root, which is the square root of the change in the objective with the sign of \\(\\psi-\\hat{\\psi}\\) where \\(\\psi\\) represents the parameter being profiled. This is the value labelled \\(\\zeta\\) in the table.\nTo review:\n\nEach row in the table is the result of re-fitting the original model with the parameter in the p column held fixed at a particular value, as shown in the column for that parameter.\nThe \\(\\zeta\\) column is the signed square root of the change in the objective from the global parameter estimates.\nThus in the block of rows where \\(\\sigma\\) is held fixed, the \\(\\zeta\\) values in rows for which \\(\\sigma<\\hat\\sigma\\) are negative and those for which \\(\\sigma > \\hat\\sigma\\) have positive values of \\(\\zeta\\).\nRows in which \\(\\zeta=0.0\\) are the global parameter estimates.", + "crumbs": [ + "Bootstrap and profiling", + "Confidence intervals from profiled objective" + ] + }, + { + "objectID": "profiling.html#profile-zeta-plots", + "href": "profiling.html#profile-zeta-plots", + "title": "Confidence intervals from profiled objective", + "section": "", + "text": "Figure 1 shows, for each of the fixed effects parameters, \\(\\zeta\\) versus the parameter value.\n\n\nCode\nzetaplot!(Figure(; resolution=(1200, 350)), pr01; ptyp='β')\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 1: ζ versus the value of the coefficient for the fixed-effects parameters in a model of response speed for the kb07 data.\n\n\n\n\n\nThe lines on these panels are read like normal probability plots, i.e. QQ plots against a standard normal distribution. Those on the \\(\\beta_2\\) and \\(\\beta_3\\) panels are, to the resolution of the plot, straight lines which indicates that the estimators of those parameters are normally distributed over the region of interest.\nThe points in the \\(\\beta_1\\) and \\(\\beta_4\\) panels are slightly over-dispersed relative to the straight line, which means that the estimators of these parameters are distributed like a T-distribution with a moderate number of degrees of freedom.\nThe profile-\\(\\zeta\\) function can be used to generate confidence intervals on the parameters\n\nconfint(pr01)\n\nDictTable with 3 columns and 8 rows:\n par estimate lower upper\n ────┬─────────────────────────────────\n β1 │ 0.531523 0.497103 0.565942\n β2 │ 0.0212959 0.0148295 0.0277621\n β3 │ 0.011218 0.00475174 0.0176844\n β4 │ 0.0698293 0.0523046 0.0873562\n σ │ 0.139458 0.13486 0.144322\n σ1 │ 0.0781364 0.0612443 0.103257\n σ2 │ 0.0452502 0.0338521 0.0618819\n σ3 │ 0.0736113 0.0600845 0.0916852\n\n\nas shown in Figure 2, which shows the absolute value of \\(\\zeta\\), which is simply the square root of the difference in the objective, versus the parameter being profiled.\n\n\nCode\nzetaplot!(Figure(; resolution=(1200, 330)), pr01; ptyp='β', absv=true)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 2: Absolute value of ζ versus value of the coefficient for the fixed-effects parameters in a model of response speed for the kb07 data. The horizontal lines are confidence intervals with nominal 50%, 80%, 90%, 95% and 99% confidence.\n\n\n\n\n\nThe 95% confidence intervals are the second horizontal lines from the top in each panel, at 1.96 on the vertical scale.\n\n\nCode\nzetaplot!(Figure(; resolution=(1200, 330)), pr01; ptyp='σ', absv=true)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 3: Absolute value of ζ versus value of the coefficient for the variance component parameters in a model of response speed for the kb07 data. The horizontal lines are confidence intervals with nominal 50%, 80%, 90%, 95% and 99% confidence.\n\n\n\n\n\nFigure 3 shows similar confidence intervals on the parameters representing standard deviations as does Figure 4 for the \\(\\theta\\) parameters.\n\n\nCode\nzetaplot!(Figure(; resolution=(1200, 330)), pr01; ptyp='θ', absv=true)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 4: Absolute value of ζ versus parameter value for the θ parameters in a model of response speed for the kb07 data. The horizontal lines are confidence intervals with nominal 50%, 80%, 90%, 95% and 99% confidence.", + "crumbs": [ + "Bootstrap and profiling", + "Confidence intervals from profiled objective" + ] + }, + { + "objectID": "contrasts_kwdyz11.html", + "href": "contrasts_kwdyz11.html", + "title": "Contrast Coding of Visual Attention Effects", + "section": "", + "text": "Code\nusing Chain\nusing DataFrames\nusing MixedModels\nusing SMLP2024: dataset\nusing StatsBase\nusing StatsModels\n\nimport ProgressMeter\nProgressMeter.ijulia_behavior(:clear);", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#seqdiffcoding", + "href": "contrasts_kwdyz11.html#seqdiffcoding", + "title": "Contrast Coding of Visual Attention Effects", + "section": "4.1 SeqDiffCoding", + "text": "4.1 SeqDiffCoding\nThe SeqDiffCoding contrast corresponds to MASS::contr.sdif() in R. The assignment of random factors such as Subj to Grouping() is necessary when the sample size is very large. We recommend to include it always, but in this tutorial we do so only in the first example.\n\nm1 = let levels = [\"val\", \"sod\", \"dos\", \"dod\"]\n contrasts = Dict(\n :CTR => SeqDiffCoding(; levels),\n :Subj => Grouping()\n )\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n389.7336\n7.0906\n54.97\n<1e-99\n55.1938\n\n\nCTR: sod\n33.7817\n3.2869\n10.28\n<1e-24\n23.2439\n\n\nCTR: dos\n13.9852\n2.3051\n6.07\n<1e-08\n10.7456\n\n\nCTR: dod\n-2.7469\n2.2140\n-1.24\n0.2147\n9.5064\n\n\nResidual\n69.8349", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#hypothesiscoding", + "href": "contrasts_kwdyz11.html#hypothesiscoding", + "title": "Contrast Coding of Visual Attention Effects", + "section": "4.2 HypothesisCoding", + "text": "4.2 HypothesisCoding\nHypothesisCoding is the most general option available. We can implement all “canned” contrasts ourselves. The next example reproduces the test statistcs from SeqDiffCoding - with a minor modification illustrating the flexibility of going beyond the default version.\n\nm1b = let levels = [\"val\", \"sod\", \"dos\", \"dod\"]\n contrasts = Dict(\n :CTR => HypothesisCoding(\n [\n -1 1 0 0\n 0 -1 1 0\n 0 0 1 -1\n ];\n levels,\n labels=[\"spt\", \"obj\", \"grv\"],\n ),\n )\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n389.7336\n7.0910\n54.96\n<1e-99\n55.1969\n\n\nCTR: spt\n33.7817\n3.2875\n10.28\n<1e-24\n23.2490\n\n\nCTR: obj\n13.9852\n2.3054\n6.07\n<1e-08\n10.7491\n\n\nCTR: grv\n2.7470\n2.2139\n1.24\n0.2147\n9.5049\n\n\nResidual\n69.8349\n\n\n\n\n\n\n\n\n\nThe difference to the preprogrammed SeqDiffCoding is that for the third contrast we changed the direction of the contrast such that the sign of the effect is positive when the result is in agreement with theoretical expectation, that is we subtract the fourth level from the third, not the third level from the fourth.", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#dummycoding", + "href": "contrasts_kwdyz11.html#dummycoding", + "title": "Contrast Coding of Visual Attention Effects", + "section": "4.3 DummyCoding", + "text": "4.3 DummyCoding\nThis contrast corresponds to contr.treatment() in R\n\nm2 = let\n contrasts = Dict(:CTR => DummyCoding(; base=\"val\"))\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n358.0914\n6.1540\n58.19\n<1e-99\n47.9101\n\n\nCTR: dod\n45.0200\n4.3634\n10.32\n<1e-24\n32.2899\n\n\nCTR: dos\n47.7669\n3.5565\n13.43\n<1e-40\n25.5361\n\n\nCTR: sod\n33.7817\n3.2874\n10.28\n<1e-24\n23.2483\n\n\nResidual\n69.8348\n\n\n\n\n\n\n\n\n\nThe DummyCoding contrast has the disadvantage that the intercept returns the mean of the level specified as base, default is the first level, not the GM.", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#ychycaeitcoding", + "href": "contrasts_kwdyz11.html#ychycaeitcoding", + "title": "Contrast Coding of Visual Attention Effects", + "section": "4.4 YchycaeitCoding", + "text": "4.4 YchycaeitCoding\nThe contrasts returned by DummyCoding may be exactly what we want. Can’t we have them, but also have the intercept estimate the GM, rather than the mean of the base level? Yes, we can! We call this “You can have your cake and it eat, too”-Coding (YchycaeitCoding). And we use HypothesisCoding to achieve this outcome.\n\nm2b = let levels = [\"val\", \"sod\", \"dos\", \"dod\"]\n contrasts = Dict(\n :CTR => HypothesisCoding(\n [\n -1 1 0 0\n -1 0 1 0\n -1 0 0 1\n ];\n levels,\n labels=levels[2:end],\n )\n )\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n389.7336\n7.0911\n54.96\n<1e-99\n55.1983\n\n\nCTR: sod\n33.7817\n3.2874\n10.28\n<1e-24\n23.2487\n\n\nCTR: dos\n47.7669\n3.5568\n13.43\n<1e-40\n25.5386\n\n\nCTR: dod\n45.0200\n4.3635\n10.32\n<1e-24\n32.2906\n\n\nResidual\n69.8348\n\n\n\n\n\n\n\n\n\nWe can simply relevel the factor or move the column with -1s for a different base.\n\nm2c = let levels = [\"val\", \"sod\", \"dos\", \"dod\"]\n contrasts = Dict(\n :CTR => HypothesisCoding(\n [\n -1/2 1/2 0 0\n -1/2 0 1/2 0\n -1/2 0 0 1/2\n ];\n levels,\n labels=levels[2:end],\n )\n )\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n389.7336\n7.0930\n54.95\n<1e-99\n55.2127\n\n\nCTR: sod\n16.8909\n1.6437\n10.28\n<1e-24\n11.6245\n\n\nCTR: dos\n23.8835\n1.7781\n13.43\n<1e-40\n12.7664\n\n\nCTR: dod\n22.5100\n2.1818\n10.32\n<1e-24\n16.1459\n\n\nResidual\n69.8349\n\n\n\n\n\n\n\n\n\nWe can simply relevel the factor or move the column with -1s for a different base.", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#effectscoding", + "href": "contrasts_kwdyz11.html#effectscoding", + "title": "Contrast Coding of Visual Attention Effects", + "section": "4.5 EffectsCoding", + "text": "4.5 EffectsCoding\nThis contrast corresponds almost to contr.sum() in R.\n\nm3 = let\n contrasts = Dict(:CTR => EffectsCoding(; base=\"dod\"))\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n389.7336\n7.0906\n54.96\n<1e-99\n55.1943\n\n\nCTR: dos\n16.1248\n1.4403\n11.20\n<1e-28\n7.3306\n\n\nCTR: sod\n2.1396\n1.3337\n1.60\n0.1087\n6.0065\n\n\nCTR: val\n-31.6422\n2.6421\n-11.98\n<1e-32\n19.9491\n\n\nResidual\n69.8349\n\n\n\n\n\n\n\n\n\nThe “almost” qualification refers to the fact that contr.sum() uses the last factor levels as default base level; EffectsCoding uses the first level.\n\nm3b = let levels = [ \"dod\", \"val\", \"sod\", \"dos\"]\n contrasts = Dict(\n :CTR => HypothesisCoding(\n [\n -1/4 3/4 -1/4 -1/4\n -1/4 -1/4 3/4 -1/4\n -1/4 -1/4 -1/4 3/4\n ];\n levels,\n labels=levels[2:end],\n )\n )\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n389.7336\n7.0910\n54.96\n<1e-99\n55.1975\n\n\nCTR: val\n-31.6422\n2.6421\n-11.98\n<1e-32\n19.9493\n\n\nCTR: sod\n2.1396\n1.3338\n1.60\n0.1087\n6.0076\n\n\nCTR: dos\n16.1248\n1.4404\n11.19\n<1e-28\n7.3317\n\n\nResidual\n69.8348\n\n\n\n\n\n\n\n\n\n\nm3c = let levels = [ \"dod\", \"val\", \"sod\", \"dos\"]\n contrasts = Dict(\n :CTR => HypothesisCoding(\n [\n -1/2 3/2 -1/2 -1/2\n -1/2 -1/2 3/2 -1/2\n -1/2 -1/2 -1/2 3/2\n ];\n levels,\n labels=levels[2:end],\n )\n )\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n389.7336\n7.0908\n54.96\n<1e-99\n55.1959\n\n\nCTR: val\n-63.2843\n5.2842\n-11.98\n<1e-32\n39.8982\n\n\nCTR: sod\n4.2791\n2.6676\n1.60\n0.1087\n12.0152\n\n\nCTR: dos\n32.2495\n2.8808\n11.19\n<1e-28\n14.6631\n\n\nResidual\n69.8349", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#helmertcoding", + "href": "contrasts_kwdyz11.html#helmertcoding", + "title": "Contrast Coding of Visual Attention Effects", + "section": "4.6 HelmertCoding", + "text": "4.6 HelmertCoding\nHelmertCoding codes each level as the difference from the average of the lower levels. With the default order of CTR levels we get the following test statistics. These contrasts are othogonal.\n\nm4 = let\n contrasts = Dict(:CTR => HelmertCoding())\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n389.7336\n7.0940\n54.94\n<1e-99\n55.2207\n\n\nCTR: dos\n1.3735\n1.1072\n1.24\n0.2148\n4.7558\n\n\nCTR: sod\n-4.2039\n0.6843\n-6.14\n<1e-09\n3.3493\n\n\nCTR: val\n-10.5474\n0.8807\n-11.98\n<1e-32\n6.6500\n\n\nResidual\n69.8348\n\n\n\n\n\n\n\n\n\n+ HeC1: (2 - 1)/2 # (391 - 358)/2\n+ HeC2: (3 - (2+1)/2)/3 # (405 - (391 + 358)/2)/3\n+ HeC3: (4 - (3+2+1)/3)/4 # (402 - (405 + 391 + 358)/3)/4", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#reverse-helmertcoding", + "href": "contrasts_kwdyz11.html#reverse-helmertcoding", + "title": "Contrast Coding of Visual Attention Effects", + "section": "4.7 Reverse HelmertCoding", + "text": "4.7 Reverse HelmertCoding\nReverse HelmertCoding codes each level as the difference from the average of the higher levels. To estimate these effects we simply reverse the order of factor levels. Of course, the contrasts are also orthogonal.\n\nm4b = let levels = reverse(StatsModels.levels(dat1.CTR))\n contrasts = Dict(:CTR => HelmertCoding(; levels))\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n389.7336\n7.0940\n54.94\n<1e-99\n55.2207\n\n\nCTR: dos\n1.3735\n1.1072\n1.24\n0.2148\n4.7558\n\n\nCTR: sod\n-4.2039\n0.6843\n-6.14\n<1e-09\n3.3493\n\n\nCTR: val\n-10.5474\n0.8807\n-11.98\n<1e-32\n6.6500\n\n\nResidual\n69.8348\n\n\n\n\n\n\n\n\n\n+ HeC1:(3 - 4)/2 # (405 - 402)/2\n+ HeC2:(2 - (3+4)/2)/3 # (391 - (405 + 402)/2)/3\n+ HeC3:(1 - (2+3+4)/3/4 # (356 -(391 + 405 + 402)/3)/4", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#anova-coding", + "href": "contrasts_kwdyz11.html#anova-coding", + "title": "Contrast Coding of Visual Attention Effects", + "section": "4.8 Anova Coding", + "text": "4.8 Anova Coding\nFactorial designs (i.e., lab experiments) are traditionally analyzed with analysis of variance. The test statistics of main effects and interactions are based on an orthogonal set of contrasts. We specify them with HypothesisCoding.\n\n4.8.1 A(2) x B(2)\nAn A(2) x B(2) design can be recast as an F(4) design with the levels (A1-B1, A1-B2, A2-B1, A2-B2). The following contrast specification returns estimates for the main effect of A, the main effect of B, and the interaction of A and B. In a figure With A on the x-axis and the levels of B shown as two lines, the interaction tests the null hypothesis that the two lines are parallel. A positive coefficient implies overadditivity (diverging lines toward the right) and a negative coefficient underadditivity (converging lines).\n\nm5 = let levels = [\"val\", \"sod\", \"dos\", \"dod\"]\n contrasts = Dict(\n :CTR => HypothesisCoding(\n [\n -1 -1 +1 +1 # A\n -1 +1 -1 +1 # B\n +1 -1 -1 +1 # A x B\n ];\n levels,\n labels=[\"A\", \"B\", \"AxB\"],\n ),\n )\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n389.7336\n7.0924\n54.95\n<1e-99\n55.2085\n\n\nCTR: A\n59.0052\n5.1818\n11.39\n<1e-29\n36.2007\n\n\nCTR: B\n31.0348\n4.6751\n6.64\n<1e-10\n31.7147\n\n\nCTR: AxB\n-36.5286\n3.0925\n-11.81\n<1e-31\n16.0032\n\n\nResidual\n69.8348\n\n\n\n\n\n\n\n\n\nIt is also helpful to see the corresponding layout of the four means for the interaction of A and B (i.e., the third contrast)\n B1 B2\n A1 +1 -1\n A2 -1 +1\nThus, interaction tests whether the difference between main diagonal and minor diagonal is different from zero.\n\n\n4.8.2 A(2) x B(2) x C(2)\nGoing beyond the four level factor; it is also helpful to see the corresponding layout of the eight means for the interaction of A and B and C.\n C1 C2\n B1 B2 B1 B2\n A1 +1 -1 A1 -1 +1\n A2 -1 +1 A2 +1 -1\n\n\n4.8.3 A(2) x B(2) x C(3)\nTO BE DONE", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#nested-coding", + "href": "contrasts_kwdyz11.html#nested-coding", + "title": "Contrast Coding of Visual Attention Effects", + "section": "4.9 Nested coding", + "text": "4.9 Nested coding\nNested contrasts are often specified as follow up as post-hoc tests for ANOVA interactions. They are orthogonal. We specify them with HypothesisCoding.\nAn A(2) x B(2) design can be recast as an F(4) design with the levels (A1-B1, A1-B2, A2-B1, A2-B2). The following contrast specification returns an estimate for the main effect of A and the effects of B nested in the two levels of A. In a figure With A on the x-axis and the levels of B shown as two lines, the second contrast tests whether A1-B1 is different from A1-B2 and the third contrast tests whether A2-B1 is different from A2-B2.\n\nm8 = let levels = [\"val\", \"sod\", \"dos\", \"dod\"]\n contrasts = Dict(\n :CTR => HypothesisCoding(\n [\n -1 -1 +1 +1\n -1 +1 0 0\n 0 0 +1 -1\n ];\n levels,\n labels=[\"do_so\", \"spt\", \"grv\"],\n ),\n )\n fit(MixedModel, form, dat1; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n389.7336\n7.0913\n54.96\n<1e-99\n55.1998\n\n\nCTR: do_so\n59.0052\n5.1826\n11.39\n<1e-29\n36.2082\n\n\nCTR: spt\n33.7817\n3.2874\n10.28\n<1e-24\n23.2481\n\n\nCTR: grv\n2.7470\n2.2142\n1.24\n0.2148\n9.5101\n\n\nResidual\n69.8348\n\n\n\n\n\n\n\n\n\nThe three contrasts for one main effect and two nested contrasts are orthogonal. There is no test of the interaction (parallelism).", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#standard-contrasts", + "href": "contrasts_kwdyz11.html#standard-contrasts", + "title": "Contrast Coding of Visual Attention Effects", + "section": "6.1 Standard contrasts", + "text": "6.1 Standard contrasts\nThe most commonly used contrasts are DummyCoding and EffectsCoding (which are similar to contr.treatment() and contr.sum() in R, respectively).", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#exotic-contrasts-rk-well", + "href": "contrasts_kwdyz11.html#exotic-contrasts-rk-well", + "title": "Contrast Coding of Visual Attention Effects", + "section": "6.2 “Exotic” contrasts (rk: well …)", + "text": "6.2 “Exotic” contrasts (rk: well …)\nWe also provide HelmertCoding and SeqDiffCoding (corresponding to base R’s contr.helmert() and MASS::contr.sdif()).", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "contrasts_kwdyz11.html#manual-contrasts", + "href": "contrasts_kwdyz11.html#manual-contrasts", + "title": "Contrast Coding of Visual Attention Effects", + "section": "6.3 Manual contrasts", + "text": "6.3 Manual contrasts\nContrastsCoding()\nThere are two ways to manually specify contrasts. First, you can specify them directly via ContrastsCoding. If you do, it’s good practice to specify the levels corresponding to the rows of the matrix, although they can be omitted in which case they’ll be inferred from the data.\nHypothesisCoding()\nA better way to specify manual contrasts is via HypothesisCoding, where each row of the matrix corresponds to the weights given to the cell means of the levels corresponding to each column (see Schad et al. (2020) for more information).", + "crumbs": [ + "Contrast coding", + "Contrast Coding of Visual Attention Effects" + ] + }, + { + "objectID": "check_emotikon_transform.html", + "href": "check_emotikon_transform.html", + "title": "Transformed and original metrics in Emotikon", + "section": "", + "text": "In Fühner et al. (2021) the original metric of two tasks (Star, S20) is time, but they were transformed to speed scores in the publication prior to computing z-scores. The critical result is the absence of evidence for the age x Sex x Test interaction. Is this interaction significant if we analyse all tasks in their original metric?\nFitting the LMM of the publication takes time, roughly 1 hour. However, if you save the model parameters (and other relevant information), you can restore the fitted model object very quickly. The notebook also illustrates this procedure." + }, + { + "objectID": "check_emotikon_transform.html#getting-the-packages-and-data", + "href": "check_emotikon_transform.html#getting-the-packages-and-data", + "title": "Transformed and original metrics in Emotikon", + "section": "1 Getting the packages and data", + "text": "1 Getting the packages and data\n\n\nCode\nusing AlgebraOfGraphics\nusing Arrow\nusing CairoMakie\nusing DataFrames\nusing DataFrameMacros\nusing MixedModels\nusing MixedModelsMakie\nusing RCall\nusing Serialization\nusing StatsBase\n\nCairoMakie.activate!(; type=\"svg\")\ndatadir = joinpath(@__DIR__, \"data\");\n\n\n\n1.1 Data and figure in publication\n\ndat = DataFrame(Arrow.Table(joinpath(datadir, \"fggk21.arrow\")))\n@transform!(dat, :a1 = :age - 8.5);\nselect!(groupby(dat, :Test), :, :score => zscore => :zScore);\ndescribe(dat)\n\n9×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nDataType\n\n\n\n\n1\nTest\n\nBPT\n\nStar_r\n0\nString\n\n\n2\nCohort\n\n2011\n\n2019\n0\nString\n\n\n3\nSchool\n\nS100043\n\nS800200\n0\nString\n\n\n4\nChild\n\nC002352\n\nC117966\n0\nString\n\n\n5\nSex\n\nfemale\n\nmale\n0\nString\n\n\n6\nage\n8.56073\n7.99452\n8.55852\n9.10609\n0\nFloat64\n\n\n7\nscore\n226.141\n1.14152\n4.65116\n1530.0\n0\nFloat64\n\n\n8\na1\n0.0607297\n-0.505476\n0.0585216\n0.606092\n0\nFloat64\n\n\n9\nzScore\n-3.91914e-13\n-3.1542\n0.00031088\n3.55078\n0\nFloat64\n\n\n\n\n\n\n\n\n1.2 Data and figure with z-scores based on original metric\n\n# dat_om = rcopy(R\"readRDS('./data/fggk21_om.rds')\"); #Don't know what the _om is\n# @transform!(dat_om, :a1 = :age - 8.5);\n# select!(groupby(dat_om, :Test), :, :score => zscore => :zScore);\n# describe(dat_om)" + }, + { + "objectID": "check_emotikon_transform.html#lmms", + "href": "check_emotikon_transform.html#lmms", + "title": "Transformed and original metrics in Emotikon", + "section": "2 LMMs", + "text": "2 LMMs\n\n2.1 Contrasts\n\ncontrasts = Dict(\n :Test => SeqDiffCoding(),\n :Sex => HelmertCoding(),\n :School => Grouping(),\n :Child => Grouping(),\n :Cohort => Grouping(),\n);\n\n\n\n2.2 Formula\n\nf1 = @formula zScore ~\n 1 +\n Test * a1 * Sex +\n (1 + Test + a1 + Sex | School) +\n (1 + Test | Child) +\n zerocorr(1 + Test | Cohort);\n\n\n\n2.3 Restore LMM m1 from publication\n\nCommand for fitting LMM m1 = fit(MixedModel, f1, dat, contrasts=contr)\nFit statistics for LMM m1: Minimizing 5179 Time: 0 Time: 1:00:38 ( 0.70 s/it)\n\n\nm1x = LinearMixedModel(f1, dat; contrasts)\nrestoreoptsum!(m1x, \"./fits/fggk21_m1_optsum.json\")\n\n┌ Warning: optsum was saved with an older version of MixedModels.jl: consider resaving.\n└ @ MixedModels ~/.julia/packages/MixedModels/peUlR/src/serialization.jl:28\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\nσ_School\nσ_Cohort\n\n\n(Intercept)\n-0.0383\n0.0108\n-3.56\n0.0004\n0.5939\n0.2024\n0.0157\n\n\nTest: Run\n-0.0228\n0.0274\n-0.83\n0.4052\n0.8384\n0.3588\n0.0651\n\n\nTest: S20_r\n-0.0147\n0.0405\n-0.36\n0.7171\n0.5825\n0.3596\n0.1107\n\n\nTest: SLJ\n0.0328\n0.0330\n0.99\n0.3198\n0.4127\n0.3027\n0.0896\n\n\nTest: Star_r\n0.0006\n0.0197\n0.03\n0.9763\n0.5574\n0.3620\n0.0313\n\n\na1\n0.2713\n0.0086\n31.63\n<1e-99\n\n0.0966\n\n\n\nSex: male\n0.2064\n0.0024\n86.55\n<1e-99\n\n0.0245\n\n\n\nTest: Run & a1\n-0.4464\n0.0131\n-34.05\n<1e-99\n\n\n\n\n\nTest: S20_r & a1\n0.1473\n0.0114\n12.97\n<1e-37\n\n\n\n\n\nTest: SLJ & a1\n-0.0068\n0.0103\n-0.66\n0.5116\n\n\n\n\n\nTest: Star_r & a1\n0.0761\n0.0111\n6.84\n<1e-11\n\n\n\n\n\nTest: Run & Sex: male\n-0.0900\n0.0037\n-24.10\n<1e-99\n\n\n\n\n\nTest: S20_r & Sex: male\n-0.0912\n0.0032\n-28.23\n<1e-99\n\n\n\n\n\nTest: SLJ & Sex: male\n0.0330\n0.0029\n11.24\n<1e-28\n\n\n\n\n\nTest: Star_r & Sex: male\n-0.0720\n0.0032\n-22.65\n<1e-99\n\n\n\n\n\na1 & Sex: male\n0.0010\n0.0069\n0.14\n0.8876\n\n\n\n\n\nTest: Run & a1 & Sex: male\n-0.0154\n0.0126\n-1.22\n0.2233\n\n\n\n\n\nTest: S20_r & a1 & Sex: male\n0.0129\n0.0109\n1.18\n0.2380\n\n\n\n\n\nTest: SLJ & a1 & Sex: male\n-0.0098\n0.0100\n-0.98\n0.3256\n\n\n\n\n\nTest: Star_r & a1 & Sex: male\n0.0166\n0.0108\n1.54\n0.1241\n\n\n\n\n\nResidual\n0.5880\n\n\n\n\n\n\n\n\n\n\n\n\nVarCorr(m1x)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\n\nChild\n(Intercept)\n0.3527294\n0.5939103\n\n\n\n\n\n\n\n\n\nTest: Run\n0.7029003\n0.8383915\n+0.11\n\n\n\n\n\n\n\n\nTest: S20_r\n0.3393356\n0.5825252\n+0.19\n-0.53\n\n\n\n\n\n\n\nTest: SLJ\n0.1702900\n0.4126621\n+0.05\n-0.14\n-0.29\n\n\n\n\n\n\nTest: Star_r\n0.3107227\n0.5574251\n-0.10\n+0.01\n-0.13\n-0.42\n\n\n\n\nSchool\n(Intercept)\n0.0409640\n0.2023957\n\n\n\n\n\n\n\n\n\nTest: Run\n0.1287690\n0.3588440\n+0.26\n\n\n\n\n\n\n\n\nTest: S20_r\n0.1293351\n0.3596319\n+0.01\n-0.57\n\n\n\n\n\n\n\nTest: SLJ\n0.0916522\n0.3027411\n-0.13\n+0.01\n-0.53\n\n\n\n\n\n\nTest: Star_r\n0.1310575\n0.3620187\n+0.26\n+0.09\n-0.06\n-0.28\n\n\n\n\n\na1\n0.0093412\n0.0966499\n+0.48\n+0.25\n-0.15\n-0.01\n+0.12\n\n\n\n\nSex: male\n0.0005999\n0.0244934\n+0.09\n+0.13\n-0.01\n+0.05\n-0.19\n+0.25\n\n\nCohort\n(Intercept)\n0.0002452\n0.0156587\n\n\n\n\n\n\n\n\n\nTest: Run\n0.0042389\n0.0651068\n.\n\n\n\n\n\n\n\n\nTest: S20_r\n0.0122535\n0.1106954\n.\n.\n\n\n\n\n\n\n\nTest: SLJ\n0.0080210\n0.0895599\n.\n.\n.\n\n\n\n\n\n\nTest: Star_r\n0.0009828\n0.0313498\n.\n.\n.\n.\n\n\n\n\nResidual\n\n0.3456872\n0.5879517\n\n\n\n\n\n\n\n\n\n\n\n\n\n2.4 Restore new LMM m1_om Star and S20 in original metric\n\nCommand for fitting LMM m1_om = fit(MixedModel, f1, dat_om, contrasts=contr)\nMinimizing 10502 Time: 0 Time: 2:09:40 ( 0.74 s/it)\nStore with: julia> saveoptsum(“./fits/fggk21_m1_om_optsum.json”, m1_om)\nOnly for short-term and when desperate: julia> serialize(“./fits/m1_om.jls”, m1_om);\n\n\n2.4.1 … restoreoptsum!()\n\nm1_om = LinearMixedModel(f1, dat; contrasts=contr);\nrestoreoptsum!(m1_om, \"./fits/fggk21_m1_om_optsum.json\");\n\n\n\n2.4.2 … deserialize()\n\nm1x_om = deserialize(\"./fits/m1_om.jls\")\n\n\nVarCorr(m1x_om)\n\n\n\n\n2.5 Residual diagnostics for LMM m1\nResidual plots for published LMM\n\n#scatter(fitted(m1x), residuals(m1x)\n\n\n#qqnorm(m1x)\n\n\n\n2.6 Residual diagnostics for LMM m1_om\nResidual plots for LMM with Star and Speed in original metric.\n\n#scatter(fitted(m1_om_v2), residuals(m1_om_v2)\n\n\n#qqnorm(m1_om_v2)\n\n\n\nFühner, T., Granacher, U., Golle, K., & Kliegl, R. (2021). Age and sex effects in physical fitness components of 108,295 third graders including 515 primary schools and 9 cohorts. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-97000-4" + }, + { + "objectID": "AoGPlots.html", + "href": "AoGPlots.html", + "title": "Creating multi-panel plots", + "section": "", + "text": "This notebook shows creating a multi-panel plot similar to Figure 2 of Fühner et al. (2021).\nThe data are available from the SMLP2024 example datasets.\n\n\nCode\nusing Arrow\nusing AlgebraOfGraphics\nusing CairoMakie # for displaying static plots\nusing DataFrames\nusing Statistics\nusing StatsBase\nusing SMLP2024: dataset\n\nCairoMakie.activate!(; type=\"svg\") # use SVG (other options include PNG)\n\n\n\ntbl = dataset(\"fggk21\")\n\nArrow.Table with 525126 rows, 7 columns, and schema:\n :Cohort String\n :School String\n :Child String\n :Sex String\n :age Float64\n :Test String\n :score Float64\n\n\n\ntypeof(tbl)\n\nArrow.Table\n\n\n\ndf = DataFrame(tbl)\ntypeof(df)\n\nDataFrame\n\n\n\n1 Creating a summary data frame\nThe response to be plotted is the mean score by Test and Sex and age, rounded to the nearest 0.1 years.\nThe first task is to round the age to 1 digit after the decimal place, which can be done with select applied to a DataFrame. In some ways this is the most complicated expression in creating the plot so we will break it down. select is applied to DataFrame(dat), which is the conversion of the Arrow.Table, dat, to a DataFrame. This is necessary because an Arrow.Table is immutable but a DataFrame can be modified.\nThe arguments after the DataFrame describe how to modify the contents. The first : indicates that all the existing columns should be included. The other expression can be pairs (created with the => operator) of the form :col => function or of the form :col => function => :newname. (See the documentation of the DataFrames package for details.)\nIn this case the function is an anonymous function of the form round.(x, digits=1) where “dot-broadcasting” is used to apply to the entire column (see this documentation for details).\n\ntransform!(df, :age, :age => (x -> x .- 8.5) => :a1) # centered age (linear)\nselect!(groupby(df, :Test), :, :score => zscore => :zScore) # z-score\ntlabels = [ # establish order and labels of tbl.Test\n \"Run\" => \"Endurance\",\n \"Star_r\" => \"Coordination\",\n \"S20_r\" => \"Speed\",\n \"SLJ\" => \"PowerLOW\",\n \"BPT\" => \"PowerUP\",\n];\n\nThe next stage is a group-apply-combine operation to group the rows by Sex, Test and rnd_age then apply mean to the zScore and also apply length to zScore to record the number in each group.\n\ndf2 = combine(\n groupby(\n select(df, :, :age => ByRow(x -> round(x; digits=1)) => :age),\n [:Sex, :Test, :age],\n ),\n :zScore => mean => :zScore,\n :zScore => length => :n,\n)\n\n120×5 DataFrame95 rows omitted\n\n\n\nRow\nSex\nTest\nage\nzScore\nn\n\n\n\nString\nString\nFloat64\nFloat64\nInt64\n\n\n\n\n1\nmale\nS20_r\n8.0\n-0.0265138\n1223\n\n\n2\nmale\nBPT\n8.0\n0.026973\n1227\n\n\n3\nmale\nSLJ\n8.0\n0.121609\n1227\n\n\n4\nmale\nStar_r\n8.0\n-0.0571726\n1186\n\n\n5\nmale\nRun\n8.0\n0.292695\n1210\n\n\n6\nfemale\nS20_r\n8.0\n-0.35164\n1411\n\n\n7\nfemale\nBPT\n8.0\n-0.610355\n1417\n\n\n8\nfemale\nSLJ\n8.0\n-0.279872\n1418\n\n\n9\nfemale\nStar_r\n8.0\n-0.268221\n1381\n\n\n10\nfemale\nRun\n8.0\n-0.245573\n1387\n\n\n11\nmale\nS20_r\n8.1\n0.0608397\n3042\n\n\n12\nmale\nBPT\n8.1\n0.0955413\n3069\n\n\n13\nmale\nSLJ\n8.1\n0.123099\n3069\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n109\nmale\nStar_r\n9.0\n0.254973\n4049\n\n\n110\nmale\nRun\n9.0\n0.258082\n4034\n\n\n111\nfemale\nS20_r\n9.1\n-0.0286172\n1154\n\n\n112\nfemale\nBPT\n9.1\n-0.0752301\n1186\n\n\n113\nfemale\nSLJ\n9.1\n-0.094587\n1174\n\n\n114\nfemale\nStar_r\n9.1\n0.00276252\n1162\n\n\n115\nfemale\nRun\n9.1\n-0.235591\n1150\n\n\n116\nmale\nS20_r\n9.1\n0.325745\n1303\n\n\n117\nmale\nBPT\n9.1\n0.616416\n1320\n\n\n118\nmale\nSLJ\n9.1\n0.267577\n1310\n\n\n119\nmale\nStar_r\n9.1\n0.254342\n1297\n\n\n120\nmale\nRun\n9.1\n0.251045\n1294\n\n\n\n\n\n\n\n\n2 Creating the plot\nThe AlgebraOfGraphics package applies operators to the results of functions such as data (specify the data table to be used), mapping (designate the roles of columns), and visual (type of visual presentation).\n\nlet\n design = mapping(:age, :zScore; color=:Sex, col=:Test)\n lines = design * linear()\n means = design * visual(Scatter; markersize=5)\n draw(data(df2) * means + data(df) * lines)\nend\n\n\n\n\n\n\n\n\n\nTBD: Relabel factor levels (Boys, Girls; fitness components for Test)\nTBD: Relevel factors; why not levels from Tables?\nTBD: Set range (7.8 to 9.2 and tick marks (8, 8.5, 9) of axes.\nTBD: Move legend in plot?\n\n\n\nFühner, T., Granacher, U., Golle, K., & Kliegl, R. (2021). Age and sex effects in physical fitness components of 108,295 third graders including 515 primary schools and 9 cohorts. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-97000-4\n\n\n\n\n\n\n Back to top", + "crumbs": [ + "Visualizations and diagnostics", + "Creating multi-panel plots" + ] + }, + { + "objectID": "pkg.html", + "href": "pkg.html", + "title": "Package management and reproducible environments", + "section": "", + "text": "Julius Krumbiegel also has a great blog post with more details on Julia environments.\nJulia packages can be configured (in a file called Project.toml) on a per-project basis. The packaged sources and compiled versions are stored in a central location, e.g. ~/.julia/packages and ~/.julia/compiled on Linux systems, but the configuration of packages to be used can be local to a project. The Pkg package is used to modify the local project’s configuration. (An alternative is “package mode” in the read-eval-print-loop or REPL, which we will show at the summer school.) Start julia in the directory of the cloned SMLP2024 repository\n\nusing Pkg # there's a package called 'Pkg' to manipulate package configs\nPkg.activate(\".\")# activate the current directory as the project\n\nIf you’ve received an environment from someone/somewhere else – such as this course repository – then you’ll need to first “instantiate” it (i.e., install all the dependencies).\n\nPkg.instantiate()# only needed the first time you work in a project\nPkg.update() # get the latest package versions compatible with the project\n\n\nPkg.status()\n\nOccasionally the Pkg.status function call will give info about new versions being available but blocked by requirements of other packages. This is to be expected - the package system is large and the web of dependencies are complex. Generally the Julia package system is very good at resolving dependencies.\n\n\n\n Back to top", + "crumbs": [ + "Getting started with Julia", + "Package management and reproducible environments" + ] + }, + { + "objectID": "index.html", + "href": "index.html", + "title": "Seventh Summer School on Statistical Methods for Linguistics and Psychology", + "section": "", + "text": "This site provides materials for the Advanced frequentist methods stream of the Summer School on Statistical Methods to be held at the University of Potsdam, 11-15 September, 2024." + }, + { + "objectID": "index.html#git", + "href": "index.html#git", + "title": "Seventh Summer School on Statistical Methods for Linguistics and Psychology", + "section": "1.1 git", + "text": "1.1 git\nWe will assume that you have git installed and are able to clone a repository from github. If not, Happy Git with R is a good place to learn about git for data science.\nThis website is built using quarto, described below, from the repository. Clone this repository with, e.g.\ngit clone https://github.com/RePsychLing/SMLP2024" + }, + { + "objectID": "index.html#julia-programming-language", + "href": "index.html#julia-programming-language", + "title": "Seventh Summer School on Statistical Methods for Linguistics and Psychology", + "section": "1.2 Julia Programming Language", + "text": "1.2 Julia Programming Language\nWe will use Julia v1.9 in the summer school. We recommend using Juliaup to install and manage Julia versions. Juliaup makes it trivial to upgrade to new Julia releases or even use old ones. Alternatively, you can download the version appropriate for your setup from here: Julia Programming Language" + }, + { + "objectID": "index.html#visual-studio-code-vs-code", + "href": "index.html#visual-studio-code-vs-code", + "title": "Seventh Summer School on Statistical Methods for Linguistics and Psychology", + "section": "1.3 Visual Studio Code (VS Code)", + "text": "1.3 Visual Studio Code (VS Code)\nWe will use VS Code IDE, that is Julia : VS Code ~ R : RStudio. You can download the version appropriate for your setup from here: VS Code" + }, + { + "objectID": "index.html#quarto", + "href": "index.html#quarto", + "title": "Seventh Summer School on Statistical Methods for Linguistics and Psychology", + "section": "1.4 Quarto", + "text": "1.4 Quarto\nThe web site and other documents for this course are rendered using a knitr-like system called Quarto. You can download the version appropriate for your setup from here: quarto" + }, + { + "objectID": "about.html", + "href": "about.html", + "title": "About", + "section": "", + "text": "About this site\n\n\n\n Back to top" + }, + { + "objectID": "useful_packages.html", + "href": "useful_packages.html", + "title": "Useful packages", + "section": "", + "text": "Unlike R, Julia does not immediately expose a huge number of functions, but instead requires loading packages (whether from the standard library or from the broader package ecosystem) for a lot of relevant functionality for statistical analysis. There are technical reasons for this, such as the ease of using the Julia package system. One further motivation is that Julia is aimed at a broader “technical computing” audience (like MATLAB or perhaps Python) and less at a “statistical analysis” audience.\nThis has two important implications:\nThis notebook is not intended to be an exhaustive list of packages, but rather to highlight a few packages that I suspect will be particularly useful. Before getting onto the packages, I have one final hint: take advantage of how easy and first-class package management in Julia is. Having good package management makes reproducible analyses much easier and avoids breaking old analyses when you start a new one. The package-manager REPL mode (activated by typing ] at the julia> prompt) is very useful.", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#reading-data", + "href": "useful_packages.html#reading-data", + "title": "Useful packages", + "section": "1.1 Reading data", + "text": "1.1 Reading data\n\nArrow.jl a high performance format for data storage, accessible in R via the arrow package and in Python via pyarrow. (Confusingly, the function for reading and writing Arrow format files in R is called read_feather and write_feather, but the modern Arrow format is distinct from the older Feather format provided by the feather package.) This is the format that we store the example and test datasets in for MixedModels.jl.\nCSV.jl useful for reading comma-separated values, tab-separated values and basically everything handled by the read.csv and read.table family of functions in R.\n\nNote that by default both Arrow.jl and CSV.jl do not return a DataFrame, but rather “column tables” – named tuples of column vectors.", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#dataframes", + "href": "useful_packages.html#dataframes", + "title": "Useful packages", + "section": "1.2 DataFrames", + "text": "1.2 DataFrames\nUnlike in R, DataFrames are not part of the base language, nor the standard library.\nDataFrames.jl provides the basic infrastructure around DataFrames, as well as its own mini language for doing the split-apply-combine approach that underlies R’s dplyr and much of the tidyverse. The DataFrames.jl documentation is the place for looking at how to e.g. read in a CSV or Arrow file as a DataFrame. Note that DataFrames.jl by default depends on CategoricalArrays.jl to handle the equivalent of factor in the R world, but there is an alternative package for factor-like array type in Julia, PooledArrays.jl. PooledArrays are simpler, but more limited than CategoricalArrays and we (Phillip and Doug) sometimes use them in our examples and simulations. The tables produced by reading an Arrow file have their own representation of factor-like data as DictEncoded arrays.\nDataFrame.jl’s mini language can be a bit daunting, if you’re used to manipulations in the style of base R or the tidyverse. For that, there are several options; recently, we’e had particularly nice experiences with DataFrameMacros.jl and Chain.jl for a convenient syntax to connect or “pipe” together successive operations. It’s your choice whether and which of these add-ons you want to use! Phillip tends to write his code using raw DataFrames.jl, but Doug really enjoys DataFrameMacros.jl.\nThe recently added Tidier collection of Julia packages is popular with those coming from the tidyverse.", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#regression", + "href": "useful_packages.html#regression", + "title": "Useful packages", + "section": "1.3 Regression", + "text": "1.3 Regression\nUnlike in R, neither formula processing nor basic regression are part of the base language or the standard library.\nThe formula syntax and basic contrast-coding schemes in Julia is provided by StatsModels.jl. By default, MixedModels.jl re-exports the @formula macro and most commonly used contrast schemes from StatsModels.jl, so you often don’t have to worry about loading StatsModels.jl directly. The same is true for GLM.jl, which provides basic linear and generalized linear models, such as ordinary least squares (OLS) regression and logistic regression, i.e. the classical, non mixed regression models.\nThe basic functionality looks quite similar to R, e.g.\njulia > lm(@formula(y ~ 1 + x), data)\njulia > glm(@formula(y ~ 1 + x), data, Binomial(), LogitLink())\nbut the more general modelling API (also used by MixedModels.jl) is also supported:\njulia > fit(LinearModel, @formula(y ~ 1 + x), mydata)\njulia > fit(\n GeneralizedLinearModel,\n @formula(y ~ 1 + x),\n data,\n Binomial(),\n LogitLink(),\n)\n(You can also specify your model matrices directly and skip the formula interface, but we don’t recommend this as it’s easy to mess up in really subtle but very problematic ways.)", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#formula-macros-and-domain-specific-languages", + "href": "useful_packages.html#formula-macros-and-domain-specific-languages", + "title": "Useful packages", + "section": "1.4 @formula, macros and domain-specific languages", + "text": "1.4 @formula, macros and domain-specific languages\nAs a sidebar: why is @formula a macro and not a normal function? Well, that’s because formulas are essentially their own domain-specific language (a variant of Wilkinson-Rogers notation) and macros are used for manipulating the language itself – or in this case, handling an entirely new, embedded language! This is also why macros are used by packages like Turing.jl and Soss.jl that define a language for Bayesian probabilistic programming like PyMC3 or Stan.", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#extensions-to-the-formula-syntax", + "href": "useful_packages.html#extensions-to-the-formula-syntax", + "title": "Useful packages", + "section": "1.5 Extensions to the formula syntax", + "text": "1.5 Extensions to the formula syntax\nThere are several ongoing efforts to extend the formula syntax to include some of the “extras” available in R, e.g. RegressionFormulae.jl to use the caret (^) notation to limit interactions to a certain order ((a+b+c)^2 generates a + b + c + a&b + a&c + b&c, but not a&b&c). Note also that Julia uses & to express interactions, not : like in R.", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#standardizing-predictors", + "href": "useful_packages.html#standardizing-predictors", + "title": "Useful packages", + "section": "1.6 Standardizing Predictors", + "text": "1.6 Standardizing Predictors\nAlthough function calls such as log can be used within Julia formulae, they must act on a rowwise basis, i.e. on observations. Transformations such as z-scoring or centering (often done with scale in R) require knowledge of the entire column. StandardizedPredictors.jl provides functions for centering, scaling, and z-scoring within the formula. These are treated as pseudo-contrasts and computed on demand, meaning that predict and effects (see next) computations will handle these transformations on new data (e.g. centering new data around the mean computed during fitting the original data) correctly and automatically.", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#effects", + "href": "useful_packages.html#effects", + "title": "Useful packages", + "section": "1.7 Effects", + "text": "1.7 Effects\nJohn Fox’s effects package in R (and the related ggeffects package for plotting these using ggplot2) provides a nice way to visualize a model’s overall view of the data. This functionality is provided by Effects.jl and works out-of-the-box with most regression model packages in Julia (including MixedModels.jl). Support for formulae with embedded functions (such as log) is not yet complete, but we’re working on it!", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#estimated-marginal-least-square-means", + "href": "useful_packages.html#estimated-marginal-least-square-means", + "title": "Useful packages", + "section": "1.8 Estimated Marginal / Least Square Means", + "text": "1.8 Estimated Marginal / Least Square Means\nEffects.jl provides a subset of the functionality (basic estimated-marginal means and exhaustive pairwise comparisons) of the R package emmeans package. However, it is often better to use sensible, hypothesis-driven contrast coding than to compute all pairwise comparisons after the fact. 😃", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#makie", + "href": "useful_packages.html#makie", + "title": "Useful packages", + "section": "3.1 Makie", + "text": "3.1 Makie\nThe Makie ecosystem is a relatively new take on graphics that aims to be both powerful and easy to use. Makie.jl itself only provides abstract definitions for many components (and is used in e.g. MixedModelsMakie.jl to define plot types for MixedModels.jl). The actual plotting and rendering is handled by a backend package such as CairoMakie.jl (good for Quarto notebooks or rending static 2D images) and GLMakie.jl (good for dynamic, interactive visuals and 3D images). AlgebraOfGraphics.jl builds a grammar of graphics upon the Makie framework. It’s a great way to get good plots very quickly, but extensive customization is still best achieved by using Makie directly.", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#plots.jl", + "href": "useful_packages.html#plots.jl", + "title": "Useful packages", + "section": "3.2 Plots.jl", + "text": "3.2 Plots.jl\nPlots.jl is the original plotting package in Julia, but we often find it difficult to work with compared to some of the other alternatives. StatsPlots.jl builds on this, adding common statistical plots, while UnicodePlots.jl renders plots as Unicode characters directly in the REPL.\nPGFPlotsX.jl is a very new package that writes directly to PGF (the format used by LaTeX’s tikz framework) and can stand alone or be used as a rendering backend for the Plots.jl ecosystem.", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#gadfly", + "href": "useful_packages.html#gadfly", + "title": "Useful packages", + "section": "3.3 Gadfly", + "text": "3.3 Gadfly\nGadfly.jl was the original attempt to create a plotting system in Julia based on the grammar of graphics (the “gg” in ggplot2). Development has largely stalled, but some functionality still exceeds AlgebraOfGraphics.jl, which has taken up the grammar of graphics mantle. Notably, the MixedModels.jl documentation still uses Gadfly as of this writing (early September 2021).", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "useful_packages.html#others", + "href": "useful_packages.html#others", + "title": "Useful packages", + "section": "3.4 Others", + "text": "3.4 Others\nThere are many other graphics packages available in Julia, often wrapping well-established frameworks such as VegaLite.", + "crumbs": [ + "Getting started with Julia", + "Useful packages" + ] + }, + { + "objectID": "selection.html", + "href": "selection.html", + "title": "SMLP2024: Advanced Frequentist Track", + "section": "", + "text": "Code\nusing Arrow\nusing CairoMakie\nusing DataFrames\nCairoMakie.activate!(; type=\"svg\") # use SVG (other options include PNG)\ntbl = Arrow.Table(\"./data/fggk21.arrow\")\n\nArrow.Table with 525126 rows, 7 columns, and schema:\n :Cohort String\n :School String\n :Child String\n :Sex String\n :age Float64\n :Test String\n :score Float64\ndf = DataFrame(tbl)\ndescribe(df)\n\n7×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nDataType\n\n\n\n\n1\nCohort\n\n2011\n\n2019\n0\nString\n\n\n2\nSchool\n\nS100043\n\nS800200\n0\nString\n\n\n3\nChild\n\nC002352\n\nC117966\n0\nString\n\n\n4\nSex\n\nfemale\n\nmale\n0\nString\n\n\n5\nage\n8.56073\n7.99452\n8.55852\n9.10609\n0\nFloat64\n\n\n6\nTest\n\nBPT\n\nStar_r\n0\nString\n\n\n7\nscore\n226.141\n1.14152\n4.65116\n1530.0\n0\nFloat64" + }, + { + "objectID": "selection.html#raw-score-density", + "href": "selection.html#raw-score-density", + "title": "SMLP2024: Advanced Frequentist Track", + "section": "1 Raw score density", + "text": "1 Raw score density\n\nlet\n fdensity = Figure(; resolution=(1000, 500))\n axs = Axis(fdensity[1, 1])\n tdf = filter(:Test => ==(test), df)\n colors = Makie.cgrad(:PuOr_4, 2; categorical=true, alpha=0.6)\n if by_sex\n density!(\n axs,\n filter(:Sex => ==(\"female\"), tdf).score;\n color=colors[1],\n label=\"Girls\",\n )\n density!(\n axs,\n filter(:Sex => ==(\"male\"), tdf).score;\n color=colors[2],\n label=\"Boys\",\n )\n axislegend(axs; position=:lt)\n else\n density!(axs, tdf.score)\n end\n fdensity\nend" + }, + { + "objectID": "singularity.html", + "href": "singularity.html", + "title": "Convergence, singularity and all that", + "section": "", + "text": "Add the packages to be used\n\n\nCode\nusing CairoMakie\nusing DataFrames\nusing LinearAlgebra\nusing MixedModels\nusing MixedModelsMakie\nusing ProgressMeter\n\nProgressMeter.ijulia_behavior(:clear)\nCairoMakie.activate!(; type=\"svg\")\n\n\nFit a model for reaction time in the sleepstudy example, preserving information on the estimation progress (the thin=1 optional argument)\n\nm01 = let f = @formula reaction ~ 1 + days + (1 + days|subj)\n fit(MixedModel, f, MixedModels.dataset(:sleepstudy); thin=1)\nend\nprint(m01)\n\nLinear mixed model fit by maximum likelihood\n reaction ~ 1 + days + (1 + days | subj)\n logLik -2 logLik AIC AICc BIC \n -875.9697 1751.9393 1763.9393 1764.4249 1783.0971\n\nVariance components:\n Column Variance Std.Dev. Corr.\nsubj (Intercept) 565.51067 23.78047\n days 32.68212 5.71683 +0.08\nResidual 654.94145 25.59182\n Number of obs: 180; levels of grouping factors: 18\n\n Fixed-effects parameters:\n──────────────────────────────────────────────────\n Coef. Std. Error z Pr(>|z|)\n──────────────────────────────────────────────────\n(Intercept) 251.405 6.63226 37.91 <1e-99\ndays 10.4673 1.50224 6.97 <1e-11\n──────────────────────────────────────────────────\n\n\nThe covariance matrix for each subject’s random effects is evaluated from its “matrix square root”, called the Cholesky factor.\n\nλ = only(m01.λ)\n\n2×2 LowerTriangular{Float64, Matrix{Float64}}:\n 0.929221 ⋅ \n 0.0181684 0.222645\n\n\nThe transpose of \\(\\lambda\\), written \\(\\lambda'\\), is an upper triangular matrix generated by “flipping” \\(\\lambda\\) about the main diagonal.\n\nλ'\n\n2×2 UpperTriangular{Float64, Adjoint{Float64, Matrix{Float64}}}:\n 0.929221 0.0181684\n ⋅ 0.222645\n\n\nThe product \\(\\lambda * \\lambda'\\) will be symmetric. The covariance matrix of the random effects, \\(\\Sigma\\), is this symmetric matrix scaled by \\(\\sigma^2\\)\n\nΣ = m01.σ^2 * λ * λ'\n\n2×2 Matrix{Float64}:\n 565.511 11.057\n 11.057 32.6821\n\n\nThe estimated variances of the random effects, which are the diagonal elements of \\(\\Sigma\\), correspond to the values shown in the table. To evaluate the covariance, isolate the correlation\n\n# m01.σρs extracts the `σρs` property of the model.\n# This property is a NamedTuple where the names\n# correspond to grouping factors - in this case, `subj`.\n# So `m01.σρs.subj.ρ` is the estimated correlation(s) for\n# this grouping factor. Because there is only one such correlation\n# we can extract it with `only()`, which also verifies that\n# there is exactly one.\nρ = only(m01.σρs.subj.ρ)\n\n0.08133216865276134\n\n\nand multiply by the standard deviations\n\nρ * sqrt(first(Σ) * last(Σ))\n\n11.057014395926759", + "crumbs": [ + "Visualizations and diagnostics", + "Convergence, singularity and all that" + ] + }, + { + "objectID": "singularity.html#what-does-it-mean-for-a-converged-model-to-be-singular", + "href": "singularity.html#what-does-it-mean-for-a-converged-model-to-be-singular", + "title": "Convergence, singularity and all that", + "section": "", + "text": "Add the packages to be used\n\n\nCode\nusing CairoMakie\nusing DataFrames\nusing LinearAlgebra\nusing MixedModels\nusing MixedModelsMakie\nusing ProgressMeter\n\nProgressMeter.ijulia_behavior(:clear)\nCairoMakie.activate!(; type=\"svg\")\n\n\nFit a model for reaction time in the sleepstudy example, preserving information on the estimation progress (the thin=1 optional argument)\n\nm01 = let f = @formula reaction ~ 1 + days + (1 + days|subj)\n fit(MixedModel, f, MixedModels.dataset(:sleepstudy); thin=1)\nend\nprint(m01)\n\nLinear mixed model fit by maximum likelihood\n reaction ~ 1 + days + (1 + days | subj)\n logLik -2 logLik AIC AICc BIC \n -875.9697 1751.9393 1763.9393 1764.4249 1783.0971\n\nVariance components:\n Column Variance Std.Dev. Corr.\nsubj (Intercept) 565.51067 23.78047\n days 32.68212 5.71683 +0.08\nResidual 654.94145 25.59182\n Number of obs: 180; levels of grouping factors: 18\n\n Fixed-effects parameters:\n──────────────────────────────────────────────────\n Coef. Std. Error z Pr(>|z|)\n──────────────────────────────────────────────────\n(Intercept) 251.405 6.63226 37.91 <1e-99\ndays 10.4673 1.50224 6.97 <1e-11\n──────────────────────────────────────────────────\n\n\nThe covariance matrix for each subject’s random effects is evaluated from its “matrix square root”, called the Cholesky factor.\n\nλ = only(m01.λ)\n\n2×2 LowerTriangular{Float64, Matrix{Float64}}:\n 0.929221 ⋅ \n 0.0181684 0.222645\n\n\nThe transpose of \\(\\lambda\\), written \\(\\lambda'\\), is an upper triangular matrix generated by “flipping” \\(\\lambda\\) about the main diagonal.\n\nλ'\n\n2×2 UpperTriangular{Float64, Adjoint{Float64, Matrix{Float64}}}:\n 0.929221 0.0181684\n ⋅ 0.222645\n\n\nThe product \\(\\lambda * \\lambda'\\) will be symmetric. The covariance matrix of the random effects, \\(\\Sigma\\), is this symmetric matrix scaled by \\(\\sigma^2\\)\n\nΣ = m01.σ^2 * λ * λ'\n\n2×2 Matrix{Float64}:\n 565.511 11.057\n 11.057 32.6821\n\n\nThe estimated variances of the random effects, which are the diagonal elements of \\(\\Sigma\\), correspond to the values shown in the table. To evaluate the covariance, isolate the correlation\n\n# m01.σρs extracts the `σρs` property of the model.\n# This property is a NamedTuple where the names\n# correspond to grouping factors - in this case, `subj`.\n# So `m01.σρs.subj.ρ` is the estimated correlation(s) for\n# this grouping factor. Because there is only one such correlation\n# we can extract it with `only()`, which also verifies that\n# there is exactly one.\nρ = only(m01.σρs.subj.ρ)\n\n0.08133216865276134\n\n\nand multiply by the standard deviations\n\nρ * sqrt(first(Σ) * last(Σ))\n\n11.057014395926759", + "crumbs": [ + "Visualizations and diagnostics", + "Convergence, singularity and all that" + ] + }, + { + "objectID": "singularity.html#the-factor-is-generated-from-a-parameter-vector", + "href": "singularity.html#the-factor-is-generated-from-a-parameter-vector", + "title": "Convergence, singularity and all that", + "section": "0.2 The factor is generated from a parameter vector", + "text": "0.2 The factor is generated from a parameter vector\nIn practice we optimize the log-likelihood with respect to a parameter vector called \\(\\theta\\) that generates \\(\\lambda\\).\n\nm01.θ\n\n3-element Vector{Float64}:\n 0.9292213162629417\n 0.018168381188609965\n 0.22264487194620683\n\n\nThe elements of this parameter vector are subject to constraints. In particular, two of the three elements have a lower bound of zero.\n\nm01.lowerbd\n\n3-element Vector{Float64}:\n 0.0\n -Inf\n 0.0\n\n\nThat is, the first and third elements of \\(\\theta\\), corresponding to diagonal elements of \\(\\lambda\\), must be non-negative, whereas the second component is unconstrained (has a lower bound of \\(-\\infty\\)).", + "crumbs": [ + "Visualizations and diagnostics", + "Convergence, singularity and all that" + ] + }, + { + "objectID": "singularity.html#progress-of-the-iterations", + "href": "singularity.html#progress-of-the-iterations", + "title": "Convergence, singularity and all that", + "section": "0.3 Progress of the iterations", + "text": "0.3 Progress of the iterations\nThe optsum.fitlog property of the model is a vector of tuples where each tuple contains the value of the \\(\\theta\\) vector and the value of the objective at that \\(\\theta\\). The fitlog always contains the first and the last evaluation. When the thin named argument is set, this property has a row for every thin’th evaluation.\n\nm01.optsum.fitlog\n\n57-element Vector{Tuple{Vector{Float64}, Float64}}:\n ([1.0, 0.0, 1.0], 1784.642296192471)\n ([1.75, 0.0, 1.0], 1790.1256369894638)\n ([1.0, 1.0, 1.0], 1798.999624496596)\n ([1.0, 0.0, 1.75], 1803.8532002844106)\n ([0.25, 0.0, 1.0], 1800.6139807455515)\n ([1.0, -1.0, 1.0], 1798.6046308389316)\n ([1.0, 0.0, 0.25], 1752.2607369909213)\n ([1.1832612965367613, -0.0086618879582246, 0.0], 1797.5876920199157)\n ([1.075, 0.0, 0.32499999999999996], 1754.9541095798672)\n ([0.8166315695343094, 0.011167254457244754, 0.28823768689703533], 1753.6956816568222)\n ⋮\n ([0.9291127454088746, 0.018179125358629283, 0.22262388947635725], 1751.93934482972)\n ([0.9291905933031118, 0.01816575364337781, 0.22264320511885913], 1751.9393444890322)\n ([0.9292542992836834, 0.018209269923966702, 0.22262081403728648], 1751.939345033819)\n ([0.9291892253327142, 0.018129789358969268, 0.22257323673727916], 1751.9393475284328)\n ([0.9292535836333305, 0.018167625568569275, 0.22264990501164184], 1751.939344490385)\n ([0.9292145039962464, 0.018171738990225547, 0.22264674302334211], 1751.9393444744132)\n ([0.9292083869346267, 0.01817147759030398, 0.22264619530985152], 1751.9393444750235)\n ([0.9292092974286392, 0.018172966192038814, 0.22265206228222742], 1751.939344506555)\n ([0.9292213162629417, 0.018168381188609965, 0.22264487194620683], 1751.9393444646903)\n\n\nThere were 57 evaluations of the objective before convergence was declared, according to rather stringent convergence criteria. We can see that the last 10 evaluations only produced changes in the fourth decimal place of the objective or even smaller. That is, effective convergence occurred after about 40 or 45 evaluations.", + "crumbs": [ + "Visualizations and diagnostics", + "Convergence, singularity and all that" + ] + }, + { + "objectID": "singularity.html#evaluating-the-random-effects-correlation-from-theta", + "href": "singularity.html#evaluating-the-random-effects-correlation-from-theta", + "title": "Convergence, singularity and all that", + "section": "2.1 Evaluating the random effects correlation from \\(\\theta\\)", + "text": "2.1 Evaluating the random effects correlation from \\(\\theta\\)\nThere is a short-cut for evaluating the correlation which is to “normalize” the second row of \\(\\lambda\\), in the sense that the row is scaled so that it has unit length.\n\nnormed = normalize!(λ[2, :])\n\n2-element Vector{Float64}:\n 0.08133216865276136\n 0.996687051356763\n\n\nproviding the correlation as\n\nfirst(normed)\n\n0.08133216865276136", + "crumbs": [ + "Visualizations and diagnostics", + "Convergence, singularity and all that" + ] + }, + { + "objectID": "singularity.html#optimizing-with-a-fixed-correlation", + "href": "singularity.html#optimizing-with-a-fixed-correlation", + "title": "Convergence, singularity and all that", + "section": "2.2 Optimizing with a fixed correlation", + "text": "2.2 Optimizing with a fixed correlation\nTo profile the correlation we need optimize the objective while fixing a value of the correlation. The way we will do this is to determine \\(\\theta_2\\) as a function of the fixed \\(\\rho\\) and \\(\\theta_3\\).\nWe need to solve \\[\n\\rho = \\frac{\\theta_2}{\\sqrt{\\theta_2^2 + \\theta_3^2}}\n\\]\nfor \\(\\theta_2\\) as a function of \\(\\rho\\) and \\(\\theta_3\\).\nNotice that \\(\\theta_2\\) and \\(\\rho\\) have the same sign. Thus it is sufficient to determine the absolute value of \\(\\theta_2\\) then transfer the sign from \\(\\rho\\). \\[\n\\theta_2^2=\\rho^2(\\theta_2^2 + \\theta_3^2)\n\\] which implies \\[\n\\theta_2^2 = \\frac{\\rho^2}{1-\\rho^2}\\theta_3^2, \\quad \\theta_3\\ge 0\n\\] and thus \\[\n\\theta_2=\\frac{\\rho}{\\sqrt{1-\\rho^2}}\\theta_3\n\\]", + "crumbs": [ + "Visualizations and diagnostics", + "Convergence, singularity and all that" + ] + }, + { + "objectID": "sleepstudy_speed.html", + "href": "sleepstudy_speed.html", + "title": "The sleepstudy: Speed - for a change …", + "section": "", + "text": "Belenky et al. (2003) reported effects of sleep deprivation across a 14-day study of 30-to-40-year old men and women holding commercial vehicle driving licenses. Their analyses are based on a subset of tasks and ratings from very large and comprehensive test and questionnaire battery (Balkin et al., 2000).\nInitially 66 subjects were assigned to one of four time-in-bed (TIB) groups with 9 hours (22:00-07:00) of sleep augmentation or 7 hours (24:00-07:00), 5 hours (02:00-07:00), and 3 hours (04:00-0:00) of sleep restrictions per night, respectively. The final sample comprised 56 subjects. The Psychomotor Vigilance Test (PVT) measures simple reaction time to a visual stimulus, presented approximately 10 times ⁄ minute (interstimulus interval varied from 2 to 10 s in 2-s increments) for 10 min and implemented in a thumb-operated, hand-held device (Dinges & Powell, 1985).\n\n\nThe study comprised 2 training days (T1, T2), one day with baseline measures (B), seven days with sleep deprivation (E1 to E7), and four recovery days (R1 to R4). T1 and T2 were devoted to training on the performance tests and familiarization with study procedures. PVT baseline testing commenced on the morning of the third day (B) and testing continued for the duration of the study (E1–E7, R1–R3; no measures were taken on R4). Bed times during T, B, and R days were 8 hours (23:00-07:00).\n\n\n\nThe PVT (along with the Stanford Sleepiness Scale) was administered as a battery four times per day (09:00, 12:00, 15:00, and 21:00 h); the battery included other tests not reported here (see Balkin et al., 2000). The sleep latency test was administered at 09:40 and 15:30 h for all groups. Subjects in the 3- and 5-h TIB groups performed an additional battery at 00:00 h and 02:00 h to occupy their additional time awake. The PVT and SSS were administered in this battery; however, as data from the 00:00 and 02:00 h sessions were not common to all TIB groups, these data were not included in the statistical analyses reported in the paper.\n\n\n\nThe authors analyzed response speed, that is (1/RT)*1000 – completely warranted according to a Box-Cox check of the current data – with mixed-model ANOVAs using group as between- and day as within-subject factors. The ANOVA was followed up with simple tests of the design effects implemented over days for each of the four groups.\n\n\n\nThe current data distributed with the RData collection is attributed to the 3-hour TIB group, but the means do not agree at all with those reported for this group in (Belenky et al., 2003, fig. 3) where the 3-hour TIB group is also based on only 13 (not 18) subjects. Specifically, the current data show a much smaller slow-down of response speed across E1 to E7 and do not reflect the recovery during R1 to R3. The current data also cover only 10 not 11 days, but it looks like only R3 is missing. The closest match of the current means was with the average of the 3-hour and 7-hour TIB groups; if only males were included, this would amount to 18 subjects. (This conjecture is based only on visual inspection of graphs.)", + "crumbs": [ + "Worked examples", + "The sleepstudy: Speed - for a change ..." + ] + }, + { + "objectID": "sleepstudy_speed.html#design", + "href": "sleepstudy_speed.html#design", + "title": "The sleepstudy: Speed - for a change …", + "section": "", + "text": "The study comprised 2 training days (T1, T2), one day with baseline measures (B), seven days with sleep deprivation (E1 to E7), and four recovery days (R1 to R4). T1 and T2 were devoted to training on the performance tests and familiarization with study procedures. PVT baseline testing commenced on the morning of the third day (B) and testing continued for the duration of the study (E1–E7, R1–R3; no measures were taken on R4). Bed times during T, B, and R days were 8 hours (23:00-07:00).", + "crumbs": [ + "Worked examples", + "The sleepstudy: Speed - for a change ..." + ] + }, + { + "objectID": "sleepstudy_speed.html#test-schedule-within-days", + "href": "sleepstudy_speed.html#test-schedule-within-days", + "title": "The sleepstudy: Speed - for a change …", + "section": "", + "text": "The PVT (along with the Stanford Sleepiness Scale) was administered as a battery four times per day (09:00, 12:00, 15:00, and 21:00 h); the battery included other tests not reported here (see Balkin et al., 2000). The sleep latency test was administered at 09:40 and 15:30 h for all groups. Subjects in the 3- and 5-h TIB groups performed an additional battery at 00:00 h and 02:00 h to occupy their additional time awake. The PVT and SSS were administered in this battery; however, as data from the 00:00 and 02:00 h sessions were not common to all TIB groups, these data were not included in the statistical analyses reported in the paper.", + "crumbs": [ + "Worked examples", + "The sleepstudy: Speed - for a change ..." + ] + }, + { + "objectID": "sleepstudy_speed.html#statistical-analyses", + "href": "sleepstudy_speed.html#statistical-analyses", + "title": "The sleepstudy: Speed - for a change …", + "section": "", + "text": "The authors analyzed response speed, that is (1/RT)*1000 – completely warranted according to a Box-Cox check of the current data – with mixed-model ANOVAs using group as between- and day as within-subject factors. The ANOVA was followed up with simple tests of the design effects implemented over days for each of the four groups.", + "crumbs": [ + "Worked examples", + "The sleepstudy: Speed - for a change ..." + ] + }, + { + "objectID": "sleepstudy_speed.html#current-data", + "href": "sleepstudy_speed.html#current-data", + "title": "The sleepstudy: Speed - for a change …", + "section": "", + "text": "The current data distributed with the RData collection is attributed to the 3-hour TIB group, but the means do not agree at all with those reported for this group in (Belenky et al., 2003, fig. 3) where the 3-hour TIB group is also based on only 13 (not 18) subjects. Specifically, the current data show a much smaller slow-down of response speed across E1 to E7 and do not reflect the recovery during R1 to R3. The current data also cover only 10 not 11 days, but it looks like only R3 is missing. The closest match of the current means was with the average of the 3-hour and 7-hour TIB groups; if only males were included, this would amount to 18 subjects. (This conjecture is based only on visual inspection of graphs.)", + "crumbs": [ + "Worked examples", + "The sleepstudy: Speed - for a change ..." + ] + }, + { + "objectID": "sleepstudy_speed.html#within-subject-simple-regressions", + "href": "sleepstudy_speed.html#within-subject-simple-regressions", + "title": "The sleepstudy: Speed - for a change …", + "section": "5.1 Within-subject simple regressions", + "text": "5.1 Within-subject simple regressions\nApplying combine to a grouped data frame like gdf produces a DataFrame with a row for each group. The permutation ord provides an ordering for the groups by increasing intercept (predicted response at day 0).\n\nwithin = combine(gdf, [:day, :speed] => simplelinreg => :coef)\n\n18×2 DataFrame\n\n\n\nRow\nSubj\ncoef\n\n\n\nString\nTuple…\n\n\n\n\n1\nS308\n(3.94806, -0.194812)\n\n\n2\nS309\n(4.87022, -0.0475185)\n\n\n3\nS310\n(4.90606, -0.120054)\n\n\n4\nS330\n(3.4449, -0.0291309)\n\n\n5\nS331\n(3.47647, -0.0498047)\n\n\n6\nS332\n(3.84436, -0.105511)\n\n\n7\nS333\n(3.60159, -0.0917378)\n\n\n8\nS334\n(4.04528, -0.133527)\n\n\n9\nS335\n(3.80451, 0.0455771)\n\n\n10\nS337\n(3.34374, -0.137744)\n\n\n11\nS349\n(4.46855, -0.170885)\n\n\n12\nS350\n(4.21414, -0.20151)\n\n\n13\nS351\n(3.80469, -0.0728582)\n\n\n14\nS352\n(3.68634, -0.144957)\n\n\n15\nS369\n(3.85384, -0.120531)\n\n\n16\nS370\n(4.52679, -0.215965)\n\n\n17\nS371\n(3.853, -0.0936243)\n\n\n18\nS372\n(3.69208, -0.113292)\n\n\n\n\n\n\nFigure 1 shows the reaction speed versus days of sleep deprivation by subject. The panels are arranged by increasing initial reaction speed starting at the lower left and proceeding across rows.\n\n\nCode\nlet\n ord = sortperm(first.(within.coef))\n labs = values(only.(keys(gdf)))[ord] # labels for panels\n f = clevelandaxes!(Figure(; resolution=(1000, 750)), labs, (2, 9))\n for (axs, sdf) in zip(f.content, gdf[ord]) # iterate over the panels and groups\n scatter!(axs, sdf.day, sdf.speed) # add the points\n coef = simplelinreg(sdf.day, sdf.speed)\n abline!(axs, first(coef), last(coef)) # add the regression line\n end\n f\nend\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n┌ Warning: abline! is deprecated and will be removed in the future. Use ablines / ablines! instead.\n│ caller = top-level scope at sleepstudy_speed.qmd:106\n└ @ Core ~/Work/SMLP2024/sleepstudy_speed.qmd:106\n\n\n\n\n\n\n\n\nFigure 1: Reaction speed (s⁻¹) versus days of sleep deprivation by subject", + "crumbs": [ + "Worked examples", + "The sleepstudy: Speed - for a change ..." + ] + }, + { + "objectID": "kb07.html", + "href": "kb07.html", + "title": "Bootstrapping a fitted model", + "section": "", + "text": "Begin by loading the packages to be used.\n\n\nCode\nusing AlgebraOfGraphics\nusing CairoMakie\nusing DataFrames\nusing MixedModels\nusing ProgressMeter\nusing Random\nusing SMLP2024: dataset\n\nCairoMakie.activate!(; type=\"svg\")\n\nProgressMeter.ijulia_behavior(:clear)\n\n\n\n1 Data set and model\nThe kb07 data (Kronmüller & Barr, 2007) are one of the datasets provided by the MixedModels package.\n\nkb07 = dataset(:kb07)\n\nArrow.Table with 1789 rows, 7 columns, and schema:\n :subj String\n :item String\n :spkr String\n :prec String\n :load String\n :rt_trunc Int16\n :rt_raw Int16\n\n\nConvert the table to a DataFrame for summary.\n\ndescribe(DataFrame(kb07))\n\n7×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nDataType\n\n\n\n\n1\nsubj\n\nS030\n\nS103\n0\nString\n\n\n2\nitem\n\nI01\n\nI32\n0\nString\n\n\n3\nspkr\n\nnew\n\nold\n0\nString\n\n\n4\nprec\n\nbreak\n\nmaintain\n0\nString\n\n\n5\nload\n\nno\n\nyes\n0\nString\n\n\n6\nrt_trunc\n2182.2\n579\n1940.0\n5171\n0\nInt16\n\n\n7\nrt_raw\n2226.24\n579\n1940.0\n15923\n0\nInt16\n\n\n\n\n\n\nThe experimental factors; spkr, prec, and load, are two-level factors. The EffectsCoding contrast is used with these to create a \\(\\pm1\\) encoding. Furthermore, Grouping contrasts are assigned to the subj and item factors. This is not a contrast per-se but an indication that these factors will be used as grouping factors for random effects and, therefore, there is no need to create a contrast matrix. For large numbers of levels in a grouping factor, an attempt to create a contrast matrix may cause memory overflow.\nIt is not important in these cases but a good practice in any case.\n\ncontrasts = merge(\n Dict(nm => EffectsCoding() for nm in (:spkr, :prec, :load)),\n Dict(nm => Grouping() for nm in (:subj, :item)),\n)\n\nThe display of an initial model fit\n\nkbm01 = let\n form = @formula(\n rt_trunc ~\n 1 +\n spkr * prec * load +\n (1 + spkr + prec + load | subj) +\n (1 + spkr + prec + load | item)\n )\n fit(MixedModel, form, kb07; contrasts)\nend\n\nMinimizing 2 Time: 0:00:00 ( 0.43 s/it)\n objective: 29353.753287212847\nMinimizing 614 Time: 0:00:01 ( 2.88 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_subj\nσ_item\n\n\n(Intercept)\n2181.6750\n77.3029\n28.22\n<1e-99\n301.8414\n362.1701\n\n\nspkr: old\n67.7490\n18.2824\n3.71\n0.0002\n43.0396\n40.5288\n\n\nprec: maintain\n-333.9206\n47.1621\n-7.08\n<1e-11\n62.1035\n246.9365\n\n\nload: yes\n78.7681\n19.5247\n4.03\n<1e-04\n65.1473\n42.1755\n\n\nspkr: old & prec: maintain\n-21.9634\n15.8062\n-1.39\n0.1647\n\n\n\n\nspkr: old & load: yes\n18.3838\n15.8062\n1.16\n0.2448\n\n\n\n\nprec: maintain & load: yes\n4.5333\n15.8062\n0.29\n0.7743\n\n\n\n\nspkr: old & prec: maintain & load: yes\n23.6052\n15.8062\n1.49\n0.1353\n\n\n\n\nResidual\n668.5061\n\n\n\n\n\n\n\n\n\n\ndoes not include the estimated correlations of the random effects.\nThe VarCorr extractor displays these.\n\nVarCorr(kbm01)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\nsubj\n(Intercept)\n91108.2428\n301.8414\n\n\n\n\n\n\nspkr: old\n1852.4098\n43.0396\n+0.78\n\n\n\n\n\nprec: maintain\n3856.8408\n62.1035\n-0.59\n+0.02\n\n\n\n\nload: yes\n4244.1689\n65.1473\n+0.36\n+0.82\n+0.53\n\n\nitem\n(Intercept)\n131167.1572\n362.1701\n\n\n\n\n\n\nspkr: old\n1642.5825\n40.5288\n+0.44\n\n\n\n\n\nprec: maintain\n60977.6107\n246.9365\n-0.69\n+0.35\n\n\n\n\nload: yes\n1778.7763\n42.1755\n+0.32\n+0.23\n-0.15\n\n\nResidual\n\n446900.4651\n668.5061\n\n\n\n\n\n\n\n\nNone of the two-factor or three-factor interaction terms in the fixed-effects are significant. In the random-effects terms only the scalar random effects and the prec random effect for item appear to be warranted, leading to the reduced formula\n\nkbm02 = let\n form = @formula(\n rt_trunc ~\n 1 + spkr + prec + load + (1 | subj) + (1 + prec | item)\n )\n fit(MixedModel, form, kb07; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n2181.8526\n77.4681\n28.16\n<1e-99\n364.7125\n298.0259\n\n\nspkr: old\n67.8790\n16.0785\n4.22\n<1e-04\n\n\n\n\nprec: maintain\n-333.7906\n47.4472\n-7.03\n<1e-11\n252.5212\n\n\n\nload: yes\n78.5904\n16.0785\n4.89\n<1e-05\n\n\n\n\nResidual\n680.0319\n\n\n\n\n\n\n\n\n\n\n\nVarCorr(kbm02)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\nitem\n(Intercept)\n133015.241\n364.713\n\n\n\n\nprec: maintain\n63766.936\n252.521\n-0.70\n\n\nsubj\n(Intercept)\n88819.437\n298.026\n\n\n\nResidual\n\n462443.388\n680.032\n\n\n\n\n\n\nThese two models are nested and can be compared with a likelihood-ratio test.\n\nMixedModels.likelihoodratiotest(kbm02, kbm01)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\nrt_trunc ~ 1 + spkr + prec + load + (1 | subj) + (1 + prec | item)\n9\n28664\n\n\n\n\n\nrt_trunc ~ 1 + spkr + prec + load + spkr & prec + spkr & load + prec & load + spkr & prec & load + (1 + spkr + prec + load | subj) + (1 + spkr + prec + load | item)\n29\n28637\n27\n20\n0.1436\n\n\n\n\n\nThe p-value of approximately 17% leads us to prefer the simpler model, kbm02, to the more complex, kbm01.\n\n\n2 A bootstrap sample\nCreate a bootstrap sample of a few thousand parameter estimates from the reduced model. The pseudo-random number generator is initialized to a fixed value for reproducibility.\n\nRandom.seed!(1234321)\nkbm02samp = parametricbootstrap(2000, kbm02)\nkbm02tbl = kbm02samp.tbl\n\nTable with 14 columns and 2000 rows:\n obj β1 β2 β3 β4 σ σ1 σ2 ⋯\n ┌───────────────────────────────────────────────────────────────────────────\n 1 │ 28769.9 2152.69 60.3921 -359.134 54.1231 702.318 421.687 212.916 ⋯\n 2 │ 28775.3 2221.81 61.7318 -349.417 71.1992 697.028 442.809 278.93 ⋯\n 3 │ 28573.6 2325.31 53.9668 -404.225 81.3407 657.746 364.354 274.392 ⋯\n 4 │ 28682.9 2274.65 52.9438 -384.66 69.6788 682.801 398.9 319.726 ⋯\n 5 │ 28616.1 2191.34 54.7785 -357.919 111.829 673.878 253.244 235.909 ⋯\n 6 │ 28634.1 2176.73 95.9793 -378.383 82.1559 671.685 510.996 309.876 ⋯\n 7 │ 28620.4 2128.07 57.6928 -267.747 60.4621 666.986 396.878 295.008 ⋯\n 8 │ 28665.9 2152.59 55.8208 -274.382 76.5128 684.746 369.091 274.446 ⋯\n 9 │ 28651.2 2088.3 76.4636 -296.14 95.6124 680.282 265.684 186.041 ⋯\n 10 │ 28679.9 2375.23 72.2422 -423.922 94.0833 680.729 425.72 287.686 ⋯\n 11 │ 28686.8 2034.11 59.9149 -224.859 59.5448 682.433 308.676 272.183 ⋯\n 12 │ 28702.2 2130.98 83.3953 -271.61 79.974 689.679 401.972 264.255 ⋯\n 13 │ 28625.0 2055.35 55.3526 -363.937 67.7128 676.818 309.148 265.792 ⋯\n 14 │ 28570.3 2160.73 81.5575 -321.913 108.979 661.524 426.871 259.827 ⋯\n 15 │ 28613.6 2237.73 36.9996 -362.443 82.1812 673.873 312.863 241.595 ⋯\n 16 │ 28681.4 2306.64 43.8801 -429.983 103.082 686.437 360.536 204.514 ⋯\n 17 │ 28577.2 2143.78 66.2959 -373.598 93.157 668.15 321.47 183.119 ⋯\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱\n\n\nOne of the uses of such a sample is to form “confidence intervals” on the parameters by obtaining the shortest interval that covers a given proportion (95%, by default) of the sample.\n\nconfint(kbm02samp)\n\nDictTable with 2 columns and 9 rows:\n par lower upper\n ────┬────────────────────\n β1 │ 2028.01 2337.92\n β2 │ 38.431 99.5944\n β3 │ -439.321 -245.864\n β4 │ 46.0262 107.511\n ρ1 │ -0.89799 -0.445596\n σ │ 655.249 701.497\n σ1 │ 261.196 448.509\n σ2 │ 175.489 312.043\n σ3 │ 228.099 357.789\n\n\nA sample like this can be used for more than just creating an interval because it approximates the distribution of the estimator. For the fixed-effects parameters the estimators are close to being normally distributed, Figure 1.\n\n\nCode\ndraw(\n data(kbm02samp.β) * mapping(:β; color=:coefname) * AlgebraOfGraphics.density();\n figure=(; resolution=(800, 450)),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 1: Comparative densities of the fixed-effects coefficients in kbm02samp\n\n\n\n\n\n\n\nCode\nlet pars = [\"σ1\", \"σ2\", \"σ3\"]\n draw(\n data(kbm02tbl) *\n mapping(pars .=> \"σ\"; color=dims(1) => renamer(pars)) *\n AlgebraOfGraphics.density();\n figure=(; resolution=(800, 450)),\n )\nend\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 2: Density plot of bootstrap samples standard deviation of random effects\n\n\n\n\n\n\n\nCode\ndraw(\n data(kbm02tbl) *\n mapping(:ρ1 => \"Correlation\") *\n AlgebraOfGraphics.density();\n figure=(; resolution=(800, 450)),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 3: Density plot of correlation parameters in bootstrap sample from model kbm02\n\n\n\n\n\n\n\n3 References\n\n\nKronmüller, E., & Barr, D. J. (2007). Perspective-free pragmatics: Broken precedents and the recovery-from-preemption hypothesis. Journal of Memory and Language, 56(3), 436–455. https://doi.org/10.1016/j.jml.2006.05.002\n\n\n\n\n\n\n Back to top", + "crumbs": [ + "Bootstrap and profiling", + "Bootstrapping a fitted model" + ] + }, + { + "objectID": "arrow.html", + "href": "arrow.html", + "title": "Saving data with Arrow", + "section": "", + "text": "1 The Arrow storage format\nThe Arrow storage format provides a language-agnostic storage and memory specification for columnar data tables, which just means “something that looks like a data frame in R”. That is, an arrow table is an ordered, named collection of columns, all of the same length.\nThe columns can be of different types including numeric values, character strings, and factor-like representations - called DictEncoded.\nAn Arrow file can be read or written from R, Python, Julia and many other languages. Somewhat confusingly in R and Python the name feather, which refers to an earlier version of the storage format, is used in some function names like read_feather.\nInternally, the SMLP2024 package uses Arrow to store all of its datasets.\n\n\n2 The Emotikon data\nThe SMLP2021 repository contains a version of the data from Fühner et al. (2021) in notebooks/data/fggk21.arrow. After that file was created there were changes in the master RDS file on the osf.io site for the project. We will recreate the Arrow file here then split it into two separate tables, one with a row for each child in the study and one with a row for each test result.\nThe Arrow package for Julia does not export any function names, which means that the function to read an Arrow file must be called as Arrow.Table. It returns a column table, as described in the Tables package. This is like a read-only data frame, which can be easily converted to a full-fledged DataFrame if desired.\nThis arrangement allows for the Arrow package not to depend on the DataFrames package, which is a heavy-weight dependency, but still easily produce a DataFrame if warranted.\nLoad the packages to be used.\n\n\nCode\nusing AlgebraOfGraphics\nusing Arrow\nusing CairoMakie\nusing Chain\nusing DataFrameMacros\nusing DataFrames\nusing Downloads\nusing KernelDensity\nusing RCall # access R from within Julia\nusing StatsBase\n\nCairoMakie.activate!(; type=\"svg\")\nusing AlgebraOfGraphics: density\n\n\n\n\n3 Downloading and importing the RDS file\nThis is similar to some of the code shown by Julius Krumbiegel on Monday. In the data directory of the emotikon project on osf.io under Data, the url for the rds data file is found to be [https://osf.io/xawdb/]. Note that we want version 2 of this file.\n\nfn = Downloads.download(\"https://osf.io/xawdb/download?version=2\");\n\n\ndfrm = rcopy(R\"readRDS($fn)\")\n\n525126×7 DataFrame525101 rows omitted\n\n\n\nRow\nCohort\nSchool\nChild\nSex\nage\nTest\nscore\n\n\n\nCat…\nCat…\nCat…\nCat…\nFloat64\nCat…\nFloat64\n\n\n\n\n1\n2013\nS100067\nC002352\nmale\n7.99452\nS20_r\n5.26316\n\n\n2\n2013\nS100067\nC002352\nmale\n7.99452\nBPT\n3.7\n\n\n3\n2013\nS100067\nC002352\nmale\n7.99452\nSLJ\n125.0\n\n\n4\n2013\nS100067\nC002352\nmale\n7.99452\nStar_r\n2.47146\n\n\n5\n2013\nS100067\nC002352\nmale\n7.99452\nRun\n1053.0\n\n\n6\n2013\nS100067\nC002353\nmale\n7.99452\nS20_r\n5.0\n\n\n7\n2013\nS100067\nC002353\nmale\n7.99452\nBPT\n4.1\n\n\n8\n2013\nS100067\nC002353\nmale\n7.99452\nSLJ\n116.0\n\n\n9\n2013\nS100067\nC002353\nmale\n7.99452\nStar_r\n1.76778\n\n\n10\n2013\nS100067\nC002353\nmale\n7.99452\nRun\n1089.0\n\n\n11\n2013\nS100067\nC002354\nmale\n7.99452\nS20_r\n4.54545\n\n\n12\n2013\nS100067\nC002354\nmale\n7.99452\nBPT\n3.9\n\n\n13\n2013\nS100067\nC002354\nmale\n7.99452\nSLJ\n111.0\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n525115\n2018\nS401470\nC117964\nmale\n9.10609\nStar_r\n1.63704\n\n\n525116\n2018\nS401470\nC117964\nmale\n9.10609\nRun\n864.0\n\n\n525117\n2018\nS401470\nC117965\nfemale\n9.10609\nS20_r\n4.65116\n\n\n525118\n2018\nS401470\nC117965\nfemale\n9.10609\nBPT\n3.8\n\n\n525119\n2018\nS401470\nC117965\nfemale\n9.10609\nSLJ\n123.0\n\n\n525120\n2018\nS401470\nC117965\nfemale\n9.10609\nStar_r\n1.52889\n\n\n525121\n2018\nS401470\nC117965\nfemale\n9.10609\nRun\n1080.0\n\n\n525122\n2018\nS800200\nC117966\nmale\n9.10609\nS20_r\n4.54545\n\n\n525123\n2018\nS800200\nC117966\nmale\n9.10609\nBPT\n3.8\n\n\n525124\n2018\nS800200\nC117966\nmale\n9.10609\nSLJ\n100.0\n\n\n525125\n2018\nS800200\nC117966\nmale\n9.10609\nStar_r\n2.18506\n\n\n525126\n2018\nS800200\nC117966\nmale\n9.10609\nRun\n990.0\n\n\n\n\n\n\nNow write this file as a Arrow file and read it back in.\n\narrowfn = joinpath(\"data\", \"fggk21.arrow\")\nArrow.write(arrowfn, dfrm; compress=:lz4)\ntbl = Arrow.Table(arrowfn)\n\nArrow.Table with 525126 rows, 7 columns, and schema:\n :Cohort String\n :School String\n :Child String\n :Sex String\n :age Float64\n :Test String\n :score Float64\n\n\n\nfilesize(arrowfn)\n\n3077850\n\n\n\ndf = DataFrame(tbl)\n\n525126×7 DataFrame525101 rows omitted\n\n\n\nRow\nCohort\nSchool\nChild\nSex\nage\nTest\nscore\n\n\n\nString\nString\nString\nString\nFloat64\nString\nFloat64\n\n\n\n\n1\n2013\nS100067\nC002352\nmale\n7.99452\nS20_r\n5.26316\n\n\n2\n2013\nS100067\nC002352\nmale\n7.99452\nBPT\n3.7\n\n\n3\n2013\nS100067\nC002352\nmale\n7.99452\nSLJ\n125.0\n\n\n4\n2013\nS100067\nC002352\nmale\n7.99452\nStar_r\n2.47146\n\n\n5\n2013\nS100067\nC002352\nmale\n7.99452\nRun\n1053.0\n\n\n6\n2013\nS100067\nC002353\nmale\n7.99452\nS20_r\n5.0\n\n\n7\n2013\nS100067\nC002353\nmale\n7.99452\nBPT\n4.1\n\n\n8\n2013\nS100067\nC002353\nmale\n7.99452\nSLJ\n116.0\n\n\n9\n2013\nS100067\nC002353\nmale\n7.99452\nStar_r\n1.76778\n\n\n10\n2013\nS100067\nC002353\nmale\n7.99452\nRun\n1089.0\n\n\n11\n2013\nS100067\nC002354\nmale\n7.99452\nS20_r\n4.54545\n\n\n12\n2013\nS100067\nC002354\nmale\n7.99452\nBPT\n3.9\n\n\n13\n2013\nS100067\nC002354\nmale\n7.99452\nSLJ\n111.0\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n525115\n2018\nS401470\nC117964\nmale\n9.10609\nStar_r\n1.63704\n\n\n525116\n2018\nS401470\nC117964\nmale\n9.10609\nRun\n864.0\n\n\n525117\n2018\nS401470\nC117965\nfemale\n9.10609\nS20_r\n4.65116\n\n\n525118\n2018\nS401470\nC117965\nfemale\n9.10609\nBPT\n3.8\n\n\n525119\n2018\nS401470\nC117965\nfemale\n9.10609\nSLJ\n123.0\n\n\n525120\n2018\nS401470\nC117965\nfemale\n9.10609\nStar_r\n1.52889\n\n\n525121\n2018\nS401470\nC117965\nfemale\n9.10609\nRun\n1080.0\n\n\n525122\n2018\nS800200\nC117966\nmale\n9.10609\nS20_r\n4.54545\n\n\n525123\n2018\nS800200\nC117966\nmale\n9.10609\nBPT\n3.8\n\n\n525124\n2018\nS800200\nC117966\nmale\n9.10609\nSLJ\n100.0\n\n\n525125\n2018\nS800200\nC117966\nmale\n9.10609\nStar_r\n2.18506\n\n\n525126\n2018\nS800200\nC117966\nmale\n9.10609\nRun\n990.0\n\n\n\n\n\n\n\n\n4 Avoiding needless repetition\nOne of the principles of relational database design is that information should not be repeated needlessly. Each row of df is determined by a combination of Child and Test, together producing a score, which can be converted to a zScore.\nThe other columns in the table, Cohort, School, age, and Sex, are properties of the Child.\nStoring these values redundantly in the full table takes up space but, more importantly, allows for inconsistency. As it stands, a given Child could be recorded as being in one Cohort for the Run test and in another Cohort for the S20_r test and nothing about the table would detect this as being an error.\nThe approach used in relational databases is to store the information for score in one table that contains only Child, Test and score, store the information for the Child in another table including Cohort, School, age and Sex. These tables can then be combined to create the table to be used for analysis by joining the different tables together.\nThe maintainers of the DataFrames package have put in a lot of work over the past few years to make joins quite efficient in Julia. Thus the processing penalty of reassembling the big table from three smaller tables is minimal.\nIt is important to note that the main advantage of using smaller tables that are joined together to produce the analysis table is the fact that the information in the analysis table is consistent by design.\n\n\n5 Creating the smaller table\n\nChild = unique(select(df, :Child, :School, :Cohort, :Sex, :age))\n\n108295×5 DataFrame108270 rows omitted\n\n\n\nRow\nChild\nSchool\nCohort\nSex\nage\n\n\n\nString\nString\nString\nString\nFloat64\n\n\n\n\n1\nC002352\nS100067\n2013\nmale\n7.99452\n\n\n2\nC002353\nS100067\n2013\nmale\n7.99452\n\n\n3\nC002354\nS100067\n2013\nmale\n7.99452\n\n\n4\nC002355\nS100122\n2013\nfemale\n7.99452\n\n\n5\nC002356\nS100146\n2013\nmale\n7.99452\n\n\n6\nC002357\nS100146\n2013\nmale\n7.99452\n\n\n7\nC002358\nS100146\n2013\nmale\n7.99452\n\n\n8\nC002359\nS100183\n2013\nfemale\n7.99452\n\n\n9\nC002360\nS100195\n2013\nfemale\n7.99452\n\n\n10\nC002361\nS100213\n2013\nmale\n7.99452\n\n\n11\nC002362\nS100237\n2013\nfemale\n7.99452\n\n\n12\nC002363\nS100237\n2013\nfemale\n7.99452\n\n\n13\nC002364\nS100250\n2013\nfemale\n7.99452\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n108284\nC117943\nS130539\n2018\nfemale\n9.10609\n\n\n108285\nC117944\nS130539\n2018\nmale\n9.10609\n\n\n108286\nC117945\nS130539\n2018\nmale\n9.10609\n\n\n108287\nC117946\nS130539\n2018\nmale\n9.10609\n\n\n108288\nC117956\nS400580\n2018\nmale\n9.10609\n\n\n108289\nC117957\nS400919\n2018\nmale\n9.10609\n\n\n108290\nC117958\nS400919\n2018\nmale\n9.10609\n\n\n108291\nC117959\nS400919\n2018\nmale\n9.10609\n\n\n108292\nC117962\nS401250\n2018\nfemale\n9.10609\n\n\n108293\nC117964\nS401470\n2018\nmale\n9.10609\n\n\n108294\nC117965\nS401470\n2018\nfemale\n9.10609\n\n\n108295\nC117966\nS800200\n2018\nmale\n9.10609\n\n\n\n\n\n\n\nlength(unique(Child.Child)) # should be 108295\n\n108295\n\n\n\nfilesize(\n Arrow.write(\"./data/fggk21_Child.arrow\", Child; compress=:lz4)\n)\n\n1774946\n\n\n\nfilesize(\n Arrow.write(\n \"./data/fggk21_Score.arrow\",\n select(df, :Child, :Test, :score);\n compress=:lz4,\n ),\n)\n\n2794058\n\n\n\n\n\n\n\n\nNote\n\n\n\nA careful examination of the file sizes versus that of ./data/fggk21.arrow will show that the separate tables combined take up more space than the original because of the compression. Compression algorithms are often more successful when applied to larger files.\n\n\nNow read the Arrow tables in and reassemble the original table.\n\nScore = DataFrame(Arrow.Table(\"./data/fggk21_Score.arrow\"))\n\n525126×3 DataFrame525101 rows omitted\n\n\n\nRow\nChild\nTest\nscore\n\n\n\nString\nString\nFloat64\n\n\n\n\n1\nC002352\nS20_r\n5.26316\n\n\n2\nC002352\nBPT\n3.7\n\n\n3\nC002352\nSLJ\n125.0\n\n\n4\nC002352\nStar_r\n2.47146\n\n\n5\nC002352\nRun\n1053.0\n\n\n6\nC002353\nS20_r\n5.0\n\n\n7\nC002353\nBPT\n4.1\n\n\n8\nC002353\nSLJ\n116.0\n\n\n9\nC002353\nStar_r\n1.76778\n\n\n10\nC002353\nRun\n1089.0\n\n\n11\nC002354\nS20_r\n4.54545\n\n\n12\nC002354\nBPT\n3.9\n\n\n13\nC002354\nSLJ\n111.0\n\n\n⋮\n⋮\n⋮\n⋮\n\n\n525115\nC117964\nStar_r\n1.63704\n\n\n525116\nC117964\nRun\n864.0\n\n\n525117\nC117965\nS20_r\n4.65116\n\n\n525118\nC117965\nBPT\n3.8\n\n\n525119\nC117965\nSLJ\n123.0\n\n\n525120\nC117965\nStar_r\n1.52889\n\n\n525121\nC117965\nRun\n1080.0\n\n\n525122\nC117966\nS20_r\n4.54545\n\n\n525123\nC117966\nBPT\n3.8\n\n\n525124\nC117966\nSLJ\n100.0\n\n\n525125\nC117966\nStar_r\n2.18506\n\n\n525126\nC117966\nRun\n990.0\n\n\n\n\n\n\nAt this point we can create the z-score column by standardizing the scores for each Test. The code to do this follows Julius’s presentation on Monday.\n\n@transform!(groupby(Score, :Test), :zScore = @bycol zscore(:score))\n\n525126×4 DataFrame525101 rows omitted\n\n\n\nRow\nChild\nTest\nscore\nzScore\n\n\n\nString\nString\nFloat64\nFloat64\n\n\n\n\n1\nC002352\nS20_r\n5.26316\n1.7913\n\n\n2\nC002352\nBPT\n3.7\n-0.0622317\n\n\n3\nC002352\nSLJ\n125.0\n-0.0336567\n\n\n4\nC002352\nStar_r\n2.47146\n1.46874\n\n\n5\nC002352\nRun\n1053.0\n0.331058\n\n\n6\nC002353\nS20_r\n5.0\n1.15471\n\n\n7\nC002353\nBPT\n4.1\n0.498354\n\n\n8\nC002353\nSLJ\n116.0\n-0.498822\n\n\n9\nC002353\nStar_r\n1.76778\n-0.9773\n\n\n10\nC002353\nRun\n1089.0\n0.574056\n\n\n11\nC002354\nS20_r\n4.54545\n0.0551481\n\n\n12\nC002354\nBPT\n3.9\n0.218061\n\n\n13\nC002354\nSLJ\n111.0\n-0.757248\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n525115\nC117964\nStar_r\n1.63704\n-1.43175\n\n\n525116\nC117964\nRun\n864.0\n-0.944681\n\n\n525117\nC117965\nS20_r\n4.65116\n0.31086\n\n\n525118\nC117965\nBPT\n3.8\n0.0779146\n\n\n525119\nC117965\nSLJ\n123.0\n-0.137027\n\n\n525120\nC117965\nStar_r\n1.52889\n-1.8077\n\n\n525121\nC117965\nRun\n1080.0\n0.513306\n\n\n525122\nC117966\nS20_r\n4.54545\n0.0551481\n\n\n525123\nC117966\nBPT\n3.8\n0.0779146\n\n\n525124\nC117966\nSLJ\n100.0\n-1.32578\n\n\n525125\nC117966\nStar_r\n2.18506\n0.473217\n\n\n525126\nC117966\nRun\n990.0\n-0.0941883\n\n\n\n\n\n\n\nChild = DataFrame(Arrow.Table(\"./data/fggk21_Child.arrow\"))\n\n108295×5 DataFrame108270 rows omitted\n\n\n\nRow\nChild\nSchool\nCohort\nSex\nage\n\n\n\nString\nString\nString\nString\nFloat64\n\n\n\n\n1\nC002352\nS100067\n2013\nmale\n7.99452\n\n\n2\nC002353\nS100067\n2013\nmale\n7.99452\n\n\n3\nC002354\nS100067\n2013\nmale\n7.99452\n\n\n4\nC002355\nS100122\n2013\nfemale\n7.99452\n\n\n5\nC002356\nS100146\n2013\nmale\n7.99452\n\n\n6\nC002357\nS100146\n2013\nmale\n7.99452\n\n\n7\nC002358\nS100146\n2013\nmale\n7.99452\n\n\n8\nC002359\nS100183\n2013\nfemale\n7.99452\n\n\n9\nC002360\nS100195\n2013\nfemale\n7.99452\n\n\n10\nC002361\nS100213\n2013\nmale\n7.99452\n\n\n11\nC002362\nS100237\n2013\nfemale\n7.99452\n\n\n12\nC002363\nS100237\n2013\nfemale\n7.99452\n\n\n13\nC002364\nS100250\n2013\nfemale\n7.99452\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n108284\nC117943\nS130539\n2018\nfemale\n9.10609\n\n\n108285\nC117944\nS130539\n2018\nmale\n9.10609\n\n\n108286\nC117945\nS130539\n2018\nmale\n9.10609\n\n\n108287\nC117946\nS130539\n2018\nmale\n9.10609\n\n\n108288\nC117956\nS400580\n2018\nmale\n9.10609\n\n\n108289\nC117957\nS400919\n2018\nmale\n9.10609\n\n\n108290\nC117958\nS400919\n2018\nmale\n9.10609\n\n\n108291\nC117959\nS400919\n2018\nmale\n9.10609\n\n\n108292\nC117962\nS401250\n2018\nfemale\n9.10609\n\n\n108293\nC117964\nS401470\n2018\nmale\n9.10609\n\n\n108294\nC117965\nS401470\n2018\nfemale\n9.10609\n\n\n108295\nC117966\nS800200\n2018\nmale\n9.10609\n\n\n\n\n\n\n\ndf1 = disallowmissing!(leftjoin(Score, Child; on=:Child))\n\n525126×8 DataFrame525101 rows omitted\n\n\n\nRow\nChild\nTest\nscore\nzScore\nSchool\nCohort\nSex\nage\n\n\n\nString\nString\nFloat64\nFloat64\nString\nString\nString\nFloat64\n\n\n\n\n1\nC002352\nS20_r\n5.26316\n1.7913\nS100067\n2013\nmale\n7.99452\n\n\n2\nC002352\nBPT\n3.7\n-0.0622317\nS100067\n2013\nmale\n7.99452\n\n\n3\nC002352\nSLJ\n125.0\n-0.0336567\nS100067\n2013\nmale\n7.99452\n\n\n4\nC002352\nStar_r\n2.47146\n1.46874\nS100067\n2013\nmale\n7.99452\n\n\n5\nC002352\nRun\n1053.0\n0.331058\nS100067\n2013\nmale\n7.99452\n\n\n6\nC002353\nS20_r\n5.0\n1.15471\nS100067\n2013\nmale\n7.99452\n\n\n7\nC002353\nBPT\n4.1\n0.498354\nS100067\n2013\nmale\n7.99452\n\n\n8\nC002353\nSLJ\n116.0\n-0.498822\nS100067\n2013\nmale\n7.99452\n\n\n9\nC002353\nStar_r\n1.76778\n-0.9773\nS100067\n2013\nmale\n7.99452\n\n\n10\nC002353\nRun\n1089.0\n0.574056\nS100067\n2013\nmale\n7.99452\n\n\n11\nC002354\nS20_r\n4.54545\n0.0551481\nS100067\n2013\nmale\n7.99452\n\n\n12\nC002354\nBPT\n3.9\n0.218061\nS100067\n2013\nmale\n7.99452\n\n\n13\nC002354\nSLJ\n111.0\n-0.757248\nS100067\n2013\nmale\n7.99452\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n525115\nC117964\nStar_r\n1.63704\n-1.43175\nS401470\n2018\nmale\n9.10609\n\n\n525116\nC117964\nRun\n864.0\n-0.944681\nS401470\n2018\nmale\n9.10609\n\n\n525117\nC117965\nS20_r\n4.65116\n0.31086\nS401470\n2018\nfemale\n9.10609\n\n\n525118\nC117965\nBPT\n3.8\n0.0779146\nS401470\n2018\nfemale\n9.10609\n\n\n525119\nC117965\nSLJ\n123.0\n-0.137027\nS401470\n2018\nfemale\n9.10609\n\n\n525120\nC117965\nStar_r\n1.52889\n-1.8077\nS401470\n2018\nfemale\n9.10609\n\n\n525121\nC117965\nRun\n1080.0\n0.513306\nS401470\n2018\nfemale\n9.10609\n\n\n525122\nC117966\nS20_r\n4.54545\n0.0551481\nS800200\n2018\nmale\n9.10609\n\n\n525123\nC117966\nBPT\n3.8\n0.0779146\nS800200\n2018\nmale\n9.10609\n\n\n525124\nC117966\nSLJ\n100.0\n-1.32578\nS800200\n2018\nmale\n9.10609\n\n\n525125\nC117966\nStar_r\n2.18506\n0.473217\nS800200\n2018\nmale\n9.10609\n\n\n525126\nC117966\nRun\n990.0\n-0.0941883\nS800200\n2018\nmale\n9.10609\n\n\n\n\n\n\n\n\n\n\n\n\nNote\n\n\n\nThe call to disallowmissing! is because the join will create columns that allow for missing values but we know that we should not get missing values in the result. This call will fail if, for some reason, missing values were created.\n\n\n\n\n6 Discovering patterns in the data\nOne of the motivations for creating the Child table was to be able to bin the ages according to the age of each child, not the age of each Child-Test combination. Not all children have all 5 test results. We can check the number of results by grouping on :Child and evaluate the number of rows in each group.\n\nnobsChild = combine(groupby(Score, :Child), nrow => :ntest)\n\n108295×2 DataFrame108270 rows omitted\n\n\n\nRow\nChild\nntest\n\n\n\nString\nInt64\n\n\n\n\n1\nC002352\n5\n\n\n2\nC002353\n5\n\n\n3\nC002354\n5\n\n\n4\nC002355\n5\n\n\n5\nC002356\n5\n\n\n6\nC002357\n5\n\n\n7\nC002358\n5\n\n\n8\nC002359\n4\n\n\n9\nC002360\n5\n\n\n10\nC002361\n4\n\n\n11\nC002362\n5\n\n\n12\nC002363\n5\n\n\n13\nC002364\n5\n\n\n⋮\n⋮\n⋮\n\n\n108284\nC117943\n5\n\n\n108285\nC117944\n5\n\n\n108286\nC117945\n5\n\n\n108287\nC117946\n5\n\n\n108288\nC117956\n5\n\n\n108289\nC117957\n4\n\n\n108290\nC117958\n5\n\n\n108291\nC117959\n5\n\n\n108292\nC117962\n5\n\n\n108293\nC117964\n5\n\n\n108294\nC117965\n5\n\n\n108295\nC117966\n5\n\n\n\n\n\n\nNow create a table of the number of children with 1, 2, …, 5 test scores.\n\ncombine(groupby(nobsChild, :ntest), nrow)\n\n5×2 DataFrame\n\n\n\nRow\nntest\nnrow\n\n\n\nInt64\nInt64\n\n\n\n\n1\n1\n462\n\n\n2\n2\n729\n\n\n3\n3\n1739\n\n\n4\n4\n8836\n\n\n5\n5\n96529\n\n\n\n\n\n\nA natural question at this point is whether there is something about those students who have few observations. For example, are they from only a few schools?\nOne approach to examining properties like is to add the number of observations for each child to the :Child table. Later we can group the table according to this :ntest to look at properties of :Child by :ntest.\n\ngdf = groupby(\n disallowmissing!(leftjoin(Child, nobsChild; on=:Child)), :ntest\n)\n\nGroupedDataFrame with 5 groups based on key: ntestFirst Group (462 rows): ntest = 1437 rows omitted\n\n\n\nRow\nChild\nSchool\nCohort\nSex\nage\nntest\n\n\n\nString\nString\nString\nString\nFloat64\nInt64\n\n\n\n\n1\nC002452\nS101175\n2013\nmale\n7.99452\n1\n\n\n2\nC002625\nS103329\n2013\nmale\n7.99452\n1\n\n\n3\nC002754\nS104814\n2013\nfemale\n7.99452\n1\n\n\n4\nC003269\nS102258\n2012\nfemale\n7.99726\n1\n\n\n5\nC003599\nS105843\n2012\nfemale\n7.99726\n1\n\n\n6\nC003807\nS100754\n2011\nmale\n8.0\n1\n\n\n7\nC003985\nS102945\n2011\nmale\n8.0\n1\n\n\n8\nC004086\nS104255\n2011\nmale\n8.0\n1\n\n\n9\nC004657\nS101400\n2014\nmale\n8.03833\n1\n\n\n10\nC005036\nS105909\n2014\nmale\n8.03833\n1\n\n\n11\nC005440\nS101023\n2019\nmale\n8.05202\n1\n\n\n12\nC005523\nS101825\n2019\nfemale\n8.05202\n1\n\n\n13\nC005697\nS103615\n2019\nmale\n8.05202\n1\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n451\nC112638\nS103718\n2015\nfemale\n9.0486\n1\n\n\n452\nC114749\nS112938\n2017\nmale\n9.06502\n1\n\n\n453\nC115460\nS101953\n2015\nmale\n9.08145\n1\n\n\n454\nC115569\nS100572\n2017\nmale\n9.08419\n1\n\n\n455\nC115587\nS100754\n2017\nfemale\n9.08419\n1\n\n\n456\nC117108\nS103196\n2018\nfemale\n9.10609\n1\n\n\n457\nC117229\nS103615\n2018\nmale\n9.10609\n1\n\n\n458\nC117230\nS103615\n2018\nmale\n9.10609\n1\n\n\n459\nC117419\nS104954\n2018\nfemale\n9.10609\n1\n\n\n460\nC117437\nS105004\n2018\nmale\n9.10609\n1\n\n\n461\nC117539\nS105636\n2018\nmale\n9.10609\n1\n\n\n462\nC117659\nS106483\n2018\nfemale\n9.10609\n1\n\n\n\n⋮Last Group (96529 rows): ntest = 596504 rows omitted\n\n\n\nRow\nChild\nSchool\nCohort\nSex\nage\nntest\n\n\n\nString\nString\nString\nString\nFloat64\nInt64\n\n\n\n\n1\nC002352\nS100067\n2013\nmale\n7.99452\n5\n\n\n2\nC002353\nS100067\n2013\nmale\n7.99452\n5\n\n\n3\nC002354\nS100067\n2013\nmale\n7.99452\n5\n\n\n4\nC002355\nS100122\n2013\nfemale\n7.99452\n5\n\n\n5\nC002356\nS100146\n2013\nmale\n7.99452\n5\n\n\n6\nC002357\nS100146\n2013\nmale\n7.99452\n5\n\n\n7\nC002358\nS100146\n2013\nmale\n7.99452\n5\n\n\n8\nC002360\nS100195\n2013\nfemale\n7.99452\n5\n\n\n9\nC002362\nS100237\n2013\nfemale\n7.99452\n5\n\n\n10\nC002363\nS100237\n2013\nfemale\n7.99452\n5\n\n\n11\nC002364\nS100250\n2013\nfemale\n7.99452\n5\n\n\n12\nC002365\nS100304\n2013\nmale\n7.99452\n5\n\n\n13\nC002366\nS100304\n2013\nmale\n7.99452\n5\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n96518\nC117942\nS130539\n2018\nmale\n9.10609\n5\n\n\n96519\nC117943\nS130539\n2018\nfemale\n9.10609\n5\n\n\n96520\nC117944\nS130539\n2018\nmale\n9.10609\n5\n\n\n96521\nC117945\nS130539\n2018\nmale\n9.10609\n5\n\n\n96522\nC117946\nS130539\n2018\nmale\n9.10609\n5\n\n\n96523\nC117956\nS400580\n2018\nmale\n9.10609\n5\n\n\n96524\nC117958\nS400919\n2018\nmale\n9.10609\n5\n\n\n96525\nC117959\nS400919\n2018\nmale\n9.10609\n5\n\n\n96526\nC117962\nS401250\n2018\nfemale\n9.10609\n5\n\n\n96527\nC117964\nS401470\n2018\nmale\n9.10609\n5\n\n\n96528\nC117965\nS401470\n2018\nfemale\n9.10609\n5\n\n\n96529\nC117966\nS800200\n2018\nmale\n9.10609\n5\n\n\n\n\n\n\nAre the sexes represented more-or-less equally?\n\ncombine(groupby(first(gdf), :Sex), nrow => :nchild)\n\n2×2 DataFrame\n\n\n\nRow\nSex\nnchild\n\n\n\nString\nInt64\n\n\n\n\n1\nmale\n230\n\n\n2\nfemale\n232\n\n\n\n\n\n\n\ncombine(groupby(last(gdf), :Sex), nrow => :nchild)\n\n2×2 DataFrame\n\n\n\nRow\nSex\nnchild\n\n\n\nString\nInt64\n\n\n\n\n1\nmale\n47552\n\n\n2\nfemale\n48977\n\n\n\n\n\n\n\n\n7 Reading Arrow files in other languages\nThere are Arrow implementations for R (the arrow package) and for Python (pyarrow).\n#| eval: false\nimport pyarrow.feather: read_table\nread_table(\"./data/fggk21.arrow\")\n#| eval: false\nlibrary(\"arrow\")\nfggk21 <- read_feather(\"./data/fggk21.arrow\")\nnrow(fggk21)\n\n\n8 References\n\n\nFühner, T., Granacher, U., Golle, K., & Kliegl, R. (2021). Age and sex effects in physical fitness components of 108,295 third graders including 515 primary schools and 9 cohorts. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-97000-4\n\n\n\n\n\n\n Back to top", + "crumbs": [ + "Getting started with Julia", + "Saving data with Arrow" + ] + }, + { + "objectID": "partial_within.html", + "href": "partial_within.html", + "title": "Partially-within subjects designs", + "section": "", + "text": "Begin by loading the packages to be used.\n\n\nCode\nusing AlgebraOfGraphics\nusing CairoMakie\nusing DataFrames\nusing MixedModels\nusing MixedModelsMakie\nusing MixedModelsSim\nusing ProgressMeter\nusing Random\n\nCairoMakie.activate!(; type=\"svg\")\n\nProgressMeter.ijulia_behavior(:clear)\n\n\n\n\nCode\nn_subj = 40\nn_item = 3\n# things are expressed as \"between\", so \"within subjects\" is \"between items\"\nitem_btwn = Dict(:frequency => [\"high\", \"medium\", \"low\"])\ndesign = simdat_crossed(MersenneTwister(42), n_subj, n_item;\n item_btwn = item_btwn)\ndesign = DataFrame(design)\n\n\n120×4 DataFrame95 rows omitted\n\n\n\nRow\nsubj\nitem\nfrequency\ndv\n\n\n\nString\nString\nString\nFloat64\n\n\n\n\n1\nS01\nI1\nhigh\n-0.556027\n\n\n2\nS02\nI1\nhigh\n-0.444383\n\n\n3\nS03\nI1\nhigh\n0.0271553\n\n\n4\nS04\nI1\nhigh\n-0.299484\n\n\n5\nS05\nI1\nhigh\n1.77786\n\n\n6\nS06\nI1\nhigh\n-1.1449\n\n\n7\nS07\nI1\nhigh\n-0.468606\n\n\n8\nS08\nI1\nhigh\n0.156143\n\n\n9\nS09\nI1\nhigh\n-2.64199\n\n\n10\nS10\nI1\nhigh\n1.00331\n\n\n11\nS11\nI1\nhigh\n1.08238\n\n\n12\nS12\nI1\nhigh\n0.187028\n\n\n13\nS13\nI1\nhigh\n0.518149\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n109\nS29\nI3\nlow\n-0.338763\n\n\n110\nS30\nI3\nlow\n-0.0953675\n\n\n111\nS31\nI3\nlow\n0.768972\n\n\n112\nS32\nI3\nlow\n1.44244\n\n\n113\nS33\nI3\nlow\n-0.275032\n\n\n114\nS34\nI3\nlow\n-0.379637\n\n\n115\nS35\nI3\nlow\n-0.722696\n\n\n116\nS36\nI3\nlow\n0.139661\n\n\n117\nS37\nI3\nlow\n-1.40934\n\n\n118\nS38\nI3\nlow\n1.05546\n\n\n119\nS39\nI3\nlow\n-2.23782\n\n\n120\nS40\nI3\nlow\n1.15915\n\n\n\n\n\n\n\n\nCode\nunique!(select(design, :item, :frequency))\n\n\n3×2 DataFrame\n\n\n\nRow\nitem\nfrequency\n\n\n\nString\nString\n\n\n\n\n1\nI1\nhigh\n\n\n2\nI2\nmedium\n\n\n3\nI3\nlow\n\n\n\n\n\n\n\n\nCode\nm0 = let contrasts, form\n contrasts = Dict(:frequency => HelmertCoding(base=\"high\"))\n form = @formula(dv ~ 1 + frequency +\n (1 + frequency | subj))\n fit(MixedModel, form, design; contrasts)\nend\n\n\nMinimizing 4 Time: 0:00:00 (27.12 ms/it)\n objective: 394.74620416420464\nMinimizing 73 Time: 0:00:00 (13.13 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_subj\n\n\n(Intercept)\n-0.0830\n0.0983\n-0.84\n0.3983\n0.5440\n\n\nfrequency: low\n0.0411\n0.1258\n0.33\n0.7438\n0.7051\n\n\nfrequency: medium\n-0.0060\n0.0572\n-0.11\n0.9160\n0.2928\n\n\nResidual\n0.5204\n\n\n\n\n\n\n\n\n\n\n\nCode\ncorrmat = [ 1 0.1 -0.2\n 0.1 1 0.1\n -0.2 0.1 1 ]\nre_subj = create_re(1.2, 1.5, 1.5; corrmat)\n\n\n3×3 LinearAlgebra.LowerTriangular{Float64, Matrix{Float64}}:\n 1.2 ⋅ ⋅ \n 0.15 1.49248 ⋅ \n -0.3 0.180907 1.45852\n\n\n\n\nCode\nθ = createθ(m0; subj=re_subj)\n\n\n6-element Vector{Float64}:\n 1.2\n 0.15000000000000002\n -0.30000000000000004\n 1.49248115565993\n 0.18090680674665818\n 1.4585173044131932\n\n\n\n\nCode\nσ = 1;\nβ = [1.0, -3, -2];\n\n\n\n\nCode\nfit!(simulate!(m0; θ, β, σ))\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_subj\n\n\n(Intercept)\n1.0024\n0.2308\n4.34\n<1e-04\n1.2898\n\n\nfrequency: low\n-2.9511\n0.2350\n-12.56\n<1e-35\n1.2281\n\n\nfrequency: medium\n-2.1689\n0.2723\n-7.97\n<1e-14\n1.6527\n\n\nResidual\n1.1845\n\n\n\n\n\n\n\n\n\n\n\nCode\nshrinkageplot(m0)\n\n\n\n\n\n\n\n\n\n\n\nCode\ncaterpillar(m0; orderby=nothing, vline_at_zero=true)\n\n\n\n\n\n\n\n\n\n\n\nCode\ndesign[!, :dv] .= response(m0)\n\n\n120-element Vector{Float64}:\n 15.057970446500043\n 5.1204354589939385\n 7.721170724963708\n 2.8917642652835136\n 3.008174534901129\n 8.242532408788222\n 4.95234926256407\n 8.26008013912291\n 5.169516496084133\n 6.988490665208847\n ⋮\n 1.8891725952397342\n 2.908433059510399\n 2.9404207880287423\n 0.7216674213195389\n -2.949507828847464\n 5.145924368618845\n 0.8248733146211791\n -2.4920339370472924\n -3.6917852507767233\n\n\n\n\nCode\ndesign_partial = filter(design) do row\n subj = parse(Int, row.subj[2:end])\n item = parse(Int, row.item[2:end])\n # for even-numbered subjects, we keep all conditions\n # for odd-numbered subjects, we keep only the two \"odd\" items,\n # i.e. the first and last conditions\n return iseven(subj) || isodd(item)\nend\nsort!(unique!(select(design_partial, :subj, :frequency)), :subj)\n\n\n100×2 DataFrame75 rows omitted\n\n\n\nRow\nsubj\nfrequency\n\n\n\nString\nString\n\n\n\n\n1\nS01\nhigh\n\n\n2\nS01\nlow\n\n\n3\nS02\nhigh\n\n\n4\nS02\nmedium\n\n\n5\nS02\nlow\n\n\n6\nS03\nhigh\n\n\n7\nS03\nlow\n\n\n8\nS04\nhigh\n\n\n9\nS04\nmedium\n\n\n10\nS04\nlow\n\n\n11\nS05\nhigh\n\n\n12\nS05\nlow\n\n\n13\nS06\nhigh\n\n\n⋮\n⋮\n⋮\n\n\n89\nS36\nmedium\n\n\n90\nS36\nlow\n\n\n91\nS37\nhigh\n\n\n92\nS37\nlow\n\n\n93\nS38\nhigh\n\n\n94\nS38\nmedium\n\n\n95\nS38\nlow\n\n\n96\nS39\nhigh\n\n\n97\nS39\nlow\n\n\n98\nS40\nhigh\n\n\n99\nS40\nmedium\n\n\n100\nS40\nlow\n\n\n\n\n\n\n\n\nCode\nm1 = let contrasts, form\n contrasts = Dict(:frequency => HelmertCoding(base=\"high\"))\n form = @formula(dv ~ 1 + frequency +\n (1 + frequency | subj))\n fit(MixedModel, form, design_partial; contrasts)\nend\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_subj\n\n\n(Intercept)\n1.2638\n0.2825\n4.47\n<1e-05\n1.3155\n\n\nfrequency: low\n-2.9511\n0.2351\n-12.56\n<1e-35\n1.2017\n\n\nfrequency: medium\n-1.9075\n0.3034\n-6.29\n<1e-09\n1.5740\n\n\nResidual\n1.2377\n\n\n\n\n\n\n\n\n\n\n\nCode\nshrinkageplot(m1)\n\n\n\n\n\n\n\n\n\n\n\nCode\ncaterpillar(m1; orderby=nothing, vline_at_zero=true)\n\n\n\n\n\n\n\n\n\n\n\n\n Back to top", + "crumbs": [ + "Worked examples", + "Partially-within subjects designs" + ] + }, + { + "objectID": "mrk17.html", + "href": "mrk17.html", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "", + "text": "Packages we (might) use.\n\nusing CategoricalArrays\nusing DataFrames\nusing MixedModels\nusing MixedModelsMakie\nusing SMLP2024: dataset\nusing Statistics: mean, std\n\n\ndat = DataFrame(dataset(:mrk17_exp1))\ndescribe(dat)\n\n9×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nDataType\n\n\n\n\n1\nsubj\n\nS01\n\nS73\n0\nString\n\n\n2\nitem\n\nACHE\n\nYEAR\n0\nString\n\n\n3\ntrial\n239.958\n2\n239.0\n480\n0\nInt16\n\n\n4\nF\n\nHF\n\nLF\n0\nString\n\n\n5\nP\n\nrel\n\nunr\n0\nString\n\n\n6\nQ\n\nclr\n\ndeg\n0\nString\n\n\n7\nlQ\n\nclr\n\ndeg\n0\nString\n\n\n8\nlT\n\nNW\n\nWD\n0\nString\n\n\n9\nrt\n647.173\n301\n601.0\n2994\n0\nInt16", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#response-covariates-and-factors", + "href": "mrk17.html#response-covariates-and-factors", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "2.1 Response, covariates, and factors", + "text": "2.1 Response, covariates, and factors\nLinear mixed models (LMMs), like many other types of statistical models, describe a relationship between a response variable and covariates that have been measured or observed along with the response. The statistical model assumes that the residuals of the fitted response (i.e., not the responses) are normally – also identically and independently – distributed. This is the first assumption of normality in the LMM. It is standard practice that model residuals are inspected and, if serious skew is indicated, that the response is Box-Cox transformed (unless not justified for theoretical reasons) to fulfill this model assumption.\nIn the following we distinguish between categorical covariates and numerical covariates. Categorical covariates are factors. The important characteristic of a factor is that, for each observed value of the response, the factor takes on the value of one of a set of discrete levels. The levels can be unordered (nominal) or ordered (ordinal). We use the term covariate when we refer to numerical covariates, that is to continuous measures with some distribution. In principle, statistical models are not constrained by the distribution of observations across levels of factors and covariates, but the distribution may lead to problems of model identification and it does implications for statistical power.\nStatistical power, especially for the detection of interactions, is best when observations are uniformly distributed across levels of factors or uniform across the values of covariates. In experimental designs, uniform distributions may be achieved by balanced assignment of subjects (or other carriers of responses) to the levels of factors or combinations of factor levels. In observational contexts, we achieve uniform distributions by stratification (e..g., on age, gender, or IQ scores). Statistical power is worse for skewed than normal distributions (I think …). Therefore, although it is not required to meet an assumption of the statistical model, it may be useful to consider Box-Cox transformations of covariates.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#nested-and-crossed-random-grouping-factors", + "href": "mrk17.html#nested-and-crossed-random-grouping-factors", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "2.2 Nested and crossed random (grouping) factors", + "text": "2.2 Nested and crossed random (grouping) factors\nIn LMMs the levels of at least one of the factors represents units in the data set that are assumed to be sampled, ideally randomly, from a population that is normally distributed with respect to the response. This is the second assumption of normal distribution in LMMs. In psychology and linguistics the observational units are often the subjects or items (e..g., texts, sentences, words, pictures) in the study. We may use numbers, such as subject identifiers, to designate the particular levels that we observed; we recommend to prepend these numbers with “S” or “I” to avoid confusion with numeric variables.\nRandom sampling is the basis of generalization from the sample to the population. The core statistics we will estimate in this context are variances and correlations of grand means and (quasi-)experimental effects. These terms will be explained below. What we want to stress here is that the estimation of (co-)variances / correlations requires a larger number of units (levels) than the estimation of means. Therefore, from a practical perspective, it is important that random factors are represented with many units.\nWhen there is more than one random factor, we must be clear about their relation. The two prototypical cases are that the factors are nested or crossed. In multilevel models, a special case of mixed models, the levels of the random factors are strictly nested. For example, at a given time, every student attends a specific class in a specific school. Students, classes, and schools could be three random factors. As soon as we look at this scenario across several school years, the nesting quickly falls apart because students may move between classes and between schools.\nIn psychology and linguistics, random factors are often crossed, for example, when every subject reads every word in every sentence in a word-by-word self-paced reading experiment (or alternatively: when every word in every sentence elicits a response from every subject). However, in an eye-movement experiment (for example), the perfect crossing on a measure like fixation duration is not attainable because of blinks or skipping of words.\nIn summary, the typical situation in experimental and observational studies with more than one random factor is partial crossing or partial nesting of levels of the random factors. Linear mixed models handle these situations very well.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#experimental-and-quasi-experimental-fixed-factors-covariates", + "href": "mrk17.html#experimental-and-quasi-experimental-fixed-factors-covariates", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "2.3 Experimental and quasi-experimental fixed factors / covariates", + "text": "2.3 Experimental and quasi-experimental fixed factors / covariates\nFixed experimental factor or covariate. In experiments the units (or levels) of the random factor(s) are assigned to manipulations implemented in their design. The researcher controls the assignment of units of the random factor(s) (e.g., subjects, items) to experimental manipulations. These manipulations are represented as factors with a fixed and discrete set of levels (e.g., training vs. control group) or as covariates associated with continuous numeric values (e.g., presentation times).\nFixed quasi-experimental factor or covariate. In observational studies (which can also be experiments) the units (or levels) of random factors may “bring along” characteristics that represent the levels of quasi-experimental factors or covariates beyond the control of the researcher. Whether a a subject is female, male, or diverse or whether a word is a noun, a verb, or an adjective are examples of quasi-experimental factors of gender or word type, respectively. Subject-related covariates are body height, body mass, and IQ scores; word-related covariates are their lengths, frequency, and cloze predictability.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#between-unit-and-within-unit-factors-covariates", + "href": "mrk17.html#between-unit-and-within-unit-factors-covariates", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "2.4 Between-unit and within-unit factors / covariates", + "text": "2.4 Between-unit and within-unit factors / covariates\nThe distinction between between-unit and within-unit factors is always relative to a random (grouping) factor of an experimental design. A between-unit factor / covariate is a factor for which every unit of the random factor is assigned to or characterized by only one level of the factor. A within-unit factor is a factor for which units of the random factor appear at every level of the factor.\nFor the typical random factor, say Subject, there is little ambiguity because we are used to the between-within distinction from ANOVAs, more specifically the F1-ANOVA. In psycholinguistics, there is the tradition to test effects also for the second random factor Item in an F2-ANOVA. Importantly, for a given fixed factor all four combinations are possible. For example, Gender is a fixed quasi-experimental between-subject / within-item factor; word frequency is a fixed quasi-experimental within-subject / between-item covariate; Prime-target relation is a fixed experimental within-subject / within-item factor (assuming that targets are presented both in a primed and in an unprimed situation); and when a training manipulation is defined by the items used in the training, then in a training-control group design, the fixed factor Group is a fixed experimental between-subject / between-item factor.\nThese distinctions are critical for setting up LMMs because variance components for (quasi-)experimental effects can only be specified for within-unit effects. Note also that loss of data (within limits), counterbalancing or blocking of items are irrelevant for these definitions.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#factor-based-contrasts-and-covariate-based-trends", + "href": "mrk17.html#factor-based-contrasts-and-covariate-based-trends", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "2.5 Factor-based contrasts and covariate-based trends", + "text": "2.5 Factor-based contrasts and covariate-based trends\nThe simplest fixed factor has two levels and the model estimates the difference between them. When we move to factors with k levels, we must decide on how we spend the k-1 degrees of freedom, that is we must specify a set of contrasts. (If we don’t do it, the program chooses DummyCoding contrasts for us.)\nThe simplest specification of a covariate is to include its linear trend, that is its slope. The slope (like a contrast) represents a difference score, that is the change in response to a one-unit change on the covariate. For covariates we must decide on the order of the trend we want to model.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#contrast--and-trend-based-fixed-effect-model-parameters", + "href": "mrk17.html#contrast--and-trend-based-fixed-effect-model-parameters", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "2.6 Contrast- and trend-based fixed-effect model parameters", + "text": "2.6 Contrast- and trend-based fixed-effect model parameters\nFixed factors and covariates are expected to have effects on the response. Fixed-effect model parameters estimate the hypothesized main and interaction effects of the study. The estimates of factors are based on contrasts; the estimates of covariates are based on trends. Conceptually, they correspond to unstandardized regression coefficients in multiple regression.\nThe intercept is a special regression coefficient; it estimates the value of the dependent variable when all fixed effects associated with factors and trends associated with covariates are zero. In experimental designs with higher-order interactions there is an advantage of specifying the LMM in such a way that the intercept estimates the grand mean (GM; mean of the means of design cells). This happens if (a) contrasts for factors are chosen such that the intercept estimates the GM (positive: EffectsCoding, SeqDifferenceCoding, or HelmertCoding contrasts; negative: DummyCoding), (b) orthogonal polynomial trends are used (Helmert, anova-based), and (c) covariates are centered on their mean before inclusion in the model. As always, there may be good theoretical reasons to depart from the default recommendation.\nThe specification of contrasts / trends does not depend on the status of the fixed factor / covariate. It does not matter whether a factor varies between or within the units of a random factor or whether it is an experimental or quasi-experimental factor. Contrasts are not specified for random (grouping) factors.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#variance-components-vcs-and-correlation-parameters-cps", + "href": "mrk17.html#variance-components-vcs-and-correlation-parameters-cps", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "2.7 Variance components (VCs) and correlation parameters (CPs)", + "text": "2.7 Variance components (VCs) and correlation parameters (CPs)\nVariance components (VCs) and correlation parameters (CPs) are within-group model parameters; they correspond to (some of the) within-unit (quasi-)experimental fixed-effect model parameters. Thus, we may be able to estimate a subject-related VC for word frequency. If we included a linear trend for word frequency, the VC estimates the subject-related variance in these slopes. We cannot estimate an item-related VC for the word-frequency slopes because there is only one frequency associated with words. Analogously, we may able to estimate an item-related VC for the effect of Group (training vs. control), but we cannot estimate a subject-related VC for this effect.\nThe within-between characteristics of fixed factors and covariates relative to the random factor(s) are features of the design of the experiment or observational study. They fundamentally constrain the specification of the LMM. That’s why it is of upmost importance to be absolutely clear about their status.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#conditional-modes-of-random-effects", + "href": "mrk17.html#conditional-modes-of-random-effects", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "2.8 Conditional modes of random effects", + "text": "2.8 Conditional modes of random effects\nIn this outline of the dimensions underlying the specification of an LMM, we have said nothing so far about the conditional modes of random effects (i.e., the results shown in caterpillar and shrinkage plots). They are not needed for model specification or model selection.\nThe VC is the prior variance of the random effects, whereas var(ranef(model)) is the variance of the posterior means/modes of the random effects. See Kliegl et al. (2010, VisualCognition); Rizopoulos (2019, stackexchange.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#abstract", + "href": "mrk17.html#abstract", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "3.1 Abstract", + "text": "3.1 Abstract\nThis semantic-priming experiment was reported in Masson, Rabe, & Kliegl (2017, Exp. 1, Memory & Cognition). It is a direct replication of an experiment reported in Masson & Kliegl (2013, Exp. 1, JEPLMC). Following a prime word a related or unrelated high- or low-frequency target word or a nonword was presented in clear or dim font. The subject’s task was to decide as quickly as possible whether the target was a word or a nonword, that is subjects performed a lexical decision task (LDT). The reaction time and the accuracy of the response were recorded. Only correct reaction times to words are included. After filtering there were 16,409 observations recorded from 73 subjects and 240 items.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#codebook", + "href": "mrk17.html#codebook", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "3.2 Codebook", + "text": "3.2 Codebook\nThe data (variables and observations) used by Masson et al. (2017) are available in file MRK17_Exp1.RDS\n\n\n\nVariable\nDescription\n\n\n\n\nSubj\nSubject identifier\n\n\nItem\nTarget (non-)word\n\n\ntrial\nTrial number\n\n\nF\nTarget frequency is high or low\n\n\nP\nPrime is related or unrelated to target\n\n\nQ\nTarget quality is clear or degraded\n\n\nlQ\nLast-trial target quality is clear or degraded\n\n\nlT\nLast-trail target requires word or nonword response\n\n\nrt\nReaction time [ms]\n\n\n\nlagQlty and lagTrgt refer to experimental conditions in the last trial.\nCorresponding indicator variables (-1/+1):", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#model-fit", + "href": "mrk17.html#model-fit", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "5.1 Model fit", + "text": "5.1 Model fit\n\ncontrasts =\n Dict( :F => EffectsCoding(; levels=[\"LF\", \"HF\"]) ,\n :P => EffectsCoding(; levels=[\"unr\", \"rel\"]),\n :Q => EffectsCoding(; levels=[\"deg\", \"clr\"]),\n :lQ =>EffectsCoding(; levels=[\"deg\", \"clr\"]),\n :lT =>EffectsCoding(; levels=[\"NW\", \"WD\"])\n );\n\nm_cpx = let\n form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +\n (1+F+P+Q+lQ+lT | subj) +\n (1 +P+Q+lQ+lT | item);\n fit(MixedModel, form, dat; contrasts)\nend\n\nVarCorr(m_cpx)\n\nMinimizing 2 Time: 0:00:00 ( 0.47 s/it)\n objective: 12236.319236009644\nMinimizing 94 Time: 0:00:02 (28.47 ms/it)\n objective: 7249.120182300852\nMinimizing 118 Time: 0:00:02 (25.01 ms/it)\n objective: 7291.763575891249\nMinimizing 127 Time: 0:00:03 (24.04 ms/it)\n objective: 7260.015819733898\nMinimizing 136 Time: 0:00:03 (23.33 ms/it)\n objective: 7265.705633144413\nMinimizing 144 Time: 0:00:03 (22.77 ms/it)\n objective: 7250.4488876317155\nMinimizing 153 Time: 0:00:03 (22.14 ms/it)\n objective: 7225.734296682795\nMinimizing 162 Time: 0:00:03 (21.57 ms/it)\n objective: 7282.113857963857\nMinimizing 171 Time: 0:00:03 (21.07 ms/it)\n objective: 7194.502401177749\nMinimizing 180 Time: 0:00:03 (20.63 ms/it)\n objective: 7191.653078945403\nMinimizing 188 Time: 0:00:03 (20.30 ms/it)\n objective: 7189.129797213707\nMinimizing 197 Time: 0:00:03 (19.91 ms/it)\n objective: 7188.046411656474\nMinimizing 206 Time: 0:00:04 (19.54 ms/it)\n objective: 7187.413963547601\nMinimizing 215 Time: 0:00:04 (19.20 ms/it)\n objective: 7185.895171538268\nMinimizing 224 Time: 0:00:04 (18.89 ms/it)\n objective: 7185.392720281486\nMinimizing 233 Time: 0:00:04 (18.60 ms/it)\n objective: 7184.568988840304\nMinimizing 242 Time: 0:00:04 (18.34 ms/it)\n objective: 7182.692960774513\nMinimizing 251 Time: 0:00:04 (18.11 ms/it)\n objective: 7182.522314101793\nMinimizing 260 Time: 0:00:04 (17.88 ms/it)\n objective: 7182.29089946668\nMinimizing 269 Time: 0:00:04 (17.68 ms/it)\n objective: 7181.754508324871\nMinimizing 278 Time: 0:00:04 (17.47 ms/it)\n objective: 7180.945383345449\nMinimizing 287 Time: 0:00:04 (17.29 ms/it)\n objective: 7180.373542670413\nMinimizing 296 Time: 0:00:05 (17.12 ms/it)\n objective: 7179.226769182374\nMinimizing 305 Time: 0:00:05 (16.96 ms/it)\n objective: 7178.784230042712\nMinimizing 314 Time: 0:00:05 (16.81 ms/it)\n objective: 7177.594055525869\nMinimizing 323 Time: 0:00:05 (16.67 ms/it)\n objective: 7177.577846900931\nMinimizing 332 Time: 0:00:05 (16.53 ms/it)\n objective: 7176.851940538352\nMinimizing 341 Time: 0:00:05 (16.40 ms/it)\n objective: 7176.3684658895745\nMinimizing 350 Time: 0:00:05 (16.28 ms/it)\n objective: 7175.803517624161\nMinimizing 359 Time: 0:00:05 (16.16 ms/it)\n objective: 7174.947370512575\nMinimizing 368 Time: 0:00:05 (16.06 ms/it)\n objective: 7173.799157161966\nMinimizing 377 Time: 0:00:06 (15.95 ms/it)\n objective: 7172.044003247749\nMinimizing 386 Time: 0:00:06 (15.86 ms/it)\n objective: 7170.623702565538\nMinimizing 395 Time: 0:00:06 (15.76 ms/it)\n objective: 7167.498474823836\nMinimizing 405 Time: 0:00:06 (15.65 ms/it)\n objective: 7166.234236515456\nMinimizing 414 Time: 0:00:06 (15.56 ms/it)\n objective: 7163.279053940776\nMinimizing 423 Time: 0:00:06 (15.47 ms/it)\n objective: 7166.484821169435\nMinimizing 432 Time: 0:00:06 (15.39 ms/it)\n objective: 7157.994744801014\nMinimizing 441 Time: 0:00:06 (15.32 ms/it)\n objective: 7156.9352798284035\nMinimizing 449 Time: 0:00:06 (15.28 ms/it)\n objective: 7153.823112245293\nMinimizing 458 Time: 0:00:06 (15.22 ms/it)\n objective: 7150.3162245249505\nMinimizing 467 Time: 0:00:07 (15.16 ms/it)\n objective: 7149.935835371041\nMinimizing 476 Time: 0:00:07 (15.09 ms/it)\n objective: 7149.555923262043\nMinimizing 485 Time: 0:00:07 (15.02 ms/it)\n objective: 7149.399721863012\nMinimizing 494 Time: 0:00:07 (15.04 ms/it)\n objective: 7149.087433332951\nMinimizing 502 Time: 0:00:07 (15.02 ms/it)\n objective: 7148.656317523455\nMinimizing 510 Time: 0:00:07 (15.00 ms/it)\n objective: 7148.592065632587\nMinimizing 518 Time: 0:00:07 (14.99 ms/it)\n objective: 7148.531024188875\nMinimizing 525 Time: 0:00:07 (14.99 ms/it)\n objective: 7148.461544048104\nMinimizing 533 Time: 0:00:07 (14.96 ms/it)\n objective: 7148.3530388399\nMinimizing 542 Time: 0:00:08 (14.90 ms/it)\n objective: 7148.2535865238815\nMinimizing 552 Time: 0:00:08 (14.83 ms/it)\n objective: 7148.096511008455\nMinimizing 560 Time: 0:00:08 (14.83 ms/it)\n objective: 7148.087202572238\nMinimizing 567 Time: 0:00:08 (14.83 ms/it)\n objective: 7147.968226099975\nMinimizing 575 Time: 0:00:08 (14.80 ms/it)\n objective: 7147.907102954974\nMinimizing 583 Time: 0:00:08 (14.78 ms/it)\n objective: 7147.8802277912\nMinimizing 591 Time: 0:00:08 (14.76 ms/it)\n objective: 7147.8572204082575\nMinimizing 599 Time: 0:00:08 (14.76 ms/it)\n objective: 7147.843379811635\nMinimizing 608 Time: 0:00:08 (14.72 ms/it)\n objective: 7147.814327096406\nMinimizing 617 Time: 0:00:09 (14.67 ms/it)\n objective: 7147.786617758407\nMinimizing 626 Time: 0:00:09 (14.63 ms/it)\n objective: 7147.746216287041\nMinimizing 635 Time: 0:00:09 (14.58 ms/it)\n objective: 7147.72352665989\nMinimizing 645 Time: 0:00:09 (14.53 ms/it)\n objective: 7147.706120026539\nMinimizing 655 Time: 0:00:09 (14.47 ms/it)\n objective: 7147.685331593435\nMinimizing 665 Time: 0:00:09 (14.42 ms/it)\n objective: 7147.667208014207\nMinimizing 674 Time: 0:00:09 (14.38 ms/it)\n objective: 7147.655838866062\nMinimizing 684 Time: 0:00:09 (14.33 ms/it)\n objective: 7147.648682816166\nMinimizing 694 Time: 0:00:09 (14.29 ms/it)\n objective: 7147.643906748403\nMinimizing 704 Time: 0:00:10 (14.25 ms/it)\n objective: 7147.640036536273\nMinimizing 713 Time: 0:00:10 (14.22 ms/it)\n objective: 7147.649157243677\nMinimizing 723 Time: 0:00:10 (14.17 ms/it)\n objective: 7147.641059189774\nMinimizing 733 Time: 0:00:10 (14.13 ms/it)\n objective: 7147.632414838774\nMinimizing 743 Time: 0:00:10 (14.09 ms/it)\n objective: 7147.630874786184\nMinimizing 752 Time: 0:00:10 (14.05 ms/it)\n objective: 7147.629970859568\nMinimizing 762 Time: 0:00:10 (14.01 ms/it)\n objective: 7147.629159600505\nMinimizing 772 Time: 0:00:10 (13.97 ms/it)\n objective: 7147.62765579575\nMinimizing 782 Time: 0:00:10 (13.93 ms/it)\n objective: 7147.626155045783\nMinimizing 792 Time: 0:00:11 (13.90 ms/it)\n objective: 7147.625473967293\nMinimizing 801 Time: 0:00:11 (13.87 ms/it)\n objective: 7147.624735383692\nMinimizing 811 Time: 0:00:11 (13.83 ms/it)\n objective: 7147.623986570402\nMinimizing 821 Time: 0:00:11 (13.80 ms/it)\n objective: 7147.623752366062\nMinimizing 831 Time: 0:00:11 (13.76 ms/it)\n objective: 7147.623424582311\nMinimizing 841 Time: 0:00:11 (13.73 ms/it)\n objective: 7147.623085873498\nMinimizing 851 Time: 0:00:11 (13.70 ms/it)\n objective: 7147.622338565887\nMinimizing 861 Time: 0:00:11 (13.66 ms/it)\n objective: 7147.621927586488\nMinimizing 871 Time: 0:00:11 (13.63 ms/it)\n objective: 7147.621667937003\nMinimizing 881 Time: 0:00:11 (13.60 ms/it)\n objective: 7147.621486368276\nMinimizing 890 Time: 0:00:12 (13.58 ms/it)\n objective: 7147.621245138782\nMinimizing 900 Time: 0:00:12 (13.56 ms/it)\n objective: 7147.62111766812\nMinimizing 909 Time: 0:00:12 (13.54 ms/it)\n objective: 7147.6208043017605\nMinimizing 918 Time: 0:00:12 (13.52 ms/it)\n objective: 7147.620751938777\nMinimizing 927 Time: 0:00:12 (13.52 ms/it)\n objective: 7147.62059723712\nMinimizing 935 Time: 0:00:12 (13.53 ms/it)\n objective: 7147.620403036227\nMinimizing 944 Time: 0:00:12 (13.51 ms/it)\n objective: 7147.620600623897\nMinimizing 953 Time: 0:00:12 (13.49 ms/it)\n objective: 7147.619235369456\nMinimizing 962 Time: 0:00:12 (13.46 ms/it)\n objective: 7147.619017957963\nMinimizing 971 Time: 0:00:13 (13.44 ms/it)\n objective: 7147.617286513275\nMinimizing 981 Time: 0:00:13 (13.42 ms/it)\n objective: 7147.617946449571\nMinimizing 991 Time: 0:00:13 (13.39 ms/it)\n objective: 7147.614530798965\nMinimizing 1001 Time: 0:00:13 (13.36 ms/it)\n objective: 7147.612787960818\nMinimizing 1011 Time: 0:00:13 (13.34 ms/it)\n objective: 7147.606365572075\nMinimizing 1021 Time: 0:00:13 (13.32 ms/it)\n objective: 7147.605624336107\nMinimizing 1031 Time: 0:00:13 (13.29 ms/it)\n objective: 7147.598997387338\nMinimizing 1040 Time: 0:00:13 (13.28 ms/it)\n objective: 7147.594605025298\nMinimizing 1049 Time: 0:00:13 (13.26 ms/it)\n objective: 7147.589251057229\nMinimizing 1058 Time: 0:00:14 (13.24 ms/it)\n objective: 7147.587092246818\nMinimizing 1067 Time: 0:00:14 (13.23 ms/it)\n objective: 7147.583649293367\nMinimizing 1077 Time: 0:00:14 (13.20 ms/it)\n objective: 7147.579369436586\nMinimizing 1087 Time: 0:00:14 (13.18 ms/it)\n objective: 7147.5754409231195\nMinimizing 1097 Time: 0:00:14 (13.16 ms/it)\n objective: 7147.572885912015\nMinimizing 1107 Time: 0:00:14 (13.14 ms/it)\n objective: 7147.569409032434\nMinimizing 1117 Time: 0:00:14 (13.12 ms/it)\n objective: 7147.567171456462\nMinimizing 1127 Time: 0:00:14 (13.10 ms/it)\n objective: 7147.566020002094\nMinimizing 1137 Time: 0:00:14 (13.08 ms/it)\n objective: 7147.564517396884\nMinimizing 1147 Time: 0:00:14 (13.07 ms/it)\n objective: 7147.563380941506\nMinimizing 1156 Time: 0:00:15 (13.05 ms/it)\n objective: 7147.562783241351\nMinimizing 1165 Time: 0:00:15 (13.04 ms/it)\n objective: 7147.561684794379\nMinimizing 1174 Time: 0:00:15 (13.02 ms/it)\n objective: 7147.560752567861\nMinimizing 1183 Time: 0:00:15 (13.01 ms/it)\n objective: 7147.560016035949\nMinimizing 1192 Time: 0:00:15 (13.00 ms/it)\n objective: 7147.559163946607\nMinimizing 1201 Time: 0:00:15 (12.98 ms/it)\n objective: 7147.559037397831\nMinimizing 1210 Time: 0:00:15 (12.97 ms/it)\n objective: 7147.558254661318\nMinimizing 1219 Time: 0:00:15 (12.96 ms/it)\n objective: 7147.5581615072715\nMinimizing 1229 Time: 0:00:15 (12.94 ms/it)\n objective: 7147.557388884788\nMinimizing 1239 Time: 0:00:16 (12.93 ms/it)\n objective: 7147.556740704144\nMinimizing 1249 Time: 0:00:16 (12.91 ms/it)\n objective: 7147.556070237347\nMinimizing 1259 Time: 0:00:16 (12.90 ms/it)\n objective: 7147.5569175676865\nMinimizing 1269 Time: 0:00:16 (12.88 ms/it)\n objective: 7147.555004775328\nMinimizing 1279 Time: 0:00:16 (12.87 ms/it)\n objective: 7147.554706891277\nMinimizing 1289 Time: 0:00:16 (12.85 ms/it)\n objective: 7147.554257919288\nMinimizing 1299 Time: 0:00:16 (12.84 ms/it)\n objective: 7147.553951891185\nMinimizing 1309 Time: 0:00:16 (12.82 ms/it)\n objective: 7147.553362379362\nMinimizing 1319 Time: 0:00:16 (12.81 ms/it)\n objective: 7147.553160958638\nMinimizing 1328 Time: 0:00:16 (12.80 ms/it)\n objective: 7147.55287535691\nMinimizing 1338 Time: 0:00:17 (12.78 ms/it)\n objective: 7147.552613773639\nMinimizing 1348 Time: 0:00:17 (12.77 ms/it)\n objective: 7147.552311581183\nMinimizing 1358 Time: 0:00:17 (12.75 ms/it)\n objective: 7147.551816636063\nMinimizing 1367 Time: 0:00:17 (12.75 ms/it)\n objective: 7147.551755227561\nMinimizing 1377 Time: 0:00:17 (12.73 ms/it)\n objective: 7147.551631565507\nMinimizing 1387 Time: 0:00:17 (12.72 ms/it)\n objective: 7147.551611470475\nMinimizing 1397 Time: 0:00:17 (12.71 ms/it)\n objective: 7147.5513987918075\nMinimizing 1407 Time: 0:00:17 (12.69 ms/it)\n objective: 7147.551219481732\nMinimizing 1417 Time: 0:00:17 (12.68 ms/it)\n objective: 7147.551171980433\nMinimizing 1427 Time: 0:00:18 (12.67 ms/it)\n objective: 7147.551061215947\nMinimizing 1437 Time: 0:00:18 (12.66 ms/it)\n objective: 7147.551117184621\nMinimizing 1447 Time: 0:00:18 (12.65 ms/it)\n objective: 7147.55091185511\nMinimizing 1457 Time: 0:00:18 (12.63 ms/it)\n objective: 7147.550822764177\nMinimizing 1467 Time: 0:00:18 (12.62 ms/it)\n objective: 7147.5507422044675\nMinimizing 1477 Time: 0:00:18 (12.61 ms/it)\n objective: 7147.550911690253\nMinimizing 1487 Time: 0:00:18 (12.60 ms/it)\n objective: 7147.550803823427\nMinimizing 1497 Time: 0:00:18 (12.58 ms/it)\n objective: 7147.550676876308\nMinimizing 1507 Time: 0:00:18 (12.57 ms/it)\n objective: 7147.55061104588\nMinimizing 1517 Time: 0:00:19 (12.56 ms/it)\n objective: 7147.5506926822745\nMinimizing 1527 Time: 0:00:19 (12.55 ms/it)\n objective: 7147.550577280057\nMinimizing 1537 Time: 0:00:19 (12.54 ms/it)\n objective: 7147.550555210411\nMinimizing 1547 Time: 0:00:19 (12.53 ms/it)\n objective: 7147.5505321694445\nMinimizing 1557 Time: 0:00:19 (12.52 ms/it)\n objective: 7147.550605309394\nMinimizing 1567 Time: 0:00:19 (12.51 ms/it)\n objective: 7147.550497050396\nMinimizing 1577 Time: 0:00:19 (12.50 ms/it)\n objective: 7147.550527636891\nMinimizing 1587 Time: 0:00:19 (12.49 ms/it)\n objective: 7147.550474180846\nMinimizing 1597 Time: 0:00:19 (12.48 ms/it)\n objective: 7147.550447320714\nMinimizing 1607 Time: 0:00:20 (12.47 ms/it)\n objective: 7147.550437475353\nMinimizing 1617 Time: 0:00:20 (12.46 ms/it)\n objective: 7147.55043172267\nMinimizing 1625 Time: 0:00:20 (12.46 ms/it)\n objective: 7147.550417227425\nMinimizing 1635 Time: 0:00:20 (12.45 ms/it)\n objective: 7147.550407555644\nMinimizing 1645 Time: 0:00:20 (12.44 ms/it)\n objective: 7147.5504031649125\nMinimizing 1654 Time: 0:00:20 (12.43 ms/it)\n objective: 7147.550395744943\nMinimizing 1664 Time: 0:00:20 (12.43 ms/it)\n objective: 7147.550391444152\nMinimizing 1674 Time: 0:00:20 (12.42 ms/it)\n objective: 7147.5503806632105\nMinimizing 1684 Time: 0:00:20 (12.41 ms/it)\n objective: 7147.550385842741\nMinimizing 1694 Time: 0:00:21 (12.40 ms/it)\n objective: 7147.550371837158\nMinimizing 1703 Time: 0:00:21 (12.40 ms/it)\n objective: 7147.550364911636\nMinimizing 1712 Time: 0:00:21 (12.39 ms/it)\n objective: 7147.550363162931\nMinimizing 1717 Time: 0:00:21 (12.41 ms/it)\n\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\nitem\n(Intercept)\n0.00320224\n0.05658830\n\n\n\n\n\n\n\n\nP: rel\n0.00012892\n0.01135418\n+0.05\n\n\n\n\n\n\n\nQ: clr\n0.00015999\n0.01264876\n+0.36\n+0.38\n\n\n\n\n\n\nlQ: clr\n0.00003476\n0.00589548\n+0.38\n+0.03\n+0.03\n\n\n\n\n\nlT: WD\n0.00015750\n0.01254988\n-0.11\n-0.87\n-0.01\n-0.35\n\n\n\nsubj\n(Intercept)\n0.03061783\n0.17497951\n\n\n\n\n\n\n\n\nF: HF\n0.00004440\n0.00666334\n+0.36\n\n\n\n\n\n\n\nP: rel\n0.00012734\n0.01128461\n+0.35\n+0.89\n\n\n\n\n\n\nQ: clr\n0.00078985\n0.02810434\n+0.41\n+0.45\n+0.73\n\n\n\n\n\nlQ: clr\n0.00011615\n0.01077740\n+0.06\n+0.17\n+0.18\n+0.58\n\n\n\n\nlT: WD\n0.00104531\n0.03233128\n+0.26\n-0.10\n-0.02\n+0.37\n+0.51\n\n\nResidual\n\n0.08569955\n0.29274486\n\n\n\n\n\n\n\n\n\n\n\nissingular(m_cpx)\n\ntrue\n\n\n\nMixedModels.PCA(m_cpx)\n\n(item = \nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . .\n P: rel 0.05 1.0 . . .\n Q: clr 0.36 0.38 1.0 . .\n lQ: clr 0.38 0.03 0.03 1.0 .\n lT: WD -0.11 -0.87 -0.01 -0.35 1.0\n\nNormalized cumulative variances:\n[0.4223, 0.6865, 0.906, 1.0, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5\n (Intercept) -0.3 -0.67 0.02 -0.67 0.07\n P: rel -0.6 0.38 0.22 -0.05 0.67\n Q: clr -0.31 -0.34 0.7 0.46 -0.28\n lQ: clr -0.31 -0.41 -0.62 0.55 0.19\n lT: WD 0.6 -0.34 0.27 0.15 0.66, subj = \nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . . .\n F: HF 0.36 1.0 . . . .\n P: rel 0.35 0.89 1.0 . . .\n Q: clr 0.41 0.45 0.73 1.0 . .\n lQ: clr 0.06 0.17 0.18 0.58 1.0 .\n lT: WD 0.26 -0.1 -0.02 0.37 0.51 1.0\n\nNormalized cumulative variances:\n[0.4768, 0.7358, 0.8812, 0.9431, 1.0, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5 PC6\n (Intercept) -0.34 0.02 0.84 0.31 -0.27 -0.11\n F: HF -0.45 0.42 -0.12 -0.41 -0.44 0.5\n P: rel -0.51 0.35 -0.16 -0.17 0.27 -0.7\n Q: clr -0.52 -0.15 -0.11 0.41 0.57 0.45\n lQ: clr -0.32 -0.51 -0.43 0.3 -0.56 -0.21\n lT: WD -0.21 -0.65 0.24 -0.67 0.16 -0.0)\n\n\nVariance-covariance matrix of random-effect structure suggests overparameterization for both subject-related and item-related components.\nWe don’t look at fixed effects before model selection.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#vcs-and-cps", + "href": "mrk17.html#vcs-and-cps", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "5.2 VCs and CPs", + "text": "5.2 VCs and CPs\nWe can also look separately at item- and subj-related VCs and CPs for subjects and items.\n\nfirst(m_cpx.λ)\n\n5×5 LinearAlgebra.LowerTriangular{Float64, Matrix{Float64}}:\n 0.193302 ⋅ ⋅ ⋅ ⋅ \n 0.00210882 0.0387279 ⋅ ⋅ ⋅ \n 0.0155967 0.0156512 0.0371304 ⋅ ⋅ \n 0.00757715 0.000160552 -0.002579 0.018479 ⋅ \n -0.00483918 -0.037088 0.0174059 -0.011658 0.0\n\n\nVP is zero for last diagonal entry; not supported by data.\n\nlast(m_cpx.λ)\n\n6×6 LinearAlgebra.LowerTriangular{Float64, Matrix{Float64}}:\n 0.59772 ⋅ ⋅ ⋅ ⋅ ⋅ \n 0.00814528 0.0212543 ⋅ ⋅ ⋅ ⋅ \n 0.0134425 0.0317765 0.0171892 ⋅ ⋅ ⋅ \n 0.0392834 0.0309145 0.0686994 0.0446995 ⋅ ⋅ \n 0.00205584 0.0057383 0.00227977 0.0362352 0.0 ⋅ \n 0.0283307 -0.0222477 0.0131949 0.0578621 0.0731409 0.0450343\n\n\nVP is zero for fourth diagonal entry; not supported by data.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#model-fit-1", + "href": "mrk17.html#model-fit-1", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "6.1 Model fit", + "text": "6.1 Model fit\nWe take out correlation parameters.\n\nm_zcp = let\n form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +\n zerocorr(1+F+P+Q+lQ+lT | subj) +\n zerocorr(1 +P+Q+lQ+lT | item);\n fit(MixedModel, form, dat; contrasts)\nend\n\nVarCorr(m_zcp)\nissingular(m_zcp)\nMixedModels.PCA(m_zcp)\n\nMinimizing 10 Time: 0:00:00 (10.81 ms/it)\n objective: 12062.171283470983\nMinimizing 21 Time: 0:00:00 (10.21 ms/it)\n objective: 11804.406038310086\nMinimizing 32 Time: 0:00:00 (10.05 ms/it)\n objective: 7418.254529819783\nMinimizing 43 Time: 0:00:00 (10.01 ms/it)\n objective: 7303.449354651699\nMinimizing 54 Time: 0:00:00 ( 9.99 ms/it)\n objective: 7248.369118002118\nMinimizing 65 Time: 0:00:00 ( 9.97 ms/it)\n objective: 7228.106822778762\nMinimizing 76 Time: 0:00:00 ( 9.94 ms/it)\n objective: 7221.239180885949\nMinimizing 87 Time: 0:00:00 ( 9.91 ms/it)\n objective: 7216.145839563162\nMinimizing 98 Time: 0:00:00 ( 9.92 ms/it)\n objective: 7210.857335837595\nMinimizing 109 Time: 0:00:01 ( 9.90 ms/it)\n objective: 7203.337839287926\nMinimizing 120 Time: 0:00:01 ( 9.87 ms/it)\n objective: 7200.461288500643\nMinimizing 131 Time: 0:00:01 ( 9.85 ms/it)\n objective: 7194.188203194858\nMinimizing 142 Time: 0:00:01 ( 9.84 ms/it)\n objective: 7188.62093968041\nMinimizing 153 Time: 0:00:01 ( 9.83 ms/it)\n objective: 7188.552220867151\nMinimizing 164 Time: 0:00:01 ( 9.84 ms/it)\n objective: 7188.538545300813\nMinimizing 174 Time: 0:00:01 ( 9.85 ms/it)\n objective: 7188.5159700931645\nMinimizing 184 Time: 0:00:01 ( 9.94 ms/it)\n objective: 7188.504641846532\nMinimizing 194 Time: 0:00:01 ( 9.99 ms/it)\n objective: 7188.502722500439\nMinimizing 203 Time: 0:00:02 (10.05 ms/it)\n objective: 7188.500891905364\nMinimizing 213 Time: 0:00:02 (10.08 ms/it)\n objective: 7188.500141354659\nMinimizing 223 Time: 0:00:02 (10.11 ms/it)\n objective: 7188.491875002939\nMinimizing 233 Time: 0:00:02 (10.13 ms/it)\n objective: 7188.49129998534\nMinimizing 243 Time: 0:00:02 (10.15 ms/it)\n objective: 7188.489739323119\nMinimizing 253 Time: 0:00:02 (10.18 ms/it)\n objective: 7188.489571414967\nMinimizing 263 Time: 0:00:02 (10.19 ms/it)\n objective: 7188.489439972358\nMinimizing 273 Time: 0:00:02 (10.21 ms/it)\n objective: 7188.489442655698\nMinimizing 282 Time: 0:00:02 (10.24 ms/it)\n objective: 7188.489422058375\nMinimizing 291 Time: 0:00:02 (10.28 ms/it)\n objective: 7188.489416642211\nMinimizing 291 Time: 0:00:02 (10.28 ms/it)\n\n\n(item = \nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . .\n P: rel 0.0 1.0 . . .\n Q: clr 0.0 0.0 1.0 . .\n lQ: clr 0.0 0.0 0.0 1.0 .\n lT: WD 0.0 0.0 0.0 0.0 1.0\n\nNormalized cumulative variances:\n[0.2, 0.4, 0.6, 0.8, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5\n (Intercept) 1.0 0.0 0.0 0.0 0.0\n P: rel 0.0 1.0 0.0 0.0 0.0\n Q: clr 0.0 0.0 1.0 0.0 0.0\n lQ: clr 0.0 0.0 0.0 1.0 0.0\n lT: WD 0.0 0.0 0.0 0.0 1.0, subj = \nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . . .\n F: HF 0.0 1.0 . . . .\n P: rel 0.0 0.0 1.0 . . .\n Q: clr 0.0 0.0 0.0 1.0 . .\n lQ: clr 0.0 0.0 0.0 0.0 1.0 .\n lT: WD 0.0 0.0 0.0 0.0 0.0 1.0\n\nNormalized cumulative variances:\n[0.1667, 0.3333, 0.5, 0.6667, 0.8333, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5 PC6\n (Intercept) 1.0 0.0 0.0 0.0 0.0 0.0\n F: HF 0.0 1.0 0.0 0.0 0.0 0.0\n P: rel 0.0 0.0 1.0 0.0 0.0 0.0\n Q: clr 0.0 0.0 0.0 0.0 1.0 0.0\n lQ: clr 0.0 0.0 0.0 1.0 0.0 0.0\n lT: WD 0.0 0.0 0.0 0.0 0.0 1.0)\n\n\n\nMixedModels.likelihoodratiotest(m_zcp, m_cpx)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + F + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lQ + lT | item)\n44\n7188\n\n\n\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + F + P + Q + lQ + lT | subj) + (1 + P + Q + lQ + lT | item)\n69\n7148\n41\n25\n0.0233\n\n\n\n\n\nLooks ok. It might be a good idea to prune the LMM by removing small VCs.\n\nm_zcp2 = let\n form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +\n zerocorr(1 +P+Q+lQ+lT | subj) +\n zerocorr(1 +P+Q +lT | item);\n fit(MixedModel, form, dat; contrasts)\nend\nVarCorr(m_zcp2)\n\nMinimizing 2 Time: 0:00:00 (56.46 ms/it)\n objective: 11209.21592702474\nMinimizing 20 Time: 0:00:00 (10.86 ms/it)\n objective: 9509.729918166682\nMinimizing 38 Time: 0:00:00 ( 8.38 ms/it)\n objective: 7311.387549009839\nMinimizing 56 Time: 0:00:00 ( 7.51 ms/it)\n objective: 7247.174405763663\nMinimizing 74 Time: 0:00:00 ( 7.06 ms/it)\n objective: 7240.910845804518\nMinimizing 93 Time: 0:00:00 ( 6.75 ms/it)\n objective: 7236.328263902658\nMinimizing 111 Time: 0:00:00 ( 6.58 ms/it)\n objective: 7227.556398242914\nMinimizing 129 Time: 0:00:00 ( 6.46 ms/it)\n objective: 7195.345776753049\nMinimizing 148 Time: 0:00:00 ( 6.33 ms/it)\n objective: 7190.682639021281\nMinimizing 167 Time: 0:00:01 ( 6.24 ms/it)\n objective: 7188.96412272415\nMinimizing 186 Time: 0:00:01 ( 6.17 ms/it)\n objective: 7188.6956407151665\nMinimizing 205 Time: 0:00:01 ( 6.12 ms/it)\n objective: 7188.575330252084\nMinimizing 224 Time: 0:00:01 ( 6.07 ms/it)\n objective: 7188.567615979038\nMinimizing 243 Time: 0:00:01 ( 6.02 ms/it)\n objective: 7188.564560144665\nMinimizing 262 Time: 0:00:01 ( 5.99 ms/it)\n objective: 7188.563992529358\nMinimizing 279 Time: 0:00:01 ( 6.00 ms/it)\n objective: 7188.563962432219\nMinimizing 298 Time: 0:00:01 ( 5.98 ms/it)\n objective: 7188.563961681864\nMinimizing 300 Time: 0:00:01 ( 5.97 ms/it)\n\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\nitem\n(Intercept)\n0.00320203\n0.05658650\n\n\n\n\n\n\n\nP: rel\n0.00007139\n0.00844898\n.\n\n\n\n\n\n\nQ: clr\n0.00014720\n0.01213252\n.\n.\n\n\n\n\n\nlT: WD\n0.00011708\n0.01082057\n.\n.\n.\n\n\n\nsubj\n(Intercept)\n0.03061099\n0.17495996\n\n\n\n\n\n\n\nP: rel\n0.00009921\n0.00996029\n.\n\n\n\n\n\n\nQ: clr\n0.00077305\n0.02780380\n.\n.\n\n\n\n\n\nlQ: clr\n0.00011889\n0.01090373\n.\n.\n.\n\n\n\n\nlT: WD\n0.00106147\n0.03258023\n.\n.\n.\n.\n\n\nResidual\n\n0.08591585\n0.29311406\n\n\n\n\n\n\n\n\n\n\nMixedModels.likelihoodratiotest(m_zcp2, m_zcp, m_cpx)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lT | item)\n42\n7189\n\n\n\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + F + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lQ + lT | item)\n44\n7188\n0\n2\n0.9634\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + F + P + Q + lQ + lT | subj) + (1 + P + Q + lQ + lT | item)\n69\n7148\n41\n25\n0.0233\n\n\n\n\n\nWe can perhaps remove some more.\n\nm_zcp3 = let\n form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +\n zerocorr(1 +Q +lT | subj) + (1 | item);\n fit(MixedModel, form, dat; contrasts)\nend\nVarCorr(m_zcp3)\n\nMinimizing 36 Time: 0:00:00 ( 2.79 ms/it)\n objective: 7204.514658399277\nMinimizing 97 Time: 0:00:00 ( 1.51 ms/it)\n\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\nitem\n(Intercept)\n0.0032049\n0.0566117\n\n\n\n\nsubj\n(Intercept)\n0.0306169\n0.1749768\n\n\n\n\n\nQ: clr\n0.0007629\n0.0276198\n.\n\n\n\n\nlT: WD\n0.0010645\n0.0326267\n.\n.\n\n\nResidual\n\n0.0864752\n0.2940667\n\n\n\n\n\n\n\n\nMixedModels.likelihoodratiotest(m_zcp3, m_zcp2, m_zcp, m_cpx)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + Q + lT | subj) + (1 | item)\n37\n7196\n\n\n\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lT | item)\n42\n7189\n8\n5\n0.1781\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + F + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lQ + lT | item)\n44\n7188\n0\n2\n0.9634\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + F + P + Q + lQ + lT | subj) + (1 + P + Q + lQ + lT | item)\n69\n7148\n41\n25\n0.0233\n\n\n\n\n\nAnd another iteration.\n\nm_zcp4 = let\n form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +\n zerocorr(1 +lT | subj) + (1 | item);\n fit(MixedModel, form, dat; contrasts)\nend\nVarCorr(m_zcp4)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\nitem\n(Intercept)\n0.0032067\n0.0566278\n\n\n\nsubj\n(Intercept)\n0.0306608\n0.1751022\n\n\n\n\nlT: WD\n0.0010896\n0.0330097\n.\n\n\nResidual\n\n0.0872330\n0.2953523\n\n\n\n\n\n\n\nMixedModels.likelihoodratiotest(m_zcp4, m_zcp3, m_zcp2, m_zcp, m_cpx)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + lT | subj) + (1 | item)\n36\n7259\n\n\n\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + Q + lT | subj) + (1 | item)\n37\n7196\n63\n1\n<1e-14\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lT | item)\n42\n7189\n8\n5\n0.1781\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + F + P + Q + lQ + lT | subj) + zerocorr(1 + P + Q + lQ + lT | item)\n44\n7188\n0\n2\n0.9634\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + F + P + Q + lQ + lT | subj) + (1 + P + Q + lQ + lT | item)\n69\n7148\n41\n25\n0.0233\n\n\n\n\n\nToo much removed. Stay with m_zcp3, but extend with CPs.\n\nm_prm = let\n form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +\n (1+ Q+lT | subj) + (1 | item);\n fit(MixedModel, form, dat; contrasts)\nend\nVarCorr(m_prm)\n\nMinimizing 132 Time: 0:00:00 ( 0.76 ms/it)\n objective: 7199.899053847559\nMinimizing 244 Time: 0:00:00 ( 0.77 ms/it)\n\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\nitem\n(Intercept)\n0.0032010\n0.0565773\n\n\n\n\nsubj\n(Intercept)\n0.0306216\n0.1749903\n\n\n\n\n\nQ: clr\n0.0007626\n0.0276159\n+0.42\n\n\n\n\nlT: WD\n0.0010621\n0.0325898\n+0.26\n+0.38\n\n\nResidual\n\n0.0864769\n0.2940695\n\n\n\n\n\n\n\n\n6.1.1 post-hoc LMM\n\nm_prm = let\n form = @formula (1000/rt) ~ 1+F*P*Q*lQ*lT +\n (1+ Q+lT | subj) + (1 | item);\n fit(MixedModel, form, dat; contrasts)\nend\nVarCorr(m_prm)\n\nMinimizing 133 Time: 0:00:00 ( 0.75 ms/it)\n objective: 7199.569489268595\nMinimizing 244 Time: 0:00:00 ( 0.76 ms/it)\n\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\nitem\n(Intercept)\n0.0032010\n0.0565773\n\n\n\n\nsubj\n(Intercept)\n0.0306216\n0.1749903\n\n\n\n\n\nQ: clr\n0.0007626\n0.0276159\n+0.42\n\n\n\n\nlT: WD\n0.0010621\n0.0325898\n+0.26\n+0.38\n\n\nResidual\n\n0.0864769\n0.2940695", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#vcs-and-cps-1", + "href": "mrk17.html#vcs-and-cps-1", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "6.2 VCs and CPs", + "text": "6.2 VCs and CPs\n\nMixedModels.likelihoodratiotest(m_zcp3, m_prm, m_cpx)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + zerocorr(1 + Q + lT | subj) + (1 | item)\n37\n7196\n\n\n\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + Q + lT | subj) + (1 | item)\n40\n7180\n16\n3\n0.0011\n\n\n:(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + F + P + Q + lQ + lT | subj) + (1 + P + Q + lQ + lT | item)\n69\n7148\n33\n29\n0.2916\n\n\n\n\n\nThe LRT favors the complex LMM, but not that χ² < 2*(χ²-dof); AIC and BIC suggest against selection.\n\ngof_summary = let\n nms = [:m_zcp3, :m_prm, :m_cpx]\n mods = eval.(nms)\n lrt = MixedModels.likelihoodratiotest(m_zcp3, m_prm, m_cpx)\n DataFrame(;\n name = nms,\n dof=dof.(mods),\n deviance=round.(deviance.(mods), digits=0),\n AIC=round.(aic.(mods),digits=0),\n AICc=round.(aicc.(mods),digits=0),\n BIC=round.(bic.(mods),digits=0),\n χ²=vcat(:., round.(lrt.tests.deviancediff, digits=0)),\n χ²_dof=vcat(:., round.(lrt.tests.dofdiff, digits=0)),\n pvalue=vcat(:., round.(lrt.tests.pvalues, digits=3))\n )\nend\n\n3×9 DataFrame\n\n\n\nRow\nname\ndof\ndeviance\nAIC\nAICc\nBIC\nχ²\nχ²_dof\npvalue\n\n\n\nSymbol\nInt64\nFloat64\nFloat64\nFloat64\nFloat64\nAny\nAny\nAny\n\n\n\n\n1\nm_zcp3\n37\n7196.0\n7270.0\n7270.0\n7555.0\n.\n.\n.\n\n\n2\nm_prm\n40\n7180.0\n7260.0\n7260.0\n7568.0\n16.0\n3.0\n0.001\n\n\n3\nm_cpx\n69\n7148.0\n7286.0\n7286.0\n7817.0\n33.0\n29.0\n0.292", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#crossed-fixed-effects", + "href": "mrk17.html#crossed-fixed-effects", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "7.1 Crossed fixed effects", + "text": "7.1 Crossed fixed effects\n\nm_mrk17_crossed =let\n form = @formula (1000/rt) ~ 1 + F*P*Q*lQ*lT +\n (1+Q | subj) + zerocorr(0+lT | subj) + zerocorr(1 + P | item) ;\n fit(MixedModel, form, dat; contrasts)\nend\n\nVarCorr(m_prm)\n\nMinimizing 67 Time: 0:00:00 ( 1.49 ms/it)\n objective: 7232.27363287469\nMinimizing 135 Time: 0:00:00 ( 1.50 ms/it)\n objective: 7186.823792113832\nMinimizing 167 Time: 0:00:00 ( 1.50 ms/it)\n\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\nitem\n(Intercept)\n0.0032010\n0.0565773\n\n\n\n\nsubj\n(Intercept)\n0.0306216\n0.1749903\n\n\n\n\n\nQ: clr\n0.0007626\n0.0276159\n+0.42\n\n\n\n\nlT: WD\n0.0010621\n0.0325898\n+0.26\n+0.38\n\n\nResidual\n\n0.0864769\n0.2940695\n\n\n\n\n\n\n\n\nshow(m_mrk17_crossed)\n\nLinear mixed model fit by maximum likelihood\n :(1000 / rt) ~ 1 + F + P + Q + lQ + lT + F & P + F & Q + P & Q + F & lQ + P & lQ + Q & lQ + F & lT + P & lT + Q & lT + lQ & lT + F & P & Q + F & P & lQ + F & Q & lQ + P & Q & lQ + F & P & lT + F & Q & lT + P & Q & lT + F & lQ & lT + P & lQ & lT + Q & lQ & lT + F & P & Q & lQ + F & P & Q & lT + F & P & lQ & lT + F & Q & lQ & lT + P & Q & lQ & lT + F & P & Q & lQ & lT + (1 + Q | subj) + zerocorr(0 + lT | subj) + zerocorr(1 + P | item)\n logLik -2 logLik AIC AICc BIC \n -3593.4086 7186.8171 7264.8171 7265.0077 7565.3349\n\nVariance components:\n Column Variance Std.Dev. Corr.\nitem (Intercept) 0.00320570 0.05661895\n P: rel 0.00006692 0.00818053 . \nsubj (Intercept) 0.03061792 0.17497978\n Q: clr 0.00076255 0.02761426 +0.42\n lT: WD 0.00106214 0.03259043 . . \nResidual 0.08640796 0.29395231\n Number of obs: 16409; levels of grouping factors: 240, 73\n\n Fixed-effects parameters:\n─────────────────────────────────────────────────────────────────────────────────────\n Coef. Std. Error z Pr(>|z|)\n─────────────────────────────────────────────────────────────────────────────────────\n(Intercept) 1.63743 0.0209305 78.23 <1e-99\nF: HF 0.019366 0.00431956 4.48 <1e-05\nP: rel 0.0188081 0.00236112 7.97 <1e-14\nQ: clr 0.042901 0.00396839 10.81 <1e-26\nlQ: clr 0.00174652 0.00231531 0.75 0.4506\nlT: WD 0.00836024 0.00446189 1.87 0.0610\nF: HF & P: rel -0.00711249 0.00236171 -3.01 0.0026\nF: HF & Q: clr -0.00141259 0.00230067 -0.61 0.5392\nP: rel & Q: clr 0.00134141 0.00230239 0.58 0.5602\nF: HF & lQ: clr -0.00105227 0.00232139 -0.45 0.6503\nP: rel & lQ: clr -0.00240742 0.00232055 -1.04 0.2995\nQ: clr & lQ: clr 0.00759804 0.00231922 3.28 0.0011\nF: HF & lT: WD 0.000536559 0.00231365 0.23 0.8166\nP: rel & lT: WD 2.93106e-5 0.00231751 0.01 0.9899\nQ: clr & lT: WD 0.0018553 0.00231773 0.80 0.4234\nlQ: clr & lT: WD -0.00524633 0.00231607 -2.27 0.0235\nF: HF & P: rel & Q: clr -0.00036224 0.00230272 -0.16 0.8750\nF: HF & P: rel & lQ: clr -0.00117753 0.00232025 -0.51 0.6118\nF: HF & Q: clr & lQ: clr 0.00273697 0.00232283 1.18 0.2387\nP: rel & Q: clr & lQ: clr -0.0039323 0.0023222 -1.69 0.0904\nF: HF & P: rel & lT: WD 0.0019864 0.00231879 0.86 0.3916\nF: HF & Q: clr & lT: WD -0.00112586 0.00231657 -0.49 0.6270\nP: rel & Q: clr & lT: WD 0.000261834 0.00231924 0.11 0.9101\nF: HF & lQ: clr & lT: WD 0.00150677 0.00232198 0.65 0.5164\nP: rel & lQ: clr & lT: WD 8.67915e-5 0.00232177 0.04 0.9702\nQ: clr & lQ: clr & lT: WD 0.00916752 0.00231827 3.95 <1e-04\nF: HF & P: rel & Q: clr & lQ: clr -0.00234443 0.00231978 -1.01 0.3122\nF: HF & P: rel & Q: clr & lT: WD -0.00148723 0.00232057 -0.64 0.5216\nF: HF & P: rel & lQ: clr & lT: WD 0.00308764 0.00232145 1.33 0.1835\nF: HF & Q: clr & lQ: clr & lT: WD 0.00393597 0.00232128 1.70 0.0900\nP: rel & Q: clr & lQ: clr & lT: WD 0.00202623 0.00232125 0.87 0.3827\nF: HF & P: rel & Q: clr & lQ: clr & lT: WD 0.00144888 0.00231838 0.62 0.5320\n─────────────────────────────────────────────────────────────────────────────────────\n\n\nFinally, a look at the fixed effects. The four-factor interaction reported in Masson & Kliegl (2013) was not replicated.", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#nested-fixed-effects", + "href": "mrk17.html#nested-fixed-effects", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "7.2 Nested fixed effects", + "text": "7.2 Nested fixed effects\n\nm_mrk17_nested =let\n form = @formula (1000/rt) ~ 1 + Q/(F/P) +\n (1+Q | subj) + zerocorr(0+lT | subj) + zerocorr(1 + P | item) ;\n fit(MixedModel, form, dat; contrasts)\nend\n\nMinimizing 74 Time: 0:00:00 ( 1.35 ms/it)\n objective: 7270.272343414747\nMinimizing 147 Time: 0:00:00 ( 1.37 ms/it)\n objective: 7237.687119961029\nMinimizing 172 Time: 0:00:00 ( 1.37 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n1.6374\n0.0209\n78.20\n<1e-99\n0.0565\n0.1751\n\n\nQ: clr\n0.0428\n0.0040\n10.74\n<1e-26\n\n0.0278\n\n\nQ: deg & F: HF\n0.0209\n0.0049\n4.26\n<1e-04\n\n\n\n\nQ: clr & F: HF\n0.0179\n0.0049\n3.65\n0.0003\n\n\n\n\nQ: deg & F: LF & P: rel\n0.0244\n0.0047\n5.18\n<1e-06\n\n\n\n\nQ: clr & F: LF & P: rel\n0.0278\n0.0047\n5.94\n<1e-08\n\n\n\n\nQ: deg & F: HF & P: rel\n0.0110\n0.0047\n2.35\n0.0186\n\n\n\n\nQ: clr & F: HF & P: rel\n0.0126\n0.0046\n2.71\n0.0067\n\n\n\n\nP: rel\n\n\n\n\n0.0086\n\n\n\nlT: WD\n\n\n\n\n\n0.0337\n\n\nResidual\n0.2944", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#nesting-within-products-of-factors", + "href": "mrk17.html#nesting-within-products-of-factors", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "8.1 Nesting within products of factors", + "text": "8.1 Nesting within products of factors\nInclude parenthesis\n\nm_mrk17_nested =let\n form = @formula (1000/rt) ~ 1 + Q/(F/P) +\n (1+Q | subj) + zerocorr(0+lT | subj) + zerocorr(1 + P | item) ;\n fit(MixedModel, form, dat; contrasts)\nend\n\nMinimizing 73 Time: 0:00:00 ( 1.38 ms/it)\n objective: 7271.786376361126\nMinimizing 147 Time: 0:00:00 ( 1.38 ms/it)\n objective: 7237.687119961029\nMinimizing 172 Time: 0:00:00 ( 1.37 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n1.6374\n0.0209\n78.20\n<1e-99\n0.0565\n0.1751\n\n\nQ: clr\n0.0428\n0.0040\n10.74\n<1e-26\n\n0.0278\n\n\nQ: deg & F: HF\n0.0209\n0.0049\n4.26\n<1e-04\n\n\n\n\nQ: clr & F: HF\n0.0179\n0.0049\n3.65\n0.0003\n\n\n\n\nQ: deg & F: LF & P: rel\n0.0244\n0.0047\n5.18\n<1e-06\n\n\n\n\nQ: clr & F: LF & P: rel\n0.0278\n0.0047\n5.94\n<1e-08\n\n\n\n\nQ: deg & F: HF & P: rel\n0.0110\n0.0047\n2.35\n0.0186\n\n\n\n\nQ: clr & F: HF & P: rel\n0.0126\n0.0046\n2.71\n0.0067\n\n\n\n\nP: rel\n\n\n\n\n0.0086\n\n\n\nlT: WD\n\n\n\n\n\n0.0337\n\n\nResidual\n0.2944", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#selection-in-fixed-effects", + "href": "mrk17.html#selection-in-fixed-effects", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "8.2 Selection in fixed effects", + "text": "8.2 Selection in fixed effects\n\nusing RegressionFormulae\n# m_prm_5 is equivalent to m_prm\nm_prm_5 = let\n form = @formula (1000/rt) ~ 1+(F+P+Q+lQ+lT)^5 + (1+Q+lT | subj) + (1 | item);\n fit(MixedModel, form, dat; contrasts)\nend;\n\nm_prm_4 = let\n form = @formula (1000/rt) ~ 1+(F+P+Q+lQ+lT)^4 + (1+Q+lT | subj) + (1 | item);\n fit(MixedModel, form, dat; contrasts)\nend;\n\nm_prm_3 = let\n form = @formula (1000/rt) ~ 1+(F+P+Q+lQ+lT)^3 + (1+Q+lT | subj) + (1 | item);\n fit(MixedModel, form, dat; contrasts)\nend;\n\nm_prm_2 = let\n form = @formula (1000/rt) ~ 1+(F+P+Q+lQ+lT)^2 + (1+Q+lT | subj) + (1 | item);\n fit(MixedModel, form, dat; contrasts)\nend;\n\nm_prm_1 = let\n form = @formula (1000/rt) ~ 1+ F+P+Q+lQ+lT + (1+Q+lT | subj) + (1 | item);\n fit(MixedModel, form, dat; contrasts)\nend;\n\n# Compare the fits\ngof_summary = let\n nms = [:m_prm_1, :m_prm_2, :m_prm_3, :m_prm_4, :m_prm_5]\n mods = eval.(nms)\n lrt = MixedModels.likelihoodratiotest(m_prm_1, m_prm_2, m_prm_3, m_prm_4, m_prm_5)\n DataFrame(;\n name = nms,\n dof=dof.(mods),\n deviance=deviance.(mods),\n AIC=aic.(mods),\n AICc=aicc.(mods),\n BIC=bic.(mods),\n χ²=vcat(:.,lrt.tests.deviancediff),\n χ²_dof=vcat(:.,lrt.tests.dofdiff),\n pvalue=vcat(:., round.(lrt.tests.pvalues, digits=3))\n )\nend\n\nMinimizing 133 Time: 0:00:00 ( 0.76 ms/it)\n objective: 7199.569489268595\nMinimizing 244 Time: 0:00:00 ( 0.76 ms/it)\nMinimizing 136 Time: 0:00:00 ( 0.74 ms/it)\n objective: 7378.040963546835\nMinimizing 273 Time: 0:00:00 ( 0.74 ms/it)\n objective: 7180.613748762729\nMinimizing 321 Time: 0:00:00 ( 0.74 ms/it)\nMinimizing 133 Time: 0:00:00 ( 0.76 ms/it)\n objective: 7210.014080210343\nMinimizing 257 Time: 0:00:00 ( 0.76 ms/it)\nMinimizing 147 Time: 0:00:00 ( 0.68 ms/it)\n objective: 7209.592926189283\nMinimizing 224 Time: 0:00:00 ( 0.68 ms/it)\nMinimizing 155 Time: 0:00:00 ( 0.65 ms/it)\n objective: 7237.619383975656\nMinimizing 241 Time: 0:00:00 ( 0.65 ms/it)\n\n\n5×9 DataFrame\n\n\n\nRow\nname\ndof\ndeviance\nAIC\nAICc\nBIC\nχ²\nχ²_dof\npvalue\n\n\n\nSymbol\nInt64\nFloat64\nFloat64\nFloat64\nFloat64\nAny\nAny\nAny\n\n\n\n\n1\nm_prm_1\n14\n7237.39\n7265.39\n7265.41\n7373.27\n.\n.\n.\n\n\n2\nm_prm_2\n24\n7209.2\n7257.2\n7257.27\n7442.13\n28.1868\n10\n0.002\n\n\n3\nm_prm_3\n34\n7187.57\n7255.57\n7255.71\n7517.56\n21.6317\n10\n0.017\n\n\n4\nm_prm_4\n39\n7180.61\n7258.61\n7258.8\n7559.13\n6.95863\n5\n0.224\n\n\n5\nm_prm_5\n40\n7180.21\n7260.21\n7260.41\n7568.44\n0.398591\n1\n0.528\n\n\n\n\n\n\nDepending on the level of precision of your hypothesis. You could stay with main effect (BIC), include 2-factor interactions (AIC; also called simple interactions) or include 3-factor interactions [χ² < 2*(χ²-dof); also called 2-way interactions].", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#posthoc-lmm", + "href": "mrk17.html#posthoc-lmm", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "8.3 Posthoc LMM", + "text": "8.3 Posthoc LMM\nWe are using only three factors for the illustruation.\n\nm_prm3 = let\n form = @formula (1000/rt) ~ 1 + lT*lQ*Q +\n (1+ Q+lT | subj) + (1 | item);\n fit(MixedModel, form, dat; contrasts)\nend\n\nMinimizing 155 Time: 0:00:00 ( 0.65 ms/it)\n objective: 7483.769438170831\nMinimizing 310 Time: 0:00:00 ( 0.65 ms/it)\n objective: 7291.145532339682\nMinimizing 343 Time: 0:00:00 ( 0.65 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n1.6376\n0.0210\n78.10\n<1e-99\n0.0596\n0.1750\n\n\nlT: WD\n0.0087\n0.0045\n1.93\n0.0530\n\n0.0328\n\n\nlQ: clr\n0.0016\n0.0023\n0.70\n0.4818\n\n\n\n\nQ: clr\n0.0428\n0.0040\n10.78\n<1e-26\n\n0.0276\n\n\nlT: WD & lQ: clr\n-0.0056\n0.0023\n-2.39\n0.0167\n\n\n\n\nlT: WD & Q: clr\n0.0020\n0.0023\n0.87\n0.3862\n\n\n\n\nlQ: clr & Q: clr\n0.0076\n0.0023\n3.26\n0.0011\n\n\n\n\nlT: WD & lQ: clr & Q: clr\n0.0092\n0.0023\n3.94\n<1e-04\n\n\n\n\nResidual\n0.2949\n\n\n\n\n\n\n\n\n\n\nThe lT & lQ & Q interactions is significant. Let’s follow it up with a post-hoc LMM, that is looking at the lQ & Q interaction in the two levels of whether the last word was a target or not.\n\nm_prm3_posthoc = let\n form = @formula (1000/rt) ~ 1 + lT/(lQ*Q) +\n (1+ Q+lT | subj) + (1 | item);\n fit(MixedModel, form, dat; contrasts)\nend\n\nMinimizing 154 Time: 0:00:00 ( 0.65 ms/it)\n objective: 7331.749225564165\nMinimizing 297 Time: 0:00:00 ( 0.65 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_item\nσ_subj\n\n\n(Intercept)\n1.6376\n0.0210\n78.10\n<1e-99\n0.0596\n0.1750\n\n\nlT: WD\n0.0087\n0.0045\n1.93\n0.0530\n\n0.0328\n\n\nlT: NW & lQ: clr\n0.0072\n0.0033\n2.19\n0.0285\n\n\n\n\nlT: WD & lQ: clr\n-0.0039\n0.0033\n-1.19\n0.2326\n\n\n\n\nlT: NW & Q: clr\n0.0408\n0.0046\n8.87\n<1e-18\n\n\n\n\nlT: WD & Q: clr\n0.0448\n0.0046\n9.73\n<1e-21\n\n\n\n\nlT: NW & lQ: clr & Q: clr\n-0.0016\n0.0033\n-0.49\n0.6275\n\n\n\n\nlT: WD & lQ: clr & Q: clr\n0.0167\n0.0033\n5.08\n<1e-06\n\n\n\n\nQ: clr\n\n\n\n\n\n0.0276\n\n\nResidual\n0.2949\n\n\n\n\n\n\n\n\n\n\nThe source of the interaction are trials where the last trial was a word target; there is no evidence for the interaction when the last trial was a nonword target.\nThe original and post-hoc LMM have the same goodness of fit.\n\n[objective(m_prm3), objective(m_prm3_posthoc)]\n\n2-element Vector{Float64}:\n 7291.145437442645\n 7291.145437447412", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "mrk17.html#info", + "href": "mrk17.html#info", + "title": "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection", + "section": "8.4 Info", + "text": "8.4 Info\n\nversioninfo()\n\nJulia Version 1.10.4\nCommit 48d4fd48430 (2024-06-04 10:41 UTC)\nBuild Info:\n Official https://julialang.org/ release\nPlatform Info:\n OS: Linux (x86_64-linux-gnu)\n CPU: 16 × Intel(R) Xeon(R) E-2288G CPU @ 3.70GHz\n WORD_SIZE: 64\n LIBM: libopenlibm\n LLVM: libLLVM-15.0.7 (ORCJIT, skylake)\nThreads: 16 default, 0 interactive, 8 GC (on 16 virtual cores)\nEnvironment:\n JULIA_LOAD_PATH = @:@stdlib", + "crumbs": [ + "Worked examples", + "RePsychLing Masson, Rabe, & Kliegl, 2017) with Julia: Specification and selection" + ] + }, + { + "objectID": "kwdyz11.html", + "href": "kwdyz11.html", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "", + "text": "We take the kwdyz11 dataset (Kliegl et al., 2011) from an experiment looking at three effects of visual cueing under four different cue-target relations (CTRs). Two horizontal rectangles are displayed above and below a central fixation point or they displayed in vertical orientation to the left and right of the fixation point. Subjects react to the onset of a small visual target occurring at one of the four ends of the two rectangles. The target is cued validly on 70% of trials by a brief flash of the corner of the rectangle at which it appears; it is cued invalidly at the three other locations 10% of the trials each.\nWe specify three contrasts for the four-level factor CTR that are derived from spatial, object-based, and attractor-like features of attention. They map onto sequential differences between appropriately ordered factor levels. At the level of fixed effects, there is the noteworthy result, that the attraction effect was estimated at 2 ms, that is clearly not significant. Nevertheless, there was a highly reliable variance component (VC) estimated for this effect. Moreover, the reliable individual differences in the attraction effect were negatively correlated with those in the spatial effect.\nUnfortunately, a few years after the publication, we determined that the reported LMM is actually singular and that the singularity is linked to a theoretically critical correlation parameter (CP) between the spatial effect and the attraction effect. Fortunately, there is also a larger dataset kkl15 from a replication and extension of this study (Kliegl et al., 2015), analyzed with kkl15.jl notebook. The critical CP (along with other fixed effects and CPs) was replicated in this study.\nA more comprehensive analysis was reported in the parsimonious mixed-model paper (Bates et al., 2015). Data and R scripts are also available in R-package RePsychLing. In this and the complementary kkl15.jl scripts, we provide some corresponding analyses with MixedModels.jl.", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#residual-over-fitted-plot", + "href": "kwdyz11.html#residual-over-fitted-plot", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "5.1 Residual-over-fitted plot", + "text": "5.1 Residual-over-fitted plot\nThe slant in residuals show a lower and upper boundary of reaction times, that is we have have too few short and too few long residuals. Not ideal, but at least width of the residual band looks similar across the fitted values, that is there is no evidence for heteroskedasticity.\n\n\nCode\nCairoMakie.activate!(; type=\"png\")\nset_aog_theme!()\ndraw(\n data((; f=fitted(m1), r=residuals(m1))) *\n mapping(:f => \"Fitted values\", :r => \"Residual from model m1\") *\n visual(Scatter);\n)\n\n\n\n\n\n\n\nFigure 3: Residuals versus the fitted values for model m1 of the log response time.\n\n\n\n\nWith many observations the scatterplot is not that informative. Contour plots or heatmaps may be an alternative.\n\n\nCode\nCairoMakie.activate!(; type=\"png\")\ndraw(\n data((; f=fitted(m1), r=residuals(m1))) *\n mapping(\n :f => \"Fitted log response time\", :r => \"Residual from model m1\"\n ) *\n density();\n)\n\n\n\n\n\n\n\nFigure 4: Heatmap of residuals versus fitted values for model m1", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#q-q-plot", + "href": "kwdyz11.html#q-q-plot", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "5.2 Q-Q plot", + "text": "5.2 Q-Q plot\nThe plot of quantiles of model residuals over corresponding quantiles of the normal distribution should yield a straight line along the main diagonal.\n\nqqnorm(residuals(m1); qqline=:none)", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#observed-and-theoretical-normal-distribution", + "href": "kwdyz11.html#observed-and-theoretical-normal-distribution", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "5.3 Observed and theoretical normal distribution", + "text": "5.3 Observed and theoretical normal distribution\nThe violation of expectation is again due to the fact that the distribution of residuals is much narrower than expected from a normal distribution, as shown in Figure 5. Overall, it does not look too bad.\n\n\nCode\nlet\n n = nrow(dat)\n dat_rz = DataFrame(;\n value=vcat(residuals(m1) ./ std(residuals(m1)), randn(n)),\n curve=vcat(fill.(\"residual\", n), fill.(\"normal\", n)),\n )\n draw(\n data(dat_rz) *\n mapping(:value => \"Standardized residuals\"; color=:curve) *\n density(; bandwidth=0.1);\n )\nend\n\n\n\n\n\n\n\nFigure 5: Kernel density plot of the standardized residuals from model m1 compared to a Gaussian", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#overlay", + "href": "kwdyz11.html#overlay", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "6.1 Overlay", + "text": "6.1 Overlay\nThe first plot overlays shrinkage-corrected conditional modes of the random effects with within-subject-based and pooled GMs and experimental effects.\nTo be done", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#caterpillar-plot", + "href": "kwdyz11.html#caterpillar-plot", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "6.2 Caterpillar plot", + "text": "6.2 Caterpillar plot\nThe caterpillar plot, Figure 6, also reveals the high correlation between spatial sod and attraction dod effects.\n\n\nCode\ncaterpillar!(\n Figure(; resolution=(800, 1000)), ranefinfo(m1, :Subj); orderby=2\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\nFigure 6: Prediction intervals on the random effects for Subj in model m1", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#shrinkage-plot", + "href": "kwdyz11.html#shrinkage-plot", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "6.3 Shrinkage plot", + "text": "6.3 Shrinkage plot\nFigure 7 provides more evidence for a problem with the visualization of the spatial sod and attraction dod CP. The corresponding panel illustrates an implosion of conditional modes.\n\n\nCode\nshrinkageplot!(Figure(; resolution=(1000, 1000)), m1)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\nFigure 7: Shrinkage plot of the conditional means of the random effects for model m1", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#generate-a-bootstrap-sample", + "href": "kwdyz11.html#generate-a-bootstrap-sample", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "7.1 Generate a bootstrap sample", + "text": "7.1 Generate a bootstrap sample\nWe generate 2500 samples for the 15 model parameters (4 fixed effect, 4 VCs, 6 CPs, and 1 residual).\n\n\nCode\nRandom.seed!(1234321)\nsamp = parametricbootstrap(2500, m1)\ntbl = samp.tbl\n\n\nTable with 26 columns and 2500 rows:\n obj β1 β2 β3 β4 σ ⋯\n ┌───────────────────────────────────────────────────────────────────\n 1 │ -12802.3 5.93392 0.0864217 0.0488879 -0.0121845 0.192002 ⋯\n 2 │ -12794.3 5.95776 0.0989845 0.0318746 -0.00666059 0.191977 ⋯\n 3 │ -12930.6 5.93669 0.0885609 0.0209795 -0.000656525 0.191496 ⋯\n 4 │ -12946.5 5.93165 0.0767126 0.0343558 -0.00499489 0.191438 ⋯\n 5 │ -12677.5 5.91938 0.098459 0.0272958 -0.0055567 0.19247 ⋯\n 6 │ -12869.7 5.96313 0.104458 0.0321628 -0.00719279 0.191751 ⋯\n 7 │ -12478.1 5.9328 0.0947867 0.0409709 -0.00461593 0.193076 ⋯\n 8 │ -12463.7 5.94439 0.0977167 0.0321098 -0.00625436 0.193115 ⋯\n 9 │ -12836.2 5.9728 0.0971358 0.0416612 6.09081e-5 0.191843 ⋯\n 10 │ -12932.3 5.94709 0.0742454 0.0505267 -0.0131469 0.191583 ⋯\n 11 │ -12736.2 5.93625 0.0728171 0.0479423 -0.0183166 0.192193 ⋯\n 12 │ -13048.0 5.89421 0.0680593 0.0481657 -0.0214161 0.191101 ⋯\n 13 │ -13313.8 5.92608 0.0892317 0.0473894 -0.0141683 0.190142 ⋯\n 14 │ -12575.0 5.89235 0.0846161 0.0341537 -0.00613088 0.192728 ⋯\n 15 │ -12833.1 5.94217 0.0774875 0.0422339 -0.00279008 0.191772 ⋯\n 16 │ -12504.5 5.94385 0.0898021 0.0429785 -0.00705666 0.192833 ⋯\n 17 │ -12875.5 5.96624 0.0976104 0.0370803 -0.00071879 0.191577 ⋯\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#shortest-coverage-interval", + "href": "kwdyz11.html#shortest-coverage-interval", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "7.2 Shortest coverage interval", + "text": "7.2 Shortest coverage interval\nThe upper limit of the interval for the critical CP CTR: sod, CTR: dod is hitting the upper wall of a perfect correlation. This is evidence of singularity. The other intervals do not exhibit such pathologies; they appear to be ok.\n\n\nCode\nconfint(samp)\n\n\nDictTable with 2 columns and 15 rows:\n par lower upper\n ────┬───────────────────────\n β1 │ 5.89917 5.97256\n β2 │ 0.0719299 0.104589\n β3 │ 0.0251177 0.0491665\n β4 │ -0.0207178 0.00268306\n ρ1 │ 0.243773 0.713001\n ρ2 │ -0.967759 0.195359\n ρ3 │ -0.643591 0.552114\n ρ4 │ -0.132618 0.739806\n ρ5 │ 0.573728 0.999995\n ρ6 │ -0.89138 0.436214\n σ │ 0.190559 0.193644\n σ1 │ 0.116566 0.168565\n σ2 │ 0.0445663 0.0698869\n σ3 │ 0.00946878 0.0409758\n σ4 │ 0.0133055 0.0376021", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#comparative-density-plots-of-bootstrapped-parameter-estimates", + "href": "kwdyz11.html#comparative-density-plots-of-bootstrapped-parameter-estimates", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "7.3 Comparative density plots of bootstrapped parameter estimates", + "text": "7.3 Comparative density plots of bootstrapped parameter estimates", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#residual", + "href": "kwdyz11.html#residual", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "7.4 Residual", + "text": "7.4 Residual\n\n\nCode\ndraw(\n data(tbl) *\n mapping(:σ => \"Residual standard deviation\") *\n density();\n)\n\n\n\n\n\n\n\nFigure 8", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#fixed-effects-wo-gm", + "href": "kwdyz11.html#fixed-effects-wo-gm", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "7.5 Fixed effects (w/o GM)", + "text": "7.5 Fixed effects (w/o GM)\nThe shortest coverage interval for the GM ranges from 376 to 404 ms. To keep the plot range small we do not include its density here.\n\n\nCode\nlabels = [\n \"CTR: sod\" => \"spatial effect\",\n \"CTR: dos\" => \"object effect\",\n \"CTR: dod\" => \"attraction effect\",\n \"(Intercept)\" => \"grand mean\",\n]\ndraw(\n data(tbl) *\n mapping(\n [:β2, :β3, :β4] .=> \"Experimental effect size [ms]\";\n color=dims(1) => renamer([\"spatial\", \"object\", \"attraction\"] .* \" effect\") =>\n \"Experimental effects\",\n ) *\n density();\n)\n\n\n\n\n\n\n\nFigure 9: Comparative density plots of the fixed-effects parameters for model m1\n\n\n\n\nThe densitiies correspond nicely with the shortest coverage intervals.", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#variance-components-vcs", + "href": "kwdyz11.html#variance-components-vcs", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "7.6 Variance components (VCs)", + "text": "7.6 Variance components (VCs)\n\n\nCode\ndraw(\n data(tbl) *\n mapping(\n [:σ1, :σ2, :σ3, :σ4] .=> \"Standard deviations [ms]\";\n color=dims(1) =>\n renamer(append!([\"Grand mean\"],[\"spatial\", \"object\", \"attraction\"] .* \" effect\")) =>\n \"Variance components\",\n ) *\n density();\n)\n\n\n\n\n\n\n\nFigure 10: Comparative density plots of the variance components for model m1\n\n\n\n\nThe VC are all very nicely defined.", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "kwdyz11.html#correlation-parameters-cps", + "href": "kwdyz11.html#correlation-parameters-cps", + "title": "RePsychLing Kliegl et al. (2011)", + "section": "7.7 Correlation parameters (CPs)", + "text": "7.7 Correlation parameters (CPs)\n\n\nCode\nlet\n labels = [\n \"(Intercept), CTR: sod\" => \"GM, spatial\",\n \"(Intercept), CTR: dos\" => \"GM, object\",\n \"CTR: sod, CTR: dos\" => \"spatial, object\",\n \"(Intercept), CTR: dod\" => \"GM, attraction\",\n \"CTR: sod, CTR: dod\" => \"spatial, attraction\",\n \"CTR: dos, CTR: dod\" => \"object, attraction\",\n ]\n draw(\n data(tbl) *\n mapping(\n [:ρ1, :ρ2, :ρ3, :ρ4, :ρ5, :ρ6] .=> \"Correlation\";\n color=dims(1) => renamer(last.(labels)) => \"Correlation parameters\",\n ) *\n density();\n )\nend\n\n\n\n\n\n\n\nFigure 11: Comparative density plots of the correlation parameters for model m1\n\n\n\n\nTwo of the CPs stand out positively. First, the correlation between GM and the spatial effect is well defined. Second, as discussed throughout this script, the CP between spatial and attraction effect is close to the 1.0 border and clearly not well defined. Therefore, this CP will be replicated with a larger sample in script kkl15.jl (Kliegl et al., 2015).", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl et al. (2011)" + ] + }, + { + "objectID": "contrasts_fggk21.html", + "href": "contrasts_fggk21.html", + "title": "Mixed Models Tutorial: Contrast Coding", + "section": "", + "text": "This script uses a subset of data reported in Fühner et al. (2021).\nTo circumvent delays associated with model fitting we work with models that are less complex than those in the reference publication. All the data to reproduce the models in the publication are used here, too; the script requires only a few changes to specify the more complex models in the paper.\nAll children were between 6.0 and 6.99 years at legal keydate (30 September) of school enrollment, that is in their ninth year of life in the third grade. To avoid delays associated with model fitting we work with a reduced data set and less complex models than those in the reference publication. The script requires only a few changes to specify the more complex models in the paper.\nThe script is structured in three main sections:", + "crumbs": [ + "Contrast coding", + "Mixed Models Tutorial: Contrast Coding" + ] + }, + { + "objectID": "contrasts_fggk21.html#packages-and-functions", + "href": "contrasts_fggk21.html#packages-and-functions", + "title": "Mixed Models Tutorial: Contrast Coding", + "section": "1.1 Packages and functions", + "text": "1.1 Packages and functions\n\n\nCode\nusing AlgebraOfGraphics\nusing CairoMakie\nusing Chain\nusing CategoricalArrays\nusing DataFrames\nusing DataFrameMacros\nusing MixedModels\nusing ProgressMeter\nusing SMLP2024: dataset\nusing Statistics\nusing StatsBase\n\nProgressMeter.ijulia_behavior(:clear);", + "crumbs": [ + "Contrast coding", + "Mixed Models Tutorial: Contrast Coding" + ] + }, + { + "objectID": "contrasts_fggk21.html#readme-for-datasetfggk21", + "href": "contrasts_fggk21.html#readme-for-datasetfggk21", + "title": "Mixed Models Tutorial: Contrast Coding", + "section": "1.2 Readme for dataset(\"fggk21\")", + "text": "1.2 Readme for dataset(\"fggk21\")\nNumber of scores: 525126\n\nCohort: 9 levels; 2011-2019\nSchool: 515 levels\nChild: 108295 levels; all children are between 8.0 and 8.99 years old\nSex: “Girls” (n=55,086), “Boys” (n= 53,209)\nage: testdate - middle of month of birthdate\nTest: 5 levels\n\nEndurance (Run): 6 minute endurance run [m]; to nearest 9m in 9x18m field\nCoordination (Star_r): star coordination run [m/s]; 9x9m field, 4 x diagonal = 50.912 m\nSpeed(S20_r): 20-meters sprint [m/s]\nMuscle power low (SLJ): standing long jump [cm]\nMuscle power up (BPT): 1-kg medicine ball push test [m]\n\nscore - see units", + "crumbs": [ + "Contrast coding", + "Mixed Models Tutorial: Contrast Coding" + ] + }, + { + "objectID": "contrasts_fggk21.html#preprocessing", + "href": "contrasts_fggk21.html#preprocessing", + "title": "Mixed Models Tutorial: Contrast Coding", + "section": "1.3 Preprocessing", + "text": "1.3 Preprocessing\n\n1.3.1 Read data\n\ntbl = dataset(:fggk21)\n\nArrow.Table with 525126 rows, 7 columns, and schema:\n :Cohort String\n :School String\n :Child String\n :Sex String\n :age Float64\n :Test String\n :score Float64\n\n\n\ndf = DataFrame(tbl)\ndescribe(df)\n\n7×7 DataFrame\n\n\n\nRow\nvariable\nmean\nmin\nmedian\nmax\nnmissing\neltype\n\n\n\nSymbol\nUnion…\nAny\nUnion…\nAny\nInt64\nDataType\n\n\n\n\n1\nCohort\n\n2011\n\n2019\n0\nString\n\n\n2\nSchool\n\nS100043\n\nS800200\n0\nString\n\n\n3\nChild\n\nC002352\n\nC117966\n0\nString\n\n\n4\nSex\n\nfemale\n\nmale\n0\nString\n\n\n5\nage\n8.56073\n7.99452\n8.55852\n9.10609\n0\nFloat64\n\n\n6\nTest\n\nBPT\n\nStar_r\n0\nString\n\n\n7\nscore\n226.141\n1.14152\n4.65116\n1530.0\n0\nFloat64\n\n\n\n\n\n\n\n\n1.3.2 Extract a stratified subsample\nWe extract a random sample of 500 children from the Sex (2) x Test (5) cells of the design. Cohort and School are random.\n\ndat = @chain df begin\n @transform(:Sex = :Sex == \"female\" ? \"Girls\" : \"Boys\")\n @groupby(:Test, :Sex)\n combine(x -> x[sample(1:nrow(x), 500), :])\nend\n\n5000×7 DataFrame4975 rows omitted\n\n\n\nRow\nTest\nSex\nCohort\nSchool\nChild\nage\nscore\n\n\n\nString\nString\nString\nString\nString\nFloat64\nFloat64\n\n\n\n\n1\nS20_r\nBoys\n2014\nS100833\nC073438\n8.70363\n4.87805\n\n\n2\nS20_r\nBoys\n2018\nS102982\nC109803\n9.02122\n4.7619\n\n\n3\nS20_r\nBoys\n2018\nS106173\nC101851\n8.93908\n4.87805\n\n\n4\nS20_r\nBoys\n2018\nS101023\nC109489\n9.02122\n4.34783\n\n\n5\nS20_r\nBoys\n2019\nS105016\nC085317\n8.80219\n4.87805\n\n\n6\nS20_r\nBoys\n2011\nS100195\nC087887\n8.83231\n5.71429\n\n\n7\nS20_r\nBoys\n2018\nS101138\nC090287\n8.85421\n4.54545\n\n\n8\nS20_r\nBoys\n2018\nS101369\nC116866\n9.10609\n4.87805\n\n\n9\nS20_r\nBoys\n2012\nS101424\nC050344\n8.50103\n4.7619\n\n\n10\nS20_r\nBoys\n2015\nS103597\nC073195\n8.69541\n4.65116\n\n\n11\nS20_r\nBoys\n2017\nS101000\nC038646\n8.39425\n5.12821\n\n\n12\nS20_r\nBoys\n2019\nS105910\nC057583\n8.55578\n4.34783\n\n\n13\nS20_r\nBoys\n2012\nS105983\nC089262\n8.83231\n5.12821\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n4989\nRun\nGirls\n2018\nS130606\nC073152\n8.69268\n963.0\n\n\n4990\nRun\nGirls\n2014\nS101710\nC054434\n8.54209\n1190.0\n\n\n4991\nRun\nGirls\n2016\nS104711\nC098099\n8.9117\n981.0\n\n\n4992\nRun\nGirls\n2015\nS100511\nC094527\n8.88159\n927.0\n\n\n4993\nRun\nGirls\n2014\nS101503\nC092320\n8.87064\n1099.0\n\n\n4994\nRun\nGirls\n2019\nS100511\nC075621\n8.71732\n828.0\n\n\n4995\nRun\nGirls\n2011\nS111442\nC061266\n8.58042\n966.0\n\n\n4996\nRun\nGirls\n2019\nS105041\nC038037\n8.38877\n1053.0\n\n\n4997\nRun\nGirls\n2012\nS100547\nC031385\n8.33402\n990.0\n\n\n4998\nRun\nGirls\n2013\nS104632\nC002731\n7.99452\n855.0\n\n\n4999\nRun\nGirls\n2016\nS103640\nC024204\n8.27105\n1000.0\n\n\n5000\nRun\nGirls\n2019\nS102131\nC095404\n8.88433\n891.0\n\n\n\n\n\n\n\n\n1.3.3 Transformations\n\ntransform!(dat, :age, :age => (x -> x .- 8.5) => :a1) # centered age (linear)\nselect!(groupby(dat, :Test), :, :score => zscore => :zScore) # z-score\n\n5000×9 DataFrame4975 rows omitted\n\n\n\nRow\nTest\nSex\nCohort\nSchool\nChild\nage\nscore\na1\nzScore\n\n\n\nString\nString\nString\nString\nString\nFloat64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\nS20_r\nBoys\n2014\nS100833\nC073438\n8.70363\n4.87805\n0.203628\n0.817771\n\n\n2\nS20_r\nBoys\n2018\nS102982\nC109803\n9.02122\n4.7619\n0.521218\n0.540353\n\n\n3\nS20_r\nBoys\n2018\nS106173\nC101851\n8.93908\n4.87805\n0.439083\n0.817771\n\n\n4\nS20_r\nBoys\n2018\nS101023\nC109489\n9.02122\n4.34783\n0.521218\n-0.448701\n\n\n5\nS20_r\nBoys\n2019\nS105016\nC085317\n8.80219\n4.87805\n0.30219\n0.817771\n\n\n6\nS20_r\nBoys\n2011\nS100195\nC087887\n8.83231\n5.71429\n0.332307\n2.81518\n\n\n7\nS20_r\nBoys\n2018\nS101138\nC090287\n8.85421\n4.54545\n0.354209\n0.0233473\n\n\n8\nS20_r\nBoys\n2018\nS101369\nC116866\n9.10609\n4.87805\n0.606092\n0.817771\n\n\n9\nS20_r\nBoys\n2012\nS101424\nC050344\n8.50103\n4.7619\n0.00102669\n0.540353\n\n\n10\nS20_r\nBoys\n2015\nS103597\nC073195\n8.69541\n4.65116\n0.195414\n0.275838\n\n\n11\nS20_r\nBoys\n2017\nS101000\nC038646\n8.39425\n5.12821\n-0.105749\n1.41529\n\n\n12\nS20_r\nBoys\n2019\nS105910\nC057583\n8.55578\n4.34783\n0.0557837\n-0.448701\n\n\n13\nS20_r\nBoys\n2012\nS105983\nC089262\n8.83231\n5.12821\n0.332307\n1.41529\n\n\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n⋮\n\n\n4989\nRun\nGirls\n2018\nS130606\nC073152\n8.69268\n963.0\n0.192676\n-0.333028\n\n\n4990\nRun\nGirls\n2014\nS101710\nC054434\n8.54209\n1190.0\n0.0420945\n1.20792\n\n\n4991\nRun\nGirls\n2016\nS104711\nC098099\n8.9117\n981.0\n0.411704\n-0.210838\n\n\n4992\nRun\nGirls\n2015\nS100511\nC094527\n8.88159\n927.0\n0.381588\n-0.577407\n\n\n4993\nRun\nGirls\n2014\nS101503\nC092320\n8.87064\n1099.0\n0.370637\n0.590183\n\n\n4994\nRun\nGirls\n2019\nS100511\nC075621\n8.71732\n828.0\n0.217317\n-1.24945\n\n\n4995\nRun\nGirls\n2011\nS111442\nC061266\n8.58042\n966.0\n0.0804244\n-0.312663\n\n\n4996\nRun\nGirls\n2019\nS105041\nC038037\n8.38877\n1053.0\n-0.111225\n0.27792\n\n\n4997\nRun\nGirls\n2012\nS100547\nC031385\n8.33402\n990.0\n-0.165982\n-0.149743\n\n\n4998\nRun\nGirls\n2013\nS104632\nC002731\n7.99452\n855.0\n-0.505476\n-1.06617\n\n\n4999\nRun\nGirls\n2016\nS103640\nC024204\n8.27105\n1000.0\n-0.228953\n-0.0818603\n\n\n5000\nRun\nGirls\n2019\nS102131\nC095404\n8.88433\n891.0\n0.384326\n-0.821786\n\n\n\n\n\n\n\ndat2 = combine(\n groupby(dat, [:Test, :Sex]),\n :score => mean,\n :score => std,\n :zScore => mean,\n :zScore => std,\n)\n\n10×6 DataFrame\n\n\n\nRow\nTest\nSex\nscore_mean\nscore_std\nzScore_mean\nzScore_std\n\n\n\nString\nString\nFloat64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\nS20_r\nBoys\n4.60142\n0.422512\n0.157034\n1.0092\n\n\n2\nBPT\nBoys\n3.9756\n0.712896\n0.379716\n0.979017\n\n\n3\nSLJ\nBoys\n128.414\n19.6343\n0.201323\n0.993676\n\n\n4\nStar_r\nBoys\n2.07246\n0.306598\n0.0975878\n1.06545\n\n\n5\nRun\nBoys\n1044.39\n150.508\n0.219487\n1.02169\n\n\n6\nS20_r\nGirls\n4.46994\n0.404632\n-0.157034\n0.96649\n\n\n7\nBPT\nGirls\n3.4226\n0.632542\n-0.379716\n0.868667\n\n\n8\nSLJ\nGirls\n120.458\n19.0898\n-0.201323\n0.96612\n\n\n9\nStar_r\nGirls\n2.01629\n0.26495\n-0.0975878\n0.920723\n\n\n10\nRun\nGirls\n979.726\n136.739\n-0.219487\n0.928228\n\n\n\n\n\n\n\n\n1.3.4 Figure of age x Sex x Test interactions\nThe main results of relevance here are shown in Figure 2 of Scientific Reports 11:17566.", + "crumbs": [ + "Contrast coding", + "Mixed Models Tutorial: Contrast Coding" + ] + }, + { + "objectID": "contrasts_fggk21.html#seqdiffcoding-contr1", + "href": "contrasts_fggk21.html#seqdiffcoding-contr1", + "title": "Mixed Models Tutorial: Contrast Coding", + "section": "2.1 SeqDiffCoding: contr1", + "text": "2.1 SeqDiffCoding: contr1\nSeqDiffCoding was used in the publication. This specification tests pairwise differences between the five neighboring levels of Test, that is:\n\nSDC1: 2-1\nSDC2: 3-2\nSDC3: 4-3\nSDC4: 5-4\n\nThe levels were sorted such that these contrasts map onto four a priori hypotheses; in other words, they are theoretically motivated pairwise comparisons. The motivation also encompasses theoretically motivated interactions with Sex. The order of levels can also be explicitly specified during contrast construction. This is very useful if levels are in a different order in the dataframe. We recommend the explicit specification to increase transparency of the code.\nThe statistical disadvantage of SeqDiffCoding is that the contrasts are not orthogonal, that is the contrasts are correlated. This is obvious from the fact that levels 2, 3, and 4 are all used in two contrasts. One consequence of this is that correlation parameters estimated between neighboring contrasts (e.g., 2-1 and 3-2) are difficult to interpret. Usually, they will be negative because assuming some practical limitation on the overall range (e.g., between levels 1 and 3), a small “2-1” effect “correlates” negatively with a larger “3-2” effect for mathematical reasons.\nObviously, the tradeoff between theoretical motivation and statistical purity is something that must be considered carefully when planning the analysis.\n\ncontr1 = merge(\n Dict(nm => Grouping() for nm in (:School, :Child, :Cohort)),\n Dict(\n :Sex => EffectsCoding(; levels=[\"Girls\", \"Boys\"]),\n :Test => SeqDiffCoding(;\n levels=[\"Run\", \"Star_r\", \"S20_r\", \"SLJ\", \"BPT\"]\n ),\n ),\n)\n\nDict{Symbol, StatsModels.AbstractContrasts} with 5 entries:\n :Child => Grouping()\n :School => Grouping()\n :Test => SeqDiffCoding([\"Run\", \"Star_r\", \"S20_r\", \"SLJ\", \"BPT\"])\n :Cohort => Grouping()\n :Sex => EffectsCoding(nothing, [\"Girls\", \"Boys\"])\n\n\n\nf_ovi_1 = @formula zScore ~ 1 + Test + (1 | Child);\n\n\nm_ovi_SeqDiff_1 = fit(MixedModel, f_ovi_1, dat; contrasts=contr1)\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n0.0001\n0.0142\n0.01\n0.9945\n0.7264\n\n\nTest: Star_r\n-0.0015\n0.0444\n-0.03\n0.9732\n\n\n\nTest: S20_r\n-0.0032\n0.0443\n-0.07\n0.9424\n\n\n\nTest: SLJ\n0.0032\n0.0444\n0.07\n0.9421\n\n\n\nTest: BPT\n0.0037\n0.0444\n0.08\n0.9334\n\n\n\nResidual\n0.6867\n\n\n\n\n\n\n\n\n\nIn this case, any differences between tests identified by the contrasts would be spurious because each test was standardized (i.e., M=0, \\(SD\\)=1). The differences could also be due to an imbalance in the number of boys and girls or in the number of missing observations for each test.\nThe primary interest in this study related to interactions of the test contrasts with and age and Sex. We start with age (linear) and its interaction with the four test contrasts.\n\nm_ovi_SeqDiff_2 = let\n form = @formula zScore ~ 1 + Test * a1 + (1 | Child)\n fit(MixedModel, form, dat; contrasts=contr1)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0197\n0.0144\n-1.36\n0.1724\n0.7196\n\n\nTest: Star_r\n-0.0181\n0.0450\n-0.40\n0.6876\n\n\n\nTest: S20_r\n-0.0079\n0.0449\n-0.17\n0.8612\n\n\n\nTest: SLJ\n0.0120\n0.0449\n0.27\n0.7899\n\n\n\nTest: BPT\n-0.0236\n0.0448\n-0.53\n0.5991\n\n\n\na1\n0.3498\n0.0494\n7.09\n<1e-11\n\n\n\nTest: Star_r & a1\n0.2903\n0.1543\n1.88\n0.0599\n\n\n\nTest: S20_r & a1\n0.0713\n0.1552\n0.46\n0.6461\n\n\n\nTest: SLJ & a1\n-0.0824\n0.1545\n-0.53\n0.5937\n\n\n\nTest: BPT & a1\n0.3568\n0.1502\n2.38\n0.0175\n\n\n\nResidual\n0.6839\n\n\n\n\n\n\n\n\n\nThe difference between older and younger childrend is larger for Star_r than for Run (0.2473). S20_r did not differ significantly from Star_r (-0.0377) and SLJ (-0.0113) The largest difference in developmental gain was between BPT and SLJ (0.3355).\nPlease note that standard errors of this LMM are anti-conservative because the LMM is missing a lot of information in the RES (e..g., contrast-related VCs snd CPs for Child, School, and Cohort.\nNext we add the main effect of Sex and its interaction with the four test contrasts.\n\nm_ovi_SeqDiff_3 = let\n form = @formula zScore ~ 1 + Test * (a1 + Sex) + (1 | Child)\n fit(MixedModel, form, dat; contrasts=contr1)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0175\n0.0140\n-1.25\n0.2123\n0.6907\n\n\nTest: Star_r\n-0.0168\n0.0438\n-0.38\n0.7023\n\n\n\nTest: S20_r\n-0.0095\n0.0438\n-0.22\n0.8274\n\n\n\nTest: SLJ\n0.0122\n0.0437\n0.28\n0.7808\n\n\n\nTest: BPT\n-0.0213\n0.0437\n-0.49\n0.6259\n\n\n\na1\n0.3223\n0.0481\n6.70\n<1e-10\n\n\n\nSex: Boys\n0.2082\n0.0138\n15.10\n<1e-50\n\n\n\nTest: Star_r & a1\n0.2837\n0.1505\n1.89\n0.0594\n\n\n\nTest: S20_r & a1\n0.0795\n0.1514\n0.52\n0.5996\n\n\n\nTest: SLJ & a1\n-0.0886\n0.1506\n-0.59\n0.5561\n\n\n\nTest: BPT & a1\n0.3107\n0.1464\n2.12\n0.0339\n\n\n\nTest: Star_r & Sex: Boys\n-0.1188\n0.0430\n-2.76\n0.0057\n\n\n\nTest: S20_r & Sex: Boys\n0.0549\n0.0429\n1.28\n0.2011\n\n\n\nTest: SLJ & Sex: Boys\n0.0481\n0.0430\n1.12\n0.2635\n\n\n\nTest: BPT & Sex: Boys\n0.1740\n0.0430\n4.05\n<1e-04\n\n\n\nResidual\n0.6758\n\n\n\n\n\n\n\n\n\nThe significant interactions with Sex reflect mostly differences related to muscle power, where the physiological constitution gives boys an advantage. The sex difference is smaller when coordination and cognition play a role – as in the Star_r test. (Caveat: SEs are estimated with an underspecified RES.)\nThe final step in this first series is to add the interactions between the three covariates. A significant interaction between any of the four Test contrasts and age (linear) x Sex was hypothesized to reflect a prepubertal signal (i.e., hormones start to rise in girls’ ninth year of life). However, this hypothesis is linked to a specific shape of the interaction: Girls would need to gain more than boys in tests of muscular power.\n\nf_ovi = @formula zScore ~ 1 + Test * a1 * Sex + (1 | Child)\nm_ovi_SeqDiff = fit(MixedModel, f_ovi, dat; contrasts=contr1)\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0181\n0.0140\n-1.29\n0.1970\n0.6929\n\n\nTest: Star_r\n-0.0177\n0.0439\n-0.40\n0.6865\n\n\n\nTest: S20_r\n-0.0086\n0.0438\n-0.20\n0.8445\n\n\n\nTest: SLJ\n0.0129\n0.0437\n0.30\n0.7677\n\n\n\nTest: BPT\n-0.0234\n0.0437\n-0.54\n0.5918\n\n\n\na1\n0.3216\n0.0481\n6.69\n<1e-10\n\n\n\nSex: Boys\n0.2051\n0.0140\n14.61\n<1e-47\n\n\n\nTest: Star_r & a1\n0.2878\n0.1506\n1.91\n0.0560\n\n\n\nTest: S20_r & a1\n0.0791\n0.1513\n0.52\n0.6013\n\n\n\nTest: SLJ & a1\n-0.0890\n0.1505\n-0.59\n0.5542\n\n\n\nTest: BPT & a1\n0.3111\n0.1464\n2.12\n0.0336\n\n\n\nTest: Star_r & Sex: Boys\n-0.1180\n0.0439\n-2.69\n0.0071\n\n\n\nTest: S20_r & Sex: Boys\n0.0565\n0.0438\n1.29\n0.1972\n\n\n\nTest: SLJ & Sex: Boys\n0.0522\n0.0437\n1.19\n0.2327\n\n\n\nTest: BPT & Sex: Boys\n0.1651\n0.0437\n3.78\n0.0002\n\n\n\na1 & Sex: Boys\n0.0510\n0.0481\n1.06\n0.2887\n\n\n\nTest: Star_r & a1 & Sex: Boys\n-0.0145\n0.1506\n-0.10\n0.9231\n\n\n\nTest: S20_r & a1 & Sex: Boys\n-0.0303\n0.1513\n-0.20\n0.8411\n\n\n\nTest: SLJ & a1 & Sex: Boys\n-0.0767\n0.1505\n-0.51\n0.6104\n\n\n\nTest: BPT & a1 & Sex: Boys\n0.1525\n0.1464\n1.04\n0.2974\n\n\n\nResidual\n0.6732\n\n\n\n\n\n\n\n\n\nThe results are very clear: Despite an abundance of statistical power there is no evidence for the differences between boys and girls in how much they gain in the ninth year of life in these five tests. The authors argue that, in this case, absence of evidence looks very much like evidence of absence of a hypothesized interaction.\nIn the next two sections we use different contrasts. Does this have a bearing on this result? We still ignore for now that we are looking at anti-conservative test statistics.", + "crumbs": [ + "Contrast coding", + "Mixed Models Tutorial: Contrast Coding" + ] + }, + { + "objectID": "contrasts_fggk21.html#helmertcoding-contr2", + "href": "contrasts_fggk21.html#helmertcoding-contr2", + "title": "Mixed Models Tutorial: Contrast Coding", + "section": "2.2 HelmertCoding: contr2", + "text": "2.2 HelmertCoding: contr2\nThe second set of contrasts uses HelmertCoding. Helmert coding codes each level as the difference from the average of the lower levels. With the default order of Test levels we get the following test statistics which we describe in reverse order of appearance in model output\n\nHeC4: 5 - mean(1,2,3,4)\nHeC3: 4 - mean(1,2,3)\nHeC2: 3 - mean(1,2)\nHeC1: 2 - 1\n\nIn the model output, HeC1 will be reported first and HeC4 last.\nThere is some justification for the HeC4 specification in a post-hoc manner because the fifth test (BPT) turned out to be different from the other four tests in that high performance is most likely not only related to physical fitness, but also to overweight/obesity, that is for a subset of children high scores on this test might be indicative of physical unfitness. A priori the SDC4 contrast 5-4 between BPT (5) and SLJ (4) was motivated because conceptually both are tests of the physical fitness component Muscular Power, BPT for upper limbs and SLJ for lower limbs, respectively.\nOne could argue that there is justification for HeC3 because Run (1), Star_r (2), and S20 (3) involve running but SLJ (4) does not. Sports scientists, however, recoil. For them it does not make much sense to average the different running tests, because they draw on completely different physiological resources; it is a variant of the old apples-and-oranges problem.\nThe justification for HeC3 is thatRun (1) and Star_r (2) draw more strongly on cardiosrespiratory Endurance than S20 (3) due to the longer duration of the runs compared to sprinting for 20 m which is a pure measure of the physical-fitness component Speed. Again, sports scientists are not very happy with this proposal.\nFinally, HeC1 contrasts the fitness components Endurance, indicated best by Run (1), and Coordination, indicated by Star_r (2). Endurance (i.e., running for 6 minutes) is considered to be the best indicator of health-related status among the five tests because it is a rather pure measure of cardiorespiratory fitness. The Star_r test requires execution of a pre-instructed sequence of forward, sideways, and backward runs. This coordination of body movements implies a demand on working memory (i.e., remembering the order of these subruns) and executive control processes, but performats also depends on endurance. HeC1 yields a measure of Coordination “corrected” for the contribution of Endurance.\nThe statistical advantage of HelmertCoding is that the resulting contrasts are orthogonal (uncorrelated). This allows for optimal partitioning of variance and statistical power. It is also more efficient to estimate “orthogonal” than “non-orthogonal” random-effect structures.\n\ncontr2 = Dict(\n :School => Grouping(),\n :Child => Grouping(),\n :Cohort => Grouping(),\n :Sex => EffectsCoding(; levels=[\"Girls\", \"Boys\"]),\n :Test => HelmertCoding(;\n levels=[\"Run\", \"Star_r\", \"S20_r\", \"SLJ\", \"BPT\"],\n ),\n);\n\n\nm_ovi_Helmert = fit(MixedModel, f_ovi, dat; contrasts=contr2)\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0181\n0.0140\n-1.29\n0.1970\n0.6929\n\n\nTest: Star_r\n-0.0088\n0.0219\n-0.40\n0.6865\n\n\n\nTest: S20_r\n-0.0058\n0.0126\n-0.46\n0.6454\n\n\n\nTest: SLJ\n0.0003\n0.0089\n0.04\n0.9713\n\n\n\nTest: BPT\n-0.0045\n0.0069\n-0.65\n0.5170\n\n\n\na1\n0.3216\n0.0481\n6.69\n<1e-10\n\n\n\nSex: Boys\n0.2051\n0.0140\n14.61\n<1e-47\n\n\n\nTest: Star_r & a1\n0.1439\n0.0753\n1.91\n0.0560\n\n\n\nTest: S20_r & a1\n0.0743\n0.0442\n1.68\n0.0929\n\n\n\nTest: SLJ & a1\n0.0149\n0.0302\n0.49\n0.6217\n\n\n\nTest: BPT & a1\n0.0712\n0.0233\n3.05\n0.0023\n\n\n\nTest: Star_r & Sex: Boys\n-0.0590\n0.0219\n-2.69\n0.0071\n\n\n\nTest: S20_r & Sex: Boys\n-0.0008\n0.0126\n-0.07\n0.9472\n\n\n\nTest: SLJ & Sex: Boys\n0.0126\n0.0089\n1.42\n0.1567\n\n\n\nTest: BPT & Sex: Boys\n0.0406\n0.0069\n5.85\n<1e-08\n\n\n\na1 & Sex: Boys\n0.0510\n0.0481\n1.06\n0.2887\n\n\n\nTest: Star_r & a1 & Sex: Boys\n-0.0073\n0.0753\n-0.10\n0.9231\n\n\n\nTest: S20_r & a1 & Sex: Boys\n-0.0125\n0.0442\n-0.28\n0.7769\n\n\n\nTest: SLJ & a1 & Sex: Boys\n-0.0254\n0.0302\n-0.84\n0.3995\n\n\n\nTest: BPT & a1 & Sex: Boys\n0.0152\n0.0233\n0.65\n0.5132\n\n\n\nResidual\n0.6732\n\n\n\n\n\n\n\n\n\nWe forego a detailed discussion of the effects, but note that again none of the interactions between age x Sex with the four test contrasts was significant.\nThe default labeling of Helmert contrasts may lead to confusions with other contrasts. Therefore, we could provide our own labels:\nlabels=[\"c2.1\", \"c3.12\", \"c4.123\", \"c5.1234\"]\nOnce the order of levels is memorized the proposed labelling is very transparent.", + "crumbs": [ + "Contrast coding", + "Mixed Models Tutorial: Contrast Coding" + ] + }, + { + "objectID": "contrasts_fggk21.html#hypothesiscoding-contr3", + "href": "contrasts_fggk21.html#hypothesiscoding-contr3", + "title": "Mixed Models Tutorial: Contrast Coding", + "section": "2.3 HypothesisCoding: contr3", + "text": "2.3 HypothesisCoding: contr3\nThe third set of contrasts uses HypothesisCoding. Hypothesis coding allows the user to specify their own a priori contrast matrix, subject to the mathematical constraint that the matrix has full rank. For example, sport scientists agree that the first four tests can be contrasted with BPT, because the difference is akin to a correction of overall physical fitness. However, they want to keep the pairwise comparisons for the first four tests.\n\nHyC1: BPT - mean(1,2,3,4)\nHyC2: Star_r - Run_r\nHyC3: Run_r - S20_r\nHyC4: S20_r - SLJ\n\n\ncontr3 = Dict(\n :School => Grouping(),\n :Child => Grouping(),\n :Cohort => Grouping(),\n :Sex => EffectsCoding(; levels=[\"Girls\", \"Boys\"]),\n :Test => HypothesisCoding(\n [\n -1 -1 -1 -1 +4\n -1 +1 0 0 0\n 0 -1 +1 0 0\n 0 0 -1 +1 0\n ];\n levels=[\"Run\", \"Star_r\", \"S20_r\", \"SLJ\", \"BPT\"],\n labels=[\"BPT-other\", \"Star-End\", \"S20-Star\", \"SLJ-S20\"],\n ),\n);\n\n\nm_ovi_Hypo = fit(MixedModel, f_ovi, dat; contrasts=contr3)\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0181\n0.0140\n-1.29\n0.1970\n0.6929\n\n\nTest: BPT-other\n-0.0899\n0.1387\n-0.65\n0.5170\n\n\n\nTest: Star-End\n-0.0177\n0.0439\n-0.40\n0.6865\n\n\n\nTest: S20-Star\n-0.0086\n0.0438\n-0.20\n0.8445\n\n\n\nTest: SLJ-S20\n0.0129\n0.0437\n0.30\n0.7677\n\n\n\na1\n0.3216\n0.0481\n6.69\n<1e-10\n\n\n\nSex: Boys\n0.2051\n0.0140\n14.61\n<1e-47\n\n\n\nTest: BPT-other & a1\n1.4231\n0.4663\n3.05\n0.0023\n\n\n\nTest: Star-End & a1\n0.2878\n0.1506\n1.91\n0.0560\n\n\n\nTest: S20-Star & a1\n0.0791\n0.1513\n0.52\n0.6013\n\n\n\nTest: SLJ-S20 & a1\n-0.0890\n0.1505\n-0.59\n0.5542\n\n\n\nTest: BPT-other & Sex: Boys\n0.8120\n0.1387\n5.85\n<1e-08\n\n\n\nTest: Star-End & Sex: Boys\n-0.1180\n0.0439\n-2.69\n0.0071\n\n\n\nTest: S20-Star & Sex: Boys\n0.0565\n0.0438\n1.29\n0.1972\n\n\n\nTest: SLJ-S20 & Sex: Boys\n0.0522\n0.0437\n1.19\n0.2327\n\n\n\na1 & Sex: Boys\n0.0510\n0.0481\n1.06\n0.2887\n\n\n\nTest: BPT-other & a1 & Sex: Boys\n0.3049\n0.4663\n0.65\n0.5132\n\n\n\nTest: Star-End & a1 & Sex: Boys\n-0.0145\n0.1506\n-0.10\n0.9231\n\n\n\nTest: S20-Star & a1 & Sex: Boys\n-0.0303\n0.1513\n-0.20\n0.8411\n\n\n\nTest: SLJ-S20 & a1 & Sex: Boys\n-0.0767\n0.1505\n-0.51\n0.6104\n\n\n\nResidual\n0.6732\n\n\n\n\n\n\n\n\n\nWith HypothesisCoding we must generate our own labels for the contrasts. The default labeling of contrasts is usually not interpretable. Therefore, we provide our own.\nAnyway, none of the interactions between age x Sex with the four Test contrasts was significant for these contrasts.\n\ncontr1b = Dict(\n :School => Grouping(),\n :Child => Grouping(),\n :Cohort => Grouping(),\n :Sex => EffectsCoding(; levels=[\"Girls\", \"Boys\"]),\n :Test => HypothesisCoding(\n [\n -1 +1 0 0 0\n 0 -1 +1 0 0\n 0 0 -1 +1 0\n 0 0 0 -1 +1\n ];\n levels=[\"Run\", \"Star_r\", \"S20_r\", \"SLJ\", \"BPT\"],\n labels=[\"Star-Run\", \"S20-Star\", \"SLJ-S20\", \"BPT-SLJ\"],\n ),\n);\n\n\nm_ovi_SeqDiff_v2 = fit(MixedModel, f_ovi, dat; contrasts=contr1b)\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0181\n0.0140\n-1.29\n0.1970\n0.6929\n\n\nTest: Star-Run\n-0.0177\n0.0439\n-0.40\n0.6865\n\n\n\nTest: S20-Star\n-0.0086\n0.0438\n-0.20\n0.8445\n\n\n\nTest: SLJ-S20\n0.0129\n0.0437\n0.30\n0.7677\n\n\n\nTest: BPT-SLJ\n-0.0234\n0.0437\n-0.54\n0.5918\n\n\n\na1\n0.3216\n0.0481\n6.69\n<1e-10\n\n\n\nSex: Boys\n0.2051\n0.0140\n14.61\n<1e-47\n\n\n\nTest: Star-Run & a1\n0.2878\n0.1506\n1.91\n0.0560\n\n\n\nTest: S20-Star & a1\n0.0791\n0.1513\n0.52\n0.6013\n\n\n\nTest: SLJ-S20 & a1\n-0.0890\n0.1505\n-0.59\n0.5542\n\n\n\nTest: BPT-SLJ & a1\n0.3111\n0.1464\n2.12\n0.0336\n\n\n\nTest: Star-Run & Sex: Boys\n-0.1180\n0.0439\n-2.69\n0.0071\n\n\n\nTest: S20-Star & Sex: Boys\n0.0565\n0.0438\n1.29\n0.1972\n\n\n\nTest: SLJ-S20 & Sex: Boys\n0.0522\n0.0437\n1.19\n0.2327\n\n\n\nTest: BPT-SLJ & Sex: Boys\n0.1651\n0.0437\n3.78\n0.0002\n\n\n\na1 & Sex: Boys\n0.0510\n0.0481\n1.06\n0.2887\n\n\n\nTest: Star-Run & a1 & Sex: Boys\n-0.0145\n0.1506\n-0.10\n0.9231\n\n\n\nTest: S20-Star & a1 & Sex: Boys\n-0.0303\n0.1513\n-0.20\n0.8411\n\n\n\nTest: SLJ-S20 & a1 & Sex: Boys\n-0.0767\n0.1505\n-0.51\n0.6104\n\n\n\nTest: BPT-SLJ & a1 & Sex: Boys\n0.1525\n0.1464\n1.04\n0.2974\n\n\n\nResidual\n0.6732\n\n\n\n\n\n\n\n\n\n\nm_zcp_SeqD = let\n form = @formula(\n zScore ~ 1 + Test * a1 * Sex + zerocorr(1 + Test | Child)\n )\n fit(MixedModel, form, dat; contrasts=contr1b)\nend\n\nMinimizing 10 Time: 0:00:00 (10.34 ms/it)\n objective: 13854.925652136491\nMinimizing 149 Time: 0:00:01 (10.57 ms/it)\n\nMinimizing 150 Time: 0:00:01 (10.67 ms/it)\n objective: 13796.200602278586\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0178\n0.0141\n-1.27\n0.2055\n0.7122\n\n\nTest: Star-Run\n-0.0143\n0.0448\n-0.32\n0.7494\n0.0703\n\n\nTest: S20-Star\n-0.0107\n0.0437\n-0.24\n0.8067\n0.7214\n\n\nTest: SLJ-S20\n0.0076\n0.0428\n0.18\n0.8595\n0.4677\n\n\nTest: BPT-SLJ\n-0.0107\n0.0431\n-0.25\n0.8033\n0.0000\n\n\na1\n0.3214\n0.0481\n6.67\n<1e-10\n\n\n\nSex: Boys\n0.2053\n0.0141\n14.61\n<1e-47\n\n\n\nTest: Star-Run & a1\n0.2899\n0.1528\n1.90\n0.0578\n\n\n\nTest: S20-Star & a1\n0.0765\n0.1505\n0.51\n0.6111\n\n\n\nTest: SLJ-S20 & a1\n-0.0896\n0.1472\n-0.61\n0.5427\n\n\n\nTest: BPT-SLJ & a1\n0.3161\n0.1442\n2.19\n0.0284\n\n\n\nTest: Star-Run & Sex: Boys\n-0.1122\n0.0448\n-2.51\n0.0122\n\n\n\nTest: S20-Star & Sex: Boys\n0.0531\n0.0437\n1.22\n0.2239\n\n\n\nTest: SLJ-S20 & Sex: Boys\n0.0501\n0.0428\n1.17\n0.2417\n\n\n\nTest: BPT-SLJ & Sex: Boys\n0.1730\n0.0431\n4.01\n<1e-04\n\n\n\na1 & Sex: Boys\n0.0493\n0.0481\n1.02\n0.3060\n\n\n\nTest: Star-Run & a1 & Sex: Boys\n-0.0077\n0.1528\n-0.05\n0.9600\n\n\n\nTest: S20-Star & a1 & Sex: Boys\n-0.0256\n0.1505\n-0.17\n0.8647\n\n\n\nTest: SLJ-S20 & a1 & Sex: Boys\n-0.0772\n0.1472\n-0.52\n0.5998\n\n\n\nTest: BPT-SLJ & a1 & Sex: Boys\n0.1645\n0.1442\n1.14\n0.2540\n\n\n\nResidual\n0.4996\n\n\n\n\n\n\n\n\n\n\nm_zcp_SeqD_2 = let\n form = @formula(\n zScore ~ 1 + Test * a1 * Sex + (0 + Test | Child)\n )\n fit(MixedModel, form, dat; contrasts=contr1b)\nend\n\nMinimizing 14 Time: 0:00:00 ( 7.45 ms/it)\n objective: 13982.580514043671\nMinimizing 26 Time: 0:00:00 ( 7.94 ms/it)\n objective: 14030.232041216525\nMinimizing 40 Time: 0:00:00 ( 7.75 ms/it)\n objective: 13809.467531689255\nMinimizing 54 Time: 0:00:00 ( 7.68 ms/it)\n objective: 13808.92595938738\nMinimizing 67 Time: 0:00:00 ( 7.69 ms/it)\n objective: 13806.975111543254\nMinimizing 80 Time: 0:00:00 ( 7.70 ms/it)\n objective: 13802.26846900888\nMinimizing 93 Time: 0:00:00 ( 7.75 ms/it)\n objective: 13794.493144568278\nMinimizing 106 Time: 0:00:00 ( 7.77 ms/it)\n objective: 13788.910178096115\nMinimizing 120 Time: 0:00:00 ( 7.76 ms/it)\n objective: 13785.10690625065\nMinimizing 134 Time: 0:00:01 ( 7.73 ms/it)\n objective: 13772.45481229144\nMinimizing 148 Time: 0:00:01 ( 7.73 ms/it)\n objective: 13761.769858317157\nMinimizing 162 Time: 0:00:01 ( 7.73 ms/it)\n objective: 13749.914617499626\nMinimizing 176 Time: 0:00:01 ( 7.70 ms/it)\n objective: 13748.901412092946\nMinimizing 190 Time: 0:00:01 ( 7.69 ms/it)\n objective: 13745.493223607971\nMinimizing 204 Time: 0:00:01 ( 7.69 ms/it)\n objective: 13739.913892056678\nMinimizing 218 Time: 0:00:01 ( 7.68 ms/it)\n objective: 13738.798251254031\nMinimizing 232 Time: 0:00:01 ( 7.65 ms/it)\n objective: 13731.696418338688\nMinimizing 246 Time: 0:00:01 ( 7.70 ms/it)\n objective: 13725.829887995456\nMinimizing 260 Time: 0:00:02 ( 7.69 ms/it)\n objective: 13715.992145258002\nMinimizing 273 Time: 0:00:02 ( 7.72 ms/it)\n objective: 13713.529683296727\nMinimizing 286 Time: 0:00:02 ( 7.74 ms/it)\n objective: 13707.94234615188\nMinimizing 299 Time: 0:00:02 ( 7.75 ms/it)\n objective: 13707.469665698492\nMinimizing 315 Time: 0:00:02 ( 7.79 ms/it)\n objective: 13702.526901215406\nMinimizing 330 Time: 0:00:02 ( 7.79 ms/it)\n objective: 13701.407982621175\nMinimizing 344 Time: 0:00:02 ( 7.78 ms/it)\n objective: 13698.046594204667\nMinimizing 357 Time: 0:00:02 ( 7.77 ms/it)\n objective: 13687.964903666249\nMinimizing 371 Time: 0:00:02 ( 7.77 ms/it)\n objective: 13681.454854918124\nMinimizing 384 Time: 0:00:02 ( 7.76 ms/it)\n objective: 13675.896375570843\nMinimizing 398 Time: 0:00:03 ( 7.76 ms/it)\n objective: 13669.884022295657\nMinimizing 412 Time: 0:00:03 ( 7.76 ms/it)\n objective: 13666.58031997356\nMinimizing 426 Time: 0:00:03 ( 7.76 ms/it)\n objective: 13654.153332034997\nMinimizing 440 Time: 0:00:03 ( 7.75 ms/it)\n objective: 13647.775301708025\nMinimizing 454 Time: 0:00:03 ( 7.74 ms/it)\n objective: 13643.014192075638\nMinimizing 468 Time: 0:00:03 ( 7.74 ms/it)\n objective: 13638.312250214072\nMinimizing 482 Time: 0:00:03 ( 7.74 ms/it)\n objective: 13634.764313388223\nMinimizing 496 Time: 0:00:03 ( 7.73 ms/it)\n objective: 13629.911540988593\nMinimizing 510 Time: 0:00:03 ( 7.73 ms/it)\n objective: 13618.510459066481\nMinimizing 523 Time: 0:00:04 ( 7.73 ms/it)\n objective: 13617.441944210696\nMinimizing 536 Time: 0:00:04 ( 7.75 ms/it)\n objective: 13615.944580738964\nMinimizing 549 Time: 0:00:04 ( 7.76 ms/it)\n objective: 13614.323330292325\nMinimizing 562 Time: 0:00:04 ( 7.76 ms/it)\n objective: 13609.210957579151\nMinimizing 575 Time: 0:00:04 ( 7.77 ms/it)\n objective: 13608.746693222904\nMinimizing 588 Time: 0:00:04 ( 7.77 ms/it)\n objective: 13600.905356946085\nMinimizing 601 Time: 0:00:04 ( 7.77 ms/it)\n objective: 13600.28561465062\nMinimizing 615 Time: 0:00:04 ( 7.76 ms/it)\n objective: 13599.201972133687\nMinimizing 629 Time: 0:00:04 ( 7.75 ms/it)\n objective: 13597.630840084275\nMinimizing 643 Time: 0:00:04 ( 7.74 ms/it)\n objective: 13587.61837899172\nMinimizing 658 Time: 0:00:05 ( 7.73 ms/it)\n objective: 13586.009605659594\nMinimizing 672 Time: 0:00:05 ( 7.72 ms/it)\n objective: 13584.544374166664\nMinimizing 686 Time: 0:00:05 ( 7.71 ms/it)\n objective: 13583.630001859437\nMinimizing 700 Time: 0:00:05 ( 7.70 ms/it)\n objective: 13582.914338085047\nMinimizing 714 Time: 0:00:05 ( 7.70 ms/it)\n objective: 13578.729620405957\nMinimizing 728 Time: 0:00:05 ( 7.69 ms/it)\n objective: 13576.410341664843\nMinimizing 743 Time: 0:00:05 ( 7.68 ms/it)\n objective: 13575.535008026138\nMinimizing 758 Time: 0:00:05 ( 7.66 ms/it)\n objective: 13569.203202734127\nMinimizing 773 Time: 0:00:05 ( 7.65 ms/it)\n objective: 13566.083881071609\nMinimizing 788 Time: 0:00:06 ( 7.64 ms/it)\n objective: 13562.327394175594\nMinimizing 803 Time: 0:00:06 ( 7.63 ms/it)\n objective: 13559.755657492522\nMinimizing 818 Time: 0:00:06 ( 7.62 ms/it)\n objective: 13550.265769897971\nMinimizing 832 Time: 0:00:06 ( 7.62 ms/it)\n objective: 13549.204684198947\nMinimizing 846 Time: 0:00:06 ( 7.62 ms/it)\n objective: 13543.78672801541\nMinimizing 860 Time: 0:00:06 ( 7.62 ms/it)\n objective: 13540.083343502723\nMinimizing 874 Time: 0:00:06 ( 7.62 ms/it)\n objective: 13532.954282739014\nMinimizing 888 Time: 0:00:06 ( 7.62 ms/it)\n objective: 13518.578757075324\nMinimizing 903 Time: 0:00:06 ( 7.61 ms/it)\n objective: 13516.583189945406\nMinimizing 918 Time: 0:00:06 ( 7.60 ms/it)\n objective: 13511.137765243213\nMinimizing 933 Time: 0:00:07 ( 7.59 ms/it)\n objective: 13508.982220839549\nMinimizing 948 Time: 0:00:07 ( 7.59 ms/it)\n objective: 13508.106646485307\nMinimizing 963 Time: 0:00:07 ( 7.58 ms/it)\n objective: 13507.38564326617\nMinimizing 978 Time: 0:00:07 ( 7.57 ms/it)\n objective: 13506.100735023087\nMinimizing 992 Time: 0:00:07 ( 7.57 ms/it)\n objective: 13505.848916881616\nMinimizing 1007 Time: 0:00:07 ( 7.56 ms/it)\n objective: 13503.662529956928\nMinimizing 1022 Time: 0:00:07 ( 7.55 ms/it)\n objective: 13503.198231182243\nMinimizing 1037 Time: 0:00:07 ( 7.54 ms/it)\n objective: 13499.10908455214\nMinimizing 1052 Time: 0:00:07 ( 7.53 ms/it)\n objective: 13498.270363498952\nMinimizing 1067 Time: 0:00:08 ( 7.52 ms/it)\n objective: 13494.61716876943\nMinimizing 1082 Time: 0:00:08 ( 7.52 ms/it)\n objective: 13494.426303182518\nMinimizing 1097 Time: 0:00:08 ( 7.51 ms/it)\n objective: 13493.949820444395\nMinimizing 1112 Time: 0:00:08 ( 7.51 ms/it)\n objective: 13492.876060336108\nMinimizing 1126 Time: 0:00:08 ( 7.50 ms/it)\n objective: 13492.319088075921\nMinimizing 1140 Time: 0:00:08 ( 7.50 ms/it)\n objective: 13492.043682367279\nMinimizing 1154 Time: 0:00:08 ( 7.50 ms/it)\n objective: 13488.238151284473\nMinimizing 1169 Time: 0:00:08 ( 7.49 ms/it)\n objective: 13487.95914316905\nMinimizing 1184 Time: 0:00:08 ( 7.49 ms/it)\n objective: 13485.29773348049\nMinimizing 1199 Time: 0:00:08 ( 7.49 ms/it)\n objective: 13484.964179269911\nMinimizing 1214 Time: 0:00:09 ( 7.48 ms/it)\n objective: 13484.472892658916\nMinimizing 1228 Time: 0:00:09 ( 7.48 ms/it)\n objective: 13484.206739258865\nMinimizing 1242 Time: 0:00:09 ( 7.48 ms/it)\n objective: 13483.872450555144\nMinimizing 1256 Time: 0:00:09 ( 7.48 ms/it)\n objective: 13483.427564764825\nMinimizing 1270 Time: 0:00:09 ( 7.48 ms/it)\n objective: 13482.660347165816\nMinimizing 1285 Time: 0:00:09 ( 7.47 ms/it)\n objective: 13480.934818693167\nMinimizing 1300 Time: 0:00:09 ( 7.47 ms/it)\n objective: 13480.415742812082\nMinimizing 1315 Time: 0:00:09 ( 7.46 ms/it)\n objective: 13478.599813143883\nMinimizing 1330 Time: 0:00:09 ( 7.46 ms/it)\n objective: 13478.152315558713\nMinimizing 1345 Time: 0:00:10 ( 7.46 ms/it)\n objective: 13477.37579554788\nMinimizing 1360 Time: 0:00:10 ( 7.45 ms/it)\n objective: 13476.929710265875\nMinimizing 1375 Time: 0:00:10 ( 7.45 ms/it)\n objective: 13476.701806857083\nMinimizing 1390 Time: 0:00:10 ( 7.44 ms/it)\n objective: 13475.606446069483\nMinimizing 1405 Time: 0:00:10 ( 7.44 ms/it)\n objective: 13474.631828306214\nMinimizing 1420 Time: 0:00:10 ( 7.44 ms/it)\n objective: 13473.865552192306\nMinimizing 1435 Time: 0:00:10 ( 7.43 ms/it)\n objective: 13472.428076141157\nMinimizing 1450 Time: 0:00:10 ( 7.43 ms/it)\n objective: 13471.530112313987\nMinimizing 1465 Time: 0:00:10 ( 7.42 ms/it)\n objective: 13470.350915251656\nMinimizing 1480 Time: 0:00:10 ( 7.42 ms/it)\n objective: 13470.168450050362\nMinimizing 1495 Time: 0:00:11 ( 7.41 ms/it)\n objective: 13469.701008994685\nMinimizing 1510 Time: 0:00:11 ( 7.41 ms/it)\n objective: 13468.923778066266\nMinimizing 1525 Time: 0:00:11 ( 7.41 ms/it)\n objective: 13464.681036416456\nMinimizing 1540 Time: 0:00:11 ( 7.40 ms/it)\n objective: 13464.42236593872\nMinimizing 1555 Time: 0:00:11 ( 7.40 ms/it)\n objective: 13463.852645869476\nMinimizing 1570 Time: 0:00:11 ( 7.39 ms/it)\n objective: 13463.362889527627\nMinimizing 1585 Time: 0:00:11 ( 7.39 ms/it)\n objective: 13463.10204613948\nMinimizing 1600 Time: 0:00:11 ( 7.38 ms/it)\n objective: 13462.543359438168\nMinimizing 1615 Time: 0:00:11 ( 7.38 ms/it)\n objective: 13462.172461736023\nMinimizing 1630 Time: 0:00:12 ( 7.38 ms/it)\n objective: 13461.449374484051\nMinimizing 1645 Time: 0:00:12 ( 7.37 ms/it)\n objective: 13461.190844090655\nMinimizing 1660 Time: 0:00:12 ( 7.37 ms/it)\n objective: 13458.674228341144\nMinimizing 1675 Time: 0:00:12 ( 7.37 ms/it)\n objective: 13458.559413870433\nMinimizing 1690 Time: 0:00:12 ( 7.37 ms/it)\n objective: 13457.809587834956\nMinimizing 1703 Time: 0:00:12 ( 7.37 ms/it)\n objective: 13457.679375278443\nMinimizing 1717 Time: 0:00:12 ( 7.38 ms/it)\n objective: 13456.458978047303\nMinimizing 1731 Time: 0:00:12 ( 7.38 ms/it)\n objective: 13456.31949736413\nMinimizing 1744 Time: 0:00:12 ( 7.38 ms/it)\n objective: 13455.712940619225\nMinimizing 1758 Time: 0:00:12 ( 7.38 ms/it)\n objective: 13453.67528533141\nMinimizing 1772 Time: 0:00:13 ( 7.38 ms/it)\n objective: 13450.595171898414\nMinimizing 1786 Time: 0:00:13 ( 7.38 ms/it)\n objective: 13450.533882012489\nMinimizing 1800 Time: 0:00:13 ( 7.38 ms/it)\n objective: 13450.346449755307\nMinimizing 1815 Time: 0:00:13 ( 7.37 ms/it)\n objective: 13450.059858557048\nMinimizing 1830 Time: 0:00:13 ( 7.37 ms/it)\n objective: 13449.955287495359\nMinimizing 1845 Time: 0:00:13 ( 7.37 ms/it)\n objective: 13449.832268378392\nMinimizing 1860 Time: 0:00:13 ( 7.37 ms/it)\n objective: 13449.482962102222\nMinimizing 1874 Time: 0:00:13 ( 7.37 ms/it)\n objective: 13449.140915889853\nMinimizing 1888 Time: 0:00:13 ( 7.37 ms/it)\n objective: 13448.870489174398\nMinimizing 1904 Time: 0:00:14 ( 7.37 ms/it)\n objective: 13447.857731424949\nMinimizing 1920 Time: 0:00:14 ( 7.37 ms/it)\n objective: 13447.409554398771\nMinimizing 1934 Time: 0:00:14 ( 7.37 ms/it)\n objective: 13445.769244726864\nMinimizing 1948 Time: 0:00:14 ( 7.37 ms/it)\n objective: 13444.935851343813\nMinimizing 1963 Time: 0:00:14 ( 7.36 ms/it)\n objective: 13444.37249914655\nMinimizing 1977 Time: 0:00:14 ( 7.36 ms/it)\n objective: 13442.32665539231\nMinimizing 1991 Time: 0:00:14 ( 7.36 ms/it)\n objective: 13442.196883789904\nMinimizing 2005 Time: 0:00:14 ( 7.36 ms/it)\n objective: 13441.999703874462\nMinimizing 2019 Time: 0:00:14 ( 7.36 ms/it)\n objective: 13441.833223438814\nMinimizing 2033 Time: 0:00:14 ( 7.36 ms/it)\n objective: 13441.773748989865\nMinimizing 2048 Time: 0:00:15 ( 7.36 ms/it)\n objective: 13441.294223560086\nMinimizing 2063 Time: 0:00:15 ( 7.36 ms/it)\n objective: 13441.258701306593\nMinimizing 2077 Time: 0:00:15 ( 7.36 ms/it)\n objective: 13440.924170162842\nMinimizing 2091 Time: 0:00:15 ( 7.36 ms/it)\n objective: 13440.8509067376\nMinimizing 2105 Time: 0:00:15 ( 7.36 ms/it)\n objective: 13440.546523816636\nMinimizing 2120 Time: 0:00:15 ( 7.36 ms/it)\n objective: 13440.203010858146\nMinimizing 2134 Time: 0:00:15 ( 7.35 ms/it)\n objective: 13439.926439225594\nMinimizing 2149 Time: 0:00:15 ( 7.35 ms/it)\n objective: 13438.816505054885\nMinimizing 2163 Time: 0:00:15 ( 7.35 ms/it)\n objective: 13436.848866931141\nMinimizing 2177 Time: 0:00:16 ( 7.35 ms/it)\n objective: 13436.647630267777\nMinimizing 2191 Time: 0:00:16 ( 7.35 ms/it)\n objective: 13435.633653786492\nMinimizing 2205 Time: 0:00:16 ( 7.35 ms/it)\n objective: 13435.44165971814\nMinimizing 2219 Time: 0:00:16 ( 7.35 ms/it)\n objective: 13434.875721214747\nMinimizing 2233 Time: 0:00:16 ( 7.35 ms/it)\n objective: 13434.795945264086\nMinimizing 2247 Time: 0:00:16 ( 7.35 ms/it)\n objective: 13434.623026371628\nMinimizing 2261 Time: 0:00:16 ( 7.35 ms/it)\n objective: 13434.420038974298\nMinimizing 2275 Time: 0:00:16 ( 7.35 ms/it)\n objective: 13434.062963390024\nMinimizing 2289 Time: 0:00:16 ( 7.35 ms/it)\n objective: 13433.588839592507\nMinimizing 2303 Time: 0:00:16 ( 7.35 ms/it)\n objective: 13433.334731066388\nMinimizing 2316 Time: 0:00:17 ( 7.36 ms/it)\n objective: 13432.535262732941\nMinimizing 2330 Time: 0:00:17 ( 7.36 ms/it)\n objective: 13432.033842502817\nMinimizing 2343 Time: 0:00:17 ( 7.36 ms/it)\n objective: 13431.8695811086\nMinimizing 2357 Time: 0:00:17 ( 7.36 ms/it)\n objective: 13431.454834187258\nMinimizing 2371 Time: 0:00:17 ( 7.36 ms/it)\n objective: 13430.793475601196\nMinimizing 2384 Time: 0:00:17 ( 7.37 ms/it)\n objective: 13430.310440057801\nMinimizing 2397 Time: 0:00:17 ( 7.37 ms/it)\n objective: 13430.055505730837\nMinimizing 2411 Time: 0:00:17 ( 7.37 ms/it)\n objective: 13428.816399686963\nMinimizing 2425 Time: 0:00:17 ( 7.37 ms/it)\n objective: 13428.590357249224\nMinimizing 2439 Time: 0:00:17 ( 7.37 ms/it)\n objective: 13428.28594486956\nMinimizing 2453 Time: 0:00:18 ( 7.36 ms/it)\n objective: 13427.991430509304\nMinimizing 2467 Time: 0:00:18 ( 7.36 ms/it)\n objective: 13427.555431311215\nMinimizing 2481 Time: 0:00:18 ( 7.36 ms/it)\n objective: 13427.129482832817\nMinimizing 2495 Time: 0:00:18 ( 7.36 ms/it)\n objective: 13426.882725501258\nMinimizing 2509 Time: 0:00:18 ( 7.36 ms/it)\n objective: 13426.628293662565\nMinimizing 2523 Time: 0:00:18 ( 7.36 ms/it)\n objective: 13426.43264934884\nMinimizing 2537 Time: 0:00:18 ( 7.36 ms/it)\n objective: 13425.966598576852\nMinimizing 2551 Time: 0:00:18 ( 7.36 ms/it)\n objective: 13425.918839314189\nMinimizing 2565 Time: 0:00:18 ( 7.36 ms/it)\n objective: 13425.549786421514\nMinimizing 2579 Time: 0:00:18 ( 7.36 ms/it)\n objective: 13425.351238154704\nMinimizing 2593 Time: 0:00:19 ( 7.36 ms/it)\n objective: 13425.20175381493\nMinimizing 2607 Time: 0:00:19 ( 7.36 ms/it)\n objective: 13424.70773917163\nMinimizing 2621 Time: 0:00:19 ( 7.36 ms/it)\n objective: 13424.567450979332\nMinimizing 2635 Time: 0:00:19 ( 7.36 ms/it)\n objective: 13423.971690599428\nMinimizing 2649 Time: 0:00:19 ( 7.36 ms/it)\n objective: 13421.990083325189\nMinimizing 2663 Time: 0:00:19 ( 7.36 ms/it)\n objective: 13421.634796240483\nMinimizing 2677 Time: 0:00:19 ( 7.36 ms/it)\n objective: 13420.922178233464\nMinimizing 2691 Time: 0:00:19 ( 7.36 ms/it)\n objective: 13420.907396531918\nMinimizing 2705 Time: 0:00:19 ( 7.36 ms/it)\n objective: 13420.664771435695\nMinimizing 2719 Time: 0:00:20 ( 7.36 ms/it)\n objective: 13420.3551541278\nMinimizing 2733 Time: 0:00:20 ( 7.36 ms/it)\n objective: 13420.288588365496\nMinimizing 2747 Time: 0:00:20 ( 7.36 ms/it)\n objective: 13420.146606798058\nMinimizing 2761 Time: 0:00:20 ( 7.36 ms/it)\n objective: 13419.13964667276\nMinimizing 2775 Time: 0:00:20 ( 7.36 ms/it)\n objective: 13417.111267245295\nMinimizing 2789 Time: 0:00:20 ( 7.36 ms/it)\n objective: 13417.001009317311\nMinimizing 2803 Time: 0:00:20 ( 7.36 ms/it)\n objective: 13416.528035796677\nMinimizing 2817 Time: 0:00:20 ( 7.36 ms/it)\n objective: 13416.130682686024\nMinimizing 2831 Time: 0:00:20 ( 7.36 ms/it)\n objective: 13415.491576423505\nMinimizing 2845 Time: 0:00:20 ( 7.36 ms/it)\n objective: 13415.464207226556\nMinimizing 2859 Time: 0:00:21 ( 7.36 ms/it)\n objective: 13414.690570045917\nMinimizing 2872 Time: 0:00:21 ( 7.36 ms/it)\n objective: 13414.590236106473\nMinimizing 2886 Time: 0:00:21 ( 7.36 ms/it)\n objective: 13414.368719160782\nMinimizing 2900 Time: 0:00:21 ( 7.36 ms/it)\n objective: 13413.691742222043\nMinimizing 2914 Time: 0:00:21 ( 7.37 ms/it)\n objective: 13411.294449664012\nMinimizing 2928 Time: 0:00:21 ( 7.37 ms/it)\n objective: 13411.011912399037\nMinimizing 2942 Time: 0:00:21 ( 7.37 ms/it)\n objective: 13410.80828546598\nMinimizing 2956 Time: 0:00:21 ( 7.37 ms/it)\n objective: 13410.423020476803\nMinimizing 2970 Time: 0:00:21 ( 7.37 ms/it)\n objective: 13410.353191679213\nMinimizing 2984 Time: 0:00:21 ( 7.37 ms/it)\n objective: 13410.266555457682\nMinimizing 2998 Time: 0:00:22 ( 7.37 ms/it)\n objective: 13410.230006247453\nMinimizing 3012 Time: 0:00:22 ( 7.37 ms/it)\n objective: 13410.166164442227\nMinimizing 3026 Time: 0:00:22 ( 7.37 ms/it)\n objective: 13409.98275589857\nMinimizing 3040 Time: 0:00:22 ( 7.37 ms/it)\n objective: 13409.864079267034\nMinimizing 3054 Time: 0:00:22 ( 7.37 ms/it)\n objective: 13409.65827262085\nMinimizing 3068 Time: 0:00:22 ( 7.37 ms/it)\n objective: 13409.497854791545\nMinimizing 3083 Time: 0:00:22 ( 7.37 ms/it)\n objective: 13409.049622542632\nMinimizing 3096 Time: 0:00:22 ( 7.38 ms/it)\n objective: 13408.989122638908\nMinimizing 3108 Time: 0:00:22 ( 7.38 ms/it)\n objective: 13408.866659970707\nMinimizing 3120 Time: 0:00:23 ( 7.39 ms/it)\n objective: 13408.818536220337\nMinimizing 3132 Time: 0:00:23 ( 7.39 ms/it)\n objective: 13408.74400325802\nMinimizing 3144 Time: 0:00:23 ( 7.40 ms/it)\n objective: 13408.59428742277\nMinimizing 3156 Time: 0:00:23 ( 7.41 ms/it)\n objective: 13408.458097941693\nMinimizing 3168 Time: 0:00:23 ( 7.41 ms/it)\n objective: 13408.161666971784\nMinimizing 3180 Time: 0:00:23 ( 7.42 ms/it)\n objective: 13407.714453381945\nMinimizing 3192 Time: 0:00:23 ( 7.42 ms/it)\n objective: 13407.532956535375\nMinimizing 3206 Time: 0:00:23 ( 7.42 ms/it)\n objective: 13407.49938616791\nMinimizing 3221 Time: 0:00:23 ( 7.42 ms/it)\n objective: 13407.366364307403\nMinimizing 3236 Time: 0:00:23 ( 7.42 ms/it)\n objective: 13407.302256937379\nMinimizing 3251 Time: 0:00:24 ( 7.41 ms/it)\n objective: 13407.178771271014\nMinimizing 3266 Time: 0:00:24 ( 7.41 ms/it)\n objective: 13407.105039817245\nMinimizing 3281 Time: 0:00:24 ( 7.41 ms/it)\n objective: 13406.806173895304\nMinimizing 3296 Time: 0:00:24 ( 7.41 ms/it)\n objective: 13406.735740287513\nMinimizing 3311 Time: 0:00:24 ( 7.41 ms/it)\n objective: 13406.380681041876\nMinimizing 3325 Time: 0:00:24 ( 7.41 ms/it)\n objective: 13406.33203176374\nMinimizing 3339 Time: 0:00:24 ( 7.40 ms/it)\n objective: 13406.245764444386\nMinimizing 3354 Time: 0:00:24 ( 7.40 ms/it)\n objective: 13406.169515524845\nMinimizing 3369 Time: 0:00:24 ( 7.40 ms/it)\n objective: 13405.978776179181\nMinimizing 3384 Time: 0:00:25 ( 7.40 ms/it)\n objective: 13405.954334776397\nMinimizing 3399 Time: 0:00:25 ( 7.40 ms/it)\n objective: 13405.567426425652\nMinimizing 3414 Time: 0:00:25 ( 7.39 ms/it)\n objective: 13405.087683400248\nMinimizing 3429 Time: 0:00:25 ( 7.39 ms/it)\n objective: 13405.015193295352\nMinimizing 3444 Time: 0:00:25 ( 7.39 ms/it)\n objective: 13404.81997576858\nMinimizing 3459 Time: 0:00:25 ( 7.39 ms/it)\n objective: 13404.572166834747\nMinimizing 3474 Time: 0:00:25 ( 7.39 ms/it)\n objective: 13404.525812108724\nMinimizing 3489 Time: 0:00:25 ( 7.38 ms/it)\n objective: 13404.250175499408\nMinimizing 3504 Time: 0:00:25 ( 7.38 ms/it)\n objective: 13404.120031650426\nMinimizing 3519 Time: 0:00:25 ( 7.38 ms/it)\n objective: 13404.008628592579\nMinimizing 3534 Time: 0:00:26 ( 7.38 ms/it)\n objective: 13403.695365184198\nMinimizing 3549 Time: 0:00:26 ( 7.38 ms/it)\n objective: 13403.451990218942\nMinimizing 3564 Time: 0:00:26 ( 7.38 ms/it)\n objective: 13403.039052210072\nMinimizing 3578 Time: 0:00:26 ( 7.37 ms/it)\n objective: 13402.757584794817\nMinimizing 3593 Time: 0:00:26 ( 7.37 ms/it)\n objective: 13402.601892271661\nMinimizing 3608 Time: 0:00:26 ( 7.37 ms/it)\n objective: 13402.22529925144\nMinimizing 3623 Time: 0:00:26 ( 7.37 ms/it)\n objective: 13402.014501068501\nMinimizing 3638 Time: 0:00:26 ( 7.37 ms/it)\n objective: 13401.59879583976\nMinimizing 3653 Time: 0:00:26 ( 7.37 ms/it)\n objective: 13401.425982988963\nMinimizing 3667 Time: 0:00:27 ( 7.37 ms/it)\n objective: 13401.106119077202\nMinimizing 3682 Time: 0:00:27 ( 7.36 ms/it)\n objective: 13400.95897524237\nMinimizing 3697 Time: 0:00:27 ( 7.36 ms/it)\n objective: 13400.84641682946\nMinimizing 3712 Time: 0:00:27 ( 7.36 ms/it)\n objective: 13400.622923207833\nMinimizing 3727 Time: 0:00:27 ( 7.36 ms/it)\n objective: 13400.531820088043\nMinimizing 3742 Time: 0:00:27 ( 7.36 ms/it)\n objective: 13400.403974232504\nMinimizing 3757 Time: 0:00:27 ( 7.36 ms/it)\n objective: 13400.027524652294\nMinimizing 3772 Time: 0:00:27 ( 7.36 ms/it)\n objective: 13399.956298843907\nMinimizing 3787 Time: 0:00:27 ( 7.35 ms/it)\n objective: 13399.795414080261\nMinimizing 3802 Time: 0:00:27 ( 7.35 ms/it)\n objective: 13399.30786695311\nMinimizing 3817 Time: 0:00:28 ( 7.35 ms/it)\n objective: 13399.015874677112\nMinimizing 3832 Time: 0:00:28 ( 7.35 ms/it)\n objective: 13398.982769213508\nMinimizing 3847 Time: 0:00:28 ( 7.35 ms/it)\n objective: 13398.751440909647\nMinimizing 3862 Time: 0:00:28 ( 7.35 ms/it)\n objective: 13398.372395532657\nMinimizing 3877 Time: 0:00:28 ( 7.35 ms/it)\n objective: 13398.338531781177\nMinimizing 3892 Time: 0:00:28 ( 7.34 ms/it)\n objective: 13398.185657952337\nMinimizing 3907 Time: 0:00:28 ( 7.34 ms/it)\n objective: 13397.906844047262\nMinimizing 3922 Time: 0:00:28 ( 7.34 ms/it)\n objective: 13397.802749459792\nMinimizing 3937 Time: 0:00:28 ( 7.34 ms/it)\n objective: 13397.628794346805\nMinimizing 3952 Time: 0:00:29 ( 7.34 ms/it)\n objective: 13397.438395749821\nMinimizing 3967 Time: 0:00:29 ( 7.34 ms/it)\n objective: 13396.853688099436\nMinimizing 3982 Time: 0:00:29 ( 7.33 ms/it)\n objective: 13396.742808832169\nMinimizing 3997 Time: 0:00:29 ( 7.33 ms/it)\n objective: 13396.691638782577\nMinimizing 4012 Time: 0:00:29 ( 7.33 ms/it)\n objective: 13396.607579192801\nMinimizing 4027 Time: 0:00:29 ( 7.33 ms/it)\n objective: 13396.434557549088\nMinimizing 4042 Time: 0:00:29 ( 7.33 ms/it)\n objective: 13396.2596687127\nMinimizing 4057 Time: 0:00:29 ( 7.33 ms/it)\n objective: 13396.050723842935\nMinimizing 4072 Time: 0:00:29 ( 7.33 ms/it)\n objective: 13395.983879727624\nMinimizing 4087 Time: 0:00:29 ( 7.33 ms/it)\n objective: 13395.615054056798\nMinimizing 4101 Time: 0:00:30 ( 7.33 ms/it)\n objective: 13395.519329171548\nMinimizing 4116 Time: 0:00:30 ( 7.32 ms/it)\n objective: 13395.224520234457\nMinimizing 4131 Time: 0:00:30 ( 7.32 ms/it)\n objective: 13394.9658075478\nMinimizing 4146 Time: 0:00:30 ( 7.32 ms/it)\n objective: 13394.791124196367\nMinimizing 4161 Time: 0:00:30 ( 7.32 ms/it)\n objective: 13394.542863835995\nMinimizing 4176 Time: 0:00:30 ( 7.32 ms/it)\n objective: 13394.444356962282\nMinimizing 4191 Time: 0:00:30 ( 7.32 ms/it)\n objective: 13394.215587366845\nMinimizing 4204 Time: 0:00:30 ( 7.33 ms/it)\n objective: 13394.052118449872\nMinimizing 4219 Time: 0:00:30 ( 7.32 ms/it)\n objective: 13393.735517275018\nMinimizing 4234 Time: 0:00:31 ( 7.32 ms/it)\n objective: 13393.44722601975\nMinimizing 4249 Time: 0:00:31 ( 7.32 ms/it)\n objective: 13392.689064977443\nMinimizing 4264 Time: 0:00:31 ( 7.32 ms/it)\n objective: 13392.5093225917\nMinimizing 4279 Time: 0:00:31 ( 7.32 ms/it)\n objective: 13392.44059960045\nMinimizing 4294 Time: 0:00:31 ( 7.32 ms/it)\n objective: 13392.393695754188\nMinimizing 4309 Time: 0:00:31 ( 7.32 ms/it)\n objective: 13392.362321211876\nMinimizing 4324 Time: 0:00:31 ( 7.31 ms/it)\n objective: 13392.171233814814\nMinimizing 4339 Time: 0:00:31 ( 7.31 ms/it)\n objective: 13392.014931868012\nMinimizing 4354 Time: 0:00:31 ( 7.31 ms/it)\n objective: 13391.834840869022\nMinimizing 4369 Time: 0:00:31 ( 7.31 ms/it)\n objective: 13391.60572624347\nMinimizing 4384 Time: 0:00:32 ( 7.31 ms/it)\n objective: 13391.4851080519\nMinimizing 4399 Time: 0:00:32 ( 7.31 ms/it)\n objective: 13391.351281604431\nMinimizing 4414 Time: 0:00:32 ( 7.31 ms/it)\n objective: 13391.02561016132\nMinimizing 4429 Time: 0:00:32 ( 7.31 ms/it)\n objective: 13391.005123049501\nMinimizing 4443 Time: 0:00:32 ( 7.31 ms/it)\n objective: 13390.969986950528\nMinimizing 4458 Time: 0:00:32 ( 7.31 ms/it)\n objective: 13390.839823986207\nMinimizing 4473 Time: 0:00:32 ( 7.31 ms/it)\n objective: 13390.77843726243\nMinimizing 4487 Time: 0:00:32 ( 7.31 ms/it)\n objective: 13390.753674325308\nMinimizing 4501 Time: 0:00:32 ( 7.31 ms/it)\n objective: 13390.628302796264\nMinimizing 4515 Time: 0:00:32 ( 7.31 ms/it)\n objective: 13390.489636480692\nMinimizing 4529 Time: 0:00:33 ( 7.31 ms/it)\n objective: 13390.268351402643\nMinimizing 4543 Time: 0:00:33 ( 7.31 ms/it)\n objective: 13390.070050332106\nMinimizing 4557 Time: 0:00:33 ( 7.31 ms/it)\n objective: 13390.031498748845\nMinimizing 4571 Time: 0:00:33 ( 7.31 ms/it)\n objective: 13390.01326925619\nMinimizing 4585 Time: 0:00:33 ( 7.31 ms/it)\n objective: 13389.683349396742\nMinimizing 4599 Time: 0:00:33 ( 7.31 ms/it)\n objective: 13389.635133653057\nMinimizing 4613 Time: 0:00:33 ( 7.31 ms/it)\n objective: 13389.504639993414\nMinimizing 4627 Time: 0:00:33 ( 7.31 ms/it)\n objective: 13389.36226139724\nMinimizing 4641 Time: 0:00:33 ( 7.31 ms/it)\n objective: 13388.960163631331\nMinimizing 4655 Time: 0:00:34 ( 7.31 ms/it)\n objective: 13388.944437128237\nMinimizing 4669 Time: 0:00:34 ( 7.31 ms/it)\n objective: 13388.830255904271\nMinimizing 4683 Time: 0:00:34 ( 7.32 ms/it)\n objective: 13388.816706349484\nMinimizing 4697 Time: 0:00:34 ( 7.32 ms/it)\n objective: 13388.750937512305\nMinimizing 4711 Time: 0:00:34 ( 7.32 ms/it)\n objective: 13388.732745979316\nMinimizing 4725 Time: 0:00:34 ( 7.32 ms/it)\n objective: 13388.477876255201\nMinimizing 4739 Time: 0:00:34 ( 7.32 ms/it)\n objective: 13388.457039927416\nMinimizing 4752 Time: 0:00:34 ( 7.32 ms/it)\n objective: 13388.279306965618\nMinimizing 4766 Time: 0:00:34 ( 7.32 ms/it)\n objective: 13388.190556589718\nMinimizing 4780 Time: 0:00:34 ( 7.32 ms/it)\n objective: 13387.951612964542\nMinimizing 4794 Time: 0:00:35 ( 7.32 ms/it)\n objective: 13387.899844082938\nMinimizing 4808 Time: 0:00:35 ( 7.32 ms/it)\n objective: 13387.474673898134\nMinimizing 4822 Time: 0:00:35 ( 7.32 ms/it)\n objective: 13387.356360838254\nMinimizing 4836 Time: 0:00:35 ( 7.32 ms/it)\n objective: 13387.319944968192\nMinimizing 4850 Time: 0:00:35 ( 7.32 ms/it)\n objective: 13387.220607053066\nMinimizing 4864 Time: 0:00:35 ( 7.33 ms/it)\n objective: 13387.093900436113\nMinimizing 4878 Time: 0:00:35 ( 7.33 ms/it)\n objective: 13386.941327755601\nMinimizing 4892 Time: 0:00:35 ( 7.33 ms/it)\n objective: 13386.874838433992\nMinimizing 4906 Time: 0:00:35 ( 7.33 ms/it)\n objective: 13386.416810829352\nMinimizing 4920 Time: 0:00:36 ( 7.33 ms/it)\n objective: 13386.320938527162\nMinimizing 4934 Time: 0:00:36 ( 7.33 ms/it)\n objective: 13386.205973585209\nMinimizing 4948 Time: 0:00:36 ( 7.33 ms/it)\n objective: 13385.97963379023\nMinimizing 4962 Time: 0:00:36 ( 7.33 ms/it)\n objective: 13385.919946994349\nMinimizing 4976 Time: 0:00:36 ( 7.33 ms/it)\n objective: 13385.133674770092\nMinimizing 4990 Time: 0:00:36 ( 7.33 ms/it)\n objective: 13384.961282848453\nMinimizing 5004 Time: 0:00:36 ( 7.33 ms/it)\n objective: 13384.939090435284\nMinimizing 5018 Time: 0:00:36 ( 7.33 ms/it)\n objective: 13384.787272787049\nMinimizing 5032 Time: 0:00:36 ( 7.33 ms/it)\n objective: 13384.660998351355\nMinimizing 5046 Time: 0:00:37 ( 7.33 ms/it)\n objective: 13384.516221564649\nMinimizing 5060 Time: 0:00:37 ( 7.33 ms/it)\n objective: 13384.667419611193\nMinimizing 5074 Time: 0:00:37 ( 7.34 ms/it)\n objective: 13384.112936695135\nMinimizing 5087 Time: 0:00:37 ( 7.34 ms/it)\n objective: 13383.952917105904\nMinimizing 5100 Time: 0:00:37 ( 7.34 ms/it)\n objective: 13383.883000572481\nMinimizing 5113 Time: 0:00:37 ( 7.34 ms/it)\n objective: 13383.869112148881\nMinimizing 5127 Time: 0:00:37 ( 7.34 ms/it)\n objective: 13383.784772963409\nMinimizing 5141 Time: 0:00:37 ( 7.34 ms/it)\n objective: 13383.676031899835\nMinimizing 5154 Time: 0:00:37 ( 7.34 ms/it)\n objective: 13383.596635321861\nMinimizing 5167 Time: 0:00:37 ( 7.34 ms/it)\n objective: 13383.420207179115\nMinimizing 5180 Time: 0:00:38 ( 7.35 ms/it)\n objective: 13383.35937420951\nMinimizing 5194 Time: 0:00:38 ( 7.35 ms/it)\n objective: 13383.412563334445\nMinimizing 5208 Time: 0:00:38 ( 7.35 ms/it)\n objective: 13383.232362762981\nMinimizing 5221 Time: 0:00:38 ( 7.35 ms/it)\n objective: 13383.104242076442\nMinimizing 5234 Time: 0:00:38 ( 7.35 ms/it)\n objective: 13383.0929790402\nMinimizing 5248 Time: 0:00:38 ( 7.35 ms/it)\n objective: 13382.859099446185\nMinimizing 5262 Time: 0:00:38 ( 7.35 ms/it)\n objective: 13382.777078811254\nMinimizing 5276 Time: 0:00:38 ( 7.35 ms/it)\n objective: 13382.586290260464\nMinimizing 5290 Time: 0:00:38 ( 7.35 ms/it)\n objective: 13382.339263743459\nMinimizing 5303 Time: 0:00:39 ( 7.35 ms/it)\n objective: 13382.330729079556\nMinimizing 5317 Time: 0:00:39 ( 7.36 ms/it)\n objective: 13382.204917918061\nMinimizing 5331 Time: 0:00:39 ( 7.36 ms/it)\n objective: 13382.121964126476\nMinimizing 5345 Time: 0:00:39 ( 7.36 ms/it)\n objective: 13381.873865505397\nMinimizing 5358 Time: 0:00:39 ( 7.36 ms/it)\n objective: 13381.855182459127\nMinimizing 5371 Time: 0:00:39 ( 7.36 ms/it)\n objective: 13381.750769383762\nMinimizing 5385 Time: 0:00:39 ( 7.36 ms/it)\n objective: 13381.710138183313\nMinimizing 5398 Time: 0:00:39 ( 7.36 ms/it)\n objective: 13381.620214353155\nMinimizing 5411 Time: 0:00:39 ( 7.36 ms/it)\n objective: 13381.466242282484\nMinimizing 5424 Time: 0:00:39 ( 7.36 ms/it)\n objective: 13380.88099694821\nMinimizing 5437 Time: 0:00:40 ( 7.36 ms/it)\n objective: 13380.852482779796\nMinimizing 5450 Time: 0:00:40 ( 7.37 ms/it)\n objective: 13380.819890855964\nMinimizing 5463 Time: 0:00:40 ( 7.37 ms/it)\n objective: 13380.736262859835\nMinimizing 5476 Time: 0:00:40 ( 7.37 ms/it)\n objective: 13380.672026715503\nMinimizing 5489 Time: 0:00:40 ( 7.37 ms/it)\n objective: 13380.506751959038\nMinimizing 5502 Time: 0:00:40 ( 7.37 ms/it)\n objective: 13380.438781375276\nMinimizing 5515 Time: 0:00:40 ( 7.37 ms/it)\n objective: 13380.368614276093\nMinimizing 5528 Time: 0:00:40 ( 7.37 ms/it)\n objective: 13380.185128732526\nMinimizing 5541 Time: 0:00:40 ( 7.37 ms/it)\n objective: 13380.136516668797\nMinimizing 5554 Time: 0:00:40 ( 7.38 ms/it)\n objective: 13379.972482921621\nMinimizing 5567 Time: 0:00:41 ( 7.38 ms/it)\n objective: 13379.939443298987\nMinimizing 5579 Time: 0:00:41 ( 7.38 ms/it)\n objective: 13379.905332863302\nMinimizing 5591 Time: 0:00:41 ( 7.38 ms/it)\n objective: 13379.86842277123\nMinimizing 5603 Time: 0:00:41 ( 7.39 ms/it)\n objective: 13379.525399804494\nMinimizing 5614 Time: 0:00:41 ( 7.39 ms/it)\n objective: 13379.403896043608\nMinimizing 5627 Time: 0:00:41 ( 7.39 ms/it)\n objective: 13378.956253257915\nMinimizing 5640 Time: 0:00:41 ( 7.39 ms/it)\n objective: 13378.045818564373\nMinimizing 5653 Time: 0:00:41 ( 7.40 ms/it)\n objective: 13377.79493796228\nMinimizing 5666 Time: 0:00:41 ( 7.40 ms/it)\n objective: 13377.668320128949\nMinimizing 5679 Time: 0:00:42 ( 7.40 ms/it)\n objective: 13377.658958734275\nMinimizing 5691 Time: 0:00:42 ( 7.40 ms/it)\n objective: 13377.637708129208\nMinimizing 5703 Time: 0:00:42 ( 7.40 ms/it)\n objective: 13377.625346839159\nMinimizing 5715 Time: 0:00:42 ( 7.41 ms/it)\n objective: 13377.617333501265\nMinimizing 5727 Time: 0:00:42 ( 7.41 ms/it)\n objective: 13377.54948557925\nMinimizing 5739 Time: 0:00:42 ( 7.41 ms/it)\n objective: 13377.502837544722\nMinimizing 5752 Time: 0:00:42 ( 7.41 ms/it)\n objective: 13377.466585695089\nMinimizing 5765 Time: 0:00:42 ( 7.42 ms/it)\n objective: 13377.456472188496\nMinimizing 5778 Time: 0:00:42 ( 7.42 ms/it)\n objective: 13377.420870036578\nMinimizing 5791 Time: 0:00:42 ( 7.42 ms/it)\n objective: 13377.406921252266\nMinimizing 5804 Time: 0:00:43 ( 7.42 ms/it)\n objective: 13377.370124584479\nMinimizing 5817 Time: 0:00:43 ( 7.42 ms/it)\n objective: 13377.35302146474\nMinimizing 5830 Time: 0:00:43 ( 7.42 ms/it)\n objective: 13377.349389796611\nMinimizing 5843 Time: 0:00:43 ( 7.42 ms/it)\n objective: 13377.34361501415\nMinimizing 5856 Time: 0:00:43 ( 7.42 ms/it)\n objective: 13377.325122791328\nMinimizing 5869 Time: 0:00:43 ( 7.42 ms/it)\n objective: 13377.29036253311\nMinimizing 5882 Time: 0:00:43 ( 7.42 ms/it)\n objective: 13377.280163644704\nMinimizing 5895 Time: 0:00:43 ( 7.42 ms/it)\n objective: 13377.246976870345\nMinimizing 5908 Time: 0:00:43 ( 7.43 ms/it)\n objective: 13377.227608630747\nMinimizing 5921 Time: 0:00:43 ( 7.43 ms/it)\n objective: 13377.198846318679\nMinimizing 5934 Time: 0:00:44 ( 7.43 ms/it)\n objective: 13377.149085639976\nMinimizing 5947 Time: 0:00:44 ( 7.43 ms/it)\n objective: 13377.130973619605\nMinimizing 5960 Time: 0:00:44 ( 7.43 ms/it)\n objective: 13377.069916933811\nMinimizing 5973 Time: 0:00:44 ( 7.43 ms/it)\n objective: 13377.054915751825\nMinimizing 5986 Time: 0:00:44 ( 7.43 ms/it)\n objective: 13376.868824740704\nMinimizing 5999 Time: 0:00:44 ( 7.43 ms/it)\n objective: 13376.79084285604\nMinimizing 6012 Time: 0:00:44 ( 7.43 ms/it)\n objective: 13376.749280155498\nMinimizing 6025 Time: 0:00:44 ( 7.43 ms/it)\n objective: 13376.682422649399\nMinimizing 6038 Time: 0:00:44 ( 7.44 ms/it)\n objective: 13376.655282042768\nMinimizing 6051 Time: 0:00:44 ( 7.44 ms/it)\n objective: 13376.585088006861\nMinimizing 6064 Time: 0:00:45 ( 7.44 ms/it)\n objective: 13376.565631842575\nMinimizing 6077 Time: 0:00:45 ( 7.44 ms/it)\n objective: 13376.46982149381\nMinimizing 6090 Time: 0:00:45 ( 7.44 ms/it)\n objective: 13376.448142946712\nMinimizing 6103 Time: 0:00:45 ( 7.44 ms/it)\n objective: 13376.434987296474\nMinimizing 6116 Time: 0:00:45 ( 7.44 ms/it)\n objective: 13376.339550814751\nMinimizing 6129 Time: 0:00:45 ( 7.44 ms/it)\n objective: 13376.333471791411\nMinimizing 6142 Time: 0:00:45 ( 7.44 ms/it)\n objective: 13376.153261703344\nMinimizing 6155 Time: 0:00:45 ( 7.44 ms/it)\n objective: 13376.124881930067\nMinimizing 6168 Time: 0:00:45 ( 7.45 ms/it)\n objective: 13376.04707555881\nMinimizing 6181 Time: 0:00:46 ( 7.45 ms/it)\n objective: 13375.978992237346\nMinimizing 6194 Time: 0:00:46 ( 7.45 ms/it)\n objective: 13375.948807614506\nMinimizing 6207 Time: 0:00:46 ( 7.45 ms/it)\n objective: 13375.688983930486\nMinimizing 6220 Time: 0:00:46 ( 7.45 ms/it)\n objective: 13375.644490464692\nMinimizing 6233 Time: 0:00:46 ( 7.45 ms/it)\n objective: 13375.639326058139\nMinimizing 6246 Time: 0:00:46 ( 7.45 ms/it)\n objective: 13375.591860274195\nMinimizing 6259 Time: 0:00:46 ( 7.45 ms/it)\n objective: 13375.568646805637\nMinimizing 6272 Time: 0:00:46 ( 7.45 ms/it)\n objective: 13375.490601102661\nMinimizing 6285 Time: 0:00:46 ( 7.45 ms/it)\n objective: 13375.474220636897\nMinimizing 6298 Time: 0:00:46 ( 7.46 ms/it)\n objective: 13375.448782051913\nMinimizing 6311 Time: 0:00:47 ( 7.46 ms/it)\n objective: 13375.42697524641\nMinimizing 6324 Time: 0:00:47 ( 7.46 ms/it)\n objective: 13375.371696402435\nMinimizing 6337 Time: 0:00:47 ( 7.46 ms/it)\n objective: 13375.357651158694\nMinimizing 6350 Time: 0:00:47 ( 7.46 ms/it)\n objective: 13375.296076487008\nMinimizing 6363 Time: 0:00:47 ( 7.46 ms/it)\n objective: 13375.24285208687\nMinimizing 6376 Time: 0:00:47 ( 7.46 ms/it)\n objective: 13375.224310229685\nMinimizing 6389 Time: 0:00:47 ( 7.46 ms/it)\n objective: 13375.201005784875\nMinimizing 6402 Time: 0:00:47 ( 7.46 ms/it)\n objective: 13375.154346305404\nMinimizing 6414 Time: 0:00:47 ( 7.47 ms/it)\n objective: 13375.116168044297\nMinimizing 6426 Time: 0:00:47 ( 7.47 ms/it)\n objective: 13375.020508141875\nMinimizing 6438 Time: 0:00:48 ( 7.47 ms/it)\n objective: 13374.981308338458\nMinimizing 6450 Time: 0:00:48 ( 7.47 ms/it)\n objective: 13374.87269590405\nMinimizing 6463 Time: 0:00:48 ( 7.47 ms/it)\n objective: 13374.66669194882\nMinimizing 6476 Time: 0:00:48 ( 7.47 ms/it)\n objective: 13374.607414354025\nMinimizing 6489 Time: 0:00:48 ( 7.47 ms/it)\n objective: 13374.557310891229\nMinimizing 6502 Time: 0:00:48 ( 7.48 ms/it)\n objective: 13374.520006731414\nMinimizing 6515 Time: 0:00:48 ( 7.48 ms/it)\n objective: 13374.427641703267\nMinimizing 6528 Time: 0:00:48 ( 7.48 ms/it)\n objective: 13374.404839222785\nMinimizing 6541 Time: 0:00:48 ( 7.48 ms/it)\n objective: 13374.331827232549\nMinimizing 6554 Time: 0:00:49 ( 7.48 ms/it)\n objective: 13374.310622945588\nMinimizing 6567 Time: 0:00:49 ( 7.48 ms/it)\n objective: 13374.224152422714\nMinimizing 6580 Time: 0:00:49 ( 7.48 ms/it)\n objective: 13374.077757193983\nMinimizing 6593 Time: 0:00:49 ( 7.48 ms/it)\n objective: 13373.99385817087\nMinimizing 6606 Time: 0:00:49 ( 7.48 ms/it)\n objective: 13373.822935274242\nMinimizing 6619 Time: 0:00:49 ( 7.48 ms/it)\n objective: 13373.782629826317\nMinimizing 6632 Time: 0:00:49 ( 7.48 ms/it)\n objective: 13373.749947493518\nMinimizing 6645 Time: 0:00:49 ( 7.48 ms/it)\n objective: 13373.715758730126\nMinimizing 6658 Time: 0:00:49 ( 7.49 ms/it)\n objective: 13373.683631695414\nMinimizing 6671 Time: 0:00:49 ( 7.49 ms/it)\n objective: 13373.606736935537\nMinimizing 6684 Time: 0:00:50 ( 7.49 ms/it)\n objective: 13373.544731723981\nMinimizing 6697 Time: 0:00:50 ( 7.49 ms/it)\n objective: 13373.455664094727\nMinimizing 6710 Time: 0:00:50 ( 7.49 ms/it)\n objective: 13373.41295664743\nMinimizing 6723 Time: 0:00:50 ( 7.49 ms/it)\n objective: 13373.321973085425\nMinimizing 6736 Time: 0:00:50 ( 7.49 ms/it)\n objective: 13373.271715735558\nMinimizing 6749 Time: 0:00:50 ( 7.49 ms/it)\n objective: 13372.982856866809\nMinimizing 6762 Time: 0:00:50 ( 7.49 ms/it)\n objective: 13373.33130436045\nMinimizing 6776 Time: 0:00:50 ( 7.49 ms/it)\n objective: 13371.965579579111\nMinimizing 6789 Time: 0:00:50 ( 7.49 ms/it)\n objective: 13371.936472508583\nMinimizing 6802 Time: 0:00:50 ( 7.49 ms/it)\n objective: 13371.898116064549\nMinimizing 6815 Time: 0:00:51 ( 7.49 ms/it)\n objective: 13371.839962030695\nMinimizing 6828 Time: 0:00:51 ( 7.50 ms/it)\n objective: 13371.786364898093\nMinimizing 6841 Time: 0:00:51 ( 7.50 ms/it)\n objective: 13371.779027829594\nMinimizing 6854 Time: 0:00:51 ( 7.50 ms/it)\n objective: 13371.652209540618\nMinimizing 6867 Time: 0:00:51 ( 7.50 ms/it)\n objective: 13371.542213625144\nMinimizing 6880 Time: 0:00:51 ( 7.50 ms/it)\n objective: 13371.45131608536\nMinimizing 6893 Time: 0:00:51 ( 7.50 ms/it)\n objective: 13371.370562589334\nMinimizing 6906 Time: 0:00:51 ( 7.50 ms/it)\n objective: 13371.364184471095\nMinimizing 6919 Time: 0:00:51 ( 7.50 ms/it)\n objective: 13371.103810562956\nMinimizing 6932 Time: 0:00:51 ( 7.50 ms/it)\n objective: 13371.697612452008\nMinimizing 6945 Time: 0:00:52 ( 7.50 ms/it)\n objective: 13370.611521213672\nMinimizing 6958 Time: 0:00:52 ( 7.50 ms/it)\n objective: 13370.590431240962\nMinimizing 6971 Time: 0:00:52 ( 7.50 ms/it)\n objective: 13370.579600251302\nMinimizing 6984 Time: 0:00:52 ( 7.50 ms/it)\n objective: 13370.569385500923\nMinimizing 6997 Time: 0:00:52 ( 7.50 ms/it)\n objective: 13370.50814392767\nMinimizing 7010 Time: 0:00:52 ( 7.50 ms/it)\n objective: 13370.485559016597\nMinimizing 7023 Time: 0:00:52 ( 7.51 ms/it)\n objective: 13370.424913073977\nMinimizing 7036 Time: 0:00:52 ( 7.51 ms/it)\n objective: 13370.15801874495\nMinimizing 7049 Time: 0:00:52 ( 7.51 ms/it)\n objective: 13370.131222709017\nMinimizing 7062 Time: 0:00:53 ( 7.51 ms/it)\n objective: 13370.115180201748\nMinimizing 7075 Time: 0:00:53 ( 7.51 ms/it)\n objective: 13370.108699870281\nMinimizing 7088 Time: 0:00:53 ( 7.51 ms/it)\n objective: 13370.081590758731\nMinimizing 7101 Time: 0:00:53 ( 7.51 ms/it)\n objective: 13370.075186012284\nMinimizing 7114 Time: 0:00:53 ( 7.51 ms/it)\n objective: 13370.026447433615\nMinimizing 7127 Time: 0:00:53 ( 7.51 ms/it)\n objective: 13370.012837310256\nMinimizing 7140 Time: 0:00:53 ( 7.51 ms/it)\n objective: 13369.980851301902\nMinimizing 7153 Time: 0:00:53 ( 7.51 ms/it)\n objective: 13369.964754558721\nMinimizing 7166 Time: 0:00:53 ( 7.51 ms/it)\n objective: 13369.913095109303\nMinimizing 7179 Time: 0:00:53 ( 7.51 ms/it)\n objective: 13369.89427704984\nMinimizing 7192 Time: 0:00:54 ( 7.51 ms/it)\n objective: 13370.178522857874\nMinimizing 7205 Time: 0:00:54 ( 7.51 ms/it)\n objective: 13369.756503244644\nMinimizing 7218 Time: 0:00:54 ( 7.52 ms/it)\n objective: 13369.743102026863\nMinimizing 7231 Time: 0:00:54 ( 7.52 ms/it)\n objective: 13369.702159832486\nMinimizing 7244 Time: 0:00:54 ( 7.52 ms/it)\n objective: 13369.651449666802\nMinimizing 7257 Time: 0:00:54 ( 7.52 ms/it)\n objective: 13369.5892650795\nMinimizing 7270 Time: 0:00:54 ( 7.52 ms/it)\n objective: 13369.580042344474\nMinimizing 7283 Time: 0:00:54 ( 7.52 ms/it)\n objective: 13369.550205761981\nMinimizing 7296 Time: 0:00:54 ( 7.52 ms/it)\n objective: 13369.53847638701\nMinimizing 7309 Time: 0:00:54 ( 7.52 ms/it)\n objective: 13369.501306758371\nMinimizing 7322 Time: 0:00:55 ( 7.52 ms/it)\n objective: 13369.475342705751\nMinimizing 7335 Time: 0:00:55 ( 7.52 ms/it)\n objective: 13369.470985072418\nMinimizing 7348 Time: 0:00:55 ( 7.52 ms/it)\n objective: 13369.41372521386\nMinimizing 7361 Time: 0:00:55 ( 7.52 ms/it)\n objective: 13369.385378006635\nMinimizing 7374 Time: 0:00:55 ( 7.52 ms/it)\n objective: 13369.357674377228\nMinimizing 7387 Time: 0:00:55 ( 7.52 ms/it)\n objective: 13369.245185528023\nMinimizing 7400 Time: 0:00:55 ( 7.52 ms/it)\n objective: 13369.232401075598\nMinimizing 7413 Time: 0:00:55 ( 7.52 ms/it)\n objective: 13369.115761863104\nMinimizing 7426 Time: 0:00:55 ( 7.52 ms/it)\n objective: 13369.022431451041\nMinimizing 7439 Time: 0:00:55 ( 7.53 ms/it)\n objective: 13369.01399809988\nMinimizing 7452 Time: 0:00:56 ( 7.53 ms/it)\n objective: 13368.9586116817\nMinimizing 7465 Time: 0:00:56 ( 7.53 ms/it)\n objective: 13368.904309428835\nMinimizing 7478 Time: 0:00:56 ( 7.53 ms/it)\n objective: 13368.849177516495\nMinimizing 7491 Time: 0:00:56 ( 7.53 ms/it)\n objective: 13368.844352713299\nMinimizing 7504 Time: 0:00:56 ( 7.53 ms/it)\n objective: 13368.88491520103\nMinimizing 7517 Time: 0:00:56 ( 7.53 ms/it)\n objective: 13368.504069727554\nMinimizing 7530 Time: 0:00:56 ( 7.53 ms/it)\n objective: 13368.316080250326\nMinimizing 7543 Time: 0:00:56 ( 7.53 ms/it)\n objective: 13368.30677029764\nMinimizing 7556 Time: 0:00:56 ( 7.53 ms/it)\n objective: 13368.265283500674\nMinimizing 7569 Time: 0:00:57 ( 7.53 ms/it)\n objective: 13368.162191943964\nMinimizing 7582 Time: 0:00:57 ( 7.53 ms/it)\n objective: 13368.042496879469\nMinimizing 7595 Time: 0:00:57 ( 7.53 ms/it)\n objective: 13367.991488810767\nMinimizing 7608 Time: 0:00:57 ( 7.53 ms/it)\n objective: 13367.96163360044\nMinimizing 7621 Time: 0:00:57 ( 7.53 ms/it)\n objective: 13367.92223936331\nMinimizing 7634 Time: 0:00:57 ( 7.53 ms/it)\n objective: 13367.907177319496\nMinimizing 7647 Time: 0:00:57 ( 7.53 ms/it)\n objective: 13367.877608048024\nMinimizing 7660 Time: 0:00:57 ( 7.53 ms/it)\n objective: 13367.702937814407\nMinimizing 7673 Time: 0:00:57 ( 7.54 ms/it)\n objective: 13367.619025732143\nMinimizing 7686 Time: 0:00:57 ( 7.54 ms/it)\n objective: 13367.421147783243\nMinimizing 7699 Time: 0:00:58 ( 7.54 ms/it)\n objective: 13367.32194098596\nMinimizing 7712 Time: 0:00:58 ( 7.54 ms/it)\n objective: 13367.252398044264\nMinimizing 7725 Time: 0:00:58 ( 7.54 ms/it)\n objective: 13367.19987273607\nMinimizing 7738 Time: 0:00:58 ( 7.54 ms/it)\n objective: 13367.188365503913\nMinimizing 7751 Time: 0:00:58 ( 7.54 ms/it)\n objective: 13367.150803571705\nMinimizing 7764 Time: 0:00:58 ( 7.54 ms/it)\n objective: 13367.144301395849\nMinimizing 7777 Time: 0:00:58 ( 7.54 ms/it)\n objective: 13367.069368550663\nMinimizing 7790 Time: 0:00:58 ( 7.54 ms/it)\n objective: 13367.058622049291\nMinimizing 7803 Time: 0:00:58 ( 7.54 ms/it)\n objective: 13367.050423728862\nMinimizing 7816 Time: 0:00:58 ( 7.54 ms/it)\n objective: 13367.025317448417\nMinimizing 7829 Time: 0:00:59 ( 7.54 ms/it)\n objective: 13367.004923522029\nMinimizing 7842 Time: 0:00:59 ( 7.54 ms/it)\n objective: 13366.958180232345\nMinimizing 7855 Time: 0:00:59 ( 7.54 ms/it)\n objective: 13366.889445437599\nMinimizing 7868 Time: 0:00:59 ( 7.54 ms/it)\n objective: 13366.873283626002\nMinimizing 7881 Time: 0:00:59 ( 7.54 ms/it)\n objective: 13366.849938914107\nMinimizing 7894 Time: 0:00:59 ( 7.55 ms/it)\n objective: 13366.813290796767\nMinimizing 7907 Time: 0:00:59 ( 7.55 ms/it)\n objective: 13366.790500041658\nMinimizing 7920 Time: 0:00:59 ( 7.55 ms/it)\n objective: 13366.751413991842\nMinimizing 7933 Time: 0:00:59 ( 7.55 ms/it)\n objective: 13366.735362938518\nMinimizing 7946 Time: 0:00:59 ( 7.55 ms/it)\n objective: 13366.707570033046\nMinimizing 7959 Time: 0:01:00 ( 7.55 ms/it)\n objective: 13366.676525630457\nMinimizing 7972 Time: 0:01:00 ( 7.55 ms/it)\n objective: 13366.57840019682\nMinimizing 7985 Time: 0:01:00 ( 7.55 ms/it)\n objective: 13366.485061246203\nMinimizing 7998 Time: 0:01:00 ( 7.55 ms/it)\n objective: 13366.457160909682\nMinimizing 8011 Time: 0:01:00 ( 7.55 ms/it)\n objective: 13366.416319651384\nMinimizing 8024 Time: 0:01:00 ( 7.55 ms/it)\n objective: 13366.403211679273\nMinimizing 8037 Time: 0:01:00 ( 7.55 ms/it)\n objective: 13366.224660476837\nMinimizing 8050 Time: 0:01:00 ( 7.55 ms/it)\n objective: 13366.120157493737\nMinimizing 8063 Time: 0:01:00 ( 7.55 ms/it)\n objective: 13366.07068632837\nMinimizing 8076 Time: 0:01:00 ( 7.55 ms/it)\n objective: 13365.858129255721\nMinimizing 8089 Time: 0:01:01 ( 7.55 ms/it)\n objective: 13365.748974905742\nMinimizing 8102 Time: 0:01:01 ( 7.55 ms/it)\n objective: 13365.729972563946\nMinimizing 8115 Time: 0:01:01 ( 7.55 ms/it)\n objective: 13365.693606154346\nMinimizing 8128 Time: 0:01:01 ( 7.55 ms/it)\n objective: 13365.601513470836\nMinimizing 8141 Time: 0:01:01 ( 7.56 ms/it)\n objective: 13365.545644254496\nMinimizing 8154 Time: 0:01:01 ( 7.56 ms/it)\n objective: 13365.535512039569\nMinimizing 8167 Time: 0:01:01 ( 7.56 ms/it)\n objective: 13365.486857557771\nMinimizing 8180 Time: 0:01:01 ( 7.56 ms/it)\n objective: 13365.404535042624\nMinimizing 8193 Time: 0:01:01 ( 7.56 ms/it)\n objective: 13365.309493955465\nMinimizing 8206 Time: 0:01:02 ( 7.56 ms/it)\n objective: 13365.282111891465\nMinimizing 8219 Time: 0:01:02 ( 7.56 ms/it)\n objective: 13365.22045874424\nMinimizing 8232 Time: 0:01:02 ( 7.56 ms/it)\n objective: 13365.202491006356\nMinimizing 8245 Time: 0:01:02 ( 7.56 ms/it)\n objective: 13365.147682533097\nMinimizing 8258 Time: 0:01:02 ( 7.56 ms/it)\n objective: 13365.01881806591\nMinimizing 8271 Time: 0:01:02 ( 7.56 ms/it)\n objective: 13364.91352119024\nMinimizing 8284 Time: 0:01:02 ( 7.56 ms/it)\n objective: 13364.909899063488\nMinimizing 8297 Time: 0:01:02 ( 7.56 ms/it)\n objective: 13364.84132873697\nMinimizing 8310 Time: 0:01:02 ( 7.56 ms/it)\n objective: 13364.798171947936\nMinimizing 8323 Time: 0:01:02 ( 7.56 ms/it)\n objective: 13364.78950660913\nMinimizing 8336 Time: 0:01:03 ( 7.56 ms/it)\n objective: 13364.737842341732\nMinimizing 8349 Time: 0:01:03 ( 7.56 ms/it)\n objective: 13364.655399637355\nMinimizing 8362 Time: 0:01:03 ( 7.56 ms/it)\n objective: 13364.584928252123\nMinimizing 8375 Time: 0:01:03 ( 7.56 ms/it)\n objective: 13364.454427006647\nMinimizing 8388 Time: 0:01:03 ( 7.57 ms/it)\n objective: 13364.449407192842\nMinimizing 8401 Time: 0:01:03 ( 7.57 ms/it)\n objective: 13364.43269589034\nMinimizing 8414 Time: 0:01:03 ( 7.57 ms/it)\n objective: 13364.387105424234\nMinimizing 8427 Time: 0:01:03 ( 7.57 ms/it)\n objective: 13364.336911130813\nMinimizing 8440 Time: 0:01:03 ( 7.57 ms/it)\n objective: 13364.31954021076\nMinimizing 8453 Time: 0:01:03 ( 7.57 ms/it)\n objective: 13364.174568437382\nMinimizing 8466 Time: 0:01:04 ( 7.57 ms/it)\n objective: 13363.990813515316\nMinimizing 8479 Time: 0:01:04 ( 7.57 ms/it)\n objective: 13363.880959640148\nMinimizing 8492 Time: 0:01:04 ( 7.57 ms/it)\n objective: 13363.872811041736\nMinimizing 8505 Time: 0:01:04 ( 7.57 ms/it)\n objective: 13363.830706508677\nMinimizing 8518 Time: 0:01:04 ( 7.57 ms/it)\n objective: 13363.791285763582\nMinimizing 8531 Time: 0:01:04 ( 7.57 ms/it)\n objective: 13363.780501061185\nMinimizing 8544 Time: 0:01:04 ( 7.57 ms/it)\n objective: 13363.698922377065\nMinimizing 8557 Time: 0:01:04 ( 7.57 ms/it)\n objective: 13363.573010990069\nMinimizing 8570 Time: 0:01:04 ( 7.57 ms/it)\n objective: 13363.559485774109\nMinimizing 8583 Time: 0:01:04 ( 7.57 ms/it)\n objective: 13363.519351053219\nMinimizing 8596 Time: 0:01:05 ( 7.57 ms/it)\n objective: 13363.469903991027\nMinimizing 8609 Time: 0:01:05 ( 7.57 ms/it)\n objective: 13363.146869581506\nMinimizing 8622 Time: 0:01:05 ( 7.57 ms/it)\n objective: 13362.974004896634\nMinimizing 8635 Time: 0:01:05 ( 7.57 ms/it)\n objective: 13362.966726134808\nMinimizing 8648 Time: 0:01:05 ( 7.57 ms/it)\n objective: 13362.876816482087\nMinimizing 8661 Time: 0:01:05 ( 7.58 ms/it)\n objective: 13362.840602212113\nMinimizing 8674 Time: 0:01:05 ( 7.58 ms/it)\n objective: 13362.808330337837\nMinimizing 8687 Time: 0:01:05 ( 7.58 ms/it)\n objective: 13362.713331427221\nMinimizing 8700 Time: 0:01:05 ( 7.58 ms/it)\n objective: 13362.52541540561\nMinimizing 8713 Time: 0:01:06 ( 7.58 ms/it)\n objective: 13362.50936264579\nMinimizing 8727 Time: 0:01:06 ( 7.58 ms/it)\n objective: 13362.481698098643\nMinimizing 8741 Time: 0:01:06 ( 7.58 ms/it)\n objective: 13362.41400605131\nMinimizing 8755 Time: 0:01:06 ( 7.58 ms/it)\n objective: 13362.333785017021\nMinimizing 8769 Time: 0:01:06 ( 7.58 ms/it)\n objective: 13362.275017465407\nMinimizing 8782 Time: 0:01:06 ( 7.58 ms/it)\n objective: 13362.192109590025\nMinimizing 8796 Time: 0:01:06 ( 7.58 ms/it)\n objective: 13361.886057994947\nMinimizing 8810 Time: 0:01:06 ( 7.58 ms/it)\n objective: 13361.8578144684\nMinimizing 8824 Time: 0:01:06 ( 7.58 ms/it)\n objective: 13361.837550150405\nMinimizing 8838 Time: 0:01:06 ( 7.58 ms/it)\n objective: 13361.801359127734\nMinimizing 8852 Time: 0:01:07 ( 7.58 ms/it)\n objective: 13361.722740446581\nMinimizing 8866 Time: 0:01:07 ( 7.58 ms/it)\n objective: 13361.805018484301\nMinimizing 8880 Time: 0:01:07 ( 7.58 ms/it)\n objective: 13361.610564986404\nMinimizing 8894 Time: 0:01:07 ( 7.58 ms/it)\n objective: 13361.481391794805\nMinimizing 8907 Time: 0:01:07 ( 7.58 ms/it)\n objective: 13361.187647229715\nMinimizing 8921 Time: 0:01:07 ( 7.58 ms/it)\n objective: 13361.118564894452\nMinimizing 8935 Time: 0:01:07 ( 7.58 ms/it)\n objective: 13361.095143978411\nMinimizing 8949 Time: 0:01:07 ( 7.58 ms/it)\n objective: 13361.060010275804\nMinimizing 8963 Time: 0:01:07 ( 7.58 ms/it)\n objective: 13361.028610137604\nMinimizing 8976 Time: 0:01:08 ( 7.58 ms/it)\n objective: 13361.016030356215\nMinimizing 8990 Time: 0:01:08 ( 7.58 ms/it)\n objective: 13360.87569488967\nMinimizing 9004 Time: 0:01:08 ( 7.58 ms/it)\n objective: 13360.86403823574\nMinimizing 9018 Time: 0:01:08 ( 7.58 ms/it)\n objective: 13360.805560081775\nMinimizing 9032 Time: 0:01:08 ( 7.58 ms/it)\n objective: 13360.797934487855\nMinimizing 9045 Time: 0:01:08 ( 7.58 ms/it)\n objective: 13360.777648840398\nMinimizing 9059 Time: 0:01:08 ( 7.58 ms/it)\n objective: 13360.719084503071\nMinimizing 9073 Time: 0:01:08 ( 7.58 ms/it)\n objective: 13360.64059255163\nMinimizing 9087 Time: 0:01:08 ( 7.58 ms/it)\n objective: 13360.606309391704\nMinimizing 9100 Time: 0:01:08 ( 7.58 ms/it)\n objective: 13360.548142264728\nMinimizing 9114 Time: 0:01:09 ( 7.58 ms/it)\n objective: 13360.528536697653\nMinimizing 9128 Time: 0:01:09 ( 7.58 ms/it)\n objective: 13360.352454508524\nMinimizing 9142 Time: 0:01:09 ( 7.58 ms/it)\n objective: 13360.243168861205\nMinimizing 9156 Time: 0:01:09 ( 7.58 ms/it)\n objective: 13360.214255512903\nMinimizing 9169 Time: 0:01:09 ( 7.58 ms/it)\n objective: 13360.04095263296\nMinimizing 9182 Time: 0:01:09 ( 7.58 ms/it)\n objective: 13359.878291541914\nMinimizing 9196 Time: 0:01:09 ( 7.58 ms/it)\n objective: 13359.864816122397\nMinimizing 9210 Time: 0:01:09 ( 7.58 ms/it)\n objective: 13359.761715725013\nMinimizing 9224 Time: 0:01:09 ( 7.58 ms/it)\n objective: 13359.654031072569\nMinimizing 9238 Time: 0:01:10 ( 7.58 ms/it)\n objective: 13359.625091722635\nMinimizing 9252 Time: 0:01:10 ( 7.58 ms/it)\n objective: 13359.345031662822\nMinimizing 9266 Time: 0:01:10 ( 7.58 ms/it)\n objective: 13359.332598017907\nMinimizing 9280 Time: 0:01:10 ( 7.58 ms/it)\n objective: 13359.21155468587\nMinimizing 9293 Time: 0:01:10 ( 7.58 ms/it)\n objective: 13359.196035343644\nMinimizing 9306 Time: 0:01:10 ( 7.58 ms/it)\n objective: 13358.865382311938\nMinimizing 9320 Time: 0:01:10 ( 7.58 ms/it)\n objective: 13358.705446093038\nMinimizing 9334 Time: 0:01:10 ( 7.58 ms/it)\n objective: 13358.687842992964\nMinimizing 9348 Time: 0:01:10 ( 7.58 ms/it)\n objective: 13358.64371003148\nMinimizing 9362 Time: 0:01:10 ( 7.58 ms/it)\n objective: 13358.573556317038\nMinimizing 9376 Time: 0:01:11 ( 7.58 ms/it)\n objective: 13358.553746106649\nMinimizing 9390 Time: 0:01:11 ( 7.58 ms/it)\n objective: 13358.633592758219\nMinimizing 9404 Time: 0:01:11 ( 7.58 ms/it)\n objective: 13358.510627141965\nMinimizing 9418 Time: 0:01:11 ( 7.58 ms/it)\n objective: 13358.470516051631\nMinimizing 9431 Time: 0:01:11 ( 7.58 ms/it)\n objective: 13358.406529329048\nMinimizing 9445 Time: 0:01:11 ( 7.58 ms/it)\n objective: 13358.200210589042\nMinimizing 9459 Time: 0:01:11 ( 7.58 ms/it)\n objective: 13358.18182786044\nMinimizing 9473 Time: 0:01:11 ( 7.58 ms/it)\n objective: 13358.1702544445\nMinimizing 9487 Time: 0:01:11 ( 7.58 ms/it)\n objective: 13358.168786482529\nMinimizing 9501 Time: 0:01:12 ( 7.58 ms/it)\n objective: 13358.14951156122\nMinimizing 9515 Time: 0:01:12 ( 7.59 ms/it)\n objective: 13358.139993503879\nMinimizing 9529 Time: 0:01:12 ( 7.59 ms/it)\n objective: 13358.03832674495\nMinimizing 9543 Time: 0:01:12 ( 7.59 ms/it)\n objective: 13358.012150576302\nMinimizing 9556 Time: 0:01:12 ( 7.59 ms/it)\n objective: 13358.004605254464\nMinimizing 9569 Time: 0:01:12 ( 7.59 ms/it)\n objective: 13357.972735239047\nMinimizing 9583 Time: 0:01:12 ( 7.59 ms/it)\n objective: 13357.964050785093\nMinimizing 9597 Time: 0:01:12 ( 7.59 ms/it)\n objective: 13357.935744345072\nMinimizing 9611 Time: 0:01:12 ( 7.59 ms/it)\n objective: 13357.888114224654\nMinimizing 9625 Time: 0:01:13 ( 7.59 ms/it)\n objective: 13357.850452371327\nMinimizing 9639 Time: 0:01:13 ( 7.59 ms/it)\n objective: 13357.7471777795\nMinimizing 9653 Time: 0:01:13 ( 7.59 ms/it)\n objective: 13357.743734510303\nMinimizing 9667 Time: 0:01:13 ( 7.59 ms/it)\n objective: 13357.671639362277\nMinimizing 9681 Time: 0:01:13 ( 7.59 ms/it)\n objective: 13357.643879831463\nMinimizing 9694 Time: 0:01:13 ( 7.59 ms/it)\n objective: 13357.58037450962\nMinimizing 9707 Time: 0:01:13 ( 7.59 ms/it)\n objective: 13357.569773701442\nMinimizing 9721 Time: 0:01:13 ( 7.59 ms/it)\n objective: 13357.48113185275\nMinimizing 9735 Time: 0:01:13 ( 7.59 ms/it)\n objective: 13357.457171079514\nMinimizing 9748 Time: 0:01:13 ( 7.59 ms/it)\n objective: 13357.38336146011\nMinimizing 9762 Time: 0:01:14 ( 7.59 ms/it)\n objective: 13357.369791779747\nMinimizing 9776 Time: 0:01:14 ( 7.59 ms/it)\n objective: 13357.289511126604\nMinimizing 9790 Time: 0:01:14 ( 7.59 ms/it)\n objective: 13357.273580989931\nMinimizing 9804 Time: 0:01:14 ( 7.59 ms/it)\n objective: 13357.108316384503\nMinimizing 9818 Time: 0:01:14 ( 7.59 ms/it)\n objective: 13356.840164088062\nMinimizing 9831 Time: 0:01:14 ( 7.59 ms/it)\n objective: 13356.831514002537\nMinimizing 9845 Time: 0:01:14 ( 7.59 ms/it)\n objective: 13356.795490035292\nMinimizing 9859 Time: 0:01:14 ( 7.59 ms/it)\n objective: 13356.785360096372\nMinimizing 9873 Time: 0:01:14 ( 7.59 ms/it)\n objective: 13356.770529168338\nMinimizing 9886 Time: 0:01:15 ( 7.59 ms/it)\n objective: 13356.732596082504\nMinimizing 9900 Time: 0:01:15 ( 7.59 ms/it)\n objective: 13356.698430170953\nMinimizing 9914 Time: 0:01:15 ( 7.59 ms/it)\n objective: 13356.675562484845\nMinimizing 9928 Time: 0:01:15 ( 7.59 ms/it)\n objective: 13356.613038422038\nMinimizing 9941 Time: 0:01:15 ( 7.59 ms/it)\n objective: 13356.508367548384\nMinimizing 9954 Time: 0:01:15 ( 7.59 ms/it)\n objective: 13356.501531919435\nMinimizing 9968 Time: 0:01:15 ( 7.59 ms/it)\n objective: 13356.470603181515\nMinimizing 9981 Time: 0:01:15 ( 7.59 ms/it)\n objective: 13356.457082423112\nMinimizing 9995 Time: 0:01:15 ( 7.59 ms/it)\n objective: 13356.426862269138\nMinimizing 10008 Time: 0:01:15 ( 7.59 ms/it)\n objective: 13356.378150232973\nMinimizing 10022 Time: 0:01:16 ( 7.59 ms/it)\n objective: 13356.373316467514\nMinimizing 10036 Time: 0:01:16 ( 7.59 ms/it)\n objective: 13356.327605296028\nMinimizing 10050 Time: 0:01:16 ( 7.59 ms/it)\n objective: 13356.317276413269\nMinimizing 10064 Time: 0:01:16 ( 7.59 ms/it)\n objective: 13356.263535978207\nMinimizing 10078 Time: 0:01:16 ( 7.59 ms/it)\n objective: 13356.251746193746\nMinimizing 10091 Time: 0:01:16 ( 7.59 ms/it)\n objective: 13356.18602528506\nMinimizing 10105 Time: 0:01:16 ( 7.59 ms/it)\n objective: 13356.071692888007\nMinimizing 10119 Time: 0:01:16 ( 7.59 ms/it)\n objective: 13356.058297816235\nMinimizing 10133 Time: 0:01:16 ( 7.59 ms/it)\n objective: 13355.980068275981\nMinimizing 10147 Time: 0:01:17 ( 7.59 ms/it)\n objective: 13355.840684352894\nMinimizing 10160 Time: 0:01:17 ( 7.59 ms/it)\n objective: 13355.825145810355\nMinimizing 10173 Time: 0:01:17 ( 7.59 ms/it)\n objective: 13355.806880024276\nMinimizing 10185 Time: 0:01:17 ( 7.59 ms/it)\n objective: 13355.7900044595\nMinimizing 10198 Time: 0:01:17 ( 7.59 ms/it)\n objective: 13355.770173163284\nMinimizing 10211 Time: 0:01:17 ( 7.59 ms/it)\n objective: 13355.731432885666\nMinimizing 10225 Time: 0:01:17 ( 7.60 ms/it)\n objective: 13355.684629451498\nMinimizing 10239 Time: 0:01:17 ( 7.60 ms/it)\n objective: 13355.639947712865\nMinimizing 10253 Time: 0:01:17 ( 7.60 ms/it)\n objective: 13355.614689875074\nMinimizing 10267 Time: 0:01:17 ( 7.60 ms/it)\n objective: 13355.559434175084\nMinimizing 10281 Time: 0:01:18 ( 7.60 ms/it)\n objective: 13355.506833567335\nMinimizing 10295 Time: 0:01:18 ( 7.60 ms/it)\n objective: 13355.493535417307\nMinimizing 10309 Time: 0:01:18 ( 7.60 ms/it)\n objective: 13355.384513433572\nMinimizing 10323 Time: 0:01:18 ( 7.60 ms/it)\n objective: 13355.224537038004\nMinimizing 10335 Time: 0:01:18 ( 7.60 ms/it)\n objective: 13355.15117978509\nMinimizing 10348 Time: 0:01:18 ( 7.60 ms/it)\n objective: 13355.119377250514\nMinimizing 10362 Time: 0:01:18 ( 7.60 ms/it)\n objective: 13355.082610773745\nMinimizing 10375 Time: 0:01:18 ( 7.60 ms/it)\n objective: 13355.034399186057\nMinimizing 10388 Time: 0:01:18 ( 7.60 ms/it)\n objective: 13355.019166930098\nMinimizing 10401 Time: 0:01:19 ( 7.60 ms/it)\n objective: 13354.954053528636\nMinimizing 10415 Time: 0:01:19 ( 7.60 ms/it)\n objective: 13354.838118348758\nMinimizing 10428 Time: 0:01:19 ( 7.60 ms/it)\n objective: 13354.829268505811\nMinimizing 10440 Time: 0:01:19 ( 7.60 ms/it)\n objective: 13354.782246079063\nMinimizing 10453 Time: 0:01:19 ( 7.60 ms/it)\n objective: 13354.692261538716\nMinimizing 10465 Time: 0:01:19 ( 7.60 ms/it)\n objective: 13354.678863413472\nMinimizing 10477 Time: 0:01:19 ( 7.60 ms/it)\n objective: 13354.622113764111\nMinimizing 10490 Time: 0:01:19 ( 7.60 ms/it)\n objective: 13354.617954296547\nMinimizing 10503 Time: 0:01:19 ( 7.60 ms/it)\n objective: 13354.568550540847\nMinimizing 10516 Time: 0:01:19 ( 7.61 ms/it)\n objective: 13354.52995235283\nMinimizing 10529 Time: 0:01:20 ( 7.61 ms/it)\n objective: 13354.48342526373\nMinimizing 10542 Time: 0:01:20 ( 7.61 ms/it)\n objective: 13354.472504086138\nMinimizing 10555 Time: 0:01:20 ( 7.61 ms/it)\n objective: 13354.431711489764\nMinimizing 10568 Time: 0:01:20 ( 7.61 ms/it)\n objective: 13354.425110274613\nMinimizing 10581 Time: 0:01:20 ( 7.61 ms/it)\n objective: 13354.415004948743\nMinimizing 10593 Time: 0:01:20 ( 7.61 ms/it)\n objective: 13354.376122501155\nMinimizing 10606 Time: 0:01:20 ( 7.61 ms/it)\n objective: 13354.258328140495\nMinimizing 10618 Time: 0:01:20 ( 7.61 ms/it)\n objective: 13354.24409545277\nMinimizing 10631 Time: 0:01:20 ( 7.61 ms/it)\n objective: 13354.213722589324\nMinimizing 10644 Time: 0:01:21 ( 7.61 ms/it)\n objective: 13354.19897918767\nMinimizing 10657 Time: 0:01:21 ( 7.61 ms/it)\n objective: 13354.17219520708\nMinimizing 10670 Time: 0:01:21 ( 7.61 ms/it)\n objective: 13354.136471808779\nMinimizing 10683 Time: 0:01:21 ( 7.62 ms/it)\n objective: 13354.096014953531\nMinimizing 10696 Time: 0:01:21 ( 7.62 ms/it)\n objective: 13354.03708133263\nMinimizing 10708 Time: 0:01:21 ( 7.62 ms/it)\n objective: 13354.03386071061\nMinimizing 10720 Time: 0:01:21 ( 7.62 ms/it)\n objective: 13353.989327666772\nMinimizing 10733 Time: 0:01:21 ( 7.62 ms/it)\n objective: 13353.959281073992\nMinimizing 10746 Time: 0:01:21 ( 7.62 ms/it)\n objective: 13353.831018776487\nMinimizing 10759 Time: 0:01:21 ( 7.62 ms/it)\n objective: 13353.726796034593\nMinimizing 10772 Time: 0:01:22 ( 7.62 ms/it)\n objective: 13353.718224597455\nMinimizing 10785 Time: 0:01:22 ( 7.62 ms/it)\n objective: 13353.676532591191\nMinimizing 10798 Time: 0:01:22 ( 7.62 ms/it)\n objective: 13353.594094531705\nMinimizing 10811 Time: 0:01:22 ( 7.62 ms/it)\n objective: 13353.458199080647\nMinimizing 10824 Time: 0:01:22 ( 7.62 ms/it)\n objective: 13353.112552658386\nMinimizing 10836 Time: 0:01:22 ( 7.62 ms/it)\n objective: 13353.060939681724\nMinimizing 10849 Time: 0:01:22 ( 7.63 ms/it)\n objective: 13352.891486021508\nMinimizing 10863 Time: 0:01:22 ( 7.63 ms/it)\n objective: 13352.846662323944\nMinimizing 10877 Time: 0:01:22 ( 7.63 ms/it)\n objective: 13352.799414215078\nMinimizing 10891 Time: 0:01:23 ( 7.63 ms/it)\n objective: 13352.728226970918\nMinimizing 10905 Time: 0:01:23 ( 7.63 ms/it)\n objective: 13352.709538284384\nMinimizing 10919 Time: 0:01:23 ( 7.63 ms/it)\n objective: 13352.591482143092\nMinimizing 10933 Time: 0:01:23 ( 7.63 ms/it)\n objective: 13352.587567225084\nMinimizing 10947 Time: 0:01:23 ( 7.63 ms/it)\n objective: 13352.559554775478\nMinimizing 10960 Time: 0:01:23 ( 7.63 ms/it)\n objective: 13352.50164528464\nMinimizing 10974 Time: 0:01:23 ( 7.63 ms/it)\n objective: 13352.496030790957\nMinimizing 10988 Time: 0:01:23 ( 7.63 ms/it)\n objective: 13352.433264243053\nMinimizing 11001 Time: 0:01:23 ( 7.63 ms/it)\n objective: 13352.392610200703\nMinimizing 11014 Time: 0:01:23 ( 7.63 ms/it)\n objective: 13352.386814995974\nMinimizing 11028 Time: 0:01:24 ( 7.63 ms/it)\n objective: 13352.369685569334\nMinimizing 11042 Time: 0:01:24 ( 7.63 ms/it)\n objective: 13352.328664215842\nMinimizing 11056 Time: 0:01:24 ( 7.63 ms/it)\n objective: 13352.294814843583\nMinimizing 11070 Time: 0:01:24 ( 7.63 ms/it)\n objective: 13352.254762036617\nMinimizing 11084 Time: 0:01:24 ( 7.63 ms/it)\n objective: 13352.209804358186\nMinimizing 11097 Time: 0:01:24 ( 7.63 ms/it)\n objective: 13352.201201765914\nMinimizing 11111 Time: 0:01:24 ( 7.63 ms/it)\n objective: 13352.187951210006\nMinimizing 11124 Time: 0:01:24 ( 7.63 ms/it)\n objective: 13352.12432845711\nMinimizing 11138 Time: 0:01:24 ( 7.63 ms/it)\n objective: 13352.113502813649\nMinimizing 11152 Time: 0:01:25 ( 7.63 ms/it)\n objective: 13352.044690670089\nMinimizing 11165 Time: 0:01:25 ( 7.63 ms/it)\n objective: 13351.995198748715\nMinimizing 11178 Time: 0:01:25 ( 7.63 ms/it)\n objective: 13351.850530314987\nMinimizing 11192 Time: 0:01:25 ( 7.63 ms/it)\n objective: 13351.781319274436\nMinimizing 11206 Time: 0:01:25 ( 7.63 ms/it)\n objective: 13351.505125089992\nMinimizing 11219 Time: 0:01:25 ( 7.63 ms/it)\n objective: 13351.461666813433\nMinimizing 11233 Time: 0:01:25 ( 7.63 ms/it)\n objective: 13351.375305019152\nMinimizing 11247 Time: 0:01:25 ( 7.63 ms/it)\n objective: 13351.268682597401\nMinimizing 11261 Time: 0:01:25 ( 7.63 ms/it)\n objective: 13351.205823858822\nMinimizing 11275 Time: 0:01:25 ( 7.63 ms/it)\n objective: 13351.158611692532\nMinimizing 11288 Time: 0:01:26 ( 7.63 ms/it)\n objective: 13350.962384291226\nMinimizing 11302 Time: 0:01:26 ( 7.63 ms/it)\n objective: 13350.932136916672\nMinimizing 11316 Time: 0:01:26 ( 7.63 ms/it)\n objective: 13350.944612548308\nMinimizing 11330 Time: 0:01:26 ( 7.63 ms/it)\n objective: 13350.482541061938\nMinimizing 11344 Time: 0:01:26 ( 7.63 ms/it)\n objective: 13350.479360717298\nMinimizing 11357 Time: 0:01:26 ( 7.63 ms/it)\n objective: 13350.434829527068\nMinimizing 11371 Time: 0:01:26 ( 7.63 ms/it)\n objective: 13350.423136170532\nMinimizing 11385 Time: 0:01:26 ( 7.63 ms/it)\n objective: 13350.323159644351\nMinimizing 11399 Time: 0:01:26 ( 7.63 ms/it)\n objective: 13350.294377892336\nMinimizing 11413 Time: 0:01:27 ( 7.63 ms/it)\n objective: 13350.259972299973\nMinimizing 11426 Time: 0:01:27 ( 7.63 ms/it)\n objective: 13350.188672974036\nMinimizing 11440 Time: 0:01:27 ( 7.63 ms/it)\n objective: 13350.00813112098\nMinimizing 11454 Time: 0:01:27 ( 7.63 ms/it)\n objective: 13349.668297528224\nMinimizing 11468 Time: 0:01:27 ( 7.63 ms/it)\n objective: 13349.4084963766\nMinimizing 11482 Time: 0:01:27 ( 7.63 ms/it)\n objective: 13349.397833178671\nMinimizing 11495 Time: 0:01:27 ( 7.63 ms/it)\n objective: 13349.391424331967\nMinimizing 11509 Time: 0:01:27 ( 7.63 ms/it)\n objective: 13349.388827066301\nMinimizing 11523 Time: 0:01:27 ( 7.63 ms/it)\n objective: 13349.32275659265\nMinimizing 11537 Time: 0:01:28 ( 7.63 ms/it)\n objective: 13349.307151949433\nMinimizing 11551 Time: 0:01:28 ( 7.63 ms/it)\n objective: 13349.287918451038\nMinimizing 11564 Time: 0:01:28 ( 7.63 ms/it)\n objective: 13349.253492136515\nMinimizing 11577 Time: 0:01:28 ( 7.63 ms/it)\n objective: 13349.231129859218\nMinimizing 11590 Time: 0:01:28 ( 7.63 ms/it)\n objective: 13349.167494999376\nMinimizing 11603 Time: 0:01:28 ( 7.63 ms/it)\n objective: 13349.150583653682\nMinimizing 11616 Time: 0:01:28 ( 7.63 ms/it)\n objective: 13349.130223942819\nMinimizing 11630 Time: 0:01:28 ( 7.63 ms/it)\n objective: 13349.04290873139\nMinimizing 11644 Time: 0:01:28 ( 7.63 ms/it)\n objective: 13349.836668721175\nMinimizing 11657 Time: 0:01:28 ( 7.63 ms/it)\n objective: 13348.74733190942\nMinimizing 11670 Time: 0:01:29 ( 7.63 ms/it)\n objective: 13348.741253256907\nMinimizing 11684 Time: 0:01:29 ( 7.63 ms/it)\n objective: 13348.713494788324\nMinimizing 11698 Time: 0:01:29 ( 7.63 ms/it)\n objective: 13348.637519376694\nMinimizing 11712 Time: 0:01:29 ( 7.63 ms/it)\n objective: 13348.630322087061\nMinimizing 11726 Time: 0:01:29 ( 7.63 ms/it)\n objective: 13348.610383664243\nMinimizing 11739 Time: 0:01:29 ( 7.63 ms/it)\n objective: 13348.639888595266\nMinimizing 11753 Time: 0:01:29 ( 7.63 ms/it)\n objective: 13348.533745111956\nMinimizing 11766 Time: 0:01:29 ( 7.63 ms/it)\n objective: 13348.527297904584\nMinimizing 11780 Time: 0:01:29 ( 7.63 ms/it)\n objective: 13348.49504227205\nMinimizing 11794 Time: 0:01:30 ( 7.63 ms/it)\n objective: 13348.481850959346\nMinimizing 11807 Time: 0:01:30 ( 7.63 ms/it)\n objective: 13348.448432735393\nMinimizing 11821 Time: 0:01:30 ( 7.63 ms/it)\n objective: 13348.338265718514\nMinimizing 11835 Time: 0:01:30 ( 7.63 ms/it)\n objective: 13348.222146126012\nMinimizing 11848 Time: 0:01:30 ( 7.63 ms/it)\n objective: 13348.21849025182\nMinimizing 11862 Time: 0:01:30 ( 7.63 ms/it)\n objective: 13348.20912309162\nMinimizing 11875 Time: 0:01:30 ( 7.63 ms/it)\n objective: 13348.187486413255\nMinimizing 11889 Time: 0:01:30 ( 7.63 ms/it)\n objective: 13348.153741997252\nMinimizing 11903 Time: 0:01:30 ( 7.63 ms/it)\n objective: 13348.126700332003\nMinimizing 11917 Time: 0:01:30 ( 7.63 ms/it)\n objective: 13348.115368754363\nMinimizing 11931 Time: 0:01:31 ( 7.63 ms/it)\n objective: 13348.082805935053\nMinimizing 11945 Time: 0:01:31 ( 7.63 ms/it)\n objective: 13347.97463443457\nMinimizing 11959 Time: 0:01:31 ( 7.63 ms/it)\n objective: 13347.89535710009\nMinimizing 11973 Time: 0:01:31 ( 7.63 ms/it)\n objective: 13347.884248236129\nMinimizing 11987 Time: 0:01:31 ( 7.63 ms/it)\n objective: 13347.864381234576\nMinimizing 12000 Time: 0:01:31 ( 7.63 ms/it)\n objective: 13347.841470582796\nMinimizing 12014 Time: 0:01:31 ( 7.63 ms/it)\n objective: 13347.780131875239\nMinimizing 12028 Time: 0:01:31 ( 7.63 ms/it)\n objective: 13347.594960365634\nMinimizing 12042 Time: 0:01:31 ( 7.63 ms/it)\n objective: 13347.574958612902\nMinimizing 12056 Time: 0:01:32 ( 7.63 ms/it)\n objective: 13347.567593316824\nMinimizing 12070 Time: 0:01:32 ( 7.63 ms/it)\n objective: 13347.55401920464\nMinimizing 12084 Time: 0:01:32 ( 7.63 ms/it)\n objective: 13347.493032846134\nMinimizing 12098 Time: 0:01:32 ( 7.63 ms/it)\n objective: 13347.48857371571\nMinimizing 12112 Time: 0:01:32 ( 7.63 ms/it)\n objective: 13347.47405385167\nMinimizing 12125 Time: 0:01:32 ( 7.63 ms/it)\n objective: 13347.41180824591\nMinimizing 12138 Time: 0:01:32 ( 7.64 ms/it)\n objective: 13347.401719889378\nMinimizing 12152 Time: 0:01:32 ( 7.64 ms/it)\n objective: 13347.369559940118\nMinimizing 12166 Time: 0:01:32 ( 7.64 ms/it)\n objective: 13347.215503778454\nMinimizing 12180 Time: 0:01:32 ( 7.64 ms/it)\n objective: 13347.202552838964\nMinimizing 12194 Time: 0:01:33 ( 7.64 ms/it)\n objective: 13347.187421355935\nMinimizing 12207 Time: 0:01:33 ( 7.64 ms/it)\n objective: 13347.184415022144\nMinimizing 12221 Time: 0:01:33 ( 7.64 ms/it)\n objective: 13347.1580186075\nMinimizing 12235 Time: 0:01:33 ( 7.64 ms/it)\n objective: 13347.136503558628\nMinimizing 12249 Time: 0:01:33 ( 7.64 ms/it)\n objective: 13347.105996971026\nMinimizing 12262 Time: 0:01:33 ( 7.64 ms/it)\n objective: 13347.096451258105\nMinimizing 12276 Time: 0:01:33 ( 7.64 ms/it)\n objective: 13347.090186884801\nMinimizing 12290 Time: 0:01:33 ( 7.64 ms/it)\n objective: 13347.025860355083\nMinimizing 12304 Time: 0:01:33 ( 7.64 ms/it)\n objective: 13346.985643587235\nMinimizing 12318 Time: 0:01:34 ( 7.64 ms/it)\n objective: 13346.962589302864\nMinimizing 12331 Time: 0:01:34 ( 7.64 ms/it)\n objective: 13346.938159191901\nMinimizing 12344 Time: 0:01:34 ( 7.64 ms/it)\n objective: 13346.927731188873\nMinimizing 12358 Time: 0:01:34 ( 7.64 ms/it)\n objective: 13346.866783892154\nMinimizing 12372 Time: 0:01:34 ( 7.64 ms/it)\n objective: 13346.85582349066\nMinimizing 12386 Time: 0:01:34 ( 7.64 ms/it)\n objective: 13346.82110853323\nMinimizing 12399 Time: 0:01:34 ( 7.64 ms/it)\n objective: 13346.764748544527\nMinimizing 12412 Time: 0:01:34 ( 7.64 ms/it)\n objective: 13346.654866133002\nMinimizing 12423 Time: 0:01:34 ( 7.64 ms/it)\n objective: 13346.48642662655\nMinimizing 12436 Time: 0:01:35 ( 7.64 ms/it)\n objective: 13346.748483082294\nMinimizing 12450 Time: 0:01:35 ( 7.64 ms/it)\n objective: 13345.628570482884\nMinimizing 12465 Time: 0:01:35 ( 7.64 ms/it)\n objective: 13345.622866117956\nMinimizing 12480 Time: 0:01:35 ( 7.64 ms/it)\n objective: 13345.622864899582\nMinimizing 12495 Time: 0:01:35 ( 7.64 ms/it)\n objective: 13345.613333107416\nMinimizing 12510 Time: 0:01:35 ( 7.64 ms/it)\n objective: 13345.6076133643\nMinimizing 12525 Time: 0:01:35 ( 7.64 ms/it)\n objective: 13345.840923904347\nMinimizing 12539 Time: 0:01:35 ( 7.64 ms/it)\n objective: 13345.546683230008\nMinimizing 12554 Time: 0:01:35 ( 7.64 ms/it)\n objective: 13345.520414555409\nMinimizing 12568 Time: 0:01:35 ( 7.64 ms/it)\n objective: 13345.482579864685\nMinimizing 12582 Time: 0:01:36 ( 7.64 ms/it)\n objective: 13345.453370938521\nMinimizing 12596 Time: 0:01:36 ( 7.64 ms/it)\n objective: 13345.414262193037\nMinimizing 12610 Time: 0:01:36 ( 7.64 ms/it)\n objective: 13345.40551057463\nMinimizing 12624 Time: 0:01:36 ( 7.64 ms/it)\n objective: 13345.330549550505\nMinimizing 12638 Time: 0:01:36 ( 7.64 ms/it)\n objective: 13345.322551443474\nMinimizing 12652 Time: 0:01:36 ( 7.63 ms/it)\n objective: 13345.309063311943\nMinimizing 12666 Time: 0:01:36 ( 7.63 ms/it)\n objective: 13345.28478984227\nMinimizing 12680 Time: 0:01:36 ( 7.64 ms/it)\n objective: 13345.271494036868\nMinimizing 12693 Time: 0:01:36 ( 7.64 ms/it)\n objective: 13345.241204354403\nMinimizing 12706 Time: 0:01:37 ( 7.64 ms/it)\n objective: 13345.151447869328\nMinimizing 12719 Time: 0:01:37 ( 7.64 ms/it)\n objective: 13345.148512951804\nMinimizing 12732 Time: 0:01:37 ( 7.64 ms/it)\n objective: 13345.144331022173\nMinimizing 12746 Time: 0:01:37 ( 7.64 ms/it)\n objective: 13345.141916181929\nMinimizing 12760 Time: 0:01:37 ( 7.64 ms/it)\n objective: 13345.128527242618\nMinimizing 12774 Time: 0:01:37 ( 7.64 ms/it)\n objective: 13345.113935825902\nMinimizing 12788 Time: 0:01:37 ( 7.64 ms/it)\n objective: 13345.108617722697\nMinimizing 12802 Time: 0:01:37 ( 7.64 ms/it)\n objective: 13345.095125132742\nMinimizing 12816 Time: 0:01:37 ( 7.64 ms/it)\n objective: 13345.082263493765\nMinimizing 12830 Time: 0:01:37 ( 7.63 ms/it)\n objective: 13345.075697140193\nMinimizing 12844 Time: 0:01:38 ( 7.63 ms/it)\n objective: 13345.039722544592\nMinimizing 12858 Time: 0:01:38 ( 7.63 ms/it)\n objective: 13345.037761111911\nMinimizing 12872 Time: 0:01:38 ( 7.63 ms/it)\n objective: 13345.1666793997\nMinimizing 12886 Time: 0:01:38 ( 7.63 ms/it)\n objective: 13344.92859330037\nMinimizing 12900 Time: 0:01:38 ( 7.63 ms/it)\n objective: 13344.925149030074\nMinimizing 12914 Time: 0:01:38 ( 7.63 ms/it)\n objective: 13344.89630675107\nMinimizing 12928 Time: 0:01:38 ( 7.63 ms/it)\n objective: 13344.892204139382\nMinimizing 12942 Time: 0:01:38 ( 7.63 ms/it)\n objective: 13344.886592941577\nMinimizing 12956 Time: 0:01:38 ( 7.63 ms/it)\n objective: 13344.872839413656\nMinimizing 12970 Time: 0:01:38 ( 7.63 ms/it)\n objective: 13344.874315630397\nMinimizing 12984 Time: 0:01:39 ( 7.63 ms/it)\n objective: 13344.79732289783\nMinimizing 12997 Time: 0:01:39 ( 7.63 ms/it)\n objective: 13344.767966937921\nMinimizing 13010 Time: 0:01:39 ( 7.63 ms/it)\n objective: 13344.763125665617\nMinimizing 13024 Time: 0:01:39 ( 7.63 ms/it)\n objective: 13344.753555207179\nMinimizing 13038 Time: 0:01:39 ( 7.63 ms/it)\n objective: 13344.736951753956\nMinimizing 13052 Time: 0:01:39 ( 7.63 ms/it)\n objective: 13344.724748820467\nMinimizing 13065 Time: 0:01:39 ( 7.63 ms/it)\n objective: 13344.667618019303\nMinimizing 13079 Time: 0:01:39 ( 7.63 ms/it)\n objective: 13344.588659589892\nMinimizing 13093 Time: 0:01:39 ( 7.63 ms/it)\n objective: 13344.576174037065\nMinimizing 13107 Time: 0:01:40 ( 7.63 ms/it)\n objective: 13344.562048646847\nMinimizing 13121 Time: 0:01:40 ( 7.63 ms/it)\n objective: 13344.48850606852\nMinimizing 13135 Time: 0:01:40 ( 7.63 ms/it)\n objective: 13344.479955785238\nMinimizing 13149 Time: 0:01:40 ( 7.63 ms/it)\n objective: 13344.459435019693\nMinimizing 13163 Time: 0:01:40 ( 7.63 ms/it)\n objective: 13344.394226295124\nMinimizing 13177 Time: 0:01:40 ( 7.63 ms/it)\n objective: 13344.38527770467\nMinimizing 13191 Time: 0:01:40 ( 7.63 ms/it)\n objective: 13344.352889354923\nMinimizing 13205 Time: 0:01:40 ( 7.63 ms/it)\n objective: 13344.332406485839\nMinimizing 13219 Time: 0:01:40 ( 7.63 ms/it)\n objective: 13344.307621613509\nMinimizing 13233 Time: 0:01:40 ( 7.63 ms/it)\n objective: 13344.23706030583\nMinimizing 13247 Time: 0:01:41 ( 7.63 ms/it)\n objective: 13344.03560074504\nMinimizing 13261 Time: 0:01:41 ( 7.63 ms/it)\n objective: 13344.00726643725\nMinimizing 13274 Time: 0:01:41 ( 7.63 ms/it)\n objective: 13343.991815181456\nMinimizing 13288 Time: 0:01:41 ( 7.63 ms/it)\n objective: 13343.967246389439\nMinimizing 13302 Time: 0:01:41 ( 7.63 ms/it)\n objective: 13343.947982783458\nMinimizing 13316 Time: 0:01:41 ( 7.63 ms/it)\n objective: 13343.898241744522\nMinimizing 13330 Time: 0:01:41 ( 7.63 ms/it)\n objective: 13343.892410396104\nMinimizing 13343 Time: 0:01:41 ( 7.63 ms/it)\n objective: 13343.857907028098\nMinimizing 13355 Time: 0:01:41 ( 7.63 ms/it)\n objective: 13343.856922619932\nMinimizing 13369 Time: 0:01:42 ( 7.63 ms/it)\n objective: 13343.806901450924\nMinimizing 13383 Time: 0:01:42 ( 7.63 ms/it)\n objective: 13343.742183542854\nMinimizing 13397 Time: 0:01:42 ( 7.63 ms/it)\n objective: 13343.588368684308\nMinimizing 13410 Time: 0:01:42 ( 7.63 ms/it)\n objective: 13343.520113863007\nMinimizing 13423 Time: 0:01:42 ( 7.63 ms/it)\n objective: 13343.515912747447\nMinimizing 13436 Time: 0:01:42 ( 7.63 ms/it)\n objective: 13343.50793322963\nMinimizing 13450 Time: 0:01:42 ( 7.63 ms/it)\n objective: 13343.503540939193\nMinimizing 13465 Time: 0:01:42 ( 7.63 ms/it)\n objective: 13343.496950676708\nMinimizing 13478 Time: 0:01:42 ( 7.63 ms/it)\n objective: 13343.449574027218\nMinimizing 13492 Time: 0:01:42 ( 7.63 ms/it)\n objective: 13343.446848708649\nMinimizing 13506 Time: 0:01:43 ( 7.63 ms/it)\n objective: 13343.43172562078\nMinimizing 13520 Time: 0:01:43 ( 7.63 ms/it)\n objective: 13343.409884976587\nMinimizing 13534 Time: 0:01:43 ( 7.63 ms/it)\n objective: 13343.384692148713\nMinimizing 13548 Time: 0:01:43 ( 7.63 ms/it)\n objective: 13343.340063787291\nMinimizing 13562 Time: 0:01:43 ( 7.63 ms/it)\n objective: 13343.326074487719\nMinimizing 13577 Time: 0:01:43 ( 7.63 ms/it)\n objective: 13344.274033716938\nMinimizing 13592 Time: 0:01:43 ( 7.63 ms/it)\n objective: 13342.755559848862\nMinimizing 13606 Time: 0:01:43 ( 7.63 ms/it)\n objective: 13342.752462506818\nMinimizing 13620 Time: 0:01:43 ( 7.63 ms/it)\n objective: 13342.751353947933\nMinimizing 13634 Time: 0:01:43 ( 7.63 ms/it)\n objective: 13342.750873648554\nMinimizing 13648 Time: 0:01:44 ( 7.63 ms/it)\n objective: 13342.74727714977\nMinimizing 13662 Time: 0:01:44 ( 7.63 ms/it)\n objective: 13342.741142379062\nMinimizing 13676 Time: 0:01:44 ( 7.63 ms/it)\n objective: 13342.69873160727\nMinimizing 13690 Time: 0:01:44 ( 7.63 ms/it)\n objective: 13342.63023211118\nMinimizing 13704 Time: 0:01:44 ( 7.63 ms/it)\n objective: 13342.629421813945\nMinimizing 13718 Time: 0:01:44 ( 7.63 ms/it)\n objective: 13342.623464409451\nMinimizing 13733 Time: 0:01:44 ( 7.62 ms/it)\n objective: 13342.611700424197\nMinimizing 13748 Time: 0:01:44 ( 7.62 ms/it)\n objective: 13342.609370531165\nMinimizing 13763 Time: 0:01:44 ( 7.62 ms/it)\n objective: 13342.603192153394\nMinimizing 13777 Time: 0:01:45 ( 7.62 ms/it)\n objective: 13342.596185176415\nMinimizing 13791 Time: 0:01:45 ( 7.62 ms/it)\n objective: 13342.5587861093\nMinimizing 13805 Time: 0:01:45 ( 7.62 ms/it)\n objective: 13342.558206853093\nMinimizing 13819 Time: 0:01:45 ( 7.62 ms/it)\n objective: 13342.556372652864\nMinimizing 13833 Time: 0:01:45 ( 7.62 ms/it)\n objective: 13342.546611815378\nMinimizing 13847 Time: 0:01:45 ( 7.62 ms/it)\n objective: 13342.532979705124\nMinimizing 13861 Time: 0:01:45 ( 7.62 ms/it)\n objective: 13342.501951278515\nMinimizing 13875 Time: 0:01:45 ( 7.62 ms/it)\n objective: 13342.438715714576\nMinimizing 13889 Time: 0:01:45 ( 7.62 ms/it)\n objective: 13342.395686358097\nMinimizing 13903 Time: 0:01:45 ( 7.62 ms/it)\n objective: 13342.384038808741\nMinimizing 13917 Time: 0:01:46 ( 7.62 ms/it)\n objective: 13342.368030303965\nMinimizing 13931 Time: 0:01:46 ( 7.62 ms/it)\n objective: 13342.34443888953\nMinimizing 13945 Time: 0:01:46 ( 7.62 ms/it)\n objective: 13342.34313225605\nMinimizing 13959 Time: 0:01:46 ( 7.62 ms/it)\n objective: 13342.334988698909\nMinimizing 13974 Time: 0:01:46 ( 7.62 ms/it)\n objective: 13342.311733475282\nMinimizing 13989 Time: 0:01:46 ( 7.62 ms/it)\n objective: 13342.289983820163\nMinimizing 14003 Time: 0:01:46 ( 7.62 ms/it)\n objective: 13342.246444806078\nMinimizing 14017 Time: 0:01:46 ( 7.62 ms/it)\n objective: 13342.244921382386\nMinimizing 14031 Time: 0:01:46 ( 7.62 ms/it)\n objective: 13342.24441355453\nMinimizing 14045 Time: 0:01:47 ( 7.62 ms/it)\n objective: 13342.218828023273\nMinimizing 14059 Time: 0:01:47 ( 7.62 ms/it)\n objective: 13342.17934070385\nMinimizing 14073 Time: 0:01:47 ( 7.62 ms/it)\n objective: 13342.06056912605\nMinimizing 14087 Time: 0:01:47 ( 7.62 ms/it)\n objective: 13342.05049441877\nMinimizing 14101 Time: 0:01:47 ( 7.62 ms/it)\n objective: 13342.026868573339\nMinimizing 14115 Time: 0:01:47 ( 7.62 ms/it)\n objective: 13342.002058981561\nMinimizing 14130 Time: 0:01:47 ( 7.62 ms/it)\n objective: 13342.001102257585\nMinimizing 14144 Time: 0:01:47 ( 7.62 ms/it)\n objective: 13342.009124869204\nMinimizing 14158 Time: 0:01:47 ( 7.62 ms/it)\n objective: 13341.959557608854\nMinimizing 14172 Time: 0:01:47 ( 7.62 ms/it)\n objective: 13341.94567407461\nMinimizing 14185 Time: 0:01:48 ( 7.62 ms/it)\n objective: 13341.928842195688\nMinimizing 14199 Time: 0:01:48 ( 7.62 ms/it)\n objective: 13341.876274474751\nMinimizing 14213 Time: 0:01:48 ( 7.62 ms/it)\n objective: 13341.87338260821\nMinimizing 14227 Time: 0:01:48 ( 7.62 ms/it)\n objective: 13341.859648768921\nMinimizing 14241 Time: 0:01:48 ( 7.62 ms/it)\n objective: 13341.833293218537\nMinimizing 14255 Time: 0:01:48 ( 7.62 ms/it)\n objective: 13341.823698525688\nMinimizing 14268 Time: 0:01:48 ( 7.62 ms/it)\n objective: 13341.779450060836\nMinimizing 14280 Time: 0:01:48 ( 7.62 ms/it)\n objective: 13341.778788315685\nMinimizing 14294 Time: 0:01:48 ( 7.62 ms/it)\n objective: 13341.749770237446\nMinimizing 14308 Time: 0:01:49 ( 7.62 ms/it)\n objective: 13341.728708525225\nMinimizing 14322 Time: 0:01:49 ( 7.62 ms/it)\n objective: 13341.718182395067\nMinimizing 14335 Time: 0:01:49 ( 7.62 ms/it)\n objective: 13341.692137467515\nMinimizing 14348 Time: 0:01:49 ( 7.62 ms/it)\n objective: 13341.679060573493\nMinimizing 14361 Time: 0:01:49 ( 7.62 ms/it)\n objective: 13341.658373906903\nMinimizing 14374 Time: 0:01:49 ( 7.62 ms/it)\n objective: 13341.634779269712\nMinimizing 14387 Time: 0:01:49 ( 7.62 ms/it)\n objective: 13341.605974389611\nMinimizing 14401 Time: 0:01:49 ( 7.62 ms/it)\n objective: 13341.482110165569\nMinimizing 14416 Time: 0:01:49 ( 7.62 ms/it)\n objective: 13341.362884887116\nMinimizing 14431 Time: 0:01:49 ( 7.62 ms/it)\n objective: 13341.361616337701\nMinimizing 14446 Time: 0:01:50 ( 7.62 ms/it)\n objective: 13341.328550239748\nMinimizing 14461 Time: 0:01:50 ( 7.62 ms/it)\n objective: 13341.318476155859\nMinimizing 14476 Time: 0:01:50 ( 7.62 ms/it)\n objective: 13341.28721405701\nMinimizing 14491 Time: 0:01:50 ( 7.62 ms/it)\n objective: 13341.206464117902\nMinimizing 14506 Time: 0:01:50 ( 7.61 ms/it)\n objective: 13341.125584964371\nMinimizing 14521 Time: 0:01:50 ( 7.61 ms/it)\n objective: 13341.124352856175\nMinimizing 14536 Time: 0:01:50 ( 7.61 ms/it)\n objective: 13341.116067249786\nMinimizing 14551 Time: 0:01:50 ( 7.61 ms/it)\n objective: 13341.108300017237\nMinimizing 14566 Time: 0:01:50 ( 7.61 ms/it)\n objective: 13341.100408753897\nMinimizing 14581 Time: 0:01:50 ( 7.61 ms/it)\n objective: 13341.054358721369\nMinimizing 14596 Time: 0:01:51 ( 7.61 ms/it)\n objective: 13340.997305768331\nMinimizing 14611 Time: 0:01:51 ( 7.61 ms/it)\n objective: 13340.989243322867\nMinimizing 14626 Time: 0:01:51 ( 7.61 ms/it)\n objective: 13340.904651490418\nMinimizing 14641 Time: 0:01:51 ( 7.61 ms/it)\n objective: 13340.85213009418\nMinimizing 14656 Time: 0:01:51 ( 7.61 ms/it)\n objective: 13340.788736080816\nMinimizing 14671 Time: 0:01:51 ( 7.61 ms/it)\n objective: 13340.631001280948\nMinimizing 14686 Time: 0:01:51 ( 7.61 ms/it)\n objective: 13340.098106181962\nMinimizing 14699 Time: 0:01:51 ( 7.61 ms/it)\n objective: 13339.802541688165\nMinimizing 14712 Time: 0:01:51 ( 7.61 ms/it)\n objective: 13339.7996311221\nMinimizing 14726 Time: 0:01:52 ( 7.61 ms/it)\n objective: 13339.786178103866\nMinimizing 14739 Time: 0:01:52 ( 7.61 ms/it)\n objective: 13339.783297457383\nMinimizing 14754 Time: 0:01:52 ( 7.61 ms/it)\n objective: 13339.746047789711\nMinimizing 14768 Time: 0:01:52 ( 7.61 ms/it)\n objective: 13339.731428983258\nMinimizing 14782 Time: 0:01:52 ( 7.61 ms/it)\n objective: 13339.67051133667\nMinimizing 14796 Time: 0:01:52 ( 7.61 ms/it)\n objective: 13339.617804636975\nMinimizing 14810 Time: 0:01:52 ( 7.61 ms/it)\n objective: 13339.538325454938\nMinimizing 14824 Time: 0:01:52 ( 7.61 ms/it)\n objective: 13339.414503420601\nMinimizing 14839 Time: 0:01:52 ( 7.61 ms/it)\n objective: 13339.408859688148\nMinimizing 14853 Time: 0:01:52 ( 7.61 ms/it)\n objective: 13339.390946794207\nMinimizing 14867 Time: 0:01:53 ( 7.61 ms/it)\n objective: 13339.378136657353\nMinimizing 14881 Time: 0:01:53 ( 7.61 ms/it)\n objective: 13339.367340318568\nMinimizing 14892 Time: 0:01:53 ( 7.61 ms/it)\n objective: 13339.354375949617\nMinimizing 14905 Time: 0:01:53 ( 7.61 ms/it)\n objective: 13339.322476467445\nMinimizing 14918 Time: 0:01:53 ( 7.61 ms/it)\n objective: 13339.22538221386\nMinimizing 14931 Time: 0:01:53 ( 7.61 ms/it)\n objective: 13339.106615442135\nMinimizing 14944 Time: 0:01:53 ( 7.61 ms/it)\n objective: 13339.093225269928\nMinimizing 14958 Time: 0:01:53 ( 7.61 ms/it)\n objective: 13339.063518638402\nMinimizing 14973 Time: 0:01:53 ( 7.61 ms/it)\n objective: 13339.046285189797\nMinimizing 14987 Time: 0:01:54 ( 7.61 ms/it)\n objective: 13339.038613203244\nMinimizing 15002 Time: 0:01:54 ( 7.61 ms/it)\n objective: 13338.997378755244\nMinimizing 15017 Time: 0:01:54 ( 7.61 ms/it)\n objective: 13338.932883187394\nMinimizing 15030 Time: 0:01:54 ( 7.61 ms/it)\n objective: 13339.051062597959\nMinimizing 15043 Time: 0:01:54 ( 7.61 ms/it)\n objective: 13338.621232862097\nMinimizing 15056 Time: 0:01:54 ( 7.61 ms/it)\n objective: 13338.617861841056\nMinimizing 15069 Time: 0:01:54 ( 7.61 ms/it)\n objective: 13338.598513041637\nMinimizing 15082 Time: 0:01:54 ( 7.61 ms/it)\n objective: 13338.590907214624\nMinimizing 15095 Time: 0:01:54 ( 7.61 ms/it)\n objective: 13338.571448459938\nMinimizing 15109 Time: 0:01:54 ( 7.61 ms/it)\n objective: 13338.55328101055\nMinimizing 15123 Time: 0:01:55 ( 7.61 ms/it)\n objective: 13338.54889314367\nMinimizing 15137 Time: 0:01:55 ( 7.61 ms/it)\n objective: 13338.452531246578\nMinimizing 15151 Time: 0:01:55 ( 7.61 ms/it)\n objective: 13338.433243870313\nMinimizing 15165 Time: 0:01:55 ( 7.61 ms/it)\n objective: 13338.416701821036\nMinimizing 15179 Time: 0:01:55 ( 7.61 ms/it)\n objective: 13338.404845290723\nMinimizing 15193 Time: 0:01:55 ( 7.61 ms/it)\n objective: 13338.385556007946\nMinimizing 15207 Time: 0:01:55 ( 7.61 ms/it)\n objective: 13338.383354799196\nMinimizing 15221 Time: 0:01:55 ( 7.61 ms/it)\n objective: 13338.37973805867\nMinimizing 15236 Time: 0:01:55 ( 7.61 ms/it)\n objective: 13338.363546701294\nMinimizing 15251 Time: 0:01:56 ( 7.61 ms/it)\n objective: 13338.349669530842\nMinimizing 15266 Time: 0:01:56 ( 7.61 ms/it)\n objective: 13338.316654233975\nMinimizing 15281 Time: 0:01:56 ( 7.61 ms/it)\n objective: 13338.312177241954\nMinimizing 15296 Time: 0:01:56 ( 7.61 ms/it)\n objective: 13338.291933278197\nMinimizing 15310 Time: 0:01:56 ( 7.61 ms/it)\n objective: 13338.186993749754\nMinimizing 15324 Time: 0:01:56 ( 7.61 ms/it)\n objective: 13338.00936621119\nMinimizing 15338 Time: 0:01:56 ( 7.61 ms/it)\n objective: 13337.995048239733\nMinimizing 15352 Time: 0:01:56 ( 7.61 ms/it)\n objective: 13337.987054526275\nMinimizing 15366 Time: 0:01:56 ( 7.61 ms/it)\n objective: 13337.976820763608\nMinimizing 15379 Time: 0:01:56 ( 7.61 ms/it)\n objective: 13337.960073338872\nMinimizing 15394 Time: 0:01:57 ( 7.61 ms/it)\n objective: 13337.935898769712\nMinimizing 15409 Time: 0:01:57 ( 7.61 ms/it)\n objective: 13337.924520801054\nMinimizing 15423 Time: 0:01:57 ( 7.61 ms/it)\n objective: 13337.921618246626\nMinimizing 15438 Time: 0:01:57 ( 7.61 ms/it)\n objective: 13337.91258408071\nMinimizing 15453 Time: 0:01:57 ( 7.60 ms/it)\n objective: 13337.893248043074\nMinimizing 15468 Time: 0:01:57 ( 7.60 ms/it)\n objective: 13337.866187868603\nMinimizing 15482 Time: 0:01:57 ( 7.60 ms/it)\n objective: 13337.82368052403\nMinimizing 15497 Time: 0:01:57 ( 7.60 ms/it)\n objective: 13337.802456079604\nMinimizing 15512 Time: 0:01:57 ( 7.60 ms/it)\n objective: 13337.778319405399\nMinimizing 15527 Time: 0:01:58 ( 7.60 ms/it)\n objective: 13337.723085633355\nMinimizing 15542 Time: 0:01:58 ( 7.60 ms/it)\n objective: 13337.681336951027\nMinimizing 15557 Time: 0:01:58 ( 7.60 ms/it)\n objective: 13337.623714578207\nMinimizing 15571 Time: 0:01:58 ( 7.60 ms/it)\n objective: 13337.620116607162\nMinimizing 15585 Time: 0:01:58 ( 7.60 ms/it)\n objective: 13337.60039693924\nMinimizing 15599 Time: 0:01:58 ( 7.60 ms/it)\n objective: 13337.53667073154\nMinimizing 15613 Time: 0:01:58 ( 7.60 ms/it)\n objective: 13337.511657091229\nMinimizing 15628 Time: 0:01:58 ( 7.60 ms/it)\n objective: 13337.37685004936\nMinimizing 15643 Time: 0:01:58 ( 7.60 ms/it)\n objective: 13337.256223656761\nMinimizing 15657 Time: 0:01:58 ( 7.60 ms/it)\n objective: 13337.224781952937\nMinimizing 15671 Time: 0:01:59 ( 7.60 ms/it)\n objective: 13337.216337961538\nMinimizing 15686 Time: 0:01:59 ( 7.60 ms/it)\n objective: 13337.19972683298\nMinimizing 15701 Time: 0:01:59 ( 7.60 ms/it)\n objective: 13337.18508632273\nMinimizing 15716 Time: 0:01:59 ( 7.60 ms/it)\n objective: 13337.110989585926\nMinimizing 15731 Time: 0:01:59 ( 7.60 ms/it)\n objective: 13337.00356813491\nMinimizing 15746 Time: 0:01:59 ( 7.59 ms/it)\n objective: 13336.990551020863\nMinimizing 15761 Time: 0:01:59 ( 7.59 ms/it)\n objective: 13336.94006346217\nMinimizing 15776 Time: 0:01:59 ( 7.59 ms/it)\n objective: 13336.933684914082\nMinimizing 15791 Time: 0:01:59 ( 7.59 ms/it)\n objective: 13336.913998128693\nMinimizing 15806 Time: 0:02:00 ( 7.59 ms/it)\n objective: 13336.87868722422\nMinimizing 15820 Time: 0:02:00 ( 7.59 ms/it)\n objective: 13336.82005116382\nMinimizing 15834 Time: 0:02:00 ( 7.59 ms/it)\n objective: 13336.70335415849\nMinimizing 15848 Time: 0:02:00 ( 7.59 ms/it)\n objective: 13336.304934579377\nMinimizing 15861 Time: 0:02:00 ( 7.59 ms/it)\n objective: 13336.26960745341\nMinimizing 15875 Time: 0:02:00 ( 7.59 ms/it)\n objective: 13336.25961314018\nMinimizing 15890 Time: 0:02:00 ( 7.59 ms/it)\n objective: 13336.257396265588\nMinimizing 15905 Time: 0:02:00 ( 7.59 ms/it)\n objective: 13336.254043617424\nMinimizing 15919 Time: 0:02:00 ( 7.59 ms/it)\n objective: 13336.246404504054\nMinimizing 15933 Time: 0:02:00 ( 7.59 ms/it)\n objective: 13336.24520488202\nMinimizing 15945 Time: 0:02:01 ( 7.59 ms/it)\n objective: 13336.232147575625\nMinimizing 15957 Time: 0:02:01 ( 7.59 ms/it)\n objective: 13336.212331401934\nMinimizing 15969 Time: 0:02:01 ( 7.60 ms/it)\n objective: 13336.1841625686\nMinimizing 15981 Time: 0:02:01 ( 7.60 ms/it)\n objective: 13336.181296923438\nMinimizing 15993 Time: 0:02:01 ( 7.60 ms/it)\n objective: 13336.156511995301\nMinimizing 16005 Time: 0:02:01 ( 7.60 ms/it)\n objective: 13336.14404342165\nMinimizing 16017 Time: 0:02:01 ( 7.60 ms/it)\n objective: 13336.022815213975\nMinimizing 16029 Time: 0:02:01 ( 7.60 ms/it)\n objective: 13335.92158471718\nMinimizing 16044 Time: 0:02:01 ( 7.60 ms/it)\n objective: 13335.919924256574\nMinimizing 16059 Time: 0:02:02 ( 7.60 ms/it)\n objective: 13335.89393574206\nMinimizing 16074 Time: 0:02:02 ( 7.60 ms/it)\n objective: 13335.844296660238\nMinimizing 16089 Time: 0:02:02 ( 7.60 ms/it)\n objective: 13335.746457684778\nMinimizing 16104 Time: 0:02:02 ( 7.60 ms/it)\n objective: 13335.745703669192\nMinimizing 16119 Time: 0:02:02 ( 7.60 ms/it)\n objective: 13335.701548804107\nMinimizing 16133 Time: 0:02:02 ( 7.60 ms/it)\n objective: 13335.626890877931\nMinimizing 16147 Time: 0:02:02 ( 7.60 ms/it)\n objective: 13335.619251927536\nMinimizing 16161 Time: 0:02:02 ( 7.60 ms/it)\n objective: 13335.549841101958\nMinimizing 16175 Time: 0:02:02 ( 7.60 ms/it)\n objective: 13335.410669319033\nMinimizing 16189 Time: 0:02:02 ( 7.60 ms/it)\n objective: 13335.223710334918\nMinimizing 16204 Time: 0:02:03 ( 7.60 ms/it)\n objective: 13335.220345227215\nMinimizing 16219 Time: 0:02:03 ( 7.59 ms/it)\n objective: 13335.218919295177\nMinimizing 16234 Time: 0:02:03 ( 7.59 ms/it)\n objective: 13335.218114378069\nMinimizing 16249 Time: 0:02:03 ( 7.59 ms/it)\n objective: 13335.21483029914\nMinimizing 16263 Time: 0:02:03 ( 7.59 ms/it)\n objective: 13335.196398677857\nMinimizing 16277 Time: 0:02:03 ( 7.59 ms/it)\n objective: 13335.12331996901\nMinimizing 16291 Time: 0:02:03 ( 7.59 ms/it)\n objective: 13335.122550750682\nMinimizing 16305 Time: 0:02:03 ( 7.59 ms/it)\n objective: 13335.120941846872\nMinimizing 16319 Time: 0:02:03 ( 7.59 ms/it)\n objective: 13335.111804640095\nMinimizing 16333 Time: 0:02:04 ( 7.59 ms/it)\n objective: 13335.10552670512\nMinimizing 16347 Time: 0:02:04 ( 7.59 ms/it)\n objective: 13335.09730867549\nMinimizing 16361 Time: 0:02:04 ( 7.59 ms/it)\n objective: 13335.08002516294\nMinimizing 16375 Time: 0:02:04 ( 7.59 ms/it)\n objective: 13335.062649100168\nMinimizing 16390 Time: 0:02:04 ( 7.59 ms/it)\n objective: 13335.003975012754\nMinimizing 16405 Time: 0:02:04 ( 7.59 ms/it)\n objective: 13334.9908595706\nMinimizing 16419 Time: 0:02:04 ( 7.59 ms/it)\n objective: 13334.923182506012\nMinimizing 16433 Time: 0:02:04 ( 7.59 ms/it)\n objective: 13334.84935115552\nMinimizing 16447 Time: 0:02:04 ( 7.59 ms/it)\n objective: 13334.843690938163\nMinimizing 16461 Time: 0:02:04 ( 7.59 ms/it)\n objective: 13334.837000614687\nMinimizing 16475 Time: 0:02:05 ( 7.59 ms/it)\n objective: 13334.827620651718\nMinimizing 16489 Time: 0:02:05 ( 7.59 ms/it)\n objective: 13334.808317676594\nMinimizing 16503 Time: 0:02:05 ( 7.59 ms/it)\n objective: 13334.803130001128\nMinimizing 16517 Time: 0:02:05 ( 7.59 ms/it)\n objective: 13334.778763942137\nMinimizing 16531 Time: 0:02:05 ( 7.59 ms/it)\n objective: 13334.752197105197\nMinimizing 16545 Time: 0:02:05 ( 7.59 ms/it)\n objective: 13334.73026654141\nMinimizing 16560 Time: 0:02:05 ( 7.59 ms/it)\n objective: 13334.667550521219\nMinimizing 16575 Time: 0:02:05 ( 7.59 ms/it)\n objective: 13334.583685550962\nMinimizing 16590 Time: 0:02:05 ( 7.59 ms/it)\n objective: 13334.58317779962\nMinimizing 16605 Time: 0:02:05 ( 7.59 ms/it)\n objective: 13334.57324292342\nMinimizing 16620 Time: 0:02:06 ( 7.59 ms/it)\n objective: 13334.567395581005\nMinimizing 16635 Time: 0:02:06 ( 7.58 ms/it)\n objective: 13334.545578969643\nMinimizing 16650 Time: 0:02:06 ( 7.58 ms/it)\n objective: 13334.543618434232\nMinimizing 16665 Time: 0:02:06 ( 7.58 ms/it)\n objective: 13334.50757851838\nMinimizing 16680 Time: 0:02:06 ( 7.58 ms/it)\n objective: 13334.475236578328\nMinimizing 16695 Time: 0:02:06 ( 7.58 ms/it)\n objective: 13334.42317210642\nMinimizing 16710 Time: 0:02:06 ( 7.58 ms/it)\n objective: 13334.364084989596\nMinimizing 16724 Time: 0:02:06 ( 7.58 ms/it)\n objective: 13334.360189547035\nMinimizing 16739 Time: 0:02:06 ( 7.58 ms/it)\n objective: 13334.341324531386\nMinimizing 16754 Time: 0:02:07 ( 7.58 ms/it)\n objective: 13334.31600599354\nMinimizing 16769 Time: 0:02:07 ( 7.58 ms/it)\n objective: 13334.305818915644\nMinimizing 16784 Time: 0:02:07 ( 7.58 ms/it)\n objective: 13334.218941403124\nMinimizing 16799 Time: 0:02:07 ( 7.58 ms/it)\n objective: 13334.08812966587\nMinimizing 16814 Time: 0:02:07 ( 7.58 ms/it)\n objective: 13334.054647581615\nMinimizing 16829 Time: 0:02:07 ( 7.58 ms/it)\n objective: 13334.051145102552\nMinimizing 16844 Time: 0:02:07 ( 7.58 ms/it)\n objective: 13334.022545259388\nMinimizing 16859 Time: 0:02:07 ( 7.58 ms/it)\n objective: 13334.013131942163\nMinimizing 16873 Time: 0:02:07 ( 7.58 ms/it)\n objective: 13333.988855241048\nMinimizing 16887 Time: 0:02:07 ( 7.58 ms/it)\n objective: 13333.895507994574\nMinimizing 16901 Time: 0:02:08 ( 7.58 ms/it)\n objective: 13333.861465142676\nMinimizing 16914 Time: 0:02:08 ( 7.58 ms/it)\n objective: 13333.846570221343\nMinimizing 16928 Time: 0:02:08 ( 7.58 ms/it)\n objective: 13333.844384498356\nMinimizing 16943 Time: 0:02:08 ( 7.58 ms/it)\n objective: 13333.841598127896\nMinimizing 16958 Time: 0:02:08 ( 7.58 ms/it)\n objective: 13333.838511294707\nMinimizing 16973 Time: 0:02:08 ( 7.58 ms/it)\n objective: 13333.831727143472\nMinimizing 16987 Time: 0:02:08 ( 7.58 ms/it)\n objective: 13333.761543254921\nMinimizing 17001 Time: 0:02:08 ( 7.58 ms/it)\n objective: 13333.759007582317\nMinimizing 17016 Time: 0:02:08 ( 7.57 ms/it)\n objective: 13333.750043274238\nMinimizing 17031 Time: 0:02:08 ( 7.57 ms/it)\n objective: 13333.742699564245\nMinimizing 17046 Time: 0:02:09 ( 7.57 ms/it)\n objective: 13333.738548880305\nMinimizing 17061 Time: 0:02:09 ( 7.57 ms/it)\n objective: 13333.70332397725\nMinimizing 17076 Time: 0:02:09 ( 7.57 ms/it)\n objective: 13333.658183210646\nMinimizing 17091 Time: 0:02:09 ( 7.57 ms/it)\n objective: 13333.609504180356\nMinimizing 17106 Time: 0:02:09 ( 7.57 ms/it)\n objective: 13333.608415762683\nMinimizing 17121 Time: 0:02:09 ( 7.57 ms/it)\n objective: 13333.602898274534\nMinimizing 17136 Time: 0:02:09 ( 7.57 ms/it)\n objective: 13333.59507447989\nMinimizing 17150 Time: 0:02:09 ( 7.57 ms/it)\n objective: 13333.589845285947\nMinimizing 17164 Time: 0:02:09 ( 7.57 ms/it)\n objective: 13333.57636132562\nMinimizing 17178 Time: 0:02:10 ( 7.57 ms/it)\n objective: 13333.559629796793\nMinimizing 17192 Time: 0:02:10 ( 7.57 ms/it)\n objective: 13333.556620464973\nMinimizing 17206 Time: 0:02:10 ( 7.57 ms/it)\n objective: 13333.543691341416\nMinimizing 17220 Time: 0:02:10 ( 7.57 ms/it)\n objective: 13333.5359053734\nMinimizing 17235 Time: 0:02:10 ( 7.57 ms/it)\n objective: 13333.500998866839\nMinimizing 17250 Time: 0:02:10 ( 7.57 ms/it)\n objective: 13333.49816048437\nMinimizing 17265 Time: 0:02:10 ( 7.57 ms/it)\n objective: 13333.462573128883\nMinimizing 17280 Time: 0:02:10 ( 7.57 ms/it)\n objective: 13333.450415160492\nMinimizing 17295 Time: 0:02:10 ( 7.57 ms/it)\n objective: 13333.349900711793\nMinimizing 17310 Time: 0:02:10 ( 7.57 ms/it)\n objective: 13333.290778406292\nMinimizing 17325 Time: 0:02:11 ( 7.57 ms/it)\n objective: 13333.28956611346\nMinimizing 17340 Time: 0:02:11 ( 7.56 ms/it)\n objective: 13333.284034536722\nMinimizing 17355 Time: 0:02:11 ( 7.56 ms/it)\n objective: 13333.270325968595\nMinimizing 17370 Time: 0:02:11 ( 7.56 ms/it)\n objective: 13333.238687290366\nMinimizing 17385 Time: 0:02:11 ( 7.56 ms/it)\n objective: 13333.228471864837\nMinimizing 17400 Time: 0:02:11 ( 7.56 ms/it)\n objective: 13333.21777517206\nMinimizing 17415 Time: 0:02:11 ( 7.56 ms/it)\n objective: 13333.20684214843\nMinimizing 17430 Time: 0:02:11 ( 7.56 ms/it)\n objective: 13333.1543679713\nMinimizing 17445 Time: 0:02:11 ( 7.56 ms/it)\n objective: 13333.142051582137\nMinimizing 17460 Time: 0:02:11 ( 7.56 ms/it)\n objective: 13333.137439363796\nMinimizing 17475 Time: 0:02:12 ( 7.56 ms/it)\n objective: 13333.116010241953\nMinimizing 17490 Time: 0:02:12 ( 7.56 ms/it)\n objective: 13333.060280116428\nMinimizing 17505 Time: 0:02:12 ( 7.56 ms/it)\n objective: 13333.050604128272\nMinimizing 17520 Time: 0:02:12 ( 7.56 ms/it)\n objective: 13333.045945147009\nMinimizing 17535 Time: 0:02:12 ( 7.56 ms/it)\n objective: 13333.027130369293\nMinimizing 17550 Time: 0:02:12 ( 7.56 ms/it)\n objective: 13333.015321131978\nMinimizing 17565 Time: 0:02:12 ( 7.55 ms/it)\n objective: 13332.97880722637\nMinimizing 17580 Time: 0:02:12 ( 7.55 ms/it)\n objective: 13332.97565210861\nMinimizing 17595 Time: 0:02:12 ( 7.55 ms/it)\n objective: 13332.983109679197\nMinimizing 17610 Time: 0:02:13 ( 7.55 ms/it)\n objective: 13332.955121745945\nMinimizing 17625 Time: 0:02:13 ( 7.55 ms/it)\n objective: 13332.937700654504\nMinimizing 17640 Time: 0:02:13 ( 7.55 ms/it)\n objective: 13332.882778018167\nMinimizing 17655 Time: 0:02:13 ( 7.55 ms/it)\n objective: 13332.88179781165\nMinimizing 17670 Time: 0:02:13 ( 7.55 ms/it)\n objective: 13332.86334774716\nMinimizing 17685 Time: 0:02:13 ( 7.55 ms/it)\n objective: 13332.816742069132\nMinimizing 17700 Time: 0:02:13 ( 7.55 ms/it)\n objective: 13332.703862158785\nMinimizing 17715 Time: 0:02:13 ( 7.55 ms/it)\n objective: 13332.701447899948\nMinimizing 17730 Time: 0:02:13 ( 7.55 ms/it)\n objective: 13332.70036918491\nMinimizing 17745 Time: 0:02:13 ( 7.55 ms/it)\n objective: 13332.691952049987\nMinimizing 17760 Time: 0:02:14 ( 7.55 ms/it)\n objective: 13332.687567772737\nMinimizing 17775 Time: 0:02:14 ( 7.55 ms/it)\n objective: 13332.678697258212\nMinimizing 17790 Time: 0:02:14 ( 7.55 ms/it)\n objective: 13332.667736668933\nMinimizing 17805 Time: 0:02:14 ( 7.54 ms/it)\n objective: 13332.646707279593\nMinimizing 17820 Time: 0:02:14 ( 7.54 ms/it)\n objective: 13332.645596742994\nMinimizing 17835 Time: 0:02:14 ( 7.54 ms/it)\n objective: 13332.641323786644\nMinimizing 17850 Time: 0:02:14 ( 7.54 ms/it)\n objective: 13332.633087447546\nMinimizing 17865 Time: 0:02:14 ( 7.54 ms/it)\n objective: 13332.619992940177\nMinimizing 17880 Time: 0:02:14 ( 7.54 ms/it)\n objective: 13332.608462807628\nMinimizing 17895 Time: 0:02:14 ( 7.54 ms/it)\n objective: 13332.600785487033\nMinimizing 17910 Time: 0:02:15 ( 7.54 ms/it)\n objective: 13332.549919360208\nMinimizing 17925 Time: 0:02:15 ( 7.54 ms/it)\n objective: 13332.501710666329\nMinimizing 17940 Time: 0:02:15 ( 7.54 ms/it)\n objective: 13332.499659351568\nMinimizing 17955 Time: 0:02:15 ( 7.54 ms/it)\n objective: 13332.491402559594\nMinimizing 17970 Time: 0:02:15 ( 7.54 ms/it)\n objective: 13332.487239124843\nMinimizing 17985 Time: 0:02:15 ( 7.54 ms/it)\n objective: 13332.45934803372\nMinimizing 18000 Time: 0:02:15 ( 7.54 ms/it)\n objective: 13332.453125551532\nMinimizing 18015 Time: 0:02:15 ( 7.54 ms/it)\n objective: 13332.446531414462\nMinimizing 18030 Time: 0:02:15 ( 7.54 ms/it)\n objective: 13332.439426579731\nMinimizing 18045 Time: 0:02:15 ( 7.54 ms/it)\n objective: 13332.417197071183\nMinimizing 18060 Time: 0:02:16 ( 7.54 ms/it)\n objective: 13332.399875101924\nMinimizing 18075 Time: 0:02:16 ( 7.53 ms/it)\n objective: 13332.34404242313\nMinimizing 18090 Time: 0:02:16 ( 7.53 ms/it)\n objective: 13332.343236198773\nMinimizing 18104 Time: 0:02:16 ( 7.53 ms/it)\n objective: 13332.333125607394\nMinimizing 18118 Time: 0:02:16 ( 7.53 ms/it)\n objective: 13332.316439239708\nMinimizing 18132 Time: 0:02:16 ( 7.53 ms/it)\n objective: 13332.291426513337\nMinimizing 18146 Time: 0:02:16 ( 7.53 ms/it)\n objective: 13332.281114748686\nMinimizing 18160 Time: 0:02:16 ( 7.53 ms/it)\n objective: 13332.237256910972\nMinimizing 18173 Time: 0:02:16 ( 7.53 ms/it)\n objective: 13332.186687238827\nMinimizing 18188 Time: 0:02:17 ( 7.53 ms/it)\n objective: 13332.179816500524\nMinimizing 18203 Time: 0:02:17 ( 7.53 ms/it)\n objective: 13332.137391900345\nMinimizing 18218 Time: 0:02:17 ( 7.53 ms/it)\n objective: 13332.135785362836\nMinimizing 18233 Time: 0:02:17 ( 7.53 ms/it)\n objective: 13332.128262044927\nMinimizing 18248 Time: 0:02:17 ( 7.53 ms/it)\n objective: 13332.116552123509\nMinimizing 18263 Time: 0:02:17 ( 7.53 ms/it)\n objective: 13332.10702422502\nMinimizing 18278 Time: 0:02:17 ( 7.53 ms/it)\n objective: 13332.093792357344\nMinimizing 18293 Time: 0:02:17 ( 7.53 ms/it)\n objective: 13332.079644275545\nMinimizing 18308 Time: 0:02:17 ( 7.53 ms/it)\n objective: 13332.077538074613\nMinimizing 18323 Time: 0:02:17 ( 7.53 ms/it)\n objective: 13332.060810137089\nMinimizing 18338 Time: 0:02:18 ( 7.53 ms/it)\n objective: 13332.032678512092\nMinimizing 18353 Time: 0:02:18 ( 7.53 ms/it)\n objective: 13332.001288815278\nMinimizing 18368 Time: 0:02:18 ( 7.53 ms/it)\n objective: 13331.841489251798\nMinimizing 18383 Time: 0:02:18 ( 7.53 ms/it)\n objective: 13331.829448723205\nMinimizing 18398 Time: 0:02:18 ( 7.53 ms/it)\n objective: 13331.820960955032\nMinimizing 18413 Time: 0:02:18 ( 7.53 ms/it)\n objective: 13331.806649847997\nMinimizing 18428 Time: 0:02:18 ( 7.53 ms/it)\n objective: 13331.791495228303\nMinimizing 18443 Time: 0:02:18 ( 7.52 ms/it)\n objective: 13331.76630928168\nMinimizing 18458 Time: 0:02:18 ( 7.52 ms/it)\n objective: 13331.743554202214\nMinimizing 18473 Time: 0:02:18 ( 7.52 ms/it)\n objective: 13331.730589842344\nMinimizing 18488 Time: 0:02:19 ( 7.52 ms/it)\n objective: 13331.728928041397\nMinimizing 18503 Time: 0:02:19 ( 7.52 ms/it)\n objective: 13331.71955521925\nMinimizing 18518 Time: 0:02:19 ( 7.52 ms/it)\n objective: 13331.68633211519\nMinimizing 18533 Time: 0:02:19 ( 7.52 ms/it)\n objective: 13331.642191328901\nMinimizing 18548 Time: 0:02:19 ( 7.52 ms/it)\n objective: 13331.483301410866\nMinimizing 18563 Time: 0:02:19 ( 7.52 ms/it)\n objective: 13331.41677610945\nMinimizing 18578 Time: 0:02:19 ( 7.52 ms/it)\n objective: 13331.409212166298\nMinimizing 18593 Time: 0:02:19 ( 7.52 ms/it)\n objective: 13331.406243641904\nMinimizing 18608 Time: 0:02:19 ( 7.52 ms/it)\n objective: 13331.402349978707\nMinimizing 18623 Time: 0:02:20 ( 7.52 ms/it)\n objective: 13331.397021771743\nMinimizing 18638 Time: 0:02:20 ( 7.52 ms/it)\n objective: 13331.387051208018\nMinimizing 18652 Time: 0:02:20 ( 7.52 ms/it)\n objective: 13331.374546016901\nMinimizing 18666 Time: 0:02:20 ( 7.52 ms/it)\n objective: 13331.30951760261\nMinimizing 18680 Time: 0:02:20 ( 7.52 ms/it)\n objective: 13331.016718128223\nMinimizing 18694 Time: 0:02:20 ( 7.52 ms/it)\n objective: 13331.016200164442\nMinimizing 18709 Time: 0:02:20 ( 7.52 ms/it)\n objective: 13331.014856694055\nMinimizing 18724 Time: 0:02:20 ( 7.52 ms/it)\n objective: 13331.000092066279\nMinimizing 18739 Time: 0:02:20 ( 7.52 ms/it)\n objective: 13330.98174820219\nMinimizing 18754 Time: 0:02:20 ( 7.52 ms/it)\n objective: 13330.923246875565\nMinimizing 18769 Time: 0:02:21 ( 7.52 ms/it)\n objective: 13331.057292039288\nMinimizing 18784 Time: 0:02:21 ( 7.52 ms/it)\n objective: 13330.820249156663\nMinimizing 18799 Time: 0:02:21 ( 7.51 ms/it)\n objective: 13330.819560777869\nMinimizing 18814 Time: 0:02:21 ( 7.51 ms/it)\n objective: 13330.81742589306\nMinimizing 18829 Time: 0:02:21 ( 7.51 ms/it)\n objective: 13330.813268829435\nMinimizing 18844 Time: 0:02:21 ( 7.51 ms/it)\n objective: 13330.809176184055\nMinimizing 18859 Time: 0:02:21 ( 7.51 ms/it)\n objective: 13330.780886687935\nMinimizing 18874 Time: 0:02:21 ( 7.51 ms/it)\n objective: 13330.735498194335\nMinimizing 18889 Time: 0:02:21 ( 7.51 ms/it)\n objective: 13330.717466289127\nMinimizing 18904 Time: 0:02:21 ( 7.51 ms/it)\n objective: 13330.671095346988\nMinimizing 18919 Time: 0:02:22 ( 7.51 ms/it)\n objective: 13330.658830652581\nMinimizing 18934 Time: 0:02:22 ( 7.51 ms/it)\n objective: 13330.646016764695\nMinimizing 18949 Time: 0:02:22 ( 7.51 ms/it)\n objective: 13330.629924788125\nMinimizing 18964 Time: 0:02:22 ( 7.51 ms/it)\n objective: 13330.575998203167\nMinimizing 18979 Time: 0:02:22 ( 7.51 ms/it)\n objective: 13330.5184689651\nMinimizing 18994 Time: 0:02:22 ( 7.51 ms/it)\n objective: 13330.51788424806\nMinimizing 19009 Time: 0:02:22 ( 7.51 ms/it)\n objective: 13330.51448597193\nMinimizing 19024 Time: 0:02:22 ( 7.51 ms/it)\n objective: 13330.509704314725\nMinimizing 19039 Time: 0:02:22 ( 7.51 ms/it)\n objective: 13330.503051528925\nMinimizing 19054 Time: 0:02:23 ( 7.51 ms/it)\n objective: 13330.479735019435\nMinimizing 19069 Time: 0:02:23 ( 7.51 ms/it)\n objective: 13330.46355300842\nMinimizing 19084 Time: 0:02:23 ( 7.51 ms/it)\n objective: 13330.432544731171\nMinimizing 19099 Time: 0:02:23 ( 7.51 ms/it)\n objective: 13330.422039944147\nMinimizing 19114 Time: 0:02:23 ( 7.51 ms/it)\n objective: 13330.358435354035\nMinimizing 19129 Time: 0:02:23 ( 7.50 ms/it)\n objective: 13330.335687806124\nMinimizing 19144 Time: 0:02:23 ( 7.50 ms/it)\n objective: 13330.334375663617\nMinimizing 19159 Time: 0:02:23 ( 7.50 ms/it)\n objective: 13330.322470435196\nMinimizing 19174 Time: 0:02:23 ( 7.50 ms/it)\n objective: 13330.277306498145\nMinimizing 19189 Time: 0:02:23 ( 7.50 ms/it)\n objective: 13330.270544234161\nMinimizing 19204 Time: 0:02:24 ( 7.50 ms/it)\n objective: 13330.209340675552\nMinimizing 19219 Time: 0:02:24 ( 7.50 ms/it)\n objective: 13330.205449058165\nMinimizing 19234 Time: 0:02:24 ( 7.50 ms/it)\n objective: 13330.199347486465\nMinimizing 19249 Time: 0:02:24 ( 7.50 ms/it)\n objective: 13330.186147581022\nMinimizing 19264 Time: 0:02:24 ( 7.50 ms/it)\n objective: 13330.15832003655\nMinimizing 19279 Time: 0:02:24 ( 7.50 ms/it)\n objective: 13330.074843961767\nMinimizing 19294 Time: 0:02:24 ( 7.50 ms/it)\n objective: 13330.02981807918\nMinimizing 19309 Time: 0:02:24 ( 7.50 ms/it)\n objective: 13330.02634915684\nMinimizing 19324 Time: 0:02:24 ( 7.50 ms/it)\n objective: 13329.999798483856\nMinimizing 19339 Time: 0:02:24 ( 7.50 ms/it)\n objective: 13329.969057325463\nMinimizing 19354 Time: 0:02:25 ( 7.50 ms/it)\n objective: 13329.963734783938\nMinimizing 19369 Time: 0:02:25 ( 7.50 ms/it)\n objective: 13329.95293151198\nMinimizing 19384 Time: 0:02:25 ( 7.49 ms/it)\n objective: 13329.933692490544\nMinimizing 19399 Time: 0:02:25 ( 7.49 ms/it)\n objective: 13329.91544806432\nMinimizing 19414 Time: 0:02:25 ( 7.49 ms/it)\n objective: 13329.897226474255\nMinimizing 19429 Time: 0:02:25 ( 7.49 ms/it)\n objective: 13329.833230712793\nMinimizing 19444 Time: 0:02:25 ( 7.49 ms/it)\n objective: 13329.83249528308\nMinimizing 19459 Time: 0:02:25 ( 7.49 ms/it)\n objective: 13329.828962245141\nMinimizing 19474 Time: 0:02:25 ( 7.49 ms/it)\n objective: 13329.825982439135\nMinimizing 19489 Time: 0:02:26 ( 7.49 ms/it)\n objective: 13329.821616552857\nMinimizing 19504 Time: 0:02:26 ( 7.49 ms/it)\n objective: 13329.814144659802\nMinimizing 19519 Time: 0:02:26 ( 7.49 ms/it)\n objective: 13329.79126935944\nMinimizing 19534 Time: 0:02:26 ( 7.49 ms/it)\n objective: 13329.789456750936\nMinimizing 19549 Time: 0:02:26 ( 7.49 ms/it)\n objective: 13329.755567503045\nMinimizing 19564 Time: 0:02:26 ( 7.49 ms/it)\n objective: 13329.75065324807\nMinimizing 19579 Time: 0:02:26 ( 7.49 ms/it)\n objective: 13329.74617634451\nMinimizing 19594 Time: 0:02:26 ( 7.49 ms/it)\n objective: 13329.729968831707\nMinimizing 19609 Time: 0:02:26 ( 7.49 ms/it)\n objective: 13329.718980641403\nMinimizing 19624 Time: 0:02:26 ( 7.49 ms/it)\n objective: 13329.652253531873\nMinimizing 19639 Time: 0:02:27 ( 7.49 ms/it)\n objective: 13329.650557134548\nMinimizing 19654 Time: 0:02:27 ( 7.49 ms/it)\n objective: 13329.647233110918\nMinimizing 19669 Time: 0:02:27 ( 7.49 ms/it)\n objective: 13329.637596124892\nMinimizing 19684 Time: 0:02:27 ( 7.49 ms/it)\n objective: 13329.629141556885\nMinimizing 19699 Time: 0:02:27 ( 7.49 ms/it)\n objective: 13329.597380038285\nMinimizing 19714 Time: 0:02:27 ( 7.49 ms/it)\n objective: 13329.590077110573\nMinimizing 19728 Time: 0:02:27 ( 7.49 ms/it)\n objective: 13329.550889750011\nMinimizing 19743 Time: 0:02:27 ( 7.49 ms/it)\n objective: 13329.535516698132\nMinimizing 19758 Time: 0:02:27 ( 7.48 ms/it)\n objective: 13329.529940206885\nMinimizing 19773 Time: 0:02:27 ( 7.48 ms/it)\n objective: 13329.501745272908\nMinimizing 19788 Time: 0:02:28 ( 7.48 ms/it)\n objective: 13329.470498716539\nMinimizing 19803 Time: 0:02:28 ( 7.48 ms/it)\n objective: 13329.4678260597\nMinimizing 19818 Time: 0:02:28 ( 7.48 ms/it)\n objective: 13329.465515735588\nMinimizing 19833 Time: 0:02:28 ( 7.48 ms/it)\n objective: 13329.45995635241\nMinimizing 19848 Time: 0:02:28 ( 7.48 ms/it)\n objective: 13329.455773567352\nMinimizing 19863 Time: 0:02:28 ( 7.48 ms/it)\n objective: 13329.448911258849\nMinimizing 19878 Time: 0:02:28 ( 7.48 ms/it)\n objective: 13329.436440422476\nMinimizing 19893 Time: 0:02:28 ( 7.48 ms/it)\n objective: 13329.426360743557\nMinimizing 19908 Time: 0:02:28 ( 7.48 ms/it)\n objective: 13329.352069562941\nMinimizing 19923 Time: 0:02:29 ( 7.48 ms/it)\n objective: 13329.347344352376\nMinimizing 19938 Time: 0:02:29 ( 7.48 ms/it)\n objective: 13329.34617265124\nMinimizing 19953 Time: 0:02:29 ( 7.48 ms/it)\n objective: 13329.34246769736\nMinimizing 19968 Time: 0:02:29 ( 7.48 ms/it)\n objective: 13329.33301840727\nMinimizing 19983 Time: 0:02:29 ( 7.48 ms/it)\n objective: 13329.325547087668\nMinimizing 19998 Time: 0:02:29 ( 7.48 ms/it)\n objective: 13329.297596178694\nMinimizing 20013 Time: 0:02:29 ( 7.48 ms/it)\n objective: 13329.266890222869\nMinimizing 20028 Time: 0:02:29 ( 7.48 ms/it)\n objective: 13329.262394117955\nMinimizing 20043 Time: 0:02:29 ( 7.47 ms/it)\n objective: 13329.25246840615\nMinimizing 20058 Time: 0:02:29 ( 7.47 ms/it)\n objective: 13329.247770604656\nMinimizing 20073 Time: 0:02:30 ( 7.47 ms/it)\n objective: 13329.20921632585\nMinimizing 20088 Time: 0:02:30 ( 7.47 ms/it)\n objective: 13329.166166704345\nMinimizing 20103 Time: 0:02:30 ( 7.47 ms/it)\n objective: 13329.163100854916\nMinimizing 20118 Time: 0:02:30 ( 7.47 ms/it)\n objective: 13329.155168256664\nMinimizing 20133 Time: 0:02:30 ( 7.47 ms/it)\n objective: 13329.144308616567\nMinimizing 20148 Time: 0:02:30 ( 7.47 ms/it)\n objective: 13329.120819068099\nMinimizing 20163 Time: 0:02:30 ( 7.47 ms/it)\n objective: 13329.10461898391\nMinimizing 20178 Time: 0:02:30 ( 7.47 ms/it)\n objective: 13329.101833808185\nMinimizing 20193 Time: 0:02:30 ( 7.47 ms/it)\n objective: 13329.0721810158\nMinimizing 20208 Time: 0:02:30 ( 7.47 ms/it)\n objective: 13329.071358179608\nMinimizing 20223 Time: 0:02:31 ( 7.47 ms/it)\n objective: 13329.058252399627\nMinimizing 20238 Time: 0:02:31 ( 7.47 ms/it)\n objective: 13329.045935743467\nMinimizing 20253 Time: 0:02:31 ( 7.47 ms/it)\n objective: 13328.997317338435\nMinimizing 20268 Time: 0:02:31 ( 7.47 ms/it)\n objective: 13328.927473802381\nMinimizing 20283 Time: 0:02:31 ( 7.47 ms/it)\n objective: 13328.92687589582\nMinimizing 20298 Time: 0:02:31 ( 7.47 ms/it)\n objective: 13328.921026396762\nMinimizing 20313 Time: 0:02:31 ( 7.47 ms/it)\n objective: 13328.913358183323\nMinimizing 20328 Time: 0:02:31 ( 7.47 ms/it)\n objective: 13328.909590178911\nMinimizing 20343 Time: 0:02:31 ( 7.47 ms/it)\n objective: 13328.902042924135\nMinimizing 20358 Time: 0:02:31 ( 7.47 ms/it)\n objective: 13328.893171037555\nMinimizing 20373 Time: 0:02:32 ( 7.46 ms/it)\n objective: 13328.884132216313\nMinimizing 20388 Time: 0:02:32 ( 7.46 ms/it)\n objective: 13328.822744374469\nMinimizing 20403 Time: 0:02:32 ( 7.46 ms/it)\n objective: 13328.816520422188\nMinimizing 20418 Time: 0:02:32 ( 7.46 ms/it)\n objective: 13328.815997290687\nMinimizing 20433 Time: 0:02:32 ( 7.46 ms/it)\n objective: 13328.815274382294\nMinimizing 20448 Time: 0:02:32 ( 7.46 ms/it)\n objective: 13328.807729363056\nMinimizing 20463 Time: 0:02:32 ( 7.46 ms/it)\n objective: 13328.797776250714\nMinimizing 20478 Time: 0:02:32 ( 7.46 ms/it)\n objective: 13328.79612250832\nMinimizing 20493 Time: 0:02:32 ( 7.46 ms/it)\n objective: 13328.789174281701\nMinimizing 20508 Time: 0:02:33 ( 7.46 ms/it)\n objective: 13328.782602541985\nMinimizing 20523 Time: 0:02:33 ( 7.46 ms/it)\n objective: 13328.75391608329\nMinimizing 20538 Time: 0:02:33 ( 7.46 ms/it)\n objective: 13328.745396715341\nMinimizing 20553 Time: 0:02:33 ( 7.46 ms/it)\n objective: 13328.74368186434\nMinimizing 20568 Time: 0:02:33 ( 7.46 ms/it)\n objective: 13328.724947656156\nMinimizing 20583 Time: 0:02:33 ( 7.46 ms/it)\n objective: 13328.719728334865\nMinimizing 20598 Time: 0:02:33 ( 7.46 ms/it)\n objective: 13328.702692766936\nMinimizing 20613 Time: 0:02:33 ( 7.46 ms/it)\n objective: 13328.696137179315\nMinimizing 20628 Time: 0:02:33 ( 7.46 ms/it)\n objective: 13328.647686955097\nMinimizing 20643 Time: 0:02:33 ( 7.46 ms/it)\n objective: 13328.613671381274\nMinimizing 20658 Time: 0:02:34 ( 7.46 ms/it)\n objective: 13328.61181658777\nMinimizing 20673 Time: 0:02:34 ( 7.46 ms/it)\n objective: 13328.57300651681\nMinimizing 20688 Time: 0:02:34 ( 7.46 ms/it)\n objective: 13328.563083097935\nMinimizing 20703 Time: 0:02:34 ( 7.46 ms/it)\n objective: 13328.562512474178\nMinimizing 20718 Time: 0:02:34 ( 7.46 ms/it)\n objective: 13328.556037998474\nMinimizing 20733 Time: 0:02:34 ( 7.45 ms/it)\n objective: 13328.552198662284\nMinimizing 20748 Time: 0:02:34 ( 7.45 ms/it)\n objective: 13328.540902033885\nMinimizing 20763 Time: 0:02:34 ( 7.45 ms/it)\n objective: 13328.540337993203\nMinimizing 20777 Time: 0:02:34 ( 7.45 ms/it)\n objective: 13328.539066489924\nMinimizing 20791 Time: 0:02:34 ( 7.45 ms/it)\n objective: 13328.536263043956\nMinimizing 20806 Time: 0:02:35 ( 7.45 ms/it)\n objective: 13328.527693420467\nMinimizing 20821 Time: 0:02:35 ( 7.45 ms/it)\n objective: 13328.509336638373\nMinimizing 20836 Time: 0:02:35 ( 7.45 ms/it)\n objective: 13328.50008276588\nMinimizing 20851 Time: 0:02:35 ( 7.45 ms/it)\n objective: 13328.497502201288\nMinimizing 20866 Time: 0:02:35 ( 7.45 ms/it)\n objective: 13328.48402564686\nMinimizing 20880 Time: 0:02:35 ( 7.45 ms/it)\n objective: 13328.479154114124\nMinimizing 20894 Time: 0:02:35 ( 7.45 ms/it)\n objective: 13328.468692663213\nMinimizing 20908 Time: 0:02:35 ( 7.45 ms/it)\n objective: 13328.467854868082\nMinimizing 20922 Time: 0:02:35 ( 7.45 ms/it)\n objective: 13328.465649320882\nMinimizing 20936 Time: 0:02:36 ( 7.45 ms/it)\n objective: 13328.459417663093\nMinimizing 20950 Time: 0:02:36 ( 7.45 ms/it)\n objective: 13328.451719262754\nMinimizing 20964 Time: 0:02:36 ( 7.45 ms/it)\n objective: 13328.433648257269\nMinimizing 20978 Time: 0:02:36 ( 7.45 ms/it)\n objective: 13328.43263866804\nMinimizing 20992 Time: 0:02:36 ( 7.45 ms/it)\n objective: 13328.423442030493\nMinimizing 21006 Time: 0:02:36 ( 7.45 ms/it)\n objective: 13328.402658757783\nMinimizing 21020 Time: 0:02:36 ( 7.45 ms/it)\n objective: 13328.380116310786\nMinimizing 21034 Time: 0:02:36 ( 7.45 ms/it)\n objective: 13328.379635922072\nMinimizing 21048 Time: 0:02:36 ( 7.45 ms/it)\n objective: 13328.374140634354\nMinimizing 21062 Time: 0:02:36 ( 7.45 ms/it)\n objective: 13328.368476607291\nMinimizing 21076 Time: 0:02:37 ( 7.45 ms/it)\n objective: 13328.35838525051\nMinimizing 21090 Time: 0:02:37 ( 7.45 ms/it)\n objective: 13328.343381699531\nMinimizing 21104 Time: 0:02:37 ( 7.45 ms/it)\n objective: 13328.339691008441\nMinimizing 21118 Time: 0:02:37 ( 7.45 ms/it)\n objective: 13328.321410691162\nMinimizing 21132 Time: 0:02:37 ( 7.45 ms/it)\n objective: 13328.318806762189\nMinimizing 21146 Time: 0:02:37 ( 7.45 ms/it)\n objective: 13328.308190752672\nMinimizing 21160 Time: 0:02:37 ( 7.45 ms/it)\n objective: 13328.302914141444\nMinimizing 21174 Time: 0:02:37 ( 7.45 ms/it)\n objective: 13328.275662781438\nMinimizing 21188 Time: 0:02:37 ( 7.45 ms/it)\n objective: 13328.267332250558\nMinimizing 21202 Time: 0:02:37 ( 7.45 ms/it)\n objective: 13328.267030443574\nMinimizing 21216 Time: 0:02:38 ( 7.45 ms/it)\n objective: 13328.26497987088\nMinimizing 21230 Time: 0:02:38 ( 7.45 ms/it)\n objective: 13328.262967928327\nMinimizing 21244 Time: 0:02:38 ( 7.45 ms/it)\n objective: 13328.258266324825\nMinimizing 21258 Time: 0:02:38 ( 7.45 ms/it)\n objective: 13328.237147518405\nMinimizing 21272 Time: 0:02:38 ( 7.45 ms/it)\n objective: 13328.231564114918\nMinimizing 21286 Time: 0:02:38 ( 7.45 ms/it)\n objective: 13328.230363839379\nMinimizing 21300 Time: 0:02:38 ( 7.45 ms/it)\n objective: 13328.219418272914\nMinimizing 21314 Time: 0:02:38 ( 7.45 ms/it)\n objective: 13328.214686666295\nMinimizing 21328 Time: 0:02:38 ( 7.45 ms/it)\n objective: 13328.329554605742\nMinimizing 21342 Time: 0:02:39 ( 7.45 ms/it)\n objective: 13328.129434561284\nMinimizing 21356 Time: 0:02:39 ( 7.45 ms/it)\n objective: 13328.058548558576\nMinimizing 21370 Time: 0:02:39 ( 7.45 ms/it)\n objective: 13328.030552163902\nMinimizing 21384 Time: 0:02:39 ( 7.45 ms/it)\n objective: 13328.030064496554\nMinimizing 21398 Time: 0:02:39 ( 7.45 ms/it)\n objective: 13328.019692689646\nMinimizing 21413 Time: 0:02:39 ( 7.45 ms/it)\n objective: 13328.016799902507\nMinimizing 21428 Time: 0:02:39 ( 7.45 ms/it)\n objective: 13328.011096218768\nMinimizing 21442 Time: 0:02:39 ( 7.45 ms/it)\n objective: 13328.004205592631\nMinimizing 21457 Time: 0:02:39 ( 7.45 ms/it)\n objective: 13327.990796425882\nMinimizing 21472 Time: 0:02:39 ( 7.45 ms/it)\n objective: 13327.923040778485\nMinimizing 21487 Time: 0:02:40 ( 7.45 ms/it)\n objective: 13327.802714455902\nMinimizing 21502 Time: 0:02:40 ( 7.45 ms/it)\n objective: 13327.80199397778\nMinimizing 21517 Time: 0:02:40 ( 7.45 ms/it)\n objective: 13327.801556733619\nMinimizing 21532 Time: 0:02:40 ( 7.45 ms/it)\n objective: 13327.799584313805\nMinimizing 21547 Time: 0:02:40 ( 7.45 ms/it)\n objective: 13327.796538176422\nMinimizing 21563 Time: 0:02:40 ( 7.45 ms/it)\n objective: 13327.792956428872\nMinimizing 21578 Time: 0:02:40 ( 7.45 ms/it)\n objective: 13327.782168812788\nMinimizing 21593 Time: 0:02:40 ( 7.45 ms/it)\n objective: 13327.76420606087\nMinimizing 21608 Time: 0:02:40 ( 7.45 ms/it)\n objective: 13327.756326261602\nMinimizing 21623 Time: 0:02:41 ( 7.45 ms/it)\n objective: 13327.742099629606\nMinimizing 21638 Time: 0:02:41 ( 7.45 ms/it)\n objective: 13327.731609365132\nMinimizing 21653 Time: 0:02:41 ( 7.45 ms/it)\n objective: 13327.691137984082\nMinimizing 21668 Time: 0:02:41 ( 7.44 ms/it)\n objective: 13327.68437747578\nMinimizing 21683 Time: 0:02:41 ( 7.44 ms/it)\n objective: 13327.683860725629\nMinimizing 21698 Time: 0:02:41 ( 7.44 ms/it)\n objective: 13327.681427554002\nMinimizing 21712 Time: 0:02:41 ( 7.44 ms/it)\n objective: 13327.676566439797\nMinimizing 21726 Time: 0:02:41 ( 7.44 ms/it)\n objective: 13327.671769973153\nMinimizing 21740 Time: 0:02:41 ( 7.44 ms/it)\n objective: 13327.668825924484\nMinimizing 21753 Time: 0:02:41 ( 7.44 ms/it)\n objective: 13327.634667787177\nMinimizing 21768 Time: 0:02:42 ( 7.44 ms/it)\n objective: 13327.555445089332\nMinimizing 21783 Time: 0:02:42 ( 7.44 ms/it)\n objective: 13327.488345760896\nMinimizing 21798 Time: 0:02:42 ( 7.44 ms/it)\n objective: 13327.483523630784\nMinimizing 21813 Time: 0:02:42 ( 7.44 ms/it)\n objective: 13327.47910430107\nMinimizing 21828 Time: 0:02:42 ( 7.44 ms/it)\n objective: 13327.47512143894\nMinimizing 21843 Time: 0:02:42 ( 7.44 ms/it)\n objective: 13327.463279945252\nMinimizing 21858 Time: 0:02:42 ( 7.44 ms/it)\n objective: 13327.45415418852\nMinimizing 21873 Time: 0:02:42 ( 7.44 ms/it)\n objective: 13327.447017111721\nMinimizing 21888 Time: 0:02:42 ( 7.44 ms/it)\n objective: 13327.446229432302\nMinimizing 21903 Time: 0:02:42 ( 7.44 ms/it)\n objective: 13327.438987079368\nMinimizing 21918 Time: 0:02:43 ( 7.44 ms/it)\n objective: 13327.429711103207\nMinimizing 21933 Time: 0:02:43 ( 7.44 ms/it)\n objective: 13327.42328607491\nMinimizing 21948 Time: 0:02:43 ( 7.44 ms/it)\n objective: 13327.420143696254\nMinimizing 21963 Time: 0:02:43 ( 7.44 ms/it)\n objective: 13327.400751692949\nMinimizing 21978 Time: 0:02:43 ( 7.44 ms/it)\n objective: 13327.362244055315\nMinimizing 21993 Time: 0:02:43 ( 7.44 ms/it)\n objective: 13327.31157680729\nMinimizing 22008 Time: 0:02:43 ( 7.44 ms/it)\n objective: 13327.25999445426\nMinimizing 22023 Time: 0:02:43 ( 7.44 ms/it)\n objective: 13327.25922332704\nMinimizing 22038 Time: 0:02:43 ( 7.44 ms/it)\n objective: 13327.2462541735\nMinimizing 22053 Time: 0:02:44 ( 7.44 ms/it)\n objective: 13327.234894800225\nMinimizing 22068 Time: 0:02:44 ( 7.44 ms/it)\n objective: 13327.224112585216\nMinimizing 22083 Time: 0:02:44 ( 7.44 ms/it)\n objective: 13327.223133419866\nMinimizing 22098 Time: 0:02:44 ( 7.44 ms/it)\n objective: 13327.210676335919\nMinimizing 22113 Time: 0:02:44 ( 7.44 ms/it)\n objective: 13327.20305388247\nMinimizing 22128 Time: 0:02:44 ( 7.44 ms/it)\n objective: 13327.19294880733\nMinimizing 22143 Time: 0:02:44 ( 7.44 ms/it)\n objective: 13327.178508058394\nMinimizing 22158 Time: 0:02:44 ( 7.44 ms/it)\n objective: 13327.06178661278\nMinimizing 22173 Time: 0:02:44 ( 7.44 ms/it)\n objective: 13326.98340218766\nMinimizing 22188 Time: 0:02:44 ( 7.44 ms/it)\n objective: 13326.981887870861\nMinimizing 22203 Time: 0:02:45 ( 7.43 ms/it)\n objective: 13326.978497942364\nMinimizing 22218 Time: 0:02:45 ( 7.43 ms/it)\n objective: 13326.974235244918\nMinimizing 22233 Time: 0:02:45 ( 7.43 ms/it)\n objective: 13326.973542317428\nMinimizing 22248 Time: 0:02:45 ( 7.43 ms/it)\n objective: 13326.969457978688\nMinimizing 22263 Time: 0:02:45 ( 7.43 ms/it)\n objective: 13326.962927732857\nMinimizing 22278 Time: 0:02:45 ( 7.43 ms/it)\n objective: 13326.960001248939\nMinimizing 22293 Time: 0:02:45 ( 7.43 ms/it)\n objective: 13326.946917483125\nMinimizing 22308 Time: 0:02:45 ( 7.43 ms/it)\n objective: 13326.824894673482\nMinimizing 22323 Time: 0:02:45 ( 7.43 ms/it)\n objective: 13326.806908089908\nMinimizing 22338 Time: 0:02:46 ( 7.43 ms/it)\n objective: 13326.802424988018\nMinimizing 22353 Time: 0:02:46 ( 7.43 ms/it)\n objective: 13326.79990728727\nMinimizing 22368 Time: 0:02:46 ( 7.43 ms/it)\n objective: 13326.789041377328\nMinimizing 22383 Time: 0:02:46 ( 7.43 ms/it)\n objective: 13326.78609542313\nMinimizing 22398 Time: 0:02:46 ( 7.43 ms/it)\n objective: 13326.785530430388\nMinimizing 22413 Time: 0:02:46 ( 7.43 ms/it)\n objective: 13326.7803489203\nMinimizing 22428 Time: 0:02:46 ( 7.43 ms/it)\n objective: 13326.767347653018\nMinimizing 22443 Time: 0:02:46 ( 7.43 ms/it)\n objective: 13326.765994263114\nMinimizing 22457 Time: 0:02:46 ( 7.43 ms/it)\n objective: 13326.714850444558\nMinimizing 22472 Time: 0:02:46 ( 7.43 ms/it)\n objective: 13326.691845574547\nMinimizing 22487 Time: 0:02:47 ( 7.43 ms/it)\n objective: 13326.691346704683\nMinimizing 22502 Time: 0:02:47 ( 7.43 ms/it)\n objective: 13326.689862591105\nMinimizing 22517 Time: 0:02:47 ( 7.43 ms/it)\n objective: 13326.688520467971\nMinimizing 22532 Time: 0:02:47 ( 7.43 ms/it)\n objective: 13326.68485120796\nMinimizing 22547 Time: 0:02:47 ( 7.43 ms/it)\n objective: 13326.679527234563\nMinimizing 22562 Time: 0:02:47 ( 7.43 ms/it)\n objective: 13326.65707452214\nMinimizing 22577 Time: 0:02:47 ( 7.43 ms/it)\n objective: 13326.629409959925\nMinimizing 22592 Time: 0:02:47 ( 7.43 ms/it)\n objective: 13326.628672901024\nMinimizing 22607 Time: 0:02:47 ( 7.43 ms/it)\n objective: 13326.616007351273\nMinimizing 22622 Time: 0:02:48 ( 7.43 ms/it)\n objective: 13326.605576266433\nMinimizing 22637 Time: 0:02:48 ( 7.43 ms/it)\n objective: 13326.603204174622\nMinimizing 22652 Time: 0:02:48 ( 7.43 ms/it)\n objective: 13326.58620426726\nMinimizing 22667 Time: 0:02:48 ( 7.43 ms/it)\n objective: 13326.585748073667\nMinimizing 22682 Time: 0:02:48 ( 7.42 ms/it)\n objective: 13326.579198923508\nMinimizing 22697 Time: 0:02:48 ( 7.42 ms/it)\n objective: 13326.57213118158\nMinimizing 22713 Time: 0:02:48 ( 7.42 ms/it)\n objective: 13326.556858142758\nMinimizing 22728 Time: 0:02:48 ( 7.42 ms/it)\n objective: 13326.501577811192\nMinimizing 22743 Time: 0:02:48 ( 7.42 ms/it)\n objective: 13326.493334192353\nMinimizing 22758 Time: 0:02:48 ( 7.42 ms/it)\n objective: 13326.485613655852\nMinimizing 22773 Time: 0:02:49 ( 7.42 ms/it)\n objective: 13326.47581872304\nMinimizing 22788 Time: 0:02:49 ( 7.42 ms/it)\n objective: 13326.473547372152\nMinimizing 22803 Time: 0:02:49 ( 7.42 ms/it)\n objective: 13326.469559979545\nMinimizing 22818 Time: 0:02:49 ( 7.42 ms/it)\n objective: 13326.414714141778\nMinimizing 22832 Time: 0:02:49 ( 7.42 ms/it)\n objective: 13326.389299300594\nMinimizing 22846 Time: 0:02:49 ( 7.42 ms/it)\n objective: 13326.388618226614\nMinimizing 22860 Time: 0:02:49 ( 7.42 ms/it)\n objective: 13326.384338204836\nMinimizing 22874 Time: 0:02:49 ( 7.42 ms/it)\n objective: 13326.370478796729\nMinimizing 22888 Time: 0:02:49 ( 7.42 ms/it)\n objective: 13326.314260219879\nMinimizing 22903 Time: 0:02:49 ( 7.42 ms/it)\n objective: 13326.205822746946\nMinimizing 22918 Time: 0:02:50 ( 7.42 ms/it)\n objective: 13326.20143877827\nMinimizing 22933 Time: 0:02:50 ( 7.42 ms/it)\n objective: 13326.196282373894\nMinimizing 22948 Time: 0:02:50 ( 7.42 ms/it)\n objective: 13326.18270598077\nMinimizing 22963 Time: 0:02:50 ( 7.42 ms/it)\n objective: 13326.160977791122\nMinimizing 22978 Time: 0:02:50 ( 7.42 ms/it)\n objective: 13326.135123221378\nMinimizing 22993 Time: 0:02:50 ( 7.42 ms/it)\n objective: 13325.980681415851\nMinimizing 23008 Time: 0:02:50 ( 7.42 ms/it)\n objective: 13325.975564473425\nMinimizing 23023 Time: 0:02:50 ( 7.42 ms/it)\n objective: 13325.973374115936\nMinimizing 23038 Time: 0:02:50 ( 7.42 ms/it)\n objective: 13325.967804808824\nMinimizing 23052 Time: 0:02:50 ( 7.42 ms/it)\n objective: 13325.964185826357\nMinimizing 23067 Time: 0:02:51 ( 7.42 ms/it)\n objective: 13325.955189713743\nMinimizing 23082 Time: 0:02:51 ( 7.42 ms/it)\n objective: 13325.943982802688\nMinimizing 23097 Time: 0:02:51 ( 7.42 ms/it)\n objective: 13325.941626119842\nMinimizing 23111 Time: 0:02:51 ( 7.42 ms/it)\n objective: 13325.92404661802\nMinimizing 23126 Time: 0:02:51 ( 7.42 ms/it)\n objective: 13325.92209266215\nMinimizing 23141 Time: 0:02:51 ( 7.42 ms/it)\n objective: 13325.8994712145\nMinimizing 23156 Time: 0:02:51 ( 7.42 ms/it)\n objective: 13325.897759617335\nMinimizing 23171 Time: 0:02:51 ( 7.42 ms/it)\n objective: 13325.894248749399\nMinimizing 23186 Time: 0:02:51 ( 7.42 ms/it)\n objective: 13325.890072594048\nMinimizing 23201 Time: 0:02:52 ( 7.42 ms/it)\n objective: 13325.888626762615\nMinimizing 23216 Time: 0:02:52 ( 7.41 ms/it)\n objective: 13325.877981894148\nMinimizing 23231 Time: 0:02:52 ( 7.41 ms/it)\n objective: 13325.860686741304\nMinimizing 23246 Time: 0:02:52 ( 7.41 ms/it)\n objective: 13325.843581273628\nMinimizing 23261 Time: 0:02:52 ( 7.41 ms/it)\n objective: 13325.841605235531\nMinimizing 23276 Time: 0:02:52 ( 7.41 ms/it)\n objective: 13325.816536874867\nMinimizing 23291 Time: 0:02:52 ( 7.41 ms/it)\n objective: 13325.815848166232\nMinimizing 23306 Time: 0:02:52 ( 7.41 ms/it)\n objective: 13325.80247579953\nMinimizing 23321 Time: 0:02:52 ( 7.41 ms/it)\n objective: 13325.782456923072\nMinimizing 23336 Time: 0:02:52 ( 7.41 ms/it)\n objective: 13325.720044661815\nMinimizing 23351 Time: 0:02:53 ( 7.41 ms/it)\n objective: 13325.710170793784\nMinimizing 23366 Time: 0:02:53 ( 7.41 ms/it)\n objective: 13325.709577160946\nMinimizing 23381 Time: 0:02:53 ( 7.41 ms/it)\n objective: 13325.707372138211\nMinimizing 23396 Time: 0:02:53 ( 7.41 ms/it)\n objective: 13325.703062869128\nMinimizing 23410 Time: 0:02:53 ( 7.41 ms/it)\n objective: 13325.701521256255\nMinimizing 23424 Time: 0:02:53 ( 7.41 ms/it)\n objective: 13325.693897443103\nMinimizing 23438 Time: 0:02:53 ( 7.41 ms/it)\n objective: 13325.692140432984\nMinimizing 23452 Time: 0:02:53 ( 7.41 ms/it)\n objective: 13325.674455428962\nMinimizing 23466 Time: 0:02:53 ( 7.41 ms/it)\n objective: 13325.667299645087\nMinimizing 23480 Time: 0:02:54 ( 7.41 ms/it)\n objective: 13325.658870860054\nMinimizing 23495 Time: 0:02:54 ( 7.41 ms/it)\n objective: 13325.6497698409\nMinimizing 23510 Time: 0:02:54 ( 7.41 ms/it)\n objective: 13325.641894808949\nMinimizing 23525 Time: 0:02:54 ( 7.41 ms/it)\n objective: 13325.617560442712\nMinimizing 23540 Time: 0:02:54 ( 7.41 ms/it)\n objective: 13325.616688481437\nMinimizing 23555 Time: 0:02:54 ( 7.41 ms/it)\n objective: 13325.610835894855\nMinimizing 23570 Time: 0:02:54 ( 7.41 ms/it)\n objective: 13325.60775627155\nMinimizing 23585 Time: 0:02:54 ( 7.41 ms/it)\n objective: 13325.595576413907\nMinimizing 23600 Time: 0:02:54 ( 7.41 ms/it)\n objective: 13325.591879340587\nMinimizing 23614 Time: 0:02:54 ( 7.41 ms/it)\n objective: 13325.574613286524\nMinimizing 23628 Time: 0:02:55 ( 7.41 ms/it)\n objective: 13325.569888214362\nMinimizing 23642 Time: 0:02:55 ( 7.41 ms/it)\n objective: 13325.556328988969\nMinimizing 23656 Time: 0:02:55 ( 7.41 ms/it)\n objective: 13325.531548377927\nMinimizing 23671 Time: 0:02:55 ( 7.41 ms/it)\n objective: 13325.513682343044\nMinimizing 23686 Time: 0:02:55 ( 7.41 ms/it)\n objective: 13325.379046877395\nMinimizing 23701 Time: 0:02:55 ( 7.41 ms/it)\n objective: 13325.374777401761\nMinimizing 23716 Time: 0:02:55 ( 7.41 ms/it)\n objective: 13325.37213175586\nMinimizing 23731 Time: 0:02:55 ( 7.41 ms/it)\n objective: 13325.365311146343\nMinimizing 23746 Time: 0:02:55 ( 7.41 ms/it)\n objective: 13325.352803217691\nMinimizing 23757 Time: 0:02:56 ( 7.41 ms/it)\n objective: 13325.348705264383\nMinimizing 23772 Time: 0:02:56 ( 7.41 ms/it)\n objective: 13325.345511018873\nMinimizing 23787 Time: 0:02:56 ( 7.41 ms/it)\n objective: 13325.333673063098\nMinimizing 23802 Time: 0:02:56 ( 7.41 ms/it)\n objective: 13325.326626249283\nMinimizing 23817 Time: 0:02:56 ( 7.41 ms/it)\n objective: 13325.280241778695\nMinimizing 23832 Time: 0:02:56 ( 7.41 ms/it)\n objective: 13325.183053750574\nMinimizing 23847 Time: 0:02:56 ( 7.41 ms/it)\n objective: 13325.171392884993\nMinimizing 23862 Time: 0:02:56 ( 7.41 ms/it)\n objective: 13325.164722867601\nMinimizing 23877 Time: 0:02:56 ( 7.41 ms/it)\n objective: 13325.14522989998\nMinimizing 23892 Time: 0:02:56 ( 7.41 ms/it)\n objective: 13325.108478127833\nMinimizing 23907 Time: 0:02:57 ( 7.41 ms/it)\n objective: 13325.096405844102\nMinimizing 23922 Time: 0:02:57 ( 7.41 ms/it)\n objective: 13325.095868863602\nMinimizing 23937 Time: 0:02:57 ( 7.41 ms/it)\n objective: 13325.094868936678\nMinimizing 23952 Time: 0:02:57 ( 7.41 ms/it)\n objective: 13325.094432796497\nMinimizing 23967 Time: 0:02:57 ( 7.40 ms/it)\n objective: 13325.090544140287\nMinimizing 23982 Time: 0:02:57 ( 7.40 ms/it)\n objective: 13325.088217323995\nMinimizing 23997 Time: 0:02:57 ( 7.40 ms/it)\n objective: 13325.07502238761\nMinimizing 24012 Time: 0:02:57 ( 7.40 ms/it)\n objective: 13325.072326211899\nMinimizing 24027 Time: 0:02:57 ( 7.40 ms/it)\n objective: 13325.07069916978\nMinimizing 24042 Time: 0:02:57 ( 7.40 ms/it)\n objective: 13325.068001290172\nMinimizing 24057 Time: 0:02:58 ( 7.40 ms/it)\n objective: 13325.061703774802\nMinimizing 24072 Time: 0:02:58 ( 7.40 ms/it)\n objective: 13325.008824948964\nMinimizing 24087 Time: 0:02:58 ( 7.40 ms/it)\n objective: 13324.904550778054\nMinimizing 24102 Time: 0:02:58 ( 7.40 ms/it)\n objective: 13324.90324775927\nMinimizing 24117 Time: 0:02:58 ( 7.40 ms/it)\n objective: 13324.90288263165\nMinimizing 24132 Time: 0:02:58 ( 7.40 ms/it)\n objective: 13324.902287951147\nMinimizing 24147 Time: 0:02:58 ( 7.40 ms/it)\n objective: 13324.902033610633\nMinimizing 24162 Time: 0:02:58 ( 7.40 ms/it)\n objective: 13324.900693161544\nMinimizing 24177 Time: 0:02:58 ( 7.40 ms/it)\n objective: 13324.900147081731\nMinimizing 24192 Time: 0:02:59 ( 7.40 ms/it)\n objective: 13324.89009152702\nMinimizing 24207 Time: 0:02:59 ( 7.40 ms/it)\n objective: 13324.887641981564\nMinimizing 24222 Time: 0:02:59 ( 7.40 ms/it)\n objective: 13324.887168602057\nMinimizing 24237 Time: 0:02:59 ( 7.40 ms/it)\n objective: 13324.885237294933\nMinimizing 24252 Time: 0:02:59 ( 7.40 ms/it)\n objective: 13324.883043830414\nMinimizing 24267 Time: 0:02:59 ( 7.40 ms/it)\n objective: 13324.877744016267\nMinimizing 24282 Time: 0:02:59 ( 7.40 ms/it)\n objective: 13324.872394857492\nMinimizing 24297 Time: 0:02:59 ( 7.40 ms/it)\n objective: 13324.869976458896\nMinimizing 24312 Time: 0:02:59 ( 7.40 ms/it)\n objective: 13324.856658037606\nMinimizing 24327 Time: 0:02:59 ( 7.40 ms/it)\n objective: 13324.842168211311\nMinimizing 24342 Time: 0:03:00 ( 7.40 ms/it)\n objective: 13324.841681454971\nMinimizing 24357 Time: 0:03:00 ( 7.40 ms/it)\n objective: 13324.83994435682\nMinimizing 24372 Time: 0:03:00 ( 7.40 ms/it)\n objective: 13324.838790802693\nMinimizing 24387 Time: 0:03:00 ( 7.40 ms/it)\n objective: 13324.830820353658\nMinimizing 24402 Time: 0:03:00 ( 7.40 ms/it)\n objective: 13324.829414578591\nMinimizing 24417 Time: 0:03:00 ( 7.40 ms/it)\n objective: 13324.824027084105\nMinimizing 24432 Time: 0:03:00 ( 7.40 ms/it)\n objective: 13324.81773927102\nMinimizing 24447 Time: 0:03:00 ( 7.40 ms/it)\n objective: 13324.81669238946\nMinimizing 24462 Time: 0:03:00 ( 7.40 ms/it)\n objective: 13324.803258516695\nMinimizing 24477 Time: 0:03:01 ( 7.40 ms/it)\n objective: 13324.802185734923\nMinimizing 24492 Time: 0:03:01 ( 7.40 ms/it)\n objective: 13324.798339415298\nMinimizing 24507 Time: 0:03:01 ( 7.40 ms/it)\n objective: 13324.791289974397\nMinimizing 24522 Time: 0:03:01 ( 7.40 ms/it)\n objective: 13324.77044996353\nMinimizing 24537 Time: 0:03:01 ( 7.40 ms/it)\n objective: 13324.764464140477\nMinimizing 24552 Time: 0:03:01 ( 7.39 ms/it)\n objective: 13324.730256418421\nMinimizing 24567 Time: 0:03:01 ( 7.39 ms/it)\n objective: 13326.022418661785\nMinimizing 24582 Time: 0:03:01 ( 7.39 ms/it)\n objective: 13324.153400086827\nMinimizing 24597 Time: 0:03:01 ( 7.39 ms/it)\n objective: 13324.150538522881\nMinimizing 24612 Time: 0:03:01 ( 7.39 ms/it)\n objective: 13324.146447345862\nMinimizing 24627 Time: 0:03:02 ( 7.39 ms/it)\n objective: 13324.14104121388\nMinimizing 24642 Time: 0:03:02 ( 7.39 ms/it)\n objective: 13324.12903281604\nMinimizing 24657 Time: 0:03:02 ( 7.39 ms/it)\n objective: 13324.12710478298\nMinimizing 24672 Time: 0:03:02 ( 7.39 ms/it)\n objective: 13324.104210816498\nMinimizing 24687 Time: 0:03:02 ( 7.39 ms/it)\n objective: 13324.102294374534\nMinimizing 24702 Time: 0:03:02 ( 7.39 ms/it)\n objective: 13324.013521233443\nMinimizing 24717 Time: 0:03:02 ( 7.39 ms/it)\n objective: 13324.012767639535\nMinimizing 24732 Time: 0:03:02 ( 7.39 ms/it)\n objective: 13323.999600773197\nMinimizing 24747 Time: 0:03:02 ( 7.39 ms/it)\n objective: 13323.973975600384\nMinimizing 24762 Time: 0:03:02 ( 7.39 ms/it)\n objective: 13323.953986663531\nMinimizing 24777 Time: 0:03:03 ( 7.39 ms/it)\n objective: 13323.951145742278\nMinimizing 24792 Time: 0:03:03 ( 7.39 ms/it)\n objective: 13323.933557569748\nMinimizing 24807 Time: 0:03:03 ( 7.39 ms/it)\n objective: 13323.926519979286\nMinimizing 24822 Time: 0:03:03 ( 7.39 ms/it)\n objective: 13323.909402831603\nMinimizing 24837 Time: 0:03:03 ( 7.39 ms/it)\n objective: 13323.908270431682\nMinimizing 24852 Time: 0:03:03 ( 7.39 ms/it)\n objective: 13323.87601469834\nMinimizing 24867 Time: 0:03:03 ( 7.39 ms/it)\n objective: 13323.84950467461\nMinimizing 24882 Time: 0:03:03 ( 7.39 ms/it)\n objective: 13323.847874241706\nMinimizing 24897 Time: 0:03:03 ( 7.39 ms/it)\n objective: 13323.837617182857\nMinimizing 24912 Time: 0:03:04 ( 7.39 ms/it)\n objective: 13323.818408620544\nMinimizing 24927 Time: 0:03:04 ( 7.39 ms/it)\n objective: 13323.815859885333\nMinimizing 24942 Time: 0:03:04 ( 7.39 ms/it)\n objective: 13323.795033596762\nMinimizing 24957 Time: 0:03:04 ( 7.39 ms/it)\n objective: 13323.793107847785\nMinimizing 24972 Time: 0:03:04 ( 7.39 ms/it)\n objective: 13323.786818467648\nMinimizing 24987 Time: 0:03:04 ( 7.38 ms/it)\n objective: 13323.767473702552\nMinimizing 25002 Time: 0:03:04 ( 7.38 ms/it)\n objective: 13323.754195074507\nMinimizing 25017 Time: 0:03:04 ( 7.38 ms/it)\n objective: 13323.74774496138\nMinimizing 25032 Time: 0:03:04 ( 7.38 ms/it)\n objective: 13323.745692061973\nMinimizing 25047 Time: 0:03:04 ( 7.38 ms/it)\n objective: 13323.714089051457\nMinimizing 25062 Time: 0:03:05 ( 7.38 ms/it)\n objective: 13323.713284750862\nMinimizing 25077 Time: 0:03:05 ( 7.38 ms/it)\n objective: 13323.70995258943\nMinimizing 25092 Time: 0:03:05 ( 7.38 ms/it)\n objective: 13323.707830206738\nMinimizing 25107 Time: 0:03:05 ( 7.38 ms/it)\n objective: 13323.695364852902\nMinimizing 25122 Time: 0:03:05 ( 7.38 ms/it)\n objective: 13323.69076560215\nMinimizing 25137 Time: 0:03:05 ( 7.38 ms/it)\n objective: 13323.666001855192\nMinimizing 25152 Time: 0:03:05 ( 7.38 ms/it)\n objective: 13323.659685598323\nMinimizing 25167 Time: 0:03:05 ( 7.38 ms/it)\n objective: 13323.657266785522\nMinimizing 25182 Time: 0:03:05 ( 7.38 ms/it)\n objective: 13323.641496908589\nMinimizing 25197 Time: 0:03:05 ( 7.38 ms/it)\n objective: 13323.638214224382\nMinimizing 25212 Time: 0:03:06 ( 7.38 ms/it)\n objective: 13323.628734145575\nMinimizing 25227 Time: 0:03:06 ( 7.38 ms/it)\n objective: 13323.600730227598\nMinimizing 25242 Time: 0:03:06 ( 7.38 ms/it)\n objective: 13323.588046129662\nMinimizing 25257 Time: 0:03:06 ( 7.38 ms/it)\n objective: 13323.579173321472\nMinimizing 25272 Time: 0:03:06 ( 7.38 ms/it)\n objective: 13323.452811563227\nMinimizing 25287 Time: 0:03:06 ( 7.38 ms/it)\n objective: 13323.433882145197\nMinimizing 25302 Time: 0:03:06 ( 7.38 ms/it)\n objective: 13323.431114574167\nMinimizing 25317 Time: 0:03:06 ( 7.38 ms/it)\n objective: 13323.422939608135\nMinimizing 25332 Time: 0:03:06 ( 7.38 ms/it)\n objective: 13323.415516952664\nMinimizing 25347 Time: 0:03:07 ( 7.38 ms/it)\n objective: 13323.408141641325\nMinimizing 25362 Time: 0:03:07 ( 7.38 ms/it)\n objective: 13323.407468803576\nMinimizing 25377 Time: 0:03:07 ( 7.38 ms/it)\n objective: 13323.406963321642\nMinimizing 25392 Time: 0:03:07 ( 7.38 ms/it)\n objective: 13323.405562594504\nMinimizing 25407 Time: 0:03:07 ( 7.38 ms/it)\n objective: 13323.39598296152\nMinimizing 25422 Time: 0:03:07 ( 7.38 ms/it)\n objective: 13323.393840426506\nMinimizing 25437 Time: 0:03:07 ( 7.38 ms/it)\n objective: 13323.39183965289\nMinimizing 25452 Time: 0:03:07 ( 7.38 ms/it)\n objective: 13323.385370975171\nMinimizing 25467 Time: 0:03:07 ( 7.38 ms/it)\n objective: 13323.373981211902\nMinimizing 25481 Time: 0:03:07 ( 7.38 ms/it)\n objective: 13323.36608427098\nMinimizing 25496 Time: 0:03:08 ( 7.38 ms/it)\n objective: 13323.332715147466\nMinimizing 25511 Time: 0:03:08 ( 7.38 ms/it)\n objective: 13323.331194170503\nMinimizing 25526 Time: 0:03:08 ( 7.38 ms/it)\n objective: 13323.321211081711\nMinimizing 25541 Time: 0:03:08 ( 7.38 ms/it)\n objective: 13323.305323059656\nMinimizing 25556 Time: 0:03:08 ( 7.38 ms/it)\n objective: 13323.30428709458\nMinimizing 25571 Time: 0:03:08 ( 7.38 ms/it)\n objective: 13323.295611087058\nMinimizing 25586 Time: 0:03:08 ( 7.38 ms/it)\n objective: 13323.280049537338\nMinimizing 25601 Time: 0:03:08 ( 7.38 ms/it)\n objective: 13323.254074576238\nMinimizing 25615 Time: 0:03:08 ( 7.38 ms/it)\n objective: 13323.253141506138\nMinimizing 25629 Time: 0:03:09 ( 7.38 ms/it)\n objective: 13323.252279394059\nMinimizing 25644 Time: 0:03:09 ( 7.38 ms/it)\n objective: 13323.245402390792\nMinimizing 25659 Time: 0:03:09 ( 7.38 ms/it)\n objective: 13323.235658150064\nMinimizing 25674 Time: 0:03:09 ( 7.37 ms/it)\n objective: 13323.220348235525\nMinimizing 25689 Time: 0:03:09 ( 7.37 ms/it)\n objective: 13323.207788501983\nMinimizing 25704 Time: 0:03:09 ( 7.37 ms/it)\n objective: 13323.201768893297\nMinimizing 25719 Time: 0:03:09 ( 7.37 ms/it)\n objective: 13323.17476754119\nMinimizing 25734 Time: 0:03:09 ( 7.37 ms/it)\n objective: 13323.092335446432\nMinimizing 25748 Time: 0:03:09 ( 7.37 ms/it)\n objective: 13323.074852658014\nMinimizing 25762 Time: 0:03:09 ( 7.37 ms/it)\n objective: 13323.07417329667\nMinimizing 25776 Time: 0:03:10 ( 7.37 ms/it)\n objective: 13323.066033492622\nMinimizing 25790 Time: 0:03:10 ( 7.37 ms/it)\n objective: 13323.057276168867\nMinimizing 25804 Time: 0:03:10 ( 7.37 ms/it)\n objective: 13323.045625411774\nMinimizing 25818 Time: 0:03:10 ( 7.37 ms/it)\n objective: 13323.037920157803\nMinimizing 25832 Time: 0:03:10 ( 7.37 ms/it)\n objective: 13323.025756669085\nMinimizing 25846 Time: 0:03:10 ( 7.37 ms/it)\n objective: 13323.003811378163\nMinimizing 25860 Time: 0:03:10 ( 7.37 ms/it)\n objective: 13323.00200276084\nMinimizing 25874 Time: 0:03:10 ( 7.37 ms/it)\n objective: 13322.980983352521\nMinimizing 25888 Time: 0:03:10 ( 7.37 ms/it)\n objective: 13322.964120379023\nMinimizing 25902 Time: 0:03:11 ( 7.37 ms/it)\n objective: 13322.938493517868\nMinimizing 25916 Time: 0:03:11 ( 7.37 ms/it)\n objective: 13322.937282661922\nMinimizing 25930 Time: 0:03:11 ( 7.37 ms/it)\n objective: 13322.925165767592\nMinimizing 25944 Time: 0:03:11 ( 7.37 ms/it)\n objective: 13322.914264750201\nMinimizing 25958 Time: 0:03:11 ( 7.37 ms/it)\n objective: 13322.893767596834\nMinimizing 25972 Time: 0:03:11 ( 7.37 ms/it)\n objective: 13322.887569413055\nMinimizing 25986 Time: 0:03:11 ( 7.37 ms/it)\n objective: 13322.856335111152\nMinimizing 26000 Time: 0:03:11 ( 7.37 ms/it)\n objective: 13322.830045455325\nMinimizing 26014 Time: 0:03:11 ( 7.37 ms/it)\n objective: 13322.829677535032\nMinimizing 26028 Time: 0:03:11 ( 7.37 ms/it)\n objective: 13322.828527920763\nMinimizing 26042 Time: 0:03:12 ( 7.37 ms/it)\n objective: 13322.823237409917\nMinimizing 26056 Time: 0:03:12 ( 7.37 ms/it)\n objective: 13322.818195578366\nMinimizing 26070 Time: 0:03:12 ( 7.37 ms/it)\n objective: 13322.80875493215\nMinimizing 26084 Time: 0:03:12 ( 7.37 ms/it)\n objective: 13322.80598625989\nMinimizing 26098 Time: 0:03:12 ( 7.37 ms/it)\n objective: 13322.793072501387\nMinimizing 26112 Time: 0:03:12 ( 7.37 ms/it)\n objective: 13322.711680029097\nMinimizing 26126 Time: 0:03:12 ( 7.37 ms/it)\n objective: 13322.622893114225\nMinimizing 26140 Time: 0:03:12 ( 7.37 ms/it)\n objective: 13322.609232675371\nMinimizing 26154 Time: 0:03:12 ( 7.37 ms/it)\n objective: 13322.60323604694\nMinimizing 26168 Time: 0:03:12 ( 7.37 ms/it)\n objective: 13322.587258746164\nMinimizing 26182 Time: 0:03:13 ( 7.37 ms/it)\n objective: 13322.585943856888\nMinimizing 26197 Time: 0:03:13 ( 7.37 ms/it)\n objective: 13322.578621595632\nMinimizing 26212 Time: 0:03:13 ( 7.37 ms/it)\n objective: 13322.564742343791\nMinimizing 26227 Time: 0:03:13 ( 7.37 ms/it)\n objective: 13322.550643448398\nMinimizing 26242 Time: 0:03:13 ( 7.37 ms/it)\n objective: 13322.397103768744\nMinimizing 26257 Time: 0:03:13 ( 7.37 ms/it)\n objective: 13322.395288103522\nMinimizing 26272 Time: 0:03:13 ( 7.37 ms/it)\n objective: 13322.392920041893\nMinimizing 26287 Time: 0:03:13 ( 7.37 ms/it)\n objective: 13322.384667976541\nMinimizing 26302 Time: 0:03:13 ( 7.37 ms/it)\n objective: 13322.367273587166\nMinimizing 26317 Time: 0:03:13 ( 7.37 ms/it)\n objective: 13322.342793149131\nMinimizing 26332 Time: 0:03:14 ( 7.37 ms/it)\n objective: 13322.333035099044\nMinimizing 26347 Time: 0:03:14 ( 7.37 ms/it)\n objective: 13322.332014789368\nMinimizing 26362 Time: 0:03:14 ( 7.37 ms/it)\n objective: 13322.331486568233\nMinimizing 26377 Time: 0:03:14 ( 7.37 ms/it)\n objective: 13322.328950214986\nMinimizing 26392 Time: 0:03:14 ( 7.37 ms/it)\n objective: 13322.321527398293\nMinimizing 26407 Time: 0:03:14 ( 7.37 ms/it)\n objective: 13322.31607248084\nMinimizing 26422 Time: 0:03:14 ( 7.37 ms/it)\n objective: 13322.301771162834\nMinimizing 26437 Time: 0:03:14 ( 7.37 ms/it)\n objective: 13322.300793330141\nMinimizing 26452 Time: 0:03:14 ( 7.37 ms/it)\n objective: 13322.29982655574\nMinimizing 26467 Time: 0:03:15 ( 7.37 ms/it)\n objective: 13322.296720845508\nMinimizing 26482 Time: 0:03:15 ( 7.37 ms/it)\n objective: 13322.292709230489\nMinimizing 26497 Time: 0:03:15 ( 7.37 ms/it)\n objective: 13322.286665776453\nMinimizing 26512 Time: 0:03:15 ( 7.37 ms/it)\n objective: 13322.285265880768\nMinimizing 26527 Time: 0:03:15 ( 7.37 ms/it)\n objective: 13322.280546433743\nMinimizing 26542 Time: 0:03:15 ( 7.37 ms/it)\n objective: 13322.251513132229\nMinimizing 26557 Time: 0:03:15 ( 7.37 ms/it)\n objective: 13322.452238480808\nMinimizing 26572 Time: 0:03:15 ( 7.37 ms/it)\n objective: 13322.208876201577\nMinimizing 26587 Time: 0:03:15 ( 7.37 ms/it)\n objective: 13322.205290361206\nMinimizing 26602 Time: 0:03:15 ( 7.37 ms/it)\n objective: 13322.202571761634\nMinimizing 26617 Time: 0:03:16 ( 7.37 ms/it)\n objective: 13322.196502808802\nMinimizing 26632 Time: 0:03:16 ( 7.37 ms/it)\n objective: 13322.194362599257\nMinimizing 26647 Time: 0:03:16 ( 7.37 ms/it)\n objective: 13322.181679792324\nMinimizing 26662 Time: 0:03:16 ( 7.37 ms/it)\n objective: 13322.179746557638\nMinimizing 26677 Time: 0:03:16 ( 7.37 ms/it)\n objective: 13322.171555529203\nMinimizing 26691 Time: 0:03:16 ( 7.37 ms/it)\n objective: 13322.15271407082\nMinimizing 26706 Time: 0:03:16 ( 7.37 ms/it)\n objective: 13322.141665783522\nMinimizing 26721 Time: 0:03:16 ( 7.36 ms/it)\n objective: 13322.141002141958\nMinimizing 26736 Time: 0:03:16 ( 7.36 ms/it)\n objective: 13322.140011221854\nMinimizing 26751 Time: 0:03:17 ( 7.36 ms/it)\n objective: 13322.138679229465\nMinimizing 26766 Time: 0:03:17 ( 7.36 ms/it)\n objective: 13322.13798806598\nMinimizing 26781 Time: 0:03:17 ( 7.36 ms/it)\n objective: 13322.131744615035\nMinimizing 26796 Time: 0:03:17 ( 7.36 ms/it)\n objective: 13322.11933034641\nMinimizing 26811 Time: 0:03:17 ( 7.36 ms/it)\n objective: 13322.11386716702\nMinimizing 26826 Time: 0:03:17 ( 7.36 ms/it)\n objective: 13322.113270087648\nMinimizing 26841 Time: 0:03:17 ( 7.36 ms/it)\n objective: 13322.109717672225\nMinimizing 26856 Time: 0:03:17 ( 7.36 ms/it)\n objective: 13322.098246238951\nMinimizing 26871 Time: 0:03:17 ( 7.36 ms/it)\n objective: 13322.086853060508\nMinimizing 26886 Time: 0:03:17 ( 7.36 ms/it)\n objective: 13322.057586892319\nMinimizing 26901 Time: 0:03:18 ( 7.36 ms/it)\n objective: 13322.006503733268\nMinimizing 26916 Time: 0:03:18 ( 7.36 ms/it)\n objective: 13322.000420133685\nMinimizing 26931 Time: 0:03:18 ( 7.36 ms/it)\n objective: 13321.996076839307\nMinimizing 26946 Time: 0:03:18 ( 7.36 ms/it)\n objective: 13321.989806150494\nMinimizing 26961 Time: 0:03:18 ( 7.36 ms/it)\n objective: 13321.983023296969\nMinimizing 26976 Time: 0:03:18 ( 7.36 ms/it)\n objective: 13321.982359693138\nMinimizing 26991 Time: 0:03:18 ( 7.36 ms/it)\n objective: 13321.971005187384\nMinimizing 27006 Time: 0:03:18 ( 7.36 ms/it)\n objective: 13321.93727157425\nMinimizing 27021 Time: 0:03:18 ( 7.36 ms/it)\n objective: 13321.775091008894\nMinimizing 27035 Time: 0:03:19 ( 7.36 ms/it)\n objective: 13320.355020921808\nMinimizing 27050 Time: 0:03:19 ( 7.36 ms/it)\n objective: 13320.268693353137\nMinimizing 27063 Time: 0:03:19 ( 7.36 ms/it)\n objective: 13320.255492047756\nMinimizing 27077 Time: 0:03:19 ( 7.36 ms/it)\n objective: 13320.24996317181\nMinimizing 27090 Time: 0:03:19 ( 7.36 ms/it)\n objective: 13320.24833526052\nMinimizing 27104 Time: 0:03:19 ( 7.36 ms/it)\n objective: 13320.247088204822\nMinimizing 27118 Time: 0:03:19 ( 7.36 ms/it)\n objective: 13320.235097612196\nMinimizing 27131 Time: 0:03:19 ( 7.36 ms/it)\n objective: 13320.216857939973\nMinimizing 27145 Time: 0:03:19 ( 7.36 ms/it)\n objective: 13320.182347763417\nMinimizing 27158 Time: 0:03:19 ( 7.36 ms/it)\n objective: 13320.135860622016\nMinimizing 27171 Time: 0:03:20 ( 7.36 ms/it)\n objective: 13320.029316663145\nMinimizing 27185 Time: 0:03:20 ( 7.36 ms/it)\n objective: 13320.02211984538\nMinimizing 27199 Time: 0:03:20 ( 7.36 ms/it)\n objective: 13320.021497332142\nMinimizing 27213 Time: 0:03:20 ( 7.36 ms/it)\n objective: 13320.017043822838\nMinimizing 27226 Time: 0:03:20 ( 7.36 ms/it)\n objective: 13320.013329172769\nMinimizing 27240 Time: 0:03:20 ( 7.36 ms/it)\n objective: 13320.008658601844\nMinimizing 27253 Time: 0:03:20 ( 7.36 ms/it)\n objective: 13320.003166060706\nMinimizing 27266 Time: 0:03:20 ( 7.36 ms/it)\n objective: 13319.996948459258\nMinimizing 27280 Time: 0:03:20 ( 7.36 ms/it)\n objective: 13319.99377048327\nMinimizing 27293 Time: 0:03:20 ( 7.36 ms/it)\n objective: 13319.998264845897\nMinimizing 27306 Time: 0:03:21 ( 7.36 ms/it)\n objective: 13319.971517492624\nMinimizing 27319 Time: 0:03:21 ( 7.36 ms/it)\n objective: 13319.953655871344\nMinimizing 27333 Time: 0:03:21 ( 7.36 ms/it)\n objective: 13319.947685920648\nMinimizing 27347 Time: 0:03:21 ( 7.37 ms/it)\n objective: 13319.941252851218\nMinimizing 27361 Time: 0:03:21 ( 7.37 ms/it)\n objective: 13319.9323335433\nMinimizing 27374 Time: 0:03:21 ( 7.37 ms/it)\n objective: 13319.913367418427\nMinimizing 27387 Time: 0:03:21 ( 7.37 ms/it)\n objective: 13319.905690780812\nMinimizing 27401 Time: 0:03:21 ( 7.37 ms/it)\n objective: 13319.875624269189\nMinimizing 27415 Time: 0:03:21 ( 7.37 ms/it)\n objective: 13319.84675169368\nMinimizing 27428 Time: 0:03:22 ( 7.37 ms/it)\n objective: 13319.846303897197\nMinimizing 27442 Time: 0:03:22 ( 7.37 ms/it)\n objective: 13319.845749282773\nMinimizing 27456 Time: 0:03:22 ( 7.37 ms/it)\n objective: 13319.845291399251\nMinimizing 27470 Time: 0:03:22 ( 7.37 ms/it)\n objective: 13319.845051824494\nMinimizing 27484 Time: 0:03:22 ( 7.37 ms/it)\n objective: 13319.844095454348\nMinimizing 27498 Time: 0:03:22 ( 7.37 ms/it)\n objective: 13319.840598749506\nMinimizing 27512 Time: 0:03:22 ( 7.37 ms/it)\n objective: 13319.837857473962\nMinimizing 27526 Time: 0:03:22 ( 7.37 ms/it)\n objective: 13319.79612609424\nMinimizing 27540 Time: 0:03:22 ( 7.37 ms/it)\n objective: 13319.789676178218\nMinimizing 27554 Time: 0:03:22 ( 7.37 ms/it)\n objective: 13319.789506520086\nMinimizing 27568 Time: 0:03:23 ( 7.37 ms/it)\n objective: 13319.789278508862\nMinimizing 27582 Time: 0:03:23 ( 7.37 ms/it)\n objective: 13319.78813816674\nMinimizing 27596 Time: 0:03:23 ( 7.37 ms/it)\n objective: 13319.787031867294\nMinimizing 27610 Time: 0:03:23 ( 7.37 ms/it)\n objective: 13319.785944017975\nMinimizing 27624 Time: 0:03:23 ( 7.37 ms/it)\n objective: 13319.781280139534\nMinimizing 27638 Time: 0:03:23 ( 7.37 ms/it)\n objective: 13319.772868442597\nMinimizing 27652 Time: 0:03:23 ( 7.37 ms/it)\n objective: 13319.772606038416\nMinimizing 27666 Time: 0:03:23 ( 7.37 ms/it)\n objective: 13319.771758236166\nMinimizing 27680 Time: 0:03:23 ( 7.37 ms/it)\n objective: 13319.769941226856\nMinimizing 27694 Time: 0:03:24 ( 7.37 ms/it)\n objective: 13319.7686089208\nMinimizing 27708 Time: 0:03:24 ( 7.37 ms/it)\n objective: 13319.768114253413\nMinimizing 27722 Time: 0:03:24 ( 7.37 ms/it)\n objective: 13319.766768285743\nMinimizing 27736 Time: 0:03:24 ( 7.37 ms/it)\n objective: 13319.76396185653\nMinimizing 27750 Time: 0:03:24 ( 7.37 ms/it)\n objective: 13319.749242424266\nMinimizing 27764 Time: 0:03:24 ( 7.37 ms/it)\n objective: 13319.73570178308\nMinimizing 27778 Time: 0:03:24 ( 7.37 ms/it)\n objective: 13319.73507797248\nMinimizing 27792 Time: 0:03:24 ( 7.37 ms/it)\n objective: 13319.732811269496\nMinimizing 27807 Time: 0:03:24 ( 7.37 ms/it)\n objective: 13319.728503624225\nMinimizing 27821 Time: 0:03:24 ( 7.37 ms/it)\n objective: 13319.72802864568\nMinimizing 27835 Time: 0:03:25 ( 7.37 ms/it)\n objective: 13319.72743548028\nMinimizing 27849 Time: 0:03:25 ( 7.37 ms/it)\n objective: 13319.727014199423\nMinimizing 27864 Time: 0:03:25 ( 7.37 ms/it)\n objective: 13319.724385208174\nMinimizing 27879 Time: 0:03:25 ( 7.37 ms/it)\n objective: 13319.71931295967\nMinimizing 27894 Time: 0:03:25 ( 7.37 ms/it)\n objective: 13319.716267866985\nMinimizing 27909 Time: 0:03:25 ( 7.37 ms/it)\n objective: 13319.710757417226\nMinimizing 27924 Time: 0:03:25 ( 7.37 ms/it)\n objective: 13319.709103234767\nMinimizing 27939 Time: 0:03:25 ( 7.37 ms/it)\n objective: 13319.70489650403\nMinimizing 27954 Time: 0:03:25 ( 7.37 ms/it)\n objective: 13319.704015958108\nMinimizing 27968 Time: 0:03:26 ( 7.37 ms/it)\n objective: 13319.701313402664\nMinimizing 27983 Time: 0:03:26 ( 7.37 ms/it)\n objective: 13319.70070770607\nMinimizing 27998 Time: 0:03:26 ( 7.37 ms/it)\n objective: 13319.699261675909\nMinimizing 28013 Time: 0:03:26 ( 7.37 ms/it)\n objective: 13319.698619921619\nMinimizing 28028 Time: 0:03:26 ( 7.37 ms/it)\n objective: 13319.694911960716\nMinimizing 28043 Time: 0:03:26 ( 7.37 ms/it)\n objective: 13319.6916300117\nMinimizing 28058 Time: 0:03:26 ( 7.37 ms/it)\n objective: 13319.690280105366\nMinimizing 28072 Time: 0:03:26 ( 7.37 ms/it)\n objective: 13319.678269729804\nMinimizing 28087 Time: 0:03:26 ( 7.37 ms/it)\n objective: 13319.677907532605\nMinimizing 28102 Time: 0:03:26 ( 7.37 ms/it)\n objective: 13319.677675337283\nMinimizing 28117 Time: 0:03:27 ( 7.37 ms/it)\n objective: 13319.67733594947\nMinimizing 28132 Time: 0:03:27 ( 7.36 ms/it)\n objective: 13319.677177272402\nMinimizing 28147 Time: 0:03:27 ( 7.36 ms/it)\n objective: 13319.675933531544\nMinimizing 28162 Time: 0:03:27 ( 7.36 ms/it)\n objective: 13319.66441256176\nMinimizing 28177 Time: 0:03:27 ( 7.36 ms/it)\n objective: 13319.657249074691\nMinimizing 28192 Time: 0:03:27 ( 7.36 ms/it)\n objective: 13319.657064692568\nMinimizing 28207 Time: 0:03:27 ( 7.36 ms/it)\n objective: 13319.655611829701\nMinimizing 28222 Time: 0:03:27 ( 7.36 ms/it)\n objective: 13319.654655720806\nMinimizing 28237 Time: 0:03:27 ( 7.36 ms/it)\n objective: 13319.654321062539\nMinimizing 28251 Time: 0:03:28 ( 7.36 ms/it)\n objective: 13319.653769916084\nMinimizing 28266 Time: 0:03:28 ( 7.36 ms/it)\n objective: 13319.653167271143\nMinimizing 28281 Time: 0:03:28 ( 7.36 ms/it)\n objective: 13319.643104551564\nMinimizing 28296 Time: 0:03:28 ( 7.36 ms/it)\n objective: 13319.642104129307\nMinimizing 28311 Time: 0:03:28 ( 7.36 ms/it)\n objective: 13319.64105514645\nMinimizing 28326 Time: 0:03:28 ( 7.36 ms/it)\n objective: 13319.640492291219\nMinimizing 28341 Time: 0:03:28 ( 7.36 ms/it)\n objective: 13319.637611923332\nMinimizing 28356 Time: 0:03:28 ( 7.36 ms/it)\n objective: 13319.637148376743\nMinimizing 28371 Time: 0:03:28 ( 7.36 ms/it)\n objective: 13319.630214671299\nMinimizing 28385 Time: 0:03:28 ( 7.36 ms/it)\n objective: 13319.62879007963\nMinimizing 28400 Time: 0:03:29 ( 7.36 ms/it)\n objective: 13319.627111768408\nMinimizing 28415 Time: 0:03:29 ( 7.36 ms/it)\n objective: 13319.623462026808\nMinimizing 28430 Time: 0:03:29 ( 7.36 ms/it)\n objective: 13319.62319760515\nMinimizing 28445 Time: 0:03:29 ( 7.36 ms/it)\n objective: 13319.622131633892\nMinimizing 28460 Time: 0:03:29 ( 7.36 ms/it)\n objective: 13319.618381701745\nMinimizing 28475 Time: 0:03:29 ( 7.36 ms/it)\n objective: 13319.617745313531\nMinimizing 28490 Time: 0:03:29 ( 7.36 ms/it)\n objective: 13319.612934042554\nMinimizing 28505 Time: 0:03:29 ( 7.36 ms/it)\n objective: 13319.60968476854\nMinimizing 28520 Time: 0:03:29 ( 7.36 ms/it)\n objective: 13319.608559428685\nMinimizing 28535 Time: 0:03:30 ( 7.36 ms/it)\n objective: 13319.603150398267\nMinimizing 28550 Time: 0:03:30 ( 7.36 ms/it)\n objective: 13319.601729654212\nMinimizing 28565 Time: 0:03:30 ( 7.36 ms/it)\n objective: 13319.59146858484\nMinimizing 28580 Time: 0:03:30 ( 7.36 ms/it)\n objective: 13319.591104162624\nMinimizing 28595 Time: 0:03:30 ( 7.36 ms/it)\n objective: 13319.585510293706\nMinimizing 28610 Time: 0:03:30 ( 7.36 ms/it)\n objective: 13319.577368768718\nMinimizing 28625 Time: 0:03:30 ( 7.36 ms/it)\n objective: 13319.577057157352\nMinimizing 28640 Time: 0:03:30 ( 7.36 ms/it)\n objective: 13319.573558962365\nMinimizing 28655 Time: 0:03:30 ( 7.36 ms/it)\n objective: 13319.573188172333\nMinimizing 28670 Time: 0:03:30 ( 7.36 ms/it)\n objective: 13319.569496985481\nMinimizing 28685 Time: 0:03:31 ( 7.36 ms/it)\n objective: 13319.541556845405\nMinimizing 28700 Time: 0:03:31 ( 7.36 ms/it)\n objective: 13319.541100967792\nMinimizing 28715 Time: 0:03:31 ( 7.36 ms/it)\n objective: 13319.539460200045\nMinimizing 28730 Time: 0:03:31 ( 7.36 ms/it)\n objective: 13319.534000743093\nMinimizing 28745 Time: 0:03:31 ( 7.36 ms/it)\n objective: 13319.533223673512\nMinimizing 28760 Time: 0:03:31 ( 7.36 ms/it)\n objective: 13319.517272859826\nMinimizing 28775 Time: 0:03:31 ( 7.36 ms/it)\n objective: 13319.506597837986\nMinimizing 28790 Time: 0:03:31 ( 7.36 ms/it)\n objective: 13319.50530365578\nMinimizing 28805 Time: 0:03:31 ( 7.36 ms/it)\n objective: 13319.50127408265\nMinimizing 28820 Time: 0:03:31 ( 7.35 ms/it)\n objective: 13319.495029655955\nMinimizing 28835 Time: 0:03:32 ( 7.35 ms/it)\n objective: 13319.493436708246\nMinimizing 28850 Time: 0:03:32 ( 7.35 ms/it)\n objective: 13319.490607815882\nMinimizing 28865 Time: 0:03:32 ( 7.35 ms/it)\n objective: 13319.483190313069\nMinimizing 28880 Time: 0:03:32 ( 7.35 ms/it)\n objective: 13319.478792590773\nMinimizing 28895 Time: 0:03:32 ( 7.35 ms/it)\n objective: 13319.476323713883\nMinimizing 28910 Time: 0:03:32 ( 7.35 ms/it)\n objective: 13319.462226738498\nMinimizing 28925 Time: 0:03:32 ( 7.35 ms/it)\n objective: 13319.46134095687\nMinimizing 28940 Time: 0:03:32 ( 7.35 ms/it)\n objective: 13319.459095027385\nMinimizing 28955 Time: 0:03:32 ( 7.35 ms/it)\n objective: 13319.457224911472\nMinimizing 28970 Time: 0:03:33 ( 7.35 ms/it)\n objective: 13319.456653249435\nMinimizing 28985 Time: 0:03:33 ( 7.35 ms/it)\n objective: 13319.447232531107\nMinimizing 29000 Time: 0:03:33 ( 7.35 ms/it)\n objective: 13319.445363997816\nMinimizing 29015 Time: 0:03:33 ( 7.35 ms/it)\n objective: 13319.441663757869\nMinimizing 29030 Time: 0:03:33 ( 7.35 ms/it)\n objective: 13319.420885132626\nMinimizing 29045 Time: 0:03:33 ( 7.35 ms/it)\n objective: 13319.414901625438\nMinimizing 29060 Time: 0:03:33 ( 7.35 ms/it)\n objective: 13319.414382389965\nMinimizing 29075 Time: 0:03:33 ( 7.35 ms/it)\n objective: 13319.412799412326\nMinimizing 29090 Time: 0:03:33 ( 7.35 ms/it)\n objective: 13319.410662610084\nMinimizing 29105 Time: 0:03:33 ( 7.35 ms/it)\n objective: 13319.392904156717\nMinimizing 29120 Time: 0:03:34 ( 7.35 ms/it)\n objective: 13319.39287942015\nMinimizing 29135 Time: 0:03:34 ( 7.35 ms/it)\n objective: 13319.392607632923\nMinimizing 29150 Time: 0:03:34 ( 7.35 ms/it)\n objective: 13319.392009263844\nMinimizing 29165 Time: 0:03:34 ( 7.35 ms/it)\n objective: 13319.391336786895\nMinimizing 29180 Time: 0:03:34 ( 7.35 ms/it)\n objective: 13319.390675376315\nMinimizing 29195 Time: 0:03:34 ( 7.35 ms/it)\n objective: 13319.383947805865\nMinimizing 29210 Time: 0:03:34 ( 7.35 ms/it)\n objective: 13319.38274871571\nMinimizing 29225 Time: 0:03:34 ( 7.35 ms/it)\n objective: 13319.38227535208\nMinimizing 29240 Time: 0:03:34 ( 7.35 ms/it)\n objective: 13319.381353344506\nMinimizing 29254 Time: 0:03:35 ( 7.35 ms/it)\n objective: 13319.380384236909\nMinimizing 29268 Time: 0:03:35 ( 7.35 ms/it)\n objective: 13319.380001343772\nMinimizing 29283 Time: 0:03:35 ( 7.35 ms/it)\n objective: 13319.375219436624\nMinimizing 29298 Time: 0:03:35 ( 7.35 ms/it)\n objective: 13319.360260937756\nMinimizing 29313 Time: 0:03:35 ( 7.35 ms/it)\n objective: 13319.360185300844\nMinimizing 29328 Time: 0:03:35 ( 7.35 ms/it)\n objective: 13319.359162387744\nMinimizing 29343 Time: 0:03:35 ( 7.35 ms/it)\n objective: 13319.357486142922\nMinimizing 29358 Time: 0:03:35 ( 7.35 ms/it)\n objective: 13319.35687510611\nMinimizing 29373 Time: 0:03:35 ( 7.35 ms/it)\n objective: 13319.350726837045\nMinimizing 29388 Time: 0:03:35 ( 7.35 ms/it)\n objective: 13319.331905344297\nMinimizing 29403 Time: 0:03:36 ( 7.35 ms/it)\n objective: 13319.331704520562\nMinimizing 29418 Time: 0:03:36 ( 7.35 ms/it)\n objective: 13319.330348258285\nMinimizing 29433 Time: 0:03:36 ( 7.35 ms/it)\n objective: 13319.325553988165\nMinimizing 29448 Time: 0:03:36 ( 7.35 ms/it)\n objective: 13319.325371520972\nMinimizing 29463 Time: 0:03:36 ( 7.35 ms/it)\n objective: 13319.322879812316\nMinimizing 29478 Time: 0:03:36 ( 7.35 ms/it)\n objective: 13319.32194799732\nMinimizing 29493 Time: 0:03:36 ( 7.35 ms/it)\n objective: 13319.321219206904\nMinimizing 29508 Time: 0:03:36 ( 7.35 ms/it)\n objective: 13319.320024299479\nMinimizing 29523 Time: 0:03:36 ( 7.35 ms/it)\n objective: 13319.317654963248\nMinimizing 29538 Time: 0:03:36 ( 7.35 ms/it)\n objective: 13319.305325047899\nMinimizing 29553 Time: 0:03:37 ( 7.35 ms/it)\n objective: 13319.300509318608\nMinimizing 29568 Time: 0:03:37 ( 7.35 ms/it)\n objective: 13319.300171874464\nMinimizing 29583 Time: 0:03:37 ( 7.35 ms/it)\n objective: 13319.299932244016\nMinimizing 29598 Time: 0:03:37 ( 7.35 ms/it)\n objective: 13319.299345807565\nMinimizing 29613 Time: 0:03:37 ( 7.35 ms/it)\n objective: 13319.298664570248\nMinimizing 29628 Time: 0:03:37 ( 7.35 ms/it)\n objective: 13319.295059581244\nMinimizing 29643 Time: 0:03:37 ( 7.34 ms/it)\n objective: 13319.280243673638\nMinimizing 29658 Time: 0:03:37 ( 7.34 ms/it)\n objective: 13319.280134142595\nMinimizing 29673 Time: 0:03:37 ( 7.34 ms/it)\n objective: 13319.279147828332\nMinimizing 29688 Time: 0:03:38 ( 7.34 ms/it)\n objective: 13319.277990260598\nMinimizing 29703 Time: 0:03:38 ( 7.34 ms/it)\n objective: 13319.27665077828\nMinimizing 29718 Time: 0:03:38 ( 7.34 ms/it)\n objective: 13319.275810941253\nMinimizing 29733 Time: 0:03:38 ( 7.34 ms/it)\n objective: 13319.267151519714\nMinimizing 29748 Time: 0:03:38 ( 7.34 ms/it)\n objective: 13319.266617043555\nMinimizing 29763 Time: 0:03:38 ( 7.34 ms/it)\n objective: 13319.265311843526\nMinimizing 29778 Time: 0:03:38 ( 7.34 ms/it)\n objective: 13319.261467029908\nMinimizing 29793 Time: 0:03:38 ( 7.34 ms/it)\n objective: 13319.252957024117\nMinimizing 29808 Time: 0:03:38 ( 7.34 ms/it)\n objective: 13319.252680339341\nMinimizing 29823 Time: 0:03:38 ( 7.34 ms/it)\n objective: 13319.252363490014\nMinimizing 29838 Time: 0:03:39 ( 7.34 ms/it)\n objective: 13319.252082797291\nMinimizing 29853 Time: 0:03:39 ( 7.34 ms/it)\n objective: 13319.249658341476\nMinimizing 29868 Time: 0:03:39 ( 7.34 ms/it)\n objective: 13319.24663186555\nMinimizing 29883 Time: 0:03:39 ( 7.34 ms/it)\n objective: 13319.245967011084\nMinimizing 29898 Time: 0:03:39 ( 7.34 ms/it)\n objective: 13319.240411905295\nMinimizing 29913 Time: 0:03:39 ( 7.34 ms/it)\n objective: 13319.238758139138\nMinimizing 29928 Time: 0:03:39 ( 7.34 ms/it)\n objective: 13319.238407833996\nMinimizing 29943 Time: 0:03:39 ( 7.34 ms/it)\n objective: 13319.237981691462\nMinimizing 29958 Time: 0:03:39 ( 7.34 ms/it)\n objective: 13319.237674684482\nMinimizing 29973 Time: 0:03:40 ( 7.34 ms/it)\n objective: 13319.235851250225\nMinimizing 29988 Time: 0:03:40 ( 7.34 ms/it)\n objective: 13319.228648968856\nMinimizing 30003 Time: 0:03:40 ( 7.34 ms/it)\n objective: 13319.228628546422\nMinimizing 30018 Time: 0:03:40 ( 7.34 ms/it)\n objective: 13319.22806362201\nMinimizing 30033 Time: 0:03:40 ( 7.34 ms/it)\n objective: 13319.22728526601\nMinimizing 30048 Time: 0:03:40 ( 7.34 ms/it)\n objective: 13319.225981076728\nMinimizing 30063 Time: 0:03:40 ( 7.34 ms/it)\n objective: 13319.22500575472\nMinimizing 30078 Time: 0:03:40 ( 7.34 ms/it)\n objective: 13319.224119420294\nMinimizing 30093 Time: 0:03:40 ( 7.34 ms/it)\n objective: 13319.22388071465\nMinimizing 30108 Time: 0:03:40 ( 7.34 ms/it)\n objective: 13319.206780291599\nMinimizing 30123 Time: 0:03:41 ( 7.34 ms/it)\n objective: 13319.17522597342\nMinimizing 30138 Time: 0:03:41 ( 7.34 ms/it)\n objective: 13319.174902652987\nMinimizing 30153 Time: 0:03:41 ( 7.34 ms/it)\n objective: 13319.173807306739\nMinimizing 30168 Time: 0:03:41 ( 7.34 ms/it)\n objective: 13319.172805273483\nMinimizing 30183 Time: 0:03:41 ( 7.34 ms/it)\n objective: 13319.17138143997\nMinimizing 30198 Time: 0:03:41 ( 7.34 ms/it)\n objective: 13319.166776528771\nMinimizing 30213 Time: 0:03:41 ( 7.34 ms/it)\n objective: 13319.160868803054\nMinimizing 30228 Time: 0:03:41 ( 7.34 ms/it)\n objective: 13319.147296470823\nMinimizing 30243 Time: 0:03:41 ( 7.34 ms/it)\n objective: 13319.12987647175\nMinimizing 30258 Time: 0:03:41 ( 7.34 ms/it)\n objective: 13319.087285741334\nMinimizing 30273 Time: 0:03:42 ( 7.34 ms/it)\n objective: 13319.086711761018\nMinimizing 30288 Time: 0:03:42 ( 7.34 ms/it)\n objective: 13319.08369201218\nMinimizing 30303 Time: 0:03:42 ( 7.34 ms/it)\n objective: 13319.08128603507\nMinimizing 30318 Time: 0:03:42 ( 7.34 ms/it)\n objective: 13319.080915554194\nMinimizing 30333 Time: 0:03:42 ( 7.34 ms/it)\n objective: 13319.072859063032\nMinimizing 30348 Time: 0:03:42 ( 7.34 ms/it)\n objective: 13319.059522382813\nMinimizing 30362 Time: 0:03:42 ( 7.34 ms/it)\n objective: 13318.984527460343\nMinimizing 30377 Time: 0:03:42 ( 7.34 ms/it)\n objective: 13318.951953823882\nMinimizing 30392 Time: 0:03:42 ( 7.34 ms/it)\n objective: 13318.951673342744\nMinimizing 30407 Time: 0:03:43 ( 7.33 ms/it)\n objective: 13318.950388999918\nMinimizing 30422 Time: 0:03:43 ( 7.33 ms/it)\n objective: 13318.949722401056\nMinimizing 30437 Time: 0:03:43 ( 7.33 ms/it)\n objective: 13318.949379876518\nMinimizing 30452 Time: 0:03:43 ( 7.33 ms/it)\n objective: 13318.948498774116\nMinimizing 30467 Time: 0:03:43 ( 7.33 ms/it)\n objective: 13318.94424841879\nMinimizing 30482 Time: 0:03:43 ( 7.33 ms/it)\n objective: 13318.938211981294\nMinimizing 30497 Time: 0:03:43 ( 7.33 ms/it)\n objective: 13318.937600586985\nMinimizing 30512 Time: 0:03:43 ( 7.33 ms/it)\n objective: 13318.926512712482\nMinimizing 30528 Time: 0:03:43 ( 7.33 ms/it)\n objective: 13318.925488212335\nMinimizing 30543 Time: 0:03:43 ( 7.33 ms/it)\n objective: 13318.921620370485\nMinimizing 30558 Time: 0:03:44 ( 7.33 ms/it)\n objective: 13318.920561537787\nMinimizing 30573 Time: 0:03:44 ( 7.33 ms/it)\n objective: 13318.919127931265\nMinimizing 30588 Time: 0:03:44 ( 7.33 ms/it)\n objective: 13318.913355053723\nMinimizing 30603 Time: 0:03:44 ( 7.33 ms/it)\n objective: 13318.894152780296\nMinimizing 30616 Time: 0:03:44 ( 7.33 ms/it)\n objective: 13318.892957417469\nMinimizing 30631 Time: 0:03:44 ( 7.33 ms/it)\n objective: 13318.892085047803\nMinimizing 30646 Time: 0:03:44 ( 7.33 ms/it)\n objective: 13318.89120573389\nMinimizing 30661 Time: 0:03:44 ( 7.33 ms/it)\n objective: 13318.890321445535\nMinimizing 30676 Time: 0:03:44 ( 7.33 ms/it)\n objective: 13318.887567563754\nMinimizing 30691 Time: 0:03:45 ( 7.33 ms/it)\n objective: 13318.88701855982\nMinimizing 30705 Time: 0:03:45 ( 7.33 ms/it)\n objective: 13318.8825619954\nMinimizing 30719 Time: 0:03:45 ( 7.33 ms/it)\n objective: 13318.877883625493\nMinimizing 30734 Time: 0:03:45 ( 7.33 ms/it)\n objective: 13318.875786700592\nMinimizing 30749 Time: 0:03:45 ( 7.33 ms/it)\n objective: 13318.85820491056\nMinimizing 30765 Time: 0:03:45 ( 7.33 ms/it)\n objective: 13318.857843792983\nMinimizing 30781 Time: 0:03:45 ( 7.33 ms/it)\n objective: 13318.85461594892\nMinimizing 30797 Time: 0:03:45 ( 7.33 ms/it)\n objective: 13318.851330874706\nMinimizing 30812 Time: 0:03:45 ( 7.33 ms/it)\n objective: 13318.828961918189\nMinimizing 30827 Time: 0:03:45 ( 7.33 ms/it)\n objective: 13318.823229199974\nMinimizing 30842 Time: 0:03:46 ( 7.33 ms/it)\n objective: 13318.822322328968\nMinimizing 30857 Time: 0:03:46 ( 7.33 ms/it)\n objective: 13318.817577720503\nMinimizing 30872 Time: 0:03:46 ( 7.33 ms/it)\n objective: 13318.81217702775\nMinimizing 30887 Time: 0:03:46 ( 7.33 ms/it)\n objective: 13318.809610448792\nMinimizing 30902 Time: 0:03:46 ( 7.33 ms/it)\n objective: 13318.809001793401\nMinimizing 30917 Time: 0:03:46 ( 7.33 ms/it)\n objective: 13318.807110029331\nMinimizing 30932 Time: 0:03:46 ( 7.33 ms/it)\n objective: 13318.799004679458\nMinimizing 30947 Time: 0:03:46 ( 7.33 ms/it)\n objective: 13318.797360402808\nMinimizing 30962 Time: 0:03:46 ( 7.33 ms/it)\n objective: 13318.796594720654\nMinimizing 30977 Time: 0:03:47 ( 7.33 ms/it)\n objective: 13318.796140296763\nMinimizing 30992 Time: 0:03:47 ( 7.33 ms/it)\n objective: 13318.795310017842\nMinimizing 31007 Time: 0:03:47 ( 7.33 ms/it)\n objective: 13318.794545561163\nMinimizing 31022 Time: 0:03:47 ( 7.33 ms/it)\n objective: 13318.785633063919\nMinimizing 31037 Time: 0:03:47 ( 7.33 ms/it)\n objective: 13318.768674989027\nMinimizing 31052 Time: 0:03:47 ( 7.33 ms/it)\n objective: 13318.767123796817\nMinimizing 31067 Time: 0:03:47 ( 7.33 ms/it)\n objective: 13318.76304193525\nMinimizing 31082 Time: 0:03:47 ( 7.33 ms/it)\n objective: 13318.761402975302\nMinimizing 31097 Time: 0:03:47 ( 7.33 ms/it)\n objective: 13318.758187083324\nMinimizing 31112 Time: 0:03:47 ( 7.33 ms/it)\n objective: 13318.746061366153\nMinimizing 31127 Time: 0:03:48 ( 7.33 ms/it)\n objective: 13318.736358005015\nMinimizing 31142 Time: 0:03:48 ( 7.33 ms/it)\n objective: 13318.736107113873\nMinimizing 31157 Time: 0:03:48 ( 7.33 ms/it)\n objective: 13318.73301025531\nMinimizing 31172 Time: 0:03:48 ( 7.33 ms/it)\n objective: 13318.730416240898\nMinimizing 31187 Time: 0:03:48 ( 7.33 ms/it)\n objective: 13318.718362650325\nMinimizing 31202 Time: 0:03:48 ( 7.33 ms/it)\n objective: 13318.71647735151\nMinimizing 31217 Time: 0:03:48 ( 7.33 ms/it)\n objective: 13318.710605837085\nMinimizing 31232 Time: 0:03:48 ( 7.33 ms/it)\n objective: 13318.696013906883\nMinimizing 31247 Time: 0:03:48 ( 7.32 ms/it)\n objective: 13318.591494342429\nMinimizing 31262 Time: 0:03:48 ( 7.32 ms/it)\n objective: 13318.555966758504\nMinimizing 31277 Time: 0:03:49 ( 7.32 ms/it)\n objective: 13318.555690192545\nMinimizing 31292 Time: 0:03:49 ( 7.32 ms/it)\n objective: 13318.555215810848\nMinimizing 31306 Time: 0:03:49 ( 7.32 ms/it)\n objective: 13318.555073483702\nMinimizing 31320 Time: 0:03:49 ( 7.32 ms/it)\n objective: 13318.553965392071\nMinimizing 31335 Time: 0:03:49 ( 7.32 ms/it)\n objective: 13318.552727107133\nMinimizing 31349 Time: 0:03:49 ( 7.32 ms/it)\n objective: 13318.552189863258\nMinimizing 31363 Time: 0:03:49 ( 7.32 ms/it)\n objective: 13318.549193947183\nMinimizing 31378 Time: 0:03:49 ( 7.32 ms/it)\n objective: 13318.511521852575\nMinimizing 31392 Time: 0:03:49 ( 7.32 ms/it)\n objective: 13318.504667198911\nMinimizing 31406 Time: 0:03:50 ( 7.32 ms/it)\n objective: 13318.504362174921\nMinimizing 31420 Time: 0:03:50 ( 7.32 ms/it)\n objective: 13318.50402450448\nMinimizing 31434 Time: 0:03:50 ( 7.32 ms/it)\n objective: 13318.503791047871\nMinimizing 31448 Time: 0:03:50 ( 7.32 ms/it)\n objective: 13318.50336429839\nMinimizing 31463 Time: 0:03:50 ( 7.32 ms/it)\n objective: 13318.503278658653\nMinimizing 31477 Time: 0:03:50 ( 7.32 ms/it)\n objective: 13318.500972039416\nMinimizing 31492 Time: 0:03:50 ( 7.32 ms/it)\n objective: 13318.496195650063\nMinimizing 31507 Time: 0:03:50 ( 7.32 ms/it)\n objective: 13318.49520516509\nMinimizing 31522 Time: 0:03:50 ( 7.32 ms/it)\n objective: 13318.490938155577\nMinimizing 31536 Time: 0:03:50 ( 7.32 ms/it)\n objective: 13318.490180608147\nMinimizing 31550 Time: 0:03:51 ( 7.32 ms/it)\n objective: 13318.489932952696\nMinimizing 31564 Time: 0:03:51 ( 7.32 ms/it)\n objective: 13318.489744292747\nMinimizing 31577 Time: 0:03:51 ( 7.32 ms/it)\n objective: 13318.488708573757\nMinimizing 31590 Time: 0:03:51 ( 7.32 ms/it)\n objective: 13318.480740670144\nMinimizing 31604 Time: 0:03:51 ( 7.32 ms/it)\n objective: 13318.48009399086\nMinimizing 31618 Time: 0:03:51 ( 7.32 ms/it)\n objective: 13318.47979039482\nMinimizing 31633 Time: 0:03:51 ( 7.32 ms/it)\n objective: 13318.479098601936\nMinimizing 31648 Time: 0:03:51 ( 7.32 ms/it)\n objective: 13318.478548612882\nMinimizing 31663 Time: 0:03:51 ( 7.32 ms/it)\n objective: 13318.468912793163\nMinimizing 31678 Time: 0:03:52 ( 7.32 ms/it)\n objective: 13318.467061815405\nMinimizing 31693 Time: 0:03:52 ( 7.32 ms/it)\n objective: 13318.466138965552\nMinimizing 31708 Time: 0:03:52 ( 7.32 ms/it)\n objective: 13318.465292640598\nMinimizing 31723 Time: 0:03:52 ( 7.32 ms/it)\n objective: 13318.461608042853\nMinimizing 31738 Time: 0:03:52 ( 7.32 ms/it)\n objective: 13318.46111039797\nMinimizing 31753 Time: 0:03:52 ( 7.32 ms/it)\n objective: 13318.46022822574\nMinimizing 31768 Time: 0:03:52 ( 7.32 ms/it)\n objective: 13318.456600592312\nMinimizing 31783 Time: 0:03:52 ( 7.32 ms/it)\n objective: 13318.453648369948\nMinimizing 31798 Time: 0:03:52 ( 7.32 ms/it)\n objective: 13318.442545409358\nMinimizing 31813 Time: 0:03:52 ( 7.32 ms/it)\n objective: 13318.440104803143\nMinimizing 31828 Time: 0:03:53 ( 7.32 ms/it)\n objective: 13318.437657174305\nMinimizing 31843 Time: 0:03:53 ( 7.32 ms/it)\n objective: 13318.436943231121\nMinimizing 31858 Time: 0:03:53 ( 7.32 ms/it)\n objective: 13318.43442024423\nMinimizing 31873 Time: 0:03:53 ( 7.32 ms/it)\n objective: 13318.427245381637\nMinimizing 31888 Time: 0:03:53 ( 7.32 ms/it)\n objective: 13318.426920206577\nMinimizing 31903 Time: 0:03:53 ( 7.32 ms/it)\n objective: 13318.426926414817\nMinimizing 31918 Time: 0:03:53 ( 7.32 ms/it)\n objective: 13318.426870108146\nMinimizing 31932 Time: 0:03:53 ( 7.32 ms/it)\n objective: 13318.42685512139\nMinimizing 31946 Time: 0:03:53 ( 7.32 ms/it)\n objective: 13318.42687163921\nMinimizing 31960 Time: 0:03:53 ( 7.32 ms/it)\n objective: 13318.426863760804\nMinimizing 31975 Time: 0:03:54 ( 7.32 ms/it)\n objective: 13318.426859661398\nMinimizing 31990 Time: 0:03:54 ( 7.32 ms/it)\n objective: 13318.426850211632\nMinimizing 32005 Time: 0:03:54 ( 7.32 ms/it)\n objective: 13318.426848752657\nMinimizing 32020 Time: 0:03:54 ( 7.32 ms/it)\n objective: 13318.426842984656\nMinimizing 32035 Time: 0:03:54 ( 7.32 ms/it)\n objective: 13318.42683950116\nMinimizing 32050 Time: 0:03:54 ( 7.32 ms/it)\n objective: 13318.426846553382\nMinimizing 32065 Time: 0:03:54 ( 7.32 ms/it)\n objective: 13318.426840526998\nMinimizing 32081 Time: 0:03:54 ( 7.32 ms/it)\n objective: 13318.426838949803\nMinimizing 32096 Time: 0:03:54 ( 7.32 ms/it)\n objective: 13318.426848268617\nMinimizing 32100 Time: 0:03:54 ( 7.32 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0183\n0.0139\n-1.32\n0.1876\n\n\n\nTest: Star-Run\n-0.0142\n0.0438\n-0.32\n0.7465\n\n\n\nTest: S20-Star\n-0.0111\n0.0409\n-0.27\n0.7862\n\n\n\nTest: SLJ-S20\n0.0099\n0.0442\n0.22\n0.8236\n\n\n\nTest: BPT-SLJ\n-0.0231\n0.0422\n-0.55\n0.5836\n\n\n\na1\n0.3167\n0.0476\n6.65\n<1e-10\n\n\n\nSex: Boys\n0.2048\n0.0139\n14.73\n<1e-48\n\n\n\nTest: Star-Run & a1\n0.2986\n0.1505\n1.98\n0.0472\n\n\n\nTest: S20-Star & a1\n0.0494\n0.1419\n0.35\n0.7275\n\n\n\nTest: SLJ-S20 & a1\n-0.0658\n0.1520\n-0.43\n0.6649\n\n\n\nTest: BPT-SLJ & a1\n0.3151\n0.1416\n2.23\n0.0261\n\n\n\nTest: Star-Run & Sex: Boys\n-0.1213\n0.0438\n-2.77\n0.0056\n\n\n\nTest: S20-Star & Sex: Boys\n0.0664\n0.0409\n1.63\n0.1041\n\n\n\nTest: SLJ-S20 & Sex: Boys\n0.0466\n0.0442\n1.05\n0.2919\n\n\n\nTest: BPT-SLJ & Sex: Boys\n0.1634\n0.0422\n3.87\n0.0001\n\n\n\na1 & Sex: Boys\n0.0502\n0.0476\n1.06\n0.2912\n\n\n\nTest: Star-Run & a1 & Sex: Boys\n-0.0114\n0.1505\n-0.08\n0.9395\n\n\n\nTest: S20-Star & a1 & Sex: Boys\n-0.0288\n0.1419\n-0.20\n0.8390\n\n\n\nTest: SLJ-S20 & a1 & Sex: Boys\n-0.0603\n0.1520\n-0.40\n0.6916\n\n\n\nTest: BPT-SLJ & a1 & Sex: Boys\n0.1415\n0.1416\n1.00\n0.3178\n\n\n\nTest: BPT\n\n\n\n\n0.9054\n\n\nTest: SLJ\n\n\n\n\n0.9717\n\n\nTest: Star_r\n\n\n\n\n0.9914\n\n\nTest: Run\n\n\n\n\n0.9717\n\n\nTest: S20_r\n\n\n\n\n0.9786\n\n\nResidual\n0.0003\n\n\n\n\n\n\n\n\n\n\nm_cpx_0_SeqDiff = let\n f_cpx_0 = @formula(\n zScore ~ 1 + Test * a1 * Sex + (0 + Test | Child)\n )\n fit(MixedModel, f_cpx_0, dat; contrasts=contr1b)\nend\n\nMinimizing 15 Time: 0:00:00 ( 6.84 ms/it)\n objective: 13919.774893920156\nMinimizing 30 Time: 0:00:00 ( 6.78 ms/it)\n objective: 13934.06374034128\nMinimizing 45 Time: 0:00:00 ( 6.76 ms/it)\n objective: 13808.794217331397\nMinimizing 60 Time: 0:00:00 ( 6.79 ms/it)\n objective: 13807.163430095352\nMinimizing 75 Time: 0:00:00 ( 6.78 ms/it)\n objective: 13805.380843030245\nMinimizing 90 Time: 0:00:00 ( 6.77 ms/it)\n objective: 13796.147954658398\nMinimizing 105 Time: 0:00:00 ( 6.78 ms/it)\n objective: 13789.167221291498\nMinimizing 120 Time: 0:00:00 ( 6.78 ms/it)\n objective: 13785.10690625065\nMinimizing 135 Time: 0:00:00 ( 6.78 ms/it)\n objective: 13770.622002111955\nMinimizing 150 Time: 0:00:01 ( 6.78 ms/it)\n objective: 13756.957204824645\nMinimizing 165 Time: 0:00:01 ( 6.78 ms/it)\n objective: 13749.738906704768\nMinimizing 180 Time: 0:00:01 ( 6.77 ms/it)\n objective: 13749.022661135381\nMinimizing 195 Time: 0:00:01 ( 6.77 ms/it)\n objective: 13743.450583952044\nMinimizing 210 Time: 0:00:01 ( 6.77 ms/it)\n objective: 13737.371538731917\nMinimizing 225 Time: 0:00:01 ( 6.77 ms/it)\n objective: 13734.843831763124\nMinimizing 240 Time: 0:00:01 ( 6.76 ms/it)\n objective: 13726.12535518125\nMinimizing 255 Time: 0:00:01 ( 6.76 ms/it)\n objective: 13718.079823864355\nMinimizing 270 Time: 0:00:01 ( 6.76 ms/it)\n objective: 13713.7998795954\nMinimizing 285 Time: 0:00:01 ( 6.76 ms/it)\n objective: 13708.429833563376\nMinimizing 300 Time: 0:00:02 ( 6.76 ms/it)\n objective: 13706.19462994505\nMinimizing 315 Time: 0:00:02 ( 6.76 ms/it)\n objective: 13702.526901215406\nMinimizing 330 Time: 0:00:02 ( 6.75 ms/it)\n objective: 13701.407982621175\nMinimizing 345 Time: 0:00:02 ( 6.75 ms/it)\n objective: 13697.216752981687\nMinimizing 360 Time: 0:00:02 ( 6.76 ms/it)\n objective: 13687.964778485086\nMinimizing 375 Time: 0:00:02 ( 6.76 ms/it)\n objective: 13680.159437377395\nMinimizing 390 Time: 0:00:02 ( 6.76 ms/it)\n objective: 13673.285851322282\nMinimizing 405 Time: 0:00:02 ( 6.75 ms/it)\n objective: 13667.51095094907\nMinimizing 420 Time: 0:00:02 ( 6.75 ms/it)\n objective: 13676.351853648895\nMinimizing 435 Time: 0:00:02 ( 6.75 ms/it)\n objective: 13648.233972394144\nMinimizing 450 Time: 0:00:03 ( 6.75 ms/it)\n objective: 13646.289861123538\nMinimizing 465 Time: 0:00:03 ( 6.75 ms/it)\n objective: 13639.024506617558\nMinimizing 480 Time: 0:00:03 ( 6.75 ms/it)\n objective: 13634.782904938736\nMinimizing 495 Time: 0:00:03 ( 6.75 ms/it)\n objective: 13630.423725385735\nMinimizing 510 Time: 0:00:03 ( 6.75 ms/it)\n objective: 13618.510459066481\nMinimizing 525 Time: 0:00:03 ( 6.75 ms/it)\n objective: 13617.232596673148\nMinimizing 540 Time: 0:00:03 ( 6.75 ms/it)\n objective: 13615.641902185882\nMinimizing 555 Time: 0:00:03 ( 6.75 ms/it)\n objective: 13611.868963963523\nMinimizing 570 Time: 0:00:03 ( 6.75 ms/it)\n objective: 13609.090749091069\nMinimizing 585 Time: 0:00:03 ( 6.75 ms/it)\n objective: 13601.752460463606\nMinimizing 600 Time: 0:00:04 ( 6.75 ms/it)\n objective: 13600.331198118234\nMinimizing 615 Time: 0:00:04 ( 6.75 ms/it)\n objective: 13599.201972133687\nMinimizing 630 Time: 0:00:04 ( 6.75 ms/it)\n objective: 13597.07032570483\nMinimizing 645 Time: 0:00:04 ( 6.75 ms/it)\n objective: 13586.41672012295\nMinimizing 660 Time: 0:00:04 ( 6.75 ms/it)\n objective: 13585.84425906483\nMinimizing 675 Time: 0:00:04 ( 6.75 ms/it)\n objective: 13584.30181207391\nMinimizing 690 Time: 0:00:04 ( 6.75 ms/it)\n objective: 13583.156506241612\nMinimizing 705 Time: 0:00:04 ( 6.75 ms/it)\n objective: 13582.366929028874\nMinimizing 720 Time: 0:00:04 ( 6.75 ms/it)\n objective: 13577.470939937491\nMinimizing 735 Time: 0:00:04 ( 6.75 ms/it)\n objective: 13575.814437644036\nMinimizing 750 Time: 0:00:05 ( 6.75 ms/it)\n objective: 13572.849954089357\nMinimizing 765 Time: 0:00:05 ( 6.75 ms/it)\n objective: 13568.097652526605\nMinimizing 780 Time: 0:00:05 ( 6.75 ms/it)\n objective: 13563.483166512051\nMinimizing 795 Time: 0:00:05 ( 6.75 ms/it)\n objective: 13560.757080004918\nMinimizing 810 Time: 0:00:05 ( 6.75 ms/it)\n objective: 13553.878850962861\nMinimizing 825 Time: 0:00:05 ( 6.75 ms/it)\n objective: 13549.3931844344\nMinimizing 840 Time: 0:00:05 ( 6.75 ms/it)\n objective: 13546.2450776345\nMinimizing 855 Time: 0:00:05 ( 6.75 ms/it)\n objective: 13540.393957147226\nMinimizing 870 Time: 0:00:05 ( 6.75 ms/it)\n objective: 13536.78564915241\nMinimizing 885 Time: 0:00:05 ( 6.75 ms/it)\n objective: 13517.485461952754\nMinimizing 900 Time: 0:00:06 ( 6.75 ms/it)\n objective: 13516.743165448926\nMinimizing 915 Time: 0:00:06 ( 6.75 ms/it)\n objective: 13513.522331677901\nMinimizing 930 Time: 0:00:06 ( 6.75 ms/it)\n objective: 13509.079679671442\nMinimizing 945 Time: 0:00:06 ( 6.75 ms/it)\n objective: 13508.197049533424\nMinimizing 960 Time: 0:00:06 ( 6.75 ms/it)\n objective: 13507.527489509594\nMinimizing 975 Time: 0:00:06 ( 6.74 ms/it)\n objective: 13506.268276394374\nMinimizing 990 Time: 0:00:06 ( 6.74 ms/it)\n objective: 13505.98630900788\nMinimizing 1005 Time: 0:00:06 ( 6.74 ms/it)\n objective: 13503.66746683239\nMinimizing 1020 Time: 0:00:06 ( 6.74 ms/it)\n objective: 13503.33522657401\nMinimizing 1035 Time: 0:00:06 ( 6.74 ms/it)\n objective: 13500.303539770604\nMinimizing 1050 Time: 0:00:07 ( 6.75 ms/it)\n objective: 13498.361545832799\nMinimizing 1065 Time: 0:00:07 ( 6.75 ms/it)\n objective: 13495.652365547001\nMinimizing 1080 Time: 0:00:07 ( 6.75 ms/it)\n objective: 13494.420022849823\nMinimizing 1095 Time: 0:00:07 ( 6.75 ms/it)\n objective: 13493.987817923211\nMinimizing 1111 Time: 0:00:07 ( 6.76 ms/it)\n objective: 13492.931869125729\nMinimizing 1127 Time: 0:00:07 ( 6.76 ms/it)\n objective: 13492.158798502358\nMinimizing 1142 Time: 0:00:07 ( 6.76 ms/it)\n objective: 13491.954239806008\nMinimizing 1157 Time: 0:00:07 ( 6.76 ms/it)\n objective: 13489.119521278852\nMinimizing 1172 Time: 0:00:07 ( 6.76 ms/it)\n objective: 13487.798822978679\nMinimizing 1187 Time: 0:00:08 ( 6.75 ms/it)\n objective: 13485.250482008072\nMinimizing 1202 Time: 0:00:08 ( 6.76 ms/it)\n objective: 13484.736374020526\nMinimizing 1217 Time: 0:00:08 ( 6.75 ms/it)\n objective: 13484.422177126231\nMinimizing 1232 Time: 0:00:08 ( 6.75 ms/it)\n objective: 13484.079088188613\nMinimizing 1247 Time: 0:00:08 ( 6.75 ms/it)\n objective: 13483.848685171259\nMinimizing 1262 Time: 0:00:08 ( 6.75 ms/it)\n objective: 13483.042934731639\nMinimizing 1277 Time: 0:00:08 ( 6.75 ms/it)\n objective: 13481.598443091214\nMinimizing 1292 Time: 0:00:08 ( 6.75 ms/it)\n objective: 13480.879115890944\nMinimizing 1307 Time: 0:00:08 ( 6.75 ms/it)\n objective: 13479.402071573073\nMinimizing 1322 Time: 0:00:08 ( 6.75 ms/it)\n objective: 13478.46694483886\nMinimizing 1337 Time: 0:00:09 ( 6.75 ms/it)\n objective: 13477.76733882117\nMinimizing 1352 Time: 0:00:09 ( 6.75 ms/it)\n objective: 13477.356396287898\nMinimizing 1367 Time: 0:00:09 ( 6.75 ms/it)\n objective: 13476.73797488567\nMinimizing 1382 Time: 0:00:09 ( 6.75 ms/it)\n objective: 13476.608043450935\nMinimizing 1397 Time: 0:00:09 ( 6.75 ms/it)\n objective: 13475.094820783583\nMinimizing 1412 Time: 0:00:09 ( 6.75 ms/it)\n objective: 13473.918005475272\nMinimizing 1427 Time: 0:00:09 ( 6.75 ms/it)\n objective: 13473.409871329422\nMinimizing 1442 Time: 0:00:09 ( 6.75 ms/it)\n objective: 13471.7466789237\nMinimizing 1457 Time: 0:00:09 ( 6.75 ms/it)\n objective: 13471.105716562059\nMinimizing 1472 Time: 0:00:09 ( 6.75 ms/it)\n objective: 13470.17220789985\nMinimizing 1487 Time: 0:00:10 ( 6.75 ms/it)\n objective: 13470.034715340327\nMinimizing 1502 Time: 0:00:10 ( 6.75 ms/it)\n objective: 13469.476169595357\nMinimizing 1517 Time: 0:00:10 ( 6.75 ms/it)\n objective: 13465.799769624056\nMinimizing 1532 Time: 0:00:10 ( 6.75 ms/it)\n objective: 13464.496539541913\nMinimizing 1547 Time: 0:00:10 ( 6.75 ms/it)\n objective: 13464.064225560396\nMinimizing 1562 Time: 0:00:10 ( 6.75 ms/it)\n objective: 13463.637813806818\nMinimizing 1577 Time: 0:00:10 ( 6.75 ms/it)\n objective: 13463.169545751429\nMinimizing 1592 Time: 0:00:10 ( 6.75 ms/it)\n objective: 13462.968788733539\nMinimizing 1607 Time: 0:00:10 ( 6.75 ms/it)\n objective: 13462.24264395712\nMinimizing 1622 Time: 0:00:10 ( 6.75 ms/it)\n objective: 13461.954158376597\nMinimizing 1637 Time: 0:00:11 ( 6.75 ms/it)\n objective: 13461.349282803356\nMinimizing 1652 Time: 0:00:11 ( 6.75 ms/it)\n objective: 13459.714563331981\nMinimizing 1667 Time: 0:00:11 ( 6.75 ms/it)\n objective: 13458.610303038193\nMinimizing 1682 Time: 0:00:11 ( 6.75 ms/it)\n objective: 13458.228239980752\nMinimizing 1697 Time: 0:00:11 ( 6.75 ms/it)\n objective: 13457.721304545666\nMinimizing 1712 Time: 0:00:11 ( 6.75 ms/it)\n objective: 13457.037324119745\nMinimizing 1727 Time: 0:00:11 ( 6.75 ms/it)\n objective: 13456.408258280819\nMinimizing 1742 Time: 0:00:11 ( 6.75 ms/it)\n objective: 13455.766895393572\nMinimizing 1757 Time: 0:00:11 ( 6.75 ms/it)\n objective: 13454.371704913545\nMinimizing 1772 Time: 0:00:11 ( 6.75 ms/it)\n objective: 13450.595171898414\nMinimizing 1787 Time: 0:00:12 ( 6.75 ms/it)\n objective: 13450.523300232227\nMinimizing 1802 Time: 0:00:12 ( 6.75 ms/it)\n objective: 13450.339610162999\nMinimizing 1817 Time: 0:00:12 ( 6.75 ms/it)\n objective: 13450.016868499246\nMinimizing 1832 Time: 0:00:12 ( 6.75 ms/it)\n objective: 13449.94140149498\nMinimizing 1847 Time: 0:00:12 ( 6.76 ms/it)\n objective: 13449.816967594066\nMinimizing 1862 Time: 0:00:12 ( 6.76 ms/it)\n objective: 13449.319770789785\nMinimizing 1877 Time: 0:00:12 ( 6.76 ms/it)\n objective: 13449.110196556918\nMinimizing 1892 Time: 0:00:12 ( 6.76 ms/it)\n objective: 13448.795751941441\nMinimizing 1907 Time: 0:00:12 ( 6.76 ms/it)\n objective: 13449.074992096692\nMinimizing 1922 Time: 0:00:12 ( 6.76 ms/it)\n objective: 13447.39077325366\nMinimizing 1937 Time: 0:00:13 ( 6.76 ms/it)\n objective: 13445.09898862688\nMinimizing 1952 Time: 0:00:13 ( 6.76 ms/it)\n objective: 13444.932383720035\nMinimizing 1967 Time: 0:00:13 ( 6.77 ms/it)\n objective: 13445.604980674863\nMinimizing 1982 Time: 0:00:13 ( 6.77 ms/it)\n objective: 13442.275084992973\nMinimizing 1997 Time: 0:00:13 ( 6.77 ms/it)\n objective: 13442.149408422498\nMinimizing 2011 Time: 0:00:13 ( 6.77 ms/it)\n objective: 13441.947494954904\nMinimizing 2025 Time: 0:00:13 ( 6.77 ms/it)\n objective: 13441.802660236346\nMinimizing 2039 Time: 0:00:13 ( 6.78 ms/it)\n objective: 13441.735454740257\nMinimizing 2053 Time: 0:00:13 ( 6.78 ms/it)\n objective: 13441.283110526892\nMinimizing 2067 Time: 0:00:14 ( 6.79 ms/it)\n objective: 13441.215844876555\nMinimizing 2081 Time: 0:00:14 ( 6.79 ms/it)\n objective: 13440.91066331276\nMinimizing 2095 Time: 0:00:14 ( 6.79 ms/it)\n objective: 13440.737892112076\nMinimizing 2110 Time: 0:00:14 ( 6.79 ms/it)\n objective: 13440.314860408005\nMinimizing 2125 Time: 0:00:14 ( 6.79 ms/it)\n objective: 13440.131066346934\nMinimizing 2140 Time: 0:00:14 ( 6.79 ms/it)\n objective: 13439.823098394561\nMinimizing 2155 Time: 0:00:14 ( 6.79 ms/it)\n objective: 13437.977472611477\nMinimizing 2170 Time: 0:00:14 ( 6.79 ms/it)\n objective: 13436.813123262982\nMinimizing 2185 Time: 0:00:14 ( 6.79 ms/it)\n objective: 13436.027968008246\nMinimizing 2200 Time: 0:00:14 ( 6.80 ms/it)\n objective: 13435.469328884566\nMinimizing 2215 Time: 0:00:15 ( 6.80 ms/it)\n objective: 13435.015305421868\nMinimizing 2230 Time: 0:00:15 ( 6.80 ms/it)\n objective: 13434.804641040762\nMinimizing 2244 Time: 0:00:15 ( 6.80 ms/it)\n objective: 13434.639119015439\nMinimizing 2258 Time: 0:00:15 ( 6.80 ms/it)\n objective: 13434.345101108986\nMinimizing 2272 Time: 0:00:15 ( 6.81 ms/it)\n objective: 13434.085744287622\nMinimizing 2286 Time: 0:00:15 ( 6.81 ms/it)\n objective: 13433.73017433774\nMinimizing 2301 Time: 0:00:15 ( 6.81 ms/it)\n objective: 13433.444091123158\nMinimizing 2316 Time: 0:00:15 ( 6.81 ms/it)\n objective: 13432.535262732941\nMinimizing 2331 Time: 0:00:15 ( 6.81 ms/it)\n objective: 13432.028432277402\nMinimizing 2346 Time: 0:00:15 ( 6.81 ms/it)\n objective: 13431.749397252977\nMinimizing 2361 Time: 0:00:16 ( 6.81 ms/it)\n objective: 13431.02924160735\nMinimizing 2376 Time: 0:00:16 ( 6.81 ms/it)\n objective: 13430.340936352091\nMinimizing 2391 Time: 0:00:16 ( 6.81 ms/it)\n objective: 13430.250705790255\nMinimizing 2406 Time: 0:00:16 ( 6.81 ms/it)\n objective: 13428.7299317184\nMinimizing 2421 Time: 0:00:16 ( 6.81 ms/it)\n objective: 13428.600177746244\nMinimizing 2436 Time: 0:00:16 ( 6.81 ms/it)\n objective: 13428.35456343239\nMinimizing 2450 Time: 0:00:16 ( 6.82 ms/it)\n objective: 13428.2381590402\nMinimizing 2464 Time: 0:00:16 ( 6.82 ms/it)\n objective: 13427.619953700036\nMinimizing 2478 Time: 0:00:16 ( 6.82 ms/it)\n objective: 13427.031311257415\nMinimizing 2493 Time: 0:00:17 ( 6.83 ms/it)\n objective: 13426.884080365933\nMinimizing 2508 Time: 0:00:17 ( 6.83 ms/it)\n objective: 13426.658416826038\nMinimizing 2523 Time: 0:00:17 ( 6.83 ms/it)\n objective: 13426.43264934884\nMinimizing 2538 Time: 0:00:17 ( 6.83 ms/it)\n objective: 13425.969080593051\nMinimizing 2553 Time: 0:00:17 ( 6.83 ms/it)\n objective: 13425.901331297908\nMinimizing 2568 Time: 0:00:17 ( 6.83 ms/it)\n objective: 13425.53287567669\nMinimizing 2583 Time: 0:00:17 ( 6.83 ms/it)\n objective: 13425.311139249592\nMinimizing 2598 Time: 0:00:17 ( 6.83 ms/it)\n objective: 13425.09968612664\nMinimizing 2613 Time: 0:00:17 ( 6.83 ms/it)\n objective: 13424.68932567427\nMinimizing 2628 Time: 0:00:17 ( 6.83 ms/it)\n objective: 13424.200897974464\nMinimizing 2643 Time: 0:00:18 ( 6.83 ms/it)\n objective: 13423.5384676123\nMinimizing 2658 Time: 0:00:18 ( 6.83 ms/it)\n objective: 13421.729598823855\nMinimizing 2673 Time: 0:00:18 ( 6.83 ms/it)\n objective: 13421.398866678508\nMinimizing 2688 Time: 0:00:18 ( 6.83 ms/it)\n objective: 13420.907891894625\nMinimizing 2703 Time: 0:00:18 ( 6.83 ms/it)\n objective: 13420.67928072399\nMinimizing 2718 Time: 0:00:18 ( 6.83 ms/it)\n objective: 13420.383694176075\nMinimizing 2733 Time: 0:00:18 ( 6.83 ms/it)\n objective: 13420.288588365496\nMinimizing 2748 Time: 0:00:18 ( 6.83 ms/it)\n objective: 13420.079692674131\nMinimizing 2763 Time: 0:00:18 ( 6.83 ms/it)\n objective: 13418.959660698398\nMinimizing 2778 Time: 0:00:18 ( 6.83 ms/it)\n objective: 13417.069232054215\nMinimizing 2793 Time: 0:00:19 ( 6.83 ms/it)\n objective: 13416.976691040429\nMinimizing 2808 Time: 0:00:19 ( 6.83 ms/it)\n objective: 13416.388248267795\nMinimizing 2823 Time: 0:00:19 ( 6.83 ms/it)\n objective: 13415.979897269957\nMinimizing 2838 Time: 0:00:19 ( 6.83 ms/it)\n objective: 13415.476616105261\nMinimizing 2855 Time: 0:00:19 ( 6.83 ms/it)\n objective: 13415.205429758818\nMinimizing 2870 Time: 0:00:19 ( 6.83 ms/it)\n objective: 13414.602298967635\nMinimizing 2885 Time: 0:00:19 ( 6.83 ms/it)\n objective: 13414.430555188512\nMinimizing 2900 Time: 0:00:19 ( 6.83 ms/it)\n objective: 13413.691742222043\nMinimizing 2915 Time: 0:00:19 ( 6.83 ms/it)\n objective: 13411.808598661191\nMinimizing 2930 Time: 0:00:20 ( 6.83 ms/it)\n objective: 13411.001015575603\nMinimizing 2945 Time: 0:00:20 ( 6.83 ms/it)\n objective: 13410.683756899736\nMinimizing 2960 Time: 0:00:20 ( 6.83 ms/it)\n objective: 13410.392192634768\nMinimizing 2975 Time: 0:00:20 ( 6.83 ms/it)\n objective: 13410.297924170125\nMinimizing 2990 Time: 0:00:20 ( 6.83 ms/it)\n objective: 13410.246482225055\nMinimizing 3005 Time: 0:00:20 ( 6.83 ms/it)\n objective: 13410.182515826004\nMinimizing 3020 Time: 0:00:20 ( 6.83 ms/it)\n objective: 13410.126147990966\nMinimizing 3035 Time: 0:00:20 ( 6.83 ms/it)\n objective: 13409.887593711617\nMinimizing 3050 Time: 0:00:20 ( 6.83 ms/it)\n objective: 13409.737112608782\nMinimizing 3065 Time: 0:00:20 ( 6.83 ms/it)\n objective: 13409.542533619075\nMinimizing 3080 Time: 0:00:21 ( 6.83 ms/it)\n objective: 13410.713853535854\nMinimizing 3095 Time: 0:00:21 ( 6.83 ms/it)\n objective: 13408.999170151452\nMinimizing 3110 Time: 0:00:21 ( 6.83 ms/it)\n objective: 13408.846049140207\nMinimizing 3125 Time: 0:00:21 ( 6.83 ms/it)\n objective: 13408.81458514851\nMinimizing 3140 Time: 0:00:21 ( 6.83 ms/it)\n objective: 13408.628653805099\nMinimizing 3155 Time: 0:00:21 ( 6.83 ms/it)\n objective: 13408.458549214956\nMinimizing 3170 Time: 0:00:21 ( 6.83 ms/it)\n objective: 13408.082805525133\nMinimizing 3186 Time: 0:00:21 ( 6.83 ms/it)\n objective: 13407.901308307417\nMinimizing 3201 Time: 0:00:21 ( 6.83 ms/it)\n objective: 13407.508925649294\nMinimizing 3216 Time: 0:00:21 ( 6.83 ms/it)\n objective: 13407.4424865896\nMinimizing 3231 Time: 0:00:22 ( 6.83 ms/it)\n objective: 13407.325513060896\nMinimizing 3246 Time: 0:00:22 ( 6.83 ms/it)\n objective: 13407.269127944564\nMinimizing 3262 Time: 0:00:22 ( 6.83 ms/it)\n objective: 13407.121673193426\nMinimizing 3277 Time: 0:00:22 ( 6.83 ms/it)\n objective: 13406.938982763691\nMinimizing 3292 Time: 0:00:22 ( 6.83 ms/it)\n objective: 13406.739485440572\nMinimizing 3307 Time: 0:00:22 ( 6.83 ms/it)\n objective: 13406.386341910991\nMinimizing 3322 Time: 0:00:22 ( 6.83 ms/it)\n objective: 13406.33130509931\nMinimizing 3337 Time: 0:00:22 ( 6.83 ms/it)\n objective: 13406.250773542939\nMinimizing 3352 Time: 0:00:22 ( 6.83 ms/it)\n objective: 13406.178296293969\nMinimizing 3367 Time: 0:00:22 ( 6.83 ms/it)\n objective: 13405.990703369345\nMinimizing 3382 Time: 0:00:23 ( 6.83 ms/it)\n objective: 13405.963545486426\nMinimizing 3397 Time: 0:00:23 ( 6.83 ms/it)\n objective: 13405.643476446741\nMinimizing 3412 Time: 0:00:23 ( 6.83 ms/it)\n objective: 13405.063527673585\nMinimizing 3427 Time: 0:00:23 ( 6.83 ms/it)\n objective: 13405.02946117951\nMinimizing 3442 Time: 0:00:23 ( 6.83 ms/it)\n objective: 13404.856109886401\nMinimizing 3457 Time: 0:00:23 ( 6.83 ms/it)\n objective: 13404.64640477307\nMinimizing 3472 Time: 0:00:23 ( 6.83 ms/it)\n objective: 13404.525980073835\nMinimizing 3487 Time: 0:00:23 ( 6.83 ms/it)\n objective: 13404.262173605995\nMinimizing 3500 Time: 0:00:23 ( 6.84 ms/it)\n objective: 13404.127115545998\nMinimizing 3513 Time: 0:00:24 ( 6.84 ms/it)\n objective: 13403.94978845527\nMinimizing 3527 Time: 0:00:24 ( 6.85 ms/it)\n objective: 13403.706382205128\nMinimizing 3541 Time: 0:00:24 ( 6.85 ms/it)\n objective: 13403.639038920504\nMinimizing 3556 Time: 0:00:24 ( 6.85 ms/it)\n objective: 13403.25089431525\nMinimizing 3571 Time: 0:00:24 ( 6.85 ms/it)\n objective: 13402.97013170414\nMinimizing 3586 Time: 0:00:24 ( 6.85 ms/it)\n objective: 13402.734308727733\nMinimizing 3601 Time: 0:00:24 ( 6.85 ms/it)\n objective: 13402.567589494363\nMinimizing 3615 Time: 0:00:24 ( 6.85 ms/it)\n objective: 13402.068284457127\nMinimizing 3629 Time: 0:00:24 ( 6.85 ms/it)\n objective: 13401.89108302733\nMinimizing 3644 Time: 0:00:24 ( 6.86 ms/it)\n objective: 13401.434992449445\nMinimizing 3659 Time: 0:00:25 ( 6.86 ms/it)\n objective: 13401.339816215353\nMinimizing 3674 Time: 0:00:25 ( 6.86 ms/it)\n objective: 13401.011237968727\nMinimizing 3689 Time: 0:00:25 ( 6.86 ms/it)\n objective: 13400.899273724535\nMinimizing 3704 Time: 0:00:25 ( 6.86 ms/it)\n objective: 13400.631030999117\nMinimizing 3718 Time: 0:00:25 ( 6.86 ms/it)\n objective: 13400.615890685847\nMinimizing 3733 Time: 0:00:25 ( 6.86 ms/it)\n objective: 13400.506604714843\nMinimizing 3748 Time: 0:00:25 ( 6.86 ms/it)\n objective: 13400.043648063256\nMinimizing 3763 Time: 0:00:25 ( 6.86 ms/it)\n objective: 13400.009337708929\nMinimizing 3778 Time: 0:00:25 ( 6.86 ms/it)\n objective: 13399.856856851271\nMinimizing 3793 Time: 0:00:26 ( 6.86 ms/it)\n objective: 13399.587067141008\nMinimizing 3808 Time: 0:00:26 ( 6.86 ms/it)\n objective: 13399.033220450634\nMinimizing 3823 Time: 0:00:26 ( 6.86 ms/it)\n objective: 13399.005206893125\nMinimizing 3838 Time: 0:00:26 ( 6.86 ms/it)\n objective: 13398.836297056536\nMinimizing 3853 Time: 0:00:26 ( 6.86 ms/it)\n objective: 13398.758431093047\nMinimizing 3868 Time: 0:00:26 ( 6.86 ms/it)\n objective: 13398.346197751132\nMinimizing 3883 Time: 0:00:26 ( 6.86 ms/it)\n objective: 13398.290112862276\nMinimizing 3897 Time: 0:00:26 ( 6.86 ms/it)\n objective: 13397.998509687663\nMinimizing 3911 Time: 0:00:26 ( 6.88 ms/it)\n objective: 13397.885113499557\nMinimizing 3922 Time: 0:00:27 ( 6.89 ms/it)\n objective: 13397.802749459792\nMinimizing 3937 Time: 0:00:27 ( 6.89 ms/it)\n objective: 13397.628794346805\nMinimizing 3952 Time: 0:00:27 ( 6.89 ms/it)\n objective: 13397.438395749821\nMinimizing 3967 Time: 0:00:27 ( 6.89 ms/it)\n objective: 13396.853688099436\nMinimizing 3982 Time: 0:00:27 ( 6.89 ms/it)\n objective: 13396.742808832169\nMinimizing 3997 Time: 0:00:27 ( 6.89 ms/it)\n objective: 13396.691638782577\nMinimizing 4012 Time: 0:00:27 ( 6.89 ms/it)\n objective: 13396.607579192801\nMinimizing 4027 Time: 0:00:27 ( 6.89 ms/it)\n objective: 13396.434557549088\nMinimizing 4042 Time: 0:00:27 ( 6.89 ms/it)\n objective: 13396.2596687127\nMinimizing 4057 Time: 0:00:27 ( 6.89 ms/it)\n objective: 13396.050723842935\nMinimizing 4072 Time: 0:00:28 ( 6.89 ms/it)\n objective: 13395.983879727624\nMinimizing 4087 Time: 0:00:28 ( 6.89 ms/it)\n objective: 13395.615054056798\nMinimizing 4102 Time: 0:00:28 ( 6.89 ms/it)\n objective: 13395.489220327814\nMinimizing 4117 Time: 0:00:28 ( 6.89 ms/it)\n objective: 13395.208256389757\nMinimizing 4132 Time: 0:00:28 ( 6.89 ms/it)\n objective: 13394.964858138323\nMinimizing 4147 Time: 0:00:28 ( 6.89 ms/it)\n objective: 13394.68285716961\nMinimizing 4162 Time: 0:00:28 ( 6.89 ms/it)\n objective: 13394.539461565291\nMinimizing 4177 Time: 0:00:28 ( 6.89 ms/it)\n objective: 13394.443118022282\nMinimizing 4192 Time: 0:00:28 ( 6.89 ms/it)\n objective: 13394.209761507474\nMinimizing 4207 Time: 0:00:28 ( 6.89 ms/it)\n objective: 13393.935845590357\nMinimizing 4222 Time: 0:00:29 ( 6.88 ms/it)\n objective: 13393.666365170902\nMinimizing 4237 Time: 0:00:29 ( 6.88 ms/it)\n objective: 13393.213668260985\nMinimizing 4252 Time: 0:00:29 ( 6.88 ms/it)\n objective: 13392.56600051163\nMinimizing 4267 Time: 0:00:29 ( 6.88 ms/it)\n objective: 13392.499011912507\nMinimizing 4283 Time: 0:00:29 ( 6.88 ms/it)\n objective: 13392.431023451369\nMinimizing 4298 Time: 0:00:29 ( 6.88 ms/it)\n objective: 13392.377736224109\nMinimizing 4314 Time: 0:00:29 ( 6.88 ms/it)\n objective: 13392.264877478905\nMinimizing 4329 Time: 0:00:29 ( 6.88 ms/it)\n objective: 13392.168621039098\nMinimizing 4344 Time: 0:00:29 ( 6.88 ms/it)\n objective: 13391.880897793235\nMinimizing 4360 Time: 0:00:29 ( 6.88 ms/it)\n objective: 13391.8287680043\nMinimizing 4375 Time: 0:00:30 ( 6.88 ms/it)\n objective: 13391.529423234104\nMinimizing 4390 Time: 0:00:30 ( 6.88 ms/it)\n objective: 13391.242169677382\nMinimizing 4405 Time: 0:00:30 ( 6.88 ms/it)\n objective: 13391.056664459808\nMinimizing 4420 Time: 0:00:30 ( 6.88 ms/it)\n objective: 13391.017208363635\nMinimizing 4435 Time: 0:00:30 ( 6.88 ms/it)\n objective: 13390.991640826513\nMinimizing 4451 Time: 0:00:30 ( 6.88 ms/it)\n objective: 13390.90667081951\nMinimizing 4467 Time: 0:00:30 ( 6.88 ms/it)\n objective: 13390.799363491387\nMinimizing 4482 Time: 0:00:30 ( 6.88 ms/it)\n objective: 13390.760200463643\nMinimizing 4497 Time: 0:00:30 ( 6.88 ms/it)\n objective: 13390.714811117723\nMinimizing 4512 Time: 0:00:31 ( 6.88 ms/it)\n objective: 13390.518470292845\nMinimizing 4527 Time: 0:00:31 ( 6.88 ms/it)\n objective: 13390.329435973516\nMinimizing 4542 Time: 0:00:31 ( 6.88 ms/it)\n objective: 13390.078339683248\nMinimizing 4557 Time: 0:00:31 ( 6.88 ms/it)\n objective: 13390.031498748845\nMinimizing 4572 Time: 0:00:31 ( 6.88 ms/it)\n objective: 13389.949992722948\nMinimizing 4587 Time: 0:00:31 ( 6.87 ms/it)\n objective: 13389.763644823739\nMinimizing 4602 Time: 0:00:31 ( 6.87 ms/it)\n objective: 13389.62784133396\nMinimizing 4617 Time: 0:00:31 ( 6.87 ms/it)\n objective: 13389.460760133683\nMinimizing 4632 Time: 0:00:31 ( 6.87 ms/it)\n objective: 13389.016476854988\nMinimizing 4647 Time: 0:00:31 ( 6.87 ms/it)\n objective: 13388.959273912442\nMinimizing 4662 Time: 0:00:32 ( 6.87 ms/it)\n objective: 13388.891763682856\nMinimizing 4677 Time: 0:00:32 ( 6.87 ms/it)\n objective: 13388.820401609235\nMinimizing 4692 Time: 0:00:32 ( 6.87 ms/it)\n objective: 13388.751192318137\nMinimizing 4707 Time: 0:00:32 ( 6.87 ms/it)\n objective: 13388.739910386357\nMinimizing 4722 Time: 0:00:32 ( 6.87 ms/it)\n objective: 13388.503706668635\nMinimizing 4737 Time: 0:00:32 ( 6.87 ms/it)\n objective: 13388.459867990445\nMinimizing 4752 Time: 0:00:32 ( 6.87 ms/it)\n objective: 13388.279306965618\nMinimizing 4767 Time: 0:00:32 ( 6.87 ms/it)\n objective: 13388.189210024313\nMinimizing 4782 Time: 0:00:32 ( 6.87 ms/it)\n objective: 13387.92470124815\nMinimizing 4797 Time: 0:00:32 ( 6.87 ms/it)\n objective: 13387.89868886891\nMinimizing 4812 Time: 0:00:33 ( 6.87 ms/it)\n objective: 13387.377972867136\nMinimizing 4827 Time: 0:00:33 ( 6.87 ms/it)\n objective: 13387.34641313323\nMinimizing 4842 Time: 0:00:33 ( 6.87 ms/it)\n objective: 13387.2694558531\nMinimizing 4857 Time: 0:00:33 ( 6.87 ms/it)\n objective: 13387.127526261815\nMinimizing 4872 Time: 0:00:33 ( 6.87 ms/it)\n objective: 13386.991657510138\nMinimizing 4887 Time: 0:00:33 ( 6.87 ms/it)\n objective: 13386.90283250594\nMinimizing 4902 Time: 0:00:33 ( 6.87 ms/it)\n objective: 13386.67754627928\nMinimizing 4917 Time: 0:00:33 ( 6.87 ms/it)\n objective: 13386.321305879152\nMinimizing 4932 Time: 0:00:33 ( 6.87 ms/it)\n objective: 13386.26595318252\nMinimizing 4947 Time: 0:00:33 ( 6.87 ms/it)\n objective: 13386.099825550475\nMinimizing 4962 Time: 0:00:34 ( 6.87 ms/it)\n objective: 13385.919946994349\nMinimizing 4977 Time: 0:00:34 ( 6.87 ms/it)\n objective: 13385.149191722252\nMinimizing 4992 Time: 0:00:34 ( 6.87 ms/it)\n objective: 13384.964423516161\nMinimizing 5007 Time: 0:00:34 ( 6.87 ms/it)\n objective: 13384.904525178492\nMinimizing 5022 Time: 0:00:34 ( 6.87 ms/it)\n objective: 13384.757298589007\nMinimizing 5038 Time: 0:00:34 ( 6.87 ms/it)\n objective: 13384.660289430067\nMinimizing 5054 Time: 0:00:34 ( 6.87 ms/it)\n objective: 13384.340072467508\nMinimizing 5069 Time: 0:00:34 ( 6.87 ms/it)\n objective: 13384.120576951216\nMinimizing 5084 Time: 0:00:34 ( 6.87 ms/it)\n objective: 13384.066943781087\nMinimizing 5099 Time: 0:00:35 ( 6.87 ms/it)\n objective: 13383.882201841698\nMinimizing 5115 Time: 0:00:35 ( 6.87 ms/it)\n objective: 13383.86097009945\nMinimizing 5130 Time: 0:00:35 ( 6.87 ms/it)\n objective: 13383.76770953826\nMinimizing 5145 Time: 0:00:35 ( 6.87 ms/it)\n objective: 13383.843311364195\nMinimizing 5160 Time: 0:00:35 ( 6.87 ms/it)\n objective: 13383.547071372173\nMinimizing 5175 Time: 0:00:35 ( 6.87 ms/it)\n objective: 13383.361992406477\nMinimizing 5190 Time: 0:00:35 ( 6.86 ms/it)\n objective: 13383.265792056372\nMinimizing 5205 Time: 0:00:35 ( 6.86 ms/it)\n objective: 13383.237556206703\nMinimizing 5220 Time: 0:00:35 ( 6.86 ms/it)\n objective: 13383.122292311135\nMinimizing 5235 Time: 0:00:35 ( 6.86 ms/it)\n objective: 13383.091164666985\nMinimizing 5250 Time: 0:00:36 ( 6.86 ms/it)\n objective: 13382.82807980945\nMinimizing 5265 Time: 0:00:36 ( 6.86 ms/it)\n objective: 13382.771836568187\nMinimizing 5280 Time: 0:00:36 ( 6.86 ms/it)\n objective: 13382.442727021204\nMinimizing 5295 Time: 0:00:36 ( 6.86 ms/it)\n objective: 13382.337126181345\nMinimizing 5310 Time: 0:00:36 ( 6.86 ms/it)\n objective: 13382.279183836028\nMinimizing 5325 Time: 0:00:36 ( 6.86 ms/it)\n objective: 13382.159433179433\nMinimizing 5340 Time: 0:00:36 ( 6.86 ms/it)\n objective: 13381.88664285669\nMinimizing 5355 Time: 0:00:36 ( 6.86 ms/it)\n objective: 13381.856052897834\nMinimizing 5370 Time: 0:00:36 ( 6.86 ms/it)\n objective: 13381.749828866079\nMinimizing 5385 Time: 0:00:36 ( 6.86 ms/it)\n objective: 13381.710138183313\nMinimizing 5400 Time: 0:00:37 ( 6.86 ms/it)\n objective: 13381.588205927212\nMinimizing 5414 Time: 0:00:37 ( 6.86 ms/it)\n objective: 13381.42252826711\nMinimizing 5429 Time: 0:00:37 ( 6.86 ms/it)\n objective: 13380.861420636233\nMinimizing 5444 Time: 0:00:37 ( 6.87 ms/it)\n objective: 13380.836075844505\nMinimizing 5459 Time: 0:00:37 ( 6.87 ms/it)\n objective: 13380.768408174998\nMinimizing 5474 Time: 0:00:37 ( 6.87 ms/it)\n objective: 13380.680055026627\nMinimizing 5489 Time: 0:00:37 ( 6.87 ms/it)\n objective: 13380.506751959038\nMinimizing 5504 Time: 0:00:37 ( 6.87 ms/it)\n objective: 13380.435753297963\nMinimizing 5519 Time: 0:00:37 ( 6.87 ms/it)\n objective: 13380.185214291741\nMinimizing 5533 Time: 0:00:38 ( 6.87 ms/it)\n objective: 13380.180568083852\nMinimizing 5548 Time: 0:00:38 ( 6.87 ms/it)\n objective: 13380.070313659075\nMinimizing 5563 Time: 0:00:38 ( 6.87 ms/it)\n objective: 13379.94142494828\nMinimizing 5578 Time: 0:00:38 ( 6.87 ms/it)\n objective: 13379.905747871795\nMinimizing 5593 Time: 0:00:38 ( 6.87 ms/it)\n objective: 13379.830238168375\nMinimizing 5608 Time: 0:00:38 ( 6.87 ms/it)\n objective: 13379.456550914256\nMinimizing 5623 Time: 0:00:38 ( 6.87 ms/it)\n objective: 13379.233924313361\nMinimizing 5638 Time: 0:00:38 ( 6.87 ms/it)\n objective: 13378.129228646212\nMinimizing 5653 Time: 0:00:38 ( 6.87 ms/it)\n objective: 13377.79493796228\nMinimizing 5668 Time: 0:00:38 ( 6.87 ms/it)\n objective: 13377.67468089141\nMinimizing 5683 Time: 0:00:39 ( 6.87 ms/it)\n objective: 13377.654639531625\nMinimizing 5698 Time: 0:00:39 ( 6.87 ms/it)\n objective: 13377.628983467628\nMinimizing 5713 Time: 0:00:39 ( 6.87 ms/it)\n objective: 13377.620588554608\nMinimizing 5728 Time: 0:00:39 ( 6.87 ms/it)\n objective: 13377.553407903593\nMinimizing 5743 Time: 0:00:39 ( 6.87 ms/it)\n objective: 13377.493974512232\nMinimizing 5758 Time: 0:00:39 ( 6.87 ms/it)\n objective: 13377.463183533386\nMinimizing 5773 Time: 0:00:39 ( 6.87 ms/it)\n objective: 13377.426853135068\nMinimizing 5788 Time: 0:00:39 ( 6.87 ms/it)\n objective: 13377.415010865836\nMinimizing 5803 Time: 0:00:39 ( 6.87 ms/it)\n objective: 13377.365693036067\nMinimizing 5818 Time: 0:00:39 ( 6.87 ms/it)\n objective: 13377.35246267174\nMinimizing 5833 Time: 0:00:40 ( 6.87 ms/it)\n objective: 13377.348444642368\nMinimizing 5848 Time: 0:00:40 ( 6.87 ms/it)\n objective: 13377.342267280903\nMinimizing 5864 Time: 0:00:40 ( 6.87 ms/it)\n objective: 13377.298746704757\nMinimizing 5879 Time: 0:00:40 ( 6.87 ms/it)\n objective: 13377.28087785084\nMinimizing 5894 Time: 0:00:40 ( 6.87 ms/it)\n objective: 13377.2489520995\nMinimizing 5909 Time: 0:00:40 ( 6.87 ms/it)\n objective: 13377.228498033066\nMinimizing 5924 Time: 0:00:40 ( 6.87 ms/it)\n objective: 13377.183408165278\nMinimizing 5939 Time: 0:00:40 ( 6.87 ms/it)\n objective: 13377.137680763175\nMinimizing 5954 Time: 0:00:40 ( 6.87 ms/it)\n objective: 13377.08602648355\nMinimizing 5969 Time: 0:00:41 ( 6.87 ms/it)\n objective: 13377.067422838132\nMinimizing 5984 Time: 0:00:41 ( 6.87 ms/it)\n objective: 13376.928430755637\nMinimizing 5999 Time: 0:00:41 ( 6.87 ms/it)\n objective: 13376.79084285604\nMinimizing 6014 Time: 0:00:41 ( 6.87 ms/it)\n objective: 13376.744470016034\nMinimizing 6029 Time: 0:00:41 ( 6.87 ms/it)\n objective: 13376.660883232995\nMinimizing 6042 Time: 0:00:41 ( 6.87 ms/it)\n objective: 13376.652006979835\nMinimizing 6056 Time: 0:00:41 ( 6.87 ms/it)\n objective: 13376.579470635406\nMinimizing 6071 Time: 0:00:41 ( 6.87 ms/it)\n objective: 13376.531513897833\nMinimizing 6086 Time: 0:00:41 ( 6.87 ms/it)\n objective: 13376.455603861272\nMinimizing 6100 Time: 0:00:41 ( 6.88 ms/it)\n objective: 13376.441846797563\nMinimizing 6115 Time: 0:00:42 ( 6.88 ms/it)\n objective: 13376.345929743504\nMinimizing 6130 Time: 0:00:42 ( 6.88 ms/it)\n objective: 13376.331609797831\nMinimizing 6145 Time: 0:00:42 ( 6.88 ms/it)\n objective: 13376.13132879656\nMinimizing 6160 Time: 0:00:42 ( 6.88 ms/it)\n objective: 13376.12029260354\nMinimizing 6175 Time: 0:00:42 ( 6.88 ms/it)\n objective: 13375.990249421564\nMinimizing 6191 Time: 0:00:42 ( 6.87 ms/it)\n objective: 13375.953330678916\nMinimizing 6207 Time: 0:00:42 ( 6.87 ms/it)\n objective: 13375.688983930486\nMinimizing 6223 Time: 0:00:42 ( 6.87 ms/it)\n objective: 13375.642318482853\nMinimizing 6238 Time: 0:00:42 ( 6.87 ms/it)\n objective: 13375.62789762528\nMinimizing 6253 Time: 0:00:42 ( 6.87 ms/it)\n objective: 13375.575289033928\nMinimizing 6268 Time: 0:00:43 ( 6.87 ms/it)\n objective: 13375.542532701766\nMinimizing 6283 Time: 0:00:43 ( 6.87 ms/it)\n objective: 13375.474428153415\nMinimizing 6298 Time: 0:00:43 ( 6.87 ms/it)\n objective: 13375.448782051913\nMinimizing 6313 Time: 0:00:43 ( 6.87 ms/it)\n objective: 13375.42897508297\nMinimizing 6328 Time: 0:00:43 ( 6.87 ms/it)\n objective: 13375.383805562073\nMinimizing 6343 Time: 0:00:43 ( 6.87 ms/it)\n objective: 13375.351178518737\nMinimizing 6358 Time: 0:00:43 ( 6.87 ms/it)\n objective: 13375.246187327422\nMinimizing 6373 Time: 0:00:43 ( 6.87 ms/it)\n objective: 13375.234876103808\nMinimizing 6388 Time: 0:00:43 ( 6.87 ms/it)\n objective: 13375.202533653195\nMinimizing 6403 Time: 0:00:44 ( 6.87 ms/it)\n objective: 13375.142522374634\nMinimizing 6418 Time: 0:00:44 ( 6.87 ms/it)\n objective: 13375.090950316771\nMinimizing 6433 Time: 0:00:44 ( 6.87 ms/it)\n objective: 13375.000291092656\nMinimizing 6448 Time: 0:00:44 ( 6.87 ms/it)\n objective: 13374.897247572851\nMinimizing 6463 Time: 0:00:44 ( 6.87 ms/it)\n objective: 13374.66669194882\nMinimizing 6478 Time: 0:00:44 ( 6.87 ms/it)\n objective: 13374.604318647675\nMinimizing 6493 Time: 0:00:44 ( 6.87 ms/it)\n objective: 13374.532537273699\nMinimizing 6508 Time: 0:00:44 ( 6.87 ms/it)\n objective: 13374.450593598376\nMinimizing 6523 Time: 0:00:44 ( 6.87 ms/it)\n objective: 13374.410235455143\nMinimizing 6538 Time: 0:00:44 ( 6.87 ms/it)\n objective: 13374.335032133211\nMinimizing 6553 Time: 0:00:45 ( 6.87 ms/it)\n objective: 13374.311289281868\nMinimizing 6568 Time: 0:00:45 ( 6.87 ms/it)\n objective: 13374.180349728704\nMinimizing 6583 Time: 0:00:45 ( 6.87 ms/it)\n objective: 13373.9969534653\nMinimizing 6598 Time: 0:00:45 ( 6.87 ms/it)\n objective: 13373.870840727373\nMinimizing 6613 Time: 0:00:45 ( 6.87 ms/it)\n objective: 13373.816123240897\nMinimizing 6628 Time: 0:00:45 ( 6.87 ms/it)\n objective: 13373.754243562558\nMinimizing 6643 Time: 0:00:45 ( 6.87 ms/it)\n objective: 13373.713547662323\nMinimizing 6656 Time: 0:00:45 ( 6.88 ms/it)\n objective: 13373.691049970112\nMinimizing 6671 Time: 0:00:45 ( 6.88 ms/it)\n objective: 13373.606736935537\nMinimizing 6686 Time: 0:00:45 ( 6.88 ms/it)\n objective: 13373.544961816653\nMinimizing 6701 Time: 0:00:46 ( 6.88 ms/it)\n objective: 13373.454028481472\nMinimizing 6716 Time: 0:00:46 ( 6.88 ms/it)\n objective: 13373.407356424868\nMinimizing 6731 Time: 0:00:46 ( 6.88 ms/it)\n objective: 13373.266439249994\nMinimizing 6746 Time: 0:00:46 ( 6.88 ms/it)\n objective: 13373.08355878386\nMinimizing 6761 Time: 0:00:46 ( 6.88 ms/it)\n objective: 13372.700572223679\nMinimizing 6776 Time: 0:00:46 ( 6.88 ms/it)\n objective: 13371.965579579111\nMinimizing 6791 Time: 0:00:46 ( 6.88 ms/it)\n objective: 13371.926754086373\nMinimizing 6806 Time: 0:00:46 ( 6.88 ms/it)\n objective: 13371.890815737628\nMinimizing 6821 Time: 0:00:46 ( 6.88 ms/it)\n objective: 13371.825391745828\nMinimizing 6836 Time: 0:00:47 ( 6.88 ms/it)\n objective: 13371.783398776293\nMinimizing 6851 Time: 0:00:47 ( 6.88 ms/it)\n objective: 13371.692600466078\nMinimizing 6866 Time: 0:00:47 ( 6.88 ms/it)\n objective: 13371.562734198014\nMinimizing 6881 Time: 0:00:47 ( 6.88 ms/it)\n objective: 13371.441694604473\nMinimizing 6896 Time: 0:00:47 ( 6.88 ms/it)\n objective: 13371.37050822581\nMinimizing 6911 Time: 0:00:47 ( 6.88 ms/it)\n objective: 13371.317465892302\nMinimizing 6927 Time: 0:00:47 ( 6.88 ms/it)\n objective: 13370.770915597685\nMinimizing 6943 Time: 0:00:47 ( 6.87 ms/it)\n objective: 13370.61077787495\nMinimizing 6958 Time: 0:00:47 ( 6.87 ms/it)\n objective: 13370.590431240962\nMinimizing 6973 Time: 0:00:47 ( 6.87 ms/it)\n objective: 13370.578297688822\nMinimizing 6988 Time: 0:00:48 ( 6.87 ms/it)\n objective: 13370.557051298892\nMinimizing 7003 Time: 0:00:48 ( 6.87 ms/it)\n objective: 13370.491426723544\nMinimizing 7018 Time: 0:00:48 ( 6.87 ms/it)\n objective: 13370.45833799184\nMinimizing 7033 Time: 0:00:48 ( 6.87 ms/it)\n objective: 13370.303519399778\nMinimizing 7048 Time: 0:00:48 ( 6.88 ms/it)\n objective: 13370.130961288618\nMinimizing 7063 Time: 0:00:48 ( 6.87 ms/it)\n objective: 13370.114242299387\nMinimizing 7078 Time: 0:00:48 ( 6.87 ms/it)\n objective: 13370.104257503328\nMinimizing 7093 Time: 0:00:48 ( 6.87 ms/it)\n objective: 13370.079336794166\nMinimizing 7108 Time: 0:00:48 ( 6.87 ms/it)\n objective: 13370.054328481412\nMinimizing 7123 Time: 0:00:48 ( 6.87 ms/it)\n objective: 13370.016732800585\nMinimizing 7138 Time: 0:00:49 ( 6.87 ms/it)\n objective: 13369.981639876583\nMinimizing 7153 Time: 0:00:49 ( 6.87 ms/it)\n objective: 13369.964754558721\nMinimizing 7168 Time: 0:00:49 ( 6.87 ms/it)\n objective: 13369.912558451513\nMinimizing 7183 Time: 0:00:49 ( 6.87 ms/it)\n objective: 13369.879356093967\nMinimizing 7198 Time: 0:00:49 ( 6.87 ms/it)\n objective: 13369.755105152108\nMinimizing 7213 Time: 0:00:49 ( 6.87 ms/it)\n objective: 13369.750238435037\nMinimizing 7228 Time: 0:00:49 ( 6.87 ms/it)\n objective: 13369.711429808987\nMinimizing 7243 Time: 0:00:49 ( 6.87 ms/it)\n objective: 13369.671289654492\nMinimizing 7258 Time: 0:00:49 ( 6.87 ms/it)\n objective: 13369.589318805323\nMinimizing 7273 Time: 0:00:49 ( 6.87 ms/it)\n objective: 13369.569123039473\nMinimizing 7288 Time: 0:00:50 ( 6.87 ms/it)\n objective: 13369.546600320944\nMinimizing 7303 Time: 0:00:50 ( 6.87 ms/it)\n objective: 13369.506954672463\nMinimizing 7318 Time: 0:00:50 ( 6.87 ms/it)\n objective: 13369.490255486591\nMinimizing 7333 Time: 0:00:50 ( 6.87 ms/it)\n objective: 13369.475027861248\nMinimizing 7348 Time: 0:00:50 ( 6.87 ms/it)\n objective: 13369.41372521386\nMinimizing 7363 Time: 0:00:50 ( 6.87 ms/it)\n objective: 13369.384289339381\nMinimizing 7378 Time: 0:00:50 ( 6.87 ms/it)\n objective: 13369.31353933843\nMinimizing 7393 Time: 0:00:50 ( 6.87 ms/it)\n objective: 13369.240696109307\nMinimizing 7408 Time: 0:00:50 ( 6.87 ms/it)\n objective: 13369.214018439532\nMinimizing 7423 Time: 0:00:51 ( 6.87 ms/it)\n objective: 13369.021392362374\nMinimizing 7437 Time: 0:00:51 ( 6.87 ms/it)\n objective: 13369.016678510023\nMinimizing 7451 Time: 0:00:51 ( 6.88 ms/it)\n objective: 13368.96123365561\nMinimizing 7465 Time: 0:00:51 ( 6.88 ms/it)\n objective: 13368.904309428835\nMinimizing 7479 Time: 0:00:51 ( 6.88 ms/it)\n objective: 13368.860184313882\nMinimizing 7493 Time: 0:00:51 ( 6.88 ms/it)\n objective: 13368.8439067129\nMinimizing 7507 Time: 0:00:51 ( 6.88 ms/it)\n objective: 13368.717814799435\nMinimizing 7522 Time: 0:00:51 ( 6.88 ms/it)\n objective: 13368.343124411134\nMinimizing 7537 Time: 0:00:51 ( 6.88 ms/it)\n objective: 13368.311637371691\nMinimizing 7552 Time: 0:00:51 ( 6.88 ms/it)\n objective: 13368.280821540306\nMinimizing 7567 Time: 0:00:52 ( 6.88 ms/it)\n objective: 13368.138206312658\nMinimizing 7582 Time: 0:00:52 ( 6.88 ms/it)\n objective: 13368.042496879469\nMinimizing 7597 Time: 0:00:52 ( 6.88 ms/it)\n objective: 13367.98447241549\nMinimizing 7612 Time: 0:00:52 ( 6.88 ms/it)\n objective: 13367.969536239238\nMinimizing 7627 Time: 0:00:52 ( 6.88 ms/it)\n objective: 13367.92077512548\nMinimizing 7642 Time: 0:00:52 ( 6.88 ms/it)\n objective: 13367.882886299041\nMinimizing 7657 Time: 0:00:52 ( 6.88 ms/it)\n objective: 13367.750793392719\nMinimizing 7672 Time: 0:00:52 ( 6.88 ms/it)\n objective: 13367.627396631491\nMinimizing 7687 Time: 0:00:52 ( 6.88 ms/it)\n objective: 13367.451115931457\nMinimizing 7702 Time: 0:00:52 ( 6.88 ms/it)\n objective: 13367.317819530603\nMinimizing 7717 Time: 0:00:53 ( 6.88 ms/it)\n objective: 13367.210093133785\nMinimizing 7732 Time: 0:00:53 ( 6.88 ms/it)\n objective: 13367.199462880133\nMinimizing 7747 Time: 0:00:53 ( 6.88 ms/it)\n objective: 13367.186724468564\nMinimizing 7762 Time: 0:00:53 ( 6.88 ms/it)\n objective: 13367.145439082844\nMinimizing 7777 Time: 0:00:53 ( 6.88 ms/it)\n objective: 13367.069368550663\nMinimizing 7792 Time: 0:00:53 ( 6.88 ms/it)\n objective: 13367.058887317871\nMinimizing 7807 Time: 0:00:53 ( 6.88 ms/it)\n objective: 13367.049009146838\nMinimizing 7822 Time: 0:00:53 ( 6.88 ms/it)\n objective: 13367.012231147062\nMinimizing 7837 Time: 0:00:53 ( 6.88 ms/it)\n objective: 13366.973243621316\nMinimizing 7852 Time: 0:00:53 ( 6.88 ms/it)\n objective: 13366.878363601681\nMinimizing 7867 Time: 0:00:54 ( 6.88 ms/it)\n objective: 13366.874336539528\nMinimizing 7882 Time: 0:00:54 ( 6.88 ms/it)\n objective: 13366.835411262255\nMinimizing 7897 Time: 0:00:54 ( 6.88 ms/it)\n objective: 13366.810178074462\nMinimizing 7912 Time: 0:00:54 ( 6.88 ms/it)\n objective: 13366.789897525887\nMinimizing 7927 Time: 0:00:54 ( 6.88 ms/it)\n objective: 13366.74776966164\nMinimizing 7942 Time: 0:00:54 ( 6.88 ms/it)\n objective: 13366.709351786172\nMinimizing 7957 Time: 0:00:54 ( 6.88 ms/it)\n objective: 13366.694821793048\nMinimizing 7972 Time: 0:00:54 ( 6.88 ms/it)\n objective: 13366.57840019682\nMinimizing 7987 Time: 0:00:54 ( 6.88 ms/it)\n objective: 13366.470667255584\nMinimizing 8002 Time: 0:00:55 ( 6.88 ms/it)\n objective: 13366.437363007673\nMinimizing 8017 Time: 0:00:55 ( 6.88 ms/it)\n objective: 13366.408919171648\nMinimizing 8032 Time: 0:00:55 ( 6.88 ms/it)\n objective: 13366.377921794956\nMinimizing 8047 Time: 0:00:55 ( 6.88 ms/it)\n objective: 13366.092883387093\nMinimizing 8062 Time: 0:00:55 ( 6.88 ms/it)\n objective: 13366.073007818377\nMinimizing 8077 Time: 0:00:55 ( 6.88 ms/it)\n objective: 13365.751529599023\nMinimizing 8092 Time: 0:00:55 ( 6.88 ms/it)\n objective: 13365.746638929042\nMinimizing 8108 Time: 0:00:55 ( 6.87 ms/it)\n objective: 13365.709298701142\nMinimizing 8123 Time: 0:00:55 ( 6.87 ms/it)\n objective: 13365.667712559138\nMinimizing 8138 Time: 0:00:55 ( 6.87 ms/it)\n objective: 13365.549607618836\nMinimizing 8153 Time: 0:00:56 ( 6.87 ms/it)\n objective: 13365.538019369007\nMinimizing 8168 Time: 0:00:56 ( 6.87 ms/it)\n objective: 13365.482405331248\nMinimizing 8183 Time: 0:00:56 ( 6.87 ms/it)\n objective: 13365.329488810814\nMinimizing 8198 Time: 0:00:56 ( 6.87 ms/it)\n objective: 13365.304353701904\nMinimizing 8214 Time: 0:00:56 ( 6.87 ms/it)\n objective: 13365.251173428536\nMinimizing 8230 Time: 0:00:56 ( 6.87 ms/it)\n objective: 13365.20377052974\nMinimizing 8245 Time: 0:00:56 ( 6.87 ms/it)\n objective: 13365.147682533097\nMinimizing 8260 Time: 0:00:56 ( 6.87 ms/it)\n objective: 13364.990440609196\nMinimizing 8275 Time: 0:00:56 ( 6.87 ms/it)\n objective: 13364.914183577748\nMinimizing 8290 Time: 0:00:56 ( 6.87 ms/it)\n objective: 13364.85599994762\nMinimizing 8305 Time: 0:00:57 ( 6.87 ms/it)\n objective: 13364.803023507819\nMinimizing 8320 Time: 0:00:57 ( 6.87 ms/it)\n objective: 13364.792236169873\nMinimizing 8335 Time: 0:00:57 ( 6.87 ms/it)\n objective: 13364.744402544049\nMinimizing 8350 Time: 0:00:57 ( 6.87 ms/it)\n objective: 13364.65439851064\nMinimizing 8365 Time: 0:00:57 ( 6.87 ms/it)\n objective: 13364.492123087126\nMinimizing 8380 Time: 0:00:57 ( 6.87 ms/it)\n objective: 13364.453285737145\nMinimizing 8395 Time: 0:00:57 ( 6.87 ms/it)\n objective: 13364.443538133251\nMinimizing 8410 Time: 0:00:57 ( 6.87 ms/it)\n objective: 13364.395906754144\nMinimizing 8425 Time: 0:00:57 ( 6.87 ms/it)\n objective: 13364.340842878904\nMinimizing 8440 Time: 0:00:58 ( 6.87 ms/it)\n objective: 13364.31954021076\nMinimizing 8454 Time: 0:00:58 ( 6.87 ms/it)\n objective: 13364.15082419027\nMinimizing 8469 Time: 0:00:58 ( 6.87 ms/it)\n objective: 13363.946133674239\nMinimizing 8484 Time: 0:00:58 ( 6.87 ms/it)\n objective: 13363.877118735014\nMinimizing 8499 Time: 0:00:58 ( 6.87 ms/it)\n objective: 13363.84631438554\nMinimizing 8514 Time: 0:00:58 ( 6.87 ms/it)\n objective: 13363.798975420104\nMinimizing 8529 Time: 0:00:58 ( 6.87 ms/it)\n objective: 13363.780800568034\nMinimizing 8544 Time: 0:00:58 ( 6.87 ms/it)\n objective: 13363.698922377065\nMinimizing 8559 Time: 0:00:58 ( 6.87 ms/it)\n objective: 13363.570915435797\nMinimizing 8574 Time: 0:00:58 ( 6.87 ms/it)\n objective: 13363.552328472368\nMinimizing 8589 Time: 0:00:59 ( 6.87 ms/it)\n objective: 13363.501190475516\nMinimizing 8604 Time: 0:00:59 ( 6.87 ms/it)\n objective: 13363.336162788073\nMinimizing 8619 Time: 0:00:59 ( 6.87 ms/it)\n objective: 13362.974966944355\nMinimizing 8634 Time: 0:00:59 ( 6.87 ms/it)\n objective: 13362.968368875212\nMinimizing 8649 Time: 0:00:59 ( 6.87 ms/it)\n objective: 13362.866489196058\nMinimizing 8664 Time: 0:00:59 ( 6.87 ms/it)\n objective: 13362.838600096045\nMinimizing 8679 Time: 0:00:59 ( 6.87 ms/it)\n objective: 13362.808891039924\nMinimizing 8694 Time: 0:00:59 ( 6.87 ms/it)\n objective: 13362.527900107394\nMinimizing 8709 Time: 0:00:59 ( 6.87 ms/it)\n objective: 13362.508007631404\nMinimizing 8724 Time: 0:00:59 ( 6.87 ms/it)\n objective: 13362.48485447964\nMinimizing 8740 Time: 0:01:00 ( 6.87 ms/it)\n objective: 13362.410317078422\nMinimizing 8755 Time: 0:01:00 ( 6.87 ms/it)\n objective: 13362.333785017021\nMinimizing 8770 Time: 0:01:00 ( 6.87 ms/it)\n objective: 13362.27268205415\nMinimizing 8785 Time: 0:01:00 ( 6.87 ms/it)\n objective: 13362.048657450992\nMinimizing 8800 Time: 0:01:00 ( 6.87 ms/it)\n objective: 13361.86490765214\nMinimizing 8815 Time: 0:01:00 ( 6.87 ms/it)\n objective: 13361.845050337884\nMinimizing 8830 Time: 0:01:00 ( 6.87 ms/it)\n objective: 13361.821267377585\nMinimizing 8845 Time: 0:01:00 ( 6.87 ms/it)\n objective: 13361.770521018887\nMinimizing 8860 Time: 0:01:00 ( 6.87 ms/it)\n objective: 13361.694090222045\nMinimizing 8875 Time: 0:01:00 ( 6.87 ms/it)\n objective: 13361.640573441233\nMinimizing 8890 Time: 0:01:01 ( 6.87 ms/it)\n objective: 13361.50920443482\nMinimizing 8905 Time: 0:01:01 ( 6.87 ms/it)\n objective: 13361.43440077248\nMinimizing 8920 Time: 0:01:01 ( 6.87 ms/it)\n objective: 13361.119453477986\nMinimizing 8935 Time: 0:01:01 ( 6.87 ms/it)\n objective: 13361.095143978411\nMinimizing 8950 Time: 0:01:01 ( 6.87 ms/it)\n objective: 13361.059780435193\nMinimizing 8965 Time: 0:01:01 ( 6.87 ms/it)\n objective: 13361.026289798669\nMinimizing 8980 Time: 0:01:01 ( 6.87 ms/it)\n objective: 13360.975520260952\nMinimizing 8995 Time: 0:01:01 ( 6.87 ms/it)\n objective: 13360.867847210837\nMinimizing 9010 Time: 0:01:01 ( 6.87 ms/it)\n objective: 13360.859843643819\nMinimizing 9025 Time: 0:01:01 ( 6.87 ms/it)\n objective: 13360.79965364631\nMinimizing 9040 Time: 0:01:02 ( 6.87 ms/it)\n objective: 13360.78628051134\nMinimizing 9055 Time: 0:01:02 ( 6.87 ms/it)\n objective: 13360.75043879226\nMinimizing 9070 Time: 0:01:02 ( 6.87 ms/it)\n objective: 13360.64209006112\nMinimizing 9085 Time: 0:01:02 ( 6.87 ms/it)\n objective: 13360.62469935129\nMinimizing 9100 Time: 0:01:02 ( 6.87 ms/it)\n objective: 13360.548142264728\nMinimizing 9115 Time: 0:01:02 ( 6.87 ms/it)\n objective: 13360.518455400656\nMinimizing 9130 Time: 0:01:02 ( 6.87 ms/it)\n objective: 13360.302842979363\nMinimizing 9145 Time: 0:01:02 ( 6.87 ms/it)\n objective: 13360.242952398636\nMinimizing 9160 Time: 0:01:02 ( 6.87 ms/it)\n objective: 13360.169740957208\nMinimizing 9175 Time: 0:01:03 ( 6.87 ms/it)\n objective: 13359.947214352076\nMinimizing 9190 Time: 0:01:03 ( 6.87 ms/it)\n objective: 13359.868346662057\nMinimizing 9205 Time: 0:01:03 ( 6.87 ms/it)\n objective: 13359.807298778796\nMinimizing 9220 Time: 0:01:03 ( 6.87 ms/it)\n objective: 13359.733491990533\nMinimizing 9235 Time: 0:01:03 ( 6.87 ms/it)\n objective: 13359.646021967448\nMinimizing 9250 Time: 0:01:03 ( 6.87 ms/it)\n objective: 13359.422486817035\nMinimizing 9265 Time: 0:01:03 ( 6.87 ms/it)\n objective: 13359.335114335146\nMinimizing 9280 Time: 0:01:03 ( 6.87 ms/it)\n objective: 13359.21155468587\nMinimizing 9295 Time: 0:01:03 ( 6.87 ms/it)\n objective: 13359.166843952218\nMinimizing 9310 Time: 0:01:03 ( 6.87 ms/it)\n objective: 13358.747913751838\nMinimizing 9325 Time: 0:01:04 ( 6.87 ms/it)\n objective: 13358.696437464743\nMinimizing 9340 Time: 0:01:04 ( 6.87 ms/it)\n objective: 13358.65018815524\nMinimizing 9355 Time: 0:01:04 ( 6.87 ms/it)\n objective: 13358.641094259729\nMinimizing 9370 Time: 0:01:04 ( 6.87 ms/it)\n objective: 13358.559805156925\nMinimizing 9385 Time: 0:01:04 ( 6.87 ms/it)\n objective: 13358.547125297358\nMinimizing 9400 Time: 0:01:04 ( 6.87 ms/it)\n objective: 13358.512463110703\nMinimizing 9415 Time: 0:01:04 ( 6.87 ms/it)\n objective: 13358.48364217821\nMinimizing 9430 Time: 0:01:04 ( 6.87 ms/it)\n objective: 13358.428020901782\nMinimizing 9445 Time: 0:01:04 ( 6.87 ms/it)\n objective: 13358.200210589042\nMinimizing 9460 Time: 0:01:04 ( 6.87 ms/it)\n objective: 13358.1787328727\nMinimizing 9475 Time: 0:01:05 ( 6.87 ms/it)\n objective: 13358.16922394975\nMinimizing 9490 Time: 0:01:05 ( 6.87 ms/it)\n objective: 13358.162856104063\nMinimizing 9505 Time: 0:01:05 ( 6.87 ms/it)\n objective: 13358.146613102224\nMinimizing 9520 Time: 0:01:05 ( 6.87 ms/it)\n objective: 13358.132933572335\nMinimizing 9535 Time: 0:01:05 ( 6.87 ms/it)\n objective: 13358.02146463866\nMinimizing 9550 Time: 0:01:05 ( 6.87 ms/it)\n objective: 13358.008704936423\nMinimizing 9565 Time: 0:01:05 ( 6.87 ms/it)\n objective: 13357.996215790721\nMinimizing 9580 Time: 0:01:05 ( 6.87 ms/it)\n objective: 13357.965784736298\nMinimizing 9595 Time: 0:01:05 ( 6.87 ms/it)\n objective: 13357.927778730067\nMinimizing 9610 Time: 0:01:06 ( 6.87 ms/it)\n objective: 13357.891812179056\nMinimizing 9625 Time: 0:01:06 ( 6.87 ms/it)\n objective: 13357.850452371327\nMinimizing 9640 Time: 0:01:06 ( 6.87 ms/it)\n objective: 13357.749120784145\nMinimizing 9655 Time: 0:01:06 ( 6.87 ms/it)\n objective: 13357.73899789466\nMinimizing 9669 Time: 0:01:06 ( 6.87 ms/it)\n objective: 13357.67472151856\nMinimizing 9684 Time: 0:01:06 ( 6.87 ms/it)\n objective: 13357.639983173358\nMinimizing 9699 Time: 0:01:06 ( 6.87 ms/it)\n objective: 13357.5706609611\nMinimizing 9714 Time: 0:01:06 ( 6.87 ms/it)\n objective: 13357.545856598612\nMinimizing 9729 Time: 0:01:06 ( 6.87 ms/it)\n objective: 13357.470692066163\nMinimizing 9744 Time: 0:01:06 ( 6.87 ms/it)\n objective: 13357.391749440772\nMinimizing 9759 Time: 0:01:07 ( 6.87 ms/it)\n objective: 13357.379048838782\nMinimizing 9774 Time: 0:01:07 ( 6.87 ms/it)\n objective: 13357.29391176656\nMinimizing 9789 Time: 0:01:07 ( 6.87 ms/it)\n objective: 13357.273432366761\nMinimizing 9804 Time: 0:01:07 ( 6.87 ms/it)\n objective: 13357.108316384503\nMinimizing 9819 Time: 0:01:07 ( 6.87 ms/it)\n objective: 13356.833747665492\nMinimizing 9834 Time: 0:01:07 ( 6.87 ms/it)\n objective: 13356.825744483125\nMinimizing 9849 Time: 0:01:07 ( 6.87 ms/it)\n objective: 13356.796599988353\nMinimizing 9864 Time: 0:01:07 ( 6.87 ms/it)\n objective: 13356.781355628991\nMinimizing 9879 Time: 0:01:07 ( 6.87 ms/it)\n objective: 13356.765395320064\nMinimizing 9894 Time: 0:01:07 ( 6.87 ms/it)\n objective: 13356.761655913862\nMinimizing 9909 Time: 0:01:08 ( 6.87 ms/it)\n objective: 13356.689726603901\nMinimizing 9924 Time: 0:01:08 ( 6.87 ms/it)\n objective: 13356.647753417856\nMinimizing 9939 Time: 0:01:08 ( 6.87 ms/it)\n objective: 13356.510383463341\nMinimizing 9954 Time: 0:01:08 ( 6.87 ms/it)\n objective: 13356.501531919435\nMinimizing 9969 Time: 0:01:08 ( 6.87 ms/it)\n objective: 13356.469089095459\nMinimizing 9984 Time: 0:01:08 ( 6.87 ms/it)\n objective: 13356.450332585562\nMinimizing 9999 Time: 0:01:08 ( 6.87 ms/it)\n objective: 13356.403033817878\nMinimizing 10014 Time: 0:01:08 ( 6.87 ms/it)\n objective: 13356.38004150764\nMinimizing 10029 Time: 0:01:08 ( 6.87 ms/it)\n objective: 13356.33580595671\nMinimizing 10044 Time: 0:01:09 ( 6.87 ms/it)\n objective: 13356.321089908\nMinimizing 10059 Time: 0:01:09 ( 6.87 ms/it)\n objective: 13356.28983799689\nMinimizing 10074 Time: 0:01:09 ( 6.87 ms/it)\n objective: 13356.253746486553\nMinimizing 10089 Time: 0:01:09 ( 6.87 ms/it)\n objective: 13356.233201505558\nMinimizing 10104 Time: 0:01:09 ( 6.87 ms/it)\n objective: 13356.071788036905\nMinimizing 10119 Time: 0:01:09 ( 6.87 ms/it)\n objective: 13356.058297816235\nMinimizing 10134 Time: 0:01:09 ( 6.87 ms/it)\n objective: 13355.967407573859\nMinimizing 10149 Time: 0:01:09 ( 6.87 ms/it)\n objective: 13355.832206948042\nMinimizing 10164 Time: 0:01:09 ( 6.87 ms/it)\n objective: 13355.824421166646\nMinimizing 10179 Time: 0:01:09 ( 6.87 ms/it)\n objective: 13355.799631444053\nMinimizing 10194 Time: 0:01:10 ( 6.87 ms/it)\n objective: 13355.777966823232\nMinimizing 10209 Time: 0:01:10 ( 6.87 ms/it)\n objective: 13355.732665592957\nMinimizing 10224 Time: 0:01:10 ( 6.87 ms/it)\n objective: 13355.70132113299\nMinimizing 10239 Time: 0:01:10 ( 6.87 ms/it)\n objective: 13355.639947712865\nMinimizing 10254 Time: 0:01:10 ( 6.87 ms/it)\n objective: 13355.612745383703\nMinimizing 10269 Time: 0:01:10 ( 6.87 ms/it)\n objective: 13355.547027447261\nMinimizing 10284 Time: 0:01:10 ( 6.87 ms/it)\n objective: 13355.504593239217\nMinimizing 10299 Time: 0:01:10 ( 6.87 ms/it)\n objective: 13355.479096091447\nMinimizing 10314 Time: 0:01:10 ( 6.87 ms/it)\n objective: 13355.30346653847\nMinimizing 10329 Time: 0:01:10 ( 6.87 ms/it)\n objective: 13355.160378422326\nMinimizing 10344 Time: 0:01:11 ( 6.87 ms/it)\n objective: 13355.140047027344\nMinimizing 10359 Time: 0:01:11 ( 6.87 ms/it)\n objective: 13355.087764095952\nMinimizing 10374 Time: 0:01:11 ( 6.87 ms/it)\n objective: 13355.037673298008\nMinimizing 10388 Time: 0:01:11 ( 6.87 ms/it)\n objective: 13355.019166930098\nMinimizing 10403 Time: 0:01:11 ( 6.87 ms/it)\n objective: 13354.947949785586\nMinimizing 10418 Time: 0:01:11 ( 6.87 ms/it)\n objective: 13354.842603864556\nMinimizing 10433 Time: 0:01:11 ( 6.87 ms/it)\n objective: 13354.821451744581\nMinimizing 10448 Time: 0:01:11 ( 6.87 ms/it)\n objective: 13354.695303323715\nMinimizing 10463 Time: 0:01:11 ( 6.87 ms/it)\n objective: 13354.681614634705\nMinimizing 10478 Time: 0:01:12 ( 6.87 ms/it)\n objective: 13354.632806485191\nMinimizing 10493 Time: 0:01:12 ( 6.87 ms/it)\n objective: 13354.610220616662\nMinimizing 10508 Time: 0:01:12 ( 6.87 ms/it)\n objective: 13354.55800435182\nMinimizing 10523 Time: 0:01:12 ( 6.87 ms/it)\n objective: 13354.497687714254\nMinimizing 10538 Time: 0:01:12 ( 6.87 ms/it)\n objective: 13354.478728011163\nMinimizing 10553 Time: 0:01:12 ( 6.87 ms/it)\n objective: 13354.442410703945\nMinimizing 10568 Time: 0:01:12 ( 6.87 ms/it)\n objective: 13354.425110274613\nMinimizing 10583 Time: 0:01:12 ( 6.87 ms/it)\n objective: 13354.413514680775\nMinimizing 10598 Time: 0:01:12 ( 6.87 ms/it)\n objective: 13354.360065265944\nMinimizing 10613 Time: 0:01:12 ( 6.88 ms/it)\n objective: 13354.22378888051\nMinimizing 10627 Time: 0:01:13 ( 6.88 ms/it)\n objective: 13354.218188786632\nMinimizing 10641 Time: 0:01:13 ( 6.88 ms/it)\n objective: 13354.206655216753\nMinimizing 10656 Time: 0:01:13 ( 6.88 ms/it)\n objective: 13354.14143136362\nMinimizing 10671 Time: 0:01:13 ( 6.88 ms/it)\n objective: 13354.135715560813\nMinimizing 10685 Time: 0:01:13 ( 6.88 ms/it)\n objective: 13354.079150386264\nMinimizing 10700 Time: 0:01:13 ( 6.88 ms/it)\n objective: 13354.038270700228\nMinimizing 10715 Time: 0:01:13 ( 6.88 ms/it)\n objective: 13354.019891930562\nMinimizing 10729 Time: 0:01:13 ( 6.88 ms/it)\n objective: 13353.97626718069\nMinimizing 10744 Time: 0:01:13 ( 6.88 ms/it)\n objective: 13353.857207170382\nMinimizing 10759 Time: 0:01:13 ( 6.88 ms/it)\n objective: 13353.726796034593\nMinimizing 10774 Time: 0:01:14 ( 6.88 ms/it)\n objective: 13353.716947599278\nMinimizing 10788 Time: 0:01:14 ( 6.88 ms/it)\n objective: 13353.658712991411\nMinimizing 10803 Time: 0:01:14 ( 6.88 ms/it)\n objective: 13353.511173341547\nMinimizing 10818 Time: 0:01:14 ( 6.88 ms/it)\n objective: 13353.390063043851\nMinimizing 10833 Time: 0:01:14 ( 6.88 ms/it)\n objective: 13353.021567532029\nMinimizing 10848 Time: 0:01:14 ( 6.88 ms/it)\n objective: 13352.897244007203\nMinimizing 10863 Time: 0:01:14 ( 6.88 ms/it)\n objective: 13352.846662323944\nMinimizing 10878 Time: 0:01:14 ( 6.88 ms/it)\n objective: 13352.796194022776\nMinimizing 10893 Time: 0:01:14 ( 6.88 ms/it)\n objective: 13352.723458942033\nMinimizing 10908 Time: 0:01:15 ( 6.88 ms/it)\n objective: 13352.68034056737\nMinimizing 10923 Time: 0:01:15 ( 6.88 ms/it)\n objective: 13352.591796056404\nMinimizing 10938 Time: 0:01:15 ( 6.88 ms/it)\n objective: 13352.572507063749\nMinimizing 10953 Time: 0:01:15 ( 6.88 ms/it)\n objective: 13352.526107956568\nMinimizing 10968 Time: 0:01:15 ( 6.88 ms/it)\n objective: 13352.493794555921\nMinimizing 10983 Time: 0:01:15 ( 6.88 ms/it)\n objective: 13352.548909451558\nMinimizing 10998 Time: 0:01:15 ( 6.88 ms/it)\n objective: 13352.39552116918\nMinimizing 11013 Time: 0:01:15 ( 6.88 ms/it)\n objective: 13352.387975489793\nMinimizing 11028 Time: 0:01:15 ( 6.88 ms/it)\n objective: 13352.369685569334\nMinimizing 11043 Time: 0:01:15 ( 6.88 ms/it)\n objective: 13352.326268345656\nMinimizing 11058 Time: 0:01:16 ( 6.88 ms/it)\n objective: 13352.292750105575\nMinimizing 11073 Time: 0:01:16 ( 6.88 ms/it)\n objective: 13352.229121718381\nMinimizing 11088 Time: 0:01:16 ( 6.88 ms/it)\n objective: 13352.20230627391\nMinimizing 11103 Time: 0:01:16 ( 6.88 ms/it)\n objective: 13352.193106757222\nMinimizing 11118 Time: 0:01:16 ( 6.88 ms/it)\n objective: 13352.171193355884\nMinimizing 11133 Time: 0:01:16 ( 6.88 ms/it)\n objective: 13352.11501445964\nMinimizing 11148 Time: 0:01:16 ( 6.88 ms/it)\n objective: 13352.061546726996\nMinimizing 11163 Time: 0:01:16 ( 6.88 ms/it)\n objective: 13352.004294380182\nMinimizing 11178 Time: 0:01:16 ( 6.88 ms/it)\n objective: 13351.850530314987\nMinimizing 11193 Time: 0:01:16 ( 6.88 ms/it)\n objective: 13352.021791293439\nMinimizing 11208 Time: 0:01:17 ( 6.88 ms/it)\n objective: 13351.855297271475\nMinimizing 11223 Time: 0:01:17 ( 6.88 ms/it)\n objective: 13351.434472677873\nMinimizing 11238 Time: 0:01:17 ( 6.88 ms/it)\n objective: 13351.330530201827\nMinimizing 11253 Time: 0:01:17 ( 6.88 ms/it)\n objective: 13351.241411714283\nMinimizing 11268 Time: 0:01:17 ( 6.88 ms/it)\n objective: 13351.196807353474\nMinimizing 11283 Time: 0:01:17 ( 6.88 ms/it)\n objective: 13350.978595137974\nMinimizing 11298 Time: 0:01:17 ( 6.88 ms/it)\n objective: 13350.953308095632\nMinimizing 11313 Time: 0:01:17 ( 6.88 ms/it)\n objective: 13350.758985716144\nMinimizing 11328 Time: 0:01:17 ( 6.88 ms/it)\n objective: 13350.488864257655\nMinimizing 11343 Time: 0:01:18 ( 6.88 ms/it)\n objective: 13350.480755878918\nMinimizing 11358 Time: 0:01:18 ( 6.88 ms/it)\n objective: 13350.436838050795\nMinimizing 11373 Time: 0:01:18 ( 6.88 ms/it)\n objective: 13350.410163312066\nMinimizing 11388 Time: 0:01:18 ( 6.88 ms/it)\n objective: 13350.304128227202\nMinimizing 11403 Time: 0:01:18 ( 6.88 ms/it)\n objective: 13350.291904704667\nMinimizing 11418 Time: 0:01:18 ( 6.88 ms/it)\n objective: 13350.234178762905\nMinimizing 11433 Time: 0:01:18 ( 6.88 ms/it)\n objective: 13350.564948266867\nMinimizing 11448 Time: 0:01:18 ( 6.88 ms/it)\n objective: 13349.670339506287\nMinimizing 11463 Time: 0:01:18 ( 6.88 ms/it)\n objective: 13349.405120001582\nMinimizing 11478 Time: 0:01:18 ( 6.88 ms/it)\n objective: 13349.40036385908\nMinimizing 11493 Time: 0:01:19 ( 6.88 ms/it)\n objective: 13349.392075374664\nMinimizing 11508 Time: 0:01:19 ( 6.88 ms/it)\n objective: 13349.388374022368\nMinimizing 11523 Time: 0:01:19 ( 6.88 ms/it)\n objective: 13349.32275659265\nMinimizing 11538 Time: 0:01:19 ( 6.88 ms/it)\n objective: 13349.306908622631\nMinimizing 11553 Time: 0:01:19 ( 6.88 ms/it)\n objective: 13349.266337743844\nMinimizing 11568 Time: 0:01:19 ( 6.88 ms/it)\n objective: 13349.253371557563\nMinimizing 11583 Time: 0:01:19 ( 6.88 ms/it)\n objective: 13349.215085777294\nMinimizing 11598 Time: 0:01:19 ( 6.88 ms/it)\n objective: 13349.15306443839\nMinimizing 11613 Time: 0:01:19 ( 6.88 ms/it)\n objective: 13349.142172054228\nMinimizing 11628 Time: 0:01:20 ( 6.88 ms/it)\n objective: 13349.058044554607\nMinimizing 11643 Time: 0:01:20 ( 6.88 ms/it)\n objective: 13349.093074445256\nMinimizing 11658 Time: 0:01:20 ( 6.88 ms/it)\n objective: 13348.750111515088\nMinimizing 11673 Time: 0:01:20 ( 6.88 ms/it)\n objective: 13348.734737960767\nMinimizing 11688 Time: 0:01:20 ( 6.88 ms/it)\n objective: 13348.706401411218\nMinimizing 11703 Time: 0:01:20 ( 6.88 ms/it)\n objective: 13348.632721919224\nMinimizing 11718 Time: 0:01:20 ( 6.88 ms/it)\n objective: 13348.626477856982\nMinimizing 11733 Time: 0:01:20 ( 6.88 ms/it)\n objective: 13348.593688355424\nMinimizing 11748 Time: 0:01:20 ( 6.88 ms/it)\n objective: 13348.546381084569\nMinimizing 11763 Time: 0:01:20 ( 6.88 ms/it)\n objective: 13348.529474397888\nMinimizing 11778 Time: 0:01:21 ( 6.88 ms/it)\n objective: 13348.498395327479\nMinimizing 11793 Time: 0:01:21 ( 6.88 ms/it)\n objective: 13348.485240423724\nMinimizing 11808 Time: 0:01:21 ( 6.88 ms/it)\n objective: 13348.441812635116\nMinimizing 11823 Time: 0:01:21 ( 6.88 ms/it)\n objective: 13348.32903826562\nMinimizing 11838 Time: 0:01:21 ( 6.88 ms/it)\n objective: 13348.221673585009\nMinimizing 11853 Time: 0:01:21 ( 6.88 ms/it)\n objective: 13348.214896810481\nMinimizing 11868 Time: 0:01:21 ( 6.88 ms/it)\n objective: 13348.199077259924\nMinimizing 11883 Time: 0:01:21 ( 6.88 ms/it)\n objective: 13348.178457346374\nMinimizing 11898 Time: 0:01:21 ( 6.88 ms/it)\n objective: 13348.128041487078\nMinimizing 11913 Time: 0:01:22 ( 6.88 ms/it)\n objective: 13348.121589186667\nMinimizing 11928 Time: 0:01:22 ( 6.88 ms/it)\n objective: 13348.08802215854\nMinimizing 11943 Time: 0:01:22 ( 6.88 ms/it)\n objective: 13348.005637507522\nMinimizing 11958 Time: 0:01:22 ( 6.88 ms/it)\n objective: 13347.893594041052\nMinimizing 11973 Time: 0:01:22 ( 6.89 ms/it)\n objective: 13347.884248236129\nMinimizing 11988 Time: 0:01:22 ( 6.89 ms/it)\n objective: 13347.86313546617\nMinimizing 12003 Time: 0:01:22 ( 6.89 ms/it)\n objective: 13347.845103490552\nMinimizing 12018 Time: 0:01:22 ( 6.89 ms/it)\n objective: 13347.766117480576\nMinimizing 12033 Time: 0:01:22 ( 6.89 ms/it)\n objective: 13347.579171432684\nMinimizing 12048 Time: 0:01:22 ( 6.89 ms/it)\n objective: 13347.570580253945\nMinimizing 12063 Time: 0:01:23 ( 6.89 ms/it)\n objective: 13347.560911486748\nMinimizing 12078 Time: 0:01:23 ( 6.89 ms/it)\n objective: 13347.512697220773\nMinimizing 12093 Time: 0:01:23 ( 6.89 ms/it)\n objective: 13347.489100522747\nMinimizing 12108 Time: 0:01:23 ( 6.89 ms/it)\n objective: 13347.483349469083\nMinimizing 12123 Time: 0:01:23 ( 6.89 ms/it)\n objective: 13347.418547751557\nMinimizing 12138 Time: 0:01:23 ( 6.89 ms/it)\n objective: 13347.401719889378\nMinimizing 12153 Time: 0:01:23 ( 6.89 ms/it)\n objective: 13347.375341032988\nMinimizing 12168 Time: 0:01:23 ( 6.89 ms/it)\n objective: 13347.244254993107\nMinimizing 12183 Time: 0:01:23 ( 6.89 ms/it)\n objective: 13347.201986245804\nMinimizing 12198 Time: 0:01:24 ( 6.89 ms/it)\n objective: 13347.183600776145\nMinimizing 12213 Time: 0:01:24 ( 6.89 ms/it)\n objective: 13347.16438171349\nMinimizing 12228 Time: 0:01:24 ( 6.89 ms/it)\n objective: 13347.138739760412\nMinimizing 12243 Time: 0:01:24 ( 6.89 ms/it)\n objective: 13347.124679966422\nMinimizing 12258 Time: 0:01:24 ( 6.89 ms/it)\n objective: 13347.098915820774\nMinimizing 12273 Time: 0:01:24 ( 6.89 ms/it)\n objective: 13347.032805347706\nMinimizing 12288 Time: 0:01:24 ( 6.89 ms/it)\n objective: 13347.026001475497\nMinimizing 12303 Time: 0:01:24 ( 6.89 ms/it)\n objective: 13346.994198280372\nMinimizing 12318 Time: 0:01:24 ( 6.89 ms/it)\n objective: 13346.962589302864\nMinimizing 12333 Time: 0:01:24 ( 6.89 ms/it)\n objective: 13346.995920389236\nMinimizing 12348 Time: 0:01:25 ( 6.89 ms/it)\n objective: 13346.917171199217\nMinimizing 12363 Time: 0:01:25 ( 6.89 ms/it)\n objective: 13346.867482292975\nMinimizing 12378 Time: 0:01:25 ( 6.89 ms/it)\n objective: 13346.849120384446\nMinimizing 12393 Time: 0:01:25 ( 6.89 ms/it)\n objective: 13346.799488106466\nMinimizing 12408 Time: 0:01:25 ( 6.89 ms/it)\n objective: 13346.686944626352\nMinimizing 12423 Time: 0:01:25 ( 6.89 ms/it)\n objective: 13346.48642662655\nMinimizing 12438 Time: 0:01:25 ( 6.89 ms/it)\n objective: 13346.92594848002\nMinimizing 12453 Time: 0:01:25 ( 6.89 ms/it)\n objective: 13345.624771598246\nMinimizing 12468 Time: 0:01:25 ( 6.89 ms/it)\n objective: 13345.622492908384\nMinimizing 12483 Time: 0:01:25 ( 6.89 ms/it)\n objective: 13345.617883374114\nMinimizing 12498 Time: 0:01:26 ( 6.89 ms/it)\n objective: 13345.614110157265\nMinimizing 12513 Time: 0:01:26 ( 6.89 ms/it)\n objective: 13345.605564614998\nMinimizing 12528 Time: 0:01:26 ( 6.89 ms/it)\n objective: 13345.554699084685\nMinimizing 12543 Time: 0:01:26 ( 6.89 ms/it)\n objective: 13345.544308497454\nMinimizing 12558 Time: 0:01:26 ( 6.89 ms/it)\n objective: 13345.491858643989\nMinimizing 12573 Time: 0:01:26 ( 6.89 ms/it)\n objective: 13345.48424000253\nMinimizing 12588 Time: 0:01:26 ( 6.89 ms/it)\n objective: 13345.416693552645\nMinimizing 12603 Time: 0:01:26 ( 6.89 ms/it)\n objective: 13345.409384650491\nMinimizing 12618 Time: 0:01:26 ( 6.89 ms/it)\n objective: 13345.365771790457\nMinimizing 12633 Time: 0:01:27 ( 6.89 ms/it)\n objective: 13345.323263412756\nMinimizing 12648 Time: 0:01:27 ( 6.89 ms/it)\n objective: 13345.315997259175\nMinimizing 12663 Time: 0:01:27 ( 6.89 ms/it)\n objective: 13345.27816786291\nMinimizing 12678 Time: 0:01:27 ( 6.89 ms/it)\n objective: 13345.265947161184\nMinimizing 12693 Time: 0:01:27 ( 6.89 ms/it)\n objective: 13345.241204354403\nMinimizing 12708 Time: 0:01:27 ( 6.89 ms/it)\n objective: 13345.151973099411\nMinimizing 12723 Time: 0:01:27 ( 6.89 ms/it)\n objective: 13345.148730351953\nMinimizing 12738 Time: 0:01:27 ( 6.89 ms/it)\n objective: 13345.143100795212\nMinimizing 12753 Time: 0:01:27 ( 6.89 ms/it)\n objective: 13345.137366424366\nMinimizing 12768 Time: 0:01:27 ( 6.89 ms/it)\n objective: 13345.124378438704\nMinimizing 12783 Time: 0:01:28 ( 6.89 ms/it)\n objective: 13345.111920154908\nMinimizing 12798 Time: 0:01:28 ( 6.89 ms/it)\n objective: 13345.102961771212\nMinimizing 12813 Time: 0:01:28 ( 6.89 ms/it)\n objective: 13345.085259086474\nMinimizing 12828 Time: 0:01:28 ( 6.89 ms/it)\n objective: 13345.07752774781\nMinimizing 12843 Time: 0:01:28 ( 6.89 ms/it)\n objective: 13345.039197739126\nMinimizing 12858 Time: 0:01:28 ( 6.89 ms/it)\n objective: 13345.037761111911\nMinimizing 12873 Time: 0:01:28 ( 6.89 ms/it)\n objective: 13344.945741753778\nMinimizing 12888 Time: 0:01:28 ( 6.89 ms/it)\n objective: 13344.928693851587\nMinimizing 12903 Time: 0:01:28 ( 6.89 ms/it)\n objective: 13344.918280682403\nMinimizing 12918 Time: 0:01:29 ( 6.89 ms/it)\n objective: 13344.895861285731\nMinimizing 12933 Time: 0:01:29 ( 6.89 ms/it)\n objective: 13344.889098035783\nMinimizing 12948 Time: 0:01:29 ( 6.89 ms/it)\n objective: 13344.884181286761\nMinimizing 12963 Time: 0:01:29 ( 6.89 ms/it)\n objective: 13344.869898359102\nMinimizing 12978 Time: 0:01:29 ( 6.89 ms/it)\n objective: 13344.857608767015\nMinimizing 12993 Time: 0:01:29 ( 6.89 ms/it)\n objective: 13344.769712908826\nMinimizing 13008 Time: 0:01:29 ( 6.89 ms/it)\n objective: 13344.763956469105\nMinimizing 13023 Time: 0:01:29 ( 6.89 ms/it)\n objective: 13344.754110087357\nMinimizing 13038 Time: 0:01:29 ( 6.89 ms/it)\n objective: 13344.736951753956\nMinimizing 13053 Time: 0:01:29 ( 6.89 ms/it)\n objective: 13344.723479470667\nMinimizing 13068 Time: 0:01:30 ( 6.89 ms/it)\n objective: 13344.656055844891\nMinimizing 13083 Time: 0:01:30 ( 6.89 ms/it)\n objective: 13344.579038085307\nMinimizing 13098 Time: 0:01:30 ( 6.89 ms/it)\n objective: 13344.572954430514\nMinimizing 13113 Time: 0:01:30 ( 6.89 ms/it)\n objective: 13344.531565643221\nMinimizing 13128 Time: 0:01:30 ( 6.89 ms/it)\n objective: 13344.481509258061\nMinimizing 13143 Time: 0:01:30 ( 6.89 ms/it)\n objective: 13344.474669982046\nMinimizing 13158 Time: 0:01:30 ( 6.89 ms/it)\n objective: 13344.421327486154\nMinimizing 13173 Time: 0:01:30 ( 6.89 ms/it)\n objective: 13344.392066209948\nMinimizing 13188 Time: 0:01:30 ( 6.89 ms/it)\n objective: 13344.358304086294\nMinimizing 13203 Time: 0:01:30 ( 6.89 ms/it)\n objective: 13344.334720780731\nMinimizing 13218 Time: 0:01:31 ( 6.89 ms/it)\n objective: 13344.308567977416\nMinimizing 13233 Time: 0:01:31 ( 6.89 ms/it)\n objective: 13344.23706030583\nMinimizing 13248 Time: 0:01:31 ( 6.89 ms/it)\n objective: 13344.041097178415\nMinimizing 13263 Time: 0:01:31 ( 6.89 ms/it)\n objective: 13344.007011748909\nMinimizing 13278 Time: 0:01:31 ( 6.89 ms/it)\n objective: 13343.98747079874\nMinimizing 13293 Time: 0:01:31 ( 6.89 ms/it)\n objective: 13343.960268394076\nMinimizing 13308 Time: 0:01:31 ( 6.89 ms/it)\n objective: 13343.930557065454\nMinimizing 13323 Time: 0:01:31 ( 6.89 ms/it)\n objective: 13343.893869436302\nMinimizing 13338 Time: 0:01:31 ( 6.89 ms/it)\n objective: 13343.872125136259\nMinimizing 13353 Time: 0:01:32 ( 6.89 ms/it)\n objective: 13343.8251938412\nMinimizing 13368 Time: 0:01:32 ( 6.89 ms/it)\n objective: 13343.80767860065\nMinimizing 13383 Time: 0:01:32 ( 6.89 ms/it)\n objective: 13343.742183542854\nMinimizing 13398 Time: 0:01:32 ( 6.89 ms/it)\n objective: 13343.524706619333\nMinimizing 13413 Time: 0:01:32 ( 6.89 ms/it)\n objective: 13343.519191973894\nMinimizing 13428 Time: 0:01:32 ( 6.89 ms/it)\n objective: 13343.513145236284\nMinimizing 13443 Time: 0:01:32 ( 6.89 ms/it)\n objective: 13343.503985637704\nMinimizing 13458 Time: 0:01:32 ( 6.89 ms/it)\n objective: 13343.500045022993\nMinimizing 13473 Time: 0:01:32 ( 6.89 ms/it)\n objective: 13343.47800719097\nMinimizing 13488 Time: 0:01:32 ( 6.89 ms/it)\n objective: 13343.4472022357\nMinimizing 13503 Time: 0:01:33 ( 6.89 ms/it)\n objective: 13343.435520621017\nMinimizing 13518 Time: 0:01:33 ( 6.89 ms/it)\n objective: 13343.388298944199\nMinimizing 13533 Time: 0:01:33 ( 6.89 ms/it)\n objective: 13343.385343308371\nMinimizing 13548 Time: 0:01:33 ( 6.89 ms/it)\n objective: 13343.340063787291\nMinimizing 13563 Time: 0:01:33 ( 6.89 ms/it)\n objective: 13343.324650076785\nMinimizing 13578 Time: 0:01:33 ( 6.89 ms/it)\n objective: 13344.13969906688\nMinimizing 13593 Time: 0:01:33 ( 6.89 ms/it)\n objective: 13342.765057540884\nMinimizing 13608 Time: 0:01:33 ( 6.89 ms/it)\n objective: 13342.751842431957\nMinimizing 13623 Time: 0:01:33 ( 6.89 ms/it)\n objective: 13342.750640718186\nMinimizing 13638 Time: 0:01:34 ( 6.89 ms/it)\n objective: 13342.748811103957\nMinimizing 13653 Time: 0:01:34 ( 6.89 ms/it)\n objective: 13342.746436606496\nMinimizing 13668 Time: 0:01:34 ( 6.89 ms/it)\n objective: 13342.731080381374\nMinimizing 13683 Time: 0:01:34 ( 6.89 ms/it)\n objective: 13342.633762626407\nMinimizing 13698 Time: 0:01:34 ( 6.89 ms/it)\n objective: 13342.629647834343\nMinimizing 13713 Time: 0:01:34 ( 6.89 ms/it)\n objective: 13342.626952746563\nMinimizing 13728 Time: 0:01:34 ( 6.89 ms/it)\n objective: 13342.612588184544\nMinimizing 13743 Time: 0:01:34 ( 6.89 ms/it)\n objective: 13342.608858129817\nMinimizing 13758 Time: 0:01:34 ( 6.89 ms/it)\n objective: 13342.60751029876\nMinimizing 13773 Time: 0:01:34 ( 6.89 ms/it)\n objective: 13342.598050464214\nMinimizing 13788 Time: 0:01:35 ( 6.89 ms/it)\n objective: 13342.569516726078\nMinimizing 13803 Time: 0:01:35 ( 6.89 ms/it)\n objective: 13342.558288428969\nMinimizing 13818 Time: 0:01:35 ( 6.89 ms/it)\n objective: 13342.54944619767\nMinimizing 13833 Time: 0:01:35 ( 6.89 ms/it)\n objective: 13342.546611815378\nMinimizing 13848 Time: 0:01:35 ( 6.89 ms/it)\n objective: 13342.542115552831\nMinimizing 13863 Time: 0:01:35 ( 6.89 ms/it)\n objective: 13342.497274557572\nMinimizing 13878 Time: 0:01:35 ( 6.89 ms/it)\n objective: 13342.42747276464\nMinimizing 13893 Time: 0:01:35 ( 6.89 ms/it)\n objective: 13342.388931177571\nMinimizing 13908 Time: 0:01:35 ( 6.89 ms/it)\n objective: 13342.381274291023\nMinimizing 13923 Time: 0:01:35 ( 6.89 ms/it)\n objective: 13342.358395179981\nMinimizing 13938 Time: 0:01:36 ( 6.89 ms/it)\n objective: 13342.343988913286\nMinimizing 13953 Time: 0:01:36 ( 6.89 ms/it)\n objective: 13342.340213374962\nMinimizing 13968 Time: 0:01:36 ( 6.89 ms/it)\n objective: 13342.31791719931\nMinimizing 13983 Time: 0:01:36 ( 6.89 ms/it)\n objective: 13342.308906502047\nMinimizing 13998 Time: 0:01:36 ( 6.89 ms/it)\n objective: 13342.254002878639\nMinimizing 14013 Time: 0:01:36 ( 6.89 ms/it)\n objective: 13342.24522380287\nMinimizing 14028 Time: 0:01:36 ( 6.89 ms/it)\n objective: 13342.226891926119\nMinimizing 14043 Time: 0:01:36 ( 6.89 ms/it)\n objective: 13342.219861720085\nMinimizing 14058 Time: 0:01:36 ( 6.89 ms/it)\n objective: 13342.186885459821\nMinimizing 14073 Time: 0:01:37 ( 6.89 ms/it)\n objective: 13342.06056912605\nMinimizing 14088 Time: 0:01:37 ( 6.89 ms/it)\n objective: 13342.050330003163\nMinimizing 14103 Time: 0:01:37 ( 6.89 ms/it)\n objective: 13342.02049463983\nMinimizing 14118 Time: 0:01:37 ( 6.89 ms/it)\n objective: 13342.00188136968\nMinimizing 14133 Time: 0:01:37 ( 6.89 ms/it)\n objective: 13342.000270805038\nMinimizing 14148 Time: 0:01:37 ( 6.89 ms/it)\n objective: 13341.959773801522\nMinimizing 14163 Time: 0:01:37 ( 6.89 ms/it)\n objective: 13341.958891604772\nMinimizing 14178 Time: 0:01:37 ( 6.89 ms/it)\n objective: 13341.938013339626\nMinimizing 14193 Time: 0:01:37 ( 6.89 ms/it)\n objective: 13341.908900629838\nMinimizing 14208 Time: 0:01:37 ( 6.89 ms/it)\n objective: 13341.87365635595\nMinimizing 14223 Time: 0:01:38 ( 6.89 ms/it)\n objective: 13341.86515817271\nMinimizing 14238 Time: 0:01:38 ( 6.89 ms/it)\n objective: 13341.82831220379\nMinimizing 14253 Time: 0:01:38 ( 6.89 ms/it)\n objective: 13341.825611467793\nMinimizing 14268 Time: 0:01:38 ( 6.89 ms/it)\n objective: 13341.779450060836\nMinimizing 14283 Time: 0:01:38 ( 6.89 ms/it)\n objective: 13341.778552667434\nMinimizing 14298 Time: 0:01:38 ( 6.89 ms/it)\n objective: 13341.736163047404\nMinimizing 14313 Time: 0:01:38 ( 6.89 ms/it)\n objective: 13341.728549369509\nMinimizing 14328 Time: 0:01:38 ( 6.89 ms/it)\n objective: 13341.703864172676\nMinimizing 14343 Time: 0:01:38 ( 6.89 ms/it)\n objective: 13341.69038413126\nMinimizing 14358 Time: 0:01:38 ( 6.89 ms/it)\n objective: 13341.671579814429\nMinimizing 14373 Time: 0:01:39 ( 6.89 ms/it)\n objective: 13341.634711431245\nMinimizing 14388 Time: 0:01:39 ( 6.89 ms/it)\n objective: 13341.60397035969\nMinimizing 14403 Time: 0:01:39 ( 6.89 ms/it)\n objective: 13341.445012514996\nMinimizing 14418 Time: 0:01:39 ( 6.89 ms/it)\n objective: 13341.365024654224\nMinimizing 14433 Time: 0:01:39 ( 6.89 ms/it)\n objective: 13341.36092733089\nMinimizing 14448 Time: 0:01:39 ( 6.90 ms/it)\n objective: 13341.326760094555\nMinimizing 14463 Time: 0:01:39 ( 6.90 ms/it)\n objective: 13341.313083627363\nMinimizing 14478 Time: 0:01:39 ( 6.90 ms/it)\n objective: 13341.281522976933\nMinimizing 14493 Time: 0:01:39 ( 6.90 ms/it)\n objective: 13341.130622068958\nMinimizing 14508 Time: 0:01:40 ( 6.90 ms/it)\n objective: 13341.125170117783\nMinimizing 14523 Time: 0:01:40 ( 6.90 ms/it)\n objective: 13341.123832796904\nMinimizing 14538 Time: 0:01:40 ( 6.90 ms/it)\n objective: 13341.115333240581\nMinimizing 14553 Time: 0:01:40 ( 6.90 ms/it)\n objective: 13341.105340322218\nMinimizing 14568 Time: 0:01:40 ( 6.90 ms/it)\n objective: 13341.10004119383\nMinimizing 14583 Time: 0:01:40 ( 6.90 ms/it)\n objective: 13341.04061021009\nMinimizing 14598 Time: 0:01:40 ( 6.90 ms/it)\n objective: 13340.998292570279\nMinimizing 14613 Time: 0:01:40 ( 6.90 ms/it)\n objective: 13340.980129442374\nMinimizing 14628 Time: 0:01:40 ( 6.90 ms/it)\n objective: 13340.887975051883\nMinimizing 14643 Time: 0:01:40 ( 6.90 ms/it)\n objective: 13340.850648001644\nMinimizing 14657 Time: 0:01:41 ( 6.90 ms/it)\n objective: 13340.775113465228\nMinimizing 14672 Time: 0:01:41 ( 6.90 ms/it)\n objective: 13340.618883324612\nMinimizing 14687 Time: 0:01:41 ( 6.90 ms/it)\n objective: 13339.93038837218\nMinimizing 14702 Time: 0:01:41 ( 6.90 ms/it)\n objective: 13339.802194371841\nMinimizing 14717 Time: 0:01:41 ( 6.90 ms/it)\n objective: 13339.79417384282\nMinimizing 14732 Time: 0:01:41 ( 6.90 ms/it)\n objective: 13339.784141392709\nMinimizing 14747 Time: 0:01:41 ( 6.90 ms/it)\n objective: 13339.778731724022\nMinimizing 14762 Time: 0:01:41 ( 6.90 ms/it)\n objective: 13339.736090131417\nMinimizing 14777 Time: 0:01:41 ( 6.90 ms/it)\n objective: 13339.716177164373\nMinimizing 14792 Time: 0:01:42 ( 6.90 ms/it)\n objective: 13339.634403322947\nMinimizing 14807 Time: 0:01:42 ( 6.90 ms/it)\n objective: 13340.087119650794\nMinimizing 14822 Time: 0:01:42 ( 6.90 ms/it)\n objective: 13339.414491383934\nMinimizing 14837 Time: 0:01:42 ( 6.90 ms/it)\n objective: 13339.411070155846\nMinimizing 14852 Time: 0:01:42 ( 6.90 ms/it)\n objective: 13339.39178276476\nMinimizing 14867 Time: 0:01:42 ( 6.90 ms/it)\n objective: 13339.378136657353\nMinimizing 14882 Time: 0:01:42 ( 6.90 ms/it)\n objective: 13339.365104074852\nMinimizing 14897 Time: 0:01:42 ( 6.90 ms/it)\n objective: 13339.344841596583\nMinimizing 14912 Time: 0:01:42 ( 6.90 ms/it)\n objective: 13339.291933889188\nMinimizing 14927 Time: 0:01:42 ( 6.90 ms/it)\n objective: 13339.098997031077\nMinimizing 14942 Time: 0:01:43 ( 6.90 ms/it)\n objective: 13339.093388413305\nMinimizing 14957 Time: 0:01:43 ( 6.90 ms/it)\n objective: 13339.06705239074\nMinimizing 14972 Time: 0:01:43 ( 6.90 ms/it)\n objective: 13339.04894409424\nMinimizing 14987 Time: 0:01:43 ( 6.90 ms/it)\n objective: 13339.038613203244\nMinimizing 15002 Time: 0:01:43 ( 6.90 ms/it)\n objective: 13338.997378755244\nMinimizing 15017 Time: 0:01:43 ( 6.90 ms/it)\n objective: 13338.932883187394\nMinimizing 15032 Time: 0:01:43 ( 6.90 ms/it)\n objective: 13338.84944278629\nMinimizing 15047 Time: 0:01:43 ( 6.90 ms/it)\n objective: 13338.621033794087\nMinimizing 15062 Time: 0:01:43 ( 6.90 ms/it)\n objective: 13338.614366147114\nMinimizing 15077 Time: 0:01:44 ( 6.90 ms/it)\n objective: 13338.59184768518\nMinimizing 15092 Time: 0:01:44 ( 6.90 ms/it)\n objective: 13338.579031379988\nMinimizing 15107 Time: 0:01:44 ( 6.90 ms/it)\n objective: 13338.558751188066\nMinimizing 15122 Time: 0:01:44 ( 6.90 ms/it)\n objective: 13338.549335210111\nMinimizing 15137 Time: 0:01:44 ( 6.90 ms/it)\n objective: 13338.452531246578\nMinimizing 15152 Time: 0:01:44 ( 6.90 ms/it)\n objective: 13338.433048698353\nMinimizing 15167 Time: 0:01:44 ( 6.90 ms/it)\n objective: 13338.433354479355\nMinimizing 15182 Time: 0:01:44 ( 6.90 ms/it)\n objective: 13338.404106597074\nMinimizing 15197 Time: 0:01:44 ( 6.90 ms/it)\n objective: 13338.382898093128\nMinimizing 15212 Time: 0:01:44 ( 6.90 ms/it)\n objective: 13338.380698813948\nMinimizing 15227 Time: 0:01:45 ( 6.90 ms/it)\n objective: 13338.376206276116\nMinimizing 15242 Time: 0:01:45 ( 6.90 ms/it)\n objective: 13338.37009423472\nMinimizing 15257 Time: 0:01:45 ( 6.90 ms/it)\n objective: 13338.332375120226\nMinimizing 15272 Time: 0:01:45 ( 6.90 ms/it)\n objective: 13338.315422826745\nMinimizing 15287 Time: 0:01:45 ( 6.90 ms/it)\n objective: 13338.30016207126\nMinimizing 15302 Time: 0:01:45 ( 6.90 ms/it)\n objective: 13338.28237848018\nMinimizing 15317 Time: 0:01:45 ( 6.90 ms/it)\n objective: 13338.058349866842\nMinimizing 15332 Time: 0:01:45 ( 6.90 ms/it)\n objective: 13337.995714274839\nMinimizing 15347 Time: 0:01:45 ( 6.90 ms/it)\n objective: 13337.991177958575\nMinimizing 15362 Time: 0:01:46 ( 6.90 ms/it)\n objective: 13337.977711082582\nMinimizing 15377 Time: 0:01:46 ( 6.90 ms/it)\n objective: 13337.964303749359\nMinimizing 15392 Time: 0:01:46 ( 6.90 ms/it)\n objective: 13337.950417868175\nMinimizing 15407 Time: 0:01:46 ( 6.90 ms/it)\n objective: 13337.924826576214\nMinimizing 15422 Time: 0:01:46 ( 6.90 ms/it)\n objective: 13337.922166207369\nMinimizing 15437 Time: 0:01:46 ( 6.90 ms/it)\n objective: 13337.915342303437\nMinimizing 15452 Time: 0:01:46 ( 6.90 ms/it)\n objective: 13337.892925831431\nMinimizing 15467 Time: 0:01:46 ( 6.90 ms/it)\n objective: 13337.870776943266\nMinimizing 15482 Time: 0:01:46 ( 6.90 ms/it)\n objective: 13337.82368052403\nMinimizing 15497 Time: 0:01:46 ( 6.90 ms/it)\n objective: 13337.802456079604\nMinimizing 15512 Time: 0:01:47 ( 6.90 ms/it)\n objective: 13337.778319405399\nMinimizing 15527 Time: 0:01:47 ( 6.90 ms/it)\n objective: 13337.723085633355\nMinimizing 15542 Time: 0:01:47 ( 6.90 ms/it)\n objective: 13337.681336951027\nMinimizing 15557 Time: 0:01:47 ( 6.90 ms/it)\n objective: 13337.623714578207\nMinimizing 15572 Time: 0:01:47 ( 6.90 ms/it)\n objective: 13337.619912843751\nMinimizing 15587 Time: 0:01:47 ( 6.90 ms/it)\n objective: 13337.592407365271\nMinimizing 15602 Time: 0:01:47 ( 6.90 ms/it)\n objective: 13337.524977112553\nMinimizing 15617 Time: 0:01:47 ( 6.90 ms/it)\n objective: 13337.501384933617\nMinimizing 15632 Time: 0:01:47 ( 6.90 ms/it)\n objective: 13337.372896101675\nMinimizing 15647 Time: 0:01:47 ( 6.90 ms/it)\n objective: 13337.244769924691\nMinimizing 15662 Time: 0:01:48 ( 6.90 ms/it)\n objective: 13337.219267041073\nMinimizing 15677 Time: 0:01:48 ( 6.90 ms/it)\n objective: 13337.209281208532\nMinimizing 15692 Time: 0:01:48 ( 6.90 ms/it)\n objective: 13337.194651381185\nMinimizing 15707 Time: 0:01:48 ( 6.90 ms/it)\n objective: 13337.163609343253\nMinimizing 15722 Time: 0:01:48 ( 6.90 ms/it)\n objective: 13337.367427068624\nMinimizing 15737 Time: 0:01:48 ( 6.90 ms/it)\n objective: 13337.000329675291\nMinimizing 15752 Time: 0:01:48 ( 6.90 ms/it)\n objective: 13336.977122713302\nMinimizing 15767 Time: 0:01:48 ( 6.90 ms/it)\n objective: 13336.94370140139\nMinimizing 15782 Time: 0:01:48 ( 6.90 ms/it)\n objective: 13336.92256336382\nMinimizing 15797 Time: 0:01:49 ( 6.90 ms/it)\n objective: 13336.912565204024\nMinimizing 15812 Time: 0:01:49 ( 6.90 ms/it)\n objective: 13336.846513392375\nMinimizing 15827 Time: 0:01:49 ( 6.90 ms/it)\n objective: 13336.756989461734\nMinimizing 15842 Time: 0:01:49 ( 6.90 ms/it)\n objective: 13336.615976333116\nMinimizing 15857 Time: 0:01:49 ( 6.90 ms/it)\n objective: 13336.269223868083\nMinimizing 15872 Time: 0:01:49 ( 6.90 ms/it)\n objective: 13336.261366299914\nMinimizing 15887 Time: 0:01:49 ( 6.90 ms/it)\n objective: 13336.25777220614\nMinimizing 15902 Time: 0:01:49 ( 6.90 ms/it)\n objective: 13336.256172290574\nMinimizing 15917 Time: 0:01:49 ( 6.90 ms/it)\n objective: 13336.251656851491\nMinimizing 15931 Time: 0:01:49 ( 6.90 ms/it)\n objective: 13336.245449235874\nMinimizing 15945 Time: 0:01:50 ( 6.90 ms/it)\n objective: 13336.232147575625\nMinimizing 15958 Time: 0:01:50 ( 6.90 ms/it)\n objective: 13336.21083419981\nMinimizing 15972 Time: 0:01:50 ( 6.90 ms/it)\n objective: 13336.182484429686\nMinimizing 15986 Time: 0:01:50 ( 6.91 ms/it)\n objective: 13336.175686567054\nMinimizing 16000 Time: 0:01:50 ( 6.91 ms/it)\n objective: 13336.14994236223\nMinimizing 16014 Time: 0:01:50 ( 6.91 ms/it)\n objective: 13336.103789755027\nMinimizing 16028 Time: 0:01:50 ( 6.91 ms/it)\n objective: 13335.925686136121\nMinimizing 16042 Time: 0:01:50 ( 6.91 ms/it)\n objective: 13335.920391065847\nMinimizing 16056 Time: 0:01:50 ( 6.91 ms/it)\n objective: 13335.896999862474\nMinimizing 16070 Time: 0:01:51 ( 6.91 ms/it)\n objective: 13335.861006309897\nMinimizing 16084 Time: 0:01:51 ( 6.91 ms/it)\n objective: 13335.777256011628\nMinimizing 16098 Time: 0:01:51 ( 6.91 ms/it)\n objective: 13335.745853048706\nMinimizing 16112 Time: 0:01:51 ( 6.91 ms/it)\n objective: 13335.74129549932\nMinimizing 16126 Time: 0:01:51 ( 6.91 ms/it)\n objective: 13335.801324112384\nMinimizing 16140 Time: 0:01:51 ( 6.91 ms/it)\n objective: 13335.624783455154\nMinimizing 16154 Time: 0:01:51 ( 6.91 ms/it)\n objective: 13335.579189488424\nMinimizing 16168 Time: 0:01:51 ( 6.91 ms/it)\n objective: 13335.415252530795\nMinimizing 16182 Time: 0:01:51 ( 6.91 ms/it)\n objective: 13335.223253945842\nMinimizing 16196 Time: 0:01:51 ( 6.91 ms/it)\n objective: 13335.221178535947\nMinimizing 16210 Time: 0:01:52 ( 6.91 ms/it)\n objective: 13335.219536509227\nMinimizing 16224 Time: 0:01:52 ( 6.91 ms/it)\n objective: 13335.21841234529\nMinimizing 16238 Time: 0:01:52 ( 6.91 ms/it)\n objective: 13335.217925056699\nMinimizing 16252 Time: 0:01:52 ( 6.91 ms/it)\n objective: 13335.210506432384\nMinimizing 16266 Time: 0:01:52 ( 6.91 ms/it)\n objective: 13335.183660413655\nMinimizing 16280 Time: 0:01:52 ( 6.91 ms/it)\n objective: 13335.133617477237\nMinimizing 16294 Time: 0:01:52 ( 6.91 ms/it)\n objective: 13335.122020143863\nMinimizing 16308 Time: 0:01:52 ( 6.91 ms/it)\n objective: 13335.118897177003\nMinimizing 16322 Time: 0:01:52 ( 6.91 ms/it)\n objective: 13335.108725729217\nMinimizing 16336 Time: 0:01:52 ( 6.91 ms/it)\n objective: 13335.105212219372\nMinimizing 16350 Time: 0:01:53 ( 6.91 ms/it)\n objective: 13335.095815820729\nMinimizing 16364 Time: 0:01:53 ( 6.91 ms/it)\n objective: 13335.06571175185\nMinimizing 16378 Time: 0:01:53 ( 6.91 ms/it)\n objective: 13335.061220820906\nMinimizing 16392 Time: 0:01:53 ( 6.92 ms/it)\n objective: 13335.008390296942\nMinimizing 16406 Time: 0:01:53 ( 6.92 ms/it)\n objective: 13334.99009610057\nMinimizing 16420 Time: 0:01:53 ( 6.92 ms/it)\n objective: 13334.916519054612\nMinimizing 16435 Time: 0:01:53 ( 6.92 ms/it)\n objective: 13334.845595854706\nMinimizing 16449 Time: 0:01:53 ( 6.92 ms/it)\n objective: 13334.843490636966\nMinimizing 16463 Time: 0:01:53 ( 6.92 ms/it)\n objective: 13334.836694603458\nMinimizing 16477 Time: 0:01:53 ( 6.92 ms/it)\n objective: 13334.82669134588\nMinimizing 16491 Time: 0:01:54 ( 6.92 ms/it)\n objective: 13334.806544282648\nMinimizing 16505 Time: 0:01:54 ( 6.92 ms/it)\n objective: 13334.801103618047\nMinimizing 16519 Time: 0:01:54 ( 6.92 ms/it)\n objective: 13334.759274538636\nMinimizing 16533 Time: 0:01:54 ( 6.92 ms/it)\n objective: 13334.751347941929\nMinimizing 16547 Time: 0:01:54 ( 6.92 ms/it)\n objective: 13334.728748197238\nMinimizing 16561 Time: 0:01:54 ( 6.92 ms/it)\n objective: 13334.64750024394\nMinimizing 16575 Time: 0:01:54 ( 6.92 ms/it)\n objective: 13334.583685550962\nMinimizing 16589 Time: 0:01:54 ( 6.92 ms/it)\n objective: 13334.583280504725\nMinimizing 16603 Time: 0:01:54 ( 6.92 ms/it)\n objective: 13334.569952924663\nMinimizing 16618 Time: 0:01:54 ( 6.92 ms/it)\n objective: 13334.567988298215\nMinimizing 16633 Time: 0:01:55 ( 6.92 ms/it)\n objective: 13334.546262842334\nMinimizing 16648 Time: 0:01:55 ( 6.92 ms/it)\n objective: 13334.544138925514\nMinimizing 16663 Time: 0:01:55 ( 6.92 ms/it)\n objective: 13334.510710211216\nMinimizing 16678 Time: 0:01:55 ( 6.92 ms/it)\n objective: 13334.476889563892\nMinimizing 16693 Time: 0:01:55 ( 6.92 ms/it)\n objective: 13334.454626424107\nMinimizing 16708 Time: 0:01:55 ( 6.92 ms/it)\n objective: 13334.362062850785\nMinimizing 16723 Time: 0:01:55 ( 6.92 ms/it)\n objective: 13334.360619089632\nMinimizing 16738 Time: 0:01:55 ( 6.92 ms/it)\n objective: 13334.341550746001\nMinimizing 16753 Time: 0:01:55 ( 6.92 ms/it)\n objective: 13334.320402974976\nMinimizing 16768 Time: 0:01:56 ( 6.92 ms/it)\n objective: 13334.307457812945\nMinimizing 16783 Time: 0:01:56 ( 6.92 ms/it)\n objective: 13334.224614883999\nMinimizing 16798 Time: 0:01:56 ( 6.92 ms/it)\n objective: 13334.095644262517\nMinimizing 16813 Time: 0:01:56 ( 6.92 ms/it)\n objective: 13334.054532773436\nMinimizing 16828 Time: 0:01:56 ( 6.92 ms/it)\n objective: 13334.052562382632\nMinimizing 16843 Time: 0:01:56 ( 6.92 ms/it)\n objective: 13334.02038898351\nMinimizing 16858 Time: 0:01:56 ( 6.92 ms/it)\n objective: 13334.014248053682\nMinimizing 16873 Time: 0:01:56 ( 6.92 ms/it)\n objective: 13333.988855241048\nMinimizing 16888 Time: 0:01:56 ( 6.92 ms/it)\n objective: 13333.867898937198\nMinimizing 16903 Time: 0:01:56 ( 6.92 ms/it)\n objective: 13333.859219855774\nMinimizing 16918 Time: 0:01:57 ( 6.92 ms/it)\n objective: 13333.846383981363\nMinimizing 16933 Time: 0:01:57 ( 6.92 ms/it)\n objective: 13333.84357290766\nMinimizing 16948 Time: 0:01:57 ( 6.92 ms/it)\n objective: 13333.84133947506\nMinimizing 16963 Time: 0:01:57 ( 6.92 ms/it)\n objective: 13333.837289346578\nMinimizing 16978 Time: 0:01:57 ( 6.92 ms/it)\n objective: 13333.819624881588\nMinimizing 16993 Time: 0:01:57 ( 6.92 ms/it)\n objective: 13333.75923950121\nMinimizing 17008 Time: 0:01:57 ( 6.92 ms/it)\n objective: 13333.758536256828\nMinimizing 17023 Time: 0:01:57 ( 6.92 ms/it)\n objective: 13333.746101364486\nMinimizing 17038 Time: 0:01:57 ( 6.92 ms/it)\n objective: 13333.741782106052\nMinimizing 17053 Time: 0:01:58 ( 6.92 ms/it)\n objective: 13333.720342352091\nMinimizing 17068 Time: 0:01:58 ( 6.92 ms/it)\n objective: 13333.688469015011\nMinimizing 17083 Time: 0:01:58 ( 6.92 ms/it)\n objective: 13333.609223999862\nMinimizing 17098 Time: 0:01:58 ( 6.92 ms/it)\n objective: 13333.608730716616\nMinimizing 17112 Time: 0:01:58 ( 6.92 ms/it)\n objective: 13333.606889065406\nMinimizing 17125 Time: 0:01:58 ( 6.92 ms/it)\n objective: 13333.600911253809\nMinimizing 17139 Time: 0:01:58 ( 6.92 ms/it)\n objective: 13333.599414144795\nMinimizing 17153 Time: 0:01:58 ( 6.92 ms/it)\n objective: 13333.58434479226\nMinimizing 17166 Time: 0:01:58 ( 6.93 ms/it)\n objective: 13333.561687261295\nMinimizing 17180 Time: 0:01:58 ( 6.93 ms/it)\n objective: 13333.559435556308\nMinimizing 17194 Time: 0:01:59 ( 6.93 ms/it)\n objective: 13333.552408369571\nMinimizing 17209 Time: 0:01:59 ( 6.93 ms/it)\n objective: 13333.54321048825\nMinimizing 17224 Time: 0:01:59 ( 6.93 ms/it)\n objective: 13333.515494220796\nMinimizing 17239 Time: 0:01:59 ( 6.93 ms/it)\n objective: 13333.499174522709\nMinimizing 17254 Time: 0:01:59 ( 6.93 ms/it)\n objective: 13333.493274650835\nMinimizing 17269 Time: 0:01:59 ( 6.93 ms/it)\n objective: 13333.455186557658\nMinimizing 17284 Time: 0:01:59 ( 6.93 ms/it)\n objective: 13333.437993298452\nMinimizing 17299 Time: 0:01:59 ( 6.93 ms/it)\n objective: 13333.320915752374\nMinimizing 17314 Time: 0:01:59 ( 6.93 ms/it)\n objective: 13333.290149521541\nMinimizing 17329 Time: 0:02:00 ( 6.93 ms/it)\n objective: 13333.289035316557\nMinimizing 17344 Time: 0:02:00 ( 6.93 ms/it)\n objective: 13333.279764576284\nMinimizing 17359 Time: 0:02:00 ( 6.93 ms/it)\n objective: 13333.262703755448\nMinimizing 17374 Time: 0:02:00 ( 6.93 ms/it)\n objective: 13333.231376314456\nMinimizing 17389 Time: 0:02:00 ( 6.93 ms/it)\n objective: 13333.22627246497\nMinimizing 17404 Time: 0:02:00 ( 6.93 ms/it)\n objective: 13333.216820631176\nMinimizing 17419 Time: 0:02:00 ( 6.93 ms/it)\n objective: 13333.180143574398\nMinimizing 17434 Time: 0:02:00 ( 6.93 ms/it)\n objective: 13333.158613650587\nMinimizing 17449 Time: 0:02:00 ( 6.93 ms/it)\n objective: 13333.14023486711\nMinimizing 17464 Time: 0:02:00 ( 6.93 ms/it)\n objective: 13333.126857057105\nMinimizing 17479 Time: 0:02:01 ( 6.93 ms/it)\n objective: 13333.106941937003\nMinimizing 17494 Time: 0:02:01 ( 6.93 ms/it)\n objective: 13333.056970260404\nMinimizing 17509 Time: 0:02:01 ( 6.93 ms/it)\n objective: 13333.050433719633\nMinimizing 17524 Time: 0:02:01 ( 6.93 ms/it)\n objective: 13333.04520694173\nMinimizing 17539 Time: 0:02:01 ( 6.93 ms/it)\n objective: 13333.02308609712\nMinimizing 17554 Time: 0:02:01 ( 6.93 ms/it)\n objective: 13333.013994719513\nMinimizing 17569 Time: 0:02:01 ( 6.93 ms/it)\n objective: 13332.977936083349\nMinimizing 17584 Time: 0:02:01 ( 6.93 ms/it)\n objective: 13332.975511619945\nMinimizing 17599 Time: 0:02:01 ( 6.93 ms/it)\n objective: 13332.958474803265\nMinimizing 17614 Time: 0:02:01 ( 6.93 ms/it)\n objective: 13332.952854860698\nMinimizing 17629 Time: 0:02:02 ( 6.93 ms/it)\n objective: 13332.919815430047\nMinimizing 17644 Time: 0:02:02 ( 6.93 ms/it)\n objective: 13332.882694401924\nMinimizing 17659 Time: 0:02:02 ( 6.93 ms/it)\n objective: 13332.878704391602\nMinimizing 17674 Time: 0:02:02 ( 6.93 ms/it)\n objective: 13332.851167182474\nMinimizing 17689 Time: 0:02:02 ( 6.93 ms/it)\n objective: 13332.788209513761\nMinimizing 17704 Time: 0:02:02 ( 6.93 ms/it)\n objective: 13332.703097644699\nMinimizing 17719 Time: 0:02:02 ( 6.93 ms/it)\n objective: 13332.701187844163\nMinimizing 17734 Time: 0:02:02 ( 6.93 ms/it)\n objective: 13332.697815801825\nMinimizing 17749 Time: 0:02:02 ( 6.93 ms/it)\n objective: 13332.690323827534\nMinimizing 17764 Time: 0:02:03 ( 6.93 ms/it)\n objective: 13332.687247730108\nMinimizing 17779 Time: 0:02:03 ( 6.93 ms/it)\n objective: 13332.673238247422\nMinimizing 17794 Time: 0:02:03 ( 6.93 ms/it)\n objective: 13332.660994644226\nMinimizing 17809 Time: 0:02:03 ( 6.93 ms/it)\n objective: 13332.64622725836\nMinimizing 17824 Time: 0:02:03 ( 6.93 ms/it)\n objective: 13332.645241794671\nMinimizing 17839 Time: 0:02:03 ( 6.93 ms/it)\n objective: 13332.640238329994\nMinimizing 17854 Time: 0:02:03 ( 6.93 ms/it)\n objective: 13332.631622952686\nMinimizing 17869 Time: 0:02:03 ( 6.93 ms/it)\n objective: 13332.612804732627\nMinimizing 17884 Time: 0:02:03 ( 6.93 ms/it)\n objective: 13332.607820559344\nMinimizing 17899 Time: 0:02:03 ( 6.92 ms/it)\n objective: 13332.600145817894\nMinimizing 17914 Time: 0:02:04 ( 6.92 ms/it)\n objective: 13332.529633013582\nMinimizing 17929 Time: 0:02:04 ( 6.92 ms/it)\n objective: 13332.501587057173\nMinimizing 17944 Time: 0:02:04 ( 6.92 ms/it)\n objective: 13332.499290583983\nMinimizing 17959 Time: 0:02:04 ( 6.92 ms/it)\n objective: 13332.489295514795\nMinimizing 17974 Time: 0:02:04 ( 6.92 ms/it)\n objective: 13332.484992574231\nMinimizing 17989 Time: 0:02:04 ( 6.92 ms/it)\n objective: 13332.454313224043\nMinimizing 18004 Time: 0:02:04 ( 6.92 ms/it)\n objective: 13332.453012897939\nMinimizing 18019 Time: 0:02:04 ( 6.92 ms/it)\n objective: 13332.444465408546\nMinimizing 18034 Time: 0:02:04 ( 6.92 ms/it)\n objective: 13332.43640637002\nMinimizing 18049 Time: 0:02:04 ( 6.92 ms/it)\n objective: 13332.415250480924\nMinimizing 18064 Time: 0:02:05 ( 6.92 ms/it)\n objective: 13332.38002569967\nMinimizing 18079 Time: 0:02:05 ( 6.92 ms/it)\n objective: 13332.343783860095\nMinimizing 18094 Time: 0:02:05 ( 6.92 ms/it)\n objective: 13332.342943183561\nMinimizing 18109 Time: 0:02:05 ( 6.92 ms/it)\n objective: 13332.328013336613\nMinimizing 18124 Time: 0:02:05 ( 6.92 ms/it)\n objective: 13332.304945329277\nMinimizing 18139 Time: 0:02:05 ( 6.92 ms/it)\n objective: 13332.288060278857\nMinimizing 18154 Time: 0:02:05 ( 6.92 ms/it)\n objective: 13332.252287369141\nMinimizing 18169 Time: 0:02:05 ( 6.92 ms/it)\n objective: 13332.194609537379\nMinimizing 18184 Time: 0:02:05 ( 6.92 ms/it)\n objective: 13332.183633060944\nMinimizing 18199 Time: 0:02:05 ( 6.92 ms/it)\n objective: 13332.144665969412\nMinimizing 18214 Time: 0:02:06 ( 6.92 ms/it)\n objective: 13332.135833717242\nMinimizing 18229 Time: 0:02:06 ( 6.92 ms/it)\n objective: 13332.13156465999\nMinimizing 18244 Time: 0:02:06 ( 6.92 ms/it)\n objective: 13332.11959062986\nMinimizing 18259 Time: 0:02:06 ( 6.92 ms/it)\n objective: 13332.110825140153\nMinimizing 18274 Time: 0:02:06 ( 6.92 ms/it)\n objective: 13332.09751962519\nMinimizing 18289 Time: 0:02:06 ( 6.92 ms/it)\n objective: 13332.079641709526\nMinimizing 18304 Time: 0:02:06 ( 6.92 ms/it)\n objective: 13332.079195920887\nMinimizing 18319 Time: 0:02:06 ( 6.92 ms/it)\n objective: 13332.05560646786\nMinimizing 18334 Time: 0:02:06 ( 6.92 ms/it)\n objective: 13332.033062607668\nMinimizing 18349 Time: 0:02:07 ( 6.92 ms/it)\n objective: 13332.017118449927\nMinimizing 18364 Time: 0:02:07 ( 6.92 ms/it)\n objective: 13331.834283003831\nMinimizing 18379 Time: 0:02:07 ( 6.92 ms/it)\n objective: 13331.829523296801\nMinimizing 18394 Time: 0:02:07 ( 6.92 ms/it)\n objective: 13331.82185353677\nMinimizing 18409 Time: 0:02:07 ( 6.92 ms/it)\n objective: 13331.811126325367\nMinimizing 18424 Time: 0:02:07 ( 6.92 ms/it)\n objective: 13331.796843724907\nMinimizing 18439 Time: 0:02:07 ( 6.92 ms/it)\n objective: 13331.773058564875\nMinimizing 18454 Time: 0:02:07 ( 6.92 ms/it)\n objective: 13331.75078281766\nMinimizing 18469 Time: 0:02:07 ( 6.92 ms/it)\n objective: 13331.740070893735\nMinimizing 18484 Time: 0:02:07 ( 6.92 ms/it)\n objective: 13331.729061820188\nMinimizing 18499 Time: 0:02:08 ( 6.92 ms/it)\n objective: 13331.727070406356\nMinimizing 18514 Time: 0:02:08 ( 6.92 ms/it)\n objective: 13331.698226752102\nMinimizing 18529 Time: 0:02:08 ( 6.92 ms/it)\n objective: 13331.662677190616\nMinimizing 18544 Time: 0:02:08 ( 6.92 ms/it)\n objective: 13331.435033106842\nMinimizing 18559 Time: 0:02:08 ( 6.92 ms/it)\n objective: 13331.418241083811\nMinimizing 18574 Time: 0:02:08 ( 6.92 ms/it)\n objective: 13331.412412329562\nMinimizing 18589 Time: 0:02:08 ( 6.92 ms/it)\n objective: 13331.406819941993\nMinimizing 18604 Time: 0:02:08 ( 6.92 ms/it)\n objective: 13331.403233328943\nMinimizing 18619 Time: 0:02:08 ( 6.92 ms/it)\n objective: 13331.398361885542\nMinimizing 18634 Time: 0:02:08 ( 6.92 ms/it)\n objective: 13331.387101725872\nMinimizing 18649 Time: 0:02:09 ( 6.92 ms/it)\n objective: 13331.38005361161\nMinimizing 18664 Time: 0:02:09 ( 6.92 ms/it)\n objective: 13331.326018027437\nMinimizing 18679 Time: 0:02:09 ( 6.92 ms/it)\n objective: 13331.04874836732\nMinimizing 18694 Time: 0:02:09 ( 6.92 ms/it)\n objective: 13331.016200164442\nMinimizing 18709 Time: 0:02:09 ( 6.92 ms/it)\n objective: 13331.014856694055\nMinimizing 18724 Time: 0:02:09 ( 6.92 ms/it)\n objective: 13331.000092066279\nMinimizing 18739 Time: 0:02:09 ( 6.92 ms/it)\n objective: 13330.98174820219\nMinimizing 18754 Time: 0:02:09 ( 6.92 ms/it)\n objective: 13330.923246875565\nMinimizing 18769 Time: 0:02:09 ( 6.92 ms/it)\n objective: 13331.057292039288\nMinimizing 18784 Time: 0:02:09 ( 6.92 ms/it)\n objective: 13330.820249156663\nMinimizing 18799 Time: 0:02:10 ( 6.92 ms/it)\n objective: 13330.819560777869\nMinimizing 18814 Time: 0:02:10 ( 6.92 ms/it)\n objective: 13330.81742589306\nMinimizing 18829 Time: 0:02:10 ( 6.92 ms/it)\n objective: 13330.813268829435\nMinimizing 18844 Time: 0:02:10 ( 6.92 ms/it)\n objective: 13330.809176184055\nMinimizing 18859 Time: 0:02:10 ( 6.92 ms/it)\n objective: 13330.780886687935\nMinimizing 18874 Time: 0:02:10 ( 6.92 ms/it)\n objective: 13330.735498194335\nMinimizing 18889 Time: 0:02:10 ( 6.92 ms/it)\n objective: 13330.717466289127\nMinimizing 18904 Time: 0:02:10 ( 6.92 ms/it)\n objective: 13330.671095346988\nMinimizing 18919 Time: 0:02:10 ( 6.92 ms/it)\n objective: 13330.658830652581\nMinimizing 18934 Time: 0:02:10 ( 6.92 ms/it)\n objective: 13330.646016764695\nMinimizing 18949 Time: 0:02:11 ( 6.92 ms/it)\n objective: 13330.629924788125\nMinimizing 18964 Time: 0:02:11 ( 6.92 ms/it)\n objective: 13330.575998203167\nMinimizing 18979 Time: 0:02:11 ( 6.92 ms/it)\n objective: 13330.5184689651\nMinimizing 18994 Time: 0:02:11 ( 6.92 ms/it)\n objective: 13330.51788424806\nMinimizing 19009 Time: 0:02:11 ( 6.92 ms/it)\n objective: 13330.51448597193\nMinimizing 19024 Time: 0:02:11 ( 6.92 ms/it)\n objective: 13330.509704314725\nMinimizing 19039 Time: 0:02:11 ( 6.92 ms/it)\n objective: 13330.503051528925\nMinimizing 19054 Time: 0:02:11 ( 6.92 ms/it)\n objective: 13330.479735019435\nMinimizing 19069 Time: 0:02:11 ( 6.92 ms/it)\n objective: 13330.46355300842\nMinimizing 19084 Time: 0:02:12 ( 6.92 ms/it)\n objective: 13330.432544731171\nMinimizing 19099 Time: 0:02:12 ( 6.92 ms/it)\n objective: 13330.422039944147\nMinimizing 19114 Time: 0:02:12 ( 6.92 ms/it)\n objective: 13330.358435354035\nMinimizing 19129 Time: 0:02:12 ( 6.92 ms/it)\n objective: 13330.335687806124\nMinimizing 19144 Time: 0:02:12 ( 6.92 ms/it)\n objective: 13330.334375663617\nMinimizing 19159 Time: 0:02:12 ( 6.92 ms/it)\n objective: 13330.322470435196\nMinimizing 19174 Time: 0:02:12 ( 6.92 ms/it)\n objective: 13330.277306498145\nMinimizing 19189 Time: 0:02:12 ( 6.92 ms/it)\n objective: 13330.270544234161\nMinimizing 19204 Time: 0:02:12 ( 6.92 ms/it)\n objective: 13330.209340675552\nMinimizing 19219 Time: 0:02:12 ( 6.92 ms/it)\n objective: 13330.205449058165\nMinimizing 19234 Time: 0:02:13 ( 6.92 ms/it)\n objective: 13330.199347486465\nMinimizing 19249 Time: 0:02:13 ( 6.92 ms/it)\n objective: 13330.186147581022\nMinimizing 19264 Time: 0:02:13 ( 6.92 ms/it)\n objective: 13330.15832003655\nMinimizing 19279 Time: 0:02:13 ( 6.92 ms/it)\n objective: 13330.074843961767\nMinimizing 19294 Time: 0:02:13 ( 6.92 ms/it)\n objective: 13330.02981807918\nMinimizing 19309 Time: 0:02:13 ( 6.92 ms/it)\n objective: 13330.02634915684\nMinimizing 19324 Time: 0:02:13 ( 6.92 ms/it)\n objective: 13329.999798483856\nMinimizing 19339 Time: 0:02:13 ( 6.92 ms/it)\n objective: 13329.969057325463\nMinimizing 19354 Time: 0:02:13 ( 6.92 ms/it)\n objective: 13329.963734783938\nMinimizing 19369 Time: 0:02:13 ( 6.92 ms/it)\n objective: 13329.95293151198\nMinimizing 19384 Time: 0:02:14 ( 6.92 ms/it)\n objective: 13329.933692490544\nMinimizing 19399 Time: 0:02:14 ( 6.92 ms/it)\n objective: 13329.91544806432\nMinimizing 19414 Time: 0:02:14 ( 6.92 ms/it)\n objective: 13329.897226474255\nMinimizing 19429 Time: 0:02:14 ( 6.92 ms/it)\n objective: 13329.833230712793\nMinimizing 19444 Time: 0:02:14 ( 6.92 ms/it)\n objective: 13329.83249528308\nMinimizing 19459 Time: 0:02:14 ( 6.92 ms/it)\n objective: 13329.828962245141\nMinimizing 19474 Time: 0:02:14 ( 6.91 ms/it)\n objective: 13329.825982439135\nMinimizing 19489 Time: 0:02:14 ( 6.91 ms/it)\n objective: 13329.821616552857\nMinimizing 19504 Time: 0:02:14 ( 6.91 ms/it)\n objective: 13329.814144659802\nMinimizing 19519 Time: 0:02:14 ( 6.91 ms/it)\n objective: 13329.79126935944\nMinimizing 19534 Time: 0:02:15 ( 6.91 ms/it)\n objective: 13329.789456750936\nMinimizing 19549 Time: 0:02:15 ( 6.91 ms/it)\n objective: 13329.755567503045\nMinimizing 19564 Time: 0:02:15 ( 6.91 ms/it)\n objective: 13329.75065324807\nMinimizing 19579 Time: 0:02:15 ( 6.91 ms/it)\n objective: 13329.74617634451\nMinimizing 19594 Time: 0:02:15 ( 6.91 ms/it)\n objective: 13329.729968831707\nMinimizing 19609 Time: 0:02:15 ( 6.91 ms/it)\n objective: 13329.718980641403\nMinimizing 19624 Time: 0:02:15 ( 6.91 ms/it)\n objective: 13329.652253531873\nMinimizing 19639 Time: 0:02:15 ( 6.91 ms/it)\n objective: 13329.650557134548\nMinimizing 19654 Time: 0:02:15 ( 6.91 ms/it)\n objective: 13329.647233110918\nMinimizing 19669 Time: 0:02:15 ( 6.91 ms/it)\n objective: 13329.637596124892\nMinimizing 19684 Time: 0:02:16 ( 6.91 ms/it)\n objective: 13329.629141556885\nMinimizing 19699 Time: 0:02:16 ( 6.91 ms/it)\n objective: 13329.597380038285\nMinimizing 19714 Time: 0:02:16 ( 6.91 ms/it)\n objective: 13329.590077110573\nMinimizing 19729 Time: 0:02:16 ( 6.91 ms/it)\n objective: 13329.550919230649\nMinimizing 19744 Time: 0:02:16 ( 6.91 ms/it)\n objective: 13329.53728489844\nMinimizing 19759 Time: 0:02:16 ( 6.91 ms/it)\n objective: 13329.528018004312\nMinimizing 19774 Time: 0:02:16 ( 6.91 ms/it)\n objective: 13329.492447316414\nMinimizing 19789 Time: 0:02:16 ( 6.91 ms/it)\n objective: 13329.468808185818\nMinimizing 19804 Time: 0:02:16 ( 6.91 ms/it)\n objective: 13329.46771172135\nMinimizing 19819 Time: 0:02:17 ( 6.91 ms/it)\n objective: 13329.465938575828\nMinimizing 19834 Time: 0:02:17 ( 6.91 ms/it)\n objective: 13329.459824371843\nMinimizing 19849 Time: 0:02:17 ( 6.91 ms/it)\n objective: 13329.455256816233\nMinimizing 19864 Time: 0:02:17 ( 6.91 ms/it)\n objective: 13329.448280938886\nMinimizing 19879 Time: 0:02:17 ( 6.91 ms/it)\n objective: 13329.4346785796\nMinimizing 19894 Time: 0:02:17 ( 6.91 ms/it)\n objective: 13329.425251753863\nMinimizing 19909 Time: 0:02:17 ( 6.91 ms/it)\n objective: 13329.351055840372\nMinimizing 19924 Time: 0:02:17 ( 6.91 ms/it)\n objective: 13329.347318027976\nMinimizing 19939 Time: 0:02:17 ( 6.91 ms/it)\n objective: 13329.346121603194\nMinimizing 19954 Time: 0:02:17 ( 6.91 ms/it)\n objective: 13329.342177374921\nMinimizing 19969 Time: 0:02:18 ( 6.91 ms/it)\n objective: 13329.33260046925\nMinimizing 19984 Time: 0:02:18 ( 6.91 ms/it)\n objective: 13329.32460494536\nMinimizing 19999 Time: 0:02:18 ( 6.91 ms/it)\n objective: 13329.289010897519\nMinimizing 20014 Time: 0:02:18 ( 6.91 ms/it)\n objective: 13329.263827779709\nMinimizing 20029 Time: 0:02:18 ( 6.91 ms/it)\n objective: 13329.262276728165\nMinimizing 20044 Time: 0:02:18 ( 6.91 ms/it)\n objective: 13329.250618800848\nMinimizing 20059 Time: 0:02:18 ( 6.91 ms/it)\n objective: 13329.247834330512\nMinimizing 20074 Time: 0:02:18 ( 6.91 ms/it)\n objective: 13329.204974296998\nMinimizing 20089 Time: 0:02:18 ( 6.91 ms/it)\n objective: 13329.16436753214\nMinimizing 20104 Time: 0:02:18 ( 6.91 ms/it)\n objective: 13329.1626961406\nMinimizing 20119 Time: 0:02:19 ( 6.91 ms/it)\n objective: 13329.154209735047\nMinimizing 20134 Time: 0:02:19 ( 6.91 ms/it)\n objective: 13329.143880958145\nMinimizing 20149 Time: 0:02:19 ( 6.91 ms/it)\n objective: 13329.117962488679\nMinimizing 20164 Time: 0:02:19 ( 6.91 ms/it)\n objective: 13329.10457216049\nMinimizing 20179 Time: 0:02:19 ( 6.91 ms/it)\n objective: 13329.099038291482\nMinimizing 20194 Time: 0:02:19 ( 6.91 ms/it)\n objective: 13329.072185730416\nMinimizing 20209 Time: 0:02:19 ( 6.91 ms/it)\n objective: 13329.071166924543\nMinimizing 20224 Time: 0:02:19 ( 6.91 ms/it)\n objective: 13329.055917316553\nMinimizing 20239 Time: 0:02:19 ( 6.91 ms/it)\n objective: 13329.04550733111\nMinimizing 20254 Time: 0:02:19 ( 6.91 ms/it)\n objective: 13328.962482216775\nMinimizing 20269 Time: 0:02:20 ( 6.91 ms/it)\n objective: 13328.927748833332\nMinimizing 20284 Time: 0:02:20 ( 6.91 ms/it)\n objective: 13328.926812721613\nMinimizing 20299 Time: 0:02:20 ( 6.91 ms/it)\n objective: 13328.919377647391\nMinimizing 20314 Time: 0:02:20 ( 6.91 ms/it)\n objective: 13328.913483455151\nMinimizing 20329 Time: 0:02:20 ( 6.91 ms/it)\n objective: 13328.909083503539\nMinimizing 20344 Time: 0:02:20 ( 6.91 ms/it)\n objective: 13328.90125764248\nMinimizing 20359 Time: 0:02:20 ( 6.91 ms/it)\n objective: 13328.891911836923\nMinimizing 20374 Time: 0:02:20 ( 6.91 ms/it)\n objective: 13328.882679207527\nMinimizing 20389 Time: 0:02:20 ( 6.91 ms/it)\n objective: 13328.831126148958\nMinimizing 20404 Time: 0:02:21 ( 6.91 ms/it)\n objective: 13328.816507312069\nMinimizing 20419 Time: 0:02:21 ( 6.91 ms/it)\n objective: 13328.815969157455\nMinimizing 20434 Time: 0:02:21 ( 6.91 ms/it)\n objective: 13328.815103480607\nMinimizing 20449 Time: 0:02:21 ( 6.91 ms/it)\n objective: 13328.807968008216\nMinimizing 20464 Time: 0:02:21 ( 6.91 ms/it)\n objective: 13328.79801758843\nMinimizing 20479 Time: 0:02:21 ( 6.91 ms/it)\n objective: 13328.79601185149\nMinimizing 20494 Time: 0:02:21 ( 6.91 ms/it)\n objective: 13328.788488880979\nMinimizing 20509 Time: 0:02:21 ( 6.91 ms/it)\n objective: 13328.782002255859\nMinimizing 20524 Time: 0:02:21 ( 6.91 ms/it)\n objective: 13328.751616276677\nMinimizing 20539 Time: 0:02:21 ( 6.91 ms/it)\n objective: 13328.745351114805\nMinimizing 20554 Time: 0:02:22 ( 6.91 ms/it)\n objective: 13328.743074135215\nMinimizing 20569 Time: 0:02:22 ( 6.91 ms/it)\n objective: 13328.723902436861\nMinimizing 20584 Time: 0:02:22 ( 6.91 ms/it)\n objective: 13328.719389493199\nMinimizing 20598 Time: 0:02:22 ( 6.91 ms/it)\n objective: 13328.702692766936\nMinimizing 20611 Time: 0:02:22 ( 6.91 ms/it)\n objective: 13328.69678127634\nMinimizing 20624 Time: 0:02:22 ( 6.91 ms/it)\n objective: 13328.661492028768\nMinimizing 20637 Time: 0:02:22 ( 6.91 ms/it)\n objective: 13328.645504165099\nMinimizing 20651 Time: 0:02:22 ( 6.91 ms/it)\n objective: 13328.6131297545\nMinimizing 20665 Time: 0:02:22 ( 6.91 ms/it)\n objective: 13328.59599418984\nMinimizing 20679 Time: 0:02:22 ( 6.91 ms/it)\n objective: 13328.563284711126\nMinimizing 20693 Time: 0:02:23 ( 6.91 ms/it)\n objective: 13328.563004483498\nMinimizing 20709 Time: 0:02:23 ( 6.91 ms/it)\n objective: 13328.561069517382\nMinimizing 20725 Time: 0:02:23 ( 6.91 ms/it)\n objective: 13328.555234422049\nMinimizing 20741 Time: 0:02:23 ( 6.91 ms/it)\n objective: 13328.549043014864\nMinimizing 20757 Time: 0:02:23 ( 6.91 ms/it)\n objective: 13328.540497406568\nMinimizing 20773 Time: 0:02:23 ( 6.91 ms/it)\n objective: 13328.540005980147\nMinimizing 20788 Time: 0:02:23 ( 6.91 ms/it)\n objective: 13328.536864956477\nMinimizing 20804 Time: 0:02:23 ( 6.91 ms/it)\n objective: 13328.531331802318\nMinimizing 20820 Time: 0:02:23 ( 6.91 ms/it)\n objective: 13328.513318566002\nMinimizing 20836 Time: 0:02:24 ( 6.91 ms/it)\n objective: 13328.50008276588\nMinimizing 20852 Time: 0:02:24 ( 6.91 ms/it)\n objective: 13328.496851011798\nMinimizing 20868 Time: 0:02:24 ( 6.91 ms/it)\n objective: 13328.484168365874\nMinimizing 20884 Time: 0:02:24 ( 6.91 ms/it)\n objective: 13328.48124605757\nMinimizing 20900 Time: 0:02:24 ( 6.91 ms/it)\n objective: 13328.46839925462\nMinimizing 20916 Time: 0:02:24 ( 6.91 ms/it)\n objective: 13328.467590104665\nMinimizing 20931 Time: 0:02:24 ( 6.91 ms/it)\n objective: 13328.461319288479\nMinimizing 20947 Time: 0:02:24 ( 6.91 ms/it)\n objective: 13328.453933112985\nMinimizing 20963 Time: 0:02:24 ( 6.91 ms/it)\n objective: 13328.43419432555\nMinimizing 20979 Time: 0:02:24 ( 6.91 ms/it)\n objective: 13328.432500284369\nMinimizing 20995 Time: 0:02:25 ( 6.91 ms/it)\n objective: 13328.421113762604\nMinimizing 21011 Time: 0:02:25 ( 6.91 ms/it)\n objective: 13328.38416793786\nMinimizing 21027 Time: 0:02:25 ( 6.91 ms/it)\n objective: 13328.379851898833\nMinimizing 21043 Time: 0:02:25 ( 6.91 ms/it)\n objective: 13328.376751316799\nMinimizing 21059 Time: 0:02:25 ( 6.91 ms/it)\n objective: 13328.369806187402\nMinimizing 21075 Time: 0:02:25 ( 6.91 ms/it)\n objective: 13328.362042835848\nMinimizing 21090 Time: 0:02:25 ( 6.91 ms/it)\n objective: 13328.343381699531\nMinimizing 21105 Time: 0:02:25 ( 6.91 ms/it)\n objective: 13328.337823394271\nMinimizing 21120 Time: 0:02:25 ( 6.91 ms/it)\n objective: 13328.321451358046\nMinimizing 21136 Time: 0:02:26 ( 6.91 ms/it)\n objective: 13328.312059720367\nMinimizing 21152 Time: 0:02:26 ( 6.91 ms/it)\n objective: 13328.30609299626\nMinimizing 21168 Time: 0:02:26 ( 6.91 ms/it)\n objective: 13328.28875842451\nMinimizing 21184 Time: 0:02:26 ( 6.91 ms/it)\n objective: 13328.267972218964\nMinimizing 21200 Time: 0:02:26 ( 6.91 ms/it)\n objective: 13328.267036784477\nMinimizing 21216 Time: 0:02:26 ( 6.91 ms/it)\n objective: 13328.26497987088\nMinimizing 21231 Time: 0:02:26 ( 6.91 ms/it)\n objective: 13328.262630383411\nMinimizing 21247 Time: 0:02:26 ( 6.91 ms/it)\n objective: 13328.25573451454\nMinimizing 21263 Time: 0:02:26 ( 6.91 ms/it)\n objective: 13328.23286972697\nMinimizing 21279 Time: 0:02:26 ( 6.91 ms/it)\n objective: 13328.231315208788\nMinimizing 21295 Time: 0:02:27 ( 6.91 ms/it)\n objective: 13328.224954260731\nMinimizing 21311 Time: 0:02:27 ( 6.91 ms/it)\n objective: 13328.217138625194\nMinimizing 21327 Time: 0:02:27 ( 6.91 ms/it)\n objective: 13328.187494608996\nMinimizing 21343 Time: 0:02:27 ( 6.91 ms/it)\n objective: 13328.12351370039\nMinimizing 21359 Time: 0:02:27 ( 6.91 ms/it)\n objective: 13328.055052723168\nMinimizing 21375 Time: 0:02:27 ( 6.90 ms/it)\n objective: 13328.030370397268\nMinimizing 21390 Time: 0:02:27 ( 6.90 ms/it)\n objective: 13328.026212190089\nMinimizing 21406 Time: 0:02:27 ( 6.90 ms/it)\n objective: 13328.018676974956\nMinimizing 21422 Time: 0:02:27 ( 6.90 ms/it)\n objective: 13328.015866306698\nMinimizing 21438 Time: 0:02:28 ( 6.90 ms/it)\n objective: 13328.006198655297\nMinimizing 21454 Time: 0:02:28 ( 6.90 ms/it)\n objective: 13327.99892970141\nMinimizing 21470 Time: 0:02:28 ( 6.90 ms/it)\n objective: 13327.928966481551\nMinimizing 21486 Time: 0:02:28 ( 6.90 ms/it)\n objective: 13328.02470796887\nMinimizing 21501 Time: 0:02:28 ( 6.90 ms/it)\n objective: 13327.801979061704\nMinimizing 21517 Time: 0:02:28 ( 6.90 ms/it)\n objective: 13327.801556733619\nMinimizing 21532 Time: 0:02:28 ( 6.90 ms/it)\n objective: 13327.799584313805\nMinimizing 21547 Time: 0:02:28 ( 6.90 ms/it)\n objective: 13327.796538176422\nMinimizing 21562 Time: 0:02:28 ( 6.90 ms/it)\n objective: 13327.793278801822\nMinimizing 21577 Time: 0:02:28 ( 6.90 ms/it)\n objective: 13327.782332322291\nMinimizing 21592 Time: 0:02:29 ( 6.90 ms/it)\n objective: 13327.764839668736\nMinimizing 21608 Time: 0:02:29 ( 6.90 ms/it)\n objective: 13327.756326261602\nMinimizing 21624 Time: 0:02:29 ( 6.90 ms/it)\n objective: 13327.741892600308\nMinimizing 21639 Time: 0:02:29 ( 6.90 ms/it)\n objective: 13327.729415401547\nMinimizing 21655 Time: 0:02:29 ( 6.90 ms/it)\n objective: 13327.690183491577\nMinimizing 21669 Time: 0:02:29 ( 6.90 ms/it)\n objective: 13327.684329612544\nMinimizing 21684 Time: 0:02:29 ( 6.90 ms/it)\n objective: 13327.683795853838\nMinimizing 21699 Time: 0:02:29 ( 6.90 ms/it)\n objective: 13327.680709614477\nMinimizing 21714 Time: 0:02:29 ( 6.90 ms/it)\n objective: 13327.676415991162\nMinimizing 21729 Time: 0:02:29 ( 6.90 ms/it)\n objective: 13327.67130588476\nMinimizing 21744 Time: 0:02:30 ( 6.90 ms/it)\n objective: 13327.662230510286\nMinimizing 21759 Time: 0:02:30 ( 6.90 ms/it)\n objective: 13327.603528149026\nMinimizing 21774 Time: 0:02:30 ( 6.90 ms/it)\n objective: 13329.111544503598\nMinimizing 21789 Time: 0:02:30 ( 6.90 ms/it)\n objective: 13327.484090964128\nMinimizing 21804 Time: 0:02:30 ( 6.90 ms/it)\n objective: 13327.482741733402\nMinimizing 21818 Time: 0:02:30 ( 6.90 ms/it)\n objective: 13327.47717121819\nMinimizing 21833 Time: 0:02:30 ( 6.90 ms/it)\n objective: 13327.467571572735\nMinimizing 21848 Time: 0:02:30 ( 6.90 ms/it)\n objective: 13327.4628520002\nMinimizing 21863 Time: 0:02:30 ( 6.90 ms/it)\n objective: 13327.452499485793\nMinimizing 21878 Time: 0:02:31 ( 6.90 ms/it)\n objective: 13327.446835663191\nMinimizing 21893 Time: 0:02:31 ( 6.90 ms/it)\n objective: 13327.445871786542\nMinimizing 21908 Time: 0:02:31 ( 6.90 ms/it)\n objective: 13327.434015690771\nMinimizing 21924 Time: 0:02:31 ( 6.90 ms/it)\n objective: 13327.443386763123\nMinimizing 21939 Time: 0:02:31 ( 6.90 ms/it)\n objective: 13327.423161290528\nMinimizing 21954 Time: 0:02:31 ( 6.90 ms/it)\n objective: 13327.414213814496\nMinimizing 21969 Time: 0:02:31 ( 6.90 ms/it)\n objective: 13327.38938667193\nMinimizing 21984 Time: 0:02:31 ( 6.90 ms/it)\n objective: 13327.32583775944\nMinimizing 21999 Time: 0:02:31 ( 6.90 ms/it)\n objective: 13327.262107619375\nMinimizing 22014 Time: 0:02:31 ( 6.90 ms/it)\n objective: 13327.259904104038\nMinimizing 22029 Time: 0:02:32 ( 6.90 ms/it)\n objective: 13327.25594936443\nMinimizing 22044 Time: 0:02:32 ( 6.90 ms/it)\n objective: 13327.23970314509\nMinimizing 22059 Time: 0:02:32 ( 6.90 ms/it)\n objective: 13327.232849535932\nMinimizing 22074 Time: 0:02:32 ( 6.90 ms/it)\n objective: 13327.223380963369\nMinimizing 22089 Time: 0:02:32 ( 6.90 ms/it)\n objective: 13327.222790139938\nMinimizing 22104 Time: 0:02:32 ( 6.90 ms/it)\n objective: 13327.205473040536\nMinimizing 22119 Time: 0:02:32 ( 6.90 ms/it)\n objective: 13327.199450099346\nMinimizing 22134 Time: 0:02:32 ( 6.90 ms/it)\n objective: 13327.190917415996\nMinimizing 22150 Time: 0:02:32 ( 6.90 ms/it)\n objective: 13327.122476330384\nMinimizing 22165 Time: 0:02:32 ( 6.90 ms/it)\n objective: 13328.560754599028\nMinimizing 22180 Time: 0:02:33 ( 6.90 ms/it)\n objective: 13326.982377612716\nMinimizing 22195 Time: 0:02:33 ( 6.90 ms/it)\n objective: 13326.980520266443\nMinimizing 22210 Time: 0:02:33 ( 6.90 ms/it)\n objective: 13326.976321790004\nMinimizing 22226 Time: 0:02:33 ( 6.90 ms/it)\n objective: 13326.973612000358\nMinimizing 22241 Time: 0:02:33 ( 6.90 ms/it)\n objective: 13326.973099495845\nMinimizing 22256 Time: 0:02:33 ( 6.90 ms/it)\n objective: 13326.966781852469\nMinimizing 22271 Time: 0:02:33 ( 6.90 ms/it)\n objective: 13326.960174615451\nMinimizing 22286 Time: 0:02:33 ( 6.90 ms/it)\n objective: 13326.957446704444\nMinimizing 22301 Time: 0:02:33 ( 6.90 ms/it)\n objective: 13326.903577389181\nMinimizing 22317 Time: 0:02:34 ( 6.90 ms/it)\n objective: 13326.81188356712\nMinimizing 22332 Time: 0:02:34 ( 6.90 ms/it)\n objective: 13326.80533651678\nMinimizing 22347 Time: 0:02:34 ( 6.90 ms/it)\n objective: 13326.80114071528\nMinimizing 22362 Time: 0:02:34 ( 6.90 ms/it)\n objective: 13326.79290784846\nMinimizing 22378 Time: 0:02:34 ( 6.90 ms/it)\n objective: 13326.786212197694\nMinimizing 22393 Time: 0:02:34 ( 6.90 ms/it)\n objective: 13326.78560582918\nMinimizing 22408 Time: 0:02:34 ( 6.90 ms/it)\n objective: 13326.78352692166\nMinimizing 22423 Time: 0:02:34 ( 6.90 ms/it)\n objective: 13326.771890382966\nMinimizing 22439 Time: 0:02:34 ( 6.90 ms/it)\n objective: 13326.766326823505\nMinimizing 22455 Time: 0:02:34 ( 6.90 ms/it)\n objective: 13326.72886697483\nMinimizing 22470 Time: 0:02:35 ( 6.90 ms/it)\n objective: 13326.691905235355\nMinimizing 22486 Time: 0:02:35 ( 6.90 ms/it)\n objective: 13326.691417169044\nMinimizing 22501 Time: 0:02:35 ( 6.90 ms/it)\n objective: 13326.689990353269\nMinimizing 22516 Time: 0:02:35 ( 6.90 ms/it)\n objective: 13326.688596868997\nMinimizing 22531 Time: 0:02:35 ( 6.90 ms/it)\n objective: 13326.68589525792\nMinimizing 22546 Time: 0:02:35 ( 6.90 ms/it)\n objective: 13326.679548932058\nMinimizing 22561 Time: 0:02:35 ( 6.90 ms/it)\n objective: 13326.665983564177\nMinimizing 22576 Time: 0:02:35 ( 6.90 ms/it)\n objective: 13326.629279067965\nMinimizing 22592 Time: 0:02:35 ( 6.90 ms/it)\n objective: 13326.628672901024\nMinimizing 22608 Time: 0:02:35 ( 6.90 ms/it)\n objective: 13326.615305702268\nMinimizing 22624 Time: 0:02:36 ( 6.90 ms/it)\n objective: 13326.605019161463\nMinimizing 22640 Time: 0:02:36 ( 6.90 ms/it)\n objective: 13326.598885612599\nMinimizing 22656 Time: 0:02:36 ( 6.90 ms/it)\n objective: 13326.586264002006\nMinimizing 22672 Time: 0:02:36 ( 6.90 ms/it)\n objective: 13326.585417005328\nMinimizing 22688 Time: 0:02:36 ( 6.90 ms/it)\n objective: 13326.5773219443\nMinimizing 22704 Time: 0:02:36 ( 6.90 ms/it)\n objective: 13326.564808780757\nMinimizing 22720 Time: 0:02:36 ( 6.90 ms/it)\n objective: 13326.539768572613\nMinimizing 22736 Time: 0:02:36 ( 6.90 ms/it)\n objective: 13326.494433528118\nMinimizing 22752 Time: 0:02:36 ( 6.90 ms/it)\n objective: 13326.49289796767\nMinimizing 22768 Time: 0:02:37 ( 6.90 ms/it)\n objective: 13326.479554432073\nMinimizing 22784 Time: 0:02:37 ( 6.90 ms/it)\n objective: 13326.473930317916\nMinimizing 22800 Time: 0:02:37 ( 6.90 ms/it)\n objective: 13326.47062744174\nMinimizing 22816 Time: 0:02:37 ( 6.90 ms/it)\n objective: 13326.429113892009\nMinimizing 22832 Time: 0:02:37 ( 6.89 ms/it)\n objective: 13326.389299300594\nMinimizing 22848 Time: 0:02:37 ( 6.89 ms/it)\n objective: 13326.388486062191\nMinimizing 22864 Time: 0:02:37 ( 6.89 ms/it)\n objective: 13326.382510892407\nMinimizing 22880 Time: 0:02:37 ( 6.89 ms/it)\n objective: 13326.360334032463\nMinimizing 22896 Time: 0:02:37 ( 6.89 ms/it)\n objective: 13326.251126782692\nMinimizing 22912 Time: 0:02:37 ( 6.89 ms/it)\n objective: 13326.201596900799\nMinimizing 22928 Time: 0:02:38 ( 6.89 ms/it)\n objective: 13326.200756263825\nMinimizing 22944 Time: 0:02:38 ( 6.89 ms/it)\n objective: 13326.186645147784\nMinimizing 22960 Time: 0:02:38 ( 6.89 ms/it)\n objective: 13326.16572093666\nMinimizing 22976 Time: 0:02:38 ( 6.89 ms/it)\n objective: 13326.13902118634\nMinimizing 22992 Time: 0:02:38 ( 6.89 ms/it)\n objective: 13325.999315058587\nMinimizing 23008 Time: 0:02:38 ( 6.89 ms/it)\n objective: 13325.975564473425\nMinimizing 23024 Time: 0:02:38 ( 6.89 ms/it)\n objective: 13325.972831541738\nMinimizing 23040 Time: 0:02:38 ( 6.89 ms/it)\n objective: 13325.967454374375\nMinimizing 23056 Time: 0:02:38 ( 6.89 ms/it)\n objective: 13325.963359604662\nMinimizing 23072 Time: 0:02:38 ( 6.89 ms/it)\n objective: 13325.95130167601\nMinimizing 23088 Time: 0:02:39 ( 6.89 ms/it)\n objective: 13325.943465455792\nMinimizing 23104 Time: 0:02:39 ( 6.89 ms/it)\n objective: 13325.933078365735\nMinimizing 23120 Time: 0:02:39 ( 6.89 ms/it)\n objective: 13325.919712834038\nMinimizing 23136 Time: 0:02:39 ( 6.89 ms/it)\n objective: 13325.899783913053\nMinimizing 23152 Time: 0:02:39 ( 6.89 ms/it)\n objective: 13325.898596879335\nMinimizing 23167 Time: 0:02:39 ( 6.89 ms/it)\n objective: 13325.894608529423\nMinimizing 23181 Time: 0:02:39 ( 6.89 ms/it)\n objective: 13325.891204203785\nMinimizing 23195 Time: 0:02:39 ( 6.89 ms/it)\n objective: 13325.889488856781\nMinimizing 23209 Time: 0:02:39 ( 6.89 ms/it)\n objective: 13325.883675640216\nMinimizing 23224 Time: 0:02:40 ( 6.89 ms/it)\n objective: 13325.866046063544\nMinimizing 23238 Time: 0:02:40 ( 6.89 ms/it)\n objective: 13325.85696938257\nMinimizing 23252 Time: 0:02:40 ( 6.89 ms/it)\n objective: 13325.84344341918\nMinimizing 23266 Time: 0:02:40 ( 6.89 ms/it)\n objective: 13325.834944784154\nMinimizing 23280 Time: 0:02:40 ( 6.89 ms/it)\n objective: 13325.816188973069\nMinimizing 23294 Time: 0:02:40 ( 6.89 ms/it)\n objective: 13325.815294583466\nMinimizing 23308 Time: 0:02:40 ( 6.89 ms/it)\n objective: 13325.801841052642\nMinimizing 23322 Time: 0:02:40 ( 6.89 ms/it)\n objective: 13325.778420452581\nMinimizing 23336 Time: 0:02:40 ( 6.89 ms/it)\n objective: 13325.720044661815\nMinimizing 23350 Time: 0:02:40 ( 6.89 ms/it)\n objective: 13325.710355581548\nMinimizing 23364 Time: 0:02:41 ( 6.89 ms/it)\n objective: 13325.709689893927\nMinimizing 23379 Time: 0:02:41 ( 6.89 ms/it)\n objective: 13325.708151664323\nMinimizing 23394 Time: 0:02:41 ( 6.89 ms/it)\n objective: 13325.703164592858\nMinimizing 23409 Time: 0:02:41 ( 6.89 ms/it)\n objective: 13325.701900485277\nMinimizing 23425 Time: 0:02:41 ( 6.89 ms/it)\n objective: 13325.693808845848\nMinimizing 23441 Time: 0:02:41 ( 6.89 ms/it)\n objective: 13325.68981808627\nMinimizing 23457 Time: 0:02:41 ( 6.89 ms/it)\n objective: 13325.671538165429\nMinimizing 23473 Time: 0:02:41 ( 6.89 ms/it)\n objective: 13325.664247808396\nMinimizing 23489 Time: 0:02:41 ( 6.89 ms/it)\n objective: 13325.651794773388\nMinimizing 23505 Time: 0:02:41 ( 6.89 ms/it)\n objective: 13325.642637510537\nMinimizing 23521 Time: 0:02:42 ( 6.89 ms/it)\n objective: 13325.619599476951\nMinimizing 23537 Time: 0:02:42 ( 6.89 ms/it)\n objective: 13325.616775359442\nMinimizing 23552 Time: 0:02:42 ( 6.89 ms/it)\n objective: 13325.612685194785\nMinimizing 23568 Time: 0:02:42 ( 6.89 ms/it)\n objective: 13325.608067382876\nMinimizing 23584 Time: 0:02:42 ( 6.89 ms/it)\n objective: 13325.595928968884\nMinimizing 23600 Time: 0:02:42 ( 6.89 ms/it)\n objective: 13325.591879340587\nMinimizing 23616 Time: 0:02:42 ( 6.89 ms/it)\n objective: 13325.573347906087\nMinimizing 23632 Time: 0:02:42 ( 6.89 ms/it)\n objective: 13325.569694765603\nMinimizing 23648 Time: 0:02:42 ( 6.89 ms/it)\n objective: 13325.538788847232\nMinimizing 23664 Time: 0:02:43 ( 6.89 ms/it)\n objective: 13325.52459089207\nMinimizing 23680 Time: 0:02:43 ( 6.89 ms/it)\n objective: 13326.438067200346\nMinimizing 23696 Time: 0:02:43 ( 6.89 ms/it)\n objective: 13325.374923427014\nMinimizing 23712 Time: 0:02:43 ( 6.89 ms/it)\n objective: 13325.373184006072\nMinimizing 23728 Time: 0:02:43 ( 6.89 ms/it)\n objective: 13325.368453568684\nMinimizing 23744 Time: 0:02:43 ( 6.89 ms/it)\n objective: 13325.353329633705\nMinimizing 23760 Time: 0:02:43 ( 6.89 ms/it)\n objective: 13325.347739125566\nMinimizing 23776 Time: 0:02:43 ( 6.89 ms/it)\n objective: 13325.347940115164\nMinimizing 23791 Time: 0:02:43 ( 6.89 ms/it)\n objective: 13325.32932654817\nMinimizing 23807 Time: 0:02:43 ( 6.89 ms/it)\n objective: 13325.325268700886\nMinimizing 23823 Time: 0:02:44 ( 6.89 ms/it)\n objective: 13325.242215933562\nMinimizing 23839 Time: 0:02:44 ( 6.89 ms/it)\n objective: 13325.172006022054\nMinimizing 23854 Time: 0:02:44 ( 6.89 ms/it)\n objective: 13325.171226626553\nMinimizing 23868 Time: 0:02:44 ( 6.89 ms/it)\n objective: 13325.156228543521\nMinimizing 23882 Time: 0:02:44 ( 6.89 ms/it)\n objective: 13325.128125648043\nMinimizing 23896 Time: 0:02:44 ( 6.89 ms/it)\n objective: 13325.10150062821\nMinimizing 23910 Time: 0:02:44 ( 6.89 ms/it)\n objective: 13325.096286907108\nMinimizing 23924 Time: 0:02:44 ( 6.89 ms/it)\n objective: 13325.095722644866\nMinimizing 23938 Time: 0:02:44 ( 6.89 ms/it)\n objective: 13325.09481461077\nMinimizing 23954 Time: 0:02:44 ( 6.89 ms/it)\n objective: 13325.094347071587\nMinimizing 23970 Time: 0:02:45 ( 6.89 ms/it)\n objective: 13325.089907856367\nMinimizing 23986 Time: 0:02:45 ( 6.89 ms/it)\n objective: 13325.087706015067\nMinimizing 24002 Time: 0:02:45 ( 6.89 ms/it)\n objective: 13325.073322889613\nMinimizing 24018 Time: 0:02:45 ( 6.89 ms/it)\n objective: 13325.071698878019\nMinimizing 24034 Time: 0:02:45 ( 6.89 ms/it)\n objective: 13325.069605354234\nMinimizing 24050 Time: 0:02:45 ( 6.89 ms/it)\n objective: 13325.06731050594\nMinimizing 24066 Time: 0:02:45 ( 6.89 ms/it)\n objective: 13325.031029600403\nMinimizing 24082 Time: 0:02:45 ( 6.89 ms/it)\n objective: 13324.928921728992\nMinimizing 24098 Time: 0:02:45 ( 6.89 ms/it)\n objective: 13324.903449190679\nMinimizing 24114 Time: 0:02:46 ( 6.88 ms/it)\n objective: 13324.902986358444\nMinimizing 24130 Time: 0:02:46 ( 6.88 ms/it)\n objective: 13324.902310571691\nMinimizing 24146 Time: 0:02:46 ( 6.88 ms/it)\n objective: 13324.9020104558\nMinimizing 24162 Time: 0:02:46 ( 6.88 ms/it)\n objective: 13324.900693161544\nMinimizing 24178 Time: 0:02:46 ( 6.88 ms/it)\n objective: 13324.900124771419\nMinimizing 24194 Time: 0:02:46 ( 6.88 ms/it)\n objective: 13324.888367541382\nMinimizing 24210 Time: 0:02:46 ( 6.88 ms/it)\n objective: 13324.88766143292\nMinimizing 24226 Time: 0:02:46 ( 6.88 ms/it)\n objective: 13324.886939615433\nMinimizing 24241 Time: 0:02:46 ( 6.88 ms/it)\n objective: 13324.884405500881\nMinimizing 24255 Time: 0:02:46 ( 6.88 ms/it)\n objective: 13324.882491166427\nMinimizing 24271 Time: 0:02:47 ( 6.88 ms/it)\n objective: 13324.873052820709\nMinimizing 24287 Time: 0:02:47 ( 6.88 ms/it)\n objective: 13324.872226935026\nMinimizing 24303 Time: 0:02:47 ( 6.88 ms/it)\n objective: 13324.86797736722\nMinimizing 24319 Time: 0:02:47 ( 6.88 ms/it)\n objective: 13324.84477803143\nMinimizing 24335 Time: 0:02:47 ( 6.88 ms/it)\n objective: 13324.841977485383\nMinimizing 24351 Time: 0:02:47 ( 6.88 ms/it)\n objective: 13324.841054861448\nMinimizing 24367 Time: 0:02:47 ( 6.88 ms/it)\n objective: 13324.839558815904\nMinimizing 24382 Time: 0:02:47 ( 6.88 ms/it)\n objective: 13324.832945818256\nMinimizing 24398 Time: 0:02:47 ( 6.88 ms/it)\n objective: 13324.829589407134\nMinimizing 24414 Time: 0:02:48 ( 6.88 ms/it)\n objective: 13324.825581579164\nMinimizing 24430 Time: 0:02:48 ( 6.88 ms/it)\n objective: 13324.817998542305\nMinimizing 24446 Time: 0:02:48 ( 6.88 ms/it)\n objective: 13324.816962923098\nMinimizing 24462 Time: 0:02:48 ( 6.88 ms/it)\n objective: 13324.803258516695\nMinimizing 24478 Time: 0:02:48 ( 6.88 ms/it)\n objective: 13324.80214209207\nMinimizing 24494 Time: 0:02:48 ( 6.88 ms/it)\n objective: 13324.797411330379\nMinimizing 24510 Time: 0:02:48 ( 6.88 ms/it)\n objective: 13324.790132120514\nMinimizing 24526 Time: 0:02:48 ( 6.88 ms/it)\n objective: 13324.766449680392\nMinimizing 24542 Time: 0:02:48 ( 6.88 ms/it)\n objective: 13324.76292447331\nMinimizing 24558 Time: 0:02:48 ( 6.88 ms/it)\n objective: 13324.678182131509\nMinimizing 24574 Time: 0:02:49 ( 6.88 ms/it)\n objective: 13324.16256173722\nMinimizing 24590 Time: 0:02:49 ( 6.88 ms/it)\n objective: 13324.151142159579\nMinimizing 24606 Time: 0:02:49 ( 6.88 ms/it)\n objective: 13324.147462384019\nMinimizing 24622 Time: 0:02:49 ( 6.88 ms/it)\n objective: 13324.143824523184\nMinimizing 24638 Time: 0:02:49 ( 6.88 ms/it)\n objective: 13324.13157006503\nMinimizing 24654 Time: 0:02:49 ( 6.88 ms/it)\n objective: 13324.127718992924\nMinimizing 24670 Time: 0:02:49 ( 6.88 ms/it)\n objective: 13324.108382862512\nMinimizing 24686 Time: 0:02:49 ( 6.88 ms/it)\n objective: 13324.03378389457\nMinimizing 24702 Time: 0:02:49 ( 6.88 ms/it)\n objective: 13324.013521233443\nMinimizing 24718 Time: 0:02:49 ( 6.88 ms/it)\n objective: 13324.012666784824\nMinimizing 24733 Time: 0:02:50 ( 6.88 ms/it)\n objective: 13324.000040468163\nMinimizing 24749 Time: 0:02:50 ( 6.88 ms/it)\n objective: 13323.968423096347\nMinimizing 24765 Time: 0:02:50 ( 6.88 ms/it)\n objective: 13323.953850897597\nMinimizing 24781 Time: 0:02:50 ( 6.88 ms/it)\n objective: 13323.94573838498\nMinimizing 24797 Time: 0:02:50 ( 6.88 ms/it)\n objective: 13323.931988113167\nMinimizing 24813 Time: 0:02:50 ( 6.88 ms/it)\n objective: 13323.9140349089\nMinimizing 24829 Time: 0:02:50 ( 6.88 ms/it)\n objective: 13323.908955066028\nMinimizing 24845 Time: 0:02:50 ( 6.88 ms/it)\n objective: 13323.89419766872\nMinimizing 24861 Time: 0:02:50 ( 6.88 ms/it)\n objective: 13323.852324141655\nMinimizing 24877 Time: 0:02:51 ( 6.88 ms/it)\n objective: 13323.848371638538\nMinimizing 24893 Time: 0:02:51 ( 6.87 ms/it)\n objective: 13323.840699705455\nMinimizing 24909 Time: 0:02:51 ( 6.87 ms/it)\n objective: 13323.81877841182\nMinimizing 24924 Time: 0:02:51 ( 6.87 ms/it)\n objective: 13323.817457392302\nMinimizing 24940 Time: 0:02:51 ( 6.87 ms/it)\n objective: 13323.837986796425\nMinimizing 24956 Time: 0:02:51 ( 6.87 ms/it)\n objective: 13323.79314238258\nMinimizing 24972 Time: 0:02:51 ( 6.87 ms/it)\n objective: 13323.786818467648\nMinimizing 24988 Time: 0:02:51 ( 6.87 ms/it)\n objective: 13323.766236201918\nMinimizing 25004 Time: 0:02:51 ( 6.87 ms/it)\n objective: 13323.750464486337\nMinimizing 25020 Time: 0:02:51 ( 6.87 ms/it)\n objective: 13323.747747567686\nMinimizing 25036 Time: 0:02:52 ( 6.87 ms/it)\n objective: 13323.743292526895\nMinimizing 25052 Time: 0:02:52 ( 6.87 ms/it)\n objective: 13323.713804753104\nMinimizing 25068 Time: 0:02:52 ( 6.87 ms/it)\n objective: 13323.712822744434\nMinimizing 25084 Time: 0:02:52 ( 6.87 ms/it)\n objective: 13323.70937624514\nMinimizing 25100 Time: 0:02:52 ( 6.87 ms/it)\n objective: 13323.704561479113\nMinimizing 25116 Time: 0:02:52 ( 6.87 ms/it)\n objective: 13323.691608221881\nMinimizing 25132 Time: 0:02:52 ( 6.87 ms/it)\n objective: 13323.67334602181\nMinimizing 25148 Time: 0:02:52 ( 6.87 ms/it)\n objective: 13323.659798414825\nMinimizing 25164 Time: 0:02:52 ( 6.87 ms/it)\n objective: 13323.65746454343\nMinimizing 25180 Time: 0:02:53 ( 6.87 ms/it)\n objective: 13323.640948931832\nMinimizing 25196 Time: 0:02:53 ( 6.87 ms/it)\n objective: 13323.638226269642\nMinimizing 25212 Time: 0:02:53 ( 6.87 ms/it)\n objective: 13323.628734145575\nMinimizing 25228 Time: 0:02:53 ( 6.87 ms/it)\n objective: 13323.597906984185\nMinimizing 25244 Time: 0:02:53 ( 6.87 ms/it)\n objective: 13323.587541658359\nMinimizing 25260 Time: 0:02:53 ( 6.87 ms/it)\n objective: 13323.554590172716\nMinimizing 25276 Time: 0:02:53 ( 6.87 ms/it)\n objective: 13323.434175322996\nMinimizing 25292 Time: 0:02:53 ( 6.87 ms/it)\n objective: 13323.433494755533\nMinimizing 25307 Time: 0:02:53 ( 6.87 ms/it)\n objective: 13323.428571804805\nMinimizing 25323 Time: 0:02:53 ( 6.87 ms/it)\n objective: 13323.422066582134\nMinimizing 25339 Time: 0:02:54 ( 6.87 ms/it)\n objective: 13323.413393725787\nMinimizing 25354 Time: 0:02:54 ( 6.87 ms/it)\n objective: 13323.407697837014\nMinimizing 25369 Time: 0:02:54 ( 6.87 ms/it)\n objective: 13323.407231690624\nMinimizing 25384 Time: 0:02:54 ( 6.87 ms/it)\n objective: 13323.4066585603\nMinimizing 25399 Time: 0:02:54 ( 6.87 ms/it)\n objective: 13323.40371533185\nMinimizing 25414 Time: 0:02:54 ( 6.87 ms/it)\n objective: 13323.39458685054\nMinimizing 25429 Time: 0:02:54 ( 6.87 ms/it)\n objective: 13323.393447367955\nMinimizing 25444 Time: 0:02:54 ( 6.87 ms/it)\n objective: 13323.38865229694\nMinimizing 25459 Time: 0:02:54 ( 6.87 ms/it)\n objective: 13323.377278377913\nMinimizing 25474 Time: 0:02:54 ( 6.87 ms/it)\n objective: 13323.372870266161\nMinimizing 25489 Time: 0:02:55 ( 6.87 ms/it)\n objective: 13323.338608801394\nMinimizing 25504 Time: 0:02:55 ( 6.87 ms/it)\n objective: 13323.331808491392\nMinimizing 25519 Time: 0:02:55 ( 6.87 ms/it)\n objective: 13323.32543553997\nMinimizing 25534 Time: 0:02:55 ( 6.87 ms/it)\n objective: 13323.308513585755\nMinimizing 25549 Time: 0:02:55 ( 6.87 ms/it)\n objective: 13323.304795191798\nMinimizing 25564 Time: 0:02:55 ( 6.87 ms/it)\n objective: 13323.301294638586\nMinimizing 25579 Time: 0:02:55 ( 6.87 ms/it)\n objective: 13323.290065557652\nMinimizing 25594 Time: 0:02:55 ( 6.87 ms/it)\n objective: 13323.256049413176\nMinimizing 25609 Time: 0:02:55 ( 6.87 ms/it)\n objective: 13323.253375282322\nMinimizing 25624 Time: 0:02:55 ( 6.87 ms/it)\n objective: 13323.252761078387\nMinimizing 25639 Time: 0:02:56 ( 6.87 ms/it)\n objective: 13323.2490163912\nMinimizing 25654 Time: 0:02:56 ( 6.87 ms/it)\n objective: 13323.238773807418\nMinimizing 25669 Time: 0:02:56 ( 6.87 ms/it)\n objective: 13323.226849554223\nMinimizing 25684 Time: 0:02:56 ( 6.87 ms/it)\n objective: 13323.20930974609\nMinimizing 25699 Time: 0:02:56 ( 6.87 ms/it)\n objective: 13323.2056473704\nMinimizing 25714 Time: 0:02:56 ( 6.87 ms/it)\n objective: 13323.178228027202\nMinimizing 25729 Time: 0:02:56 ( 6.87 ms/it)\n objective: 13323.13293205484\nMinimizing 25744 Time: 0:02:56 ( 6.87 ms/it)\n objective: 13323.075127984703\nMinimizing 25759 Time: 0:02:56 ( 6.87 ms/it)\n objective: 13323.074458831194\nMinimizing 25774 Time: 0:02:57 ( 6.87 ms/it)\n objective: 13323.068142148768\nMinimizing 25789 Time: 0:02:57 ( 6.87 ms/it)\n objective: 13323.057414702387\nMinimizing 25804 Time: 0:02:57 ( 6.87 ms/it)\n objective: 13323.045625411774\nMinimizing 25819 Time: 0:02:57 ( 6.87 ms/it)\n objective: 13323.03783585751\nMinimizing 25834 Time: 0:02:57 ( 6.87 ms/it)\n objective: 13323.018657660461\nMinimizing 25849 Time: 0:02:57 ( 6.87 ms/it)\n objective: 13323.003403881725\nMinimizing 25864 Time: 0:02:57 ( 6.87 ms/it)\n objective: 13322.99810934886\nMinimizing 25879 Time: 0:02:57 ( 6.87 ms/it)\n objective: 13322.9729513809\nMinimizing 25894 Time: 0:02:57 ( 6.87 ms/it)\n objective: 13322.938968131013\nMinimizing 25909 Time: 0:02:57 ( 6.87 ms/it)\n objective: 13322.93806398478\nMinimizing 25924 Time: 0:02:58 ( 6.87 ms/it)\n objective: 13322.929529983026\nMinimizing 25939 Time: 0:02:58 ( 6.87 ms/it)\n objective: 13322.918040594188\nMinimizing 25954 Time: 0:02:58 ( 6.87 ms/it)\n objective: 13322.898878752792\nMinimizing 25969 Time: 0:02:58 ( 6.87 ms/it)\n objective: 13322.888402011275\nMinimizing 25984 Time: 0:02:58 ( 6.87 ms/it)\n objective: 13322.872896282599\nMinimizing 25999 Time: 0:02:58 ( 6.87 ms/it)\n objective: 13322.830113556178\nMinimizing 26014 Time: 0:02:58 ( 6.87 ms/it)\n objective: 13322.829677535032\nMinimizing 26029 Time: 0:02:58 ( 6.87 ms/it)\n objective: 13322.828304435447\nMinimizing 26044 Time: 0:02:58 ( 6.87 ms/it)\n objective: 13322.822400007892\nMinimizing 26059 Time: 0:02:58 ( 6.87 ms/it)\n objective: 13322.817119775675\nMinimizing 26074 Time: 0:02:59 ( 6.87 ms/it)\n objective: 13322.807389417154\nMinimizing 26089 Time: 0:02:59 ( 6.87 ms/it)\n objective: 13322.803204736949\nMinimizing 26104 Time: 0:02:59 ( 6.87 ms/it)\n objective: 13322.785861942204\nMinimizing 26119 Time: 0:02:59 ( 6.87 ms/it)\n objective: 13322.692722944514\nMinimizing 26134 Time: 0:02:59 ( 6.87 ms/it)\n objective: 13322.609752601988\nMinimizing 26149 Time: 0:02:59 ( 6.87 ms/it)\n objective: 13322.603516203802\nMinimizing 26164 Time: 0:02:59 ( 6.87 ms/it)\n objective: 13322.591255335486\nMinimizing 26179 Time: 0:02:59 ( 6.87 ms/it)\n objective: 13322.586202882914\nMinimizing 26194 Time: 0:02:59 ( 6.87 ms/it)\n objective: 13322.594188662246\nMinimizing 26209 Time: 0:02:59 ( 6.87 ms/it)\n objective: 13322.565544025274\nMinimizing 26224 Time: 0:03:00 ( 6.87 ms/it)\n objective: 13322.55949812493\nMinimizing 26239 Time: 0:03:00 ( 6.87 ms/it)\n objective: 13322.406509473818\nMinimizing 26254 Time: 0:03:00 ( 6.87 ms/it)\n objective: 13322.395590189743\nMinimizing 26269 Time: 0:03:00 ( 6.87 ms/it)\n objective: 13322.393859894291\nMinimizing 26284 Time: 0:03:00 ( 6.87 ms/it)\n objective: 13322.385943464134\nMinimizing 26299 Time: 0:03:00 ( 6.87 ms/it)\n objective: 13322.369477663582\nMinimizing 26314 Time: 0:03:00 ( 6.87 ms/it)\n objective: 13322.34905602473\nMinimizing 26329 Time: 0:03:00 ( 6.87 ms/it)\n objective: 13322.333796735838\nMinimizing 26344 Time: 0:03:00 ( 6.87 ms/it)\n objective: 13322.332139578954\nMinimizing 26359 Time: 0:03:01 ( 6.87 ms/it)\n objective: 13322.331663799996\nMinimizing 26374 Time: 0:03:01 ( 6.87 ms/it)\n objective: 13322.329607107575\nMinimizing 26389 Time: 0:03:01 ( 6.87 ms/it)\n objective: 13322.32342997234\nMinimizing 26404 Time: 0:03:01 ( 6.87 ms/it)\n objective: 13322.319395621336\nMinimizing 26419 Time: 0:03:01 ( 6.87 ms/it)\n objective: 13322.301532050958\nMinimizing 26434 Time: 0:03:01 ( 6.87 ms/it)\n objective: 13322.30090986214\nMinimizing 26449 Time: 0:03:01 ( 6.87 ms/it)\n objective: 13322.300128561488\nMinimizing 26464 Time: 0:03:01 ( 6.87 ms/it)\n objective: 13322.298473328701\nMinimizing 26479 Time: 0:03:01 ( 6.87 ms/it)\n objective: 13322.293163996146\nMinimizing 26494 Time: 0:03:01 ( 6.87 ms/it)\n objective: 13322.290279489287\nMinimizing 26509 Time: 0:03:02 ( 6.87 ms/it)\n objective: 13322.285388918506\nMinimizing 26524 Time: 0:03:02 ( 6.87 ms/it)\n objective: 13322.282342203762\nMinimizing 26539 Time: 0:03:02 ( 6.87 ms/it)\n objective: 13322.255137888918\nMinimizing 26554 Time: 0:03:02 ( 6.87 ms/it)\n objective: 13322.233192796892\nMinimizing 26569 Time: 0:03:02 ( 6.87 ms/it)\n objective: 13322.209166855464\nMinimizing 26584 Time: 0:03:02 ( 6.87 ms/it)\n objective: 13322.207201735087\nMinimizing 26599 Time: 0:03:02 ( 6.87 ms/it)\n objective: 13322.203106752553\nMinimizing 26614 Time: 0:03:02 ( 6.87 ms/it)\n objective: 13322.196700367946\nMinimizing 26629 Time: 0:03:02 ( 6.87 ms/it)\n objective: 13322.195682090081\nMinimizing 26644 Time: 0:03:02 ( 6.87 ms/it)\n objective: 13322.181781145337\nMinimizing 26659 Time: 0:03:03 ( 6.87 ms/it)\n objective: 13322.17987567914\nMinimizing 26674 Time: 0:03:03 ( 6.87 ms/it)\n objective: 13322.173563138465\nMinimizing 26689 Time: 0:03:03 ( 6.87 ms/it)\n objective: 13322.147485170193\nMinimizing 26704 Time: 0:03:03 ( 6.87 ms/it)\n objective: 13322.141737244005\nMinimizing 26719 Time: 0:03:03 ( 6.87 ms/it)\n objective: 13322.141230367415\nMinimizing 26734 Time: 0:03:03 ( 6.87 ms/it)\n objective: 13322.140126766579\nMinimizing 26749 Time: 0:03:03 ( 6.87 ms/it)\n objective: 13322.138709305771\nMinimizing 26764 Time: 0:03:03 ( 6.87 ms/it)\n objective: 13322.138082907855\nMinimizing 26779 Time: 0:03:03 ( 6.87 ms/it)\n objective: 13322.132185935756\nMinimizing 26794 Time: 0:03:03 ( 6.87 ms/it)\n objective: 13322.120911144331\nMinimizing 26809 Time: 0:03:04 ( 6.87 ms/it)\n objective: 13322.113991364298\nMinimizing 26824 Time: 0:03:04 ( 6.87 ms/it)\n objective: 13322.113387134857\nMinimizing 26839 Time: 0:03:04 ( 6.87 ms/it)\n objective: 13322.11046375033\nMinimizing 26854 Time: 0:03:04 ( 6.87 ms/it)\n objective: 13322.099336957108\nMinimizing 26869 Time: 0:03:04 ( 6.87 ms/it)\n objective: 13322.084524258113\nMinimizing 26884 Time: 0:03:04 ( 6.87 ms/it)\n objective: 13322.063839652139\nMinimizing 26899 Time: 0:03:04 ( 6.87 ms/it)\n objective: 13322.0155499369\nMinimizing 26914 Time: 0:03:04 ( 6.87 ms/it)\n objective: 13322.000558433385\nMinimizing 26929 Time: 0:03:04 ( 6.87 ms/it)\n objective: 13321.997768479632\nMinimizing 26944 Time: 0:03:04 ( 6.87 ms/it)\n objective: 13321.989875511877\nMinimizing 26959 Time: 0:03:05 ( 6.87 ms/it)\n objective: 13321.984484698885\nMinimizing 26974 Time: 0:03:05 ( 6.87 ms/it)\n objective: 13321.982594697605\nMinimizing 26989 Time: 0:03:05 ( 6.87 ms/it)\n objective: 13321.974760900397\nMinimizing 27004 Time: 0:03:05 ( 6.87 ms/it)\n objective: 13321.941009857852\nMinimizing 27019 Time: 0:03:05 ( 6.87 ms/it)\n objective: 13321.806900048294\nMinimizing 27034 Time: 0:03:05 ( 6.87 ms/it)\n objective: 13321.422455843116\nMinimizing 27049 Time: 0:03:05 ( 6.87 ms/it)\n objective: 13320.268549029715\nMinimizing 27064 Time: 0:03:05 ( 6.87 ms/it)\n objective: 13320.255488383424\nMinimizing 27079 Time: 0:03:05 ( 6.87 ms/it)\n objective: 13320.250728623287\nMinimizing 27094 Time: 0:03:06 ( 6.87 ms/it)\n objective: 13320.24796584739\nMinimizing 27109 Time: 0:03:06 ( 6.87 ms/it)\n objective: 13320.2452907572\nMinimizing 27124 Time: 0:03:06 ( 6.87 ms/it)\n objective: 13320.223813572185\nMinimizing 27139 Time: 0:03:06 ( 6.87 ms/it)\n objective: 13320.208108329185\nMinimizing 27154 Time: 0:03:06 ( 6.87 ms/it)\n objective: 13320.148346046684\nMinimizing 27169 Time: 0:03:06 ( 6.87 ms/it)\n objective: 13320.052495868484\nMinimizing 27184 Time: 0:03:06 ( 6.87 ms/it)\n objective: 13320.021999924502\nMinimizing 27199 Time: 0:03:06 ( 6.87 ms/it)\n objective: 13320.021497332142\nMinimizing 27214 Time: 0:03:06 ( 6.87 ms/it)\n objective: 13320.016425152353\nMinimizing 27229 Time: 0:03:06 ( 6.87 ms/it)\n objective: 13320.012925402028\nMinimizing 27244 Time: 0:03:07 ( 6.87 ms/it)\n objective: 13320.007580268633\nMinimizing 27259 Time: 0:03:07 ( 6.87 ms/it)\n objective: 13319.99838856989\nMinimizing 27274 Time: 0:03:07 ( 6.87 ms/it)\n objective: 13319.994593361116\nMinimizing 27289 Time: 0:03:07 ( 6.87 ms/it)\n objective: 13319.98299152295\nMinimizing 27304 Time: 0:03:07 ( 6.87 ms/it)\n objective: 13319.971892012793\nMinimizing 27319 Time: 0:03:07 ( 6.87 ms/it)\n objective: 13319.953655871344\nMinimizing 27334 Time: 0:03:07 ( 6.87 ms/it)\n objective: 13319.947623654865\nMinimizing 27349 Time: 0:03:07 ( 6.87 ms/it)\n objective: 13319.938840763323\nMinimizing 27364 Time: 0:03:07 ( 6.87 ms/it)\n objective: 13319.931863788632\nMinimizing 27379 Time: 0:03:07 ( 6.87 ms/it)\n objective: 13319.907144755925\nMinimizing 27394 Time: 0:03:08 ( 6.87 ms/it)\n objective: 13319.904510039138\nMinimizing 27409 Time: 0:03:08 ( 6.87 ms/it)\n objective: 13319.848195948405\nMinimizing 27424 Time: 0:03:08 ( 6.87 ms/it)\n objective: 13319.846388926875\nMinimizing 27439 Time: 0:03:08 ( 6.86 ms/it)\n objective: 13319.845897955325\nMinimizing 27454 Time: 0:03:08 ( 6.86 ms/it)\n objective: 13319.845384788816\nMinimizing 27469 Time: 0:03:08 ( 6.86 ms/it)\n objective: 13319.845057090832\nMinimizing 27484 Time: 0:03:08 ( 6.86 ms/it)\n objective: 13319.844095454348\nMinimizing 27499 Time: 0:03:08 ( 6.86 ms/it)\n objective: 13319.840902387266\nMinimizing 27514 Time: 0:03:08 ( 6.86 ms/it)\n objective: 13319.834684381596\nMinimizing 27529 Time: 0:03:08 ( 6.86 ms/it)\n objective: 13319.82131133374\nMinimizing 27544 Time: 0:03:09 ( 6.86 ms/it)\n objective: 13319.789587220163\nMinimizing 27559 Time: 0:03:09 ( 6.86 ms/it)\n objective: 13319.789443854082\nMinimizing 27574 Time: 0:03:09 ( 6.86 ms/it)\n objective: 13319.789026905608\nMinimizing 27589 Time: 0:03:09 ( 6.86 ms/it)\n objective: 13319.787153598852\nMinimizing 27604 Time: 0:03:09 ( 6.86 ms/it)\n objective: 13319.78665749525\nMinimizing 27619 Time: 0:03:09 ( 6.86 ms/it)\n objective: 13319.783850553227\nMinimizing 27634 Time: 0:03:09 ( 6.86 ms/it)\n objective: 13319.773207589213\nMinimizing 27649 Time: 0:03:09 ( 6.86 ms/it)\n objective: 13319.772689421778\nMinimizing 27664 Time: 0:03:09 ( 6.87 ms/it)\n objective: 13319.77213598913\nMinimizing 27679 Time: 0:03:10 ( 6.87 ms/it)\n objective: 13319.77019615729\nMinimizing 27694 Time: 0:03:10 ( 6.87 ms/it)\n objective: 13319.7686089208\nMinimizing 27709 Time: 0:03:10 ( 6.87 ms/it)\n objective: 13319.768075620057\nMinimizing 27724 Time: 0:03:10 ( 6.87 ms/it)\n objective: 13319.766322738826\nMinimizing 27739 Time: 0:03:10 ( 6.87 ms/it)\n objective: 13319.763331976617\nMinimizing 27754 Time: 0:03:10 ( 6.87 ms/it)\n objective: 13319.735881350993\nMinimizing 27769 Time: 0:03:10 ( 6.87 ms/it)\n objective: 13319.735561155161\nMinimizing 27784 Time: 0:03:10 ( 6.87 ms/it)\n objective: 13319.734453965473\nMinimizing 27799 Time: 0:03:10 ( 6.87 ms/it)\n objective: 13319.730423380228\nMinimizing 27814 Time: 0:03:10 ( 6.87 ms/it)\n objective: 13319.728180403981\nMinimizing 27829 Time: 0:03:11 ( 6.86 ms/it)\n objective: 13319.72781844296\nMinimizing 27844 Time: 0:03:11 ( 6.86 ms/it)\n objective: 13319.727311298688\nMinimizing 27859 Time: 0:03:11 ( 6.86 ms/it)\n objective: 13319.726190035217\nMinimizing 27874 Time: 0:03:11 ( 6.86 ms/it)\n objective: 13319.71980743183\nMinimizing 27889 Time: 0:03:11 ( 6.86 ms/it)\n objective: 13319.718782030264\nMinimizing 27904 Time: 0:03:11 ( 6.86 ms/it)\n objective: 13319.710864137538\nMinimizing 27919 Time: 0:03:11 ( 6.86 ms/it)\n objective: 13319.710562760447\nMinimizing 27934 Time: 0:03:11 ( 6.86 ms/it)\n objective: 13319.705356261344\nMinimizing 27949 Time: 0:03:11 ( 6.86 ms/it)\n objective: 13319.70447774035\nMinimizing 27964 Time: 0:03:11 ( 6.86 ms/it)\n objective: 13319.701828833466\nMinimizing 27979 Time: 0:03:12 ( 6.86 ms/it)\n objective: 13319.700727715084\nMinimizing 27994 Time: 0:03:12 ( 6.86 ms/it)\n objective: 13319.699621296662\nMinimizing 28009 Time: 0:03:12 ( 6.86 ms/it)\n objective: 13319.69867906229\nMinimizing 28024 Time: 0:03:12 ( 6.86 ms/it)\n objective: 13319.696766442576\nMinimizing 28039 Time: 0:03:12 ( 6.86 ms/it)\n objective: 13319.691624293875\nMinimizing 28054 Time: 0:03:12 ( 6.86 ms/it)\n objective: 13319.690739939979\nMinimizing 28069 Time: 0:03:12 ( 6.86 ms/it)\n objective: 13319.680722921854\nMinimizing 28084 Time: 0:03:12 ( 6.86 ms/it)\n objective: 13319.677997205537\nMinimizing 28099 Time: 0:03:12 ( 6.86 ms/it)\n objective: 13319.677811397996\nMinimizing 28114 Time: 0:03:12 ( 6.86 ms/it)\n objective: 13319.67738327969\nMinimizing 28129 Time: 0:03:13 ( 6.86 ms/it)\n objective: 13319.677226634798\nMinimizing 28144 Time: 0:03:13 ( 6.86 ms/it)\n objective: 13319.676019415681\nMinimizing 28159 Time: 0:03:13 ( 6.86 ms/it)\n objective: 13319.669088710667\nMinimizing 28174 Time: 0:03:13 ( 6.86 ms/it)\n objective: 13319.65728714707\nMinimizing 28189 Time: 0:03:13 ( 6.86 ms/it)\n objective: 13319.657030676506\nMinimizing 28204 Time: 0:03:13 ( 6.86 ms/it)\n objective: 13319.655993959444\nMinimizing 28219 Time: 0:03:13 ( 6.86 ms/it)\n objective: 13319.65466138946\nMinimizing 28234 Time: 0:03:13 ( 6.86 ms/it)\n objective: 13319.654402434448\nMinimizing 28249 Time: 0:03:13 ( 6.86 ms/it)\n objective: 13319.653831912918\nMinimizing 28264 Time: 0:03:14 ( 6.86 ms/it)\n objective: 13319.653259204628\nMinimizing 28279 Time: 0:03:14 ( 6.86 ms/it)\n objective: 13319.644774496875\nMinimizing 28294 Time: 0:03:14 ( 6.86 ms/it)\n objective: 13319.642204099\nMinimizing 28309 Time: 0:03:14 ( 6.86 ms/it)\n objective: 13319.641135271682\nMinimizing 28324 Time: 0:03:14 ( 6.86 ms/it)\n objective: 13319.640681949008\nMinimizing 28339 Time: 0:03:14 ( 6.86 ms/it)\n objective: 13319.638560322928\nMinimizing 28354 Time: 0:03:14 ( 6.86 ms/it)\n objective: 13319.637176410644\nMinimizing 28369 Time: 0:03:14 ( 6.86 ms/it)\n objective: 13319.63155167397\nMinimizing 28384 Time: 0:03:14 ( 6.86 ms/it)\n objective: 13319.628818496829\nMinimizing 28399 Time: 0:03:14 ( 6.86 ms/it)\n objective: 13319.627807899102\nMinimizing 28414 Time: 0:03:15 ( 6.86 ms/it)\n objective: 13319.623492199447\nMinimizing 28429 Time: 0:03:15 ( 6.86 ms/it)\n objective: 13319.623167782906\nMinimizing 28444 Time: 0:03:15 ( 6.86 ms/it)\n objective: 13319.622348744932\nMinimizing 28459 Time: 0:03:15 ( 6.86 ms/it)\n objective: 13319.618558170128\nMinimizing 28474 Time: 0:03:15 ( 6.86 ms/it)\n objective: 13319.6178068476\nMinimizing 28489 Time: 0:03:15 ( 6.86 ms/it)\n objective: 13319.614513968307\nMinimizing 28504 Time: 0:03:15 ( 6.86 ms/it)\n objective: 13319.609692625687\nMinimizing 28519 Time: 0:03:15 ( 6.86 ms/it)\n objective: 13319.608866740964\nMinimizing 28534 Time: 0:03:15 ( 6.86 ms/it)\n objective: 13319.603108909461\nMinimizing 28549 Time: 0:03:15 ( 6.86 ms/it)\n objective: 13319.601996628408\nMinimizing 28564 Time: 0:03:16 ( 6.86 ms/it)\n objective: 13319.591364269916\nMinimizing 28579 Time: 0:03:16 ( 6.86 ms/it)\n objective: 13319.59113128393\nMinimizing 28594 Time: 0:03:16 ( 6.86 ms/it)\n objective: 13319.586932560574\nMinimizing 28609 Time: 0:03:16 ( 6.86 ms/it)\n objective: 13319.577102816227\nMinimizing 28624 Time: 0:03:16 ( 6.86 ms/it)\n objective: 13319.57699748849\nMinimizing 28639 Time: 0:03:16 ( 6.86 ms/it)\n objective: 13319.573669211997\nMinimizing 28654 Time: 0:03:16 ( 6.86 ms/it)\n objective: 13319.573296881368\nMinimizing 28669 Time: 0:03:16 ( 6.86 ms/it)\n objective: 13319.569988628704\nMinimizing 28684 Time: 0:03:16 ( 6.86 ms/it)\n objective: 13319.554054480686\nMinimizing 28699 Time: 0:03:16 ( 6.86 ms/it)\n objective: 13319.541212009935\nMinimizing 28714 Time: 0:03:17 ( 6.86 ms/it)\n objective: 13319.539904969075\nMinimizing 28729 Time: 0:03:17 ( 6.86 ms/it)\n objective: 13319.534360820122\nMinimizing 28744 Time: 0:03:17 ( 6.86 ms/it)\n objective: 13319.533288538762\nMinimizing 28759 Time: 0:03:17 ( 6.86 ms/it)\n objective: 13319.517475092274\nMinimizing 28774 Time: 0:03:17 ( 6.86 ms/it)\n objective: 13319.506605894669\nMinimizing 28789 Time: 0:03:17 ( 6.86 ms/it)\n objective: 13319.505671887411\nMinimizing 28804 Time: 0:03:17 ( 6.86 ms/it)\n objective: 13319.501485319357\nMinimizing 28819 Time: 0:03:17 ( 6.86 ms/it)\n objective: 13319.495530651257\nMinimizing 28834 Time: 0:03:17 ( 6.86 ms/it)\n objective: 13319.493524352743\nMinimizing 28849 Time: 0:03:17 ( 6.86 ms/it)\n objective: 13319.4907841393\nMinimizing 28864 Time: 0:03:18 ( 6.86 ms/it)\n objective: 13319.484216522906\nMinimizing 28879 Time: 0:03:18 ( 6.86 ms/it)\n objective: 13319.478785074141\nMinimizing 28894 Time: 0:03:18 ( 6.86 ms/it)\n objective: 13319.476925632305\nMinimizing 28909 Time: 0:03:18 ( 6.86 ms/it)\n objective: 13319.461900662136\nMinimizing 28924 Time: 0:03:18 ( 6.86 ms/it)\n objective: 13319.4614213553\nMinimizing 28939 Time: 0:03:18 ( 6.86 ms/it)\n objective: 13319.459380531785\nMinimizing 28954 Time: 0:03:18 ( 6.86 ms/it)\n objective: 13319.457299720787\nMinimizing 28969 Time: 0:03:18 ( 6.86 ms/it)\n objective: 13319.456737670174\nMinimizing 28984 Time: 0:03:18 ( 6.86 ms/it)\n objective: 13319.446059792332\nMinimizing 28999 Time: 0:03:18 ( 6.86 ms/it)\n objective: 13319.445503213952\nMinimizing 29014 Time: 0:03:19 ( 6.86 ms/it)\n objective: 13319.442146683781\nMinimizing 29029 Time: 0:03:19 ( 6.86 ms/it)\n objective: 13319.415166991312\nMinimizing 29044 Time: 0:03:19 ( 6.86 ms/it)\n objective: 13319.41487045007\nMinimizing 29059 Time: 0:03:19 ( 6.86 ms/it)\n objective: 13319.414466591668\nMinimizing 29074 Time: 0:03:19 ( 6.86 ms/it)\n objective: 13319.41299531865\nMinimizing 29089 Time: 0:03:19 ( 6.86 ms/it)\n objective: 13319.410858698539\nMinimizing 29104 Time: 0:03:19 ( 6.86 ms/it)\n objective: 13319.394128321961\nMinimizing 29119 Time: 0:03:19 ( 6.86 ms/it)\n objective: 13319.392837391308\nMinimizing 29134 Time: 0:03:19 ( 6.86 ms/it)\n objective: 13319.392651456335\nMinimizing 29149 Time: 0:03:20 ( 6.86 ms/it)\n objective: 13319.392090191497\nMinimizing 29164 Time: 0:03:20 ( 6.86 ms/it)\n objective: 13319.391447792324\nMinimizing 29179 Time: 0:03:20 ( 6.86 ms/it)\n objective: 13319.390749635233\nMinimizing 29194 Time: 0:03:20 ( 6.86 ms/it)\n objective: 13319.383641825247\nMinimizing 29209 Time: 0:03:20 ( 6.86 ms/it)\n objective: 13319.382706084856\nMinimizing 29224 Time: 0:03:20 ( 6.86 ms/it)\n objective: 13319.38233220455\nMinimizing 29239 Time: 0:03:20 ( 6.86 ms/it)\n objective: 13319.381453434966\nMinimizing 29254 Time: 0:03:20 ( 6.86 ms/it)\n objective: 13319.380384236909\nMinimizing 29269 Time: 0:03:20 ( 6.86 ms/it)\n objective: 13319.37983905754\nMinimizing 29284 Time: 0:03:20 ( 6.86 ms/it)\n objective: 13319.37318632497\nMinimizing 29299 Time: 0:03:21 ( 6.86 ms/it)\n objective: 13319.360341063817\nMinimizing 29314 Time: 0:03:21 ( 6.86 ms/it)\n objective: 13319.360157723713\nMinimizing 29329 Time: 0:03:21 ( 6.86 ms/it)\n objective: 13319.359211586256\nMinimizing 29344 Time: 0:03:21 ( 6.86 ms/it)\n objective: 13319.357355823944\nMinimizing 29359 Time: 0:03:21 ( 6.86 ms/it)\n objective: 13319.356791910817\nMinimizing 29374 Time: 0:03:21 ( 6.86 ms/it)\n objective: 13319.349634279424\nMinimizing 29389 Time: 0:03:21 ( 6.86 ms/it)\n objective: 13319.33225182908\nMinimizing 29404 Time: 0:03:21 ( 6.86 ms/it)\n objective: 13319.331679539988\nMinimizing 29419 Time: 0:03:21 ( 6.86 ms/it)\n objective: 13319.330117062593\nMinimizing 29434 Time: 0:03:21 ( 6.86 ms/it)\n objective: 13319.325558373574\nMinimizing 29449 Time: 0:03:22 ( 6.86 ms/it)\n objective: 13319.32533681416\nMinimizing 29464 Time: 0:03:22 ( 6.86 ms/it)\n objective: 13319.322487801095\nMinimizing 29479 Time: 0:03:22 ( 6.86 ms/it)\n objective: 13319.32188202138\nMinimizing 29494 Time: 0:03:22 ( 6.86 ms/it)\n objective: 13319.32115749891\nMinimizing 29509 Time: 0:03:22 ( 6.86 ms/it)\n objective: 13319.319919370362\nMinimizing 29524 Time: 0:03:22 ( 6.86 ms/it)\n objective: 13319.317433819335\nMinimizing 29539 Time: 0:03:22 ( 6.86 ms/it)\n objective: 13319.303214779968\nMinimizing 29554 Time: 0:03:22 ( 6.86 ms/it)\n objective: 13319.30049935331\nMinimizing 29569 Time: 0:03:22 ( 6.86 ms/it)\n objective: 13319.300145254194\nMinimizing 29584 Time: 0:03:22 ( 6.86 ms/it)\n objective: 13319.299859899693\nMinimizing 29599 Time: 0:03:23 ( 6.86 ms/it)\n objective: 13319.299305651613\nMinimizing 29614 Time: 0:03:23 ( 6.86 ms/it)\n objective: 13319.298650861601\nMinimizing 29629 Time: 0:03:23 ( 6.86 ms/it)\n objective: 13319.307562207527\nMinimizing 29644 Time: 0:03:23 ( 6.86 ms/it)\n objective: 13319.280197813845\nMinimizing 29659 Time: 0:03:23 ( 6.86 ms/it)\n objective: 13319.280041465448\nMinimizing 29674 Time: 0:03:23 ( 6.86 ms/it)\n objective: 13319.279121145533\nMinimizing 29689 Time: 0:03:23 ( 6.86 ms/it)\n objective: 13319.277906575444\nMinimizing 29704 Time: 0:03:23 ( 6.86 ms/it)\n objective: 13319.27648061463\nMinimizing 29719 Time: 0:03:23 ( 6.86 ms/it)\n objective: 13319.27572350124\nMinimizing 29734 Time: 0:03:24 ( 6.86 ms/it)\n objective: 13319.266880042254\nMinimizing 29749 Time: 0:03:24 ( 6.86 ms/it)\n objective: 13319.266561918645\nMinimizing 29764 Time: 0:03:24 ( 6.86 ms/it)\n objective: 13319.265186968245\nMinimizing 29779 Time: 0:03:24 ( 6.86 ms/it)\n objective: 13319.25982279767\nMinimizing 29794 Time: 0:03:24 ( 6.86 ms/it)\n objective: 13319.252946444889\nMinimizing 29809 Time: 0:03:24 ( 6.86 ms/it)\n objective: 13319.25265336885\nMinimizing 29824 Time: 0:03:24 ( 6.86 ms/it)\n objective: 13319.25234013448\nMinimizing 29839 Time: 0:03:24 ( 6.86 ms/it)\n objective: 13319.25199416552\nMinimizing 29854 Time: 0:03:24 ( 6.86 ms/it)\n objective: 13319.248944443927\nMinimizing 29869 Time: 0:03:24 ( 6.86 ms/it)\n objective: 13319.246634433468\nMinimizing 29884 Time: 0:03:25 ( 6.86 ms/it)\n objective: 13319.245844556339\nMinimizing 29899 Time: 0:03:25 ( 6.86 ms/it)\n objective: 13319.24041514429\nMinimizing 29914 Time: 0:03:25 ( 6.86 ms/it)\n objective: 13319.238714900013\nMinimizing 29929 Time: 0:03:25 ( 6.86 ms/it)\n objective: 13319.238333718167\nMinimizing 29944 Time: 0:03:25 ( 6.86 ms/it)\n objective: 13319.237901613335\nMinimizing 29959 Time: 0:03:25 ( 6.86 ms/it)\n objective: 13319.237569940946\nMinimizing 29974 Time: 0:03:25 ( 6.86 ms/it)\n objective: 13319.235634601806\nMinimizing 29989 Time: 0:03:25 ( 6.86 ms/it)\n objective: 13319.228652360864\nMinimizing 30004 Time: 0:03:25 ( 6.86 ms/it)\n objective: 13319.228561906508\nMinimizing 30019 Time: 0:03:25 ( 6.86 ms/it)\n objective: 13319.228046899982\nMinimizing 30034 Time: 0:03:26 ( 6.86 ms/it)\n objective: 13319.22710673805\nMinimizing 30049 Time: 0:03:26 ( 6.86 ms/it)\n objective: 13319.225931763169\nMinimizing 30064 Time: 0:03:26 ( 6.86 ms/it)\n objective: 13319.224867787387\nMinimizing 30079 Time: 0:03:26 ( 6.86 ms/it)\n objective: 13319.22414478629\nMinimizing 30094 Time: 0:03:26 ( 6.86 ms/it)\n objective: 13319.220894244238\nMinimizing 30109 Time: 0:03:26 ( 6.86 ms/it)\n objective: 13319.223722022303\nMinimizing 30124 Time: 0:03:26 ( 6.86 ms/it)\n objective: 13319.175184773034\nMinimizing 30139 Time: 0:03:26 ( 6.86 ms/it)\n objective: 13319.174862641274\nMinimizing 30154 Time: 0:03:26 ( 6.86 ms/it)\n objective: 13319.173213871778\nMinimizing 30169 Time: 0:03:26 ( 6.86 ms/it)\n objective: 13319.17276937126\nMinimizing 30184 Time: 0:03:27 ( 6.86 ms/it)\n objective: 13319.171252959379\nMinimizing 30199 Time: 0:03:27 ( 6.86 ms/it)\n objective: 13319.166585182014\nMinimizing 30214 Time: 0:03:27 ( 6.86 ms/it)\n objective: 13319.158402624322\nMinimizing 30229 Time: 0:03:27 ( 6.86 ms/it)\n objective: 13319.147387968056\nMinimizing 30244 Time: 0:03:27 ( 6.86 ms/it)\n objective: 13319.113353657434\nMinimizing 30259 Time: 0:03:27 ( 6.86 ms/it)\n objective: 13319.087326319117\nMinimizing 30274 Time: 0:03:27 ( 6.86 ms/it)\n objective: 13319.086572416621\nMinimizing 30289 Time: 0:03:27 ( 6.86 ms/it)\n objective: 13319.083536452148\nMinimizing 30304 Time: 0:03:27 ( 6.86 ms/it)\n objective: 13319.08128510078\nMinimizing 30319 Time: 0:03:27 ( 6.86 ms/it)\n objective: 13319.080896167245\nMinimizing 30334 Time: 0:03:28 ( 6.86 ms/it)\n objective: 13319.071622916323\nMinimizing 30349 Time: 0:03:28 ( 6.86 ms/it)\n objective: 13319.058759014864\nMinimizing 30364 Time: 0:03:28 ( 6.86 ms/it)\n objective: 13318.967923063654\nMinimizing 30379 Time: 0:03:28 ( 6.86 ms/it)\n objective: 13318.95189589198\nMinimizing 30394 Time: 0:03:28 ( 6.86 ms/it)\n objective: 13318.951559366978\nMinimizing 30409 Time: 0:03:28 ( 6.86 ms/it)\n objective: 13318.95016177863\nMinimizing 30424 Time: 0:03:28 ( 6.86 ms/it)\n objective: 13318.949680432095\nMinimizing 30439 Time: 0:03:28 ( 6.86 ms/it)\n objective: 13318.949289823096\nMinimizing 30454 Time: 0:03:28 ( 6.86 ms/it)\n objective: 13318.948242705825\nMinimizing 30469 Time: 0:03:29 ( 6.86 ms/it)\n objective: 13318.942267297578\nMinimizing 30484 Time: 0:03:29 ( 6.86 ms/it)\n objective: 13318.938201766781\nMinimizing 30499 Time: 0:03:29 ( 6.86 ms/it)\n objective: 13318.936820964635\nMinimizing 30514 Time: 0:03:29 ( 6.86 ms/it)\n objective: 13318.925886723679\nMinimizing 30529 Time: 0:03:29 ( 6.86 ms/it)\n objective: 13318.925435335113\nMinimizing 30544 Time: 0:03:29 ( 6.86 ms/it)\n objective: 13318.920956352958\nMinimizing 30559 Time: 0:03:29 ( 6.86 ms/it)\n objective: 13318.920537958053\nMinimizing 30574 Time: 0:03:29 ( 6.86 ms/it)\n objective: 13318.918995824351\nMinimizing 30589 Time: 0:03:29 ( 6.86 ms/it)\n objective: 13318.922051668735\nMinimizing 30604 Time: 0:03:29 ( 6.86 ms/it)\n objective: 13318.89733898679\nMinimizing 30619 Time: 0:03:30 ( 6.86 ms/it)\n objective: 13318.892939684898\nMinimizing 30634 Time: 0:03:30 ( 6.86 ms/it)\n objective: 13318.891872379463\nMinimizing 30649 Time: 0:03:30 ( 6.86 ms/it)\n objective: 13318.891054441905\nMinimizing 30664 Time: 0:03:30 ( 6.86 ms/it)\n objective: 13318.889973928206\nMinimizing 30679 Time: 0:03:30 ( 6.86 ms/it)\n objective: 13318.88753616187\nMinimizing 30694 Time: 0:03:30 ( 6.86 ms/it)\n objective: 13318.886504811424\nMinimizing 30709 Time: 0:03:30 ( 6.86 ms/it)\n objective: 13318.881112211311\nMinimizing 30724 Time: 0:03:30 ( 6.86 ms/it)\n objective: 13318.877419668235\nMinimizing 30739 Time: 0:03:30 ( 6.86 ms/it)\n objective: 13318.868394685516\nMinimizing 30754 Time: 0:03:30 ( 6.86 ms/it)\n objective: 13318.8581605988\nMinimizing 30769 Time: 0:03:31 ( 6.86 ms/it)\n objective: 13318.85713138587\nMinimizing 30784 Time: 0:03:31 ( 6.86 ms/it)\n objective: 13318.854234413273\nMinimizing 30799 Time: 0:03:31 ( 6.86 ms/it)\n objective: 13318.850390373613\nMinimizing 30814 Time: 0:03:31 ( 6.86 ms/it)\n objective: 13318.837969755055\nMinimizing 30829 Time: 0:03:31 ( 6.86 ms/it)\n objective: 13318.823183799788\nMinimizing 30844 Time: 0:03:31 ( 6.86 ms/it)\n objective: 13318.821716557752\nMinimizing 30859 Time: 0:03:31 ( 6.86 ms/it)\n objective: 13318.816919184566\nMinimizing 30874 Time: 0:03:31 ( 6.86 ms/it)\n objective: 13318.811515493187\nMinimizing 30889 Time: 0:03:31 ( 6.86 ms/it)\n objective: 13318.809563593051\nMinimizing 30904 Time: 0:03:31 ( 6.86 ms/it)\n objective: 13318.808869929693\nMinimizing 30919 Time: 0:03:32 ( 6.86 ms/it)\n objective: 13318.80621255796\nMinimizing 30934 Time: 0:03:32 ( 6.86 ms/it)\n objective: 13318.798838254312\nMinimizing 30949 Time: 0:03:32 ( 6.86 ms/it)\n objective: 13318.797319051882\nMinimizing 30964 Time: 0:03:32 ( 6.86 ms/it)\n objective: 13318.796533040295\nMinimizing 30979 Time: 0:03:32 ( 6.86 ms/it)\n objective: 13318.795993607971\nMinimizing 30994 Time: 0:03:32 ( 6.86 ms/it)\n objective: 13318.79496820386\nMinimizing 31009 Time: 0:03:32 ( 6.86 ms/it)\n objective: 13318.794424001258\nMinimizing 31024 Time: 0:03:32 ( 6.86 ms/it)\n objective: 13318.77881645714\nMinimizing 31039 Time: 0:03:32 ( 6.86 ms/it)\n objective: 13318.768578322823\nMinimizing 31054 Time: 0:03:32 ( 6.86 ms/it)\n objective: 13318.766335496519\nMinimizing 31069 Time: 0:03:33 ( 6.86 ms/it)\n objective: 13318.762424403569\nMinimizing 31084 Time: 0:03:33 ( 6.86 ms/it)\n objective: 13318.76133113807\nMinimizing 31099 Time: 0:03:33 ( 6.86 ms/it)\n objective: 13318.757385488585\nMinimizing 31114 Time: 0:03:33 ( 6.86 ms/it)\n objective: 13318.742703805518\nMinimizing 31129 Time: 0:03:33 ( 6.86 ms/it)\n objective: 13318.736372023443\nMinimizing 31144 Time: 0:03:33 ( 6.86 ms/it)\n objective: 13318.73592477999\nMinimizing 31159 Time: 0:03:33 ( 6.86 ms/it)\n objective: 13318.733248780263\nMinimizing 31174 Time: 0:03:33 ( 6.86 ms/it)\n objective: 13318.72997828864\nMinimizing 31189 Time: 0:03:33 ( 6.86 ms/it)\n objective: 13318.71687801517\nMinimizing 31204 Time: 0:03:34 ( 6.86 ms/it)\n objective: 13318.716341016494\nMinimizing 31219 Time: 0:03:34 ( 6.86 ms/it)\n objective: 13318.73195776678\nMinimizing 31234 Time: 0:03:34 ( 6.86 ms/it)\n objective: 13318.69436955129\nMinimizing 31249 Time: 0:03:34 ( 6.86 ms/it)\n objective: 13318.567047572127\nMinimizing 31264 Time: 0:03:34 ( 6.86 ms/it)\n objective: 13318.555847161522\nMinimizing 31279 Time: 0:03:34 ( 6.86 ms/it)\n objective: 13318.555585473441\nMinimizing 31294 Time: 0:03:34 ( 6.86 ms/it)\n objective: 13318.555218169116\nMinimizing 31309 Time: 0:03:34 ( 6.86 ms/it)\n objective: 13318.555014153942\nMinimizing 31324 Time: 0:03:34 ( 6.86 ms/it)\n objective: 13318.553477215974\nMinimizing 31339 Time: 0:03:34 ( 6.86 ms/it)\n objective: 13318.552698611835\nMinimizing 31354 Time: 0:03:35 ( 6.86 ms/it)\n objective: 13318.551583247012\nMinimizing 31369 Time: 0:03:35 ( 6.86 ms/it)\n objective: 13318.525152111979\nMinimizing 31384 Time: 0:03:35 ( 6.86 ms/it)\n objective: 13318.50466107596\nMinimizing 31399 Time: 0:03:35 ( 6.86 ms/it)\n objective: 13318.504483145924\nMinimizing 31414 Time: 0:03:35 ( 6.86 ms/it)\n objective: 13318.504148584718\nMinimizing 31429 Time: 0:03:35 ( 6.86 ms/it)\n objective: 13318.503907172766\nMinimizing 31444 Time: 0:03:35 ( 6.86 ms/it)\n objective: 13318.503521238177\nMinimizing 31459 Time: 0:03:35 ( 6.86 ms/it)\n objective: 13318.503289076325\nMinimizing 31474 Time: 0:03:35 ( 6.86 ms/it)\n objective: 13318.502459596319\nMinimizing 31489 Time: 0:03:35 ( 6.86 ms/it)\n objective: 13318.49627119019\nMinimizing 31504 Time: 0:03:36 ( 6.86 ms/it)\n objective: 13318.495306164172\nMinimizing 31519 Time: 0:03:36 ( 6.86 ms/it)\n objective: 13318.492374510402\nMinimizing 31534 Time: 0:03:36 ( 6.86 ms/it)\n objective: 13318.490290971793\nMinimizing 31549 Time: 0:03:36 ( 6.86 ms/it)\n objective: 13318.48985730666\nMinimizing 31564 Time: 0:03:36 ( 6.86 ms/it)\n objective: 13318.489744292747\nMinimizing 31579 Time: 0:03:36 ( 6.86 ms/it)\n objective: 13318.486726732954\nMinimizing 31593 Time: 0:03:36 ( 6.86 ms/it)\n objective: 13318.48022669289\nMinimizing 31608 Time: 0:03:36 ( 6.86 ms/it)\n objective: 13318.480097212654\nMinimizing 31623 Time: 0:03:36 ( 6.86 ms/it)\n objective: 13318.479483446848\nMinimizing 31638 Time: 0:03:36 ( 6.86 ms/it)\n objective: 13318.47910676575\nMinimizing 31652 Time: 0:03:37 ( 6.86 ms/it)\n objective: 13318.477661737634\nMinimizing 31666 Time: 0:03:37 ( 6.86 ms/it)\n objective: 13318.467330348692\nMinimizing 31678 Time: 0:03:37 ( 6.86 ms/it)\n objective: 13318.467061815405\nMinimizing 31692 Time: 0:03:37 ( 6.86 ms/it)\n objective: 13318.4662447788\nMinimizing 31706 Time: 0:03:37 ( 6.86 ms/it)\n objective: 13318.465420935856\nMinimizing 31720 Time: 0:03:37 ( 6.86 ms/it)\n objective: 13318.461528615968\nMinimizing 31735 Time: 0:03:37 ( 6.86 ms/it)\n objective: 13318.461141356514\nMinimizing 31749 Time: 0:03:37 ( 6.86 ms/it)\n objective: 13318.46025494275\nMinimizing 31763 Time: 0:03:37 ( 6.86 ms/it)\n objective: 13318.458797042593\nMinimizing 31777 Time: 0:03:38 ( 6.86 ms/it)\n objective: 13318.454866237691\nMinimizing 31792 Time: 0:03:38 ( 6.86 ms/it)\n objective: 13318.448094710475\nMinimizing 31806 Time: 0:03:38 ( 6.86 ms/it)\n objective: 13318.442435281817\nMinimizing 31820 Time: 0:03:38 ( 6.86 ms/it)\n objective: 13318.438825138\nMinimizing 31834 Time: 0:03:38 ( 6.86 ms/it)\n objective: 13318.437178064822\nMinimizing 31847 Time: 0:03:38 ( 6.86 ms/it)\n objective: 13318.436841250936\nMinimizing 31861 Time: 0:03:38 ( 6.86 ms/it)\n objective: 13318.434138978802\nMinimizing 31875 Time: 0:03:38 ( 6.86 ms/it)\n objective: 13318.427068281642\nMinimizing 31889 Time: 0:03:38 ( 6.86 ms/it)\n objective: 13318.426858292281\nMinimizing 31903 Time: 0:03:38 ( 6.86 ms/it)\n objective: 13318.426926414817\nMinimizing 31918 Time: 0:03:39 ( 6.86 ms/it)\n objective: 13318.426870108146\nMinimizing 31933 Time: 0:03:39 ( 6.86 ms/it)\n objective: 13318.426866771188\nMinimizing 31948 Time: 0:03:39 ( 6.86 ms/it)\n objective: 13318.42686730239\nMinimizing 31963 Time: 0:03:39 ( 6.86 ms/it)\n objective: 13318.426851992088\nMinimizing 31978 Time: 0:03:39 ( 6.86 ms/it)\n objective: 13318.42686271423\nMinimizing 31993 Time: 0:03:39 ( 6.86 ms/it)\n objective: 13318.42684531596\nMinimizing 32008 Time: 0:03:39 ( 6.86 ms/it)\n objective: 13318.426845584909\nMinimizing 32023 Time: 0:03:39 ( 6.86 ms/it)\n objective: 13318.426848744784\nMinimizing 32038 Time: 0:03:39 ( 6.86 ms/it)\n objective: 13318.426841513778\nMinimizing 32053 Time: 0:03:40 ( 6.86 ms/it)\n objective: 13318.426840389177\nMinimizing 32068 Time: 0:03:40 ( 6.86 ms/it)\n objective: 13318.426845112364\nMinimizing 32083 Time: 0:03:40 ( 6.86 ms/it)\n objective: 13318.426842506873\nMinimizing 32098 Time: 0:03:40 ( 6.86 ms/it)\n objective: 13318.42684245354\nMinimizing 32100 Time: 0:03:40 ( 6.86 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0183\n0.0139\n-1.32\n0.1876\n\n\n\nTest: Star-Run\n-0.0142\n0.0438\n-0.32\n0.7465\n\n\n\nTest: S20-Star\n-0.0111\n0.0409\n-0.27\n0.7862\n\n\n\nTest: SLJ-S20\n0.0099\n0.0442\n0.22\n0.8236\n\n\n\nTest: BPT-SLJ\n-0.0231\n0.0422\n-0.55\n0.5836\n\n\n\na1\n0.3167\n0.0476\n6.65\n<1e-10\n\n\n\nSex: Boys\n0.2048\n0.0139\n14.73\n<1e-48\n\n\n\nTest: Star-Run & a1\n0.2986\n0.1505\n1.98\n0.0472\n\n\n\nTest: S20-Star & a1\n0.0494\n0.1419\n0.35\n0.7275\n\n\n\nTest: SLJ-S20 & a1\n-0.0658\n0.1520\n-0.43\n0.6649\n\n\n\nTest: BPT-SLJ & a1\n0.3151\n0.1416\n2.23\n0.0261\n\n\n\nTest: Star-Run & Sex: Boys\n-0.1213\n0.0438\n-2.77\n0.0056\n\n\n\nTest: S20-Star & Sex: Boys\n0.0664\n0.0409\n1.63\n0.1041\n\n\n\nTest: SLJ-S20 & Sex: Boys\n0.0466\n0.0442\n1.05\n0.2919\n\n\n\nTest: BPT-SLJ & Sex: Boys\n0.1634\n0.0422\n3.87\n0.0001\n\n\n\na1 & Sex: Boys\n0.0502\n0.0476\n1.06\n0.2912\n\n\n\nTest: Star-Run & a1 & Sex: Boys\n-0.0114\n0.1505\n-0.08\n0.9395\n\n\n\nTest: S20-Star & a1 & Sex: Boys\n-0.0288\n0.1419\n-0.20\n0.8390\n\n\n\nTest: SLJ-S20 & a1 & Sex: Boys\n-0.0603\n0.1520\n-0.40\n0.6916\n\n\n\nTest: BPT-SLJ & a1 & Sex: Boys\n0.1415\n0.1416\n1.00\n0.3178\n\n\n\nTest: BPT\n\n\n\n\n0.9054\n\n\nTest: SLJ\n\n\n\n\n0.9717\n\n\nTest: Star_r\n\n\n\n\n0.9914\n\n\nTest: Run\n\n\n\n\n0.9717\n\n\nTest: S20_r\n\n\n\n\n0.9786\n\n\nResidual\n0.0003\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_cpx_0_SeqDiff)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\nChild\nTest: Run\n0.944206222\n0.971702743\n\n\n\n\n\n\n\nTest: Star_r\n0.982837416\n0.991381569\n+0.64\n\n\n\n\n\n\nTest: S20_r\n0.957593363\n0.978566995\n+0.22\n+0.78\n\n\n\n\n\nTest: SLJ\n0.944136095\n0.971666659\n+0.51\n+0.12\n-0.54\n\n\n\n\nTest: BPT\n0.819784002\n0.905419241\n+0.15\n+0.42\n+0.11\n+0.38\n\n\nResidual\n\n0.000000096\n0.000309326\n\n\n\n\n\n\n\n\n\n\nm_cpx_0_SeqDiff.PCA\n\n(Child = \nPrincipal components based on correlation matrix\n Test: Run 1.0 . . . .\n Test: Star_r 0.64 1.0 . . .\n Test: S20_r 0.22 0.78 1.0 . .\n Test: SLJ 0.51 0.12 -0.54 1.0 .\n Test: BPT 0.15 0.42 0.11 0.38 1.0\n\nNormalized cumulative variances:\n[0.4586, 0.803, 0.9738, 1.0, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5\n Test: Run -0.5 0.24 -0.56 0.61 -0.0\n Test: Star_r -0.64 -0.16 -0.02 -0.48 -0.58\n Test: S20_r -0.42 -0.59 0.03 -0.09 0.69\n Test: SLJ -0.19 0.71 -0.07 -0.51 0.44\n Test: BPT -0.36 0.25 0.82 0.36 -0.0,)\n\n\n\nf_cpx_1 = @formula(\n zScore ~ 1 + Test * a1 * Sex + (1 + Test | Child)\n)\nm_cpx_1_SeqDiff =\nfit(MixedModel, f_cpx_1, dat; contrasts=contr1b)\n\nMinimizing 15 Time: 0:00:00 ( 6.75 ms/it)\n objective: 13992.236627340779\nMinimizing 30 Time: 0:00:00 ( 6.78 ms/it)\n objective: 13841.827333118908\nMinimizing 46 Time: 0:00:00 ( 6.75 ms/it)\n objective: 13777.493789325295\nMinimizing 62 Time: 0:00:00 ( 6.73 ms/it)\n objective: 13772.872983178971\nMinimizing 77 Time: 0:00:00 ( 6.72 ms/it)\n objective: 13760.898044900383\nMinimizing 92 Time: 0:00:00 ( 6.72 ms/it)\n objective: 13757.94612351193\nMinimizing 107 Time: 0:00:00 ( 6.72 ms/it)\n objective: 13748.357981786798\nMinimizing 122 Time: 0:00:00 ( 6.72 ms/it)\n objective: 13744.757616729485\nMinimizing 137 Time: 0:00:00 ( 6.72 ms/it)\n objective: 13739.966470629659\nMinimizing 152 Time: 0:00:01 ( 6.72 ms/it)\n objective: 13716.545957536473\nMinimizing 167 Time: 0:00:01 ( 6.72 ms/it)\n objective: 13710.064283256732\nMinimizing 182 Time: 0:00:01 ( 6.73 ms/it)\n objective: 13696.051134898527\nMinimizing 197 Time: 0:00:01 ( 6.73 ms/it)\n objective: 13692.114899055618\nMinimizing 212 Time: 0:00:01 ( 6.73 ms/it)\n objective: 13685.070924484331\nMinimizing 227 Time: 0:00:01 ( 6.73 ms/it)\n objective: 13672.89278358193\nMinimizing 242 Time: 0:00:01 ( 6.73 ms/it)\n objective: 13661.980753634167\nMinimizing 257 Time: 0:00:01 ( 6.73 ms/it)\n objective: 13652.330741699696\nMinimizing 273 Time: 0:00:01 ( 6.72 ms/it)\n objective: 13643.691435874693\nMinimizing 289 Time: 0:00:01 ( 6.72 ms/it)\n objective: 13642.458938908409\nMinimizing 305 Time: 0:00:02 ( 6.72 ms/it)\n objective: 13639.567256285494\nMinimizing 321 Time: 0:00:02 ( 6.72 ms/it)\n objective: 13635.270641935873\nMinimizing 336 Time: 0:00:02 ( 6.72 ms/it)\n objective: 13632.134726806788\nMinimizing 351 Time: 0:00:02 ( 6.72 ms/it)\n objective: 13628.761272475658\nMinimizing 366 Time: 0:00:02 ( 6.73 ms/it)\n objective: 13624.189653118381\nMinimizing 381 Time: 0:00:02 ( 6.73 ms/it)\n objective: 13620.312662915818\nMinimizing 396 Time: 0:00:02 ( 6.73 ms/it)\n objective: 13618.417326557159\nMinimizing 411 Time: 0:00:02 ( 6.74 ms/it)\n objective: 13609.36978720955\nMinimizing 426 Time: 0:00:02 ( 6.74 ms/it)\n objective: 13607.234474665878\nMinimizing 441 Time: 0:00:02 ( 6.75 ms/it)\n objective: 13604.504124525956\nMinimizing 456 Time: 0:00:03 ( 6.76 ms/it)\n objective: 13602.045160800197\nMinimizing 471 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13600.801804760627\nMinimizing 486 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13598.14423866852\nMinimizing 501 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13597.34932065606\nMinimizing 516 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13595.704406463956\nMinimizing 531 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13594.245308691123\nMinimizing 546 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13593.789234561955\nMinimizing 562 Time: 0:00:03 ( 6.76 ms/it)\n objective: 13592.585058519155\nMinimizing 577 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13588.988369878443\nMinimizing 592 Time: 0:00:04 ( 6.76 ms/it)\n objective: 13588.288747463237\nMinimizing 607 Time: 0:00:04 ( 6.76 ms/it)\n objective: 13587.355794477331\nMinimizing 623 Time: 0:00:04 ( 6.76 ms/it)\n objective: 13586.7483568133\nMinimizing 638 Time: 0:00:04 ( 6.76 ms/it)\n objective: 13583.043956016008\nMinimizing 653 Time: 0:00:04 ( 6.77 ms/it)\n objective: 13582.832811241413\nMinimizing 668 Time: 0:00:04 ( 6.77 ms/it)\n objective: 13574.479359698835\nMinimizing 683 Time: 0:00:04 ( 6.77 ms/it)\n objective: 13568.874482939911\nMinimizing 698 Time: 0:00:04 ( 6.77 ms/it)\n objective: 13566.419593540963\nMinimizing 713 Time: 0:00:04 ( 6.77 ms/it)\n objective: 13560.72496586899\nMinimizing 728 Time: 0:00:04 ( 6.77 ms/it)\n objective: 13554.750402167076\nMinimizing 743 Time: 0:00:05 ( 6.77 ms/it)\n objective: 13541.189813718764\nMinimizing 758 Time: 0:00:05 ( 6.76 ms/it)\n objective: 13494.999215824471\nMinimizing 773 Time: 0:00:05 ( 6.78 ms/it)\n objective: 13492.648850050697\nMinimizing 786 Time: 0:00:05 ( 6.80 ms/it)\n objective: 13488.098061394303\nMinimizing 801 Time: 0:00:05 ( 6.80 ms/it)\n objective: 13487.32443868601\nMinimizing 816 Time: 0:00:05 ( 6.80 ms/it)\n objective: 13485.804641650837\nMinimizing 831 Time: 0:00:05 ( 6.80 ms/it)\n objective: 13484.972100949992\nMinimizing 846 Time: 0:00:05 ( 6.80 ms/it)\n objective: 13482.206646936727\nMinimizing 861 Time: 0:00:05 ( 6.80 ms/it)\n objective: 13481.957981714644\nMinimizing 876 Time: 0:00:05 ( 6.81 ms/it)\n objective: 13480.668897363736\nMinimizing 890 Time: 0:00:06 ( 6.81 ms/it)\n objective: 13477.848993872809\nMinimizing 905 Time: 0:00:06 ( 6.82 ms/it)\n objective: 13477.61909756872\nMinimizing 920 Time: 0:00:06 ( 6.82 ms/it)\n objective: 13477.08038639943\nMinimizing 935 Time: 0:00:06 ( 6.82 ms/it)\n objective: 13476.438883850664\nMinimizing 950 Time: 0:00:06 ( 6.82 ms/it)\n objective: 13475.129701195961\nMinimizing 965 Time: 0:00:06 ( 6.81 ms/it)\n objective: 13473.015547506504\nMinimizing 980 Time: 0:00:06 ( 6.81 ms/it)\n objective: 13468.897406319942\nMinimizing 996 Time: 0:00:06 ( 6.81 ms/it)\n objective: 13468.531470264665\nMinimizing 1011 Time: 0:00:06 ( 6.81 ms/it)\n objective: 13462.405574791788\nMinimizing 1026 Time: 0:00:06 ( 6.81 ms/it)\n objective: 13453.976129003247\nMinimizing 1041 Time: 0:00:07 ( 6.81 ms/it)\n objective: 13433.67933769874\nMinimizing 1056 Time: 0:00:07 ( 6.81 ms/it)\n objective: 13421.061688833033\nMinimizing 1071 Time: 0:00:07 ( 6.81 ms/it)\n objective: 13419.311719043399\nMinimizing 1086 Time: 0:00:07 ( 6.82 ms/it)\n objective: 13416.643668426506\nMinimizing 1101 Time: 0:00:07 ( 6.82 ms/it)\n objective: 13412.486559248086\nMinimizing 1116 Time: 0:00:07 ( 6.82 ms/it)\n objective: 13408.443196157146\nMinimizing 1131 Time: 0:00:07 ( 6.81 ms/it)\n objective: 13403.757208356918\nMinimizing 1147 Time: 0:00:07 ( 6.81 ms/it)\n objective: 13403.131604382761\nMinimizing 1163 Time: 0:00:07 ( 6.81 ms/it)\n objective: 13398.354195700864\nMinimizing 1178 Time: 0:00:08 ( 6.81 ms/it)\n objective: 13395.26226109868\nMinimizing 1193 Time: 0:00:08 ( 6.81 ms/it)\n objective: 13391.908013950517\nMinimizing 1208 Time: 0:00:08 ( 6.81 ms/it)\n objective: 13391.74473191856\nMinimizing 1223 Time: 0:00:08 ( 6.81 ms/it)\n objective: 13389.736211543415\nMinimizing 1238 Time: 0:00:08 ( 6.80 ms/it)\n objective: 13388.349268089878\nMinimizing 1253 Time: 0:00:08 ( 6.80 ms/it)\n objective: 13384.994787373966\nMinimizing 1268 Time: 0:00:08 ( 6.80 ms/it)\n objective: 13383.408577161244\nMinimizing 1283 Time: 0:00:08 ( 6.80 ms/it)\n objective: 13379.77084859628\nMinimizing 1298 Time: 0:00:08 ( 6.80 ms/it)\n objective: 13378.356226256132\nMinimizing 1313 Time: 0:00:08 ( 6.80 ms/it)\n objective: 13371.204840040773\nMinimizing 1328 Time: 0:00:09 ( 6.80 ms/it)\n objective: 13368.552162300708\nMinimizing 1343 Time: 0:00:09 ( 6.80 ms/it)\n objective: 13360.956304682499\nMinimizing 1358 Time: 0:00:09 ( 6.80 ms/it)\n objective: 13351.888021394683\nMinimizing 1373 Time: 0:00:09 ( 6.80 ms/it)\n objective: 13344.62582498425\nMinimizing 1388 Time: 0:00:09 ( 6.80 ms/it)\n objective: 13336.39649426828\nMinimizing 1403 Time: 0:00:09 ( 6.80 ms/it)\n objective: 13324.383979126607\nMinimizing 1418 Time: 0:00:09 ( 6.80 ms/it)\n objective: 13322.61377139554\nMinimizing 1433 Time: 0:00:09 ( 6.80 ms/it)\n objective: 13320.57700703424\nMinimizing 1448 Time: 0:00:09 ( 6.79 ms/it)\n objective: 13318.4552940086\nMinimizing 1463 Time: 0:00:09 ( 6.79 ms/it)\n objective: 13316.762270926934\nMinimizing 1478 Time: 0:00:10 ( 6.79 ms/it)\n objective: 13314.408444318266\nMinimizing 1493 Time: 0:00:10 ( 6.79 ms/it)\n objective: 13313.767416919145\nMinimizing 1509 Time: 0:00:10 ( 6.79 ms/it)\n objective: 13312.854553939309\nMinimizing 1524 Time: 0:00:10 ( 6.79 ms/it)\n objective: 13311.419052878919\nMinimizing 1539 Time: 0:00:10 ( 6.79 ms/it)\n objective: 13310.612248396574\nMinimizing 1554 Time: 0:00:10 ( 6.79 ms/it)\n objective: 13309.0929672551\nMinimizing 1569 Time: 0:00:10 ( 6.79 ms/it)\n objective: 13308.660574705282\nMinimizing 1584 Time: 0:00:10 ( 6.79 ms/it)\n objective: 13305.877100199563\nMinimizing 1600 Time: 0:00:10 ( 6.79 ms/it)\n objective: 13302.793196995844\nMinimizing 1615 Time: 0:00:10 ( 6.79 ms/it)\n objective: 13286.848302695595\nMinimizing 1630 Time: 0:00:11 ( 6.79 ms/it)\n objective: 13281.248852850098\nMinimizing 1645 Time: 0:00:11 ( 6.79 ms/it)\n objective: 13273.201509256221\nMinimizing 1660 Time: 0:00:11 ( 6.79 ms/it)\n objective: 13269.96410347159\nMinimizing 1676 Time: 0:00:11 ( 6.79 ms/it)\n objective: 13265.32337813532\nMinimizing 1691 Time: 0:00:11 ( 6.79 ms/it)\n objective: 13258.242799779066\nMinimizing 1706 Time: 0:00:11 ( 6.79 ms/it)\n objective: 13224.157705357982\nMinimizing 1722 Time: 0:00:11 ( 6.79 ms/it)\n objective: 13218.836364054543\nMinimizing 1737 Time: 0:00:11 ( 6.79 ms/it)\n objective: 13172.04866272863\nMinimizing 1752 Time: 0:00:11 ( 6.79 ms/it)\n objective: 13170.738575918338\nMinimizing 1768 Time: 0:00:11 ( 6.79 ms/it)\n objective: 13167.32485531435\nMinimizing 1783 Time: 0:00:12 ( 6.79 ms/it)\n objective: 13165.006059175896\nMinimizing 1799 Time: 0:00:12 ( 6.79 ms/it)\n objective: 13163.136445673794\nMinimizing 1814 Time: 0:00:12 ( 6.79 ms/it)\n objective: 13160.840416817664\nMinimizing 1830 Time: 0:00:12 ( 6.79 ms/it)\n objective: 13158.762434916745\nMinimizing 1845 Time: 0:00:12 ( 6.79 ms/it)\n objective: 13156.638211053461\nMinimizing 1860 Time: 0:00:12 ( 6.79 ms/it)\n objective: 13150.721932839908\nMinimizing 1875 Time: 0:00:12 ( 6.78 ms/it)\n objective: 13145.595892946672\nMinimizing 1890 Time: 0:00:12 ( 6.78 ms/it)\n objective: 13144.827830569018\nMinimizing 1905 Time: 0:00:12 ( 6.78 ms/it)\n objective: 13144.730932205304\nMinimizing 1921 Time: 0:00:13 ( 6.78 ms/it)\n objective: 13144.71833535924\nMinimizing 1936 Time: 0:00:13 ( 6.78 ms/it)\n objective: 13144.718762883233\nMinimizing 1953 Time: 0:00:13 ( 6.79 ms/it)\n objective: 13144.709759888443\nMinimizing 1970 Time: 0:00:13 ( 6.79 ms/it)\n objective: 13144.709424346802\nMinimizing 1985 Time: 0:00:13 ( 6.79 ms/it)\n objective: 13144.715370203223\nMinimizing 2000 Time: 0:00:13 ( 6.79 ms/it)\n objective: 13144.710256644277\nMinimizing 2016 Time: 0:00:13 ( 6.79 ms/it)\n objective: 13144.721392364707\nMinimizing 2032 Time: 0:00:13 ( 6.79 ms/it)\n objective: 13144.704198067135\nMinimizing 2047 Time: 0:00:13 ( 6.79 ms/it)\n objective: 13144.705960082254\nMinimizing 2063 Time: 0:00:13 ( 6.78 ms/it)\n objective: 13144.72094398216\nMinimizing 2078 Time: 0:00:14 ( 6.78 ms/it)\n objective: 13144.706341214944\nMinimizing 2093 Time: 0:00:14 ( 6.78 ms/it)\n objective: 13144.708741448092\nMinimizing 2108 Time: 0:00:14 ( 6.78 ms/it)\n objective: 13144.706141769188\nMinimizing 2123 Time: 0:00:14 ( 6.78 ms/it)\n objective: 13144.705232628257\nMinimizing 2138 Time: 0:00:14 ( 6.78 ms/it)\n objective: 13144.705378644489\nMinimizing 2153 Time: 0:00:14 ( 6.78 ms/it)\n objective: 13144.709370245488\nMinimizing 2167 Time: 0:00:14 ( 6.78 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0191\n0.0140\n-1.36\n0.1727\n0.5852\n\n\nTest: Star-Run\n-0.0130\n0.0445\n-0.29\n0.7698\n0.8496\n\n\nTest: S20-Star\n-0.0097\n0.0446\n-0.22\n0.8282\n0.6785\n\n\nTest: SLJ-S20\n0.0093\n0.0441\n0.21\n0.8331\n0.8706\n\n\nTest: BPT-SLJ\n-0.0197\n0.0424\n-0.46\n0.6422\n0.9671\n\n\na1\n0.3201\n0.0479\n6.69\n<1e-10\n\n\n\nSex: Boys\n0.2053\n0.0140\n14.69\n<1e-48\n\n\n\nTest: Star-Run & a1\n0.2958\n0.1526\n1.94\n0.0526\n\n\n\nTest: S20-Star & a1\n0.0631\n0.1543\n0.41\n0.6824\n\n\n\nTest: SLJ-S20 & a1\n-0.0902\n0.1521\n-0.59\n0.5532\n\n\n\nTest: BPT-SLJ & a1\n0.3325\n0.1421\n2.34\n0.0193\n\n\n\nTest: Star-Run & Sex: Boys\n-0.1263\n0.0445\n-2.84\n0.0046\n\n\n\nTest: S20-Star & Sex: Boys\n0.0715\n0.0446\n1.61\n0.1084\n\n\n\nTest: SLJ-S20 & Sex: Boys\n0.0426\n0.0441\n0.97\n0.3335\n\n\n\nTest: BPT-SLJ & Sex: Boys\n0.1669\n0.0424\n3.94\n<1e-04\n\n\n\na1 & Sex: Boys\n0.0444\n0.0479\n0.93\n0.3536\n\n\n\nTest: Star-Run & a1 & Sex: Boys\n-0.0142\n0.1526\n-0.09\n0.9259\n\n\n\nTest: S20-Star & a1 & Sex: Boys\n-0.0226\n0.1543\n-0.15\n0.8836\n\n\n\nTest: SLJ-S20 & a1 & Sex: Boys\n-0.0506\n0.1521\n-0.33\n0.7395\n\n\n\nTest: BPT-SLJ & a1 & Sex: Boys\n0.1397\n0.1421\n0.98\n0.3255\n\n\n\nResidual\n0.0000\n\n\n\n\n\n\n\n\n\n\nm_cpx_1_SeqDiff.PCA\n\n(Child = \nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . .\n Test: Star-Run 0.75 1.0 . . .\n Test: S20-Star 0.04 0.67 1.0 . .\n Test: SLJ-S20 -0.44 -0.19 0.42 1.0 .\n Test: BPT-SLJ -0.07 -0.24 -0.37 -0.32 1.0\n\nNormalized cumulative variances:\n[0.4315, 0.7875, 0.9292, 1.0, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5\n (Intercept) -0.47 0.46 -0.17 -0.59 0.43\n Test: Star-Run -0.66 0.14 0.2 0.06 -0.71\n Test: S20-Star -0.49 -0.41 0.45 0.33 0.53\n Test: SLJ-S20 0.03 -0.67 0.14 -0.7 -0.17\n Test: BPT-SLJ 0.31 0.39 0.84 -0.19 0.01,)", + "crumbs": [ + "Contrast coding", + "Mixed Models Tutorial: Contrast Coding" + ] + }, + { + "objectID": "contrasts_fggk21.html#pca-based-hypothesiscoding-contr4", + "href": "contrasts_fggk21.html#pca-based-hypothesiscoding-contr4", + "title": "Mixed Models Tutorial: Contrast Coding", + "section": "2.4 PCA-based HypothesisCoding: contr4", + "text": "2.4 PCA-based HypothesisCoding: contr4\nThe fourth set of contrasts uses HypothesisCoding to specify the set of contrasts implementing the loadings of the four principle components of the published LMM based on test scores, not test effects (contrasts) - coarse-grained, that is roughly according to their signs. This is actually a very interesting and plausible solution nobody had proposed a priori.\n\nPC1: BPT - Run_r\nPC2: (Star_r + S20_r + SLJ) - (BPT + Run_r)\nPC3: Star_r - (S20_r + SLJ)\nPC4: S20_r - SLJ\n\nPC1 contrasts the worst and the best indicator of physical health; PC2 contrasts these two against the core indicators of physical fitness; PC3 contrasts the cognitive and the physical tests within the narrow set of physical fitness components; and PC4, finally, contrasts two types of lower muscular fitness differing in speed and power.\n\ncontr4 = Dict(\n :School => Grouping(),\n :Child => Grouping(),\n :Cohort => Grouping(),\n :Sex => EffectsCoding(; levels=[\"Girls\", \"Boys\"]),\n :Test => HypothesisCoding(\n [\n -1 0 0 0 +1\n -3 +2 +2 +2 -3\n 0 +2 -1 -1 0\n 0 0 +1 -1 0\n ];\n levels=[\"Run\", \"Star_r\", \"S20_r\", \"SLJ\", \"BPT\"],\n labels=[\"c5.1\", \"c234.15\", \"c2.34\", \"c3.4\"],\n ),\n);\n\n\nm_cpx_1_PC = fit(MixedModel, f_cpx_1, dat; contrasts=contr4)\n\nMinimizing 15 Time: 0:00:00 ( 6.88 ms/it)\n objective: 13816.332602330232\nMinimizing 30 Time: 0:00:00 ( 6.82 ms/it)\n objective: 13819.656571653324\nMinimizing 45 Time: 0:00:00 ( 6.79 ms/it)\n objective: 13791.585455961931\nMinimizing 60 Time: 0:00:00 ( 6.77 ms/it)\n objective: 13780.67571280646\nMinimizing 75 Time: 0:00:00 ( 6.78 ms/it)\n objective: 13773.874746283942\nMinimizing 90 Time: 0:00:00 ( 6.78 ms/it)\n objective: 13772.086478567438\nMinimizing 105 Time: 0:00:00 ( 6.80 ms/it)\n objective: 13764.66194285501\nMinimizing 120 Time: 0:00:00 ( 6.81 ms/it)\n objective: 13758.522570130917\nMinimizing 135 Time: 0:00:00 ( 6.81 ms/it)\n objective: 13744.732814488541\nMinimizing 150 Time: 0:00:01 ( 6.81 ms/it)\n objective: 13733.668211491575\nMinimizing 165 Time: 0:00:01 ( 6.81 ms/it)\n objective: 13732.36340006553\nMinimizing 180 Time: 0:00:01 ( 6.82 ms/it)\n objective: 13728.264261401562\nMinimizing 195 Time: 0:00:01 ( 6.81 ms/it)\n objective: 13723.675896663455\nMinimizing 210 Time: 0:00:01 ( 6.81 ms/it)\n objective: 13708.984335873063\nMinimizing 226 Time: 0:00:01 ( 6.80 ms/it)\n objective: 13706.154390410527\nMinimizing 241 Time: 0:00:01 ( 6.79 ms/it)\n objective: 13695.813163832274\nMinimizing 256 Time: 0:00:01 ( 6.79 ms/it)\n objective: 13687.171487634814\nMinimizing 271 Time: 0:00:01 ( 6.79 ms/it)\n objective: 13681.930135484894\nMinimizing 287 Time: 0:00:01 ( 6.78 ms/it)\n objective: 13667.212654789928\nMinimizing 303 Time: 0:00:02 ( 6.78 ms/it)\n objective: 13661.137116128333\nMinimizing 318 Time: 0:00:02 ( 6.78 ms/it)\n objective: 13648.711313588408\nMinimizing 333 Time: 0:00:02 ( 6.78 ms/it)\n objective: 13640.226573704029\nMinimizing 348 Time: 0:00:02 ( 6.79 ms/it)\n objective: 13617.082625055522\nMinimizing 363 Time: 0:00:02 ( 6.78 ms/it)\n objective: 13615.026428660676\nMinimizing 378 Time: 0:00:02 ( 6.78 ms/it)\n objective: 13610.071531565009\nMinimizing 393 Time: 0:00:02 ( 6.78 ms/it)\n objective: 13606.914276340576\nMinimizing 409 Time: 0:00:02 ( 6.78 ms/it)\n objective: 13601.226598413992\nMinimizing 424 Time: 0:00:02 ( 6.77 ms/it)\n objective: 13592.804836582302\nMinimizing 439 Time: 0:00:02 ( 6.77 ms/it)\n objective: 13568.28721410715\nMinimizing 454 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13534.056522645864\nMinimizing 469 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13529.915286408752\nMinimizing 485 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13514.271096737903\nMinimizing 500 Time: 0:00:03 ( 6.76 ms/it)\n objective: 13509.704795192432\nMinimizing 515 Time: 0:00:03 ( 6.76 ms/it)\n objective: 13502.135581336523\nMinimizing 530 Time: 0:00:03 ( 6.76 ms/it)\n objective: 13506.742422807103\nMinimizing 545 Time: 0:00:03 ( 6.76 ms/it)\n objective: 13463.385223559373\nMinimizing 560 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13462.810806506597\nMinimizing 575 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13456.78806683152\nMinimizing 590 Time: 0:00:03 ( 6.77 ms/it)\n objective: 13454.98161661686\nMinimizing 605 Time: 0:00:04 ( 6.77 ms/it)\n objective: 13452.632935365473\nMinimizing 620 Time: 0:00:04 ( 6.77 ms/it)\n objective: 13449.7338857047\nMinimizing 635 Time: 0:00:04 ( 6.78 ms/it)\n objective: 13446.226209510103\nMinimizing 649 Time: 0:00:04 ( 6.79 ms/it)\n objective: 13443.323829727764\nMinimizing 664 Time: 0:00:04 ( 6.80 ms/it)\n objective: 13440.292480622637\nMinimizing 678 Time: 0:00:04 ( 6.81 ms/it)\n objective: 13430.352151089013\nMinimizing 693 Time: 0:00:04 ( 6.81 ms/it)\n objective: 13426.67388857832\nMinimizing 708 Time: 0:00:04 ( 6.82 ms/it)\n objective: 13424.94699773231\nMinimizing 723 Time: 0:00:04 ( 6.83 ms/it)\n objective: 13424.095826991637\nMinimizing 738 Time: 0:00:05 ( 6.83 ms/it)\n objective: 13423.12578855658\nMinimizing 753 Time: 0:00:05 ( 6.82 ms/it)\n objective: 13422.366024823343\nMinimizing 768 Time: 0:00:05 ( 6.82 ms/it)\n objective: 13418.14667611393\nMinimizing 783 Time: 0:00:05 ( 6.82 ms/it)\n objective: 13416.68757398241\nMinimizing 798 Time: 0:00:05 ( 6.82 ms/it)\n objective: 13410.005246821544\nMinimizing 813 Time: 0:00:05 ( 6.82 ms/it)\n objective: 13397.903233037963\nMinimizing 828 Time: 0:00:05 ( 6.82 ms/it)\n objective: 13382.120781459795\nMinimizing 843 Time: 0:00:05 ( 6.82 ms/it)\n objective: 13366.505614135458\nMinimizing 858 Time: 0:00:05 ( 6.82 ms/it)\n objective: 13364.025405890228\nMinimizing 873 Time: 0:00:05 ( 6.82 ms/it)\n objective: 13344.389366420699\nMinimizing 888 Time: 0:00:06 ( 6.83 ms/it)\n objective: 13344.033429230505\nMinimizing 903 Time: 0:00:06 ( 6.83 ms/it)\n objective: 13340.611014517257\nMinimizing 918 Time: 0:00:06 ( 6.83 ms/it)\n objective: 13338.890144004705\nMinimizing 933 Time: 0:00:06 ( 6.83 ms/it)\n objective: 13336.292964622058\nMinimizing 948 Time: 0:00:06 ( 6.84 ms/it)\n objective: 13334.410375687352\nMinimizing 963 Time: 0:00:06 ( 6.84 ms/it)\n objective: 13330.7964364582\nMinimizing 977 Time: 0:00:06 ( 6.85 ms/it)\n objective: 13327.077580748039\nMinimizing 992 Time: 0:00:06 ( 6.85 ms/it)\n objective: 13325.67516979968\nMinimizing 1006 Time: 0:00:06 ( 6.86 ms/it)\n objective: 13323.546768697706\nMinimizing 1020 Time: 0:00:07 ( 6.86 ms/it)\n objective: 13320.731071181115\nMinimizing 1034 Time: 0:00:07 ( 6.87 ms/it)\n objective: 13317.376753146702\nMinimizing 1048 Time: 0:00:07 ( 6.88 ms/it)\n objective: 13316.511893543866\nMinimizing 1062 Time: 0:00:07 ( 6.89 ms/it)\n objective: 13316.011429793944\nMinimizing 1075 Time: 0:00:07 ( 6.90 ms/it)\n objective: 13312.653803260066\nMinimizing 1089 Time: 0:00:07 ( 6.92 ms/it)\n objective: 13311.798952526675\nMinimizing 1103 Time: 0:00:07 ( 6.92 ms/it)\n objective: 13311.153731812388\nMinimizing 1117 Time: 0:00:07 ( 6.93 ms/it)\n objective: 13309.648125015112\nMinimizing 1131 Time: 0:00:07 ( 6.94 ms/it)\n objective: 13309.426799283116\nMinimizing 1144 Time: 0:00:07 ( 6.95 ms/it)\n objective: 13307.433029087246\nMinimizing 1159 Time: 0:00:08 ( 6.95 ms/it)\n objective: 13301.755871152942\nMinimizing 1174 Time: 0:00:08 ( 6.95 ms/it)\n objective: 13294.200399732013\nMinimizing 1189 Time: 0:00:08 ( 6.95 ms/it)\n objective: 13290.387146921057\nMinimizing 1204 Time: 0:00:08 ( 6.95 ms/it)\n objective: 13287.385817066097\nMinimizing 1219 Time: 0:00:08 ( 6.94 ms/it)\n objective: 13279.505459288368\nMinimizing 1234 Time: 0:00:08 ( 6.94 ms/it)\n objective: 13273.29319327083\nMinimizing 1249 Time: 0:00:08 ( 6.94 ms/it)\n objective: 13260.459047355165\nMinimizing 1264 Time: 0:00:08 ( 6.94 ms/it)\n objective: 13239.77648263349\nMinimizing 1279 Time: 0:00:08 ( 6.94 ms/it)\n objective: 13232.372454307391\nMinimizing 1294 Time: 0:00:08 ( 6.94 ms/it)\n objective: 13219.20806476663\nMinimizing 1309 Time: 0:00:09 ( 6.94 ms/it)\n objective: 13207.397301005229\nMinimizing 1324 Time: 0:00:09 ( 6.94 ms/it)\n objective: 13207.04314413754\nMinimizing 1339 Time: 0:00:09 ( 6.94 ms/it)\n objective: 13199.883679693623\nMinimizing 1355 Time: 0:00:09 ( 6.95 ms/it)\n objective: 13196.533831483379\nMinimizing 1371 Time: 0:00:09 ( 6.95 ms/it)\n objective: 13190.483032705422\nMinimizing 1385 Time: 0:00:09 ( 6.95 ms/it)\n objective: 13183.917344881687\nMinimizing 1400 Time: 0:00:09 ( 6.96 ms/it)\n objective: 13180.88049893771\nMinimizing 1415 Time: 0:00:09 ( 6.96 ms/it)\n objective: 13172.490587439534\nMinimizing 1430 Time: 0:00:09 ( 6.95 ms/it)\n objective: 13158.24100738352\nMinimizing 1445 Time: 0:00:10 ( 6.95 ms/it)\n objective: 13157.580295724736\nMinimizing 1460 Time: 0:00:10 ( 6.95 ms/it)\n objective: 13156.772916139555\nMinimizing 1475 Time: 0:00:10 ( 6.95 ms/it)\n objective: 13155.054826345557\nMinimizing 1490 Time: 0:00:10 ( 6.95 ms/it)\n objective: 13150.94076273813\nMinimizing 1505 Time: 0:00:10 ( 6.95 ms/it)\n objective: 13147.509740719834\nMinimizing 1520 Time: 0:00:10 ( 6.95 ms/it)\n objective: 13140.819616088833\nMinimizing 1535 Time: 0:00:10 ( 6.95 ms/it)\n objective: 13137.350813194833\nMinimizing 1550 Time: 0:00:10 ( 6.95 ms/it)\n objective: 13136.919053304591\nMinimizing 1565 Time: 0:00:10 ( 6.95 ms/it)\n objective: 13134.483498100337\nMinimizing 1580 Time: 0:00:10 ( 6.95 ms/it)\n objective: 13134.447373534378\nMinimizing 1595 Time: 0:00:11 ( 6.95 ms/it)\n objective: 13134.431338372917\nMinimizing 1610 Time: 0:00:11 ( 6.95 ms/it)\n objective: 13134.429919027709\nMinimizing 1625 Time: 0:00:11 ( 6.95 ms/it)\n objective: 13134.432391109323\nMinimizing 1640 Time: 0:00:11 ( 6.95 ms/it)\n objective: 13134.434131062284\nMinimizing 1655 Time: 0:00:11 ( 6.95 ms/it)\n objective: 13134.43136893095\nMinimizing 1670 Time: 0:00:11 ( 6.95 ms/it)\n objective: 13134.434588496297\nMinimizing 1685 Time: 0:00:11 ( 6.95 ms/it)\n objective: 13134.432497580332\nMinimizing 1700 Time: 0:00:11 ( 6.95 ms/it)\n objective: 13134.42282214512\nMinimizing 1715 Time: 0:00:11 ( 6.95 ms/it)\n objective: 13134.435660578849\nMinimizing 1730 Time: 0:00:12 ( 6.95 ms/it)\n objective: 13134.431615684036\nMinimizing 1743 Time: 0:00:12 ( 6.97 ms/it)\n objective: 13134.437980213028\nMinimizing 1757 Time: 0:00:12 ( 6.97 ms/it)\n objective: 13134.43474466115\nMinimizing 1772 Time: 0:00:12 ( 6.97 ms/it)\n objective: 13134.43327911022\nMinimizing 1788 Time: 0:00:12 ( 6.96 ms/it)\n objective: 13134.427159230705\nMinimizing 1799 Time: 0:00:12 ( 6.96 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0182\n0.0140\n-1.30\n0.1942\n0.6870\n\n\nTest: c5.1\n-0.0403\n0.0429\n-0.94\n0.3481\n1.3839\n\n\nTest: c234.15\n0.0162\n0.1678\n0.10\n0.9231\n0.6312\n\n\nTest: c2.34\n0.0099\n0.0758\n0.13\n0.8958\n0.9852\n\n\nTest: c3.4\n-0.0137\n0.0445\n-0.31\n0.7584\n1.5296\n\n\na1\n0.3207\n0.0480\n6.67\n<1e-10\n\n\n\nSex: Boys\n0.2052\n0.0140\n14.63\n<1e-47\n\n\n\nTest: c5.1 & a1\n0.6160\n0.1469\n4.19\n<1e-04\n\n\n\nTest: c234.15 & a1\n0.0641\n0.5728\n0.11\n0.9108\n\n\n\nTest: c2.34 & a1\n-0.0629\n0.2584\n-0.24\n0.8077\n\n\n\nTest: c3.4 & a1\n0.0742\n0.1533\n0.48\n0.6284\n\n\n\nTest: c5.1 & Sex: Boys\n0.1573\n0.0429\n3.67\n0.0002\n\n\n\nTest: c234.15 & Sex: Boys\n-0.8202\n0.1678\n-4.89\n<1e-05\n\n\n\nTest: c2.34 & Sex: Boys\n-0.1837\n0.0758\n-2.42\n0.0154\n\n\n\nTest: c3.4 & Sex: Boys\n-0.0465\n0.0445\n-1.04\n0.2964\n\n\n\na1 & Sex: Boys\n0.0508\n0.0480\n1.06\n0.2908\n\n\n\nTest: c5.1 & a1 & Sex: Boys\n0.0384\n0.1469\n0.26\n0.7937\n\n\n\nTest: c234.15 & a1 & Sex: Boys\n-0.3568\n0.5728\n-0.62\n0.5333\n\n\n\nTest: c2.34 & a1 & Sex: Boys\n0.1448\n0.2584\n0.56\n0.5752\n\n\n\nTest: c3.4 & a1 & Sex: Boys\n0.0608\n0.1533\n0.40\n0.6916\n\n\n\nResidual\n0.0000\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_cpx_1_PC)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\nChild\n(Intercept)\n0.47198187\n0.68700937\n\n\n\n\n\n\n\nTest: c5.1\n1.91517989\n1.38390025\n-0.11\n\n\n\n\n\n\nTest: c234.15\n0.39840370\n0.63119228\n+0.83\n-0.48\n\n\n\n\n\nTest: c2.34\n0.97057584\n0.98517807\n+0.69\n-0.15\n+0.88\n\n\n\n\nTest: c3.4\n2.33977540\n1.52963244\n+0.12\n+0.08\n+0.40\n+0.63\n\n\nResidual\n\n0.00000000\n0.00001040\n\n\n\n\n\n\n\n\n\n\nm_cpx_1_PC.PCA\n\n(Child = \nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . .\n Test: c5.1 -0.11 1.0 . . .\n Test: c234.15 0.83 -0.48 1.0 . .\n Test: c2.34 0.69 -0.15 0.88 1.0 .\n Test: c3.4 0.12 0.08 0.4 0.63 1.0\n\nNormalized cumulative variances:\n[0.5849, 0.8176, 0.9767, 1.0, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5\n (Intercept) -0.48 -0.12 0.61 0.5 -0.38\n Test: c5.1 0.2 0.73 0.57 -0.1 0.3\n Test: c234.15 -0.57 -0.18 0.02 -0.0 0.8\n Test: c2.34 -0.55 0.2 -0.01 -0.73 -0.35\n Test: c3.4 -0.32 0.61 -0.55 0.46 -0.08,)\n\n\nThere is a numerical interaction with a z-value > 2.0 for the first PCA (i.e., BPT - Run_r). This interaction would really need to be replicated to be taken seriously. It is probably due to larger “unfitness” gains in boys than girls (i.e., in BPT) relative to the slightly larger health-related “fitness” gains of girls than boys (i.e., in Run_r).\n\ncontr4b = merge(\n Dict(nm => Grouping() for nm in (:School, :Child, :Cohort)),\n Dict(\n :Sex => EffectsCoding(; levels=[\"Girls\", \"Boys\"]),\n :Test => HypothesisCoding(\n [\n 0.49 -0.04 0.20 0.03 -0.85\n 0.70 -0.56 -0.21 -0.13 0.37\n 0.31 0.68 -0.56 -0.35 0.00\n 0.04 0.08 0.61 -0.78 0.13\n ];\n levels=[\"Run\", \"Star_r\", \"S20_r\", \"SLJ\", \"BPT\"],\n labels=[\"c5.1\", \"c234.15\", \"c12.34\", \"c3.4\"],\n ),\n ),\n);\n\n\nm_cpx_1_PC_2 = fit(MixedModel, f_cpx_1, dat; contrasts=contr4b)\n\nMinimizing 15 Time: 0:00:00 ( 6.95 ms/it)\n objective: 13850.24151055307\nMinimizing 30 Time: 0:00:00 ( 6.95 ms/it)\n objective: 13838.10560061899\nMinimizing 45 Time: 0:00:00 ( 6.94 ms/it)\n objective: 13780.927582361033\nMinimizing 60 Time: 0:00:00 ( 6.94 ms/it)\n objective: 13772.847096673908\nMinimizing 75 Time: 0:00:00 ( 6.92 ms/it)\n objective: 13761.023906063292\nMinimizing 90 Time: 0:00:00 ( 6.92 ms/it)\n objective: 13712.810888035232\nMinimizing 105 Time: 0:00:00 ( 6.94 ms/it)\n objective: 13703.180631551688\nMinimizing 120 Time: 0:00:00 ( 6.95 ms/it)\n objective: 13692.380747939407\nMinimizing 135 Time: 0:00:00 ( 6.95 ms/it)\n objective: 13684.220472205616\nMinimizing 150 Time: 0:00:01 ( 6.94 ms/it)\n objective: 13677.886588077183\nMinimizing 165 Time: 0:00:01 ( 6.92 ms/it)\n objective: 13671.394614708945\nMinimizing 180 Time: 0:00:01 ( 6.91 ms/it)\n objective: 13660.661746517948\nMinimizing 195 Time: 0:00:01 ( 6.91 ms/it)\n objective: 13636.166198863193\nMinimizing 210 Time: 0:00:01 ( 6.91 ms/it)\n objective: 13630.469487943494\nMinimizing 225 Time: 0:00:01 ( 6.91 ms/it)\n objective: 13627.278645578153\nMinimizing 240 Time: 0:00:01 ( 6.90 ms/it)\n objective: 13620.449884102767\nMinimizing 255 Time: 0:00:01 ( 6.90 ms/it)\n objective: 13610.459682344223\nMinimizing 270 Time: 0:00:01 ( 6.89 ms/it)\n objective: 13598.363790982221\nMinimizing 285 Time: 0:00:01 ( 6.89 ms/it)\n objective: 13596.620097651725\nMinimizing 300 Time: 0:00:02 ( 6.89 ms/it)\n objective: 13591.281476002816\nMinimizing 315 Time: 0:00:02 ( 6.88 ms/it)\n objective: 13584.031821209064\nMinimizing 330 Time: 0:00:02 ( 6.88 ms/it)\n objective: 13574.017253943832\nMinimizing 345 Time: 0:00:02 ( 6.88 ms/it)\n objective: 13555.855304774563\nMinimizing 360 Time: 0:00:02 ( 6.88 ms/it)\n objective: 13552.806626147372\nMinimizing 375 Time: 0:00:02 ( 6.88 ms/it)\n objective: 13547.092330697484\nMinimizing 390 Time: 0:00:02 ( 6.88 ms/it)\n objective: 13532.911687696724\nMinimizing 405 Time: 0:00:02 ( 6.88 ms/it)\n objective: 13520.872533328955\nMinimizing 420 Time: 0:00:02 ( 6.89 ms/it)\n objective: 13518.365460546862\nMinimizing 435 Time: 0:00:02 ( 6.89 ms/it)\n objective: 13515.797058975942\nMinimizing 450 Time: 0:00:03 ( 6.89 ms/it)\n objective: 13514.065460224185\nMinimizing 465 Time: 0:00:03 ( 6.89 ms/it)\n objective: 13502.387613892985\nMinimizing 480 Time: 0:00:03 ( 6.89 ms/it)\n objective: 13501.371486198099\nMinimizing 495 Time: 0:00:03 ( 6.89 ms/it)\n objective: 13495.650745079183\nMinimizing 510 Time: 0:00:03 ( 6.89 ms/it)\n objective: 13493.574736526003\nMinimizing 525 Time: 0:00:03 ( 6.89 ms/it)\n objective: 13487.24491238776\nMinimizing 540 Time: 0:00:03 ( 6.89 ms/it)\n objective: 13479.970415348675\nMinimizing 555 Time: 0:00:03 ( 6.89 ms/it)\n objective: 13467.40530566145\nMinimizing 570 Time: 0:00:03 ( 6.89 ms/it)\n objective: 13466.37840979021\nMinimizing 585 Time: 0:00:04 ( 6.88 ms/it)\n objective: 13444.808014244067\nMinimizing 600 Time: 0:00:04 ( 6.88 ms/it)\n objective: 13416.153199135573\nMinimizing 615 Time: 0:00:04 ( 6.88 ms/it)\n objective: 13412.752158354255\nMinimizing 630 Time: 0:00:04 ( 6.88 ms/it)\n objective: 13403.054644450516\nMinimizing 645 Time: 0:00:04 ( 6.88 ms/it)\n objective: 13401.161340292405\nMinimizing 660 Time: 0:00:04 ( 6.88 ms/it)\n objective: 13399.522092879924\nMinimizing 675 Time: 0:00:04 ( 6.88 ms/it)\n objective: 13395.327109975005\nMinimizing 690 Time: 0:00:04 ( 6.88 ms/it)\n objective: 13391.840157092141\nMinimizing 705 Time: 0:00:04 ( 6.88 ms/it)\n objective: 13387.823883136785\nMinimizing 720 Time: 0:00:04 ( 6.88 ms/it)\n objective: 13384.034588419112\nMinimizing 737 Time: 0:00:05 ( 6.90 ms/it)\n objective: 13376.996328404828\nMinimizing 754 Time: 0:00:05 ( 6.89 ms/it)\n objective: 13366.136746271266\nMinimizing 769 Time: 0:00:05 ( 6.89 ms/it)\n objective: 13331.19146440213\nMinimizing 784 Time: 0:00:05 ( 6.89 ms/it)\n objective: 13322.826223245342\nMinimizing 799 Time: 0:00:05 ( 6.89 ms/it)\n objective: 13294.31975397146\nMinimizing 814 Time: 0:00:05 ( 6.89 ms/it)\n objective: 13291.596260652514\nMinimizing 829 Time: 0:00:05 ( 6.88 ms/it)\n objective: 13279.664025394144\nMinimizing 844 Time: 0:00:05 ( 6.88 ms/it)\n objective: 13278.088882631753\nMinimizing 859 Time: 0:00:05 ( 6.88 ms/it)\n objective: 13276.429779799728\nMinimizing 874 Time: 0:00:06 ( 6.88 ms/it)\n objective: 13275.05837348754\nMinimizing 889 Time: 0:00:06 ( 6.88 ms/it)\n objective: 13268.619728254474\nMinimizing 904 Time: 0:00:06 ( 6.88 ms/it)\n objective: 13267.842004174803\nMinimizing 919 Time: 0:00:06 ( 6.88 ms/it)\n objective: 13262.838248273736\nMinimizing 934 Time: 0:00:06 ( 6.88 ms/it)\n objective: 13260.90469525801\nMinimizing 949 Time: 0:00:06 ( 6.88 ms/it)\n objective: 13260.588538399563\nMinimizing 964 Time: 0:00:06 ( 6.88 ms/it)\n objective: 13260.523629822099\nMinimizing 979 Time: 0:00:06 ( 6.88 ms/it)\n objective: 13259.208036389507\nMinimizing 993 Time: 0:00:06 ( 6.89 ms/it)\n objective: 13258.201803771197\nMinimizing 1007 Time: 0:00:06 ( 6.90 ms/it)\n objective: 13256.83947944238\nMinimizing 1021 Time: 0:00:07 ( 6.91 ms/it)\n objective: 13256.705809842271\nMinimizing 1035 Time: 0:00:07 ( 6.91 ms/it)\n objective: 13255.863057219874\nMinimizing 1049 Time: 0:00:07 ( 6.92 ms/it)\n objective: 13255.736158535015\nMinimizing 1063 Time: 0:00:07 ( 6.92 ms/it)\n objective: 13255.63587310417\nMinimizing 1077 Time: 0:00:07 ( 6.93 ms/it)\n objective: 13255.519203594042\nMinimizing 1091 Time: 0:00:07 ( 6.93 ms/it)\n objective: 13255.389298437833\nMinimizing 1105 Time: 0:00:07 ( 6.93 ms/it)\n objective: 13255.118205318227\nMinimizing 1119 Time: 0:00:07 ( 6.94 ms/it)\n objective: 13255.088864998266\nMinimizing 1133 Time: 0:00:07 ( 6.94 ms/it)\n objective: 13255.062787202129\nMinimizing 1147 Time: 0:00:07 ( 6.95 ms/it)\n objective: 13255.061423943407\nMinimizing 1161 Time: 0:00:08 ( 6.95 ms/it)\n objective: 13255.056097710563\nMinimizing 1175 Time: 0:00:08 ( 6.95 ms/it)\n objective: 13255.056234650416\nMinimizing 1190 Time: 0:00:08 ( 6.96 ms/it)\n objective: 13255.055002824854\nMinimizing 1204 Time: 0:00:08 ( 6.96 ms/it)\n objective: 13255.054838904995\nMinimizing 1218 Time: 0:00:08 ( 6.97 ms/it)\n objective: 13255.05491801395\nMinimizing 1232 Time: 0:00:08 ( 6.98 ms/it)\n objective: 13255.054799779056\nMinimizing 1246 Time: 0:00:08 ( 6.98 ms/it)\n objective: 13255.054791186776\nMinimizing 1259 Time: 0:00:08 ( 6.99 ms/it)\n objective: 13255.054784246706\nMinimizing 1273 Time: 0:00:08 ( 7.00 ms/it)\n objective: 13255.054746274604\nMinimizing 1287 Time: 0:00:09 ( 7.00 ms/it)\n objective: 13255.054704167065\nMinimizing 1301 Time: 0:00:09 ( 7.01 ms/it)\n objective: 13255.054710118318\nMinimizing 1315 Time: 0:00:09 ( 7.02 ms/it)\n objective: 13255.054771622381\nMinimizing 1329 Time: 0:00:09 ( 7.02 ms/it)\n objective: 13255.05471708742\nMinimizing 1343 Time: 0:00:09 ( 7.02 ms/it)\n objective: 13255.054834318915\nMinimizing 1357 Time: 0:00:09 ( 7.03 ms/it)\n objective: 13255.054715139035\nMinimizing 1371 Time: 0:00:09 ( 7.03 ms/it)\n objective: 13255.054669005374\nMinimizing 1385 Time: 0:00:09 ( 7.04 ms/it)\n objective: 13255.054763583583\nMinimizing 1399 Time: 0:00:09 ( 7.04 ms/it)\n objective: 13255.054697954416\nMinimizing 1403 Time: 0:00:09 ( 7.04 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0159\n0.0141\n-1.13\n0.2594\n0.6435\n\n\nTest: c5.1\n0.0245\n0.0301\n0.81\n0.4160\n0.9323\n\n\nTest: c234.15\n0.0059\n0.0312\n0.19\n0.8488\n0.5608\n\n\nTest: c12.34\n0.0013\n0.0285\n0.04\n0.9643\n0.4152\n\n\nTest: c3.4\n-0.0060\n0.0313\n-0.19\n0.8482\n1.1259\n\n\na1\n0.3125\n0.0482\n6.49\n<1e-10\n\n\n\nSex: Boys\n0.1973\n0.0141\n14.03\n<1e-43\n\n\n\nTest: c5.1 & a1\n-0.3806\n0.1023\n-3.72\n0.0002\n\n\n\nTest: c234.15 & a1\n-0.1120\n0.1065\n-1.05\n0.2927\n\n\n\nTest: c12.34 & a1\n-0.1509\n0.0985\n-1.53\n0.1256\n\n\n\nTest: c3.4 & a1\n0.0659\n0.1071\n0.62\n0.5385\n\n\n\nTest: c5.1 & Sex: Boys\n-0.1415\n0.0301\n-4.70\n<1e-05\n\n\n\nTest: c234.15 & Sex: Boys\n0.1375\n0.0312\n4.41\n<1e-04\n\n\n\nTest: c12.34 & Sex: Boys\n-0.0572\n0.0285\n-2.01\n0.0448\n\n\n\nTest: c3.4 & Sex: Boys\n-0.0171\n0.0313\n-0.54\n0.5863\n\n\n\na1 & Sex: Boys\n0.0491\n0.0482\n1.02\n0.3085\n\n\n\nTest: c5.1 & a1 & Sex: Boys\n-0.0498\n0.1023\n-0.49\n0.6263\n\n\n\nTest: c234.15 & a1 & Sex: Boys\n0.0510\n0.1065\n0.48\n0.6317\n\n\n\nTest: c12.34 & a1 & Sex: Boys\n0.0235\n0.0985\n0.24\n0.8118\n\n\n\nTest: c3.4 & a1 & Sex: Boys\n0.0716\n0.1071\n0.67\n0.5040\n\n\n\nResidual\n0.0001\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_cpx_1_PC_2)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\nChild\n(Intercept)\n0.414151310\n0.643545888\n\n\n\n\n\n\n\nTest: c5.1\n0.869173571\n0.932294788\n+0.26\n\n\n\n\n\n\nTest: c234.15\n0.314451282\n0.560759558\n-0.08\n-0.41\n\n\n\n\n\nTest: c12.34\n0.172427319\n0.415243686\n+0.93\n+0.54\n-0.04\n\n\n\n\nTest: c3.4\n1.267608783\n1.125881336\n-0.07\n+0.26\n-0.97\n-0.15\n\n\nResidual\n\n0.000000010\n0.000101193\n\n\n\n\n\n\n\n\n\n\nm_cpx_1_PC_2.PCA\n\n(Child = \nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . .\n Test: c5.1 0.26 1.0 . . .\n Test: c234.15 -0.08 -0.41 1.0 . .\n Test: c12.34 0.93 0.54 -0.04 1.0 .\n Test: c3.4 -0.07 0.26 -0.97 -0.15 1.0\n\nNormalized cumulative variances:\n[0.4658, 0.8678, 0.9987, 1.0, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5\n (Intercept) -0.45 0.43 0.49 0.03 0.61\n Test: c5.1 -0.49 0.01 -0.82 -0.05 0.29\n Test: c234.15 0.45 0.51 -0.17 -0.7 0.14\n Test: c12.34 -0.49 0.47 0.05 -0.13 -0.72\n Test: c3.4 -0.34 -0.58 0.24 -0.7 -0.0,)\n\n\n\nf_zcp_1 = @formula(zScore ~ 1 + Test*a1*Sex + zerocorr(1 + Test | Child))\nm_zcp_1_PC_2 = fit(MixedModel, f_zcp_1, dat; contrasts=contr4b)\n\nMinimizing 21 Time: 0:00:00 ( 4.89 ms/it)\n objective: 13783.233343890173\nMinimizing 42 Time: 0:00:00 ( 4.91 ms/it)\n objective: 13681.464929881493\nMinimizing 62 Time: 0:00:00 ( 4.96 ms/it)\n objective: 13668.548336113578\nMinimizing 83 Time: 0:00:00 ( 4.95 ms/it)\n objective: 13661.29491583141\nMinimizing 104 Time: 0:00:00 ( 4.95 ms/it)\n objective: 13639.038750316176\nMinimizing 125 Time: 0:00:00 ( 4.94 ms/it)\n objective: 13625.056668479876\nMinimizing 145 Time: 0:00:00 ( 4.95 ms/it)\n objective: 13621.198908831146\nMinimizing 166 Time: 0:00:00 ( 4.95 ms/it)\n objective: 13611.319217932381\nMinimizing 187 Time: 0:00:00 ( 4.95 ms/it)\n objective: 13608.36643724149\nMinimizing 208 Time: 0:00:01 ( 4.95 ms/it)\n objective: 13597.737356090172\nMinimizing 229 Time: 0:00:01 ( 4.95 ms/it)\n objective: 13546.149390347673\nMinimizing 250 Time: 0:00:01 ( 4.95 ms/it)\n objective: 13508.091954147378\nMinimizing 271 Time: 0:00:01 ( 4.94 ms/it)\n objective: 13448.258506957267\nMinimizing 292 Time: 0:00:01 ( 4.93 ms/it)\n objective: 13441.56761418299\nMinimizing 313 Time: 0:00:01 ( 4.93 ms/it)\n objective: 13438.392783886804\nMinimizing 334 Time: 0:00:01 ( 4.93 ms/it)\n objective: 13432.59101495325\nMinimizing 355 Time: 0:00:01 ( 4.93 ms/it)\n objective: 13423.460288299873\nMinimizing 377 Time: 0:00:01 ( 4.92 ms/it)\n objective: 13354.965375328546\nMinimizing 398 Time: 0:00:01 ( 4.92 ms/it)\n objective: 13266.235515146866\nMinimizing 419 Time: 0:00:02 ( 4.92 ms/it)\n objective: 13225.004241497227\nMinimizing 440 Time: 0:00:02 ( 4.92 ms/it)\n objective: 13143.428498894005\nMinimizing 461 Time: 0:00:02 ( 4.92 ms/it)\n objective: 13091.28040350812\nMinimizing 482 Time: 0:00:02 ( 4.91 ms/it)\n objective: 13058.26570571358\nMinimizing 505 Time: 0:00:02 ( 4.94 ms/it)\n objective: 13005.383286547469\nMinimizing 526 Time: 0:00:02 ( 4.94 ms/it)\n objective: 13005.287061735376\nMinimizing 547 Time: 0:00:02 ( 4.93 ms/it)\n objective: 13007.14799122524\nMinimizing 568 Time: 0:00:02 ( 4.93 ms/it)\n objective: 13006.541079391318\nMinimizing 589 Time: 0:00:02 ( 4.93 ms/it)\n objective: 13004.64820935615\nMinimizing 607 Time: 0:00:02 ( 4.93 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0172\n0.0142\n-1.21\n0.2248\n0.7313\n\n\nTest: c5.1\n0.0226\n0.0305\n0.74\n0.4590\n0.6944\n\n\nTest: c234.15\n0.0030\n0.0304\n0.10\n0.9219\n0.3699\n\n\nTest: c12.34\n0.0103\n0.0318\n0.33\n0.7445\n0.9437\n\n\nTest: c3.4\n-0.0084\n0.0317\n-0.26\n0.7916\n0.7067\n\n\na1\n0.3129\n0.0488\n6.42\n<1e-09\n\n\n\nSex: Boys\n0.1966\n0.0142\n13.84\n<1e-42\n\n\n\nTest: c5.1 & a1\n-0.3817\n0.1032\n-3.70\n0.0002\n\n\n\nTest: c234.15 & a1\n-0.1079\n0.1038\n-1.04\n0.2989\n\n\n\nTest: c12.34 & a1\n-0.1352\n0.1094\n-1.24\n0.2168\n\n\n\nTest: c3.4 & a1\n0.0858\n0.1085\n0.79\n0.4290\n\n\n\nTest: c5.1 & Sex: Boys\n-0.1458\n0.0305\n-4.78\n<1e-05\n\n\n\nTest: c234.15 & Sex: Boys\n0.1359\n0.0304\n4.47\n<1e-05\n\n\n\nTest: c12.34 & Sex: Boys\n-0.0368\n0.0318\n-1.16\n0.2470\n\n\n\nTest: c3.4 & Sex: Boys\n-0.0188\n0.0317\n-0.59\n0.5545\n\n\n\na1 & Sex: Boys\n0.0453\n0.0488\n0.93\n0.3530\n\n\n\nTest: c5.1 & a1 & Sex: Boys\n-0.0476\n0.1032\n-0.46\n0.6448\n\n\n\nTest: c234.15 & a1 & Sex: Boys\n0.0387\n0.1038\n0.37\n0.7095\n\n\n\nTest: c12.34 & a1 & Sex: Boys\n0.0534\n0.1094\n0.49\n0.6259\n\n\n\nTest: c3.4 & a1 & Sex: Boys\n0.0690\n0.1085\n0.64\n0.5250\n\n\n\nResidual\n0.0000\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_zcp_1_PC_2)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\nChild\n(Intercept)\n0.53475523\n0.73126960\n\n\n\n\n\n\n\nTest: c5.1\n0.48225729\n0.69444747\n.\n\n\n\n\n\n\nTest: c234.15\n0.13684026\n0.36991926\n.\n.\n\n\n\n\n\nTest: c12.34\n0.89057374\n0.94370214\n.\n.\n.\n\n\n\n\nTest: c3.4\n0.49946393\n0.70672762\n.\n.\n.\n.\n\n\nResidual\n\n0.00000000\n0.00000085\n\n\n\n\n\n\n\n\n\n\nMixedModels.likelihoodratiotest(m_zcp_1_PC_2, m_cpx_1_PC_2)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Child)\n26\n13004\n\n\n\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 + Test | Child)\n36\n13255\n-251\n10\nNaN", + "crumbs": [ + "Contrast coding", + "Mixed Models Tutorial: Contrast Coding" + ] + }, + { + "objectID": "contrasts_fggk21.html#contrasts-are-re-parameterizations-of-the-same-model", + "href": "contrasts_fggk21.html#contrasts-are-re-parameterizations-of-the-same-model", + "title": "Mixed Models Tutorial: Contrast Coding", + "section": "3.1 Contrasts are re-parameterizations of the same model", + "text": "3.1 Contrasts are re-parameterizations of the same model\nThe choice of contrast does not affect the model objective, in other words, they all yield the same goodness of fit. It does not matter whether a contrast is orthogonal or not.\n\n[\n objective(m_ovi_SeqDiff),\n objective(m_ovi_Helmert),\n objective(m_ovi_Hypo),\n]\n\n3-element Vector{Float64}:\n 13810.00650113982\n 13810.006501139793\n 13810.006501139791", + "crumbs": [ + "Contrast coding", + "Mixed Models Tutorial: Contrast Coding" + ] + }, + { + "objectID": "contrasts_fggk21.html#vcs-and-cps-depend-on-contrast-coding", + "href": "contrasts_fggk21.html#vcs-and-cps-depend-on-contrast-coding", + "title": "Mixed Models Tutorial: Contrast Coding", + "section": "3.2 VCs and CPs depend on contrast coding", + "text": "3.2 VCs and CPs depend on contrast coding\nTrivially, the meaning of a contrast depends on its definition. Consequently, the contrast specification has a big effect on the random-effect structure. As an illustration, we refit the LMMs with variance components (VCs) and correlation parameters (CPs) for Child-related contrasts of Test. Unfortunately, it is not easy, actually rather quite difficult, to grasp the meaning of correlations of contrast-based effects; they represent two-way interactions.\n\nbegin\n f_Child = @formula zScore ~\n 1 + Test * a1 * Sex + (1 + Test | Child)\n m_Child_SDC = fit(MixedModel, f_Child, dat; contrasts=contr1)\n m_Child_HeC = fit(MixedModel, f_Child, dat; contrasts=contr2)\n m_Child_HyC = fit(MixedModel, f_Child, dat; contrasts=contr3)\n m_Child_PCA = fit(MixedModel, f_Child, dat; contrasts=contr4)\nend\n\nMinimizing 14 Time: 0:00:00 ( 7.38 ms/it)\n objective: 13864.841025986228\nMinimizing 28 Time: 0:00:00 ( 7.40 ms/it)\n objective: 13845.893578754876\nMinimizing 42 Time: 0:00:00 ( 7.36 ms/it)\n objective: 13779.642766649653\nMinimizing 56 Time: 0:00:00 ( 7.43 ms/it)\n objective: 13774.420150810312\nMinimizing 70 Time: 0:00:00 ( 7.42 ms/it)\n objective: 13763.255128405614\nMinimizing 84 Time: 0:00:00 ( 7.46 ms/it)\n objective: 13758.347505903877\nMinimizing 99 Time: 0:00:00 ( 7.41 ms/it)\n objective: 13756.880079794735\nMinimizing 114 Time: 0:00:00 ( 7.38 ms/it)\n objective: 13748.769476413026\nMinimizing 128 Time: 0:00:00 ( 7.36 ms/it)\n objective: 13745.93833595192\nMinimizing 142 Time: 0:00:01 ( 7.36 ms/it)\n objective: 13719.337037060759\nMinimizing 156 Time: 0:00:01 ( 7.35 ms/it)\n objective: 13713.02271569964\nMinimizing 171 Time: 0:00:01 ( 7.33 ms/it)\n objective: 13710.233015094\nMinimizing 185 Time: 0:00:01 ( 7.33 ms/it)\n objective: 13705.921017381152\nMinimizing 199 Time: 0:00:01 ( 7.32 ms/it)\n objective: 13703.845507302389\nMinimizing 213 Time: 0:00:01 ( 7.32 ms/it)\n objective: 13701.708647031715\nMinimizing 227 Time: 0:00:01 ( 7.31 ms/it)\n objective: 13699.71787171471\nMinimizing 241 Time: 0:00:01 ( 7.31 ms/it)\n objective: 13690.619306364151\nMinimizing 255 Time: 0:00:01 ( 7.31 ms/it)\n objective: 13690.29357377909\nMinimizing 269 Time: 0:00:01 ( 7.31 ms/it)\n objective: 13687.474526688906\nMinimizing 284 Time: 0:00:02 ( 7.30 ms/it)\n objective: 13687.04984767899\nMinimizing 298 Time: 0:00:02 ( 7.30 ms/it)\n objective: 13686.375572547107\nMinimizing 312 Time: 0:00:02 ( 7.30 ms/it)\n objective: 13685.048629360046\nMinimizing 326 Time: 0:00:02 ( 7.31 ms/it)\n objective: 13683.45090568854\nMinimizing 340 Time: 0:00:02 ( 7.33 ms/it)\n objective: 13670.28900574682\nMinimizing 354 Time: 0:00:02 ( 7.33 ms/it)\n objective: 13666.130497207325\nMinimizing 368 Time: 0:00:02 ( 7.34 ms/it)\n objective: 13663.176077165926\nMinimizing 382 Time: 0:00:02 ( 7.33 ms/it)\n objective: 13657.89791477644\nMinimizing 396 Time: 0:00:02 ( 7.33 ms/it)\n objective: 13653.787699088523\nMinimizing 410 Time: 0:00:03 ( 7.34 ms/it)\n objective: 13651.978565149546\nMinimizing 424 Time: 0:00:03 ( 7.34 ms/it)\n objective: 13650.257119077532\nMinimizing 438 Time: 0:00:03 ( 7.35 ms/it)\n objective: 13596.718990553814\nMinimizing 452 Time: 0:00:03 ( 7.35 ms/it)\n objective: 13595.870964499867\nMinimizing 466 Time: 0:00:03 ( 7.35 ms/it)\n objective: 13587.115499796888\nMinimizing 480 Time: 0:00:03 ( 7.36 ms/it)\n objective: 13583.500285415314\nMinimizing 494 Time: 0:00:03 ( 7.36 ms/it)\n objective: 13578.21380550712\nMinimizing 508 Time: 0:00:03 ( 7.36 ms/it)\n objective: 13573.672392208908\nMinimizing 522 Time: 0:00:03 ( 7.36 ms/it)\n objective: 13561.744968132942\nMinimizing 536 Time: 0:00:03 ( 7.36 ms/it)\n objective: 13556.121370537869\nMinimizing 550 Time: 0:00:04 ( 7.36 ms/it)\n objective: 13536.968713369119\nMinimizing 564 Time: 0:00:04 ( 7.35 ms/it)\n objective: 13525.032867750167\nMinimizing 578 Time: 0:00:04 ( 7.36 ms/it)\n objective: 13524.904133885815\nMinimizing 592 Time: 0:00:04 ( 7.35 ms/it)\n objective: 13523.363880581528\nMinimizing 606 Time: 0:00:04 ( 7.35 ms/it)\n objective: 13521.632787209572\nMinimizing 621 Time: 0:00:04 ( 7.34 ms/it)\n objective: 13520.78772943154\nMinimizing 635 Time: 0:00:04 ( 7.34 ms/it)\n objective: 13518.695643430212\nMinimizing 649 Time: 0:00:04 ( 7.35 ms/it)\n objective: 13517.036591524637\nMinimizing 663 Time: 0:00:04 ( 7.35 ms/it)\n objective: 13510.7797885588\nMinimizing 677 Time: 0:00:04 ( 7.35 ms/it)\n objective: 13510.176477523295\nMinimizing 691 Time: 0:00:05 ( 7.35 ms/it)\n objective: 13506.246601124869\nMinimizing 705 Time: 0:00:05 ( 7.35 ms/it)\n objective: 13503.56922521325\nMinimizing 719 Time: 0:00:05 ( 7.36 ms/it)\n objective: 13502.614113315482\nMinimizing 732 Time: 0:00:05 ( 7.37 ms/it)\n objective: 13497.016552788686\nMinimizing 745 Time: 0:00:05 ( 7.38 ms/it)\n objective: 13489.782425035293\nMinimizing 757 Time: 0:00:05 ( 7.40 ms/it)\n objective: 13488.94185046242\nMinimizing 770 Time: 0:00:05 ( 7.41 ms/it)\n objective: 13488.627314452053\nMinimizing 783 Time: 0:00:05 ( 7.42 ms/it)\n objective: 13487.64272385076\nMinimizing 797 Time: 0:00:05 ( 7.42 ms/it)\n objective: 13485.080584581614\nMinimizing 811 Time: 0:00:06 ( 7.42 ms/it)\n objective: 13484.637993051896\nMinimizing 825 Time: 0:00:06 ( 7.42 ms/it)\n objective: 13482.7367410328\nMinimizing 839 Time: 0:00:06 ( 7.42 ms/it)\n objective: 13482.481133163245\nMinimizing 853 Time: 0:00:06 ( 7.42 ms/it)\n objective: 13480.919770174864\nMinimizing 867 Time: 0:00:06 ( 7.42 ms/it)\n objective: 13479.338020027433\nMinimizing 881 Time: 0:00:06 ( 7.42 ms/it)\n objective: 13478.268180507002\nMinimizing 895 Time: 0:00:06 ( 7.42 ms/it)\n objective: 13474.89802321393\nMinimizing 910 Time: 0:00:06 ( 7.42 ms/it)\n objective: 13474.818139830619\nMinimizing 924 Time: 0:00:06 ( 7.42 ms/it)\n objective: 13473.899599287739\nMinimizing 938 Time: 0:00:06 ( 7.42 ms/it)\n objective: 13472.991812527842\nMinimizing 951 Time: 0:00:07 ( 7.44 ms/it)\n objective: 13472.554082196628\nMinimizing 964 Time: 0:00:07 ( 7.45 ms/it)\n objective: 13468.538171214794\nMinimizing 977 Time: 0:00:07 ( 7.45 ms/it)\n objective: 13467.639716883765\nMinimizing 992 Time: 0:00:07 ( 7.45 ms/it)\n objective: 13467.578593657796\nMinimizing 1007 Time: 0:00:07 ( 7.44 ms/it)\n objective: 13466.23114230048\nMinimizing 1022 Time: 0:00:07 ( 7.43 ms/it)\n objective: 13464.516615018845\nMinimizing 1036 Time: 0:00:07 ( 7.43 ms/it)\n objective: 13463.980091173093\nMinimizing 1050 Time: 0:00:07 ( 7.43 ms/it)\n objective: 13462.773479719443\nMinimizing 1065 Time: 0:00:07 ( 7.43 ms/it)\n objective: 13462.216706877145\nMinimizing 1080 Time: 0:00:08 ( 7.42 ms/it)\n objective: 13461.358939771475\nMinimizing 1095 Time: 0:00:08 ( 7.41 ms/it)\n objective: 13458.882016874835\nMinimizing 1110 Time: 0:00:08 ( 7.41 ms/it)\n objective: 13458.617667955717\nMinimizing 1125 Time: 0:00:08 ( 7.40 ms/it)\n objective: 13457.543642033306\nMinimizing 1140 Time: 0:00:08 ( 7.39 ms/it)\n objective: 13456.274499247207\nMinimizing 1155 Time: 0:00:08 ( 7.39 ms/it)\n objective: 13454.019270400633\nMinimizing 1170 Time: 0:00:08 ( 7.39 ms/it)\n objective: 13453.999404741633\nMinimizing 1185 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13453.081996300418\nMinimizing 1200 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13451.84992165603\nMinimizing 1215 Time: 0:00:08 ( 7.37 ms/it)\n objective: 13451.461555001071\nMinimizing 1230 Time: 0:00:09 ( 7.36 ms/it)\n objective: 13450.057680870392\nMinimizing 1245 Time: 0:00:09 ( 7.36 ms/it)\n objective: 13448.822191192878\nMinimizing 1260 Time: 0:00:09 ( 7.35 ms/it)\n objective: 13445.389702814973\nMinimizing 1275 Time: 0:00:09 ( 7.35 ms/it)\n objective: 13428.298074435195\nMinimizing 1290 Time: 0:00:09 ( 7.35 ms/it)\n objective: 13419.14473015425\nMinimizing 1305 Time: 0:00:09 ( 7.34 ms/it)\n objective: 13414.257278873745\nMinimizing 1320 Time: 0:00:09 ( 7.34 ms/it)\n objective: 13403.679148710464\nMinimizing 1335 Time: 0:00:09 ( 7.33 ms/it)\n objective: 13396.232708204952\nMinimizing 1350 Time: 0:00:09 ( 7.33 ms/it)\n objective: 13350.927349250676\nMinimizing 1365 Time: 0:00:09 ( 7.32 ms/it)\n objective: 13339.274454825776\nMinimizing 1380 Time: 0:00:10 ( 7.32 ms/it)\n objective: 13335.342672513361\nMinimizing 1395 Time: 0:00:10 ( 7.32 ms/it)\n objective: 13329.844363219323\nMinimizing 1410 Time: 0:00:10 ( 7.32 ms/it)\n objective: 13318.587893036965\nMinimizing 1425 Time: 0:00:10 ( 7.31 ms/it)\n objective: 13318.483025400827\nMinimizing 1440 Time: 0:00:10 ( 7.31 ms/it)\n objective: 13317.603252419402\nMinimizing 1455 Time: 0:00:10 ( 7.31 ms/it)\n objective: 13316.97518029595\nMinimizing 1470 Time: 0:00:10 ( 7.30 ms/it)\n objective: 13316.072121185236\nMinimizing 1485 Time: 0:00:10 ( 7.30 ms/it)\n objective: 13315.359104246527\nMinimizing 1500 Time: 0:00:10 ( 7.30 ms/it)\n objective: 13313.924189835161\nMinimizing 1515 Time: 0:00:11 ( 7.29 ms/it)\n objective: 13313.797955305534\nMinimizing 1530 Time: 0:00:11 ( 7.29 ms/it)\n objective: 13311.316728242207\nMinimizing 1545 Time: 0:00:11 ( 7.29 ms/it)\n objective: 13311.254955979413\nMinimizing 1560 Time: 0:00:11 ( 7.29 ms/it)\n objective: 13310.752448072686\nMinimizing 1575 Time: 0:00:11 ( 7.28 ms/it)\n objective: 13310.263050807902\nMinimizing 1590 Time: 0:00:11 ( 7.28 ms/it)\n objective: 13309.18452747412\nMinimizing 1605 Time: 0:00:11 ( 7.28 ms/it)\n objective: 13307.424914636562\nMinimizing 1620 Time: 0:00:11 ( 7.27 ms/it)\n objective: 13306.590932389605\nMinimizing 1635 Time: 0:00:11 ( 7.27 ms/it)\n objective: 13304.340360901668\nMinimizing 1649 Time: 0:00:11 ( 7.27 ms/it)\n objective: 13304.231132097222\nMinimizing 1664 Time: 0:00:12 ( 7.27 ms/it)\n objective: 13304.038819103182\nMinimizing 1679 Time: 0:00:12 ( 7.27 ms/it)\n objective: 13303.680926864385\nMinimizing 1694 Time: 0:00:12 ( 7.26 ms/it)\n objective: 13303.419516689974\nMinimizing 1709 Time: 0:00:12 ( 7.26 ms/it)\n objective: 13302.681011149776\nMinimizing 1724 Time: 0:00:12 ( 7.26 ms/it)\n objective: 13302.011843218294\nMinimizing 1739 Time: 0:00:12 ( 7.26 ms/it)\n objective: 13301.443772030732\nMinimizing 1754 Time: 0:00:12 ( 7.25 ms/it)\n objective: 13300.149896853356\nMinimizing 1769 Time: 0:00:12 ( 7.25 ms/it)\n objective: 13299.8611757061\nMinimizing 1784 Time: 0:00:12 ( 7.25 ms/it)\n objective: 13299.58302719945\nMinimizing 1798 Time: 0:00:13 ( 7.25 ms/it)\n objective: 13299.122255901151\nMinimizing 1813 Time: 0:00:13 ( 7.25 ms/it)\n objective: 13298.51099639821\nMinimizing 1828 Time: 0:00:13 ( 7.25 ms/it)\n objective: 13297.229667072825\nMinimizing 1843 Time: 0:00:13 ( 7.25 ms/it)\n objective: 13296.744068662185\nMinimizing 1858 Time: 0:00:13 ( 7.25 ms/it)\n objective: 13295.195782863346\nMinimizing 1873 Time: 0:00:13 ( 7.24 ms/it)\n objective: 13294.97360063158\nMinimizing 1888 Time: 0:00:13 ( 7.24 ms/it)\n objective: 13294.294746616506\nMinimizing 1903 Time: 0:00:13 ( 7.24 ms/it)\n objective: 13292.943878194666\nMinimizing 1918 Time: 0:00:13 ( 7.24 ms/it)\n objective: 13292.491278275134\nMinimizing 1933 Time: 0:00:13 ( 7.23 ms/it)\n objective: 13292.397969650163\nMinimizing 1947 Time: 0:00:14 ( 7.23 ms/it)\n objective: 13291.894899531384\nMinimizing 1962 Time: 0:00:14 ( 7.23 ms/it)\n objective: 13291.356819490858\nMinimizing 1977 Time: 0:00:14 ( 7.23 ms/it)\n objective: 13290.752636017758\nMinimizing 1992 Time: 0:00:14 ( 7.23 ms/it)\n objective: 13289.566208013915\nMinimizing 2007 Time: 0:00:14 ( 7.23 ms/it)\n objective: 13288.537436593717\nMinimizing 2022 Time: 0:00:14 ( 7.23 ms/it)\n objective: 13288.037102696675\nMinimizing 2037 Time: 0:00:14 ( 7.22 ms/it)\n objective: 13287.095374361612\nMinimizing 2052 Time: 0:00:14 ( 7.22 ms/it)\n objective: 13286.236736689549\nMinimizing 2067 Time: 0:00:14 ( 7.22 ms/it)\n objective: 13284.47508208947\nMinimizing 2082 Time: 0:00:15 ( 7.22 ms/it)\n objective: 13282.954953712266\nMinimizing 2097 Time: 0:00:15 ( 7.22 ms/it)\n objective: 13279.51512109107\nMinimizing 2112 Time: 0:00:15 ( 7.22 ms/it)\n objective: 13275.962618584832\nMinimizing 2126 Time: 0:00:15 ( 7.22 ms/it)\n objective: 13265.934409907422\nMinimizing 2140 Time: 0:00:15 ( 7.22 ms/it)\n objective: 13254.338813870418\nMinimizing 2155 Time: 0:00:15 ( 7.21 ms/it)\n objective: 13252.290857140033\nMinimizing 2170 Time: 0:00:15 ( 7.21 ms/it)\n objective: 13250.185037324627\nMinimizing 2185 Time: 0:00:15 ( 7.21 ms/it)\n objective: 13244.551926471962\nMinimizing 2200 Time: 0:00:15 ( 7.21 ms/it)\n objective: 13238.150409325623\nMinimizing 2217 Time: 0:00:15 ( 7.21 ms/it)\n objective: 13232.850150665225\nMinimizing 2233 Time: 0:00:16 ( 7.21 ms/it)\n objective: 13226.338353252722\nMinimizing 2248 Time: 0:00:16 ( 7.21 ms/it)\n objective: 13223.438841768424\nMinimizing 2262 Time: 0:00:16 ( 7.21 ms/it)\n objective: 13218.113302238213\nMinimizing 2276 Time: 0:00:16 ( 7.21 ms/it)\n objective: 13217.731166378071\nMinimizing 2291 Time: 0:00:16 ( 7.21 ms/it)\n objective: 13214.768506554843\nMinimizing 2306 Time: 0:00:16 ( 7.21 ms/it)\n objective: 13211.085951421177\nMinimizing 2321 Time: 0:00:16 ( 7.21 ms/it)\n objective: 13201.440930079378\nMinimizing 2336 Time: 0:00:16 ( 7.21 ms/it)\n objective: 13196.688185621882\nMinimizing 2351 Time: 0:00:16 ( 7.21 ms/it)\n objective: 13188.13659060486\nMinimizing 2366 Time: 0:00:17 ( 7.21 ms/it)\n objective: 13180.288196244132\nMinimizing 2381 Time: 0:00:17 ( 7.20 ms/it)\n objective: 13164.871640060272\nMinimizing 2396 Time: 0:00:17 ( 7.20 ms/it)\n objective: 13160.4412493243\nMinimizing 2411 Time: 0:00:17 ( 7.20 ms/it)\n objective: 13146.791395771419\nMinimizing 2425 Time: 0:00:17 ( 7.20 ms/it)\n objective: 13144.898178043746\nMinimizing 2440 Time: 0:00:17 ( 7.20 ms/it)\n objective: 13136.092751673175\nMinimizing 2455 Time: 0:00:17 ( 7.20 ms/it)\n objective: 13112.94757467517\nMinimizing 2470 Time: 0:00:17 ( 7.20 ms/it)\n objective: 13109.07036788974\nMinimizing 2485 Time: 0:00:17 ( 7.20 ms/it)\n objective: 13109.041576801654\nMinimizing 2500 Time: 0:00:18 ( 7.20 ms/it)\n objective: 13109.001231908522\nMinimizing 2515 Time: 0:00:18 ( 7.20 ms/it)\n objective: 13109.015337759716\nMinimizing 2530 Time: 0:00:18 ( 7.20 ms/it)\n objective: 13109.034734655841\nMinimizing 2545 Time: 0:00:18 ( 7.20 ms/it)\n objective: 13109.038868225558\nMinimizing 2560 Time: 0:00:18 ( 7.20 ms/it)\n objective: 13109.061958145263\nMinimizing 2575 Time: 0:00:18 ( 7.20 ms/it)\n objective: 13109.01477411139\nMinimizing 2589 Time: 0:00:18 ( 7.20 ms/it)\n objective: 13109.003292939946\nMinimizing 2604 Time: 0:00:18 ( 7.20 ms/it)\n objective: 13109.042902489833\nMinimizing 2618 Time: 0:00:18 ( 7.20 ms/it)\n objective: 13109.006932114848\nMinimizing 2633 Time: 0:00:18 ( 7.20 ms/it)\n objective: 13109.006471395347\nMinimizing 2648 Time: 0:00:19 ( 7.20 ms/it)\n objective: 13109.009539637394\nMinimizing 2663 Time: 0:00:19 ( 7.20 ms/it)\n objective: 13109.003172373152\nMinimizing 2678 Time: 0:00:19 ( 7.19 ms/it)\n objective: 13109.006089056216\nMinimizing 2690 Time: 0:00:19 ( 7.19 ms/it)\nMinimizing 15 Time: 0:00:00 ( 7.03 ms/it)\n objective: 14484.818769769932\nMinimizing 30 Time: 0:00:00 ( 7.01 ms/it)\n objective: 15149.245792406973\nMinimizing 45 Time: 0:00:00 ( 7.04 ms/it)\n objective: 13769.912489896697\nMinimizing 59 Time: 0:00:00 ( 7.15 ms/it)\n objective: 13761.74191805981\nMinimizing 73 Time: 0:00:00 ( 7.17 ms/it)\n objective: 13758.810515322486\nMinimizing 87 Time: 0:00:00 ( 7.17 ms/it)\n objective: 13747.89144631826\nMinimizing 102 Time: 0:00:00 ( 7.15 ms/it)\n objective: 13738.014532122419\nMinimizing 117 Time: 0:00:00 ( 7.13 ms/it)\n objective: 13731.266747706104\nMinimizing 132 Time: 0:00:00 ( 7.12 ms/it)\n objective: 13724.618309431458\nMinimizing 147 Time: 0:00:01 ( 7.12 ms/it)\n objective: 13654.800970509314\nMinimizing 162 Time: 0:00:01 ( 7.10 ms/it)\n objective: 13653.23607155866\nMinimizing 177 Time: 0:00:01 ( 7.09 ms/it)\n objective: 13647.684656378384\nMinimizing 192 Time: 0:00:01 ( 7.09 ms/it)\n objective: 13644.340551829799\nMinimizing 207 Time: 0:00:01 ( 7.08 ms/it)\n objective: 13639.157628992602\nMinimizing 222 Time: 0:00:01 ( 7.08 ms/it)\n objective: 13637.212982312521\nMinimizing 236 Time: 0:00:01 ( 7.09 ms/it)\n objective: 13631.420008504178\nMinimizing 250 Time: 0:00:01 ( 7.12 ms/it)\n objective: 13627.76036240306\nMinimizing 264 Time: 0:00:01 ( 7.13 ms/it)\n objective: 13618.13712888892\nMinimizing 278 Time: 0:00:01 ( 7.15 ms/it)\n objective: 13602.718359392646\nMinimizing 292 Time: 0:00:02 ( 7.17 ms/it)\n objective: 13594.795656013492\nMinimizing 306 Time: 0:00:02 ( 7.17 ms/it)\n objective: 13592.296898878354\nMinimizing 320 Time: 0:00:02 ( 7.18 ms/it)\n objective: 13589.80172428917\nMinimizing 334 Time: 0:00:02 ( 7.19 ms/it)\n objective: 13587.694269563737\nMinimizing 348 Time: 0:00:02 ( 7.21 ms/it)\n objective: 13586.165199506271\nMinimizing 362 Time: 0:00:02 ( 7.21 ms/it)\n objective: 13583.971722038295\nMinimizing 376 Time: 0:00:02 ( 7.22 ms/it)\n objective: 13579.163713327973\nMinimizing 390 Time: 0:00:02 ( 7.23 ms/it)\n objective: 13572.566887500128\nMinimizing 404 Time: 0:00:02 ( 7.24 ms/it)\n objective: 13571.997138661252\nMinimizing 418 Time: 0:00:03 ( 7.24 ms/it)\n objective: 13569.486490841591\nMinimizing 431 Time: 0:00:03 ( 7.26 ms/it)\n objective: 13568.964181688567\nMinimizing 445 Time: 0:00:03 ( 7.27 ms/it)\n objective: 13567.440966488095\nMinimizing 459 Time: 0:00:03 ( 7.27 ms/it)\n objective: 13566.783662910664\nMinimizing 473 Time: 0:00:03 ( 7.27 ms/it)\n objective: 13565.013170825518\nMinimizing 487 Time: 0:00:03 ( 7.27 ms/it)\n objective: 13562.508977192181\nMinimizing 501 Time: 0:00:03 ( 7.28 ms/it)\n objective: 13554.218956510995\nMinimizing 515 Time: 0:00:03 ( 7.28 ms/it)\n objective: 13553.000339553691\nMinimizing 529 Time: 0:00:03 ( 7.28 ms/it)\n objective: 13549.8952039791\nMinimizing 543 Time: 0:00:03 ( 7.29 ms/it)\n objective: 13546.692917706656\nMinimizing 557 Time: 0:00:04 ( 7.29 ms/it)\n objective: 13545.77669800381\nMinimizing 571 Time: 0:00:04 ( 7.29 ms/it)\n objective: 13541.315990069146\nMinimizing 585 Time: 0:00:04 ( 7.29 ms/it)\n objective: 13537.015846784452\nMinimizing 599 Time: 0:00:04 ( 7.29 ms/it)\n objective: 13537.941379972595\nMinimizing 613 Time: 0:00:04 ( 7.29 ms/it)\n objective: 13526.733567803945\nMinimizing 627 Time: 0:00:04 ( 7.30 ms/it)\n objective: 13517.545489108263\nMinimizing 641 Time: 0:00:04 ( 7.31 ms/it)\n objective: 13512.95366684183\nMinimizing 655 Time: 0:00:04 ( 7.31 ms/it)\n objective: 13508.438718048426\nMinimizing 669 Time: 0:00:04 ( 7.31 ms/it)\n objective: 13507.36919131755\nMinimizing 683 Time: 0:00:04 ( 7.31 ms/it)\n objective: 13503.892437927789\nMinimizing 697 Time: 0:00:05 ( 7.32 ms/it)\n objective: 13500.878851006266\nMinimizing 711 Time: 0:00:05 ( 7.32 ms/it)\n objective: 13500.540697427685\nMinimizing 725 Time: 0:00:05 ( 7.32 ms/it)\n objective: 13499.647851582413\nMinimizing 739 Time: 0:00:05 ( 7.32 ms/it)\n objective: 13497.31636822351\nMinimizing 753 Time: 0:00:05 ( 7.32 ms/it)\n objective: 13496.74235689282\nMinimizing 767 Time: 0:00:05 ( 7.32 ms/it)\n objective: 13495.236254695941\nMinimizing 781 Time: 0:00:05 ( 7.33 ms/it)\n objective: 13494.460546471433\nMinimizing 794 Time: 0:00:05 ( 7.34 ms/it)\n objective: 13493.116948996358\nMinimizing 808 Time: 0:00:05 ( 7.34 ms/it)\n objective: 13491.55222243048\nMinimizing 821 Time: 0:00:06 ( 7.35 ms/it)\n objective: 13485.298372171179\nMinimizing 834 Time: 0:00:06 ( 7.36 ms/it)\n objective: 13484.846928479092\nMinimizing 848 Time: 0:00:06 ( 7.36 ms/it)\n objective: 13481.798310757702\nMinimizing 862 Time: 0:00:06 ( 7.36 ms/it)\n objective: 13481.54557346375\nMinimizing 876 Time: 0:00:06 ( 7.36 ms/it)\n objective: 13479.242262358726\nMinimizing 890 Time: 0:00:06 ( 7.36 ms/it)\n objective: 13477.776281089456\nMinimizing 904 Time: 0:00:06 ( 7.36 ms/it)\n objective: 13471.293038345124\nMinimizing 918 Time: 0:00:06 ( 7.36 ms/it)\n objective: 13470.261187016251\nMinimizing 932 Time: 0:00:06 ( 7.36 ms/it)\n objective: 13462.889959603483\nMinimizing 946 Time: 0:00:06 ( 7.36 ms/it)\n objective: 13458.245608738893\nMinimizing 960 Time: 0:00:07 ( 7.37 ms/it)\n objective: 13448.82679411889\nMinimizing 974 Time: 0:00:07 ( 7.37 ms/it)\n objective: 13446.907386952858\nMinimizing 988 Time: 0:00:07 ( 7.37 ms/it)\n objective: 13445.662653153297\nMinimizing 1002 Time: 0:00:07 ( 7.37 ms/it)\n objective: 13440.418761966415\nMinimizing 1016 Time: 0:00:07 ( 7.37 ms/it)\n objective: 13436.471265100285\nMinimizing 1030 Time: 0:00:07 ( 7.37 ms/it)\n objective: 13435.588412697325\nMinimizing 1044 Time: 0:00:07 ( 7.37 ms/it)\n objective: 13431.482927769925\nMinimizing 1057 Time: 0:00:07 ( 7.38 ms/it)\n objective: 13422.652037011016\nMinimizing 1071 Time: 0:00:07 ( 7.38 ms/it)\n objective: 13403.243319638073\nMinimizing 1085 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13400.098319343466\nMinimizing 1099 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13392.128289924643\nMinimizing 1113 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13390.187741748465\nMinimizing 1127 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13388.262281691314\nMinimizing 1141 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13385.439643098529\nMinimizing 1155 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13380.929415013677\nMinimizing 1169 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13366.210191090278\nMinimizing 1183 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13353.708329809568\nMinimizing 1197 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13352.423984292298\nMinimizing 1211 Time: 0:00:08 ( 7.38 ms/it)\n objective: 13350.44711648039\nMinimizing 1225 Time: 0:00:09 ( 7.38 ms/it)\n objective: 13347.356347077643\nMinimizing 1239 Time: 0:00:09 ( 7.38 ms/it)\n objective: 13345.241849598242\nMinimizing 1252 Time: 0:00:09 ( 7.38 ms/it)\n objective: 13342.73638115912\nMinimizing 1266 Time: 0:00:09 ( 7.38 ms/it)\n objective: 13340.971598195058\nMinimizing 1280 Time: 0:00:09 ( 7.39 ms/it)\n objective: 13337.339740364754\nMinimizing 1294 Time: 0:00:09 ( 7.38 ms/it)\n objective: 13336.211162891705\nMinimizing 1308 Time: 0:00:09 ( 7.38 ms/it)\n objective: 13328.602816670158\nMinimizing 1322 Time: 0:00:09 ( 7.38 ms/it)\n objective: 13328.241141181046\nMinimizing 1336 Time: 0:00:09 ( 7.38 ms/it)\n objective: 13326.212511028396\nMinimizing 1351 Time: 0:00:09 ( 7.39 ms/it)\n objective: 13325.046683382912\nMinimizing 1366 Time: 0:00:10 ( 7.39 ms/it)\n objective: 13324.624948563855\nMinimizing 1380 Time: 0:00:10 ( 7.39 ms/it)\n objective: 13323.218171707762\nMinimizing 1394 Time: 0:00:10 ( 7.39 ms/it)\n objective: 13321.328471264103\nMinimizing 1408 Time: 0:00:10 ( 7.39 ms/it)\n objective: 13320.5146919408\nMinimizing 1422 Time: 0:00:10 ( 7.39 ms/it)\n objective: 13319.747185641492\nMinimizing 1436 Time: 0:00:10 ( 7.39 ms/it)\n objective: 13317.807118322351\nMinimizing 1450 Time: 0:00:10 ( 7.39 ms/it)\n objective: 13316.583428790953\nMinimizing 1463 Time: 0:00:10 ( 7.39 ms/it)\n objective: 13316.369017908946\nMinimizing 1476 Time: 0:00:10 ( 7.40 ms/it)\n objective: 13314.720825314682\nMinimizing 1490 Time: 0:00:11 ( 7.40 ms/it)\n objective: 13313.942277059949\nMinimizing 1504 Time: 0:00:11 ( 7.40 ms/it)\n objective: 13310.55964418828\nMinimizing 1518 Time: 0:00:11 ( 7.40 ms/it)\n objective: 13308.084344920353\nMinimizing 1532 Time: 0:00:11 ( 7.40 ms/it)\n objective: 13306.527777598953\nMinimizing 1546 Time: 0:00:11 ( 7.41 ms/it)\n objective: 13305.292557824127\nMinimizing 1560 Time: 0:00:11 ( 7.41 ms/it)\n objective: 13304.99530357393\nMinimizing 1574 Time: 0:00:11 ( 7.41 ms/it)\n objective: 13304.21735900284\nMinimizing 1588 Time: 0:00:11 ( 7.41 ms/it)\n objective: 13303.882313797236\nMinimizing 1602 Time: 0:00:11 ( 7.41 ms/it)\n objective: 13300.523697426834\nMinimizing 1616 Time: 0:00:11 ( 7.41 ms/it)\n objective: 13300.326373889926\nMinimizing 1630 Time: 0:00:12 ( 7.41 ms/it)\n objective: 13300.099524684862\nMinimizing 1644 Time: 0:00:12 ( 7.41 ms/it)\n objective: 13300.039316488837\nMinimizing 1658 Time: 0:00:12 ( 7.41 ms/it)\n objective: 13300.016441446089\nMinimizing 1672 Time: 0:00:12 ( 7.41 ms/it)\n objective: 13299.992881394806\nMinimizing 1686 Time: 0:00:12 ( 7.41 ms/it)\n objective: 13299.986797741367\nMinimizing 1699 Time: 0:00:12 ( 7.42 ms/it)\n objective: 13299.985735700582\nMinimizing 1713 Time: 0:00:12 ( 7.42 ms/it)\n objective: 13299.985038568862\nMinimizing 1727 Time: 0:00:12 ( 7.42 ms/it)\n objective: 13299.98481352735\nMinimizing 1741 Time: 0:00:12 ( 7.42 ms/it)\n objective: 13299.983782423922\nMinimizing 1755 Time: 0:00:13 ( 7.42 ms/it)\n objective: 13299.97619429884\nMinimizing 1769 Time: 0:00:13 ( 7.42 ms/it)\n objective: 13299.975995964516\nMinimizing 1783 Time: 0:00:13 ( 7.41 ms/it)\n objective: 13299.9729981102\nMinimizing 1797 Time: 0:00:13 ( 7.41 ms/it)\n objective: 13299.96839557476\nMinimizing 1811 Time: 0:00:13 ( 7.41 ms/it)\n objective: 13299.967537347824\nMinimizing 1825 Time: 0:00:13 ( 7.41 ms/it)\n objective: 13299.961759416285\nMinimizing 1839 Time: 0:00:13 ( 7.41 ms/it)\n objective: 13299.960301776562\nMinimizing 1853 Time: 0:00:13 ( 7.41 ms/it)\n objective: 13299.959559300827\nMinimizing 1867 Time: 0:00:13 ( 7.41 ms/it)\n objective: 13299.958113731467\nMinimizing 1881 Time: 0:00:13 ( 7.41 ms/it)\n objective: 13299.954657530849\nMinimizing 1895 Time: 0:00:14 ( 7.41 ms/it)\n objective: 13299.918198116953\nMinimizing 1909 Time: 0:00:14 ( 7.40 ms/it)\n objective: 13299.91660130379\nMinimizing 1923 Time: 0:00:14 ( 7.40 ms/it)\n objective: 13299.916022577483\nMinimizing 1937 Time: 0:00:14 ( 7.41 ms/it)\n objective: 13299.915282904272\nMinimizing 1951 Time: 0:00:14 ( 7.41 ms/it)\n objective: 13299.91275801108\nMinimizing 1965 Time: 0:00:14 ( 7.41 ms/it)\n objective: 13299.911371354538\nMinimizing 1979 Time: 0:00:14 ( 7.40 ms/it)\n objective: 13299.907843613837\nMinimizing 1994 Time: 0:00:14 ( 7.40 ms/it)\n objective: 13299.887608155754\nMinimizing 2008 Time: 0:00:14 ( 7.40 ms/it)\n objective: 13299.886695265886\nMinimizing 2022 Time: 0:00:14 ( 7.40 ms/it)\n objective: 13299.883886907977\nMinimizing 2037 Time: 0:00:15 ( 7.40 ms/it)\n objective: 13299.867251779331\nMinimizing 2051 Time: 0:00:15 ( 7.40 ms/it)\n objective: 13299.780878727397\nMinimizing 2065 Time: 0:00:15 ( 7.40 ms/it)\n objective: 13299.716876275386\nMinimizing 2079 Time: 0:00:15 ( 7.39 ms/it)\n objective: 13299.704992282015\nMinimizing 2093 Time: 0:00:15 ( 7.39 ms/it)\n objective: 13299.7023027518\nMinimizing 2107 Time: 0:00:15 ( 7.39 ms/it)\n objective: 13299.702110452912\nMinimizing 2121 Time: 0:00:15 ( 7.39 ms/it)\n objective: 13299.699853481201\nMinimizing 2136 Time: 0:00:15 ( 7.39 ms/it)\n objective: 13299.695840824585\nMinimizing 2151 Time: 0:00:15 ( 7.39 ms/it)\n objective: 13299.689033835588\nMinimizing 2166 Time: 0:00:15 ( 7.39 ms/it)\n objective: 13299.686354496705\nMinimizing 2180 Time: 0:00:16 ( 7.38 ms/it)\n objective: 13299.684333503523\nMinimizing 2195 Time: 0:00:16 ( 7.38 ms/it)\n objective: 13299.67685816715\nMinimizing 2210 Time: 0:00:16 ( 7.38 ms/it)\n objective: 13299.671855180815\nMinimizing 2225 Time: 0:00:16 ( 7.38 ms/it)\n objective: 13299.669831754843\nMinimizing 2240 Time: 0:00:16 ( 7.37 ms/it)\n objective: 13299.664821198458\nMinimizing 2255 Time: 0:00:16 ( 7.37 ms/it)\n objective: 13299.66390414616\nMinimizing 2270 Time: 0:00:16 ( 7.37 ms/it)\n objective: 13299.663356312478\nMinimizing 2285 Time: 0:00:16 ( 7.36 ms/it)\n objective: 13299.66278192561\nMinimizing 2300 Time: 0:00:16 ( 7.36 ms/it)\n objective: 13299.661119690383\nMinimizing 2315 Time: 0:00:17 ( 7.36 ms/it)\n objective: 13299.638889618582\nMinimizing 2330 Time: 0:00:17 ( 7.35 ms/it)\n objective: 13299.635406443267\nMinimizing 2345 Time: 0:00:17 ( 7.35 ms/it)\n objective: 13299.634563153246\nMinimizing 2360 Time: 0:00:17 ( 7.35 ms/it)\n objective: 13299.631799048322\nMinimizing 2375 Time: 0:00:17 ( 7.34 ms/it)\n objective: 13299.630767917333\nMinimizing 2390 Time: 0:00:17 ( 7.34 ms/it)\n objective: 13299.622092789927\nMinimizing 2405 Time: 0:00:17 ( 7.34 ms/it)\n objective: 13299.62150079933\nMinimizing 2420 Time: 0:00:17 ( 7.34 ms/it)\n objective: 13299.619576690864\nMinimizing 2434 Time: 0:00:17 ( 7.34 ms/it)\n objective: 13299.618630976212\nMinimizing 2449 Time: 0:00:17 ( 7.34 ms/it)\n objective: 13299.590949282428\nMinimizing 2464 Time: 0:00:18 ( 7.33 ms/it)\n objective: 13299.587271440381\nMinimizing 2479 Time: 0:00:18 ( 7.33 ms/it)\n objective: 13299.584474582123\nMinimizing 2494 Time: 0:00:18 ( 7.33 ms/it)\n objective: 13299.582613714403\nMinimizing 2509 Time: 0:00:18 ( 7.33 ms/it)\n objective: 13299.582530809057\nMinimizing 2524 Time: 0:00:18 ( 7.32 ms/it)\n objective: 13299.581651118206\nMinimizing 2539 Time: 0:00:18 ( 7.32 ms/it)\n objective: 13299.580380826883\nMinimizing 2554 Time: 0:00:18 ( 7.32 ms/it)\n objective: 13299.575468890718\nMinimizing 2569 Time: 0:00:18 ( 7.32 ms/it)\n objective: 13299.574941438987\nMinimizing 2584 Time: 0:00:18 ( 7.31 ms/it)\n objective: 13299.57473547748\nMinimizing 2599 Time: 0:00:19 ( 7.31 ms/it)\n objective: 13299.574675681244\nMinimizing 2614 Time: 0:00:19 ( 7.31 ms/it)\n objective: 13299.574759437543\nMinimizing 2629 Time: 0:00:19 ( 7.31 ms/it)\n objective: 13299.57465236247\nMinimizing 2644 Time: 0:00:19 ( 7.31 ms/it)\n objective: 13299.574693544753\nMinimizing 2659 Time: 0:00:19 ( 7.31 ms/it)\n objective: 13299.574655547316\nMinimizing 2673 Time: 0:00:19 ( 7.31 ms/it)\n objective: 13299.574664291795\nMinimizing 2687 Time: 0:00:19 ( 7.31 ms/it)\n objective: 13299.574650705501\nMinimizing 2702 Time: 0:00:19 ( 7.30 ms/it)\n objective: 13299.574662029554\nMinimizing 2717 Time: 0:00:19 ( 7.30 ms/it)\n objective: 13299.57465825256\nMinimizing 2732 Time: 0:00:19 ( 7.30 ms/it)\n objective: 13299.574659096776\nMinimizing 2747 Time: 0:00:20 ( 7.30 ms/it)\n objective: 13299.574653715434\nMinimizing 2754 Time: 0:00:20 ( 7.30 ms/it)\nMinimizing 15 Time: 0:00:00 ( 6.88 ms/it)\n objective: 13919.888179726298\nMinimizing 30 Time: 0:00:00 ( 6.87 ms/it)\n objective: 13815.883938504396\nMinimizing 45 Time: 0:00:00 ( 6.89 ms/it)\n objective: 13786.33833284796\nMinimizing 60 Time: 0:00:00 ( 6.89 ms/it)\n objective: 13784.43109247562\nMinimizing 75 Time: 0:00:00 ( 6.89 ms/it)\n objective: 13780.643883138247\nMinimizing 90 Time: 0:00:00 ( 6.91 ms/it)\n objective: 13778.159421052334\nMinimizing 105 Time: 0:00:00 ( 6.92 ms/it)\n objective: 13748.771484139796\nMinimizing 120 Time: 0:00:00 ( 6.93 ms/it)\n objective: 13742.232397522686\nMinimizing 135 Time: 0:00:00 ( 6.93 ms/it)\n objective: 13735.058257565554\nMinimizing 150 Time: 0:00:01 ( 6.93 ms/it)\n objective: 13728.752825850144\nMinimizing 165 Time: 0:00:01 ( 6.93 ms/it)\n objective: 13722.711479649493\nMinimizing 180 Time: 0:00:01 ( 6.94 ms/it)\n objective: 13721.51956269921\nMinimizing 194 Time: 0:00:01 ( 6.97 ms/it)\n objective: 13718.383898739745\nMinimizing 209 Time: 0:00:01 ( 6.98 ms/it)\n objective: 13714.643657575401\nMinimizing 224 Time: 0:00:01 ( 6.97 ms/it)\n objective: 13714.081910390509\nMinimizing 239 Time: 0:00:01 ( 6.97 ms/it)\n objective: 13710.919152905153\nMinimizing 254 Time: 0:00:01 ( 6.97 ms/it)\n objective: 13708.377044422363\nMinimizing 269 Time: 0:00:01 ( 6.97 ms/it)\n objective: 13707.531207325808\nMinimizing 284 Time: 0:00:01 ( 6.97 ms/it)\n objective: 13704.781395081307\nMinimizing 299 Time: 0:00:02 ( 6.97 ms/it)\n objective: 13703.318775764576\nMinimizing 314 Time: 0:00:02 ( 6.97 ms/it)\n objective: 13700.864623521473\nMinimizing 329 Time: 0:00:02 ( 6.97 ms/it)\n objective: 13696.941601516359\nMinimizing 344 Time: 0:00:02 ( 6.97 ms/it)\n objective: 13688.26463997189\nMinimizing 359 Time: 0:00:02 ( 6.97 ms/it)\n objective: 13684.813655986602\nMinimizing 374 Time: 0:00:02 ( 6.97 ms/it)\n objective: 13676.32159524804\nMinimizing 389 Time: 0:00:02 ( 6.97 ms/it)\n objective: 13662.599234298144\nMinimizing 404 Time: 0:00:02 ( 6.97 ms/it)\n objective: 13642.145530001137\nMinimizing 419 Time: 0:00:02 ( 6.97 ms/it)\n objective: 13632.808208863455\nMinimizing 434 Time: 0:00:03 ( 6.97 ms/it)\n objective: 13627.330102655375\nMinimizing 449 Time: 0:00:03 ( 6.96 ms/it)\n objective: 13605.865467742173\nMinimizing 464 Time: 0:00:03 ( 6.97 ms/it)\n objective: 13603.213778256239\nMinimizing 478 Time: 0:00:03 ( 6.98 ms/it)\n objective: 13599.3686192823\nMinimizing 492 Time: 0:00:03 ( 6.99 ms/it)\n objective: 13587.380071440111\nMinimizing 507 Time: 0:00:03 ( 6.99 ms/it)\n objective: 13564.684557248402\nMinimizing 522 Time: 0:00:03 ( 6.99 ms/it)\n objective: 13560.285636726672\nMinimizing 536 Time: 0:00:03 ( 6.99 ms/it)\n objective: 13553.942275228554\nMinimizing 551 Time: 0:00:03 ( 6.99 ms/it)\n objective: 13551.432660416802\nMinimizing 566 Time: 0:00:03 ( 6.99 ms/it)\n objective: 13546.68115545698\nMinimizing 581 Time: 0:00:04 ( 6.99 ms/it)\n objective: 13542.457680659856\nMinimizing 596 Time: 0:00:04 ( 6.99 ms/it)\n objective: 13531.282504736759\nMinimizing 611 Time: 0:00:04 ( 6.99 ms/it)\n objective: 13507.686555040127\nMinimizing 626 Time: 0:00:04 ( 6.99 ms/it)\n objective: 13501.795122595562\nMinimizing 642 Time: 0:00:04 ( 7.00 ms/it)\n objective: 13488.233557449079\nMinimizing 658 Time: 0:00:04 ( 7.00 ms/it)\n objective: 13469.388268558534\nMinimizing 673 Time: 0:00:04 ( 6.99 ms/it)\n objective: 13464.469901100732\nMinimizing 688 Time: 0:00:04 ( 6.99 ms/it)\n objective: 13463.11122495486\nMinimizing 703 Time: 0:00:04 ( 6.99 ms/it)\n objective: 13462.282211793492\nMinimizing 718 Time: 0:00:05 ( 6.99 ms/it)\n objective: 13460.87544580958\nMinimizing 733 Time: 0:00:05 ( 6.98 ms/it)\n objective: 13458.552168422742\nMinimizing 748 Time: 0:00:05 ( 6.98 ms/it)\n objective: 13458.34861197463\nMinimizing 763 Time: 0:00:05 ( 6.98 ms/it)\n objective: 13455.94230646456\nMinimizing 778 Time: 0:00:05 ( 6.98 ms/it)\n objective: 13454.57298623756\nMinimizing 793 Time: 0:00:05 ( 6.98 ms/it)\n objective: 13448.645169117874\nMinimizing 808 Time: 0:00:05 ( 6.98 ms/it)\n objective: 13448.61643171533\nMinimizing 823 Time: 0:00:05 ( 6.98 ms/it)\n objective: 13447.566058433054\nMinimizing 838 Time: 0:00:05 ( 6.98 ms/it)\n objective: 13446.591300746077\nMinimizing 853 Time: 0:00:05 ( 6.98 ms/it)\n objective: 13445.207405540794\nMinimizing 868 Time: 0:00:06 ( 6.97 ms/it)\n objective: 13443.570739741306\nMinimizing 883 Time: 0:00:06 ( 6.97 ms/it)\n objective: 13439.295178485554\nMinimizing 898 Time: 0:00:06 ( 6.97 ms/it)\n objective: 13438.37428408036\nMinimizing 913 Time: 0:00:06 ( 6.97 ms/it)\n objective: 13436.702654517001\nMinimizing 928 Time: 0:00:06 ( 6.97 ms/it)\n objective: 13434.755561863487\nMinimizing 943 Time: 0:00:06 ( 6.97 ms/it)\n objective: 13432.637122840591\nMinimizing 958 Time: 0:00:06 ( 6.97 ms/it)\n objective: 13424.008808011087\nMinimizing 973 Time: 0:00:06 ( 6.97 ms/it)\n objective: 13423.410638560272\nMinimizing 988 Time: 0:00:06 ( 6.97 ms/it)\n objective: 13415.272322237295\nMinimizing 1003 Time: 0:00:06 ( 6.97 ms/it)\n objective: 13412.38322487795\nMinimizing 1018 Time: 0:00:07 ( 6.97 ms/it)\n objective: 13401.797763866794\nMinimizing 1033 Time: 0:00:07 ( 6.97 ms/it)\n objective: 13384.65012255188\nMinimizing 1048 Time: 0:00:07 ( 6.97 ms/it)\n objective: 13355.424734630244\nMinimizing 1063 Time: 0:00:07 ( 6.97 ms/it)\n objective: 13350.87330040599\nMinimizing 1078 Time: 0:00:07 ( 6.97 ms/it)\n objective: 13349.023877738146\nMinimizing 1093 Time: 0:00:07 ( 6.97 ms/it)\n objective: 13343.796538168943\nMinimizing 1108 Time: 0:00:07 ( 6.96 ms/it)\n objective: 13329.905809950593\nMinimizing 1123 Time: 0:00:07 ( 6.96 ms/it)\n objective: 13318.125331359799\nMinimizing 1138 Time: 0:00:07 ( 6.96 ms/it)\n objective: 13315.558830251946\nMinimizing 1153 Time: 0:00:08 ( 6.96 ms/it)\n objective: 13306.260512878187\nMinimizing 1168 Time: 0:00:08 ( 6.96 ms/it)\n objective: 13304.46073174938\nMinimizing 1183 Time: 0:00:08 ( 6.96 ms/it)\n objective: 13294.11897909896\nMinimizing 1198 Time: 0:00:08 ( 6.96 ms/it)\n objective: 13293.423984179884\nMinimizing 1213 Time: 0:00:08 ( 6.96 ms/it)\n objective: 13288.038433850394\nMinimizing 1228 Time: 0:00:08 ( 6.96 ms/it)\n objective: 13279.134590073561\nMinimizing 1243 Time: 0:00:08 ( 6.96 ms/it)\n objective: 13277.59210004643\nMinimizing 1257 Time: 0:00:08 ( 6.97 ms/it)\n objective: 13276.592823444138\nMinimizing 1271 Time: 0:00:08 ( 6.98 ms/it)\n objective: 13275.14079082162\nMinimizing 1285 Time: 0:00:08 ( 6.98 ms/it)\n objective: 13273.95376134415\nMinimizing 1299 Time: 0:00:09 ( 6.98 ms/it)\n objective: 13271.896365992914\nMinimizing 1313 Time: 0:00:09 ( 6.99 ms/it)\n objective: 13268.022286465668\nMinimizing 1327 Time: 0:00:09 ( 6.99 ms/it)\n objective: 13266.960992555294\nMinimizing 1341 Time: 0:00:09 ( 6.99 ms/it)\n objective: 13264.20269177662\nMinimizing 1355 Time: 0:00:09 ( 7.00 ms/it)\n objective: 13264.056451031138\nMinimizing 1369 Time: 0:00:09 ( 7.00 ms/it)\n objective: 13262.014099073072\nMinimizing 1383 Time: 0:00:09 ( 7.01 ms/it)\n objective: 13261.39514845531\nMinimizing 1397 Time: 0:00:09 ( 7.01 ms/it)\n objective: 13258.973968907303\nMinimizing 1411 Time: 0:00:09 ( 7.02 ms/it)\n objective: 13252.131282131086\nMinimizing 1425 Time: 0:00:10 ( 7.02 ms/it)\n objective: 13242.019296540384\nMinimizing 1439 Time: 0:00:10 ( 7.03 ms/it)\n objective: 13240.161053406526\nMinimizing 1453 Time: 0:00:10 ( 7.03 ms/it)\n objective: 13232.076313444995\nMinimizing 1467 Time: 0:00:10 ( 7.03 ms/it)\n objective: 13230.218340676787\nMinimizing 1481 Time: 0:00:10 ( 7.04 ms/it)\n objective: 13225.47226706834\nMinimizing 1495 Time: 0:00:10 ( 7.04 ms/it)\n objective: 13221.133445492596\nMinimizing 1508 Time: 0:00:10 ( 7.05 ms/it)\n objective: 13213.376585905382\nMinimizing 1522 Time: 0:00:10 ( 7.05 ms/it)\n objective: 13196.480471017174\nMinimizing 1536 Time: 0:00:10 ( 7.06 ms/it)\n objective: 13176.58414446337\nMinimizing 1550 Time: 0:00:10 ( 7.06 ms/it)\n objective: 13176.124813405811\nMinimizing 1564 Time: 0:00:11 ( 7.06 ms/it)\n objective: 13176.122626118784\nMinimizing 1578 Time: 0:00:11 ( 7.06 ms/it)\n objective: 13176.122256583723\nMinimizing 1592 Time: 0:00:11 ( 7.07 ms/it)\n objective: 13176.122205140986\nMinimizing 1606 Time: 0:00:11 ( 7.07 ms/it)\n objective: 13176.12100398689\nMinimizing 1620 Time: 0:00:11 ( 7.07 ms/it)\n objective: 13176.11691388562\nMinimizing 1634 Time: 0:00:11 ( 7.08 ms/it)\n objective: 13176.120781069796\nMinimizing 1648 Time: 0:00:11 ( 7.08 ms/it)\n objective: 13176.121256129176\nMinimizing 1662 Time: 0:00:11 ( 7.09 ms/it)\n objective: 13176.121651204958\nMinimizing 1676 Time: 0:00:11 ( 7.09 ms/it)\n objective: 13176.121153868342\nMinimizing 1690 Time: 0:00:11 ( 7.09 ms/it)\n objective: 13176.120604542695\nMinimizing 1704 Time: 0:00:12 ( 7.09 ms/it)\n objective: 13176.121718868628\nMinimizing 1718 Time: 0:00:12 ( 7.09 ms/it)\n objective: 13176.12001931647\nMinimizing 1732 Time: 0:00:12 ( 7.09 ms/it)\n objective: 13176.12056770247\nMinimizing 1746 Time: 0:00:12 ( 7.10 ms/it)\n objective: 13176.12362314183\nMinimizing 1760 Time: 0:00:12 ( 7.10 ms/it)\n objective: 13176.122318892172\nMinimizing 1774 Time: 0:00:12 ( 7.10 ms/it)\n objective: 13176.119855460041\nMinimizing 1774 Time: 0:00:12 ( 7.10 ms/it)\nMinimizing 14 Time: 0:00:00 ( 7.34 ms/it)\n objective: 13799.690289755828\nMinimizing 28 Time: 0:00:00 ( 7.40 ms/it)\n objective: 13814.98771423492\nMinimizing 42 Time: 0:00:00 ( 7.40 ms/it)\n objective: 13791.53328914472\nMinimizing 56 Time: 0:00:00 ( 7.39 ms/it)\n objective: 13784.223194780694\nMinimizing 70 Time: 0:00:00 ( 7.39 ms/it)\n objective: 13776.902801456363\nMinimizing 84 Time: 0:00:00 ( 7.43 ms/it)\n objective: 13773.23052871109\nMinimizing 98 Time: 0:00:00 ( 7.44 ms/it)\n objective: 13767.8482604022\nMinimizing 112 Time: 0:00:00 ( 7.43 ms/it)\n objective: 13761.47186604549\nMinimizing 126 Time: 0:00:00 ( 7.41 ms/it)\n objective: 13756.858539470975\nMinimizing 140 Time: 0:00:01 ( 7.41 ms/it)\n objective: 13737.366960877882\nMinimizing 154 Time: 0:00:01 ( 7.42 ms/it)\n objective: 13733.454930466967\nMinimizing 168 Time: 0:00:01 ( 7.44 ms/it)\n objective: 13731.63266226235\nMinimizing 182 Time: 0:00:01 ( 7.46 ms/it)\n objective: 13727.74135783894\nMinimizing 196 Time: 0:00:01 ( 7.46 ms/it)\n objective: 13723.11371396379\nMinimizing 210 Time: 0:00:01 ( 7.47 ms/it)\n objective: 13708.984335873063\nMinimizing 224 Time: 0:00:01 ( 7.47 ms/it)\n objective: 13706.349013616393\nMinimizing 238 Time: 0:00:01 ( 7.47 ms/it)\n objective: 13701.587847825613\nMinimizing 252 Time: 0:00:01 ( 7.46 ms/it)\n objective: 13687.990772626152\nMinimizing 266 Time: 0:00:01 ( 7.45 ms/it)\n objective: 13685.677862680417\nMinimizing 280 Time: 0:00:02 ( 7.44 ms/it)\n objective: 13676.107302250028\nMinimizing 294 Time: 0:00:02 ( 7.45 ms/it)\n objective: 13666.898255152362\nMinimizing 308 Time: 0:00:02 ( 7.46 ms/it)\n objective: 13650.318569806805\nMinimizing 322 Time: 0:00:02 ( 7.46 ms/it)\n objective: 13648.261527641524\nMinimizing 336 Time: 0:00:02 ( 7.46 ms/it)\n objective: 13637.863939047435\nMinimizing 350 Time: 0:00:02 ( 7.46 ms/it)\n objective: 13630.575572279373\nMinimizing 364 Time: 0:00:02 ( 7.47 ms/it)\n objective: 13614.88660370166\nMinimizing 378 Time: 0:00:02 ( 7.47 ms/it)\n objective: 13610.071531565009\nMinimizing 392 Time: 0:00:02 ( 7.46 ms/it)\n objective: 13607.239932400262\nMinimizing 406 Time: 0:00:03 ( 7.46 ms/it)\n objective: 13602.74249920054\nMinimizing 420 Time: 0:00:03 ( 7.46 ms/it)\n objective: 13598.932864674047\nMinimizing 434 Time: 0:00:03 ( 7.46 ms/it)\n objective: 13577.700711742516\nMinimizing 448 Time: 0:00:03 ( 7.46 ms/it)\n objective: 13540.98406420846\nMinimizing 462 Time: 0:00:03 ( 7.46 ms/it)\n objective: 13532.706236090708\nMinimizing 476 Time: 0:00:03 ( 7.46 ms/it)\n objective: 13520.379128555374\nMinimizing 490 Time: 0:00:03 ( 7.46 ms/it)\n objective: 13513.302198592966\nMinimizing 504 Time: 0:00:03 ( 7.46 ms/it)\n objective: 13508.088088117613\nMinimizing 518 Time: 0:00:03 ( 7.46 ms/it)\n objective: 13500.338741727523\nMinimizing 532 Time: 0:00:03 ( 7.45 ms/it)\n objective: 13492.837756229885\nMinimizing 546 Time: 0:00:04 ( 7.46 ms/it)\n objective: 13465.566937206575\nMinimizing 560 Time: 0:00:04 ( 7.46 ms/it)\n objective: 13462.810806506597\nMinimizing 574 Time: 0:00:04 ( 7.46 ms/it)\n objective: 13456.77155015275\nMinimizing 588 Time: 0:00:04 ( 7.46 ms/it)\n objective: 13455.020788706599\nMinimizing 602 Time: 0:00:04 ( 7.46 ms/it)\n objective: 13453.754699343932\nMinimizing 616 Time: 0:00:04 ( 7.46 ms/it)\n objective: 13450.692854851492\nMinimizing 630 Time: 0:00:04 ( 7.46 ms/it)\n objective: 13448.528232286815\nMinimizing 644 Time: 0:00:04 ( 7.46 ms/it)\n objective: 13443.455904456314\nMinimizing 658 Time: 0:00:04 ( 7.46 ms/it)\n objective: 13442.133160955593\nMinimizing 672 Time: 0:00:05 ( 7.45 ms/it)\n objective: 13437.000825261872\nMinimizing 686 Time: 0:00:05 ( 7.45 ms/it)\n objective: 13426.941025741398\nMinimizing 700 Time: 0:00:05 ( 7.45 ms/it)\n objective: 13426.602804233313\nMinimizing 714 Time: 0:00:05 ( 7.45 ms/it)\n objective: 13424.676702505378\nMinimizing 727 Time: 0:00:05 ( 7.46 ms/it)\n objective: 13423.693517994077\nMinimizing 740 Time: 0:00:05 ( 7.47 ms/it)\n objective: 13422.970916159189\nMinimizing 753 Time: 0:00:05 ( 7.48 ms/it)\n objective: 13422.366024823343\nMinimizing 766 Time: 0:00:05 ( 7.49 ms/it)\n objective: 13417.735615639387\nMinimizing 779 Time: 0:00:05 ( 7.50 ms/it)\n objective: 13417.103077060594\nMinimizing 793 Time: 0:00:05 ( 7.50 ms/it)\n objective: 13412.083820849708\nMinimizing 807 Time: 0:00:06 ( 7.49 ms/it)\n objective: 13402.651882951526\nMinimizing 821 Time: 0:00:06 ( 7.49 ms/it)\n objective: 13394.434166434541\nMinimizing 835 Time: 0:00:06 ( 7.49 ms/it)\n objective: 13375.566997549904\nMinimizing 849 Time: 0:00:06 ( 7.49 ms/it)\n objective: 13366.072629510745\nMinimizing 863 Time: 0:00:06 ( 7.49 ms/it)\n objective: 13356.16795565781\nMinimizing 877 Time: 0:00:06 ( 7.48 ms/it)\n objective: 13344.647086932076\nMinimizing 891 Time: 0:00:06 ( 7.48 ms/it)\n objective: 13343.92449473987\nMinimizing 905 Time: 0:00:06 ( 7.48 ms/it)\n objective: 13340.338483102038\nMinimizing 919 Time: 0:00:06 ( 7.49 ms/it)\n objective: 13338.464328736067\nMinimizing 933 Time: 0:00:06 ( 7.48 ms/it)\n objective: 13336.292964622058\nMinimizing 947 Time: 0:00:07 ( 7.48 ms/it)\n objective: 13334.4332020742\nMinimizing 961 Time: 0:00:07 ( 7.49 ms/it)\n objective: 13331.56700019956\nMinimizing 975 Time: 0:00:07 ( 7.49 ms/it)\n objective: 13328.17346719306\nMinimizing 989 Time: 0:00:07 ( 7.48 ms/it)\n objective: 13325.702470719727\nMinimizing 1003 Time: 0:00:07 ( 7.48 ms/it)\n objective: 13324.023172127942\nMinimizing 1017 Time: 0:00:07 ( 7.48 ms/it)\n objective: 13320.680490710627\nMinimizing 1031 Time: 0:00:07 ( 7.48 ms/it)\n objective: 13318.093795479333\nMinimizing 1045 Time: 0:00:07 ( 7.48 ms/it)\n objective: 13316.52141299202\nMinimizing 1059 Time: 0:00:07 ( 7.47 ms/it)\n objective: 13316.400689379763\nMinimizing 1073 Time: 0:00:08 ( 7.48 ms/it)\n objective: 13313.380829079018\nMinimizing 1087 Time: 0:00:08 ( 7.48 ms/it)\n objective: 13311.809365524692\nMinimizing 1100 Time: 0:00:08 ( 7.48 ms/it)\n objective: 13311.647481701904\nMinimizing 1114 Time: 0:00:08 ( 7.48 ms/it)\n objective: 13309.703775092261\nMinimizing 1128 Time: 0:00:08 ( 7.48 ms/it)\n objective: 13309.600251360622\nMinimizing 1142 Time: 0:00:08 ( 7.47 ms/it)\n objective: 13307.588390650577\nMinimizing 1156 Time: 0:00:08 ( 7.47 ms/it)\n objective: 13302.61931799214\nMinimizing 1170 Time: 0:00:08 ( 7.47 ms/it)\n objective: 13296.309421655256\nMinimizing 1184 Time: 0:00:08 ( 7.47 ms/it)\n objective: 13292.065207682012\nMinimizing 1198 Time: 0:00:08 ( 7.47 ms/it)\n objective: 13288.750471524778\nMinimizing 1212 Time: 0:00:09 ( 7.46 ms/it)\n objective: 13284.53302863962\nMinimizing 1226 Time: 0:00:09 ( 7.46 ms/it)\n objective: 13278.725620617566\nMinimizing 1240 Time: 0:00:09 ( 7.47 ms/it)\n objective: 13272.254949326918\nMinimizing 1254 Time: 0:00:09 ( 7.46 ms/it)\n objective: 13247.357388104036\nMinimizing 1268 Time: 0:00:09 ( 7.46 ms/it)\n objective: 13235.296743677274\nMinimizing 1282 Time: 0:00:09 ( 7.46 ms/it)\n objective: 13231.563146665052\nMinimizing 1296 Time: 0:00:09 ( 7.46 ms/it)\n objective: 13216.005783504253\nMinimizing 1310 Time: 0:00:09 ( 7.46 ms/it)\n objective: 13207.247931517064\nMinimizing 1324 Time: 0:00:09 ( 7.46 ms/it)\n objective: 13207.04314413754\nMinimizing 1338 Time: 0:00:09 ( 7.45 ms/it)\n objective: 13200.474647403462\nMinimizing 1352 Time: 0:00:10 ( 7.45 ms/it)\n objective: 13197.289365754332\nMinimizing 1366 Time: 0:00:10 ( 7.45 ms/it)\n objective: 13192.429658603141\nMinimizing 1380 Time: 0:00:10 ( 7.45 ms/it)\n objective: 13187.930481763717\nMinimizing 1394 Time: 0:00:10 ( 7.45 ms/it)\n objective: 13182.061168171422\nMinimizing 1408 Time: 0:00:10 ( 7.45 ms/it)\n objective: 13178.252241522467\nMinimizing 1422 Time: 0:00:10 ( 7.45 ms/it)\n objective: 13165.31283846611\nMinimizing 1436 Time: 0:00:10 ( 7.45 ms/it)\n objective: 13157.735247360528\nMinimizing 1450 Time: 0:00:10 ( 7.45 ms/it)\n objective: 13157.199392885057\nMinimizing 1464 Time: 0:00:10 ( 7.45 ms/it)\n objective: 13156.50308670917\nMinimizing 1478 Time: 0:00:11 ( 7.45 ms/it)\n objective: 13154.813020916175\nMinimizing 1492 Time: 0:00:11 ( 7.45 ms/it)\n objective: 13150.31367202608\nMinimizing 1506 Time: 0:00:11 ( 7.45 ms/it)\n objective: 13147.247403814355\nMinimizing 1520 Time: 0:00:11 ( 7.45 ms/it)\n objective: 13140.819616088833\nMinimizing 1534 Time: 0:00:11 ( 7.44 ms/it)\n objective: 13137.304527376531\nMinimizing 1548 Time: 0:00:11 ( 7.44 ms/it)\n objective: 13137.00180729988\nMinimizing 1562 Time: 0:00:11 ( 7.44 ms/it)\n objective: 13134.553269421682\nMinimizing 1576 Time: 0:00:11 ( 7.44 ms/it)\n objective: 13134.482516308737\nMinimizing 1590 Time: 0:00:11 ( 7.44 ms/it)\n objective: 13134.427148029063\nMinimizing 1604 Time: 0:00:11 ( 7.44 ms/it)\n objective: 13134.433973989173\nMinimizing 1618 Time: 0:00:12 ( 7.44 ms/it)\n objective: 13134.433782834356\nMinimizing 1632 Time: 0:00:12 ( 7.44 ms/it)\n objective: 13134.424998009024\nMinimizing 1646 Time: 0:00:12 ( 7.44 ms/it)\n objective: 13134.432448809108\nMinimizing 1660 Time: 0:00:12 ( 7.44 ms/it)\n objective: 13134.439745825148\nMinimizing 1674 Time: 0:00:12 ( 7.46 ms/it)\n objective: 13134.435048403218\nMinimizing 1688 Time: 0:00:12 ( 7.46 ms/it)\n objective: 13134.438355184306\nMinimizing 1702 Time: 0:00:12 ( 7.46 ms/it)\n objective: 13134.434327619849\nMinimizing 1716 Time: 0:00:12 ( 7.46 ms/it)\n objective: 13134.436954488774\nMinimizing 1730 Time: 0:00:12 ( 7.45 ms/it)\n objective: 13134.431615684036\nMinimizing 1744 Time: 0:00:12 ( 7.45 ms/it)\n objective: 13134.423937564177\nMinimizing 1758 Time: 0:00:13 ( 7.45 ms/it)\n objective: 13134.433076300586\nMinimizing 1772 Time: 0:00:13 ( 7.46 ms/it)\n objective: 13134.43327911022\nMinimizing 1786 Time: 0:00:13 ( 7.46 ms/it)\n objective: 13134.4345713199\nMinimizing 1799 Time: 0:00:13 ( 7.46 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0182\n0.0140\n-1.30\n0.1942\n0.6870\n\n\nTest: c5.1\n-0.0403\n0.0429\n-0.94\n0.3481\n1.3839\n\n\nTest: c234.15\n0.0162\n0.1678\n0.10\n0.9231\n0.6312\n\n\nTest: c2.34\n0.0099\n0.0758\n0.13\n0.8958\n0.9852\n\n\nTest: c3.4\n-0.0137\n0.0445\n-0.31\n0.7584\n1.5296\n\n\na1\n0.3207\n0.0480\n6.67\n<1e-10\n\n\n\nSex: Boys\n0.2052\n0.0140\n14.63\n<1e-47\n\n\n\nTest: c5.1 & a1\n0.6160\n0.1469\n4.19\n<1e-04\n\n\n\nTest: c234.15 & a1\n0.0641\n0.5728\n0.11\n0.9108\n\n\n\nTest: c2.34 & a1\n-0.0629\n0.2584\n-0.24\n0.8077\n\n\n\nTest: c3.4 & a1\n0.0742\n0.1533\n0.48\n0.6284\n\n\n\nTest: c5.1 & Sex: Boys\n0.1573\n0.0429\n3.67\n0.0002\n\n\n\nTest: c234.15 & Sex: Boys\n-0.8202\n0.1678\n-4.89\n<1e-05\n\n\n\nTest: c2.34 & Sex: Boys\n-0.1837\n0.0758\n-2.42\n0.0154\n\n\n\nTest: c3.4 & Sex: Boys\n-0.0465\n0.0445\n-1.04\n0.2964\n\n\n\na1 & Sex: Boys\n0.0508\n0.0480\n1.06\n0.2908\n\n\n\nTest: c5.1 & a1 & Sex: Boys\n0.0384\n0.1469\n0.26\n0.7937\n\n\n\nTest: c234.15 & a1 & Sex: Boys\n-0.3568\n0.5728\n-0.62\n0.5333\n\n\n\nTest: c2.34 & a1 & Sex: Boys\n0.1448\n0.2584\n0.56\n0.5752\n\n\n\nTest: c3.4 & a1 & Sex: Boys\n0.0608\n0.1533\n0.40\n0.6916\n\n\n\nResidual\n0.0000\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_Child_SDC)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\nChild\n(Intercept)\n0.45678315\n0.67585735\n\n\n\n\n\n\n\nTest: Star_r\n0.53615183\n0.73222389\n+0.75\n\n\n\n\n\n\nTest: S20_r\n0.46625602\n0.68282942\n+0.04\n+0.68\n\n\n\n\n\nTest: SLJ\n0.21929072\n0.46828487\n-0.51\n+0.18\n+0.83\n\n\n\n\nTest: BPT\n0.79542520\n0.89186613\n-0.17\n-0.37\n-0.35\n-0.24\n\n\nResidual\n\n0.00000000\n0.00000691\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_Child_HeC)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\nChild\n(Intercept)\n0.361522753\n0.601267621\n\n\n\n\n\n\n\nTest: Star_r\n0.179715119\n0.423928200\n+0.11\n\n\n\n\n\n\nTest: S20_r\n0.057451969\n0.239691403\n-0.40\n+0.35\n\n\n\n\n\nTest: SLJ\n0.119372818\n0.345503717\n-0.32\n+0.04\n-0.60\n\n\n\n\nTest: BPT\n0.035272708\n0.187810298\n-0.12\n-0.26\n-0.19\n+0.39\n\n\nResidual\n\n0.000000049\n0.000221448\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_Child_HyC)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\nChild\n(Intercept)\n0.46286734\n0.68034355\n\n\n\n\n\n\n\nTest: BPT-other\n1.27680406\n1.12995755\n+0.88\n\n\n\n\n\n\nTest: Star-End\n0.95080634\n0.97509299\n+0.46\n-0.01\n\n\n\n\n\nTest: S20-Star\n1.26380582\n1.12419118\n-0.10\n-0.15\n+0.17\n\n\n\n\nTest: SLJ-S20\n0.29366656\n0.54191011\n-0.11\n+0.22\n-0.67\n+0.18\n\n\nResidual\n\n0.00000000\n0.00001522\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_Child_PCA)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\nChild\n(Intercept)\n0.47198187\n0.68700937\n\n\n\n\n\n\n\nTest: c5.1\n1.91517989\n1.38390025\n-0.11\n\n\n\n\n\n\nTest: c234.15\n0.39840370\n0.63119228\n+0.83\n-0.48\n\n\n\n\n\nTest: c2.34\n0.97057584\n0.98517807\n+0.69\n-0.15\n+0.88\n\n\n\n\nTest: c3.4\n2.33977540\n1.52963244\n+0.12\n+0.08\n+0.40\n+0.63\n\n\nResidual\n\n0.00000000\n0.00001040\n\n\n\n\n\n\n\n\n\nThe CPs for the various contrasts are in line with expectations. For the SDC we observe substantial negative CPs between neighboring contrasts. For the orthogonal HeC, all CPs are small; they are uncorrelated. HyC contains some of the SDC contrasts and we observe again the negative CPs. The (roughly) PCA-based contrasts are small with one exception; there is a sizeable CP of +.41 between GM and the core of adjusted physical fitness (c234.15).\nDo these differences in CPs imply that we can move to zcpLMMs when we have orthogonal contrasts? We pursue this question with by refitting the four LMMs with zerocorr() and compare the goodness of fit.\n\nbegin\n f_Child0 = @formula zScore ~\n 1 + Test * a1 * Sex + zerocorr(1 + Test | Child)\n m_Child_SDC0 = fit(MixedModel, f_Child0, dat; contrasts=contr1)\n m_Child_HeC0 = fit(MixedModel, f_Child0, dat; contrasts=contr2)\n m_Child_HyC0 = fit(MixedModel, f_Child0, dat; contrasts=contr3)\n m_Child_PCA0 = fit(MixedModel, f_Child0, dat; contrasts=contr4)\nend\n\nMinimizing 20 Time: 0:00:00 ( 5.16 ms/it)\n objective: 13801.840082943849\nMinimizing 41 Time: 0:00:00 ( 5.08 ms/it)\n objective: 13798.088511524951\nMinimizing 62 Time: 0:00:00 ( 5.06 ms/it)\n objective: 13796.729778581182\nMinimizing 82 Time: 0:00:00 ( 5.05 ms/it)\n objective: 13796.212345608132\nMinimizing 102 Time: 0:00:00 ( 5.07 ms/it)\n objective: 13796.20566915371\nMinimizing 121 Time: 0:00:00 ( 5.12 ms/it)\n objective: 13796.201088517759\nMinimizing 141 Time: 0:00:00 ( 5.10 ms/it)\n objective: 13796.20062481871\nMinimizing 161 Time: 0:00:00 ( 5.10 ms/it)\n objective: 13796.200603124427\nMinimizing 166 Time: 0:00:00 ( 5.09 ms/it)\nMinimizing 20 Time: 0:00:00 ( 5.02 ms/it)\n objective: 13777.86936071649\nMinimizing 40 Time: 0:00:00 ( 5.05 ms/it)\n objective: 13751.435654438266\nMinimizing 60 Time: 0:00:00 ( 5.10 ms/it)\n objective: 13744.158578651432\nMinimizing 79 Time: 0:00:00 ( 5.21 ms/it)\n objective: 13722.102325285614\nMinimizing 99 Time: 0:00:00 ( 5.19 ms/it)\n objective: 13698.864914585596\nMinimizing 119 Time: 0:00:00 ( 5.17 ms/it)\n objective: 13675.037724363621\nMinimizing 139 Time: 0:00:00 ( 5.15 ms/it)\n objective: 13669.500265406032\nMinimizing 159 Time: 0:00:00 ( 5.19 ms/it)\n objective: 13666.601484438732\nMinimizing 178 Time: 0:00:00 ( 5.20 ms/it)\n objective: 13664.62857516346\nMinimizing 198 Time: 0:00:01 ( 5.19 ms/it)\n objective: 13658.994251554324\nMinimizing 218 Time: 0:00:01 ( 5.18 ms/it)\n objective: 13651.869567634305\nMinimizing 238 Time: 0:00:01 ( 5.20 ms/it)\n objective: 13647.157274289502\nMinimizing 258 Time: 0:00:01 ( 5.19 ms/it)\n objective: 13643.99421763185\nMinimizing 278 Time: 0:00:01 ( 5.20 ms/it)\n objective: 13634.071910279778\nMinimizing 298 Time: 0:00:01 ( 5.20 ms/it)\n objective: 13611.253260503185\nMinimizing 318 Time: 0:00:01 ( 5.20 ms/it)\n objective: 13572.593989700017\nMinimizing 338 Time: 0:00:01 ( 5.19 ms/it)\n objective: 13553.878759422732\nMinimizing 358 Time: 0:00:01 ( 5.19 ms/it)\n objective: 13547.856240491867\nMinimizing 378 Time: 0:00:01 ( 5.18 ms/it)\n objective: 13502.19686150935\nMinimizing 398 Time: 0:00:02 ( 5.18 ms/it)\n objective: 13474.224490691908\nMinimizing 418 Time: 0:00:02 ( 5.17 ms/it)\n objective: 13425.90477785928\nMinimizing 438 Time: 0:00:02 ( 5.16 ms/it)\n objective: 13412.427282714802\nMinimizing 458 Time: 0:00:02 ( 5.18 ms/it)\n objective: 13403.801051929535\nMinimizing 478 Time: 0:00:02 ( 5.18 ms/it)\n objective: 13388.673016598339\nMinimizing 498 Time: 0:00:02 ( 5.18 ms/it)\n objective: 13386.288818941037\nMinimizing 518 Time: 0:00:02 ( 5.18 ms/it)\n objective: 13377.441475318126\nMinimizing 538 Time: 0:00:02 ( 5.17 ms/it)\n objective: 13370.259104194884\nMinimizing 558 Time: 0:00:02 ( 5.17 ms/it)\n objective: 13367.369029424292\nMinimizing 578 Time: 0:00:02 ( 5.17 ms/it)\n objective: 13365.101712476157\nMinimizing 599 Time: 0:00:03 ( 5.17 ms/it)\n objective: 13361.308738745414\nMinimizing 619 Time: 0:00:03 ( 5.16 ms/it)\n objective: 13357.450535290518\nMinimizing 640 Time: 0:00:03 ( 5.16 ms/it)\n objective: 13350.20610579828\nMinimizing 660 Time: 0:00:03 ( 5.16 ms/it)\n objective: 13334.739287137127\nMinimizing 680 Time: 0:00:03 ( 5.15 ms/it)\n objective: 13328.795718790963\nMinimizing 700 Time: 0:00:03 ( 5.15 ms/it)\n objective: 13283.718608480907\nMinimizing 720 Time: 0:00:03 ( 5.15 ms/it)\n objective: 13267.202844101223\nMinimizing 740 Time: 0:00:03 ( 5.15 ms/it)\n objective: 13193.519004086498\nMinimizing 760 Time: 0:00:03 ( 5.16 ms/it)\n objective: 13183.016734053206\nMinimizing 779 Time: 0:00:04 ( 5.16 ms/it)\n objective: 13178.856704928665\nMinimizing 799 Time: 0:00:04 ( 5.16 ms/it)\n objective: 13167.353935667721\nMinimizing 819 Time: 0:00:04 ( 5.16 ms/it)\n objective: 13159.669089089628\nMinimizing 839 Time: 0:00:04 ( 5.16 ms/it)\n objective: 13145.049961976925\nMinimizing 859 Time: 0:00:04 ( 5.16 ms/it)\n objective: 13103.90558579212\nMinimizing 880 Time: 0:00:04 ( 5.15 ms/it)\n objective: 13068.957720565246\nMinimizing 900 Time: 0:00:04 ( 5.15 ms/it)\n objective: 13067.831647235886\nMinimizing 920 Time: 0:00:04 ( 5.15 ms/it)\n objective: 13067.78037017677\nMinimizing 941 Time: 0:00:04 ( 5.15 ms/it)\n objective: 13067.78825339333\nMinimizing 962 Time: 0:00:04 ( 5.14 ms/it)\n objective: 13067.795915521361\nMinimizing 984 Time: 0:00:05 ( 5.13 ms/it)\n objective: 13067.718710527042\nMinimizing 988 Time: 0:00:05 ( 5.13 ms/it)\nMinimizing 22 Time: 0:00:00 ( 4.69 ms/it)\n objective: 13790.073907703707\nMinimizing 44 Time: 0:00:00 ( 4.69 ms/it)\n objective: 13771.670965021149\nMinimizing 66 Time: 0:00:00 ( 4.71 ms/it)\n objective: 13755.664061426653\nMinimizing 87 Time: 0:00:00 ( 4.76 ms/it)\n objective: 13747.465550982335\nMinimizing 108 Time: 0:00:00 ( 4.82 ms/it)\n objective: 13730.301500466248\nMinimizing 130 Time: 0:00:00 ( 4.80 ms/it)\n objective: 13688.67645685529\nMinimizing 151 Time: 0:00:00 ( 4.80 ms/it)\n objective: 13608.293077886097\nMinimizing 172 Time: 0:00:00 ( 4.86 ms/it)\n objective: 13599.473504072888\nMinimizing 194 Time: 0:00:00 ( 4.85 ms/it)\n objective: 13591.277410855968\nMinimizing 216 Time: 0:00:01 ( 4.84 ms/it)\n objective: 13581.084125263322\nMinimizing 238 Time: 0:00:01 ( 4.83 ms/it)\n objective: 13547.159759337155\nMinimizing 260 Time: 0:00:01 ( 4.82 ms/it)\n objective: 13539.059128529203\nMinimizing 282 Time: 0:00:01 ( 4.81 ms/it)\n objective: 13536.754523006224\nMinimizing 304 Time: 0:00:01 ( 4.81 ms/it)\n objective: 13534.084209389424\nMinimizing 326 Time: 0:00:01 ( 4.80 ms/it)\n objective: 13528.59897845464\nMinimizing 347 Time: 0:00:01 ( 4.80 ms/it)\n objective: 13528.056359095957\nMinimizing 369 Time: 0:00:01 ( 4.80 ms/it)\n objective: 13519.60772759054\nMinimizing 391 Time: 0:00:01 ( 4.80 ms/it)\n objective: 13492.50350522164\nMinimizing 412 Time: 0:00:01 ( 4.80 ms/it)\n objective: 13409.822939818652\nMinimizing 433 Time: 0:00:02 ( 4.82 ms/it)\n objective: 13373.494186909578\nMinimizing 455 Time: 0:00:02 ( 4.82 ms/it)\n objective: 13353.467999600267\nMinimizing 477 Time: 0:00:02 ( 4.81 ms/it)\n objective: 13307.067361501671\nMinimizing 498 Time: 0:00:02 ( 4.82 ms/it)\n objective: 13301.080914859456\nMinimizing 520 Time: 0:00:02 ( 4.81 ms/it)\n objective: 13262.430734973896\nMinimizing 542 Time: 0:00:02 ( 4.81 ms/it)\n objective: 13220.083469635225\nMinimizing 564 Time: 0:00:02 ( 4.81 ms/it)\n objective: 13143.58809747886\nMinimizing 585 Time: 0:00:02 ( 4.81 ms/it)\n objective: 13109.042536250374\nMinimizing 606 Time: 0:00:02 ( 4.81 ms/it)\n objective: 13067.524690211401\nMinimizing 628 Time: 0:00:03 ( 4.81 ms/it)\n objective: 13065.942942702968\nMinimizing 650 Time: 0:00:03 ( 4.80 ms/it)\n objective: 13066.532255472805\nMinimizing 672 Time: 0:00:03 ( 4.80 ms/it)\n objective: 13066.566853468365\nMinimizing 694 Time: 0:00:03 ( 4.80 ms/it)\n objective: 13066.880043387835\nMinimizing 707 Time: 0:00:03 ( 4.79 ms/it)\nMinimizing 22 Time: 0:00:00 ( 4.68 ms/it)\n objective: 13794.154003683772\nMinimizing 43 Time: 0:00:00 ( 4.76 ms/it)\n objective: 13756.627746982058\nMinimizing 64 Time: 0:00:00 ( 4.89 ms/it)\n objective: 13705.382967517973\nMinimizing 85 Time: 0:00:00 ( 4.86 ms/it)\n objective: 13685.52768416985\nMinimizing 107 Time: 0:00:00 ( 4.84 ms/it)\n objective: 13671.312378892324\nMinimizing 129 Time: 0:00:00 ( 4.82 ms/it)\n objective: 13661.69728603626\nMinimizing 151 Time: 0:00:00 ( 4.81 ms/it)\n objective: 13604.925277998052\nMinimizing 172 Time: 0:00:00 ( 4.83 ms/it)\n objective: 13596.698452328332\nMinimizing 193 Time: 0:00:00 ( 4.84 ms/it)\n objective: 13574.922962707264\nMinimizing 215 Time: 0:00:01 ( 4.83 ms/it)\n objective: 13508.247427506576\nMinimizing 237 Time: 0:00:01 ( 4.82 ms/it)\n objective: 13497.034128687104\nMinimizing 259 Time: 0:00:01 ( 4.82 ms/it)\n objective: 13456.883701287552\nMinimizing 280 Time: 0:00:01 ( 4.82 ms/it)\n objective: 13439.835180333714\nMinimizing 302 Time: 0:00:01 ( 4.81 ms/it)\n objective: 13437.083616760072\nMinimizing 323 Time: 0:00:01 ( 4.81 ms/it)\n objective: 13419.10168892931\nMinimizing 344 Time: 0:00:01 ( 4.81 ms/it)\n objective: 13357.418124709591\nMinimizing 366 Time: 0:00:01 ( 4.81 ms/it)\n objective: 13343.382880526886\nMinimizing 387 Time: 0:00:01 ( 4.82 ms/it)\n objective: 13331.687094072098\nMinimizing 407 Time: 0:00:01 ( 4.84 ms/it)\n objective: 13320.712612247444\nMinimizing 428 Time: 0:00:02 ( 4.84 ms/it)\n objective: 13238.815642997404\nMinimizing 449 Time: 0:00:02 ( 4.84 ms/it)\n objective: 13173.645232293493\nMinimizing 471 Time: 0:00:02 ( 4.83 ms/it)\n objective: 13152.210993701869\nMinimizing 492 Time: 0:00:02 ( 4.84 ms/it)\n objective: 13131.427762340492\nMinimizing 513 Time: 0:00:02 ( 4.84 ms/it)\n objective: 13106.628157707397\nMinimizing 535 Time: 0:00:02 ( 4.83 ms/it)\n objective: 13053.284148149643\nMinimizing 557 Time: 0:00:02 ( 4.83 ms/it)\n objective: 13050.517145347476\nMinimizing 579 Time: 0:00:02 ( 4.83 ms/it)\n objective: 13050.065897029577\nMinimizing 601 Time: 0:00:02 ( 4.82 ms/it)\n objective: 13049.660644023228\nMinimizing 622 Time: 0:00:03 ( 4.83 ms/it)\n objective: 13050.174055599113\nMinimizing 644 Time: 0:00:03 ( 4.82 ms/it)\n objective: 13050.670904996397\nMinimizing 658 Time: 0:00:03 ( 4.82 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Child\n\n\n(Intercept)\n-0.0180\n0.0141\n-1.28\n0.2009\n0.7670\n\n\nTest: c5.1\n-0.0390\n0.0447\n-0.87\n0.3828\n1.2320\n\n\nTest: c234.15\n-0.0073\n0.1700\n-0.04\n0.9660\n0.0698\n\n\nTest: c2.34\n0.0105\n0.0729\n0.14\n0.8859\n1.4253\n\n\nTest: c3.4\n-0.0120\n0.0447\n-0.27\n0.7876\n1.1523\n\n\na1\n0.3210\n0.0483\n6.64\n<1e-10\n\n\n\nSex: Boys\n0.2051\n0.0141\n14.54\n<1e-47\n\n\n\nTest: c5.1 & a1\n0.6230\n0.1530\n4.07\n<1e-04\n\n\n\nTest: c234.15 & a1\n0.0763\n0.5809\n0.13\n0.8955\n\n\n\nTest: c2.34 & a1\n-0.0799\n0.2480\n-0.32\n0.7473\n\n\n\nTest: c3.4 & a1\n0.0901\n0.1538\n0.59\n0.5581\n\n\n\nTest: c5.1 & Sex: Boys\n0.1581\n0.0447\n3.54\n0.0004\n\n\n\nTest: c234.15 & Sex: Boys\n-0.8295\n0.1700\n-4.88\n<1e-05\n\n\n\nTest: c2.34 & Sex: Boys\n-0.1587\n0.0729\n-2.18\n0.0296\n\n\n\nTest: c3.4 & Sex: Boys\n-0.0543\n0.0447\n-1.21\n0.2244\n\n\n\na1 & Sex: Boys\n0.0494\n0.0483\n1.02\n0.3064\n\n\n\nTest: c5.1 & a1 & Sex: Boys\n0.0278\n0.1530\n0.18\n0.8559\n\n\n\nTest: c234.15 & a1 & Sex: Boys\n-0.4123\n0.5809\n-0.71\n0.4778\n\n\n\nTest: c2.34 & a1 & Sex: Boys\n0.1670\n0.2480\n0.67\n0.5006\n\n\n\nTest: c3.4 & a1 & Sex: Boys\n0.0888\n0.1538\n0.58\n0.5639\n\n\n\nResidual\n0.0000\n\n\n\n\n\n\n\n\n\n\nMixedModels.likelihoodratiotest(m_Child_SDC0, m_Child_SDC)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Child)\n26\n13796\n\n\n\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 + Test | Child)\n36\n13109\n687\n10\n<1e-99\n\n\n\n\n\n\nMixedModels.likelihoodratiotest(m_Child_HeC0, m_Child_HeC)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Child)\n26\n13068\n\n\n\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 + Test | Child)\n36\n13300\n-232\n10\nNaN\n\n\n\n\n\n\nMixedModels.likelihoodratiotest(m_Child_HyC0, m_Child_HyC)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Child)\n26\n13066\n\n\n\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 + Test | Child)\n36\n13176\n-110\n10\nNaN\n\n\n\n\n\n\nMixedModels.likelihoodratiotest(m_Child_PCA0, m_Child_PCA)\n\n\n\n\n\nmodel-dof\ndeviance\nχ²\nχ²-dof\nP(>χ²)\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + zerocorr(1 + Test | Child)\n26\n13049\n\n\n\n\n\nzScore ~ 1 + Test + a1 + Sex + Test & a1 + Test & Sex + a1 & Sex + Test & a1 & Sex + (1 + Test | Child)\n36\n13134\n-85\n10\nNaN\n\n\n\n\n\nObviously, we can not drop CPs from any of the LMMs. The full LMMs all have the same objective, but we can compare the goodness-of-fit statistics of zcpLMMs more directly.\n\nbegin\n zcpLMM = [\"SDC0\", \"HeC0\", \"HyC0\", \"PCA0\"]\n mods = [m_Child_SDC0, m_Child_HeC0, m_Child_HyC0, m_Child_PCA0]\n gof_summary = sort!(\n DataFrame(;\n zcpLMM=zcpLMM,\n dof=dof.(mods),\n deviance=deviance.(mods),\n AIC=aic.(mods),\n BIC=bic.(mods),\n ),\n :deviance,\n )\nend\n\n4×5 DataFrame\n\n\n\nRow\nzcpLMM\ndof\ndeviance\nAIC\nBIC\n\n\n\nString\nInt64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\nPCA0\n26\n13049.4\n13101.4\n13270.8\n\n\n2\nHyC0\n26\n13065.8\n13117.8\n13287.2\n\n\n3\nHeC0\n26\n13067.7\n13119.7\n13289.1\n\n\n4\nSDC0\n26\n13796.2\n13848.2\n14017.6\n\n\n\n\n\n\nThe best fit was obtained for the PCA-based zcpLMM. Somewhat surprisingly the second best fit was obtained for the SDC. The relatively poor performance of HeC-based zcpLMM is puzzling to me. I thought it might be related to imbalance in design in the present data, but this does not appear to be the case. The same comparison of SequentialDifferenceCoding and Helmert Coding also showed a worse fit for the zcp-HeC LMM than the zcp-SDC LMM.", + "crumbs": [ + "Contrast coding", + "Mixed Models Tutorial: Contrast Coding" + ] + }, + { + "objectID": "contrasts_fggk21.html#vcs-and-cps-depend-on-random-factor", + "href": "contrasts_fggk21.html#vcs-and-cps-depend-on-random-factor", + "title": "Mixed Models Tutorial: Contrast Coding", + "section": "3.3 VCs and CPs depend on random factor", + "text": "3.3 VCs and CPs depend on random factor\nVCs and CPs resulting from a set of test contrasts can also be estimated for the random factor School. Of course, these VCs and CPs may look different from the ones we just estimated for Child.\nThe effect of age (i.e., developmental gain) varies within School. Therefore, we also include its VCs and CPs in this model; the school-related VC for Sex was not significant.\n\nf_School = @formula zScore ~\n 1 + Test * a1 * Sex + (1 + Test + a1 | School);\nm_School_SeqDiff = fit(MixedModel, f_School, dat; contrasts=contr1);\nm_School_Helmert = fit(MixedModel, f_School, dat; contrasts=contr2);\nm_School_Hypo = fit(MixedModel, f_School, dat; contrasts=contr3);\nm_School_PCA = fit(MixedModel, f_School, dat; contrasts=contr4);\n\nMinimizing 70 Time: 0:00:00 ( 1.44 ms/it)\n objective: 13811.64529664444\nMinimizing 205 Time: 0:00:00 ( 0.98 ms/it)\n objective: 13774.773213004657\nMinimizing 341 Time: 0:00:00 ( 0.89 ms/it)\n objective: 13773.325252364972\nMinimizing 477 Time: 0:00:00 ( 0.85 ms/it)\n objective: 13772.694297112774\nMinimizing 611 Time: 0:00:00 ( 0.83 ms/it)\n objective: 13772.671154527705\nMinimizing 745 Time: 0:00:00 ( 0.82 ms/it)\n objective: 13772.670793562127\nMinimizing 792 Time: 0:00:00 ( 0.81 ms/it)\n\nMinimizing 793 Time: 0:00:00 ( 0.81 ms/it)\n objective: 13772.670780690043\nMinimizing 135 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13840.443004276474\nMinimizing 271 Time: 0:00:00 ( 0.74 ms/it)\n objective: 13805.146355123181\nMinimizing 405 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13801.599266262609\nMinimizing 540 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13801.36258054134\nMinimizing 675 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13801.3294428708\nMinimizing 810 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13801.318010056366\nMinimizing 944 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13801.315003946416\nMinimizing 1080 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13801.316409231427\nMinimizing 1214 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13801.31406667234\nMinimizing 1271 Time: 0:00:00 ( 0.75 ms/it)\nMinimizing 137 Time: 0:00:00 ( 0.73 ms/it)\n objective: 13780.812348816995\nMinimizing 273 Time: 0:00:00 ( 0.74 ms/it)\n objective: 13773.116746328047\nMinimizing 409 Time: 0:00:00 ( 0.74 ms/it)\n objective: 13772.734800127791\nMinimizing 544 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13772.674846291791\nMinimizing 675 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13772.671079144462\nMinimizing 810 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13772.670785512491\nMinimizing 830 Time: 0:00:00 ( 0.75 ms/it)\n\nMinimizing 831 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13772.67078220487\nMinimizing 137 Time: 0:00:00 ( 0.73 ms/it)\n objective: 13776.235548768575\nMinimizing 273 Time: 0:00:00 ( 0.74 ms/it)\n objective: 13772.972354322641\nMinimizing 408 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13772.724964654355\nMinimizing 542 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13772.675673083704\nMinimizing 677 Time: 0:00:00 ( 0.75 ms/it)\n objective: 13772.671035917809\nMinimizing 772 Time: 0:00:00 ( 0.75 ms/it)\n\n\n\nVarCorr(m_School_SeqDiff)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\nSchool\n(Intercept)\n0.027986\n0.167291\n\n\n\n\n\n\n\n\nTest: Star_r\n0.193948\n0.440395\n+0.01\n\n\n\n\n\n\n\nTest: S20_r\n0.130505\n0.361255\n+0.12\n-0.77\n\n\n\n\n\n\nTest: SLJ\n0.185494\n0.430690\n-0.15\n+0.51\n-0.67\n\n\n\n\n\nTest: BPT\n0.086521\n0.294144\n-0.35\n-0.63\n+0.45\n-0.66\n\n\n\n\na1\n0.152653\n0.390708\n+0.23\n+0.04\n+0.34\n-0.24\n+0.29\n\n\nResidual\n\n0.837562\n0.915184\n\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_School_Helmert)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\nSchool\n(Intercept)\n0.0000000\n0.0000000\n\n\n\n\n\n\n\n\nTest: Star_r\n0.0509601\n0.2257435\n+NaN\n\n\n\n\n\n\n\nTest: S20_r\n0.0061410\n0.0783648\n+NaN\n-0.17\n\n\n\n\n\n\nTest: SLJ\n0.0088294\n0.0939650\n+NaN\n+0.47\n-0.29\n\n\n\n\n\nTest: BPT\n0.0010655\n0.0326421\n+NaN\n-0.35\n-0.13\n+0.47\n\n\n\n\na1\n0.1782216\n0.4221630\n+NaN\n+0.08\n+0.62\n-0.06\n+0.35\n\n\nResidual\n\n0.8659680\n0.9305740\n\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_School_Hypo)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\nSchool\n(Intercept)\n0.027988\n0.167297\n\n\n\n\n\n\n\n\nTest: BPT-other\n0.721457\n0.849386\n-0.59\n\n\n\n\n\n\n\nTest: Star-End\n0.193971\n0.440421\n+0.01\n-0.23\n\n\n\n\n\n\nTest: S20-Star\n0.130471\n0.361208\n+0.12\n+0.06\n-0.77\n\n\n\n\n\nTest: SLJ-S20\n0.185461\n0.430652\n-0.15\n+0.30\n+0.51\n-0.67\n\n\n\n\na1\n0.152663\n0.390721\n+0.23\n+0.34\n+0.04\n+0.34\n-0.24\n\n\nResidual\n\n0.837564\n0.915185\n\n\n\n\n\n\n\n\n\n\n\nVarCorr(m_School_PCA)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\nSchool\n(Intercept)\n0.027989\n0.167300\n\n\n\n\n\n\n\n\nTest: c5.1\n0.101829\n0.319106\n-0.36\n\n\n\n\n\n\n\nTest: c234.15\n2.405034\n1.550817\n+0.28\n+0.42\n\n\n\n\n\n\nTest: c2.34\n0.288511\n0.537132\n-0.04\n+0.02\n+0.37\n\n\n\n\n\nTest: c3.4\n0.185385\n0.430564\n+0.15\n-0.68\n-0.37\n-0.10\n\n\n\n\na1\n0.152625\n0.390673\n+0.23\n+0.38\n+0.02\n-0.26\n+0.24\n\n\nResidual\n\n0.837575\n0.915191\n\n\n\n\n\n\n\n\n\n\nWe compare again how much of the fit resides in the CPs.\n\nbegin\n f_School0 = @formula zScore ~\n 1 + Test * a1 * Sex + zerocorr(1 + Test + a1 | School)\n m_School_SDC0 = fit(MixedModel, f_School0, dat; contrasts=contr1)\n m_School_HeC0 = fit(MixedModel, f_School0, dat; contrasts=contr2)\n m_School_HyC0 = fit(MixedModel, f_School0, dat; contrasts=contr3)\n m_School_PCA0 = fit(MixedModel, f_School0, dat; contrasts=contr4)\n #\n zcpLMM2 = [\"SDC0\", \"HeC0\", \"HyC0\", \"PCA0\"]\n mods2 = [\n m_School_SDC0, m_School_HeC0, m_School_HyC0, m_School_PCA0\n ]\n gof_summary2 = sort!(\n DataFrame(;\n zcpLMM=zcpLMM2,\n dof=dof.(mods2),\n deviance=deviance.(mods2),\n AIC=aic.(mods2),\n BIC=bic.(mods2),\n ),\n :deviance,\n )\nend\n\n4×5 DataFrame\n\n\n\nRow\nzcpLMM\ndof\ndeviance\nAIC\nBIC\n\n\n\nString\nInt64\nFloat64\nFloat64\nFloat64\n\n\n\n\n1\nHeC0\n27\n13789.5\n13843.5\n14019.5\n\n\n2\nPCA0\n27\n13793.7\n13847.7\n14023.6\n\n\n3\nHyC0\n27\n13797.1\n13851.1\n14027.0\n\n\n4\nSDC0\n27\n13801.2\n13855.2\n14031.2\n\n\n\n\n\n\nFor the random factor School the Helmert contrast, followed by PCA-based contrasts have least information in the CPs; SDC has the largest contribution from CPs. Interesting.", + "crumbs": [ + "Contrast coding", + "Mixed Models Tutorial: Contrast Coding" + ] + }, + { + "objectID": "kkl15.html", + "href": "kkl15.html", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "", + "text": "Kliegl et al. (2015) is a follow-up to Kliegl et al. (2011) (see also script kwdyz11.qmd) from an experiment looking at a variety of effects of visual cueing under four different cue-target relations (CTRs). In this experiment two rectangles are displayed (1) in horizontal orientation , (2) in vertical orientation, (3) in left diagonal orientation, or in (4) right diagonal orientation relative to a central fixation point. Subjects react to the onset of a small or a large visual target occurring at one of the four ends of the two rectangles. The target is cued validly on 70% of trials by a brief flash of the corner of the rectangle at which it appears; it is cued invalidly at the three other locations 10% of the trials each. This implies a latent imbalance in design that is not visible in the repeated-measures ANOVA, but we will show its effect in the random-effect structure and conditional modes.\nThere are a couple of differences between the first and this follow-up experiment, rendering it more a conceptual than a direct replication. First, the original experiment was carried out at Peking University and this follow-up at Potsdam University. Second, diagonal orientations of rectangles and large target sizes were not part of the design of Kliegl et al. (2011).\nWe specify three contrasts for the four-level factor CTR that are derived from spatial, object-based, and attractor-like features of attention. They map onto sequential differences between appropriately ordered factor levels. Replicating Kliegl et al. (2011), the attraction effect was not significant as a fixed effect, but yielded a highly reliable variance component (VC; i.e., reliable individual differences in positive and negative attraction effects cancel the fixed effect). Moreover, these individual differences in the attraction effect were negatively correlated with those in the spatial effect.\nThis comparison is of interest because a few years after the publication of Kliegl et al. (2011), the theoretically critical correlation parameter (CP) between the spatial effect and the attraction effect was determined as the source of a non-singular LMM in that paper. The present study served the purpose to estimate this parameter with a larger sample and a wider variety of experimental conditions.\nHere we also include two additional experimental manipulations of target size and orientation of cue rectangle. A similar analysis was reported in the parsimonious mixed-model paper (Bates et al., 2015); it was also used in a paper of GAMMs (Baayen et al., 2017). Data and R scripts of those analyses are also available in R-package RePsychLing.\nThe analysis is based on log-transformed reaction times lrt, indicated by a boxcox() check of model residuals.\nIn this vignette we focus on the reduction of model complexity. And we start with a quote:\n“Neither the [maximal] nor the [minimal] linear mixed models are appropriate for most repeated measures analysis. Using the [maximal] model is generally wasteful and costly in terms of statiscal power for testing hypotheses. On the other hand, the [minimal] model fails to account for nontrivial correlation among repeated measurements. This results in inflated [T]ype I error rates when non-negligible correlation does in fact exist. We can usually find middle ground, a covariance model that adequately accounts for correlation but is more parsimonious than the [maximal] model. Doing so allows us full control over [T]ype I error rates without needlessly sacrificing power.”\nStroup, W. W. (2012, p. 185). Generalized linear mixed models: Modern concepts, methods and applica?ons. CRC Press, Boca Raton.", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#contrasts", + "href": "kkl15.html#contrasts", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "4.1 Contrasts", + "text": "4.1 Contrasts\n\ncontrasts = Dict(\n :Subj => Grouping(),\n :CTR => SeqDiffCoding(; levels=[\"val\", \"sod\", \"dos\", \"dod\"]),\n :cardinal => EffectsCoding(; levels=[\"cardinal\", \"diagonal\"]),\n :size => EffectsCoding(; levels=[\"big\", \"small\"])\n)\n\nDict{Symbol, StatsModels.AbstractContrasts} with 4 entries:\n :CTR => SeqDiffCoding([\"val\", \"sod\", \"dos\", \"dod\"])\n :size => EffectsCoding(nothing, [\"big\", \"small\"])\n :Subj => Grouping()\n :cardinal => EffectsCoding(nothing, [\"cardinal\", \"diagonal\"])\n\n\n\nm_max_rt = let\n form = @formula rt ~ 1 + CTR * size * cardinal +\n (1 + CTR * size * cardinal | Subj)\n fit(MixedModel, form, dat; contrasts)\nend\n\nMinimizing 2 Time: 0:00:00 (53.17 ms/it)\n objective: 601241.5133290171\nMinimizing 1002 Time: 0:00:01 ( 1.79 ms/it)\n objective: 599333.7272274668\nMinimizing 1408 Time: 0:00:02 ( 1.66 ms/it)\n objective: 599308.5351230928\nMinimizing 1489 Time: 0:00:02 ( 1.64 ms/it)\n objective: 599306.5889190858\nMinimizing 1570 Time: 0:00:02 ( 1.62 ms/it)\n objective: 599304.9501835523\nMinimizing 1649 Time: 0:00:02 ( 1.61 ms/it)\n objective: 599303.8187455853\nMinimizing 1727 Time: 0:00:02 ( 1.61 ms/it)\n objective: 599301.3769616229\nMinimizing 1799 Time: 0:00:02 ( 1.60 ms/it)\n objective: 599300.3700002148\nMinimizing 1872 Time: 0:00:02 ( 1.60 ms/it)\n objective: 599299.4958301473\nMinimizing 1943 Time: 0:00:03 ( 1.59 ms/it)\n objective: 599299.0448406646\nMinimizing 2010 Time: 0:00:03 ( 1.59 ms/it)\n objective: 599298.6887895833\nMinimizing 2087 Time: 0:00:03 ( 1.58 ms/it)\n objective: 599298.5089541689\nMinimizing 2162 Time: 0:00:03 ( 1.58 ms/it)\n objective: 599298.3506449385\nMinimizing 2233 Time: 0:00:03 ( 1.57 ms/it)\n objective: 599298.2531667919\nMinimizing 2307 Time: 0:00:03 ( 1.57 ms/it)\n objective: 599298.1823067392\nMinimizing 2379 Time: 0:00:03 ( 1.56 ms/it)\n objective: 599298.1003545397\nMinimizing 2449 Time: 0:00:03 ( 1.56 ms/it)\n objective: 599298.0592483006\nMinimizing 2519 Time: 0:00:03 ( 1.57 ms/it)\n objective: 599298.016369703\nMinimizing 2585 Time: 0:00:04 ( 1.57 ms/it)\n objective: 599297.9868262913\nMinimizing 2651 Time: 0:00:04 ( 1.57 ms/it)\n objective: 599297.9623177209\nMinimizing 2717 Time: 0:00:04 ( 1.57 ms/it)\n objective: 599297.9362295533\nMinimizing 2783 Time: 0:00:04 ( 1.57 ms/it)\n objective: 599297.9158610143\nMinimizing 2847 Time: 0:00:04 ( 1.57 ms/it)\n objective: 599297.8907719204\nMinimizing 2913 Time: 0:00:04 ( 1.57 ms/it)\n objective: 599297.8655307669\nMinimizing 2977 Time: 0:00:04 ( 1.57 ms/it)\n objective: 599297.838000219\nMinimizing 3044 Time: 0:00:04 ( 1.57 ms/it)\n objective: 599297.8130767004\nMinimizing 3109 Time: 0:00:04 ( 1.57 ms/it)\n objective: 599297.7940609268\nMinimizing 3177 Time: 0:00:04 ( 1.57 ms/it)\n objective: 599297.7789314522\nMinimizing 3243 Time: 0:00:05 ( 1.57 ms/it)\n objective: 599297.7661259501\nMinimizing 3309 Time: 0:00:05 ( 1.57 ms/it)\n objective: 599297.7549728255\nMinimizing 3375 Time: 0:00:05 ( 1.57 ms/it)\n objective: 599297.7460114947\nMinimizing 3440 Time: 0:00:05 ( 1.57 ms/it)\n objective: 599297.73789173\nMinimizing 3507 Time: 0:00:05 ( 1.57 ms/it)\n objective: 599297.7294974972\nMinimizing 3574 Time: 0:00:05 ( 1.57 ms/it)\n objective: 599297.7197337943\nMinimizing 3639 Time: 0:00:05 ( 1.57 ms/it)\n objective: 599297.71324412\nMinimizing 3704 Time: 0:00:05 ( 1.57 ms/it)\n objective: 599297.7098021851\nMinimizing 3770 Time: 0:00:05 ( 1.57 ms/it)\n objective: 599297.7007812862\nMinimizing 3838 Time: 0:00:06 ( 1.57 ms/it)\n objective: 599297.6951566301\nMinimizing 3904 Time: 0:00:06 ( 1.57 ms/it)\n objective: 599297.6904034589\nMinimizing 3971 Time: 0:00:06 ( 1.57 ms/it)\n objective: 599297.682983504\nMinimizing 4038 Time: 0:00:06 ( 1.57 ms/it)\n objective: 599297.6760234906\nMinimizing 4112 Time: 0:00:06 ( 1.57 ms/it)\n objective: 599297.6708892871\nMinimizing 4180 Time: 0:00:06 ( 1.57 ms/it)\n objective: 599297.6661708716\nMinimizing 4246 Time: 0:00:06 ( 1.57 ms/it)\n objective: 599297.6605394017\nMinimizing 4311 Time: 0:00:06 ( 1.57 ms/it)\n objective: 599297.6580177272\nMinimizing 4385 Time: 0:00:06 ( 1.57 ms/it)\n objective: 599297.6559655914\nMinimizing 4457 Time: 0:00:06 ( 1.56 ms/it)\n objective: 599297.6544785632\nMinimizing 4528 Time: 0:00:07 ( 1.56 ms/it)\n objective: 599297.654285845\nMinimizing 4595 Time: 0:00:07 ( 1.56 ms/it)\n objective: 599297.6534127444\nMinimizing 4667 Time: 0:00:07 ( 1.57 ms/it)\n\nMinimizing 4668 Time: 0:00:07 ( 1.57 ms/it)\n objective: 599297.6521322867\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n308.4078\n5.8382\n52.83\n<1e-99\n0.0000\n\n\nCTR: sod\n23.2685\n2.5669\n9.06\n<1e-18\n5.9416\n\n\nCTR: dos\n13.0801\n1.5434\n8.47\n<1e-16\n4.0401\n\n\nCTR: dod\n2.7724\n2.0918\n1.33\n0.1850\n13.6316\n\n\nsize: small\n26.4130\n5.8382\n4.52\n<1e-05\n54.0253\n\n\ncardinal: diagonal\n6.6749\n1.7882\n3.73\n0.0002\n11.4760\n\n\nCTR: sod & size: small\n8.7925\n2.5669\n3.43\n0.0006\n21.4147\n\n\nCTR: dos & size: small\n-0.7065\n1.5434\n-0.46\n0.6472\n7.8354\n\n\nCTR: dod & size: small\n7.4983\n2.0918\n3.58\n0.0003\n7.9016\n\n\nCTR: sod & cardinal: diagonal\n3.6318\n1.1024\n3.29\n0.0010\n3.5394\n\n\nCTR: dos & cardinal: diagonal\n1.3340\n1.2448\n1.07\n0.2839\n1.8675\n\n\nCTR: dod & cardinal: diagonal\n-0.2245\n1.3222\n-0.17\n0.8652\n4.1664\n\n\nsize: small & cardinal: diagonal\n2.0502\n1.7882\n1.15\n0.2516\n11.4335\n\n\nCTR: sod & size: small & cardinal: diagonal\n-0.7638\n1.1024\n-0.69\n0.4884\n4.3866\n\n\nCTR: dos & size: small & cardinal: diagonal\n-0.1555\n1.2448\n-0.12\n0.9006\n1.6228\n\n\nCTR: dod & size: small & cardinal: diagonal\n4.1767\n1.3222\n3.16\n0.0016\n2.2306\n\n\nResidual\n63.0091\n\n\n\n\n\n\n\n\n\n\nm_cpx_rt = let\n form = @formula rt ~ 1 + CTR * size * cardinal +\n (1 + CTR + size + cardinal | Subj)\n fit(MixedModel, form, dat; contrasts)\nend\n\nMinimizing 302 Time: 0:00:00 ( 0.33 ms/it)\n objective: 599405.9108670889\nMinimizing 915 Time: 0:00:00 ( 0.22 ms/it)\n objective: 599403.6000686976\nMinimizing 924 Time: 0:00:00 ( 0.22 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n308.4075\n6.0636\n50.86\n<1e-99\n40.4009\n\n\nCTR: sod\n23.2723\n2.4767\n9.40\n<1e-20\n21.3210\n\n\nCTR: dos\n13.1098\n1.4624\n8.96\n<1e-18\n7.5108\n\n\nCTR: dod\n2.7142\n1.9178\n1.42\n0.1570\n13.7084\n\n\nsize: small\n26.3865\n6.0636\n4.35\n<1e-04\n38.9502\n\n\ncardinal: diagonal\n6.6505\n1.7319\n3.84\n0.0001\n15.6643\n\n\nCTR: sod & size: small\n8.7956\n2.4767\n3.55\n0.0004\n\n\n\nCTR: dos & size: small\n-0.7230\n1.4624\n-0.49\n0.6210\n\n\n\nCTR: dod & size: small\n7.4188\n1.9178\n3.87\n0.0001\n\n\n\nCTR: sod & cardinal: diagonal\n3.6413\n0.9210\n3.95\n<1e-04\n\n\n\nCTR: dos & cardinal: diagonal\n1.3278\n1.2176\n1.09\n0.2755\n\n\n\nCTR: dod & cardinal: diagonal\n-0.3145\n1.2217\n-0.26\n0.7968\n\n\n\nsize: small & cardinal: diagonal\n2.0471\n1.7319\n1.18\n0.2372\n\n\n\nCTR: sod & size: small & cardinal: diagonal\n-0.7610\n0.9210\n-0.83\n0.4086\n\n\n\nCTR: dos & size: small & cardinal: diagonal\n-0.1335\n1.2176\n-0.11\n0.9127\n\n\n\nCTR: dod & size: small & cardinal: diagonal\n4.1117\n1.2217\n3.37\n0.0008\n\n\n\nResidual\n63.1010", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#box-cox", + "href": "kkl15.html#box-cox", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "4.2 Box-Cox", + "text": "4.2 Box-Cox\n\nbc1 = fit(BoxCoxTransformation, m_max_rt)\n\n\nbc2 = fit(BoxCoxTransformation, m_cpx_rt)\n\nMinimizing 669 Time: 0:00:00 ( 0.16 ms/it)\n objective: -24614.57221183604\nMinimizing 1134 Time: 0:00:00 ( 0.16 ms/it)\nMinimizing 666 Time: 0:00:00 ( 0.16 ms/it)\n objective: -7521.599763635946\nMinimizing 894 Time: 0:00:00 ( 0.16 ms/it)\nMinimizing 668 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27394.902843102427\nMinimizing 1173 Time: 0:00:00 ( 0.16 ms/it)\n\nMinimizing 1174 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27395.25159135472\nMinimizing 670 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27948.912130319124\nMinimizing 1293 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27948.924023849817\nMinimizing 1293 Time: 0:00:00 ( 0.16 ms/it)\nMinimizing 670 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27948.757616138104\nMinimizing 1292 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27949.950395857046\nMinimizing 1496 Time: 0:00:00 ( 0.16 ms/it)\n\nMinimizing 1497 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27949.950563489412\nMinimizing 670 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27949.955411736297\nMinimizing 1295 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27949.95958768511\nMinimizing 1899 Time: 0:00:00 ( 0.17 ms/it)\n objective: -27949.961513601214\nMinimizing 2357 Time: 0:00:00 ( 0.17 ms/it)\nMinimizing 661 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27949.66778994192\nMinimizing 1100 Time: 0:00:00 ( 0.16 ms/it)\nMinimizing 661 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27948.215871258206\nMinimizing 1278 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27948.380519227445\nMinimizing 1499 Time: 0:00:00 ( 0.16 ms/it)\nMinimizing 667 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27948.211640368994\nMinimizing 1285 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27949.93485052913\nMinimizing 1312 Time: 0:00:00 ( 0.16 ms/it)\nMinimizing 661 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27949.922438171878\nMinimizing 1216 Time: 0:00:00 ( 0.16 ms/it)\nMinimizing 663 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27947.646299135915\nMinimizing 1189 Time: 0:00:00 ( 0.16 ms/it)\nMinimizing 668 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27948.357951161892\nMinimizing 948 Time: 0:00:00 ( 0.16 ms/it)\nMinimizing 669 Time: 0:00:00 ( 0.16 ms/it)\n objective: -27948.135260234907\nMinimizing 1288 Time: 0:00:00 ( 0.17 ms/it)\n objective: -27948.35069788193\nMinimizing 1394 Time: 0:00:00 ( 0.17 ms/it)\nMinimizing 571 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27948.556266149943\nMinimizing 1107 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27949.943733548123\nMinimizing 1577 Time: 0:00:00 ( 0.19 ms/it)\nMinimizing 571 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27947.752620644653\nMinimizing 1105 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27948.35024549874\nMinimizing 1211 Time: 0:00:00 ( 0.19 ms/it)\nMinimizing 571 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27946.898268664925\nMinimizing 1106 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27948.352563028613\nMinimizing 1442 Time: 0:00:00 ( 0.19 ms/it)\nMinimizing 571 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27945.4392929145\nMinimizing 1107 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27949.894231694314\nMinimizing 1632 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27949.952141882208\nMinimizing 1690 Time: 0:00:00 ( 0.19 ms/it)\nMinimizing 571 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27948.002337963364\nMinimizing 1106 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27949.916584711973\nMinimizing 1764 Time: 0:00:00 ( 0.22 ms/it)\nMinimizing 572 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27949.87281263115\nMinimizing 920 Time: 0:00:00 ( 0.19 ms/it)\nMinimizing 571 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27946.451386707562\nMinimizing 1107 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27948.382683844105\nMinimizing 1632 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27948.40667584678\nMinimizing 2156 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27949.946197014546\nMinimizing 2301 Time: 0:00:00 ( 0.19 ms/it)\nMinimizing 572 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27947.144567634066\nMinimizing 1108 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27949.94662606359\nMinimizing 1629 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27949.954183490412\nMinimizing 1683 Time: 0:00:00 ( 0.19 ms/it)\nMinimizing 573 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27946.396120890593\nMinimizing 1109 Time: 0:00:00 ( 0.19 ms/it)\n objective: -27949.86351095891\nMinimizing 1621 Time: 0:00:00 ( 0.19 ms/it)\n\n\nBox-Cox transformation\n\nestimated λ: -0.7011\nresultant transformation:\n\n y^-0.7 - 1\n------------\n -0.7\n\n\n\nboxcoxplot(bc2; conf_level=0.95)\n\nClear evidence for skew. Traditionally, we used log transforms for reaction times. even stronger than log. We stay with log for now. Could try 1/sqrt(rt).", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#zero-correlation-parameter-lmm-1", + "href": "kkl15.html#zero-correlation-parameter-lmm-1", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "6.1 Zero-correlation parameter LMM (1)", + "text": "6.1 Zero-correlation parameter LMM (1)\nForce CPs to zero.\n\nm_zcp1 = let\n form = @formula log(rt) ~ 1 + CTR * size * cardinal +\n zerocorr(1 + CTR * size * cardinal | Subj)\n fit(MixedModel, form, dat; contrasts)\nend\n\nMinimizing 359 Time: 0:00:00 ( 0.28 ms/it)\n objective: -24555.63858432091\nMinimizing 622 Time: 0:00:00 ( 0.27 ms/it)\n\nMinimizing 623 Time: 0:00:00 ( 0.27 ms/it)\n objective: -24588.607470697283\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n5.6910\n0.0173\n329.51\n<1e-99\n0.0024\n\n\nCTR: sod\n0.0744\n0.0077\n9.66\n<1e-21\n0.0369\n\n\nCTR: dos\n0.0409\n0.0045\n9.09\n<1e-19\n0.0003\n\n\nCTR: dod\n0.0015\n0.0053\n0.29\n0.7749\n0.0012\n\n\nsize: small\n0.0921\n0.0173\n5.33\n<1e-07\n0.1598\n\n\ncardinal: diagonal\n0.0204\n0.0052\n3.92\n<1e-04\n0.0159\n\n\nCTR: sod & size: small\n0.0244\n0.0077\n3.17\n0.0015\n0.0555\n\n\nCTR: dos & size: small\n-0.0053\n0.0045\n-1.19\n0.2344\n0.0240\n\n\nCTR: dod & size: small\n0.0181\n0.0053\n3.39\n0.0007\n0.0357\n\n\nCTR: sod & cardinal: diagonal\n0.0101\n0.0033\n3.03\n0.0025\n0.0151\n\n\nCTR: dos & cardinal: diagonal\n0.0046\n0.0037\n1.26\n0.2092\n0.0000\n\n\nCTR: dod & cardinal: diagonal\n-0.0055\n0.0037\n-1.49\n0.1358\n0.0016\n\n\nsize: small & cardinal: diagonal\n0.0041\n0.0052\n0.79\n0.4291\n0.0444\n\n\nCTR: sod & size: small & cardinal: diagonal\n-0.0032\n0.0033\n-0.95\n0.3427\n0.0082\n\n\nCTR: dos & size: small & cardinal: diagonal\n-0.0005\n0.0037\n-0.13\n0.8945\n0.0000\n\n\nCTR: dod & size: small & cardinal: diagonal\n0.0109\n0.0037\n2.94\n0.0033\n0.0031\n\n\nResidual\n0.1903\n\n\n\n\n\n\n\n\n\n\nissingular(m_zcp1)\n\ntrue\n\n\n\nonly(MixedModels.PCA(m_zcp1))\n\n\nPrincipal components based on correlation matrix\n (Intercept) … . . . . . . .\n CTR: sod . . . . . . .\n CTR: dos . . . . . . .\n CTR: dod . . . . . . .\n size: small . . . . . . .\n CTR: sod & size: small … . . . . . . .\n CTR: dos & size: small . . . . . . .\n CTR: dod & size: small . . . . . . .\n cardinal: diagonal . . . . . . .\n CTR: sod & cardinal: diagonal 1.0 . . . . . .\n CTR: dos & cardinal: diagonal … 0.0 0.0 . . . . .\n CTR: dod & cardinal: diagonal 0.0 0.0 1.0 . . . .\n size: small & cardinal: diagonal 0.0 0.0 0.0 1.0 . . .\n CTR: sod & size: small & cardinal: diagonal 0.0 0.0 0.0 0.0 1.0 . .\n CTR: dos & size: small & cardinal: diagonal 0.0 0.0 0.0 0.0 0.0 0.0 .\n CTR: dod & size: small & cardinal: diagonal … 0.0 0.0 0.0 0.0 0.0 0.0 1.0\n\nNormalized cumulative variances:\n[0.0714, 0.1429, 0.2143, 0.2857, 0.3571, 0.4286, 0.5, 0.5714, 0.6429, 0.7143, 0.7857, 0.8571, 0.9286, 1.0, 1.0, 1.0]\n\nComponent loadings\n … PC13 PC14 PC15 PC16\n (Intercept) 0.0 0.0 0.0 0.0\n CTR: sod 0.0 0.0 0.0 0.0\n CTR: dos 0.0 0.0 0.0 0.0\n CTR: dod 1.0 0.0 0.0 0.0\n size: small … 0.0 0.0 0.0 0.0\n CTR: sod & size: small 0.0 1.0 0.0 0.0\n CTR: dos & size: small 0.0 0.0 0.0 0.0\n CTR: dod & size: small 0.0 0.0 0.0 0.0\n cardinal: diagonal 0.0 0.0 0.0 0.0\n CTR: sod & cardinal: diagonal … 0.0 0.0 0.0 0.0\n CTR: dos & cardinal: diagonal 0.0 0.0 NaN 0.0\n CTR: dod & cardinal: diagonal 0.0 0.0 0.0 0.0\n size: small & cardinal: diagonal 0.0 0.0 0.0 0.0\n CTR: sod & size: small & cardinal: diagonal 0.0 0.0 0.0 0.0\n CTR: dos & size: small & cardinal: diagonal … 0.0 0.0 0.0 NaN\n CTR: dod & size: small & cardinal: diagonal 0.0 0.0 0.0 0.0\n\n\n\nVarCorr(m_zcp1)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nSubj\n(Intercept)\n0.000005718\n0.002391288\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nCTR: sod\n0.001359814\n0.036875659\n.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nCTR: dos\n0.000000064\n0.000252674\n.\n.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nCTR: dod\n0.000001426\n0.001193943\n.\n.\n.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nsize: small\n0.025533020\n0.159790549\n.\n.\n.\n.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nCTR: sod & size: small\n0.003075211\n0.055454589\n.\n.\n.\n.\n.\n\n\n\n\n\n\n\n\n\n\n\n\n\nCTR: dos & size: small\n0.000578116\n0.024044053\n.\n.\n.\n.\n.\n.\n\n\n\n\n\n\n\n\n\n\n\n\nCTR: dod & size: small\n0.001275837\n0.035718859\n.\n.\n.\n.\n.\n.\n.\n\n\n\n\n\n\n\n\n\n\n\ncardinal: diagonal\n0.000252244\n0.015882183\n.\n.\n.\n.\n.\n.\n.\n.\n\n\n\n\n\n\n\n\n\n\nCTR: sod & cardinal: diagonal\n0.000227732\n0.015090802\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\n\n\n\n\n\n\n\n\nCTR: dos & cardinal: diagonal\n0.000000000\n0.000000000\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\n\n\n\n\n\n\n\nCTR: dod & cardinal: diagonal\n0.000002586\n0.001607978\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\n\n\n\n\n\n\nsize: small & cardinal: diagonal\n0.001973966\n0.044429339\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\n\n\n\n\n\nCTR: sod & size: small & cardinal: diagonal\n0.000067412\n0.008210458\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\n\n\n\n\nCTR: dos & size: small & cardinal: diagonal\n0.000000000\n0.000000000\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\n\n\n\nCTR: dod & size: small & cardinal: diagonal\n0.000009423\n0.003069670\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\n\nResidual\n\n0.036196638\n0.190254141", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#reduced-zcp-lmm", + "href": "kkl15.html#reduced-zcp-lmm", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "6.2 Reduced zcp LMM", + "text": "6.2 Reduced zcp LMM\nTake out VC for interactions.\n\nm_zcp1_rdc = let\n form = @formula log(rt) ~ 1 + CTR * size * cardinal +\n zerocorr(1 + CTR + size + cardinal | Subj)\n fit(MixedModel, form, dat; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n5.6911\n0.0173\n329.28\n<1e-99\n0.1572\n\n\nCTR: sod\n0.0745\n0.0077\n9.66\n<1e-21\n0.0666\n\n\nCTR: dos\n0.0409\n0.0045\n9.09\n<1e-19\n0.0240\n\n\nCTR: dod\n0.0015\n0.0053\n0.29\n0.7751\n0.0357\n\n\nsize: small\n0.0920\n0.0173\n5.33\n<1e-06\n0.0295\n\n\ncardinal: diagonal\n0.0204\n0.0053\n3.88\n0.0001\n0.0476\n\n\nCTR: sod & size: small\n0.0244\n0.0077\n3.17\n0.0015\n\n\n\nCTR: dos & size: small\n-0.0054\n0.0045\n-1.20\n0.2317\n\n\n\nCTR: dod & size: small\n0.0181\n0.0053\n3.39\n0.0007\n\n\n\nCTR: sod & cardinal: diagonal\n0.0101\n0.0028\n3.64\n0.0003\n\n\n\nCTR: dos & cardinal: diagonal\n0.0046\n0.0037\n1.25\n0.2125\n\n\n\nCTR: dod & cardinal: diagonal\n-0.0056\n0.0037\n-1.51\n0.1317\n\n\n\nsize: small & cardinal: diagonal\n0.0042\n0.0053\n0.79\n0.4286\n\n\n\nCTR: sod & size: small & cardinal: diagonal\n-0.0031\n0.0028\n-1.12\n0.2610\n\n\n\nCTR: dos & size: small & cardinal: diagonal\n-0.0005\n0.0037\n-0.12\n0.9023\n\n\n\nCTR: dod & size: small & cardinal: diagonal\n0.0109\n0.0037\n2.95\n0.0032\n\n\n\nResidual\n0.1904\n\n\n\n\n\n\n\n\n\n\nissingular(m_zcp1_rdc)\n\nfalse\n\n\n\nonly(MixedModels.PCA(m_zcp1_rdc))\n\n\nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . . .\n CTR: sod 0.0 1.0 . . . .\n CTR: dos 0.0 0.0 1.0 . . .\n CTR: dod 0.0 0.0 0.0 1.0 . .\n size: small 0.0 0.0 0.0 0.0 1.0 .\n cardinal: diagonal 0.0 0.0 0.0 0.0 0.0 1.0\n\nNormalized cumulative variances:\n[0.1667, 0.3333, 0.5, 0.6667, 0.8333, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5 PC6\n (Intercept) 1.0 0.0 0.0 0.0 0.0 0.0\n CTR: sod 0.0 1.0 0.0 0.0 0.0 0.0\n CTR: dos 0.0 0.0 1.0 0.0 0.0 0.0\n CTR: dod 0.0 0.0 0.0 1.0 0.0 0.0\n size: small 0.0 0.0 0.0 0.0 1.0 0.0\n cardinal: diagonal 0.0 0.0 0.0 0.0 0.0 1.0\n\n\n\nVarCorr(m_zcp1_rdc)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\nSubj\n(Intercept)\n0.0247021\n0.1571690\n\n\n\n\n\n\n\n\nCTR: sod\n0.0044409\n0.0666397\n.\n\n\n\n\n\n\n\nCTR: dos\n0.0005763\n0.0240059\n.\n.\n\n\n\n\n\n\nCTR: dod\n0.0012752\n0.0357104\n.\n.\n.\n\n\n\n\n\nsize: small\n0.0008722\n0.0295334\n.\n.\n.\n.\n\n\n\n\ncardinal: diagonal\n0.0022678\n0.0476216\n.\n.\n.\n.\n.\n\n\nResidual\n\n0.0362601\n0.1904207", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#model-comparison-1", + "href": "kkl15.html#model-comparison-1", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "6.3 Model comparison 1", + "text": "6.3 Model comparison 1\nLet’s compare the three models.\n\ngof_summary = let\n nms = [:m_zcp1_rdc, :m_zcp1, :m_max]\n mods = eval.(nms)\n lrt = MixedModels.likelihoodratiotest(m_zcp1_rdc, m_zcp1, m_max)\n DataFrame(;\n name = nms,\n dof=dof.(mods),\n deviance=round.(deviance.(mods), digits=0),\n AIC=round.(aic.(mods),digits=0),\n AICc=round.(aicc.(mods),digits=0),\n BIC=round.(bic.(mods),digits=0),\n χ²=vcat(:., round.(lrt.tests.deviancediff, digits=0)),\n χ²_dof=vcat(:., round.(lrt.tests.dofdiff, digits=0)),\n pvalue=vcat(:., round.(lrt.tests.pvalues, digits=3))\n )\nend\n\n3×9 DataFrame\n\n\n\nRow\nname\ndof\ndeviance\nAIC\nAICc\nBIC\nχ²\nχ²_dof\npvalue\n\n\n\nSymbol\nInt64\nFloat64\nFloat64\nFloat64\nFloat64\nAny\nAny\nAny\n\n\n\n\n1\nm_zcp1_rdc\n23\n-24558.0\n-24512.0\n-24512.0\n-24308.0\n.\n.\n.\n\n\n2\nm_zcp1\n33\n-24589.0\n-24523.0\n-24523.0\n-24229.0\n30.0\n10.0\n0.001\n\n\n3\nm_max\n153\n-24691.0\n-24385.0\n-24384.0\n-23025.0\n103.0\n120.0\n0.872", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#parsimonious-lmm-1", + "href": "kkl15.html#parsimonious-lmm-1", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "6.4 Parsimonious LMM (1)", + "text": "6.4 Parsimonious LMM (1)\nExtend zcp-reduced LMM with CPs\n\nm_prm1 = let\n form = @formula log(rt) ~ 1 + CTR * size * cardinal +\n (1 + CTR + size + cardinal | Subj)\n fit(MixedModel, form, dat; contrasts)\nend\n\nMinimizing 756 Time: 0:00:00 ( 0.14 ms/it)\n objective: -24615.345521636216\nMinimizing 882 Time: 0:00:00 ( 0.15 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n5.6911\n0.0176\n323.58\n<1e-99\n0.1117\n\n\nCTR: sod\n0.0745\n0.0076\n9.75\n<1e-21\n0.0660\n\n\nCTR: dos\n0.0408\n0.0043\n9.42\n<1e-20\n0.0213\n\n\nCTR: dod\n0.0017\n0.0051\n0.34\n0.7374\n0.0326\n\n\nsize: small\n0.0921\n0.0176\n5.23\n<1e-06\n0.1184\n\n\ncardinal: diagonal\n0.0204\n0.0053\n3.88\n0.0001\n0.0476\n\n\nCTR: sod & size: small\n0.0244\n0.0076\n3.19\n0.0014\n\n\n\nCTR: dos & size: small\n-0.0054\n0.0043\n-1.24\n0.2156\n\n\n\nCTR: dod & size: small\n0.0181\n0.0051\n3.55\n0.0004\n\n\n\nCTR: sod & cardinal: diagonal\n0.0101\n0.0028\n3.64\n0.0003\n\n\n\nCTR: dos & cardinal: diagonal\n0.0046\n0.0037\n1.25\n0.2118\n\n\n\nCTR: dod & cardinal: diagonal\n-0.0056\n0.0037\n-1.51\n0.1301\n\n\n\nsize: small & cardinal: diagonal\n0.0042\n0.0053\n0.79\n0.4272\n\n\n\nCTR: sod & size: small & cardinal: diagonal\n-0.0031\n0.0028\n-1.13\n0.2597\n\n\n\nCTR: dos & size: small & cardinal: diagonal\n-0.0005\n0.0037\n-0.13\n0.8997\n\n\n\nCTR: dod & size: small & cardinal: diagonal\n0.0110\n0.0037\n2.98\n0.0029\n\n\n\nResidual\n0.1904\n\n\n\n\n\n\n\n\n\n\nissingular(m_prm1)\n\nfalse\n\n\n\nonly(MixedModels.PCA(m_prm1))\n\n\nPrincipal components based on correlation matrix\n (Intercept) 1.0 . . . . .\n CTR: sod 0.68 1.0 . . . .\n CTR: dos 0.17 -0.08 1.0 . . .\n CTR: dod 0.83 0.6 0.28 1.0 . .\n size: small -0.42 -0.15 -0.11 -0.03 1.0 .\n cardinal: diagonal 0.05 -0.05 -0.02 0.05 0.07 1.0\n\nNormalized cumulative variances:\n[0.4224, 0.6045, 0.7798, 0.9344, 0.9891, 1.0]\n\nComponent loadings\n PC1 PC2 PC3 PC4 PC5 PC6\n (Intercept) -0.6 -0.0 0.02 0.11 -0.31 0.73\n CTR: sod -0.5 0.27 0.31 -0.11 0.75 -0.09\n CTR: dos -0.16 -0.54 -0.71 -0.23 0.35 0.07\n CTR: dod -0.55 0.16 -0.2 -0.26 -0.45 -0.6\n size: small 0.24 0.58 -0.25 -0.67 -0.0 0.31\n cardinal: diagonal -0.01 0.52 -0.55 0.64 0.12 -0.03\n\n\n\nVarCorr(m_prm1)\n\n\n\n\n\nColumn\nVariance\nStd.Dev\nCorr.\n\n\n\n\n\n\nSubj\n(Intercept)\n0.0124760\n0.1116961\n\n\n\n\n\n\n\n\nCTR: sod\n0.0043557\n0.0659974\n+0.68\n\n\n\n\n\n\n\nCTR: dos\n0.0004536\n0.0212982\n+0.17\n-0.08\n\n\n\n\n\n\nCTR: dod\n0.0010643\n0.0326233\n+0.83\n+0.60\n+0.28\n\n\n\n\n\nsize: small\n0.0140120\n0.1183725\n-0.42\n-0.15\n-0.11\n-0.03\n\n\n\n\ncardinal: diagonal\n0.0022687\n0.0476312\n+0.05\n-0.05\n-0.02\n+0.05\n+0.07\n\n\nResidual\n\n0.0362661\n0.1904365\n\n\n\n\n\n\n\n\n\n\nWe note that the critical correlation parameter between spatial (sod) and attraction (dod) is now estimated at .60 – not that close to the 1.0 boundary that caused singularity in Kliegl et al. (2011).", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#model-comparison-2", + "href": "kkl15.html#model-comparison-2", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "6.5 Model comparison 2", + "text": "6.5 Model comparison 2\n\ngof_summary = let\n nms = [:m_zcp1_rdc, :m_prm1, :m_max]\n mods = eval.(nms)\n lrt = MixedModels.likelihoodratiotest(m_prm1, m_zcp1, m_max)\n DataFrame(;\n name = nms,\n dof=dof.(mods),\n deviance=round.(deviance.(mods), digits=0),\n AIC=round.(aic.(mods),digits=0),\n AICc=round.(aicc.(mods),digits=0),\n BIC=round.(bic.(mods),digits=0),\n χ²=vcat(:., round.(lrt.tests.deviancediff, digits=0)),\n χ²_dof=vcat(:., round.(lrt.tests.dofdiff, digits=0)),\n pvalue=vcat(:., round.(lrt.tests.pvalues, digits=3))\n )\nend\n\n3×9 DataFrame\n\n\n\nRow\nname\ndof\ndeviance\nAIC\nAICc\nBIC\nχ²\nχ²_dof\npvalue\n\n\n\nSymbol\nInt64\nFloat64\nFloat64\nFloat64\nFloat64\nAny\nAny\nAny\n\n\n\n\n1\nm_zcp1_rdc\n23\n-24558.0\n-24512.0\n-24512.0\n-24308.0\n.\n.\n.\n\n\n2\nm_prm1\n38\n-24615.0\n-24539.0\n-24539.0\n-24201.0\n27.0\n5.0\n0.0\n\n\n3\nm_max\n153\n-24691.0\n-24385.0\n-24384.0\n-23025.0\n76.0\n115.0\n0.998", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#complex-lmm", + "href": "kkl15.html#complex-lmm", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "7.1 Complex LMM", + "text": "7.1 Complex LMM\nTake out interaction VCs.\n\nm_cpx = let\n form = @formula log(rt) ~ 1 + CTR * size * cardinal +\n (1 + CTR + size + cardinal | Subj)\n fit(MixedModel, form, dat; contrasts)\nend\n\nMinimizing 747 Time: 0:00:00 ( 0.14 ms/it)\n objective: -24615.344285958156\nMinimizing 882 Time: 0:00:00 ( 0.14 ms/it)\n\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n5.6911\n0.0176\n323.58\n<1e-99\n0.1117\n\n\nCTR: sod\n0.0745\n0.0076\n9.75\n<1e-21\n0.0660\n\n\nCTR: dos\n0.0408\n0.0043\n9.42\n<1e-20\n0.0213\n\n\nCTR: dod\n0.0017\n0.0051\n0.34\n0.7374\n0.0326\n\n\nsize: small\n0.0921\n0.0176\n5.23\n<1e-06\n0.1184\n\n\ncardinal: diagonal\n0.0204\n0.0053\n3.88\n0.0001\n0.0476\n\n\nCTR: sod & size: small\n0.0244\n0.0076\n3.19\n0.0014\n\n\n\nCTR: dos & size: small\n-0.0054\n0.0043\n-1.24\n0.2156\n\n\n\nCTR: dod & size: small\n0.0181\n0.0051\n3.55\n0.0004\n\n\n\nCTR: sod & cardinal: diagonal\n0.0101\n0.0028\n3.64\n0.0003\n\n\n\nCTR: dos & cardinal: diagonal\n0.0046\n0.0037\n1.25\n0.2118\n\n\n\nCTR: dod & cardinal: diagonal\n-0.0056\n0.0037\n-1.51\n0.1301\n\n\n\nsize: small & cardinal: diagonal\n0.0042\n0.0053\n0.79\n0.4272\n\n\n\nCTR: sod & size: small & cardinal: diagonal\n-0.0031\n0.0028\n-1.13\n0.2597\n\n\n\nCTR: dos & size: small & cardinal: diagonal\n-0.0005\n0.0037\n-0.13\n0.8997\n\n\n\nCTR: dod & size: small & cardinal: diagonal\n0.0110\n0.0037\n2.98\n0.0029\n\n\n\nResidual\n0.1904", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#zero-correlation-parameter-lmm-2", + "href": "kkl15.html#zero-correlation-parameter-lmm-2", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "7.2 Zero-correlation parameter LMM (2)", + "text": "7.2 Zero-correlation parameter LMM (2)\nTake out interaction VCs.\n\nm_zcp2 = let\n form = @formula log(rt) ~ 1 + CTR * size * cardinal +\n zerocorr(1 + CTR + size + cardinal | Subj)\n fit(MixedModel, form, dat; contrasts)\nend\n\n\n\n\n\nEst.\nSE\nz\np\nσ_Subj\n\n\n(Intercept)\n5.6911\n0.0173\n329.28\n<1e-99\n0.1572\n\n\nCTR: sod\n0.0745\n0.0077\n9.66\n<1e-21\n0.0666\n\n\nCTR: dos\n0.0409\n0.0045\n9.09\n<1e-19\n0.0240\n\n\nCTR: dod\n0.0015\n0.0053\n0.29\n0.7751\n0.0357\n\n\nsize: small\n0.0920\n0.0173\n5.33\n<1e-06\n0.0295\n\n\ncardinal: diagonal\n0.0204\n0.0053\n3.88\n0.0001\n0.0476\n\n\nCTR: sod & size: small\n0.0244\n0.0077\n3.17\n0.0015\n\n\n\nCTR: dos & size: small\n-0.0054\n0.0045\n-1.20\n0.2317\n\n\n\nCTR: dod & size: small\n0.0181\n0.0053\n3.39\n0.0007\n\n\n\nCTR: sod & cardinal: diagonal\n0.0101\n0.0028\n3.64\n0.0003\n\n\n\nCTR: dos & cardinal: diagonal\n0.0046\n0.0037\n1.25\n0.2125\n\n\n\nCTR: dod & cardinal: diagonal\n-0.0056\n0.0037\n-1.51\n0.1317\n\n\n\nsize: small & cardinal: diagonal\n0.0042\n0.0053\n0.79\n0.4286\n\n\n\nCTR: sod & size: small & cardinal: diagonal\n-0.0031\n0.0028\n-1.12\n0.2610\n\n\n\nCTR: dos & size: small & cardinal: diagonal\n-0.0005\n0.0037\n-0.12\n0.9023\n\n\n\nCTR: dod & size: small & cardinal: diagonal\n0.0109\n0.0037\n2.95\n0.0032\n\n\n\nResidual\n0.1904", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#model-comparison-3", + "href": "kkl15.html#model-comparison-3", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "7.3 Model comparison 3", + "text": "7.3 Model comparison 3\n\ngof_summary = let\n nms = [:m_zcp2, :m_cpx, :m_max]\n mods = eval.(nms)\n lrt = MixedModels.likelihoodratiotest(m_zcp2, m_cpx, m_max)\n DataFrame(;\n name = nms,\n dof=dof.(mods),\n deviance=round.(deviance.(mods), digits=0),\n AIC=round.(aic.(mods),digits=0),\n AICc=round.(aicc.(mods),digits=0),\n BIC=round.(bic.(mods),digits=0),\n χ²=vcat(:., round.(lrt.tests.deviancediff, digits=0)),\n χ²_dof=vcat(:., round.(lrt.tests.dofdiff, digits=0)),\n pvalue=vcat(:., round.(lrt.tests.pvalues, digits=3))\n )\nend\n\n3×9 DataFrame\n\n\n\nRow\nname\ndof\ndeviance\nAIC\nAICc\nBIC\nχ²\nχ²_dof\npvalue\n\n\n\nSymbol\nInt64\nFloat64\nFloat64\nFloat64\nFloat64\nAny\nAny\nAny\n\n\n\n\n1\nm_zcp2\n23\n-24558.0\n-24512.0\n-24512.0\n-24308.0\n.\n.\n.\n\n\n2\nm_cpx\n38\n-24615.0\n-24539.0\n-24539.0\n-24201.0\n57.0\n15.0\n0.0\n\n\n3\nm_max\n153\n-24691.0\n-24385.0\n-24384.0\n-23025.0\n76.0\n115.0\n0.998", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#residual-over-fitted-plot", + "href": "kkl15.html#residual-over-fitted-plot", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "10.1 Residual-over-fitted plot", + "text": "10.1 Residual-over-fitted plot\nThe slant in residuals show a lower and upper boundary of reaction times, that is we have have too few short and too few long residuals. Not ideal, but at least width of the residual band looks similar across the fitted values, that is there is no evidence for heteroskedasticity.\n\n\nCode\nCairoMakie.activate!(; type=\"png\")\nscatter(fitted(m_prm1), residuals(m_prm1); alpha=0.3)\n\n\n\n\n\n\n\nFigure 3: Residuals versus fitted values for model m1\n\n\n\n\nWith many observations the scatterplot is not that informative. Contour plots or heatmaps may be an alternative.\n\n\nCode\nset_aog_theme!()\ndraw(\n data((; f=fitted(m_prm1), r=residuals(m_prm1))) *\n mapping(\n :f => \"Fitted values from m1\", :r => \"Residuals from m1\"\n ) *\n density();\n)\n\n\n\n\n\n\n\nFigure 4: Heatmap of residuals versus fitted values for model m1", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#q-q-plot", + "href": "kkl15.html#q-q-plot", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "10.2 Q-Q plot", + "text": "10.2 Q-Q plot\nThe plot of quantiles of model residuals over corresponding quantiles of the normal distribution should yield a straight line along the main diagonal.\n\n\nCode\nCairoMakie.activate!(; type=\"png\")\nqqnorm(\n residuals(m_prm1);\n qqline=:none,\n axis=(;\n xlabel=\"Standard normal quantiles\",\n ylabel=\"Quantiles of the residuals from model m1\",\n ),\n)\n\n\n\n\n\n\n\nFigure 5: Quantile-quantile plot of the residuals for model m1 versus a standard normal", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#observed-and-theoretical-normal-distribution", + "href": "kkl15.html#observed-and-theoretical-normal-distribution", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "10.3 Observed and theoretical normal distribution", + "text": "10.3 Observed and theoretical normal distribution\n******We****** can see this in this plot. Overall, it does not look too bad.\n\n\nCode\nCairoMakie.activate!(; type=\"svg\")\nlet\n n = nrow(dat)\n dat_rz = (;\n value=vcat(residuals(m_prm1) ./ std(residuals(m_prm1)), randn(n)),\n curve=repeat([\"residual\", \"normal\"]; inner=n),\n )\n draw(\n data(dat_rz) *\n mapping(:value; color=:curve) *\n density(; bandwidth=0.1);\n )\nend\n\n\n\n\n\n\n\n\nFigure 6: Kernel density plot of the standardized residuals for model m1 versus a standard normal", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#caterpillar-plot", + "href": "kkl15.html#caterpillar-plot", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "11.1 Caterpillar plot", + "text": "11.1 Caterpillar plot\n\n\nCode\ncm1 = only(ranefinfo(m_prm1))\ncaterpillar!(Figure(; resolution=(800, 1200)), cm1; orderby=2)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 7: Prediction intervals of the subject random effects in model m1", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#shrinkage-plot", + "href": "kkl15.html#shrinkage-plot", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "11.2 Shrinkage plot", + "text": "11.2 Shrinkage plot\n\n11.2.1 Log-transformed reaction times (LMM m_prm1)\n\n\nCode\nshrinkageplot!(Figure(; resolution=(1000, 1200)), m_prm1)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 8: Shrinkage plots of the subject random effects in model m1L", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#generate-a-bootstrap-sample", + "href": "kkl15.html#generate-a-bootstrap-sample", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "12.1 Generate a bootstrap sample", + "text": "12.1 Generate a bootstrap sample\nWe generate 2500 samples for the 15 model parameters (4 fixed effect, 7 VCs, 15 CPs, and 1 residual).\n\nsamp = parametricbootstrap(MersenneTwister(1234321), 2500, m_prm1;\n optsum_overrides=(; ftol_rel=1e-8));\n\n\ntbl = samp.tbl\n\nTable with 60 columns and 2500 rows:\n obj β01 β02 β03 β04 β05 ⋯\n ┌────────────────────────────────────────────────────────────────────\n 1 │ -25134.3 5.68049 0.0689586 0.0393162 -0.00115769 0.0780895 ⋯\n 2 │ -24466.7 5.66099 0.0666992 0.0419148 -0.00303919 0.101403 ⋯\n 3 │ -24448.9 5.69085 0.0782739 0.0370164 0.00841092 0.0970059 ⋯\n 4 │ -24614.9 5.6972 0.0762013 0.0462014 -0.00324924 0.0631257 ⋯\n 5 │ -25168.8 5.68336 0.068479 0.0386334 0.00257696 0.117804 ⋯\n 6 │ -24729.2 5.69257 0.0843594 0.0442014 0.00172628 0.0942692 ⋯\n 7 │ -24499.0 5.67623 0.0705523 0.0418438 -0.00636468 0.0768522 ⋯\n 8 │ -24804.4 5.69366 0.0765241 0.0457697 0.00292816 0.0869247 ⋯\n 9 │ -24396.7 5.68366 0.0814744 0.0495446 -0.00164492 0.101197 ⋯\n 10 │ -24577.8 5.6861 0.0794278 0.0344818 0.00620175 0.0818131 ⋯\n 11 │ -25086.6 5.6658 0.0662387 0.0445268 -0.000810468 0.0947995 ⋯\n 12 │ -24364.5 5.67832 0.0697527 0.0446342 -0.000982085 0.102116 ⋯\n 13 │ -24972.8 5.6935 0.0729605 0.0421222 -0.00304358 0.0842434 ⋯\n 14 │ -24616.6 5.70284 0.0766981 0.0432196 -0.00502166 0.0714031 ⋯\n 15 │ -24772.4 5.68248 0.0825491 0.0371784 0.00389937 0.088786 ⋯\n 16 │ -24632.9 5.69897 0.0718701 0.0408715 0.000512973 0.0851757 ⋯\n 17 │ -24397.0 5.70738 0.0970143 0.0363716 0.00149944 0.0856153 ⋯\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋱", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#shortest-coverage-interval", + "href": "kkl15.html#shortest-coverage-interval", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "12.2 Shortest coverage interval", + "text": "12.2 Shortest coverage interval\n\nconfint(samp)\n\nDictTable with 2 columns and 38 rows:\n par lower upper\n ────┬────────────────────────\n β01 │ 5.65513 5.72404\n β02 │ 0.06039 0.0904927\n β03 │ 0.0325719 0.0489381\n β04 │ -0.00815893 0.0115762\n β05 │ 0.0580068 0.124695\n β06 │ 0.0104524 0.0312924\n β07 │ 0.00972538 0.0388313\n β08 │ -0.0138485 0.00313372\n β09 │ 0.00790021 0.0275058\n β10 │ 0.00485266 0.0157063\n β11 │ -0.00250178 0.0117922\n β12 │ -0.0124778 0.00200615\n β13 │ -0.00597154 0.0135989\n β14 │ -0.00828454 0.0022754\n β15 │ -0.00784482 0.00657597\n β16 │ 0.00368085 0.0180065\n ρ01 │ 0.38489 1.0\n ⋮ │ ⋮ ⋮\n\n\nWe can also visualize the shortest coverage intervals for fixed effects with the ridgeplot() command:\n\n\nCode\nridgeplot(samp; show_intercept=false)\n\n\n\n\n\n\n\n\nFigure 9: Ridge plot of fixed-effects bootstrap samples from model m1L", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + }, + { + "objectID": "kkl15.html#comparative-density-plots-of-bootstrapped-parameter-estimates", + "href": "kkl15.html#comparative-density-plots-of-bootstrapped-parameter-estimates", + "title": "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity", + "section": "12.3 Comparative density plots of bootstrapped parameter estimates", + "text": "12.3 Comparative density plots of bootstrapped parameter estimates\n\n12.3.1 Residual\n\n\nCode\ndraw(\n data(tbl) *\n mapping(:σ => \"Residual\") *\n density();\n figure=(; resolution=(800, 400)),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 10: Kernel density estimate from bootstrap samples of the residual standard deviation for model m_prm1\n\n\n\n\n\n\n\n12.3.2 Fixed effects and associated variance components (w/o GM)\nThe shortest coverage interval for the GM ranges from x to x ms and the associate variance component from .x to .x. To keep the plot range small we do not include their densities here.\n\n\nCode\nrn = renamer([\n \"(Intercept)\" => \"GM\",\n \"CTR: sod\" => \"spatial effect\",\n \"CTR: dos\" => \"object effect\",\n \"CTR: dod\" => \"attraction effect\",\n \"(Intercept), CTR: sod\" => \"GM, spatial\",\n \"(Intercept), CTR: dos\" => \"GM, object\",\n \"CTR: sod, CTR: dos\" => \"spatial, object\",\n \"(Intercept), CTR: dod\" => \"GM, attraction\",\n \"CTR: sod, CTR: dod\" => \"spatial, attraction\",\n \"CTR: dos, CTR: dod\" => \"object, attraction\",\n])\ndraw(\n data(tbl) *\n mapping(\n [:β02, :β03, :β04] .=> \"Experimental effect size [ms]\";\n color=dims(1) =>\n renamer([\"spatial effect\", \"object effect\", \"attraction effect\"]) =>\n \"Experimental effects\",\n ) *\n density();\n figure=(; resolution=(800, 350)),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 11: Kernel density estimate from bootstrap samples of the fixed effects for model m_prm1\n\n\n\n\n\nThe densitiies correspond nicely with the shortest coverage intervals.\n\n\nCode\ndraw(\n data(tbl) *\n mapping(\n [:σ2, :σ3, :σ4] .=> \"Standard deviations [ms]\";\n color=dims(1) =>\n renamer([\"spatial effect\", \"object effect\", \"attraction effect\"]) =>\n \"Variance components\",\n ) *\n density();\n figure=(; resolution=(800, 350)),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 12: Kernel density estimate from bootstrap samples of the standard deviations for model m1L (excluding Grand Mean)\n\n\n\n\n\nThe VC are all very nicely defined.\n\n\n12.3.3 Correlation parameters (CPs)\n\n\nCode\ndraw(\n data(tbl) *\n mapping(\n [:ρ01, :ρ02, :ρ03, :ρ04, :ρ05, :ρ06] .=> \"Correlation\";\n color=dims(1) =>\n renamer([\"GM, spatial\", \"GM, object\", \"spatial, object\",\n \"GM, attraction\", \"spatial, attraction\", \"object, attraction\"]) =>\n \"Correlation parameters\",\n ) *\n density();\n figure=(; resolution=(800, 350)),\n)\n\n\n┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.\n└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227\n\n\n\n\n\n\n\n\nFigure 13: Kernel density estimate from bootstrap samples of the standard deviations for model m1L\n\n\n\n\n\nThree CPs stand out positively, the correlation between GM and the spatial effect, GM and attraction effect, and the correlation between spatial and attraction effects. The second CP was positive, but not significant in the first study. The third CP replicates a CP that was judged questionable in script kwdyz11.jl.\nThe three remaining CPs are not well defined for log-transformed reaction times; they only fit noise and should be removed. It is also possible that fitting the complex experimental design (including target size and rectangle orientation) will lead to more acceptable estimates. The corresponding plot based on LMM m1_rt for raw reaction times still shows them with very wide distributions, but acceptable.", + "crumbs": [ + "Worked examples", + "RePsychLing Kliegl, Kuschela, & Laubrock (2015)- Reduction of Model Complexity" + ] + } +] \ No newline at end of file diff --git a/selection.html b/selection.html new file mode 100644 index 0000000..e4c59bc --- /dev/null +++ b/selection.html @@ -0,0 +1,1115 @@ + + + + + + + + + + + +selection – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +
+
+Code +
using Arrow
+using CairoMakie
+using DataFrames
+CairoMakie.activate!(; type="svg") # use SVG (other options include PNG)
+
+
+
+
tbl = Arrow.Table("./data/fggk21.arrow")
+
+
Arrow.Table with 525126 rows, 7 columns, and schema:
+ :Cohort  String
+ :School  String
+ :Child   String
+ :Sex     String
+ :age     Float64
+ :Test    String
+ :score   Float64
+
+
+
+
df = DataFrame(tbl)
+describe(df)
+
+
7×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1Cohort201120190String
2SchoolS100043S8002000String
3ChildC002352C1179660String
4Sexfemalemale0String
5age8.560737.994528.558529.106090Float64
6TestBPTStar_r0String
7score226.1411.141524.651161530.00Float64
+
+
+
+
+

1 Raw score density

+
+
let
+  fdensity = Figure(; resolution=(1000, 500))
+  axs = Axis(fdensity[1, 1])
+  tdf = filter(:Test => ==(test), df)
+  colors = Makie.cgrad(:PuOr_4, 2; categorical=true, alpha=0.6)
+  if by_sex
+    density!(
+      axs,
+      filter(:Sex => ==("female"), tdf).score;
+      color=colors[1],
+      label="Girls",
+    )
+    density!(
+      axs,
+      filter(:Sex => ==("male"), tdf).score;
+      color=colors[2],
+      label="Boys",
+    )
+    axislegend(axs; position=:lt)
+  else
+    density!(axs, tdf.score)
+  end
+  fdensity
+end
+
+ + +
+ + Back to top
+ +
+ + + + + \ No newline at end of file diff --git a/shrinkageplot.html b/shrinkageplot.html new file mode 100644 index 0000000..57c75c0 --- /dev/null +++ b/shrinkageplot.html @@ -0,0 +1,2135 @@ + + + + + + + + + + + +More on shrinkage plots – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

More on shrinkage plots

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + + +
+
+Code +
using CairoMakie
+using DataFrames
+using LinearAlgebra
+using MixedModels
+using MixedModelsMakie
+using Random
+using ProgressMeter
+
+ProgressMeter.ijulia_behavior(:clear);
+
+
+

Load the kb07 data set (don’t tell Reinhold that I used these data).

+
+
kb07 = MixedModels.dataset(:kb07)
+
+
Arrow.Table with 1789 rows, 7 columns, and schema:
+ :subj      String
+ :item      String
+ :spkr      String
+ :prec      String
+ :load      String
+ :rt_trunc  Int16
+ :rt_raw    Int16
+
+
+
+
contrasts = Dict(
+  :subj => Grouping(),
+  :item => Grouping(),
+  :spkr => HelmertCoding(),
+  :prec => HelmertCoding(),
+  :load => HelmertCoding(),
+)
+m1 = let
+  form = @formula(
+    rt_trunc ~
+      1 +
+      spkr * prec * load +
+      (1 + spkr + prec + load | subj) +
+      (1 + spkr + prec + load | item)
+  )
+  fit(MixedModel, form, kb07; contrasts)
+end
+
+
Minimizing 2    Time: 0:00:00 ( 0.46  s/it)
+  objective:  29353.753287212847
+Minimizing 614    Time: 0:00:01 ( 3.05 ms/it)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_subjσ_item
(Intercept)2181.675077.302928.22<1e-99301.8414362.1701
spkr: old67.749018.28243.710.000243.039640.5288
prec: maintain-333.920647.1621-7.08<1e-1162.1035246.9365
load: yes78.768119.52474.03<1e-0465.147342.1755
spkr: old & prec: maintain-21.963415.8062-1.390.1647
spkr: old & load: yes18.383815.80621.160.2448
prec: maintain & load: yes4.533315.80620.290.7743
spkr: old & prec: maintain & load: yes23.605215.80621.490.1353
Residual668.5061
+
+
+
+
VarCorr(m1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
subj(Intercept)91108.2428301.8414
spkr: old1852.409843.0396+0.78
prec: maintain3856.840862.1035-0.59+0.02
load: yes4244.168965.1473+0.36+0.82+0.53
item(Intercept)131167.1572362.1701
spkr: old1642.582540.5288+0.44
prec: maintain60977.6107246.9365-0.69+0.35
load: yes1778.776342.1755+0.32+0.23-0.15
Residual446900.4651668.5061
+
+
+
+
issingular(m1)
+
+
true
+
+
+
+
print(m1)
+
+
Linear mixed model fit by maximum likelihood
+ rt_trunc ~ 1 + spkr + prec + load + spkr & prec + spkr & load + prec & load + spkr & prec & load + (1 + spkr + prec + load | subj) + (1 + spkr + prec + load | item)
+    logLik   -2 logLik      AIC         AICc        BIC     
+ -14318.5697  28637.1394  28695.1394  28696.1286  28854.3323
+
+Variance components:
+             Column       Variance  Std.Dev.   Corr.
+subj     (Intercept)      91108.2428 301.8414
+         spkr: old         1852.4098  43.0396 +0.78
+         prec: maintain    3856.8408  62.1035 -0.59 +0.02
+         load: yes         4244.1689  65.1473 +0.36 +0.82 +0.53
+item     (Intercept)     131167.1572 362.1701
+         spkr: old         1642.5825  40.5288 +0.44
+         prec: maintain   60977.6107 246.9365 -0.69 +0.35
+         load: yes         1778.7763  42.1755 +0.32 +0.23 -0.15
+Residual                 446900.4651 668.5061
+ Number of obs: 1789; levels of grouping factors: 56, 32
+
+  Fixed-effects parameters:
+───────────────────────────────────────────────────────────────────────────────
+                                             Coef.  Std. Error      z  Pr(>|z|)
+───────────────────────────────────────────────────────────────────────────────
+(Intercept)                             2181.67        77.3029  28.22    <1e-99
+spkr: old                                 67.749       18.2824   3.71    0.0002
+prec: maintain                          -333.921       47.1621  -7.08    <1e-11
+load: yes                                 78.7681      19.5247   4.03    <1e-04
+spkr: old & prec: maintain               -21.9634      15.8062  -1.39    0.1647
+spkr: old & load: yes                     18.3838      15.8062   1.16    0.2448
+prec: maintain & load: yes                 4.53334     15.8062   0.29    0.7743
+spkr: old & prec: maintain & load: yes    23.6052      15.8062   1.49    0.1353
+───────────────────────────────────────────────────────────────────────────────
+
+
+
+

1 Expressing the covariance of random effects

+

Earlier today we mentioned that the parameters being optimized are from a “matrix square root” of the covariance matrix for the random effects. There is one such lower triangular matrix for each grouping factor.

+
+
l1 = first(m1.λ)   # Cholesky factor of relative covariance for subj
+
+
4×4 LowerTriangular{Float64, Matrix{Float64}}:
+  0.451516    ⋅          ⋅          ⋅ 
+  0.0502699  0.0402238   ⋅          ⋅ 
+ -0.0551878  0.072667   0.0174361   ⋅ 
+  0.0351047  0.084743   0.0329117  0.0
+
+
+

Notice the zero on the diagonal. A triangular matrix with zeros on the diagonal is singular.

+
+
l2 = last(m1.λ)    # this one is also singular
+
+
4×4 LowerTriangular{Float64, Matrix{Float64}}:
+  0.54176     ⋅            ⋅         ⋅ 
+  0.0267807  0.0543902     ⋅         ⋅ 
+ -0.253141   0.269008     0.0        ⋅ 
+  0.0200425  0.00611238  -0.044718  0.0392618
+
+
+

To regenerate the covariance matrix we need to know that the covariance is not the square of l1, it is l1 * l1' (so that the result is symmetric) and multiplied by σ̂²

+
+
Σ₁ = varest(m1) .* (l1 * l1')
+
+
4×4 Matrix{Float64}:
+  91108.2   10143.6     -11136.0     7083.52
+  10143.6    1852.41        66.4352  2311.99
+ -11136.0      66.4352    3856.84    2142.67
+   7083.52   2311.99      2142.67    4244.17
+
+
+
+
diag(Σ₁)  # compare to the variance column in the VarCorr output
+
+
4-element Vector{Float64}:
+ 91108.24276565886
+  1852.4098101273987
+  3856.8407541614683
+  4244.168853804923
+
+
+
+
sqrt.(diag(Σ₁))
+
+
4-element Vector{Float64}:
+ 301.8414198973674
+  43.03963069227475
+  62.10346813312014
+  65.1472858514069
+
+
+
+
+

2 Shrinkage plots

+
+
+Code +
shrinkageplot(m1)
+
+
+
+
+
+ +
+
+Figure 1: Shrinkage plot of model m1 +
+
+
+
+
+

The upper left panel shows the perfect negative correlation for those two components of the random effects.

+
+
shrinkageplot(m1, :item)
+
+
+
+

+
+
+
+
+
+
X1 = Int.(m1.X')
+
+
8×1789 Matrix{Int64}:
+  1   1   1   1   1  1   1   1   1   1  …   1   1   1   1   1   1   1  1   1
+ -1   1   1  -1  -1  1   1  -1  -1   1      1  -1  -1   1   1  -1  -1  1   1
+ -1   1  -1   1  -1  1  -1   1  -1   1     -1   1  -1   1  -1   1  -1  1  -1
+  1  -1  -1  -1  -1  1   1   1   1  -1      1   1   1  -1  -1  -1  -1  1   1
+  1   1  -1  -1   1  1  -1  -1   1   1     -1  -1   1   1  -1  -1   1  1  -1
+ -1  -1  -1   1   1  1   1  -1  -1  -1  …   1  -1  -1  -1  -1   1   1  1   1
+ -1  -1   1  -1   1  1  -1   1  -1  -1     -1   1  -1  -1   1  -1   1  1  -1
+  1  -1   1   1  -1  1  -1  -1   1  -1     -1  -1   1  -1   1   1  -1  1  -1
+
+
+
+
X1 * X1'
+
+
8×8 Matrix{Int64}:
+ 1789    -1    -1     3    -3     1     1     3
+   -1  1789    -3     1    -1     3     3     1
+   -1    -3  1789     1    -1     3     3     1
+    3     1     1  1789     3    -1    -1    -3
+   -3    -1    -1     3  1789     1     1     3
+    1     3     3    -1     1  1789    -3    -1
+    1     3     3    -1     1    -3  1789    -1
+    3     1     1    -3     3    -1    -1  1789
+
+
+
+
+

3 How to interpret a shrinkage plot

+
    +
  • Extreme shrinkage (shrunk to a line or to a point) is easy to interpret - the term is not providing benefit and can be removed.
    • +
  • When the range of the blue dots (shrunk values) is comparable to those of the red dots (unshrunk) it indicates that the term after shrinkage is about as strong as without shrinkage.
    • +
  • By itself, this doesn’t mean that the term is important. In some ways you need to get a feeling for the absolute magnitude of the random effects in addition to the relative magnitude.
    • +
  • Small magnitude and small relative magnitude indicate you can drop that term
    • +
+
+
+

4 Conclusions from these plots

+
    +
  • Only the intercept for the subj appears to be contributing explanatory power
    • +
  • For the item both the intercept and the spkr appear to be contributing
    • +
+
+
m2 = let
+  form = @formula(
+    rt_trunc ~
+      1 + prec * spkr * load + (1 | subj) + (1 + prec | item)
+  )
+  fit(MixedModel, form, kb07; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)2181.758277.470928.16<1e-99364.7286298.1109
prec: maintain-333.858247.4629-7.03<1e-11252.6687
spkr: old67.811416.05264.22<1e-04
load: yes78.684916.05254.90<1e-06
prec: maintain & spkr: old-21.880216.0525-1.360.1729
prec: maintain & load: yes4.471016.05260.280.7806
spkr: old & load: yes18.321416.05261.140.2537
prec: maintain & spkr: old & load: yes23.521916.05251.470.1428
Residual678.9318
+
+
+
+
VarCorr(m2)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)133026.916364.729
prec: maintain63841.497252.669-0.70
subj(Intercept)88870.081298.111
Residual460948.432678.932
+
+
+
+
+Code +
shrinkageplot(m2)
+
+
+
+
+
+ +
+
+Figure 2: Shrinkage plot of model m2 +
+
+
+
+
+
+
m3 = let
+  form = @formula(
+    rt_trunc ~
+      1 + prec + spkr + load + (1 | subj) + (1 + prec | item)
+  )
+  fit(MixedModel, form, kb07; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)2181.852677.468128.16<1e-99364.7125298.0259
prec: maintain-333.790647.4472-7.03<1e-11252.5212
spkr: old67.879016.07854.22<1e-04
load: yes78.590416.07854.89<1e-05
Residual680.0319
+
+
+
+
VarCorr(m3)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)133015.243364.713
prec: maintain63766.936252.521-0.70
subj(Intercept)88819.436298.026
Residual462443.388680.032
+
+
+
+
rng = Random.seed!(1234321);
+
+
+
m3btstrp = parametricbootstrap(rng, 2000, m3);
+
+
+
DataFrame(shortestcovint(m3btstrp))
+
+
9×5 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowtypegroupnameslowerupper
StringString?String?Float64Float64
1βmissing(Intercept)2030.642335.35
2βmissingprec: maintain-418.691-237.661
3βmissingspkr: old36.082397.0289
4βmissingload: yes47.7298109.975
5σitem(Intercept)264.8450.381
6σitemprec: maintain170.059310.399
7ρitem(Intercept), prec: maintain-0.909187-0.475621
8σsubj(Intercept)229.638356.51
9σresidualmissing657.775701.589
+
+
+
+
+
ridgeplot(m3btstrp)
+
+
+
+
+ +
+
+Figure 3: Ridge plot of the fixed-effects coefficients from the bootstrap sample +
+
+
+
+
+
+
ridgeplot(m3btstrp; show_intercept=false)
+
+
+
+
+ +
+
+Figure 4: Ridge plot of the fixed-effects coefficients from the bootstrap sample (with the intercept) +
+
+
+
+
+
+
m4 = let
+  form = @formula(
+    rt_trunc ~
+      1 + prec + spkr + load + (1 + prec | item) + (1 | subj)
+  )
+  fit(MixedModel, form, kb07; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_itemσ_subj
(Intercept)2181.852677.468128.16<1e-99364.7125298.0259
prec: maintain-333.790647.4472-7.03<1e-11252.5212
spkr: old67.879016.07854.22<1e-04
load: yes78.590416.07854.89<1e-05
Residual680.0319
+
+
+
+
m4bstrp = parametricbootstrap(rng, 2000, m4);
+
+
+
ridgeplot(m4bstrp; show_intercept=false)
+
+
+
+

+
+
+
+
+
+
DataFrame(shortestcovint(m4bstrp))
+
+
9×5 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowtypegroupnameslowerupper
StringString?String?Float64Float64
1βmissing(Intercept)2035.632335.48
2βmissingprec: maintain-425.693-241.478
3βmissingspkr: old37.7216100.267
4βmissingload: yes50.2233112.426
5σitem(Intercept)263.272452.286
6σitemprec: maintain171.223313.796
7ρitem(Intercept), prec: maintain-0.917884-0.4704
8σsubj(Intercept)228.256357.775
9σresidualmissing656.872701.941
+
+
+
+
+
VarCorr(m4)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
item(Intercept)133015.243364.713
prec: maintain63766.936252.521-0.70
subj(Intercept)88819.436298.026
Residual462443.388680.032
+
+
+
+
+Code +
let mods = [m1, m2, m4]
+  DataFrame(;
+    geomdof=(sum ∘ leverage).(mods),
+    npar=dof.(mods),
+    deviance=deviance.(mods),
+    AIC=aic.(mods),
+    BIC=bic.(mods),
+    AICc=aicc.(mods),
+  )
+end
+
+
+
3×6 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowgeomdofnpardevianceAICBICAICc
Float64Int64Float64Float64Float64Float64
1131.5512928637.128695.128854.328696.1
2107.5431328658.528684.528755.828684.7
3103.478928663.928681.928731.328682.0
+
+
+
+
+
scatter(fitted(m4), residuals(m4))
+
+
+
+
+ +
+
+Figure 5: Residuals versus fitted values for model m4 +
+
+
+
+
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/shrinkageplot_files/figure-html/cell-14-output-1.svg b/shrinkageplot_files/figure-html/cell-14-output-1.svg new file mode 100644 index 0000000..d884b3f --- /dev/null +++ b/shrinkageplot_files/figure-html/cell-14-output-1.svg @@ -0,0 +1,1560 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/shrinkageplot_files/figure-html/cell-29-output-1.svg b/shrinkageplot_files/figure-html/cell-29-output-1.svg new file mode 100644 index 0000000..71025c3 --- /dev/null +++ b/shrinkageplot_files/figure-html/cell-29-output-1.svg @@ -0,0 +1,561 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/shrinkageplot_files/figure-html/fig-m1shrinkage-output-1.svg b/shrinkageplot_files/figure-html/fig-m1shrinkage-output-1.svg new file mode 100644 index 0000000..9345815 --- /dev/null +++ b/shrinkageplot_files/figure-html/fig-m1shrinkage-output-1.svg @@ -0,0 +1,2062 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/shrinkageplot_files/figure-html/fig-m2shrinkage-output-1.svg b/shrinkageplot_files/figure-html/fig-m2shrinkage-output-1.svg new file mode 100644 index 0000000..5c78422 --- /dev/null +++ b/shrinkageplot_files/figure-html/fig-m2shrinkage-output-1.svg @@ -0,0 +1,399 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/shrinkageplot_files/figure-html/fig-ridgeplot-output-1.svg b/shrinkageplot_files/figure-html/fig-ridgeplot-output-1.svg new file mode 100644 index 0000000..277e169 --- /dev/null +++ b/shrinkageplot_files/figure-html/fig-ridgeplot-output-1.svg @@ -0,0 +1,595 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/shrinkageplot_files/figure-html/fig-ridgeplotnoint-output-1.svg b/shrinkageplot_files/figure-html/fig-ridgeplotnoint-output-1.svg new file mode 100644 index 0000000..1484181 --- /dev/null +++ b/shrinkageplot_files/figure-html/fig-ridgeplotnoint-output-1.svg @@ -0,0 +1,564 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/shrinkageplot_files/figure-html/fig-scatterm4-output-1.svg b/shrinkageplot_files/figure-html/fig-scatterm4-output-1.svg new file mode 100644 index 0000000..79d5d69 --- /dev/null +++ b/shrinkageplot_files/figure-html/fig-scatterm4-output-1.svg @@ -0,0 +1,1930 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/singularity.html b/singularity.html new file mode 100644 index 0000000..6f4fd54 --- /dev/null +++ b/singularity.html @@ -0,0 +1,1202 @@ + + + + + + + + + + + +Convergence, singularity and all that – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Convergence, singularity and all that

+
+ + + +
+ +
+
Author
+
+

Douglas Bates

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +
+

0.1 What does it mean for a converged model to be singular?

+

Add the packages to be used

+
+
+Code +
using CairoMakie
+using DataFrames
+using LinearAlgebra
+using MixedModels
+using MixedModelsMakie
+using ProgressMeter
+
+ProgressMeter.ijulia_behavior(:clear)
+CairoMakie.activate!(; type="svg")
+
+
+

Fit a model for reaction time in the sleepstudy example, preserving information on the estimation progress (the thin=1 optional argument)

+
+
m01 = let f = @formula reaction ~ 1 + days + (1 + days|subj)
+  fit(MixedModel, f, MixedModels.dataset(:sleepstudy); thin=1)
+end
+print(m01)
+
+
Linear mixed model fit by maximum likelihood
+ reaction ~ 1 + days + (1 + days | subj)
+   logLik   -2 logLik     AIC       AICc        BIC    
+  -875.9697  1751.9393  1763.9393  1764.4249  1783.0971
+
+Variance components:
+            Column    Variance Std.Dev.   Corr.
+subj     (Intercept)  565.51067 23.78047
+         days          32.68212  5.71683 +0.08
+Residual              654.94145 25.59182
+ Number of obs: 180; levels of grouping factors: 18
+
+  Fixed-effects parameters:
+──────────────────────────────────────────────────
+                Coef.  Std. Error      z  Pr(>|z|)
+──────────────────────────────────────────────────
+(Intercept)  251.405      6.63226  37.91    <1e-99
+days          10.4673     1.50224   6.97    <1e-11
+──────────────────────────────────────────────────
+
+
+

The covariance matrix for each subject’s random effects is evaluated from its “matrix square root”, called the Cholesky factor.

+
+
λ = only(m01.λ)
+
+
2×2 LowerTriangular{Float64, Matrix{Float64}}:
+ 0.929221    ⋅ 
+ 0.0181684  0.222645
+
+
+

The transpose of \(\lambda\), written \(\lambda'\), is an upper triangular matrix generated by “flipping” \(\lambda\) about the main diagonal.

+
+
λ'
+
+
2×2 UpperTriangular{Float64, Adjoint{Float64, Matrix{Float64}}}:
+ 0.929221  0.0181684
+  ⋅        0.222645
+
+
+

The product \(\lambda * \lambda'\) will be symmetric. The covariance matrix of the random effects, \(\Sigma\), is this symmetric matrix scaled by \(\sigma^2\)

+
+
Σ = m01.σ^2 * λ * λ'
+
+
2×2 Matrix{Float64}:
+ 565.511  11.057
+  11.057  32.6821
+
+
+

The estimated variances of the random effects, which are the diagonal elements of \(\Sigma\), correspond to the values shown in the table. To evaluate the covariance, isolate the correlation

+
+
# m01.σρs extracts the `σρs` property of the model.
+# This property is a NamedTuple where the names
+# correspond to grouping factors - in this case, `subj`.
+# So `m01.σρs.subj.ρ` is the estimated correlation(s) for
+# this grouping factor.  Because there is only one such correlation
+# we can extract it with `only()`, which also verifies that
+# there is exactly one.
+ρ = only(m01.σρs.subj.ρ)
+
+
0.08133216865276134
+
+
+

and multiply by the standard deviations

+
+
ρ * sqrt(first(Σ) * last(Σ))
+
+
11.057014395926759
+
+
+
+
+

0.2 The factor is generated from a parameter vector

+

In practice we optimize the log-likelihood with respect to a parameter vector called \(\theta\) that generates \(\lambda\).

+
+
m01.θ
+
+
3-element Vector{Float64}:
+ 0.9292213162629417
+ 0.018168381188609965
+ 0.22264487194620683
+
+
+

The elements of this parameter vector are subject to constraints. In particular, two of the three elements have a lower bound of zero.

+
+
m01.lowerbd
+
+
3-element Vector{Float64}:
+   0.0
+ -Inf
+   0.0
+
+
+

That is, the first and third elements of \(\theta\), corresponding to diagonal elements of \(\lambda\), must be non-negative, whereas the second component is unconstrained (has a lower bound of \(-\infty\)).

+
+
+

0.3 Progress of the iterations

+

The optsum.fitlog property of the model is a vector of tuples where each tuple contains the value of the \(\theta\) vector and the value of the objective at that \(\theta\). The fitlog always contains the first and the last evaluation. When the thin named argument is set, this property has a row for every thin’th evaluation.

+
+
m01.optsum.fitlog
+
+
57-element Vector{Tuple{Vector{Float64}, Float64}}:
+ ([1.0, 0.0, 1.0], 1784.642296192471)
+ ([1.75, 0.0, 1.0], 1790.1256369894638)
+ ([1.0, 1.0, 1.0], 1798.999624496596)
+ ([1.0, 0.0, 1.75], 1803.8532002844106)
+ ([0.25, 0.0, 1.0], 1800.6139807455515)
+ ([1.0, -1.0, 1.0], 1798.6046308389316)
+ ([1.0, 0.0, 0.25], 1752.2607369909213)
+ ([1.1832612965367613, -0.0086618879582246, 0.0], 1797.5876920199157)
+ ([1.075, 0.0, 0.32499999999999996], 1754.9541095798672)
+ ([0.8166315695343094, 0.011167254457244754, 0.28823768689703533], 1753.6956816568222)
+ ⋮
+ ([0.9291127454088746, 0.018179125358629283, 0.22262388947635725], 1751.93934482972)
+ ([0.9291905933031118, 0.01816575364337781, 0.22264320511885913], 1751.9393444890322)
+ ([0.9292542992836834, 0.018209269923966702, 0.22262081403728648], 1751.939345033819)
+ ([0.9291892253327142, 0.018129789358969268, 0.22257323673727916], 1751.9393475284328)
+ ([0.9292535836333305, 0.018167625568569275, 0.22264990501164184], 1751.939344490385)
+ ([0.9292145039962464, 0.018171738990225547, 0.22264674302334211], 1751.9393444744132)
+ ([0.9292083869346267, 0.01817147759030398, 0.22264619530985152], 1751.9393444750235)
+ ([0.9292092974286392, 0.018172966192038814, 0.22265206228222742], 1751.939344506555)
+ ([0.9292213162629417, 0.018168381188609965, 0.22264487194620683], 1751.9393444646903)
+
+
+

There were 57 evaluations of the objective before convergence was declared, according to rather stringent convergence criteria. We can see that the last 10 evaluations only produced changes in the fourth decimal place of the objective or even smaller. That is, effective convergence occurred after about 40 or 45 evaluations.

+
+
+

1 A model that converges to a degenerate distribution

+

When the iterations converge to a value of \(\theta\) that is on the boundary, which means that one of the diagonal elements of \(\lambda\) is exactly zero, we say that \(\lambda\) is singular, in the sense that its inverse does not exist.

+

Another, more evocative characterization, is that the distribution of the random effects is a degenerate distribution, in the sense that the values of the random effects are constrained to a plane (technically, a linear subspace) in the space of all possible random effects.

+

This is shown in section 3.5.3 of Embrace Uncertainty, especially Figure 3.13.

+
+
+

2 Appendix

+

These are some appendices, mostly so that I can keep track of them

+
+

2.1 Evaluating the random effects correlation from \(\theta\)

+

There is a short-cut for evaluating the correlation which is to “normalize” the second row of \(\lambda\), in the sense that the row is scaled so that it has unit length.

+
+
normed = normalize!(λ[2, :])
+
+
2-element Vector{Float64}:
+ 0.08133216865276136
+ 0.996687051356763
+
+
+

providing the correlation as

+
+
first(normed)
+
+
0.08133216865276136
+
+
+
+
+

2.2 Optimizing with a fixed correlation

+

To profile the correlation we need optimize the objective while fixing a value of the correlation. The way we will do this is to determine \(\theta_2\) as a function of the fixed \(\rho\) and \(\theta_3\).

+

We need to solve \[ +\rho = \frac{\theta_2}{\sqrt{\theta_2^2 + \theta_3^2}} +\]

+

for \(\theta_2\) as a function of \(\rho\) and \(\theta_3\).

+

Notice that \(\theta_2\) and \(\rho\) have the same sign. Thus it is sufficient to determine the absolute value of \(\theta_2\) then transfer the sign from \(\rho\). \[ +\theta_2^2=\rho^2(\theta_2^2 + \theta_3^2) +\] which implies \[ +\theta_2^2 = \frac{\rho^2}{1-\rho^2}\theta_3^2, \quad \theta_3\ge 0 +\] and thus \[ +\theta_2=\frac{\rho}{\sqrt{1-\rho^2}}\theta_3 +\]

+ + +
+
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/site_libs/bootstrap/bootstrap-dark.min.css b/site_libs/bootstrap/bootstrap-dark.min.css new file mode 100644 index 0000000..ccb52e3 --- /dev/null +++ b/site_libs/bootstrap/bootstrap-dark.min.css @@ -0,0 +1,12 @@ +/*! + * Bootstrap v5.3.1 (https://getbootstrap.com/) + * Copyright 2011-2023 The Bootstrap Authors + * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) + */@import"https://fonts.googleapis.com/css2?family=Lato:ital,wght@0,400;0,700;1,400&display=swap";:root,[data-bs-theme=light]{--bs-blue: #375a7f;--bs-indigo: #6610f2;--bs-purple: #6f42c1;--bs-pink: #e83e8c;--bs-red: #e74c3c;--bs-orange: #fd7e14;--bs-yellow: #f39c12;--bs-green: #00bc8c;--bs-teal: #20c997;--bs-cyan: #3498db;--bs-black: #000;--bs-white: #fff;--bs-gray: #6c757d;--bs-gray-dark: #343a40;--bs-gray-100: #f8f9fa;--bs-gray-200: #ebebeb;--bs-gray-300: #dee2e6;--bs-gray-400: #ced4da;--bs-gray-500: #adb5bd;--bs-gray-600: #6c757d;--bs-gray-700: #444;--bs-gray-800: #343a40;--bs-gray-900: #222;--bs-default: #434343;--bs-primary: #375a7f;--bs-secondary: #434343;--bs-success: #00bc8c;--bs-info: #3498db;--bs-warning: #f39c12;--bs-danger: #e74c3c;--bs-light: #6f6f6f;--bs-dark: #2d2d2d;--bs-default-rgb: 67, 67, 67;--bs-primary-rgb: 55, 90, 127;--bs-secondary-rgb: 67, 67, 67;--bs-success-rgb: 0, 188, 140;--bs-info-rgb: 52, 152, 219;--bs-warning-rgb: 243, 156, 18;--bs-danger-rgb: 231, 76, 60;--bs-light-rgb: 111, 111, 111;--bs-dark-rgb: 45, 45, 45;--bs-primary-text-emphasis: #162433;--bs-secondary-text-emphasis: #1b1b1b;--bs-success-text-emphasis: #004b38;--bs-info-text-emphasis: #153d58;--bs-warning-text-emphasis: #613e07;--bs-danger-text-emphasis: #5c1e18;--bs-light-text-emphasis: #444;--bs-dark-text-emphasis: #444;--bs-primary-bg-subtle: #d7dee5;--bs-secondary-bg-subtle: #d9d9d9;--bs-success-bg-subtle: #ccf2e8;--bs-info-bg-subtle: #d6eaf8;--bs-warning-bg-subtle: #fdebd0;--bs-danger-bg-subtle: #fadbd8;--bs-light-bg-subtle: #fcfcfd;--bs-dark-bg-subtle: #ced4da;--bs-primary-border-subtle: #afbdcc;--bs-secondary-border-subtle: #b4b4b4;--bs-success-border-subtle: #99e4d1;--bs-info-border-subtle: #aed6f1;--bs-warning-border-subtle: #fad7a0;--bs-danger-border-subtle: #f5b7b1;--bs-light-border-subtle: #ebebeb;--bs-dark-border-subtle: #adb5bd;--bs-white-rgb: 255, 255, 255;--bs-black-rgb: 0, 0, 0;--bs-font-sans-serif: Lato, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";--bs-font-monospace: SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace;--bs-gradient: linear-gradient(180deg, rgba(255, 255, 255, 0.15), rgba(255, 255, 255, 0));--bs-root-font-size: 17px;--bs-body-font-family: Lato, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";--bs-body-font-size:1rem;--bs-body-font-weight: 400;--bs-body-line-height: 1.5;--bs-body-color: #fff;--bs-body-color-rgb: 255, 255, 255;--bs-body-bg: #222;--bs-body-bg-rgb: 34, 34, 34;--bs-emphasis-color: #000;--bs-emphasis-color-rgb: 0, 0, 0;--bs-secondary-color: rgba(255, 255, 255, 0.75);--bs-secondary-color-rgb: 255, 255, 255;--bs-secondary-bg: #ebebeb;--bs-secondary-bg-rgb: 235, 235, 235;--bs-tertiary-color: rgba(255, 255, 255, 0.5);--bs-tertiary-color-rgb: 255, 255, 255;--bs-tertiary-bg: #f8f9fa;--bs-tertiary-bg-rgb: 248, 249, 250;--bs-heading-color: inherit;--bs-link-color: #00bc8c;--bs-link-color-rgb: 0, 188, 140;--bs-link-decoration: underline;--bs-link-hover-color: #009670;--bs-link-hover-color-rgb: 0, 150, 112;--bs-code-color: #7d12ba;--bs-highlight-bg: #fdebd0;--bs-border-width: 1px;--bs-border-style: solid;--bs-border-color: #dee2e6;--bs-border-color-translucent: rgba(0, 0, 0, 0.175);--bs-border-radius: 0.25rem;--bs-border-radius-sm: 0.2em;--bs-border-radius-lg: 0.5rem;--bs-border-radius-xl: 1rem;--bs-border-radius-xxl: 2rem;--bs-border-radius-2xl: var(--bs-border-radius-xxl);--bs-border-radius-pill: 50rem;--bs-box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);--bs-box-shadow-sm: 0 0.125rem 0.25rem rgba(0, 0, 0, 0.075);--bs-box-shadow-lg: 0 1rem 3rem rgba(0, 0, 0, 0.175);--bs-box-shadow-inset: inset 0 1px 2px rgba(0, 0, 0, 0.075);--bs-focus-ring-width: 0.25rem;--bs-focus-ring-opacity: 0.25;--bs-focus-ring-color: rgba(55, 90, 127, 0.25);--bs-form-valid-color: #00bc8c;--bs-form-valid-border-color: #00bc8c;--bs-form-invalid-color: #e74c3c;--bs-form-invalid-border-color: #e74c3c}[data-bs-theme=dark]{color-scheme:dark;--bs-body-color: #dee2e6;--bs-body-color-rgb: 222, 226, 230;--bs-body-bg: #222;--bs-body-bg-rgb: 34, 34, 34;--bs-emphasis-color: #fff;--bs-emphasis-color-rgb: 255, 255, 255;--bs-secondary-color: rgba(222, 226, 230, 0.75);--bs-secondary-color-rgb: 222, 226, 230;--bs-secondary-bg: #343a40;--bs-secondary-bg-rgb: 52, 58, 64;--bs-tertiary-color: rgba(222, 226, 230, 0.5);--bs-tertiary-color-rgb: 222, 226, 230;--bs-tertiary-bg: #2b2e31;--bs-tertiary-bg-rgb: 43, 46, 49;--bs-primary-text-emphasis: #879cb2;--bs-secondary-text-emphasis: #8e8e8e;--bs-success-text-emphasis: #66d7ba;--bs-info-text-emphasis: #85c1e9;--bs-warning-text-emphasis: #f8c471;--bs-danger-text-emphasis: #f1948a;--bs-light-text-emphasis: #f8f9fa;--bs-dark-text-emphasis: #dee2e6;--bs-primary-bg-subtle: #0b1219;--bs-secondary-bg-subtle: #0d0d0d;--bs-success-bg-subtle: #00261c;--bs-info-bg-subtle: #0a1e2c;--bs-warning-bg-subtle: #311f04;--bs-danger-bg-subtle: #2e0f0c;--bs-light-bg-subtle: #343a40;--bs-dark-bg-subtle: #1a1d20;--bs-primary-border-subtle: #21364c;--bs-secondary-border-subtle: #282828;--bs-success-border-subtle: #007154;--bs-info-border-subtle: #1f5b83;--bs-warning-border-subtle: #925e0b;--bs-danger-border-subtle: #8b2e24;--bs-light-border-subtle: #444;--bs-dark-border-subtle: #343a40;--bs-heading-color: inherit;--bs-link-color: #879cb2;--bs-link-hover-color: #9fb0c1;--bs-link-color-rgb: 135, 156, 178;--bs-link-hover-color-rgb: 159, 176, 193;--bs-code-color: white;--bs-border-color: #444;--bs-border-color-translucent: rgba(255, 255, 255, 0.15);--bs-form-valid-color: #66d7ba;--bs-form-valid-border-color: #66d7ba;--bs-form-invalid-color: #f1948a;--bs-form-invalid-border-color: #f1948a}*,*::before,*::after{box-sizing:border-box}:root{font-size:var(--bs-root-font-size)}body{margin:0;font-family:var(--bs-body-font-family);font-size:var(--bs-body-font-size);font-weight:var(--bs-body-font-weight);line-height:var(--bs-body-line-height);color:var(--bs-body-color);text-align:var(--bs-body-text-align);background-color:var(--bs-body-bg);-webkit-text-size-adjust:100%;-webkit-tap-highlight-color:rgba(0,0,0,0)}hr{margin:1rem 0;color:inherit;border:0;border-top:1px solid;opacity:.25}h6,.h6,h5,.h5,h4,.h4,h3,.h3,h2,.h2,h1,.h1{margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2;color:var(--bs-heading-color)}h1,.h1{font-size:calc(1.325rem + 0.9vw)}@media(min-width: 1200px){h1,.h1{font-size:2rem}}h2,.h2{font-size:calc(1.29rem + 0.48vw)}@media(min-width: 1200px){h2,.h2{font-size:1.65rem}}h3,.h3{font-size:calc(1.27rem + 0.24vw)}@media(min-width: 1200px){h3,.h3{font-size:1.45rem}}h4,.h4{font-size:1.25rem}h5,.h5{font-size:1.1rem}h6,.h6{font-size:1rem}p{margin-top:0;margin-bottom:1rem}abbr[title]{text-decoration:underline dotted;-webkit-text-decoration:underline dotted;-moz-text-decoration:underline dotted;-ms-text-decoration:underline dotted;-o-text-decoration:underline dotted;cursor:help;text-decoration-skip-ink:none}address{margin-bottom:1rem;font-style:normal;line-height:inherit}ol,ul{padding-left:2rem}ol,ul,dl{margin-top:0;margin-bottom:1rem}ol ol,ul ul,ol ul,ul ol{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem;padding:.625rem 1.25rem;border-left:.25rem solid #ebebeb}blockquote p:last-child,blockquote ul:last-child,blockquote ol:last-child{margin-bottom:0}b,strong{font-weight:bolder}small,.small{font-size:0.875em}mark,.mark{padding:.1875em;background-color:var(--bs-highlight-bg)}sub,sup{position:relative;font-size:0.75em;line-height:0;vertical-align:baseline}sub{bottom:-0.25em}sup{top:-0.5em}a{color:rgba(var(--bs-link-color-rgb), var(--bs-link-opacity, 1));text-decoration:underline;-webkit-text-decoration:underline;-moz-text-decoration:underline;-ms-text-decoration:underline;-o-text-decoration:underline}a:hover{--bs-link-color-rgb: var(--bs-link-hover-color-rgb)}a:not([href]):not([class]),a:not([href]):not([class]):hover{color:inherit;text-decoration:none}pre,code,kbd,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em}pre{display:block;margin-top:0;margin-bottom:1rem;overflow:auto;font-size:0.875em;color:inherit;background-color:#000;padding:.5rem;border:1px solid var(--bs-border-color, #dee2e6);border-radius:.25rem}pre code{background-color:rgba(0,0,0,0);font-size:inherit;color:inherit;word-break:normal}code{font-size:0.875em;color:var(--bs-code-color);background-color:#000;border-radius:.25rem;padding:.125rem .25rem;word-wrap:break-word}a>code{color:inherit}kbd{padding:.4rem .4rem;font-size:0.875em;color:#222;background-color:#fff;border-radius:.2em}kbd kbd{padding:0;font-size:1em}figure{margin:0 0 1rem}img,svg{vertical-align:middle}table{caption-side:bottom;border-collapse:collapse}caption{padding-top:.5rem;padding-bottom:.5rem;color:rgba(255,255,255,.75);text-align:left}th{text-align:inherit;text-align:-webkit-match-parent}thead,tbody,tfoot,tr,td,th{border-color:inherit;border-style:solid;border-width:0}label{display:inline-block}button{border-radius:0}button:focus:not(:focus-visible){outline:0}input,button,select,optgroup,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,select{text-transform:none}[role=button]{cursor:pointer}select{word-wrap:normal}select:disabled{opacity:1}[list]:not([type=date]):not([type=datetime-local]):not([type=month]):not([type=week]):not([type=time])::-webkit-calendar-picker-indicator{display:none !important}button,[type=button],[type=reset],[type=submit]{-webkit-appearance:button}button:not(:disabled),[type=button]:not(:disabled),[type=reset]:not(:disabled),[type=submit]:not(:disabled){cursor:pointer}::-moz-focus-inner{padding:0;border-style:none}textarea{resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{float:left;width:100%;padding:0;margin-bottom:.5rem;font-size:calc(1.275rem + 0.3vw);line-height:inherit}@media(min-width: 1200px){legend{font-size:1.5rem}}legend+*{clear:left}::-webkit-datetime-edit-fields-wrapper,::-webkit-datetime-edit-text,::-webkit-datetime-edit-minute,::-webkit-datetime-edit-hour-field,::-webkit-datetime-edit-day-field,::-webkit-datetime-edit-month-field,::-webkit-datetime-edit-year-field{padding:0}::-webkit-inner-spin-button{height:auto}[type=search]{-webkit-appearance:textfield;outline-offset:-2px}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-color-swatch-wrapper{padding:0}::file-selector-button{font:inherit;-webkit-appearance:button}output{display:inline-block}iframe{border:0}summary{display:list-item;cursor:pointer}progress{vertical-align:baseline}[hidden]{display:none !important}.lead{font-size:1.25rem;font-weight:300}.display-1{font-size:calc(1.625rem + 4.5vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-1{font-size:5rem}}.display-2{font-size:calc(1.575rem + 3.9vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-2{font-size:4.5rem}}.display-3{font-size:calc(1.525rem + 3.3vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-3{font-size:4rem}}.display-4{font-size:calc(1.475rem + 2.7vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-4{font-size:3.5rem}}.display-5{font-size:calc(1.425rem + 2.1vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-5{font-size:3rem}}.display-6{font-size:calc(1.375rem + 1.5vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-6{font-size:2.5rem}}.list-unstyled{padding-left:0;list-style:none}.list-inline{padding-left:0;list-style:none}.list-inline-item{display:inline-block}.list-inline-item:not(:last-child){margin-right:.5rem}.initialism{font-size:0.875em;text-transform:uppercase}.blockquote{margin-bottom:1rem;font-size:1.25rem}.blockquote>:last-child{margin-bottom:0}.blockquote-footer{margin-top:-1rem;margin-bottom:1rem;font-size:0.875em;color:#6c757d}.blockquote-footer::before{content:"— "}.img-fluid{max-width:100%;height:auto}.img-thumbnail{padding:.25rem;background-color:#222;border:1px solid #dee2e6;border-radius:.25rem;max-width:100%;height:auto}.figure{display:inline-block}.figure-img{margin-bottom:.5rem;line-height:1}.figure-caption{font-size:0.875em;color:rgba(255,255,255,.75)}.container,.container-fluid,.container-xxl,.container-xl,.container-lg,.container-md,.container-sm{--bs-gutter-x: 1.5rem;--bs-gutter-y: 0;width:100%;padding-right:calc(var(--bs-gutter-x)*.5);padding-left:calc(var(--bs-gutter-x)*.5);margin-right:auto;margin-left:auto}@media(min-width: 576px){.container-sm,.container{max-width:540px}}@media(min-width: 768px){.container-md,.container-sm,.container{max-width:720px}}@media(min-width: 992px){.container-lg,.container-md,.container-sm,.container{max-width:960px}}@media(min-width: 1200px){.container-xl,.container-lg,.container-md,.container-sm,.container{max-width:1140px}}@media(min-width: 1400px){.container-xxl,.container-xl,.container-lg,.container-md,.container-sm,.container{max-width:1320px}}:root{--bs-breakpoint-xs: 0;--bs-breakpoint-sm: 576px;--bs-breakpoint-md: 768px;--bs-breakpoint-lg: 992px;--bs-breakpoint-xl: 1200px;--bs-breakpoint-xxl: 1400px}.grid{display:grid;grid-template-rows:repeat(var(--bs-rows, 1), 1fr);grid-template-columns:repeat(var(--bs-columns, 12), 1fr);gap:var(--bs-gap, 1.5rem)}.grid .g-col-1{grid-column:auto/span 1}.grid .g-col-2{grid-column:auto/span 2}.grid .g-col-3{grid-column:auto/span 3}.grid .g-col-4{grid-column:auto/span 4}.grid .g-col-5{grid-column:auto/span 5}.grid .g-col-6{grid-column:auto/span 6}.grid .g-col-7{grid-column:auto/span 7}.grid .g-col-8{grid-column:auto/span 8}.grid .g-col-9{grid-column:auto/span 9}.grid .g-col-10{grid-column:auto/span 10}.grid .g-col-11{grid-column:auto/span 11}.grid .g-col-12{grid-column:auto/span 12}.grid .g-start-1{grid-column-start:1}.grid .g-start-2{grid-column-start:2}.grid .g-start-3{grid-column-start:3}.grid .g-start-4{grid-column-start:4}.grid .g-start-5{grid-column-start:5}.grid .g-start-6{grid-column-start:6}.grid .g-start-7{grid-column-start:7}.grid .g-start-8{grid-column-start:8}.grid .g-start-9{grid-column-start:9}.grid .g-start-10{grid-column-start:10}.grid .g-start-11{grid-column-start:11}@media(min-width: 576px){.grid .g-col-sm-1{grid-column:auto/span 1}.grid .g-col-sm-2{grid-column:auto/span 2}.grid .g-col-sm-3{grid-column:auto/span 3}.grid .g-col-sm-4{grid-column:auto/span 4}.grid .g-col-sm-5{grid-column:auto/span 5}.grid .g-col-sm-6{grid-column:auto/span 6}.grid .g-col-sm-7{grid-column:auto/span 7}.grid .g-col-sm-8{grid-column:auto/span 8}.grid .g-col-sm-9{grid-column:auto/span 9}.grid .g-col-sm-10{grid-column:auto/span 10}.grid .g-col-sm-11{grid-column:auto/span 11}.grid .g-col-sm-12{grid-column:auto/span 12}.grid .g-start-sm-1{grid-column-start:1}.grid .g-start-sm-2{grid-column-start:2}.grid .g-start-sm-3{grid-column-start:3}.grid .g-start-sm-4{grid-column-start:4}.grid .g-start-sm-5{grid-column-start:5}.grid .g-start-sm-6{grid-column-start:6}.grid .g-start-sm-7{grid-column-start:7}.grid .g-start-sm-8{grid-column-start:8}.grid .g-start-sm-9{grid-column-start:9}.grid .g-start-sm-10{grid-column-start:10}.grid .g-start-sm-11{grid-column-start:11}}@media(min-width: 768px){.grid .g-col-md-1{grid-column:auto/span 1}.grid .g-col-md-2{grid-column:auto/span 2}.grid .g-col-md-3{grid-column:auto/span 3}.grid .g-col-md-4{grid-column:auto/span 4}.grid .g-col-md-5{grid-column:auto/span 5}.grid .g-col-md-6{grid-column:auto/span 6}.grid .g-col-md-7{grid-column:auto/span 7}.grid .g-col-md-8{grid-column:auto/span 8}.grid .g-col-md-9{grid-column:auto/span 9}.grid .g-col-md-10{grid-column:auto/span 10}.grid .g-col-md-11{grid-column:auto/span 11}.grid .g-col-md-12{grid-column:auto/span 12}.grid .g-start-md-1{grid-column-start:1}.grid .g-start-md-2{grid-column-start:2}.grid .g-start-md-3{grid-column-start:3}.grid .g-start-md-4{grid-column-start:4}.grid .g-start-md-5{grid-column-start:5}.grid .g-start-md-6{grid-column-start:6}.grid .g-start-md-7{grid-column-start:7}.grid .g-start-md-8{grid-column-start:8}.grid .g-start-md-9{grid-column-start:9}.grid .g-start-md-10{grid-column-start:10}.grid .g-start-md-11{grid-column-start:11}}@media(min-width: 992px){.grid .g-col-lg-1{grid-column:auto/span 1}.grid .g-col-lg-2{grid-column:auto/span 2}.grid .g-col-lg-3{grid-column:auto/span 3}.grid .g-col-lg-4{grid-column:auto/span 4}.grid .g-col-lg-5{grid-column:auto/span 5}.grid .g-col-lg-6{grid-column:auto/span 6}.grid .g-col-lg-7{grid-column:auto/span 7}.grid .g-col-lg-8{grid-column:auto/span 8}.grid .g-col-lg-9{grid-column:auto/span 9}.grid .g-col-lg-10{grid-column:auto/span 10}.grid .g-col-lg-11{grid-column:auto/span 11}.grid .g-col-lg-12{grid-column:auto/span 12}.grid .g-start-lg-1{grid-column-start:1}.grid .g-start-lg-2{grid-column-start:2}.grid .g-start-lg-3{grid-column-start:3}.grid .g-start-lg-4{grid-column-start:4}.grid .g-start-lg-5{grid-column-start:5}.grid .g-start-lg-6{grid-column-start:6}.grid .g-start-lg-7{grid-column-start:7}.grid .g-start-lg-8{grid-column-start:8}.grid .g-start-lg-9{grid-column-start:9}.grid .g-start-lg-10{grid-column-start:10}.grid .g-start-lg-11{grid-column-start:11}}@media(min-width: 1200px){.grid .g-col-xl-1{grid-column:auto/span 1}.grid .g-col-xl-2{grid-column:auto/span 2}.grid .g-col-xl-3{grid-column:auto/span 3}.grid .g-col-xl-4{grid-column:auto/span 4}.grid .g-col-xl-5{grid-column:auto/span 5}.grid .g-col-xl-6{grid-column:auto/span 6}.grid .g-col-xl-7{grid-column:auto/span 7}.grid .g-col-xl-8{grid-column:auto/span 8}.grid .g-col-xl-9{grid-column:auto/span 9}.grid .g-col-xl-10{grid-column:auto/span 10}.grid .g-col-xl-11{grid-column:auto/span 11}.grid .g-col-xl-12{grid-column:auto/span 12}.grid .g-start-xl-1{grid-column-start:1}.grid .g-start-xl-2{grid-column-start:2}.grid .g-start-xl-3{grid-column-start:3}.grid .g-start-xl-4{grid-column-start:4}.grid .g-start-xl-5{grid-column-start:5}.grid .g-start-xl-6{grid-column-start:6}.grid .g-start-xl-7{grid-column-start:7}.grid .g-start-xl-8{grid-column-start:8}.grid .g-start-xl-9{grid-column-start:9}.grid .g-start-xl-10{grid-column-start:10}.grid .g-start-xl-11{grid-column-start:11}}@media(min-width: 1400px){.grid .g-col-xxl-1{grid-column:auto/span 1}.grid .g-col-xxl-2{grid-column:auto/span 2}.grid .g-col-xxl-3{grid-column:auto/span 3}.grid .g-col-xxl-4{grid-column:auto/span 4}.grid .g-col-xxl-5{grid-column:auto/span 5}.grid .g-col-xxl-6{grid-column:auto/span 6}.grid .g-col-xxl-7{grid-column:auto/span 7}.grid .g-col-xxl-8{grid-column:auto/span 8}.grid .g-col-xxl-9{grid-column:auto/span 9}.grid .g-col-xxl-10{grid-column:auto/span 10}.grid .g-col-xxl-11{grid-column:auto/span 11}.grid .g-col-xxl-12{grid-column:auto/span 12}.grid .g-start-xxl-1{grid-column-start:1}.grid .g-start-xxl-2{grid-column-start:2}.grid .g-start-xxl-3{grid-column-start:3}.grid .g-start-xxl-4{grid-column-start:4}.grid .g-start-xxl-5{grid-column-start:5}.grid .g-start-xxl-6{grid-column-start:6}.grid .g-start-xxl-7{grid-column-start:7}.grid .g-start-xxl-8{grid-column-start:8}.grid .g-start-xxl-9{grid-column-start:9}.grid .g-start-xxl-10{grid-column-start:10}.grid .g-start-xxl-11{grid-column-start:11}}.table{--bs-table-color-type: initial;--bs-table-bg-type: initial;--bs-table-color-state: initial;--bs-table-bg-state: initial;--bs-table-color: #fff;--bs-table-bg: #222;--bs-table-border-color: #434343;--bs-table-accent-bg: transparent;--bs-table-striped-color: #fff;--bs-table-striped-bg: rgba(0, 0, 0, 0.05);--bs-table-active-color: #fff;--bs-table-active-bg: rgba(0, 0, 0, 0.1);--bs-table-hover-color: #fff;--bs-table-hover-bg: rgba(0, 0, 0, 0.075);width:100%;margin-bottom:1rem;vertical-align:top;border-color:var(--bs-table-border-color)}.table>:not(caption)>*>*{padding:.5rem .5rem;color:var(--bs-table-color-state, var(--bs-table-color-type, var(--bs-table-color)));background-color:var(--bs-table-bg);border-bottom-width:1px;box-shadow:inset 0 0 0 9999px var(--bs-table-bg-state, var(--bs-table-bg-type, var(--bs-table-accent-bg)))}.table>tbody{vertical-align:inherit}.table>thead{vertical-align:bottom}.table-group-divider{border-top:calc(1px*2) solid #fff}.caption-top{caption-side:top}.table-sm>:not(caption)>*>*{padding:.25rem .25rem}.table-bordered>:not(caption)>*{border-width:1px 0}.table-bordered>:not(caption)>*>*{border-width:0 1px}.table-borderless>:not(caption)>*>*{border-bottom-width:0}.table-borderless>:not(:first-child){border-top-width:0}.table-striped>tbody>tr:nth-of-type(odd)>*{--bs-table-color-type: var(--bs-table-striped-color);--bs-table-bg-type: var(--bs-table-striped-bg)}.table-striped-columns>:not(caption)>tr>:nth-child(even){--bs-table-color-type: var(--bs-table-striped-color);--bs-table-bg-type: var(--bs-table-striped-bg)}.table-active{--bs-table-color-state: var(--bs-table-active-color);--bs-table-bg-state: var(--bs-table-active-bg)}.table-hover>tbody>tr:hover>*{--bs-table-color-state: var(--bs-table-hover-color);--bs-table-bg-state: var(--bs-table-hover-bg)}.table-primary{--bs-table-color: #fff;--bs-table-bg: #375a7f;--bs-table-border-color: #4b6b8c;--bs-table-striped-bg: #416285;--bs-table-striped-color: #fff;--bs-table-active-bg: #4b6b8c;--bs-table-active-color: #fff;--bs-table-hover-bg: #466689;--bs-table-hover-color: #fff;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-secondary{--bs-table-color: #fff;--bs-table-bg: #434343;--bs-table-border-color: #565656;--bs-table-striped-bg: #4c4c4c;--bs-table-striped-color: #fff;--bs-table-active-bg: #565656;--bs-table-active-color: #fff;--bs-table-hover-bg: #515151;--bs-table-hover-color: #fff;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-success{--bs-table-color: #fff;--bs-table-bg: #00bc8c;--bs-table-border-color: #1ac398;--bs-table-striped-bg: #0dbf92;--bs-table-striped-color: #fff;--bs-table-active-bg: #1ac398;--bs-table-active-color: #fff;--bs-table-hover-bg: #13c195;--bs-table-hover-color: #fff;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-info{--bs-table-color: #fff;--bs-table-bg: #3498db;--bs-table-border-color: #48a2df;--bs-table-striped-bg: #3e9ddd;--bs-table-striped-color: #fff;--bs-table-active-bg: #48a2df;--bs-table-active-color: #fff;--bs-table-hover-bg: #43a0de;--bs-table-hover-color: #fff;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-warning{--bs-table-color: #fff;--bs-table-bg: #f39c12;--bs-table-border-color: #f4a62a;--bs-table-striped-bg: #f4a11e;--bs-table-striped-color: #fff;--bs-table-active-bg: #f4a62a;--bs-table-active-color: #fff;--bs-table-hover-bg: #f4a324;--bs-table-hover-color: #fff;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-danger{--bs-table-color: #fff;--bs-table-bg: #e74c3c;--bs-table-border-color: #e95e50;--bs-table-striped-bg: #e85546;--bs-table-striped-color: #fff;--bs-table-active-bg: #e95e50;--bs-table-active-color: #fff;--bs-table-hover-bg: #e9594b;--bs-table-hover-color: #fff;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-light{--bs-table-color: #fff;--bs-table-bg: #6f6f6f;--bs-table-border-color: #7d7d7d;--bs-table-striped-bg: #767676;--bs-table-striped-color: #fff;--bs-table-active-bg: #7d7d7d;--bs-table-active-color: #fff;--bs-table-hover-bg: #7a7a7a;--bs-table-hover-color: #fff;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-dark{--bs-table-color: #fff;--bs-table-bg: #2d2d2d;--bs-table-border-color: #424242;--bs-table-striped-bg: #383838;--bs-table-striped-color: #fff;--bs-table-active-bg: #424242;--bs-table-active-color: #fff;--bs-table-hover-bg: #3d3d3d;--bs-table-hover-color: #fff;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-responsive{overflow-x:auto;-webkit-overflow-scrolling:touch}@media(max-width: 575.98px){.table-responsive-sm{overflow-x:auto;-webkit-overflow-scrolling:touch}}@media(max-width: 767.98px){.table-responsive-md{overflow-x:auto;-webkit-overflow-scrolling:touch}}@media(max-width: 991.98px){.table-responsive-lg{overflow-x:auto;-webkit-overflow-scrolling:touch}}@media(max-width: 1199.98px){.table-responsive-xl{overflow-x:auto;-webkit-overflow-scrolling:touch}}@media(max-width: 1399.98px){.table-responsive-xxl{overflow-x:auto;-webkit-overflow-scrolling:touch}}.form-label,.shiny-input-container .control-label{margin-bottom:.5rem}.col-form-label{padding-top:calc(0.375rem + 1px);padding-bottom:calc(0.375rem + 1px);margin-bottom:0;font-size:inherit;line-height:1.5}.col-form-label-lg{padding-top:calc(0.5rem + 1px);padding-bottom:calc(0.5rem + 1px);font-size:1.25rem}.col-form-label-sm{padding-top:calc(0.25rem + 1px);padding-bottom:calc(0.25rem + 1px);font-size:0.875rem}.form-text{margin-top:.25rem;font-size:0.875em;color:rgba(255,255,255,.75)}.form-control{display:block;width:100%;padding:.375rem .75rem;font-size:1rem;font-weight:400;line-height:1.5;color:#2d2d2d;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:#fff;background-clip:padding-box;border:1px solid #adb5bd;border-radius:.25rem;transition:border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-control{transition:none}}.form-control[type=file]{overflow:hidden}.form-control[type=file]:not(:disabled):not([readonly]){cursor:pointer}.form-control:focus{color:#2d2d2d;background-color:#fff;border-color:#9badbf;outline:0;box-shadow:0 0 0 .25rem rgba(55,90,127,.25)}.form-control::-webkit-date-and-time-value{min-width:85px;height:1.5em;margin:0}.form-control::-webkit-datetime-edit{display:block;padding:0}.form-control::placeholder{color:#595959;opacity:1}.form-control:disabled{background-color:#ebebeb;opacity:1}.form-control::file-selector-button{padding:.375rem .75rem;margin:-0.375rem -0.75rem;margin-inline-end:.75rem;color:#6f6f6f;background-color:#434343;pointer-events:none;border-color:inherit;border-style:solid;border-width:0;border-inline-end-width:1px;border-radius:0;transition:color .15s ease-in-out,background-color .15s ease-in-out,border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-control::file-selector-button{transition:none}}.form-control:hover:not(:disabled):not([readonly])::file-selector-button{background-color:#363636}.form-control-plaintext{display:block;width:100%;padding:.375rem 0;margin-bottom:0;line-height:1.5;color:#fff;background-color:rgba(0,0,0,0);border:solid rgba(0,0,0,0);border-width:1px 0}.form-control-plaintext:focus{outline:0}.form-control-plaintext.form-control-sm,.form-control-plaintext.form-control-lg{padding-right:0;padding-left:0}.form-control-sm{min-height:calc(1.5em + 0.5rem + calc(1px * 2));padding:.25rem .5rem;font-size:0.875rem;border-radius:.2em}.form-control-sm::file-selector-button{padding:.25rem .5rem;margin:-0.25rem -0.5rem;margin-inline-end:.5rem}.form-control-lg{min-height:calc(1.5em + 1rem + calc(1px * 2));padding:.5rem 1rem;font-size:1.25rem;border-radius:.5rem}.form-control-lg::file-selector-button{padding:.5rem 1rem;margin:-0.5rem -1rem;margin-inline-end:1rem}textarea.form-control{min-height:calc(1.5em + 0.75rem + calc(1px * 2))}textarea.form-control-sm{min-height:calc(1.5em + 0.5rem + calc(1px * 2))}textarea.form-control-lg{min-height:calc(1.5em + 1rem + calc(1px * 2))}.form-control-color{width:3rem;height:calc(1.5em + 0.75rem + calc(1px * 2));padding:.375rem}.form-control-color:not(:disabled):not([readonly]){cursor:pointer}.form-control-color::-moz-color-swatch{border:0 !important;border-radius:.25rem}.form-control-color::-webkit-color-swatch{border:0 !important;border-radius:.25rem}.form-control-color.form-control-sm{height:calc(1.5em + 0.5rem + calc(1px * 2))}.form-control-color.form-control-lg{height:calc(1.5em + 1rem + calc(1px * 2))}.form-select{--bs-form-select-bg-img: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16'%3e%3cpath fill='none' stroke='%23343a40' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' d='m2 5 6 6 6-6'/%3e%3c/svg%3e");display:block;width:100%;padding:.375rem 2.25rem .375rem .75rem;font-size:1rem;font-weight:400;line-height:1.5;color:#2d2d2d;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:#fff;background-image:var(--bs-form-select-bg-img),var(--bs-form-select-bg-icon, none);background-repeat:no-repeat;background-position:right .75rem center;background-size:16px 12px;border:1px solid #adb5bd;border-radius:.25rem;transition:border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-select{transition:none}}.form-select:focus{border-color:#9badbf;outline:0;box-shadow:0 0 0 .25rem rgba(55,90,127,.25)}.form-select[multiple],.form-select[size]:not([size="1"]){padding-right:.75rem;background-image:none}.form-select:disabled{color:#595959;background-color:#ebebeb}.form-select:-moz-focusring{color:rgba(0,0,0,0);text-shadow:0 0 0 #2d2d2d}.form-select-sm{padding-top:.25rem;padding-bottom:.25rem;padding-left:.5rem;font-size:0.875rem;border-radius:.2em}.form-select-lg{padding-top:.5rem;padding-bottom:.5rem;padding-left:1rem;font-size:1.25rem;border-radius:.5rem}[data-bs-theme=dark] .form-select{--bs-form-select-bg-img: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16'%3e%3cpath fill='none' stroke='%23dee2e6' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' d='m2 5 6 6 6-6'/%3e%3c/svg%3e")}.form-check,.shiny-input-container .checkbox,.shiny-input-container .radio{display:block;min-height:1.5rem;padding-left:0;margin-bottom:.125rem}.form-check .form-check-input,.form-check .shiny-input-container .checkbox input,.form-check .shiny-input-container .radio input,.shiny-input-container .checkbox .form-check-input,.shiny-input-container .checkbox .shiny-input-container .checkbox input,.shiny-input-container .checkbox .shiny-input-container .radio input,.shiny-input-container .radio .form-check-input,.shiny-input-container .radio .shiny-input-container .checkbox input,.shiny-input-container .radio .shiny-input-container .radio input{float:left;margin-left:0}.form-check-reverse{padding-right:0;padding-left:0;text-align:right}.form-check-reverse .form-check-input{float:right;margin-right:0;margin-left:0}.form-check-input,.shiny-input-container .checkbox input,.shiny-input-container .checkbox-inline input,.shiny-input-container .radio input,.shiny-input-container .radio-inline input{--bs-form-check-bg: #fff;width:1em;height:1em;margin-top:.25em;vertical-align:top;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:var(--bs-form-check-bg);background-image:var(--bs-form-check-bg-image);background-repeat:no-repeat;background-position:center;background-size:contain;border:none;print-color-adjust:exact}.form-check-input[type=checkbox],.shiny-input-container .checkbox input[type=checkbox],.shiny-input-container .checkbox-inline input[type=checkbox],.shiny-input-container .radio input[type=checkbox],.shiny-input-container .radio-inline input[type=checkbox]{border-radius:.25em}.form-check-input[type=radio],.shiny-input-container .checkbox input[type=radio],.shiny-input-container .checkbox-inline input[type=radio],.shiny-input-container .radio input[type=radio],.shiny-input-container .radio-inline input[type=radio]{border-radius:50%}.form-check-input:active,.shiny-input-container .checkbox input:active,.shiny-input-container .checkbox-inline input:active,.shiny-input-container .radio input:active,.shiny-input-container .radio-inline input:active{filter:brightness(90%)}.form-check-input:focus,.shiny-input-container .checkbox input:focus,.shiny-input-container .checkbox-inline input:focus,.shiny-input-container .radio input:focus,.shiny-input-container .radio-inline input:focus{border-color:#9badbf;outline:0;box-shadow:0 0 0 .25rem rgba(55,90,127,.25)}.form-check-input:checked,.shiny-input-container .checkbox input:checked,.shiny-input-container .checkbox-inline input:checked,.shiny-input-container .radio input:checked,.shiny-input-container .radio-inline input:checked{background-color:#375a7f;border-color:#375a7f}.form-check-input:checked[type=checkbox],.shiny-input-container .checkbox input:checked[type=checkbox],.shiny-input-container .checkbox-inline input:checked[type=checkbox],.shiny-input-container .radio input:checked[type=checkbox],.shiny-input-container .radio-inline input:checked[type=checkbox]{--bs-form-check-bg-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 20 20'%3e%3cpath fill='none' stroke='%23fff' stroke-linecap='round' stroke-linejoin='round' stroke-width='3' d='m6 10 3 3 6-6'/%3e%3c/svg%3e")}.form-check-input:checked[type=radio],.shiny-input-container .checkbox input:checked[type=radio],.shiny-input-container .checkbox-inline input:checked[type=radio],.shiny-input-container .radio input:checked[type=radio],.shiny-input-container .radio-inline input:checked[type=radio]{--bs-form-check-bg-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='2' fill='%23fff'/%3e%3c/svg%3e")}.form-check-input[type=checkbox]:indeterminate,.shiny-input-container .checkbox input[type=checkbox]:indeterminate,.shiny-input-container .checkbox-inline input[type=checkbox]:indeterminate,.shiny-input-container .radio input[type=checkbox]:indeterminate,.shiny-input-container .radio-inline input[type=checkbox]:indeterminate{background-color:#375a7f;border-color:#375a7f;--bs-form-check-bg-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 20 20'%3e%3cpath fill='none' stroke='%23fff' stroke-linecap='round' stroke-linejoin='round' stroke-width='3' d='M6 10h8'/%3e%3c/svg%3e")}.form-check-input:disabled,.shiny-input-container .checkbox input:disabled,.shiny-input-container .checkbox-inline input:disabled,.shiny-input-container .radio input:disabled,.shiny-input-container .radio-inline input:disabled{pointer-events:none;filter:none;opacity:.5}.form-check-input[disabled]~.form-check-label,.form-check-input[disabled]~span,.form-check-input:disabled~.form-check-label,.form-check-input:disabled~span,.shiny-input-container .checkbox input[disabled]~.form-check-label,.shiny-input-container .checkbox input[disabled]~span,.shiny-input-container .checkbox input:disabled~.form-check-label,.shiny-input-container .checkbox input:disabled~span,.shiny-input-container .checkbox-inline input[disabled]~.form-check-label,.shiny-input-container .checkbox-inline input[disabled]~span,.shiny-input-container .checkbox-inline input:disabled~.form-check-label,.shiny-input-container .checkbox-inline input:disabled~span,.shiny-input-container .radio input[disabled]~.form-check-label,.shiny-input-container .radio input[disabled]~span,.shiny-input-container .radio input:disabled~.form-check-label,.shiny-input-container .radio input:disabled~span,.shiny-input-container .radio-inline input[disabled]~.form-check-label,.shiny-input-container .radio-inline input[disabled]~span,.shiny-input-container .radio-inline input:disabled~.form-check-label,.shiny-input-container .radio-inline input:disabled~span{cursor:default;opacity:.5}.form-check-label,.shiny-input-container .checkbox label,.shiny-input-container .checkbox-inline label,.shiny-input-container .radio label,.shiny-input-container .radio-inline label{cursor:pointer}.form-switch{padding-left:2.5em}.form-switch .form-check-input{--bs-form-switch-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='rgba%280, 0, 0, 0.25%29'/%3e%3c/svg%3e");width:2em;margin-left:-2.5em;background-image:var(--bs-form-switch-bg);background-position:left center;border-radius:2em;transition:background-position .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-switch .form-check-input{transition:none}}.form-switch .form-check-input:focus{--bs-form-switch-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='%239badbf'/%3e%3c/svg%3e")}.form-switch .form-check-input:checked{background-position:right center;--bs-form-switch-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='%23fff'/%3e%3c/svg%3e")}.form-switch.form-check-reverse{padding-right:2.5em;padding-left:0}.form-switch.form-check-reverse .form-check-input{margin-right:-2.5em;margin-left:0}.form-check-inline{display:inline-block;margin-right:1rem}.btn-check{position:absolute;clip:rect(0, 0, 0, 0);pointer-events:none}.btn-check[disabled]+.btn,.btn-check:disabled+.btn{pointer-events:none;filter:none;opacity:.65}[data-bs-theme=dark] .form-switch .form-check-input:not(:checked):not(:focus){--bs-form-switch-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='rgba%28255, 255, 255, 0.25%29'/%3e%3c/svg%3e")}.form-range{width:100%;height:1.5rem;padding:0;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:rgba(0,0,0,0)}.form-range:focus{outline:0}.form-range:focus::-webkit-slider-thumb{box-shadow:0 0 0 1px #222,0 0 0 .25rem rgba(55,90,127,.25)}.form-range:focus::-moz-range-thumb{box-shadow:0 0 0 1px #222,0 0 0 .25rem rgba(55,90,127,.25)}.form-range::-moz-focus-outer{border:0}.form-range::-webkit-slider-thumb{width:1rem;height:1rem;margin-top:-0.25rem;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:#375a7f;border:0;border-radius:1rem;transition:background-color .15s ease-in-out,border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-range::-webkit-slider-thumb{transition:none}}.form-range::-webkit-slider-thumb:active{background-color:#c3ced9}.form-range::-webkit-slider-runnable-track{width:100%;height:.5rem;color:rgba(0,0,0,0);cursor:pointer;background-color:#f8f9fa;border-color:rgba(0,0,0,0);border-radius:1rem}.form-range::-moz-range-thumb{width:1rem;height:1rem;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:#375a7f;border:0;border-radius:1rem;transition:background-color .15s ease-in-out,border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-range::-moz-range-thumb{transition:none}}.form-range::-moz-range-thumb:active{background-color:#c3ced9}.form-range::-moz-range-track{width:100%;height:.5rem;color:rgba(0,0,0,0);cursor:pointer;background-color:#f8f9fa;border-color:rgba(0,0,0,0);border-radius:1rem}.form-range:disabled{pointer-events:none}.form-range:disabled::-webkit-slider-thumb{background-color:rgba(255,255,255,.75)}.form-range:disabled::-moz-range-thumb{background-color:rgba(255,255,255,.75)}.form-floating{position:relative}.form-floating>.form-control,.form-floating>.form-control-plaintext,.form-floating>.form-select{height:calc(3.5rem + calc(1px * 2));min-height:calc(3.5rem + calc(1px * 2));line-height:1.25}.form-floating>label{position:absolute;top:0;left:0;z-index:2;height:100%;padding:1rem .75rem;overflow:hidden;text-align:start;text-overflow:ellipsis;white-space:nowrap;pointer-events:none;border:1px solid rgba(0,0,0,0);transform-origin:0 0;transition:opacity .1s ease-in-out,transform .1s ease-in-out}@media(prefers-reduced-motion: reduce){.form-floating>label{transition:none}}.form-floating>.form-control,.form-floating>.form-control-plaintext{padding:1rem .75rem}.form-floating>.form-control::placeholder,.form-floating>.form-control-plaintext::placeholder{color:rgba(0,0,0,0)}.form-floating>.form-control:focus,.form-floating>.form-control:not(:placeholder-shown),.form-floating>.form-control-plaintext:focus,.form-floating>.form-control-plaintext:not(:placeholder-shown){padding-top:1.625rem;padding-bottom:.625rem}.form-floating>.form-control:-webkit-autofill,.form-floating>.form-control-plaintext:-webkit-autofill{padding-top:1.625rem;padding-bottom:.625rem}.form-floating>.form-select{padding-top:1.625rem;padding-bottom:.625rem}.form-floating>.form-control:focus~label,.form-floating>.form-control:not(:placeholder-shown)~label,.form-floating>.form-control-plaintext~label,.form-floating>.form-select~label{color:rgba(var(--bs-body-color-rgb), 0.65);transform:scale(0.85) translateY(-0.5rem) translateX(0.15rem)}.form-floating>.form-control:focus~label::after,.form-floating>.form-control:not(:placeholder-shown)~label::after,.form-floating>.form-control-plaintext~label::after,.form-floating>.form-select~label::after{position:absolute;inset:1rem .375rem;z-index:-1;height:1.5em;content:"";background-color:#fff;border-radius:.25rem}.form-floating>.form-control:-webkit-autofill~label{color:rgba(var(--bs-body-color-rgb), 0.65);transform:scale(0.85) translateY(-0.5rem) translateX(0.15rem)}.form-floating>.form-control-plaintext~label{border-width:1px 0}.form-floating>:disabled~label,.form-floating>.form-control:disabled~label{color:#6c757d}.form-floating>:disabled~label::after,.form-floating>.form-control:disabled~label::after{background-color:#ebebeb}.input-group{position:relative;display:flex;display:-webkit-flex;flex-wrap:wrap;-webkit-flex-wrap:wrap;align-items:stretch;-webkit-align-items:stretch;width:100%}.input-group>.form-control,.input-group>.form-select,.input-group>.form-floating{position:relative;flex:1 1 auto;-webkit-flex:1 1 auto;width:1%;min-width:0}.input-group>.form-control:focus,.input-group>.form-select:focus,.input-group>.form-floating:focus-within{z-index:5}.input-group .btn{position:relative;z-index:2}.input-group .btn:focus{z-index:5}.input-group-text{display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;padding:.375rem .75rem;font-size:1rem;font-weight:400;line-height:1.5;color:#6f6f6f;text-align:center;white-space:nowrap;background-color:#434343;border:1px solid #adb5bd;border-radius:.25rem}.input-group-lg>.form-control,.input-group-lg>.form-select,.input-group-lg>.input-group-text,.input-group-lg>.btn{padding:.5rem 1rem;font-size:1.25rem;border-radius:.5rem}.input-group-sm>.form-control,.input-group-sm>.form-select,.input-group-sm>.input-group-text,.input-group-sm>.btn{padding:.25rem .5rem;font-size:0.875rem;border-radius:.2em}.input-group-lg>.form-select,.input-group-sm>.form-select{padding-right:3rem}.input-group:not(.has-validation)>:not(:last-child):not(.dropdown-toggle):not(.dropdown-menu):not(.form-floating),.input-group:not(.has-validation)>.dropdown-toggle:nth-last-child(n+3),.input-group:not(.has-validation)>.form-floating:not(:last-child)>.form-control,.input-group:not(.has-validation)>.form-floating:not(:last-child)>.form-select{border-top-right-radius:0;border-bottom-right-radius:0}.input-group.has-validation>:nth-last-child(n+3):not(.dropdown-toggle):not(.dropdown-menu):not(.form-floating),.input-group.has-validation>.dropdown-toggle:nth-last-child(n+4),.input-group.has-validation>.form-floating:nth-last-child(n+3)>.form-control,.input-group.has-validation>.form-floating:nth-last-child(n+3)>.form-select{border-top-right-radius:0;border-bottom-right-radius:0}.input-group>:not(:first-child):not(.dropdown-menu):not(.valid-tooltip):not(.valid-feedback):not(.invalid-tooltip):not(.invalid-feedback){margin-left:calc(1px*-1);border-top-left-radius:0;border-bottom-left-radius:0}.input-group>.form-floating:not(:first-child)>.form-control,.input-group>.form-floating:not(:first-child)>.form-select{border-top-left-radius:0;border-bottom-left-radius:0}.valid-feedback{display:none;width:100%;margin-top:.25rem;font-size:0.875em;color:#00bc8c}.valid-tooltip{position:absolute;top:100%;z-index:5;display:none;max-width:100%;padding:.25rem .5rem;margin-top:.1rem;font-size:0.875rem;color:#fff;background-color:#00bc8c;border-radius:.25rem}.was-validated :valid~.valid-feedback,.was-validated :valid~.valid-tooltip,.is-valid~.valid-feedback,.is-valid~.valid-tooltip{display:block}.was-validated .form-control:valid,.form-control.is-valid{border-color:#00bc8c;padding-right:calc(1.5em + 0.75rem);background-image:url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 8 8'%3e%3cpath fill='%2300bc8c' d='M2.3 6.73.6 4.53c-.4-1.04.46-1.4 1.1-.8l1.1 1.4 3.4-3.8c.6-.63 1.6-.27 1.2.7l-4 4.6c-.43.5-.8.4-1.1.1z'/%3e%3c/svg%3e");background-repeat:no-repeat;background-position:right calc(0.375em + 0.1875rem) center;background-size:calc(0.75em + 0.375rem) calc(0.75em + 0.375rem)}.was-validated .form-control:valid:focus,.form-control.is-valid:focus{border-color:#00bc8c;box-shadow:0 0 0 .25rem rgba(0,188,140,.25)}.was-validated textarea.form-control:valid,textarea.form-control.is-valid{padding-right:calc(1.5em + 0.75rem);background-position:top calc(0.375em + 0.1875rem) right calc(0.375em + 0.1875rem)}.was-validated .form-select:valid,.form-select.is-valid{border-color:#00bc8c}.was-validated .form-select:valid:not([multiple]):not([size]),.was-validated .form-select:valid:not([multiple])[size="1"],.form-select.is-valid:not([multiple]):not([size]),.form-select.is-valid:not([multiple])[size="1"]{--bs-form-select-bg-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 8 8'%3e%3cpath fill='%2300bc8c' d='M2.3 6.73.6 4.53c-.4-1.04.46-1.4 1.1-.8l1.1 1.4 3.4-3.8c.6-.63 1.6-.27 1.2.7l-4 4.6c-.43.5-.8.4-1.1.1z'/%3e%3c/svg%3e");padding-right:4.125rem;background-position:right .75rem center,center right 2.25rem;background-size:16px 12px,calc(0.75em + 0.375rem) calc(0.75em + 0.375rem)}.was-validated .form-select:valid:focus,.form-select.is-valid:focus{border-color:#00bc8c;box-shadow:0 0 0 .25rem rgba(0,188,140,.25)}.was-validated .form-control-color:valid,.form-control-color.is-valid{width:calc(3rem + calc(1.5em + 0.75rem))}.was-validated .form-check-input:valid,.form-check-input.is-valid{border-color:#00bc8c}.was-validated .form-check-input:valid:checked,.form-check-input.is-valid:checked{background-color:#00bc8c}.was-validated .form-check-input:valid:focus,.form-check-input.is-valid:focus{box-shadow:0 0 0 .25rem rgba(0,188,140,.25)}.was-validated .form-check-input:valid~.form-check-label,.form-check-input.is-valid~.form-check-label{color:#00bc8c}.form-check-inline .form-check-input~.valid-feedback{margin-left:.5em}.was-validated .input-group>.form-control:not(:focus):valid,.input-group>.form-control:not(:focus).is-valid,.was-validated .input-group>.form-select:not(:focus):valid,.input-group>.form-select:not(:focus).is-valid,.was-validated .input-group>.form-floating:not(:focus-within):valid,.input-group>.form-floating:not(:focus-within).is-valid{z-index:3}.invalid-feedback{display:none;width:100%;margin-top:.25rem;font-size:0.875em;color:#e74c3c}.invalid-tooltip{position:absolute;top:100%;z-index:5;display:none;max-width:100%;padding:.25rem .5rem;margin-top:.1rem;font-size:0.875rem;color:#fff;background-color:#e74c3c;border-radius:.25rem}.was-validated :invalid~.invalid-feedback,.was-validated :invalid~.invalid-tooltip,.is-invalid~.invalid-feedback,.is-invalid~.invalid-tooltip{display:block}.was-validated .form-control:invalid,.form-control.is-invalid{border-color:#e74c3c;padding-right:calc(1.5em + 0.75rem);background-image:url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 12 12' width='12' height='12' fill='none' stroke='%23e74c3c'%3e%3ccircle cx='6' cy='6' r='4.5'/%3e%3cpath stroke-linejoin='round' d='M5.8 3.6h.4L6 6.5z'/%3e%3ccircle cx='6' cy='8.2' r='.6' fill='%23e74c3c' stroke='none'/%3e%3c/svg%3e");background-repeat:no-repeat;background-position:right calc(0.375em + 0.1875rem) center;background-size:calc(0.75em + 0.375rem) calc(0.75em + 0.375rem)}.was-validated .form-control:invalid:focus,.form-control.is-invalid:focus{border-color:#e74c3c;box-shadow:0 0 0 .25rem rgba(231,76,60,.25)}.was-validated textarea.form-control:invalid,textarea.form-control.is-invalid{padding-right:calc(1.5em + 0.75rem);background-position:top calc(0.375em + 0.1875rem) right calc(0.375em + 0.1875rem)}.was-validated .form-select:invalid,.form-select.is-invalid{border-color:#e74c3c}.was-validated .form-select:invalid:not([multiple]):not([size]),.was-validated .form-select:invalid:not([multiple])[size="1"],.form-select.is-invalid:not([multiple]):not([size]),.form-select.is-invalid:not([multiple])[size="1"]{--bs-form-select-bg-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 12 12' width='12' height='12' fill='none' stroke='%23e74c3c'%3e%3ccircle cx='6' cy='6' r='4.5'/%3e%3cpath stroke-linejoin='round' d='M5.8 3.6h.4L6 6.5z'/%3e%3ccircle cx='6' cy='8.2' r='.6' fill='%23e74c3c' stroke='none'/%3e%3c/svg%3e");padding-right:4.125rem;background-position:right .75rem center,center right 2.25rem;background-size:16px 12px,calc(0.75em + 0.375rem) calc(0.75em + 0.375rem)}.was-validated .form-select:invalid:focus,.form-select.is-invalid:focus{border-color:#e74c3c;box-shadow:0 0 0 .25rem rgba(231,76,60,.25)}.was-validated .form-control-color:invalid,.form-control-color.is-invalid{width:calc(3rem + calc(1.5em + 0.75rem))}.was-validated .form-check-input:invalid,.form-check-input.is-invalid{border-color:#e74c3c}.was-validated .form-check-input:invalid:checked,.form-check-input.is-invalid:checked{background-color:#e74c3c}.was-validated .form-check-input:invalid:focus,.form-check-input.is-invalid:focus{box-shadow:0 0 0 .25rem rgba(231,76,60,.25)}.was-validated .form-check-input:invalid~.form-check-label,.form-check-input.is-invalid~.form-check-label{color:#e74c3c}.form-check-inline .form-check-input~.invalid-feedback{margin-left:.5em}.was-validated .input-group>.form-control:not(:focus):invalid,.input-group>.form-control:not(:focus).is-invalid,.was-validated .input-group>.form-select:not(:focus):invalid,.input-group>.form-select:not(:focus).is-invalid,.was-validated .input-group>.form-floating:not(:focus-within):invalid,.input-group>.form-floating:not(:focus-within).is-invalid{z-index:4}.btn{--bs-btn-padding-x: 0.75rem;--bs-btn-padding-y: 0.375rem;--bs-btn-font-family: ;--bs-btn-font-size:1rem;--bs-btn-font-weight: 400;--bs-btn-line-height: 1.5;--bs-btn-color: #fff;--bs-btn-bg: transparent;--bs-btn-border-width: 1px;--bs-btn-border-color: transparent;--bs-btn-border-radius: 0.25rem;--bs-btn-hover-border-color: transparent;--bs-btn-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.15), 0 1px 1px rgba(0, 0, 0, 0.075);--bs-btn-disabled-opacity: 0.65;--bs-btn-focus-box-shadow: 0 0 0 0.25rem rgba(var(--bs-btn-focus-shadow-rgb), .5);display:inline-block;padding:var(--bs-btn-padding-y) var(--bs-btn-padding-x);font-family:var(--bs-btn-font-family);font-size:var(--bs-btn-font-size);font-weight:var(--bs-btn-font-weight);line-height:var(--bs-btn-line-height);color:var(--bs-btn-color);text-align:center;text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;vertical-align:middle;cursor:pointer;user-select:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;-o-user-select:none;border:var(--bs-btn-border-width) solid var(--bs-btn-border-color);border-radius:var(--bs-btn-border-radius);background-color:var(--bs-btn-bg);transition:color .15s ease-in-out,background-color .15s ease-in-out,border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.btn{transition:none}}.btn:hover{color:var(--bs-btn-hover-color);background-color:var(--bs-btn-hover-bg);border-color:var(--bs-btn-hover-border-color)}.btn-check+.btn:hover{color:var(--bs-btn-color);background-color:var(--bs-btn-bg);border-color:var(--bs-btn-border-color)}.btn:focus-visible{color:var(--bs-btn-hover-color);background-color:var(--bs-btn-hover-bg);border-color:var(--bs-btn-hover-border-color);outline:0;box-shadow:var(--bs-btn-focus-box-shadow)}.btn-check:focus-visible+.btn{border-color:var(--bs-btn-hover-border-color);outline:0;box-shadow:var(--bs-btn-focus-box-shadow)}.btn-check:checked+.btn,:not(.btn-check)+.btn:active,.btn:first-child:active,.btn.active,.btn.show{color:var(--bs-btn-active-color);background-color:var(--bs-btn-active-bg);border-color:var(--bs-btn-active-border-color)}.btn-check:checked+.btn:focus-visible,:not(.btn-check)+.btn:active:focus-visible,.btn:first-child:active:focus-visible,.btn.active:focus-visible,.btn.show:focus-visible{box-shadow:var(--bs-btn-focus-box-shadow)}.btn:disabled,.btn.disabled,fieldset:disabled .btn{color:var(--bs-btn-disabled-color);pointer-events:none;background-color:var(--bs-btn-disabled-bg);border-color:var(--bs-btn-disabled-border-color);opacity:var(--bs-btn-disabled-opacity)}.btn-default{--bs-btn-color: #fff;--bs-btn-bg: #434343;--bs-btn-border-color: #434343;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #393939;--bs-btn-hover-border-color: #363636;--bs-btn-focus-shadow-rgb: 95, 95, 95;--bs-btn-active-color: #fff;--bs-btn-active-bg: #363636;--bs-btn-active-border-color: #323232;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #434343;--bs-btn-disabled-border-color: #434343}.btn-primary{--bs-btn-color: #fff;--bs-btn-bg: #375a7f;--bs-btn-border-color: #375a7f;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #2f4d6c;--bs-btn-hover-border-color: #2c4866;--bs-btn-focus-shadow-rgb: 85, 115, 146;--bs-btn-active-color: #fff;--bs-btn-active-bg: #2c4866;--bs-btn-active-border-color: #29445f;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #375a7f;--bs-btn-disabled-border-color: #375a7f}.btn-secondary{--bs-btn-color: #fff;--bs-btn-bg: #434343;--bs-btn-border-color: #434343;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #393939;--bs-btn-hover-border-color: #363636;--bs-btn-focus-shadow-rgb: 95, 95, 95;--bs-btn-active-color: #fff;--bs-btn-active-bg: #363636;--bs-btn-active-border-color: #323232;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #434343;--bs-btn-disabled-border-color: #434343}.btn-success{--bs-btn-color: #fff;--bs-btn-bg: #00bc8c;--bs-btn-border-color: #00bc8c;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #00a077;--bs-btn-hover-border-color: #009670;--bs-btn-focus-shadow-rgb: 38, 198, 157;--bs-btn-active-color: #fff;--bs-btn-active-bg: #009670;--bs-btn-active-border-color: #008d69;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #00bc8c;--bs-btn-disabled-border-color: #00bc8c}.btn-info{--bs-btn-color: #fff;--bs-btn-bg: #3498db;--bs-btn-border-color: #3498db;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #2c81ba;--bs-btn-hover-border-color: #2a7aaf;--bs-btn-focus-shadow-rgb: 82, 167, 224;--bs-btn-active-color: #fff;--bs-btn-active-bg: #2a7aaf;--bs-btn-active-border-color: #2772a4;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #3498db;--bs-btn-disabled-border-color: #3498db}.btn-warning{--bs-btn-color: #fff;--bs-btn-bg: #f39c12;--bs-btn-border-color: #f39c12;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #cf850f;--bs-btn-hover-border-color: #c27d0e;--bs-btn-focus-shadow-rgb: 245, 171, 54;--bs-btn-active-color: #fff;--bs-btn-active-bg: #c27d0e;--bs-btn-active-border-color: #b6750e;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #f39c12;--bs-btn-disabled-border-color: #f39c12}.btn-danger{--bs-btn-color: #fff;--bs-btn-bg: #e74c3c;--bs-btn-border-color: #e74c3c;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #c44133;--bs-btn-hover-border-color: #b93d30;--bs-btn-focus-shadow-rgb: 235, 103, 89;--bs-btn-active-color: #fff;--bs-btn-active-bg: #b93d30;--bs-btn-active-border-color: #ad392d;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #e74c3c;--bs-btn-disabled-border-color: #e74c3c}.btn-light{--bs-btn-color: #fff;--bs-btn-bg: #6f6f6f;--bs-btn-border-color: #6f6f6f;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #5e5e5e;--bs-btn-hover-border-color: #595959;--bs-btn-focus-shadow-rgb: 133, 133, 133;--bs-btn-active-color: #fff;--bs-btn-active-bg: #595959;--bs-btn-active-border-color: #535353;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #6f6f6f;--bs-btn-disabled-border-color: #6f6f6f}.btn-dark{--bs-btn-color: #fff;--bs-btn-bg: #2d2d2d;--bs-btn-border-color: #2d2d2d;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #4d4d4d;--bs-btn-hover-border-color: #424242;--bs-btn-focus-shadow-rgb: 77, 77, 77;--bs-btn-active-color: #fff;--bs-btn-active-bg: #575757;--bs-btn-active-border-color: #424242;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #2d2d2d;--bs-btn-disabled-border-color: #2d2d2d}.btn-outline-default{--bs-btn-color: #434343;--bs-btn-border-color: #434343;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #434343;--bs-btn-hover-border-color: #434343;--bs-btn-focus-shadow-rgb: 67, 67, 67;--bs-btn-active-color: #fff;--bs-btn-active-bg: #434343;--bs-btn-active-border-color: #434343;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #434343;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #434343;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-primary{--bs-btn-color: #375a7f;--bs-btn-border-color: #375a7f;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #375a7f;--bs-btn-hover-border-color: #375a7f;--bs-btn-focus-shadow-rgb: 55, 90, 127;--bs-btn-active-color: #fff;--bs-btn-active-bg: #375a7f;--bs-btn-active-border-color: #375a7f;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #375a7f;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #375a7f;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-secondary{--bs-btn-color: #434343;--bs-btn-border-color: #434343;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #434343;--bs-btn-hover-border-color: #434343;--bs-btn-focus-shadow-rgb: 67, 67, 67;--bs-btn-active-color: #fff;--bs-btn-active-bg: #434343;--bs-btn-active-border-color: #434343;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #434343;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #434343;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-success{--bs-btn-color: #00bc8c;--bs-btn-border-color: #00bc8c;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #00bc8c;--bs-btn-hover-border-color: #00bc8c;--bs-btn-focus-shadow-rgb: 0, 188, 140;--bs-btn-active-color: #fff;--bs-btn-active-bg: #00bc8c;--bs-btn-active-border-color: #00bc8c;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #00bc8c;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #00bc8c;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-info{--bs-btn-color: #3498db;--bs-btn-border-color: #3498db;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #3498db;--bs-btn-hover-border-color: #3498db;--bs-btn-focus-shadow-rgb: 52, 152, 219;--bs-btn-active-color: #fff;--bs-btn-active-bg: #3498db;--bs-btn-active-border-color: #3498db;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #3498db;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #3498db;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-warning{--bs-btn-color: #f39c12;--bs-btn-border-color: #f39c12;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #f39c12;--bs-btn-hover-border-color: #f39c12;--bs-btn-focus-shadow-rgb: 243, 156, 18;--bs-btn-active-color: #fff;--bs-btn-active-bg: #f39c12;--bs-btn-active-border-color: #f39c12;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #f39c12;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #f39c12;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-danger{--bs-btn-color: #e74c3c;--bs-btn-border-color: #e74c3c;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #e74c3c;--bs-btn-hover-border-color: #e74c3c;--bs-btn-focus-shadow-rgb: 231, 76, 60;--bs-btn-active-color: #fff;--bs-btn-active-bg: #e74c3c;--bs-btn-active-border-color: #e74c3c;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #e74c3c;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #e74c3c;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-light{--bs-btn-color: #6f6f6f;--bs-btn-border-color: #6f6f6f;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #6f6f6f;--bs-btn-hover-border-color: #6f6f6f;--bs-btn-focus-shadow-rgb: 111, 111, 111;--bs-btn-active-color: #fff;--bs-btn-active-bg: #6f6f6f;--bs-btn-active-border-color: #6f6f6f;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #6f6f6f;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #6f6f6f;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-dark{--bs-btn-color: #2d2d2d;--bs-btn-border-color: #2d2d2d;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #2d2d2d;--bs-btn-hover-border-color: #2d2d2d;--bs-btn-focus-shadow-rgb: 45, 45, 45;--bs-btn-active-color: #fff;--bs-btn-active-bg: #2d2d2d;--bs-btn-active-border-color: #2d2d2d;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #2d2d2d;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #2d2d2d;--bs-btn-bg: transparent;--bs-gradient: none}.btn-link{--bs-btn-font-weight: 400;--bs-btn-color: #00bc8c;--bs-btn-bg: transparent;--bs-btn-border-color: transparent;--bs-btn-hover-color: #009670;--bs-btn-hover-border-color: transparent;--bs-btn-active-color: #009670;--bs-btn-active-border-color: transparent;--bs-btn-disabled-color: #6c757d;--bs-btn-disabled-border-color: transparent;--bs-btn-box-shadow: 0 0 0 #000;--bs-btn-focus-shadow-rgb: 38, 198, 157;text-decoration:underline;-webkit-text-decoration:underline;-moz-text-decoration:underline;-ms-text-decoration:underline;-o-text-decoration:underline}.btn-link:focus-visible{color:var(--bs-btn-color)}.btn-link:hover{color:var(--bs-btn-hover-color)}.btn-lg,.btn-group-lg>.btn{--bs-btn-padding-y: 0.5rem;--bs-btn-padding-x: 1rem;--bs-btn-font-size:1.25rem;--bs-btn-border-radius: 0.5rem}.btn-sm,.btn-group-sm>.btn{--bs-btn-padding-y: 0.25rem;--bs-btn-padding-x: 0.5rem;--bs-btn-font-size:0.875rem;--bs-btn-border-radius: 0.2em}.fade{transition:opacity .15s linear}@media(prefers-reduced-motion: reduce){.fade{transition:none}}.fade:not(.show){opacity:0}.collapse:not(.show){display:none}.collapsing{height:0;overflow:hidden;transition:height .2s ease}@media(prefers-reduced-motion: reduce){.collapsing{transition:none}}.collapsing.collapse-horizontal{width:0;height:auto;transition:width .35s ease}@media(prefers-reduced-motion: reduce){.collapsing.collapse-horizontal{transition:none}}.dropup,.dropend,.dropdown,.dropstart,.dropup-center,.dropdown-center{position:relative}.dropdown-toggle{white-space:nowrap}.dropdown-toggle::after{display:inline-block;margin-left:.255em;vertical-align:.255em;content:"";border-top:.3em solid;border-right:.3em solid rgba(0,0,0,0);border-bottom:0;border-left:.3em solid rgba(0,0,0,0)}.dropdown-toggle:empty::after{margin-left:0}.dropdown-menu{--bs-dropdown-zindex: 1000;--bs-dropdown-min-width: 10rem;--bs-dropdown-padding-x: 0;--bs-dropdown-padding-y: 0.5rem;--bs-dropdown-spacer: 0.125rem;--bs-dropdown-font-size:1rem;--bs-dropdown-color: #fff;--bs-dropdown-bg: #222;--bs-dropdown-border-color: #434343;--bs-dropdown-border-radius: 0.25rem;--bs-dropdown-border-width: 1px;--bs-dropdown-inner-border-radius: calc(0.25rem - 1px);--bs-dropdown-divider-bg: #434343;--bs-dropdown-divider-margin-y: 0.5rem;--bs-dropdown-box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);--bs-dropdown-link-color: #fff;--bs-dropdown-link-hover-color: #fff;--bs-dropdown-link-hover-bg: #375a7f;--bs-dropdown-link-active-color: #fff;--bs-dropdown-link-active-bg: #375a7f;--bs-dropdown-link-disabled-color: rgba(255, 255, 255, 0.5);--bs-dropdown-item-padding-x: 1rem;--bs-dropdown-item-padding-y: 0.25rem;--bs-dropdown-header-color: #6c757d;--bs-dropdown-header-padding-x: 1rem;--bs-dropdown-header-padding-y: 0.5rem;position:absolute;z-index:var(--bs-dropdown-zindex);display:none;min-width:var(--bs-dropdown-min-width);padding:var(--bs-dropdown-padding-y) var(--bs-dropdown-padding-x);margin:0;font-size:var(--bs-dropdown-font-size);color:var(--bs-dropdown-color);text-align:left;list-style:none;background-color:var(--bs-dropdown-bg);background-clip:padding-box;border:var(--bs-dropdown-border-width) solid var(--bs-dropdown-border-color);border-radius:var(--bs-dropdown-border-radius)}.dropdown-menu[data-bs-popper]{top:100%;left:0;margin-top:var(--bs-dropdown-spacer)}.dropdown-menu-start{--bs-position: start}.dropdown-menu-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-end{--bs-position: end}.dropdown-menu-end[data-bs-popper]{right:0;left:auto}@media(min-width: 576px){.dropdown-menu-sm-start{--bs-position: start}.dropdown-menu-sm-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-sm-end{--bs-position: end}.dropdown-menu-sm-end[data-bs-popper]{right:0;left:auto}}@media(min-width: 768px){.dropdown-menu-md-start{--bs-position: start}.dropdown-menu-md-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-md-end{--bs-position: end}.dropdown-menu-md-end[data-bs-popper]{right:0;left:auto}}@media(min-width: 992px){.dropdown-menu-lg-start{--bs-position: start}.dropdown-menu-lg-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-lg-end{--bs-position: end}.dropdown-menu-lg-end[data-bs-popper]{right:0;left:auto}}@media(min-width: 1200px){.dropdown-menu-xl-start{--bs-position: start}.dropdown-menu-xl-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-xl-end{--bs-position: end}.dropdown-menu-xl-end[data-bs-popper]{right:0;left:auto}}@media(min-width: 1400px){.dropdown-menu-xxl-start{--bs-position: start}.dropdown-menu-xxl-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-xxl-end{--bs-position: end}.dropdown-menu-xxl-end[data-bs-popper]{right:0;left:auto}}.dropup .dropdown-menu[data-bs-popper]{top:auto;bottom:100%;margin-top:0;margin-bottom:var(--bs-dropdown-spacer)}.dropup .dropdown-toggle::after{display:inline-block;margin-left:.255em;vertical-align:.255em;content:"";border-top:0;border-right:.3em solid rgba(0,0,0,0);border-bottom:.3em solid;border-left:.3em solid rgba(0,0,0,0)}.dropup .dropdown-toggle:empty::after{margin-left:0}.dropend .dropdown-menu[data-bs-popper]{top:0;right:auto;left:100%;margin-top:0;margin-left:var(--bs-dropdown-spacer)}.dropend .dropdown-toggle::after{display:inline-block;margin-left:.255em;vertical-align:.255em;content:"";border-top:.3em solid rgba(0,0,0,0);border-right:0;border-bottom:.3em solid rgba(0,0,0,0);border-left:.3em solid}.dropend .dropdown-toggle:empty::after{margin-left:0}.dropend .dropdown-toggle::after{vertical-align:0}.dropstart .dropdown-menu[data-bs-popper]{top:0;right:100%;left:auto;margin-top:0;margin-right:var(--bs-dropdown-spacer)}.dropstart .dropdown-toggle::after{display:inline-block;margin-left:.255em;vertical-align:.255em;content:""}.dropstart .dropdown-toggle::after{display:none}.dropstart .dropdown-toggle::before{display:inline-block;margin-right:.255em;vertical-align:.255em;content:"";border-top:.3em solid rgba(0,0,0,0);border-right:.3em solid;border-bottom:.3em solid rgba(0,0,0,0)}.dropstart .dropdown-toggle:empty::after{margin-left:0}.dropstart .dropdown-toggle::before{vertical-align:0}.dropdown-divider{height:0;margin:var(--bs-dropdown-divider-margin-y) 0;overflow:hidden;border-top:1px solid var(--bs-dropdown-divider-bg);opacity:1}.dropdown-item{display:block;width:100%;padding:var(--bs-dropdown-item-padding-y) var(--bs-dropdown-item-padding-x);clear:both;font-weight:400;color:var(--bs-dropdown-link-color);text-align:inherit;text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;white-space:nowrap;background-color:rgba(0,0,0,0);border:0;border-radius:var(--bs-dropdown-item-border-radius, 0)}.dropdown-item:hover,.dropdown-item:focus{color:var(--bs-dropdown-link-hover-color);background-color:var(--bs-dropdown-link-hover-bg)}.dropdown-item.active,.dropdown-item:active{color:var(--bs-dropdown-link-active-color);text-decoration:none;background-color:var(--bs-dropdown-link-active-bg)}.dropdown-item.disabled,.dropdown-item:disabled{color:var(--bs-dropdown-link-disabled-color);pointer-events:none;background-color:rgba(0,0,0,0)}.dropdown-menu.show{display:block}.dropdown-header{display:block;padding:var(--bs-dropdown-header-padding-y) var(--bs-dropdown-header-padding-x);margin-bottom:0;font-size:0.875rem;color:var(--bs-dropdown-header-color);white-space:nowrap}.dropdown-item-text{display:block;padding:var(--bs-dropdown-item-padding-y) var(--bs-dropdown-item-padding-x);color:var(--bs-dropdown-link-color)}.dropdown-menu-dark{--bs-dropdown-color: #dee2e6;--bs-dropdown-bg: #343a40;--bs-dropdown-border-color: #434343;--bs-dropdown-box-shadow: ;--bs-dropdown-link-color: #dee2e6;--bs-dropdown-link-hover-color: #fff;--bs-dropdown-divider-bg: #434343;--bs-dropdown-link-hover-bg: rgba(255, 255, 255, 0.15);--bs-dropdown-link-active-color: #fff;--bs-dropdown-link-active-bg: #375a7f;--bs-dropdown-link-disabled-color: #adb5bd;--bs-dropdown-header-color: #adb5bd}.btn-group,.btn-group-vertical{position:relative;display:inline-flex;vertical-align:middle}.btn-group>.btn,.btn-group-vertical>.btn{position:relative;flex:1 1 auto;-webkit-flex:1 1 auto}.btn-group>.btn-check:checked+.btn,.btn-group>.btn-check:focus+.btn,.btn-group>.btn:hover,.btn-group>.btn:focus,.btn-group>.btn:active,.btn-group>.btn.active,.btn-group-vertical>.btn-check:checked+.btn,.btn-group-vertical>.btn-check:focus+.btn,.btn-group-vertical>.btn:hover,.btn-group-vertical>.btn:focus,.btn-group-vertical>.btn:active,.btn-group-vertical>.btn.active{z-index:1}.btn-toolbar{display:flex;display:-webkit-flex;flex-wrap:wrap;-webkit-flex-wrap:wrap;justify-content:flex-start;-webkit-justify-content:flex-start}.btn-toolbar .input-group{width:auto}.btn-group{border-radius:.25rem}.btn-group>:not(.btn-check:first-child)+.btn,.btn-group>.btn-group:not(:first-child){margin-left:calc(1px*-1)}.btn-group>.btn:not(:last-child):not(.dropdown-toggle),.btn-group>.btn.dropdown-toggle-split:first-child,.btn-group>.btn-group:not(:last-child)>.btn{border-top-right-radius:0;border-bottom-right-radius:0}.btn-group>.btn:nth-child(n+3),.btn-group>:not(.btn-check)+.btn,.btn-group>.btn-group:not(:first-child)>.btn{border-top-left-radius:0;border-bottom-left-radius:0}.dropdown-toggle-split{padding-right:.5625rem;padding-left:.5625rem}.dropdown-toggle-split::after,.dropup .dropdown-toggle-split::after,.dropend .dropdown-toggle-split::after{margin-left:0}.dropstart .dropdown-toggle-split::before{margin-right:0}.btn-sm+.dropdown-toggle-split,.btn-group-sm>.btn+.dropdown-toggle-split{padding-right:.375rem;padding-left:.375rem}.btn-lg+.dropdown-toggle-split,.btn-group-lg>.btn+.dropdown-toggle-split{padding-right:.75rem;padding-left:.75rem}.btn-group-vertical{flex-direction:column;-webkit-flex-direction:column;align-items:flex-start;-webkit-align-items:flex-start;justify-content:center;-webkit-justify-content:center}.btn-group-vertical>.btn,.btn-group-vertical>.btn-group{width:100%}.btn-group-vertical>.btn:not(:first-child),.btn-group-vertical>.btn-group:not(:first-child){margin-top:calc(1px*-1)}.btn-group-vertical>.btn:not(:last-child):not(.dropdown-toggle),.btn-group-vertical>.btn-group:not(:last-child)>.btn{border-bottom-right-radius:0;border-bottom-left-radius:0}.btn-group-vertical>.btn~.btn,.btn-group-vertical>.btn-group:not(:first-child)>.btn{border-top-left-radius:0;border-top-right-radius:0}.nav{--bs-nav-link-padding-x: 2rem;--bs-nav-link-padding-y: 0.5rem;--bs-nav-link-font-weight: ;--bs-nav-link-color: #00bc8c;--bs-nav-link-hover-color: #009670;--bs-nav-link-disabled-color: #6f6f6f;display:flex;display:-webkit-flex;flex-wrap:wrap;-webkit-flex-wrap:wrap;padding-left:0;margin-bottom:0;list-style:none}.nav-link{display:block;padding:var(--bs-nav-link-padding-y) var(--bs-nav-link-padding-x);font-size:var(--bs-nav-link-font-size);font-weight:var(--bs-nav-link-font-weight);color:var(--bs-nav-link-color);text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;background:none;border:0;transition:color .15s ease-in-out,background-color .15s ease-in-out,border-color .15s ease-in-out}@media(prefers-reduced-motion: reduce){.nav-link{transition:none}}.nav-link:hover,.nav-link:focus{color:var(--bs-nav-link-hover-color)}.nav-link:focus-visible{outline:0;box-shadow:0 0 0 .25rem rgba(55,90,127,.25)}.nav-link.disabled,.nav-link:disabled{color:var(--bs-nav-link-disabled-color);pointer-events:none;cursor:default}.nav-tabs{--bs-nav-tabs-border-width: 1px;--bs-nav-tabs-border-color: #434343;--bs-nav-tabs-border-radius: 0.25rem;--bs-nav-tabs-link-hover-border-color: #434343 #434343 transparent;--bs-nav-tabs-link-active-color: #fff;--bs-nav-tabs-link-active-bg: #222;--bs-nav-tabs-link-active-border-color: #434343 #434343 transparent;border-bottom:var(--bs-nav-tabs-border-width) solid var(--bs-nav-tabs-border-color)}.nav-tabs .nav-link{margin-bottom:calc(-1*var(--bs-nav-tabs-border-width));border:var(--bs-nav-tabs-border-width) solid rgba(0,0,0,0);border-top-left-radius:var(--bs-nav-tabs-border-radius);border-top-right-radius:var(--bs-nav-tabs-border-radius)}.nav-tabs .nav-link:hover,.nav-tabs .nav-link:focus{isolation:isolate;border-color:var(--bs-nav-tabs-link-hover-border-color)}.nav-tabs .nav-link.active,.nav-tabs .nav-item.show .nav-link{color:var(--bs-nav-tabs-link-active-color);background-color:var(--bs-nav-tabs-link-active-bg);border-color:var(--bs-nav-tabs-link-active-border-color)}.nav-tabs .dropdown-menu{margin-top:calc(-1*var(--bs-nav-tabs-border-width));border-top-left-radius:0;border-top-right-radius:0}.nav-pills{--bs-nav-pills-border-radius: 0.25rem;--bs-nav-pills-link-active-color: #fff;--bs-nav-pills-link-active-bg: #375a7f}.nav-pills .nav-link{border-radius:var(--bs-nav-pills-border-radius)}.nav-pills .nav-link.active,.nav-pills .show>.nav-link{color:var(--bs-nav-pills-link-active-color);background-color:var(--bs-nav-pills-link-active-bg)}.nav-underline{--bs-nav-underline-gap: 1rem;--bs-nav-underline-border-width: 0.125rem;--bs-nav-underline-link-active-color: #000;gap:var(--bs-nav-underline-gap)}.nav-underline .nav-link{padding-right:0;padding-left:0;border-bottom:var(--bs-nav-underline-border-width) solid rgba(0,0,0,0)}.nav-underline .nav-link:hover,.nav-underline .nav-link:focus{border-bottom-color:currentcolor}.nav-underline .nav-link.active,.nav-underline .show>.nav-link{font-weight:700;color:var(--bs-nav-underline-link-active-color);border-bottom-color:currentcolor}.nav-fill>.nav-link,.nav-fill .nav-item{flex:1 1 auto;-webkit-flex:1 1 auto;text-align:center}.nav-justified>.nav-link,.nav-justified .nav-item{flex-basis:0;-webkit-flex-basis:0;flex-grow:1;-webkit-flex-grow:1;text-align:center}.nav-fill .nav-item .nav-link,.nav-justified .nav-item .nav-link{width:100%}.tab-content>.tab-pane{display:none}.tab-content>.active{display:block}.navbar{--bs-navbar-padding-x: 0;--bs-navbar-padding-y: 1rem;--bs-navbar-color: #dee2e6;--bs-navbar-hover-color: rgba(250, 254, 253, 0.8);--bs-navbar-disabled-color: rgba(222, 226, 230, 0.75);--bs-navbar-active-color: #fafefd;--bs-navbar-brand-padding-y: 0.3125rem;--bs-navbar-brand-margin-end: 1rem;--bs-navbar-brand-font-size: 1.25rem;--bs-navbar-brand-color: #dee2e6;--bs-navbar-brand-hover-color: #fafefd;--bs-navbar-nav-link-padding-x: 0.5rem;--bs-navbar-toggler-padding-y: 0.25;--bs-navbar-toggler-padding-x: 0;--bs-navbar-toggler-font-size: 1.25rem;--bs-navbar-toggler-icon-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'%3e%3cpath stroke='%23dee2e6' stroke-linecap='round' stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/%3e%3c/svg%3e");--bs-navbar-toggler-border-color: rgba(222, 226, 230, 0);--bs-navbar-toggler-border-radius: 0.25rem;--bs-navbar-toggler-focus-width: 0.25rem;--bs-navbar-toggler-transition: box-shadow 0.15s ease-in-out;position:relative;display:flex;display:-webkit-flex;flex-wrap:wrap;-webkit-flex-wrap:wrap;align-items:center;-webkit-align-items:center;justify-content:space-between;-webkit-justify-content:space-between;padding:var(--bs-navbar-padding-y) var(--bs-navbar-padding-x)}.navbar>.container,.navbar>.container-fluid,.navbar>.container-sm,.navbar>.container-md,.navbar>.container-lg,.navbar>.container-xl,.navbar>.container-xxl{display:flex;display:-webkit-flex;flex-wrap:inherit;-webkit-flex-wrap:inherit;align-items:center;-webkit-align-items:center;justify-content:space-between;-webkit-justify-content:space-between}.navbar-brand{padding-top:var(--bs-navbar-brand-padding-y);padding-bottom:var(--bs-navbar-brand-padding-y);margin-right:var(--bs-navbar-brand-margin-end);font-size:var(--bs-navbar-brand-font-size);color:var(--bs-navbar-brand-color);text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;white-space:nowrap}.navbar-brand:hover,.navbar-brand:focus{color:var(--bs-navbar-brand-hover-color)}.navbar-nav{--bs-nav-link-padding-x: 0;--bs-nav-link-padding-y: 0.5rem;--bs-nav-link-font-weight: ;--bs-nav-link-color: var(--bs-navbar-color);--bs-nav-link-hover-color: var(--bs-navbar-hover-color);--bs-nav-link-disabled-color: var(--bs-navbar-disabled-color);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;padding-left:0;margin-bottom:0;list-style:none}.navbar-nav .nav-link.active,.navbar-nav .nav-link.show{color:var(--bs-navbar-active-color)}.navbar-nav .dropdown-menu{position:static}.navbar-text{padding-top:.5rem;padding-bottom:.5rem;color:var(--bs-navbar-color)}.navbar-text a,.navbar-text a:hover,.navbar-text a:focus{color:var(--bs-navbar-active-color)}.navbar-collapse{flex-basis:100%;-webkit-flex-basis:100%;flex-grow:1;-webkit-flex-grow:1;align-items:center;-webkit-align-items:center}.navbar-toggler{padding:var(--bs-navbar-toggler-padding-y) var(--bs-navbar-toggler-padding-x);font-size:var(--bs-navbar-toggler-font-size);line-height:1;color:var(--bs-navbar-color);background-color:rgba(0,0,0,0);border:var(--bs-border-width) solid var(--bs-navbar-toggler-border-color);border-radius:var(--bs-navbar-toggler-border-radius);transition:var(--bs-navbar-toggler-transition)}@media(prefers-reduced-motion: reduce){.navbar-toggler{transition:none}}.navbar-toggler:hover{text-decoration:none}.navbar-toggler:focus{text-decoration:none;outline:0;box-shadow:0 0 0 var(--bs-navbar-toggler-focus-width)}.navbar-toggler-icon{display:inline-block;width:1.5em;height:1.5em;vertical-align:middle;background-image:var(--bs-navbar-toggler-icon-bg);background-repeat:no-repeat;background-position:center;background-size:100%}.navbar-nav-scroll{max-height:var(--bs-scroll-height, 75vh);overflow-y:auto}@media(min-width: 576px){.navbar-expand-sm{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand-sm .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand-sm .navbar-nav .dropdown-menu{position:absolute}.navbar-expand-sm .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand-sm .navbar-nav-scroll{overflow:visible}.navbar-expand-sm .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand-sm .navbar-toggler{display:none}.navbar-expand-sm .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand-sm .offcanvas .offcanvas-header{display:none}.navbar-expand-sm .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}}@media(min-width: 768px){.navbar-expand-md{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand-md .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand-md .navbar-nav .dropdown-menu{position:absolute}.navbar-expand-md .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand-md .navbar-nav-scroll{overflow:visible}.navbar-expand-md .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand-md .navbar-toggler{display:none}.navbar-expand-md .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand-md .offcanvas .offcanvas-header{display:none}.navbar-expand-md .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}}@media(min-width: 992px){.navbar-expand-lg{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand-lg .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand-lg .navbar-nav .dropdown-menu{position:absolute}.navbar-expand-lg .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand-lg .navbar-nav-scroll{overflow:visible}.navbar-expand-lg .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand-lg .navbar-toggler{display:none}.navbar-expand-lg .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand-lg .offcanvas .offcanvas-header{display:none}.navbar-expand-lg .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}}@media(min-width: 1200px){.navbar-expand-xl{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand-xl .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand-xl .navbar-nav .dropdown-menu{position:absolute}.navbar-expand-xl .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand-xl .navbar-nav-scroll{overflow:visible}.navbar-expand-xl .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand-xl .navbar-toggler{display:none}.navbar-expand-xl .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand-xl .offcanvas .offcanvas-header{display:none}.navbar-expand-xl .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}}@media(min-width: 1400px){.navbar-expand-xxl{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand-xxl .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand-xxl .navbar-nav .dropdown-menu{position:absolute}.navbar-expand-xxl .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand-xxl .navbar-nav-scroll{overflow:visible}.navbar-expand-xxl .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand-xxl .navbar-toggler{display:none}.navbar-expand-xxl .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand-xxl .offcanvas .offcanvas-header{display:none}.navbar-expand-xxl .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}}.navbar-expand{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand .navbar-nav .dropdown-menu{position:absolute}.navbar-expand .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand .navbar-nav-scroll{overflow:visible}.navbar-expand .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand .navbar-toggler{display:none}.navbar-expand .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand .offcanvas .offcanvas-header{display:none}.navbar-expand .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}.navbar-dark,.navbar[data-bs-theme=dark]{--bs-navbar-color: #dee2e6;--bs-navbar-hover-color: rgba(250, 254, 253, 0.8);--bs-navbar-disabled-color: rgba(222, 226, 230, 0.75);--bs-navbar-active-color: #fafefd;--bs-navbar-brand-color: #dee2e6;--bs-navbar-brand-hover-color: #fafefd;--bs-navbar-toggler-border-color: rgba(222, 226, 230, 0);--bs-navbar-toggler-icon-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'%3e%3cpath stroke='%23dee2e6' stroke-linecap='round' stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/%3e%3c/svg%3e")}[data-bs-theme=dark] .navbar-toggler-icon{--bs-navbar-toggler-icon-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'%3e%3cpath stroke='%23dee2e6' stroke-linecap='round' stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/%3e%3c/svg%3e")}.card{--bs-card-spacer-y: 1rem;--bs-card-spacer-x: 1rem;--bs-card-title-spacer-y: 0.5rem;--bs-card-title-color: ;--bs-card-subtitle-color: ;--bs-card-border-width: 1px;--bs-card-border-color: rgba(0, 0, 0, 0.175);--bs-card-border-radius: 0.25rem;--bs-card-box-shadow: ;--bs-card-inner-border-radius: calc(0.25rem - 1px);--bs-card-cap-padding-y: 0.5rem;--bs-card-cap-padding-x: 1rem;--bs-card-cap-bg: rgba(52, 58, 64, 0.25);--bs-card-cap-color: ;--bs-card-height: ;--bs-card-color: ;--bs-card-bg: #2d2d2d;--bs-card-img-overlay-padding: 1rem;--bs-card-group-margin: 0.75rem;position:relative;display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;min-width:0;height:var(--bs-card-height);color:var(--bs-body-color);word-wrap:break-word;background-color:var(--bs-card-bg);background-clip:border-box;border:var(--bs-card-border-width) solid var(--bs-card-border-color);border-radius:var(--bs-card-border-radius)}.card>hr{margin-right:0;margin-left:0}.card>.list-group{border-top:inherit;border-bottom:inherit}.card>.list-group:first-child{border-top-width:0;border-top-left-radius:var(--bs-card-inner-border-radius);border-top-right-radius:var(--bs-card-inner-border-radius)}.card>.list-group:last-child{border-bottom-width:0;border-bottom-right-radius:var(--bs-card-inner-border-radius);border-bottom-left-radius:var(--bs-card-inner-border-radius)}.card>.card-header+.list-group,.card>.list-group+.card-footer{border-top:0}.card-body{flex:1 1 auto;-webkit-flex:1 1 auto;padding:var(--bs-card-spacer-y) var(--bs-card-spacer-x);color:var(--bs-card-color)}.card-title{margin-bottom:var(--bs-card-title-spacer-y);color:var(--bs-card-title-color)}.card-subtitle{margin-top:calc(-0.5*var(--bs-card-title-spacer-y));margin-bottom:0;color:var(--bs-card-subtitle-color)}.card-text:last-child{margin-bottom:0}.card-link+.card-link{margin-left:var(--bs-card-spacer-x)}.card-header{padding:var(--bs-card-cap-padding-y) var(--bs-card-cap-padding-x);margin-bottom:0;color:var(--bs-card-cap-color);background-color:var(--bs-card-cap-bg);border-bottom:var(--bs-card-border-width) solid var(--bs-card-border-color)}.card-header:first-child{border-radius:var(--bs-card-inner-border-radius) var(--bs-card-inner-border-radius) 0 0}.card-footer{padding:var(--bs-card-cap-padding-y) var(--bs-card-cap-padding-x);color:var(--bs-card-cap-color);background-color:var(--bs-card-cap-bg);border-top:var(--bs-card-border-width) solid var(--bs-card-border-color)}.card-footer:last-child{border-radius:0 0 var(--bs-card-inner-border-radius) var(--bs-card-inner-border-radius)}.card-header-tabs{margin-right:calc(-0.5*var(--bs-card-cap-padding-x));margin-bottom:calc(-1*var(--bs-card-cap-padding-y));margin-left:calc(-0.5*var(--bs-card-cap-padding-x));border-bottom:0}.card-header-tabs .nav-link.active{background-color:var(--bs-card-bg);border-bottom-color:var(--bs-card-bg)}.card-header-pills{margin-right:calc(-0.5*var(--bs-card-cap-padding-x));margin-left:calc(-0.5*var(--bs-card-cap-padding-x))}.card-img-overlay{position:absolute;top:0;right:0;bottom:0;left:0;padding:var(--bs-card-img-overlay-padding);border-radius:var(--bs-card-inner-border-radius)}.card-img,.card-img-top,.card-img-bottom{width:100%}.card-img,.card-img-top{border-top-left-radius:var(--bs-card-inner-border-radius);border-top-right-radius:var(--bs-card-inner-border-radius)}.card-img,.card-img-bottom{border-bottom-right-radius:var(--bs-card-inner-border-radius);border-bottom-left-radius:var(--bs-card-inner-border-radius)}.card-group>.card{margin-bottom:var(--bs-card-group-margin)}@media(min-width: 576px){.card-group{display:flex;display:-webkit-flex;flex-flow:row wrap;-webkit-flex-flow:row wrap}.card-group>.card{flex:1 0 0%;-webkit-flex:1 0 0%;margin-bottom:0}.card-group>.card+.card{margin-left:0;border-left:0}.card-group>.card:not(:last-child){border-top-right-radius:0;border-bottom-right-radius:0}.card-group>.card:not(:last-child) .card-img-top,.card-group>.card:not(:last-child) .card-header{border-top-right-radius:0}.card-group>.card:not(:last-child) .card-img-bottom,.card-group>.card:not(:last-child) .card-footer{border-bottom-right-radius:0}.card-group>.card:not(:first-child){border-top-left-radius:0;border-bottom-left-radius:0}.card-group>.card:not(:first-child) .card-img-top,.card-group>.card:not(:first-child) .card-header{border-top-left-radius:0}.card-group>.card:not(:first-child) .card-img-bottom,.card-group>.card:not(:first-child) .card-footer{border-bottom-left-radius:0}}.accordion{--bs-accordion-color: #fff;--bs-accordion-bg: #222;--bs-accordion-transition: color 0.15s ease-in-out, background-color 0.15s ease-in-out, border-color 0.15s ease-in-out, box-shadow 0.15s ease-in-out, border-radius 0.15s ease;--bs-accordion-border-color: #dee2e6;--bs-accordion-border-width: 1px;--bs-accordion-border-radius: 0.25rem;--bs-accordion-inner-border-radius: calc(0.25rem - 1px);--bs-accordion-btn-padding-x: 1.25rem;--bs-accordion-btn-padding-y: 1rem;--bs-accordion-btn-color: #fff;--bs-accordion-btn-bg: #222;--bs-accordion-btn-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23fff'%3e%3cpath fill-rule='evenodd' d='M1.646 4.646a.5.5 0 0 1 .708 0L8 10.293l5.646-5.647a.5.5 0 0 1 .708.708l-6 6a.5.5 0 0 1-.708 0l-6-6a.5.5 0 0 1 0-.708z'/%3e%3c/svg%3e");--bs-accordion-btn-icon-width: 1.25rem;--bs-accordion-btn-icon-transform: rotate(-180deg);--bs-accordion-btn-icon-transition: transform 0.2s ease-in-out;--bs-accordion-btn-active-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23162433'%3e%3cpath fill-rule='evenodd' d='M1.646 4.646a.5.5 0 0 1 .708 0L8 10.293l5.646-5.647a.5.5 0 0 1 .708.708l-6 6a.5.5 0 0 1-.708 0l-6-6a.5.5 0 0 1 0-.708z'/%3e%3c/svg%3e");--bs-accordion-btn-focus-border-color: #9badbf;--bs-accordion-btn-focus-box-shadow: 0 0 0 0.25rem rgba(55, 90, 127, 0.25);--bs-accordion-body-padding-x: 1.25rem;--bs-accordion-body-padding-y: 1rem;--bs-accordion-active-color: #162433;--bs-accordion-active-bg: #d7dee5}.accordion-button{position:relative;display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;width:100%;padding:var(--bs-accordion-btn-padding-y) var(--bs-accordion-btn-padding-x);font-size:1rem;color:var(--bs-accordion-btn-color);text-align:left;background-color:var(--bs-accordion-btn-bg);border:0;border-radius:0;overflow-anchor:none;transition:var(--bs-accordion-transition)}@media(prefers-reduced-motion: reduce){.accordion-button{transition:none}}.accordion-button:not(.collapsed){color:var(--bs-accordion-active-color);background-color:var(--bs-accordion-active-bg);box-shadow:inset 0 calc(-1*var(--bs-accordion-border-width)) 0 var(--bs-accordion-border-color)}.accordion-button:not(.collapsed)::after{background-image:var(--bs-accordion-btn-active-icon);transform:var(--bs-accordion-btn-icon-transform)}.accordion-button::after{flex-shrink:0;-webkit-flex-shrink:0;width:var(--bs-accordion-btn-icon-width);height:var(--bs-accordion-btn-icon-width);margin-left:auto;content:"";background-image:var(--bs-accordion-btn-icon);background-repeat:no-repeat;background-size:var(--bs-accordion-btn-icon-width);transition:var(--bs-accordion-btn-icon-transition)}@media(prefers-reduced-motion: reduce){.accordion-button::after{transition:none}}.accordion-button:hover{z-index:2}.accordion-button:focus{z-index:3;border-color:var(--bs-accordion-btn-focus-border-color);outline:0;box-shadow:var(--bs-accordion-btn-focus-box-shadow)}.accordion-header{margin-bottom:0}.accordion-item{color:var(--bs-accordion-color);background-color:var(--bs-accordion-bg);border:var(--bs-accordion-border-width) solid var(--bs-accordion-border-color)}.accordion-item:first-of-type{border-top-left-radius:var(--bs-accordion-border-radius);border-top-right-radius:var(--bs-accordion-border-radius)}.accordion-item:first-of-type .accordion-button{border-top-left-radius:var(--bs-accordion-inner-border-radius);border-top-right-radius:var(--bs-accordion-inner-border-radius)}.accordion-item:not(:first-of-type){border-top:0}.accordion-item:last-of-type{border-bottom-right-radius:var(--bs-accordion-border-radius);border-bottom-left-radius:var(--bs-accordion-border-radius)}.accordion-item:last-of-type .accordion-button.collapsed{border-bottom-right-radius:var(--bs-accordion-inner-border-radius);border-bottom-left-radius:var(--bs-accordion-inner-border-radius)}.accordion-item:last-of-type .accordion-collapse{border-bottom-right-radius:var(--bs-accordion-border-radius);border-bottom-left-radius:var(--bs-accordion-border-radius)}.accordion-body{padding:var(--bs-accordion-body-padding-y) var(--bs-accordion-body-padding-x)}.accordion-flush .accordion-collapse{border-width:0}.accordion-flush .accordion-item{border-right:0;border-left:0;border-radius:0}.accordion-flush .accordion-item:first-child{border-top:0}.accordion-flush .accordion-item:last-child{border-bottom:0}.accordion-flush .accordion-item .accordion-button,.accordion-flush .accordion-item .accordion-button.collapsed{border-radius:0}[data-bs-theme=dark] .accordion-button::after{--bs-accordion-btn-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23879cb2'%3e%3cpath fill-rule='evenodd' d='M1.646 4.646a.5.5 0 0 1 .708 0L8 10.293l5.646-5.647a.5.5 0 0 1 .708.708l-6 6a.5.5 0 0 1-.708 0l-6-6a.5.5 0 0 1 0-.708z'/%3e%3c/svg%3e");--bs-accordion-btn-active-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23879cb2'%3e%3cpath fill-rule='evenodd' d='M1.646 4.646a.5.5 0 0 1 .708 0L8 10.293l5.646-5.647a.5.5 0 0 1 .708.708l-6 6a.5.5 0 0 1-.708 0l-6-6a.5.5 0 0 1 0-.708z'/%3e%3c/svg%3e")}.breadcrumb{--bs-breadcrumb-padding-x: 0.75rem;--bs-breadcrumb-padding-y: 0.375rem;--bs-breadcrumb-margin-bottom: 1rem;--bs-breadcrumb-bg: #434343;--bs-breadcrumb-border-radius: 0.25rem;--bs-breadcrumb-divider-color: rgba(255, 255, 255, 0.75);--bs-breadcrumb-item-padding-x: 0.5rem;--bs-breadcrumb-item-active-color: rgba(255, 255, 255, 0.75);display:flex;display:-webkit-flex;flex-wrap:wrap;-webkit-flex-wrap:wrap;padding:var(--bs-breadcrumb-padding-y) var(--bs-breadcrumb-padding-x);margin-bottom:var(--bs-breadcrumb-margin-bottom);font-size:var(--bs-breadcrumb-font-size);list-style:none;background-color:var(--bs-breadcrumb-bg);border-radius:var(--bs-breadcrumb-border-radius)}.breadcrumb-item+.breadcrumb-item{padding-left:var(--bs-breadcrumb-item-padding-x)}.breadcrumb-item+.breadcrumb-item::before{float:left;padding-right:var(--bs-breadcrumb-item-padding-x);color:var(--bs-breadcrumb-divider-color);content:var(--bs-breadcrumb-divider, ">") /* rtl: var(--bs-breadcrumb-divider, ">") */}.breadcrumb-item.active{color:var(--bs-breadcrumb-item-active-color)}.pagination{--bs-pagination-padding-x: 0.75rem;--bs-pagination-padding-y: 0.375rem;--bs-pagination-font-size:1rem;--bs-pagination-color: #fff;--bs-pagination-bg: #00bc8c;--bs-pagination-border-width: 0;--bs-pagination-border-color: transparent;--bs-pagination-border-radius: 0.25rem;--bs-pagination-hover-color: #fff;--bs-pagination-hover-bg: #00efb2;--bs-pagination-hover-border-color: transparent;--bs-pagination-focus-color: #009670;--bs-pagination-focus-bg: #ebebeb;--bs-pagination-focus-box-shadow: 0 0 0 0.25rem rgba(55, 90, 127, 0.25);--bs-pagination-active-color: #fff;--bs-pagination-active-bg: #00efb2;--bs-pagination-active-border-color: transparent;--bs-pagination-disabled-color: #fff;--bs-pagination-disabled-bg: #007053;--bs-pagination-disabled-border-color: transparent;display:flex;display:-webkit-flex;padding-left:0;list-style:none}.page-link{position:relative;display:block;padding:var(--bs-pagination-padding-y) var(--bs-pagination-padding-x);font-size:var(--bs-pagination-font-size);color:var(--bs-pagination-color);text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;background-color:var(--bs-pagination-bg);border:var(--bs-pagination-border-width) solid var(--bs-pagination-border-color);transition:color .15s ease-in-out,background-color .15s ease-in-out,border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.page-link{transition:none}}.page-link:hover{z-index:2;color:var(--bs-pagination-hover-color);background-color:var(--bs-pagination-hover-bg);border-color:var(--bs-pagination-hover-border-color)}.page-link:focus{z-index:3;color:var(--bs-pagination-focus-color);background-color:var(--bs-pagination-focus-bg);outline:0;box-shadow:var(--bs-pagination-focus-box-shadow)}.page-link.active,.active>.page-link{z-index:3;color:var(--bs-pagination-active-color);background-color:var(--bs-pagination-active-bg);border-color:var(--bs-pagination-active-border-color)}.page-link.disabled,.disabled>.page-link{color:var(--bs-pagination-disabled-color);pointer-events:none;background-color:var(--bs-pagination-disabled-bg);border-color:var(--bs-pagination-disabled-border-color)}.page-item:not(:first-child) .page-link{margin-left:calc(0*-1)}.page-item:first-child .page-link{border-top-left-radius:var(--bs-pagination-border-radius);border-bottom-left-radius:var(--bs-pagination-border-radius)}.page-item:last-child .page-link{border-top-right-radius:var(--bs-pagination-border-radius);border-bottom-right-radius:var(--bs-pagination-border-radius)}.pagination-lg{--bs-pagination-padding-x: 1.5rem;--bs-pagination-padding-y: 0.75rem;--bs-pagination-font-size:1.25rem;--bs-pagination-border-radius: 0.5rem}.pagination-sm{--bs-pagination-padding-x: 0.5rem;--bs-pagination-padding-y: 0.25rem;--bs-pagination-font-size:0.875rem;--bs-pagination-border-radius: 0.2em}.badge{--bs-badge-padding-x: 0.65em;--bs-badge-padding-y: 0.35em;--bs-badge-font-size:0.75em;--bs-badge-font-weight: 700;--bs-badge-color: #fff;--bs-badge-border-radius: 0.25rem;display:inline-block;padding:var(--bs-badge-padding-y) var(--bs-badge-padding-x);font-size:var(--bs-badge-font-size);font-weight:var(--bs-badge-font-weight);line-height:1;color:var(--bs-badge-color);text-align:center;white-space:nowrap;vertical-align:baseline;border-radius:var(--bs-badge-border-radius)}.badge:empty{display:none}.btn .badge{position:relative;top:-1px}.alert{--bs-alert-bg: transparent;--bs-alert-padding-x: 1rem;--bs-alert-padding-y: 1rem;--bs-alert-margin-bottom: 1rem;--bs-alert-color: inherit;--bs-alert-border-color: transparent;--bs-alert-border: 1px solid var(--bs-alert-border-color);--bs-alert-border-radius: 0.25rem;--bs-alert-link-color: inherit;position:relative;padding:var(--bs-alert-padding-y) var(--bs-alert-padding-x);margin-bottom:var(--bs-alert-margin-bottom);color:var(--bs-alert-color);background-color:var(--bs-alert-bg);border:var(--bs-alert-border);border-radius:var(--bs-alert-border-radius)}.alert-heading{color:inherit}.alert-link{font-weight:700;color:var(--bs-alert-link-color)}.alert-dismissible{padding-right:3rem}.alert-dismissible .btn-close{position:absolute;top:0;right:0;z-index:2;padding:1.25rem 1rem}.alert-default{--bs-alert-color: var(--bs-default-text-emphasis);--bs-alert-bg: var(--bs-default-bg-subtle);--bs-alert-border-color: var(--bs-default-border-subtle);--bs-alert-link-color: var(--bs-default-text-emphasis)}.alert-primary{--bs-alert-color: var(--bs-primary-text-emphasis);--bs-alert-bg: var(--bs-primary-bg-subtle);--bs-alert-border-color: var(--bs-primary-border-subtle);--bs-alert-link-color: var(--bs-primary-text-emphasis)}.alert-secondary{--bs-alert-color: var(--bs-secondary-text-emphasis);--bs-alert-bg: var(--bs-secondary-bg-subtle);--bs-alert-border-color: var(--bs-secondary-border-subtle);--bs-alert-link-color: var(--bs-secondary-text-emphasis)}.alert-success{--bs-alert-color: var(--bs-success-text-emphasis);--bs-alert-bg: var(--bs-success-bg-subtle);--bs-alert-border-color: var(--bs-success-border-subtle);--bs-alert-link-color: var(--bs-success-text-emphasis)}.alert-info{--bs-alert-color: var(--bs-info-text-emphasis);--bs-alert-bg: var(--bs-info-bg-subtle);--bs-alert-border-color: var(--bs-info-border-subtle);--bs-alert-link-color: var(--bs-info-text-emphasis)}.alert-warning{--bs-alert-color: var(--bs-warning-text-emphasis);--bs-alert-bg: var(--bs-warning-bg-subtle);--bs-alert-border-color: var(--bs-warning-border-subtle);--bs-alert-link-color: var(--bs-warning-text-emphasis)}.alert-danger{--bs-alert-color: var(--bs-danger-text-emphasis);--bs-alert-bg: var(--bs-danger-bg-subtle);--bs-alert-border-color: var(--bs-danger-border-subtle);--bs-alert-link-color: var(--bs-danger-text-emphasis)}.alert-light{--bs-alert-color: var(--bs-light-text-emphasis);--bs-alert-bg: var(--bs-light-bg-subtle);--bs-alert-border-color: var(--bs-light-border-subtle);--bs-alert-link-color: var(--bs-light-text-emphasis)}.alert-dark{--bs-alert-color: var(--bs-dark-text-emphasis);--bs-alert-bg: var(--bs-dark-bg-subtle);--bs-alert-border-color: var(--bs-dark-border-subtle);--bs-alert-link-color: var(--bs-dark-text-emphasis)}@keyframes progress-bar-stripes{0%{background-position-x:1rem}}.progress,.progress-stacked{--bs-progress-height: 1rem;--bs-progress-font-size:0.75rem;--bs-progress-bg: #434343;--bs-progress-border-radius: 0.25rem;--bs-progress-box-shadow: inset 0 1px 2px rgba(0, 0, 0, 0.075);--bs-progress-bar-color: #fff;--bs-progress-bar-bg: #375a7f;--bs-progress-bar-transition: width 0.6s ease;display:flex;display:-webkit-flex;height:var(--bs-progress-height);overflow:hidden;font-size:var(--bs-progress-font-size);background-color:var(--bs-progress-bg);border-radius:var(--bs-progress-border-radius)}.progress-bar{display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;justify-content:center;-webkit-justify-content:center;overflow:hidden;color:var(--bs-progress-bar-color);text-align:center;white-space:nowrap;background-color:var(--bs-progress-bar-bg);transition:var(--bs-progress-bar-transition)}@media(prefers-reduced-motion: reduce){.progress-bar{transition:none}}.progress-bar-striped{background-image:linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);background-size:var(--bs-progress-height) var(--bs-progress-height)}.progress-stacked>.progress{overflow:visible}.progress-stacked>.progress>.progress-bar{width:100%}.progress-bar-animated{animation:1s linear infinite progress-bar-stripes}@media(prefers-reduced-motion: reduce){.progress-bar-animated{animation:none}}.list-group{--bs-list-group-color: #fff;--bs-list-group-bg: #2d2d2d;--bs-list-group-border-color: #434343;--bs-list-group-border-width: 1px;--bs-list-group-border-radius: 0.25rem;--bs-list-group-item-padding-x: 1rem;--bs-list-group-item-padding-y: 0.5rem;--bs-list-group-action-color: rgba(255, 255, 255, 0.75);--bs-list-group-action-hover-color: #fff;--bs-list-group-action-hover-bg: #434343;--bs-list-group-action-active-color: #fff;--bs-list-group-action-active-bg: #222;--bs-list-group-disabled-color: rgba(255, 255, 255, 0.75);--bs-list-group-disabled-bg: #2d2d2d;--bs-list-group-active-color: #fff;--bs-list-group-active-bg: #375a7f;--bs-list-group-active-border-color: #375a7f;display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;padding-left:0;margin-bottom:0;border-radius:var(--bs-list-group-border-radius)}.list-group-numbered{list-style-type:none;counter-reset:section}.list-group-numbered>.list-group-item::before{content:counters(section, ".") ". ";counter-increment:section}.list-group-item-action{width:100%;color:var(--bs-list-group-action-color);text-align:inherit}.list-group-item-action:hover,.list-group-item-action:focus{z-index:1;color:var(--bs-list-group-action-hover-color);text-decoration:none;background-color:var(--bs-list-group-action-hover-bg)}.list-group-item-action:active{color:var(--bs-list-group-action-active-color);background-color:var(--bs-list-group-action-active-bg)}.list-group-item{position:relative;display:block;padding:var(--bs-list-group-item-padding-y) var(--bs-list-group-item-padding-x);color:var(--bs-list-group-color);text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;background-color:var(--bs-list-group-bg);border:var(--bs-list-group-border-width) solid var(--bs-list-group-border-color)}.list-group-item:first-child{border-top-left-radius:inherit;border-top-right-radius:inherit}.list-group-item:last-child{border-bottom-right-radius:inherit;border-bottom-left-radius:inherit}.list-group-item.disabled,.list-group-item:disabled{color:var(--bs-list-group-disabled-color);pointer-events:none;background-color:var(--bs-list-group-disabled-bg)}.list-group-item.active{z-index:2;color:var(--bs-list-group-active-color);background-color:var(--bs-list-group-active-bg);border-color:var(--bs-list-group-active-border-color)}.list-group-item+.list-group-item{border-top-width:0}.list-group-item+.list-group-item.active{margin-top:calc(-1*var(--bs-list-group-border-width));border-top-width:var(--bs-list-group-border-width)}.list-group-horizontal{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal>.list-group-item.active{margin-top:0}.list-group-horizontal>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}@media(min-width: 576px){.list-group-horizontal-sm{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal-sm>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal-sm>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal-sm>.list-group-item.active{margin-top:0}.list-group-horizontal-sm>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal-sm>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}}@media(min-width: 768px){.list-group-horizontal-md{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal-md>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal-md>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal-md>.list-group-item.active{margin-top:0}.list-group-horizontal-md>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal-md>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}}@media(min-width: 992px){.list-group-horizontal-lg{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal-lg>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal-lg>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal-lg>.list-group-item.active{margin-top:0}.list-group-horizontal-lg>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal-lg>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}}@media(min-width: 1200px){.list-group-horizontal-xl{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal-xl>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal-xl>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal-xl>.list-group-item.active{margin-top:0}.list-group-horizontal-xl>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal-xl>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}}@media(min-width: 1400px){.list-group-horizontal-xxl{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal-xxl>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal-xxl>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal-xxl>.list-group-item.active{margin-top:0}.list-group-horizontal-xxl>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal-xxl>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}}.list-group-flush{border-radius:0}.list-group-flush>.list-group-item{border-width:0 0 var(--bs-list-group-border-width)}.list-group-flush>.list-group-item:last-child{border-bottom-width:0}.list-group-item-default{--bs-list-group-color: var(--bs-default-text-emphasis);--bs-list-group-bg: var(--bs-default-bg-subtle);--bs-list-group-border-color: var(--bs-default-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-default-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-default-border-subtle);--bs-list-group-active-color: var(--bs-default-bg-subtle);--bs-list-group-active-bg: var(--bs-default-text-emphasis);--bs-list-group-active-border-color: var(--bs-default-text-emphasis)}.list-group-item-primary{--bs-list-group-color: var(--bs-primary-text-emphasis);--bs-list-group-bg: var(--bs-primary-bg-subtle);--bs-list-group-border-color: var(--bs-primary-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-primary-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-primary-border-subtle);--bs-list-group-active-color: var(--bs-primary-bg-subtle);--bs-list-group-active-bg: var(--bs-primary-text-emphasis);--bs-list-group-active-border-color: var(--bs-primary-text-emphasis)}.list-group-item-secondary{--bs-list-group-color: var(--bs-secondary-text-emphasis);--bs-list-group-bg: var(--bs-secondary-bg-subtle);--bs-list-group-border-color: var(--bs-secondary-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-secondary-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-secondary-border-subtle);--bs-list-group-active-color: var(--bs-secondary-bg-subtle);--bs-list-group-active-bg: var(--bs-secondary-text-emphasis);--bs-list-group-active-border-color: var(--bs-secondary-text-emphasis)}.list-group-item-success{--bs-list-group-color: var(--bs-success-text-emphasis);--bs-list-group-bg: var(--bs-success-bg-subtle);--bs-list-group-border-color: var(--bs-success-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-success-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-success-border-subtle);--bs-list-group-active-color: var(--bs-success-bg-subtle);--bs-list-group-active-bg: var(--bs-success-text-emphasis);--bs-list-group-active-border-color: var(--bs-success-text-emphasis)}.list-group-item-info{--bs-list-group-color: var(--bs-info-text-emphasis);--bs-list-group-bg: var(--bs-info-bg-subtle);--bs-list-group-border-color: var(--bs-info-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-info-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-info-border-subtle);--bs-list-group-active-color: var(--bs-info-bg-subtle);--bs-list-group-active-bg: var(--bs-info-text-emphasis);--bs-list-group-active-border-color: var(--bs-info-text-emphasis)}.list-group-item-warning{--bs-list-group-color: var(--bs-warning-text-emphasis);--bs-list-group-bg: var(--bs-warning-bg-subtle);--bs-list-group-border-color: var(--bs-warning-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-warning-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-warning-border-subtle);--bs-list-group-active-color: var(--bs-warning-bg-subtle);--bs-list-group-active-bg: var(--bs-warning-text-emphasis);--bs-list-group-active-border-color: var(--bs-warning-text-emphasis)}.list-group-item-danger{--bs-list-group-color: var(--bs-danger-text-emphasis);--bs-list-group-bg: var(--bs-danger-bg-subtle);--bs-list-group-border-color: var(--bs-danger-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-danger-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-danger-border-subtle);--bs-list-group-active-color: var(--bs-danger-bg-subtle);--bs-list-group-active-bg: var(--bs-danger-text-emphasis);--bs-list-group-active-border-color: var(--bs-danger-text-emphasis)}.list-group-item-light{--bs-list-group-color: var(--bs-light-text-emphasis);--bs-list-group-bg: var(--bs-light-bg-subtle);--bs-list-group-border-color: var(--bs-light-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-light-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-light-border-subtle);--bs-list-group-active-color: var(--bs-light-bg-subtle);--bs-list-group-active-bg: var(--bs-light-text-emphasis);--bs-list-group-active-border-color: var(--bs-light-text-emphasis)}.list-group-item-dark{--bs-list-group-color: var(--bs-dark-text-emphasis);--bs-list-group-bg: var(--bs-dark-bg-subtle);--bs-list-group-border-color: var(--bs-dark-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-dark-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-dark-border-subtle);--bs-list-group-active-color: var(--bs-dark-bg-subtle);--bs-list-group-active-bg: var(--bs-dark-text-emphasis);--bs-list-group-active-border-color: var(--bs-dark-text-emphasis)}.btn-close{--bs-btn-close-color: #fff;--bs-btn-close-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23fff'%3e%3cpath d='M.293.293a1 1 0 0 1 1.414 0L8 6.586 14.293.293a1 1 0 1 1 1.414 1.414L9.414 8l6.293 6.293a1 1 0 0 1-1.414 1.414L8 9.414l-6.293 6.293a1 1 0 0 1-1.414-1.414L6.586 8 .293 1.707a1 1 0 0 1 0-1.414z'/%3e%3c/svg%3e");--bs-btn-close-opacity: 0.4;--bs-btn-close-hover-opacity: 1;--bs-btn-close-focus-shadow: 0 0 0 0.25rem rgba(55, 90, 127, 0.25);--bs-btn-close-focus-opacity: 1;--bs-btn-close-disabled-opacity: 0.25;--bs-btn-close-white-filter: invert(1) grayscale(100%) brightness(200%);box-sizing:content-box;width:1em;height:1em;padding:.25em .25em;color:var(--bs-btn-close-color);background:rgba(0,0,0,0) var(--bs-btn-close-bg) center/1em auto no-repeat;border:0;border-radius:.25rem;opacity:var(--bs-btn-close-opacity)}.btn-close:hover{color:var(--bs-btn-close-color);text-decoration:none;opacity:var(--bs-btn-close-hover-opacity)}.btn-close:focus{outline:0;box-shadow:var(--bs-btn-close-focus-shadow);opacity:var(--bs-btn-close-focus-opacity)}.btn-close:disabled,.btn-close.disabled{pointer-events:none;user-select:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;-o-user-select:none;opacity:var(--bs-btn-close-disabled-opacity)}.btn-close-white{filter:var(--bs-btn-close-white-filter)}[data-bs-theme=dark] .btn-close{filter:var(--bs-btn-close-white-filter)}.toast{--bs-toast-zindex: 1090;--bs-toast-padding-x: 0.75rem;--bs-toast-padding-y: 0.5rem;--bs-toast-spacing: 1.5rem;--bs-toast-max-width: 350px;--bs-toast-font-size:0.875rem;--bs-toast-color: ;--bs-toast-bg: #434343;--bs-toast-border-width: 1px;--bs-toast-border-color: rgba(0, 0, 0, 0.175);--bs-toast-border-radius: 0.25rem;--bs-toast-box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);--bs-toast-header-color: rgba(255, 255, 255, 0.75);--bs-toast-header-bg: #2d2d2d;--bs-toast-header-border-color: rgba(0, 0, 0, 0.175);width:var(--bs-toast-max-width);max-width:100%;font-size:var(--bs-toast-font-size);color:var(--bs-toast-color);pointer-events:auto;background-color:var(--bs-toast-bg);background-clip:padding-box;border:var(--bs-toast-border-width) solid var(--bs-toast-border-color);box-shadow:var(--bs-toast-box-shadow);border-radius:var(--bs-toast-border-radius)}.toast.showing{opacity:0}.toast:not(.show){display:none}.toast-container{--bs-toast-zindex: 1090;position:absolute;z-index:var(--bs-toast-zindex);width:max-content;width:-webkit-max-content;width:-moz-max-content;width:-ms-max-content;width:-o-max-content;max-width:100%;pointer-events:none}.toast-container>:not(:last-child){margin-bottom:var(--bs-toast-spacing)}.toast-header{display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;padding:var(--bs-toast-padding-y) var(--bs-toast-padding-x);color:var(--bs-toast-header-color);background-color:var(--bs-toast-header-bg);background-clip:padding-box;border-bottom:var(--bs-toast-border-width) solid var(--bs-toast-header-border-color);border-top-left-radius:calc(var(--bs-toast-border-radius) - var(--bs-toast-border-width));border-top-right-radius:calc(var(--bs-toast-border-radius) - var(--bs-toast-border-width))}.toast-header .btn-close{margin-right:calc(-0.5*var(--bs-toast-padding-x));margin-left:var(--bs-toast-padding-x)}.toast-body{padding:var(--bs-toast-padding-x);word-wrap:break-word}.modal{--bs-modal-zindex: 1055;--bs-modal-width: 500px;--bs-modal-padding: 1rem;--bs-modal-margin: 0.5rem;--bs-modal-color: ;--bs-modal-bg: #2d2d2d;--bs-modal-border-color: #434343;--bs-modal-border-width: 1px;--bs-modal-border-radius: 0.5rem;--bs-modal-box-shadow: 0 0.125rem 0.25rem rgba(0, 0, 0, 0.075);--bs-modal-inner-border-radius: calc(0.5rem - 1px);--bs-modal-header-padding-x: 1rem;--bs-modal-header-padding-y: 1rem;--bs-modal-header-padding: 1rem 1rem;--bs-modal-header-border-color: #434343;--bs-modal-header-border-width: 1px;--bs-modal-title-line-height: 1.5;--bs-modal-footer-gap: 0.5rem;--bs-modal-footer-bg: ;--bs-modal-footer-border-color: #434343;--bs-modal-footer-border-width: 1px;position:fixed;top:0;left:0;z-index:var(--bs-modal-zindex);display:none;width:100%;height:100%;overflow-x:hidden;overflow-y:auto;outline:0}.modal-dialog{position:relative;width:auto;margin:var(--bs-modal-margin);pointer-events:none}.modal.fade .modal-dialog{transition:transform .3s ease-out;transform:translate(0, -50px)}@media(prefers-reduced-motion: reduce){.modal.fade .modal-dialog{transition:none}}.modal.show .modal-dialog{transform:none}.modal.modal-static .modal-dialog{transform:scale(1.02)}.modal-dialog-scrollable{height:calc(100% - var(--bs-modal-margin)*2)}.modal-dialog-scrollable .modal-content{max-height:100%;overflow:hidden}.modal-dialog-scrollable .modal-body{overflow-y:auto}.modal-dialog-centered{display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;min-height:calc(100% - var(--bs-modal-margin)*2)}.modal-content{position:relative;display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;width:100%;color:var(--bs-modal-color);pointer-events:auto;background-color:var(--bs-modal-bg);background-clip:padding-box;border:var(--bs-modal-border-width) solid var(--bs-modal-border-color);border-radius:var(--bs-modal-border-radius);outline:0}.modal-backdrop{--bs-backdrop-zindex: 1050;--bs-backdrop-bg: #000;--bs-backdrop-opacity: 0.5;position:fixed;top:0;left:0;z-index:var(--bs-backdrop-zindex);width:100vw;height:100vh;background-color:var(--bs-backdrop-bg)}.modal-backdrop.fade{opacity:0}.modal-backdrop.show{opacity:var(--bs-backdrop-opacity)}.modal-header{display:flex;display:-webkit-flex;flex-shrink:0;-webkit-flex-shrink:0;align-items:center;-webkit-align-items:center;justify-content:space-between;-webkit-justify-content:space-between;padding:var(--bs-modal-header-padding);border-bottom:var(--bs-modal-header-border-width) solid var(--bs-modal-header-border-color);border-top-left-radius:var(--bs-modal-inner-border-radius);border-top-right-radius:var(--bs-modal-inner-border-radius)}.modal-header .btn-close{padding:calc(var(--bs-modal-header-padding-y)*.5) calc(var(--bs-modal-header-padding-x)*.5);margin:calc(-0.5*var(--bs-modal-header-padding-y)) calc(-0.5*var(--bs-modal-header-padding-x)) calc(-0.5*var(--bs-modal-header-padding-y)) auto}.modal-title{margin-bottom:0;line-height:var(--bs-modal-title-line-height)}.modal-body{position:relative;flex:1 1 auto;-webkit-flex:1 1 auto;padding:var(--bs-modal-padding)}.modal-footer{display:flex;display:-webkit-flex;flex-shrink:0;-webkit-flex-shrink:0;flex-wrap:wrap;-webkit-flex-wrap:wrap;align-items:center;-webkit-align-items:center;justify-content:flex-end;-webkit-justify-content:flex-end;padding:calc(var(--bs-modal-padding) - var(--bs-modal-footer-gap)*.5);background-color:var(--bs-modal-footer-bg);border-top:var(--bs-modal-footer-border-width) solid var(--bs-modal-footer-border-color);border-bottom-right-radius:var(--bs-modal-inner-border-radius);border-bottom-left-radius:var(--bs-modal-inner-border-radius)}.modal-footer>*{margin:calc(var(--bs-modal-footer-gap)*.5)}@media(min-width: 576px){.modal{--bs-modal-margin: 1.75rem;--bs-modal-box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15)}.modal-dialog{max-width:var(--bs-modal-width);margin-right:auto;margin-left:auto}.modal-sm{--bs-modal-width: 300px}}@media(min-width: 992px){.modal-lg,.modal-xl{--bs-modal-width: 800px}}@media(min-width: 1200px){.modal-xl{--bs-modal-width: 1140px}}.modal-fullscreen{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen .modal-header,.modal-fullscreen .modal-footer{border-radius:0}.modal-fullscreen .modal-body{overflow-y:auto}@media(max-width: 575.98px){.modal-fullscreen-sm-down{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen-sm-down .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen-sm-down .modal-header,.modal-fullscreen-sm-down .modal-footer{border-radius:0}.modal-fullscreen-sm-down .modal-body{overflow-y:auto}}@media(max-width: 767.98px){.modal-fullscreen-md-down{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen-md-down .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen-md-down .modal-header,.modal-fullscreen-md-down .modal-footer{border-radius:0}.modal-fullscreen-md-down .modal-body{overflow-y:auto}}@media(max-width: 991.98px){.modal-fullscreen-lg-down{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen-lg-down .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen-lg-down .modal-header,.modal-fullscreen-lg-down .modal-footer{border-radius:0}.modal-fullscreen-lg-down .modal-body{overflow-y:auto}}@media(max-width: 1199.98px){.modal-fullscreen-xl-down{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen-xl-down .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen-xl-down .modal-header,.modal-fullscreen-xl-down .modal-footer{border-radius:0}.modal-fullscreen-xl-down .modal-body{overflow-y:auto}}@media(max-width: 1399.98px){.modal-fullscreen-xxl-down{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen-xxl-down .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen-xxl-down .modal-header,.modal-fullscreen-xxl-down .modal-footer{border-radius:0}.modal-fullscreen-xxl-down .modal-body{overflow-y:auto}}.tooltip{--bs-tooltip-zindex: 1080;--bs-tooltip-max-width: 200px;--bs-tooltip-padding-x: 0.5rem;--bs-tooltip-padding-y: 0.25rem;--bs-tooltip-margin: ;--bs-tooltip-font-size:0.875rem;--bs-tooltip-color: #222;--bs-tooltip-bg: #000;--bs-tooltip-border-radius: 0.25rem;--bs-tooltip-opacity: 0.9;--bs-tooltip-arrow-width: 0.8rem;--bs-tooltip-arrow-height: 0.4rem;z-index:var(--bs-tooltip-zindex);display:block;margin:var(--bs-tooltip-margin);font-family:Lato,-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";font-style:normal;font-weight:400;line-height:1.5;text-align:left;text-align:start;text-decoration:none;text-shadow:none;text-transform:none;letter-spacing:normal;word-break:normal;white-space:normal;word-spacing:normal;line-break:auto;font-size:var(--bs-tooltip-font-size);word-wrap:break-word;opacity:0}.tooltip.show{opacity:var(--bs-tooltip-opacity)}.tooltip .tooltip-arrow{display:block;width:var(--bs-tooltip-arrow-width);height:var(--bs-tooltip-arrow-height)}.tooltip .tooltip-arrow::before{position:absolute;content:"";border-color:rgba(0,0,0,0);border-style:solid}.bs-tooltip-top .tooltip-arrow,.bs-tooltip-auto[data-popper-placement^=top] .tooltip-arrow{bottom:calc(-1*var(--bs-tooltip-arrow-height))}.bs-tooltip-top .tooltip-arrow::before,.bs-tooltip-auto[data-popper-placement^=top] .tooltip-arrow::before{top:-1px;border-width:var(--bs-tooltip-arrow-height) calc(var(--bs-tooltip-arrow-width)*.5) 0;border-top-color:var(--bs-tooltip-bg)}.bs-tooltip-end .tooltip-arrow,.bs-tooltip-auto[data-popper-placement^=right] .tooltip-arrow{left:calc(-1*var(--bs-tooltip-arrow-height));width:var(--bs-tooltip-arrow-height);height:var(--bs-tooltip-arrow-width)}.bs-tooltip-end .tooltip-arrow::before,.bs-tooltip-auto[data-popper-placement^=right] .tooltip-arrow::before{right:-1px;border-width:calc(var(--bs-tooltip-arrow-width)*.5) var(--bs-tooltip-arrow-height) calc(var(--bs-tooltip-arrow-width)*.5) 0;border-right-color:var(--bs-tooltip-bg)}.bs-tooltip-bottom .tooltip-arrow,.bs-tooltip-auto[data-popper-placement^=bottom] .tooltip-arrow{top:calc(-1*var(--bs-tooltip-arrow-height))}.bs-tooltip-bottom .tooltip-arrow::before,.bs-tooltip-auto[data-popper-placement^=bottom] .tooltip-arrow::before{bottom:-1px;border-width:0 calc(var(--bs-tooltip-arrow-width)*.5) var(--bs-tooltip-arrow-height);border-bottom-color:var(--bs-tooltip-bg)}.bs-tooltip-start .tooltip-arrow,.bs-tooltip-auto[data-popper-placement^=left] .tooltip-arrow{right:calc(-1*var(--bs-tooltip-arrow-height));width:var(--bs-tooltip-arrow-height);height:var(--bs-tooltip-arrow-width)}.bs-tooltip-start .tooltip-arrow::before,.bs-tooltip-auto[data-popper-placement^=left] .tooltip-arrow::before{left:-1px;border-width:calc(var(--bs-tooltip-arrow-width)*.5) 0 calc(var(--bs-tooltip-arrow-width)*.5) var(--bs-tooltip-arrow-height);border-left-color:var(--bs-tooltip-bg)}.tooltip-inner{max-width:var(--bs-tooltip-max-width);padding:var(--bs-tooltip-padding-y) var(--bs-tooltip-padding-x);color:var(--bs-tooltip-color);text-align:center;background-color:var(--bs-tooltip-bg);border-radius:var(--bs-tooltip-border-radius)}.popover{--bs-popover-zindex: 1070;--bs-popover-max-width: 276px;--bs-popover-font-size:0.875rem;--bs-popover-bg: #2d2d2d;--bs-popover-border-width: 1px;--bs-popover-border-color: rgba(0, 0, 0, 0.175);--bs-popover-border-radius: 0.5rem;--bs-popover-inner-border-radius: calc(0.5rem - 1px);--bs-popover-box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);--bs-popover-header-padding-x: 1rem;--bs-popover-header-padding-y: 0.5rem;--bs-popover-header-font-size:1rem;--bs-popover-header-color: inherit;--bs-popover-header-bg: #434343;--bs-popover-body-padding-x: 1rem;--bs-popover-body-padding-y: 1rem;--bs-popover-body-color: #fff;--bs-popover-arrow-width: 1rem;--bs-popover-arrow-height: 0.5rem;--bs-popover-arrow-border: var(--bs-popover-border-color);z-index:var(--bs-popover-zindex);display:block;max-width:var(--bs-popover-max-width);font-family:Lato,-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";font-style:normal;font-weight:400;line-height:1.5;text-align:left;text-align:start;text-decoration:none;text-shadow:none;text-transform:none;letter-spacing:normal;word-break:normal;white-space:normal;word-spacing:normal;line-break:auto;font-size:var(--bs-popover-font-size);word-wrap:break-word;background-color:var(--bs-popover-bg);background-clip:padding-box;border:var(--bs-popover-border-width) solid var(--bs-popover-border-color);border-radius:var(--bs-popover-border-radius)}.popover .popover-arrow{display:block;width:var(--bs-popover-arrow-width);height:var(--bs-popover-arrow-height)}.popover .popover-arrow::before,.popover .popover-arrow::after{position:absolute;display:block;content:"";border-color:rgba(0,0,0,0);border-style:solid;border-width:0}.bs-popover-top>.popover-arrow,.bs-popover-auto[data-popper-placement^=top]>.popover-arrow{bottom:calc(-1*(var(--bs-popover-arrow-height)) - var(--bs-popover-border-width))}.bs-popover-top>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=top]>.popover-arrow::before,.bs-popover-top>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=top]>.popover-arrow::after{border-width:var(--bs-popover-arrow-height) calc(var(--bs-popover-arrow-width)*.5) 0}.bs-popover-top>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=top]>.popover-arrow::before{bottom:0;border-top-color:var(--bs-popover-arrow-border)}.bs-popover-top>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=top]>.popover-arrow::after{bottom:var(--bs-popover-border-width);border-top-color:var(--bs-popover-bg)}.bs-popover-end>.popover-arrow,.bs-popover-auto[data-popper-placement^=right]>.popover-arrow{left:calc(-1*(var(--bs-popover-arrow-height)) - var(--bs-popover-border-width));width:var(--bs-popover-arrow-height);height:var(--bs-popover-arrow-width)}.bs-popover-end>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=right]>.popover-arrow::before,.bs-popover-end>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=right]>.popover-arrow::after{border-width:calc(var(--bs-popover-arrow-width)*.5) var(--bs-popover-arrow-height) calc(var(--bs-popover-arrow-width)*.5) 0}.bs-popover-end>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=right]>.popover-arrow::before{left:0;border-right-color:var(--bs-popover-arrow-border)}.bs-popover-end>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=right]>.popover-arrow::after{left:var(--bs-popover-border-width);border-right-color:var(--bs-popover-bg)}.bs-popover-bottom>.popover-arrow,.bs-popover-auto[data-popper-placement^=bottom]>.popover-arrow{top:calc(-1*(var(--bs-popover-arrow-height)) - var(--bs-popover-border-width))}.bs-popover-bottom>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=bottom]>.popover-arrow::before,.bs-popover-bottom>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=bottom]>.popover-arrow::after{border-width:0 calc(var(--bs-popover-arrow-width)*.5) var(--bs-popover-arrow-height)}.bs-popover-bottom>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=bottom]>.popover-arrow::before{top:0;border-bottom-color:var(--bs-popover-arrow-border)}.bs-popover-bottom>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=bottom]>.popover-arrow::after{top:var(--bs-popover-border-width);border-bottom-color:var(--bs-popover-bg)}.bs-popover-bottom .popover-header::before,.bs-popover-auto[data-popper-placement^=bottom] .popover-header::before{position:absolute;top:0;left:50%;display:block;width:var(--bs-popover-arrow-width);margin-left:calc(-0.5*var(--bs-popover-arrow-width));content:"";border-bottom:var(--bs-popover-border-width) solid var(--bs-popover-header-bg)}.bs-popover-start>.popover-arrow,.bs-popover-auto[data-popper-placement^=left]>.popover-arrow{right:calc(-1*(var(--bs-popover-arrow-height)) - var(--bs-popover-border-width));width:var(--bs-popover-arrow-height);height:var(--bs-popover-arrow-width)}.bs-popover-start>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=left]>.popover-arrow::before,.bs-popover-start>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=left]>.popover-arrow::after{border-width:calc(var(--bs-popover-arrow-width)*.5) 0 calc(var(--bs-popover-arrow-width)*.5) var(--bs-popover-arrow-height)}.bs-popover-start>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=left]>.popover-arrow::before{right:0;border-left-color:var(--bs-popover-arrow-border)}.bs-popover-start>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=left]>.popover-arrow::after{right:var(--bs-popover-border-width);border-left-color:var(--bs-popover-bg)}.popover-header{padding:var(--bs-popover-header-padding-y) var(--bs-popover-header-padding-x);margin-bottom:0;font-size:var(--bs-popover-header-font-size);color:var(--bs-popover-header-color);background-color:var(--bs-popover-header-bg);border-bottom:var(--bs-popover-border-width) solid var(--bs-popover-border-color);border-top-left-radius:var(--bs-popover-inner-border-radius);border-top-right-radius:var(--bs-popover-inner-border-radius)}.popover-header:empty{display:none}.popover-body{padding:var(--bs-popover-body-padding-y) var(--bs-popover-body-padding-x);color:var(--bs-popover-body-color)}.carousel{position:relative}.carousel.pointer-event{touch-action:pan-y;-webkit-touch-action:pan-y;-moz-touch-action:pan-y;-ms-touch-action:pan-y;-o-touch-action:pan-y}.carousel-inner{position:relative;width:100%;overflow:hidden}.carousel-inner::after{display:block;clear:both;content:""}.carousel-item{position:relative;display:none;float:left;width:100%;margin-right:-100%;backface-visibility:hidden;-webkit-backface-visibility:hidden;-moz-backface-visibility:hidden;-ms-backface-visibility:hidden;-o-backface-visibility:hidden;transition:transform .6s ease-in-out}@media(prefers-reduced-motion: reduce){.carousel-item{transition:none}}.carousel-item.active,.carousel-item-next,.carousel-item-prev{display:block}.carousel-item-next:not(.carousel-item-start),.active.carousel-item-end{transform:translateX(100%)}.carousel-item-prev:not(.carousel-item-end),.active.carousel-item-start{transform:translateX(-100%)}.carousel-fade .carousel-item{opacity:0;transition-property:opacity;transform:none}.carousel-fade .carousel-item.active,.carousel-fade .carousel-item-next.carousel-item-start,.carousel-fade .carousel-item-prev.carousel-item-end{z-index:1;opacity:1}.carousel-fade .active.carousel-item-start,.carousel-fade .active.carousel-item-end{z-index:0;opacity:0;transition:opacity 0s .6s}@media(prefers-reduced-motion: reduce){.carousel-fade .active.carousel-item-start,.carousel-fade .active.carousel-item-end{transition:none}}.carousel-control-prev,.carousel-control-next{position:absolute;top:0;bottom:0;z-index:1;display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;justify-content:center;-webkit-justify-content:center;width:15%;padding:0;color:#fff;text-align:center;background:none;border:0;opacity:.5;transition:opacity .15s ease}@media(prefers-reduced-motion: reduce){.carousel-control-prev,.carousel-control-next{transition:none}}.carousel-control-prev:hover,.carousel-control-prev:focus,.carousel-control-next:hover,.carousel-control-next:focus{color:#fff;text-decoration:none;outline:0;opacity:.9}.carousel-control-prev{left:0}.carousel-control-next{right:0}.carousel-control-prev-icon,.carousel-control-next-icon{display:inline-block;width:2rem;height:2rem;background-repeat:no-repeat;background-position:50%;background-size:100% 100%}.carousel-control-prev-icon{background-image:url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23fff'%3e%3cpath d='M11.354 1.646a.5.5 0 0 1 0 .708L5.707 8l5.647 5.646a.5.5 0 0 1-.708.708l-6-6a.5.5 0 0 1 0-.708l6-6a.5.5 0 0 1 .708 0z'/%3e%3c/svg%3e")}.carousel-control-next-icon{background-image:url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23fff'%3e%3cpath d='M4.646 1.646a.5.5 0 0 1 .708 0l6 6a.5.5 0 0 1 0 .708l-6 6a.5.5 0 0 1-.708-.708L10.293 8 4.646 2.354a.5.5 0 0 1 0-.708z'/%3e%3c/svg%3e")}.carousel-indicators{position:absolute;right:0;bottom:0;left:0;z-index:2;display:flex;display:-webkit-flex;justify-content:center;-webkit-justify-content:center;padding:0;margin-right:15%;margin-bottom:1rem;margin-left:15%}.carousel-indicators [data-bs-target]{box-sizing:content-box;flex:0 1 auto;-webkit-flex:0 1 auto;width:30px;height:3px;padding:0;margin-right:3px;margin-left:3px;text-indent:-999px;cursor:pointer;background-color:#fff;background-clip:padding-box;border:0;border-top:10px solid rgba(0,0,0,0);border-bottom:10px solid rgba(0,0,0,0);opacity:.5;transition:opacity .6s ease}@media(prefers-reduced-motion: reduce){.carousel-indicators [data-bs-target]{transition:none}}.carousel-indicators .active{opacity:1}.carousel-caption{position:absolute;right:15%;bottom:1.25rem;left:15%;padding-top:1.25rem;padding-bottom:1.25rem;color:#fff;text-align:center}.carousel-dark .carousel-control-prev-icon,.carousel-dark .carousel-control-next-icon{filter:invert(1) grayscale(100)}.carousel-dark .carousel-indicators [data-bs-target]{background-color:#000}.carousel-dark .carousel-caption{color:#000}[data-bs-theme=dark] .carousel .carousel-control-prev-icon,[data-bs-theme=dark] .carousel .carousel-control-next-icon,[data-bs-theme=dark].carousel .carousel-control-prev-icon,[data-bs-theme=dark].carousel .carousel-control-next-icon{filter:invert(1) grayscale(100)}[data-bs-theme=dark] .carousel .carousel-indicators [data-bs-target],[data-bs-theme=dark].carousel .carousel-indicators [data-bs-target]{background-color:#000}[data-bs-theme=dark] .carousel .carousel-caption,[data-bs-theme=dark].carousel .carousel-caption{color:#000}.spinner-grow,.spinner-border{display:inline-block;width:var(--bs-spinner-width);height:var(--bs-spinner-height);vertical-align:var(--bs-spinner-vertical-align);border-radius:50%;animation:var(--bs-spinner-animation-speed) linear infinite var(--bs-spinner-animation-name)}@keyframes spinner-border{to{transform:rotate(360deg) /* rtl:ignore */}}.spinner-border{--bs-spinner-width: 2rem;--bs-spinner-height: 2rem;--bs-spinner-vertical-align: -0.125em;--bs-spinner-border-width: 0.25em;--bs-spinner-animation-speed: 0.75s;--bs-spinner-animation-name: spinner-border;border:var(--bs-spinner-border-width) solid currentcolor;border-right-color:rgba(0,0,0,0)}.spinner-border-sm{--bs-spinner-width: 1rem;--bs-spinner-height: 1rem;--bs-spinner-border-width: 0.2em}@keyframes spinner-grow{0%{transform:scale(0)}50%{opacity:1;transform:none}}.spinner-grow{--bs-spinner-width: 2rem;--bs-spinner-height: 2rem;--bs-spinner-vertical-align: -0.125em;--bs-spinner-animation-speed: 0.75s;--bs-spinner-animation-name: spinner-grow;background-color:currentcolor;opacity:0}.spinner-grow-sm{--bs-spinner-width: 1rem;--bs-spinner-height: 1rem}@media(prefers-reduced-motion: reduce){.spinner-border,.spinner-grow{--bs-spinner-animation-speed: 1.5s}}.offcanvas,.offcanvas-xxl,.offcanvas-xl,.offcanvas-lg,.offcanvas-md,.offcanvas-sm{--bs-offcanvas-zindex: 1045;--bs-offcanvas-width: 400px;--bs-offcanvas-height: 30vh;--bs-offcanvas-padding-x: 1rem;--bs-offcanvas-padding-y: 1rem;--bs-offcanvas-color: #fff;--bs-offcanvas-bg: #222;--bs-offcanvas-border-width: 1px;--bs-offcanvas-border-color: #434343;--bs-offcanvas-box-shadow: 0 0.125rem 0.25rem rgba(0, 0, 0, 0.075);--bs-offcanvas-transition: transform 0.3s ease-in-out;--bs-offcanvas-title-line-height: 1.5}@media(max-width: 575.98px){.offcanvas-sm{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}}@media(max-width: 575.98px)and (prefers-reduced-motion: reduce){.offcanvas-sm{transition:none}}@media(max-width: 575.98px){.offcanvas-sm.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas-sm.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas-sm.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas-sm.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas-sm.showing,.offcanvas-sm.show:not(.hiding){transform:none}.offcanvas-sm.showing,.offcanvas-sm.hiding,.offcanvas-sm.show{visibility:visible}}@media(min-width: 576px){.offcanvas-sm{--bs-offcanvas-height: auto;--bs-offcanvas-border-width: 0;background-color:rgba(0,0,0,0) !important}.offcanvas-sm .offcanvas-header{display:none}.offcanvas-sm .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible;background-color:rgba(0,0,0,0) !important}}@media(max-width: 767.98px){.offcanvas-md{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}}@media(max-width: 767.98px)and (prefers-reduced-motion: reduce){.offcanvas-md{transition:none}}@media(max-width: 767.98px){.offcanvas-md.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas-md.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas-md.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas-md.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas-md.showing,.offcanvas-md.show:not(.hiding){transform:none}.offcanvas-md.showing,.offcanvas-md.hiding,.offcanvas-md.show{visibility:visible}}@media(min-width: 768px){.offcanvas-md{--bs-offcanvas-height: auto;--bs-offcanvas-border-width: 0;background-color:rgba(0,0,0,0) !important}.offcanvas-md .offcanvas-header{display:none}.offcanvas-md .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible;background-color:rgba(0,0,0,0) !important}}@media(max-width: 991.98px){.offcanvas-lg{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}}@media(max-width: 991.98px)and (prefers-reduced-motion: reduce){.offcanvas-lg{transition:none}}@media(max-width: 991.98px){.offcanvas-lg.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas-lg.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas-lg.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas-lg.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas-lg.showing,.offcanvas-lg.show:not(.hiding){transform:none}.offcanvas-lg.showing,.offcanvas-lg.hiding,.offcanvas-lg.show{visibility:visible}}@media(min-width: 992px){.offcanvas-lg{--bs-offcanvas-height: auto;--bs-offcanvas-border-width: 0;background-color:rgba(0,0,0,0) !important}.offcanvas-lg .offcanvas-header{display:none}.offcanvas-lg .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible;background-color:rgba(0,0,0,0) !important}}@media(max-width: 1199.98px){.offcanvas-xl{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}}@media(max-width: 1199.98px)and (prefers-reduced-motion: reduce){.offcanvas-xl{transition:none}}@media(max-width: 1199.98px){.offcanvas-xl.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas-xl.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas-xl.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas-xl.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas-xl.showing,.offcanvas-xl.show:not(.hiding){transform:none}.offcanvas-xl.showing,.offcanvas-xl.hiding,.offcanvas-xl.show{visibility:visible}}@media(min-width: 1200px){.offcanvas-xl{--bs-offcanvas-height: auto;--bs-offcanvas-border-width: 0;background-color:rgba(0,0,0,0) !important}.offcanvas-xl .offcanvas-header{display:none}.offcanvas-xl .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible;background-color:rgba(0,0,0,0) !important}}@media(max-width: 1399.98px){.offcanvas-xxl{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}}@media(max-width: 1399.98px)and (prefers-reduced-motion: reduce){.offcanvas-xxl{transition:none}}@media(max-width: 1399.98px){.offcanvas-xxl.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas-xxl.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas-xxl.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas-xxl.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas-xxl.showing,.offcanvas-xxl.show:not(.hiding){transform:none}.offcanvas-xxl.showing,.offcanvas-xxl.hiding,.offcanvas-xxl.show{visibility:visible}}@media(min-width: 1400px){.offcanvas-xxl{--bs-offcanvas-height: auto;--bs-offcanvas-border-width: 0;background-color:rgba(0,0,0,0) !important}.offcanvas-xxl .offcanvas-header{display:none}.offcanvas-xxl .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible;background-color:rgba(0,0,0,0) !important}}.offcanvas{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}@media(prefers-reduced-motion: reduce){.offcanvas{transition:none}}.offcanvas.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas.showing,.offcanvas.show:not(.hiding){transform:none}.offcanvas.showing,.offcanvas.hiding,.offcanvas.show{visibility:visible}.offcanvas-backdrop{position:fixed;top:0;left:0;z-index:1040;width:100vw;height:100vh;background-color:#000}.offcanvas-backdrop.fade{opacity:0}.offcanvas-backdrop.show{opacity:.5}.offcanvas-header{display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;justify-content:space-between;-webkit-justify-content:space-between;padding:var(--bs-offcanvas-padding-y) var(--bs-offcanvas-padding-x)}.offcanvas-header .btn-close{padding:calc(var(--bs-offcanvas-padding-y)*.5) calc(var(--bs-offcanvas-padding-x)*.5);margin-top:calc(-0.5*var(--bs-offcanvas-padding-y));margin-right:calc(-0.5*var(--bs-offcanvas-padding-x));margin-bottom:calc(-0.5*var(--bs-offcanvas-padding-y))}.offcanvas-title{margin-bottom:0;line-height:var(--bs-offcanvas-title-line-height)}.offcanvas-body{flex-grow:1;-webkit-flex-grow:1;padding:var(--bs-offcanvas-padding-y) var(--bs-offcanvas-padding-x);overflow-y:auto}.placeholder{display:inline-block;min-height:1em;vertical-align:middle;cursor:wait;background-color:currentcolor;opacity:.5}.placeholder.btn::before{display:inline-block;content:""}.placeholder-xs{min-height:.6em}.placeholder-sm{min-height:.8em}.placeholder-lg{min-height:1.2em}.placeholder-glow .placeholder{animation:placeholder-glow 2s ease-in-out infinite}@keyframes placeholder-glow{50%{opacity:.2}}.placeholder-wave{mask-image:linear-gradient(130deg, #000 55%, rgba(0, 0, 0, 0.8) 75%, #000 95%);-webkit-mask-image:linear-gradient(130deg, #000 55%, rgba(0, 0, 0, 0.8) 75%, #000 95%);mask-size:200% 100%;-webkit-mask-size:200% 100%;animation:placeholder-wave 2s linear infinite}@keyframes placeholder-wave{100%{mask-position:-200% 0%;-webkit-mask-position:-200% 0%}}.clearfix::after{display:block;clear:both;content:""}.text-bg-default{color:#fff !important;background-color:RGBA(var(--bs-default-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-primary{color:#fff !important;background-color:RGBA(var(--bs-primary-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-secondary{color:#fff !important;background-color:RGBA(var(--bs-secondary-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-success{color:#fff !important;background-color:RGBA(var(--bs-success-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-info{color:#fff !important;background-color:RGBA(var(--bs-info-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-warning{color:#fff !important;background-color:RGBA(var(--bs-warning-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-danger{color:#fff !important;background-color:RGBA(var(--bs-danger-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-light{color:#fff !important;background-color:RGBA(var(--bs-light-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-dark{color:#fff !important;background-color:RGBA(var(--bs-dark-rgb), var(--bs-bg-opacity, 1)) !important}.link-default{color:RGBA(var(--bs-default-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-default-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-default:hover,.link-default:focus{color:RGBA(54, 54, 54, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(54, 54, 54, var(--bs-link-underline-opacity, 1)) !important}.link-primary{color:RGBA(var(--bs-primary-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-primary-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-primary:hover,.link-primary:focus{color:RGBA(44, 72, 102, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(44, 72, 102, var(--bs-link-underline-opacity, 1)) !important}.link-secondary{color:RGBA(var(--bs-secondary-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-secondary-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-secondary:hover,.link-secondary:focus{color:RGBA(54, 54, 54, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(54, 54, 54, var(--bs-link-underline-opacity, 1)) !important}.link-success{color:RGBA(var(--bs-success-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-success-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-success:hover,.link-success:focus{color:RGBA(0, 150, 112, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(0, 150, 112, var(--bs-link-underline-opacity, 1)) !important}.link-info{color:RGBA(var(--bs-info-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-info-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-info:hover,.link-info:focus{color:RGBA(42, 122, 175, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(42, 122, 175, var(--bs-link-underline-opacity, 1)) !important}.link-warning{color:RGBA(var(--bs-warning-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-warning-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-warning:hover,.link-warning:focus{color:RGBA(194, 125, 14, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(194, 125, 14, var(--bs-link-underline-opacity, 1)) !important}.link-danger{color:RGBA(var(--bs-danger-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-danger-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-danger:hover,.link-danger:focus{color:RGBA(185, 61, 48, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(185, 61, 48, var(--bs-link-underline-opacity, 1)) !important}.link-light{color:RGBA(var(--bs-light-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-light-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-light:hover,.link-light:focus{color:RGBA(89, 89, 89, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(89, 89, 89, var(--bs-link-underline-opacity, 1)) !important}.link-dark{color:RGBA(var(--bs-dark-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-dark-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-dark:hover,.link-dark:focus{color:RGBA(36, 36, 36, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(36, 36, 36, var(--bs-link-underline-opacity, 1)) !important}.link-body-emphasis{color:RGBA(var(--bs-emphasis-color-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-emphasis-color-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-body-emphasis:hover,.link-body-emphasis:focus{color:RGBA(var(--bs-emphasis-color-rgb), var(--bs-link-opacity, 0.75)) !important;text-decoration-color:RGBA(var(--bs-emphasis-color-rgb), var(--bs-link-underline-opacity, 0.75)) !important}.focus-ring:focus{outline:0;box-shadow:var(--bs-focus-ring-x, 0) var(--bs-focus-ring-y, 0) var(--bs-focus-ring-blur, 0) var(--bs-focus-ring-width) var(--bs-focus-ring-color)}.icon-link{display:inline-flex;gap:.375rem;align-items:center;-webkit-align-items:center;text-decoration-color:rgba(var(--bs-link-color-rgb), var(--bs-link-opacity, 0.5));text-underline-offset:.25em;backface-visibility:hidden;-webkit-backface-visibility:hidden;-moz-backface-visibility:hidden;-ms-backface-visibility:hidden;-o-backface-visibility:hidden}.icon-link>.bi{flex-shrink:0;-webkit-flex-shrink:0;width:1em;height:1em;fill:currentcolor;transition:.2s ease-in-out transform}@media(prefers-reduced-motion: reduce){.icon-link>.bi{transition:none}}.icon-link-hover:hover>.bi,.icon-link-hover:focus-visible>.bi{transform:var(--bs-icon-link-transform, translate3d(0.25em, 0, 0))}.ratio{position:relative;width:100%}.ratio::before{display:block;padding-top:var(--bs-aspect-ratio);content:""}.ratio>*{position:absolute;top:0;left:0;width:100%;height:100%}.ratio-1x1{--bs-aspect-ratio: 100%}.ratio-4x3{--bs-aspect-ratio: 75%}.ratio-16x9{--bs-aspect-ratio: 56.25%}.ratio-21x9{--bs-aspect-ratio: 42.8571428571%}.fixed-top{position:fixed;top:0;right:0;left:0;z-index:1030}.fixed-bottom{position:fixed;right:0;bottom:0;left:0;z-index:1030}.sticky-top{position:sticky;top:0;z-index:1020}.sticky-bottom{position:sticky;bottom:0;z-index:1020}@media(min-width: 576px){.sticky-sm-top{position:sticky;top:0;z-index:1020}.sticky-sm-bottom{position:sticky;bottom:0;z-index:1020}}@media(min-width: 768px){.sticky-md-top{position:sticky;top:0;z-index:1020}.sticky-md-bottom{position:sticky;bottom:0;z-index:1020}}@media(min-width: 992px){.sticky-lg-top{position:sticky;top:0;z-index:1020}.sticky-lg-bottom{position:sticky;bottom:0;z-index:1020}}@media(min-width: 1200px){.sticky-xl-top{position:sticky;top:0;z-index:1020}.sticky-xl-bottom{position:sticky;bottom:0;z-index:1020}}@media(min-width: 1400px){.sticky-xxl-top{position:sticky;top:0;z-index:1020}.sticky-xxl-bottom{position:sticky;bottom:0;z-index:1020}}.hstack{display:flex;display:-webkit-flex;flex-direction:row;-webkit-flex-direction:row;align-items:center;-webkit-align-items:center;align-self:stretch;-webkit-align-self:stretch}.vstack{display:flex;display:-webkit-flex;flex:1 1 auto;-webkit-flex:1 1 auto;flex-direction:column;-webkit-flex-direction:column;align-self:stretch;-webkit-align-self:stretch}.visually-hidden,.visually-hidden-focusable:not(:focus):not(:focus-within){width:1px !important;height:1px !important;padding:0 !important;margin:-1px !important;overflow:hidden !important;clip:rect(0, 0, 0, 0) !important;white-space:nowrap !important;border:0 !important}.visually-hidden:not(caption),.visually-hidden-focusable:not(:focus):not(:focus-within):not(caption){position:absolute !important}.stretched-link::after{position:absolute;top:0;right:0;bottom:0;left:0;z-index:1;content:""}.text-truncate{overflow:hidden;text-overflow:ellipsis;white-space:nowrap}.vr{display:inline-block;align-self:stretch;-webkit-align-self:stretch;width:1px;min-height:1em;background-color:currentcolor;opacity:.25}.align-baseline{vertical-align:baseline !important}.align-top{vertical-align:top !important}.align-middle{vertical-align:middle !important}.align-bottom{vertical-align:bottom !important}.align-text-bottom{vertical-align:text-bottom !important}.align-text-top{vertical-align:text-top !important}.float-start{float:left !important}.float-end{float:right !important}.float-none{float:none !important}.object-fit-contain{object-fit:contain !important}.object-fit-cover{object-fit:cover !important}.object-fit-fill{object-fit:fill !important}.object-fit-scale{object-fit:scale-down !important}.object-fit-none{object-fit:none !important}.opacity-0{opacity:0 !important}.opacity-25{opacity:.25 !important}.opacity-50{opacity:.5 !important}.opacity-75{opacity:.75 !important}.opacity-100{opacity:1 !important}.overflow-auto{overflow:auto !important}.overflow-hidden{overflow:hidden !important}.overflow-visible{overflow:visible !important}.overflow-scroll{overflow:scroll !important}.overflow-x-auto{overflow-x:auto !important}.overflow-x-hidden{overflow-x:hidden !important}.overflow-x-visible{overflow-x:visible !important}.overflow-x-scroll{overflow-x:scroll !important}.overflow-y-auto{overflow-y:auto !important}.overflow-y-hidden{overflow-y:hidden !important}.overflow-y-visible{overflow-y:visible !important}.overflow-y-scroll{overflow-y:scroll !important}.d-inline{display:inline !important}.d-inline-block{display:inline-block !important}.d-block{display:block !important}.d-grid{display:grid !important}.d-inline-grid{display:inline-grid !important}.d-table{display:table !important}.d-table-row{display:table-row !important}.d-table-cell{display:table-cell !important}.d-flex{display:flex !important}.d-inline-flex{display:inline-flex !important}.d-none{display:none !important}.shadow{box-shadow:0 .5rem 1rem rgba(0,0,0,.15) !important}.shadow-sm{box-shadow:0 .125rem .25rem rgba(0,0,0,.075) !important}.shadow-lg{box-shadow:0 1rem 3rem rgba(0,0,0,.175) !important}.shadow-none{box-shadow:none !important}.focus-ring-default{--bs-focus-ring-color: rgba(var(--bs-default-rgb), var(--bs-focus-ring-opacity))}.focus-ring-primary{--bs-focus-ring-color: rgba(var(--bs-primary-rgb), var(--bs-focus-ring-opacity))}.focus-ring-secondary{--bs-focus-ring-color: rgba(var(--bs-secondary-rgb), var(--bs-focus-ring-opacity))}.focus-ring-success{--bs-focus-ring-color: rgba(var(--bs-success-rgb), var(--bs-focus-ring-opacity))}.focus-ring-info{--bs-focus-ring-color: rgba(var(--bs-info-rgb), var(--bs-focus-ring-opacity))}.focus-ring-warning{--bs-focus-ring-color: rgba(var(--bs-warning-rgb), var(--bs-focus-ring-opacity))}.focus-ring-danger{--bs-focus-ring-color: rgba(var(--bs-danger-rgb), var(--bs-focus-ring-opacity))}.focus-ring-light{--bs-focus-ring-color: rgba(var(--bs-light-rgb), var(--bs-focus-ring-opacity))}.focus-ring-dark{--bs-focus-ring-color: rgba(var(--bs-dark-rgb), var(--bs-focus-ring-opacity))}.position-static{position:static !important}.position-relative{position:relative !important}.position-absolute{position:absolute !important}.position-fixed{position:fixed !important}.position-sticky{position:sticky !important}.top-0{top:0 !important}.top-50{top:50% !important}.top-100{top:100% !important}.bottom-0{bottom:0 !important}.bottom-50{bottom:50% !important}.bottom-100{bottom:100% !important}.start-0{left:0 !important}.start-50{left:50% !important}.start-100{left:100% !important}.end-0{right:0 !important}.end-50{right:50% !important}.end-100{right:100% !important}.translate-middle{transform:translate(-50%, -50%) !important}.translate-middle-x{transform:translateX(-50%) !important}.translate-middle-y{transform:translateY(-50%) !important}.border{border:var(--bs-border-width) var(--bs-border-style) var(--bs-border-color) !important}.border-0{border:0 !important}.border-top{border-top:var(--bs-border-width) var(--bs-border-style) var(--bs-border-color) !important}.border-top-0{border-top:0 !important}.border-end{border-right:var(--bs-border-width) var(--bs-border-style) var(--bs-border-color) !important}.border-end-0{border-right:0 !important}.border-bottom{border-bottom:var(--bs-border-width) var(--bs-border-style) var(--bs-border-color) !important}.border-bottom-0{border-bottom:0 !important}.border-start{border-left:var(--bs-border-width) var(--bs-border-style) var(--bs-border-color) !important}.border-start-0{border-left:0 !important}.border-default{--bs-border-opacity: 1;border-color:rgba(var(--bs-default-rgb), var(--bs-border-opacity)) !important}.border-primary{--bs-border-opacity: 1;border-color:rgba(var(--bs-primary-rgb), var(--bs-border-opacity)) !important}.border-secondary{--bs-border-opacity: 1;border-color:rgba(var(--bs-secondary-rgb), var(--bs-border-opacity)) !important}.border-success{--bs-border-opacity: 1;border-color:rgba(var(--bs-success-rgb), var(--bs-border-opacity)) !important}.border-info{--bs-border-opacity: 1;border-color:rgba(var(--bs-info-rgb), var(--bs-border-opacity)) !important}.border-warning{--bs-border-opacity: 1;border-color:rgba(var(--bs-warning-rgb), var(--bs-border-opacity)) !important}.border-danger{--bs-border-opacity: 1;border-color:rgba(var(--bs-danger-rgb), var(--bs-border-opacity)) !important}.border-light{--bs-border-opacity: 1;border-color:rgba(var(--bs-light-rgb), var(--bs-border-opacity)) !important}.border-dark{--bs-border-opacity: 1;border-color:rgba(var(--bs-dark-rgb), var(--bs-border-opacity)) !important}.border-black{--bs-border-opacity: 1;border-color:rgba(var(--bs-black-rgb), var(--bs-border-opacity)) !important}.border-white{--bs-border-opacity: 1;border-color:rgba(var(--bs-white-rgb), var(--bs-border-opacity)) !important}.border-primary-subtle{border-color:var(--bs-primary-border-subtle) !important}.border-secondary-subtle{border-color:var(--bs-secondary-border-subtle) !important}.border-success-subtle{border-color:var(--bs-success-border-subtle) !important}.border-info-subtle{border-color:var(--bs-info-border-subtle) !important}.border-warning-subtle{border-color:var(--bs-warning-border-subtle) !important}.border-danger-subtle{border-color:var(--bs-danger-border-subtle) !important}.border-light-subtle{border-color:var(--bs-light-border-subtle) !important}.border-dark-subtle{border-color:var(--bs-dark-border-subtle) !important}.border-1{border-width:1px !important}.border-2{border-width:2px !important}.border-3{border-width:3px !important}.border-4{border-width:4px !important}.border-5{border-width:5px !important}.border-opacity-10{--bs-border-opacity: 0.1}.border-opacity-25{--bs-border-opacity: 0.25}.border-opacity-50{--bs-border-opacity: 0.5}.border-opacity-75{--bs-border-opacity: 0.75}.border-opacity-100{--bs-border-opacity: 1}.w-25{width:25% !important}.w-50{width:50% !important}.w-75{width:75% !important}.w-100{width:100% !important}.w-auto{width:auto !important}.mw-100{max-width:100% !important}.vw-100{width:100vw !important}.min-vw-100{min-width:100vw !important}.h-25{height:25% !important}.h-50{height:50% !important}.h-75{height:75% !important}.h-100{height:100% !important}.h-auto{height:auto !important}.mh-100{max-height:100% !important}.vh-100{height:100vh !important}.min-vh-100{min-height:100vh !important}.flex-fill{flex:1 1 auto !important}.flex-row{flex-direction:row !important}.flex-column{flex-direction:column !important}.flex-row-reverse{flex-direction:row-reverse !important}.flex-column-reverse{flex-direction:column-reverse !important}.flex-grow-0{flex-grow:0 !important}.flex-grow-1{flex-grow:1 !important}.flex-shrink-0{flex-shrink:0 !important}.flex-shrink-1{flex-shrink:1 !important}.flex-wrap{flex-wrap:wrap !important}.flex-nowrap{flex-wrap:nowrap !important}.flex-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-start{justify-content:flex-start !important}.justify-content-end{justify-content:flex-end !important}.justify-content-center{justify-content:center !important}.justify-content-between{justify-content:space-between !important}.justify-content-around{justify-content:space-around !important}.justify-content-evenly{justify-content:space-evenly !important}.align-items-start{align-items:flex-start !important}.align-items-end{align-items:flex-end !important}.align-items-center{align-items:center !important}.align-items-baseline{align-items:baseline !important}.align-items-stretch{align-items:stretch !important}.align-content-start{align-content:flex-start !important}.align-content-end{align-content:flex-end !important}.align-content-center{align-content:center !important}.align-content-between{align-content:space-between !important}.align-content-around{align-content:space-around !important}.align-content-stretch{align-content:stretch !important}.align-self-auto{align-self:auto !important}.align-self-start{align-self:flex-start !important}.align-self-end{align-self:flex-end !important}.align-self-center{align-self:center !important}.align-self-baseline{align-self:baseline !important}.align-self-stretch{align-self:stretch !important}.order-first{order:-1 !important}.order-0{order:0 !important}.order-1{order:1 !important}.order-2{order:2 !important}.order-3{order:3 !important}.order-4{order:4 !important}.order-5{order:5 !important}.order-last{order:6 !important}.m-0{margin:0 !important}.m-1{margin:.25rem !important}.m-2{margin:.5rem !important}.m-3{margin:1rem !important}.m-4{margin:1.5rem !important}.m-5{margin:3rem !important}.m-auto{margin:auto !important}.mx-0{margin-right:0 !important;margin-left:0 !important}.mx-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-3{margin-right:1rem !important;margin-left:1rem !important}.mx-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-5{margin-right:3rem !important;margin-left:3rem !important}.mx-auto{margin-right:auto !important;margin-left:auto !important}.my-0{margin-top:0 !important;margin-bottom:0 !important}.my-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-0{margin-top:0 !important}.mt-1{margin-top:.25rem !important}.mt-2{margin-top:.5rem !important}.mt-3{margin-top:1rem !important}.mt-4{margin-top:1.5rem !important}.mt-5{margin-top:3rem !important}.mt-auto{margin-top:auto !important}.me-0{margin-right:0 !important}.me-1{margin-right:.25rem !important}.me-2{margin-right:.5rem !important}.me-3{margin-right:1rem !important}.me-4{margin-right:1.5rem !important}.me-5{margin-right:3rem !important}.me-auto{margin-right:auto !important}.mb-0{margin-bottom:0 !important}.mb-1{margin-bottom:.25rem !important}.mb-2{margin-bottom:.5rem !important}.mb-3{margin-bottom:1rem !important}.mb-4{margin-bottom:1.5rem !important}.mb-5{margin-bottom:3rem !important}.mb-auto{margin-bottom:auto !important}.ms-0{margin-left:0 !important}.ms-1{margin-left:.25rem !important}.ms-2{margin-left:.5rem !important}.ms-3{margin-left:1rem !important}.ms-4{margin-left:1.5rem !important}.ms-5{margin-left:3rem !important}.ms-auto{margin-left:auto !important}.p-0{padding:0 !important}.p-1{padding:.25rem !important}.p-2{padding:.5rem !important}.p-3{padding:1rem !important}.p-4{padding:1.5rem !important}.p-5{padding:3rem !important}.px-0{padding-right:0 !important;padding-left:0 !important}.px-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-3{padding-right:1rem !important;padding-left:1rem !important}.px-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-5{padding-right:3rem !important;padding-left:3rem !important}.py-0{padding-top:0 !important;padding-bottom:0 !important}.py-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-0{padding-top:0 !important}.pt-1{padding-top:.25rem !important}.pt-2{padding-top:.5rem !important}.pt-3{padding-top:1rem !important}.pt-4{padding-top:1.5rem !important}.pt-5{padding-top:3rem !important}.pe-0{padding-right:0 !important}.pe-1{padding-right:.25rem !important}.pe-2{padding-right:.5rem !important}.pe-3{padding-right:1rem !important}.pe-4{padding-right:1.5rem !important}.pe-5{padding-right:3rem !important}.pb-0{padding-bottom:0 !important}.pb-1{padding-bottom:.25rem !important}.pb-2{padding-bottom:.5rem !important}.pb-3{padding-bottom:1rem !important}.pb-4{padding-bottom:1.5rem !important}.pb-5{padding-bottom:3rem !important}.ps-0{padding-left:0 !important}.ps-1{padding-left:.25rem !important}.ps-2{padding-left:.5rem !important}.ps-3{padding-left:1rem !important}.ps-4{padding-left:1.5rem !important}.ps-5{padding-left:3rem !important}.gap-0{gap:0 !important}.gap-1{gap:.25rem !important}.gap-2{gap:.5rem !important}.gap-3{gap:1rem !important}.gap-4{gap:1.5rem !important}.gap-5{gap:3rem !important}.row-gap-0{row-gap:0 !important}.row-gap-1{row-gap:.25rem !important}.row-gap-2{row-gap:.5rem !important}.row-gap-3{row-gap:1rem !important}.row-gap-4{row-gap:1.5rem !important}.row-gap-5{row-gap:3rem !important}.column-gap-0{column-gap:0 !important}.column-gap-1{column-gap:.25rem !important}.column-gap-2{column-gap:.5rem !important}.column-gap-3{column-gap:1rem !important}.column-gap-4{column-gap:1.5rem !important}.column-gap-5{column-gap:3rem !important}.font-monospace{font-family:var(--bs-font-monospace) !important}.fs-1{font-size:calc(1.325rem + 0.9vw) !important}.fs-2{font-size:calc(1.29rem + 0.48vw) !important}.fs-3{font-size:calc(1.27rem + 0.24vw) !important}.fs-4{font-size:1.25rem !important}.fs-5{font-size:1.1rem !important}.fs-6{font-size:1rem !important}.fst-italic{font-style:italic !important}.fst-normal{font-style:normal !important}.fw-lighter{font-weight:lighter !important}.fw-light{font-weight:300 !important}.fw-normal{font-weight:400 !important}.fw-medium{font-weight:500 !important}.fw-semibold{font-weight:600 !important}.fw-bold{font-weight:700 !important}.fw-bolder{font-weight:bolder !important}.lh-1{line-height:1 !important}.lh-sm{line-height:1.25 !important}.lh-base{line-height:1.5 !important}.lh-lg{line-height:2 !important}.text-start{text-align:left !important}.text-end{text-align:right !important}.text-center{text-align:center !important}.text-decoration-none{text-decoration:none !important}.text-decoration-underline{text-decoration:underline !important}.text-decoration-line-through{text-decoration:line-through !important}.text-lowercase{text-transform:lowercase !important}.text-uppercase{text-transform:uppercase !important}.text-capitalize{text-transform:capitalize !important}.text-wrap{white-space:normal !important}.text-nowrap{white-space:nowrap !important}.text-break{word-wrap:break-word !important;word-break:break-word !important}.text-default{--bs-text-opacity: 1;color:rgba(var(--bs-default-rgb), var(--bs-text-opacity)) !important}.text-primary{--bs-text-opacity: 1;color:rgba(var(--bs-primary-rgb), var(--bs-text-opacity)) !important}.text-secondary{--bs-text-opacity: 1;color:rgba(var(--bs-secondary-rgb), var(--bs-text-opacity)) !important}.text-success{--bs-text-opacity: 1;color:rgba(var(--bs-success-rgb), var(--bs-text-opacity)) !important}.text-info{--bs-text-opacity: 1;color:rgba(var(--bs-info-rgb), var(--bs-text-opacity)) !important}.text-warning{--bs-text-opacity: 1;color:rgba(var(--bs-warning-rgb), var(--bs-text-opacity)) !important}.text-danger{--bs-text-opacity: 1;color:rgba(var(--bs-danger-rgb), var(--bs-text-opacity)) !important}.text-light{--bs-text-opacity: 1;color:rgba(var(--bs-light-rgb), var(--bs-text-opacity)) !important}.text-dark{--bs-text-opacity: 1;color:rgba(var(--bs-dark-rgb), var(--bs-text-opacity)) !important}.text-black{--bs-text-opacity: 1;color:rgba(var(--bs-black-rgb), var(--bs-text-opacity)) !important}.text-white{--bs-text-opacity: 1;color:rgba(var(--bs-white-rgb), var(--bs-text-opacity)) !important}.text-body{--bs-text-opacity: 1;color:rgba(var(--bs-body-color-rgb), var(--bs-text-opacity)) !important}.text-muted{--bs-text-opacity: 1;color:var(--bs-secondary-color) !important}.text-black-50{--bs-text-opacity: 1;color:rgba(0,0,0,.5) !important}.text-white-50{--bs-text-opacity: 1;color:rgba(255,255,255,.5) !important}.text-body-secondary{--bs-text-opacity: 1;color:var(--bs-secondary-color) !important}.text-body-tertiary{--bs-text-opacity: 1;color:var(--bs-tertiary-color) !important}.text-body-emphasis{--bs-text-opacity: 1;color:var(--bs-emphasis-color) !important}.text-reset{--bs-text-opacity: 1;color:inherit !important}.text-opacity-25{--bs-text-opacity: 0.25}.text-opacity-50{--bs-text-opacity: 0.5}.text-opacity-75{--bs-text-opacity: 0.75}.text-opacity-100{--bs-text-opacity: 1}.text-primary-emphasis{color:var(--bs-primary-text-emphasis) !important}.text-secondary-emphasis{color:var(--bs-secondary-text-emphasis) !important}.text-success-emphasis{color:var(--bs-success-text-emphasis) !important}.text-info-emphasis{color:var(--bs-info-text-emphasis) !important}.text-warning-emphasis{color:var(--bs-warning-text-emphasis) !important}.text-danger-emphasis{color:var(--bs-danger-text-emphasis) !important}.text-light-emphasis{color:var(--bs-light-text-emphasis) !important}.text-dark-emphasis{color:var(--bs-dark-text-emphasis) !important}.link-opacity-10{--bs-link-opacity: 0.1}.link-opacity-10-hover:hover{--bs-link-opacity: 0.1}.link-opacity-25{--bs-link-opacity: 0.25}.link-opacity-25-hover:hover{--bs-link-opacity: 0.25}.link-opacity-50{--bs-link-opacity: 0.5}.link-opacity-50-hover:hover{--bs-link-opacity: 0.5}.link-opacity-75{--bs-link-opacity: 0.75}.link-opacity-75-hover:hover{--bs-link-opacity: 0.75}.link-opacity-100{--bs-link-opacity: 1}.link-opacity-100-hover:hover{--bs-link-opacity: 1}.link-offset-1{text-underline-offset:.125em !important}.link-offset-1-hover:hover{text-underline-offset:.125em !important}.link-offset-2{text-underline-offset:.25em !important}.link-offset-2-hover:hover{text-underline-offset:.25em !important}.link-offset-3{text-underline-offset:.375em !important}.link-offset-3-hover:hover{text-underline-offset:.375em !important}.link-underline-default{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-default-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-primary{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-primary-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-secondary{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-secondary-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-success{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-success-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-info{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-info-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-warning{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-warning-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-danger{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-danger-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-light{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-light-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-dark{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-dark-rgb), var(--bs-link-underline-opacity)) !important}.link-underline{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-link-color-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-underline-opacity-0{--bs-link-underline-opacity: 0}.link-underline-opacity-0-hover:hover{--bs-link-underline-opacity: 0}.link-underline-opacity-10{--bs-link-underline-opacity: 0.1}.link-underline-opacity-10-hover:hover{--bs-link-underline-opacity: 0.1}.link-underline-opacity-25{--bs-link-underline-opacity: 0.25}.link-underline-opacity-25-hover:hover{--bs-link-underline-opacity: 0.25}.link-underline-opacity-50{--bs-link-underline-opacity: 0.5}.link-underline-opacity-50-hover:hover{--bs-link-underline-opacity: 0.5}.link-underline-opacity-75{--bs-link-underline-opacity: 0.75}.link-underline-opacity-75-hover:hover{--bs-link-underline-opacity: 0.75}.link-underline-opacity-100{--bs-link-underline-opacity: 1}.link-underline-opacity-100-hover:hover{--bs-link-underline-opacity: 1}.bg-default{--bs-bg-opacity: 1;background-color:rgba(var(--bs-default-rgb), var(--bs-bg-opacity)) !important}.bg-primary{--bs-bg-opacity: 1;background-color:rgba(var(--bs-primary-rgb), var(--bs-bg-opacity)) !important}.bg-secondary{--bs-bg-opacity: 1;background-color:rgba(var(--bs-secondary-rgb), var(--bs-bg-opacity)) !important}.bg-success{--bs-bg-opacity: 1;background-color:rgba(var(--bs-success-rgb), var(--bs-bg-opacity)) !important}.bg-info{--bs-bg-opacity: 1;background-color:rgba(var(--bs-info-rgb), var(--bs-bg-opacity)) !important}.bg-warning{--bs-bg-opacity: 1;background-color:rgba(var(--bs-warning-rgb), var(--bs-bg-opacity)) !important}.bg-danger{--bs-bg-opacity: 1;background-color:rgba(var(--bs-danger-rgb), var(--bs-bg-opacity)) !important}.bg-light{--bs-bg-opacity: 1;background-color:rgba(var(--bs-light-rgb), var(--bs-bg-opacity)) !important}.bg-dark{--bs-bg-opacity: 1;background-color:rgba(var(--bs-dark-rgb), var(--bs-bg-opacity)) !important}.bg-black{--bs-bg-opacity: 1;background-color:rgba(var(--bs-black-rgb), var(--bs-bg-opacity)) !important}.bg-white{--bs-bg-opacity: 1;background-color:rgba(var(--bs-white-rgb), var(--bs-bg-opacity)) !important}.bg-body{--bs-bg-opacity: 1;background-color:rgba(var(--bs-body-bg-rgb), var(--bs-bg-opacity)) !important}.bg-transparent{--bs-bg-opacity: 1;background-color:rgba(0,0,0,0) !important}.bg-body-secondary{--bs-bg-opacity: 1;background-color:rgba(var(--bs-secondary-bg-rgb), var(--bs-bg-opacity)) !important}.bg-body-tertiary{--bs-bg-opacity: 1;background-color:rgba(var(--bs-tertiary-bg-rgb), var(--bs-bg-opacity)) !important}.bg-opacity-10{--bs-bg-opacity: 0.1}.bg-opacity-25{--bs-bg-opacity: 0.25}.bg-opacity-50{--bs-bg-opacity: 0.5}.bg-opacity-75{--bs-bg-opacity: 0.75}.bg-opacity-100{--bs-bg-opacity: 1}.bg-primary-subtle{background-color:var(--bs-primary-bg-subtle) !important}.bg-secondary-subtle{background-color:var(--bs-secondary-bg-subtle) !important}.bg-success-subtle{background-color:var(--bs-success-bg-subtle) !important}.bg-info-subtle{background-color:var(--bs-info-bg-subtle) !important}.bg-warning-subtle{background-color:var(--bs-warning-bg-subtle) !important}.bg-danger-subtle{background-color:var(--bs-danger-bg-subtle) !important}.bg-light-subtle{background-color:var(--bs-light-bg-subtle) !important}.bg-dark-subtle{background-color:var(--bs-dark-bg-subtle) !important}.bg-gradient{background-image:var(--bs-gradient) !important}.user-select-all{user-select:all !important}.user-select-auto{user-select:auto !important}.user-select-none{user-select:none !important}.pe-none{pointer-events:none !important}.pe-auto{pointer-events:auto !important}.rounded{border-radius:var(--bs-border-radius) !important}.rounded-0{border-radius:0 !important}.rounded-1{border-radius:var(--bs-border-radius-sm) !important}.rounded-2{border-radius:var(--bs-border-radius) !important}.rounded-3{border-radius:var(--bs-border-radius-lg) !important}.rounded-4{border-radius:var(--bs-border-radius-xl) !important}.rounded-5{border-radius:var(--bs-border-radius-xxl) !important}.rounded-circle{border-radius:50% !important}.rounded-pill{border-radius:var(--bs-border-radius-pill) !important}.rounded-top{border-top-left-radius:var(--bs-border-radius) !important;border-top-right-radius:var(--bs-border-radius) !important}.rounded-top-0{border-top-left-radius:0 !important;border-top-right-radius:0 !important}.rounded-top-1{border-top-left-radius:var(--bs-border-radius-sm) !important;border-top-right-radius:var(--bs-border-radius-sm) !important}.rounded-top-2{border-top-left-radius:var(--bs-border-radius) !important;border-top-right-radius:var(--bs-border-radius) !important}.rounded-top-3{border-top-left-radius:var(--bs-border-radius-lg) !important;border-top-right-radius:var(--bs-border-radius-lg) !important}.rounded-top-4{border-top-left-radius:var(--bs-border-radius-xl) !important;border-top-right-radius:var(--bs-border-radius-xl) !important}.rounded-top-5{border-top-left-radius:var(--bs-border-radius-xxl) !important;border-top-right-radius:var(--bs-border-radius-xxl) !important}.rounded-top-circle{border-top-left-radius:50% !important;border-top-right-radius:50% !important}.rounded-top-pill{border-top-left-radius:var(--bs-border-radius-pill) !important;border-top-right-radius:var(--bs-border-radius-pill) !important}.rounded-end{border-top-right-radius:var(--bs-border-radius) !important;border-bottom-right-radius:var(--bs-border-radius) !important}.rounded-end-0{border-top-right-radius:0 !important;border-bottom-right-radius:0 !important}.rounded-end-1{border-top-right-radius:var(--bs-border-radius-sm) !important;border-bottom-right-radius:var(--bs-border-radius-sm) !important}.rounded-end-2{border-top-right-radius:var(--bs-border-radius) !important;border-bottom-right-radius:var(--bs-border-radius) !important}.rounded-end-3{border-top-right-radius:var(--bs-border-radius-lg) !important;border-bottom-right-radius:var(--bs-border-radius-lg) !important}.rounded-end-4{border-top-right-radius:var(--bs-border-radius-xl) !important;border-bottom-right-radius:var(--bs-border-radius-xl) !important}.rounded-end-5{border-top-right-radius:var(--bs-border-radius-xxl) !important;border-bottom-right-radius:var(--bs-border-radius-xxl) !important}.rounded-end-circle{border-top-right-radius:50% !important;border-bottom-right-radius:50% !important}.rounded-end-pill{border-top-right-radius:var(--bs-border-radius-pill) !important;border-bottom-right-radius:var(--bs-border-radius-pill) !important}.rounded-bottom{border-bottom-right-radius:var(--bs-border-radius) !important;border-bottom-left-radius:var(--bs-border-radius) !important}.rounded-bottom-0{border-bottom-right-radius:0 !important;border-bottom-left-radius:0 !important}.rounded-bottom-1{border-bottom-right-radius:var(--bs-border-radius-sm) !important;border-bottom-left-radius:var(--bs-border-radius-sm) !important}.rounded-bottom-2{border-bottom-right-radius:var(--bs-border-radius) !important;border-bottom-left-radius:var(--bs-border-radius) !important}.rounded-bottom-3{border-bottom-right-radius:var(--bs-border-radius-lg) !important;border-bottom-left-radius:var(--bs-border-radius-lg) !important}.rounded-bottom-4{border-bottom-right-radius:var(--bs-border-radius-xl) !important;border-bottom-left-radius:var(--bs-border-radius-xl) !important}.rounded-bottom-5{border-bottom-right-radius:var(--bs-border-radius-xxl) !important;border-bottom-left-radius:var(--bs-border-radius-xxl) !important}.rounded-bottom-circle{border-bottom-right-radius:50% !important;border-bottom-left-radius:50% !important}.rounded-bottom-pill{border-bottom-right-radius:var(--bs-border-radius-pill) !important;border-bottom-left-radius:var(--bs-border-radius-pill) !important}.rounded-start{border-bottom-left-radius:var(--bs-border-radius) !important;border-top-left-radius:var(--bs-border-radius) !important}.rounded-start-0{border-bottom-left-radius:0 !important;border-top-left-radius:0 !important}.rounded-start-1{border-bottom-left-radius:var(--bs-border-radius-sm) !important;border-top-left-radius:var(--bs-border-radius-sm) !important}.rounded-start-2{border-bottom-left-radius:var(--bs-border-radius) !important;border-top-left-radius:var(--bs-border-radius) !important}.rounded-start-3{border-bottom-left-radius:var(--bs-border-radius-lg) !important;border-top-left-radius:var(--bs-border-radius-lg) !important}.rounded-start-4{border-bottom-left-radius:var(--bs-border-radius-xl) !important;border-top-left-radius:var(--bs-border-radius-xl) !important}.rounded-start-5{border-bottom-left-radius:var(--bs-border-radius-xxl) !important;border-top-left-radius:var(--bs-border-radius-xxl) !important}.rounded-start-circle{border-bottom-left-radius:50% !important;border-top-left-radius:50% !important}.rounded-start-pill{border-bottom-left-radius:var(--bs-border-radius-pill) !important;border-top-left-radius:var(--bs-border-radius-pill) !important}.visible{visibility:visible !important}.invisible{visibility:hidden !important}.z-n1{z-index:-1 !important}.z-0{z-index:0 !important}.z-1{z-index:1 !important}.z-2{z-index:2 !important}.z-3{z-index:3 !important}@media(min-width: 576px){.float-sm-start{float:left !important}.float-sm-end{float:right !important}.float-sm-none{float:none !important}.object-fit-sm-contain{object-fit:contain !important}.object-fit-sm-cover{object-fit:cover !important}.object-fit-sm-fill{object-fit:fill !important}.object-fit-sm-scale{object-fit:scale-down !important}.object-fit-sm-none{object-fit:none !important}.d-sm-inline{display:inline !important}.d-sm-inline-block{display:inline-block !important}.d-sm-block{display:block !important}.d-sm-grid{display:grid !important}.d-sm-inline-grid{display:inline-grid !important}.d-sm-table{display:table !important}.d-sm-table-row{display:table-row !important}.d-sm-table-cell{display:table-cell !important}.d-sm-flex{display:flex !important}.d-sm-inline-flex{display:inline-flex !important}.d-sm-none{display:none !important}.flex-sm-fill{flex:1 1 auto !important}.flex-sm-row{flex-direction:row !important}.flex-sm-column{flex-direction:column !important}.flex-sm-row-reverse{flex-direction:row-reverse !important}.flex-sm-column-reverse{flex-direction:column-reverse !important}.flex-sm-grow-0{flex-grow:0 !important}.flex-sm-grow-1{flex-grow:1 !important}.flex-sm-shrink-0{flex-shrink:0 !important}.flex-sm-shrink-1{flex-shrink:1 !important}.flex-sm-wrap{flex-wrap:wrap !important}.flex-sm-nowrap{flex-wrap:nowrap !important}.flex-sm-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-sm-start{justify-content:flex-start !important}.justify-content-sm-end{justify-content:flex-end !important}.justify-content-sm-center{justify-content:center !important}.justify-content-sm-between{justify-content:space-between !important}.justify-content-sm-around{justify-content:space-around !important}.justify-content-sm-evenly{justify-content:space-evenly !important}.align-items-sm-start{align-items:flex-start !important}.align-items-sm-end{align-items:flex-end !important}.align-items-sm-center{align-items:center !important}.align-items-sm-baseline{align-items:baseline !important}.align-items-sm-stretch{align-items:stretch !important}.align-content-sm-start{align-content:flex-start !important}.align-content-sm-end{align-content:flex-end !important}.align-content-sm-center{align-content:center !important}.align-content-sm-between{align-content:space-between !important}.align-content-sm-around{align-content:space-around !important}.align-content-sm-stretch{align-content:stretch !important}.align-self-sm-auto{align-self:auto !important}.align-self-sm-start{align-self:flex-start !important}.align-self-sm-end{align-self:flex-end !important}.align-self-sm-center{align-self:center !important}.align-self-sm-baseline{align-self:baseline !important}.align-self-sm-stretch{align-self:stretch !important}.order-sm-first{order:-1 !important}.order-sm-0{order:0 !important}.order-sm-1{order:1 !important}.order-sm-2{order:2 !important}.order-sm-3{order:3 !important}.order-sm-4{order:4 !important}.order-sm-5{order:5 !important}.order-sm-last{order:6 !important}.m-sm-0{margin:0 !important}.m-sm-1{margin:.25rem !important}.m-sm-2{margin:.5rem !important}.m-sm-3{margin:1rem !important}.m-sm-4{margin:1.5rem !important}.m-sm-5{margin:3rem !important}.m-sm-auto{margin:auto !important}.mx-sm-0{margin-right:0 !important;margin-left:0 !important}.mx-sm-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-sm-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-sm-3{margin-right:1rem !important;margin-left:1rem !important}.mx-sm-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-sm-5{margin-right:3rem !important;margin-left:3rem !important}.mx-sm-auto{margin-right:auto !important;margin-left:auto !important}.my-sm-0{margin-top:0 !important;margin-bottom:0 !important}.my-sm-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-sm-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-sm-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-sm-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-sm-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-sm-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-sm-0{margin-top:0 !important}.mt-sm-1{margin-top:.25rem !important}.mt-sm-2{margin-top:.5rem !important}.mt-sm-3{margin-top:1rem !important}.mt-sm-4{margin-top:1.5rem !important}.mt-sm-5{margin-top:3rem !important}.mt-sm-auto{margin-top:auto !important}.me-sm-0{margin-right:0 !important}.me-sm-1{margin-right:.25rem !important}.me-sm-2{margin-right:.5rem !important}.me-sm-3{margin-right:1rem !important}.me-sm-4{margin-right:1.5rem !important}.me-sm-5{margin-right:3rem !important}.me-sm-auto{margin-right:auto !important}.mb-sm-0{margin-bottom:0 !important}.mb-sm-1{margin-bottom:.25rem !important}.mb-sm-2{margin-bottom:.5rem !important}.mb-sm-3{margin-bottom:1rem !important}.mb-sm-4{margin-bottom:1.5rem !important}.mb-sm-5{margin-bottom:3rem !important}.mb-sm-auto{margin-bottom:auto !important}.ms-sm-0{margin-left:0 !important}.ms-sm-1{margin-left:.25rem !important}.ms-sm-2{margin-left:.5rem !important}.ms-sm-3{margin-left:1rem !important}.ms-sm-4{margin-left:1.5rem !important}.ms-sm-5{margin-left:3rem !important}.ms-sm-auto{margin-left:auto !important}.p-sm-0{padding:0 !important}.p-sm-1{padding:.25rem !important}.p-sm-2{padding:.5rem !important}.p-sm-3{padding:1rem !important}.p-sm-4{padding:1.5rem !important}.p-sm-5{padding:3rem !important}.px-sm-0{padding-right:0 !important;padding-left:0 !important}.px-sm-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-sm-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-sm-3{padding-right:1rem !important;padding-left:1rem !important}.px-sm-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-sm-5{padding-right:3rem !important;padding-left:3rem !important}.py-sm-0{padding-top:0 !important;padding-bottom:0 !important}.py-sm-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-sm-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-sm-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-sm-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-sm-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-sm-0{padding-top:0 !important}.pt-sm-1{padding-top:.25rem !important}.pt-sm-2{padding-top:.5rem !important}.pt-sm-3{padding-top:1rem !important}.pt-sm-4{padding-top:1.5rem !important}.pt-sm-5{padding-top:3rem !important}.pe-sm-0{padding-right:0 !important}.pe-sm-1{padding-right:.25rem !important}.pe-sm-2{padding-right:.5rem !important}.pe-sm-3{padding-right:1rem !important}.pe-sm-4{padding-right:1.5rem !important}.pe-sm-5{padding-right:3rem !important}.pb-sm-0{padding-bottom:0 !important}.pb-sm-1{padding-bottom:.25rem !important}.pb-sm-2{padding-bottom:.5rem !important}.pb-sm-3{padding-bottom:1rem !important}.pb-sm-4{padding-bottom:1.5rem !important}.pb-sm-5{padding-bottom:3rem !important}.ps-sm-0{padding-left:0 !important}.ps-sm-1{padding-left:.25rem !important}.ps-sm-2{padding-left:.5rem !important}.ps-sm-3{padding-left:1rem !important}.ps-sm-4{padding-left:1.5rem !important}.ps-sm-5{padding-left:3rem !important}.gap-sm-0{gap:0 !important}.gap-sm-1{gap:.25rem !important}.gap-sm-2{gap:.5rem !important}.gap-sm-3{gap:1rem !important}.gap-sm-4{gap:1.5rem !important}.gap-sm-5{gap:3rem !important}.row-gap-sm-0{row-gap:0 !important}.row-gap-sm-1{row-gap:.25rem !important}.row-gap-sm-2{row-gap:.5rem !important}.row-gap-sm-3{row-gap:1rem !important}.row-gap-sm-4{row-gap:1.5rem !important}.row-gap-sm-5{row-gap:3rem !important}.column-gap-sm-0{column-gap:0 !important}.column-gap-sm-1{column-gap:.25rem !important}.column-gap-sm-2{column-gap:.5rem !important}.column-gap-sm-3{column-gap:1rem !important}.column-gap-sm-4{column-gap:1.5rem !important}.column-gap-sm-5{column-gap:3rem !important}.text-sm-start{text-align:left !important}.text-sm-end{text-align:right !important}.text-sm-center{text-align:center !important}}@media(min-width: 768px){.float-md-start{float:left !important}.float-md-end{float:right !important}.float-md-none{float:none !important}.object-fit-md-contain{object-fit:contain !important}.object-fit-md-cover{object-fit:cover !important}.object-fit-md-fill{object-fit:fill !important}.object-fit-md-scale{object-fit:scale-down !important}.object-fit-md-none{object-fit:none !important}.d-md-inline{display:inline !important}.d-md-inline-block{display:inline-block !important}.d-md-block{display:block !important}.d-md-grid{display:grid !important}.d-md-inline-grid{display:inline-grid !important}.d-md-table{display:table !important}.d-md-table-row{display:table-row !important}.d-md-table-cell{display:table-cell !important}.d-md-flex{display:flex !important}.d-md-inline-flex{display:inline-flex !important}.d-md-none{display:none !important}.flex-md-fill{flex:1 1 auto !important}.flex-md-row{flex-direction:row !important}.flex-md-column{flex-direction:column !important}.flex-md-row-reverse{flex-direction:row-reverse !important}.flex-md-column-reverse{flex-direction:column-reverse !important}.flex-md-grow-0{flex-grow:0 !important}.flex-md-grow-1{flex-grow:1 !important}.flex-md-shrink-0{flex-shrink:0 !important}.flex-md-shrink-1{flex-shrink:1 !important}.flex-md-wrap{flex-wrap:wrap !important}.flex-md-nowrap{flex-wrap:nowrap !important}.flex-md-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-md-start{justify-content:flex-start !important}.justify-content-md-end{justify-content:flex-end !important}.justify-content-md-center{justify-content:center !important}.justify-content-md-between{justify-content:space-between !important}.justify-content-md-around{justify-content:space-around !important}.justify-content-md-evenly{justify-content:space-evenly !important}.align-items-md-start{align-items:flex-start !important}.align-items-md-end{align-items:flex-end !important}.align-items-md-center{align-items:center !important}.align-items-md-baseline{align-items:baseline !important}.align-items-md-stretch{align-items:stretch !important}.align-content-md-start{align-content:flex-start !important}.align-content-md-end{align-content:flex-end !important}.align-content-md-center{align-content:center !important}.align-content-md-between{align-content:space-between !important}.align-content-md-around{align-content:space-around !important}.align-content-md-stretch{align-content:stretch !important}.align-self-md-auto{align-self:auto !important}.align-self-md-start{align-self:flex-start !important}.align-self-md-end{align-self:flex-end !important}.align-self-md-center{align-self:center !important}.align-self-md-baseline{align-self:baseline !important}.align-self-md-stretch{align-self:stretch !important}.order-md-first{order:-1 !important}.order-md-0{order:0 !important}.order-md-1{order:1 !important}.order-md-2{order:2 !important}.order-md-3{order:3 !important}.order-md-4{order:4 !important}.order-md-5{order:5 !important}.order-md-last{order:6 !important}.m-md-0{margin:0 !important}.m-md-1{margin:.25rem !important}.m-md-2{margin:.5rem !important}.m-md-3{margin:1rem !important}.m-md-4{margin:1.5rem !important}.m-md-5{margin:3rem !important}.m-md-auto{margin:auto !important}.mx-md-0{margin-right:0 !important;margin-left:0 !important}.mx-md-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-md-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-md-3{margin-right:1rem !important;margin-left:1rem !important}.mx-md-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-md-5{margin-right:3rem !important;margin-left:3rem !important}.mx-md-auto{margin-right:auto !important;margin-left:auto !important}.my-md-0{margin-top:0 !important;margin-bottom:0 !important}.my-md-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-md-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-md-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-md-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-md-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-md-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-md-0{margin-top:0 !important}.mt-md-1{margin-top:.25rem !important}.mt-md-2{margin-top:.5rem !important}.mt-md-3{margin-top:1rem !important}.mt-md-4{margin-top:1.5rem !important}.mt-md-5{margin-top:3rem !important}.mt-md-auto{margin-top:auto !important}.me-md-0{margin-right:0 !important}.me-md-1{margin-right:.25rem !important}.me-md-2{margin-right:.5rem !important}.me-md-3{margin-right:1rem !important}.me-md-4{margin-right:1.5rem !important}.me-md-5{margin-right:3rem !important}.me-md-auto{margin-right:auto !important}.mb-md-0{margin-bottom:0 !important}.mb-md-1{margin-bottom:.25rem !important}.mb-md-2{margin-bottom:.5rem !important}.mb-md-3{margin-bottom:1rem !important}.mb-md-4{margin-bottom:1.5rem !important}.mb-md-5{margin-bottom:3rem !important}.mb-md-auto{margin-bottom:auto !important}.ms-md-0{margin-left:0 !important}.ms-md-1{margin-left:.25rem !important}.ms-md-2{margin-left:.5rem !important}.ms-md-3{margin-left:1rem !important}.ms-md-4{margin-left:1.5rem !important}.ms-md-5{margin-left:3rem !important}.ms-md-auto{margin-left:auto !important}.p-md-0{padding:0 !important}.p-md-1{padding:.25rem !important}.p-md-2{padding:.5rem !important}.p-md-3{padding:1rem !important}.p-md-4{padding:1.5rem !important}.p-md-5{padding:3rem !important}.px-md-0{padding-right:0 !important;padding-left:0 !important}.px-md-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-md-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-md-3{padding-right:1rem !important;padding-left:1rem !important}.px-md-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-md-5{padding-right:3rem !important;padding-left:3rem !important}.py-md-0{padding-top:0 !important;padding-bottom:0 !important}.py-md-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-md-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-md-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-md-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-md-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-md-0{padding-top:0 !important}.pt-md-1{padding-top:.25rem !important}.pt-md-2{padding-top:.5rem !important}.pt-md-3{padding-top:1rem !important}.pt-md-4{padding-top:1.5rem !important}.pt-md-5{padding-top:3rem !important}.pe-md-0{padding-right:0 !important}.pe-md-1{padding-right:.25rem !important}.pe-md-2{padding-right:.5rem !important}.pe-md-3{padding-right:1rem !important}.pe-md-4{padding-right:1.5rem !important}.pe-md-5{padding-right:3rem !important}.pb-md-0{padding-bottom:0 !important}.pb-md-1{padding-bottom:.25rem !important}.pb-md-2{padding-bottom:.5rem !important}.pb-md-3{padding-bottom:1rem !important}.pb-md-4{padding-bottom:1.5rem !important}.pb-md-5{padding-bottom:3rem !important}.ps-md-0{padding-left:0 !important}.ps-md-1{padding-left:.25rem !important}.ps-md-2{padding-left:.5rem !important}.ps-md-3{padding-left:1rem !important}.ps-md-4{padding-left:1.5rem !important}.ps-md-5{padding-left:3rem !important}.gap-md-0{gap:0 !important}.gap-md-1{gap:.25rem !important}.gap-md-2{gap:.5rem !important}.gap-md-3{gap:1rem !important}.gap-md-4{gap:1.5rem !important}.gap-md-5{gap:3rem !important}.row-gap-md-0{row-gap:0 !important}.row-gap-md-1{row-gap:.25rem !important}.row-gap-md-2{row-gap:.5rem !important}.row-gap-md-3{row-gap:1rem !important}.row-gap-md-4{row-gap:1.5rem !important}.row-gap-md-5{row-gap:3rem !important}.column-gap-md-0{column-gap:0 !important}.column-gap-md-1{column-gap:.25rem !important}.column-gap-md-2{column-gap:.5rem !important}.column-gap-md-3{column-gap:1rem !important}.column-gap-md-4{column-gap:1.5rem !important}.column-gap-md-5{column-gap:3rem !important}.text-md-start{text-align:left !important}.text-md-end{text-align:right !important}.text-md-center{text-align:center !important}}@media(min-width: 992px){.float-lg-start{float:left !important}.float-lg-end{float:right !important}.float-lg-none{float:none !important}.object-fit-lg-contain{object-fit:contain !important}.object-fit-lg-cover{object-fit:cover !important}.object-fit-lg-fill{object-fit:fill !important}.object-fit-lg-scale{object-fit:scale-down !important}.object-fit-lg-none{object-fit:none !important}.d-lg-inline{display:inline !important}.d-lg-inline-block{display:inline-block !important}.d-lg-block{display:block !important}.d-lg-grid{display:grid !important}.d-lg-inline-grid{display:inline-grid !important}.d-lg-table{display:table !important}.d-lg-table-row{display:table-row !important}.d-lg-table-cell{display:table-cell !important}.d-lg-flex{display:flex !important}.d-lg-inline-flex{display:inline-flex !important}.d-lg-none{display:none !important}.flex-lg-fill{flex:1 1 auto !important}.flex-lg-row{flex-direction:row !important}.flex-lg-column{flex-direction:column !important}.flex-lg-row-reverse{flex-direction:row-reverse !important}.flex-lg-column-reverse{flex-direction:column-reverse !important}.flex-lg-grow-0{flex-grow:0 !important}.flex-lg-grow-1{flex-grow:1 !important}.flex-lg-shrink-0{flex-shrink:0 !important}.flex-lg-shrink-1{flex-shrink:1 !important}.flex-lg-wrap{flex-wrap:wrap !important}.flex-lg-nowrap{flex-wrap:nowrap !important}.flex-lg-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-lg-start{justify-content:flex-start !important}.justify-content-lg-end{justify-content:flex-end !important}.justify-content-lg-center{justify-content:center !important}.justify-content-lg-between{justify-content:space-between !important}.justify-content-lg-around{justify-content:space-around !important}.justify-content-lg-evenly{justify-content:space-evenly !important}.align-items-lg-start{align-items:flex-start !important}.align-items-lg-end{align-items:flex-end !important}.align-items-lg-center{align-items:center !important}.align-items-lg-baseline{align-items:baseline !important}.align-items-lg-stretch{align-items:stretch !important}.align-content-lg-start{align-content:flex-start !important}.align-content-lg-end{align-content:flex-end !important}.align-content-lg-center{align-content:center !important}.align-content-lg-between{align-content:space-between !important}.align-content-lg-around{align-content:space-around !important}.align-content-lg-stretch{align-content:stretch !important}.align-self-lg-auto{align-self:auto !important}.align-self-lg-start{align-self:flex-start !important}.align-self-lg-end{align-self:flex-end !important}.align-self-lg-center{align-self:center !important}.align-self-lg-baseline{align-self:baseline !important}.align-self-lg-stretch{align-self:stretch !important}.order-lg-first{order:-1 !important}.order-lg-0{order:0 !important}.order-lg-1{order:1 !important}.order-lg-2{order:2 !important}.order-lg-3{order:3 !important}.order-lg-4{order:4 !important}.order-lg-5{order:5 !important}.order-lg-last{order:6 !important}.m-lg-0{margin:0 !important}.m-lg-1{margin:.25rem !important}.m-lg-2{margin:.5rem !important}.m-lg-3{margin:1rem !important}.m-lg-4{margin:1.5rem !important}.m-lg-5{margin:3rem !important}.m-lg-auto{margin:auto !important}.mx-lg-0{margin-right:0 !important;margin-left:0 !important}.mx-lg-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-lg-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-lg-3{margin-right:1rem !important;margin-left:1rem !important}.mx-lg-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-lg-5{margin-right:3rem !important;margin-left:3rem !important}.mx-lg-auto{margin-right:auto !important;margin-left:auto !important}.my-lg-0{margin-top:0 !important;margin-bottom:0 !important}.my-lg-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-lg-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-lg-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-lg-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-lg-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-lg-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-lg-0{margin-top:0 !important}.mt-lg-1{margin-top:.25rem !important}.mt-lg-2{margin-top:.5rem !important}.mt-lg-3{margin-top:1rem !important}.mt-lg-4{margin-top:1.5rem !important}.mt-lg-5{margin-top:3rem !important}.mt-lg-auto{margin-top:auto !important}.me-lg-0{margin-right:0 !important}.me-lg-1{margin-right:.25rem !important}.me-lg-2{margin-right:.5rem !important}.me-lg-3{margin-right:1rem !important}.me-lg-4{margin-right:1.5rem !important}.me-lg-5{margin-right:3rem !important}.me-lg-auto{margin-right:auto !important}.mb-lg-0{margin-bottom:0 !important}.mb-lg-1{margin-bottom:.25rem !important}.mb-lg-2{margin-bottom:.5rem !important}.mb-lg-3{margin-bottom:1rem !important}.mb-lg-4{margin-bottom:1.5rem !important}.mb-lg-5{margin-bottom:3rem !important}.mb-lg-auto{margin-bottom:auto !important}.ms-lg-0{margin-left:0 !important}.ms-lg-1{margin-left:.25rem !important}.ms-lg-2{margin-left:.5rem !important}.ms-lg-3{margin-left:1rem !important}.ms-lg-4{margin-left:1.5rem !important}.ms-lg-5{margin-left:3rem !important}.ms-lg-auto{margin-left:auto !important}.p-lg-0{padding:0 !important}.p-lg-1{padding:.25rem !important}.p-lg-2{padding:.5rem !important}.p-lg-3{padding:1rem !important}.p-lg-4{padding:1.5rem !important}.p-lg-5{padding:3rem !important}.px-lg-0{padding-right:0 !important;padding-left:0 !important}.px-lg-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-lg-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-lg-3{padding-right:1rem !important;padding-left:1rem !important}.px-lg-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-lg-5{padding-right:3rem !important;padding-left:3rem !important}.py-lg-0{padding-top:0 !important;padding-bottom:0 !important}.py-lg-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-lg-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-lg-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-lg-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-lg-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-lg-0{padding-top:0 !important}.pt-lg-1{padding-top:.25rem !important}.pt-lg-2{padding-top:.5rem !important}.pt-lg-3{padding-top:1rem !important}.pt-lg-4{padding-top:1.5rem !important}.pt-lg-5{padding-top:3rem !important}.pe-lg-0{padding-right:0 !important}.pe-lg-1{padding-right:.25rem !important}.pe-lg-2{padding-right:.5rem !important}.pe-lg-3{padding-right:1rem !important}.pe-lg-4{padding-right:1.5rem !important}.pe-lg-5{padding-right:3rem !important}.pb-lg-0{padding-bottom:0 !important}.pb-lg-1{padding-bottom:.25rem !important}.pb-lg-2{padding-bottom:.5rem !important}.pb-lg-3{padding-bottom:1rem !important}.pb-lg-4{padding-bottom:1.5rem !important}.pb-lg-5{padding-bottom:3rem !important}.ps-lg-0{padding-left:0 !important}.ps-lg-1{padding-left:.25rem !important}.ps-lg-2{padding-left:.5rem !important}.ps-lg-3{padding-left:1rem !important}.ps-lg-4{padding-left:1.5rem !important}.ps-lg-5{padding-left:3rem !important}.gap-lg-0{gap:0 !important}.gap-lg-1{gap:.25rem !important}.gap-lg-2{gap:.5rem !important}.gap-lg-3{gap:1rem !important}.gap-lg-4{gap:1.5rem !important}.gap-lg-5{gap:3rem !important}.row-gap-lg-0{row-gap:0 !important}.row-gap-lg-1{row-gap:.25rem !important}.row-gap-lg-2{row-gap:.5rem !important}.row-gap-lg-3{row-gap:1rem !important}.row-gap-lg-4{row-gap:1.5rem !important}.row-gap-lg-5{row-gap:3rem !important}.column-gap-lg-0{column-gap:0 !important}.column-gap-lg-1{column-gap:.25rem !important}.column-gap-lg-2{column-gap:.5rem !important}.column-gap-lg-3{column-gap:1rem !important}.column-gap-lg-4{column-gap:1.5rem !important}.column-gap-lg-5{column-gap:3rem !important}.text-lg-start{text-align:left !important}.text-lg-end{text-align:right !important}.text-lg-center{text-align:center !important}}@media(min-width: 1200px){.float-xl-start{float:left !important}.float-xl-end{float:right !important}.float-xl-none{float:none !important}.object-fit-xl-contain{object-fit:contain !important}.object-fit-xl-cover{object-fit:cover !important}.object-fit-xl-fill{object-fit:fill !important}.object-fit-xl-scale{object-fit:scale-down !important}.object-fit-xl-none{object-fit:none !important}.d-xl-inline{display:inline !important}.d-xl-inline-block{display:inline-block !important}.d-xl-block{display:block !important}.d-xl-grid{display:grid !important}.d-xl-inline-grid{display:inline-grid !important}.d-xl-table{display:table !important}.d-xl-table-row{display:table-row !important}.d-xl-table-cell{display:table-cell !important}.d-xl-flex{display:flex !important}.d-xl-inline-flex{display:inline-flex !important}.d-xl-none{display:none !important}.flex-xl-fill{flex:1 1 auto !important}.flex-xl-row{flex-direction:row !important}.flex-xl-column{flex-direction:column !important}.flex-xl-row-reverse{flex-direction:row-reverse !important}.flex-xl-column-reverse{flex-direction:column-reverse !important}.flex-xl-grow-0{flex-grow:0 !important}.flex-xl-grow-1{flex-grow:1 !important}.flex-xl-shrink-0{flex-shrink:0 !important}.flex-xl-shrink-1{flex-shrink:1 !important}.flex-xl-wrap{flex-wrap:wrap !important}.flex-xl-nowrap{flex-wrap:nowrap !important}.flex-xl-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-xl-start{justify-content:flex-start !important}.justify-content-xl-end{justify-content:flex-end !important}.justify-content-xl-center{justify-content:center !important}.justify-content-xl-between{justify-content:space-between !important}.justify-content-xl-around{justify-content:space-around !important}.justify-content-xl-evenly{justify-content:space-evenly !important}.align-items-xl-start{align-items:flex-start !important}.align-items-xl-end{align-items:flex-end !important}.align-items-xl-center{align-items:center !important}.align-items-xl-baseline{align-items:baseline !important}.align-items-xl-stretch{align-items:stretch !important}.align-content-xl-start{align-content:flex-start !important}.align-content-xl-end{align-content:flex-end !important}.align-content-xl-center{align-content:center !important}.align-content-xl-between{align-content:space-between !important}.align-content-xl-around{align-content:space-around !important}.align-content-xl-stretch{align-content:stretch !important}.align-self-xl-auto{align-self:auto !important}.align-self-xl-start{align-self:flex-start !important}.align-self-xl-end{align-self:flex-end !important}.align-self-xl-center{align-self:center !important}.align-self-xl-baseline{align-self:baseline !important}.align-self-xl-stretch{align-self:stretch !important}.order-xl-first{order:-1 !important}.order-xl-0{order:0 !important}.order-xl-1{order:1 !important}.order-xl-2{order:2 !important}.order-xl-3{order:3 !important}.order-xl-4{order:4 !important}.order-xl-5{order:5 !important}.order-xl-last{order:6 !important}.m-xl-0{margin:0 !important}.m-xl-1{margin:.25rem !important}.m-xl-2{margin:.5rem !important}.m-xl-3{margin:1rem !important}.m-xl-4{margin:1.5rem !important}.m-xl-5{margin:3rem !important}.m-xl-auto{margin:auto !important}.mx-xl-0{margin-right:0 !important;margin-left:0 !important}.mx-xl-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-xl-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-xl-3{margin-right:1rem !important;margin-left:1rem !important}.mx-xl-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-xl-5{margin-right:3rem !important;margin-left:3rem !important}.mx-xl-auto{margin-right:auto !important;margin-left:auto !important}.my-xl-0{margin-top:0 !important;margin-bottom:0 !important}.my-xl-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-xl-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-xl-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-xl-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-xl-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-xl-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-xl-0{margin-top:0 !important}.mt-xl-1{margin-top:.25rem !important}.mt-xl-2{margin-top:.5rem !important}.mt-xl-3{margin-top:1rem !important}.mt-xl-4{margin-top:1.5rem !important}.mt-xl-5{margin-top:3rem !important}.mt-xl-auto{margin-top:auto !important}.me-xl-0{margin-right:0 !important}.me-xl-1{margin-right:.25rem !important}.me-xl-2{margin-right:.5rem !important}.me-xl-3{margin-right:1rem !important}.me-xl-4{margin-right:1.5rem !important}.me-xl-5{margin-right:3rem !important}.me-xl-auto{margin-right:auto !important}.mb-xl-0{margin-bottom:0 !important}.mb-xl-1{margin-bottom:.25rem !important}.mb-xl-2{margin-bottom:.5rem !important}.mb-xl-3{margin-bottom:1rem !important}.mb-xl-4{margin-bottom:1.5rem !important}.mb-xl-5{margin-bottom:3rem !important}.mb-xl-auto{margin-bottom:auto !important}.ms-xl-0{margin-left:0 !important}.ms-xl-1{margin-left:.25rem !important}.ms-xl-2{margin-left:.5rem !important}.ms-xl-3{margin-left:1rem !important}.ms-xl-4{margin-left:1.5rem !important}.ms-xl-5{margin-left:3rem !important}.ms-xl-auto{margin-left:auto !important}.p-xl-0{padding:0 !important}.p-xl-1{padding:.25rem !important}.p-xl-2{padding:.5rem !important}.p-xl-3{padding:1rem !important}.p-xl-4{padding:1.5rem !important}.p-xl-5{padding:3rem !important}.px-xl-0{padding-right:0 !important;padding-left:0 !important}.px-xl-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-xl-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-xl-3{padding-right:1rem !important;padding-left:1rem !important}.px-xl-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-xl-5{padding-right:3rem !important;padding-left:3rem !important}.py-xl-0{padding-top:0 !important;padding-bottom:0 !important}.py-xl-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-xl-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-xl-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-xl-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-xl-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-xl-0{padding-top:0 !important}.pt-xl-1{padding-top:.25rem !important}.pt-xl-2{padding-top:.5rem !important}.pt-xl-3{padding-top:1rem !important}.pt-xl-4{padding-top:1.5rem !important}.pt-xl-5{padding-top:3rem !important}.pe-xl-0{padding-right:0 !important}.pe-xl-1{padding-right:.25rem !important}.pe-xl-2{padding-right:.5rem !important}.pe-xl-3{padding-right:1rem !important}.pe-xl-4{padding-right:1.5rem !important}.pe-xl-5{padding-right:3rem !important}.pb-xl-0{padding-bottom:0 !important}.pb-xl-1{padding-bottom:.25rem !important}.pb-xl-2{padding-bottom:.5rem !important}.pb-xl-3{padding-bottom:1rem !important}.pb-xl-4{padding-bottom:1.5rem !important}.pb-xl-5{padding-bottom:3rem !important}.ps-xl-0{padding-left:0 !important}.ps-xl-1{padding-left:.25rem !important}.ps-xl-2{padding-left:.5rem !important}.ps-xl-3{padding-left:1rem !important}.ps-xl-4{padding-left:1.5rem !important}.ps-xl-5{padding-left:3rem !important}.gap-xl-0{gap:0 !important}.gap-xl-1{gap:.25rem !important}.gap-xl-2{gap:.5rem !important}.gap-xl-3{gap:1rem !important}.gap-xl-4{gap:1.5rem !important}.gap-xl-5{gap:3rem !important}.row-gap-xl-0{row-gap:0 !important}.row-gap-xl-1{row-gap:.25rem !important}.row-gap-xl-2{row-gap:.5rem !important}.row-gap-xl-3{row-gap:1rem !important}.row-gap-xl-4{row-gap:1.5rem !important}.row-gap-xl-5{row-gap:3rem !important}.column-gap-xl-0{column-gap:0 !important}.column-gap-xl-1{column-gap:.25rem !important}.column-gap-xl-2{column-gap:.5rem !important}.column-gap-xl-3{column-gap:1rem !important}.column-gap-xl-4{column-gap:1.5rem !important}.column-gap-xl-5{column-gap:3rem !important}.text-xl-start{text-align:left !important}.text-xl-end{text-align:right !important}.text-xl-center{text-align:center !important}}@media(min-width: 1400px){.float-xxl-start{float:left !important}.float-xxl-end{float:right !important}.float-xxl-none{float:none !important}.object-fit-xxl-contain{object-fit:contain !important}.object-fit-xxl-cover{object-fit:cover !important}.object-fit-xxl-fill{object-fit:fill !important}.object-fit-xxl-scale{object-fit:scale-down !important}.object-fit-xxl-none{object-fit:none !important}.d-xxl-inline{display:inline !important}.d-xxl-inline-block{display:inline-block !important}.d-xxl-block{display:block !important}.d-xxl-grid{display:grid !important}.d-xxl-inline-grid{display:inline-grid !important}.d-xxl-table{display:table !important}.d-xxl-table-row{display:table-row !important}.d-xxl-table-cell{display:table-cell !important}.d-xxl-flex{display:flex !important}.d-xxl-inline-flex{display:inline-flex !important}.d-xxl-none{display:none !important}.flex-xxl-fill{flex:1 1 auto !important}.flex-xxl-row{flex-direction:row !important}.flex-xxl-column{flex-direction:column !important}.flex-xxl-row-reverse{flex-direction:row-reverse !important}.flex-xxl-column-reverse{flex-direction:column-reverse !important}.flex-xxl-grow-0{flex-grow:0 !important}.flex-xxl-grow-1{flex-grow:1 !important}.flex-xxl-shrink-0{flex-shrink:0 !important}.flex-xxl-shrink-1{flex-shrink:1 !important}.flex-xxl-wrap{flex-wrap:wrap !important}.flex-xxl-nowrap{flex-wrap:nowrap !important}.flex-xxl-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-xxl-start{justify-content:flex-start !important}.justify-content-xxl-end{justify-content:flex-end !important}.justify-content-xxl-center{justify-content:center !important}.justify-content-xxl-between{justify-content:space-between !important}.justify-content-xxl-around{justify-content:space-around !important}.justify-content-xxl-evenly{justify-content:space-evenly !important}.align-items-xxl-start{align-items:flex-start !important}.align-items-xxl-end{align-items:flex-end !important}.align-items-xxl-center{align-items:center !important}.align-items-xxl-baseline{align-items:baseline !important}.align-items-xxl-stretch{align-items:stretch !important}.align-content-xxl-start{align-content:flex-start !important}.align-content-xxl-end{align-content:flex-end !important}.align-content-xxl-center{align-content:center !important}.align-content-xxl-between{align-content:space-between !important}.align-content-xxl-around{align-content:space-around !important}.align-content-xxl-stretch{align-content:stretch !important}.align-self-xxl-auto{align-self:auto !important}.align-self-xxl-start{align-self:flex-start !important}.align-self-xxl-end{align-self:flex-end !important}.align-self-xxl-center{align-self:center !important}.align-self-xxl-baseline{align-self:baseline !important}.align-self-xxl-stretch{align-self:stretch !important}.order-xxl-first{order:-1 !important}.order-xxl-0{order:0 !important}.order-xxl-1{order:1 !important}.order-xxl-2{order:2 !important}.order-xxl-3{order:3 !important}.order-xxl-4{order:4 !important}.order-xxl-5{order:5 !important}.order-xxl-last{order:6 !important}.m-xxl-0{margin:0 !important}.m-xxl-1{margin:.25rem !important}.m-xxl-2{margin:.5rem !important}.m-xxl-3{margin:1rem !important}.m-xxl-4{margin:1.5rem !important}.m-xxl-5{margin:3rem !important}.m-xxl-auto{margin:auto !important}.mx-xxl-0{margin-right:0 !important;margin-left:0 !important}.mx-xxl-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-xxl-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-xxl-3{margin-right:1rem !important;margin-left:1rem !important}.mx-xxl-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-xxl-5{margin-right:3rem !important;margin-left:3rem !important}.mx-xxl-auto{margin-right:auto !important;margin-left:auto !important}.my-xxl-0{margin-top:0 !important;margin-bottom:0 !important}.my-xxl-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-xxl-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-xxl-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-xxl-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-xxl-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-xxl-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-xxl-0{margin-top:0 !important}.mt-xxl-1{margin-top:.25rem !important}.mt-xxl-2{margin-top:.5rem !important}.mt-xxl-3{margin-top:1rem !important}.mt-xxl-4{margin-top:1.5rem !important}.mt-xxl-5{margin-top:3rem !important}.mt-xxl-auto{margin-top:auto !important}.me-xxl-0{margin-right:0 !important}.me-xxl-1{margin-right:.25rem !important}.me-xxl-2{margin-right:.5rem !important}.me-xxl-3{margin-right:1rem !important}.me-xxl-4{margin-right:1.5rem !important}.me-xxl-5{margin-right:3rem !important}.me-xxl-auto{margin-right:auto !important}.mb-xxl-0{margin-bottom:0 !important}.mb-xxl-1{margin-bottom:.25rem !important}.mb-xxl-2{margin-bottom:.5rem !important}.mb-xxl-3{margin-bottom:1rem !important}.mb-xxl-4{margin-bottom:1.5rem !important}.mb-xxl-5{margin-bottom:3rem !important}.mb-xxl-auto{margin-bottom:auto !important}.ms-xxl-0{margin-left:0 !important}.ms-xxl-1{margin-left:.25rem !important}.ms-xxl-2{margin-left:.5rem !important}.ms-xxl-3{margin-left:1rem !important}.ms-xxl-4{margin-left:1.5rem !important}.ms-xxl-5{margin-left:3rem !important}.ms-xxl-auto{margin-left:auto !important}.p-xxl-0{padding:0 !important}.p-xxl-1{padding:.25rem !important}.p-xxl-2{padding:.5rem !important}.p-xxl-3{padding:1rem !important}.p-xxl-4{padding:1.5rem !important}.p-xxl-5{padding:3rem !important}.px-xxl-0{padding-right:0 !important;padding-left:0 !important}.px-xxl-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-xxl-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-xxl-3{padding-right:1rem !important;padding-left:1rem !important}.px-xxl-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-xxl-5{padding-right:3rem !important;padding-left:3rem !important}.py-xxl-0{padding-top:0 !important;padding-bottom:0 !important}.py-xxl-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-xxl-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-xxl-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-xxl-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-xxl-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-xxl-0{padding-top:0 !important}.pt-xxl-1{padding-top:.25rem !important}.pt-xxl-2{padding-top:.5rem !important}.pt-xxl-3{padding-top:1rem !important}.pt-xxl-4{padding-top:1.5rem !important}.pt-xxl-5{padding-top:3rem !important}.pe-xxl-0{padding-right:0 !important}.pe-xxl-1{padding-right:.25rem !important}.pe-xxl-2{padding-right:.5rem !important}.pe-xxl-3{padding-right:1rem !important}.pe-xxl-4{padding-right:1.5rem !important}.pe-xxl-5{padding-right:3rem !important}.pb-xxl-0{padding-bottom:0 !important}.pb-xxl-1{padding-bottom:.25rem !important}.pb-xxl-2{padding-bottom:.5rem !important}.pb-xxl-3{padding-bottom:1rem !important}.pb-xxl-4{padding-bottom:1.5rem !important}.pb-xxl-5{padding-bottom:3rem !important}.ps-xxl-0{padding-left:0 !important}.ps-xxl-1{padding-left:.25rem !important}.ps-xxl-2{padding-left:.5rem !important}.ps-xxl-3{padding-left:1rem !important}.ps-xxl-4{padding-left:1.5rem !important}.ps-xxl-5{padding-left:3rem !important}.gap-xxl-0{gap:0 !important}.gap-xxl-1{gap:.25rem !important}.gap-xxl-2{gap:.5rem !important}.gap-xxl-3{gap:1rem !important}.gap-xxl-4{gap:1.5rem !important}.gap-xxl-5{gap:3rem !important}.row-gap-xxl-0{row-gap:0 !important}.row-gap-xxl-1{row-gap:.25rem !important}.row-gap-xxl-2{row-gap:.5rem !important}.row-gap-xxl-3{row-gap:1rem !important}.row-gap-xxl-4{row-gap:1.5rem !important}.row-gap-xxl-5{row-gap:3rem !important}.column-gap-xxl-0{column-gap:0 !important}.column-gap-xxl-1{column-gap:.25rem !important}.column-gap-xxl-2{column-gap:.5rem !important}.column-gap-xxl-3{column-gap:1rem !important}.column-gap-xxl-4{column-gap:1.5rem !important}.column-gap-xxl-5{column-gap:3rem !important}.text-xxl-start{text-align:left !important}.text-xxl-end{text-align:right !important}.text-xxl-center{text-align:center !important}}.bg-default{color:#fff}.bg-primary{color:#fff}.bg-secondary{color:#fff}.bg-success{color:#fff}.bg-info{color:#fff}.bg-warning{color:#fff}.bg-danger{color:#fff}.bg-light{color:#fff}.bg-dark{color:#fff}@media(min-width: 1200px){.fs-1{font-size:2rem !important}.fs-2{font-size:1.65rem !important}.fs-3{font-size:1.45rem !important}}@media print{.d-print-inline{display:inline !important}.d-print-inline-block{display:inline-block !important}.d-print-block{display:block !important}.d-print-grid{display:grid !important}.d-print-inline-grid{display:inline-grid !important}.d-print-table{display:table !important}.d-print-table-row{display:table-row !important}.d-print-table-cell{display:table-cell !important}.d-print-flex{display:flex !important}.d-print-inline-flex{display:inline-flex !important}.d-print-none{display:none !important}}:root{--bslib-spacer: 1rem;--bslib-mb-spacer: var(--bslib-spacer, 1rem)}.bslib-mb-spacing{margin-bottom:var(--bslib-mb-spacer)}.bslib-gap-spacing{gap:var(--bslib-mb-spacer)}.bslib-gap-spacing>.bslib-mb-spacing,.bslib-gap-spacing>.form-group,.bslib-gap-spacing>p,.bslib-gap-spacing>pre{margin-bottom:0}.html-fill-container>.html-fill-item.bslib-mb-spacing{margin-bottom:0}.tab-content>.tab-pane.html-fill-container{display:none}.tab-content>.active.html-fill-container{display:flex}.tab-content.html-fill-container{padding:0}.bg-blue{--bslib-color-bg: #375a7f;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-blue{--bslib-color-fg: #375a7f;color:var(--bslib-color-fg)}.bg-indigo{--bslib-color-bg: #6610f2;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-indigo{--bslib-color-fg: #6610f2;color:var(--bslib-color-fg)}.bg-purple{--bslib-color-bg: #6f42c1;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-purple{--bslib-color-fg: #6f42c1;color:var(--bslib-color-fg)}.bg-pink{--bslib-color-bg: #e83e8c;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-pink{--bslib-color-fg: #e83e8c;color:var(--bslib-color-fg)}.bg-red{--bslib-color-bg: #e74c3c;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-red{--bslib-color-fg: #e74c3c;color:var(--bslib-color-fg)}.bg-orange{--bslib-color-bg: #fd7e14;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-orange{--bslib-color-fg: #fd7e14;color:var(--bslib-color-fg)}.bg-yellow{--bslib-color-bg: #f39c12;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-yellow{--bslib-color-fg: #f39c12;color:var(--bslib-color-fg)}.bg-green{--bslib-color-bg: #00bc8c;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-green{--bslib-color-fg: #00bc8c;color:var(--bslib-color-fg)}.bg-teal{--bslib-color-bg: #20c997;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-teal{--bslib-color-fg: #20c997;color:var(--bslib-color-fg)}.bg-cyan{--bslib-color-bg: #3498db;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-cyan{--bslib-color-fg: #3498db;color:var(--bslib-color-fg)}.text-default{--bslib-color-fg: #434343}.bg-default{--bslib-color-bg: #434343;--bslib-color-fg: #fff}.text-primary{--bslib-color-fg: #375a7f}.bg-primary{--bslib-color-bg: #375a7f;--bslib-color-fg: #fff}.text-secondary{--bslib-color-fg: #434343}.bg-secondary{--bslib-color-bg: #434343;--bslib-color-fg: #fff}.text-success{--bslib-color-fg: #00bc8c}.bg-success{--bslib-color-bg: #00bc8c;--bslib-color-fg: #fff}.text-info{--bslib-color-fg: #3498db}.bg-info{--bslib-color-bg: #3498db;--bslib-color-fg: #fff}.text-warning{--bslib-color-fg: #f39c12}.bg-warning{--bslib-color-bg: #f39c12;--bslib-color-fg: #fff}.text-danger{--bslib-color-fg: #e74c3c}.bg-danger{--bslib-color-bg: #e74c3c;--bslib-color-fg: #fff}.text-light{--bslib-color-fg: #6f6f6f}.bg-light{--bslib-color-bg: #6f6f6f;--bslib-color-fg: #fff}.text-dark{--bslib-color-fg: #2d2d2d}.bg-dark{--bslib-color-bg: #2d2d2d;--bslib-color-fg: #fff}.bg-gradient-blue-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #4a3cad;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #4a3cad;color:#fff}.bg-gradient-blue-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4d5099;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4d5099;color:#fff}.bg-gradient-blue-pink{--bslib-color-fg: #fff;--bslib-color-bg: #7e4f84;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #7e4f84;color:#fff}.bg-gradient-blue-red{--bslib-color-fg: #fff;--bslib-color-bg: #7d5464;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #7d5464;color:#fff}.bg-gradient-blue-orange{--bslib-color-fg: #fff;--bslib-color-bg: #866854;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #866854;color:#fff}.bg-gradient-blue-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #827453;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #827453;color:#fff}.bg-gradient-blue-green{--bslib-color-fg: #fff;--bslib-color-bg: #218184;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #218184;color:#fff}.bg-gradient-blue-teal{--bslib-color-fg: #fff;--bslib-color-bg: #2e8689;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #2e8689;color:#fff}.bg-gradient-blue-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #3673a4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #3673a4;color:#fff}.bg-gradient-indigo-blue{--bslib-color-fg: #fff;--bslib-color-bg: #532ec4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #532ec4;color:#fff}.bg-gradient-indigo-purple{--bslib-color-fg: #fff;--bslib-color-bg: #6a24de;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #6a24de;color:#fff}.bg-gradient-indigo-pink{--bslib-color-fg: #fff;--bslib-color-bg: #9a22c9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #9a22c9;color:#fff}.bg-gradient-indigo-red{--bslib-color-fg: #fff;--bslib-color-bg: #9a28a9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #9a28a9;color:#fff}.bg-gradient-indigo-orange{--bslib-color-fg: #fff;--bslib-color-bg: #a23c99;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #a23c99;color:#fff}.bg-gradient-indigo-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #9e4898;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #9e4898;color:#fff}.bg-gradient-indigo-green{--bslib-color-fg: #fff;--bslib-color-bg: #3d55c9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #3d55c9;color:#fff}.bg-gradient-indigo-teal{--bslib-color-fg: #fff;--bslib-color-bg: #4a5ace;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #4a5ace;color:#fff}.bg-gradient-indigo-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #5246e9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #5246e9;color:#fff}.bg-gradient-purple-blue{--bslib-color-fg: #fff;--bslib-color-bg: #594ca7;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #594ca7;color:#fff}.bg-gradient-purple-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #6b2ed5;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #6b2ed5;color:#fff}.bg-gradient-purple-pink{--bslib-color-fg: #fff;--bslib-color-bg: #9f40ac;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #9f40ac;color:#fff}.bg-gradient-purple-red{--bslib-color-fg: #fff;--bslib-color-bg: #9f468c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #9f468c;color:#fff}.bg-gradient-purple-orange{--bslib-color-fg: #fff;--bslib-color-bg: #a85a7c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #a85a7c;color:#fff}.bg-gradient-purple-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #a4667b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #a4667b;color:#fff}.bg-gradient-purple-green{--bslib-color-fg: #fff;--bslib-color-bg: #4373ac;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #4373ac;color:#fff}.bg-gradient-purple-teal{--bslib-color-fg: #fff;--bslib-color-bg: #4f78b0;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #4f78b0;color:#fff}.bg-gradient-purple-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #5764cb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #5764cb;color:#fff}.bg-gradient-pink-blue{--bslib-color-fg: #fff;--bslib-color-bg: #a14987;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #a14987;color:#fff}.bg-gradient-pink-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #b42cb5;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #b42cb5;color:#fff}.bg-gradient-pink-purple{--bslib-color-fg: #fff;--bslib-color-bg: #b840a1;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #b840a1;color:#fff}.bg-gradient-pink-red{--bslib-color-fg: #fff;--bslib-color-bg: #e8446c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #e8446c;color:#fff}.bg-gradient-pink-orange{--bslib-color-fg: #fff;--bslib-color-bg: #f0585c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #f0585c;color:#fff}.bg-gradient-pink-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #ec645b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #ec645b;color:#fff}.bg-gradient-pink-green{--bslib-color-fg: #fff;--bslib-color-bg: #8b708c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #8b708c;color:#fff}.bg-gradient-pink-teal{--bslib-color-fg: #fff;--bslib-color-bg: #987690;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #987690;color:#fff}.bg-gradient-pink-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #a062ac;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #a062ac;color:#fff}.bg-gradient-red-blue{--bslib-color-fg: #fff;--bslib-color-bg: #a15257;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #a15257;color:#fff}.bg-gradient-red-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #b33485;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #b33485;color:#fff}.bg-gradient-red-purple{--bslib-color-fg: #fff;--bslib-color-bg: #b74871;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #b74871;color:#fff}.bg-gradient-red-pink{--bslib-color-fg: #fff;--bslib-color-bg: #e7465c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #e7465c;color:#fff}.bg-gradient-red-orange{--bslib-color-fg: #fff;--bslib-color-bg: #f0602c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #f0602c;color:#fff}.bg-gradient-red-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #ec6c2b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #ec6c2b;color:#fff}.bg-gradient-red-green{--bslib-color-fg: #fff;--bslib-color-bg: #8b795c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #8b795c;color:#fff}.bg-gradient-red-teal{--bslib-color-fg: #fff;--bslib-color-bg: #977e60;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #977e60;color:#fff}.bg-gradient-red-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #9f6a7c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #9f6a7c;color:#fff}.bg-gradient-orange-blue{--bslib-color-fg: #fff;--bslib-color-bg: #ae703f;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #ae703f;color:#fff}.bg-gradient-orange-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #c1526d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #c1526d;color:#fff}.bg-gradient-orange-purple{--bslib-color-fg: #fff;--bslib-color-bg: #c46659;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #c46659;color:#fff}.bg-gradient-orange-pink{--bslib-color-fg: #fff;--bslib-color-bg: #f56444;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #f56444;color:#fff}.bg-gradient-orange-red{--bslib-color-fg: #fff;--bslib-color-bg: #f46a24;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #f46a24;color:#fff}.bg-gradient-orange-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #f98a13;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #f98a13;color:#fff}.bg-gradient-orange-green{--bslib-color-fg: #fff;--bslib-color-bg: #989744;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #989744;color:#fff}.bg-gradient-orange-teal{--bslib-color-fg: #fff;--bslib-color-bg: #a59c48;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #a59c48;color:#fff}.bg-gradient-orange-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #ad8864;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #ad8864;color:#fff}.bg-gradient-yellow-blue{--bslib-color-fg: #fff;--bslib-color-bg: #a8823e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #a8823e;color:#fff}.bg-gradient-yellow-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #bb646c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #bb646c;color:#fff}.bg-gradient-yellow-purple{--bslib-color-fg: #fff;--bslib-color-bg: #be7858;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #be7858;color:#fff}.bg-gradient-yellow-pink{--bslib-color-fg: #fff;--bslib-color-bg: #ef7643;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #ef7643;color:#fff}.bg-gradient-yellow-red{--bslib-color-fg: #fff;--bslib-color-bg: #ee7c23;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #ee7c23;color:#fff}.bg-gradient-yellow-orange{--bslib-color-fg: #fff;--bslib-color-bg: #f79013;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #f79013;color:#fff}.bg-gradient-yellow-green{--bslib-color-fg: #fff;--bslib-color-bg: #92a943;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #92a943;color:#fff}.bg-gradient-yellow-teal{--bslib-color-fg: #fff;--bslib-color-bg: #9fae47;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #9fae47;color:#fff}.bg-gradient-yellow-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #a79a62;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #a79a62;color:#fff}.bg-gradient-green-blue{--bslib-color-fg: #fff;--bslib-color-bg: #169587;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #169587;color:#fff}.bg-gradient-green-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #2977b5;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #2977b5;color:#fff}.bg-gradient-green-purple{--bslib-color-fg: #fff;--bslib-color-bg: #2c8ba1;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #2c8ba1;color:#fff}.bg-gradient-green-pink{--bslib-color-fg: #fff;--bslib-color-bg: #5d8a8c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #5d8a8c;color:#fff}.bg-gradient-green-red{--bslib-color-fg: #fff;--bslib-color-bg: #5c8f6c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #5c8f6c;color:#fff}.bg-gradient-green-orange{--bslib-color-fg: #fff;--bslib-color-bg: #65a35c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #65a35c;color:#fff}.bg-gradient-green-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #61af5b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #61af5b;color:#fff}.bg-gradient-green-teal{--bslib-color-fg: #fff;--bslib-color-bg: #0dc190;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #0dc190;color:#fff}.bg-gradient-green-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #15aeac;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #15aeac;color:#fff}.bg-gradient-teal-blue{--bslib-color-fg: #fff;--bslib-color-bg: #299d8d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #299d8d;color:#fff}.bg-gradient-teal-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #3c7fbb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #3c7fbb;color:#fff}.bg-gradient-teal-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4093a8;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4093a8;color:#fff}.bg-gradient-teal-pink{--bslib-color-fg: #fff;--bslib-color-bg: #709193;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #709193;color:#fff}.bg-gradient-teal-red{--bslib-color-fg: #fff;--bslib-color-bg: #709773;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #709773;color:#fff}.bg-gradient-teal-orange{--bslib-color-fg: #fff;--bslib-color-bg: #78ab63;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #78ab63;color:#fff}.bg-gradient-teal-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #74b762;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #74b762;color:#fff}.bg-gradient-teal-green{--bslib-color-fg: #fff;--bslib-color-bg: #13c493;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #13c493;color:#fff}.bg-gradient-teal-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #28b5b2;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #28b5b2;color:#fff}.bg-gradient-cyan-blue{--bslib-color-fg: #fff;--bslib-color-bg: #357fb6;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #357fb6;color:#fff}.bg-gradient-cyan-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #4862e4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #4862e4;color:#fff}.bg-gradient-cyan-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4c76d1;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4c76d1;color:#fff}.bg-gradient-cyan-pink{--bslib-color-fg: #fff;--bslib-color-bg: #7c74bb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #7c74bb;color:#fff}.bg-gradient-cyan-red{--bslib-color-fg: #fff;--bslib-color-bg: #7c7a9b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #7c7a9b;color:#fff}.bg-gradient-cyan-orange{--bslib-color-fg: #fff;--bslib-color-bg: #848e8b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #848e8b;color:#fff}.bg-gradient-cyan-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #809a8b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #809a8b;color:#fff}.bg-gradient-cyan-green{--bslib-color-fg: #fff;--bslib-color-bg: #1fa6bb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #1fa6bb;color:#fff}.bg-gradient-cyan-teal{--bslib-color-fg: #fff;--bslib-color-bg: #2cacc0;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #2cacc0;color:#fff}.bg-blue{--bslib-color-bg: #375a7f;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-blue{--bslib-color-fg: #375a7f;color:var(--bslib-color-fg)}.bg-indigo{--bslib-color-bg: #6610f2;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-indigo{--bslib-color-fg: #6610f2;color:var(--bslib-color-fg)}.bg-purple{--bslib-color-bg: #6f42c1;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-purple{--bslib-color-fg: #6f42c1;color:var(--bslib-color-fg)}.bg-pink{--bslib-color-bg: #e83e8c;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-pink{--bslib-color-fg: #e83e8c;color:var(--bslib-color-fg)}.bg-red{--bslib-color-bg: #e74c3c;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-red{--bslib-color-fg: #e74c3c;color:var(--bslib-color-fg)}.bg-orange{--bslib-color-bg: #fd7e14;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-orange{--bslib-color-fg: #fd7e14;color:var(--bslib-color-fg)}.bg-yellow{--bslib-color-bg: #f39c12;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-yellow{--bslib-color-fg: #f39c12;color:var(--bslib-color-fg)}.bg-green{--bslib-color-bg: #00bc8c;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-green{--bslib-color-fg: #00bc8c;color:var(--bslib-color-fg)}.bg-teal{--bslib-color-bg: #20c997;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-teal{--bslib-color-fg: #20c997;color:var(--bslib-color-fg)}.bg-cyan{--bslib-color-bg: #3498db;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-cyan{--bslib-color-fg: #3498db;color:var(--bslib-color-fg)}.text-default{--bslib-color-fg: #434343}.bg-default{--bslib-color-bg: #434343;--bslib-color-fg: #fff}.text-primary{--bslib-color-fg: #375a7f}.bg-primary{--bslib-color-bg: #375a7f;--bslib-color-fg: #fff}.text-secondary{--bslib-color-fg: #434343}.bg-secondary{--bslib-color-bg: #434343;--bslib-color-fg: #fff}.text-success{--bslib-color-fg: #00bc8c}.bg-success{--bslib-color-bg: #00bc8c;--bslib-color-fg: #fff}.text-info{--bslib-color-fg: #3498db}.bg-info{--bslib-color-bg: #3498db;--bslib-color-fg: #fff}.text-warning{--bslib-color-fg: #f39c12}.bg-warning{--bslib-color-bg: #f39c12;--bslib-color-fg: #fff}.text-danger{--bslib-color-fg: #e74c3c}.bg-danger{--bslib-color-bg: #e74c3c;--bslib-color-fg: #fff}.text-light{--bslib-color-fg: #6f6f6f}.bg-light{--bslib-color-bg: #6f6f6f;--bslib-color-fg: #fff}.text-dark{--bslib-color-fg: #2d2d2d}.bg-dark{--bslib-color-bg: #2d2d2d;--bslib-color-fg: #fff}.bg-gradient-blue-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #4a3cad;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #4a3cad;color:#fff}.bg-gradient-blue-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4d5099;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4d5099;color:#fff}.bg-gradient-blue-pink{--bslib-color-fg: #fff;--bslib-color-bg: #7e4f84;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #7e4f84;color:#fff}.bg-gradient-blue-red{--bslib-color-fg: #fff;--bslib-color-bg: #7d5464;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #7d5464;color:#fff}.bg-gradient-blue-orange{--bslib-color-fg: #fff;--bslib-color-bg: #866854;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #866854;color:#fff}.bg-gradient-blue-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #827453;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #827453;color:#fff}.bg-gradient-blue-green{--bslib-color-fg: #fff;--bslib-color-bg: #218184;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #218184;color:#fff}.bg-gradient-blue-teal{--bslib-color-fg: #fff;--bslib-color-bg: #2e8689;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #2e8689;color:#fff}.bg-gradient-blue-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #3673a4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #375a7f var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #3673a4;color:#fff}.bg-gradient-indigo-blue{--bslib-color-fg: #fff;--bslib-color-bg: #532ec4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #532ec4;color:#fff}.bg-gradient-indigo-purple{--bslib-color-fg: #fff;--bslib-color-bg: #6a24de;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #6a24de;color:#fff}.bg-gradient-indigo-pink{--bslib-color-fg: #fff;--bslib-color-bg: #9a22c9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #9a22c9;color:#fff}.bg-gradient-indigo-red{--bslib-color-fg: #fff;--bslib-color-bg: #9a28a9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #9a28a9;color:#fff}.bg-gradient-indigo-orange{--bslib-color-fg: #fff;--bslib-color-bg: #a23c99;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #a23c99;color:#fff}.bg-gradient-indigo-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #9e4898;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #9e4898;color:#fff}.bg-gradient-indigo-green{--bslib-color-fg: #fff;--bslib-color-bg: #3d55c9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #3d55c9;color:#fff}.bg-gradient-indigo-teal{--bslib-color-fg: #fff;--bslib-color-bg: #4a5ace;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #4a5ace;color:#fff}.bg-gradient-indigo-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #5246e9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #5246e9;color:#fff}.bg-gradient-purple-blue{--bslib-color-fg: #fff;--bslib-color-bg: #594ca7;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #594ca7;color:#fff}.bg-gradient-purple-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #6b2ed5;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #6b2ed5;color:#fff}.bg-gradient-purple-pink{--bslib-color-fg: #fff;--bslib-color-bg: #9f40ac;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #9f40ac;color:#fff}.bg-gradient-purple-red{--bslib-color-fg: #fff;--bslib-color-bg: #9f468c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #9f468c;color:#fff}.bg-gradient-purple-orange{--bslib-color-fg: #fff;--bslib-color-bg: #a85a7c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #a85a7c;color:#fff}.bg-gradient-purple-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #a4667b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #a4667b;color:#fff}.bg-gradient-purple-green{--bslib-color-fg: #fff;--bslib-color-bg: #4373ac;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #4373ac;color:#fff}.bg-gradient-purple-teal{--bslib-color-fg: #fff;--bslib-color-bg: #4f78b0;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #4f78b0;color:#fff}.bg-gradient-purple-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #5764cb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #5764cb;color:#fff}.bg-gradient-pink-blue{--bslib-color-fg: #fff;--bslib-color-bg: #a14987;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #a14987;color:#fff}.bg-gradient-pink-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #b42cb5;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #b42cb5;color:#fff}.bg-gradient-pink-purple{--bslib-color-fg: #fff;--bslib-color-bg: #b840a1;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #b840a1;color:#fff}.bg-gradient-pink-red{--bslib-color-fg: #fff;--bslib-color-bg: #e8446c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #e8446c;color:#fff}.bg-gradient-pink-orange{--bslib-color-fg: #fff;--bslib-color-bg: #f0585c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #f0585c;color:#fff}.bg-gradient-pink-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #ec645b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #ec645b;color:#fff}.bg-gradient-pink-green{--bslib-color-fg: #fff;--bslib-color-bg: #8b708c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #8b708c;color:#fff}.bg-gradient-pink-teal{--bslib-color-fg: #fff;--bslib-color-bg: #987690;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #987690;color:#fff}.bg-gradient-pink-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #a062ac;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #a062ac;color:#fff}.bg-gradient-red-blue{--bslib-color-fg: #fff;--bslib-color-bg: #a15257;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #a15257;color:#fff}.bg-gradient-red-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #b33485;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #b33485;color:#fff}.bg-gradient-red-purple{--bslib-color-fg: #fff;--bslib-color-bg: #b74871;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #b74871;color:#fff}.bg-gradient-red-pink{--bslib-color-fg: #fff;--bslib-color-bg: #e7465c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #e7465c;color:#fff}.bg-gradient-red-orange{--bslib-color-fg: #fff;--bslib-color-bg: #f0602c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #f0602c;color:#fff}.bg-gradient-red-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #ec6c2b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #ec6c2b;color:#fff}.bg-gradient-red-green{--bslib-color-fg: #fff;--bslib-color-bg: #8b795c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #8b795c;color:#fff}.bg-gradient-red-teal{--bslib-color-fg: #fff;--bslib-color-bg: #977e60;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #977e60;color:#fff}.bg-gradient-red-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #9f6a7c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e74c3c var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #9f6a7c;color:#fff}.bg-gradient-orange-blue{--bslib-color-fg: #fff;--bslib-color-bg: #ae703f;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #ae703f;color:#fff}.bg-gradient-orange-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #c1526d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #c1526d;color:#fff}.bg-gradient-orange-purple{--bslib-color-fg: #fff;--bslib-color-bg: #c46659;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #c46659;color:#fff}.bg-gradient-orange-pink{--bslib-color-fg: #fff;--bslib-color-bg: #f56444;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #f56444;color:#fff}.bg-gradient-orange-red{--bslib-color-fg: #fff;--bslib-color-bg: #f46a24;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #f46a24;color:#fff}.bg-gradient-orange-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #f98a13;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #f98a13;color:#fff}.bg-gradient-orange-green{--bslib-color-fg: #fff;--bslib-color-bg: #989744;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #989744;color:#fff}.bg-gradient-orange-teal{--bslib-color-fg: #fff;--bslib-color-bg: #a59c48;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #a59c48;color:#fff}.bg-gradient-orange-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #ad8864;background:linear-gradient(var(--bg-gradient-deg, 140deg), #fd7e14 var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #ad8864;color:#fff}.bg-gradient-yellow-blue{--bslib-color-fg: #fff;--bslib-color-bg: #a8823e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #a8823e;color:#fff}.bg-gradient-yellow-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #bb646c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #bb646c;color:#fff}.bg-gradient-yellow-purple{--bslib-color-fg: #fff;--bslib-color-bg: #be7858;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #be7858;color:#fff}.bg-gradient-yellow-pink{--bslib-color-fg: #fff;--bslib-color-bg: #ef7643;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #ef7643;color:#fff}.bg-gradient-yellow-red{--bslib-color-fg: #fff;--bslib-color-bg: #ee7c23;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #ee7c23;color:#fff}.bg-gradient-yellow-orange{--bslib-color-fg: #fff;--bslib-color-bg: #f79013;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #f79013;color:#fff}.bg-gradient-yellow-green{--bslib-color-fg: #fff;--bslib-color-bg: #92a943;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #92a943;color:#fff}.bg-gradient-yellow-teal{--bslib-color-fg: #fff;--bslib-color-bg: #9fae47;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #9fae47;color:#fff}.bg-gradient-yellow-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #a79a62;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f39c12 var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #a79a62;color:#fff}.bg-gradient-green-blue{--bslib-color-fg: #fff;--bslib-color-bg: #169587;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #169587;color:#fff}.bg-gradient-green-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #2977b5;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #2977b5;color:#fff}.bg-gradient-green-purple{--bslib-color-fg: #fff;--bslib-color-bg: #2c8ba1;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #2c8ba1;color:#fff}.bg-gradient-green-pink{--bslib-color-fg: #fff;--bslib-color-bg: #5d8a8c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #5d8a8c;color:#fff}.bg-gradient-green-red{--bslib-color-fg: #fff;--bslib-color-bg: #5c8f6c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #5c8f6c;color:#fff}.bg-gradient-green-orange{--bslib-color-fg: #fff;--bslib-color-bg: #65a35c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #65a35c;color:#fff}.bg-gradient-green-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #61af5b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #61af5b;color:#fff}.bg-gradient-green-teal{--bslib-color-fg: #fff;--bslib-color-bg: #0dc190;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #0dc190;color:#fff}.bg-gradient-green-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #15aeac;background:linear-gradient(var(--bg-gradient-deg, 140deg), #00bc8c var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #15aeac;color:#fff}.bg-gradient-teal-blue{--bslib-color-fg: #fff;--bslib-color-bg: #299d8d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #299d8d;color:#fff}.bg-gradient-teal-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #3c7fbb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #3c7fbb;color:#fff}.bg-gradient-teal-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4093a8;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4093a8;color:#fff}.bg-gradient-teal-pink{--bslib-color-fg: #fff;--bslib-color-bg: #709193;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #709193;color:#fff}.bg-gradient-teal-red{--bslib-color-fg: #fff;--bslib-color-bg: #709773;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #709773;color:#fff}.bg-gradient-teal-orange{--bslib-color-fg: #fff;--bslib-color-bg: #78ab63;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #78ab63;color:#fff}.bg-gradient-teal-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #74b762;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #74b762;color:#fff}.bg-gradient-teal-green{--bslib-color-fg: #fff;--bslib-color-bg: #13c493;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #13c493;color:#fff}.bg-gradient-teal-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #28b5b2;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #3498db var(--bg-gradient-end, 180%)) #28b5b2;color:#fff}.bg-gradient-cyan-blue{--bslib-color-fg: #fff;--bslib-color-bg: #357fb6;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #375a7f var(--bg-gradient-end, 180%)) #357fb6;color:#fff}.bg-gradient-cyan-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #4862e4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #4862e4;color:#fff}.bg-gradient-cyan-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4c76d1;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4c76d1;color:#fff}.bg-gradient-cyan-pink{--bslib-color-fg: #fff;--bslib-color-bg: #7c74bb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #7c74bb;color:#fff}.bg-gradient-cyan-red{--bslib-color-fg: #fff;--bslib-color-bg: #7c7a9b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #e74c3c var(--bg-gradient-end, 180%)) #7c7a9b;color:#fff}.bg-gradient-cyan-orange{--bslib-color-fg: #fff;--bslib-color-bg: #848e8b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #fd7e14 var(--bg-gradient-end, 180%)) #848e8b;color:#fff}.bg-gradient-cyan-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #809a8b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #f39c12 var(--bg-gradient-end, 180%)) #809a8b;color:#fff}.bg-gradient-cyan-green{--bslib-color-fg: #fff;--bslib-color-bg: #1fa6bb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #00bc8c var(--bg-gradient-end, 180%)) #1fa6bb;color:#fff}.bg-gradient-cyan-teal{--bslib-color-fg: #fff;--bslib-color-bg: #2cacc0;background:linear-gradient(var(--bg-gradient-deg, 140deg), #3498db var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #2cacc0;color:#fff}:root{--bslib-spacer: 1rem;--bslib-mb-spacer: var(--bslib-spacer, 1rem)}.bslib-mb-spacing{margin-bottom:var(--bslib-mb-spacer)}.bslib-gap-spacing{gap:var(--bslib-mb-spacer)}.bslib-gap-spacing>.bslib-mb-spacing,.bslib-gap-spacing>.form-group,.bslib-gap-spacing>p,.bslib-gap-spacing>pre{margin-bottom:0}.html-fill-container>.html-fill-item.bslib-mb-spacing{margin-bottom:0}.tab-content>.tab-pane.html-fill-container{display:none}.tab-content>.active.html-fill-container{display:flex}.tab-content.html-fill-container{padding:0}.accordion .accordion-header{font-size:calc(1.29rem + 0.48vw);margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2;color:var(--bs-heading-color);margin-bottom:0}@media(min-width: 1200px){.accordion .accordion-header{font-size:1.65rem}}.accordion .accordion-icon:not(:empty){margin-right:.75rem;display:flex}.accordion .accordion-button:not(.collapsed){box-shadow:none}.accordion .accordion-button:not(.collapsed):focus{box-shadow:var(--bs-accordion-btn-focus-box-shadow)}.bslib-card{overflow:auto}.bslib-card .card-body+.card-body{padding-top:0}.bslib-card .card-body{overflow:auto}.bslib-card .card-body p{margin-top:0}.bslib-card .card-body p:last-child{margin-bottom:0}.bslib-card .card-body{max-height:var(--bslib-card-body-max-height, none)}.bslib-card[data-full-screen=true]>.card-body{max-height:var(--bslib-card-body-max-height-full-screen, none)}.bslib-card .card-header .form-group{margin-bottom:0}.bslib-card .card-header .selectize-control{margin-bottom:0}.bslib-card .card-header .selectize-control .item{margin-right:1.15rem}.bslib-card .card-footer{margin-top:auto}.bslib-card .bslib-navs-card-title{display:flex;flex-wrap:wrap;justify-content:space-between;align-items:center}.bslib-card .bslib-navs-card-title .nav{margin-left:auto}.bslib-card .bslib-sidebar-layout:not([data-bslib-sidebar-border=true]){border:none}.bslib-card .bslib-sidebar-layout:not([data-bslib-sidebar-border-radius=true]){border-top-left-radius:0;border-top-right-radius:0}[data-full-screen=true]{position:fixed;inset:3.5rem 1rem 1rem;height:auto !important;max-height:none !important;width:auto !important;z-index:1070}.bslib-full-screen-enter{display:none;position:absolute;bottom:var(--bslib-full-screen-enter-bottom, 0.2rem);right:var(--bslib-full-screen-enter-right, 0);top:var(--bslib-full-screen-enter-top);left:var(--bslib-full-screen-enter-left);color:var(--bslib-color-fg, var(--bs-card-color));background-color:var(--bslib-color-bg, var(--bs-card-bg, var(--bs-body-bg)));border:var(--bs-card-border-width) solid var(--bslib-color-fg, var(--bs-card-border-color));box-shadow:0 2px 4px rgba(0,0,0,.15);margin:.2rem .4rem;padding:.55rem !important;font-size:.8rem;cursor:pointer;opacity:.7;z-index:1070}.bslib-full-screen-enter:hover{opacity:1}.card[data-full-screen=false]:hover>*>.bslib-full-screen-enter{display:block}.bslib-has-full-screen .card:hover>*>.bslib-full-screen-enter{display:none}@media(max-width: 575.98px){.bslib-full-screen-enter{display:none !important}}.bslib-full-screen-exit{position:relative;top:1.35rem;font-size:.9rem;cursor:pointer;text-decoration:none;display:flex;float:right;margin-right:2.15rem;align-items:center;color:rgba(var(--bs-body-bg-rgb), 0.8)}.bslib-full-screen-exit:hover{color:rgba(var(--bs-body-bg-rgb), 1)}.bslib-full-screen-exit svg{margin-left:.5rem;font-size:1.5rem}#bslib-full-screen-overlay{position:fixed;inset:0;background-color:rgba(var(--bs-body-color-rgb), 0.6);backdrop-filter:blur(2px);-webkit-backdrop-filter:blur(2px);z-index:1069;animation:bslib-full-screen-overlay-enter 400ms cubic-bezier(0.6, 0.02, 0.65, 1) forwards}@keyframes bslib-full-screen-overlay-enter{0%{opacity:0}100%{opacity:1}}.bslib-grid{display:grid !important;gap:var(--bslib-spacer, 1rem);height:var(--bslib-grid-height)}.bslib-grid.grid{grid-template-columns:repeat(var(--bs-columns, 12), minmax(0, 1fr));grid-template-rows:unset;grid-auto-rows:var(--bslib-grid--row-heights);--bslib-grid--row-heights--xs: unset;--bslib-grid--row-heights--sm: unset;--bslib-grid--row-heights--md: unset;--bslib-grid--row-heights--lg: unset;--bslib-grid--row-heights--xl: unset;--bslib-grid--row-heights--xxl: unset}.bslib-grid.grid.bslib-grid--row-heights--xs{--bslib-grid--row-heights: var(--bslib-grid--row-heights--xs)}@media(min-width: 576px){.bslib-grid.grid.bslib-grid--row-heights--sm{--bslib-grid--row-heights: var(--bslib-grid--row-heights--sm)}}@media(min-width: 768px){.bslib-grid.grid.bslib-grid--row-heights--md{--bslib-grid--row-heights: var(--bslib-grid--row-heights--md)}}@media(min-width: 992px){.bslib-grid.grid.bslib-grid--row-heights--lg{--bslib-grid--row-heights: var(--bslib-grid--row-heights--lg)}}@media(min-width: 1200px){.bslib-grid.grid.bslib-grid--row-heights--xl{--bslib-grid--row-heights: var(--bslib-grid--row-heights--xl)}}@media(min-width: 1400px){.bslib-grid.grid.bslib-grid--row-heights--xxl{--bslib-grid--row-heights: var(--bslib-grid--row-heights--xxl)}}.bslib-grid>*>.shiny-input-container{width:100%}.bslib-grid-item{grid-column:auto/span 1}@media(max-width: 767.98px){.bslib-grid-item{grid-column:1/-1}}@media(max-width: 575.98px){.bslib-grid{grid-template-columns:1fr !important;height:var(--bslib-grid-height-mobile)}.bslib-grid.grid{height:unset !important;grid-auto-rows:var(--bslib-grid--row-heights--xs, auto)}}@media(min-width: 576px){.nav:not(.nav-hidden){display:flex !important;display:-webkit-flex !important}.nav:not(.nav-hidden):not(.nav-stacked):not(.flex-column){float:none !important}.nav:not(.nav-hidden):not(.nav-stacked):not(.flex-column)>.bslib-nav-spacer{margin-left:auto !important}.nav:not(.nav-hidden):not(.nav-stacked):not(.flex-column)>.form-inline{margin-top:auto;margin-bottom:auto}.nav:not(.nav-hidden).nav-stacked{flex-direction:column;-webkit-flex-direction:column;height:100%}.nav:not(.nav-hidden).nav-stacked>.bslib-nav-spacer{margin-top:auto !important}}html{height:100%}.bslib-page-fill{width:100%;height:100%;margin:0;padding:var(--bslib-spacer, 1rem);gap:var(--bslib-spacer, 1rem)}@media(max-width: 575.98px){.bslib-page-fill{height:var(--bslib-page-fill-mobile-height, auto)}}.navbar+.container-fluid:has(>.tab-content>.tab-pane.active.html-fill-container),.navbar+.container-sm:has(>.tab-content>.tab-pane.active.html-fill-container),.navbar+.container-md:has(>.tab-content>.tab-pane.active.html-fill-container),.navbar+.container-lg:has(>.tab-content>.tab-pane.active.html-fill-container),.navbar+.container-xl:has(>.tab-content>.tab-pane.active.html-fill-container),.navbar+.container-xxl:has(>.tab-content>.tab-pane.active.html-fill-container){padding-left:0;padding-right:0}.navbar+.container-fluid>.tab-content>.tab-pane.active.html-fill-container,.navbar+.container-sm>.tab-content>.tab-pane.active.html-fill-container,.navbar+.container-md>.tab-content>.tab-pane.active.html-fill-container,.navbar+.container-lg>.tab-content>.tab-pane.active.html-fill-container,.navbar+.container-xl>.tab-content>.tab-pane.active.html-fill-container,.navbar+.container-xxl>.tab-content>.tab-pane.active.html-fill-container{padding:var(--bslib-spacer, 1rem);gap:var(--bslib-spacer, 1rem)}.navbar+.container-fluid>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child),.navbar+.container-sm>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child),.navbar+.container-md>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child),.navbar+.container-lg>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child),.navbar+.container-xl>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child),.navbar+.container-xxl>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child){padding:0}.navbar+.container-fluid>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]),.navbar+.container-sm>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]),.navbar+.container-md>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]),.navbar+.container-lg>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]),.navbar+.container-xl>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]),.navbar+.container-xxl>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]){border-left:none;border-right:none;border-bottom:none}.navbar+.container-fluid>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]),.navbar+.container-sm>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]),.navbar+.container-md>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]),.navbar+.container-lg>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]),.navbar+.container-xl>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]),.navbar+.container-xxl>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]){border-radius:0}.navbar+div>.bslib-sidebar-layout{border-top:var(--bslib-sidebar-border)}:root{--bslib-page-sidebar-title-bg: #375a7f;--bslib-page-sidebar-title-color: #fff}.bslib-page-title{background-color:var(--bslib-page-sidebar-title-bg);color:var(--bslib-page-sidebar-title-color);font-size:1.25rem;font-weight:300;padding:var(--bslib-spacer, 1rem);padding-left:1.5rem;margin-bottom:0;border-bottom:1px solid #dee2e6}.bslib-sidebar-layout{--bslib-sidebar-transition-duration: 500ms;--bslib-sidebar-transition-easing-x: cubic-bezier(0.8, 0.78, 0.22, 1.07);--bslib-sidebar-border: var(--bs-card-border-width, 1px) solid var(--bs-card-border-color, rgba(0, 0, 0, 0.175));--bslib-sidebar-border-radius: var(--bs-border-radius);--bslib-sidebar-vert-border: var(--bs-card-border-width, 1px) solid var(--bs-card-border-color, rgba(0, 0, 0, 0.175));--bslib-sidebar-bg: rgba(var(--bs-emphasis-color-rgb, 0, 0, 0), 0.05);--bslib-sidebar-fg: var(--bs-emphasis-color, black);--bslib-sidebar-main-fg: var(--bs-card-color, var(--bs-body-color));--bslib-sidebar-main-bg: var(--bs-card-bg, var(--bs-body-bg));--bslib-sidebar-toggle-bg: rgba(var(--bs-emphasis-color-rgb, 0, 0, 0), 0.1);--bslib-sidebar-padding: calc(var(--bslib-spacer) * 1.5);--bslib-sidebar-icon-size: var(--bslib-spacer, 1rem);--bslib-sidebar-icon-button-size: calc(var(--bslib-sidebar-icon-size, 1rem) * 2);--bslib-sidebar-padding-icon: calc(var(--bslib-sidebar-icon-button-size, 2rem) * 1.5);--bslib-collapse-toggle-border-radius: var(--bs-border-radius, 0.25rem);--bslib-collapse-toggle-transform: 0deg;--bslib-sidebar-toggle-transition-easing: cubic-bezier(1, 0, 0, 1);--bslib-collapse-toggle-right-transform: 180deg;--bslib-sidebar-column-main: minmax(0, 1fr);display:grid !important;grid-template-columns:min(100% - var(--bslib-sidebar-icon-size),var(--bslib-sidebar-width, 250px)) var(--bslib-sidebar-column-main);position:relative;transition:grid-template-columns ease-in-out var(--bslib-sidebar-transition-duration);border:var(--bslib-sidebar-border);border-radius:var(--bslib-sidebar-border-radius)}@media(prefers-reduced-motion: reduce){.bslib-sidebar-layout{transition:none}}.bslib-sidebar-layout[data-bslib-sidebar-border=false]{border:none}.bslib-sidebar-layout[data-bslib-sidebar-border-radius=false]{border-radius:initial}.bslib-sidebar-layout>.main,.bslib-sidebar-layout>.sidebar{grid-row:1/2;border-radius:inherit;overflow:auto}.bslib-sidebar-layout>.main{grid-column:2/3;border-top-left-radius:0;border-bottom-left-radius:0;padding:var(--bslib-sidebar-padding);transition:padding var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration);color:var(--bslib-sidebar-main-fg);background-color:var(--bslib-sidebar-main-bg)}.bslib-sidebar-layout>.sidebar{grid-column:1/2;width:100%;height:100%;border-right:var(--bslib-sidebar-vert-border);border-top-right-radius:0;border-bottom-right-radius:0;color:var(--bslib-sidebar-fg);background-color:var(--bslib-sidebar-bg);backdrop-filter:blur(5px)}.bslib-sidebar-layout>.sidebar>.sidebar-content{display:flex;flex-direction:column;gap:var(--bslib-spacer, 1rem);padding:var(--bslib-sidebar-padding);padding-top:var(--bslib-sidebar-padding-icon)}.bslib-sidebar-layout>.sidebar>.sidebar-content>:last-child:not(.sidebar-title){margin-bottom:0}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion{margin-left:calc(-1*var(--bslib-sidebar-padding));margin-right:calc(-1*var(--bslib-sidebar-padding))}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion:last-child{margin-bottom:calc(-1*var(--bslib-sidebar-padding))}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion:not(:last-child){margin-bottom:1rem}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion .accordion-body{display:flex;flex-direction:column}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion:not(:first-child) .accordion-item:first-child{border-top:var(--bs-accordion-border-width) solid var(--bs-accordion-border-color)}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion:not(:last-child) .accordion-item:last-child{border-bottom:var(--bs-accordion-border-width) solid var(--bs-accordion-border-color)}.bslib-sidebar-layout>.sidebar>.sidebar-content.has-accordion>.sidebar-title{border-bottom:none;padding-bottom:0}.bslib-sidebar-layout>.sidebar .shiny-input-container{width:100%}.bslib-sidebar-layout[data-bslib-sidebar-open=always]>.sidebar>.sidebar-content{padding-top:var(--bslib-sidebar-padding)}.bslib-sidebar-layout>.collapse-toggle{grid-row:1/2;grid-column:1/2;display:inline-flex;align-items:center;position:absolute;right:calc(var(--bslib-sidebar-icon-size));top:calc(var(--bslib-sidebar-icon-size, 1rem)/2);border:none;border-radius:var(--bslib-collapse-toggle-border-radius);height:var(--bslib-sidebar-icon-button-size, 2rem);width:var(--bslib-sidebar-icon-button-size, 2rem);display:flex;align-items:center;justify-content:center;padding:0;color:var(--bslib-sidebar-fg);background-color:unset;transition:color var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration),top var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration),right var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration),left var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration)}.bslib-sidebar-layout>.collapse-toggle:hover{background-color:var(--bslib-sidebar-toggle-bg)}.bslib-sidebar-layout>.collapse-toggle>.collapse-icon{opacity:.8;width:var(--bslib-sidebar-icon-size);height:var(--bslib-sidebar-icon-size);transform:rotateY(var(--bslib-collapse-toggle-transform));transition:transform var(--bslib-sidebar-toggle-transition-easing) var(--bslib-sidebar-transition-duration)}.bslib-sidebar-layout>.collapse-toggle:hover>.collapse-icon{opacity:1}.bslib-sidebar-layout .sidebar-title{font-size:1.25rem;line-height:1.25;margin-top:0;margin-bottom:1rem;padding-bottom:1rem;border-bottom:var(--bslib-sidebar-border)}.bslib-sidebar-layout.sidebar-right{grid-template-columns:var(--bslib-sidebar-column-main) min(100% - var(--bslib-sidebar-icon-size),var(--bslib-sidebar-width, 250px))}.bslib-sidebar-layout.sidebar-right>.main{grid-column:1/2;border-top-right-radius:0;border-bottom-right-radius:0;border-top-left-radius:inherit;border-bottom-left-radius:inherit}.bslib-sidebar-layout.sidebar-right>.sidebar{grid-column:2/3;border-right:none;border-left:var(--bslib-sidebar-vert-border);border-top-left-radius:0;border-bottom-left-radius:0}.bslib-sidebar-layout.sidebar-right>.collapse-toggle{grid-column:2/3;left:var(--bslib-sidebar-icon-size);right:unset;border:var(--bslib-collapse-toggle-border)}.bslib-sidebar-layout.sidebar-right>.collapse-toggle>.collapse-icon{transform:rotateY(var(--bslib-collapse-toggle-right-transform))}.bslib-sidebar-layout.sidebar-collapsed{--bslib-collapse-toggle-transform: 180deg;--bslib-collapse-toggle-right-transform: 0deg;--bslib-sidebar-vert-border: none;grid-template-columns:0 minmax(0, 1fr)}.bslib-sidebar-layout.sidebar-collapsed.sidebar-right{grid-template-columns:minmax(0, 1fr) 0}.bslib-sidebar-layout.sidebar-collapsed:not(.transitioning)>.sidebar>*{display:none}.bslib-sidebar-layout.sidebar-collapsed>.main{border-radius:inherit}.bslib-sidebar-layout.sidebar-collapsed:not(.sidebar-right)>.main{padding-left:var(--bslib-sidebar-padding-icon)}.bslib-sidebar-layout.sidebar-collapsed.sidebar-right>.main{padding-right:var(--bslib-sidebar-padding-icon)}.bslib-sidebar-layout.sidebar-collapsed>.collapse-toggle{color:var(--bslib-sidebar-main-fg);top:calc(var(--bslib-sidebar-overlap-counter, 0)*(var(--bslib-sidebar-icon-size) + var(--bslib-sidebar-padding)) + var(--bslib-sidebar-icon-size, 1rem)/2);right:calc(-2.5*var(--bslib-sidebar-icon-size) - var(--bs-card-border-width, 1px))}.bslib-sidebar-layout.sidebar-collapsed.sidebar-right>.collapse-toggle{left:calc(-2.5*var(--bslib-sidebar-icon-size) - var(--bs-card-border-width, 1px));right:unset}@media(min-width: 576px){.bslib-sidebar-layout.transitioning>.sidebar>.sidebar-content{display:none}}@media(max-width: 575.98px){.bslib-sidebar-layout[data-bslib-sidebar-open=desktop]{--bslib-sidebar-js-init-collapsed: true}.bslib-sidebar-layout>.sidebar,.bslib-sidebar-layout.sidebar-right>.sidebar{border:none}.bslib-sidebar-layout>.main,.bslib-sidebar-layout.sidebar-right>.main{grid-column:1/3}.bslib-sidebar-layout[data-bslib-sidebar-open=always]{display:block !important}.bslib-sidebar-layout[data-bslib-sidebar-open=always]>.sidebar{max-height:var(--bslib-sidebar-max-height-mobile);overflow-y:auto;border-top:var(--bslib-sidebar-vert-border)}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]){grid-template-columns:100% 0}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]):not(.sidebar-collapsed)>.sidebar{z-index:1}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]):not(.sidebar-collapsed)>.collapse-toggle{z-index:1}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]).sidebar-right{grid-template-columns:0 100%}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]).sidebar-collapsed{grid-template-columns:0 100%}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]).sidebar-collapsed.sidebar-right{grid-template-columns:100% 0}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]):not(.sidebar-right)>.main{padding-left:var(--bslib-sidebar-padding-icon)}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]).sidebar-right>.main{padding-right:var(--bslib-sidebar-padding-icon)}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always])>.main{opacity:0;transition:opacity var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration)}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]).sidebar-collapsed>.main{opacity:1}}:root{--bslib-value-box-shadow: none;--bslib-value-box-border-width-auto-yes: var(--bslib-value-box-border-width-baseline);--bslib-value-box-border-width-auto-no: 0;--bslib-value-box-border-width-baseline: 1px}.bslib-value-box{border-width:var(--bslib-value-box-border-width-auto-no, var(--bslib-value-box-border-width-baseline));container-name:bslib-value-box;container-type:inline-size}.bslib-value-box.card{box-shadow:var(--bslib-value-box-shadow)}.bslib-value-box.border-auto{border-width:var(--bslib-value-box-border-width-auto-yes, var(--bslib-value-box-border-width-baseline))}.bslib-value-box.default{--bslib-value-box-bg-default: var(--bs-card-bg, #222);--bslib-value-box-border-color-default: var(--bs-card-border-color, rgba(0, 0, 0, 0.175));color:var(--bslib-value-box-color);background-color:var(--bslib-value-box-bg, var(--bslib-value-box-bg-default));border-color:var(--bslib-value-box-border-color, var(--bslib-value-box-border-color-default))}.bslib-value-box .value-box-grid{display:grid;grid-template-areas:"left right";align-items:center;overflow:hidden}.bslib-value-box .value-box-showcase{height:100%;max-height:var(---bslib-value-box-showcase-max-h, 100%)}.bslib-value-box .value-box-showcase,.bslib-value-box .value-box-showcase>.html-fill-item{width:100%}.bslib-value-box[data-full-screen=true] .value-box-showcase{max-height:var(---bslib-value-box-showcase-max-h-fs, 100%)}@media screen and (min-width: 575.98px){@container bslib-value-box (max-width: 300px){.bslib-value-box:not(.showcase-bottom) .value-box-grid{grid-template-columns:1fr !important;grid-template-rows:auto auto;grid-template-areas:"top" "bottom"}.bslib-value-box:not(.showcase-bottom) .value-box-grid .value-box-showcase{grid-area:top !important}.bslib-value-box:not(.showcase-bottom) .value-box-grid .value-box-area{grid-area:bottom !important;justify-content:end}}}.bslib-value-box .value-box-area{justify-content:center;padding:1.5rem 1rem;font-size:.9rem;font-weight:500}.bslib-value-box .value-box-area *{margin-bottom:0;margin-top:0}.bslib-value-box .value-box-title{font-size:1rem;margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2}.bslib-value-box .value-box-title:empty::after{content:" "}.bslib-value-box .value-box-value{font-size:calc(1.29rem + 0.48vw);margin-top:0;margin-bottom:.5rem;font-weight:500;line-height:1.2}@media(min-width: 1200px){.bslib-value-box .value-box-value{font-size:1.65rem}}.bslib-value-box .value-box-value:empty::after{content:" "}.bslib-value-box .value-box-showcase{align-items:center;justify-content:center;margin-top:auto;margin-bottom:auto;padding:1rem}.bslib-value-box .value-box-showcase .bi,.bslib-value-box .value-box-showcase .fa,.bslib-value-box .value-box-showcase .fab,.bslib-value-box .value-box-showcase .fas,.bslib-value-box .value-box-showcase .far{opacity:.85;min-width:50px;max-width:125%}.bslib-value-box .value-box-showcase .bi,.bslib-value-box .value-box-showcase .fa,.bslib-value-box .value-box-showcase .fab,.bslib-value-box .value-box-showcase .fas,.bslib-value-box .value-box-showcase .far{font-size:4rem}.bslib-value-box.showcase-top-right .value-box-grid{grid-template-columns:1fr var(---bslib-value-box-showcase-w, 50%)}.bslib-value-box.showcase-top-right .value-box-grid .value-box-showcase{grid-area:right;margin-left:auto;align-self:start;align-items:end;padding-left:0;padding-bottom:0}.bslib-value-box.showcase-top-right .value-box-grid .value-box-area{grid-area:left;align-self:end}.bslib-value-box.showcase-top-right[data-full-screen=true] .value-box-grid{grid-template-columns:auto var(---bslib-value-box-showcase-w-fs, 1fr)}.bslib-value-box.showcase-top-right[data-full-screen=true] .value-box-grid>div{align-self:center}.bslib-value-box.showcase-top-right:not([data-full-screen=true]) .value-box-showcase{margin-top:0}@container bslib-value-box (max-width: 300px){.bslib-value-box.showcase-top-right:not([data-full-screen=true]) .value-box-grid .value-box-showcase{padding-left:1rem}}.bslib-value-box.showcase-left-center .value-box-grid{grid-template-columns:var(---bslib-value-box-showcase-w, 30%) auto}.bslib-value-box.showcase-left-center[data-full-screen=true] .value-box-grid{grid-template-columns:var(---bslib-value-box-showcase-w-fs, 1fr) auto}.bslib-value-box.showcase-left-center:not([data-fill-screen=true]) .value-box-grid .value-box-showcase{grid-area:left}.bslib-value-box.showcase-left-center:not([data-fill-screen=true]) .value-box-grid .value-box-area{grid-area:right}.bslib-value-box.showcase-bottom .value-box-grid{grid-template-columns:1fr;grid-template-rows:1fr var(---bslib-value-box-showcase-h, auto);grid-template-areas:"top" "bottom";overflow:hidden}.bslib-value-box.showcase-bottom .value-box-grid .value-box-showcase{grid-area:bottom;padding:0;margin:0}.bslib-value-box.showcase-bottom .value-box-grid .value-box-area{grid-area:top}.bslib-value-box.showcase-bottom[data-full-screen=true] .value-box-grid{grid-template-rows:1fr var(---bslib-value-box-showcase-h-fs, 2fr)}.bslib-value-box.showcase-bottom[data-full-screen=true] .value-box-grid .value-box-showcase{padding:1rem}[data-bs-theme=dark] .bslib-value-box{--bslib-value-box-shadow: 0 0.5rem 1rem rgb(0 0 0 / 50%)}.html-fill-container{display:flex;flex-direction:column;min-height:0;min-width:0}.html-fill-container>.html-fill-item{flex:1 1 auto;min-height:0;min-width:0}.html-fill-container>:not(.html-fill-item){flex:0 0 auto}.quarto-container{min-height:calc(100vh - 132px)}body.hypothesis-enabled #quarto-header{margin-right:16px}footer.footer .nav-footer,#quarto-header>nav{padding-left:1em;padding-right:1em}footer.footer div.nav-footer p:first-child{margin-top:0}footer.footer div.nav-footer p:last-child{margin-bottom:0}#quarto-content>*{padding-top:14px}#quarto-content>#quarto-sidebar-glass{padding-top:0px}@media(max-width: 991.98px){#quarto-content>*{padding-top:0}#quarto-content .subtitle{padding-top:14px}#quarto-content section:first-of-type h2:first-of-type,#quarto-content section:first-of-type .h2:first-of-type{margin-top:1rem}}.headroom-target,header.headroom{will-change:transform;transition:position 200ms linear;transition:all 200ms linear}header.headroom--pinned{transform:translateY(0%)}header.headroom--unpinned{transform:translateY(-100%)}.navbar-container{width:100%}.navbar-brand{overflow:hidden;text-overflow:ellipsis}.navbar-brand-container{max-width:calc(100% - 115px);min-width:0;display:flex;align-items:center}@media(min-width: 992px){.navbar-brand-container{margin-right:1em}}.navbar-brand.navbar-brand-logo{margin-right:4px;display:inline-flex}.navbar-toggler{flex-basis:content;flex-shrink:0}.navbar .navbar-brand-container{order:2}.navbar .navbar-toggler{order:1}.navbar .navbar-container>.navbar-nav{order:20}.navbar .navbar-container>.navbar-brand-container{margin-left:0 !important;margin-right:0 !important}.navbar .navbar-collapse{order:20}.navbar #quarto-search{order:4;margin-left:auto}.navbar .navbar-toggler{margin-right:.5em}.navbar-collapse .quarto-navbar-tools{margin-left:.5em}.navbar-logo{max-height:24px;width:auto;padding-right:4px}nav .nav-item:not(.compact){padding-top:1px}nav .nav-link i,nav .dropdown-item i{padding-right:1px}.navbar-expand-lg .navbar-nav .nav-link{padding-left:.6rem;padding-right:.6rem}nav .nav-item.compact .nav-link{padding-left:.5rem;padding-right:.5rem;font-size:1.1rem}.navbar .quarto-navbar-tools{order:3}.navbar .quarto-navbar-tools div.dropdown{display:inline-block}.navbar .quarto-navbar-tools .quarto-navigation-tool{color:#dee2e6}.navbar .quarto-navbar-tools .quarto-navigation-tool:hover{color:#fafefd}.navbar-nav .dropdown-menu{min-width:220px;font-size:.9rem}.navbar .navbar-nav .nav-link.dropdown-toggle::after{opacity:.75;vertical-align:.175em}.navbar ul.dropdown-menu{padding-top:0;padding-bottom:0}.navbar .dropdown-header{text-transform:uppercase;font-size:.8rem;padding:0 .5rem}.navbar .dropdown-item{padding:.4rem .5rem}.navbar .dropdown-item>i.bi{margin-left:.1rem;margin-right:.25em}.sidebar #quarto-search{margin-top:-1px}.sidebar #quarto-search svg.aa-SubmitIcon{width:16px;height:16px}.sidebar-navigation a{color:inherit}.sidebar-title{margin-top:.25rem;padding-bottom:.5rem;font-size:1.3rem;line-height:1.6rem;visibility:visible}.sidebar-title>a{font-size:inherit;text-decoration:none}.sidebar-title .sidebar-tools-main{margin-top:-6px}@media(max-width: 991.98px){#quarto-sidebar div.sidebar-header{padding-top:.2em}}.sidebar-header-stacked .sidebar-title{margin-top:.6rem}.sidebar-logo{max-width:90%;padding-bottom:.5rem}.sidebar-logo-link{text-decoration:none}.sidebar-navigation li a{text-decoration:none}.sidebar-navigation .quarto-navigation-tool{opacity:.7;font-size:.875rem}#quarto-sidebar>nav>.sidebar-tools-main{margin-left:14px}.sidebar-tools-main{display:inline-flex;margin-left:0px;order:2}.sidebar-tools-main:not(.tools-wide){vertical-align:middle}.sidebar-navigation .quarto-navigation-tool.dropdown-toggle::after{display:none}.sidebar.sidebar-navigation>*{padding-top:1em}.sidebar-item{margin-bottom:.2em;line-height:1rem;margin-top:.4rem}.sidebar-section{padding-left:.5em;padding-bottom:.2em}.sidebar-item .sidebar-item-container{display:flex;justify-content:space-between;cursor:pointer}.sidebar-item-toggle:hover{cursor:pointer}.sidebar-item .sidebar-item-toggle .bi{font-size:.7rem;text-align:center}.sidebar-item .sidebar-item-toggle .bi-chevron-right::before{transition:transform 200ms ease}.sidebar-item .sidebar-item-toggle[aria-expanded=false] .bi-chevron-right::before{transform:none}.sidebar-item .sidebar-item-toggle[aria-expanded=true] .bi-chevron-right::before{transform:rotate(90deg)}.sidebar-item-text{width:100%}.sidebar-navigation .sidebar-divider{margin-left:0;margin-right:0;margin-top:.5rem;margin-bottom:.5rem}@media(max-width: 991.98px){.quarto-secondary-nav{display:block}.quarto-secondary-nav button.quarto-search-button{padding-right:0em;padding-left:2em}.quarto-secondary-nav button.quarto-btn-toggle{margin-left:-0.75rem;margin-right:.15rem}.quarto-secondary-nav nav.quarto-title-breadcrumbs{display:none}.quarto-secondary-nav nav.quarto-page-breadcrumbs{display:flex;align-items:center;padding-right:1em;margin-left:-0.25em}.quarto-secondary-nav nav.quarto-page-breadcrumbs a{text-decoration:none}.quarto-secondary-nav nav.quarto-page-breadcrumbs ol.breadcrumb{margin-bottom:0}}@media(min-width: 992px){.quarto-secondary-nav{display:none}}.quarto-title-breadcrumbs .breadcrumb{margin-bottom:.5em;font-size:.9rem}.quarto-title-breadcrumbs .breadcrumb li:last-of-type a{color:#6c757d}.quarto-secondary-nav .quarto-btn-toggle{color:#adadad}.quarto-secondary-nav[aria-expanded=false] .quarto-btn-toggle .bi-chevron-right::before{transform:none}.quarto-secondary-nav[aria-expanded=true] .quarto-btn-toggle .bi-chevron-right::before{transform:rotate(90deg)}.quarto-secondary-nav .quarto-btn-toggle .bi-chevron-right::before{transition:transform 200ms ease}.quarto-secondary-nav{cursor:pointer}.no-decor{text-decoration:none}.quarto-secondary-nav-title{margin-top:.3em;color:#adadad;padding-top:4px}.quarto-secondary-nav nav.quarto-page-breadcrumbs{color:#adadad}.quarto-secondary-nav nav.quarto-page-breadcrumbs a{color:#adadad}.quarto-secondary-nav nav.quarto-page-breadcrumbs a:hover{color:rgba(26,195,152,.8)}.quarto-secondary-nav nav.quarto-page-breadcrumbs .breadcrumb-item::before{color:#7a7a7a}.breadcrumb-item{line-height:1.2rem}div.sidebar-item-container{color:#adadad}div.sidebar-item-container:hover,div.sidebar-item-container:focus{color:rgba(26,195,152,.8)}div.sidebar-item-container.disabled{color:rgba(173,173,173,.75)}div.sidebar-item-container .active,div.sidebar-item-container .show>.nav-link,div.sidebar-item-container .sidebar-link>code{color:#1ac398}div.sidebar.sidebar-navigation.rollup.quarto-sidebar-toggle-contents,nav.sidebar.sidebar-navigation:not(.rollup){background-color:#222}@media(max-width: 991.98px){.sidebar-navigation .sidebar-item a,.nav-page .nav-page-text,.sidebar-navigation{font-size:1rem}.sidebar-navigation ul.sidebar-section.depth1 .sidebar-section-item{font-size:1.1rem}.sidebar-logo{display:none}.sidebar.sidebar-navigation{position:static;border-bottom:1px solid #434343}.sidebar.sidebar-navigation.collapsing{position:fixed;z-index:1000}.sidebar.sidebar-navigation.show{position:fixed;z-index:1000}.sidebar.sidebar-navigation{min-height:100%}nav.quarto-secondary-nav{background-color:#222;border-bottom:1px solid #434343}.quarto-banner nav.quarto-secondary-nav{background-color:#375a7f;color:#dee2e6;border-top:1px solid #434343}.sidebar .sidebar-footer{visibility:visible;padding-top:1rem;position:inherit}.sidebar-tools-collapse{display:block}}#quarto-sidebar{transition:width .15s ease-in}#quarto-sidebar>*{padding-right:1em}@media(max-width: 991.98px){#quarto-sidebar .sidebar-menu-container{white-space:nowrap;min-width:225px}#quarto-sidebar.show{transition:width .15s ease-out}}@media(min-width: 992px){#quarto-sidebar{display:flex;flex-direction:column}.nav-page .nav-page-text,.sidebar-navigation .sidebar-section .sidebar-item{font-size:.875rem}.sidebar-navigation .sidebar-item{font-size:.925rem}.sidebar.sidebar-navigation{display:block;position:sticky}.sidebar-search{width:100%}.sidebar .sidebar-footer{visibility:visible}}@media(min-width: 992px){#quarto-sidebar-glass{display:none}}@media(max-width: 991.98px){#quarto-sidebar-glass{position:fixed;top:0;bottom:0;left:0;right:0;background-color:rgba(255,255,255,0);transition:background-color .15s ease-in;z-index:-1}#quarto-sidebar-glass.collapsing{z-index:1000}#quarto-sidebar-glass.show{transition:background-color .15s ease-out;background-color:rgba(102,102,102,.4);z-index:1000}}.sidebar .sidebar-footer{padding:.5rem 1rem;align-self:flex-end;color:#6c757d;width:100%}.quarto-page-breadcrumbs .breadcrumb-item+.breadcrumb-item,.quarto-page-breadcrumbs .breadcrumb-item{padding-right:.33em;padding-left:0}.quarto-page-breadcrumbs .breadcrumb-item::before{padding-right:.33em}.quarto-sidebar-footer{font-size:.875em}.sidebar-section .bi-chevron-right{vertical-align:middle}.sidebar-section .bi-chevron-right::before{font-size:.9em}.notransition{-webkit-transition:none !important;-moz-transition:none !important;-o-transition:none !important;transition:none !important}.btn:focus:not(:focus-visible){box-shadow:none}.page-navigation{display:flex;justify-content:space-between}.nav-page{padding-bottom:.75em}.nav-page .bi{font-size:1.8rem;vertical-align:middle}.nav-page .nav-page-text{padding-left:.25em;padding-right:.25em}.nav-page a{color:#6c757d;text-decoration:none;display:flex;align-items:center}.nav-page a:hover{color:#009670}.nav-footer .toc-actions{padding-bottom:.5em;padding-top:.5em}.nav-footer .toc-actions a,.nav-footer .toc-actions a:hover{text-decoration:none}.nav-footer .toc-actions ul{display:flex;list-style:none}.nav-footer .toc-actions ul :first-child{margin-left:auto}.nav-footer .toc-actions ul :last-child{margin-right:auto}.nav-footer .toc-actions ul li{padding-right:1.5em}.nav-footer .toc-actions ul li i.bi{padding-right:.4em}.nav-footer .toc-actions ul li:last-of-type{padding-right:0}.nav-footer{display:flex;flex-direction:row;flex-wrap:wrap;justify-content:space-between;align-items:baseline;text-align:center;padding-top:.5rem;padding-bottom:.5rem;background-color:#222}body.nav-fixed{padding-top:82px}.nav-footer-contents{color:#6c757d;margin-top:.25rem}.nav-footer{min-height:3.5em;color:#8a8a8a}.nav-footer a{color:#8a8a8a}.nav-footer .nav-footer-left{font-size:.825em}.nav-footer .nav-footer-center{font-size:.825em}.nav-footer .nav-footer-right{font-size:.825em}.nav-footer-left .footer-items,.nav-footer-center .footer-items,.nav-footer-right .footer-items{display:inline-flex;padding-top:.3em;padding-bottom:.3em;margin-bottom:0em}.nav-footer-left .footer-items .nav-link,.nav-footer-center .footer-items .nav-link,.nav-footer-right .footer-items .nav-link{padding-left:.6em;padding-right:.6em}@media(min-width: 768px){.nav-footer-left{flex:1 1 0px;text-align:left}}@media(max-width: 575.98px){.nav-footer-left{margin-bottom:1em;flex:100%}}@media(min-width: 768px){.nav-footer-right{flex:1 1 0px;text-align:right}}@media(max-width: 575.98px){.nav-footer-right{margin-bottom:1em;flex:100%}}.nav-footer-center{text-align:center;min-height:3em}@media(min-width: 768px){.nav-footer-center{flex:1 1 0px}}.nav-footer-center .footer-items{justify-content:center}@media(max-width: 767.98px){.nav-footer-center{margin-bottom:1em;flex:100%}}@media(max-width: 767.98px){.nav-footer-center{margin-top:3em;order:10}}.navbar .quarto-reader-toggle.reader .quarto-reader-toggle-btn{background-color:#dee2e6;border-radius:3px}@media(max-width: 991.98px){.quarto-reader-toggle{display:none}}.quarto-reader-toggle.reader.quarto-navigation-tool .quarto-reader-toggle-btn{background-color:#adadad;border-radius:3px}.quarto-reader-toggle .quarto-reader-toggle-btn{display:inline-flex;padding-left:.2em;padding-right:.2em;margin-left:-0.2em;margin-right:-0.2em;text-align:center}.navbar .quarto-reader-toggle:not(.reader) .bi::before{background-image:url('data:image/svg+xml,')}.navbar .quarto-reader-toggle.reader .bi::before{background-image:url('data:image/svg+xml,')}.sidebar-navigation .quarto-reader-toggle:not(.reader) .bi::before{background-image:url('data:image/svg+xml,')}.sidebar-navigation .quarto-reader-toggle.reader .bi::before{background-image:url('data:image/svg+xml,')}#quarto-back-to-top{display:none;position:fixed;bottom:50px;background-color:#222;border-radius:.25rem;box-shadow:0 .2rem .5rem #6c757d,0 0 .05rem #6c757d;color:#6c757d;text-decoration:none;font-size:.9em;text-align:center;left:50%;padding:.4rem .8rem;transform:translate(-50%, 0)}#quarto-announcement{padding:.5em;display:flex;justify-content:space-between;margin-bottom:0;font-size:.9em}#quarto-announcement .quarto-announcement-content{margin-right:auto}#quarto-announcement .quarto-announcement-content p{margin-bottom:0}#quarto-announcement .quarto-announcement-icon{margin-right:.5em;font-size:1.2em;margin-top:-0.15em}#quarto-announcement .quarto-announcement-action{cursor:pointer}.aa-DetachedSearchButtonQuery{display:none}.aa-DetachedOverlay ul.aa-List,#quarto-search-results ul.aa-List{list-style:none;padding-left:0}.aa-DetachedOverlay .aa-Panel,#quarto-search-results .aa-Panel{background-color:#222;position:absolute;z-index:2000}#quarto-search-results .aa-Panel{max-width:400px}#quarto-search input{font-size:.925rem}@media(min-width: 992px){.navbar #quarto-search{margin-left:.25rem;order:999}}.navbar.navbar-expand-sm #quarto-search,.navbar.navbar-expand-md #quarto-search{order:999}@media(min-width: 992px){.navbar .quarto-navbar-tools{order:900}}@media(min-width: 992px){.navbar .quarto-navbar-tools.tools-end{margin-left:auto !important}}@media(max-width: 991.98px){#quarto-sidebar .sidebar-search{display:none}}#quarto-sidebar .sidebar-search .aa-Autocomplete{width:100%}.navbar .aa-Autocomplete .aa-Form{width:180px}.navbar #quarto-search.type-overlay .aa-Autocomplete{width:40px}.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form{background-color:inherit;border:none}.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form:focus-within{box-shadow:none;outline:none}.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form .aa-InputWrapper{display:none}.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form .aa-InputWrapper:focus-within{display:inherit}.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form .aa-Label svg,.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form .aa-LoadingIndicator svg{width:26px;height:26px;color:#dee2e6;opacity:1}.navbar #quarto-search.type-overlay .aa-Autocomplete svg.aa-SubmitIcon{width:26px;height:26px;color:#dee2e6;opacity:1}.aa-Autocomplete .aa-Form,.aa-DetachedFormContainer .aa-Form{align-items:center;background-color:#fff;border:1px solid #adb5bd;border-radius:.25rem;color:#2d2d2d;display:flex;line-height:1em;margin:0;position:relative;width:100%}.aa-Autocomplete .aa-Form:focus-within,.aa-DetachedFormContainer .aa-Form:focus-within{box-shadow:rgba(55,90,127,.6) 0 0 0 1px;outline:currentColor none medium}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix{align-items:center;display:flex;flex-shrink:0;order:1}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-Label,.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-Label,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator{cursor:initial;flex-shrink:0;padding:0;text-align:left}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-Label svg,.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator svg,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-Label svg,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator svg{color:#2d2d2d;opacity:.5}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-SubmitButton,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-SubmitButton{appearance:none;background:none;border:0;margin:0}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator{align-items:center;display:flex;justify-content:center}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator[hidden],.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator[hidden]{display:none}.aa-Autocomplete .aa-Form .aa-InputWrapper,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper{order:3;position:relative;width:100%}.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input{appearance:none;background:none;border:0;color:#2d2d2d;font:inherit;height:calc(1.5em + .1rem + 2px);padding:0;width:100%}.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input::placeholder,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input::placeholder{color:#2d2d2d;opacity:.8}.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input:focus,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input:focus{border-color:none;box-shadow:none;outline:none}.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-decoration,.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-cancel-button,.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-results-button,.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-results-decoration,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-decoration,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-cancel-button,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-results-button,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-results-decoration{display:none}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix{align-items:center;display:flex;order:4}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-ClearButton,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-ClearButton{align-items:center;background:none;border:0;color:#2d2d2d;opacity:.8;cursor:pointer;display:flex;margin:0;width:calc(1.5em + .1rem + 2px)}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-ClearButton:hover,.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-ClearButton:focus,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-ClearButton:hover,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-ClearButton:focus{color:#2d2d2d;opacity:.8}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-ClearButton[hidden],.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-ClearButton[hidden]{display:none}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-ClearButton svg,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-ClearButton svg{width:calc(1.5em + 0.75rem + calc(1px * 2))}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-CopyButton,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-CopyButton{border:none;align-items:center;background:none;color:#2d2d2d;opacity:.4;font-size:.7rem;cursor:pointer;display:none;margin:0;width:calc(1em + .1rem + 2px)}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-CopyButton:hover,.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-CopyButton:focus,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-CopyButton:hover,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-CopyButton:focus{color:#2d2d2d;opacity:.8}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-CopyButton[hidden],.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-CopyButton[hidden]{display:none}.aa-PanelLayout:empty{display:none}.quarto-search-no-results.no-query{display:none}.aa-Source:has(.no-query){display:none}#quarto-search-results .aa-Panel{border:solid #adb5bd 1px}#quarto-search-results .aa-SourceNoResults{width:398px}.aa-DetachedOverlay .aa-Panel,#quarto-search-results .aa-Panel{max-height:65vh;overflow-y:auto;font-size:.925rem}.aa-DetachedOverlay .aa-SourceNoResults,#quarto-search-results .aa-SourceNoResults{height:60px;display:flex;justify-content:center;align-items:center}.aa-DetachedOverlay .search-error,#quarto-search-results .search-error{padding-top:10px;padding-left:20px;padding-right:20px;cursor:default}.aa-DetachedOverlay .search-error .search-error-title,#quarto-search-results .search-error .search-error-title{font-size:1.1rem;margin-bottom:.5rem}.aa-DetachedOverlay .search-error .search-error-title .search-error-icon,#quarto-search-results .search-error .search-error-title .search-error-icon{margin-right:8px}.aa-DetachedOverlay .search-error .search-error-text,#quarto-search-results .search-error .search-error-text{font-weight:300}.aa-DetachedOverlay .search-result-text,#quarto-search-results .search-result-text{font-weight:300;overflow:hidden;text-overflow:ellipsis;display:-webkit-box;-webkit-line-clamp:2;-webkit-box-orient:vertical;line-height:1.2rem;max-height:2.4rem}.aa-DetachedOverlay .aa-SourceHeader .search-result-header,#quarto-search-results .aa-SourceHeader .search-result-header{font-size:.875rem;background-color:#2f2f2f;padding-left:14px;padding-bottom:4px;padding-top:4px}.aa-DetachedOverlay .aa-SourceHeader .search-result-header-no-results,#quarto-search-results .aa-SourceHeader .search-result-header-no-results{display:none}.aa-DetachedOverlay .aa-SourceFooter .algolia-search-logo,#quarto-search-results .aa-SourceFooter .algolia-search-logo{width:110px;opacity:.85;margin:8px;float:right}.aa-DetachedOverlay .search-result-section,#quarto-search-results .search-result-section{font-size:.925em}.aa-DetachedOverlay a.search-result-link,#quarto-search-results a.search-result-link{color:inherit;text-decoration:none}.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item,#quarto-search-results li.aa-Item[aria-selected=true] .search-item{background-color:#375a7f}.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item.search-result-more,.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item .search-result-section,.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item .search-result-text,.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item .search-result-title-container,.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item .search-result-text-container,#quarto-search-results li.aa-Item[aria-selected=true] .search-item.search-result-more,#quarto-search-results li.aa-Item[aria-selected=true] .search-item .search-result-section,#quarto-search-results li.aa-Item[aria-selected=true] .search-item .search-result-text,#quarto-search-results li.aa-Item[aria-selected=true] .search-item .search-result-title-container,#quarto-search-results li.aa-Item[aria-selected=true] .search-item .search-result-text-container{color:#fff;background-color:#375a7f}.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item mark.search-match,.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item .search-match.mark,#quarto-search-results li.aa-Item[aria-selected=true] .search-item mark.search-match,#quarto-search-results li.aa-Item[aria-selected=true] .search-item .search-match.mark{color:#fff;background-color:#2b4663}.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item,#quarto-search-results li.aa-Item[aria-selected=false] .search-item{background-color:#2d2d2d}.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item.search-result-more,.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item .search-result-section,.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item .search-result-text,.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item .search-result-title-container,.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item .search-result-text-container,#quarto-search-results li.aa-Item[aria-selected=false] .search-item.search-result-more,#quarto-search-results li.aa-Item[aria-selected=false] .search-item .search-result-section,#quarto-search-results li.aa-Item[aria-selected=false] .search-item .search-result-text,#quarto-search-results li.aa-Item[aria-selected=false] .search-item .search-result-title-container,#quarto-search-results li.aa-Item[aria-selected=false] .search-item .search-result-text-container{color:#fff}.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item mark.search-match,.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item .search-match.mark,#quarto-search-results li.aa-Item[aria-selected=false] .search-item mark.search-match,#quarto-search-results li.aa-Item[aria-selected=false] .search-item .search-match.mark{color:inherit;background-color:#000}.aa-DetachedOverlay .aa-Item .search-result-doc:not(.document-selectable) .search-result-title-container,#quarto-search-results .aa-Item .search-result-doc:not(.document-selectable) .search-result-title-container{background-color:#2d2d2d;color:#fff}.aa-DetachedOverlay .aa-Item .search-result-doc:not(.document-selectable) .search-result-text-container,#quarto-search-results .aa-Item .search-result-doc:not(.document-selectable) .search-result-text-container{padding-top:0px}.aa-DetachedOverlay li.aa-Item .search-result-doc.document-selectable .search-result-text-container,#quarto-search-results li.aa-Item .search-result-doc.document-selectable .search-result-text-container{margin-top:-4px}.aa-DetachedOverlay .aa-Item,#quarto-search-results .aa-Item{cursor:pointer}.aa-DetachedOverlay .aa-Item .search-item,#quarto-search-results .aa-Item .search-item{border-left:none;border-right:none;border-top:none;background-color:#2d2d2d;border-color:#adb5bd;color:#fff}.aa-DetachedOverlay .aa-Item .search-item p,#quarto-search-results .aa-Item .search-item p{margin-top:0;margin-bottom:0}.aa-DetachedOverlay .aa-Item .search-item i.bi,#quarto-search-results .aa-Item .search-item i.bi{padding-left:8px;padding-right:8px;font-size:1.3em}.aa-DetachedOverlay .aa-Item .search-item .search-result-title,#quarto-search-results .aa-Item .search-item .search-result-title{margin-top:.3em;margin-bottom:0em}.aa-DetachedOverlay .aa-Item .search-item .search-result-crumbs,#quarto-search-results .aa-Item .search-item .search-result-crumbs{white-space:nowrap;text-overflow:ellipsis;font-size:.8em;font-weight:300;margin-right:1em}.aa-DetachedOverlay .aa-Item .search-item .search-result-crumbs:not(.search-result-crumbs-wrap),#quarto-search-results .aa-Item .search-item .search-result-crumbs:not(.search-result-crumbs-wrap){max-width:30%;margin-left:auto;margin-top:.5em;margin-bottom:.1rem}.aa-DetachedOverlay .aa-Item .search-item .search-result-crumbs.search-result-crumbs-wrap,#quarto-search-results .aa-Item .search-item .search-result-crumbs.search-result-crumbs-wrap{flex-basis:100%;margin-top:0em;margin-bottom:.2em;margin-left:37px}.aa-DetachedOverlay .aa-Item .search-result-title-container,#quarto-search-results .aa-Item .search-result-title-container{font-size:1em;display:flex;flex-wrap:wrap;padding:6px 4px 6px 4px}.aa-DetachedOverlay .aa-Item .search-result-text-container,#quarto-search-results .aa-Item .search-result-text-container{padding-bottom:8px;padding-right:8px;margin-left:42px}.aa-DetachedOverlay .aa-Item .search-result-doc-section,.aa-DetachedOverlay .aa-Item .search-result-more,#quarto-search-results .aa-Item .search-result-doc-section,#quarto-search-results .aa-Item .search-result-more{padding-top:8px;padding-bottom:8px;padding-left:44px}.aa-DetachedOverlay .aa-Item .search-result-more,#quarto-search-results .aa-Item .search-result-more{font-size:.8em;font-weight:400}.aa-DetachedOverlay .aa-Item .search-result-doc,#quarto-search-results .aa-Item .search-result-doc{border-top:1px solid #adb5bd}.aa-DetachedSearchButton{background:none;border:none}.aa-DetachedSearchButton .aa-DetachedSearchButtonPlaceholder{display:none}.navbar .aa-DetachedSearchButton .aa-DetachedSearchButtonIcon{color:#dee2e6}.sidebar-tools-collapse #quarto-search,.sidebar-tools-main #quarto-search{display:inline}.sidebar-tools-collapse #quarto-search .aa-Autocomplete,.sidebar-tools-main #quarto-search .aa-Autocomplete{display:inline}.sidebar-tools-collapse #quarto-search .aa-DetachedSearchButton,.sidebar-tools-main #quarto-search .aa-DetachedSearchButton{padding-left:4px;padding-right:4px}.sidebar-tools-collapse #quarto-search .aa-DetachedSearchButton .aa-DetachedSearchButtonIcon,.sidebar-tools-main #quarto-search .aa-DetachedSearchButton .aa-DetachedSearchButtonIcon{color:#adadad}.sidebar-tools-collapse #quarto-search .aa-DetachedSearchButton .aa-DetachedSearchButtonIcon .aa-SubmitIcon,.sidebar-tools-main #quarto-search .aa-DetachedSearchButton .aa-DetachedSearchButtonIcon .aa-SubmitIcon{margin-top:-3px}.aa-DetachedContainer{background:rgba(34,34,34,.65);width:90%;bottom:0;box-shadow:rgba(173,181,189,.6) 0 0 0 1px;outline:currentColor none medium;display:flex;flex-direction:column;left:0;margin:0;overflow:hidden;padding:0;position:fixed;right:0;top:0;z-index:1101}.aa-DetachedContainer::after{height:32px}.aa-DetachedContainer .aa-SourceHeader{margin:var(--aa-spacing-half) 0 var(--aa-spacing-half) 2px}.aa-DetachedContainer .aa-Panel{background-color:#222;border-radius:0;box-shadow:none;flex-grow:1;margin:0;padding:0;position:relative}.aa-DetachedContainer .aa-PanelLayout{bottom:0;box-shadow:none;left:0;margin:0;max-height:none;overflow-y:auto;position:absolute;right:0;top:0;width:100%}.aa-DetachedFormContainer{background-color:#222;border-bottom:1px solid #adb5bd;display:flex;flex-direction:row;justify-content:space-between;margin:0;padding:.5em}.aa-DetachedCancelButton{background:none;font-size:.8em;border:0;border-radius:3px;color:#fff;cursor:pointer;margin:0 0 0 .5em;padding:0 .5em}.aa-DetachedCancelButton:hover,.aa-DetachedCancelButton:focus{box-shadow:rgba(55,90,127,.6) 0 0 0 1px;outline:currentColor none medium}.aa-DetachedContainer--modal{bottom:inherit;height:auto;margin:0 auto;position:absolute;top:100px;border-radius:6px;max-width:850px}@media(max-width: 575.98px){.aa-DetachedContainer--modal{width:100%;top:0px;border-radius:0px;border:none}}.aa-DetachedContainer--modal .aa-PanelLayout{max-height:var(--aa-detached-modal-max-height);padding-bottom:var(--aa-spacing-half);position:static}.aa-Detached{height:100vh;overflow:hidden}.aa-DetachedOverlay{background-color:rgba(255,255,255,.4);position:fixed;left:0;right:0;top:0;margin:0;padding:0;height:100vh;z-index:1100}.quarto-dashboard.nav-fixed.dashboard-sidebar #quarto-content.quarto-dashboard-content{padding:0em}.quarto-dashboard #quarto-content.quarto-dashboard-content{padding:1em}.quarto-dashboard #quarto-content.quarto-dashboard-content>*{padding-top:0}@media(min-width: 576px){.quarto-dashboard{height:100%}}.quarto-dashboard .card.valuebox.bslib-card.bg-primary{background-color:#375a7f !important}.quarto-dashboard .card.valuebox.bslib-card.bg-secondary{background-color:#434343 !important}.quarto-dashboard .card.valuebox.bslib-card.bg-success{background-color:#00bc8c !important}.quarto-dashboard .card.valuebox.bslib-card.bg-info{background-color:#3498db !important}.quarto-dashboard .card.valuebox.bslib-card.bg-warning{background-color:#f39c12 !important}.quarto-dashboard .card.valuebox.bslib-card.bg-danger{background-color:#e74c3c !important}.quarto-dashboard .card.valuebox.bslib-card.bg-light{background-color:#6f6f6f !important}.quarto-dashboard .card.valuebox.bslib-card.bg-dark{background-color:#2d2d2d !important}.quarto-dashboard.dashboard-fill{display:flex;flex-direction:column}.quarto-dashboard #quarto-appendix{display:none}.quarto-dashboard #quarto-header #quarto-dashboard-header{border-top:solid 1px #4673a3;border-bottom:solid 1px #4673a3}.quarto-dashboard #quarto-header #quarto-dashboard-header>nav{padding-left:1em;padding-right:1em}.quarto-dashboard #quarto-header #quarto-dashboard-header>nav .navbar-brand-container{padding-left:0}.quarto-dashboard #quarto-header #quarto-dashboard-header .navbar-toggler{margin-right:0}.quarto-dashboard #quarto-header #quarto-dashboard-header .navbar-toggler-icon{height:1em;width:1em;background-image:url('data:image/svg+xml,')}.quarto-dashboard #quarto-header #quarto-dashboard-header .navbar-brand-container{padding-right:1em}.quarto-dashboard #quarto-header #quarto-dashboard-header .navbar-title{font-size:1.1em}.quarto-dashboard #quarto-header #quarto-dashboard-header .navbar-nav{font-size:.9em}.quarto-dashboard #quarto-dashboard-header .navbar{padding:0}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-container{padding-left:1em}.quarto-dashboard #quarto-dashboard-header .navbar.slim .navbar-brand-container .nav-link,.quarto-dashboard #quarto-dashboard-header .navbar.slim .navbar-nav .nav-link{padding:.7em}.quarto-dashboard #quarto-dashboard-header .navbar .quarto-color-scheme-toggle{order:9}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-toggler{margin-left:.5em;order:10}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-nav .nav-link{padding:.5em;height:100%;display:flex;align-items:center}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-nav .active{background-color:#436e9b}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-brand-container{padding:.5em .5em .5em 0;display:flex;flex-direction:row;margin-right:2em;align-items:center}@media(max-width: 767.98px){.quarto-dashboard #quarto-dashboard-header .navbar .navbar-brand-container{margin-right:auto}}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-collapse{align-self:stretch}@media(min-width: 768px){.quarto-dashboard #quarto-dashboard-header .navbar .navbar-collapse{order:8}}@media(max-width: 767.98px){.quarto-dashboard #quarto-dashboard-header .navbar .navbar-collapse{order:1000;padding-bottom:.5em}}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-collapse .navbar-nav{align-self:stretch}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-title{font-size:1.25em;line-height:1.1em;display:flex;flex-direction:row;flex-wrap:wrap;align-items:baseline}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-title .navbar-title-text{margin-right:.4em}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-title a{text-decoration:none;color:inherit}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-subtitle,.quarto-dashboard #quarto-dashboard-header .navbar .navbar-author{font-size:.9rem;margin-right:.5em}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-author{margin-left:auto}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-logo{max-height:48px;min-height:30px;object-fit:cover;margin-right:1em}.quarto-dashboard #quarto-dashboard-header .navbar .quarto-dashboard-links{order:9;padding-right:1em}.quarto-dashboard #quarto-dashboard-header .navbar .quarto-dashboard-link-text{margin-left:.25em}.quarto-dashboard #quarto-dashboard-header .navbar .quarto-dashboard-link{padding-right:0em;padding-left:.7em;text-decoration:none;color:#dee2e6}.quarto-dashboard .page-layout-custom .tab-content{padding:0;border:none}.quarto-dashboard-img-contain{height:100%;width:100%;object-fit:contain}@media(max-width: 575.98px){.quarto-dashboard .bslib-grid{grid-template-rows:minmax(1em, max-content) !important}.quarto-dashboard .sidebar-content{height:inherit}.quarto-dashboard .page-layout-custom{min-height:100vh}}.quarto-dashboard.dashboard-toolbar>.page-layout-custom,.quarto-dashboard.dashboard-sidebar>.page-layout-custom{padding:0}.quarto-dashboard .quarto-dashboard-content.quarto-dashboard-pages{padding:0}.quarto-dashboard .callout{margin-bottom:0;margin-top:0}.quarto-dashboard .html-fill-container figure{overflow:hidden}.quarto-dashboard bslib-tooltip .rounded-pill{border:solid #6c757d 1px}.quarto-dashboard bslib-tooltip .rounded-pill .svg{fill:#fff}.quarto-dashboard .tabset .dashboard-card-no-title .nav-tabs{margin-left:0;margin-right:auto}.quarto-dashboard .tabset .tab-content{border:none}.quarto-dashboard .tabset .card-header .nav-link[role=tab]{margin-top:-6px;padding-top:6px;padding-bottom:6px}.quarto-dashboard .card.valuebox,.quarto-dashboard .card.bslib-value-box{min-height:3rem}.quarto-dashboard .card.valuebox .card-body,.quarto-dashboard .card.bslib-value-box .card-body{padding:0}.quarto-dashboard .bslib-value-box .value-box-value{font-size:clamp(.1em,15cqw,5em)}.quarto-dashboard .bslib-value-box .value-box-showcase .bi{font-size:clamp(.1em,max(18cqw,5.2cqh),5em);text-align:center;height:1em}.quarto-dashboard .bslib-value-box .value-box-showcase .bi::before{vertical-align:1em}.quarto-dashboard .bslib-value-box .value-box-area{margin-top:auto;margin-bottom:auto}.quarto-dashboard .card figure.quarto-float{display:flex;flex-direction:column;align-items:center}.quarto-dashboard .dashboard-scrolling{padding:1em}.quarto-dashboard .full-height{height:100%}.quarto-dashboard .showcase-bottom .value-box-grid{display:grid;grid-template-columns:1fr;grid-template-rows:1fr auto;grid-template-areas:"top" "bottom"}.quarto-dashboard .showcase-bottom .value-box-grid .value-box-showcase{grid-area:bottom;padding:0;margin:0}.quarto-dashboard .showcase-bottom .value-box-grid .value-box-showcase i.bi{font-size:4rem}.quarto-dashboard .showcase-bottom .value-box-grid .value-box-area{grid-area:top}.quarto-dashboard .tab-content{margin-bottom:0}.quarto-dashboard .bslib-card .bslib-navs-card-title{justify-content:stretch;align-items:end}.quarto-dashboard .card-header{display:flex;flex-wrap:wrap;justify-content:space-between}.quarto-dashboard .card-header .card-title{display:flex;flex-direction:column;justify-content:center;margin-bottom:0}.quarto-dashboard .tabset .card-toolbar{margin-bottom:1em}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout{border:none;gap:var(--bslib-spacer, 1rem)}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout>.main{padding:0}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout>.sidebar{border-radius:.25rem;border:1px solid rgba(0,0,0,.175)}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout>.collapse-toggle{display:none}@media(max-width: 767.98px){.quarto-dashboard .bslib-grid>.bslib-sidebar-layout{grid-template-columns:1fr;grid-template-rows:max-content 1fr}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout>.main{grid-column:1;grid-row:2}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout .sidebar{grid-column:1;grid-row:1}}.quarto-dashboard .sidebar-right .sidebar{padding-left:2.5em}.quarto-dashboard .sidebar-right .collapse-toggle{left:2px}.quarto-dashboard .quarto-dashboard .sidebar-right button.collapse-toggle:not(.transitioning){left:unset}.quarto-dashboard aside.sidebar{padding-left:1em;padding-right:1em;background-color:rgba(52,58,64,.25);color:#fff}.quarto-dashboard .bslib-sidebar-layout>div.main{padding:.7em}.quarto-dashboard .bslib-sidebar-layout button.collapse-toggle{margin-top:.3em}.quarto-dashboard .bslib-sidebar-layout .collapse-toggle{top:0}.quarto-dashboard .bslib-sidebar-layout.sidebar-collapsed:not(.transitioning):not(.sidebar-right) .collapse-toggle{left:2px}.quarto-dashboard .sidebar>section>.h3:first-of-type{margin-top:0em}.quarto-dashboard .sidebar .h3,.quarto-dashboard .sidebar .h4,.quarto-dashboard .sidebar .h5,.quarto-dashboard .sidebar .h6{margin-top:.5em}.quarto-dashboard .sidebar form{flex-direction:column;align-items:start;margin-bottom:1em}.quarto-dashboard .sidebar form div[class*=oi-][class$=-input]{flex-direction:column}.quarto-dashboard .sidebar form[class*=oi-][class$=-toggle]{flex-direction:row-reverse;align-items:center;justify-content:start}.quarto-dashboard .sidebar form input[type=range]{margin-top:.5em;margin-right:.8em;margin-left:1em}.quarto-dashboard .sidebar label{width:fit-content}.quarto-dashboard .sidebar .card-body{margin-bottom:2em}.quarto-dashboard .sidebar .shiny-input-container{margin-bottom:1em}.quarto-dashboard .sidebar .shiny-options-group{margin-top:0}.quarto-dashboard .sidebar .control-label{margin-bottom:.3em}.quarto-dashboard .card .card-body .quarto-layout-row{align-items:stretch}.quarto-dashboard .toolbar{font-size:.9em;display:flex;flex-direction:row;border-top:solid 1px silver;padding:1em;flex-wrap:wrap;background-color:rgba(52,58,64,.25)}.quarto-dashboard .toolbar .cell-output-display{display:flex}.quarto-dashboard .toolbar .shiny-input-container{padding-bottom:.5em;margin-bottom:.5em;width:inherit}.quarto-dashboard .toolbar .shiny-input-container>.checkbox:first-child{margin-top:6px}.quarto-dashboard .toolbar>*:last-child{margin-right:0}.quarto-dashboard .toolbar>*>*{margin-right:1em;align-items:baseline}.quarto-dashboard .toolbar>*>*>a{text-decoration:none;margin-top:auto;margin-bottom:auto}.quarto-dashboard .toolbar .shiny-input-container{padding-bottom:0;margin-bottom:0}.quarto-dashboard .toolbar .shiny-input-container>*{flex-shrink:0;flex-grow:0}.quarto-dashboard .toolbar .form-group.shiny-input-container:not([role=group])>label{margin-bottom:0}.quarto-dashboard .toolbar .shiny-input-container.no-baseline{align-items:start;padding-top:6px}.quarto-dashboard .toolbar .shiny-input-container{display:flex;align-items:baseline}.quarto-dashboard .toolbar .shiny-input-container label{padding-right:.4em}.quarto-dashboard .toolbar .shiny-input-container .bslib-input-switch{margin-top:6px}.quarto-dashboard .toolbar input[type=text]{line-height:1;width:inherit}.quarto-dashboard .toolbar .input-daterange{width:inherit}.quarto-dashboard .toolbar .input-daterange input[type=text]{height:2.4em;width:10em}.quarto-dashboard .toolbar .input-daterange .input-group-addon{height:auto;padding:0;margin-left:-5px !important;margin-right:-5px}.quarto-dashboard .toolbar .input-daterange .input-group-addon .input-group-text{padding-top:0;padding-bottom:0;height:100%}.quarto-dashboard .toolbar span.irs.irs--shiny{width:10em}.quarto-dashboard .toolbar span.irs.irs--shiny .irs-line{top:9px}.quarto-dashboard .toolbar span.irs.irs--shiny .irs-min,.quarto-dashboard .toolbar span.irs.irs--shiny .irs-max,.quarto-dashboard .toolbar span.irs.irs--shiny .irs-from,.quarto-dashboard .toolbar span.irs.irs--shiny .irs-to,.quarto-dashboard .toolbar span.irs.irs--shiny .irs-single{top:20px}.quarto-dashboard .toolbar span.irs.irs--shiny .irs-bar{top:8px}.quarto-dashboard .toolbar span.irs.irs--shiny .irs-handle{top:0px}.quarto-dashboard .toolbar .shiny-input-checkboxgroup>label{margin-top:6px}.quarto-dashboard .toolbar .shiny-input-checkboxgroup>.shiny-options-group{margin-top:0;align-items:baseline}.quarto-dashboard .toolbar .shiny-input-radiogroup>label{margin-top:6px}.quarto-dashboard .toolbar .shiny-input-radiogroup>.shiny-options-group{align-items:baseline;margin-top:0}.quarto-dashboard .toolbar .shiny-input-radiogroup>.shiny-options-group>.radio{margin-right:.3em}.quarto-dashboard .toolbar .form-select{padding-top:.2em;padding-bottom:.2em}.quarto-dashboard .toolbar .shiny-input-select{min-width:6em}.quarto-dashboard .toolbar div.checkbox{margin-bottom:0px}.quarto-dashboard .toolbar>.checkbox:first-child{margin-top:6px}.quarto-dashboard .toolbar form{width:fit-content}.quarto-dashboard .toolbar form label{padding-top:.2em;padding-bottom:.2em;width:fit-content}.quarto-dashboard .toolbar form input[type=date]{width:fit-content}.quarto-dashboard .toolbar form input[type=color]{width:3em}.quarto-dashboard .toolbar form button{padding:.4em}.quarto-dashboard .toolbar form select{width:fit-content}.quarto-dashboard .toolbar>*{font-size:.9em;flex-grow:0}.quarto-dashboard .toolbar .shiny-input-container label{margin-bottom:1px}.quarto-dashboard .toolbar-bottom{margin-top:1em;margin-bottom:0 !important;order:2}.quarto-dashboard .quarto-dashboard-content>.dashboard-toolbar-container>.toolbar-content>.tab-content>.tab-pane>*:not(.bslib-sidebar-layout){padding:1em}.quarto-dashboard .quarto-dashboard-content>.dashboard-toolbar-container>.toolbar-content>*:not(.tab-content){padding:1em}.quarto-dashboard .quarto-dashboard-content>.tab-content>.dashboard-page>.dashboard-toolbar-container>.toolbar-content,.quarto-dashboard .quarto-dashboard-content>.tab-content>.dashboard-page:not(.dashboard-sidebar-container)>*:not(.dashboard-toolbar-container){padding:1em}.quarto-dashboard .toolbar-content{padding:0}.quarto-dashboard .quarto-dashboard-content.quarto-dashboard-pages .tab-pane>.dashboard-toolbar-container .toolbar{border-radius:0;margin-bottom:0}.quarto-dashboard .dashboard-toolbar-container.toolbar-toplevel .toolbar{border-bottom:1px solid rgba(0,0,0,.175)}.quarto-dashboard .dashboard-toolbar-container.toolbar-toplevel .toolbar-bottom{margin-top:0}.quarto-dashboard .dashboard-toolbar-container:not(.toolbar-toplevel) .toolbar{margin-bottom:1em;border-top:none;border-radius:.25rem;border:1px solid rgba(0,0,0,.175)}.quarto-dashboard .vega-embed.has-actions details{width:1.7em;height:2em;position:absolute !important;top:0;right:0}.quarto-dashboard .dashboard-toolbar-container{padding:0}.quarto-dashboard .card .card-header p:last-child,.quarto-dashboard .card .card-footer p:last-child{margin-bottom:0}.quarto-dashboard .card .card-body>.h4:first-child{margin-top:0}.quarto-dashboard .card .card-body{z-index:4}@media(max-width: 767.98px){.quarto-dashboard .card .card-body .itables div.dataTables_wrapper div.dataTables_length,.quarto-dashboard .card .card-body .itables div.dataTables_wrapper div.dataTables_info,.quarto-dashboard .card .card-body .itables div.dataTables_wrapper div.dataTables_paginate{text-align:initial}.quarto-dashboard .card .card-body .itables div.dataTables_wrapper div.dataTables_filter{text-align:right}.quarto-dashboard .card .card-body .itables div.dataTables_wrapper div.dataTables_paginate ul.pagination{justify-content:initial}}.quarto-dashboard .card .card-body .itables .dataTables_wrapper{display:flex;flex-wrap:wrap;justify-content:space-between;align-items:center;padding-top:0}.quarto-dashboard .card .card-body .itables .dataTables_wrapper table{flex-shrink:0}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dt-buttons{margin-bottom:.5em;margin-left:auto;width:fit-content;float:right}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dt-buttons.btn-group{background:#222;border:none}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dt-buttons .btn-secondary{background-color:#222;background-image:none;border:solid #dee2e6 1px;padding:.2em .7em}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dt-buttons .btn span{font-size:.8em;color:#fff}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_info{margin-left:.5em;margin-bottom:.5em;padding-top:0}@media(min-width: 768px){.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_info{font-size:.875em}}@media(max-width: 767.98px){.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_info{font-size:.8em}}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_filter{margin-bottom:.5em;font-size:.875em}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_filter input[type=search]{padding:1px 5px 1px 5px;font-size:.875em}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_length{flex-basis:1 1 50%;margin-bottom:.5em;font-size:.875em}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_length select{padding:.4em 3em .4em .5em;font-size:.875em;margin-left:.2em;margin-right:.2em}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_paginate{flex-shrink:0}@media(min-width: 768px){.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_paginate{margin-left:auto}}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_paginate ul.pagination .paginate_button .page-link{font-size:.8em}.quarto-dashboard .card .card-footer{font-size:.9em}.quarto-dashboard .card .card-toolbar{display:flex;flex-grow:1;flex-direction:row;width:100%;flex-wrap:wrap}.quarto-dashboard .card .card-toolbar>*{font-size:.8em;flex-grow:0}.quarto-dashboard .card .card-toolbar>.card-title{font-size:1em;flex-grow:1;align-self:flex-start;margin-top:.1em}.quarto-dashboard .card .card-toolbar .cell-output-display{display:flex}.quarto-dashboard .card .card-toolbar .shiny-input-container{padding-bottom:.5em;margin-bottom:.5em;width:inherit}.quarto-dashboard .card .card-toolbar .shiny-input-container>.checkbox:first-child{margin-top:6px}.quarto-dashboard .card .card-toolbar>*:last-child{margin-right:0}.quarto-dashboard .card .card-toolbar>*>*{margin-right:1em;align-items:baseline}.quarto-dashboard .card .card-toolbar>*>*>a{text-decoration:none;margin-top:auto;margin-bottom:auto}.quarto-dashboard .card .card-toolbar form{width:fit-content}.quarto-dashboard .card .card-toolbar form label{padding-top:.2em;padding-bottom:.2em;width:fit-content}.quarto-dashboard .card .card-toolbar form input[type=date]{width:fit-content}.quarto-dashboard .card .card-toolbar form input[type=color]{width:3em}.quarto-dashboard .card .card-toolbar form button{padding:.4em}.quarto-dashboard .card .card-toolbar form select{width:fit-content}.quarto-dashboard .card .card-toolbar .cell-output-display{display:flex}.quarto-dashboard .card .card-toolbar .shiny-input-container{padding-bottom:.5em;margin-bottom:.5em;width:inherit}.quarto-dashboard .card .card-toolbar .shiny-input-container>.checkbox:first-child{margin-top:6px}.quarto-dashboard .card .card-toolbar>*:last-child{margin-right:0}.quarto-dashboard .card .card-toolbar>*>*{margin-right:1em;align-items:baseline}.quarto-dashboard .card .card-toolbar>*>*>a{text-decoration:none;margin-top:auto;margin-bottom:auto}.quarto-dashboard .card .card-toolbar .shiny-input-container{padding-bottom:0;margin-bottom:0}.quarto-dashboard .card .card-toolbar .shiny-input-container>*{flex-shrink:0;flex-grow:0}.quarto-dashboard .card .card-toolbar .form-group.shiny-input-container:not([role=group])>label{margin-bottom:0}.quarto-dashboard .card .card-toolbar .shiny-input-container.no-baseline{align-items:start;padding-top:6px}.quarto-dashboard .card .card-toolbar .shiny-input-container{display:flex;align-items:baseline}.quarto-dashboard .card .card-toolbar .shiny-input-container label{padding-right:.4em}.quarto-dashboard .card .card-toolbar .shiny-input-container .bslib-input-switch{margin-top:6px}.quarto-dashboard .card .card-toolbar input[type=text]{line-height:1;width:inherit}.quarto-dashboard .card .card-toolbar .input-daterange{width:inherit}.quarto-dashboard .card .card-toolbar .input-daterange input[type=text]{height:2.4em;width:10em}.quarto-dashboard .card .card-toolbar .input-daterange .input-group-addon{height:auto;padding:0;margin-left:-5px !important;margin-right:-5px}.quarto-dashboard .card .card-toolbar .input-daterange .input-group-addon .input-group-text{padding-top:0;padding-bottom:0;height:100%}.quarto-dashboard .card .card-toolbar span.irs.irs--shiny{width:10em}.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-line{top:9px}.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-min,.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-max,.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-from,.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-to,.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-single{top:20px}.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-bar{top:8px}.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-handle{top:0px}.quarto-dashboard .card .card-toolbar .shiny-input-checkboxgroup>label{margin-top:6px}.quarto-dashboard .card .card-toolbar .shiny-input-checkboxgroup>.shiny-options-group{margin-top:0;align-items:baseline}.quarto-dashboard .card .card-toolbar .shiny-input-radiogroup>label{margin-top:6px}.quarto-dashboard .card .card-toolbar .shiny-input-radiogroup>.shiny-options-group{align-items:baseline;margin-top:0}.quarto-dashboard .card .card-toolbar .shiny-input-radiogroup>.shiny-options-group>.radio{margin-right:.3em}.quarto-dashboard .card .card-toolbar .form-select{padding-top:.2em;padding-bottom:.2em}.quarto-dashboard .card .card-toolbar .shiny-input-select{min-width:6em}.quarto-dashboard .card .card-toolbar div.checkbox{margin-bottom:0px}.quarto-dashboard .card .card-toolbar>.checkbox:first-child{margin-top:6px}.quarto-dashboard .card-body>table>thead{border-top:none}.quarto-dashboard .card-body>.table>:not(caption)>*>*{background-color:#2d2d2d}.tableFloatingHeaderOriginal{background-color:#2d2d2d;position:sticky !important;top:0 !important}.dashboard-data-table{margin-top:-1px}div.value-box-area span.observablehq--number{font-size:calc(clamp(.1em,15cqw,5em)*1.25);line-height:1.2;color:inherit;font-family:var(--bs-body-font-family)}.quarto-listing{padding-bottom:1em}.listing-pagination{padding-top:.5em}ul.pagination{float:right;padding-left:8px;padding-top:.5em}ul.pagination li{padding-right:.75em}ul.pagination li.disabled a,ul.pagination li.active a{color:#fff;text-decoration:none}ul.pagination li:last-of-type{padding-right:0}.listing-actions-group{display:flex}.listing-actions-group .form-select,.listing-actions-group .form-control{background-color:#222;color:#fff}.quarto-listing-filter{margin-bottom:1em;width:200px;margin-left:auto}.quarto-listing-sort{margin-bottom:1em;margin-right:auto;width:auto}.quarto-listing-sort .input-group-text{font-size:.8em}.input-group-text{border-right:none}.quarto-listing-sort select.form-select{font-size:.8em}.listing-no-matching{text-align:center;padding-top:2em;padding-bottom:3em;font-size:1em}#quarto-margin-sidebar .quarto-listing-category{padding-top:0;font-size:1rem}#quarto-margin-sidebar .quarto-listing-category-title{cursor:pointer;font-weight:600;font-size:1rem}.quarto-listing-category .category{cursor:pointer}.quarto-listing-category .category.active{font-weight:600}.quarto-listing-category.category-cloud{display:flex;flex-wrap:wrap;align-items:baseline}.quarto-listing-category.category-cloud .category{padding-right:5px}.quarto-listing-category.category-cloud .category-cloud-1{font-size:.75em}.quarto-listing-category.category-cloud .category-cloud-2{font-size:.95em}.quarto-listing-category.category-cloud .category-cloud-3{font-size:1.15em}.quarto-listing-category.category-cloud .category-cloud-4{font-size:1.35em}.quarto-listing-category.category-cloud .category-cloud-5{font-size:1.55em}.quarto-listing-category.category-cloud .category-cloud-6{font-size:1.75em}.quarto-listing-category.category-cloud .category-cloud-7{font-size:1.95em}.quarto-listing-category.category-cloud .category-cloud-8{font-size:2.15em}.quarto-listing-category.category-cloud .category-cloud-9{font-size:2.35em}.quarto-listing-category.category-cloud .category-cloud-10{font-size:2.55em}.quarto-listing-cols-1{grid-template-columns:repeat(1, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-1{grid-template-columns:repeat(1, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-1{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-2{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-2{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-2{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-3{grid-template-columns:repeat(3, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-3{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-3{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-4{grid-template-columns:repeat(4, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-4{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-4{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-5{grid-template-columns:repeat(5, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-5{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-5{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-6{grid-template-columns:repeat(6, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-6{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-6{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-7{grid-template-columns:repeat(7, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-7{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-7{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-8{grid-template-columns:repeat(8, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-8{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-8{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-9{grid-template-columns:repeat(9, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-9{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-9{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-10{grid-template-columns:repeat(10, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-10{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-10{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-11{grid-template-columns:repeat(11, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-11{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-11{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-12{grid-template-columns:repeat(12, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-12{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-12{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-grid{gap:1.5em}.quarto-grid-item.borderless{border:none}.quarto-grid-item.borderless .listing-categories .listing-category:last-of-type,.quarto-grid-item.borderless .listing-categories .listing-category:first-of-type{padding-left:0}.quarto-grid-item.borderless .listing-categories .listing-category{border:0}.quarto-grid-link{text-decoration:none;color:inherit}.quarto-grid-link:hover{text-decoration:none;color:inherit}.quarto-grid-item h5.title,.quarto-grid-item .title.h5{margin-top:0;margin-bottom:0}.quarto-grid-item .card-footer{display:flex;justify-content:space-between;font-size:.8em}.quarto-grid-item .card-footer p{margin-bottom:0}.quarto-grid-item p.card-img-top{margin-bottom:0}.quarto-grid-item p.card-img-top>img{object-fit:cover}.quarto-grid-item .card-other-values{margin-top:.5em;font-size:.8em}.quarto-grid-item .card-other-values tr{margin-bottom:.5em}.quarto-grid-item .card-other-values tr>td:first-of-type{font-weight:600;padding-right:1em;padding-left:1em;vertical-align:top}.quarto-grid-item div.post-contents{display:flex;flex-direction:column;text-decoration:none;height:100%}.quarto-grid-item .listing-item-img-placeholder{background-color:rgba(52,58,64,.25);flex-shrink:0}.quarto-grid-item .card-attribution{padding-top:1em;display:flex;gap:1em;text-transform:uppercase;color:#6c757d;font-weight:500;flex-grow:10;align-items:flex-end}.quarto-grid-item .description{padding-bottom:1em}.quarto-grid-item .card-attribution .date{align-self:flex-end}.quarto-grid-item .card-attribution.justify{justify-content:space-between}.quarto-grid-item .card-attribution.start{justify-content:flex-start}.quarto-grid-item .card-attribution.end{justify-content:flex-end}.quarto-grid-item .card-title{margin-bottom:.1em}.quarto-grid-item .card-subtitle{padding-top:.25em}.quarto-grid-item .card-text{font-size:.9em}.quarto-grid-item .listing-reading-time{padding-bottom:.25em}.quarto-grid-item .card-text-small{font-size:.8em}.quarto-grid-item .card-subtitle.subtitle{font-size:.9em;font-weight:600;padding-bottom:.5em}.quarto-grid-item .listing-categories{display:flex;flex-wrap:wrap;padding-bottom:5px}.quarto-grid-item .listing-categories .listing-category{color:#6c757d;border:solid #6c757d 1px;border-radius:.25rem;text-transform:uppercase;font-size:.65em;padding-left:.5em;padding-right:.5em;padding-top:.15em;padding-bottom:.15em;cursor:pointer;margin-right:4px;margin-bottom:4px}.quarto-grid-item.card-right{text-align:right}.quarto-grid-item.card-right .listing-categories{justify-content:flex-end}.quarto-grid-item.card-left{text-align:left}.quarto-grid-item.card-center{text-align:center}.quarto-grid-item.card-center .listing-description{text-align:justify}.quarto-grid-item.card-center .listing-categories{justify-content:center}table.quarto-listing-table td.image{padding:0px}table.quarto-listing-table td.image img{width:100%;max-width:50px;object-fit:contain}table.quarto-listing-table a{text-decoration:none;word-break:keep-all}table.quarto-listing-table th a{color:inherit}table.quarto-listing-table th a.asc:after{margin-bottom:-2px;margin-left:5px;display:inline-block;height:1rem;width:1rem;background-repeat:no-repeat;background-size:1rem 1rem;background-image:url('data:image/svg+xml,');content:""}table.quarto-listing-table th a.desc:after{margin-bottom:-2px;margin-left:5px;display:inline-block;height:1rem;width:1rem;background-repeat:no-repeat;background-size:1rem 1rem;background-image:url('data:image/svg+xml,');content:""}table.quarto-listing-table.table-hover td{cursor:pointer}.quarto-post.image-left{flex-direction:row}.quarto-post.image-right{flex-direction:row-reverse}@media(max-width: 767.98px){.quarto-post.image-right,.quarto-post.image-left{gap:0em;flex-direction:column}.quarto-post .metadata{padding-bottom:1em;order:2}.quarto-post .body{order:1}.quarto-post .thumbnail{order:3}}.list.quarto-listing-default div:last-of-type{border-bottom:none}@media(min-width: 992px){.quarto-listing-container-default{margin-right:2em}}div.quarto-post{display:flex;gap:2em;margin-bottom:1.5em;border-bottom:1px solid #dee2e6}@media(max-width: 767.98px){div.quarto-post{padding-bottom:1em}}div.quarto-post .metadata{flex-basis:20%;flex-grow:0;margin-top:.2em;flex-shrink:10}div.quarto-post .thumbnail{flex-basis:30%;flex-grow:0;flex-shrink:0}div.quarto-post .thumbnail img{margin-top:.4em;width:100%;object-fit:cover}div.quarto-post .body{flex-basis:45%;flex-grow:1;flex-shrink:0}div.quarto-post .body h3.listing-title,div.quarto-post .body .listing-title.h3{margin-top:0px;margin-bottom:0px;border-bottom:none}div.quarto-post .body .listing-subtitle{font-size:.875em;margin-bottom:.5em;margin-top:.2em}div.quarto-post .body .description{font-size:.9em}div.quarto-post .body pre code{white-space:pre-wrap}div.quarto-post a{color:#fff;text-decoration:none}div.quarto-post .metadata{display:flex;flex-direction:column;font-size:.8em;font-family:Lato,-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,"Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";flex-basis:33%}div.quarto-post .listing-categories{display:flex;flex-wrap:wrap;padding-bottom:5px}div.quarto-post .listing-categories .listing-category{color:#6c757d;border:solid #6c757d 1px;border-radius:.25rem;text-transform:uppercase;font-size:.65em;padding-left:.5em;padding-right:.5em;padding-top:.15em;padding-bottom:.15em;cursor:pointer;margin-right:4px;margin-bottom:4px}div.quarto-post .listing-description{margin-bottom:.5em}div.quarto-about-jolla{display:flex !important;flex-direction:column;align-items:center;margin-top:10%;padding-bottom:1em}div.quarto-about-jolla .about-image{object-fit:cover;margin-left:auto;margin-right:auto;margin-bottom:1.5em}div.quarto-about-jolla img.round{border-radius:50%}div.quarto-about-jolla img.rounded{border-radius:10px}div.quarto-about-jolla .quarto-title h1.title,div.quarto-about-jolla .quarto-title .title.h1{text-align:center}div.quarto-about-jolla .quarto-title .description{text-align:center}div.quarto-about-jolla h2,div.quarto-about-jolla .h2{border-bottom:none}div.quarto-about-jolla .about-sep{width:60%}div.quarto-about-jolla main{text-align:center}div.quarto-about-jolla .about-links{display:flex}@media(min-width: 992px){div.quarto-about-jolla .about-links{flex-direction:row;column-gap:.8em;row-gap:15px;flex-wrap:wrap}}@media(max-width: 991.98px){div.quarto-about-jolla .about-links{flex-direction:column;row-gap:1em;width:100%;padding-bottom:1.5em}}div.quarto-about-jolla .about-link{color:#fff;text-decoration:none;border:solid 1px}@media(min-width: 992px){div.quarto-about-jolla .about-link{font-size:.8em;padding:.25em .5em;border-radius:4px}}@media(max-width: 991.98px){div.quarto-about-jolla .about-link{font-size:1.1em;padding:.5em .5em;text-align:center;border-radius:6px}}div.quarto-about-jolla .about-link:hover{color:#00bc8c}div.quarto-about-jolla .about-link i.bi{margin-right:.15em}div.quarto-about-solana{display:flex !important;flex-direction:column;padding-top:3em !important;padding-bottom:1em}div.quarto-about-solana .about-entity{display:flex !important;align-items:start;justify-content:space-between}@media(min-width: 992px){div.quarto-about-solana .about-entity{flex-direction:row}}@media(max-width: 991.98px){div.quarto-about-solana .about-entity{flex-direction:column-reverse;align-items:center;text-align:center}}div.quarto-about-solana .about-entity .entity-contents{display:flex;flex-direction:column}@media(max-width: 767.98px){div.quarto-about-solana .about-entity .entity-contents{width:100%}}div.quarto-about-solana .about-entity .about-image{object-fit:cover}@media(max-width: 991.98px){div.quarto-about-solana .about-entity .about-image{margin-bottom:1.5em}}div.quarto-about-solana .about-entity img.round{border-radius:50%}div.quarto-about-solana .about-entity img.rounded{border-radius:10px}div.quarto-about-solana .about-entity .about-links{display:flex;justify-content:left;padding-bottom:1.2em}@media(min-width: 992px){div.quarto-about-solana .about-entity .about-links{flex-direction:row;column-gap:.8em;row-gap:15px;flex-wrap:wrap}}@media(max-width: 991.98px){div.quarto-about-solana .about-entity .about-links{flex-direction:column;row-gap:1em;width:100%;padding-bottom:1.5em}}div.quarto-about-solana .about-entity .about-link{color:#fff;text-decoration:none;border:solid 1px}@media(min-width: 992px){div.quarto-about-solana .about-entity .about-link{font-size:.8em;padding:.25em .5em;border-radius:4px}}@media(max-width: 991.98px){div.quarto-about-solana .about-entity .about-link{font-size:1.1em;padding:.5em .5em;text-align:center;border-radius:6px}}div.quarto-about-solana .about-entity .about-link:hover{color:#00bc8c}div.quarto-about-solana .about-entity .about-link i.bi{margin-right:.15em}div.quarto-about-solana .about-contents{padding-right:1.5em;flex-basis:0;flex-grow:1}div.quarto-about-solana .about-contents main.content{margin-top:0}div.quarto-about-solana .about-contents h2,div.quarto-about-solana .about-contents .h2{border-bottom:none}div.quarto-about-trestles{display:flex !important;flex-direction:row;padding-top:3em !important;padding-bottom:1em}@media(max-width: 991.98px){div.quarto-about-trestles{flex-direction:column;padding-top:0em !important}}div.quarto-about-trestles .about-entity{display:flex !important;flex-direction:column;align-items:center;text-align:center;padding-right:1em}@media(min-width: 992px){div.quarto-about-trestles .about-entity{flex:0 0 42%}}div.quarto-about-trestles .about-entity .about-image{object-fit:cover;margin-bottom:1.5em}div.quarto-about-trestles .about-entity img.round{border-radius:50%}div.quarto-about-trestles .about-entity img.rounded{border-radius:10px}div.quarto-about-trestles .about-entity .about-links{display:flex;justify-content:center}@media(min-width: 992px){div.quarto-about-trestles .about-entity .about-links{flex-direction:row;column-gap:.8em;row-gap:15px;flex-wrap:wrap}}@media(max-width: 991.98px){div.quarto-about-trestles .about-entity .about-links{flex-direction:column;row-gap:1em;width:100%;padding-bottom:1.5em}}div.quarto-about-trestles .about-entity .about-link{color:#fff;text-decoration:none;border:solid 1px}@media(min-width: 992px){div.quarto-about-trestles .about-entity .about-link{font-size:.8em;padding:.25em .5em;border-radius:4px}}@media(max-width: 991.98px){div.quarto-about-trestles .about-entity .about-link{font-size:1.1em;padding:.5em .5em;text-align:center;border-radius:6px}}div.quarto-about-trestles .about-entity .about-link:hover{color:#00bc8c}div.quarto-about-trestles .about-entity .about-link i.bi{margin-right:.15em}div.quarto-about-trestles .about-contents{flex-basis:0;flex-grow:1}div.quarto-about-trestles .about-contents h2,div.quarto-about-trestles .about-contents .h2{border-bottom:none}@media(min-width: 992px){div.quarto-about-trestles .about-contents{border-left:solid 1px #dee2e6;padding-left:1.5em}}div.quarto-about-trestles .about-contents main.content{margin-top:0}div.quarto-about-marquee{padding-bottom:1em}div.quarto-about-marquee .about-contents{display:flex;flex-direction:column}div.quarto-about-marquee .about-image{max-height:550px;margin-bottom:1.5em;object-fit:cover}div.quarto-about-marquee img.round{border-radius:50%}div.quarto-about-marquee img.rounded{border-radius:10px}div.quarto-about-marquee h2,div.quarto-about-marquee .h2{border-bottom:none}div.quarto-about-marquee .about-links{display:flex;justify-content:center;padding-top:1.5em}@media(min-width: 992px){div.quarto-about-marquee .about-links{flex-direction:row;column-gap:.8em;row-gap:15px;flex-wrap:wrap}}@media(max-width: 991.98px){div.quarto-about-marquee .about-links{flex-direction:column;row-gap:1em;width:100%;padding-bottom:1.5em}}div.quarto-about-marquee .about-link{color:#fff;text-decoration:none;border:solid 1px}@media(min-width: 992px){div.quarto-about-marquee .about-link{font-size:.8em;padding:.25em .5em;border-radius:4px}}@media(max-width: 991.98px){div.quarto-about-marquee .about-link{font-size:1.1em;padding:.5em .5em;text-align:center;border-radius:6px}}div.quarto-about-marquee .about-link:hover{color:#00bc8c}div.quarto-about-marquee .about-link i.bi{margin-right:.15em}@media(min-width: 992px){div.quarto-about-marquee .about-link{border:none}}div.quarto-about-broadside{display:flex;flex-direction:column;padding-bottom:1em}div.quarto-about-broadside .about-main{display:flex !important;padding-top:0 !important}@media(min-width: 992px){div.quarto-about-broadside .about-main{flex-direction:row;align-items:flex-start}}@media(max-width: 991.98px){div.quarto-about-broadside .about-main{flex-direction:column}}@media(max-width: 991.98px){div.quarto-about-broadside .about-main .about-entity{flex-shrink:0;width:100%;height:450px;margin-bottom:1.5em;background-size:cover;background-repeat:no-repeat}}@media(min-width: 992px){div.quarto-about-broadside .about-main .about-entity{flex:0 10 50%;margin-right:1.5em;width:100%;height:100%;background-size:100%;background-repeat:no-repeat}}div.quarto-about-broadside .about-main .about-contents{padding-top:14px;flex:0 0 50%}div.quarto-about-broadside h2,div.quarto-about-broadside .h2{border-bottom:none}div.quarto-about-broadside .about-sep{margin-top:1.5em;width:60%;align-self:center}div.quarto-about-broadside .about-links{display:flex;justify-content:center;column-gap:20px;padding-top:1.5em}@media(min-width: 992px){div.quarto-about-broadside .about-links{flex-direction:row;column-gap:.8em;row-gap:15px;flex-wrap:wrap}}@media(max-width: 991.98px){div.quarto-about-broadside .about-links{flex-direction:column;row-gap:1em;width:100%;padding-bottom:1.5em}}div.quarto-about-broadside .about-link{color:#fff;text-decoration:none;border:solid 1px}@media(min-width: 992px){div.quarto-about-broadside .about-link{font-size:.8em;padding:.25em .5em;border-radius:4px}}@media(max-width: 991.98px){div.quarto-about-broadside .about-link{font-size:1.1em;padding:.5em .5em;text-align:center;border-radius:6px}}div.quarto-about-broadside .about-link:hover{color:#00bc8c}div.quarto-about-broadside .about-link i.bi{margin-right:.15em}@media(min-width: 992px){div.quarto-about-broadside .about-link{border:none}}.tippy-box[data-theme~=quarto]{background-color:#222;border:solid 1px #dee2e6;border-radius:.25rem;color:#fff;font-size:.875rem}.tippy-box[data-theme~=quarto]>.tippy-backdrop{background-color:#222}.tippy-box[data-theme~=quarto]>.tippy-arrow:after,.tippy-box[data-theme~=quarto]>.tippy-svg-arrow:after{content:"";position:absolute;z-index:-1}.tippy-box[data-theme~=quarto]>.tippy-arrow:after{border-color:rgba(0,0,0,0);border-style:solid}.tippy-box[data-placement^=top]>.tippy-arrow:before{bottom:-6px}.tippy-box[data-placement^=bottom]>.tippy-arrow:before{top:-6px}.tippy-box[data-placement^=right]>.tippy-arrow:before{left:-6px}.tippy-box[data-placement^=left]>.tippy-arrow:before{right:-6px}.tippy-box[data-theme~=quarto][data-placement^=top]>.tippy-arrow:before{border-top-color:#222}.tippy-box[data-theme~=quarto][data-placement^=top]>.tippy-arrow:after{border-top-color:#dee2e6;border-width:7px 7px 0;top:17px;left:1px}.tippy-box[data-theme~=quarto][data-placement^=top]>.tippy-svg-arrow>svg{top:16px}.tippy-box[data-theme~=quarto][data-placement^=top]>.tippy-svg-arrow:after{top:17px}.tippy-box[data-theme~=quarto][data-placement^=bottom]>.tippy-arrow:before{border-bottom-color:#222;bottom:16px}.tippy-box[data-theme~=quarto][data-placement^=bottom]>.tippy-arrow:after{border-bottom-color:#dee2e6;border-width:0 7px 7px;bottom:17px;left:1px}.tippy-box[data-theme~=quarto][data-placement^=bottom]>.tippy-svg-arrow>svg{bottom:15px}.tippy-box[data-theme~=quarto][data-placement^=bottom]>.tippy-svg-arrow:after{bottom:17px}.tippy-box[data-theme~=quarto][data-placement^=left]>.tippy-arrow:before{border-left-color:#222}.tippy-box[data-theme~=quarto][data-placement^=left]>.tippy-arrow:after{border-left-color:#dee2e6;border-width:7px 0 7px 7px;left:17px;top:1px}.tippy-box[data-theme~=quarto][data-placement^=left]>.tippy-svg-arrow>svg{left:11px}.tippy-box[data-theme~=quarto][data-placement^=left]>.tippy-svg-arrow:after{left:12px}.tippy-box[data-theme~=quarto][data-placement^=right]>.tippy-arrow:before{border-right-color:#222;right:16px}.tippy-box[data-theme~=quarto][data-placement^=right]>.tippy-arrow:after{border-width:7px 7px 7px 0;right:17px;top:1px;border-right-color:#dee2e6}.tippy-box[data-theme~=quarto][data-placement^=right]>.tippy-svg-arrow>svg{right:11px}.tippy-box[data-theme~=quarto][data-placement^=right]>.tippy-svg-arrow:after{right:12px}.tippy-box[data-theme~=quarto]>.tippy-svg-arrow{fill:#fff}.tippy-box[data-theme~=quarto]>.tippy-svg-arrow:after{background-image:url(data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMTYiIGhlaWdodD0iNiIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48cGF0aCBkPSJNMCA2czEuNzk2LS4wMTMgNC42Ny0zLjYxNUM1Ljg1MS45IDYuOTMuMDA2IDggMGMxLjA3LS4wMDYgMi4xNDguODg3IDMuMzQzIDIuMzg1QzE0LjIzMyA2LjAwNSAxNiA2IDE2IDZIMHoiIGZpbGw9InJnYmEoMCwgOCwgMTYsIDAuMikiLz48L3N2Zz4=);background-size:16px 6px;width:16px;height:6px}.top-right{position:absolute;top:1em;right:1em}.visually-hidden{border:0;clip:rect(0 0 0 0);height:auto;margin:0;overflow:hidden;padding:0;position:absolute;width:1px;white-space:nowrap}.hidden{display:none !important}.zindex-bottom{z-index:-1 !important}figure.figure{display:block}.quarto-layout-panel{margin-bottom:1em}.quarto-layout-panel>figure{width:100%}.quarto-layout-panel>figure>figcaption,.quarto-layout-panel>.panel-caption{margin-top:10pt}.quarto-layout-panel>.table-caption{margin-top:0px}.table-caption p{margin-bottom:.5em}.quarto-layout-row{display:flex;flex-direction:row;align-items:flex-start}.quarto-layout-valign-top{align-items:flex-start}.quarto-layout-valign-bottom{align-items:flex-end}.quarto-layout-valign-center{align-items:center}.quarto-layout-cell{position:relative;margin-right:20px}.quarto-layout-cell:last-child{margin-right:0}.quarto-layout-cell figure,.quarto-layout-cell>p{margin:.2em}.quarto-layout-cell img{max-width:100%}.quarto-layout-cell .html-widget{width:100% !important}.quarto-layout-cell div figure p{margin:0}.quarto-layout-cell figure{display:block;margin-inline-start:0;margin-inline-end:0}.quarto-layout-cell table{display:inline-table}.quarto-layout-cell-subref figcaption,figure .quarto-layout-row figure figcaption{text-align:center;font-style:italic}.quarto-figure{position:relative;margin-bottom:1em}.quarto-figure>figure{width:100%;margin-bottom:0}.quarto-figure-left>figure>p,.quarto-figure-left>figure>div{text-align:left}.quarto-figure-center>figure>p,.quarto-figure-center>figure>div{text-align:center}.quarto-figure-right>figure>p,.quarto-figure-right>figure>div{text-align:right}.quarto-figure>figure>div.cell-annotation,.quarto-figure>figure>div code{text-align:left}figure>p:empty{display:none}figure>p:first-child{margin-top:0;margin-bottom:0}figure>figcaption.quarto-float-caption-bottom{margin-bottom:.5em}figure>figcaption.quarto-float-caption-top{margin-top:.5em}div[id^=tbl-]{position:relative}.quarto-figure>.anchorjs-link{position:absolute;top:.6em;right:.5em}div[id^=tbl-]>.anchorjs-link{position:absolute;top:.7em;right:.3em}.quarto-figure:hover>.anchorjs-link,div[id^=tbl-]:hover>.anchorjs-link,h2:hover>.anchorjs-link,.h2:hover>.anchorjs-link,h3:hover>.anchorjs-link,.h3:hover>.anchorjs-link,h4:hover>.anchorjs-link,.h4:hover>.anchorjs-link,h5:hover>.anchorjs-link,.h5:hover>.anchorjs-link,h6:hover>.anchorjs-link,.h6:hover>.anchorjs-link,.reveal-anchorjs-link>.anchorjs-link{opacity:1}#title-block-header{margin-block-end:1rem;position:relative;margin-top:-1px}#title-block-header .abstract{margin-block-start:1rem}#title-block-header .abstract .abstract-title{font-weight:600}#title-block-header a{text-decoration:none}#title-block-header .author,#title-block-header .date,#title-block-header .doi{margin-block-end:.2rem}#title-block-header .quarto-title-block>div{display:flex}#title-block-header .quarto-title-block>div>h1,#title-block-header .quarto-title-block>div>.h1{flex-grow:1}#title-block-header .quarto-title-block>div>button{flex-shrink:0;height:2.25rem;margin-top:0}@media(min-width: 992px){#title-block-header .quarto-title-block>div>button{margin-top:5px}}tr.header>th>p:last-of-type{margin-bottom:0px}table,table.table{margin-top:.5rem;margin-bottom:.5rem}caption,.table-caption{padding-top:.5rem;padding-bottom:.5rem;text-align:center}figure.quarto-float-tbl figcaption.quarto-float-caption-top{margin-top:.5rem;margin-bottom:.25rem;text-align:center}figure.quarto-float-tbl figcaption.quarto-float-caption-bottom{padding-top:.25rem;margin-bottom:.5rem;text-align:center}.utterances{max-width:none;margin-left:-8px}iframe{margin-bottom:1em}details{margin-bottom:1em}details[show]{margin-bottom:0}details>summary{color:#6c757d}details>summary>p:only-child{display:inline}pre.sourceCode,code.sourceCode{position:relative}dd code:not(.sourceCode),p code:not(.sourceCode){white-space:pre-wrap}code{white-space:pre}@media print{code{white-space:pre-wrap}}pre>code{display:block}pre>code.sourceCode{white-space:pre}pre>code.sourceCode>span>a:first-child::before{text-decoration:none}pre.code-overflow-wrap>code.sourceCode{white-space:pre-wrap}pre.code-overflow-scroll>code.sourceCode{white-space:pre}code a:any-link{color:inherit;text-decoration:none}code a:hover{color:inherit;text-decoration:underline}ul.task-list{padding-left:1em}[data-tippy-root]{display:inline-block}.tippy-content .footnote-back{display:none}.footnote-back{margin-left:.2em}.tippy-content{overflow-x:auto}.quarto-embedded-source-code{display:none}.quarto-unresolved-ref{font-weight:600}.quarto-cover-image{max-width:35%;float:right;margin-left:30px}.cell-output-display .widget-subarea{margin-bottom:1em}.cell-output-display:not(.no-overflow-x),.knitsql-table:not(.no-overflow-x){overflow-x:auto}.panel-input{margin-bottom:1em}.panel-input>div,.panel-input>div>div{display:inline-block;vertical-align:top;padding-right:12px}.panel-input>p:last-child{margin-bottom:0}.layout-sidebar{margin-bottom:1em}.layout-sidebar .tab-content{border:none}.tab-content>.page-columns.active{display:grid}div.sourceCode>iframe{width:100%;height:300px;margin-bottom:-0.5em}a{text-underline-offset:3px}div.ansi-escaped-output{font-family:monospace;display:block}/*! +* +* ansi colors from IPython notebook's +* +* we also add `bright-[color]-` synonyms for the `-[color]-intense` classes since +* that seems to be what ansi_up emits +* +*/.ansi-black-fg{color:#3e424d}.ansi-black-bg{background-color:#3e424d}.ansi-black-intense-black,.ansi-bright-black-fg{color:#282c36}.ansi-black-intense-black,.ansi-bright-black-bg{background-color:#282c36}.ansi-red-fg{color:#e75c58}.ansi-red-bg{background-color:#e75c58}.ansi-red-intense-red,.ansi-bright-red-fg{color:#b22b31}.ansi-red-intense-red,.ansi-bright-red-bg{background-color:#b22b31}.ansi-green-fg{color:#00a250}.ansi-green-bg{background-color:#00a250}.ansi-green-intense-green,.ansi-bright-green-fg{color:#007427}.ansi-green-intense-green,.ansi-bright-green-bg{background-color:#007427}.ansi-yellow-fg{color:#ddb62b}.ansi-yellow-bg{background-color:#ddb62b}.ansi-yellow-intense-yellow,.ansi-bright-yellow-fg{color:#b27d12}.ansi-yellow-intense-yellow,.ansi-bright-yellow-bg{background-color:#b27d12}.ansi-blue-fg{color:#208ffb}.ansi-blue-bg{background-color:#208ffb}.ansi-blue-intense-blue,.ansi-bright-blue-fg{color:#0065ca}.ansi-blue-intense-blue,.ansi-bright-blue-bg{background-color:#0065ca}.ansi-magenta-fg{color:#d160c4}.ansi-magenta-bg{background-color:#d160c4}.ansi-magenta-intense-magenta,.ansi-bright-magenta-fg{color:#a03196}.ansi-magenta-intense-magenta,.ansi-bright-magenta-bg{background-color:#a03196}.ansi-cyan-fg{color:#60c6c8}.ansi-cyan-bg{background-color:#60c6c8}.ansi-cyan-intense-cyan,.ansi-bright-cyan-fg{color:#258f8f}.ansi-cyan-intense-cyan,.ansi-bright-cyan-bg{background-color:#258f8f}.ansi-white-fg{color:#c5c1b4}.ansi-white-bg{background-color:#c5c1b4}.ansi-white-intense-white,.ansi-bright-white-fg{color:#a1a6b2}.ansi-white-intense-white,.ansi-bright-white-bg{background-color:#a1a6b2}.ansi-default-inverse-fg{color:#fff}.ansi-default-inverse-bg{background-color:#000}.ansi-bold{font-weight:bold}.ansi-underline{text-decoration:underline}:root{--quarto-body-bg: #222;--quarto-body-color: #fff;--quarto-text-muted: #6c757d;--quarto-border-color: #434343;--quarto-border-width: 1px;--quarto-border-radius: 0.25rem}table.gt_table{color:var(--quarto-body-color);font-size:1em;width:100%;background-color:rgba(0,0,0,0);border-top-width:inherit;border-bottom-width:inherit;border-color:var(--quarto-border-color)}table.gt_table th.gt_column_spanner_outer{color:var(--quarto-body-color);background-color:rgba(0,0,0,0);border-top-width:inherit;border-bottom-width:inherit;border-color:var(--quarto-border-color)}table.gt_table th.gt_col_heading{color:var(--quarto-body-color);font-weight:bold;background-color:rgba(0,0,0,0)}table.gt_table thead.gt_col_headings{border-bottom:1px solid currentColor;border-top-width:inherit;border-top-color:var(--quarto-border-color)}table.gt_table thead.gt_col_headings:not(:first-child){border-top-width:1px;border-top-color:var(--quarto-border-color)}table.gt_table td.gt_row{border-bottom-width:1px;border-bottom-color:var(--quarto-border-color);border-top-width:0px}table.gt_table tbody.gt_table_body{border-top-width:1px;border-bottom-width:1px;border-bottom-color:var(--quarto-border-color);border-top-color:currentColor}div.columns{display:initial;gap:initial}div.column{display:inline-block;overflow-x:initial;vertical-align:top;width:50%}.code-annotation-tip-content{word-wrap:break-word}.code-annotation-container-hidden{display:none !important}dl.code-annotation-container-grid{display:grid;grid-template-columns:min-content auto}dl.code-annotation-container-grid dt{grid-column:1}dl.code-annotation-container-grid dd{grid-column:2}pre.sourceCode.code-annotation-code{padding-right:0}code.sourceCode .code-annotation-anchor{z-index:100;position:relative;float:right;background-color:rgba(0,0,0,0)}input[type=checkbox]{margin-right:.5ch}:root{--mermaid-bg-color: #222;--mermaid-edge-color: #434343;--mermaid-node-fg-color: #fff;--mermaid-fg-color: #fff;--mermaid-fg-color--lighter: white;--mermaid-fg-color--lightest: white;--mermaid-font-family: Lato, -apple-system, BlinkMacSystemFont, Segoe UI, Roboto, Helvetica Neue, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji, Segoe UI Symbol;--mermaid-label-bg-color: #222;--mermaid-label-fg-color: #375a7f;--mermaid-node-bg-color: rgba(55, 90, 127, 0.1);--mermaid-node-fg-color: #fff}@media print{:root{font-size:11pt}#quarto-sidebar,#TOC,.nav-page{display:none}.page-columns .content{grid-column-start:page-start}.fixed-top{position:relative}.panel-caption,.figure-caption,figcaption{color:#666}}.code-copy-button{position:absolute;top:0;right:0;border:0;margin-top:5px;margin-right:5px;background-color:rgba(0,0,0,0);z-index:3}.code-copy-button:focus{outline:none}.code-copy-button-tooltip{font-size:.75em}pre.sourceCode:hover>.code-copy-button>.bi::before{display:inline-block;height:1rem;width:1rem;content:"";vertical-align:-0.125em;background-image:url('data:image/svg+xml,');background-repeat:no-repeat;background-size:1rem 1rem}pre.sourceCode:hover>.code-copy-button-checked>.bi::before{background-image:url('data:image/svg+xml,')}pre.sourceCode:hover>.code-copy-button:hover>.bi::before{background-image:url('data:image/svg+xml,')}pre.sourceCode:hover>.code-copy-button-checked:hover>.bi::before{background-image:url('data:image/svg+xml,')}main ol ol,main ul ul,main ol ul,main ul ol{margin-bottom:1em}ul>li:not(:has(>p))>ul,ol>li:not(:has(>p))>ul,ul>li:not(:has(>p))>ol,ol>li:not(:has(>p))>ol{margin-bottom:0}ul>li:not(:has(>p))>ul>li:has(>p),ol>li:not(:has(>p))>ul>li:has(>p),ul>li:not(:has(>p))>ol>li:has(>p),ol>li:not(:has(>p))>ol>li:has(>p){margin-top:1rem}body{margin:0}main.page-columns>header>h1.title,main.page-columns>header>.title.h1{margin-bottom:0}@media(min-width: 992px){body .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset] 35px [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(500px, calc(850px - 3em)) [body-content-end] 1.5em [body-end] 35px [body-end-outset] minmax(75px, 145px) [page-end-inset] 35px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.fullcontent:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset] 35px [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(500px, calc(850px - 3em)) [body-content-end] 1.5em [body-end] 35px [body-end-outset] 35px [page-end-inset page-end] 5fr [screen-end-inset] 1.5em}body.slimcontent:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset] 35px [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(500px, calc(850px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(0px, 200px) [page-end-inset] 35px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.listing:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start] minmax(50px, 100px) [page-start-inset] 50px [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(500px, calc(850px - 3em)) [body-content-end] 3em [body-end] 50px [body-end-outset] minmax(0px, 250px) [page-end-inset] minmax(50px, 100px) [page-end] 1fr [screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 35px [page-start-inset] minmax(0px, 175px) [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(450px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(0px, 200px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 35px [page-start-inset] minmax(0px, 175px) [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(450px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(0px, 200px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] minmax(25px, 50px) [page-start-inset] minmax(50px, 150px) [body-start-outset] minmax(25px, 50px) [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] minmax(25px, 50px) [body-end-outset] minmax(50px, 150px) [page-end-inset] minmax(25px, 50px) [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start] minmax(50px, 100px) [page-start-inset] 50px [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(500px, calc(1000px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(50px, 100px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked.fullcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start] minmax(50px, 100px) [page-start-inset] 50px [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(500px, calc(1000px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating.fullcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 50px [page-start-inset] minmax(50px, 150px) [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked.slimcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start] minmax(50px, 100px) [page-start-inset] 50px [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(450px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(0px, 200px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked.listing .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start] minmax(50px, 100px) [page-start-inset] 50px [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(500px, calc(1000px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(0px, 200px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating.slimcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 50px [page-start-inset] minmax(50px, 150px) [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(450px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(50px, 150px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating.listing .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] minmax(25px, 50px) [page-start-inset] minmax(50px, 150px) [body-start-outset] minmax(25px, 50px) [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] minmax(25px, 50px) [body-end-outset] minmax(50px, 150px) [page-end-inset] minmax(25px, 50px) [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}}@media(max-width: 991.98px){body .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset] 5fr [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] 35px [body-end-outset] minmax(75px, 145px) [page-end-inset] 35px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.fullcontent:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset] 5fr [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.slimcontent:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset] 5fr [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] 35px [body-end-outset] minmax(75px, 145px) [page-end-inset] 35px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.listing:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset] 5fr [body-start] 1.5em [body-content-start] minmax(500px, calc(1250px - 3em)) [body-content-end body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 35px [page-start-inset] minmax(0px, 145px) [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(450px, calc(800px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 35px [page-start-inset] minmax(0px, 145px) [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(450px, calc(800px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset body-start-outset body-start] 1.5em [body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(75px, 150px) [page-end-inset] 25px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(25px, 50px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked.fullcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(500px, calc(1000px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating.fullcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset body-start-outset body-start] 1em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 4fr [screen-end-inset] 1.5em [screen-end]}body.docked.slimcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(25px, 50px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked.listing .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(25px, 50px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating.slimcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset body-start-outset body-start] 1em [body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 35px [body-end-outset] minmax(75px, 145px) [page-end-inset] 35px [page-end] 4fr [screen-end-inset] 1.5em [screen-end]}body.floating.listing .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset body-start-outset body-start] 1em [body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(75px, 150px) [page-end-inset] 25px [page-end] 4fr [screen-end-inset] 1.5em [screen-end]}}@media(max-width: 767.98px){body .page-columns,body.fullcontent:not(.floating):not(.docked) .page-columns,body.slimcontent:not(.floating):not(.docked) .page-columns,body.docked .page-columns,body.docked.slimcontent .page-columns,body.docked.fullcontent .page-columns,body.floating .page-columns,body.floating.slimcontent .page-columns,body.floating.fullcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(0px, 1fr) [body-content-end body-end body-end-outset page-end-inset page-end screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(0px, 1fr) [body-content-end body-end body-end-outset page-end-inset page-end screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(0px, 1fr) [body-content-end body-end body-end-outset page-end-inset page-end screen-end-inset] 1.5em [screen-end]}nav[role=doc-toc]{display:none}}body,.page-row-navigation{grid-template-rows:[page-top] max-content [contents-top] max-content [contents-bottom] max-content [page-bottom]}.page-rows-contents{grid-template-rows:[content-top] minmax(max-content, 1fr) [content-bottom] minmax(60px, max-content) [page-bottom]}.page-full{grid-column:screen-start/screen-end !important}.page-columns>*{grid-column:body-content-start/body-content-end}.page-columns.column-page>*{grid-column:page-start/page-end}.page-columns.column-page-left .page-columns.page-full>*,.page-columns.column-page-left>*{grid-column:page-start/body-content-end}.page-columns.column-page-right .page-columns.page-full>*,.page-columns.column-page-right>*{grid-column:body-content-start/page-end}.page-rows{grid-auto-rows:auto}.header{grid-column:screen-start/screen-end;grid-row:page-top/contents-top}#quarto-content{padding:0;grid-column:screen-start/screen-end;grid-row:contents-top/contents-bottom}body.floating .sidebar.sidebar-navigation{grid-column:page-start/body-start;grid-row:content-top/page-bottom}body.docked .sidebar.sidebar-navigation{grid-column:screen-start/body-start;grid-row:content-top/page-bottom}.sidebar.toc-left{grid-column:page-start/body-start;grid-row:content-top/page-bottom}.sidebar.margin-sidebar{grid-column:body-end/page-end;grid-row:content-top/page-bottom}.page-columns .content{grid-column:body-content-start/body-content-end;grid-row:content-top/content-bottom;align-content:flex-start}.page-columns .page-navigation{grid-column:body-content-start/body-content-end;grid-row:content-bottom/page-bottom}.page-columns .footer{grid-column:screen-start/screen-end;grid-row:contents-bottom/page-bottom}.page-columns .column-body{grid-column:body-content-start/body-content-end}.page-columns .column-body-fullbleed{grid-column:body-start/body-end}.page-columns .column-body-outset{grid-column:body-start-outset/body-end-outset;z-index:998;opacity:.999}.page-columns .column-body-outset table{background:#222}.page-columns .column-body-outset-left{grid-column:body-start-outset/body-content-end;z-index:998;opacity:.999}.page-columns .column-body-outset-left table{background:#222}.page-columns .column-body-outset-right{grid-column:body-content-start/body-end-outset;z-index:998;opacity:.999}.page-columns .column-body-outset-right table{background:#222}.page-columns .column-page{grid-column:page-start/page-end;z-index:998;opacity:.999}.page-columns .column-page table{background:#222}.page-columns .column-page-inset{grid-column:page-start-inset/page-end-inset;z-index:998;opacity:.999}.page-columns .column-page-inset table{background:#222}.page-columns .column-page-inset-left{grid-column:page-start-inset/body-content-end;z-index:998;opacity:.999}.page-columns .column-page-inset-left table{background:#222}.page-columns .column-page-inset-right{grid-column:body-content-start/page-end-inset;z-index:998;opacity:.999}.page-columns .column-page-inset-right figcaption table{background:#222}.page-columns .column-page-left{grid-column:page-start/body-content-end;z-index:998;opacity:.999}.page-columns .column-page-left table{background:#222}.page-columns .column-page-right{grid-column:body-content-start/page-end;z-index:998;opacity:.999}.page-columns .column-page-right figcaption table{background:#222}#quarto-content.page-columns #quarto-margin-sidebar,#quarto-content.page-columns #quarto-sidebar{z-index:1}@media(max-width: 991.98px){#quarto-content.page-columns #quarto-margin-sidebar.collapse,#quarto-content.page-columns #quarto-sidebar.collapse,#quarto-content.page-columns #quarto-margin-sidebar.collapsing,#quarto-content.page-columns #quarto-sidebar.collapsing{z-index:1055}}#quarto-content.page-columns main.column-page,#quarto-content.page-columns main.column-page-right,#quarto-content.page-columns main.column-page-left{z-index:0}.page-columns .column-screen-inset{grid-column:screen-start-inset/screen-end-inset;z-index:998;opacity:.999}.page-columns .column-screen-inset table{background:#222}.page-columns .column-screen-inset-left{grid-column:screen-start-inset/body-content-end;z-index:998;opacity:.999}.page-columns .column-screen-inset-left table{background:#222}.page-columns .column-screen-inset-right{grid-column:body-content-start/screen-end-inset;z-index:998;opacity:.999}.page-columns .column-screen-inset-right table{background:#222}.page-columns .column-screen{grid-column:screen-start/screen-end;z-index:998;opacity:.999}.page-columns .column-screen table{background:#222}.page-columns .column-screen-left{grid-column:screen-start/body-content-end;z-index:998;opacity:.999}.page-columns .column-screen-left table{background:#222}.page-columns .column-screen-right{grid-column:body-content-start/screen-end;z-index:998;opacity:.999}.page-columns .column-screen-right table{background:#222}.page-columns .column-screen-inset-shaded{grid-column:screen-start/screen-end;padding:1em;background:#6f6f6f;z-index:998;opacity:.999;margin-bottom:1em}.zindex-content{z-index:998;opacity:.999}.zindex-modal{z-index:1055;opacity:.999}.zindex-over-content{z-index:999;opacity:.999}img.img-fluid.column-screen,img.img-fluid.column-screen-inset-shaded,img.img-fluid.column-screen-inset,img.img-fluid.column-screen-inset-left,img.img-fluid.column-screen-inset-right,img.img-fluid.column-screen-left,img.img-fluid.column-screen-right{width:100%}@media(min-width: 992px){.margin-caption,div.aside,aside:not(.footnotes):not(.sidebar),.column-margin{grid-column:body-end/page-end !important;z-index:998}.column-sidebar{grid-column:page-start/body-start !important;z-index:998}.column-leftmargin{grid-column:screen-start-inset/body-start !important;z-index:998}.no-row-height{height:1em;overflow:visible}}@media(max-width: 991.98px){.margin-caption,div.aside,aside:not(.footnotes):not(.sidebar),.column-margin{grid-column:body-end/page-end !important;z-index:998}.no-row-height{height:1em;overflow:visible}.page-columns.page-full{overflow:visible}.page-columns.toc-left .margin-caption,.page-columns.toc-left div.aside,.page-columns.toc-left aside:not(.footnotes):not(.sidebar),.page-columns.toc-left .column-margin{grid-column:body-content-start/body-content-end !important;z-index:998;opacity:.999}.page-columns.toc-left .no-row-height{height:initial;overflow:initial}}@media(max-width: 767.98px){.margin-caption,div.aside,aside:not(.footnotes):not(.sidebar),.column-margin{grid-column:body-content-start/body-content-end !important;z-index:998;opacity:.999}.no-row-height{height:initial;overflow:initial}#quarto-margin-sidebar{display:none}#quarto-sidebar-toc-left{display:none}.hidden-sm{display:none}}.panel-grid{display:grid;grid-template-rows:repeat(1, 1fr);grid-template-columns:repeat(24, 1fr);gap:1em}.panel-grid .g-col-1{grid-column:auto/span 1}.panel-grid .g-col-2{grid-column:auto/span 2}.panel-grid .g-col-3{grid-column:auto/span 3}.panel-grid .g-col-4{grid-column:auto/span 4}.panel-grid .g-col-5{grid-column:auto/span 5}.panel-grid .g-col-6{grid-column:auto/span 6}.panel-grid .g-col-7{grid-column:auto/span 7}.panel-grid .g-col-8{grid-column:auto/span 8}.panel-grid .g-col-9{grid-column:auto/span 9}.panel-grid .g-col-10{grid-column:auto/span 10}.panel-grid .g-col-11{grid-column:auto/span 11}.panel-grid .g-col-12{grid-column:auto/span 12}.panel-grid .g-col-13{grid-column:auto/span 13}.panel-grid .g-col-14{grid-column:auto/span 14}.panel-grid .g-col-15{grid-column:auto/span 15}.panel-grid .g-col-16{grid-column:auto/span 16}.panel-grid .g-col-17{grid-column:auto/span 17}.panel-grid .g-col-18{grid-column:auto/span 18}.panel-grid .g-col-19{grid-column:auto/span 19}.panel-grid .g-col-20{grid-column:auto/span 20}.panel-grid .g-col-21{grid-column:auto/span 21}.panel-grid .g-col-22{grid-column:auto/span 22}.panel-grid .g-col-23{grid-column:auto/span 23}.panel-grid .g-col-24{grid-column:auto/span 24}.panel-grid .g-start-1{grid-column-start:1}.panel-grid .g-start-2{grid-column-start:2}.panel-grid .g-start-3{grid-column-start:3}.panel-grid .g-start-4{grid-column-start:4}.panel-grid .g-start-5{grid-column-start:5}.panel-grid .g-start-6{grid-column-start:6}.panel-grid .g-start-7{grid-column-start:7}.panel-grid .g-start-8{grid-column-start:8}.panel-grid .g-start-9{grid-column-start:9}.panel-grid .g-start-10{grid-column-start:10}.panel-grid .g-start-11{grid-column-start:11}.panel-grid .g-start-12{grid-column-start:12}.panel-grid .g-start-13{grid-column-start:13}.panel-grid .g-start-14{grid-column-start:14}.panel-grid .g-start-15{grid-column-start:15}.panel-grid .g-start-16{grid-column-start:16}.panel-grid .g-start-17{grid-column-start:17}.panel-grid .g-start-18{grid-column-start:18}.panel-grid .g-start-19{grid-column-start:19}.panel-grid .g-start-20{grid-column-start:20}.panel-grid .g-start-21{grid-column-start:21}.panel-grid .g-start-22{grid-column-start:22}.panel-grid .g-start-23{grid-column-start:23}@media(min-width: 576px){.panel-grid .g-col-sm-1{grid-column:auto/span 1}.panel-grid .g-col-sm-2{grid-column:auto/span 2}.panel-grid .g-col-sm-3{grid-column:auto/span 3}.panel-grid .g-col-sm-4{grid-column:auto/span 4}.panel-grid .g-col-sm-5{grid-column:auto/span 5}.panel-grid .g-col-sm-6{grid-column:auto/span 6}.panel-grid .g-col-sm-7{grid-column:auto/span 7}.panel-grid .g-col-sm-8{grid-column:auto/span 8}.panel-grid .g-col-sm-9{grid-column:auto/span 9}.panel-grid .g-col-sm-10{grid-column:auto/span 10}.panel-grid .g-col-sm-11{grid-column:auto/span 11}.panel-grid .g-col-sm-12{grid-column:auto/span 12}.panel-grid .g-col-sm-13{grid-column:auto/span 13}.panel-grid .g-col-sm-14{grid-column:auto/span 14}.panel-grid .g-col-sm-15{grid-column:auto/span 15}.panel-grid .g-col-sm-16{grid-column:auto/span 16}.panel-grid .g-col-sm-17{grid-column:auto/span 17}.panel-grid .g-col-sm-18{grid-column:auto/span 18}.panel-grid .g-col-sm-19{grid-column:auto/span 19}.panel-grid .g-col-sm-20{grid-column:auto/span 20}.panel-grid .g-col-sm-21{grid-column:auto/span 21}.panel-grid .g-col-sm-22{grid-column:auto/span 22}.panel-grid .g-col-sm-23{grid-column:auto/span 23}.panel-grid .g-col-sm-24{grid-column:auto/span 24}.panel-grid .g-start-sm-1{grid-column-start:1}.panel-grid .g-start-sm-2{grid-column-start:2}.panel-grid .g-start-sm-3{grid-column-start:3}.panel-grid .g-start-sm-4{grid-column-start:4}.panel-grid .g-start-sm-5{grid-column-start:5}.panel-grid .g-start-sm-6{grid-column-start:6}.panel-grid .g-start-sm-7{grid-column-start:7}.panel-grid .g-start-sm-8{grid-column-start:8}.panel-grid .g-start-sm-9{grid-column-start:9}.panel-grid .g-start-sm-10{grid-column-start:10}.panel-grid .g-start-sm-11{grid-column-start:11}.panel-grid .g-start-sm-12{grid-column-start:12}.panel-grid .g-start-sm-13{grid-column-start:13}.panel-grid .g-start-sm-14{grid-column-start:14}.panel-grid .g-start-sm-15{grid-column-start:15}.panel-grid .g-start-sm-16{grid-column-start:16}.panel-grid .g-start-sm-17{grid-column-start:17}.panel-grid .g-start-sm-18{grid-column-start:18}.panel-grid .g-start-sm-19{grid-column-start:19}.panel-grid .g-start-sm-20{grid-column-start:20}.panel-grid .g-start-sm-21{grid-column-start:21}.panel-grid .g-start-sm-22{grid-column-start:22}.panel-grid .g-start-sm-23{grid-column-start:23}}@media(min-width: 768px){.panel-grid .g-col-md-1{grid-column:auto/span 1}.panel-grid .g-col-md-2{grid-column:auto/span 2}.panel-grid .g-col-md-3{grid-column:auto/span 3}.panel-grid .g-col-md-4{grid-column:auto/span 4}.panel-grid .g-col-md-5{grid-column:auto/span 5}.panel-grid .g-col-md-6{grid-column:auto/span 6}.panel-grid .g-col-md-7{grid-column:auto/span 7}.panel-grid .g-col-md-8{grid-column:auto/span 8}.panel-grid .g-col-md-9{grid-column:auto/span 9}.panel-grid .g-col-md-10{grid-column:auto/span 10}.panel-grid .g-col-md-11{grid-column:auto/span 11}.panel-grid .g-col-md-12{grid-column:auto/span 12}.panel-grid .g-col-md-13{grid-column:auto/span 13}.panel-grid .g-col-md-14{grid-column:auto/span 14}.panel-grid .g-col-md-15{grid-column:auto/span 15}.panel-grid .g-col-md-16{grid-column:auto/span 16}.panel-grid .g-col-md-17{grid-column:auto/span 17}.panel-grid .g-col-md-18{grid-column:auto/span 18}.panel-grid .g-col-md-19{grid-column:auto/span 19}.panel-grid .g-col-md-20{grid-column:auto/span 20}.panel-grid .g-col-md-21{grid-column:auto/span 21}.panel-grid .g-col-md-22{grid-column:auto/span 22}.panel-grid .g-col-md-23{grid-column:auto/span 23}.panel-grid .g-col-md-24{grid-column:auto/span 24}.panel-grid .g-start-md-1{grid-column-start:1}.panel-grid .g-start-md-2{grid-column-start:2}.panel-grid .g-start-md-3{grid-column-start:3}.panel-grid .g-start-md-4{grid-column-start:4}.panel-grid .g-start-md-5{grid-column-start:5}.panel-grid .g-start-md-6{grid-column-start:6}.panel-grid .g-start-md-7{grid-column-start:7}.panel-grid .g-start-md-8{grid-column-start:8}.panel-grid .g-start-md-9{grid-column-start:9}.panel-grid .g-start-md-10{grid-column-start:10}.panel-grid .g-start-md-11{grid-column-start:11}.panel-grid .g-start-md-12{grid-column-start:12}.panel-grid .g-start-md-13{grid-column-start:13}.panel-grid .g-start-md-14{grid-column-start:14}.panel-grid .g-start-md-15{grid-column-start:15}.panel-grid .g-start-md-16{grid-column-start:16}.panel-grid .g-start-md-17{grid-column-start:17}.panel-grid .g-start-md-18{grid-column-start:18}.panel-grid .g-start-md-19{grid-column-start:19}.panel-grid .g-start-md-20{grid-column-start:20}.panel-grid .g-start-md-21{grid-column-start:21}.panel-grid .g-start-md-22{grid-column-start:22}.panel-grid .g-start-md-23{grid-column-start:23}}@media(min-width: 992px){.panel-grid .g-col-lg-1{grid-column:auto/span 1}.panel-grid .g-col-lg-2{grid-column:auto/span 2}.panel-grid .g-col-lg-3{grid-column:auto/span 3}.panel-grid .g-col-lg-4{grid-column:auto/span 4}.panel-grid .g-col-lg-5{grid-column:auto/span 5}.panel-grid .g-col-lg-6{grid-column:auto/span 6}.panel-grid .g-col-lg-7{grid-column:auto/span 7}.panel-grid .g-col-lg-8{grid-column:auto/span 8}.panel-grid .g-col-lg-9{grid-column:auto/span 9}.panel-grid .g-col-lg-10{grid-column:auto/span 10}.panel-grid .g-col-lg-11{grid-column:auto/span 11}.panel-grid .g-col-lg-12{grid-column:auto/span 12}.panel-grid .g-col-lg-13{grid-column:auto/span 13}.panel-grid .g-col-lg-14{grid-column:auto/span 14}.panel-grid .g-col-lg-15{grid-column:auto/span 15}.panel-grid .g-col-lg-16{grid-column:auto/span 16}.panel-grid .g-col-lg-17{grid-column:auto/span 17}.panel-grid .g-col-lg-18{grid-column:auto/span 18}.panel-grid .g-col-lg-19{grid-column:auto/span 19}.panel-grid .g-col-lg-20{grid-column:auto/span 20}.panel-grid .g-col-lg-21{grid-column:auto/span 21}.panel-grid .g-col-lg-22{grid-column:auto/span 22}.panel-grid .g-col-lg-23{grid-column:auto/span 23}.panel-grid .g-col-lg-24{grid-column:auto/span 24}.panel-grid .g-start-lg-1{grid-column-start:1}.panel-grid .g-start-lg-2{grid-column-start:2}.panel-grid .g-start-lg-3{grid-column-start:3}.panel-grid .g-start-lg-4{grid-column-start:4}.panel-grid .g-start-lg-5{grid-column-start:5}.panel-grid .g-start-lg-6{grid-column-start:6}.panel-grid .g-start-lg-7{grid-column-start:7}.panel-grid .g-start-lg-8{grid-column-start:8}.panel-grid .g-start-lg-9{grid-column-start:9}.panel-grid .g-start-lg-10{grid-column-start:10}.panel-grid .g-start-lg-11{grid-column-start:11}.panel-grid .g-start-lg-12{grid-column-start:12}.panel-grid .g-start-lg-13{grid-column-start:13}.panel-grid .g-start-lg-14{grid-column-start:14}.panel-grid .g-start-lg-15{grid-column-start:15}.panel-grid .g-start-lg-16{grid-column-start:16}.panel-grid .g-start-lg-17{grid-column-start:17}.panel-grid .g-start-lg-18{grid-column-start:18}.panel-grid .g-start-lg-19{grid-column-start:19}.panel-grid .g-start-lg-20{grid-column-start:20}.panel-grid .g-start-lg-21{grid-column-start:21}.panel-grid .g-start-lg-22{grid-column-start:22}.panel-grid .g-start-lg-23{grid-column-start:23}}@media(min-width: 1200px){.panel-grid .g-col-xl-1{grid-column:auto/span 1}.panel-grid .g-col-xl-2{grid-column:auto/span 2}.panel-grid .g-col-xl-3{grid-column:auto/span 3}.panel-grid .g-col-xl-4{grid-column:auto/span 4}.panel-grid .g-col-xl-5{grid-column:auto/span 5}.panel-grid .g-col-xl-6{grid-column:auto/span 6}.panel-grid .g-col-xl-7{grid-column:auto/span 7}.panel-grid .g-col-xl-8{grid-column:auto/span 8}.panel-grid .g-col-xl-9{grid-column:auto/span 9}.panel-grid .g-col-xl-10{grid-column:auto/span 10}.panel-grid .g-col-xl-11{grid-column:auto/span 11}.panel-grid .g-col-xl-12{grid-column:auto/span 12}.panel-grid .g-col-xl-13{grid-column:auto/span 13}.panel-grid .g-col-xl-14{grid-column:auto/span 14}.panel-grid .g-col-xl-15{grid-column:auto/span 15}.panel-grid .g-col-xl-16{grid-column:auto/span 16}.panel-grid .g-col-xl-17{grid-column:auto/span 17}.panel-grid .g-col-xl-18{grid-column:auto/span 18}.panel-grid .g-col-xl-19{grid-column:auto/span 19}.panel-grid .g-col-xl-20{grid-column:auto/span 20}.panel-grid .g-col-xl-21{grid-column:auto/span 21}.panel-grid .g-col-xl-22{grid-column:auto/span 22}.panel-grid .g-col-xl-23{grid-column:auto/span 23}.panel-grid .g-col-xl-24{grid-column:auto/span 24}.panel-grid .g-start-xl-1{grid-column-start:1}.panel-grid .g-start-xl-2{grid-column-start:2}.panel-grid .g-start-xl-3{grid-column-start:3}.panel-grid .g-start-xl-4{grid-column-start:4}.panel-grid .g-start-xl-5{grid-column-start:5}.panel-grid .g-start-xl-6{grid-column-start:6}.panel-grid .g-start-xl-7{grid-column-start:7}.panel-grid .g-start-xl-8{grid-column-start:8}.panel-grid .g-start-xl-9{grid-column-start:9}.panel-grid .g-start-xl-10{grid-column-start:10}.panel-grid .g-start-xl-11{grid-column-start:11}.panel-grid .g-start-xl-12{grid-column-start:12}.panel-grid .g-start-xl-13{grid-column-start:13}.panel-grid .g-start-xl-14{grid-column-start:14}.panel-grid .g-start-xl-15{grid-column-start:15}.panel-grid .g-start-xl-16{grid-column-start:16}.panel-grid .g-start-xl-17{grid-column-start:17}.panel-grid .g-start-xl-18{grid-column-start:18}.panel-grid .g-start-xl-19{grid-column-start:19}.panel-grid .g-start-xl-20{grid-column-start:20}.panel-grid .g-start-xl-21{grid-column-start:21}.panel-grid .g-start-xl-22{grid-column-start:22}.panel-grid .g-start-xl-23{grid-column-start:23}}@media(min-width: 1400px){.panel-grid .g-col-xxl-1{grid-column:auto/span 1}.panel-grid .g-col-xxl-2{grid-column:auto/span 2}.panel-grid .g-col-xxl-3{grid-column:auto/span 3}.panel-grid .g-col-xxl-4{grid-column:auto/span 4}.panel-grid .g-col-xxl-5{grid-column:auto/span 5}.panel-grid .g-col-xxl-6{grid-column:auto/span 6}.panel-grid .g-col-xxl-7{grid-column:auto/span 7}.panel-grid .g-col-xxl-8{grid-column:auto/span 8}.panel-grid .g-col-xxl-9{grid-column:auto/span 9}.panel-grid .g-col-xxl-10{grid-column:auto/span 10}.panel-grid .g-col-xxl-11{grid-column:auto/span 11}.panel-grid .g-col-xxl-12{grid-column:auto/span 12}.panel-grid .g-col-xxl-13{grid-column:auto/span 13}.panel-grid .g-col-xxl-14{grid-column:auto/span 14}.panel-grid .g-col-xxl-15{grid-column:auto/span 15}.panel-grid .g-col-xxl-16{grid-column:auto/span 16}.panel-grid .g-col-xxl-17{grid-column:auto/span 17}.panel-grid .g-col-xxl-18{grid-column:auto/span 18}.panel-grid .g-col-xxl-19{grid-column:auto/span 19}.panel-grid .g-col-xxl-20{grid-column:auto/span 20}.panel-grid .g-col-xxl-21{grid-column:auto/span 21}.panel-grid .g-col-xxl-22{grid-column:auto/span 22}.panel-grid .g-col-xxl-23{grid-column:auto/span 23}.panel-grid .g-col-xxl-24{grid-column:auto/span 24}.panel-grid .g-start-xxl-1{grid-column-start:1}.panel-grid .g-start-xxl-2{grid-column-start:2}.panel-grid .g-start-xxl-3{grid-column-start:3}.panel-grid .g-start-xxl-4{grid-column-start:4}.panel-grid .g-start-xxl-5{grid-column-start:5}.panel-grid .g-start-xxl-6{grid-column-start:6}.panel-grid .g-start-xxl-7{grid-column-start:7}.panel-grid .g-start-xxl-8{grid-column-start:8}.panel-grid .g-start-xxl-9{grid-column-start:9}.panel-grid .g-start-xxl-10{grid-column-start:10}.panel-grid .g-start-xxl-11{grid-column-start:11}.panel-grid .g-start-xxl-12{grid-column-start:12}.panel-grid .g-start-xxl-13{grid-column-start:13}.panel-grid .g-start-xxl-14{grid-column-start:14}.panel-grid .g-start-xxl-15{grid-column-start:15}.panel-grid .g-start-xxl-16{grid-column-start:16}.panel-grid .g-start-xxl-17{grid-column-start:17}.panel-grid .g-start-xxl-18{grid-column-start:18}.panel-grid .g-start-xxl-19{grid-column-start:19}.panel-grid .g-start-xxl-20{grid-column-start:20}.panel-grid .g-start-xxl-21{grid-column-start:21}.panel-grid .g-start-xxl-22{grid-column-start:22}.panel-grid .g-start-xxl-23{grid-column-start:23}}main{margin-top:1em;margin-bottom:1em}h1,.h1,h2,.h2{color:inherit;margin-top:2rem;margin-bottom:1rem;font-weight:600}h1.title,.title.h1{margin-top:0}main.content>section:first-of-type>h2:first-child,main.content>section:first-of-type>.h2:first-child{margin-top:0}h2,.h2{border-bottom:1px solid #434343;padding-bottom:.5rem}h3,.h3{font-weight:600}h3,.h3,h4,.h4{opacity:.9;margin-top:1.5rem}h5,.h5,h6,.h6{opacity:.9}.header-section-number{color:#bfbfbf}.nav-link.active .header-section-number{color:inherit}mark,.mark{padding:0em}.panel-caption,.figure-caption,.subfigure-caption,.table-caption,figcaption,caption{font-size:.9rem;color:#bfbfbf}.quarto-layout-cell[data-ref-parent] caption{color:#bfbfbf}.column-margin figcaption,.margin-caption,div.aside,aside,.column-margin{color:#bfbfbf;font-size:.825rem}.panel-caption.margin-caption{text-align:inherit}.column-margin.column-container p{margin-bottom:0}.column-margin.column-container>*:not(.collapse):first-child{padding-bottom:.5em;display:block}.column-margin.column-container>*:not(.collapse):not(:first-child){padding-top:.5em;padding-bottom:.5em;display:block}.column-margin.column-container>*.collapse:not(.show){display:none}@media(min-width: 768px){.column-margin.column-container .callout-margin-content:first-child{margin-top:4.5em}.column-margin.column-container .callout-margin-content-simple:first-child{margin-top:3.5em}}.margin-caption>*{padding-top:.5em;padding-bottom:.5em}@media(max-width: 767.98px){.quarto-layout-row{flex-direction:column}}.nav-tabs .nav-item{margin-top:1px;cursor:pointer}.tab-content{margin-top:0px;border-left:#434343 1px solid;border-right:#434343 1px solid;border-bottom:#434343 1px solid;margin-left:0;padding:1em;margin-bottom:1em}@media(max-width: 767.98px){.layout-sidebar{margin-left:0;margin-right:0}}.panel-sidebar,.panel-sidebar .form-control,.panel-input,.panel-input .form-control,.selectize-dropdown{font-size:.9rem}.panel-sidebar .form-control,.panel-input .form-control{padding-top:.1rem}.tab-pane div.sourceCode{margin-top:0px}.tab-pane>p{padding-top:0}.tab-pane>p:nth-child(1){padding-top:0}.tab-pane>p:last-child{margin-bottom:0}.tab-pane>pre:last-child{margin-bottom:0}.tab-content>.tab-pane:not(.active){display:none !important}div.sourceCode{background-color:#000;border:1px solid #000;border-radius:.25rem}pre.sourceCode{background-color:rgba(0,0,0,0)}pre.sourceCode{border:none;font-size:.875em;overflow:visible !important;padding:.4em}.callout pre.sourceCode{padding-left:0}div.sourceCode{overflow-y:hidden}.callout div.sourceCode{margin-left:initial}.blockquote{font-size:inherit;padding-left:1rem;padding-right:1.5rem;color:#bfbfbf}.blockquote h1:first-child,.blockquote .h1:first-child,.blockquote h2:first-child,.blockquote .h2:first-child,.blockquote h3:first-child,.blockquote .h3:first-child,.blockquote h4:first-child,.blockquote .h4:first-child,.blockquote h5:first-child,.blockquote .h5:first-child{margin-top:0}pre{background-color:initial;padding:initial;border:initial}p pre code:not(.sourceCode),li pre code:not(.sourceCode),pre code:not(.sourceCode){background-color:initial}p code:not(.sourceCode),li code:not(.sourceCode),td code:not(.sourceCode){background-color:#000;padding:.2em}nav p code:not(.sourceCode),nav li code:not(.sourceCode),nav td code:not(.sourceCode){background-color:rgba(0,0,0,0);padding:0}td code:not(.sourceCode){white-space:pre-wrap}#quarto-embedded-source-code-modal>.modal-dialog{max-width:1000px;padding-left:1.75rem;padding-right:1.75rem}#quarto-embedded-source-code-modal>.modal-dialog>.modal-content>.modal-body{padding:0}#quarto-embedded-source-code-modal>.modal-dialog>.modal-content>.modal-body div.sourceCode{margin:0;padding:.2rem .2rem;border-radius:0px;border:none}#quarto-embedded-source-code-modal>.modal-dialog>.modal-content>.modal-header{padding:.7rem}.code-tools-button{font-size:1rem;padding:.15rem .15rem;margin-left:5px;color:#6c757d;background-color:rgba(0,0,0,0);transition:initial;cursor:pointer}.code-tools-button>.bi::before{display:inline-block;height:1rem;width:1rem;content:"";vertical-align:-0.125em;background-image:url('data:image/svg+xml,');background-repeat:no-repeat;background-size:1rem 1rem}.code-tools-button:hover>.bi::before{background-image:url('data:image/svg+xml,')}#quarto-embedded-source-code-modal .code-copy-button>.bi::before{background-image:url('data:image/svg+xml,')}#quarto-embedded-source-code-modal .code-copy-button-checked>.bi::before{background-image:url('data:image/svg+xml,')}.sidebar{will-change:top;transition:top 200ms linear;position:sticky;overflow-y:auto;padding-top:1.2em;max-height:100vh}.sidebar.toc-left,.sidebar.margin-sidebar{top:0px;padding-top:1em}.sidebar.quarto-banner-title-block-sidebar>*{padding-top:1.65em}figure .quarto-notebook-link{margin-top:.5em}.quarto-notebook-link{font-size:.75em;color:#6c757d;margin-bottom:1em;text-decoration:none;display:block}.quarto-notebook-link:hover{text-decoration:underline;color:#00bc8c}.quarto-notebook-link::before{display:inline-block;height:.75rem;width:.75rem;margin-bottom:0em;margin-right:.25em;content:"";vertical-align:-0.125em;background-image:url('data:image/svg+xml,');background-repeat:no-repeat;background-size:.75rem .75rem}.toc-actions i.bi,.quarto-code-links i.bi,.quarto-other-links i.bi,.quarto-alternate-notebooks i.bi,.quarto-alternate-formats i.bi{margin-right:.4em;font-size:.8rem}.quarto-other-links-text-target .quarto-code-links i.bi,.quarto-other-links-text-target .quarto-other-links i.bi{margin-right:.2em}.quarto-other-formats-text-target .quarto-alternate-formats i.bi{margin-right:.1em}.toc-actions i.bi.empty,.quarto-code-links i.bi.empty,.quarto-other-links i.bi.empty,.quarto-alternate-notebooks i.bi.empty,.quarto-alternate-formats i.bi.empty{padding-left:1em}.quarto-notebook h2,.quarto-notebook .h2{border-bottom:none}.quarto-notebook .cell-container{display:flex}.quarto-notebook .cell-container .cell{flex-grow:4}.quarto-notebook .cell-container .cell-decorator{padding-top:1.5em;padding-right:1em;text-align:right}.quarto-notebook .cell-container.code-fold .cell-decorator{padding-top:3em}.quarto-notebook .cell-code code{white-space:pre-wrap}.quarto-notebook .cell .cell-output-stderr pre code,.quarto-notebook .cell .cell-output-stdout pre code{white-space:pre-wrap;overflow-wrap:anywhere}.toc-actions,.quarto-alternate-formats,.quarto-other-links,.quarto-code-links,.quarto-alternate-notebooks{padding-left:0em}.sidebar .toc-actions a,.sidebar .quarto-alternate-formats a,.sidebar .quarto-other-links a,.sidebar .quarto-code-links a,.sidebar .quarto-alternate-notebooks a,.sidebar nav[role=doc-toc] a{text-decoration:none}.sidebar .toc-actions a:hover,.sidebar .quarto-other-links a:hover,.sidebar .quarto-code-links a:hover,.sidebar .quarto-alternate-formats a:hover,.sidebar .quarto-alternate-notebooks a:hover{color:#00bc8c}.sidebar .toc-actions h2,.sidebar .toc-actions .h2,.sidebar .quarto-code-links h2,.sidebar .quarto-code-links .h2,.sidebar .quarto-other-links h2,.sidebar .quarto-other-links .h2,.sidebar .quarto-alternate-notebooks h2,.sidebar .quarto-alternate-notebooks .h2,.sidebar .quarto-alternate-formats h2,.sidebar .quarto-alternate-formats .h2,.sidebar nav[role=doc-toc]>h2,.sidebar nav[role=doc-toc]>.h2{font-weight:500;margin-bottom:.2rem;margin-top:.3rem;font-family:inherit;border-bottom:0;padding-bottom:0;padding-top:0px}.sidebar .toc-actions>h2,.sidebar .toc-actions>.h2,.sidebar .quarto-code-links>h2,.sidebar .quarto-code-links>.h2,.sidebar .quarto-other-links>h2,.sidebar .quarto-other-links>.h2,.sidebar .quarto-alternate-notebooks>h2,.sidebar .quarto-alternate-notebooks>.h2,.sidebar .quarto-alternate-formats>h2,.sidebar .quarto-alternate-formats>.h2{font-size:.8rem}.sidebar nav[role=doc-toc]>h2,.sidebar nav[role=doc-toc]>.h2{font-size:.875rem}.sidebar nav[role=doc-toc]>ul a{border-left:1px solid #ebebeb;padding-left:.6rem}.sidebar .toc-actions h2>ul a,.sidebar .toc-actions .h2>ul a,.sidebar .quarto-code-links h2>ul a,.sidebar .quarto-code-links .h2>ul a,.sidebar .quarto-other-links h2>ul a,.sidebar .quarto-other-links .h2>ul a,.sidebar .quarto-alternate-notebooks h2>ul a,.sidebar .quarto-alternate-notebooks .h2>ul a,.sidebar .quarto-alternate-formats h2>ul a,.sidebar .quarto-alternate-formats .h2>ul a{border-left:none;padding-left:.6rem}.sidebar .toc-actions ul a:empty,.sidebar .quarto-code-links ul a:empty,.sidebar .quarto-other-links ul a:empty,.sidebar .quarto-alternate-notebooks ul a:empty,.sidebar .quarto-alternate-formats ul a:empty,.sidebar nav[role=doc-toc]>ul a:empty{display:none}.sidebar .toc-actions ul,.sidebar .quarto-code-links ul,.sidebar .quarto-other-links ul,.sidebar .quarto-alternate-notebooks ul,.sidebar .quarto-alternate-formats ul{padding-left:0;list-style:none}.sidebar nav[role=doc-toc] ul{list-style:none;padding-left:0;list-style:none}.sidebar nav[role=doc-toc]>ul{margin-left:.45em}.quarto-margin-sidebar nav[role=doc-toc]{padding-left:.5em}.sidebar .toc-actions>ul,.sidebar .quarto-code-links>ul,.sidebar .quarto-other-links>ul,.sidebar .quarto-alternate-notebooks>ul,.sidebar .quarto-alternate-formats>ul{font-size:.8rem}.sidebar nav[role=doc-toc]>ul{font-size:.875rem}.sidebar .toc-actions ul li a,.sidebar .quarto-code-links ul li a,.sidebar .quarto-other-links ul li a,.sidebar .quarto-alternate-notebooks ul li a,.sidebar .quarto-alternate-formats ul li a,.sidebar nav[role=doc-toc]>ul li a{line-height:1.1rem;padding-bottom:.2rem;padding-top:.2rem;color:inherit}.sidebar nav[role=doc-toc] ul>li>ul>li>a{padding-left:1.2em}.sidebar nav[role=doc-toc] ul>li>ul>li>ul>li>a{padding-left:2.4em}.sidebar nav[role=doc-toc] ul>li>ul>li>ul>li>ul>li>a{padding-left:3.6em}.sidebar nav[role=doc-toc] ul>li>ul>li>ul>li>ul>li>ul>li>a{padding-left:4.8em}.sidebar nav[role=doc-toc] ul>li>ul>li>ul>li>ul>li>ul>li>ul>li>a{padding-left:6em}.sidebar nav[role=doc-toc] ul>li>a.active,.sidebar nav[role=doc-toc] ul>li>ul>li>a.active{border-left:1px solid #00bc8c;color:#00bc8c !important}.sidebar nav[role=doc-toc] ul>li>a:hover,.sidebar nav[role=doc-toc] ul>li>ul>li>a:hover{color:#00bc8c !important}kbd,.kbd{color:#fff;background-color:#f8f9fa;border:1px solid;border-radius:5px;border-color:#434343}.quarto-appendix-contents div.hanging-indent{margin-left:0em}.quarto-appendix-contents div.hanging-indent div.csl-entry{margin-left:1em;text-indent:-1em}.citation a,.footnote-ref{text-decoration:none}.footnotes ol{padding-left:1em}.tippy-content>*{margin-bottom:.7em}.tippy-content>*:last-child{margin-bottom:0}.callout{margin-top:1.25rem;margin-bottom:1.25rem;border-radius:.25rem;overflow-wrap:break-word}.callout .callout-title-container{overflow-wrap:anywhere}.callout.callout-style-simple{padding:.4em .7em;border-left:5px solid;border-right:1px solid #434343;border-top:1px solid #434343;border-bottom:1px solid #434343}.callout.callout-style-default{border-left:5px solid;border-right:1px solid #434343;border-top:1px solid #434343;border-bottom:1px solid #434343}.callout .callout-body-container{flex-grow:1}.callout.callout-style-simple .callout-body{font-size:.9rem;font-weight:400}.callout.callout-style-default .callout-body{font-size:.9rem;font-weight:400}.callout:not(.no-icon).callout-titled.callout-style-simple .callout-body{padding-left:1.6em}.callout.callout-titled>.callout-header{padding-top:.2em;margin-bottom:-0.2em}.callout.callout-style-simple>div.callout-header{border-bottom:none;font-size:.9rem;font-weight:600;opacity:75%}.callout.callout-style-default>div.callout-header{border-bottom:none;font-weight:600;opacity:85%;font-size:.9rem;padding-left:.5em;padding-right:.5em}.callout.callout-style-default .callout-body{padding-left:.5em;padding-right:.5em}.callout.callout-style-default .callout-body>:first-child{padding-top:.5rem;margin-top:0}.callout>div.callout-header[data-bs-toggle=collapse]{cursor:pointer}.callout.callout-style-default .callout-header[aria-expanded=false],.callout.callout-style-default .callout-header[aria-expanded=true]{padding-top:0px;margin-bottom:0px;align-items:center}.callout.callout-titled .callout-body>:last-child:not(.sourceCode),.callout.callout-titled .callout-body>div>:last-child:not(.sourceCode){padding-bottom:.5rem;margin-bottom:0}.callout:not(.callout-titled) .callout-body>:first-child,.callout:not(.callout-titled) .callout-body>div>:first-child{margin-top:.25rem}.callout:not(.callout-titled) .callout-body>:last-child,.callout:not(.callout-titled) .callout-body>div>:last-child{margin-bottom:.2rem}.callout.callout-style-simple .callout-icon::before,.callout.callout-style-simple .callout-toggle::before{height:1rem;width:1rem;display:inline-block;content:"";background-repeat:no-repeat;background-size:1rem 1rem}.callout.callout-style-default .callout-icon::before,.callout.callout-style-default .callout-toggle::before{height:.9rem;width:.9rem;display:inline-block;content:"";background-repeat:no-repeat;background-size:.9rem .9rem}.callout.callout-style-default .callout-toggle::before{margin-top:5px}.callout .callout-btn-toggle .callout-toggle::before{transition:transform .2s linear}.callout .callout-header[aria-expanded=false] .callout-toggle::before{transform:rotate(-90deg)}.callout .callout-header[aria-expanded=true] .callout-toggle::before{transform:none}.callout.callout-style-simple:not(.no-icon) div.callout-icon-container{padding-top:.2em;padding-right:.55em}.callout.callout-style-default:not(.no-icon) div.callout-icon-container{padding-top:.1em;padding-right:.35em}.callout.callout-style-default:not(.no-icon) div.callout-title-container{margin-top:-1px}.callout.callout-style-default.callout-caution:not(.no-icon) div.callout-icon-container{padding-top:.3em;padding-right:.35em}.callout>.callout-body>.callout-icon-container>.no-icon,.callout>.callout-header>.callout-icon-container>.no-icon{display:none}div.callout.callout{border-left-color:#6c757d}div.callout.callout-style-default>.callout-header{background-color:#6c757d}div.callout-note.callout{border-left-color:#375a7f}div.callout-note.callout-style-default>.callout-header{background-color:#111b26}div.callout-note:not(.callout-titled) .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-note.callout-titled .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-note .callout-toggle::before{background-image:url('data:image/svg+xml,')}div.callout-tip.callout{border-left-color:#00bc8c}div.callout-tip.callout-style-default>.callout-header{background-color:#00382a}div.callout-tip:not(.callout-titled) .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-tip.callout-titled .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-tip .callout-toggle::before{background-image:url('data:image/svg+xml,')}div.callout-warning.callout{border-left-color:#f39c12}div.callout-warning.callout-style-default>.callout-header{background-color:#492f05}div.callout-warning:not(.callout-titled) .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-warning.callout-titled .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-warning .callout-toggle::before{background-image:url('data:image/svg+xml,')}div.callout-caution.callout{border-left-color:#fd7e14}div.callout-caution.callout-style-default>.callout-header{background-color:#4c2606}div.callout-caution:not(.callout-titled) .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-caution.callout-titled .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-caution .callout-toggle::before{background-image:url('data:image/svg+xml,')}div.callout-important.callout{border-left-color:#e74c3c}div.callout-important.callout-style-default>.callout-header{background-color:#451712}div.callout-important:not(.callout-titled) .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-important.callout-titled .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-important .callout-toggle::before{background-image:url('data:image/svg+xml,')}.quarto-toggle-container{display:flex;align-items:center}.quarto-reader-toggle .bi::before,.quarto-color-scheme-toggle .bi::before{display:inline-block;height:1rem;width:1rem;content:"";background-repeat:no-repeat;background-size:1rem 1rem}.sidebar-navigation{padding-left:20px}.navbar{background-color:#375a7f;color:#dee2e6}.navbar .quarto-color-scheme-toggle:not(.alternate) .bi::before{background-image:url('data:image/svg+xml,')}.navbar .quarto-color-scheme-toggle.alternate .bi::before{background-image:url('data:image/svg+xml,')}.sidebar-navigation .quarto-color-scheme-toggle:not(.alternate) .bi::before{background-image:url('data:image/svg+xml,')}.sidebar-navigation .quarto-color-scheme-toggle.alternate .bi::before{background-image:url('data:image/svg+xml,')}.quarto-sidebar-toggle{border-color:#dee2e6;border-bottom-left-radius:.25rem;border-bottom-right-radius:.25rem;border-style:solid;border-width:1px;overflow:hidden;border-top-width:0px;padding-top:0px !important}.quarto-sidebar-toggle-title{cursor:pointer;padding-bottom:2px;margin-left:.25em;text-align:center;font-weight:400;font-size:.775em}#quarto-content .quarto-sidebar-toggle{background:#272727}#quarto-content .quarto-sidebar-toggle-title{color:#fff}.quarto-sidebar-toggle-icon{color:#dee2e6;margin-right:.5em;float:right;transition:transform .2s ease}.quarto-sidebar-toggle-icon::before{padding-top:5px}.quarto-sidebar-toggle.expanded .quarto-sidebar-toggle-icon{transform:rotate(-180deg)}.quarto-sidebar-toggle.expanded .quarto-sidebar-toggle-title{border-bottom:solid #dee2e6 1px}.quarto-sidebar-toggle-contents{background-color:#222;padding-right:10px;padding-left:10px;margin-top:0px !important;transition:max-height .5s ease}.quarto-sidebar-toggle.expanded .quarto-sidebar-toggle-contents{padding-top:1em;padding-bottom:10px}@media(max-width: 767.98px){.sidebar-menu-container{padding-bottom:5em}}.quarto-sidebar-toggle:not(.expanded) .quarto-sidebar-toggle-contents{padding-top:0px !important;padding-bottom:0px}nav[role=doc-toc]{z-index:1020}#quarto-sidebar>*,nav[role=doc-toc]>*{transition:opacity .1s ease,border .1s ease}#quarto-sidebar.slow>*,nav[role=doc-toc].slow>*{transition:opacity .4s ease,border .4s ease}.quarto-color-scheme-toggle:not(.alternate).top-right .bi::before{background-image:url('data:image/svg+xml,')}.quarto-color-scheme-toggle.alternate.top-right .bi::before{background-image:url('data:image/svg+xml,')}#quarto-appendix.default{border-top:1px solid #dee2e6}#quarto-appendix.default{background-color:#222;padding-top:1.5em;margin-top:2em;z-index:998}#quarto-appendix.default .quarto-appendix-heading{margin-top:0;line-height:1.4em;font-weight:600;opacity:.9;border-bottom:none;margin-bottom:0}#quarto-appendix.default .footnotes ol,#quarto-appendix.default .footnotes ol li>p:last-of-type,#quarto-appendix.default .quarto-appendix-contents>p:last-of-type{margin-bottom:0}#quarto-appendix.default .footnotes ol{margin-left:.5em}#quarto-appendix.default .quarto-appendix-secondary-label{margin-bottom:.4em}#quarto-appendix.default .quarto-appendix-bibtex{font-size:.7em;padding:1em;border:solid 1px #dee2e6;margin-bottom:1em}#quarto-appendix.default .quarto-appendix-bibtex code.sourceCode{white-space:pre-wrap}#quarto-appendix.default .quarto-appendix-citeas{font-size:.9em;padding:1em;border:solid 1px #dee2e6;margin-bottom:1em}#quarto-appendix.default .quarto-appendix-heading{font-size:1em !important}#quarto-appendix.default *[role=doc-endnotes]>ol,#quarto-appendix.default .quarto-appendix-contents>*:not(h2):not(.h2){font-size:.9em}#quarto-appendix.default section{padding-bottom:1.5em}#quarto-appendix.default section *[role=doc-endnotes],#quarto-appendix.default section>*:not(a){opacity:.9;word-wrap:break-word}.btn.btn-quarto,div.cell-output-display .btn-quarto{--bs-btn-color: #d9d9d9;--bs-btn-bg: #434343;--bs-btn-border-color: #434343;--bs-btn-hover-color: #d9d9d9;--bs-btn-hover-bg: #5f5f5f;--bs-btn-hover-border-color: #565656;--bs-btn-focus-shadow-rgb: 90, 90, 90;--bs-btn-active-color: #fff;--bs-btn-active-bg: dimgray;--bs-btn-active-border-color: #565656;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #434343;--bs-btn-disabled-border-color: #434343}nav.quarto-secondary-nav.color-navbar{background-color:#375a7f;color:#dee2e6}nav.quarto-secondary-nav.color-navbar h1,nav.quarto-secondary-nav.color-navbar .h1,nav.quarto-secondary-nav.color-navbar .quarto-btn-toggle{color:#dee2e6}@media(max-width: 991.98px){body.nav-sidebar .quarto-title-banner{margin-bottom:0;padding-bottom:1em}body.nav-sidebar #title-block-header{margin-block-end:0}}p.subtitle{margin-top:.25em;margin-bottom:.5em}code a:any-link{color:inherit;text-decoration-color:#6c757d}/*! dark */div.observablehq table thead tr th{background-color:var(--bs-body-bg)}input,button,select,optgroup,textarea{background-color:var(--bs-body-bg)}.code-annotated .code-copy-button{margin-right:1.25em;margin-top:0;padding-bottom:0;padding-top:3px}.code-annotation-gutter-bg{background-color:#222}.code-annotation-gutter{background-color:#000}.code-annotation-gutter,.code-annotation-gutter-bg{height:100%;width:calc(20px + .5em);position:absolute;top:0;right:0}dl.code-annotation-container-grid dt{margin-right:1em;margin-top:.25rem}dl.code-annotation-container-grid dt{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;color:#e6e6e6;border:solid #e6e6e6 1px;border-radius:50%;height:22px;width:22px;line-height:22px;font-size:11px;text-align:center;vertical-align:middle;text-decoration:none}dl.code-annotation-container-grid dt[data-target-cell]{cursor:pointer}dl.code-annotation-container-grid dt[data-target-cell].code-annotation-active{color:#222;border:solid #aaa 1px;background-color:#aaa}pre.code-annotation-code{padding-top:0;padding-bottom:0}pre.code-annotation-code code{z-index:3}#code-annotation-line-highlight-gutter{width:100%;border-top:solid rgba(170,170,170,.2666666667) 1px;border-bottom:solid rgba(170,170,170,.2666666667) 1px;z-index:2;background-color:rgba(170,170,170,.1333333333)}#code-annotation-line-highlight{margin-left:-4em;width:calc(100% + 4em);border-top:solid rgba(170,170,170,.2666666667) 1px;border-bottom:solid rgba(170,170,170,.2666666667) 1px;z-index:2;background-color:rgba(170,170,170,.1333333333)}code.sourceCode .code-annotation-anchor.code-annotation-active{background-color:var(--quarto-hl-normal-color, #aaaaaa);border:solid var(--quarto-hl-normal-color, #aaaaaa) 1px;color:#000;font-weight:bolder}code.sourceCode .code-annotation-anchor{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;color:var(--quarto-hl-co-color);border:solid var(--quarto-hl-co-color) 1px;border-radius:50%;height:18px;width:18px;font-size:9px;margin-top:2px}code.sourceCode button.code-annotation-anchor{padding:2px;user-select:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;-o-user-select:none}code.sourceCode a.code-annotation-anchor{line-height:18px;text-align:center;vertical-align:middle;cursor:default;text-decoration:none}@media print{.page-columns .column-screen-inset{grid-column:page-start-inset/page-end-inset;z-index:998;opacity:.999}.page-columns .column-screen-inset table{background:#222}.page-columns .column-screen-inset-left{grid-column:page-start-inset/body-content-end;z-index:998;opacity:.999}.page-columns .column-screen-inset-left table{background:#222}.page-columns .column-screen-inset-right{grid-column:body-content-start/page-end-inset;z-index:998;opacity:.999}.page-columns .column-screen-inset-right table{background:#222}.page-columns .column-screen{grid-column:page-start/page-end;z-index:998;opacity:.999}.page-columns .column-screen table{background:#222}.page-columns .column-screen-left{grid-column:page-start/body-content-end;z-index:998;opacity:.999}.page-columns .column-screen-left table{background:#222}.page-columns .column-screen-right{grid-column:body-content-start/page-end;z-index:998;opacity:.999}.page-columns .column-screen-right table{background:#222}.page-columns .column-screen-inset-shaded{grid-column:page-start-inset/page-end-inset;padding:1em;background:#6f6f6f;z-index:998;opacity:.999;margin-bottom:1em}}.quarto-video{margin-bottom:1em}.table{border-top:1px solid #fff;border-bottom:1px solid #fff}.table>thead{border-top-width:0;border-bottom:1px solid #fff}.table a{word-break:break-word}.table>:not(caption)>*>*{background-color:unset;color:unset}#quarto-document-content .crosstalk-input .checkbox input[type=checkbox],#quarto-document-content .crosstalk-input .checkbox-inline input[type=checkbox]{position:unset;margin-top:unset;margin-left:unset}#quarto-document-content .row{margin-left:unset;margin-right:unset}.quarto-xref{white-space:nowrap}#quarto-draft-alert{margin-top:0px;margin-bottom:0px;padding:.3em;text-align:center;font-size:.9em}#quarto-draft-alert i{margin-right:.3em}a.external:after{content:"";background-image:url('data:image/svg+xml,');background-size:contain;background-repeat:no-repeat;background-position:center center;margin-left:.2em;padding-right:.75em}div.sourceCode code a.external:after{content:none}a.external:after:hover{cursor:pointer}.quarto-ext-icon{display:inline-block;font-size:.75em;padding-left:.3em}.code-with-filename .code-with-filename-file{margin-bottom:0;padding-bottom:2px;padding-top:2px;padding-left:.7em;border:var(--quarto-border-width) solid var(--quarto-border-color);border-radius:var(--quarto-border-radius);border-bottom:0;border-bottom-left-radius:0%;border-bottom-right-radius:0%}.code-with-filename div.sourceCode,.reveal .code-with-filename div.sourceCode{margin-top:0;border-top-left-radius:0%;border-top-right-radius:0%}.code-with-filename .code-with-filename-file pre{margin-bottom:0}.code-with-filename .code-with-filename-file{background-color:rgba(219,219,219,.8)}.quarto-dark .code-with-filename .code-with-filename-file{background-color:#555}.code-with-filename .code-with-filename-file strong{font-weight:400}.blockquote-footer{color:#595959}.form-floating>label,.form-floating>.form-control:focus~label,.form-floating>.form-control:not(:placeholder-shown)~label{color:#595959}.nav-tabs .nav-link,.nav-tabs .nav-link.active,.nav-tabs .nav-link.active:focus,.nav-tabs .nav-link.active:hover,.nav-tabs .nav-item.open .nav-link,.nav-tabs .nav-item.open .nav-link:focus,.nav-tabs .nav-item.open .nav-link:hover,.nav-pills .nav-link,.nav-pills .nav-link.active,.nav-pills .nav-link.active:focus,.nav-pills .nav-link.active:hover,.nav-pills .nav-item.open .nav-link,.nav-pills .nav-item.open .nav-link:focus,.nav-pills .nav-item.open .nav-link:hover{color:#fff}.breadcrumb a{color:#fff}.pagination a:hover{text-decoration:none}.alert{color:#fff;border:none}.alert a,.alert .alert-link{color:#fff;text-decoration:underline}.alert-default{background-color:#434343}.alert-primary{background-color:#375a7f}.alert-secondary{background-color:#434343}.alert-success{background-color:#00bc8c}.alert-info{background-color:#3498db}.alert-warning{background-color:#f39c12}.alert-danger{background-color:#e74c3c}.alert-light{background-color:#6f6f6f}.alert-dark{background-color:#2d2d2d}.tooltip{--bs-tooltip-bg: var(--bs-tertiary-bg);--bs-tooltip-color: var(--bs-emphasis-color)}.quarto-title-banner{margin-bottom:1em;color:#dee2e6;background:#375a7f}.quarto-title-banner a{color:#dee2e6}.quarto-title-banner h1,.quarto-title-banner .h1,.quarto-title-banner h2,.quarto-title-banner .h2{color:#dee2e6}.quarto-title-banner .code-tools-button{color:#a4afba}.quarto-title-banner .code-tools-button:hover{color:#dee2e6}.quarto-title-banner .code-tools-button>.bi::before{background-image:url('data:image/svg+xml,')}.quarto-title-banner .code-tools-button:hover>.bi::before{background-image:url('data:image/svg+xml,')}.quarto-title-banner .quarto-title .title{font-weight:600}.quarto-title-banner .quarto-categories{margin-top:.75em}@media(min-width: 992px){.quarto-title-banner{padding-top:2.5em;padding-bottom:2.5em}}@media(max-width: 991.98px){.quarto-title-banner{padding-top:1em;padding-bottom:1em}}@media(max-width: 767.98px){body.hypothesis-enabled #title-block-header>*{padding-right:20px}}main.quarto-banner-title-block>section:first-child>h2,main.quarto-banner-title-block>section:first-child>.h2,main.quarto-banner-title-block>section:first-child>h3,main.quarto-banner-title-block>section:first-child>.h3,main.quarto-banner-title-block>section:first-child>h4,main.quarto-banner-title-block>section:first-child>.h4{margin-top:0}.quarto-title .quarto-categories{display:flex;flex-wrap:wrap;row-gap:.5em;column-gap:.4em;padding-bottom:.5em;margin-top:.75em}.quarto-title .quarto-categories .quarto-category{padding:.25em .75em;font-size:.65em;text-transform:uppercase;border:solid 1px;border-radius:.25rem;opacity:.6}.quarto-title .quarto-categories .quarto-category a{color:inherit}.quarto-title-meta-container{display:grid;grid-template-columns:1fr auto}.quarto-title-meta-column-end{display:flex;flex-direction:column;padding-left:1em}.quarto-title-meta-column-end a .bi{margin-right:.3em}#title-block-header.quarto-title-block.default .quarto-title-meta{display:grid;grid-template-columns:repeat(2, 1fr);grid-column-gap:1em}#title-block-header.quarto-title-block.default .quarto-title .title{margin-bottom:0}#title-block-header.quarto-title-block.default .quarto-title-author-orcid img{margin-top:-0.2em;height:.8em;width:.8em}#title-block-header.quarto-title-block.default .quarto-title-author-email{opacity:.7}#title-block-header.quarto-title-block.default .quarto-description p:last-of-type{margin-bottom:0}#title-block-header.quarto-title-block.default .quarto-title-meta-contents p,#title-block-header.quarto-title-block.default .quarto-title-authors p,#title-block-header.quarto-title-block.default .quarto-title-affiliations p{margin-bottom:.1em}#title-block-header.quarto-title-block.default .quarto-title-meta-heading{text-transform:uppercase;margin-top:1em;font-size:.8em;opacity:.8;font-weight:400}#title-block-header.quarto-title-block.default .quarto-title-meta-contents{font-size:.9em}#title-block-header.quarto-title-block.default .quarto-title-meta-contents p.affiliation:last-of-type{margin-bottom:.1em}#title-block-header.quarto-title-block.default p.affiliation{margin-bottom:.1em}#title-block-header.quarto-title-block.default .keywords,#title-block-header.quarto-title-block.default .description,#title-block-header.quarto-title-block.default .abstract{margin-top:0}#title-block-header.quarto-title-block.default .keywords>p,#title-block-header.quarto-title-block.default .description>p,#title-block-header.quarto-title-block.default .abstract>p{font-size:.9em}#title-block-header.quarto-title-block.default .keywords>p:last-of-type,#title-block-header.quarto-title-block.default .description>p:last-of-type,#title-block-header.quarto-title-block.default .abstract>p:last-of-type{margin-bottom:0}#title-block-header.quarto-title-block.default .keywords .block-title,#title-block-header.quarto-title-block.default .description .block-title,#title-block-header.quarto-title-block.default .abstract .block-title{margin-top:1em;text-transform:uppercase;font-size:.8em;opacity:.8;font-weight:400}#title-block-header.quarto-title-block.default .quarto-title-meta-author{display:grid;grid-template-columns:minmax(max-content, 1fr) 1fr;grid-column-gap:1em}.quarto-title-tools-only{display:flex;justify-content:right} diff --git a/site_libs/bootstrap/bootstrap-icons.css b/site_libs/bootstrap/bootstrap-icons.css new file mode 100644 index 0000000..285e444 --- /dev/null +++ b/site_libs/bootstrap/bootstrap-icons.css @@ -0,0 +1,2078 @@ +/*! + * Bootstrap Icons v1.11.1 (https://icons.getbootstrap.com/) + * Copyright 2019-2023 The Bootstrap Authors + * Licensed under MIT (https://github.com/twbs/icons/blob/main/LICENSE) + */ + +@font-face { + font-display: block; + font-family: "bootstrap-icons"; + src: +url("./bootstrap-icons.woff?2820a3852bdb9a5832199cc61cec4e65") format("woff"); +} + +.bi::before, +[class^="bi-"]::before, +[class*=" bi-"]::before { + display: inline-block; + font-family: bootstrap-icons !important; + font-style: normal; + font-weight: normal !important; + font-variant: normal; + text-transform: none; + line-height: 1; + vertical-align: -.125em; + -webkit-font-smoothing: antialiased; + -moz-osx-font-smoothing: grayscale; +} + +.bi-123::before { content: "\f67f"; } +.bi-alarm-fill::before { content: "\f101"; } +.bi-alarm::before { content: "\f102"; } +.bi-align-bottom::before { content: "\f103"; } +.bi-align-center::before { content: "\f104"; } +.bi-align-end::before { content: "\f105"; } +.bi-align-middle::before { content: "\f106"; } +.bi-align-start::before { content: "\f107"; } +.bi-align-top::before { content: "\f108"; } +.bi-alt::before { content: "\f109"; } +.bi-app-indicator::before { content: "\f10a"; } +.bi-app::before { content: "\f10b"; } +.bi-archive-fill::before { content: "\f10c"; } +.bi-archive::before { content: "\f10d"; } +.bi-arrow-90deg-down::before { content: "\f10e"; } +.bi-arrow-90deg-left::before { content: "\f10f"; } +.bi-arrow-90deg-right::before { content: "\f110"; } +.bi-arrow-90deg-up::before { content: "\f111"; } +.bi-arrow-bar-down::before { content: "\f112"; } +.bi-arrow-bar-left::before { content: "\f113"; } +.bi-arrow-bar-right::before { content: "\f114"; } +.bi-arrow-bar-up::before { content: "\f115"; } +.bi-arrow-clockwise::before { content: "\f116"; } +.bi-arrow-counterclockwise::before { content: "\f117"; } +.bi-arrow-down-circle-fill::before { content: "\f118"; } +.bi-arrow-down-circle::before { content: "\f119"; } +.bi-arrow-down-left-circle-fill::before { content: "\f11a"; } +.bi-arrow-down-left-circle::before { content: "\f11b"; } +.bi-arrow-down-left-square-fill::before { content: "\f11c"; } +.bi-arrow-down-left-square::before { content: "\f11d"; } +.bi-arrow-down-left::before { content: "\f11e"; } +.bi-arrow-down-right-circle-fill::before { content: "\f11f"; } +.bi-arrow-down-right-circle::before { content: "\f120"; } +.bi-arrow-down-right-square-fill::before { content: "\f121"; } +.bi-arrow-down-right-square::before { content: "\f122"; } +.bi-arrow-down-right::before { content: "\f123"; } +.bi-arrow-down-short::before { content: "\f124"; } +.bi-arrow-down-square-fill::before { content: "\f125"; } +.bi-arrow-down-square::before { content: "\f126"; } +.bi-arrow-down-up::before { content: "\f127"; } +.bi-arrow-down::before { content: "\f128"; } +.bi-arrow-left-circle-fill::before { content: "\f129"; } +.bi-arrow-left-circle::before { content: "\f12a"; } +.bi-arrow-left-right::before { content: "\f12b"; } +.bi-arrow-left-short::before { content: "\f12c"; } +.bi-arrow-left-square-fill::before { content: "\f12d"; } +.bi-arrow-left-square::before { content: "\f12e"; } +.bi-arrow-left::before { content: "\f12f"; } +.bi-arrow-repeat::before { content: "\f130"; } +.bi-arrow-return-left::before { content: "\f131"; } +.bi-arrow-return-right::before { content: "\f132"; } +.bi-arrow-right-circle-fill::before { content: "\f133"; } +.bi-arrow-right-circle::before { content: "\f134"; } +.bi-arrow-right-short::before { content: "\f135"; } +.bi-arrow-right-square-fill::before { content: "\f136"; } +.bi-arrow-right-square::before { content: "\f137"; } +.bi-arrow-right::before { content: "\f138"; } +.bi-arrow-up-circle-fill::before { content: "\f139"; } +.bi-arrow-up-circle::before { content: "\f13a"; } +.bi-arrow-up-left-circle-fill::before { content: "\f13b"; } +.bi-arrow-up-left-circle::before { content: "\f13c"; } +.bi-arrow-up-left-square-fill::before { content: "\f13d"; } +.bi-arrow-up-left-square::before { content: "\f13e"; } +.bi-arrow-up-left::before { content: "\f13f"; } +.bi-arrow-up-right-circle-fill::before { content: "\f140"; } +.bi-arrow-up-right-circle::before { content: "\f141"; } +.bi-arrow-up-right-square-fill::before { content: "\f142"; } +.bi-arrow-up-right-square::before { content: "\f143"; } +.bi-arrow-up-right::before { content: "\f144"; } +.bi-arrow-up-short::before { content: "\f145"; } +.bi-arrow-up-square-fill::before { content: "\f146"; } +.bi-arrow-up-square::before { content: "\f147"; } +.bi-arrow-up::before { content: "\f148"; } +.bi-arrows-angle-contract::before { content: "\f149"; } +.bi-arrows-angle-expand::before { content: "\f14a"; } +.bi-arrows-collapse::before { content: "\f14b"; } +.bi-arrows-expand::before { content: "\f14c"; } +.bi-arrows-fullscreen::before { content: "\f14d"; } +.bi-arrows-move::before { content: "\f14e"; } +.bi-aspect-ratio-fill::before { content: "\f14f"; } +.bi-aspect-ratio::before { content: "\f150"; } +.bi-asterisk::before { content: "\f151"; } +.bi-at::before { content: "\f152"; } +.bi-award-fill::before { content: "\f153"; } +.bi-award::before { content: "\f154"; } +.bi-back::before { content: "\f155"; } +.bi-backspace-fill::before { content: "\f156"; } +.bi-backspace-reverse-fill::before { content: "\f157"; } +.bi-backspace-reverse::before { content: "\f158"; } +.bi-backspace::before { content: "\f159"; } +.bi-badge-3d-fill::before { content: "\f15a"; } +.bi-badge-3d::before { content: "\f15b"; } +.bi-badge-4k-fill::before { content: "\f15c"; } +.bi-badge-4k::before { content: "\f15d"; } +.bi-badge-8k-fill::before { content: "\f15e"; } +.bi-badge-8k::before { content: "\f15f"; } +.bi-badge-ad-fill::before { content: "\f160"; } +.bi-badge-ad::before { content: "\f161"; } +.bi-badge-ar-fill::before { content: "\f162"; } +.bi-badge-ar::before { content: "\f163"; } +.bi-badge-cc-fill::before { content: "\f164"; } +.bi-badge-cc::before { content: "\f165"; } +.bi-badge-hd-fill::before { content: "\f166"; } +.bi-badge-hd::before { content: "\f167"; } +.bi-badge-tm-fill::before { content: "\f168"; } +.bi-badge-tm::before { content: "\f169"; } +.bi-badge-vo-fill::before { content: "\f16a"; } +.bi-badge-vo::before { content: "\f16b"; } +.bi-badge-vr-fill::before { content: "\f16c"; } +.bi-badge-vr::before { content: "\f16d"; } +.bi-badge-wc-fill::before { content: "\f16e"; } +.bi-badge-wc::before { content: "\f16f"; } +.bi-bag-check-fill::before { content: "\f170"; } +.bi-bag-check::before { content: "\f171"; } +.bi-bag-dash-fill::before { content: "\f172"; } +.bi-bag-dash::before { content: "\f173"; } +.bi-bag-fill::before { content: "\f174"; } +.bi-bag-plus-fill::before { content: "\f175"; } +.bi-bag-plus::before { content: "\f176"; } +.bi-bag-x-fill::before { content: "\f177"; } +.bi-bag-x::before { content: "\f178"; } +.bi-bag::before { content: "\f179"; } +.bi-bar-chart-fill::before { content: "\f17a"; } +.bi-bar-chart-line-fill::before { content: "\f17b"; } +.bi-bar-chart-line::before { content: "\f17c"; } +.bi-bar-chart-steps::before { content: "\f17d"; } +.bi-bar-chart::before { content: "\f17e"; } +.bi-basket-fill::before { content: "\f17f"; } +.bi-basket::before { content: "\f180"; } +.bi-basket2-fill::before { content: "\f181"; } +.bi-basket2::before { content: "\f182"; } +.bi-basket3-fill::before { content: "\f183"; } +.bi-basket3::before { content: "\f184"; } +.bi-battery-charging::before { content: "\f185"; } +.bi-battery-full::before { content: "\f186"; } +.bi-battery-half::before { content: "\f187"; } +.bi-battery::before { content: "\f188"; } +.bi-bell-fill::before { content: "\f189"; } +.bi-bell::before { content: "\f18a"; } +.bi-bezier::before { content: "\f18b"; } +.bi-bezier2::before { content: "\f18c"; } +.bi-bicycle::before { content: "\f18d"; } +.bi-binoculars-fill::before { content: "\f18e"; } +.bi-binoculars::before { content: "\f18f"; } +.bi-blockquote-left::before { content: "\f190"; } +.bi-blockquote-right::before { content: "\f191"; } +.bi-book-fill::before { content: "\f192"; } +.bi-book-half::before { content: "\f193"; } +.bi-book::before { content: "\f194"; } +.bi-bookmark-check-fill::before { content: "\f195"; } +.bi-bookmark-check::before { content: "\f196"; } +.bi-bookmark-dash-fill::before { content: "\f197"; } +.bi-bookmark-dash::before { content: "\f198"; } +.bi-bookmark-fill::before { content: "\f199"; } +.bi-bookmark-heart-fill::before { content: "\f19a"; } +.bi-bookmark-heart::before { content: "\f19b"; } +.bi-bookmark-plus-fill::before { content: "\f19c"; } +.bi-bookmark-plus::before { content: "\f19d"; } +.bi-bookmark-star-fill::before { content: "\f19e"; } +.bi-bookmark-star::before { content: "\f19f"; } +.bi-bookmark-x-fill::before { content: "\f1a0"; } +.bi-bookmark-x::before { content: "\f1a1"; } +.bi-bookmark::before { content: "\f1a2"; } +.bi-bookmarks-fill::before { content: "\f1a3"; } +.bi-bookmarks::before { content: "\f1a4"; } +.bi-bookshelf::before { content: "\f1a5"; } +.bi-bootstrap-fill::before { content: "\f1a6"; } +.bi-bootstrap-reboot::before { content: "\f1a7"; } +.bi-bootstrap::before { content: "\f1a8"; } +.bi-border-all::before { content: "\f1a9"; } +.bi-border-bottom::before { content: "\f1aa"; } +.bi-border-center::before { content: "\f1ab"; } +.bi-border-inner::before { content: "\f1ac"; } +.bi-border-left::before { content: "\f1ad"; } +.bi-border-middle::before { content: "\f1ae"; } +.bi-border-outer::before { content: "\f1af"; } +.bi-border-right::before { content: "\f1b0"; } +.bi-border-style::before { content: "\f1b1"; } +.bi-border-top::before { content: "\f1b2"; } +.bi-border-width::before { content: "\f1b3"; } +.bi-border::before { content: "\f1b4"; } +.bi-bounding-box-circles::before { content: "\f1b5"; } +.bi-bounding-box::before { content: "\f1b6"; } +.bi-box-arrow-down-left::before { content: "\f1b7"; } +.bi-box-arrow-down-right::before { content: "\f1b8"; } +.bi-box-arrow-down::before { content: "\f1b9"; } +.bi-box-arrow-in-down-left::before { content: "\f1ba"; } +.bi-box-arrow-in-down-right::before { content: "\f1bb"; } +.bi-box-arrow-in-down::before { content: "\f1bc"; } +.bi-box-arrow-in-left::before { content: "\f1bd"; } +.bi-box-arrow-in-right::before { content: "\f1be"; } +.bi-box-arrow-in-up-left::before { content: "\f1bf"; } +.bi-box-arrow-in-up-right::before { content: "\f1c0"; } +.bi-box-arrow-in-up::before { content: "\f1c1"; } +.bi-box-arrow-left::before { content: "\f1c2"; } +.bi-box-arrow-right::before { content: "\f1c3"; } +.bi-box-arrow-up-left::before { content: "\f1c4"; } +.bi-box-arrow-up-right::before { content: "\f1c5"; } +.bi-box-arrow-up::before { content: "\f1c6"; } +.bi-box-seam::before { content: "\f1c7"; } +.bi-box::before { content: "\f1c8"; } +.bi-braces::before { content: "\f1c9"; } +.bi-bricks::before { content: "\f1ca"; } +.bi-briefcase-fill::before { content: "\f1cb"; } +.bi-briefcase::before { content: "\f1cc"; } +.bi-brightness-alt-high-fill::before { content: "\f1cd"; } +.bi-brightness-alt-high::before { content: "\f1ce"; } +.bi-brightness-alt-low-fill::before { content: "\f1cf"; } +.bi-brightness-alt-low::before { content: "\f1d0"; } +.bi-brightness-high-fill::before { content: "\f1d1"; } +.bi-brightness-high::before { content: "\f1d2"; } +.bi-brightness-low-fill::before { content: "\f1d3"; } +.bi-brightness-low::before { content: "\f1d4"; } +.bi-broadcast-pin::before { content: "\f1d5"; } +.bi-broadcast::before { content: "\f1d6"; } +.bi-brush-fill::before { content: "\f1d7"; } +.bi-brush::before { content: "\f1d8"; } +.bi-bucket-fill::before { content: "\f1d9"; } +.bi-bucket::before { content: "\f1da"; } +.bi-bug-fill::before { content: "\f1db"; } +.bi-bug::before { content: "\f1dc"; } +.bi-building::before { content: "\f1dd"; } +.bi-bullseye::before { content: "\f1de"; } +.bi-calculator-fill::before { content: "\f1df"; } +.bi-calculator::before { content: "\f1e0"; } +.bi-calendar-check-fill::before { content: "\f1e1"; } +.bi-calendar-check::before { content: "\f1e2"; } +.bi-calendar-date-fill::before { content: "\f1e3"; } +.bi-calendar-date::before { content: "\f1e4"; } +.bi-calendar-day-fill::before { content: "\f1e5"; } +.bi-calendar-day::before { content: "\f1e6"; } +.bi-calendar-event-fill::before { content: "\f1e7"; } +.bi-calendar-event::before { content: "\f1e8"; } +.bi-calendar-fill::before { content: "\f1e9"; } +.bi-calendar-minus-fill::before { content: "\f1ea"; } +.bi-calendar-minus::before { content: "\f1eb"; } +.bi-calendar-month-fill::before { content: "\f1ec"; } +.bi-calendar-month::before { content: "\f1ed"; } +.bi-calendar-plus-fill::before { content: "\f1ee"; } +.bi-calendar-plus::before { content: "\f1ef"; } +.bi-calendar-range-fill::before { content: "\f1f0"; } +.bi-calendar-range::before { content: "\f1f1"; } +.bi-calendar-week-fill::before { content: "\f1f2"; } +.bi-calendar-week::before { content: "\f1f3"; } +.bi-calendar-x-fill::before { content: "\f1f4"; } +.bi-calendar-x::before { content: "\f1f5"; } +.bi-calendar::before { content: "\f1f6"; } +.bi-calendar2-check-fill::before { content: "\f1f7"; } +.bi-calendar2-check::before { content: "\f1f8"; } +.bi-calendar2-date-fill::before { content: "\f1f9"; } +.bi-calendar2-date::before { content: "\f1fa"; } +.bi-calendar2-day-fill::before { content: "\f1fb"; } +.bi-calendar2-day::before { content: "\f1fc"; } +.bi-calendar2-event-fill::before { content: "\f1fd"; } +.bi-calendar2-event::before { content: "\f1fe"; } +.bi-calendar2-fill::before { content: "\f1ff"; } +.bi-calendar2-minus-fill::before { content: "\f200"; } +.bi-calendar2-minus::before { content: "\f201"; } +.bi-calendar2-month-fill::before { content: "\f202"; } +.bi-calendar2-month::before { content: "\f203"; } +.bi-calendar2-plus-fill::before { content: "\f204"; } +.bi-calendar2-plus::before { content: "\f205"; } +.bi-calendar2-range-fill::before { content: "\f206"; } +.bi-calendar2-range::before { content: "\f207"; } +.bi-calendar2-week-fill::before { content: "\f208"; } +.bi-calendar2-week::before { content: "\f209"; } +.bi-calendar2-x-fill::before { content: "\f20a"; } +.bi-calendar2-x::before { content: "\f20b"; } +.bi-calendar2::before { content: "\f20c"; } +.bi-calendar3-event-fill::before { content: "\f20d"; } +.bi-calendar3-event::before { content: "\f20e"; } +.bi-calendar3-fill::before { content: "\f20f"; } +.bi-calendar3-range-fill::before { content: "\f210"; } +.bi-calendar3-range::before { content: "\f211"; } +.bi-calendar3-week-fill::before { content: "\f212"; } +.bi-calendar3-week::before { content: "\f213"; } +.bi-calendar3::before { content: "\f214"; } +.bi-calendar4-event::before { content: "\f215"; } +.bi-calendar4-range::before { content: "\f216"; } +.bi-calendar4-week::before { content: "\f217"; } +.bi-calendar4::before { content: "\f218"; } +.bi-camera-fill::before { content: "\f219"; } +.bi-camera-reels-fill::before { content: "\f21a"; } +.bi-camera-reels::before { content: "\f21b"; } +.bi-camera-video-fill::before { content: "\f21c"; } +.bi-camera-video-off-fill::before { content: "\f21d"; } +.bi-camera-video-off::before { content: "\f21e"; } +.bi-camera-video::before { content: "\f21f"; } +.bi-camera::before { content: "\f220"; } +.bi-camera2::before { content: "\f221"; } +.bi-capslock-fill::before { content: "\f222"; } +.bi-capslock::before { content: "\f223"; } +.bi-card-checklist::before { content: "\f224"; } +.bi-card-heading::before { content: "\f225"; } +.bi-card-image::before { content: "\f226"; } +.bi-card-list::before { content: "\f227"; } +.bi-card-text::before { content: "\f228"; } +.bi-caret-down-fill::before { content: "\f229"; } +.bi-caret-down-square-fill::before { content: "\f22a"; } +.bi-caret-down-square::before { content: "\f22b"; } +.bi-caret-down::before { content: "\f22c"; } +.bi-caret-left-fill::before { content: "\f22d"; } +.bi-caret-left-square-fill::before { content: "\f22e"; } +.bi-caret-left-square::before { content: "\f22f"; } +.bi-caret-left::before { content: "\f230"; } +.bi-caret-right-fill::before { content: "\f231"; } +.bi-caret-right-square-fill::before { content: "\f232"; } +.bi-caret-right-square::before { content: "\f233"; } +.bi-caret-right::before { content: "\f234"; } +.bi-caret-up-fill::before { content: "\f235"; } +.bi-caret-up-square-fill::before { content: "\f236"; } +.bi-caret-up-square::before { content: "\f237"; } +.bi-caret-up::before { content: "\f238"; } +.bi-cart-check-fill::before { content: "\f239"; } +.bi-cart-check::before { content: "\f23a"; } +.bi-cart-dash-fill::before { content: "\f23b"; } +.bi-cart-dash::before { content: "\f23c"; } +.bi-cart-fill::before { content: "\f23d"; } +.bi-cart-plus-fill::before { content: "\f23e"; } +.bi-cart-plus::before { content: "\f23f"; } +.bi-cart-x-fill::before { content: "\f240"; } +.bi-cart-x::before { content: "\f241"; } +.bi-cart::before { content: "\f242"; } +.bi-cart2::before { content: "\f243"; } +.bi-cart3::before { content: "\f244"; } +.bi-cart4::before { content: "\f245"; } +.bi-cash-stack::before { content: "\f246"; } +.bi-cash::before { content: "\f247"; } +.bi-cast::before { content: "\f248"; } +.bi-chat-dots-fill::before { content: "\f249"; } +.bi-chat-dots::before { content: "\f24a"; } +.bi-chat-fill::before { content: "\f24b"; } +.bi-chat-left-dots-fill::before { content: "\f24c"; } +.bi-chat-left-dots::before { content: "\f24d"; } +.bi-chat-left-fill::before { content: "\f24e"; } +.bi-chat-left-quote-fill::before { content: "\f24f"; } +.bi-chat-left-quote::before { content: "\f250"; } +.bi-chat-left-text-fill::before { content: "\f251"; } +.bi-chat-left-text::before { content: "\f252"; } +.bi-chat-left::before { content: "\f253"; } +.bi-chat-quote-fill::before { content: "\f254"; } +.bi-chat-quote::before { content: "\f255"; } +.bi-chat-right-dots-fill::before { content: "\f256"; } +.bi-chat-right-dots::before { content: "\f257"; } +.bi-chat-right-fill::before { content: "\f258"; } +.bi-chat-right-quote-fill::before { content: "\f259"; } +.bi-chat-right-quote::before { content: "\f25a"; } +.bi-chat-right-text-fill::before { content: "\f25b"; } +.bi-chat-right-text::before { content: "\f25c"; } +.bi-chat-right::before { content: "\f25d"; } +.bi-chat-square-dots-fill::before { content: "\f25e"; } +.bi-chat-square-dots::before { content: "\f25f"; } +.bi-chat-square-fill::before { content: "\f260"; } +.bi-chat-square-quote-fill::before { content: "\f261"; } +.bi-chat-square-quote::before { content: "\f262"; } +.bi-chat-square-text-fill::before { content: "\f263"; } +.bi-chat-square-text::before { content: "\f264"; } +.bi-chat-square::before { content: "\f265"; } +.bi-chat-text-fill::before { content: "\f266"; } +.bi-chat-text::before { content: "\f267"; } +.bi-chat::before { content: "\f268"; } +.bi-check-all::before { content: "\f269"; } +.bi-check-circle-fill::before { content: "\f26a"; } +.bi-check-circle::before { content: "\f26b"; } +.bi-check-square-fill::before { content: "\f26c"; } +.bi-check-square::before { content: "\f26d"; } +.bi-check::before { content: "\f26e"; } +.bi-check2-all::before { content: "\f26f"; } +.bi-check2-circle::before { content: "\f270"; } +.bi-check2-square::before { content: "\f271"; } +.bi-check2::before { content: "\f272"; } +.bi-chevron-bar-contract::before { content: "\f273"; } +.bi-chevron-bar-down::before { content: "\f274"; } +.bi-chevron-bar-expand::before { content: "\f275"; } +.bi-chevron-bar-left::before { content: "\f276"; } +.bi-chevron-bar-right::before { content: "\f277"; } +.bi-chevron-bar-up::before { content: "\f278"; } +.bi-chevron-compact-down::before { content: "\f279"; } +.bi-chevron-compact-left::before { content: "\f27a"; } +.bi-chevron-compact-right::before { content: "\f27b"; } +.bi-chevron-compact-up::before { content: "\f27c"; } +.bi-chevron-contract::before { content: "\f27d"; } +.bi-chevron-double-down::before { content: "\f27e"; } +.bi-chevron-double-left::before { content: "\f27f"; } +.bi-chevron-double-right::before { content: "\f280"; } +.bi-chevron-double-up::before { content: "\f281"; } +.bi-chevron-down::before { content: "\f282"; } +.bi-chevron-expand::before { content: "\f283"; } +.bi-chevron-left::before { content: "\f284"; } +.bi-chevron-right::before { content: "\f285"; } +.bi-chevron-up::before { content: "\f286"; } +.bi-circle-fill::before { content: "\f287"; } +.bi-circle-half::before { content: "\f288"; } +.bi-circle-square::before { content: "\f289"; } +.bi-circle::before { content: "\f28a"; } +.bi-clipboard-check::before { content: "\f28b"; } +.bi-clipboard-data::before { content: "\f28c"; } +.bi-clipboard-minus::before { content: "\f28d"; } +.bi-clipboard-plus::before { content: "\f28e"; } +.bi-clipboard-x::before { content: "\f28f"; } +.bi-clipboard::before { content: "\f290"; } +.bi-clock-fill::before { content: "\f291"; } +.bi-clock-history::before { content: "\f292"; } +.bi-clock::before { content: "\f293"; } +.bi-cloud-arrow-down-fill::before { content: "\f294"; } +.bi-cloud-arrow-down::before { content: "\f295"; } +.bi-cloud-arrow-up-fill::before { content: "\f296"; } +.bi-cloud-arrow-up::before { content: "\f297"; } +.bi-cloud-check-fill::before { content: "\f298"; } +.bi-cloud-check::before { content: "\f299"; } +.bi-cloud-download-fill::before { content: "\f29a"; } +.bi-cloud-download::before { content: "\f29b"; } +.bi-cloud-drizzle-fill::before { content: "\f29c"; } +.bi-cloud-drizzle::before { content: "\f29d"; } +.bi-cloud-fill::before { content: "\f29e"; } +.bi-cloud-fog-fill::before { content: "\f29f"; } +.bi-cloud-fog::before { content: "\f2a0"; } +.bi-cloud-fog2-fill::before { content: "\f2a1"; } +.bi-cloud-fog2::before { content: "\f2a2"; } +.bi-cloud-hail-fill::before { content: "\f2a3"; } +.bi-cloud-hail::before { content: "\f2a4"; } +.bi-cloud-haze-fill::before { content: "\f2a6"; } +.bi-cloud-haze::before { content: "\f2a7"; } +.bi-cloud-haze2-fill::before { content: "\f2a8"; } +.bi-cloud-lightning-fill::before { content: "\f2a9"; } +.bi-cloud-lightning-rain-fill::before { content: "\f2aa"; } +.bi-cloud-lightning-rain::before { content: "\f2ab"; } +.bi-cloud-lightning::before { content: "\f2ac"; } +.bi-cloud-minus-fill::before { content: "\f2ad"; } +.bi-cloud-minus::before { content: "\f2ae"; } +.bi-cloud-moon-fill::before { content: "\f2af"; } +.bi-cloud-moon::before { content: "\f2b0"; } +.bi-cloud-plus-fill::before { content: "\f2b1"; } +.bi-cloud-plus::before { content: "\f2b2"; } +.bi-cloud-rain-fill::before { content: "\f2b3"; } +.bi-cloud-rain-heavy-fill::before { content: "\f2b4"; } +.bi-cloud-rain-heavy::before { content: "\f2b5"; } +.bi-cloud-rain::before { content: "\f2b6"; } +.bi-cloud-slash-fill::before { content: "\f2b7"; } +.bi-cloud-slash::before { content: "\f2b8"; } +.bi-cloud-sleet-fill::before { content: "\f2b9"; } +.bi-cloud-sleet::before { content: "\f2ba"; } +.bi-cloud-snow-fill::before { content: "\f2bb"; } +.bi-cloud-snow::before { content: "\f2bc"; } +.bi-cloud-sun-fill::before { content: "\f2bd"; } +.bi-cloud-sun::before { content: "\f2be"; } +.bi-cloud-upload-fill::before { content: "\f2bf"; } +.bi-cloud-upload::before { content: "\f2c0"; } +.bi-cloud::before { content: "\f2c1"; } +.bi-clouds-fill::before { content: "\f2c2"; } +.bi-clouds::before { content: "\f2c3"; } +.bi-cloudy-fill::before { content: "\f2c4"; } +.bi-cloudy::before { content: "\f2c5"; } +.bi-code-slash::before { content: "\f2c6"; } +.bi-code-square::before { content: "\f2c7"; } +.bi-code::before { content: "\f2c8"; } +.bi-collection-fill::before { content: "\f2c9"; } +.bi-collection-play-fill::before { content: "\f2ca"; } +.bi-collection-play::before { content: "\f2cb"; } +.bi-collection::before { content: "\f2cc"; } +.bi-columns-gap::before { content: "\f2cd"; } +.bi-columns::before { content: "\f2ce"; } +.bi-command::before { content: "\f2cf"; } +.bi-compass-fill::before { content: "\f2d0"; } +.bi-compass::before { content: "\f2d1"; } +.bi-cone-striped::before { content: "\f2d2"; } +.bi-cone::before { content: "\f2d3"; } +.bi-controller::before { content: "\f2d4"; } +.bi-cpu-fill::before { content: "\f2d5"; } +.bi-cpu::before { content: "\f2d6"; } +.bi-credit-card-2-back-fill::before { content: "\f2d7"; } +.bi-credit-card-2-back::before { content: "\f2d8"; } +.bi-credit-card-2-front-fill::before { content: "\f2d9"; } +.bi-credit-card-2-front::before { content: "\f2da"; } +.bi-credit-card-fill::before { content: "\f2db"; } +.bi-credit-card::before { content: "\f2dc"; } +.bi-crop::before { content: "\f2dd"; } +.bi-cup-fill::before { content: "\f2de"; } +.bi-cup-straw::before { content: "\f2df"; } +.bi-cup::before { content: "\f2e0"; } +.bi-cursor-fill::before { content: "\f2e1"; } +.bi-cursor-text::before { content: "\f2e2"; } +.bi-cursor::before { content: "\f2e3"; } +.bi-dash-circle-dotted::before { content: "\f2e4"; } +.bi-dash-circle-fill::before { content: "\f2e5"; } +.bi-dash-circle::before { content: "\f2e6"; } +.bi-dash-square-dotted::before { content: "\f2e7"; } +.bi-dash-square-fill::before { content: "\f2e8"; } +.bi-dash-square::before { content: "\f2e9"; } +.bi-dash::before { content: "\f2ea"; } +.bi-diagram-2-fill::before { content: "\f2eb"; } +.bi-diagram-2::before { content: "\f2ec"; } +.bi-diagram-3-fill::before { content: "\f2ed"; } +.bi-diagram-3::before { content: "\f2ee"; } +.bi-diamond-fill::before { content: "\f2ef"; } +.bi-diamond-half::before { content: "\f2f0"; } +.bi-diamond::before { content: "\f2f1"; } +.bi-dice-1-fill::before { content: "\f2f2"; } +.bi-dice-1::before { content: "\f2f3"; } +.bi-dice-2-fill::before { content: "\f2f4"; } +.bi-dice-2::before { content: "\f2f5"; } +.bi-dice-3-fill::before { content: "\f2f6"; } +.bi-dice-3::before { content: "\f2f7"; } +.bi-dice-4-fill::before { content: "\f2f8"; } +.bi-dice-4::before { content: "\f2f9"; } +.bi-dice-5-fill::before { content: "\f2fa"; } +.bi-dice-5::before { content: "\f2fb"; } +.bi-dice-6-fill::before { content: "\f2fc"; } +.bi-dice-6::before { content: "\f2fd"; } +.bi-disc-fill::before { content: "\f2fe"; } +.bi-disc::before { content: "\f2ff"; } +.bi-discord::before { content: "\f300"; } +.bi-display-fill::before { content: "\f301"; } +.bi-display::before { content: "\f302"; } +.bi-distribute-horizontal::before { content: "\f303"; } +.bi-distribute-vertical::before { content: "\f304"; } +.bi-door-closed-fill::before { content: "\f305"; } +.bi-door-closed::before { content: "\f306"; } +.bi-door-open-fill::before { content: "\f307"; } +.bi-door-open::before { content: "\f308"; } +.bi-dot::before { content: "\f309"; } +.bi-download::before { content: "\f30a"; } +.bi-droplet-fill::before { content: "\f30b"; } +.bi-droplet-half::before { content: "\f30c"; } +.bi-droplet::before { content: "\f30d"; } +.bi-earbuds::before { content: "\f30e"; } +.bi-easel-fill::before { content: "\f30f"; } +.bi-easel::before { content: "\f310"; } +.bi-egg-fill::before { content: "\f311"; } +.bi-egg-fried::before { content: "\f312"; } +.bi-egg::before { content: "\f313"; } +.bi-eject-fill::before { content: "\f314"; } +.bi-eject::before { content: "\f315"; } +.bi-emoji-angry-fill::before { content: "\f316"; } +.bi-emoji-angry::before { content: "\f317"; } +.bi-emoji-dizzy-fill::before { content: "\f318"; } +.bi-emoji-dizzy::before { content: "\f319"; } +.bi-emoji-expressionless-fill::before { content: "\f31a"; } +.bi-emoji-expressionless::before { content: "\f31b"; } +.bi-emoji-frown-fill::before { content: "\f31c"; } +.bi-emoji-frown::before { content: "\f31d"; } +.bi-emoji-heart-eyes-fill::before { content: "\f31e"; } +.bi-emoji-heart-eyes::before { content: "\f31f"; } +.bi-emoji-laughing-fill::before { content: "\f320"; } +.bi-emoji-laughing::before { content: "\f321"; } +.bi-emoji-neutral-fill::before { content: "\f322"; } +.bi-emoji-neutral::before { content: "\f323"; } +.bi-emoji-smile-fill::before { content: "\f324"; } +.bi-emoji-smile-upside-down-fill::before { content: "\f325"; } +.bi-emoji-smile-upside-down::before { content: "\f326"; } +.bi-emoji-smile::before { content: "\f327"; } +.bi-emoji-sunglasses-fill::before { content: "\f328"; } +.bi-emoji-sunglasses::before { content: "\f329"; } +.bi-emoji-wink-fill::before { content: "\f32a"; } +.bi-emoji-wink::before { content: "\f32b"; } +.bi-envelope-fill::before { content: "\f32c"; } +.bi-envelope-open-fill::before { content: "\f32d"; } +.bi-envelope-open::before { content: "\f32e"; } +.bi-envelope::before { content: "\f32f"; } +.bi-eraser-fill::before { content: "\f330"; } +.bi-eraser::before { content: "\f331"; } +.bi-exclamation-circle-fill::before { content: "\f332"; } +.bi-exclamation-circle::before { content: "\f333"; } +.bi-exclamation-diamond-fill::before { content: "\f334"; } +.bi-exclamation-diamond::before { content: "\f335"; } +.bi-exclamation-octagon-fill::before { content: "\f336"; } +.bi-exclamation-octagon::before { content: "\f337"; } +.bi-exclamation-square-fill::before { content: "\f338"; } +.bi-exclamation-square::before { content: "\f339"; } +.bi-exclamation-triangle-fill::before { content: "\f33a"; } +.bi-exclamation-triangle::before { content: "\f33b"; } +.bi-exclamation::before { content: "\f33c"; } +.bi-exclude::before { content: "\f33d"; } +.bi-eye-fill::before { content: "\f33e"; } +.bi-eye-slash-fill::before { content: "\f33f"; } +.bi-eye-slash::before { content: "\f340"; } +.bi-eye::before { content: "\f341"; } +.bi-eyedropper::before { content: "\f342"; } +.bi-eyeglasses::before { content: "\f343"; } +.bi-facebook::before { content: "\f344"; } +.bi-file-arrow-down-fill::before { content: "\f345"; } +.bi-file-arrow-down::before { content: "\f346"; } +.bi-file-arrow-up-fill::before { content: "\f347"; } +.bi-file-arrow-up::before { content: "\f348"; } +.bi-file-bar-graph-fill::before { content: "\f349"; } +.bi-file-bar-graph::before { content: "\f34a"; } +.bi-file-binary-fill::before { content: "\f34b"; } +.bi-file-binary::before { content: "\f34c"; } +.bi-file-break-fill::before { content: "\f34d"; } +.bi-file-break::before { content: "\f34e"; } +.bi-file-check-fill::before { content: "\f34f"; } +.bi-file-check::before { content: "\f350"; } +.bi-file-code-fill::before { content: "\f351"; } +.bi-file-code::before { content: "\f352"; } +.bi-file-diff-fill::before { content: "\f353"; } +.bi-file-diff::before { content: "\f354"; } +.bi-file-earmark-arrow-down-fill::before { content: "\f355"; } +.bi-file-earmark-arrow-down::before { content: "\f356"; } +.bi-file-earmark-arrow-up-fill::before { content: "\f357"; } +.bi-file-earmark-arrow-up::before { content: "\f358"; } +.bi-file-earmark-bar-graph-fill::before { content: "\f359"; } +.bi-file-earmark-bar-graph::before { content: "\f35a"; } +.bi-file-earmark-binary-fill::before { content: "\f35b"; } +.bi-file-earmark-binary::before { content: "\f35c"; } +.bi-file-earmark-break-fill::before { content: "\f35d"; } +.bi-file-earmark-break::before { content: "\f35e"; } +.bi-file-earmark-check-fill::before { content: "\f35f"; } +.bi-file-earmark-check::before { content: "\f360"; } +.bi-file-earmark-code-fill::before { content: "\f361"; } +.bi-file-earmark-code::before { content: "\f362"; } +.bi-file-earmark-diff-fill::before { content: "\f363"; } +.bi-file-earmark-diff::before { content: "\f364"; } +.bi-file-earmark-easel-fill::before { content: "\f365"; } +.bi-file-earmark-easel::before { content: "\f366"; } +.bi-file-earmark-excel-fill::before { content: "\f367"; } +.bi-file-earmark-excel::before { content: "\f368"; } +.bi-file-earmark-fill::before { content: "\f369"; } +.bi-file-earmark-font-fill::before { content: "\f36a"; } +.bi-file-earmark-font::before { content: "\f36b"; } +.bi-file-earmark-image-fill::before { content: "\f36c"; } +.bi-file-earmark-image::before { content: "\f36d"; } +.bi-file-earmark-lock-fill::before { content: "\f36e"; } +.bi-file-earmark-lock::before { content: "\f36f"; } +.bi-file-earmark-lock2-fill::before { content: "\f370"; } +.bi-file-earmark-lock2::before { content: "\f371"; } +.bi-file-earmark-medical-fill::before { content: "\f372"; } +.bi-file-earmark-medical::before { content: "\f373"; } +.bi-file-earmark-minus-fill::before { content: "\f374"; } +.bi-file-earmark-minus::before { content: "\f375"; } +.bi-file-earmark-music-fill::before { content: "\f376"; } +.bi-file-earmark-music::before { content: "\f377"; } +.bi-file-earmark-person-fill::before { content: "\f378"; } +.bi-file-earmark-person::before { content: "\f379"; } +.bi-file-earmark-play-fill::before { content: "\f37a"; } +.bi-file-earmark-play::before { content: "\f37b"; } +.bi-file-earmark-plus-fill::before { content: "\f37c"; } +.bi-file-earmark-plus::before { content: "\f37d"; } +.bi-file-earmark-post-fill::before { content: "\f37e"; } +.bi-file-earmark-post::before { content: "\f37f"; } +.bi-file-earmark-ppt-fill::before { content: "\f380"; } +.bi-file-earmark-ppt::before { content: "\f381"; } +.bi-file-earmark-richtext-fill::before { content: "\f382"; } +.bi-file-earmark-richtext::before { content: "\f383"; } +.bi-file-earmark-ruled-fill::before { content: "\f384"; } +.bi-file-earmark-ruled::before { content: "\f385"; } +.bi-file-earmark-slides-fill::before { content: "\f386"; } +.bi-file-earmark-slides::before { content: "\f387"; } +.bi-file-earmark-spreadsheet-fill::before { content: "\f388"; } +.bi-file-earmark-spreadsheet::before { content: "\f389"; } +.bi-file-earmark-text-fill::before { content: "\f38a"; } +.bi-file-earmark-text::before { content: "\f38b"; } +.bi-file-earmark-word-fill::before { content: "\f38c"; } +.bi-file-earmark-word::before { content: "\f38d"; } +.bi-file-earmark-x-fill::before { content: "\f38e"; } +.bi-file-earmark-x::before { content: "\f38f"; } +.bi-file-earmark-zip-fill::before { content: "\f390"; } +.bi-file-earmark-zip::before { content: "\f391"; } +.bi-file-earmark::before { content: "\f392"; } +.bi-file-easel-fill::before { content: "\f393"; } +.bi-file-easel::before { content: "\f394"; } +.bi-file-excel-fill::before { content: "\f395"; } +.bi-file-excel::before { content: "\f396"; } +.bi-file-fill::before { content: "\f397"; } +.bi-file-font-fill::before { content: "\f398"; } +.bi-file-font::before { content: "\f399"; } +.bi-file-image-fill::before { content: "\f39a"; } +.bi-file-image::before { content: "\f39b"; } +.bi-file-lock-fill::before { content: "\f39c"; } +.bi-file-lock::before { content: "\f39d"; } +.bi-file-lock2-fill::before { content: "\f39e"; } +.bi-file-lock2::before { content: "\f39f"; } +.bi-file-medical-fill::before { content: "\f3a0"; } +.bi-file-medical::before { content: "\f3a1"; } +.bi-file-minus-fill::before { content: "\f3a2"; } +.bi-file-minus::before { content: "\f3a3"; } +.bi-file-music-fill::before { content: "\f3a4"; } +.bi-file-music::before { content: "\f3a5"; } +.bi-file-person-fill::before { content: "\f3a6"; } +.bi-file-person::before { content: "\f3a7"; } +.bi-file-play-fill::before { content: "\f3a8"; } +.bi-file-play::before { content: "\f3a9"; } +.bi-file-plus-fill::before { content: "\f3aa"; } +.bi-file-plus::before { content: "\f3ab"; } +.bi-file-post-fill::before { content: "\f3ac"; } +.bi-file-post::before { content: "\f3ad"; } +.bi-file-ppt-fill::before { content: "\f3ae"; } +.bi-file-ppt::before { content: "\f3af"; } +.bi-file-richtext-fill::before { content: "\f3b0"; } +.bi-file-richtext::before { content: "\f3b1"; } +.bi-file-ruled-fill::before { content: "\f3b2"; } +.bi-file-ruled::before { content: "\f3b3"; } +.bi-file-slides-fill::before { content: "\f3b4"; } +.bi-file-slides::before { content: "\f3b5"; } +.bi-file-spreadsheet-fill::before { content: "\f3b6"; } +.bi-file-spreadsheet::before { content: "\f3b7"; } +.bi-file-text-fill::before { content: "\f3b8"; } +.bi-file-text::before { content: "\f3b9"; } +.bi-file-word-fill::before { content: "\f3ba"; } +.bi-file-word::before { content: "\f3bb"; } +.bi-file-x-fill::before { content: "\f3bc"; } +.bi-file-x::before { content: "\f3bd"; } +.bi-file-zip-fill::before { content: "\f3be"; } +.bi-file-zip::before { content: "\f3bf"; } +.bi-file::before { content: "\f3c0"; } +.bi-files-alt::before { content: "\f3c1"; } +.bi-files::before { content: "\f3c2"; } +.bi-film::before { content: "\f3c3"; } +.bi-filter-circle-fill::before { content: "\f3c4"; } +.bi-filter-circle::before { content: "\f3c5"; } +.bi-filter-left::before { content: "\f3c6"; } +.bi-filter-right::before { content: "\f3c7"; } +.bi-filter-square-fill::before { content: "\f3c8"; } +.bi-filter-square::before { content: "\f3c9"; } +.bi-filter::before { content: "\f3ca"; } +.bi-flag-fill::before { content: "\f3cb"; } +.bi-flag::before { content: "\f3cc"; } +.bi-flower1::before { content: "\f3cd"; } +.bi-flower2::before { content: "\f3ce"; } +.bi-flower3::before { content: "\f3cf"; } +.bi-folder-check::before { content: "\f3d0"; } +.bi-folder-fill::before { content: "\f3d1"; } +.bi-folder-minus::before { content: "\f3d2"; } +.bi-folder-plus::before { content: "\f3d3"; } +.bi-folder-symlink-fill::before { content: "\f3d4"; } +.bi-folder-symlink::before { content: "\f3d5"; } +.bi-folder-x::before { content: "\f3d6"; } +.bi-folder::before { content: "\f3d7"; } +.bi-folder2-open::before { content: "\f3d8"; } +.bi-folder2::before { content: "\f3d9"; } +.bi-fonts::before { content: "\f3da"; } +.bi-forward-fill::before { content: "\f3db"; } +.bi-forward::before { content: "\f3dc"; } +.bi-front::before { content: "\f3dd"; } +.bi-fullscreen-exit::before { content: "\f3de"; } +.bi-fullscreen::before { content: "\f3df"; } +.bi-funnel-fill::before { content: "\f3e0"; } +.bi-funnel::before { content: "\f3e1"; } +.bi-gear-fill::before { content: "\f3e2"; } +.bi-gear-wide-connected::before { content: "\f3e3"; } +.bi-gear-wide::before { content: "\f3e4"; } +.bi-gear::before { content: "\f3e5"; } +.bi-gem::before { content: "\f3e6"; } +.bi-geo-alt-fill::before { content: "\f3e7"; } +.bi-geo-alt::before { content: "\f3e8"; } +.bi-geo-fill::before { content: "\f3e9"; } +.bi-geo::before { content: "\f3ea"; } +.bi-gift-fill::before { content: "\f3eb"; } +.bi-gift::before { content: "\f3ec"; } +.bi-github::before { content: "\f3ed"; } +.bi-globe::before { content: "\f3ee"; } +.bi-globe2::before { content: "\f3ef"; } +.bi-google::before { content: "\f3f0"; } +.bi-graph-down::before { content: "\f3f1"; } +.bi-graph-up::before { content: "\f3f2"; } +.bi-grid-1x2-fill::before { content: "\f3f3"; } +.bi-grid-1x2::before { content: "\f3f4"; } +.bi-grid-3x2-gap-fill::before { content: "\f3f5"; } +.bi-grid-3x2-gap::before { content: "\f3f6"; } +.bi-grid-3x2::before { content: "\f3f7"; } +.bi-grid-3x3-gap-fill::before { content: "\f3f8"; } +.bi-grid-3x3-gap::before { content: "\f3f9"; } +.bi-grid-3x3::before { content: "\f3fa"; } +.bi-grid-fill::before { content: "\f3fb"; } +.bi-grid::before { content: "\f3fc"; } +.bi-grip-horizontal::before { content: "\f3fd"; } +.bi-grip-vertical::before { content: "\f3fe"; } +.bi-hammer::before { content: "\f3ff"; } +.bi-hand-index-fill::before { content: "\f400"; } +.bi-hand-index-thumb-fill::before { content: "\f401"; } +.bi-hand-index-thumb::before { content: "\f402"; } +.bi-hand-index::before { content: "\f403"; } +.bi-hand-thumbs-down-fill::before { content: "\f404"; } +.bi-hand-thumbs-down::before { content: "\f405"; } +.bi-hand-thumbs-up-fill::before { content: "\f406"; } +.bi-hand-thumbs-up::before { content: "\f407"; } +.bi-handbag-fill::before { content: "\f408"; } +.bi-handbag::before { content: "\f409"; } +.bi-hash::before { content: "\f40a"; } +.bi-hdd-fill::before { content: "\f40b"; } +.bi-hdd-network-fill::before { content: "\f40c"; } +.bi-hdd-network::before { content: "\f40d"; } +.bi-hdd-rack-fill::before { content: "\f40e"; } +.bi-hdd-rack::before { content: "\f40f"; } +.bi-hdd-stack-fill::before { content: "\f410"; } +.bi-hdd-stack::before { content: "\f411"; } +.bi-hdd::before { content: "\f412"; } +.bi-headphones::before { content: "\f413"; } +.bi-headset::before { content: "\f414"; } +.bi-heart-fill::before { content: "\f415"; } +.bi-heart-half::before { content: "\f416"; } +.bi-heart::before { content: "\f417"; } +.bi-heptagon-fill::before { content: "\f418"; } +.bi-heptagon-half::before { content: "\f419"; } +.bi-heptagon::before { content: "\f41a"; } +.bi-hexagon-fill::before { content: "\f41b"; } +.bi-hexagon-half::before { content: "\f41c"; } +.bi-hexagon::before { content: "\f41d"; } +.bi-hourglass-bottom::before { content: "\f41e"; } +.bi-hourglass-split::before { content: "\f41f"; } +.bi-hourglass-top::before { content: "\f420"; } +.bi-hourglass::before { content: "\f421"; } +.bi-house-door-fill::before { content: "\f422"; } +.bi-house-door::before { content: "\f423"; } +.bi-house-fill::before { content: "\f424"; } +.bi-house::before { content: "\f425"; } +.bi-hr::before { content: "\f426"; } +.bi-hurricane::before { content: "\f427"; } +.bi-image-alt::before { content: "\f428"; } +.bi-image-fill::before { content: "\f429"; } +.bi-image::before { content: "\f42a"; } +.bi-images::before { content: "\f42b"; } +.bi-inbox-fill::before { content: "\f42c"; } +.bi-inbox::before { content: "\f42d"; } +.bi-inboxes-fill::before { content: "\f42e"; } +.bi-inboxes::before { content: "\f42f"; } +.bi-info-circle-fill::before { content: "\f430"; } +.bi-info-circle::before { content: "\f431"; } +.bi-info-square-fill::before { content: "\f432"; } +.bi-info-square::before { content: "\f433"; } +.bi-info::before { content: "\f434"; } +.bi-input-cursor-text::before { content: "\f435"; } +.bi-input-cursor::before { content: "\f436"; } +.bi-instagram::before { content: "\f437"; } +.bi-intersect::before { content: "\f438"; } +.bi-journal-album::before { content: "\f439"; } +.bi-journal-arrow-down::before { content: "\f43a"; } +.bi-journal-arrow-up::before { content: "\f43b"; } +.bi-journal-bookmark-fill::before { content: "\f43c"; } +.bi-journal-bookmark::before { content: "\f43d"; } +.bi-journal-check::before { content: "\f43e"; } +.bi-journal-code::before { content: "\f43f"; } +.bi-journal-medical::before { content: "\f440"; } +.bi-journal-minus::before { content: "\f441"; } +.bi-journal-plus::before { content: "\f442"; } +.bi-journal-richtext::before { content: "\f443"; } +.bi-journal-text::before { content: "\f444"; } +.bi-journal-x::before { content: "\f445"; } +.bi-journal::before { content: "\f446"; } +.bi-journals::before { content: "\f447"; } +.bi-joystick::before { content: "\f448"; } +.bi-justify-left::before { content: "\f449"; } +.bi-justify-right::before { content: "\f44a"; } +.bi-justify::before { content: "\f44b"; } +.bi-kanban-fill::before { content: "\f44c"; } +.bi-kanban::before { content: "\f44d"; } +.bi-key-fill::before { content: "\f44e"; } +.bi-key::before { content: "\f44f"; } +.bi-keyboard-fill::before { content: "\f450"; } +.bi-keyboard::before { content: "\f451"; } +.bi-ladder::before { content: "\f452"; } +.bi-lamp-fill::before { content: "\f453"; } +.bi-lamp::before { content: "\f454"; } +.bi-laptop-fill::before { content: "\f455"; } +.bi-laptop::before { content: "\f456"; } +.bi-layer-backward::before { content: "\f457"; } +.bi-layer-forward::before { content: "\f458"; } +.bi-layers-fill::before { content: "\f459"; } +.bi-layers-half::before { content: "\f45a"; } +.bi-layers::before { content: "\f45b"; } +.bi-layout-sidebar-inset-reverse::before { content: "\f45c"; } +.bi-layout-sidebar-inset::before { content: "\f45d"; } +.bi-layout-sidebar-reverse::before { content: "\f45e"; } +.bi-layout-sidebar::before { content: "\f45f"; } +.bi-layout-split::before { content: "\f460"; } +.bi-layout-text-sidebar-reverse::before { content: "\f461"; } +.bi-layout-text-sidebar::before { content: "\f462"; } +.bi-layout-text-window-reverse::before { content: "\f463"; } +.bi-layout-text-window::before { content: "\f464"; } +.bi-layout-three-columns::before { content: "\f465"; } +.bi-layout-wtf::before { content: "\f466"; } +.bi-life-preserver::before { content: "\f467"; } +.bi-lightbulb-fill::before { content: "\f468"; } +.bi-lightbulb-off-fill::before { content: "\f469"; } +.bi-lightbulb-off::before { content: "\f46a"; } +.bi-lightbulb::before { content: "\f46b"; } +.bi-lightning-charge-fill::before { content: "\f46c"; } +.bi-lightning-charge::before { content: "\f46d"; } +.bi-lightning-fill::before { content: "\f46e"; } +.bi-lightning::before { content: "\f46f"; } +.bi-link-45deg::before { content: "\f470"; } +.bi-link::before { content: "\f471"; } +.bi-linkedin::before { content: "\f472"; } +.bi-list-check::before { content: "\f473"; } +.bi-list-nested::before { content: "\f474"; } +.bi-list-ol::before { content: "\f475"; } +.bi-list-stars::before { content: "\f476"; } +.bi-list-task::before { content: "\f477"; } +.bi-list-ul::before { content: "\f478"; } +.bi-list::before { content: "\f479"; } +.bi-lock-fill::before { content: "\f47a"; } +.bi-lock::before { content: "\f47b"; } +.bi-mailbox::before { content: "\f47c"; } +.bi-mailbox2::before { content: "\f47d"; } +.bi-map-fill::before { content: "\f47e"; } +.bi-map::before { content: "\f47f"; } +.bi-markdown-fill::before { content: "\f480"; } +.bi-markdown::before { content: "\f481"; } +.bi-mask::before { content: "\f482"; } +.bi-megaphone-fill::before { content: "\f483"; } +.bi-megaphone::before { content: "\f484"; } +.bi-menu-app-fill::before { content: "\f485"; } +.bi-menu-app::before { content: "\f486"; } +.bi-menu-button-fill::before { content: "\f487"; } +.bi-menu-button-wide-fill::before { content: "\f488"; } +.bi-menu-button-wide::before { content: "\f489"; } +.bi-menu-button::before { content: "\f48a"; } +.bi-menu-down::before { content: "\f48b"; } +.bi-menu-up::before { content: "\f48c"; } +.bi-mic-fill::before { content: "\f48d"; } +.bi-mic-mute-fill::before { content: "\f48e"; } +.bi-mic-mute::before { content: "\f48f"; } +.bi-mic::before { content: "\f490"; } +.bi-minecart-loaded::before { content: "\f491"; } +.bi-minecart::before { content: "\f492"; } +.bi-moisture::before { content: "\f493"; } +.bi-moon-fill::before { content: "\f494"; } +.bi-moon-stars-fill::before { content: "\f495"; } +.bi-moon-stars::before { content: "\f496"; } +.bi-moon::before { content: "\f497"; } +.bi-mouse-fill::before { content: "\f498"; } +.bi-mouse::before { content: "\f499"; } +.bi-mouse2-fill::before { content: "\f49a"; } +.bi-mouse2::before { content: "\f49b"; } +.bi-mouse3-fill::before { content: "\f49c"; } +.bi-mouse3::before { content: "\f49d"; } +.bi-music-note-beamed::before { content: "\f49e"; } +.bi-music-note-list::before { content: "\f49f"; } +.bi-music-note::before { content: "\f4a0"; } +.bi-music-player-fill::before { content: "\f4a1"; } +.bi-music-player::before { content: "\f4a2"; } +.bi-newspaper::before { content: "\f4a3"; } +.bi-node-minus-fill::before { content: "\f4a4"; } +.bi-node-minus::before { content: "\f4a5"; } +.bi-node-plus-fill::before { content: "\f4a6"; } +.bi-node-plus::before { content: "\f4a7"; } +.bi-nut-fill::before { content: "\f4a8"; } +.bi-nut::before { content: "\f4a9"; } +.bi-octagon-fill::before { content: "\f4aa"; } +.bi-octagon-half::before { content: "\f4ab"; } +.bi-octagon::before { content: "\f4ac"; } +.bi-option::before { content: "\f4ad"; } +.bi-outlet::before { content: "\f4ae"; } +.bi-paint-bucket::before { content: "\f4af"; } +.bi-palette-fill::before { content: "\f4b0"; } +.bi-palette::before { content: "\f4b1"; } +.bi-palette2::before { content: "\f4b2"; } +.bi-paperclip::before { content: "\f4b3"; } +.bi-paragraph::before { content: "\f4b4"; } +.bi-patch-check-fill::before { content: "\f4b5"; } +.bi-patch-check::before { content: "\f4b6"; } +.bi-patch-exclamation-fill::before { content: "\f4b7"; } +.bi-patch-exclamation::before { content: "\f4b8"; } +.bi-patch-minus-fill::before { content: "\f4b9"; } +.bi-patch-minus::before { content: "\f4ba"; } +.bi-patch-plus-fill::before { content: "\f4bb"; } +.bi-patch-plus::before { content: "\f4bc"; } +.bi-patch-question-fill::before { content: "\f4bd"; } +.bi-patch-question::before { content: "\f4be"; } +.bi-pause-btn-fill::before { content: "\f4bf"; } +.bi-pause-btn::before { content: "\f4c0"; } +.bi-pause-circle-fill::before { content: "\f4c1"; } +.bi-pause-circle::before { content: "\f4c2"; } +.bi-pause-fill::before { content: "\f4c3"; } +.bi-pause::before { content: "\f4c4"; } +.bi-peace-fill::before { content: "\f4c5"; } +.bi-peace::before { content: "\f4c6"; } +.bi-pen-fill::before { content: "\f4c7"; } +.bi-pen::before { content: "\f4c8"; } +.bi-pencil-fill::before { content: "\f4c9"; } +.bi-pencil-square::before { content: "\f4ca"; } +.bi-pencil::before { content: "\f4cb"; } +.bi-pentagon-fill::before { content: "\f4cc"; } +.bi-pentagon-half::before { content: "\f4cd"; } +.bi-pentagon::before { content: "\f4ce"; } +.bi-people-fill::before { content: "\f4cf"; } +.bi-people::before { content: "\f4d0"; } +.bi-percent::before { content: "\f4d1"; } +.bi-person-badge-fill::before { content: "\f4d2"; } +.bi-person-badge::before { content: "\f4d3"; } +.bi-person-bounding-box::before { content: "\f4d4"; } +.bi-person-check-fill::before { content: "\f4d5"; } +.bi-person-check::before { content: "\f4d6"; } +.bi-person-circle::before { content: "\f4d7"; } +.bi-person-dash-fill::before { content: "\f4d8"; } +.bi-person-dash::before { content: "\f4d9"; } +.bi-person-fill::before { content: "\f4da"; } +.bi-person-lines-fill::before { content: "\f4db"; } +.bi-person-plus-fill::before { content: "\f4dc"; } +.bi-person-plus::before { content: "\f4dd"; } +.bi-person-square::before { content: "\f4de"; } +.bi-person-x-fill::before { content: "\f4df"; } +.bi-person-x::before { content: "\f4e0"; } +.bi-person::before { content: "\f4e1"; } +.bi-phone-fill::before { content: "\f4e2"; } +.bi-phone-landscape-fill::before { content: "\f4e3"; } +.bi-phone-landscape::before { content: "\f4e4"; } +.bi-phone-vibrate-fill::before { content: "\f4e5"; } +.bi-phone-vibrate::before { content: "\f4e6"; } +.bi-phone::before { content: "\f4e7"; } +.bi-pie-chart-fill::before { content: "\f4e8"; } +.bi-pie-chart::before { content: "\f4e9"; } +.bi-pin-angle-fill::before { content: "\f4ea"; } +.bi-pin-angle::before { content: "\f4eb"; } +.bi-pin-fill::before { content: "\f4ec"; } +.bi-pin::before { content: "\f4ed"; } +.bi-pip-fill::before { content: "\f4ee"; } +.bi-pip::before { content: "\f4ef"; } +.bi-play-btn-fill::before { content: "\f4f0"; } +.bi-play-btn::before { content: "\f4f1"; } +.bi-play-circle-fill::before { content: "\f4f2"; } +.bi-play-circle::before { content: "\f4f3"; } +.bi-play-fill::before { content: "\f4f4"; } +.bi-play::before { content: "\f4f5"; } +.bi-plug-fill::before { content: "\f4f6"; } +.bi-plug::before { content: "\f4f7"; } +.bi-plus-circle-dotted::before { content: "\f4f8"; } +.bi-plus-circle-fill::before { content: "\f4f9"; } +.bi-plus-circle::before { content: "\f4fa"; } +.bi-plus-square-dotted::before { content: "\f4fb"; } +.bi-plus-square-fill::before { content: "\f4fc"; } +.bi-plus-square::before { content: "\f4fd"; } +.bi-plus::before { content: "\f4fe"; } +.bi-power::before { content: "\f4ff"; } +.bi-printer-fill::before { content: "\f500"; } +.bi-printer::before { content: "\f501"; } +.bi-puzzle-fill::before { content: "\f502"; } +.bi-puzzle::before { content: "\f503"; } +.bi-question-circle-fill::before { content: "\f504"; } +.bi-question-circle::before { content: "\f505"; } +.bi-question-diamond-fill::before { content: "\f506"; } +.bi-question-diamond::before { content: "\f507"; } +.bi-question-octagon-fill::before { content: "\f508"; } +.bi-question-octagon::before { content: "\f509"; } +.bi-question-square-fill::before { content: "\f50a"; } +.bi-question-square::before { content: "\f50b"; } +.bi-question::before { content: "\f50c"; } +.bi-rainbow::before { content: "\f50d"; } +.bi-receipt-cutoff::before { content: "\f50e"; } +.bi-receipt::before { content: "\f50f"; } +.bi-reception-0::before { content: "\f510"; } +.bi-reception-1::before { content: "\f511"; } +.bi-reception-2::before { content: "\f512"; } +.bi-reception-3::before { content: "\f513"; } +.bi-reception-4::before { content: "\f514"; } +.bi-record-btn-fill::before { content: "\f515"; } +.bi-record-btn::before { content: "\f516"; } +.bi-record-circle-fill::before { content: "\f517"; } +.bi-record-circle::before { content: "\f518"; } +.bi-record-fill::before { content: "\f519"; } +.bi-record::before { content: "\f51a"; } +.bi-record2-fill::before { content: "\f51b"; } +.bi-record2::before { content: "\f51c"; } +.bi-reply-all-fill::before { content: "\f51d"; } +.bi-reply-all::before { content: "\f51e"; } +.bi-reply-fill::before { content: "\f51f"; } +.bi-reply::before { content: "\f520"; } +.bi-rss-fill::before { content: "\f521"; } +.bi-rss::before { content: "\f522"; } +.bi-rulers::before { content: "\f523"; } +.bi-save-fill::before { content: "\f524"; } +.bi-save::before { content: "\f525"; } +.bi-save2-fill::before { content: "\f526"; } +.bi-save2::before { content: "\f527"; } +.bi-scissors::before { content: "\f528"; } +.bi-screwdriver::before { content: "\f529"; } +.bi-search::before { content: "\f52a"; } +.bi-segmented-nav::before { content: "\f52b"; } +.bi-server::before { content: "\f52c"; } +.bi-share-fill::before { content: "\f52d"; } +.bi-share::before { content: "\f52e"; } +.bi-shield-check::before { content: "\f52f"; } +.bi-shield-exclamation::before { content: "\f530"; } +.bi-shield-fill-check::before { content: "\f531"; } +.bi-shield-fill-exclamation::before { content: "\f532"; } +.bi-shield-fill-minus::before { content: "\f533"; } +.bi-shield-fill-plus::before { content: "\f534"; } +.bi-shield-fill-x::before { content: "\f535"; } +.bi-shield-fill::before { content: "\f536"; } +.bi-shield-lock-fill::before { content: "\f537"; } +.bi-shield-lock::before { content: "\f538"; } +.bi-shield-minus::before { content: "\f539"; } +.bi-shield-plus::before { content: "\f53a"; } +.bi-shield-shaded::before { content: "\f53b"; } +.bi-shield-slash-fill::before { content: "\f53c"; } +.bi-shield-slash::before { content: "\f53d"; } +.bi-shield-x::before { content: "\f53e"; } +.bi-shield::before { content: "\f53f"; } +.bi-shift-fill::before { content: "\f540"; } +.bi-shift::before { content: "\f541"; } +.bi-shop-window::before { content: "\f542"; } +.bi-shop::before { content: "\f543"; } +.bi-shuffle::before { content: "\f544"; } +.bi-signpost-2-fill::before { content: "\f545"; } +.bi-signpost-2::before { content: "\f546"; } +.bi-signpost-fill::before { content: "\f547"; } +.bi-signpost-split-fill::before { content: "\f548"; } +.bi-signpost-split::before { content: "\f549"; } +.bi-signpost::before { content: "\f54a"; } +.bi-sim-fill::before { content: "\f54b"; } +.bi-sim::before { content: "\f54c"; } +.bi-skip-backward-btn-fill::before { content: "\f54d"; } +.bi-skip-backward-btn::before { content: "\f54e"; } +.bi-skip-backward-circle-fill::before { content: "\f54f"; } +.bi-skip-backward-circle::before { content: "\f550"; } +.bi-skip-backward-fill::before { content: "\f551"; } +.bi-skip-backward::before { content: "\f552"; } +.bi-skip-end-btn-fill::before { content: "\f553"; } +.bi-skip-end-btn::before { content: "\f554"; } +.bi-skip-end-circle-fill::before { content: "\f555"; } +.bi-skip-end-circle::before { content: "\f556"; } +.bi-skip-end-fill::before { content: "\f557"; } +.bi-skip-end::before { content: "\f558"; } +.bi-skip-forward-btn-fill::before { content: "\f559"; } +.bi-skip-forward-btn::before { content: "\f55a"; } +.bi-skip-forward-circle-fill::before { content: "\f55b"; } +.bi-skip-forward-circle::before { content: "\f55c"; } +.bi-skip-forward-fill::before { content: "\f55d"; } +.bi-skip-forward::before { content: "\f55e"; } +.bi-skip-start-btn-fill::before { content: "\f55f"; } +.bi-skip-start-btn::before { content: "\f560"; } +.bi-skip-start-circle-fill::before { content: "\f561"; } +.bi-skip-start-circle::before { content: "\f562"; } +.bi-skip-start-fill::before { content: "\f563"; } +.bi-skip-start::before { content: "\f564"; } +.bi-slack::before { content: "\f565"; } +.bi-slash-circle-fill::before { content: "\f566"; } +.bi-slash-circle::before { content: "\f567"; } +.bi-slash-square-fill::before { content: "\f568"; } +.bi-slash-square::before { content: "\f569"; } +.bi-slash::before { content: "\f56a"; } +.bi-sliders::before { content: "\f56b"; } +.bi-smartwatch::before { content: "\f56c"; } +.bi-snow::before { content: "\f56d"; } +.bi-snow2::before { content: "\f56e"; } +.bi-snow3::before { content: "\f56f"; } +.bi-sort-alpha-down-alt::before { content: "\f570"; } +.bi-sort-alpha-down::before { content: "\f571"; } +.bi-sort-alpha-up-alt::before { content: "\f572"; } +.bi-sort-alpha-up::before { content: "\f573"; } +.bi-sort-down-alt::before { content: "\f574"; } +.bi-sort-down::before { content: "\f575"; } +.bi-sort-numeric-down-alt::before { content: "\f576"; } +.bi-sort-numeric-down::before { content: "\f577"; } +.bi-sort-numeric-up-alt::before { content: "\f578"; } +.bi-sort-numeric-up::before { content: "\f579"; } +.bi-sort-up-alt::before { content: "\f57a"; } +.bi-sort-up::before { content: "\f57b"; } +.bi-soundwave::before { content: "\f57c"; } +.bi-speaker-fill::before { content: "\f57d"; } +.bi-speaker::before { content: "\f57e"; } +.bi-speedometer::before { content: "\f57f"; } +.bi-speedometer2::before { content: "\f580"; } +.bi-spellcheck::before { content: "\f581"; } +.bi-square-fill::before { content: "\f582"; } +.bi-square-half::before { content: "\f583"; } +.bi-square::before { content: "\f584"; } +.bi-stack::before { content: "\f585"; } +.bi-star-fill::before { content: "\f586"; } +.bi-star-half::before { content: "\f587"; } +.bi-star::before { content: "\f588"; } +.bi-stars::before { content: "\f589"; } +.bi-stickies-fill::before { content: "\f58a"; } +.bi-stickies::before { content: "\f58b"; } +.bi-sticky-fill::before { content: "\f58c"; } +.bi-sticky::before { content: "\f58d"; } +.bi-stop-btn-fill::before { content: "\f58e"; } +.bi-stop-btn::before { content: "\f58f"; } +.bi-stop-circle-fill::before { content: "\f590"; } +.bi-stop-circle::before { content: "\f591"; } +.bi-stop-fill::before { content: "\f592"; } +.bi-stop::before { content: "\f593"; } +.bi-stoplights-fill::before { content: "\f594"; } +.bi-stoplights::before { content: "\f595"; } +.bi-stopwatch-fill::before { content: "\f596"; } +.bi-stopwatch::before { content: "\f597"; } +.bi-subtract::before { content: "\f598"; } +.bi-suit-club-fill::before { content: "\f599"; } +.bi-suit-club::before { content: "\f59a"; } +.bi-suit-diamond-fill::before { content: "\f59b"; } +.bi-suit-diamond::before { content: "\f59c"; } +.bi-suit-heart-fill::before { content: "\f59d"; } +.bi-suit-heart::before { content: "\f59e"; } +.bi-suit-spade-fill::before { content: "\f59f"; } +.bi-suit-spade::before { content: "\f5a0"; } +.bi-sun-fill::before { content: "\f5a1"; } +.bi-sun::before { content: "\f5a2"; } +.bi-sunglasses::before { content: "\f5a3"; } +.bi-sunrise-fill::before { content: "\f5a4"; } +.bi-sunrise::before { content: "\f5a5"; } +.bi-sunset-fill::before { content: "\f5a6"; } +.bi-sunset::before { content: "\f5a7"; } +.bi-symmetry-horizontal::before { content: "\f5a8"; } +.bi-symmetry-vertical::before { content: "\f5a9"; } +.bi-table::before { content: "\f5aa"; } +.bi-tablet-fill::before { content: "\f5ab"; } +.bi-tablet-landscape-fill::before { content: "\f5ac"; } +.bi-tablet-landscape::before { content: "\f5ad"; } +.bi-tablet::before { content: "\f5ae"; } +.bi-tag-fill::before { content: "\f5af"; } +.bi-tag::before { content: "\f5b0"; } +.bi-tags-fill::before { content: "\f5b1"; } +.bi-tags::before { content: "\f5b2"; } +.bi-telegram::before { content: "\f5b3"; } +.bi-telephone-fill::before { content: "\f5b4"; } +.bi-telephone-forward-fill::before { content: "\f5b5"; } +.bi-telephone-forward::before { content: "\f5b6"; } +.bi-telephone-inbound-fill::before { content: "\f5b7"; } +.bi-telephone-inbound::before { content: "\f5b8"; } +.bi-telephone-minus-fill::before { content: "\f5b9"; } +.bi-telephone-minus::before { content: "\f5ba"; } +.bi-telephone-outbound-fill::before { content: "\f5bb"; } +.bi-telephone-outbound::before { content: "\f5bc"; } +.bi-telephone-plus-fill::before { content: "\f5bd"; } +.bi-telephone-plus::before { content: "\f5be"; } +.bi-telephone-x-fill::before { content: "\f5bf"; } +.bi-telephone-x::before { content: "\f5c0"; } +.bi-telephone::before { content: "\f5c1"; } +.bi-terminal-fill::before { content: "\f5c2"; } +.bi-terminal::before { content: "\f5c3"; } +.bi-text-center::before { content: "\f5c4"; } +.bi-text-indent-left::before { content: "\f5c5"; } +.bi-text-indent-right::before { content: "\f5c6"; } +.bi-text-left::before { content: "\f5c7"; } +.bi-text-paragraph::before { content: "\f5c8"; } +.bi-text-right::before { content: "\f5c9"; } +.bi-textarea-resize::before { content: "\f5ca"; } +.bi-textarea-t::before { content: "\f5cb"; } +.bi-textarea::before { content: "\f5cc"; } +.bi-thermometer-half::before { content: "\f5cd"; } +.bi-thermometer-high::before { content: "\f5ce"; } +.bi-thermometer-low::before { content: "\f5cf"; } +.bi-thermometer-snow::before { content: "\f5d0"; } +.bi-thermometer-sun::before { content: "\f5d1"; } +.bi-thermometer::before { content: "\f5d2"; } +.bi-three-dots-vertical::before { content: "\f5d3"; } +.bi-three-dots::before { content: "\f5d4"; } +.bi-toggle-off::before { content: "\f5d5"; } +.bi-toggle-on::before { content: "\f5d6"; } +.bi-toggle2-off::before { content: "\f5d7"; } +.bi-toggle2-on::before { content: "\f5d8"; } +.bi-toggles::before { content: "\f5d9"; } +.bi-toggles2::before { content: "\f5da"; } +.bi-tools::before { content: "\f5db"; } +.bi-tornado::before { content: "\f5dc"; } +.bi-trash-fill::before { content: "\f5dd"; } +.bi-trash::before { content: "\f5de"; } +.bi-trash2-fill::before { content: "\f5df"; } +.bi-trash2::before { content: "\f5e0"; } +.bi-tree-fill::before { content: "\f5e1"; } +.bi-tree::before { content: "\f5e2"; } +.bi-triangle-fill::before { content: "\f5e3"; } +.bi-triangle-half::before { content: "\f5e4"; } +.bi-triangle::before { content: "\f5e5"; } +.bi-trophy-fill::before { content: "\f5e6"; } +.bi-trophy::before { content: "\f5e7"; } +.bi-tropical-storm::before { content: "\f5e8"; } +.bi-truck-flatbed::before { content: "\f5e9"; } +.bi-truck::before { content: "\f5ea"; } +.bi-tsunami::before { content: "\f5eb"; } +.bi-tv-fill::before { content: "\f5ec"; } +.bi-tv::before { content: "\f5ed"; } +.bi-twitch::before { content: "\f5ee"; } +.bi-twitter::before { content: "\f5ef"; } +.bi-type-bold::before { content: "\f5f0"; } +.bi-type-h1::before { content: "\f5f1"; } +.bi-type-h2::before { content: "\f5f2"; } +.bi-type-h3::before { content: "\f5f3"; } +.bi-type-italic::before { content: "\f5f4"; } +.bi-type-strikethrough::before { content: "\f5f5"; } +.bi-type-underline::before { content: "\f5f6"; } +.bi-type::before { content: "\f5f7"; } +.bi-ui-checks-grid::before { content: "\f5f8"; } +.bi-ui-checks::before { content: "\f5f9"; } +.bi-ui-radios-grid::before { content: "\f5fa"; } +.bi-ui-radios::before { content: "\f5fb"; } +.bi-umbrella-fill::before { content: "\f5fc"; } +.bi-umbrella::before { content: "\f5fd"; } +.bi-union::before { content: "\f5fe"; } +.bi-unlock-fill::before { content: "\f5ff"; } +.bi-unlock::before { content: "\f600"; } +.bi-upc-scan::before { content: "\f601"; } +.bi-upc::before { content: "\f602"; } +.bi-upload::before { content: "\f603"; } +.bi-vector-pen::before { content: "\f604"; } +.bi-view-list::before { content: "\f605"; } +.bi-view-stacked::before { content: "\f606"; } +.bi-vinyl-fill::before { content: "\f607"; } +.bi-vinyl::before { content: "\f608"; } +.bi-voicemail::before { content: "\f609"; } +.bi-volume-down-fill::before { content: "\f60a"; } +.bi-volume-down::before { content: "\f60b"; } +.bi-volume-mute-fill::before { content: "\f60c"; } +.bi-volume-mute::before { content: "\f60d"; } +.bi-volume-off-fill::before { content: "\f60e"; } +.bi-volume-off::before { content: "\f60f"; } +.bi-volume-up-fill::before { content: "\f610"; } +.bi-volume-up::before { content: "\f611"; } +.bi-vr::before { content: "\f612"; } +.bi-wallet-fill::before { content: "\f613"; } +.bi-wallet::before { content: "\f614"; } +.bi-wallet2::before { content: "\f615"; } +.bi-watch::before { content: "\f616"; } +.bi-water::before { content: "\f617"; } +.bi-whatsapp::before { content: "\f618"; } +.bi-wifi-1::before { content: "\f619"; } +.bi-wifi-2::before { content: "\f61a"; } +.bi-wifi-off::before { content: "\f61b"; } +.bi-wifi::before { content: "\f61c"; } +.bi-wind::before { content: "\f61d"; } +.bi-window-dock::before { content: "\f61e"; } +.bi-window-sidebar::before { content: "\f61f"; } +.bi-window::before { content: "\f620"; } +.bi-wrench::before { content: "\f621"; } +.bi-x-circle-fill::before { content: "\f622"; } +.bi-x-circle::before { content: "\f623"; } +.bi-x-diamond-fill::before { content: "\f624"; } +.bi-x-diamond::before { content: "\f625"; } +.bi-x-octagon-fill::before { content: "\f626"; } +.bi-x-octagon::before { content: "\f627"; } +.bi-x-square-fill::before { content: "\f628"; } +.bi-x-square::before { content: "\f629"; } +.bi-x::before { content: "\f62a"; } +.bi-youtube::before { content: "\f62b"; } +.bi-zoom-in::before { content: "\f62c"; } +.bi-zoom-out::before { content: "\f62d"; } +.bi-bank::before { content: "\f62e"; } +.bi-bank2::before { content: "\f62f"; } +.bi-bell-slash-fill::before { content: "\f630"; } +.bi-bell-slash::before { content: "\f631"; } +.bi-cash-coin::before { content: "\f632"; } +.bi-check-lg::before { content: "\f633"; } +.bi-coin::before { content: "\f634"; } +.bi-currency-bitcoin::before { content: "\f635"; } +.bi-currency-dollar::before { content: "\f636"; } +.bi-currency-euro::before { content: "\f637"; } +.bi-currency-exchange::before { content: "\f638"; } +.bi-currency-pound::before { content: "\f639"; } +.bi-currency-yen::before { content: "\f63a"; } +.bi-dash-lg::before { content: "\f63b"; } +.bi-exclamation-lg::before { content: "\f63c"; } +.bi-file-earmark-pdf-fill::before { content: "\f63d"; } +.bi-file-earmark-pdf::before { content: "\f63e"; } +.bi-file-pdf-fill::before { content: "\f63f"; } +.bi-file-pdf::before { content: "\f640"; } +.bi-gender-ambiguous::before { content: "\f641"; } +.bi-gender-female::before { content: "\f642"; } +.bi-gender-male::before { content: "\f643"; } +.bi-gender-trans::before { content: "\f644"; } +.bi-headset-vr::before { content: "\f645"; } +.bi-info-lg::before { content: "\f646"; } +.bi-mastodon::before { content: "\f647"; } +.bi-messenger::before { content: "\f648"; } +.bi-piggy-bank-fill::before { content: "\f649"; } +.bi-piggy-bank::before { content: "\f64a"; } +.bi-pin-map-fill::before { content: "\f64b"; } +.bi-pin-map::before { content: "\f64c"; } +.bi-plus-lg::before { content: "\f64d"; } +.bi-question-lg::before { content: "\f64e"; } +.bi-recycle::before { content: "\f64f"; } +.bi-reddit::before { content: "\f650"; } +.bi-safe-fill::before { content: "\f651"; } +.bi-safe2-fill::before { content: "\f652"; } +.bi-safe2::before { content: "\f653"; } +.bi-sd-card-fill::before { content: "\f654"; } +.bi-sd-card::before { content: "\f655"; } +.bi-skype::before { content: "\f656"; } +.bi-slash-lg::before { content: "\f657"; } +.bi-translate::before { content: "\f658"; } +.bi-x-lg::before { content: "\f659"; } +.bi-safe::before { content: "\f65a"; } +.bi-apple::before { content: "\f65b"; } +.bi-microsoft::before { content: "\f65d"; } +.bi-windows::before { content: "\f65e"; } +.bi-behance::before { content: "\f65c"; } +.bi-dribbble::before { content: "\f65f"; } +.bi-line::before { content: "\f660"; } +.bi-medium::before { content: "\f661"; } +.bi-paypal::before { content: "\f662"; } +.bi-pinterest::before { content: "\f663"; } +.bi-signal::before { content: "\f664"; } +.bi-snapchat::before { content: "\f665"; } +.bi-spotify::before { content: "\f666"; } +.bi-stack-overflow::before { content: "\f667"; } +.bi-strava::before { content: "\f668"; } +.bi-wordpress::before { content: "\f669"; } +.bi-vimeo::before { content: "\f66a"; } +.bi-activity::before { content: "\f66b"; } +.bi-easel2-fill::before { content: "\f66c"; } +.bi-easel2::before { content: "\f66d"; } +.bi-easel3-fill::before { content: "\f66e"; } +.bi-easel3::before { content: "\f66f"; } +.bi-fan::before { content: "\f670"; } +.bi-fingerprint::before { content: "\f671"; } +.bi-graph-down-arrow::before { content: "\f672"; } +.bi-graph-up-arrow::before { content: "\f673"; } +.bi-hypnotize::before { content: "\f674"; } +.bi-magic::before { content: "\f675"; } +.bi-person-rolodex::before { content: "\f676"; } +.bi-person-video::before { content: "\f677"; } +.bi-person-video2::before { content: "\f678"; } +.bi-person-video3::before { content: "\f679"; } +.bi-person-workspace::before { content: "\f67a"; } +.bi-radioactive::before { content: "\f67b"; } +.bi-webcam-fill::before { content: "\f67c"; } +.bi-webcam::before { content: "\f67d"; } +.bi-yin-yang::before { content: "\f67e"; } +.bi-bandaid-fill::before { content: "\f680"; } +.bi-bandaid::before { content: "\f681"; } +.bi-bluetooth::before { content: "\f682"; } +.bi-body-text::before { content: "\f683"; } +.bi-boombox::before { content: "\f684"; } +.bi-boxes::before { content: "\f685"; } +.bi-dpad-fill::before { content: "\f686"; } +.bi-dpad::before { content: "\f687"; } +.bi-ear-fill::before { content: "\f688"; } +.bi-ear::before { content: "\f689"; } +.bi-envelope-check-fill::before { content: "\f68b"; } +.bi-envelope-check::before { content: "\f68c"; } +.bi-envelope-dash-fill::before { content: "\f68e"; } +.bi-envelope-dash::before { content: "\f68f"; } +.bi-envelope-exclamation-fill::before { content: "\f691"; } +.bi-envelope-exclamation::before { content: "\f692"; } +.bi-envelope-plus-fill::before { content: "\f693"; } +.bi-envelope-plus::before { content: "\f694"; } +.bi-envelope-slash-fill::before { content: "\f696"; } +.bi-envelope-slash::before { content: "\f697"; } +.bi-envelope-x-fill::before { content: "\f699"; } +.bi-envelope-x::before { content: "\f69a"; } +.bi-explicit-fill::before { content: "\f69b"; } +.bi-explicit::before { content: "\f69c"; } +.bi-git::before { content: "\f69d"; } +.bi-infinity::before { content: "\f69e"; } +.bi-list-columns-reverse::before { content: "\f69f"; } +.bi-list-columns::before { content: "\f6a0"; } +.bi-meta::before { content: "\f6a1"; } +.bi-nintendo-switch::before { content: "\f6a4"; } +.bi-pc-display-horizontal::before { content: "\f6a5"; } +.bi-pc-display::before { content: "\f6a6"; } +.bi-pc-horizontal::before { content: "\f6a7"; } +.bi-pc::before { content: "\f6a8"; } +.bi-playstation::before { content: "\f6a9"; } +.bi-plus-slash-minus::before { content: "\f6aa"; } +.bi-projector-fill::before { content: "\f6ab"; } +.bi-projector::before { content: "\f6ac"; } +.bi-qr-code-scan::before { content: "\f6ad"; } +.bi-qr-code::before { content: "\f6ae"; } +.bi-quora::before { content: "\f6af"; } +.bi-quote::before { content: "\f6b0"; } +.bi-robot::before { content: "\f6b1"; } +.bi-send-check-fill::before { content: "\f6b2"; } +.bi-send-check::before { content: "\f6b3"; } +.bi-send-dash-fill::before { content: "\f6b4"; } +.bi-send-dash::before { content: "\f6b5"; } +.bi-send-exclamation-fill::before { content: "\f6b7"; } +.bi-send-exclamation::before { content: "\f6b8"; } +.bi-send-fill::before { content: "\f6b9"; } +.bi-send-plus-fill::before { content: "\f6ba"; } +.bi-send-plus::before { content: "\f6bb"; } +.bi-send-slash-fill::before { content: "\f6bc"; } +.bi-send-slash::before { content: "\f6bd"; } +.bi-send-x-fill::before { content: "\f6be"; } +.bi-send-x::before { content: "\f6bf"; } +.bi-send::before { content: "\f6c0"; } +.bi-steam::before { content: "\f6c1"; } +.bi-terminal-dash::before { content: "\f6c3"; } +.bi-terminal-plus::before { content: "\f6c4"; } +.bi-terminal-split::before { content: "\f6c5"; } +.bi-ticket-detailed-fill::before { content: "\f6c6"; } +.bi-ticket-detailed::before { content: "\f6c7"; } +.bi-ticket-fill::before { content: "\f6c8"; } +.bi-ticket-perforated-fill::before { content: "\f6c9"; } +.bi-ticket-perforated::before { content: "\f6ca"; } +.bi-ticket::before { content: "\f6cb"; } +.bi-tiktok::before { content: "\f6cc"; } +.bi-window-dash::before { content: "\f6cd"; } +.bi-window-desktop::before { content: "\f6ce"; } +.bi-window-fullscreen::before { content: "\f6cf"; } +.bi-window-plus::before { content: "\f6d0"; } +.bi-window-split::before { content: "\f6d1"; } +.bi-window-stack::before { content: "\f6d2"; } +.bi-window-x::before { content: "\f6d3"; } +.bi-xbox::before { content: "\f6d4"; } +.bi-ethernet::before { content: "\f6d5"; } +.bi-hdmi-fill::before { content: "\f6d6"; } +.bi-hdmi::before { content: "\f6d7"; } +.bi-usb-c-fill::before { content: "\f6d8"; } +.bi-usb-c::before { content: "\f6d9"; } +.bi-usb-fill::before { content: "\f6da"; } +.bi-usb-plug-fill::before { content: "\f6db"; } +.bi-usb-plug::before { content: "\f6dc"; } +.bi-usb-symbol::before { content: "\f6dd"; } +.bi-usb::before { content: "\f6de"; } +.bi-boombox-fill::before { content: "\f6df"; } +.bi-displayport::before { content: "\f6e1"; } +.bi-gpu-card::before { content: "\f6e2"; } +.bi-memory::before { content: "\f6e3"; } +.bi-modem-fill::before { content: "\f6e4"; } +.bi-modem::before { content: "\f6e5"; } +.bi-motherboard-fill::before { content: "\f6e6"; } +.bi-motherboard::before { content: "\f6e7"; } +.bi-optical-audio-fill::before { content: "\f6e8"; } +.bi-optical-audio::before { content: "\f6e9"; } +.bi-pci-card::before { content: "\f6ea"; } +.bi-router-fill::before { content: "\f6eb"; } +.bi-router::before { content: "\f6ec"; } +.bi-thunderbolt-fill::before { content: "\f6ef"; } +.bi-thunderbolt::before { content: "\f6f0"; } +.bi-usb-drive-fill::before { content: "\f6f1"; } +.bi-usb-drive::before { content: "\f6f2"; } +.bi-usb-micro-fill::before { content: "\f6f3"; } +.bi-usb-micro::before { content: "\f6f4"; } +.bi-usb-mini-fill::before { content: "\f6f5"; } +.bi-usb-mini::before { content: "\f6f6"; } +.bi-cloud-haze2::before { content: "\f6f7"; } +.bi-device-hdd-fill::before { content: "\f6f8"; } +.bi-device-hdd::before { content: "\f6f9"; } +.bi-device-ssd-fill::before { content: "\f6fa"; } +.bi-device-ssd::before { content: "\f6fb"; } +.bi-displayport-fill::before { content: "\f6fc"; } +.bi-mortarboard-fill::before { content: "\f6fd"; } +.bi-mortarboard::before { content: "\f6fe"; } +.bi-terminal-x::before { content: "\f6ff"; } +.bi-arrow-through-heart-fill::before { content: "\f700"; } +.bi-arrow-through-heart::before { content: "\f701"; } +.bi-badge-sd-fill::before { content: "\f702"; } +.bi-badge-sd::before { content: "\f703"; } +.bi-bag-heart-fill::before { content: "\f704"; } +.bi-bag-heart::before { content: "\f705"; } +.bi-balloon-fill::before { content: "\f706"; } +.bi-balloon-heart-fill::before { content: "\f707"; } +.bi-balloon-heart::before { content: "\f708"; } +.bi-balloon::before { content: "\f709"; } +.bi-box2-fill::before { content: "\f70a"; } +.bi-box2-heart-fill::before { content: "\f70b"; } +.bi-box2-heart::before { content: "\f70c"; } +.bi-box2::before { content: "\f70d"; } +.bi-braces-asterisk::before { content: "\f70e"; } +.bi-calendar-heart-fill::before { content: "\f70f"; } +.bi-calendar-heart::before { content: "\f710"; } +.bi-calendar2-heart-fill::before { content: "\f711"; } +.bi-calendar2-heart::before { content: "\f712"; } +.bi-chat-heart-fill::before { content: "\f713"; } +.bi-chat-heart::before { content: "\f714"; } +.bi-chat-left-heart-fill::before { content: "\f715"; } +.bi-chat-left-heart::before { content: "\f716"; } +.bi-chat-right-heart-fill::before { content: "\f717"; } +.bi-chat-right-heart::before { content: "\f718"; } +.bi-chat-square-heart-fill::before { content: "\f719"; } +.bi-chat-square-heart::before { content: "\f71a"; } +.bi-clipboard-check-fill::before { content: "\f71b"; } +.bi-clipboard-data-fill::before { content: "\f71c"; } +.bi-clipboard-fill::before { content: "\f71d"; } +.bi-clipboard-heart-fill::before { content: "\f71e"; } +.bi-clipboard-heart::before { content: "\f71f"; } +.bi-clipboard-minus-fill::before { content: "\f720"; } +.bi-clipboard-plus-fill::before { content: "\f721"; } +.bi-clipboard-pulse::before { content: "\f722"; } +.bi-clipboard-x-fill::before { content: "\f723"; } +.bi-clipboard2-check-fill::before { content: "\f724"; } +.bi-clipboard2-check::before { content: "\f725"; } +.bi-clipboard2-data-fill::before { content: "\f726"; } +.bi-clipboard2-data::before { content: "\f727"; } +.bi-clipboard2-fill::before { content: "\f728"; } +.bi-clipboard2-heart-fill::before { content: "\f729"; } +.bi-clipboard2-heart::before { content: "\f72a"; } +.bi-clipboard2-minus-fill::before { content: "\f72b"; } +.bi-clipboard2-minus::before { content: "\f72c"; } +.bi-clipboard2-plus-fill::before { content: "\f72d"; } +.bi-clipboard2-plus::before { content: "\f72e"; } +.bi-clipboard2-pulse-fill::before { content: "\f72f"; } +.bi-clipboard2-pulse::before { content: "\f730"; } +.bi-clipboard2-x-fill::before { content: "\f731"; } +.bi-clipboard2-x::before { content: "\f732"; } +.bi-clipboard2::before { content: "\f733"; } +.bi-emoji-kiss-fill::before { content: "\f734"; } +.bi-emoji-kiss::before { content: "\f735"; } +.bi-envelope-heart-fill::before { content: "\f736"; } +.bi-envelope-heart::before { content: "\f737"; } +.bi-envelope-open-heart-fill::before { content: "\f738"; } +.bi-envelope-open-heart::before { content: "\f739"; } +.bi-envelope-paper-fill::before { content: "\f73a"; } +.bi-envelope-paper-heart-fill::before { content: "\f73b"; } +.bi-envelope-paper-heart::before { content: "\f73c"; } +.bi-envelope-paper::before { content: "\f73d"; } +.bi-filetype-aac::before { content: "\f73e"; } +.bi-filetype-ai::before { content: "\f73f"; } +.bi-filetype-bmp::before { content: "\f740"; } +.bi-filetype-cs::before { content: "\f741"; } +.bi-filetype-css::before { content: "\f742"; } +.bi-filetype-csv::before { content: "\f743"; } +.bi-filetype-doc::before { content: "\f744"; } +.bi-filetype-docx::before { content: "\f745"; } +.bi-filetype-exe::before { content: "\f746"; } +.bi-filetype-gif::before { content: "\f747"; } +.bi-filetype-heic::before { content: "\f748"; } +.bi-filetype-html::before { content: "\f749"; } +.bi-filetype-java::before { content: "\f74a"; } +.bi-filetype-jpg::before { content: "\f74b"; } +.bi-filetype-js::before { content: "\f74c"; } +.bi-filetype-jsx::before { content: "\f74d"; } +.bi-filetype-key::before { content: "\f74e"; } +.bi-filetype-m4p::before { content: "\f74f"; } +.bi-filetype-md::before { content: "\f750"; } +.bi-filetype-mdx::before { content: "\f751"; } +.bi-filetype-mov::before { content: "\f752"; } +.bi-filetype-mp3::before { content: "\f753"; } +.bi-filetype-mp4::before { content: "\f754"; } +.bi-filetype-otf::before { content: "\f755"; } +.bi-filetype-pdf::before { content: "\f756"; } +.bi-filetype-php::before { content: "\f757"; } +.bi-filetype-png::before { content: "\f758"; } +.bi-filetype-ppt::before { content: "\f75a"; } +.bi-filetype-psd::before { content: "\f75b"; } +.bi-filetype-py::before { content: "\f75c"; } +.bi-filetype-raw::before { content: "\f75d"; } +.bi-filetype-rb::before { content: "\f75e"; } +.bi-filetype-sass::before { content: "\f75f"; } +.bi-filetype-scss::before { content: "\f760"; } +.bi-filetype-sh::before { content: "\f761"; } +.bi-filetype-svg::before { content: "\f762"; } +.bi-filetype-tiff::before { content: "\f763"; } +.bi-filetype-tsx::before { content: "\f764"; } +.bi-filetype-ttf::before { content: "\f765"; } +.bi-filetype-txt::before { content: "\f766"; } +.bi-filetype-wav::before { content: "\f767"; } +.bi-filetype-woff::before { content: "\f768"; } +.bi-filetype-xls::before { content: "\f76a"; } +.bi-filetype-xml::before { content: "\f76b"; } +.bi-filetype-yml::before { content: "\f76c"; } +.bi-heart-arrow::before { content: "\f76d"; } +.bi-heart-pulse-fill::before { content: "\f76e"; } +.bi-heart-pulse::before { content: "\f76f"; } +.bi-heartbreak-fill::before { content: "\f770"; } +.bi-heartbreak::before { content: "\f771"; } +.bi-hearts::before { content: "\f772"; } +.bi-hospital-fill::before { content: "\f773"; } +.bi-hospital::before { content: "\f774"; } +.bi-house-heart-fill::before { content: "\f775"; } +.bi-house-heart::before { content: "\f776"; } +.bi-incognito::before { content: "\f777"; } +.bi-magnet-fill::before { content: "\f778"; } +.bi-magnet::before { content: "\f779"; } +.bi-person-heart::before { content: "\f77a"; } +.bi-person-hearts::before { content: "\f77b"; } +.bi-phone-flip::before { content: "\f77c"; } +.bi-plugin::before { content: "\f77d"; } +.bi-postage-fill::before { content: "\f77e"; } +.bi-postage-heart-fill::before { content: "\f77f"; } +.bi-postage-heart::before { content: "\f780"; } +.bi-postage::before { content: "\f781"; } +.bi-postcard-fill::before { content: "\f782"; } +.bi-postcard-heart-fill::before { content: "\f783"; } +.bi-postcard-heart::before { content: "\f784"; } +.bi-postcard::before { content: "\f785"; } +.bi-search-heart-fill::before { content: "\f786"; } +.bi-search-heart::before { content: "\f787"; } +.bi-sliders2-vertical::before { content: "\f788"; } +.bi-sliders2::before { content: "\f789"; } +.bi-trash3-fill::before { content: "\f78a"; } +.bi-trash3::before { content: "\f78b"; } +.bi-valentine::before { content: "\f78c"; } +.bi-valentine2::before { content: "\f78d"; } +.bi-wrench-adjustable-circle-fill::before { content: "\f78e"; } +.bi-wrench-adjustable-circle::before { content: "\f78f"; } +.bi-wrench-adjustable::before { content: "\f790"; } +.bi-filetype-json::before { content: "\f791"; } +.bi-filetype-pptx::before { content: "\f792"; } +.bi-filetype-xlsx::before { content: "\f793"; } +.bi-1-circle-fill::before { content: "\f796"; } +.bi-1-circle::before { content: "\f797"; } +.bi-1-square-fill::before { content: "\f798"; } +.bi-1-square::before { content: "\f799"; } +.bi-2-circle-fill::before { content: "\f79c"; } +.bi-2-circle::before { content: "\f79d"; } +.bi-2-square-fill::before { content: "\f79e"; } +.bi-2-square::before { content: "\f79f"; } +.bi-3-circle-fill::before { content: "\f7a2"; } +.bi-3-circle::before { content: "\f7a3"; } +.bi-3-square-fill::before { content: "\f7a4"; } +.bi-3-square::before { content: "\f7a5"; } +.bi-4-circle-fill::before { content: "\f7a8"; } +.bi-4-circle::before { content: "\f7a9"; } +.bi-4-square-fill::before { content: "\f7aa"; } +.bi-4-square::before { content: "\f7ab"; } +.bi-5-circle-fill::before { content: "\f7ae"; } +.bi-5-circle::before { content: "\f7af"; } +.bi-5-square-fill::before { content: "\f7b0"; } +.bi-5-square::before { content: "\f7b1"; } +.bi-6-circle-fill::before { content: "\f7b4"; } +.bi-6-circle::before { content: "\f7b5"; } +.bi-6-square-fill::before { content: "\f7b6"; } +.bi-6-square::before { content: "\f7b7"; } +.bi-7-circle-fill::before { content: "\f7ba"; } +.bi-7-circle::before { content: "\f7bb"; } +.bi-7-square-fill::before { content: "\f7bc"; } +.bi-7-square::before { content: "\f7bd"; } +.bi-8-circle-fill::before { content: "\f7c0"; } +.bi-8-circle::before { content: "\f7c1"; } +.bi-8-square-fill::before { content: "\f7c2"; } +.bi-8-square::before { content: "\f7c3"; } +.bi-9-circle-fill::before { content: "\f7c6"; } +.bi-9-circle::before { content: "\f7c7"; } +.bi-9-square-fill::before { content: "\f7c8"; } +.bi-9-square::before { content: "\f7c9"; } +.bi-airplane-engines-fill::before { content: "\f7ca"; } +.bi-airplane-engines::before { content: "\f7cb"; } +.bi-airplane-fill::before { content: "\f7cc"; } +.bi-airplane::before { content: "\f7cd"; } +.bi-alexa::before { content: "\f7ce"; } +.bi-alipay::before { content: "\f7cf"; } +.bi-android::before { content: "\f7d0"; } +.bi-android2::before { content: "\f7d1"; } +.bi-box-fill::before { content: "\f7d2"; } +.bi-box-seam-fill::before { content: "\f7d3"; } +.bi-browser-chrome::before { content: "\f7d4"; } +.bi-browser-edge::before { content: "\f7d5"; } +.bi-browser-firefox::before { content: "\f7d6"; } +.bi-browser-safari::before { content: "\f7d7"; } +.bi-c-circle-fill::before { content: "\f7da"; } +.bi-c-circle::before { content: "\f7db"; } +.bi-c-square-fill::before { content: "\f7dc"; } +.bi-c-square::before { content: "\f7dd"; } +.bi-capsule-pill::before { content: "\f7de"; } +.bi-capsule::before { content: "\f7df"; } +.bi-car-front-fill::before { content: "\f7e0"; } +.bi-car-front::before { content: "\f7e1"; } +.bi-cassette-fill::before { content: "\f7e2"; } +.bi-cassette::before { content: "\f7e3"; } +.bi-cc-circle-fill::before { content: "\f7e6"; } +.bi-cc-circle::before { content: "\f7e7"; } +.bi-cc-square-fill::before { content: "\f7e8"; } +.bi-cc-square::before { content: "\f7e9"; } +.bi-cup-hot-fill::before { content: "\f7ea"; } +.bi-cup-hot::before { content: "\f7eb"; } +.bi-currency-rupee::before { content: "\f7ec"; } +.bi-dropbox::before { content: "\f7ed"; } +.bi-escape::before { content: "\f7ee"; } +.bi-fast-forward-btn-fill::before { content: "\f7ef"; } +.bi-fast-forward-btn::before { content: "\f7f0"; } +.bi-fast-forward-circle-fill::before { content: "\f7f1"; } +.bi-fast-forward-circle::before { content: "\f7f2"; } +.bi-fast-forward-fill::before { content: "\f7f3"; } +.bi-fast-forward::before { content: "\f7f4"; } +.bi-filetype-sql::before { content: "\f7f5"; } +.bi-fire::before { content: "\f7f6"; } +.bi-google-play::before { content: "\f7f7"; } +.bi-h-circle-fill::before { content: "\f7fa"; } +.bi-h-circle::before { content: "\f7fb"; } +.bi-h-square-fill::before { content: "\f7fc"; } +.bi-h-square::before { content: "\f7fd"; } +.bi-indent::before { content: "\f7fe"; } +.bi-lungs-fill::before { content: "\f7ff"; } +.bi-lungs::before { content: "\f800"; } +.bi-microsoft-teams::before { content: "\f801"; } +.bi-p-circle-fill::before { content: "\f804"; } +.bi-p-circle::before { content: "\f805"; } +.bi-p-square-fill::before { content: "\f806"; } +.bi-p-square::before { content: "\f807"; } +.bi-pass-fill::before { content: "\f808"; } +.bi-pass::before { content: "\f809"; } +.bi-prescription::before { content: "\f80a"; } +.bi-prescription2::before { content: "\f80b"; } +.bi-r-circle-fill::before { content: "\f80e"; } +.bi-r-circle::before { content: "\f80f"; } +.bi-r-square-fill::before { content: "\f810"; } +.bi-r-square::before { content: "\f811"; } +.bi-repeat-1::before { content: "\f812"; } +.bi-repeat::before { content: "\f813"; } +.bi-rewind-btn-fill::before { content: "\f814"; } +.bi-rewind-btn::before { content: "\f815"; } +.bi-rewind-circle-fill::before { content: "\f816"; } +.bi-rewind-circle::before { content: "\f817"; } +.bi-rewind-fill::before { content: "\f818"; } +.bi-rewind::before { content: "\f819"; } +.bi-train-freight-front-fill::before { content: "\f81a"; } +.bi-train-freight-front::before { content: "\f81b"; } +.bi-train-front-fill::before { content: "\f81c"; } +.bi-train-front::before { content: "\f81d"; } +.bi-train-lightrail-front-fill::before { content: "\f81e"; } +.bi-train-lightrail-front::before { content: "\f81f"; } +.bi-truck-front-fill::before { content: "\f820"; } +.bi-truck-front::before { content: "\f821"; } +.bi-ubuntu::before { content: "\f822"; } +.bi-unindent::before { content: "\f823"; } +.bi-unity::before { content: "\f824"; } +.bi-universal-access-circle::before { content: "\f825"; } +.bi-universal-access::before { content: "\f826"; } +.bi-virus::before { content: "\f827"; } +.bi-virus2::before { content: "\f828"; } +.bi-wechat::before { content: "\f829"; } +.bi-yelp::before { content: "\f82a"; } +.bi-sign-stop-fill::before { content: "\f82b"; } +.bi-sign-stop-lights-fill::before { content: "\f82c"; } +.bi-sign-stop-lights::before { content: "\f82d"; } +.bi-sign-stop::before { content: "\f82e"; } +.bi-sign-turn-left-fill::before { content: "\f82f"; } +.bi-sign-turn-left::before { content: "\f830"; } +.bi-sign-turn-right-fill::before { content: "\f831"; } +.bi-sign-turn-right::before { content: "\f832"; } +.bi-sign-turn-slight-left-fill::before { content: "\f833"; } +.bi-sign-turn-slight-left::before { content: "\f834"; } +.bi-sign-turn-slight-right-fill::before { content: "\f835"; } +.bi-sign-turn-slight-right::before { content: "\f836"; } +.bi-sign-yield-fill::before { content: "\f837"; } +.bi-sign-yield::before { content: "\f838"; } +.bi-ev-station-fill::before { content: "\f839"; } +.bi-ev-station::before { content: "\f83a"; } +.bi-fuel-pump-diesel-fill::before { content: "\f83b"; } +.bi-fuel-pump-diesel::before { content: "\f83c"; } +.bi-fuel-pump-fill::before { content: "\f83d"; } +.bi-fuel-pump::before { content: "\f83e"; } +.bi-0-circle-fill::before { content: "\f83f"; } +.bi-0-circle::before { content: "\f840"; } +.bi-0-square-fill::before { content: "\f841"; } +.bi-0-square::before { content: "\f842"; } +.bi-rocket-fill::before { content: "\f843"; } +.bi-rocket-takeoff-fill::before { content: "\f844"; } +.bi-rocket-takeoff::before { content: "\f845"; } +.bi-rocket::before { content: "\f846"; } +.bi-stripe::before { content: "\f847"; } +.bi-subscript::before { content: "\f848"; } +.bi-superscript::before { content: "\f849"; } +.bi-trello::before { content: "\f84a"; } +.bi-envelope-at-fill::before { content: "\f84b"; } +.bi-envelope-at::before { content: "\f84c"; } +.bi-regex::before { content: "\f84d"; } +.bi-text-wrap::before { content: "\f84e"; } +.bi-sign-dead-end-fill::before { content: "\f84f"; } +.bi-sign-dead-end::before { content: "\f850"; } +.bi-sign-do-not-enter-fill::before { content: "\f851"; } +.bi-sign-do-not-enter::before { content: "\f852"; } +.bi-sign-intersection-fill::before { content: "\f853"; } +.bi-sign-intersection-side-fill::before { content: "\f854"; } +.bi-sign-intersection-side::before { content: "\f855"; } +.bi-sign-intersection-t-fill::before { content: "\f856"; } +.bi-sign-intersection-t::before { content: "\f857"; } +.bi-sign-intersection-y-fill::before { content: "\f858"; } +.bi-sign-intersection-y::before { content: "\f859"; } +.bi-sign-intersection::before { content: "\f85a"; } +.bi-sign-merge-left-fill::before { content: "\f85b"; } +.bi-sign-merge-left::before { content: "\f85c"; } +.bi-sign-merge-right-fill::before { content: "\f85d"; } +.bi-sign-merge-right::before { content: "\f85e"; } +.bi-sign-no-left-turn-fill::before { content: "\f85f"; } +.bi-sign-no-left-turn::before { content: "\f860"; } +.bi-sign-no-parking-fill::before { content: "\f861"; } +.bi-sign-no-parking::before { content: "\f862"; } +.bi-sign-no-right-turn-fill::before { content: "\f863"; } +.bi-sign-no-right-turn::before { content: "\f864"; } +.bi-sign-railroad-fill::before { content: "\f865"; } +.bi-sign-railroad::before { content: "\f866"; } +.bi-building-add::before { content: "\f867"; } +.bi-building-check::before { content: "\f868"; } +.bi-building-dash::before { content: "\f869"; } +.bi-building-down::before { content: "\f86a"; } +.bi-building-exclamation::before { content: "\f86b"; } +.bi-building-fill-add::before { content: "\f86c"; } +.bi-building-fill-check::before { content: "\f86d"; } +.bi-building-fill-dash::before { content: "\f86e"; } +.bi-building-fill-down::before { content: "\f86f"; } +.bi-building-fill-exclamation::before { content: "\f870"; } +.bi-building-fill-gear::before { content: "\f871"; } +.bi-building-fill-lock::before { content: "\f872"; } +.bi-building-fill-slash::before { content: "\f873"; } +.bi-building-fill-up::before { content: "\f874"; } +.bi-building-fill-x::before { content: "\f875"; } +.bi-building-fill::before { content: "\f876"; } +.bi-building-gear::before { content: "\f877"; } +.bi-building-lock::before { content: "\f878"; } +.bi-building-slash::before { content: "\f879"; } +.bi-building-up::before { content: "\f87a"; } +.bi-building-x::before { content: "\f87b"; } +.bi-buildings-fill::before { content: "\f87c"; } +.bi-buildings::before { content: "\f87d"; } +.bi-bus-front-fill::before { content: "\f87e"; } +.bi-bus-front::before { content: "\f87f"; } +.bi-ev-front-fill::before { content: "\f880"; } +.bi-ev-front::before { content: "\f881"; } +.bi-globe-americas::before { content: "\f882"; } +.bi-globe-asia-australia::before { content: "\f883"; } +.bi-globe-central-south-asia::before { content: "\f884"; } +.bi-globe-europe-africa::before { content: "\f885"; } +.bi-house-add-fill::before { content: "\f886"; } +.bi-house-add::before { content: "\f887"; } +.bi-house-check-fill::before { content: "\f888"; } +.bi-house-check::before { content: "\f889"; } +.bi-house-dash-fill::before { content: "\f88a"; } +.bi-house-dash::before { content: "\f88b"; } +.bi-house-down-fill::before { content: "\f88c"; } +.bi-house-down::before { content: "\f88d"; } +.bi-house-exclamation-fill::before { content: "\f88e"; } +.bi-house-exclamation::before { content: "\f88f"; } +.bi-house-gear-fill::before { content: "\f890"; } +.bi-house-gear::before { content: "\f891"; } +.bi-house-lock-fill::before { content: "\f892"; } +.bi-house-lock::before { content: "\f893"; } +.bi-house-slash-fill::before { content: "\f894"; } +.bi-house-slash::before { content: "\f895"; } +.bi-house-up-fill::before { content: "\f896"; } +.bi-house-up::before { content: "\f897"; } +.bi-house-x-fill::before { content: "\f898"; } +.bi-house-x::before { content: "\f899"; } +.bi-person-add::before { content: "\f89a"; } +.bi-person-down::before { content: "\f89b"; } +.bi-person-exclamation::before { content: "\f89c"; } +.bi-person-fill-add::before { content: "\f89d"; } +.bi-person-fill-check::before { content: "\f89e"; } +.bi-person-fill-dash::before { content: "\f89f"; } +.bi-person-fill-down::before { content: "\f8a0"; } +.bi-person-fill-exclamation::before { content: "\f8a1"; } +.bi-person-fill-gear::before { content: "\f8a2"; } +.bi-person-fill-lock::before { content: "\f8a3"; } +.bi-person-fill-slash::before { content: "\f8a4"; } +.bi-person-fill-up::before { content: "\f8a5"; } +.bi-person-fill-x::before { content: "\f8a6"; } +.bi-person-gear::before { content: "\f8a7"; } +.bi-person-lock::before { content: "\f8a8"; } +.bi-person-slash::before { content: "\f8a9"; } +.bi-person-up::before { content: "\f8aa"; } +.bi-scooter::before { content: "\f8ab"; } +.bi-taxi-front-fill::before { content: "\f8ac"; } +.bi-taxi-front::before { content: "\f8ad"; } +.bi-amd::before { content: "\f8ae"; } +.bi-database-add::before { content: "\f8af"; } +.bi-database-check::before { content: "\f8b0"; } +.bi-database-dash::before { content: "\f8b1"; } +.bi-database-down::before { content: "\f8b2"; } +.bi-database-exclamation::before { content: "\f8b3"; } +.bi-database-fill-add::before { content: "\f8b4"; } +.bi-database-fill-check::before { content: "\f8b5"; } +.bi-database-fill-dash::before { content: "\f8b6"; } +.bi-database-fill-down::before { content: "\f8b7"; } +.bi-database-fill-exclamation::before { content: "\f8b8"; } +.bi-database-fill-gear::before { content: "\f8b9"; } +.bi-database-fill-lock::before { content: "\f8ba"; } +.bi-database-fill-slash::before { content: "\f8bb"; } +.bi-database-fill-up::before { content: "\f8bc"; } +.bi-database-fill-x::before { content: "\f8bd"; } +.bi-database-fill::before { content: "\f8be"; } +.bi-database-gear::before { content: "\f8bf"; } +.bi-database-lock::before { content: "\f8c0"; } +.bi-database-slash::before { content: "\f8c1"; } +.bi-database-up::before { content: "\f8c2"; } +.bi-database-x::before { content: "\f8c3"; } +.bi-database::before { content: "\f8c4"; } +.bi-houses-fill::before { content: "\f8c5"; } +.bi-houses::before { content: "\f8c6"; } +.bi-nvidia::before { content: "\f8c7"; } +.bi-person-vcard-fill::before { content: "\f8c8"; } +.bi-person-vcard::before { content: "\f8c9"; } +.bi-sina-weibo::before { content: "\f8ca"; } +.bi-tencent-qq::before { content: "\f8cb"; } +.bi-wikipedia::before { content: "\f8cc"; } +.bi-alphabet-uppercase::before { content: "\f2a5"; } +.bi-alphabet::before { content: "\f68a"; } +.bi-amazon::before { content: "\f68d"; } +.bi-arrows-collapse-vertical::before { content: "\f690"; } +.bi-arrows-expand-vertical::before { content: "\f695"; } +.bi-arrows-vertical::before { content: "\f698"; } +.bi-arrows::before { content: "\f6a2"; } +.bi-ban-fill::before { content: "\f6a3"; } +.bi-ban::before { content: "\f6b6"; } +.bi-bing::before { content: "\f6c2"; } +.bi-cake::before { content: "\f6e0"; } +.bi-cake2::before { content: "\f6ed"; } +.bi-cookie::before { content: "\f6ee"; } +.bi-copy::before { content: "\f759"; } +.bi-crosshair::before { content: "\f769"; } +.bi-crosshair2::before { content: "\f794"; } +.bi-emoji-astonished-fill::before { content: "\f795"; } +.bi-emoji-astonished::before { content: "\f79a"; } +.bi-emoji-grimace-fill::before { content: "\f79b"; } +.bi-emoji-grimace::before { content: "\f7a0"; } +.bi-emoji-grin-fill::before { content: "\f7a1"; } +.bi-emoji-grin::before { content: "\f7a6"; } +.bi-emoji-surprise-fill::before { content: "\f7a7"; } +.bi-emoji-surprise::before { content: "\f7ac"; } +.bi-emoji-tear-fill::before { content: "\f7ad"; } +.bi-emoji-tear::before { content: "\f7b2"; } +.bi-envelope-arrow-down-fill::before { content: "\f7b3"; } +.bi-envelope-arrow-down::before { content: "\f7b8"; } +.bi-envelope-arrow-up-fill::before { content: "\f7b9"; } +.bi-envelope-arrow-up::before { content: "\f7be"; } +.bi-feather::before { content: "\f7bf"; } +.bi-feather2::before { content: "\f7c4"; } +.bi-floppy-fill::before { content: "\f7c5"; } +.bi-floppy::before { content: "\f7d8"; } +.bi-floppy2-fill::before { content: "\f7d9"; } +.bi-floppy2::before { content: "\f7e4"; } +.bi-gitlab::before { content: "\f7e5"; } +.bi-highlighter::before { content: "\f7f8"; } +.bi-marker-tip::before { content: "\f802"; } +.bi-nvme-fill::before { content: "\f803"; } +.bi-nvme::before { content: "\f80c"; } +.bi-opencollective::before { content: "\f80d"; } +.bi-pci-card-network::before { content: "\f8cd"; } +.bi-pci-card-sound::before { content: "\f8ce"; } +.bi-radar::before { content: "\f8cf"; } +.bi-send-arrow-down-fill::before { content: "\f8d0"; } +.bi-send-arrow-down::before { content: "\f8d1"; } +.bi-send-arrow-up-fill::before { content: "\f8d2"; } +.bi-send-arrow-up::before { content: "\f8d3"; } +.bi-sim-slash-fill::before { content: "\f8d4"; } +.bi-sim-slash::before { content: "\f8d5"; } +.bi-sourceforge::before { content: "\f8d6"; } +.bi-substack::before { content: "\f8d7"; } +.bi-threads-fill::before { content: "\f8d8"; } +.bi-threads::before { content: "\f8d9"; } +.bi-transparency::before { content: "\f8da"; } +.bi-twitter-x::before { content: "\f8db"; } +.bi-type-h4::before { content: "\f8dc"; } +.bi-type-h5::before { content: "\f8dd"; } +.bi-type-h6::before { content: "\f8de"; } +.bi-backpack-fill::before { content: "\f8df"; } +.bi-backpack::before { content: "\f8e0"; } +.bi-backpack2-fill::before { content: "\f8e1"; } +.bi-backpack2::before { content: "\f8e2"; } +.bi-backpack3-fill::before { content: "\f8e3"; } +.bi-backpack3::before { content: "\f8e4"; } +.bi-backpack4-fill::before { content: "\f8e5"; } +.bi-backpack4::before { content: "\f8e6"; } +.bi-brilliance::before { content: "\f8e7"; } +.bi-cake-fill::before { content: "\f8e8"; } +.bi-cake2-fill::before { content: "\f8e9"; } +.bi-duffle-fill::before { content: "\f8ea"; } +.bi-duffle::before { content: "\f8eb"; } +.bi-exposure::before { content: "\f8ec"; } +.bi-gender-neuter::before { content: "\f8ed"; } +.bi-highlights::before { content: "\f8ee"; } +.bi-luggage-fill::before { content: "\f8ef"; } +.bi-luggage::before { content: "\f8f0"; } +.bi-mailbox-flag::before { content: "\f8f1"; } +.bi-mailbox2-flag::before { content: "\f8f2"; } +.bi-noise-reduction::before { content: "\f8f3"; } +.bi-passport-fill::before { content: "\f8f4"; } +.bi-passport::before { content: "\f8f5"; } +.bi-person-arms-up::before { content: "\f8f6"; } +.bi-person-raised-hand::before { content: "\f8f7"; } +.bi-person-standing-dress::before { content: "\f8f8"; } +.bi-person-standing::before { content: "\f8f9"; } +.bi-person-walking::before { content: "\f8fa"; } +.bi-person-wheelchair::before { content: "\f8fb"; } +.bi-shadows::before { content: "\f8fc"; } +.bi-suitcase-fill::before { content: "\f8fd"; } +.bi-suitcase-lg-fill::before { content: "\f8fe"; } +.bi-suitcase-lg::before { content: "\f8ff"; } +.bi-suitcase::before { content: "\f900"; } +.bi-suitcase2-fill::before { content: "\f901"; } +.bi-suitcase2::before { content: "\f902"; } +.bi-vignette::before { content: "\f903"; } diff --git a/site_libs/bootstrap/bootstrap-icons.woff b/site_libs/bootstrap/bootstrap-icons.woff new file mode 100644 index 0000000..dbeeb05 Binary files /dev/null and b/site_libs/bootstrap/bootstrap-icons.woff differ diff --git a/site_libs/bootstrap/bootstrap.min.css b/site_libs/bootstrap/bootstrap.min.css new file mode 100644 index 0000000..37d5ec7 --- /dev/null +++ b/site_libs/bootstrap/bootstrap.min.css @@ -0,0 +1,12 @@ +/*! + * Bootstrap v5.3.1 (https://getbootstrap.com/) + * Copyright 2011-2023 The Bootstrap Authors + * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) + */@import"https://fonts.googleapis.com/css2?family=Roboto:wght@400;500;700&display=swap";:root,[data-bs-theme=light]{--bs-blue: #325d88;--bs-indigo: #6610f2;--bs-purple: #6f42c1;--bs-pink: #e83e8c;--bs-red: #d9534f;--bs-orange: #f47c3c;--bs-yellow: #ffc107;--bs-green: #93c54b;--bs-teal: #20c997;--bs-cyan: #29abe0;--bs-black: #000;--bs-white: #fff;--bs-gray: #6c757d;--bs-gray-dark: #343a40;--bs-gray-100: #f8f9fa;--bs-gray-200: #f8f5f0;--bs-gray-300: #dee2e6;--bs-gray-400: #ced4da;--bs-gray-500: #adb5bd;--bs-gray-600: #6c757d;--bs-gray-700: #495057;--bs-gray-800: #343a40;--bs-gray-900: #212529;--bs-default: #6c757d;--bs-primary: #325d88;--bs-secondary: #6c757d;--bs-success: #93c54b;--bs-info: #29abe0;--bs-warning: #f47c3c;--bs-danger: #d9534f;--bs-light: #f8f5f0;--bs-dark: #343a40;--bs-default-rgb: 108, 117, 125;--bs-primary-rgb: 50, 93, 136;--bs-secondary-rgb: 108, 117, 125;--bs-success-rgb: 147, 197, 75;--bs-info-rgb: 41, 171, 224;--bs-warning-rgb: 244, 124, 60;--bs-danger-rgb: 217, 83, 79;--bs-light-rgb: 248, 245, 240;--bs-dark-rgb: 52, 58, 64;--bs-primary-text-emphasis: #142536;--bs-secondary-text-emphasis: #2b2f32;--bs-success-text-emphasis: #3b4f1e;--bs-info-text-emphasis: #10445a;--bs-warning-text-emphasis: #623218;--bs-danger-text-emphasis: #572120;--bs-light-text-emphasis: #495057;--bs-dark-text-emphasis: #495057;--bs-primary-bg-subtle: #d6dfe7;--bs-secondary-bg-subtle: #e2e3e5;--bs-success-bg-subtle: #e9f3db;--bs-info-bg-subtle: #d4eef9;--bs-warning-bg-subtle: #fde5d8;--bs-danger-bg-subtle: #f7dddc;--bs-light-bg-subtle: #fcfcfd;--bs-dark-bg-subtle: #ced4da;--bs-primary-border-subtle: #adbecf;--bs-secondary-border-subtle: #c4c8cb;--bs-success-border-subtle: #d4e8b7;--bs-info-border-subtle: #a9ddf3;--bs-warning-border-subtle: #fbcbb1;--bs-danger-border-subtle: #f0bab9;--bs-light-border-subtle: #f8f5f0;--bs-dark-border-subtle: #adb5bd;--bs-white-rgb: 255, 255, 255;--bs-black-rgb: 0, 0, 0;--bs-font-sans-serif: Roboto, -apple-system, BlinkMacSystemFont, "Segoe UI", "Helvetica Neue", Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";--bs-font-monospace: SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace;--bs-gradient: linear-gradient(180deg, rgba(255, 255, 255, 0.15), rgba(255, 255, 255, 0));--bs-root-font-size: 17px;--bs-body-font-family: Roboto, -apple-system, BlinkMacSystemFont, "Segoe UI", "Helvetica Neue", Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";--bs-body-font-size:1rem;--bs-body-font-weight: 400;--bs-body-line-height: 1.5;--bs-body-color: #343a40;--bs-body-color-rgb: 52, 58, 64;--bs-body-bg: #fff;--bs-body-bg-rgb: 255, 255, 255;--bs-emphasis-color: #000;--bs-emphasis-color-rgb: 0, 0, 0;--bs-secondary-color: rgba(52, 58, 64, 0.75);--bs-secondary-color-rgb: 52, 58, 64;--bs-secondary-bg: #f8f5f0;--bs-secondary-bg-rgb: 248, 245, 240;--bs-tertiary-color: rgba(52, 58, 64, 0.5);--bs-tertiary-color-rgb: 52, 58, 64;--bs-tertiary-bg: #f8f9fa;--bs-tertiary-bg-rgb: 248, 249, 250;--bs-heading-color: inherit;--bs-link-color: #93c54b;--bs-link-color-rgb: 147, 197, 75;--bs-link-decoration: underline;--bs-link-hover-color: #769e3c;--bs-link-hover-color-rgb: 118, 158, 60;--bs-code-color: #7d12ba;--bs-highlight-bg: #fff3cd;--bs-border-width: 1px;--bs-border-style: solid;--bs-border-color: #dee2e6;--bs-border-color-translucent: rgba(0, 0, 0, 0.175);--bs-border-radius: 0.25rem;--bs-border-radius-sm: 0.2em;--bs-border-radius-lg: 0.5rem;--bs-border-radius-xl: 1rem;--bs-border-radius-xxl: 2rem;--bs-border-radius-2xl: var(--bs-border-radius-xxl);--bs-border-radius-pill: 50rem;--bs-box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);--bs-box-shadow-sm: 0 0.125rem 0.25rem rgba(0, 0, 0, 0.075);--bs-box-shadow-lg: 0 1rem 3rem rgba(0, 0, 0, 0.175);--bs-box-shadow-inset: inset 0 1px 2px rgba(0, 0, 0, 0.075);--bs-focus-ring-width: 0.25rem;--bs-focus-ring-opacity: 0.25;--bs-focus-ring-color: rgba(50, 93, 136, 0.25);--bs-form-valid-color: #93c54b;--bs-form-valid-border-color: #93c54b;--bs-form-invalid-color: #d9534f;--bs-form-invalid-border-color: #d9534f}[data-bs-theme=dark]{color-scheme:dark;--bs-body-color: #dee2e6;--bs-body-color-rgb: 222, 226, 230;--bs-body-bg: #212529;--bs-body-bg-rgb: 33, 37, 41;--bs-emphasis-color: #fff;--bs-emphasis-color-rgb: 255, 255, 255;--bs-secondary-color: rgba(222, 226, 230, 0.75);--bs-secondary-color-rgb: 222, 226, 230;--bs-secondary-bg: #343a40;--bs-secondary-bg-rgb: 52, 58, 64;--bs-tertiary-color: rgba(222, 226, 230, 0.5);--bs-tertiary-color-rgb: 222, 226, 230;--bs-tertiary-bg: #2b3035;--bs-tertiary-bg-rgb: 43, 48, 53;--bs-primary-text-emphasis: #849eb8;--bs-secondary-text-emphasis: #a7acb1;--bs-success-text-emphasis: #bedc93;--bs-info-text-emphasis: #7fcdec;--bs-warning-text-emphasis: #f8b08a;--bs-danger-text-emphasis: #e89895;--bs-light-text-emphasis: #f8f9fa;--bs-dark-text-emphasis: #dee2e6;--bs-primary-bg-subtle: #0a131b;--bs-secondary-bg-subtle: #161719;--bs-success-bg-subtle: #1d270f;--bs-info-bg-subtle: #08222d;--bs-warning-bg-subtle: #31190c;--bs-danger-bg-subtle: #2b1110;--bs-light-bg-subtle: #343a40;--bs-dark-bg-subtle: #1a1d20;--bs-primary-border-subtle: #1e3852;--bs-secondary-border-subtle: #41464b;--bs-success-border-subtle: #58762d;--bs-info-border-subtle: #196786;--bs-warning-border-subtle: #924a24;--bs-danger-border-subtle: #82322f;--bs-light-border-subtle: #495057;--bs-dark-border-subtle: #343a40;--bs-heading-color: inherit;--bs-link-color: #849eb8;--bs-link-hover-color: #9db1c6;--bs-link-color-rgb: 132, 158, 184;--bs-link-hover-color-rgb: 157, 177, 198;--bs-code-color: white;--bs-border-color: #495057;--bs-border-color-translucent: rgba(255, 255, 255, 0.15);--bs-form-valid-color: #bedc93;--bs-form-valid-border-color: #bedc93;--bs-form-invalid-color: #e89895;--bs-form-invalid-border-color: #e89895}*,*::before,*::after{box-sizing:border-box}:root{font-size:var(--bs-root-font-size)}body{margin:0;font-family:var(--bs-body-font-family);font-size:var(--bs-body-font-size);font-weight:var(--bs-body-font-weight);line-height:var(--bs-body-line-height);color:var(--bs-body-color);text-align:var(--bs-body-text-align);background-color:var(--bs-body-bg);-webkit-text-size-adjust:100%;-webkit-tap-highlight-color:rgba(0,0,0,0)}hr{margin:1rem 0;color:inherit;border:0;border-top:1px solid;opacity:.25}h6,.h6,h5,.h5,h4,.h4,h3,.h3,h2,.h2,h1,.h1{margin-top:0;margin-bottom:.5rem;font-weight:400;line-height:1.2;color:var(--bs-heading-color)}h1,.h1{font-size:calc(1.325rem + 0.9vw)}@media(min-width: 1200px){h1,.h1{font-size:2rem}}h2,.h2{font-size:calc(1.29rem + 0.48vw)}@media(min-width: 1200px){h2,.h2{font-size:1.65rem}}h3,.h3{font-size:calc(1.27rem + 0.24vw)}@media(min-width: 1200px){h3,.h3{font-size:1.45rem}}h4,.h4{font-size:1.25rem}h5,.h5{font-size:1.1rem}h6,.h6{font-size:1rem}p{margin-top:0;margin-bottom:1rem}abbr[title]{text-decoration:underline dotted;-webkit-text-decoration:underline dotted;-moz-text-decoration:underline dotted;-ms-text-decoration:underline dotted;-o-text-decoration:underline dotted;cursor:help;text-decoration-skip-ink:none}address{margin-bottom:1rem;font-style:normal;line-height:inherit}ol,ul{padding-left:2rem}ol,ul,dl{margin-top:0;margin-bottom:1rem}ol ol,ul ul,ol ul,ul ol{margin-bottom:0}dt{font-weight:700}dd{margin-bottom:.5rem;margin-left:0}blockquote{margin:0 0 1rem;padding:.625rem 1.25rem;border-left:.25rem solid #f8f5f0}blockquote p:last-child,blockquote ul:last-child,blockquote ol:last-child{margin-bottom:0}b,strong{font-weight:bolder}small,.small{font-size:0.875em}mark,.mark{padding:.1875em;background-color:var(--bs-highlight-bg)}sub,sup{position:relative;font-size:0.75em;line-height:0;vertical-align:baseline}sub{bottom:-0.25em}sup{top:-0.5em}a{color:rgba(var(--bs-link-color-rgb), var(--bs-link-opacity, 1));text-decoration:underline;-webkit-text-decoration:underline;-moz-text-decoration:underline;-ms-text-decoration:underline;-o-text-decoration:underline}a:hover{--bs-link-color-rgb: var(--bs-link-hover-color-rgb)}a:not([href]):not([class]),a:not([href]):not([class]):hover{color:inherit;text-decoration:none}pre,code,kbd,samp{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;font-size:1em}pre{display:block;margin-top:0;margin-bottom:1rem;overflow:auto;font-size:0.875em;color:#000;background-color:#f8f5f0;padding:.5rem;border:1px solid var(--bs-border-color, #dee2e6);border-radius:.25rem}pre code{background-color:rgba(0,0,0,0);font-size:inherit;color:inherit;word-break:normal}code{font-size:0.875em;color:var(--bs-code-color);background-color:#f8f5f0;border-radius:.25rem;padding:.125rem .25rem;word-wrap:break-word}a>code{color:inherit}kbd{padding:.4rem .4rem;font-size:0.875em;color:#fff;background-color:#343a40;border-radius:.2em}kbd kbd{padding:0;font-size:1em}figure{margin:0 0 1rem}img,svg{vertical-align:middle}table{caption-side:bottom;border-collapse:collapse}caption{padding-top:.5rem;padding-bottom:.5rem;color:rgba(52,58,64,.75);text-align:left}th{text-align:inherit;text-align:-webkit-match-parent}thead,tbody,tfoot,tr,td,th{border-color:inherit;border-style:solid;border-width:0}label{display:inline-block}button{border-radius:0}button:focus:not(:focus-visible){outline:0}input,button,select,optgroup,textarea{margin:0;font-family:inherit;font-size:inherit;line-height:inherit}button,select{text-transform:none}[role=button]{cursor:pointer}select{word-wrap:normal}select:disabled{opacity:1}[list]:not([type=date]):not([type=datetime-local]):not([type=month]):not([type=week]):not([type=time])::-webkit-calendar-picker-indicator{display:none !important}button,[type=button],[type=reset],[type=submit]{-webkit-appearance:button}button:not(:disabled),[type=button]:not(:disabled),[type=reset]:not(:disabled),[type=submit]:not(:disabled){cursor:pointer}::-moz-focus-inner{padding:0;border-style:none}textarea{resize:vertical}fieldset{min-width:0;padding:0;margin:0;border:0}legend{float:left;width:100%;padding:0;margin-bottom:.5rem;font-size:calc(1.275rem + 0.3vw);line-height:inherit}@media(min-width: 1200px){legend{font-size:1.5rem}}legend+*{clear:left}::-webkit-datetime-edit-fields-wrapper,::-webkit-datetime-edit-text,::-webkit-datetime-edit-minute,::-webkit-datetime-edit-hour-field,::-webkit-datetime-edit-day-field,::-webkit-datetime-edit-month-field,::-webkit-datetime-edit-year-field{padding:0}::-webkit-inner-spin-button{height:auto}[type=search]{-webkit-appearance:textfield;outline-offset:-2px}::-webkit-search-decoration{-webkit-appearance:none}::-webkit-color-swatch-wrapper{padding:0}::file-selector-button{font:inherit;-webkit-appearance:button}output{display:inline-block}iframe{border:0}summary{display:list-item;cursor:pointer}progress{vertical-align:baseline}[hidden]{display:none !important}.lead{font-size:1.25rem;font-weight:300}.display-1{font-size:calc(1.625rem + 4.5vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-1{font-size:5rem}}.display-2{font-size:calc(1.575rem + 3.9vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-2{font-size:4.5rem}}.display-3{font-size:calc(1.525rem + 3.3vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-3{font-size:4rem}}.display-4{font-size:calc(1.475rem + 2.7vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-4{font-size:3.5rem}}.display-5{font-size:calc(1.425rem + 2.1vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-5{font-size:3rem}}.display-6{font-size:calc(1.375rem + 1.5vw);font-weight:300;line-height:1.2}@media(min-width: 1200px){.display-6{font-size:2.5rem}}.list-unstyled{padding-left:0;list-style:none}.list-inline{padding-left:0;list-style:none}.list-inline-item{display:inline-block}.list-inline-item:not(:last-child){margin-right:.5rem}.initialism{font-size:0.875em;text-transform:uppercase}.blockquote{margin-bottom:1rem;font-size:1.25rem}.blockquote>:last-child{margin-bottom:0}.blockquote-footer{margin-top:-1rem;margin-bottom:1rem;font-size:0.875em;color:#6c757d}.blockquote-footer::before{content:"— "}.img-fluid{max-width:100%;height:auto}.img-thumbnail{padding:.25rem;background-color:#fff;border:1px solid #dee2e6;border-radius:.25rem;max-width:100%;height:auto}.figure{display:inline-block}.figure-img{margin-bottom:.5rem;line-height:1}.figure-caption{font-size:0.875em;color:rgba(52,58,64,.75)}.container,.container-fluid,.container-xxl,.container-xl,.container-lg,.container-md,.container-sm{--bs-gutter-x: 1.5rem;--bs-gutter-y: 0;width:100%;padding-right:calc(var(--bs-gutter-x)*.5);padding-left:calc(var(--bs-gutter-x)*.5);margin-right:auto;margin-left:auto}@media(min-width: 576px){.container-sm,.container{max-width:540px}}@media(min-width: 768px){.container-md,.container-sm,.container{max-width:720px}}@media(min-width: 992px){.container-lg,.container-md,.container-sm,.container{max-width:960px}}@media(min-width: 1200px){.container-xl,.container-lg,.container-md,.container-sm,.container{max-width:1140px}}@media(min-width: 1400px){.container-xxl,.container-xl,.container-lg,.container-md,.container-sm,.container{max-width:1320px}}:root{--bs-breakpoint-xs: 0;--bs-breakpoint-sm: 576px;--bs-breakpoint-md: 768px;--bs-breakpoint-lg: 992px;--bs-breakpoint-xl: 1200px;--bs-breakpoint-xxl: 1400px}.grid{display:grid;grid-template-rows:repeat(var(--bs-rows, 1), 1fr);grid-template-columns:repeat(var(--bs-columns, 12), 1fr);gap:var(--bs-gap, 1.5rem)}.grid .g-col-1{grid-column:auto/span 1}.grid .g-col-2{grid-column:auto/span 2}.grid .g-col-3{grid-column:auto/span 3}.grid .g-col-4{grid-column:auto/span 4}.grid .g-col-5{grid-column:auto/span 5}.grid .g-col-6{grid-column:auto/span 6}.grid .g-col-7{grid-column:auto/span 7}.grid .g-col-8{grid-column:auto/span 8}.grid .g-col-9{grid-column:auto/span 9}.grid .g-col-10{grid-column:auto/span 10}.grid .g-col-11{grid-column:auto/span 11}.grid .g-col-12{grid-column:auto/span 12}.grid .g-start-1{grid-column-start:1}.grid .g-start-2{grid-column-start:2}.grid .g-start-3{grid-column-start:3}.grid .g-start-4{grid-column-start:4}.grid .g-start-5{grid-column-start:5}.grid .g-start-6{grid-column-start:6}.grid .g-start-7{grid-column-start:7}.grid .g-start-8{grid-column-start:8}.grid .g-start-9{grid-column-start:9}.grid .g-start-10{grid-column-start:10}.grid .g-start-11{grid-column-start:11}@media(min-width: 576px){.grid .g-col-sm-1{grid-column:auto/span 1}.grid .g-col-sm-2{grid-column:auto/span 2}.grid .g-col-sm-3{grid-column:auto/span 3}.grid .g-col-sm-4{grid-column:auto/span 4}.grid .g-col-sm-5{grid-column:auto/span 5}.grid .g-col-sm-6{grid-column:auto/span 6}.grid .g-col-sm-7{grid-column:auto/span 7}.grid .g-col-sm-8{grid-column:auto/span 8}.grid .g-col-sm-9{grid-column:auto/span 9}.grid .g-col-sm-10{grid-column:auto/span 10}.grid .g-col-sm-11{grid-column:auto/span 11}.grid .g-col-sm-12{grid-column:auto/span 12}.grid .g-start-sm-1{grid-column-start:1}.grid .g-start-sm-2{grid-column-start:2}.grid .g-start-sm-3{grid-column-start:3}.grid .g-start-sm-4{grid-column-start:4}.grid .g-start-sm-5{grid-column-start:5}.grid .g-start-sm-6{grid-column-start:6}.grid .g-start-sm-7{grid-column-start:7}.grid .g-start-sm-8{grid-column-start:8}.grid .g-start-sm-9{grid-column-start:9}.grid .g-start-sm-10{grid-column-start:10}.grid .g-start-sm-11{grid-column-start:11}}@media(min-width: 768px){.grid .g-col-md-1{grid-column:auto/span 1}.grid .g-col-md-2{grid-column:auto/span 2}.grid .g-col-md-3{grid-column:auto/span 3}.grid .g-col-md-4{grid-column:auto/span 4}.grid .g-col-md-5{grid-column:auto/span 5}.grid .g-col-md-6{grid-column:auto/span 6}.grid .g-col-md-7{grid-column:auto/span 7}.grid .g-col-md-8{grid-column:auto/span 8}.grid .g-col-md-9{grid-column:auto/span 9}.grid .g-col-md-10{grid-column:auto/span 10}.grid .g-col-md-11{grid-column:auto/span 11}.grid .g-col-md-12{grid-column:auto/span 12}.grid .g-start-md-1{grid-column-start:1}.grid .g-start-md-2{grid-column-start:2}.grid .g-start-md-3{grid-column-start:3}.grid .g-start-md-4{grid-column-start:4}.grid .g-start-md-5{grid-column-start:5}.grid .g-start-md-6{grid-column-start:6}.grid .g-start-md-7{grid-column-start:7}.grid .g-start-md-8{grid-column-start:8}.grid .g-start-md-9{grid-column-start:9}.grid .g-start-md-10{grid-column-start:10}.grid .g-start-md-11{grid-column-start:11}}@media(min-width: 992px){.grid .g-col-lg-1{grid-column:auto/span 1}.grid .g-col-lg-2{grid-column:auto/span 2}.grid .g-col-lg-3{grid-column:auto/span 3}.grid .g-col-lg-4{grid-column:auto/span 4}.grid .g-col-lg-5{grid-column:auto/span 5}.grid .g-col-lg-6{grid-column:auto/span 6}.grid .g-col-lg-7{grid-column:auto/span 7}.grid .g-col-lg-8{grid-column:auto/span 8}.grid .g-col-lg-9{grid-column:auto/span 9}.grid .g-col-lg-10{grid-column:auto/span 10}.grid .g-col-lg-11{grid-column:auto/span 11}.grid .g-col-lg-12{grid-column:auto/span 12}.grid .g-start-lg-1{grid-column-start:1}.grid .g-start-lg-2{grid-column-start:2}.grid .g-start-lg-3{grid-column-start:3}.grid .g-start-lg-4{grid-column-start:4}.grid .g-start-lg-5{grid-column-start:5}.grid .g-start-lg-6{grid-column-start:6}.grid .g-start-lg-7{grid-column-start:7}.grid .g-start-lg-8{grid-column-start:8}.grid .g-start-lg-9{grid-column-start:9}.grid .g-start-lg-10{grid-column-start:10}.grid .g-start-lg-11{grid-column-start:11}}@media(min-width: 1200px){.grid .g-col-xl-1{grid-column:auto/span 1}.grid .g-col-xl-2{grid-column:auto/span 2}.grid .g-col-xl-3{grid-column:auto/span 3}.grid .g-col-xl-4{grid-column:auto/span 4}.grid .g-col-xl-5{grid-column:auto/span 5}.grid .g-col-xl-6{grid-column:auto/span 6}.grid .g-col-xl-7{grid-column:auto/span 7}.grid .g-col-xl-8{grid-column:auto/span 8}.grid .g-col-xl-9{grid-column:auto/span 9}.grid .g-col-xl-10{grid-column:auto/span 10}.grid .g-col-xl-11{grid-column:auto/span 11}.grid .g-col-xl-12{grid-column:auto/span 12}.grid .g-start-xl-1{grid-column-start:1}.grid .g-start-xl-2{grid-column-start:2}.grid .g-start-xl-3{grid-column-start:3}.grid .g-start-xl-4{grid-column-start:4}.grid .g-start-xl-5{grid-column-start:5}.grid .g-start-xl-6{grid-column-start:6}.grid .g-start-xl-7{grid-column-start:7}.grid .g-start-xl-8{grid-column-start:8}.grid .g-start-xl-9{grid-column-start:9}.grid .g-start-xl-10{grid-column-start:10}.grid .g-start-xl-11{grid-column-start:11}}@media(min-width: 1400px){.grid .g-col-xxl-1{grid-column:auto/span 1}.grid .g-col-xxl-2{grid-column:auto/span 2}.grid .g-col-xxl-3{grid-column:auto/span 3}.grid .g-col-xxl-4{grid-column:auto/span 4}.grid .g-col-xxl-5{grid-column:auto/span 5}.grid .g-col-xxl-6{grid-column:auto/span 6}.grid .g-col-xxl-7{grid-column:auto/span 7}.grid .g-col-xxl-8{grid-column:auto/span 8}.grid .g-col-xxl-9{grid-column:auto/span 9}.grid .g-col-xxl-10{grid-column:auto/span 10}.grid .g-col-xxl-11{grid-column:auto/span 11}.grid .g-col-xxl-12{grid-column:auto/span 12}.grid .g-start-xxl-1{grid-column-start:1}.grid .g-start-xxl-2{grid-column-start:2}.grid .g-start-xxl-3{grid-column-start:3}.grid .g-start-xxl-4{grid-column-start:4}.grid .g-start-xxl-5{grid-column-start:5}.grid .g-start-xxl-6{grid-column-start:6}.grid .g-start-xxl-7{grid-column-start:7}.grid .g-start-xxl-8{grid-column-start:8}.grid .g-start-xxl-9{grid-column-start:9}.grid .g-start-xxl-10{grid-column-start:10}.grid .g-start-xxl-11{grid-column-start:11}}.table{--bs-table-color-type: initial;--bs-table-bg-type: initial;--bs-table-color-state: initial;--bs-table-bg-state: initial;--bs-table-color: #343a40;--bs-table-bg: #fff;--bs-table-border-color: #dee2e6;--bs-table-accent-bg: transparent;--bs-table-striped-color: #343a40;--bs-table-striped-bg: rgba(0, 0, 0, 0.05);--bs-table-active-color: #343a40;--bs-table-active-bg: rgba(0, 0, 0, 0.1);--bs-table-hover-color: #343a40;--bs-table-hover-bg: rgba(0, 0, 0, 0.075);width:100%;margin-bottom:1rem;vertical-align:top;border-color:var(--bs-table-border-color)}.table>:not(caption)>*>*{padding:.5rem .5rem;color:var(--bs-table-color-state, var(--bs-table-color-type, var(--bs-table-color)));background-color:var(--bs-table-bg);border-bottom-width:1px;box-shadow:inset 0 0 0 9999px var(--bs-table-bg-state, var(--bs-table-bg-type, var(--bs-table-accent-bg)))}.table>tbody{vertical-align:inherit}.table>thead{vertical-align:bottom}.table-group-divider{border-top:calc(1px*2) solid #b2bac1}.caption-top{caption-side:top}.table-sm>:not(caption)>*>*{padding:.25rem .25rem}.table-bordered>:not(caption)>*{border-width:1px 0}.table-bordered>:not(caption)>*>*{border-width:0 1px}.table-borderless>:not(caption)>*>*{border-bottom-width:0}.table-borderless>:not(:first-child){border-top-width:0}.table-striped>tbody>tr:nth-of-type(odd)>*{--bs-table-color-type: var(--bs-table-striped-color);--bs-table-bg-type: var(--bs-table-striped-bg)}.table-striped-columns>:not(caption)>tr>:nth-child(even){--bs-table-color-type: var(--bs-table-striped-color);--bs-table-bg-type: var(--bs-table-striped-bg)}.table-active{--bs-table-color-state: var(--bs-table-active-color);--bs-table-bg-state: var(--bs-table-active-bg)}.table-hover>tbody>tr:hover>*{--bs-table-color-state: var(--bs-table-hover-color);--bs-table-bg-state: var(--bs-table-hover-bg)}.table-primary{--bs-table-color: #000;--bs-table-bg: #d6dfe7;--bs-table-border-color: #c1c9d0;--bs-table-striped-bg: #cbd4db;--bs-table-striped-color: #000;--bs-table-active-bg: #c1c9d0;--bs-table-active-color: #000;--bs-table-hover-bg: #c6ced6;--bs-table-hover-color: #000;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-secondary{--bs-table-color: #000;--bs-table-bg: #e2e3e5;--bs-table-border-color: #cbccce;--bs-table-striped-bg: #d7d8da;--bs-table-striped-color: #000;--bs-table-active-bg: #cbccce;--bs-table-active-color: #000;--bs-table-hover-bg: #d1d2d4;--bs-table-hover-color: #000;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-success{--bs-table-color: #000;--bs-table-bg: #e9f3db;--bs-table-border-color: #d2dbc5;--bs-table-striped-bg: #dde7d0;--bs-table-striped-color: #000;--bs-table-active-bg: #d2dbc5;--bs-table-active-color: #000;--bs-table-hover-bg: #d8e1cb;--bs-table-hover-color: #000;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-info{--bs-table-color: #000;--bs-table-bg: #d4eef9;--bs-table-border-color: #bfd6e0;--bs-table-striped-bg: #c9e2ed;--bs-table-striped-color: #000;--bs-table-active-bg: #bfd6e0;--bs-table-active-color: #000;--bs-table-hover-bg: #c4dce6;--bs-table-hover-color: #000;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-warning{--bs-table-color: #000;--bs-table-bg: #fde5d8;--bs-table-border-color: #e4cec2;--bs-table-striped-bg: #f0dacd;--bs-table-striped-color: #000;--bs-table-active-bg: #e4cec2;--bs-table-active-color: #000;--bs-table-hover-bg: #ead4c8;--bs-table-hover-color: #000;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-danger{--bs-table-color: #000;--bs-table-bg: #f7dddc;--bs-table-border-color: #dec7c6;--bs-table-striped-bg: #ebd2d1;--bs-table-striped-color: #000;--bs-table-active-bg: #dec7c6;--bs-table-active-color: #000;--bs-table-hover-bg: #e4cccc;--bs-table-hover-color: #000;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-light{--bs-table-color: #000;--bs-table-bg: #f8f5f0;--bs-table-border-color: #dfddd8;--bs-table-striped-bg: #ece9e4;--bs-table-striped-color: #000;--bs-table-active-bg: #dfddd8;--bs-table-active-color: #000;--bs-table-hover-bg: #e5e3de;--bs-table-hover-color: #000;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-dark{--bs-table-color: #fff;--bs-table-bg: #343a40;--bs-table-border-color: #484e53;--bs-table-striped-bg: #3e444a;--bs-table-striped-color: #fff;--bs-table-active-bg: #484e53;--bs-table-active-color: #fff;--bs-table-hover-bg: #43494e;--bs-table-hover-color: #fff;color:var(--bs-table-color);border-color:var(--bs-table-border-color)}.table-responsive{overflow-x:auto;-webkit-overflow-scrolling:touch}@media(max-width: 575.98px){.table-responsive-sm{overflow-x:auto;-webkit-overflow-scrolling:touch}}@media(max-width: 767.98px){.table-responsive-md{overflow-x:auto;-webkit-overflow-scrolling:touch}}@media(max-width: 991.98px){.table-responsive-lg{overflow-x:auto;-webkit-overflow-scrolling:touch}}@media(max-width: 1199.98px){.table-responsive-xl{overflow-x:auto;-webkit-overflow-scrolling:touch}}@media(max-width: 1399.98px){.table-responsive-xxl{overflow-x:auto;-webkit-overflow-scrolling:touch}}.form-label,.shiny-input-container .control-label{margin-bottom:.5rem}.col-form-label{padding-top:calc(0.375rem + 1px);padding-bottom:calc(0.375rem + 1px);margin-bottom:0;font-size:inherit;line-height:1.5}.col-form-label-lg{padding-top:calc(0.5rem + 1px);padding-bottom:calc(0.5rem + 1px);font-size:1.25rem}.col-form-label-sm{padding-top:calc(0.25rem + 1px);padding-bottom:calc(0.25rem + 1px);font-size:0.875rem}.form-text{margin-top:.25rem;font-size:0.875em;color:rgba(52,58,64,.75)}.form-control{display:block;width:100%;padding:.375rem .75rem;font-size:1rem;font-weight:400;line-height:1.5;color:#343a40;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:#fff;background-clip:padding-box;border:1px solid #dee2e6;border-radius:.25rem;transition:border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-control{transition:none}}.form-control[type=file]{overflow:hidden}.form-control[type=file]:not(:disabled):not([readonly]){cursor:pointer}.form-control:focus{color:#343a40;background-color:#fff;border-color:#99aec4;outline:0;box-shadow:0 0 0 .25rem rgba(50,93,136,.25)}.form-control::-webkit-date-and-time-value{min-width:85px;height:1.5em;margin:0}.form-control::-webkit-datetime-edit{display:block;padding:0}.form-control::placeholder{color:rgba(52,58,64,.75);opacity:1}.form-control:disabled{background-color:#f8f5f0;opacity:1}.form-control::file-selector-button{padding:.375rem .75rem;margin:-0.375rem -0.75rem;margin-inline-end:.75rem;color:#343a40;background-color:#f8f9fa;pointer-events:none;border-color:inherit;border-style:solid;border-width:0;border-inline-end-width:1px;border-radius:0;transition:color .15s ease-in-out,background-color .15s ease-in-out,border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-control::file-selector-button{transition:none}}.form-control:hover:not(:disabled):not([readonly])::file-selector-button{background-color:#f8f5f0}.form-control-plaintext{display:block;width:100%;padding:.375rem 0;margin-bottom:0;line-height:1.5;color:#343a40;background-color:rgba(0,0,0,0);border:solid rgba(0,0,0,0);border-width:1px 0}.form-control-plaintext:focus{outline:0}.form-control-plaintext.form-control-sm,.form-control-plaintext.form-control-lg{padding-right:0;padding-left:0}.form-control-sm{min-height:calc(1.5em + 0.5rem + calc(1px * 2));padding:.25rem .5rem;font-size:0.875rem;border-radius:.2em}.form-control-sm::file-selector-button{padding:.25rem .5rem;margin:-0.25rem -0.5rem;margin-inline-end:.5rem}.form-control-lg{min-height:calc(1.5em + 1rem + calc(1px * 2));padding:.5rem 1rem;font-size:1.25rem;border-radius:.5rem}.form-control-lg::file-selector-button{padding:.5rem 1rem;margin:-0.5rem -1rem;margin-inline-end:1rem}textarea.form-control{min-height:calc(1.5em + 0.75rem + calc(1px * 2))}textarea.form-control-sm{min-height:calc(1.5em + 0.5rem + calc(1px * 2))}textarea.form-control-lg{min-height:calc(1.5em + 1rem + calc(1px * 2))}.form-control-color{width:3rem;height:calc(1.5em + 0.75rem + calc(1px * 2));padding:.375rem}.form-control-color:not(:disabled):not([readonly]){cursor:pointer}.form-control-color::-moz-color-swatch{border:0 !important;border-radius:.25rem}.form-control-color::-webkit-color-swatch{border:0 !important;border-radius:.25rem}.form-control-color.form-control-sm{height:calc(1.5em + 0.5rem + calc(1px * 2))}.form-control-color.form-control-lg{height:calc(1.5em + 1rem + calc(1px * 2))}.form-select{--bs-form-select-bg-img: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16'%3e%3cpath fill='none' stroke='%23343a40' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' d='m2 5 6 6 6-6'/%3e%3c/svg%3e");display:block;width:100%;padding:.375rem 2.25rem .375rem .75rem;font-size:1rem;font-weight:400;line-height:1.5;color:#343a40;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:#fff;background-image:var(--bs-form-select-bg-img),var(--bs-form-select-bg-icon, none);background-repeat:no-repeat;background-position:right .75rem center;background-size:16px 12px;border:1px solid #dee2e6;border-radius:.25rem;transition:border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-select{transition:none}}.form-select:focus{border-color:#99aec4;outline:0;box-shadow:0 0 0 .25rem rgba(50,93,136,.25)}.form-select[multiple],.form-select[size]:not([size="1"]){padding-right:.75rem;background-image:none}.form-select:disabled{background-color:#f8f5f0}.form-select:-moz-focusring{color:rgba(0,0,0,0);text-shadow:0 0 0 #343a40}.form-select-sm{padding-top:.25rem;padding-bottom:.25rem;padding-left:.5rem;font-size:0.875rem;border-radius:.2em}.form-select-lg{padding-top:.5rem;padding-bottom:.5rem;padding-left:1rem;font-size:1.25rem;border-radius:.5rem}[data-bs-theme=dark] .form-select{--bs-form-select-bg-img: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16'%3e%3cpath fill='none' stroke='%23dee2e6' stroke-linecap='round' stroke-linejoin='round' stroke-width='2' d='m2 5 6 6 6-6'/%3e%3c/svg%3e")}.form-check,.shiny-input-container .checkbox,.shiny-input-container .radio{display:block;min-height:1.5rem;padding-left:0;margin-bottom:.125rem}.form-check .form-check-input,.form-check .shiny-input-container .checkbox input,.form-check .shiny-input-container .radio input,.shiny-input-container .checkbox .form-check-input,.shiny-input-container .checkbox .shiny-input-container .checkbox input,.shiny-input-container .checkbox .shiny-input-container .radio input,.shiny-input-container .radio .form-check-input,.shiny-input-container .radio .shiny-input-container .checkbox input,.shiny-input-container .radio .shiny-input-container .radio input{float:left;margin-left:0}.form-check-reverse{padding-right:0;padding-left:0;text-align:right}.form-check-reverse .form-check-input{float:right;margin-right:0;margin-left:0}.form-check-input,.shiny-input-container .checkbox input,.shiny-input-container .checkbox-inline input,.shiny-input-container .radio input,.shiny-input-container .radio-inline input{--bs-form-check-bg: #fff;width:1em;height:1em;margin-top:.25em;vertical-align:top;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:var(--bs-form-check-bg);background-image:var(--bs-form-check-bg-image);background-repeat:no-repeat;background-position:center;background-size:contain;border:1px solid #dee2e6;print-color-adjust:exact}.form-check-input[type=checkbox],.shiny-input-container .checkbox input[type=checkbox],.shiny-input-container .checkbox-inline input[type=checkbox],.shiny-input-container .radio input[type=checkbox],.shiny-input-container .radio-inline input[type=checkbox]{border-radius:.25em}.form-check-input[type=radio],.shiny-input-container .checkbox input[type=radio],.shiny-input-container .checkbox-inline input[type=radio],.shiny-input-container .radio input[type=radio],.shiny-input-container .radio-inline input[type=radio]{border-radius:50%}.form-check-input:active,.shiny-input-container .checkbox input:active,.shiny-input-container .checkbox-inline input:active,.shiny-input-container .radio input:active,.shiny-input-container .radio-inline input:active{filter:brightness(90%)}.form-check-input:focus,.shiny-input-container .checkbox input:focus,.shiny-input-container .checkbox-inline input:focus,.shiny-input-container .radio input:focus,.shiny-input-container .radio-inline input:focus{border-color:#99aec4;outline:0;box-shadow:0 0 0 .25rem rgba(50,93,136,.25)}.form-check-input:checked,.shiny-input-container .checkbox input:checked,.shiny-input-container .checkbox-inline input:checked,.shiny-input-container .radio input:checked,.shiny-input-container .radio-inline input:checked{background-color:#325d88;border-color:#325d88}.form-check-input:checked[type=checkbox],.shiny-input-container .checkbox input:checked[type=checkbox],.shiny-input-container .checkbox-inline input:checked[type=checkbox],.shiny-input-container .radio input:checked[type=checkbox],.shiny-input-container .radio-inline input:checked[type=checkbox]{--bs-form-check-bg-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 20 20'%3e%3cpath fill='none' stroke='%23fff' stroke-linecap='round' stroke-linejoin='round' stroke-width='3' d='m6 10 3 3 6-6'/%3e%3c/svg%3e")}.form-check-input:checked[type=radio],.shiny-input-container .checkbox input:checked[type=radio],.shiny-input-container .checkbox-inline input:checked[type=radio],.shiny-input-container .radio input:checked[type=radio],.shiny-input-container .radio-inline input:checked[type=radio]{--bs-form-check-bg-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='2' fill='%23fff'/%3e%3c/svg%3e")}.form-check-input[type=checkbox]:indeterminate,.shiny-input-container .checkbox input[type=checkbox]:indeterminate,.shiny-input-container .checkbox-inline input[type=checkbox]:indeterminate,.shiny-input-container .radio input[type=checkbox]:indeterminate,.shiny-input-container .radio-inline input[type=checkbox]:indeterminate{background-color:#325d88;border-color:#325d88;--bs-form-check-bg-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 20 20'%3e%3cpath fill='none' stroke='%23fff' stroke-linecap='round' stroke-linejoin='round' stroke-width='3' d='M6 10h8'/%3e%3c/svg%3e")}.form-check-input:disabled,.shiny-input-container .checkbox input:disabled,.shiny-input-container .checkbox-inline input:disabled,.shiny-input-container .radio input:disabled,.shiny-input-container .radio-inline input:disabled{pointer-events:none;filter:none;opacity:.5}.form-check-input[disabled]~.form-check-label,.form-check-input[disabled]~span,.form-check-input:disabled~.form-check-label,.form-check-input:disabled~span,.shiny-input-container .checkbox input[disabled]~.form-check-label,.shiny-input-container .checkbox input[disabled]~span,.shiny-input-container .checkbox input:disabled~.form-check-label,.shiny-input-container .checkbox input:disabled~span,.shiny-input-container .checkbox-inline input[disabled]~.form-check-label,.shiny-input-container .checkbox-inline input[disabled]~span,.shiny-input-container .checkbox-inline input:disabled~.form-check-label,.shiny-input-container .checkbox-inline input:disabled~span,.shiny-input-container .radio input[disabled]~.form-check-label,.shiny-input-container .radio input[disabled]~span,.shiny-input-container .radio input:disabled~.form-check-label,.shiny-input-container .radio input:disabled~span,.shiny-input-container .radio-inline input[disabled]~.form-check-label,.shiny-input-container .radio-inline input[disabled]~span,.shiny-input-container .radio-inline input:disabled~.form-check-label,.shiny-input-container .radio-inline input:disabled~span{cursor:default;opacity:.5}.form-check-label,.shiny-input-container .checkbox label,.shiny-input-container .checkbox-inline label,.shiny-input-container .radio label,.shiny-input-container .radio-inline label{cursor:pointer}.form-switch{padding-left:2.5em}.form-switch .form-check-input{--bs-form-switch-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='rgba%280, 0, 0, 0.25%29'/%3e%3c/svg%3e");width:2em;margin-left:-2.5em;background-image:var(--bs-form-switch-bg);background-position:left center;border-radius:2em;transition:background-position .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-switch .form-check-input{transition:none}}.form-switch .form-check-input:focus{--bs-form-switch-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='%2399aec4'/%3e%3c/svg%3e")}.form-switch .form-check-input:checked{background-position:right center;--bs-form-switch-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='%23fff'/%3e%3c/svg%3e")}.form-switch.form-check-reverse{padding-right:2.5em;padding-left:0}.form-switch.form-check-reverse .form-check-input{margin-right:-2.5em;margin-left:0}.form-check-inline{display:inline-block;margin-right:1rem}.btn-check{position:absolute;clip:rect(0, 0, 0, 0);pointer-events:none}.btn-check[disabled]+.btn,.btn-check:disabled+.btn{pointer-events:none;filter:none;opacity:.65}[data-bs-theme=dark] .form-switch .form-check-input:not(:checked):not(:focus){--bs-form-switch-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='rgba%28255, 255, 255, 0.25%29'/%3e%3c/svg%3e")}.form-range{width:100%;height:1.5rem;padding:0;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:rgba(0,0,0,0)}.form-range:focus{outline:0}.form-range:focus::-webkit-slider-thumb{box-shadow:0 0 0 1px #fff,0 0 0 .25rem rgba(50,93,136,.25)}.form-range:focus::-moz-range-thumb{box-shadow:0 0 0 1px #fff,0 0 0 .25rem rgba(50,93,136,.25)}.form-range::-moz-focus-outer{border:0}.form-range::-webkit-slider-thumb{width:1rem;height:1rem;margin-top:-0.25rem;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:#325d88;border:0;border-radius:1rem;transition:background-color .15s ease-in-out,border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-range::-webkit-slider-thumb{transition:none}}.form-range::-webkit-slider-thumb:active{background-color:#c2cedb}.form-range::-webkit-slider-runnable-track{width:100%;height:.5rem;color:rgba(0,0,0,0);cursor:pointer;background-color:#f8f9fa;border-color:rgba(0,0,0,0);border-radius:1rem}.form-range::-moz-range-thumb{width:1rem;height:1rem;appearance:none;-webkit-appearance:none;-moz-appearance:none;-ms-appearance:none;-o-appearance:none;background-color:#325d88;border:0;border-radius:1rem;transition:background-color .15s ease-in-out,border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.form-range::-moz-range-thumb{transition:none}}.form-range::-moz-range-thumb:active{background-color:#c2cedb}.form-range::-moz-range-track{width:100%;height:.5rem;color:rgba(0,0,0,0);cursor:pointer;background-color:#f8f9fa;border-color:rgba(0,0,0,0);border-radius:1rem}.form-range:disabled{pointer-events:none}.form-range:disabled::-webkit-slider-thumb{background-color:rgba(52,58,64,.75)}.form-range:disabled::-moz-range-thumb{background-color:rgba(52,58,64,.75)}.form-floating{position:relative}.form-floating>.form-control,.form-floating>.form-control-plaintext,.form-floating>.form-select{height:calc(3.5rem + calc(1px * 2));min-height:calc(3.5rem + calc(1px * 2));line-height:1.25}.form-floating>label{position:absolute;top:0;left:0;z-index:2;height:100%;padding:1rem .75rem;overflow:hidden;text-align:start;text-overflow:ellipsis;white-space:nowrap;pointer-events:none;border:1px solid rgba(0,0,0,0);transform-origin:0 0;transition:opacity .1s ease-in-out,transform .1s ease-in-out}@media(prefers-reduced-motion: reduce){.form-floating>label{transition:none}}.form-floating>.form-control,.form-floating>.form-control-plaintext{padding:1rem .75rem}.form-floating>.form-control::placeholder,.form-floating>.form-control-plaintext::placeholder{color:rgba(0,0,0,0)}.form-floating>.form-control:focus,.form-floating>.form-control:not(:placeholder-shown),.form-floating>.form-control-plaintext:focus,.form-floating>.form-control-plaintext:not(:placeholder-shown){padding-top:1.625rem;padding-bottom:.625rem}.form-floating>.form-control:-webkit-autofill,.form-floating>.form-control-plaintext:-webkit-autofill{padding-top:1.625rem;padding-bottom:.625rem}.form-floating>.form-select{padding-top:1.625rem;padding-bottom:.625rem}.form-floating>.form-control:focus~label,.form-floating>.form-control:not(:placeholder-shown)~label,.form-floating>.form-control-plaintext~label,.form-floating>.form-select~label{color:rgba(var(--bs-body-color-rgb), 0.65);transform:scale(0.85) translateY(-0.5rem) translateX(0.15rem)}.form-floating>.form-control:focus~label::after,.form-floating>.form-control:not(:placeholder-shown)~label::after,.form-floating>.form-control-plaintext~label::after,.form-floating>.form-select~label::after{position:absolute;inset:1rem .375rem;z-index:-1;height:1.5em;content:"";background-color:#fff;border-radius:.25rem}.form-floating>.form-control:-webkit-autofill~label{color:rgba(var(--bs-body-color-rgb), 0.65);transform:scale(0.85) translateY(-0.5rem) translateX(0.15rem)}.form-floating>.form-control-plaintext~label{border-width:1px 0}.form-floating>:disabled~label,.form-floating>.form-control:disabled~label{color:#6c757d}.form-floating>:disabled~label::after,.form-floating>.form-control:disabled~label::after{background-color:#f8f5f0}.input-group{position:relative;display:flex;display:-webkit-flex;flex-wrap:wrap;-webkit-flex-wrap:wrap;align-items:stretch;-webkit-align-items:stretch;width:100%}.input-group>.form-control,.input-group>.form-select,.input-group>.form-floating{position:relative;flex:1 1 auto;-webkit-flex:1 1 auto;width:1%;min-width:0}.input-group>.form-control:focus,.input-group>.form-select:focus,.input-group>.form-floating:focus-within{z-index:5}.input-group .btn{position:relative;z-index:2}.input-group .btn:focus{z-index:5}.input-group-text{display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;padding:.375rem .75rem;font-size:1rem;font-weight:400;line-height:1.5;color:#343a40;text-align:center;white-space:nowrap;background-color:#f8f9fa;border:1px solid #dee2e6;border-radius:.25rem}.input-group-lg>.form-control,.input-group-lg>.form-select,.input-group-lg>.input-group-text,.input-group-lg>.btn{padding:.5rem 1rem;font-size:1.25rem;border-radius:.5rem}.input-group-sm>.form-control,.input-group-sm>.form-select,.input-group-sm>.input-group-text,.input-group-sm>.btn{padding:.25rem .5rem;font-size:0.875rem;border-radius:.2em}.input-group-lg>.form-select,.input-group-sm>.form-select{padding-right:3rem}.input-group:not(.has-validation)>:not(:last-child):not(.dropdown-toggle):not(.dropdown-menu):not(.form-floating),.input-group:not(.has-validation)>.dropdown-toggle:nth-last-child(n+3),.input-group:not(.has-validation)>.form-floating:not(:last-child)>.form-control,.input-group:not(.has-validation)>.form-floating:not(:last-child)>.form-select{border-top-right-radius:0;border-bottom-right-radius:0}.input-group.has-validation>:nth-last-child(n+3):not(.dropdown-toggle):not(.dropdown-menu):not(.form-floating),.input-group.has-validation>.dropdown-toggle:nth-last-child(n+4),.input-group.has-validation>.form-floating:nth-last-child(n+3)>.form-control,.input-group.has-validation>.form-floating:nth-last-child(n+3)>.form-select{border-top-right-radius:0;border-bottom-right-radius:0}.input-group>:not(:first-child):not(.dropdown-menu):not(.valid-tooltip):not(.valid-feedback):not(.invalid-tooltip):not(.invalid-feedback){margin-left:calc(1px*-1);border-top-left-radius:0;border-bottom-left-radius:0}.input-group>.form-floating:not(:first-child)>.form-control,.input-group>.form-floating:not(:first-child)>.form-select{border-top-left-radius:0;border-bottom-left-radius:0}.valid-feedback{display:none;width:100%;margin-top:.25rem;font-size:0.875em;color:#93c54b}.valid-tooltip{position:absolute;top:100%;z-index:5;display:none;max-width:100%;padding:.25rem .5rem;margin-top:.1rem;font-size:0.875rem;color:#fff;background-color:#93c54b;border-radius:.25rem}.was-validated :valid~.valid-feedback,.was-validated :valid~.valid-tooltip,.is-valid~.valid-feedback,.is-valid~.valid-tooltip{display:block}.was-validated .form-control:valid,.form-control.is-valid{border-color:#93c54b;padding-right:calc(1.5em + 0.75rem);background-image:url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 8 8'%3e%3cpath fill='%2393c54b' d='M2.3 6.73.6 4.53c-.4-1.04.46-1.4 1.1-.8l1.1 1.4 3.4-3.8c.6-.63 1.6-.27 1.2.7l-4 4.6c-.43.5-.8.4-1.1.1z'/%3e%3c/svg%3e");background-repeat:no-repeat;background-position:right calc(0.375em + 0.1875rem) center;background-size:calc(0.75em + 0.375rem) calc(0.75em + 0.375rem)}.was-validated .form-control:valid:focus,.form-control.is-valid:focus{border-color:#93c54b;box-shadow:0 0 0 .25rem rgba(147,197,75,.25)}.was-validated textarea.form-control:valid,textarea.form-control.is-valid{padding-right:calc(1.5em + 0.75rem);background-position:top calc(0.375em + 0.1875rem) right calc(0.375em + 0.1875rem)}.was-validated .form-select:valid,.form-select.is-valid{border-color:#93c54b}.was-validated .form-select:valid:not([multiple]):not([size]),.was-validated .form-select:valid:not([multiple])[size="1"],.form-select.is-valid:not([multiple]):not([size]),.form-select.is-valid:not([multiple])[size="1"]{--bs-form-select-bg-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 8 8'%3e%3cpath fill='%2393c54b' d='M2.3 6.73.6 4.53c-.4-1.04.46-1.4 1.1-.8l1.1 1.4 3.4-3.8c.6-.63 1.6-.27 1.2.7l-4 4.6c-.43.5-.8.4-1.1.1z'/%3e%3c/svg%3e");padding-right:4.125rem;background-position:right .75rem center,center right 2.25rem;background-size:16px 12px,calc(0.75em + 0.375rem) calc(0.75em + 0.375rem)}.was-validated .form-select:valid:focus,.form-select.is-valid:focus{border-color:#93c54b;box-shadow:0 0 0 .25rem rgba(147,197,75,.25)}.was-validated .form-control-color:valid,.form-control-color.is-valid{width:calc(3rem + calc(1.5em + 0.75rem))}.was-validated .form-check-input:valid,.form-check-input.is-valid{border-color:#93c54b}.was-validated .form-check-input:valid:checked,.form-check-input.is-valid:checked{background-color:#93c54b}.was-validated .form-check-input:valid:focus,.form-check-input.is-valid:focus{box-shadow:0 0 0 .25rem rgba(147,197,75,.25)}.was-validated .form-check-input:valid~.form-check-label,.form-check-input.is-valid~.form-check-label{color:#93c54b}.form-check-inline .form-check-input~.valid-feedback{margin-left:.5em}.was-validated .input-group>.form-control:not(:focus):valid,.input-group>.form-control:not(:focus).is-valid,.was-validated .input-group>.form-select:not(:focus):valid,.input-group>.form-select:not(:focus).is-valid,.was-validated .input-group>.form-floating:not(:focus-within):valid,.input-group>.form-floating:not(:focus-within).is-valid{z-index:3}.invalid-feedback{display:none;width:100%;margin-top:.25rem;font-size:0.875em;color:#d9534f}.invalid-tooltip{position:absolute;top:100%;z-index:5;display:none;max-width:100%;padding:.25rem .5rem;margin-top:.1rem;font-size:0.875rem;color:#fff;background-color:#d9534f;border-radius:.25rem}.was-validated :invalid~.invalid-feedback,.was-validated :invalid~.invalid-tooltip,.is-invalid~.invalid-feedback,.is-invalid~.invalid-tooltip{display:block}.was-validated .form-control:invalid,.form-control.is-invalid{border-color:#d9534f;padding-right:calc(1.5em + 0.75rem);background-image:url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 12 12' width='12' height='12' fill='none' stroke='%23d9534f'%3e%3ccircle cx='6' cy='6' r='4.5'/%3e%3cpath stroke-linejoin='round' d='M5.8 3.6h.4L6 6.5z'/%3e%3ccircle cx='6' cy='8.2' r='.6' fill='%23d9534f' stroke='none'/%3e%3c/svg%3e");background-repeat:no-repeat;background-position:right calc(0.375em + 0.1875rem) center;background-size:calc(0.75em + 0.375rem) calc(0.75em + 0.375rem)}.was-validated .form-control:invalid:focus,.form-control.is-invalid:focus{border-color:#d9534f;box-shadow:0 0 0 .25rem rgba(217,83,79,.25)}.was-validated textarea.form-control:invalid,textarea.form-control.is-invalid{padding-right:calc(1.5em + 0.75rem);background-position:top calc(0.375em + 0.1875rem) right calc(0.375em + 0.1875rem)}.was-validated .form-select:invalid,.form-select.is-invalid{border-color:#d9534f}.was-validated .form-select:invalid:not([multiple]):not([size]),.was-validated .form-select:invalid:not([multiple])[size="1"],.form-select.is-invalid:not([multiple]):not([size]),.form-select.is-invalid:not([multiple])[size="1"]{--bs-form-select-bg-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 12 12' width='12' height='12' fill='none' stroke='%23d9534f'%3e%3ccircle cx='6' cy='6' r='4.5'/%3e%3cpath stroke-linejoin='round' d='M5.8 3.6h.4L6 6.5z'/%3e%3ccircle cx='6' cy='8.2' r='.6' fill='%23d9534f' stroke='none'/%3e%3c/svg%3e");padding-right:4.125rem;background-position:right .75rem center,center right 2.25rem;background-size:16px 12px,calc(0.75em + 0.375rem) calc(0.75em + 0.375rem)}.was-validated .form-select:invalid:focus,.form-select.is-invalid:focus{border-color:#d9534f;box-shadow:0 0 0 .25rem rgba(217,83,79,.25)}.was-validated .form-control-color:invalid,.form-control-color.is-invalid{width:calc(3rem + calc(1.5em + 0.75rem))}.was-validated .form-check-input:invalid,.form-check-input.is-invalid{border-color:#d9534f}.was-validated .form-check-input:invalid:checked,.form-check-input.is-invalid:checked{background-color:#d9534f}.was-validated .form-check-input:invalid:focus,.form-check-input.is-invalid:focus{box-shadow:0 0 0 .25rem rgba(217,83,79,.25)}.was-validated .form-check-input:invalid~.form-check-label,.form-check-input.is-invalid~.form-check-label{color:#d9534f}.form-check-inline .form-check-input~.invalid-feedback{margin-left:.5em}.was-validated .input-group>.form-control:not(:focus):invalid,.input-group>.form-control:not(:focus).is-invalid,.was-validated .input-group>.form-select:not(:focus):invalid,.input-group>.form-select:not(:focus).is-invalid,.was-validated .input-group>.form-floating:not(:focus-within):invalid,.input-group>.form-floating:not(:focus-within).is-invalid{z-index:4}.btn{--bs-btn-padding-x: 0.75rem;--bs-btn-padding-y: 0.375rem;--bs-btn-font-family: ;--bs-btn-font-size:1rem;--bs-btn-font-weight: 400;--bs-btn-line-height: 1.5;--bs-btn-color: #343a40;--bs-btn-bg: transparent;--bs-btn-border-width: 1px;--bs-btn-border-color: transparent;--bs-btn-border-radius: 0.25rem;--bs-btn-hover-border-color: transparent;--bs-btn-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.15), 0 1px 1px rgba(0, 0, 0, 0.075);--bs-btn-disabled-opacity: 0.65;--bs-btn-focus-box-shadow: 0 0 0 0.25rem rgba(var(--bs-btn-focus-shadow-rgb), .5);display:inline-block;padding:var(--bs-btn-padding-y) var(--bs-btn-padding-x);font-family:var(--bs-btn-font-family);font-size:var(--bs-btn-font-size);font-weight:var(--bs-btn-font-weight);line-height:var(--bs-btn-line-height);color:var(--bs-btn-color);text-align:center;text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;vertical-align:middle;cursor:pointer;user-select:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;-o-user-select:none;border:var(--bs-btn-border-width) solid var(--bs-btn-border-color);border-radius:var(--bs-btn-border-radius);background-color:var(--bs-btn-bg);transition:color .15s ease-in-out,background-color .15s ease-in-out,border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.btn{transition:none}}.btn:hover{color:var(--bs-btn-hover-color);background-color:var(--bs-btn-hover-bg);border-color:var(--bs-btn-hover-border-color)}.btn-check+.btn:hover{color:var(--bs-btn-color);background-color:var(--bs-btn-bg);border-color:var(--bs-btn-border-color)}.btn:focus-visible{color:var(--bs-btn-hover-color);background-color:var(--bs-btn-hover-bg);border-color:var(--bs-btn-hover-border-color);outline:0;box-shadow:var(--bs-btn-focus-box-shadow)}.btn-check:focus-visible+.btn{border-color:var(--bs-btn-hover-border-color);outline:0;box-shadow:var(--bs-btn-focus-box-shadow)}.btn-check:checked+.btn,:not(.btn-check)+.btn:active,.btn:first-child:active,.btn.active,.btn.show{color:var(--bs-btn-active-color);background-color:var(--bs-btn-active-bg);border-color:var(--bs-btn-active-border-color)}.btn-check:checked+.btn:focus-visible,:not(.btn-check)+.btn:active:focus-visible,.btn:first-child:active:focus-visible,.btn.active:focus-visible,.btn.show:focus-visible{box-shadow:var(--bs-btn-focus-box-shadow)}.btn:disabled,.btn.disabled,fieldset:disabled .btn{color:var(--bs-btn-disabled-color);pointer-events:none;background-color:var(--bs-btn-disabled-bg);border-color:var(--bs-btn-disabled-border-color);opacity:var(--bs-btn-disabled-opacity)}.btn-default{--bs-btn-color: #fff;--bs-btn-bg: #6c757d;--bs-btn-border-color: #6c757d;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #5c636a;--bs-btn-hover-border-color: #565e64;--bs-btn-focus-shadow-rgb: 130, 138, 145;--bs-btn-active-color: #fff;--bs-btn-active-bg: #565e64;--bs-btn-active-border-color: #51585e;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #6c757d;--bs-btn-disabled-border-color: #6c757d}.btn-primary{--bs-btn-color: #fff;--bs-btn-bg: #325d88;--bs-btn-border-color: #325d88;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #2b4f74;--bs-btn-hover-border-color: #284a6d;--bs-btn-focus-shadow-rgb: 81, 117, 154;--bs-btn-active-color: #fff;--bs-btn-active-bg: #284a6d;--bs-btn-active-border-color: #264666;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #325d88;--bs-btn-disabled-border-color: #325d88}.btn-secondary{--bs-btn-color: #fff;--bs-btn-bg: #6c757d;--bs-btn-border-color: #6c757d;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #5c636a;--bs-btn-hover-border-color: #565e64;--bs-btn-focus-shadow-rgb: 130, 138, 145;--bs-btn-active-color: #fff;--bs-btn-active-bg: #565e64;--bs-btn-active-border-color: #51585e;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #6c757d;--bs-btn-disabled-border-color: #6c757d}.btn-success{--bs-btn-color: #fff;--bs-btn-bg: #93c54b;--bs-btn-border-color: #93c54b;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #7da740;--bs-btn-hover-border-color: #769e3c;--bs-btn-focus-shadow-rgb: 163, 206, 102;--bs-btn-active-color: #fff;--bs-btn-active-bg: #769e3c;--bs-btn-active-border-color: #6e9438;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #93c54b;--bs-btn-disabled-border-color: #93c54b}.btn-info{--bs-btn-color: #fff;--bs-btn-bg: #29abe0;--bs-btn-border-color: #29abe0;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #2391be;--bs-btn-hover-border-color: #2189b3;--bs-btn-focus-shadow-rgb: 73, 184, 229;--bs-btn-active-color: #fff;--bs-btn-active-bg: #2189b3;--bs-btn-active-border-color: #1f80a8;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #29abe0;--bs-btn-disabled-border-color: #29abe0}.btn-warning{--bs-btn-color: #fff;--bs-btn-bg: #f47c3c;--bs-btn-border-color: #f47c3c;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #cf6933;--bs-btn-hover-border-color: #c36330;--bs-btn-focus-shadow-rgb: 246, 144, 89;--bs-btn-active-color: #fff;--bs-btn-active-bg: #c36330;--bs-btn-active-border-color: #b75d2d;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #f47c3c;--bs-btn-disabled-border-color: #f47c3c}.btn-danger{--bs-btn-color: #fff;--bs-btn-bg: #d9534f;--bs-btn-border-color: #d9534f;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #b84743;--bs-btn-hover-border-color: #ae423f;--bs-btn-focus-shadow-rgb: 223, 109, 105;--bs-btn-active-color: #fff;--bs-btn-active-bg: #ae423f;--bs-btn-active-border-color: #a33e3b;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #d9534f;--bs-btn-disabled-border-color: #d9534f}.btn-light{--bs-btn-color: #000;--bs-btn-bg: #f8f5f0;--bs-btn-border-color: #f8f5f0;--bs-btn-hover-color: #000;--bs-btn-hover-bg: #d3d0cc;--bs-btn-hover-border-color: #c6c4c0;--bs-btn-focus-shadow-rgb: 211, 208, 204;--bs-btn-active-color: #000;--bs-btn-active-bg: #c6c4c0;--bs-btn-active-border-color: #bab8b4;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #000;--bs-btn-disabled-bg: #f8f5f0;--bs-btn-disabled-border-color: #f8f5f0}.btn-dark{--bs-btn-color: #fff;--bs-btn-bg: #343a40;--bs-btn-border-color: #343a40;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #52585d;--bs-btn-hover-border-color: #484e53;--bs-btn-focus-shadow-rgb: 82, 88, 93;--bs-btn-active-color: #fff;--bs-btn-active-bg: #5d6166;--bs-btn-active-border-color: #484e53;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #343a40;--bs-btn-disabled-border-color: #343a40}.btn-outline-default{--bs-btn-color: #6c757d;--bs-btn-border-color: #6c757d;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #6c757d;--bs-btn-hover-border-color: #6c757d;--bs-btn-focus-shadow-rgb: 108, 117, 125;--bs-btn-active-color: #fff;--bs-btn-active-bg: #6c757d;--bs-btn-active-border-color: #6c757d;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #6c757d;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #6c757d;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-primary{--bs-btn-color: #325d88;--bs-btn-border-color: #325d88;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #325d88;--bs-btn-hover-border-color: #325d88;--bs-btn-focus-shadow-rgb: 50, 93, 136;--bs-btn-active-color: #fff;--bs-btn-active-bg: #325d88;--bs-btn-active-border-color: #325d88;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #325d88;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #325d88;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-secondary{--bs-btn-color: #6c757d;--bs-btn-border-color: #6c757d;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #6c757d;--bs-btn-hover-border-color: #6c757d;--bs-btn-focus-shadow-rgb: 108, 117, 125;--bs-btn-active-color: #fff;--bs-btn-active-bg: #6c757d;--bs-btn-active-border-color: #6c757d;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #6c757d;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #6c757d;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-success{--bs-btn-color: #93c54b;--bs-btn-border-color: #93c54b;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #93c54b;--bs-btn-hover-border-color: #93c54b;--bs-btn-focus-shadow-rgb: 147, 197, 75;--bs-btn-active-color: #fff;--bs-btn-active-bg: #93c54b;--bs-btn-active-border-color: #93c54b;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #93c54b;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #93c54b;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-info{--bs-btn-color: #29abe0;--bs-btn-border-color: #29abe0;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #29abe0;--bs-btn-hover-border-color: #29abe0;--bs-btn-focus-shadow-rgb: 41, 171, 224;--bs-btn-active-color: #fff;--bs-btn-active-bg: #29abe0;--bs-btn-active-border-color: #29abe0;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #29abe0;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #29abe0;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-warning{--bs-btn-color: #f47c3c;--bs-btn-border-color: #f47c3c;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #f47c3c;--bs-btn-hover-border-color: #f47c3c;--bs-btn-focus-shadow-rgb: 244, 124, 60;--bs-btn-active-color: #fff;--bs-btn-active-bg: #f47c3c;--bs-btn-active-border-color: #f47c3c;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #f47c3c;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #f47c3c;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-danger{--bs-btn-color: #d9534f;--bs-btn-border-color: #d9534f;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #d9534f;--bs-btn-hover-border-color: #d9534f;--bs-btn-focus-shadow-rgb: 217, 83, 79;--bs-btn-active-color: #fff;--bs-btn-active-bg: #d9534f;--bs-btn-active-border-color: #d9534f;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #d9534f;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #d9534f;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-light{--bs-btn-color: #f8f5f0;--bs-btn-border-color: #f8f5f0;--bs-btn-hover-color: #000;--bs-btn-hover-bg: #f8f5f0;--bs-btn-hover-border-color: #f8f5f0;--bs-btn-focus-shadow-rgb: 248, 245, 240;--bs-btn-active-color: #000;--bs-btn-active-bg: #f8f5f0;--bs-btn-active-border-color: #f8f5f0;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #f8f5f0;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #f8f5f0;--bs-btn-bg: transparent;--bs-gradient: none}.btn-outline-dark{--bs-btn-color: #343a40;--bs-btn-border-color: #343a40;--bs-btn-hover-color: #fff;--bs-btn-hover-bg: #343a40;--bs-btn-hover-border-color: #343a40;--bs-btn-focus-shadow-rgb: 52, 58, 64;--bs-btn-active-color: #fff;--bs-btn-active-bg: #343a40;--bs-btn-active-border-color: #343a40;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #343a40;--bs-btn-disabled-bg: transparent;--bs-btn-disabled-border-color: #343a40;--bs-btn-bg: transparent;--bs-gradient: none}.btn-link{--bs-btn-font-weight: 400;--bs-btn-color: #93c54b;--bs-btn-bg: transparent;--bs-btn-border-color: transparent;--bs-btn-hover-color: #769e3c;--bs-btn-hover-border-color: transparent;--bs-btn-active-color: #769e3c;--bs-btn-active-border-color: transparent;--bs-btn-disabled-color: #6c757d;--bs-btn-disabled-border-color: transparent;--bs-btn-box-shadow: 0 0 0 #000;--bs-btn-focus-shadow-rgb: 163, 206, 102;text-decoration:underline;-webkit-text-decoration:underline;-moz-text-decoration:underline;-ms-text-decoration:underline;-o-text-decoration:underline}.btn-link:focus-visible{color:var(--bs-btn-color)}.btn-link:hover{color:var(--bs-btn-hover-color)}.btn-lg,.btn-group-lg>.btn{--bs-btn-padding-y: 0.5rem;--bs-btn-padding-x: 1rem;--bs-btn-font-size:1.25rem;--bs-btn-border-radius: 0.5rem}.btn-sm,.btn-group-sm>.btn{--bs-btn-padding-y: 0.25rem;--bs-btn-padding-x: 0.5rem;--bs-btn-font-size:0.875rem;--bs-btn-border-radius: 0.2em}.fade{transition:opacity .15s linear}@media(prefers-reduced-motion: reduce){.fade{transition:none}}.fade:not(.show){opacity:0}.collapse:not(.show){display:none}.collapsing{height:0;overflow:hidden;transition:height .2s ease}@media(prefers-reduced-motion: reduce){.collapsing{transition:none}}.collapsing.collapse-horizontal{width:0;height:auto;transition:width .35s ease}@media(prefers-reduced-motion: reduce){.collapsing.collapse-horizontal{transition:none}}.dropup,.dropend,.dropdown,.dropstart,.dropup-center,.dropdown-center{position:relative}.dropdown-toggle{white-space:nowrap}.dropdown-toggle::after{display:inline-block;margin-left:.255em;vertical-align:.255em;content:"";border-top:.3em solid;border-right:.3em solid rgba(0,0,0,0);border-bottom:0;border-left:.3em solid rgba(0,0,0,0)}.dropdown-toggle:empty::after{margin-left:0}.dropdown-menu{--bs-dropdown-zindex: 1000;--bs-dropdown-min-width: 10rem;--bs-dropdown-padding-x: 0;--bs-dropdown-padding-y: 0.5rem;--bs-dropdown-spacer: 0.125rem;--bs-dropdown-font-size:1rem;--bs-dropdown-color: #343a40;--bs-dropdown-bg: #fff;--bs-dropdown-border-color: rgba(0, 0, 0, 0.175);--bs-dropdown-border-radius: 0.25rem;--bs-dropdown-border-width: 1px;--bs-dropdown-inner-border-radius: calc(0.25rem - 1px);--bs-dropdown-divider-bg: rgba(0, 0, 0, 0.175);--bs-dropdown-divider-margin-y: 0.5rem;--bs-dropdown-box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);--bs-dropdown-link-color: #6c757d;--bs-dropdown-link-hover-color: #6c757d;--bs-dropdown-link-hover-bg: #f8f5f0;--bs-dropdown-link-active-color: #6c757d;--bs-dropdown-link-active-bg: #f8f5f0;--bs-dropdown-link-disabled-color: rgba(52, 58, 64, 0.5);--bs-dropdown-item-padding-x: 1rem;--bs-dropdown-item-padding-y: 0.25rem;--bs-dropdown-header-color: #6c757d;--bs-dropdown-header-padding-x: 1rem;--bs-dropdown-header-padding-y: 0.5rem;position:absolute;z-index:var(--bs-dropdown-zindex);display:none;min-width:var(--bs-dropdown-min-width);padding:var(--bs-dropdown-padding-y) var(--bs-dropdown-padding-x);margin:0;font-size:var(--bs-dropdown-font-size);color:var(--bs-dropdown-color);text-align:left;list-style:none;background-color:var(--bs-dropdown-bg);background-clip:padding-box;border:var(--bs-dropdown-border-width) solid var(--bs-dropdown-border-color);border-radius:var(--bs-dropdown-border-radius)}.dropdown-menu[data-bs-popper]{top:100%;left:0;margin-top:var(--bs-dropdown-spacer)}.dropdown-menu-start{--bs-position: start}.dropdown-menu-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-end{--bs-position: end}.dropdown-menu-end[data-bs-popper]{right:0;left:auto}@media(min-width: 576px){.dropdown-menu-sm-start{--bs-position: start}.dropdown-menu-sm-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-sm-end{--bs-position: end}.dropdown-menu-sm-end[data-bs-popper]{right:0;left:auto}}@media(min-width: 768px){.dropdown-menu-md-start{--bs-position: start}.dropdown-menu-md-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-md-end{--bs-position: end}.dropdown-menu-md-end[data-bs-popper]{right:0;left:auto}}@media(min-width: 992px){.dropdown-menu-lg-start{--bs-position: start}.dropdown-menu-lg-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-lg-end{--bs-position: end}.dropdown-menu-lg-end[data-bs-popper]{right:0;left:auto}}@media(min-width: 1200px){.dropdown-menu-xl-start{--bs-position: start}.dropdown-menu-xl-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-xl-end{--bs-position: end}.dropdown-menu-xl-end[data-bs-popper]{right:0;left:auto}}@media(min-width: 1400px){.dropdown-menu-xxl-start{--bs-position: start}.dropdown-menu-xxl-start[data-bs-popper]{right:auto;left:0}.dropdown-menu-xxl-end{--bs-position: end}.dropdown-menu-xxl-end[data-bs-popper]{right:0;left:auto}}.dropup .dropdown-menu[data-bs-popper]{top:auto;bottom:100%;margin-top:0;margin-bottom:var(--bs-dropdown-spacer)}.dropup .dropdown-toggle::after{display:inline-block;margin-left:.255em;vertical-align:.255em;content:"";border-top:0;border-right:.3em solid rgba(0,0,0,0);border-bottom:.3em solid;border-left:.3em solid rgba(0,0,0,0)}.dropup .dropdown-toggle:empty::after{margin-left:0}.dropend .dropdown-menu[data-bs-popper]{top:0;right:auto;left:100%;margin-top:0;margin-left:var(--bs-dropdown-spacer)}.dropend .dropdown-toggle::after{display:inline-block;margin-left:.255em;vertical-align:.255em;content:"";border-top:.3em solid rgba(0,0,0,0);border-right:0;border-bottom:.3em solid rgba(0,0,0,0);border-left:.3em solid}.dropend .dropdown-toggle:empty::after{margin-left:0}.dropend .dropdown-toggle::after{vertical-align:0}.dropstart .dropdown-menu[data-bs-popper]{top:0;right:100%;left:auto;margin-top:0;margin-right:var(--bs-dropdown-spacer)}.dropstart .dropdown-toggle::after{display:inline-block;margin-left:.255em;vertical-align:.255em;content:""}.dropstart .dropdown-toggle::after{display:none}.dropstart .dropdown-toggle::before{display:inline-block;margin-right:.255em;vertical-align:.255em;content:"";border-top:.3em solid rgba(0,0,0,0);border-right:.3em solid;border-bottom:.3em solid rgba(0,0,0,0)}.dropstart .dropdown-toggle:empty::after{margin-left:0}.dropstart .dropdown-toggle::before{vertical-align:0}.dropdown-divider{height:0;margin:var(--bs-dropdown-divider-margin-y) 0;overflow:hidden;border-top:1px solid var(--bs-dropdown-divider-bg);opacity:1}.dropdown-item{display:block;width:100%;padding:var(--bs-dropdown-item-padding-y) var(--bs-dropdown-item-padding-x);clear:both;font-weight:400;color:var(--bs-dropdown-link-color);text-align:inherit;text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;white-space:nowrap;background-color:rgba(0,0,0,0);border:0;border-radius:var(--bs-dropdown-item-border-radius, 0)}.dropdown-item:hover,.dropdown-item:focus{color:var(--bs-dropdown-link-hover-color);background-color:var(--bs-dropdown-link-hover-bg)}.dropdown-item.active,.dropdown-item:active{color:var(--bs-dropdown-link-active-color);text-decoration:none;background-color:var(--bs-dropdown-link-active-bg)}.dropdown-item.disabled,.dropdown-item:disabled{color:var(--bs-dropdown-link-disabled-color);pointer-events:none;background-color:rgba(0,0,0,0)}.dropdown-menu.show{display:block}.dropdown-header{display:block;padding:var(--bs-dropdown-header-padding-y) var(--bs-dropdown-header-padding-x);margin-bottom:0;font-size:0.875rem;color:var(--bs-dropdown-header-color);white-space:nowrap}.dropdown-item-text{display:block;padding:var(--bs-dropdown-item-padding-y) var(--bs-dropdown-item-padding-x);color:var(--bs-dropdown-link-color)}.dropdown-menu-dark{--bs-dropdown-color: #dee2e6;--bs-dropdown-bg: #343a40;--bs-dropdown-border-color: rgba(0, 0, 0, 0.175);--bs-dropdown-box-shadow: ;--bs-dropdown-link-color: #dee2e6;--bs-dropdown-link-hover-color: #fff;--bs-dropdown-divider-bg: rgba(0, 0, 0, 0.175);--bs-dropdown-link-hover-bg: rgba(255, 255, 255, 0.15);--bs-dropdown-link-active-color: #6c757d;--bs-dropdown-link-active-bg: #f8f5f0;--bs-dropdown-link-disabled-color: #adb5bd;--bs-dropdown-header-color: #adb5bd}.btn-group,.btn-group-vertical{position:relative;display:inline-flex;vertical-align:middle}.btn-group>.btn,.btn-group-vertical>.btn{position:relative;flex:1 1 auto;-webkit-flex:1 1 auto}.btn-group>.btn-check:checked+.btn,.btn-group>.btn-check:focus+.btn,.btn-group>.btn:hover,.btn-group>.btn:focus,.btn-group>.btn:active,.btn-group>.btn.active,.btn-group-vertical>.btn-check:checked+.btn,.btn-group-vertical>.btn-check:focus+.btn,.btn-group-vertical>.btn:hover,.btn-group-vertical>.btn:focus,.btn-group-vertical>.btn:active,.btn-group-vertical>.btn.active{z-index:1}.btn-toolbar{display:flex;display:-webkit-flex;flex-wrap:wrap;-webkit-flex-wrap:wrap;justify-content:flex-start;-webkit-justify-content:flex-start}.btn-toolbar .input-group{width:auto}.btn-group{border-radius:.25rem}.btn-group>:not(.btn-check:first-child)+.btn,.btn-group>.btn-group:not(:first-child){margin-left:calc(1px*-1)}.btn-group>.btn:not(:last-child):not(.dropdown-toggle),.btn-group>.btn.dropdown-toggle-split:first-child,.btn-group>.btn-group:not(:last-child)>.btn{border-top-right-radius:0;border-bottom-right-radius:0}.btn-group>.btn:nth-child(n+3),.btn-group>:not(.btn-check)+.btn,.btn-group>.btn-group:not(:first-child)>.btn{border-top-left-radius:0;border-bottom-left-radius:0}.dropdown-toggle-split{padding-right:.5625rem;padding-left:.5625rem}.dropdown-toggle-split::after,.dropup .dropdown-toggle-split::after,.dropend .dropdown-toggle-split::after{margin-left:0}.dropstart .dropdown-toggle-split::before{margin-right:0}.btn-sm+.dropdown-toggle-split,.btn-group-sm>.btn+.dropdown-toggle-split{padding-right:.375rem;padding-left:.375rem}.btn-lg+.dropdown-toggle-split,.btn-group-lg>.btn+.dropdown-toggle-split{padding-right:.75rem;padding-left:.75rem}.btn-group-vertical{flex-direction:column;-webkit-flex-direction:column;align-items:flex-start;-webkit-align-items:flex-start;justify-content:center;-webkit-justify-content:center}.btn-group-vertical>.btn,.btn-group-vertical>.btn-group{width:100%}.btn-group-vertical>.btn:not(:first-child),.btn-group-vertical>.btn-group:not(:first-child){margin-top:calc(1px*-1)}.btn-group-vertical>.btn:not(:last-child):not(.dropdown-toggle),.btn-group-vertical>.btn-group:not(:last-child)>.btn{border-bottom-right-radius:0;border-bottom-left-radius:0}.btn-group-vertical>.btn~.btn,.btn-group-vertical>.btn-group:not(:first-child)>.btn{border-top-left-radius:0;border-top-right-radius:0}.nav{--bs-nav-link-padding-x: 0.9rem;--bs-nav-link-padding-y: 0.5rem;--bs-nav-link-font-weight: ;--bs-nav-link-color: #93c54b;--bs-nav-link-hover-color: #769e3c;--bs-nav-link-disabled-color: #dee2e6;display:flex;display:-webkit-flex;flex-wrap:wrap;-webkit-flex-wrap:wrap;padding-left:0;margin-bottom:0;list-style:none}.nav-link{display:block;padding:var(--bs-nav-link-padding-y) var(--bs-nav-link-padding-x);font-size:var(--bs-nav-link-font-size);font-weight:var(--bs-nav-link-font-weight);color:var(--bs-nav-link-color);text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;background:none;border:0;transition:color .15s ease-in-out,background-color .15s ease-in-out,border-color .15s ease-in-out}@media(prefers-reduced-motion: reduce){.nav-link{transition:none}}.nav-link:hover,.nav-link:focus{color:var(--bs-nav-link-hover-color)}.nav-link:focus-visible{outline:0;box-shadow:0 0 0 .25rem rgba(50,93,136,.25)}.nav-link.disabled,.nav-link:disabled{color:var(--bs-nav-link-disabled-color);pointer-events:none;cursor:default}.nav-tabs{--bs-nav-tabs-border-width: 1px;--bs-nav-tabs-border-color: #dee2e6;--bs-nav-tabs-border-radius: 0.25rem;--bs-nav-tabs-link-hover-border-color: #dee2e6;--bs-nav-tabs-link-active-color: #000;--bs-nav-tabs-link-active-bg: #fff;--bs-nav-tabs-link-active-border-color: #dee2e6 #dee2e6 #fff;border-bottom:var(--bs-nav-tabs-border-width) solid var(--bs-nav-tabs-border-color)}.nav-tabs .nav-link{margin-bottom:calc(-1*var(--bs-nav-tabs-border-width));border:var(--bs-nav-tabs-border-width) solid rgba(0,0,0,0);border-top-left-radius:var(--bs-nav-tabs-border-radius);border-top-right-radius:var(--bs-nav-tabs-border-radius)}.nav-tabs .nav-link:hover,.nav-tabs .nav-link:focus{isolation:isolate;border-color:var(--bs-nav-tabs-link-hover-border-color)}.nav-tabs .nav-link.active,.nav-tabs .nav-item.show .nav-link{color:var(--bs-nav-tabs-link-active-color);background-color:var(--bs-nav-tabs-link-active-bg);border-color:var(--bs-nav-tabs-link-active-border-color)}.nav-tabs .dropdown-menu{margin-top:calc(-1*var(--bs-nav-tabs-border-width));border-top-left-radius:0;border-top-right-radius:0}.nav-pills{--bs-nav-pills-border-radius: 0.25rem;--bs-nav-pills-link-active-color: #6c757d;--bs-nav-pills-link-active-bg: #f8f5f0}.nav-pills .nav-link{border-radius:var(--bs-nav-pills-border-radius)}.nav-pills .nav-link.active,.nav-pills .show>.nav-link{color:var(--bs-nav-pills-link-active-color);background-color:var(--bs-nav-pills-link-active-bg)}.nav-underline{--bs-nav-underline-gap: 1rem;--bs-nav-underline-border-width: 0.125rem;--bs-nav-underline-link-active-color: #000;gap:var(--bs-nav-underline-gap)}.nav-underline .nav-link{padding-right:0;padding-left:0;border-bottom:var(--bs-nav-underline-border-width) solid rgba(0,0,0,0)}.nav-underline .nav-link:hover,.nav-underline .nav-link:focus{border-bottom-color:currentcolor}.nav-underline .nav-link.active,.nav-underline .show>.nav-link{font-weight:700;color:var(--bs-nav-underline-link-active-color);border-bottom-color:currentcolor}.nav-fill>.nav-link,.nav-fill .nav-item{flex:1 1 auto;-webkit-flex:1 1 auto;text-align:center}.nav-justified>.nav-link,.nav-justified .nav-item{flex-basis:0;-webkit-flex-basis:0;flex-grow:1;-webkit-flex-grow:1;text-align:center}.nav-fill .nav-item .nav-link,.nav-justified .nav-item .nav-link{width:100%}.tab-content>.tab-pane{display:none}.tab-content>.active{display:block}.navbar{--bs-navbar-padding-x: 0;--bs-navbar-padding-y: 0.5rem;--bs-navbar-color: rgba(255, 255, 255, 0.7);--bs-navbar-hover-color: rgba(254, 254, 253, 0.8);--bs-navbar-disabled-color: rgba(255, 255, 255, 0.75);--bs-navbar-active-color: #fefefd;--bs-navbar-brand-padding-y: 0.3125rem;--bs-navbar-brand-margin-end: 1rem;--bs-navbar-brand-font-size: 1.25rem;--bs-navbar-brand-color: rgba(255, 255, 255, 0.7);--bs-navbar-brand-hover-color: #fefefd;--bs-navbar-nav-link-padding-x: 0.5rem;--bs-navbar-toggler-padding-y: 0.25;--bs-navbar-toggler-padding-x: 0;--bs-navbar-toggler-font-size: 1.25rem;--bs-navbar-toggler-icon-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'%3e%3cpath stroke='rgba%28255, 255, 255, 0.7%29' stroke-linecap='round' stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/%3e%3c/svg%3e");--bs-navbar-toggler-border-color: rgba(255, 255, 255, 0);--bs-navbar-toggler-border-radius: 0.25rem;--bs-navbar-toggler-focus-width: 0.25rem;--bs-navbar-toggler-transition: box-shadow 0.15s ease-in-out;position:relative;display:flex;display:-webkit-flex;flex-wrap:wrap;-webkit-flex-wrap:wrap;align-items:center;-webkit-align-items:center;justify-content:space-between;-webkit-justify-content:space-between;padding:var(--bs-navbar-padding-y) var(--bs-navbar-padding-x)}.navbar>.container,.navbar>.container-fluid,.navbar>.container-sm,.navbar>.container-md,.navbar>.container-lg,.navbar>.container-xl,.navbar>.container-xxl{display:flex;display:-webkit-flex;flex-wrap:inherit;-webkit-flex-wrap:inherit;align-items:center;-webkit-align-items:center;justify-content:space-between;-webkit-justify-content:space-between}.navbar-brand{padding-top:var(--bs-navbar-brand-padding-y);padding-bottom:var(--bs-navbar-brand-padding-y);margin-right:var(--bs-navbar-brand-margin-end);font-size:var(--bs-navbar-brand-font-size);color:var(--bs-navbar-brand-color);text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;white-space:nowrap}.navbar-brand:hover,.navbar-brand:focus{color:var(--bs-navbar-brand-hover-color)}.navbar-nav{--bs-nav-link-padding-x: 0;--bs-nav-link-padding-y: 0.5rem;--bs-nav-link-font-weight: ;--bs-nav-link-color: var(--bs-navbar-color);--bs-nav-link-hover-color: var(--bs-navbar-hover-color);--bs-nav-link-disabled-color: var(--bs-navbar-disabled-color);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;padding-left:0;margin-bottom:0;list-style:none}.navbar-nav .nav-link.active,.navbar-nav .nav-link.show{color:var(--bs-navbar-active-color)}.navbar-nav .dropdown-menu{position:static}.navbar-text{padding-top:.5rem;padding-bottom:.5rem;color:var(--bs-navbar-color)}.navbar-text a,.navbar-text a:hover,.navbar-text a:focus{color:var(--bs-navbar-active-color)}.navbar-collapse{flex-basis:100%;-webkit-flex-basis:100%;flex-grow:1;-webkit-flex-grow:1;align-items:center;-webkit-align-items:center}.navbar-toggler{padding:var(--bs-navbar-toggler-padding-y) var(--bs-navbar-toggler-padding-x);font-size:var(--bs-navbar-toggler-font-size);line-height:1;color:var(--bs-navbar-color);background-color:rgba(0,0,0,0);border:var(--bs-border-width) solid var(--bs-navbar-toggler-border-color);border-radius:var(--bs-navbar-toggler-border-radius);transition:var(--bs-navbar-toggler-transition)}@media(prefers-reduced-motion: reduce){.navbar-toggler{transition:none}}.navbar-toggler:hover{text-decoration:none}.navbar-toggler:focus{text-decoration:none;outline:0;box-shadow:0 0 0 var(--bs-navbar-toggler-focus-width)}.navbar-toggler-icon{display:inline-block;width:1.5em;height:1.5em;vertical-align:middle;background-image:var(--bs-navbar-toggler-icon-bg);background-repeat:no-repeat;background-position:center;background-size:100%}.navbar-nav-scroll{max-height:var(--bs-scroll-height, 75vh);overflow-y:auto}@media(min-width: 576px){.navbar-expand-sm{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand-sm .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand-sm .navbar-nav .dropdown-menu{position:absolute}.navbar-expand-sm .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand-sm .navbar-nav-scroll{overflow:visible}.navbar-expand-sm .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand-sm .navbar-toggler{display:none}.navbar-expand-sm .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand-sm .offcanvas .offcanvas-header{display:none}.navbar-expand-sm .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}}@media(min-width: 768px){.navbar-expand-md{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand-md .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand-md .navbar-nav .dropdown-menu{position:absolute}.navbar-expand-md .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand-md .navbar-nav-scroll{overflow:visible}.navbar-expand-md .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand-md .navbar-toggler{display:none}.navbar-expand-md .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand-md .offcanvas .offcanvas-header{display:none}.navbar-expand-md .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}}@media(min-width: 992px){.navbar-expand-lg{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand-lg .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand-lg .navbar-nav .dropdown-menu{position:absolute}.navbar-expand-lg .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand-lg .navbar-nav-scroll{overflow:visible}.navbar-expand-lg .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand-lg .navbar-toggler{display:none}.navbar-expand-lg .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand-lg .offcanvas .offcanvas-header{display:none}.navbar-expand-lg .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}}@media(min-width: 1200px){.navbar-expand-xl{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand-xl .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand-xl .navbar-nav .dropdown-menu{position:absolute}.navbar-expand-xl .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand-xl .navbar-nav-scroll{overflow:visible}.navbar-expand-xl .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand-xl .navbar-toggler{display:none}.navbar-expand-xl .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand-xl .offcanvas .offcanvas-header{display:none}.navbar-expand-xl .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}}@media(min-width: 1400px){.navbar-expand-xxl{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand-xxl .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand-xxl .navbar-nav .dropdown-menu{position:absolute}.navbar-expand-xxl .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand-xxl .navbar-nav-scroll{overflow:visible}.navbar-expand-xxl .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand-xxl .navbar-toggler{display:none}.navbar-expand-xxl .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand-xxl .offcanvas .offcanvas-header{display:none}.navbar-expand-xxl .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}}.navbar-expand{flex-wrap:nowrap;-webkit-flex-wrap:nowrap;justify-content:flex-start;-webkit-justify-content:flex-start}.navbar-expand .navbar-nav{flex-direction:row;-webkit-flex-direction:row}.navbar-expand .navbar-nav .dropdown-menu{position:absolute}.navbar-expand .navbar-nav .nav-link{padding-right:var(--bs-navbar-nav-link-padding-x);padding-left:var(--bs-navbar-nav-link-padding-x)}.navbar-expand .navbar-nav-scroll{overflow:visible}.navbar-expand .navbar-collapse{display:flex !important;display:-webkit-flex !important;flex-basis:auto;-webkit-flex-basis:auto}.navbar-expand .navbar-toggler{display:none}.navbar-expand .offcanvas{position:static;z-index:auto;flex-grow:1;-webkit-flex-grow:1;width:auto !important;height:auto !important;visibility:visible !important;background-color:rgba(0,0,0,0) !important;border:0 !important;transform:none !important;transition:none}.navbar-expand .offcanvas .offcanvas-header{display:none}.navbar-expand .offcanvas .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible}.navbar-dark,.navbar[data-bs-theme=dark]{--bs-navbar-color: rgba(255, 255, 255, 0.7);--bs-navbar-hover-color: rgba(254, 254, 253, 0.8);--bs-navbar-disabled-color: rgba(255, 255, 255, 0.75);--bs-navbar-active-color: #fefefd;--bs-navbar-brand-color: rgba(255, 255, 255, 0.7);--bs-navbar-brand-hover-color: #fefefd;--bs-navbar-toggler-border-color: rgba(255, 255, 255, 0);--bs-navbar-toggler-icon-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'%3e%3cpath stroke='rgba%28255, 255, 255, 0.7%29' stroke-linecap='round' stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/%3e%3c/svg%3e")}[data-bs-theme=dark] .navbar-toggler-icon{--bs-navbar-toggler-icon-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 30 30'%3e%3cpath stroke='rgba%28255, 255, 255, 0.7%29' stroke-linecap='round' stroke-miterlimit='10' stroke-width='2' d='M4 7h22M4 15h22M4 23h22'/%3e%3c/svg%3e")}.card{--bs-card-spacer-y: 1rem;--bs-card-spacer-x: 1rem;--bs-card-title-spacer-y: 0.5rem;--bs-card-title-color: ;--bs-card-subtitle-color: ;--bs-card-border-width: 1px;--bs-card-border-color: rgba(222, 226, 230, 0.75);--bs-card-border-radius: 0.25rem;--bs-card-box-shadow: ;--bs-card-inner-border-radius: calc(0.25rem - 1px);--bs-card-cap-padding-y: 0.5rem;--bs-card-cap-padding-x: 1rem;--bs-card-cap-bg: rgba(52, 58, 64, 0.25);--bs-card-cap-color: ;--bs-card-height: ;--bs-card-color: ;--bs-card-bg: #fff;--bs-card-img-overlay-padding: 1rem;--bs-card-group-margin: 0.75rem;position:relative;display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;min-width:0;height:var(--bs-card-height);color:var(--bs-body-color);word-wrap:break-word;background-color:var(--bs-card-bg);background-clip:border-box;border:var(--bs-card-border-width) solid var(--bs-card-border-color);border-radius:var(--bs-card-border-radius)}.card>hr{margin-right:0;margin-left:0}.card>.list-group{border-top:inherit;border-bottom:inherit}.card>.list-group:first-child{border-top-width:0;border-top-left-radius:var(--bs-card-inner-border-radius);border-top-right-radius:var(--bs-card-inner-border-radius)}.card>.list-group:last-child{border-bottom-width:0;border-bottom-right-radius:var(--bs-card-inner-border-radius);border-bottom-left-radius:var(--bs-card-inner-border-radius)}.card>.card-header+.list-group,.card>.list-group+.card-footer{border-top:0}.card-body{flex:1 1 auto;-webkit-flex:1 1 auto;padding:var(--bs-card-spacer-y) var(--bs-card-spacer-x);color:var(--bs-card-color)}.card-title{margin-bottom:var(--bs-card-title-spacer-y);color:var(--bs-card-title-color)}.card-subtitle{margin-top:calc(-0.5*var(--bs-card-title-spacer-y));margin-bottom:0;color:var(--bs-card-subtitle-color)}.card-text:last-child{margin-bottom:0}.card-link+.card-link{margin-left:var(--bs-card-spacer-x)}.card-header{padding:var(--bs-card-cap-padding-y) var(--bs-card-cap-padding-x);margin-bottom:0;color:var(--bs-card-cap-color);background-color:var(--bs-card-cap-bg);border-bottom:var(--bs-card-border-width) solid var(--bs-card-border-color)}.card-header:first-child{border-radius:var(--bs-card-inner-border-radius) var(--bs-card-inner-border-radius) 0 0}.card-footer{padding:var(--bs-card-cap-padding-y) var(--bs-card-cap-padding-x);color:var(--bs-card-cap-color);background-color:var(--bs-card-cap-bg);border-top:var(--bs-card-border-width) solid var(--bs-card-border-color)}.card-footer:last-child{border-radius:0 0 var(--bs-card-inner-border-radius) var(--bs-card-inner-border-radius)}.card-header-tabs{margin-right:calc(-0.5*var(--bs-card-cap-padding-x));margin-bottom:calc(-1*var(--bs-card-cap-padding-y));margin-left:calc(-0.5*var(--bs-card-cap-padding-x));border-bottom:0}.card-header-tabs .nav-link.active{background-color:var(--bs-card-bg);border-bottom-color:var(--bs-card-bg)}.card-header-pills{margin-right:calc(-0.5*var(--bs-card-cap-padding-x));margin-left:calc(-0.5*var(--bs-card-cap-padding-x))}.card-img-overlay{position:absolute;top:0;right:0;bottom:0;left:0;padding:var(--bs-card-img-overlay-padding);border-radius:var(--bs-card-inner-border-radius)}.card-img,.card-img-top,.card-img-bottom{width:100%}.card-img,.card-img-top{border-top-left-radius:var(--bs-card-inner-border-radius);border-top-right-radius:var(--bs-card-inner-border-radius)}.card-img,.card-img-bottom{border-bottom-right-radius:var(--bs-card-inner-border-radius);border-bottom-left-radius:var(--bs-card-inner-border-radius)}.card-group>.card{margin-bottom:var(--bs-card-group-margin)}@media(min-width: 576px){.card-group{display:flex;display:-webkit-flex;flex-flow:row wrap;-webkit-flex-flow:row wrap}.card-group>.card{flex:1 0 0%;-webkit-flex:1 0 0%;margin-bottom:0}.card-group>.card+.card{margin-left:0;border-left:0}.card-group>.card:not(:last-child){border-top-right-radius:0;border-bottom-right-radius:0}.card-group>.card:not(:last-child) .card-img-top,.card-group>.card:not(:last-child) .card-header{border-top-right-radius:0}.card-group>.card:not(:last-child) .card-img-bottom,.card-group>.card:not(:last-child) .card-footer{border-bottom-right-radius:0}.card-group>.card:not(:first-child){border-top-left-radius:0;border-bottom-left-radius:0}.card-group>.card:not(:first-child) .card-img-top,.card-group>.card:not(:first-child) .card-header{border-top-left-radius:0}.card-group>.card:not(:first-child) .card-img-bottom,.card-group>.card:not(:first-child) .card-footer{border-bottom-left-radius:0}}.accordion{--bs-accordion-color: #343a40;--bs-accordion-bg: #fff;--bs-accordion-transition: color 0.15s ease-in-out, background-color 0.15s ease-in-out, border-color 0.15s ease-in-out, box-shadow 0.15s ease-in-out, border-radius 0.15s ease;--bs-accordion-border-color: #dee2e6;--bs-accordion-border-width: 1px;--bs-accordion-border-radius: 0.25rem;--bs-accordion-inner-border-radius: calc(0.25rem - 1px);--bs-accordion-btn-padding-x: 1.25rem;--bs-accordion-btn-padding-y: 1rem;--bs-accordion-btn-color: #343a40;--bs-accordion-btn-bg: #fff;--bs-accordion-btn-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23343a40'%3e%3cpath fill-rule='evenodd' d='M1.646 4.646a.5.5 0 0 1 .708 0L8 10.293l5.646-5.647a.5.5 0 0 1 .708.708l-6 6a.5.5 0 0 1-.708 0l-6-6a.5.5 0 0 1 0-.708z'/%3e%3c/svg%3e");--bs-accordion-btn-icon-width: 1.25rem;--bs-accordion-btn-icon-transform: rotate(-180deg);--bs-accordion-btn-icon-transition: transform 0.2s ease-in-out;--bs-accordion-btn-active-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23142536'%3e%3cpath fill-rule='evenodd' d='M1.646 4.646a.5.5 0 0 1 .708 0L8 10.293l5.646-5.647a.5.5 0 0 1 .708.708l-6 6a.5.5 0 0 1-.708 0l-6-6a.5.5 0 0 1 0-.708z'/%3e%3c/svg%3e");--bs-accordion-btn-focus-border-color: #99aec4;--bs-accordion-btn-focus-box-shadow: 0 0 0 0.25rem rgba(50, 93, 136, 0.25);--bs-accordion-body-padding-x: 1.25rem;--bs-accordion-body-padding-y: 1rem;--bs-accordion-active-color: #142536;--bs-accordion-active-bg: #d6dfe7}.accordion-button{position:relative;display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;width:100%;padding:var(--bs-accordion-btn-padding-y) var(--bs-accordion-btn-padding-x);font-size:1rem;color:var(--bs-accordion-btn-color);text-align:left;background-color:var(--bs-accordion-btn-bg);border:0;border-radius:0;overflow-anchor:none;transition:var(--bs-accordion-transition)}@media(prefers-reduced-motion: reduce){.accordion-button{transition:none}}.accordion-button:not(.collapsed){color:var(--bs-accordion-active-color);background-color:var(--bs-accordion-active-bg);box-shadow:inset 0 calc(-1*var(--bs-accordion-border-width)) 0 var(--bs-accordion-border-color)}.accordion-button:not(.collapsed)::after{background-image:var(--bs-accordion-btn-active-icon);transform:var(--bs-accordion-btn-icon-transform)}.accordion-button::after{flex-shrink:0;-webkit-flex-shrink:0;width:var(--bs-accordion-btn-icon-width);height:var(--bs-accordion-btn-icon-width);margin-left:auto;content:"";background-image:var(--bs-accordion-btn-icon);background-repeat:no-repeat;background-size:var(--bs-accordion-btn-icon-width);transition:var(--bs-accordion-btn-icon-transition)}@media(prefers-reduced-motion: reduce){.accordion-button::after{transition:none}}.accordion-button:hover{z-index:2}.accordion-button:focus{z-index:3;border-color:var(--bs-accordion-btn-focus-border-color);outline:0;box-shadow:var(--bs-accordion-btn-focus-box-shadow)}.accordion-header{margin-bottom:0}.accordion-item{color:var(--bs-accordion-color);background-color:var(--bs-accordion-bg);border:var(--bs-accordion-border-width) solid var(--bs-accordion-border-color)}.accordion-item:first-of-type{border-top-left-radius:var(--bs-accordion-border-radius);border-top-right-radius:var(--bs-accordion-border-radius)}.accordion-item:first-of-type .accordion-button{border-top-left-radius:var(--bs-accordion-inner-border-radius);border-top-right-radius:var(--bs-accordion-inner-border-radius)}.accordion-item:not(:first-of-type){border-top:0}.accordion-item:last-of-type{border-bottom-right-radius:var(--bs-accordion-border-radius);border-bottom-left-radius:var(--bs-accordion-border-radius)}.accordion-item:last-of-type .accordion-button.collapsed{border-bottom-right-radius:var(--bs-accordion-inner-border-radius);border-bottom-left-radius:var(--bs-accordion-inner-border-radius)}.accordion-item:last-of-type .accordion-collapse{border-bottom-right-radius:var(--bs-accordion-border-radius);border-bottom-left-radius:var(--bs-accordion-border-radius)}.accordion-body{padding:var(--bs-accordion-body-padding-y) var(--bs-accordion-body-padding-x)}.accordion-flush .accordion-collapse{border-width:0}.accordion-flush .accordion-item{border-right:0;border-left:0;border-radius:0}.accordion-flush .accordion-item:first-child{border-top:0}.accordion-flush .accordion-item:last-child{border-bottom:0}.accordion-flush .accordion-item .accordion-button,.accordion-flush .accordion-item .accordion-button.collapsed{border-radius:0}[data-bs-theme=dark] .accordion-button::after{--bs-accordion-btn-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23849eb8'%3e%3cpath fill-rule='evenodd' d='M1.646 4.646a.5.5 0 0 1 .708 0L8 10.293l5.646-5.647a.5.5 0 0 1 .708.708l-6 6a.5.5 0 0 1-.708 0l-6-6a.5.5 0 0 1 0-.708z'/%3e%3c/svg%3e");--bs-accordion-btn-active-icon: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23849eb8'%3e%3cpath fill-rule='evenodd' d='M1.646 4.646a.5.5 0 0 1 .708 0L8 10.293l5.646-5.647a.5.5 0 0 1 .708.708l-6 6a.5.5 0 0 1-.708 0l-6-6a.5.5 0 0 1 0-.708z'/%3e%3c/svg%3e")}.breadcrumb{--bs-breadcrumb-padding-x: 0.75rem;--bs-breadcrumb-padding-y: 0.375rem;--bs-breadcrumb-margin-bottom: 1rem;--bs-breadcrumb-bg: #f8f5f0;--bs-breadcrumb-border-radius: 0.25rem;--bs-breadcrumb-divider-color: rgba(52, 58, 64, 0.75);--bs-breadcrumb-item-padding-x: 0.5rem;--bs-breadcrumb-item-active-color: rgba(52, 58, 64, 0.75);display:flex;display:-webkit-flex;flex-wrap:wrap;-webkit-flex-wrap:wrap;padding:var(--bs-breadcrumb-padding-y) var(--bs-breadcrumb-padding-x);margin-bottom:var(--bs-breadcrumb-margin-bottom);font-size:var(--bs-breadcrumb-font-size);list-style:none;background-color:var(--bs-breadcrumb-bg);border-radius:var(--bs-breadcrumb-border-radius)}.breadcrumb-item+.breadcrumb-item{padding-left:var(--bs-breadcrumb-item-padding-x)}.breadcrumb-item+.breadcrumb-item::before{float:left;padding-right:var(--bs-breadcrumb-item-padding-x);color:var(--bs-breadcrumb-divider-color);content:var(--bs-breadcrumb-divider, ">") /* rtl: var(--bs-breadcrumb-divider, ">") */}.breadcrumb-item.active{color:var(--bs-breadcrumb-item-active-color)}.pagination{--bs-pagination-padding-x: 0.75rem;--bs-pagination-padding-y: 0.375rem;--bs-pagination-font-size:1rem;--bs-pagination-color: #6c757d;--bs-pagination-bg: #f8f5f0;--bs-pagination-border-width: 1px;--bs-pagination-border-color: #dee2e6;--bs-pagination-border-radius: 0.25rem;--bs-pagination-hover-color: #6c757d;--bs-pagination-hover-bg: #f8f9fa;--bs-pagination-hover-border-color: #dee2e6;--bs-pagination-focus-color: #769e3c;--bs-pagination-focus-bg: #f8f5f0;--bs-pagination-focus-box-shadow: 0 0 0 0.25rem rgba(50, 93, 136, 0.25);--bs-pagination-active-color: #6c757d;--bs-pagination-active-bg: #dee2e6;--bs-pagination-active-border-color: #dee2e6;--bs-pagination-disabled-color: #dee2e6;--bs-pagination-disabled-bg: #f8f5f0;--bs-pagination-disabled-border-color: #dee2e6;display:flex;display:-webkit-flex;padding-left:0;list-style:none}.page-link{position:relative;display:block;padding:var(--bs-pagination-padding-y) var(--bs-pagination-padding-x);font-size:var(--bs-pagination-font-size);color:var(--bs-pagination-color);text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;background-color:var(--bs-pagination-bg);border:var(--bs-pagination-border-width) solid var(--bs-pagination-border-color);transition:color .15s ease-in-out,background-color .15s ease-in-out,border-color .15s ease-in-out,box-shadow .15s ease-in-out}@media(prefers-reduced-motion: reduce){.page-link{transition:none}}.page-link:hover{z-index:2;color:var(--bs-pagination-hover-color);background-color:var(--bs-pagination-hover-bg);border-color:var(--bs-pagination-hover-border-color)}.page-link:focus{z-index:3;color:var(--bs-pagination-focus-color);background-color:var(--bs-pagination-focus-bg);outline:0;box-shadow:var(--bs-pagination-focus-box-shadow)}.page-link.active,.active>.page-link{z-index:3;color:var(--bs-pagination-active-color);background-color:var(--bs-pagination-active-bg);border-color:var(--bs-pagination-active-border-color)}.page-link.disabled,.disabled>.page-link{color:var(--bs-pagination-disabled-color);pointer-events:none;background-color:var(--bs-pagination-disabled-bg);border-color:var(--bs-pagination-disabled-border-color)}.page-item:not(:first-child) .page-link{margin-left:calc(1px*-1)}.page-item:first-child .page-link{border-top-left-radius:var(--bs-pagination-border-radius);border-bottom-left-radius:var(--bs-pagination-border-radius)}.page-item:last-child .page-link{border-top-right-radius:var(--bs-pagination-border-radius);border-bottom-right-radius:var(--bs-pagination-border-radius)}.pagination-lg{--bs-pagination-padding-x: 1.5rem;--bs-pagination-padding-y: 0.75rem;--bs-pagination-font-size:1.25rem;--bs-pagination-border-radius: 0.5rem}.pagination-sm{--bs-pagination-padding-x: 0.5rem;--bs-pagination-padding-y: 0.25rem;--bs-pagination-font-size:0.875rem;--bs-pagination-border-radius: 0.2em}.badge{--bs-badge-padding-x: 0.65em;--bs-badge-padding-y: 0.35em;--bs-badge-font-size:0.75em;--bs-badge-font-weight: 700;--bs-badge-color: #fff;--bs-badge-border-radius: 0.25rem;display:inline-block;padding:var(--bs-badge-padding-y) var(--bs-badge-padding-x);font-size:var(--bs-badge-font-size);font-weight:var(--bs-badge-font-weight);line-height:1;color:var(--bs-badge-color);text-align:center;white-space:nowrap;vertical-align:baseline;border-radius:var(--bs-badge-border-radius)}.badge:empty{display:none}.btn .badge{position:relative;top:-1px}.alert{--bs-alert-bg: transparent;--bs-alert-padding-x: 1rem;--bs-alert-padding-y: 1rem;--bs-alert-margin-bottom: 1rem;--bs-alert-color: inherit;--bs-alert-border-color: transparent;--bs-alert-border: 1px solid var(--bs-alert-border-color);--bs-alert-border-radius: 0.25rem;--bs-alert-link-color: inherit;position:relative;padding:var(--bs-alert-padding-y) var(--bs-alert-padding-x);margin-bottom:var(--bs-alert-margin-bottom);color:var(--bs-alert-color);background-color:var(--bs-alert-bg);border:var(--bs-alert-border);border-radius:var(--bs-alert-border-radius)}.alert-heading{color:inherit}.alert-link{font-weight:700;color:var(--bs-alert-link-color)}.alert-dismissible{padding-right:3rem}.alert-dismissible .btn-close{position:absolute;top:0;right:0;z-index:2;padding:1.25rem 1rem}.alert-default{--bs-alert-color: var(--bs-default-text-emphasis);--bs-alert-bg: var(--bs-default-bg-subtle);--bs-alert-border-color: var(--bs-default-border-subtle);--bs-alert-link-color: var(--bs-default-text-emphasis)}.alert-primary{--bs-alert-color: var(--bs-primary-text-emphasis);--bs-alert-bg: var(--bs-primary-bg-subtle);--bs-alert-border-color: var(--bs-primary-border-subtle);--bs-alert-link-color: var(--bs-primary-text-emphasis)}.alert-secondary{--bs-alert-color: var(--bs-secondary-text-emphasis);--bs-alert-bg: var(--bs-secondary-bg-subtle);--bs-alert-border-color: var(--bs-secondary-border-subtle);--bs-alert-link-color: var(--bs-secondary-text-emphasis)}.alert-success{--bs-alert-color: var(--bs-success-text-emphasis);--bs-alert-bg: var(--bs-success-bg-subtle);--bs-alert-border-color: var(--bs-success-border-subtle);--bs-alert-link-color: var(--bs-success-text-emphasis)}.alert-info{--bs-alert-color: var(--bs-info-text-emphasis);--bs-alert-bg: var(--bs-info-bg-subtle);--bs-alert-border-color: var(--bs-info-border-subtle);--bs-alert-link-color: var(--bs-info-text-emphasis)}.alert-warning{--bs-alert-color: var(--bs-warning-text-emphasis);--bs-alert-bg: var(--bs-warning-bg-subtle);--bs-alert-border-color: var(--bs-warning-border-subtle);--bs-alert-link-color: var(--bs-warning-text-emphasis)}.alert-danger{--bs-alert-color: var(--bs-danger-text-emphasis);--bs-alert-bg: var(--bs-danger-bg-subtle);--bs-alert-border-color: var(--bs-danger-border-subtle);--bs-alert-link-color: var(--bs-danger-text-emphasis)}.alert-light{--bs-alert-color: var(--bs-light-text-emphasis);--bs-alert-bg: var(--bs-light-bg-subtle);--bs-alert-border-color: var(--bs-light-border-subtle);--bs-alert-link-color: var(--bs-light-text-emphasis)}.alert-dark{--bs-alert-color: var(--bs-dark-text-emphasis);--bs-alert-bg: var(--bs-dark-bg-subtle);--bs-alert-border-color: var(--bs-dark-border-subtle);--bs-alert-link-color: var(--bs-dark-text-emphasis)}@keyframes progress-bar-stripes{0%{background-position-x:1rem}}.progress,.progress-stacked{--bs-progress-height: 1rem;--bs-progress-font-size:0.75rem;--bs-progress-bg: #dee2e6;--bs-progress-border-radius: 10px;--bs-progress-box-shadow: inset 0 1px 2px rgba(0, 0, 0, 0.075);--bs-progress-bar-color: #325d88;--bs-progress-bar-bg: #325d88;--bs-progress-bar-transition: width 0.6s ease;display:flex;display:-webkit-flex;height:var(--bs-progress-height);overflow:hidden;font-size:var(--bs-progress-font-size);background-color:var(--bs-progress-bg);border-radius:var(--bs-progress-border-radius)}.progress-bar{display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;justify-content:center;-webkit-justify-content:center;overflow:hidden;color:var(--bs-progress-bar-color);text-align:center;white-space:nowrap;background-color:var(--bs-progress-bar-bg);transition:var(--bs-progress-bar-transition)}@media(prefers-reduced-motion: reduce){.progress-bar{transition:none}}.progress-bar-striped{background-image:linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);background-size:var(--bs-progress-height) var(--bs-progress-height)}.progress-stacked>.progress{overflow:visible}.progress-stacked>.progress>.progress-bar{width:100%}.progress-bar-animated{animation:1s linear infinite progress-bar-stripes}@media(prefers-reduced-motion: reduce){.progress-bar-animated{animation:none}}.list-group{--bs-list-group-color: #343a40;--bs-list-group-bg: #fff;--bs-list-group-border-color: #dee2e6;--bs-list-group-border-width: 1px;--bs-list-group-border-radius: 0.25rem;--bs-list-group-item-padding-x: 1rem;--bs-list-group-item-padding-y: 0.5rem;--bs-list-group-action-color: #343a40;--bs-list-group-action-hover-color: #000;--bs-list-group-action-hover-bg: #f8f5f0;--bs-list-group-action-active-color: #343a40;--bs-list-group-action-active-bg: #dee2e6;--bs-list-group-disabled-color: #adb5bd;--bs-list-group-disabled-bg: #fff;--bs-list-group-active-color: #343a40;--bs-list-group-active-bg: #f8f5f0;--bs-list-group-active-border-color: #dee2e6;display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;padding-left:0;margin-bottom:0;border-radius:var(--bs-list-group-border-radius)}.list-group-numbered{list-style-type:none;counter-reset:section}.list-group-numbered>.list-group-item::before{content:counters(section, ".") ". ";counter-increment:section}.list-group-item-action{width:100%;color:var(--bs-list-group-action-color);text-align:inherit}.list-group-item-action:hover,.list-group-item-action:focus{z-index:1;color:var(--bs-list-group-action-hover-color);text-decoration:none;background-color:var(--bs-list-group-action-hover-bg)}.list-group-item-action:active{color:var(--bs-list-group-action-active-color);background-color:var(--bs-list-group-action-active-bg)}.list-group-item{position:relative;display:block;padding:var(--bs-list-group-item-padding-y) var(--bs-list-group-item-padding-x);color:var(--bs-list-group-color);text-decoration:none;-webkit-text-decoration:none;-moz-text-decoration:none;-ms-text-decoration:none;-o-text-decoration:none;background-color:var(--bs-list-group-bg);border:var(--bs-list-group-border-width) solid var(--bs-list-group-border-color)}.list-group-item:first-child{border-top-left-radius:inherit;border-top-right-radius:inherit}.list-group-item:last-child{border-bottom-right-radius:inherit;border-bottom-left-radius:inherit}.list-group-item.disabled,.list-group-item:disabled{color:var(--bs-list-group-disabled-color);pointer-events:none;background-color:var(--bs-list-group-disabled-bg)}.list-group-item.active{z-index:2;color:var(--bs-list-group-active-color);background-color:var(--bs-list-group-active-bg);border-color:var(--bs-list-group-active-border-color)}.list-group-item+.list-group-item{border-top-width:0}.list-group-item+.list-group-item.active{margin-top:calc(-1*var(--bs-list-group-border-width));border-top-width:var(--bs-list-group-border-width)}.list-group-horizontal{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal>.list-group-item.active{margin-top:0}.list-group-horizontal>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}@media(min-width: 576px){.list-group-horizontal-sm{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal-sm>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal-sm>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal-sm>.list-group-item.active{margin-top:0}.list-group-horizontal-sm>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal-sm>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}}@media(min-width: 768px){.list-group-horizontal-md{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal-md>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal-md>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal-md>.list-group-item.active{margin-top:0}.list-group-horizontal-md>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal-md>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}}@media(min-width: 992px){.list-group-horizontal-lg{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal-lg>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal-lg>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal-lg>.list-group-item.active{margin-top:0}.list-group-horizontal-lg>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal-lg>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}}@media(min-width: 1200px){.list-group-horizontal-xl{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal-xl>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal-xl>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal-xl>.list-group-item.active{margin-top:0}.list-group-horizontal-xl>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal-xl>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}}@media(min-width: 1400px){.list-group-horizontal-xxl{flex-direction:row;-webkit-flex-direction:row}.list-group-horizontal-xxl>.list-group-item:first-child:not(:last-child){border-bottom-left-radius:var(--bs-list-group-border-radius);border-top-right-radius:0}.list-group-horizontal-xxl>.list-group-item:last-child:not(:first-child){border-top-right-radius:var(--bs-list-group-border-radius);border-bottom-left-radius:0}.list-group-horizontal-xxl>.list-group-item.active{margin-top:0}.list-group-horizontal-xxl>.list-group-item+.list-group-item{border-top-width:var(--bs-list-group-border-width);border-left-width:0}.list-group-horizontal-xxl>.list-group-item+.list-group-item.active{margin-left:calc(-1*var(--bs-list-group-border-width));border-left-width:var(--bs-list-group-border-width)}}.list-group-flush{border-radius:0}.list-group-flush>.list-group-item{border-width:0 0 var(--bs-list-group-border-width)}.list-group-flush>.list-group-item:last-child{border-bottom-width:0}.list-group-item-default{--bs-list-group-color: var(--bs-default-text-emphasis);--bs-list-group-bg: var(--bs-default-bg-subtle);--bs-list-group-border-color: var(--bs-default-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-default-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-default-border-subtle);--bs-list-group-active-color: var(--bs-default-bg-subtle);--bs-list-group-active-bg: var(--bs-default-text-emphasis);--bs-list-group-active-border-color: var(--bs-default-text-emphasis)}.list-group-item-primary{--bs-list-group-color: var(--bs-primary-text-emphasis);--bs-list-group-bg: var(--bs-primary-bg-subtle);--bs-list-group-border-color: var(--bs-primary-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-primary-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-primary-border-subtle);--bs-list-group-active-color: var(--bs-primary-bg-subtle);--bs-list-group-active-bg: var(--bs-primary-text-emphasis);--bs-list-group-active-border-color: var(--bs-primary-text-emphasis)}.list-group-item-secondary{--bs-list-group-color: var(--bs-secondary-text-emphasis);--bs-list-group-bg: var(--bs-secondary-bg-subtle);--bs-list-group-border-color: var(--bs-secondary-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-secondary-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-secondary-border-subtle);--bs-list-group-active-color: var(--bs-secondary-bg-subtle);--bs-list-group-active-bg: var(--bs-secondary-text-emphasis);--bs-list-group-active-border-color: var(--bs-secondary-text-emphasis)}.list-group-item-success{--bs-list-group-color: var(--bs-success-text-emphasis);--bs-list-group-bg: var(--bs-success-bg-subtle);--bs-list-group-border-color: var(--bs-success-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-success-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-success-border-subtle);--bs-list-group-active-color: var(--bs-success-bg-subtle);--bs-list-group-active-bg: var(--bs-success-text-emphasis);--bs-list-group-active-border-color: var(--bs-success-text-emphasis)}.list-group-item-info{--bs-list-group-color: var(--bs-info-text-emphasis);--bs-list-group-bg: var(--bs-info-bg-subtle);--bs-list-group-border-color: var(--bs-info-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-info-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-info-border-subtle);--bs-list-group-active-color: var(--bs-info-bg-subtle);--bs-list-group-active-bg: var(--bs-info-text-emphasis);--bs-list-group-active-border-color: var(--bs-info-text-emphasis)}.list-group-item-warning{--bs-list-group-color: var(--bs-warning-text-emphasis);--bs-list-group-bg: var(--bs-warning-bg-subtle);--bs-list-group-border-color: var(--bs-warning-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-warning-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-warning-border-subtle);--bs-list-group-active-color: var(--bs-warning-bg-subtle);--bs-list-group-active-bg: var(--bs-warning-text-emphasis);--bs-list-group-active-border-color: var(--bs-warning-text-emphasis)}.list-group-item-danger{--bs-list-group-color: var(--bs-danger-text-emphasis);--bs-list-group-bg: var(--bs-danger-bg-subtle);--bs-list-group-border-color: var(--bs-danger-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-danger-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-danger-border-subtle);--bs-list-group-active-color: var(--bs-danger-bg-subtle);--bs-list-group-active-bg: var(--bs-danger-text-emphasis);--bs-list-group-active-border-color: var(--bs-danger-text-emphasis)}.list-group-item-light{--bs-list-group-color: var(--bs-light-text-emphasis);--bs-list-group-bg: var(--bs-light-bg-subtle);--bs-list-group-border-color: var(--bs-light-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-light-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-light-border-subtle);--bs-list-group-active-color: var(--bs-light-bg-subtle);--bs-list-group-active-bg: var(--bs-light-text-emphasis);--bs-list-group-active-border-color: var(--bs-light-text-emphasis)}.list-group-item-dark{--bs-list-group-color: var(--bs-dark-text-emphasis);--bs-list-group-bg: var(--bs-dark-bg-subtle);--bs-list-group-border-color: var(--bs-dark-border-subtle);--bs-list-group-action-hover-color: var(--bs-emphasis-color);--bs-list-group-action-hover-bg: var(--bs-dark-border-subtle);--bs-list-group-action-active-color: var(--bs-emphasis-color);--bs-list-group-action-active-bg: var(--bs-dark-border-subtle);--bs-list-group-active-color: var(--bs-dark-bg-subtle);--bs-list-group-active-bg: var(--bs-dark-text-emphasis);--bs-list-group-active-border-color: var(--bs-dark-text-emphasis)}.btn-close{--bs-btn-close-color: #fff;--bs-btn-close-bg: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23fff'%3e%3cpath d='M.293.293a1 1 0 0 1 1.414 0L8 6.586 14.293.293a1 1 0 1 1 1.414 1.414L9.414 8l6.293 6.293a1 1 0 0 1-1.414 1.414L8 9.414l-6.293 6.293a1 1 0 0 1-1.414-1.414L6.586 8 .293 1.707a1 1 0 0 1 0-1.414z'/%3e%3c/svg%3e");--bs-btn-close-opacity: 0.8;--bs-btn-close-hover-opacity: 1;--bs-btn-close-focus-shadow: 0 0 0 0.25rem rgba(50, 93, 136, 0.25);--bs-btn-close-focus-opacity: 1;--bs-btn-close-disabled-opacity: 0.25;--bs-btn-close-white-filter: invert(1) grayscale(100%) brightness(200%);box-sizing:content-box;width:1em;height:1em;padding:.25em .25em;color:var(--bs-btn-close-color);background:rgba(0,0,0,0) var(--bs-btn-close-bg) center/1em auto no-repeat;border:0;border-radius:.25rem;opacity:var(--bs-btn-close-opacity)}.btn-close:hover{color:var(--bs-btn-close-color);text-decoration:none;opacity:var(--bs-btn-close-hover-opacity)}.btn-close:focus{outline:0;box-shadow:var(--bs-btn-close-focus-shadow);opacity:var(--bs-btn-close-focus-opacity)}.btn-close:disabled,.btn-close.disabled{pointer-events:none;user-select:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;-o-user-select:none;opacity:var(--bs-btn-close-disabled-opacity)}.btn-close-white{filter:var(--bs-btn-close-white-filter)}[data-bs-theme=dark] .btn-close{filter:var(--bs-btn-close-white-filter)}.toast{--bs-toast-zindex: 1090;--bs-toast-padding-x: 0.75rem;--bs-toast-padding-y: 0.5rem;--bs-toast-spacing: 1.5rem;--bs-toast-max-width: 350px;--bs-toast-font-size:0.875rem;--bs-toast-color: ;--bs-toast-bg: rgba(255, 255, 255, 0.85);--bs-toast-border-width: 1px;--bs-toast-border-color: rgba(0, 0, 0, 0.175);--bs-toast-border-radius: 0.25rem;--bs-toast-box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);--bs-toast-header-color: rgba(52, 58, 64, 0.75);--bs-toast-header-bg: rgba(255, 255, 255, 0.85);--bs-toast-header-border-color: rgba(0, 0, 0, 0.175);width:var(--bs-toast-max-width);max-width:100%;font-size:var(--bs-toast-font-size);color:var(--bs-toast-color);pointer-events:auto;background-color:var(--bs-toast-bg);background-clip:padding-box;border:var(--bs-toast-border-width) solid var(--bs-toast-border-color);box-shadow:var(--bs-toast-box-shadow);border-radius:var(--bs-toast-border-radius)}.toast.showing{opacity:0}.toast:not(.show){display:none}.toast-container{--bs-toast-zindex: 1090;position:absolute;z-index:var(--bs-toast-zindex);width:max-content;width:-webkit-max-content;width:-moz-max-content;width:-ms-max-content;width:-o-max-content;max-width:100%;pointer-events:none}.toast-container>:not(:last-child){margin-bottom:var(--bs-toast-spacing)}.toast-header{display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;padding:var(--bs-toast-padding-y) var(--bs-toast-padding-x);color:var(--bs-toast-header-color);background-color:var(--bs-toast-header-bg);background-clip:padding-box;border-bottom:var(--bs-toast-border-width) solid var(--bs-toast-header-border-color);border-top-left-radius:calc(var(--bs-toast-border-radius) - var(--bs-toast-border-width));border-top-right-radius:calc(var(--bs-toast-border-radius) - var(--bs-toast-border-width))}.toast-header .btn-close{margin-right:calc(-0.5*var(--bs-toast-padding-x));margin-left:var(--bs-toast-padding-x)}.toast-body{padding:var(--bs-toast-padding-x);word-wrap:break-word}.modal{--bs-modal-zindex: 1055;--bs-modal-width: 500px;--bs-modal-padding: 1rem;--bs-modal-margin: 0.5rem;--bs-modal-color: ;--bs-modal-bg: #fff;--bs-modal-border-color: #dee2e6;--bs-modal-border-width: 1px;--bs-modal-border-radius: 0.5rem;--bs-modal-box-shadow: 0 0.125rem 0.25rem rgba(0, 0, 0, 0.075);--bs-modal-inner-border-radius: calc(0.5rem - 1px);--bs-modal-header-padding-x: 1rem;--bs-modal-header-padding-y: 1rem;--bs-modal-header-padding: 1rem 1rem;--bs-modal-header-border-color: #dee2e6;--bs-modal-header-border-width: 1px;--bs-modal-title-line-height: 1.5;--bs-modal-footer-gap: 0.5rem;--bs-modal-footer-bg: ;--bs-modal-footer-border-color: #dee2e6;--bs-modal-footer-border-width: 1px;position:fixed;top:0;left:0;z-index:var(--bs-modal-zindex);display:none;width:100%;height:100%;overflow-x:hidden;overflow-y:auto;outline:0}.modal-dialog{position:relative;width:auto;margin:var(--bs-modal-margin);pointer-events:none}.modal.fade .modal-dialog{transition:transform .3s ease-out;transform:translate(0, -50px)}@media(prefers-reduced-motion: reduce){.modal.fade .modal-dialog{transition:none}}.modal.show .modal-dialog{transform:none}.modal.modal-static .modal-dialog{transform:scale(1.02)}.modal-dialog-scrollable{height:calc(100% - var(--bs-modal-margin)*2)}.modal-dialog-scrollable .modal-content{max-height:100%;overflow:hidden}.modal-dialog-scrollable .modal-body{overflow-y:auto}.modal-dialog-centered{display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;min-height:calc(100% - var(--bs-modal-margin)*2)}.modal-content{position:relative;display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;width:100%;color:var(--bs-modal-color);pointer-events:auto;background-color:var(--bs-modal-bg);background-clip:padding-box;border:var(--bs-modal-border-width) solid var(--bs-modal-border-color);border-radius:var(--bs-modal-border-radius);outline:0}.modal-backdrop{--bs-backdrop-zindex: 1050;--bs-backdrop-bg: #000;--bs-backdrop-opacity: 0.5;position:fixed;top:0;left:0;z-index:var(--bs-backdrop-zindex);width:100vw;height:100vh;background-color:var(--bs-backdrop-bg)}.modal-backdrop.fade{opacity:0}.modal-backdrop.show{opacity:var(--bs-backdrop-opacity)}.modal-header{display:flex;display:-webkit-flex;flex-shrink:0;-webkit-flex-shrink:0;align-items:center;-webkit-align-items:center;justify-content:space-between;-webkit-justify-content:space-between;padding:var(--bs-modal-header-padding);border-bottom:var(--bs-modal-header-border-width) solid var(--bs-modal-header-border-color);border-top-left-radius:var(--bs-modal-inner-border-radius);border-top-right-radius:var(--bs-modal-inner-border-radius)}.modal-header .btn-close{padding:calc(var(--bs-modal-header-padding-y)*.5) calc(var(--bs-modal-header-padding-x)*.5);margin:calc(-0.5*var(--bs-modal-header-padding-y)) calc(-0.5*var(--bs-modal-header-padding-x)) calc(-0.5*var(--bs-modal-header-padding-y)) auto}.modal-title{margin-bottom:0;line-height:var(--bs-modal-title-line-height)}.modal-body{position:relative;flex:1 1 auto;-webkit-flex:1 1 auto;padding:var(--bs-modal-padding)}.modal-footer{display:flex;display:-webkit-flex;flex-shrink:0;-webkit-flex-shrink:0;flex-wrap:wrap;-webkit-flex-wrap:wrap;align-items:center;-webkit-align-items:center;justify-content:flex-end;-webkit-justify-content:flex-end;padding:calc(var(--bs-modal-padding) - var(--bs-modal-footer-gap)*.5);background-color:var(--bs-modal-footer-bg);border-top:var(--bs-modal-footer-border-width) solid var(--bs-modal-footer-border-color);border-bottom-right-radius:var(--bs-modal-inner-border-radius);border-bottom-left-radius:var(--bs-modal-inner-border-radius)}.modal-footer>*{margin:calc(var(--bs-modal-footer-gap)*.5)}@media(min-width: 576px){.modal{--bs-modal-margin: 1.75rem;--bs-modal-box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15)}.modal-dialog{max-width:var(--bs-modal-width);margin-right:auto;margin-left:auto}.modal-sm{--bs-modal-width: 300px}}@media(min-width: 992px){.modal-lg,.modal-xl{--bs-modal-width: 800px}}@media(min-width: 1200px){.modal-xl{--bs-modal-width: 1140px}}.modal-fullscreen{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen .modal-header,.modal-fullscreen .modal-footer{border-radius:0}.modal-fullscreen .modal-body{overflow-y:auto}@media(max-width: 575.98px){.modal-fullscreen-sm-down{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen-sm-down .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen-sm-down .modal-header,.modal-fullscreen-sm-down .modal-footer{border-radius:0}.modal-fullscreen-sm-down .modal-body{overflow-y:auto}}@media(max-width: 767.98px){.modal-fullscreen-md-down{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen-md-down .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen-md-down .modal-header,.modal-fullscreen-md-down .modal-footer{border-radius:0}.modal-fullscreen-md-down .modal-body{overflow-y:auto}}@media(max-width: 991.98px){.modal-fullscreen-lg-down{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen-lg-down .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen-lg-down .modal-header,.modal-fullscreen-lg-down .modal-footer{border-radius:0}.modal-fullscreen-lg-down .modal-body{overflow-y:auto}}@media(max-width: 1199.98px){.modal-fullscreen-xl-down{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen-xl-down .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen-xl-down .modal-header,.modal-fullscreen-xl-down .modal-footer{border-radius:0}.modal-fullscreen-xl-down .modal-body{overflow-y:auto}}@media(max-width: 1399.98px){.modal-fullscreen-xxl-down{width:100vw;max-width:none;height:100%;margin:0}.modal-fullscreen-xxl-down .modal-content{height:100%;border:0;border-radius:0}.modal-fullscreen-xxl-down .modal-header,.modal-fullscreen-xxl-down .modal-footer{border-radius:0}.modal-fullscreen-xxl-down .modal-body{overflow-y:auto}}.tooltip{--bs-tooltip-zindex: 1080;--bs-tooltip-max-width: 200px;--bs-tooltip-padding-x: 0.5rem;--bs-tooltip-padding-y: 0.25rem;--bs-tooltip-margin: ;--bs-tooltip-font-size:0.875rem;--bs-tooltip-color: #fff;--bs-tooltip-bg: #000;--bs-tooltip-border-radius: 0.25rem;--bs-tooltip-opacity: 0.9;--bs-tooltip-arrow-width: 0.8rem;--bs-tooltip-arrow-height: 0.4rem;z-index:var(--bs-tooltip-zindex);display:block;margin:var(--bs-tooltip-margin);font-family:Roboto,-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";font-style:normal;font-weight:400;line-height:1.5;text-align:left;text-align:start;text-decoration:none;text-shadow:none;text-transform:none;letter-spacing:normal;word-break:normal;white-space:normal;word-spacing:normal;line-break:auto;font-size:var(--bs-tooltip-font-size);word-wrap:break-word;opacity:0}.tooltip.show{opacity:var(--bs-tooltip-opacity)}.tooltip .tooltip-arrow{display:block;width:var(--bs-tooltip-arrow-width);height:var(--bs-tooltip-arrow-height)}.tooltip .tooltip-arrow::before{position:absolute;content:"";border-color:rgba(0,0,0,0);border-style:solid}.bs-tooltip-top .tooltip-arrow,.bs-tooltip-auto[data-popper-placement^=top] .tooltip-arrow{bottom:calc(-1*var(--bs-tooltip-arrow-height))}.bs-tooltip-top .tooltip-arrow::before,.bs-tooltip-auto[data-popper-placement^=top] .tooltip-arrow::before{top:-1px;border-width:var(--bs-tooltip-arrow-height) calc(var(--bs-tooltip-arrow-width)*.5) 0;border-top-color:var(--bs-tooltip-bg)}.bs-tooltip-end .tooltip-arrow,.bs-tooltip-auto[data-popper-placement^=right] .tooltip-arrow{left:calc(-1*var(--bs-tooltip-arrow-height));width:var(--bs-tooltip-arrow-height);height:var(--bs-tooltip-arrow-width)}.bs-tooltip-end .tooltip-arrow::before,.bs-tooltip-auto[data-popper-placement^=right] .tooltip-arrow::before{right:-1px;border-width:calc(var(--bs-tooltip-arrow-width)*.5) var(--bs-tooltip-arrow-height) calc(var(--bs-tooltip-arrow-width)*.5) 0;border-right-color:var(--bs-tooltip-bg)}.bs-tooltip-bottom .tooltip-arrow,.bs-tooltip-auto[data-popper-placement^=bottom] .tooltip-arrow{top:calc(-1*var(--bs-tooltip-arrow-height))}.bs-tooltip-bottom .tooltip-arrow::before,.bs-tooltip-auto[data-popper-placement^=bottom] .tooltip-arrow::before{bottom:-1px;border-width:0 calc(var(--bs-tooltip-arrow-width)*.5) var(--bs-tooltip-arrow-height);border-bottom-color:var(--bs-tooltip-bg)}.bs-tooltip-start .tooltip-arrow,.bs-tooltip-auto[data-popper-placement^=left] .tooltip-arrow{right:calc(-1*var(--bs-tooltip-arrow-height));width:var(--bs-tooltip-arrow-height);height:var(--bs-tooltip-arrow-width)}.bs-tooltip-start .tooltip-arrow::before,.bs-tooltip-auto[data-popper-placement^=left] .tooltip-arrow::before{left:-1px;border-width:calc(var(--bs-tooltip-arrow-width)*.5) 0 calc(var(--bs-tooltip-arrow-width)*.5) var(--bs-tooltip-arrow-height);border-left-color:var(--bs-tooltip-bg)}.tooltip-inner{max-width:var(--bs-tooltip-max-width);padding:var(--bs-tooltip-padding-y) var(--bs-tooltip-padding-x);color:var(--bs-tooltip-color);text-align:center;background-color:var(--bs-tooltip-bg);border-radius:var(--bs-tooltip-border-radius)}.popover{--bs-popover-zindex: 1070;--bs-popover-max-width: 276px;--bs-popover-font-size:0.875rem;--bs-popover-bg: #fff;--bs-popover-border-width: 1px;--bs-popover-border-color: rgba(0, 0, 0, 0.175);--bs-popover-border-radius: 0.5rem;--bs-popover-inner-border-radius: calc(0.5rem - 1px);--bs-popover-box-shadow: 0 0.5rem 1rem rgba(0, 0, 0, 0.15);--bs-popover-header-padding-x: 1rem;--bs-popover-header-padding-y: 0.5rem;--bs-popover-header-font-size:1rem;--bs-popover-header-color: inherit;--bs-popover-header-bg: #f8f5f0;--bs-popover-body-padding-x: 1rem;--bs-popover-body-padding-y: 1rem;--bs-popover-body-color: #343a40;--bs-popover-arrow-width: 1rem;--bs-popover-arrow-height: 0.5rem;--bs-popover-arrow-border: var(--bs-popover-border-color);z-index:var(--bs-popover-zindex);display:block;max-width:var(--bs-popover-max-width);font-family:Roboto,-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";font-style:normal;font-weight:400;line-height:1.5;text-align:left;text-align:start;text-decoration:none;text-shadow:none;text-transform:none;letter-spacing:normal;word-break:normal;white-space:normal;word-spacing:normal;line-break:auto;font-size:var(--bs-popover-font-size);word-wrap:break-word;background-color:var(--bs-popover-bg);background-clip:padding-box;border:var(--bs-popover-border-width) solid var(--bs-popover-border-color);border-radius:var(--bs-popover-border-radius)}.popover .popover-arrow{display:block;width:var(--bs-popover-arrow-width);height:var(--bs-popover-arrow-height)}.popover .popover-arrow::before,.popover .popover-arrow::after{position:absolute;display:block;content:"";border-color:rgba(0,0,0,0);border-style:solid;border-width:0}.bs-popover-top>.popover-arrow,.bs-popover-auto[data-popper-placement^=top]>.popover-arrow{bottom:calc(-1*(var(--bs-popover-arrow-height)) - var(--bs-popover-border-width))}.bs-popover-top>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=top]>.popover-arrow::before,.bs-popover-top>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=top]>.popover-arrow::after{border-width:var(--bs-popover-arrow-height) calc(var(--bs-popover-arrow-width)*.5) 0}.bs-popover-top>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=top]>.popover-arrow::before{bottom:0;border-top-color:var(--bs-popover-arrow-border)}.bs-popover-top>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=top]>.popover-arrow::after{bottom:var(--bs-popover-border-width);border-top-color:var(--bs-popover-bg)}.bs-popover-end>.popover-arrow,.bs-popover-auto[data-popper-placement^=right]>.popover-arrow{left:calc(-1*(var(--bs-popover-arrow-height)) - var(--bs-popover-border-width));width:var(--bs-popover-arrow-height);height:var(--bs-popover-arrow-width)}.bs-popover-end>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=right]>.popover-arrow::before,.bs-popover-end>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=right]>.popover-arrow::after{border-width:calc(var(--bs-popover-arrow-width)*.5) var(--bs-popover-arrow-height) calc(var(--bs-popover-arrow-width)*.5) 0}.bs-popover-end>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=right]>.popover-arrow::before{left:0;border-right-color:var(--bs-popover-arrow-border)}.bs-popover-end>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=right]>.popover-arrow::after{left:var(--bs-popover-border-width);border-right-color:var(--bs-popover-bg)}.bs-popover-bottom>.popover-arrow,.bs-popover-auto[data-popper-placement^=bottom]>.popover-arrow{top:calc(-1*(var(--bs-popover-arrow-height)) - var(--bs-popover-border-width))}.bs-popover-bottom>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=bottom]>.popover-arrow::before,.bs-popover-bottom>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=bottom]>.popover-arrow::after{border-width:0 calc(var(--bs-popover-arrow-width)*.5) var(--bs-popover-arrow-height)}.bs-popover-bottom>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=bottom]>.popover-arrow::before{top:0;border-bottom-color:var(--bs-popover-arrow-border)}.bs-popover-bottom>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=bottom]>.popover-arrow::after{top:var(--bs-popover-border-width);border-bottom-color:var(--bs-popover-bg)}.bs-popover-bottom .popover-header::before,.bs-popover-auto[data-popper-placement^=bottom] .popover-header::before{position:absolute;top:0;left:50%;display:block;width:var(--bs-popover-arrow-width);margin-left:calc(-0.5*var(--bs-popover-arrow-width));content:"";border-bottom:var(--bs-popover-border-width) solid var(--bs-popover-header-bg)}.bs-popover-start>.popover-arrow,.bs-popover-auto[data-popper-placement^=left]>.popover-arrow{right:calc(-1*(var(--bs-popover-arrow-height)) - var(--bs-popover-border-width));width:var(--bs-popover-arrow-height);height:var(--bs-popover-arrow-width)}.bs-popover-start>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=left]>.popover-arrow::before,.bs-popover-start>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=left]>.popover-arrow::after{border-width:calc(var(--bs-popover-arrow-width)*.5) 0 calc(var(--bs-popover-arrow-width)*.5) var(--bs-popover-arrow-height)}.bs-popover-start>.popover-arrow::before,.bs-popover-auto[data-popper-placement^=left]>.popover-arrow::before{right:0;border-left-color:var(--bs-popover-arrow-border)}.bs-popover-start>.popover-arrow::after,.bs-popover-auto[data-popper-placement^=left]>.popover-arrow::after{right:var(--bs-popover-border-width);border-left-color:var(--bs-popover-bg)}.popover-header{padding:var(--bs-popover-header-padding-y) var(--bs-popover-header-padding-x);margin-bottom:0;font-size:var(--bs-popover-header-font-size);color:var(--bs-popover-header-color);background-color:var(--bs-popover-header-bg);border-bottom:var(--bs-popover-border-width) solid var(--bs-popover-border-color);border-top-left-radius:var(--bs-popover-inner-border-radius);border-top-right-radius:var(--bs-popover-inner-border-radius)}.popover-header:empty{display:none}.popover-body{padding:var(--bs-popover-body-padding-y) var(--bs-popover-body-padding-x);color:var(--bs-popover-body-color)}.carousel{position:relative}.carousel.pointer-event{touch-action:pan-y;-webkit-touch-action:pan-y;-moz-touch-action:pan-y;-ms-touch-action:pan-y;-o-touch-action:pan-y}.carousel-inner{position:relative;width:100%;overflow:hidden}.carousel-inner::after{display:block;clear:both;content:""}.carousel-item{position:relative;display:none;float:left;width:100%;margin-right:-100%;backface-visibility:hidden;-webkit-backface-visibility:hidden;-moz-backface-visibility:hidden;-ms-backface-visibility:hidden;-o-backface-visibility:hidden;transition:transform .6s ease-in-out}@media(prefers-reduced-motion: reduce){.carousel-item{transition:none}}.carousel-item.active,.carousel-item-next,.carousel-item-prev{display:block}.carousel-item-next:not(.carousel-item-start),.active.carousel-item-end{transform:translateX(100%)}.carousel-item-prev:not(.carousel-item-end),.active.carousel-item-start{transform:translateX(-100%)}.carousel-fade .carousel-item{opacity:0;transition-property:opacity;transform:none}.carousel-fade .carousel-item.active,.carousel-fade .carousel-item-next.carousel-item-start,.carousel-fade .carousel-item-prev.carousel-item-end{z-index:1;opacity:1}.carousel-fade .active.carousel-item-start,.carousel-fade .active.carousel-item-end{z-index:0;opacity:0;transition:opacity 0s .6s}@media(prefers-reduced-motion: reduce){.carousel-fade .active.carousel-item-start,.carousel-fade .active.carousel-item-end{transition:none}}.carousel-control-prev,.carousel-control-next{position:absolute;top:0;bottom:0;z-index:1;display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;justify-content:center;-webkit-justify-content:center;width:15%;padding:0;color:#fff;text-align:center;background:none;border:0;opacity:.5;transition:opacity .15s ease}@media(prefers-reduced-motion: reduce){.carousel-control-prev,.carousel-control-next{transition:none}}.carousel-control-prev:hover,.carousel-control-prev:focus,.carousel-control-next:hover,.carousel-control-next:focus{color:#fff;text-decoration:none;outline:0;opacity:.9}.carousel-control-prev{left:0}.carousel-control-next{right:0}.carousel-control-prev-icon,.carousel-control-next-icon{display:inline-block;width:2rem;height:2rem;background-repeat:no-repeat;background-position:50%;background-size:100% 100%}.carousel-control-prev-icon{background-image:url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23fff'%3e%3cpath d='M11.354 1.646a.5.5 0 0 1 0 .708L5.707 8l5.647 5.646a.5.5 0 0 1-.708.708l-6-6a.5.5 0 0 1 0-.708l6-6a.5.5 0 0 1 .708 0z'/%3e%3c/svg%3e")}.carousel-control-next-icon{background-image:url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23fff'%3e%3cpath d='M4.646 1.646a.5.5 0 0 1 .708 0l6 6a.5.5 0 0 1 0 .708l-6 6a.5.5 0 0 1-.708-.708L10.293 8 4.646 2.354a.5.5 0 0 1 0-.708z'/%3e%3c/svg%3e")}.carousel-indicators{position:absolute;right:0;bottom:0;left:0;z-index:2;display:flex;display:-webkit-flex;justify-content:center;-webkit-justify-content:center;padding:0;margin-right:15%;margin-bottom:1rem;margin-left:15%}.carousel-indicators [data-bs-target]{box-sizing:content-box;flex:0 1 auto;-webkit-flex:0 1 auto;width:30px;height:3px;padding:0;margin-right:3px;margin-left:3px;text-indent:-999px;cursor:pointer;background-color:#fff;background-clip:padding-box;border:0;border-top:10px solid rgba(0,0,0,0);border-bottom:10px solid rgba(0,0,0,0);opacity:.5;transition:opacity .6s ease}@media(prefers-reduced-motion: reduce){.carousel-indicators [data-bs-target]{transition:none}}.carousel-indicators .active{opacity:1}.carousel-caption{position:absolute;right:15%;bottom:1.25rem;left:15%;padding-top:1.25rem;padding-bottom:1.25rem;color:#fff;text-align:center}.carousel-dark .carousel-control-prev-icon,.carousel-dark .carousel-control-next-icon{filter:invert(1) grayscale(100)}.carousel-dark .carousel-indicators [data-bs-target]{background-color:#000}.carousel-dark .carousel-caption{color:#000}[data-bs-theme=dark] .carousel .carousel-control-prev-icon,[data-bs-theme=dark] .carousel .carousel-control-next-icon,[data-bs-theme=dark].carousel .carousel-control-prev-icon,[data-bs-theme=dark].carousel .carousel-control-next-icon{filter:invert(1) grayscale(100)}[data-bs-theme=dark] .carousel .carousel-indicators [data-bs-target],[data-bs-theme=dark].carousel .carousel-indicators [data-bs-target]{background-color:#000}[data-bs-theme=dark] .carousel .carousel-caption,[data-bs-theme=dark].carousel .carousel-caption{color:#000}.spinner-grow,.spinner-border{display:inline-block;width:var(--bs-spinner-width);height:var(--bs-spinner-height);vertical-align:var(--bs-spinner-vertical-align);border-radius:50%;animation:var(--bs-spinner-animation-speed) linear infinite var(--bs-spinner-animation-name)}@keyframes spinner-border{to{transform:rotate(360deg) /* rtl:ignore */}}.spinner-border{--bs-spinner-width: 2rem;--bs-spinner-height: 2rem;--bs-spinner-vertical-align: -0.125em;--bs-spinner-border-width: 0.25em;--bs-spinner-animation-speed: 0.75s;--bs-spinner-animation-name: spinner-border;border:var(--bs-spinner-border-width) solid currentcolor;border-right-color:rgba(0,0,0,0)}.spinner-border-sm{--bs-spinner-width: 1rem;--bs-spinner-height: 1rem;--bs-spinner-border-width: 0.2em}@keyframes spinner-grow{0%{transform:scale(0)}50%{opacity:1;transform:none}}.spinner-grow{--bs-spinner-width: 2rem;--bs-spinner-height: 2rem;--bs-spinner-vertical-align: -0.125em;--bs-spinner-animation-speed: 0.75s;--bs-spinner-animation-name: spinner-grow;background-color:currentcolor;opacity:0}.spinner-grow-sm{--bs-spinner-width: 1rem;--bs-spinner-height: 1rem}@media(prefers-reduced-motion: reduce){.spinner-border,.spinner-grow{--bs-spinner-animation-speed: 1.5s}}.offcanvas,.offcanvas-xxl,.offcanvas-xl,.offcanvas-lg,.offcanvas-md,.offcanvas-sm{--bs-offcanvas-zindex: 1045;--bs-offcanvas-width: 400px;--bs-offcanvas-height: 30vh;--bs-offcanvas-padding-x: 1rem;--bs-offcanvas-padding-y: 1rem;--bs-offcanvas-color: #343a40;--bs-offcanvas-bg: #fff;--bs-offcanvas-border-width: 1px;--bs-offcanvas-border-color: #dee2e6;--bs-offcanvas-box-shadow: 0 0.125rem 0.25rem rgba(0, 0, 0, 0.075);--bs-offcanvas-transition: transform 0.3s ease-in-out;--bs-offcanvas-title-line-height: 1.5}@media(max-width: 575.98px){.offcanvas-sm{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}}@media(max-width: 575.98px)and (prefers-reduced-motion: reduce){.offcanvas-sm{transition:none}}@media(max-width: 575.98px){.offcanvas-sm.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas-sm.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas-sm.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas-sm.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas-sm.showing,.offcanvas-sm.show:not(.hiding){transform:none}.offcanvas-sm.showing,.offcanvas-sm.hiding,.offcanvas-sm.show{visibility:visible}}@media(min-width: 576px){.offcanvas-sm{--bs-offcanvas-height: auto;--bs-offcanvas-border-width: 0;background-color:rgba(0,0,0,0) !important}.offcanvas-sm .offcanvas-header{display:none}.offcanvas-sm .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible;background-color:rgba(0,0,0,0) !important}}@media(max-width: 767.98px){.offcanvas-md{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}}@media(max-width: 767.98px)and (prefers-reduced-motion: reduce){.offcanvas-md{transition:none}}@media(max-width: 767.98px){.offcanvas-md.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas-md.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas-md.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas-md.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas-md.showing,.offcanvas-md.show:not(.hiding){transform:none}.offcanvas-md.showing,.offcanvas-md.hiding,.offcanvas-md.show{visibility:visible}}@media(min-width: 768px){.offcanvas-md{--bs-offcanvas-height: auto;--bs-offcanvas-border-width: 0;background-color:rgba(0,0,0,0) !important}.offcanvas-md .offcanvas-header{display:none}.offcanvas-md .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible;background-color:rgba(0,0,0,0) !important}}@media(max-width: 991.98px){.offcanvas-lg{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}}@media(max-width: 991.98px)and (prefers-reduced-motion: reduce){.offcanvas-lg{transition:none}}@media(max-width: 991.98px){.offcanvas-lg.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas-lg.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas-lg.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas-lg.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas-lg.showing,.offcanvas-lg.show:not(.hiding){transform:none}.offcanvas-lg.showing,.offcanvas-lg.hiding,.offcanvas-lg.show{visibility:visible}}@media(min-width: 992px){.offcanvas-lg{--bs-offcanvas-height: auto;--bs-offcanvas-border-width: 0;background-color:rgba(0,0,0,0) !important}.offcanvas-lg .offcanvas-header{display:none}.offcanvas-lg .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible;background-color:rgba(0,0,0,0) !important}}@media(max-width: 1199.98px){.offcanvas-xl{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}}@media(max-width: 1199.98px)and (prefers-reduced-motion: reduce){.offcanvas-xl{transition:none}}@media(max-width: 1199.98px){.offcanvas-xl.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas-xl.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas-xl.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas-xl.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas-xl.showing,.offcanvas-xl.show:not(.hiding){transform:none}.offcanvas-xl.showing,.offcanvas-xl.hiding,.offcanvas-xl.show{visibility:visible}}@media(min-width: 1200px){.offcanvas-xl{--bs-offcanvas-height: auto;--bs-offcanvas-border-width: 0;background-color:rgba(0,0,0,0) !important}.offcanvas-xl .offcanvas-header{display:none}.offcanvas-xl .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible;background-color:rgba(0,0,0,0) !important}}@media(max-width: 1399.98px){.offcanvas-xxl{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}}@media(max-width: 1399.98px)and (prefers-reduced-motion: reduce){.offcanvas-xxl{transition:none}}@media(max-width: 1399.98px){.offcanvas-xxl.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas-xxl.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas-xxl.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas-xxl.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas-xxl.showing,.offcanvas-xxl.show:not(.hiding){transform:none}.offcanvas-xxl.showing,.offcanvas-xxl.hiding,.offcanvas-xxl.show{visibility:visible}}@media(min-width: 1400px){.offcanvas-xxl{--bs-offcanvas-height: auto;--bs-offcanvas-border-width: 0;background-color:rgba(0,0,0,0) !important}.offcanvas-xxl .offcanvas-header{display:none}.offcanvas-xxl .offcanvas-body{display:flex;display:-webkit-flex;flex-grow:0;-webkit-flex-grow:0;padding:0;overflow-y:visible;background-color:rgba(0,0,0,0) !important}}.offcanvas{position:fixed;bottom:0;z-index:var(--bs-offcanvas-zindex);display:flex;display:-webkit-flex;flex-direction:column;-webkit-flex-direction:column;max-width:100%;color:var(--bs-offcanvas-color);visibility:hidden;background-color:var(--bs-offcanvas-bg);background-clip:padding-box;outline:0;transition:var(--bs-offcanvas-transition)}@media(prefers-reduced-motion: reduce){.offcanvas{transition:none}}.offcanvas.offcanvas-start{top:0;left:0;width:var(--bs-offcanvas-width);border-right:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(-100%)}.offcanvas.offcanvas-end{top:0;right:0;width:var(--bs-offcanvas-width);border-left:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateX(100%)}.offcanvas.offcanvas-top{top:0;right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-bottom:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(-100%)}.offcanvas.offcanvas-bottom{right:0;left:0;height:var(--bs-offcanvas-height);max-height:100%;border-top:var(--bs-offcanvas-border-width) solid var(--bs-offcanvas-border-color);transform:translateY(100%)}.offcanvas.showing,.offcanvas.show:not(.hiding){transform:none}.offcanvas.showing,.offcanvas.hiding,.offcanvas.show{visibility:visible}.offcanvas-backdrop{position:fixed;top:0;left:0;z-index:1040;width:100vw;height:100vh;background-color:#000}.offcanvas-backdrop.fade{opacity:0}.offcanvas-backdrop.show{opacity:.5}.offcanvas-header{display:flex;display:-webkit-flex;align-items:center;-webkit-align-items:center;justify-content:space-between;-webkit-justify-content:space-between;padding:var(--bs-offcanvas-padding-y) var(--bs-offcanvas-padding-x)}.offcanvas-header .btn-close{padding:calc(var(--bs-offcanvas-padding-y)*.5) calc(var(--bs-offcanvas-padding-x)*.5);margin-top:calc(-0.5*var(--bs-offcanvas-padding-y));margin-right:calc(-0.5*var(--bs-offcanvas-padding-x));margin-bottom:calc(-0.5*var(--bs-offcanvas-padding-y))}.offcanvas-title{margin-bottom:0;line-height:var(--bs-offcanvas-title-line-height)}.offcanvas-body{flex-grow:1;-webkit-flex-grow:1;padding:var(--bs-offcanvas-padding-y) var(--bs-offcanvas-padding-x);overflow-y:auto}.placeholder{display:inline-block;min-height:1em;vertical-align:middle;cursor:wait;background-color:currentcolor;opacity:.5}.placeholder.btn::before{display:inline-block;content:""}.placeholder-xs{min-height:.6em}.placeholder-sm{min-height:.8em}.placeholder-lg{min-height:1.2em}.placeholder-glow .placeholder{animation:placeholder-glow 2s ease-in-out infinite}@keyframes placeholder-glow{50%{opacity:.2}}.placeholder-wave{mask-image:linear-gradient(130deg, #000 55%, rgba(0, 0, 0, 0.8) 75%, #000 95%);-webkit-mask-image:linear-gradient(130deg, #000 55%, rgba(0, 0, 0, 0.8) 75%, #000 95%);mask-size:200% 100%;-webkit-mask-size:200% 100%;animation:placeholder-wave 2s linear infinite}@keyframes placeholder-wave{100%{mask-position:-200% 0%;-webkit-mask-position:-200% 0%}}.clearfix::after{display:block;clear:both;content:""}.text-bg-default{color:#fff !important;background-color:RGBA(var(--bs-default-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-primary{color:#fff !important;background-color:RGBA(var(--bs-primary-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-secondary{color:#fff !important;background-color:RGBA(var(--bs-secondary-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-success{color:#fff !important;background-color:RGBA(var(--bs-success-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-info{color:#fff !important;background-color:RGBA(var(--bs-info-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-warning{color:#fff !important;background-color:RGBA(var(--bs-warning-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-danger{color:#fff !important;background-color:RGBA(var(--bs-danger-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-light{color:#000 !important;background-color:RGBA(var(--bs-light-rgb), var(--bs-bg-opacity, 1)) !important}.text-bg-dark{color:#fff !important;background-color:RGBA(var(--bs-dark-rgb), var(--bs-bg-opacity, 1)) !important}.link-default{color:RGBA(var(--bs-default-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-default-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-default:hover,.link-default:focus{color:RGBA(86, 94, 100, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(86, 94, 100, var(--bs-link-underline-opacity, 1)) !important}.link-primary{color:RGBA(var(--bs-primary-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-primary-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-primary:hover,.link-primary:focus{color:RGBA(40, 74, 109, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(40, 74, 109, var(--bs-link-underline-opacity, 1)) !important}.link-secondary{color:RGBA(var(--bs-secondary-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-secondary-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-secondary:hover,.link-secondary:focus{color:RGBA(86, 94, 100, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(86, 94, 100, var(--bs-link-underline-opacity, 1)) !important}.link-success{color:RGBA(var(--bs-success-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-success-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-success:hover,.link-success:focus{color:RGBA(118, 158, 60, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(118, 158, 60, var(--bs-link-underline-opacity, 1)) !important}.link-info{color:RGBA(var(--bs-info-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-info-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-info:hover,.link-info:focus{color:RGBA(33, 137, 179, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(33, 137, 179, var(--bs-link-underline-opacity, 1)) !important}.link-warning{color:RGBA(var(--bs-warning-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-warning-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-warning:hover,.link-warning:focus{color:RGBA(195, 99, 48, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(195, 99, 48, var(--bs-link-underline-opacity, 1)) !important}.link-danger{color:RGBA(var(--bs-danger-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-danger-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-danger:hover,.link-danger:focus{color:RGBA(174, 66, 63, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(174, 66, 63, var(--bs-link-underline-opacity, 1)) !important}.link-light{color:RGBA(var(--bs-light-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-light-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-light:hover,.link-light:focus{color:RGBA(249, 247, 243, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(249, 247, 243, var(--bs-link-underline-opacity, 1)) !important}.link-dark{color:RGBA(var(--bs-dark-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-dark-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-dark:hover,.link-dark:focus{color:RGBA(42, 46, 51, var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(42, 46, 51, var(--bs-link-underline-opacity, 1)) !important}.link-body-emphasis{color:RGBA(var(--bs-emphasis-color-rgb), var(--bs-link-opacity, 1)) !important;text-decoration-color:RGBA(var(--bs-emphasis-color-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-body-emphasis:hover,.link-body-emphasis:focus{color:RGBA(var(--bs-emphasis-color-rgb), var(--bs-link-opacity, 0.75)) !important;text-decoration-color:RGBA(var(--bs-emphasis-color-rgb), var(--bs-link-underline-opacity, 0.75)) !important}.focus-ring:focus{outline:0;box-shadow:var(--bs-focus-ring-x, 0) var(--bs-focus-ring-y, 0) var(--bs-focus-ring-blur, 0) var(--bs-focus-ring-width) var(--bs-focus-ring-color)}.icon-link{display:inline-flex;gap:.375rem;align-items:center;-webkit-align-items:center;text-decoration-color:rgba(var(--bs-link-color-rgb), var(--bs-link-opacity, 0.5));text-underline-offset:.25em;backface-visibility:hidden;-webkit-backface-visibility:hidden;-moz-backface-visibility:hidden;-ms-backface-visibility:hidden;-o-backface-visibility:hidden}.icon-link>.bi{flex-shrink:0;-webkit-flex-shrink:0;width:1em;height:1em;fill:currentcolor;transition:.2s ease-in-out transform}@media(prefers-reduced-motion: reduce){.icon-link>.bi{transition:none}}.icon-link-hover:hover>.bi,.icon-link-hover:focus-visible>.bi{transform:var(--bs-icon-link-transform, translate3d(0.25em, 0, 0))}.ratio{position:relative;width:100%}.ratio::before{display:block;padding-top:var(--bs-aspect-ratio);content:""}.ratio>*{position:absolute;top:0;left:0;width:100%;height:100%}.ratio-1x1{--bs-aspect-ratio: 100%}.ratio-4x3{--bs-aspect-ratio: 75%}.ratio-16x9{--bs-aspect-ratio: 56.25%}.ratio-21x9{--bs-aspect-ratio: 42.8571428571%}.fixed-top{position:fixed;top:0;right:0;left:0;z-index:1030}.fixed-bottom{position:fixed;right:0;bottom:0;left:0;z-index:1030}.sticky-top{position:sticky;top:0;z-index:1020}.sticky-bottom{position:sticky;bottom:0;z-index:1020}@media(min-width: 576px){.sticky-sm-top{position:sticky;top:0;z-index:1020}.sticky-sm-bottom{position:sticky;bottom:0;z-index:1020}}@media(min-width: 768px){.sticky-md-top{position:sticky;top:0;z-index:1020}.sticky-md-bottom{position:sticky;bottom:0;z-index:1020}}@media(min-width: 992px){.sticky-lg-top{position:sticky;top:0;z-index:1020}.sticky-lg-bottom{position:sticky;bottom:0;z-index:1020}}@media(min-width: 1200px){.sticky-xl-top{position:sticky;top:0;z-index:1020}.sticky-xl-bottom{position:sticky;bottom:0;z-index:1020}}@media(min-width: 1400px){.sticky-xxl-top{position:sticky;top:0;z-index:1020}.sticky-xxl-bottom{position:sticky;bottom:0;z-index:1020}}.hstack{display:flex;display:-webkit-flex;flex-direction:row;-webkit-flex-direction:row;align-items:center;-webkit-align-items:center;align-self:stretch;-webkit-align-self:stretch}.vstack{display:flex;display:-webkit-flex;flex:1 1 auto;-webkit-flex:1 1 auto;flex-direction:column;-webkit-flex-direction:column;align-self:stretch;-webkit-align-self:stretch}.visually-hidden,.visually-hidden-focusable:not(:focus):not(:focus-within){width:1px !important;height:1px !important;padding:0 !important;margin:-1px !important;overflow:hidden !important;clip:rect(0, 0, 0, 0) !important;white-space:nowrap !important;border:0 !important}.visually-hidden:not(caption),.visually-hidden-focusable:not(:focus):not(:focus-within):not(caption){position:absolute !important}.stretched-link::after{position:absolute;top:0;right:0;bottom:0;left:0;z-index:1;content:""}.text-truncate{overflow:hidden;text-overflow:ellipsis;white-space:nowrap}.vr{display:inline-block;align-self:stretch;-webkit-align-self:stretch;width:1px;min-height:1em;background-color:currentcolor;opacity:.25}.align-baseline{vertical-align:baseline !important}.align-top{vertical-align:top !important}.align-middle{vertical-align:middle !important}.align-bottom{vertical-align:bottom !important}.align-text-bottom{vertical-align:text-bottom !important}.align-text-top{vertical-align:text-top !important}.float-start{float:left !important}.float-end{float:right !important}.float-none{float:none !important}.object-fit-contain{object-fit:contain !important}.object-fit-cover{object-fit:cover !important}.object-fit-fill{object-fit:fill !important}.object-fit-scale{object-fit:scale-down !important}.object-fit-none{object-fit:none !important}.opacity-0{opacity:0 !important}.opacity-25{opacity:.25 !important}.opacity-50{opacity:.5 !important}.opacity-75{opacity:.75 !important}.opacity-100{opacity:1 !important}.overflow-auto{overflow:auto !important}.overflow-hidden{overflow:hidden !important}.overflow-visible{overflow:visible !important}.overflow-scroll{overflow:scroll !important}.overflow-x-auto{overflow-x:auto !important}.overflow-x-hidden{overflow-x:hidden !important}.overflow-x-visible{overflow-x:visible !important}.overflow-x-scroll{overflow-x:scroll !important}.overflow-y-auto{overflow-y:auto !important}.overflow-y-hidden{overflow-y:hidden !important}.overflow-y-visible{overflow-y:visible !important}.overflow-y-scroll{overflow-y:scroll !important}.d-inline{display:inline !important}.d-inline-block{display:inline-block !important}.d-block{display:block !important}.d-grid{display:grid !important}.d-inline-grid{display:inline-grid !important}.d-table{display:table !important}.d-table-row{display:table-row !important}.d-table-cell{display:table-cell !important}.d-flex{display:flex !important}.d-inline-flex{display:inline-flex !important}.d-none{display:none !important}.shadow{box-shadow:0 .5rem 1rem rgba(0,0,0,.15) !important}.shadow-sm{box-shadow:0 .125rem .25rem rgba(0,0,0,.075) !important}.shadow-lg{box-shadow:0 1rem 3rem rgba(0,0,0,.175) !important}.shadow-none{box-shadow:none !important}.focus-ring-default{--bs-focus-ring-color: rgba(var(--bs-default-rgb), var(--bs-focus-ring-opacity))}.focus-ring-primary{--bs-focus-ring-color: rgba(var(--bs-primary-rgb), var(--bs-focus-ring-opacity))}.focus-ring-secondary{--bs-focus-ring-color: rgba(var(--bs-secondary-rgb), var(--bs-focus-ring-opacity))}.focus-ring-success{--bs-focus-ring-color: rgba(var(--bs-success-rgb), var(--bs-focus-ring-opacity))}.focus-ring-info{--bs-focus-ring-color: rgba(var(--bs-info-rgb), var(--bs-focus-ring-opacity))}.focus-ring-warning{--bs-focus-ring-color: rgba(var(--bs-warning-rgb), var(--bs-focus-ring-opacity))}.focus-ring-danger{--bs-focus-ring-color: rgba(var(--bs-danger-rgb), var(--bs-focus-ring-opacity))}.focus-ring-light{--bs-focus-ring-color: rgba(var(--bs-light-rgb), var(--bs-focus-ring-opacity))}.focus-ring-dark{--bs-focus-ring-color: rgba(var(--bs-dark-rgb), var(--bs-focus-ring-opacity))}.position-static{position:static !important}.position-relative{position:relative !important}.position-absolute{position:absolute !important}.position-fixed{position:fixed !important}.position-sticky{position:sticky !important}.top-0{top:0 !important}.top-50{top:50% !important}.top-100{top:100% !important}.bottom-0{bottom:0 !important}.bottom-50{bottom:50% !important}.bottom-100{bottom:100% !important}.start-0{left:0 !important}.start-50{left:50% !important}.start-100{left:100% !important}.end-0{right:0 !important}.end-50{right:50% !important}.end-100{right:100% !important}.translate-middle{transform:translate(-50%, -50%) !important}.translate-middle-x{transform:translateX(-50%) !important}.translate-middle-y{transform:translateY(-50%) !important}.border{border:var(--bs-border-width) var(--bs-border-style) var(--bs-border-color) !important}.border-0{border:0 !important}.border-top{border-top:var(--bs-border-width) var(--bs-border-style) var(--bs-border-color) !important}.border-top-0{border-top:0 !important}.border-end{border-right:var(--bs-border-width) var(--bs-border-style) var(--bs-border-color) !important}.border-end-0{border-right:0 !important}.border-bottom{border-bottom:var(--bs-border-width) var(--bs-border-style) var(--bs-border-color) !important}.border-bottom-0{border-bottom:0 !important}.border-start{border-left:var(--bs-border-width) var(--bs-border-style) var(--bs-border-color) !important}.border-start-0{border-left:0 !important}.border-default{--bs-border-opacity: 1;border-color:rgba(var(--bs-default-rgb), var(--bs-border-opacity)) !important}.border-primary{--bs-border-opacity: 1;border-color:rgba(var(--bs-primary-rgb), var(--bs-border-opacity)) !important}.border-secondary{--bs-border-opacity: 1;border-color:rgba(var(--bs-secondary-rgb), var(--bs-border-opacity)) !important}.border-success{--bs-border-opacity: 1;border-color:rgba(var(--bs-success-rgb), var(--bs-border-opacity)) !important}.border-info{--bs-border-opacity: 1;border-color:rgba(var(--bs-info-rgb), var(--bs-border-opacity)) !important}.border-warning{--bs-border-opacity: 1;border-color:rgba(var(--bs-warning-rgb), var(--bs-border-opacity)) !important}.border-danger{--bs-border-opacity: 1;border-color:rgba(var(--bs-danger-rgb), var(--bs-border-opacity)) !important}.border-light{--bs-border-opacity: 1;border-color:rgba(var(--bs-light-rgb), var(--bs-border-opacity)) !important}.border-dark{--bs-border-opacity: 1;border-color:rgba(var(--bs-dark-rgb), var(--bs-border-opacity)) !important}.border-black{--bs-border-opacity: 1;border-color:rgba(var(--bs-black-rgb), var(--bs-border-opacity)) !important}.border-white{--bs-border-opacity: 1;border-color:rgba(var(--bs-white-rgb), var(--bs-border-opacity)) !important}.border-primary-subtle{border-color:var(--bs-primary-border-subtle) !important}.border-secondary-subtle{border-color:var(--bs-secondary-border-subtle) !important}.border-success-subtle{border-color:var(--bs-success-border-subtle) !important}.border-info-subtle{border-color:var(--bs-info-border-subtle) !important}.border-warning-subtle{border-color:var(--bs-warning-border-subtle) !important}.border-danger-subtle{border-color:var(--bs-danger-border-subtle) !important}.border-light-subtle{border-color:var(--bs-light-border-subtle) !important}.border-dark-subtle{border-color:var(--bs-dark-border-subtle) !important}.border-1{border-width:1px !important}.border-2{border-width:2px !important}.border-3{border-width:3px !important}.border-4{border-width:4px !important}.border-5{border-width:5px !important}.border-opacity-10{--bs-border-opacity: 0.1}.border-opacity-25{--bs-border-opacity: 0.25}.border-opacity-50{--bs-border-opacity: 0.5}.border-opacity-75{--bs-border-opacity: 0.75}.border-opacity-100{--bs-border-opacity: 1}.w-25{width:25% !important}.w-50{width:50% !important}.w-75{width:75% !important}.w-100{width:100% !important}.w-auto{width:auto !important}.mw-100{max-width:100% !important}.vw-100{width:100vw !important}.min-vw-100{min-width:100vw !important}.h-25{height:25% !important}.h-50{height:50% !important}.h-75{height:75% !important}.h-100{height:100% !important}.h-auto{height:auto !important}.mh-100{max-height:100% !important}.vh-100{height:100vh !important}.min-vh-100{min-height:100vh !important}.flex-fill{flex:1 1 auto !important}.flex-row{flex-direction:row !important}.flex-column{flex-direction:column !important}.flex-row-reverse{flex-direction:row-reverse !important}.flex-column-reverse{flex-direction:column-reverse !important}.flex-grow-0{flex-grow:0 !important}.flex-grow-1{flex-grow:1 !important}.flex-shrink-0{flex-shrink:0 !important}.flex-shrink-1{flex-shrink:1 !important}.flex-wrap{flex-wrap:wrap !important}.flex-nowrap{flex-wrap:nowrap !important}.flex-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-start{justify-content:flex-start !important}.justify-content-end{justify-content:flex-end !important}.justify-content-center{justify-content:center !important}.justify-content-between{justify-content:space-between !important}.justify-content-around{justify-content:space-around !important}.justify-content-evenly{justify-content:space-evenly !important}.align-items-start{align-items:flex-start !important}.align-items-end{align-items:flex-end !important}.align-items-center{align-items:center !important}.align-items-baseline{align-items:baseline !important}.align-items-stretch{align-items:stretch !important}.align-content-start{align-content:flex-start !important}.align-content-end{align-content:flex-end !important}.align-content-center{align-content:center !important}.align-content-between{align-content:space-between !important}.align-content-around{align-content:space-around !important}.align-content-stretch{align-content:stretch !important}.align-self-auto{align-self:auto !important}.align-self-start{align-self:flex-start !important}.align-self-end{align-self:flex-end !important}.align-self-center{align-self:center !important}.align-self-baseline{align-self:baseline !important}.align-self-stretch{align-self:stretch !important}.order-first{order:-1 !important}.order-0{order:0 !important}.order-1{order:1 !important}.order-2{order:2 !important}.order-3{order:3 !important}.order-4{order:4 !important}.order-5{order:5 !important}.order-last{order:6 !important}.m-0{margin:0 !important}.m-1{margin:.25rem !important}.m-2{margin:.5rem !important}.m-3{margin:1rem !important}.m-4{margin:1.5rem !important}.m-5{margin:3rem !important}.m-auto{margin:auto !important}.mx-0{margin-right:0 !important;margin-left:0 !important}.mx-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-3{margin-right:1rem !important;margin-left:1rem !important}.mx-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-5{margin-right:3rem !important;margin-left:3rem !important}.mx-auto{margin-right:auto !important;margin-left:auto !important}.my-0{margin-top:0 !important;margin-bottom:0 !important}.my-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-0{margin-top:0 !important}.mt-1{margin-top:.25rem !important}.mt-2{margin-top:.5rem !important}.mt-3{margin-top:1rem !important}.mt-4{margin-top:1.5rem !important}.mt-5{margin-top:3rem !important}.mt-auto{margin-top:auto !important}.me-0{margin-right:0 !important}.me-1{margin-right:.25rem !important}.me-2{margin-right:.5rem !important}.me-3{margin-right:1rem !important}.me-4{margin-right:1.5rem !important}.me-5{margin-right:3rem !important}.me-auto{margin-right:auto !important}.mb-0{margin-bottom:0 !important}.mb-1{margin-bottom:.25rem !important}.mb-2{margin-bottom:.5rem !important}.mb-3{margin-bottom:1rem !important}.mb-4{margin-bottom:1.5rem !important}.mb-5{margin-bottom:3rem !important}.mb-auto{margin-bottom:auto !important}.ms-0{margin-left:0 !important}.ms-1{margin-left:.25rem !important}.ms-2{margin-left:.5rem !important}.ms-3{margin-left:1rem !important}.ms-4{margin-left:1.5rem !important}.ms-5{margin-left:3rem !important}.ms-auto{margin-left:auto !important}.p-0{padding:0 !important}.p-1{padding:.25rem !important}.p-2{padding:.5rem !important}.p-3{padding:1rem !important}.p-4{padding:1.5rem !important}.p-5{padding:3rem !important}.px-0{padding-right:0 !important;padding-left:0 !important}.px-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-3{padding-right:1rem !important;padding-left:1rem !important}.px-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-5{padding-right:3rem !important;padding-left:3rem !important}.py-0{padding-top:0 !important;padding-bottom:0 !important}.py-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-0{padding-top:0 !important}.pt-1{padding-top:.25rem !important}.pt-2{padding-top:.5rem !important}.pt-3{padding-top:1rem !important}.pt-4{padding-top:1.5rem !important}.pt-5{padding-top:3rem !important}.pe-0{padding-right:0 !important}.pe-1{padding-right:.25rem !important}.pe-2{padding-right:.5rem !important}.pe-3{padding-right:1rem !important}.pe-4{padding-right:1.5rem !important}.pe-5{padding-right:3rem !important}.pb-0{padding-bottom:0 !important}.pb-1{padding-bottom:.25rem !important}.pb-2{padding-bottom:.5rem !important}.pb-3{padding-bottom:1rem !important}.pb-4{padding-bottom:1.5rem !important}.pb-5{padding-bottom:3rem !important}.ps-0{padding-left:0 !important}.ps-1{padding-left:.25rem !important}.ps-2{padding-left:.5rem !important}.ps-3{padding-left:1rem !important}.ps-4{padding-left:1.5rem !important}.ps-5{padding-left:3rem !important}.gap-0{gap:0 !important}.gap-1{gap:.25rem !important}.gap-2{gap:.5rem !important}.gap-3{gap:1rem !important}.gap-4{gap:1.5rem !important}.gap-5{gap:3rem !important}.row-gap-0{row-gap:0 !important}.row-gap-1{row-gap:.25rem !important}.row-gap-2{row-gap:.5rem !important}.row-gap-3{row-gap:1rem !important}.row-gap-4{row-gap:1.5rem !important}.row-gap-5{row-gap:3rem !important}.column-gap-0{column-gap:0 !important}.column-gap-1{column-gap:.25rem !important}.column-gap-2{column-gap:.5rem !important}.column-gap-3{column-gap:1rem !important}.column-gap-4{column-gap:1.5rem !important}.column-gap-5{column-gap:3rem !important}.font-monospace{font-family:var(--bs-font-monospace) !important}.fs-1{font-size:calc(1.325rem + 0.9vw) !important}.fs-2{font-size:calc(1.29rem + 0.48vw) !important}.fs-3{font-size:calc(1.27rem + 0.24vw) !important}.fs-4{font-size:1.25rem !important}.fs-5{font-size:1.1rem !important}.fs-6{font-size:1rem !important}.fst-italic{font-style:italic !important}.fst-normal{font-style:normal !important}.fw-lighter{font-weight:lighter !important}.fw-light{font-weight:300 !important}.fw-normal{font-weight:400 !important}.fw-medium{font-weight:500 !important}.fw-semibold{font-weight:600 !important}.fw-bold{font-weight:700 !important}.fw-bolder{font-weight:bolder !important}.lh-1{line-height:1 !important}.lh-sm{line-height:1.25 !important}.lh-base{line-height:1.5 !important}.lh-lg{line-height:2 !important}.text-start{text-align:left !important}.text-end{text-align:right !important}.text-center{text-align:center !important}.text-decoration-none{text-decoration:none !important}.text-decoration-underline{text-decoration:underline !important}.text-decoration-line-through{text-decoration:line-through !important}.text-lowercase{text-transform:lowercase !important}.text-uppercase{text-transform:uppercase !important}.text-capitalize{text-transform:capitalize !important}.text-wrap{white-space:normal !important}.text-nowrap{white-space:nowrap !important}.text-break{word-wrap:break-word !important;word-break:break-word !important}.text-default{--bs-text-opacity: 1;color:rgba(var(--bs-default-rgb), var(--bs-text-opacity)) !important}.text-primary{--bs-text-opacity: 1;color:rgba(var(--bs-primary-rgb), var(--bs-text-opacity)) !important}.text-secondary{--bs-text-opacity: 1;color:rgba(var(--bs-secondary-rgb), var(--bs-text-opacity)) !important}.text-success{--bs-text-opacity: 1;color:rgba(var(--bs-success-rgb), var(--bs-text-opacity)) !important}.text-info{--bs-text-opacity: 1;color:rgba(var(--bs-info-rgb), var(--bs-text-opacity)) !important}.text-warning{--bs-text-opacity: 1;color:rgba(var(--bs-warning-rgb), var(--bs-text-opacity)) !important}.text-danger{--bs-text-opacity: 1;color:rgba(var(--bs-danger-rgb), var(--bs-text-opacity)) !important}.text-light{--bs-text-opacity: 1;color:rgba(var(--bs-light-rgb), var(--bs-text-opacity)) !important}.text-dark{--bs-text-opacity: 1;color:rgba(var(--bs-dark-rgb), var(--bs-text-opacity)) !important}.text-black{--bs-text-opacity: 1;color:rgba(var(--bs-black-rgb), var(--bs-text-opacity)) !important}.text-white{--bs-text-opacity: 1;color:rgba(var(--bs-white-rgb), var(--bs-text-opacity)) !important}.text-body{--bs-text-opacity: 1;color:rgba(var(--bs-body-color-rgb), var(--bs-text-opacity)) !important}.text-muted{--bs-text-opacity: 1;color:var(--bs-secondary-color) !important}.text-black-50{--bs-text-opacity: 1;color:rgba(0,0,0,.5) !important}.text-white-50{--bs-text-opacity: 1;color:rgba(255,255,255,.5) !important}.text-body-secondary{--bs-text-opacity: 1;color:var(--bs-secondary-color) !important}.text-body-tertiary{--bs-text-opacity: 1;color:var(--bs-tertiary-color) !important}.text-body-emphasis{--bs-text-opacity: 1;color:var(--bs-emphasis-color) !important}.text-reset{--bs-text-opacity: 1;color:inherit !important}.text-opacity-25{--bs-text-opacity: 0.25}.text-opacity-50{--bs-text-opacity: 0.5}.text-opacity-75{--bs-text-opacity: 0.75}.text-opacity-100{--bs-text-opacity: 1}.text-primary-emphasis{color:var(--bs-primary-text-emphasis) !important}.text-secondary-emphasis{color:var(--bs-secondary-text-emphasis) !important}.text-success-emphasis{color:var(--bs-success-text-emphasis) !important}.text-info-emphasis{color:var(--bs-info-text-emphasis) !important}.text-warning-emphasis{color:var(--bs-warning-text-emphasis) !important}.text-danger-emphasis{color:var(--bs-danger-text-emphasis) !important}.text-light-emphasis{color:var(--bs-light-text-emphasis) !important}.text-dark-emphasis{color:var(--bs-dark-text-emphasis) !important}.link-opacity-10{--bs-link-opacity: 0.1}.link-opacity-10-hover:hover{--bs-link-opacity: 0.1}.link-opacity-25{--bs-link-opacity: 0.25}.link-opacity-25-hover:hover{--bs-link-opacity: 0.25}.link-opacity-50{--bs-link-opacity: 0.5}.link-opacity-50-hover:hover{--bs-link-opacity: 0.5}.link-opacity-75{--bs-link-opacity: 0.75}.link-opacity-75-hover:hover{--bs-link-opacity: 0.75}.link-opacity-100{--bs-link-opacity: 1}.link-opacity-100-hover:hover{--bs-link-opacity: 1}.link-offset-1{text-underline-offset:.125em !important}.link-offset-1-hover:hover{text-underline-offset:.125em !important}.link-offset-2{text-underline-offset:.25em !important}.link-offset-2-hover:hover{text-underline-offset:.25em !important}.link-offset-3{text-underline-offset:.375em !important}.link-offset-3-hover:hover{text-underline-offset:.375em !important}.link-underline-default{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-default-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-primary{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-primary-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-secondary{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-secondary-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-success{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-success-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-info{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-info-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-warning{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-warning-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-danger{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-danger-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-light{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-light-rgb), var(--bs-link-underline-opacity)) !important}.link-underline-dark{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-dark-rgb), var(--bs-link-underline-opacity)) !important}.link-underline{--bs-link-underline-opacity: 1;text-decoration-color:rgba(var(--bs-link-color-rgb), var(--bs-link-underline-opacity, 1)) !important}.link-underline-opacity-0{--bs-link-underline-opacity: 0}.link-underline-opacity-0-hover:hover{--bs-link-underline-opacity: 0}.link-underline-opacity-10{--bs-link-underline-opacity: 0.1}.link-underline-opacity-10-hover:hover{--bs-link-underline-opacity: 0.1}.link-underline-opacity-25{--bs-link-underline-opacity: 0.25}.link-underline-opacity-25-hover:hover{--bs-link-underline-opacity: 0.25}.link-underline-opacity-50{--bs-link-underline-opacity: 0.5}.link-underline-opacity-50-hover:hover{--bs-link-underline-opacity: 0.5}.link-underline-opacity-75{--bs-link-underline-opacity: 0.75}.link-underline-opacity-75-hover:hover{--bs-link-underline-opacity: 0.75}.link-underline-opacity-100{--bs-link-underline-opacity: 1}.link-underline-opacity-100-hover:hover{--bs-link-underline-opacity: 1}.bg-default{--bs-bg-opacity: 1;background-color:rgba(var(--bs-default-rgb), var(--bs-bg-opacity)) !important}.bg-primary{--bs-bg-opacity: 1;background-color:rgba(var(--bs-primary-rgb), var(--bs-bg-opacity)) !important}.bg-secondary{--bs-bg-opacity: 1;background-color:rgba(var(--bs-secondary-rgb), var(--bs-bg-opacity)) !important}.bg-success{--bs-bg-opacity: 1;background-color:rgba(var(--bs-success-rgb), var(--bs-bg-opacity)) !important}.bg-info{--bs-bg-opacity: 1;background-color:rgba(var(--bs-info-rgb), var(--bs-bg-opacity)) !important}.bg-warning{--bs-bg-opacity: 1;background-color:rgba(var(--bs-warning-rgb), var(--bs-bg-opacity)) !important}.bg-danger{--bs-bg-opacity: 1;background-color:rgba(var(--bs-danger-rgb), var(--bs-bg-opacity)) !important}.bg-light{--bs-bg-opacity: 1;background-color:rgba(var(--bs-light-rgb), var(--bs-bg-opacity)) !important}.bg-dark{--bs-bg-opacity: 1;background-color:rgba(var(--bs-dark-rgb), var(--bs-bg-opacity)) !important}.bg-black{--bs-bg-opacity: 1;background-color:rgba(var(--bs-black-rgb), var(--bs-bg-opacity)) !important}.bg-white{--bs-bg-opacity: 1;background-color:rgba(var(--bs-white-rgb), var(--bs-bg-opacity)) !important}.bg-body{--bs-bg-opacity: 1;background-color:rgba(var(--bs-body-bg-rgb), var(--bs-bg-opacity)) !important}.bg-transparent{--bs-bg-opacity: 1;background-color:rgba(0,0,0,0) !important}.bg-body-secondary{--bs-bg-opacity: 1;background-color:rgba(var(--bs-secondary-bg-rgb), var(--bs-bg-opacity)) !important}.bg-body-tertiary{--bs-bg-opacity: 1;background-color:rgba(var(--bs-tertiary-bg-rgb), var(--bs-bg-opacity)) !important}.bg-opacity-10{--bs-bg-opacity: 0.1}.bg-opacity-25{--bs-bg-opacity: 0.25}.bg-opacity-50{--bs-bg-opacity: 0.5}.bg-opacity-75{--bs-bg-opacity: 0.75}.bg-opacity-100{--bs-bg-opacity: 1}.bg-primary-subtle{background-color:var(--bs-primary-bg-subtle) !important}.bg-secondary-subtle{background-color:var(--bs-secondary-bg-subtle) !important}.bg-success-subtle{background-color:var(--bs-success-bg-subtle) !important}.bg-info-subtle{background-color:var(--bs-info-bg-subtle) !important}.bg-warning-subtle{background-color:var(--bs-warning-bg-subtle) !important}.bg-danger-subtle{background-color:var(--bs-danger-bg-subtle) !important}.bg-light-subtle{background-color:var(--bs-light-bg-subtle) !important}.bg-dark-subtle{background-color:var(--bs-dark-bg-subtle) !important}.bg-gradient{background-image:var(--bs-gradient) !important}.user-select-all{user-select:all !important}.user-select-auto{user-select:auto !important}.user-select-none{user-select:none !important}.pe-none{pointer-events:none !important}.pe-auto{pointer-events:auto !important}.rounded{border-radius:var(--bs-border-radius) !important}.rounded-0{border-radius:0 !important}.rounded-1{border-radius:var(--bs-border-radius-sm) !important}.rounded-2{border-radius:var(--bs-border-radius) !important}.rounded-3{border-radius:var(--bs-border-radius-lg) !important}.rounded-4{border-radius:var(--bs-border-radius-xl) !important}.rounded-5{border-radius:var(--bs-border-radius-xxl) !important}.rounded-circle{border-radius:50% !important}.rounded-pill{border-radius:var(--bs-border-radius-pill) !important}.rounded-top{border-top-left-radius:var(--bs-border-radius) !important;border-top-right-radius:var(--bs-border-radius) !important}.rounded-top-0{border-top-left-radius:0 !important;border-top-right-radius:0 !important}.rounded-top-1{border-top-left-radius:var(--bs-border-radius-sm) !important;border-top-right-radius:var(--bs-border-radius-sm) !important}.rounded-top-2{border-top-left-radius:var(--bs-border-radius) !important;border-top-right-radius:var(--bs-border-radius) !important}.rounded-top-3{border-top-left-radius:var(--bs-border-radius-lg) !important;border-top-right-radius:var(--bs-border-radius-lg) !important}.rounded-top-4{border-top-left-radius:var(--bs-border-radius-xl) !important;border-top-right-radius:var(--bs-border-radius-xl) !important}.rounded-top-5{border-top-left-radius:var(--bs-border-radius-xxl) !important;border-top-right-radius:var(--bs-border-radius-xxl) !important}.rounded-top-circle{border-top-left-radius:50% !important;border-top-right-radius:50% !important}.rounded-top-pill{border-top-left-radius:var(--bs-border-radius-pill) !important;border-top-right-radius:var(--bs-border-radius-pill) !important}.rounded-end{border-top-right-radius:var(--bs-border-radius) !important;border-bottom-right-radius:var(--bs-border-radius) !important}.rounded-end-0{border-top-right-radius:0 !important;border-bottom-right-radius:0 !important}.rounded-end-1{border-top-right-radius:var(--bs-border-radius-sm) !important;border-bottom-right-radius:var(--bs-border-radius-sm) !important}.rounded-end-2{border-top-right-radius:var(--bs-border-radius) !important;border-bottom-right-radius:var(--bs-border-radius) !important}.rounded-end-3{border-top-right-radius:var(--bs-border-radius-lg) !important;border-bottom-right-radius:var(--bs-border-radius-lg) !important}.rounded-end-4{border-top-right-radius:var(--bs-border-radius-xl) !important;border-bottom-right-radius:var(--bs-border-radius-xl) !important}.rounded-end-5{border-top-right-radius:var(--bs-border-radius-xxl) !important;border-bottom-right-radius:var(--bs-border-radius-xxl) !important}.rounded-end-circle{border-top-right-radius:50% !important;border-bottom-right-radius:50% !important}.rounded-end-pill{border-top-right-radius:var(--bs-border-radius-pill) !important;border-bottom-right-radius:var(--bs-border-radius-pill) !important}.rounded-bottom{border-bottom-right-radius:var(--bs-border-radius) !important;border-bottom-left-radius:var(--bs-border-radius) !important}.rounded-bottom-0{border-bottom-right-radius:0 !important;border-bottom-left-radius:0 !important}.rounded-bottom-1{border-bottom-right-radius:var(--bs-border-radius-sm) !important;border-bottom-left-radius:var(--bs-border-radius-sm) !important}.rounded-bottom-2{border-bottom-right-radius:var(--bs-border-radius) !important;border-bottom-left-radius:var(--bs-border-radius) !important}.rounded-bottom-3{border-bottom-right-radius:var(--bs-border-radius-lg) !important;border-bottom-left-radius:var(--bs-border-radius-lg) !important}.rounded-bottom-4{border-bottom-right-radius:var(--bs-border-radius-xl) !important;border-bottom-left-radius:var(--bs-border-radius-xl) !important}.rounded-bottom-5{border-bottom-right-radius:var(--bs-border-radius-xxl) !important;border-bottom-left-radius:var(--bs-border-radius-xxl) !important}.rounded-bottom-circle{border-bottom-right-radius:50% !important;border-bottom-left-radius:50% !important}.rounded-bottom-pill{border-bottom-right-radius:var(--bs-border-radius-pill) !important;border-bottom-left-radius:var(--bs-border-radius-pill) !important}.rounded-start{border-bottom-left-radius:var(--bs-border-radius) !important;border-top-left-radius:var(--bs-border-radius) !important}.rounded-start-0{border-bottom-left-radius:0 !important;border-top-left-radius:0 !important}.rounded-start-1{border-bottom-left-radius:var(--bs-border-radius-sm) !important;border-top-left-radius:var(--bs-border-radius-sm) !important}.rounded-start-2{border-bottom-left-radius:var(--bs-border-radius) !important;border-top-left-radius:var(--bs-border-radius) !important}.rounded-start-3{border-bottom-left-radius:var(--bs-border-radius-lg) !important;border-top-left-radius:var(--bs-border-radius-lg) !important}.rounded-start-4{border-bottom-left-radius:var(--bs-border-radius-xl) !important;border-top-left-radius:var(--bs-border-radius-xl) !important}.rounded-start-5{border-bottom-left-radius:var(--bs-border-radius-xxl) !important;border-top-left-radius:var(--bs-border-radius-xxl) !important}.rounded-start-circle{border-bottom-left-radius:50% !important;border-top-left-radius:50% !important}.rounded-start-pill{border-bottom-left-radius:var(--bs-border-radius-pill) !important;border-top-left-radius:var(--bs-border-radius-pill) !important}.visible{visibility:visible !important}.invisible{visibility:hidden !important}.z-n1{z-index:-1 !important}.z-0{z-index:0 !important}.z-1{z-index:1 !important}.z-2{z-index:2 !important}.z-3{z-index:3 !important}@media(min-width: 576px){.float-sm-start{float:left !important}.float-sm-end{float:right !important}.float-sm-none{float:none !important}.object-fit-sm-contain{object-fit:contain !important}.object-fit-sm-cover{object-fit:cover !important}.object-fit-sm-fill{object-fit:fill !important}.object-fit-sm-scale{object-fit:scale-down !important}.object-fit-sm-none{object-fit:none !important}.d-sm-inline{display:inline !important}.d-sm-inline-block{display:inline-block !important}.d-sm-block{display:block !important}.d-sm-grid{display:grid !important}.d-sm-inline-grid{display:inline-grid !important}.d-sm-table{display:table !important}.d-sm-table-row{display:table-row !important}.d-sm-table-cell{display:table-cell !important}.d-sm-flex{display:flex !important}.d-sm-inline-flex{display:inline-flex !important}.d-sm-none{display:none !important}.flex-sm-fill{flex:1 1 auto !important}.flex-sm-row{flex-direction:row !important}.flex-sm-column{flex-direction:column !important}.flex-sm-row-reverse{flex-direction:row-reverse !important}.flex-sm-column-reverse{flex-direction:column-reverse !important}.flex-sm-grow-0{flex-grow:0 !important}.flex-sm-grow-1{flex-grow:1 !important}.flex-sm-shrink-0{flex-shrink:0 !important}.flex-sm-shrink-1{flex-shrink:1 !important}.flex-sm-wrap{flex-wrap:wrap !important}.flex-sm-nowrap{flex-wrap:nowrap !important}.flex-sm-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-sm-start{justify-content:flex-start !important}.justify-content-sm-end{justify-content:flex-end !important}.justify-content-sm-center{justify-content:center !important}.justify-content-sm-between{justify-content:space-between !important}.justify-content-sm-around{justify-content:space-around !important}.justify-content-sm-evenly{justify-content:space-evenly !important}.align-items-sm-start{align-items:flex-start !important}.align-items-sm-end{align-items:flex-end !important}.align-items-sm-center{align-items:center !important}.align-items-sm-baseline{align-items:baseline !important}.align-items-sm-stretch{align-items:stretch !important}.align-content-sm-start{align-content:flex-start !important}.align-content-sm-end{align-content:flex-end !important}.align-content-sm-center{align-content:center !important}.align-content-sm-between{align-content:space-between !important}.align-content-sm-around{align-content:space-around !important}.align-content-sm-stretch{align-content:stretch !important}.align-self-sm-auto{align-self:auto !important}.align-self-sm-start{align-self:flex-start !important}.align-self-sm-end{align-self:flex-end !important}.align-self-sm-center{align-self:center !important}.align-self-sm-baseline{align-self:baseline !important}.align-self-sm-stretch{align-self:stretch !important}.order-sm-first{order:-1 !important}.order-sm-0{order:0 !important}.order-sm-1{order:1 !important}.order-sm-2{order:2 !important}.order-sm-3{order:3 !important}.order-sm-4{order:4 !important}.order-sm-5{order:5 !important}.order-sm-last{order:6 !important}.m-sm-0{margin:0 !important}.m-sm-1{margin:.25rem !important}.m-sm-2{margin:.5rem !important}.m-sm-3{margin:1rem !important}.m-sm-4{margin:1.5rem !important}.m-sm-5{margin:3rem !important}.m-sm-auto{margin:auto !important}.mx-sm-0{margin-right:0 !important;margin-left:0 !important}.mx-sm-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-sm-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-sm-3{margin-right:1rem !important;margin-left:1rem !important}.mx-sm-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-sm-5{margin-right:3rem !important;margin-left:3rem !important}.mx-sm-auto{margin-right:auto !important;margin-left:auto !important}.my-sm-0{margin-top:0 !important;margin-bottom:0 !important}.my-sm-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-sm-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-sm-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-sm-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-sm-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-sm-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-sm-0{margin-top:0 !important}.mt-sm-1{margin-top:.25rem !important}.mt-sm-2{margin-top:.5rem !important}.mt-sm-3{margin-top:1rem !important}.mt-sm-4{margin-top:1.5rem !important}.mt-sm-5{margin-top:3rem !important}.mt-sm-auto{margin-top:auto !important}.me-sm-0{margin-right:0 !important}.me-sm-1{margin-right:.25rem !important}.me-sm-2{margin-right:.5rem !important}.me-sm-3{margin-right:1rem !important}.me-sm-4{margin-right:1.5rem !important}.me-sm-5{margin-right:3rem !important}.me-sm-auto{margin-right:auto !important}.mb-sm-0{margin-bottom:0 !important}.mb-sm-1{margin-bottom:.25rem !important}.mb-sm-2{margin-bottom:.5rem !important}.mb-sm-3{margin-bottom:1rem !important}.mb-sm-4{margin-bottom:1.5rem !important}.mb-sm-5{margin-bottom:3rem !important}.mb-sm-auto{margin-bottom:auto !important}.ms-sm-0{margin-left:0 !important}.ms-sm-1{margin-left:.25rem !important}.ms-sm-2{margin-left:.5rem !important}.ms-sm-3{margin-left:1rem !important}.ms-sm-4{margin-left:1.5rem !important}.ms-sm-5{margin-left:3rem !important}.ms-sm-auto{margin-left:auto !important}.p-sm-0{padding:0 !important}.p-sm-1{padding:.25rem !important}.p-sm-2{padding:.5rem !important}.p-sm-3{padding:1rem !important}.p-sm-4{padding:1.5rem !important}.p-sm-5{padding:3rem !important}.px-sm-0{padding-right:0 !important;padding-left:0 !important}.px-sm-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-sm-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-sm-3{padding-right:1rem !important;padding-left:1rem !important}.px-sm-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-sm-5{padding-right:3rem !important;padding-left:3rem !important}.py-sm-0{padding-top:0 !important;padding-bottom:0 !important}.py-sm-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-sm-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-sm-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-sm-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-sm-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-sm-0{padding-top:0 !important}.pt-sm-1{padding-top:.25rem !important}.pt-sm-2{padding-top:.5rem !important}.pt-sm-3{padding-top:1rem !important}.pt-sm-4{padding-top:1.5rem !important}.pt-sm-5{padding-top:3rem !important}.pe-sm-0{padding-right:0 !important}.pe-sm-1{padding-right:.25rem !important}.pe-sm-2{padding-right:.5rem !important}.pe-sm-3{padding-right:1rem !important}.pe-sm-4{padding-right:1.5rem !important}.pe-sm-5{padding-right:3rem !important}.pb-sm-0{padding-bottom:0 !important}.pb-sm-1{padding-bottom:.25rem !important}.pb-sm-2{padding-bottom:.5rem !important}.pb-sm-3{padding-bottom:1rem !important}.pb-sm-4{padding-bottom:1.5rem !important}.pb-sm-5{padding-bottom:3rem !important}.ps-sm-0{padding-left:0 !important}.ps-sm-1{padding-left:.25rem !important}.ps-sm-2{padding-left:.5rem !important}.ps-sm-3{padding-left:1rem !important}.ps-sm-4{padding-left:1.5rem !important}.ps-sm-5{padding-left:3rem !important}.gap-sm-0{gap:0 !important}.gap-sm-1{gap:.25rem !important}.gap-sm-2{gap:.5rem !important}.gap-sm-3{gap:1rem !important}.gap-sm-4{gap:1.5rem !important}.gap-sm-5{gap:3rem !important}.row-gap-sm-0{row-gap:0 !important}.row-gap-sm-1{row-gap:.25rem !important}.row-gap-sm-2{row-gap:.5rem !important}.row-gap-sm-3{row-gap:1rem !important}.row-gap-sm-4{row-gap:1.5rem !important}.row-gap-sm-5{row-gap:3rem !important}.column-gap-sm-0{column-gap:0 !important}.column-gap-sm-1{column-gap:.25rem !important}.column-gap-sm-2{column-gap:.5rem !important}.column-gap-sm-3{column-gap:1rem !important}.column-gap-sm-4{column-gap:1.5rem !important}.column-gap-sm-5{column-gap:3rem !important}.text-sm-start{text-align:left !important}.text-sm-end{text-align:right !important}.text-sm-center{text-align:center !important}}@media(min-width: 768px){.float-md-start{float:left !important}.float-md-end{float:right !important}.float-md-none{float:none !important}.object-fit-md-contain{object-fit:contain !important}.object-fit-md-cover{object-fit:cover !important}.object-fit-md-fill{object-fit:fill !important}.object-fit-md-scale{object-fit:scale-down !important}.object-fit-md-none{object-fit:none !important}.d-md-inline{display:inline !important}.d-md-inline-block{display:inline-block !important}.d-md-block{display:block !important}.d-md-grid{display:grid !important}.d-md-inline-grid{display:inline-grid !important}.d-md-table{display:table !important}.d-md-table-row{display:table-row !important}.d-md-table-cell{display:table-cell !important}.d-md-flex{display:flex !important}.d-md-inline-flex{display:inline-flex !important}.d-md-none{display:none !important}.flex-md-fill{flex:1 1 auto !important}.flex-md-row{flex-direction:row !important}.flex-md-column{flex-direction:column !important}.flex-md-row-reverse{flex-direction:row-reverse !important}.flex-md-column-reverse{flex-direction:column-reverse !important}.flex-md-grow-0{flex-grow:0 !important}.flex-md-grow-1{flex-grow:1 !important}.flex-md-shrink-0{flex-shrink:0 !important}.flex-md-shrink-1{flex-shrink:1 !important}.flex-md-wrap{flex-wrap:wrap !important}.flex-md-nowrap{flex-wrap:nowrap !important}.flex-md-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-md-start{justify-content:flex-start !important}.justify-content-md-end{justify-content:flex-end !important}.justify-content-md-center{justify-content:center !important}.justify-content-md-between{justify-content:space-between !important}.justify-content-md-around{justify-content:space-around !important}.justify-content-md-evenly{justify-content:space-evenly !important}.align-items-md-start{align-items:flex-start !important}.align-items-md-end{align-items:flex-end !important}.align-items-md-center{align-items:center !important}.align-items-md-baseline{align-items:baseline !important}.align-items-md-stretch{align-items:stretch !important}.align-content-md-start{align-content:flex-start !important}.align-content-md-end{align-content:flex-end !important}.align-content-md-center{align-content:center !important}.align-content-md-between{align-content:space-between !important}.align-content-md-around{align-content:space-around !important}.align-content-md-stretch{align-content:stretch !important}.align-self-md-auto{align-self:auto !important}.align-self-md-start{align-self:flex-start !important}.align-self-md-end{align-self:flex-end !important}.align-self-md-center{align-self:center !important}.align-self-md-baseline{align-self:baseline !important}.align-self-md-stretch{align-self:stretch !important}.order-md-first{order:-1 !important}.order-md-0{order:0 !important}.order-md-1{order:1 !important}.order-md-2{order:2 !important}.order-md-3{order:3 !important}.order-md-4{order:4 !important}.order-md-5{order:5 !important}.order-md-last{order:6 !important}.m-md-0{margin:0 !important}.m-md-1{margin:.25rem !important}.m-md-2{margin:.5rem !important}.m-md-3{margin:1rem !important}.m-md-4{margin:1.5rem !important}.m-md-5{margin:3rem !important}.m-md-auto{margin:auto !important}.mx-md-0{margin-right:0 !important;margin-left:0 !important}.mx-md-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-md-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-md-3{margin-right:1rem !important;margin-left:1rem !important}.mx-md-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-md-5{margin-right:3rem !important;margin-left:3rem !important}.mx-md-auto{margin-right:auto !important;margin-left:auto !important}.my-md-0{margin-top:0 !important;margin-bottom:0 !important}.my-md-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-md-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-md-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-md-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-md-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-md-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-md-0{margin-top:0 !important}.mt-md-1{margin-top:.25rem !important}.mt-md-2{margin-top:.5rem !important}.mt-md-3{margin-top:1rem !important}.mt-md-4{margin-top:1.5rem !important}.mt-md-5{margin-top:3rem !important}.mt-md-auto{margin-top:auto !important}.me-md-0{margin-right:0 !important}.me-md-1{margin-right:.25rem !important}.me-md-2{margin-right:.5rem !important}.me-md-3{margin-right:1rem !important}.me-md-4{margin-right:1.5rem !important}.me-md-5{margin-right:3rem !important}.me-md-auto{margin-right:auto !important}.mb-md-0{margin-bottom:0 !important}.mb-md-1{margin-bottom:.25rem !important}.mb-md-2{margin-bottom:.5rem !important}.mb-md-3{margin-bottom:1rem !important}.mb-md-4{margin-bottom:1.5rem !important}.mb-md-5{margin-bottom:3rem !important}.mb-md-auto{margin-bottom:auto !important}.ms-md-0{margin-left:0 !important}.ms-md-1{margin-left:.25rem !important}.ms-md-2{margin-left:.5rem !important}.ms-md-3{margin-left:1rem !important}.ms-md-4{margin-left:1.5rem !important}.ms-md-5{margin-left:3rem !important}.ms-md-auto{margin-left:auto !important}.p-md-0{padding:0 !important}.p-md-1{padding:.25rem !important}.p-md-2{padding:.5rem !important}.p-md-3{padding:1rem !important}.p-md-4{padding:1.5rem !important}.p-md-5{padding:3rem !important}.px-md-0{padding-right:0 !important;padding-left:0 !important}.px-md-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-md-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-md-3{padding-right:1rem !important;padding-left:1rem !important}.px-md-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-md-5{padding-right:3rem !important;padding-left:3rem !important}.py-md-0{padding-top:0 !important;padding-bottom:0 !important}.py-md-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-md-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-md-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-md-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-md-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-md-0{padding-top:0 !important}.pt-md-1{padding-top:.25rem !important}.pt-md-2{padding-top:.5rem !important}.pt-md-3{padding-top:1rem !important}.pt-md-4{padding-top:1.5rem !important}.pt-md-5{padding-top:3rem !important}.pe-md-0{padding-right:0 !important}.pe-md-1{padding-right:.25rem !important}.pe-md-2{padding-right:.5rem !important}.pe-md-3{padding-right:1rem !important}.pe-md-4{padding-right:1.5rem !important}.pe-md-5{padding-right:3rem !important}.pb-md-0{padding-bottom:0 !important}.pb-md-1{padding-bottom:.25rem !important}.pb-md-2{padding-bottom:.5rem !important}.pb-md-3{padding-bottom:1rem !important}.pb-md-4{padding-bottom:1.5rem !important}.pb-md-5{padding-bottom:3rem !important}.ps-md-0{padding-left:0 !important}.ps-md-1{padding-left:.25rem !important}.ps-md-2{padding-left:.5rem !important}.ps-md-3{padding-left:1rem !important}.ps-md-4{padding-left:1.5rem !important}.ps-md-5{padding-left:3rem !important}.gap-md-0{gap:0 !important}.gap-md-1{gap:.25rem !important}.gap-md-2{gap:.5rem !important}.gap-md-3{gap:1rem !important}.gap-md-4{gap:1.5rem !important}.gap-md-5{gap:3rem !important}.row-gap-md-0{row-gap:0 !important}.row-gap-md-1{row-gap:.25rem !important}.row-gap-md-2{row-gap:.5rem !important}.row-gap-md-3{row-gap:1rem !important}.row-gap-md-4{row-gap:1.5rem !important}.row-gap-md-5{row-gap:3rem !important}.column-gap-md-0{column-gap:0 !important}.column-gap-md-1{column-gap:.25rem !important}.column-gap-md-2{column-gap:.5rem !important}.column-gap-md-3{column-gap:1rem !important}.column-gap-md-4{column-gap:1.5rem !important}.column-gap-md-5{column-gap:3rem !important}.text-md-start{text-align:left !important}.text-md-end{text-align:right !important}.text-md-center{text-align:center !important}}@media(min-width: 992px){.float-lg-start{float:left !important}.float-lg-end{float:right !important}.float-lg-none{float:none !important}.object-fit-lg-contain{object-fit:contain !important}.object-fit-lg-cover{object-fit:cover !important}.object-fit-lg-fill{object-fit:fill !important}.object-fit-lg-scale{object-fit:scale-down !important}.object-fit-lg-none{object-fit:none !important}.d-lg-inline{display:inline !important}.d-lg-inline-block{display:inline-block !important}.d-lg-block{display:block !important}.d-lg-grid{display:grid !important}.d-lg-inline-grid{display:inline-grid !important}.d-lg-table{display:table !important}.d-lg-table-row{display:table-row !important}.d-lg-table-cell{display:table-cell !important}.d-lg-flex{display:flex !important}.d-lg-inline-flex{display:inline-flex !important}.d-lg-none{display:none !important}.flex-lg-fill{flex:1 1 auto !important}.flex-lg-row{flex-direction:row !important}.flex-lg-column{flex-direction:column !important}.flex-lg-row-reverse{flex-direction:row-reverse !important}.flex-lg-column-reverse{flex-direction:column-reverse !important}.flex-lg-grow-0{flex-grow:0 !important}.flex-lg-grow-1{flex-grow:1 !important}.flex-lg-shrink-0{flex-shrink:0 !important}.flex-lg-shrink-1{flex-shrink:1 !important}.flex-lg-wrap{flex-wrap:wrap !important}.flex-lg-nowrap{flex-wrap:nowrap !important}.flex-lg-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-lg-start{justify-content:flex-start !important}.justify-content-lg-end{justify-content:flex-end !important}.justify-content-lg-center{justify-content:center !important}.justify-content-lg-between{justify-content:space-between !important}.justify-content-lg-around{justify-content:space-around !important}.justify-content-lg-evenly{justify-content:space-evenly !important}.align-items-lg-start{align-items:flex-start !important}.align-items-lg-end{align-items:flex-end !important}.align-items-lg-center{align-items:center !important}.align-items-lg-baseline{align-items:baseline !important}.align-items-lg-stretch{align-items:stretch !important}.align-content-lg-start{align-content:flex-start !important}.align-content-lg-end{align-content:flex-end !important}.align-content-lg-center{align-content:center !important}.align-content-lg-between{align-content:space-between !important}.align-content-lg-around{align-content:space-around !important}.align-content-lg-stretch{align-content:stretch !important}.align-self-lg-auto{align-self:auto !important}.align-self-lg-start{align-self:flex-start !important}.align-self-lg-end{align-self:flex-end !important}.align-self-lg-center{align-self:center !important}.align-self-lg-baseline{align-self:baseline !important}.align-self-lg-stretch{align-self:stretch !important}.order-lg-first{order:-1 !important}.order-lg-0{order:0 !important}.order-lg-1{order:1 !important}.order-lg-2{order:2 !important}.order-lg-3{order:3 !important}.order-lg-4{order:4 !important}.order-lg-5{order:5 !important}.order-lg-last{order:6 !important}.m-lg-0{margin:0 !important}.m-lg-1{margin:.25rem !important}.m-lg-2{margin:.5rem !important}.m-lg-3{margin:1rem !important}.m-lg-4{margin:1.5rem !important}.m-lg-5{margin:3rem !important}.m-lg-auto{margin:auto !important}.mx-lg-0{margin-right:0 !important;margin-left:0 !important}.mx-lg-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-lg-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-lg-3{margin-right:1rem !important;margin-left:1rem !important}.mx-lg-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-lg-5{margin-right:3rem !important;margin-left:3rem !important}.mx-lg-auto{margin-right:auto !important;margin-left:auto !important}.my-lg-0{margin-top:0 !important;margin-bottom:0 !important}.my-lg-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-lg-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-lg-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-lg-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-lg-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-lg-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-lg-0{margin-top:0 !important}.mt-lg-1{margin-top:.25rem !important}.mt-lg-2{margin-top:.5rem !important}.mt-lg-3{margin-top:1rem !important}.mt-lg-4{margin-top:1.5rem !important}.mt-lg-5{margin-top:3rem !important}.mt-lg-auto{margin-top:auto !important}.me-lg-0{margin-right:0 !important}.me-lg-1{margin-right:.25rem !important}.me-lg-2{margin-right:.5rem !important}.me-lg-3{margin-right:1rem !important}.me-lg-4{margin-right:1.5rem !important}.me-lg-5{margin-right:3rem !important}.me-lg-auto{margin-right:auto !important}.mb-lg-0{margin-bottom:0 !important}.mb-lg-1{margin-bottom:.25rem !important}.mb-lg-2{margin-bottom:.5rem !important}.mb-lg-3{margin-bottom:1rem !important}.mb-lg-4{margin-bottom:1.5rem !important}.mb-lg-5{margin-bottom:3rem !important}.mb-lg-auto{margin-bottom:auto !important}.ms-lg-0{margin-left:0 !important}.ms-lg-1{margin-left:.25rem !important}.ms-lg-2{margin-left:.5rem !important}.ms-lg-3{margin-left:1rem !important}.ms-lg-4{margin-left:1.5rem !important}.ms-lg-5{margin-left:3rem !important}.ms-lg-auto{margin-left:auto !important}.p-lg-0{padding:0 !important}.p-lg-1{padding:.25rem !important}.p-lg-2{padding:.5rem !important}.p-lg-3{padding:1rem !important}.p-lg-4{padding:1.5rem !important}.p-lg-5{padding:3rem !important}.px-lg-0{padding-right:0 !important;padding-left:0 !important}.px-lg-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-lg-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-lg-3{padding-right:1rem !important;padding-left:1rem !important}.px-lg-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-lg-5{padding-right:3rem !important;padding-left:3rem !important}.py-lg-0{padding-top:0 !important;padding-bottom:0 !important}.py-lg-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-lg-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-lg-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-lg-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-lg-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-lg-0{padding-top:0 !important}.pt-lg-1{padding-top:.25rem !important}.pt-lg-2{padding-top:.5rem !important}.pt-lg-3{padding-top:1rem !important}.pt-lg-4{padding-top:1.5rem !important}.pt-lg-5{padding-top:3rem !important}.pe-lg-0{padding-right:0 !important}.pe-lg-1{padding-right:.25rem !important}.pe-lg-2{padding-right:.5rem !important}.pe-lg-3{padding-right:1rem !important}.pe-lg-4{padding-right:1.5rem !important}.pe-lg-5{padding-right:3rem !important}.pb-lg-0{padding-bottom:0 !important}.pb-lg-1{padding-bottom:.25rem !important}.pb-lg-2{padding-bottom:.5rem !important}.pb-lg-3{padding-bottom:1rem !important}.pb-lg-4{padding-bottom:1.5rem !important}.pb-lg-5{padding-bottom:3rem !important}.ps-lg-0{padding-left:0 !important}.ps-lg-1{padding-left:.25rem !important}.ps-lg-2{padding-left:.5rem !important}.ps-lg-3{padding-left:1rem !important}.ps-lg-4{padding-left:1.5rem !important}.ps-lg-5{padding-left:3rem !important}.gap-lg-0{gap:0 !important}.gap-lg-1{gap:.25rem !important}.gap-lg-2{gap:.5rem !important}.gap-lg-3{gap:1rem !important}.gap-lg-4{gap:1.5rem !important}.gap-lg-5{gap:3rem !important}.row-gap-lg-0{row-gap:0 !important}.row-gap-lg-1{row-gap:.25rem !important}.row-gap-lg-2{row-gap:.5rem !important}.row-gap-lg-3{row-gap:1rem !important}.row-gap-lg-4{row-gap:1.5rem !important}.row-gap-lg-5{row-gap:3rem !important}.column-gap-lg-0{column-gap:0 !important}.column-gap-lg-1{column-gap:.25rem !important}.column-gap-lg-2{column-gap:.5rem !important}.column-gap-lg-3{column-gap:1rem !important}.column-gap-lg-4{column-gap:1.5rem !important}.column-gap-lg-5{column-gap:3rem !important}.text-lg-start{text-align:left !important}.text-lg-end{text-align:right !important}.text-lg-center{text-align:center !important}}@media(min-width: 1200px){.float-xl-start{float:left !important}.float-xl-end{float:right !important}.float-xl-none{float:none !important}.object-fit-xl-contain{object-fit:contain !important}.object-fit-xl-cover{object-fit:cover !important}.object-fit-xl-fill{object-fit:fill !important}.object-fit-xl-scale{object-fit:scale-down !important}.object-fit-xl-none{object-fit:none !important}.d-xl-inline{display:inline !important}.d-xl-inline-block{display:inline-block !important}.d-xl-block{display:block !important}.d-xl-grid{display:grid !important}.d-xl-inline-grid{display:inline-grid !important}.d-xl-table{display:table !important}.d-xl-table-row{display:table-row !important}.d-xl-table-cell{display:table-cell !important}.d-xl-flex{display:flex !important}.d-xl-inline-flex{display:inline-flex !important}.d-xl-none{display:none !important}.flex-xl-fill{flex:1 1 auto !important}.flex-xl-row{flex-direction:row !important}.flex-xl-column{flex-direction:column !important}.flex-xl-row-reverse{flex-direction:row-reverse !important}.flex-xl-column-reverse{flex-direction:column-reverse !important}.flex-xl-grow-0{flex-grow:0 !important}.flex-xl-grow-1{flex-grow:1 !important}.flex-xl-shrink-0{flex-shrink:0 !important}.flex-xl-shrink-1{flex-shrink:1 !important}.flex-xl-wrap{flex-wrap:wrap !important}.flex-xl-nowrap{flex-wrap:nowrap !important}.flex-xl-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-xl-start{justify-content:flex-start !important}.justify-content-xl-end{justify-content:flex-end !important}.justify-content-xl-center{justify-content:center !important}.justify-content-xl-between{justify-content:space-between !important}.justify-content-xl-around{justify-content:space-around !important}.justify-content-xl-evenly{justify-content:space-evenly !important}.align-items-xl-start{align-items:flex-start !important}.align-items-xl-end{align-items:flex-end !important}.align-items-xl-center{align-items:center !important}.align-items-xl-baseline{align-items:baseline !important}.align-items-xl-stretch{align-items:stretch !important}.align-content-xl-start{align-content:flex-start !important}.align-content-xl-end{align-content:flex-end !important}.align-content-xl-center{align-content:center !important}.align-content-xl-between{align-content:space-between !important}.align-content-xl-around{align-content:space-around !important}.align-content-xl-stretch{align-content:stretch !important}.align-self-xl-auto{align-self:auto !important}.align-self-xl-start{align-self:flex-start !important}.align-self-xl-end{align-self:flex-end !important}.align-self-xl-center{align-self:center !important}.align-self-xl-baseline{align-self:baseline !important}.align-self-xl-stretch{align-self:stretch !important}.order-xl-first{order:-1 !important}.order-xl-0{order:0 !important}.order-xl-1{order:1 !important}.order-xl-2{order:2 !important}.order-xl-3{order:3 !important}.order-xl-4{order:4 !important}.order-xl-5{order:5 !important}.order-xl-last{order:6 !important}.m-xl-0{margin:0 !important}.m-xl-1{margin:.25rem !important}.m-xl-2{margin:.5rem !important}.m-xl-3{margin:1rem !important}.m-xl-4{margin:1.5rem !important}.m-xl-5{margin:3rem !important}.m-xl-auto{margin:auto !important}.mx-xl-0{margin-right:0 !important;margin-left:0 !important}.mx-xl-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-xl-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-xl-3{margin-right:1rem !important;margin-left:1rem !important}.mx-xl-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-xl-5{margin-right:3rem !important;margin-left:3rem !important}.mx-xl-auto{margin-right:auto !important;margin-left:auto !important}.my-xl-0{margin-top:0 !important;margin-bottom:0 !important}.my-xl-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-xl-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-xl-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-xl-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-xl-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-xl-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-xl-0{margin-top:0 !important}.mt-xl-1{margin-top:.25rem !important}.mt-xl-2{margin-top:.5rem !important}.mt-xl-3{margin-top:1rem !important}.mt-xl-4{margin-top:1.5rem !important}.mt-xl-5{margin-top:3rem !important}.mt-xl-auto{margin-top:auto !important}.me-xl-0{margin-right:0 !important}.me-xl-1{margin-right:.25rem !important}.me-xl-2{margin-right:.5rem !important}.me-xl-3{margin-right:1rem !important}.me-xl-4{margin-right:1.5rem !important}.me-xl-5{margin-right:3rem !important}.me-xl-auto{margin-right:auto !important}.mb-xl-0{margin-bottom:0 !important}.mb-xl-1{margin-bottom:.25rem !important}.mb-xl-2{margin-bottom:.5rem !important}.mb-xl-3{margin-bottom:1rem !important}.mb-xl-4{margin-bottom:1.5rem !important}.mb-xl-5{margin-bottom:3rem !important}.mb-xl-auto{margin-bottom:auto !important}.ms-xl-0{margin-left:0 !important}.ms-xl-1{margin-left:.25rem !important}.ms-xl-2{margin-left:.5rem !important}.ms-xl-3{margin-left:1rem !important}.ms-xl-4{margin-left:1.5rem !important}.ms-xl-5{margin-left:3rem !important}.ms-xl-auto{margin-left:auto !important}.p-xl-0{padding:0 !important}.p-xl-1{padding:.25rem !important}.p-xl-2{padding:.5rem !important}.p-xl-3{padding:1rem !important}.p-xl-4{padding:1.5rem !important}.p-xl-5{padding:3rem !important}.px-xl-0{padding-right:0 !important;padding-left:0 !important}.px-xl-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-xl-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-xl-3{padding-right:1rem !important;padding-left:1rem !important}.px-xl-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-xl-5{padding-right:3rem !important;padding-left:3rem !important}.py-xl-0{padding-top:0 !important;padding-bottom:0 !important}.py-xl-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-xl-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-xl-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-xl-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-xl-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-xl-0{padding-top:0 !important}.pt-xl-1{padding-top:.25rem !important}.pt-xl-2{padding-top:.5rem !important}.pt-xl-3{padding-top:1rem !important}.pt-xl-4{padding-top:1.5rem !important}.pt-xl-5{padding-top:3rem !important}.pe-xl-0{padding-right:0 !important}.pe-xl-1{padding-right:.25rem !important}.pe-xl-2{padding-right:.5rem !important}.pe-xl-3{padding-right:1rem !important}.pe-xl-4{padding-right:1.5rem !important}.pe-xl-5{padding-right:3rem !important}.pb-xl-0{padding-bottom:0 !important}.pb-xl-1{padding-bottom:.25rem !important}.pb-xl-2{padding-bottom:.5rem !important}.pb-xl-3{padding-bottom:1rem !important}.pb-xl-4{padding-bottom:1.5rem !important}.pb-xl-5{padding-bottom:3rem !important}.ps-xl-0{padding-left:0 !important}.ps-xl-1{padding-left:.25rem !important}.ps-xl-2{padding-left:.5rem !important}.ps-xl-3{padding-left:1rem !important}.ps-xl-4{padding-left:1.5rem !important}.ps-xl-5{padding-left:3rem !important}.gap-xl-0{gap:0 !important}.gap-xl-1{gap:.25rem !important}.gap-xl-2{gap:.5rem !important}.gap-xl-3{gap:1rem !important}.gap-xl-4{gap:1.5rem !important}.gap-xl-5{gap:3rem !important}.row-gap-xl-0{row-gap:0 !important}.row-gap-xl-1{row-gap:.25rem !important}.row-gap-xl-2{row-gap:.5rem !important}.row-gap-xl-3{row-gap:1rem !important}.row-gap-xl-4{row-gap:1.5rem !important}.row-gap-xl-5{row-gap:3rem !important}.column-gap-xl-0{column-gap:0 !important}.column-gap-xl-1{column-gap:.25rem !important}.column-gap-xl-2{column-gap:.5rem !important}.column-gap-xl-3{column-gap:1rem !important}.column-gap-xl-4{column-gap:1.5rem !important}.column-gap-xl-5{column-gap:3rem !important}.text-xl-start{text-align:left !important}.text-xl-end{text-align:right !important}.text-xl-center{text-align:center !important}}@media(min-width: 1400px){.float-xxl-start{float:left !important}.float-xxl-end{float:right !important}.float-xxl-none{float:none !important}.object-fit-xxl-contain{object-fit:contain !important}.object-fit-xxl-cover{object-fit:cover !important}.object-fit-xxl-fill{object-fit:fill !important}.object-fit-xxl-scale{object-fit:scale-down !important}.object-fit-xxl-none{object-fit:none !important}.d-xxl-inline{display:inline !important}.d-xxl-inline-block{display:inline-block !important}.d-xxl-block{display:block !important}.d-xxl-grid{display:grid !important}.d-xxl-inline-grid{display:inline-grid !important}.d-xxl-table{display:table !important}.d-xxl-table-row{display:table-row !important}.d-xxl-table-cell{display:table-cell !important}.d-xxl-flex{display:flex !important}.d-xxl-inline-flex{display:inline-flex !important}.d-xxl-none{display:none !important}.flex-xxl-fill{flex:1 1 auto !important}.flex-xxl-row{flex-direction:row !important}.flex-xxl-column{flex-direction:column !important}.flex-xxl-row-reverse{flex-direction:row-reverse !important}.flex-xxl-column-reverse{flex-direction:column-reverse !important}.flex-xxl-grow-0{flex-grow:0 !important}.flex-xxl-grow-1{flex-grow:1 !important}.flex-xxl-shrink-0{flex-shrink:0 !important}.flex-xxl-shrink-1{flex-shrink:1 !important}.flex-xxl-wrap{flex-wrap:wrap !important}.flex-xxl-nowrap{flex-wrap:nowrap !important}.flex-xxl-wrap-reverse{flex-wrap:wrap-reverse !important}.justify-content-xxl-start{justify-content:flex-start !important}.justify-content-xxl-end{justify-content:flex-end !important}.justify-content-xxl-center{justify-content:center !important}.justify-content-xxl-between{justify-content:space-between !important}.justify-content-xxl-around{justify-content:space-around !important}.justify-content-xxl-evenly{justify-content:space-evenly !important}.align-items-xxl-start{align-items:flex-start !important}.align-items-xxl-end{align-items:flex-end !important}.align-items-xxl-center{align-items:center !important}.align-items-xxl-baseline{align-items:baseline !important}.align-items-xxl-stretch{align-items:stretch !important}.align-content-xxl-start{align-content:flex-start !important}.align-content-xxl-end{align-content:flex-end !important}.align-content-xxl-center{align-content:center !important}.align-content-xxl-between{align-content:space-between !important}.align-content-xxl-around{align-content:space-around !important}.align-content-xxl-stretch{align-content:stretch !important}.align-self-xxl-auto{align-self:auto !important}.align-self-xxl-start{align-self:flex-start !important}.align-self-xxl-end{align-self:flex-end !important}.align-self-xxl-center{align-self:center !important}.align-self-xxl-baseline{align-self:baseline !important}.align-self-xxl-stretch{align-self:stretch !important}.order-xxl-first{order:-1 !important}.order-xxl-0{order:0 !important}.order-xxl-1{order:1 !important}.order-xxl-2{order:2 !important}.order-xxl-3{order:3 !important}.order-xxl-4{order:4 !important}.order-xxl-5{order:5 !important}.order-xxl-last{order:6 !important}.m-xxl-0{margin:0 !important}.m-xxl-1{margin:.25rem !important}.m-xxl-2{margin:.5rem !important}.m-xxl-3{margin:1rem !important}.m-xxl-4{margin:1.5rem !important}.m-xxl-5{margin:3rem !important}.m-xxl-auto{margin:auto !important}.mx-xxl-0{margin-right:0 !important;margin-left:0 !important}.mx-xxl-1{margin-right:.25rem !important;margin-left:.25rem !important}.mx-xxl-2{margin-right:.5rem !important;margin-left:.5rem !important}.mx-xxl-3{margin-right:1rem !important;margin-left:1rem !important}.mx-xxl-4{margin-right:1.5rem !important;margin-left:1.5rem !important}.mx-xxl-5{margin-right:3rem !important;margin-left:3rem !important}.mx-xxl-auto{margin-right:auto !important;margin-left:auto !important}.my-xxl-0{margin-top:0 !important;margin-bottom:0 !important}.my-xxl-1{margin-top:.25rem !important;margin-bottom:.25rem !important}.my-xxl-2{margin-top:.5rem !important;margin-bottom:.5rem !important}.my-xxl-3{margin-top:1rem !important;margin-bottom:1rem !important}.my-xxl-4{margin-top:1.5rem !important;margin-bottom:1.5rem !important}.my-xxl-5{margin-top:3rem !important;margin-bottom:3rem !important}.my-xxl-auto{margin-top:auto !important;margin-bottom:auto !important}.mt-xxl-0{margin-top:0 !important}.mt-xxl-1{margin-top:.25rem !important}.mt-xxl-2{margin-top:.5rem !important}.mt-xxl-3{margin-top:1rem !important}.mt-xxl-4{margin-top:1.5rem !important}.mt-xxl-5{margin-top:3rem !important}.mt-xxl-auto{margin-top:auto !important}.me-xxl-0{margin-right:0 !important}.me-xxl-1{margin-right:.25rem !important}.me-xxl-2{margin-right:.5rem !important}.me-xxl-3{margin-right:1rem !important}.me-xxl-4{margin-right:1.5rem !important}.me-xxl-5{margin-right:3rem !important}.me-xxl-auto{margin-right:auto !important}.mb-xxl-0{margin-bottom:0 !important}.mb-xxl-1{margin-bottom:.25rem !important}.mb-xxl-2{margin-bottom:.5rem !important}.mb-xxl-3{margin-bottom:1rem !important}.mb-xxl-4{margin-bottom:1.5rem !important}.mb-xxl-5{margin-bottom:3rem !important}.mb-xxl-auto{margin-bottom:auto !important}.ms-xxl-0{margin-left:0 !important}.ms-xxl-1{margin-left:.25rem !important}.ms-xxl-2{margin-left:.5rem !important}.ms-xxl-3{margin-left:1rem !important}.ms-xxl-4{margin-left:1.5rem !important}.ms-xxl-5{margin-left:3rem !important}.ms-xxl-auto{margin-left:auto !important}.p-xxl-0{padding:0 !important}.p-xxl-1{padding:.25rem !important}.p-xxl-2{padding:.5rem !important}.p-xxl-3{padding:1rem !important}.p-xxl-4{padding:1.5rem !important}.p-xxl-5{padding:3rem !important}.px-xxl-0{padding-right:0 !important;padding-left:0 !important}.px-xxl-1{padding-right:.25rem !important;padding-left:.25rem !important}.px-xxl-2{padding-right:.5rem !important;padding-left:.5rem !important}.px-xxl-3{padding-right:1rem !important;padding-left:1rem !important}.px-xxl-4{padding-right:1.5rem !important;padding-left:1.5rem !important}.px-xxl-5{padding-right:3rem !important;padding-left:3rem !important}.py-xxl-0{padding-top:0 !important;padding-bottom:0 !important}.py-xxl-1{padding-top:.25rem !important;padding-bottom:.25rem !important}.py-xxl-2{padding-top:.5rem !important;padding-bottom:.5rem !important}.py-xxl-3{padding-top:1rem !important;padding-bottom:1rem !important}.py-xxl-4{padding-top:1.5rem !important;padding-bottom:1.5rem !important}.py-xxl-5{padding-top:3rem !important;padding-bottom:3rem !important}.pt-xxl-0{padding-top:0 !important}.pt-xxl-1{padding-top:.25rem !important}.pt-xxl-2{padding-top:.5rem !important}.pt-xxl-3{padding-top:1rem !important}.pt-xxl-4{padding-top:1.5rem !important}.pt-xxl-5{padding-top:3rem !important}.pe-xxl-0{padding-right:0 !important}.pe-xxl-1{padding-right:.25rem !important}.pe-xxl-2{padding-right:.5rem !important}.pe-xxl-3{padding-right:1rem !important}.pe-xxl-4{padding-right:1.5rem !important}.pe-xxl-5{padding-right:3rem !important}.pb-xxl-0{padding-bottom:0 !important}.pb-xxl-1{padding-bottom:.25rem !important}.pb-xxl-2{padding-bottom:.5rem !important}.pb-xxl-3{padding-bottom:1rem !important}.pb-xxl-4{padding-bottom:1.5rem !important}.pb-xxl-5{padding-bottom:3rem !important}.ps-xxl-0{padding-left:0 !important}.ps-xxl-1{padding-left:.25rem !important}.ps-xxl-2{padding-left:.5rem !important}.ps-xxl-3{padding-left:1rem !important}.ps-xxl-4{padding-left:1.5rem !important}.ps-xxl-5{padding-left:3rem !important}.gap-xxl-0{gap:0 !important}.gap-xxl-1{gap:.25rem !important}.gap-xxl-2{gap:.5rem !important}.gap-xxl-3{gap:1rem !important}.gap-xxl-4{gap:1.5rem !important}.gap-xxl-5{gap:3rem !important}.row-gap-xxl-0{row-gap:0 !important}.row-gap-xxl-1{row-gap:.25rem !important}.row-gap-xxl-2{row-gap:.5rem !important}.row-gap-xxl-3{row-gap:1rem !important}.row-gap-xxl-4{row-gap:1.5rem !important}.row-gap-xxl-5{row-gap:3rem !important}.column-gap-xxl-0{column-gap:0 !important}.column-gap-xxl-1{column-gap:.25rem !important}.column-gap-xxl-2{column-gap:.5rem !important}.column-gap-xxl-3{column-gap:1rem !important}.column-gap-xxl-4{column-gap:1.5rem !important}.column-gap-xxl-5{column-gap:3rem !important}.text-xxl-start{text-align:left !important}.text-xxl-end{text-align:right !important}.text-xxl-center{text-align:center !important}}.bg-default{color:#fff}.bg-primary{color:#fff}.bg-secondary{color:#fff}.bg-success{color:#fff}.bg-info{color:#fff}.bg-warning{color:#fff}.bg-danger{color:#fff}.bg-light{color:#000}.bg-dark{color:#fff}@media(min-width: 1200px){.fs-1{font-size:2rem !important}.fs-2{font-size:1.65rem !important}.fs-3{font-size:1.45rem !important}}@media print{.d-print-inline{display:inline !important}.d-print-inline-block{display:inline-block !important}.d-print-block{display:block !important}.d-print-grid{display:grid !important}.d-print-inline-grid{display:inline-grid !important}.d-print-table{display:table !important}.d-print-table-row{display:table-row !important}.d-print-table-cell{display:table-cell !important}.d-print-flex{display:flex !important}.d-print-inline-flex{display:inline-flex !important}.d-print-none{display:none !important}}:root{--bslib-spacer: 1rem;--bslib-mb-spacer: var(--bslib-spacer, 1rem)}.bslib-mb-spacing{margin-bottom:var(--bslib-mb-spacer)}.bslib-gap-spacing{gap:var(--bslib-mb-spacer)}.bslib-gap-spacing>.bslib-mb-spacing,.bslib-gap-spacing>.form-group,.bslib-gap-spacing>p,.bslib-gap-spacing>pre{margin-bottom:0}.html-fill-container>.html-fill-item.bslib-mb-spacing{margin-bottom:0}.tab-content>.tab-pane.html-fill-container{display:none}.tab-content>.active.html-fill-container{display:flex}.tab-content.html-fill-container{padding:0}.bg-blue{--bslib-color-bg: #325d88;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-blue{--bslib-color-fg: #325d88;color:var(--bslib-color-fg)}.bg-indigo{--bslib-color-bg: #6610f2;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-indigo{--bslib-color-fg: #6610f2;color:var(--bslib-color-fg)}.bg-purple{--bslib-color-bg: #6f42c1;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-purple{--bslib-color-fg: #6f42c1;color:var(--bslib-color-fg)}.bg-pink{--bslib-color-bg: #e83e8c;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-pink{--bslib-color-fg: #e83e8c;color:var(--bslib-color-fg)}.bg-red{--bslib-color-bg: #d9534f;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-red{--bslib-color-fg: #d9534f;color:var(--bslib-color-fg)}.bg-orange{--bslib-color-bg: #f47c3c;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-orange{--bslib-color-fg: #f47c3c;color:var(--bslib-color-fg)}.bg-yellow{--bslib-color-bg: #ffc107;--bslib-color-fg: #000;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-yellow{--bslib-color-fg: #ffc107;color:var(--bslib-color-fg)}.bg-green{--bslib-color-bg: #93c54b;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-green{--bslib-color-fg: #93c54b;color:var(--bslib-color-fg)}.bg-teal{--bslib-color-bg: #20c997;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-teal{--bslib-color-fg: #20c997;color:var(--bslib-color-fg)}.bg-cyan{--bslib-color-bg: #29abe0;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-cyan{--bslib-color-fg: #29abe0;color:var(--bslib-color-fg)}.text-default{--bslib-color-fg: #6c757d}.bg-default{--bslib-color-bg: #6c757d;--bslib-color-fg: #fff}.text-primary{--bslib-color-fg: #325d88}.bg-primary{--bslib-color-bg: #325d88;--bslib-color-fg: #fff}.text-secondary{--bslib-color-fg: #6c757d}.bg-secondary{--bslib-color-bg: #6c757d;--bslib-color-fg: #fff}.text-success{--bslib-color-fg: #93c54b}.bg-success{--bslib-color-bg: #93c54b;--bslib-color-fg: #fff}.text-info{--bslib-color-fg: #29abe0}.bg-info{--bslib-color-bg: #29abe0;--bslib-color-fg: #fff}.text-warning{--bslib-color-fg: #f47c3c}.bg-warning{--bslib-color-bg: #f47c3c;--bslib-color-fg: #fff}.text-danger{--bslib-color-fg: #d9534f}.bg-danger{--bslib-color-bg: #d9534f;--bslib-color-fg: #fff}.text-light{--bslib-color-fg: #f8f5f0}.bg-light{--bslib-color-bg: #f8f5f0;--bslib-color-fg: #000}.text-dark{--bslib-color-fg: #343a40}.bg-dark{--bslib-color-bg: #343a40;--bslib-color-fg: #fff}.bg-gradient-blue-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #473eb2;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #473eb2;color:#fff}.bg-gradient-blue-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4a529f;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4a529f;color:#fff}.bg-gradient-blue-pink{--bslib-color-fg: #fff;--bslib-color-bg: #7b518a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #7b518a;color:#fff}.bg-gradient-blue-red{--bslib-color-fg: #fff;--bslib-color-bg: #755971;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #755971;color:#fff}.bg-gradient-blue-orange{--bslib-color-fg: #fff;--bslib-color-bg: #80696a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #80696a;color:#fff}.bg-gradient-blue-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #848554;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #848554;color:#fff}.bg-gradient-blue-green{--bslib-color-fg: #fff;--bslib-color-bg: #598770;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #598770;color:#fff}.bg-gradient-blue-teal{--bslib-color-fg: #fff;--bslib-color-bg: #2b888e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #2b888e;color:#fff}.bg-gradient-blue-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #2e7cab;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #2e7cab;color:#fff}.bg-gradient-indigo-blue{--bslib-color-fg: #fff;--bslib-color-bg: #512fc8;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #512fc8;color:#fff}.bg-gradient-indigo-purple{--bslib-color-fg: #fff;--bslib-color-bg: #6a24de;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #6a24de;color:#fff}.bg-gradient-indigo-pink{--bslib-color-fg: #fff;--bslib-color-bg: #9a22c9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #9a22c9;color:#fff}.bg-gradient-indigo-red{--bslib-color-fg: #fff;--bslib-color-bg: #942bb1;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #942bb1;color:#fff}.bg-gradient-indigo-orange{--bslib-color-fg: #fff;--bslib-color-bg: #9f3ba9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #9f3ba9;color:#fff}.bg-gradient-indigo-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #a35794;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #a35794;color:#fff}.bg-gradient-indigo-green{--bslib-color-fg: #fff;--bslib-color-bg: #7858af;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #7858af;color:#fff}.bg-gradient-indigo-teal{--bslib-color-fg: #fff;--bslib-color-bg: #4a5ace;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #4a5ace;color:#fff}.bg-gradient-indigo-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #4e4eeb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #4e4eeb;color:#fff}.bg-gradient-purple-blue{--bslib-color-fg: #fff;--bslib-color-bg: #574daa;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #574daa;color:#fff}.bg-gradient-purple-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #6b2ed5;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #6b2ed5;color:#fff}.bg-gradient-purple-pink{--bslib-color-fg: #fff;--bslib-color-bg: #9f40ac;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #9f40ac;color:#fff}.bg-gradient-purple-red{--bslib-color-fg: #fff;--bslib-color-bg: #994993;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #994993;color:#fff}.bg-gradient-purple-orange{--bslib-color-fg: #fff;--bslib-color-bg: #a4598c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #a4598c;color:#fff}.bg-gradient-purple-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #a97577;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #a97577;color:#fff}.bg-gradient-purple-green{--bslib-color-fg: #fff;--bslib-color-bg: #7d7692;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #7d7692;color:#fff}.bg-gradient-purple-teal{--bslib-color-fg: #fff;--bslib-color-bg: #4f78b0;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #4f78b0;color:#fff}.bg-gradient-purple-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #536ccd;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #536ccd;color:#fff}.bg-gradient-pink-blue{--bslib-color-fg: #fff;--bslib-color-bg: #9f4a8a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #9f4a8a;color:#fff}.bg-gradient-pink-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #b42cb5;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #b42cb5;color:#fff}.bg-gradient-pink-purple{--bslib-color-fg: #fff;--bslib-color-bg: #b840a1;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #b840a1;color:#fff}.bg-gradient-pink-red{--bslib-color-fg: #fff;--bslib-color-bg: #e24674;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #e24674;color:#fff}.bg-gradient-pink-orange{--bslib-color-fg: #fff;--bslib-color-bg: #ed576c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #ed576c;color:#fff}.bg-gradient-pink-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #f17257;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #f17257;color:#fff}.bg-gradient-pink-green{--bslib-color-fg: #fff;--bslib-color-bg: #c67472;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #c67472;color:#fff}.bg-gradient-pink-teal{--bslib-color-fg: #fff;--bslib-color-bg: #987690;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #987690;color:#fff}.bg-gradient-pink-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #9c6aae;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #9c6aae;color:#fff}.bg-gradient-red-blue{--bslib-color-fg: #fff;--bslib-color-bg: #965766;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #965766;color:#fff}.bg-gradient-red-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #ab3890;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #ab3890;color:#fff}.bg-gradient-red-purple{--bslib-color-fg: #fff;--bslib-color-bg: #af4c7d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #af4c7d;color:#fff}.bg-gradient-red-pink{--bslib-color-fg: #fff;--bslib-color-bg: #df4b67;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #df4b67;color:#fff}.bg-gradient-red-orange{--bslib-color-fg: #fff;--bslib-color-bg: #e46347;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #e46347;color:#fff}.bg-gradient-red-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #e87f32;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #e87f32;color:#fff}.bg-gradient-red-green{--bslib-color-fg: #fff;--bslib-color-bg: #bd814d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #bd814d;color:#fff}.bg-gradient-red-teal{--bslib-color-fg: #fff;--bslib-color-bg: #8f826c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #8f826c;color:#fff}.bg-gradient-red-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #937689;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #937689;color:#fff}.bg-gradient-orange-blue{--bslib-color-fg: #fff;--bslib-color-bg: #a6705a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #a6705a;color:#fff}.bg-gradient-orange-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #bb5185;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #bb5185;color:#fff}.bg-gradient-orange-purple{--bslib-color-fg: #fff;--bslib-color-bg: #bf6571;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #bf6571;color:#fff}.bg-gradient-orange-pink{--bslib-color-fg: #fff;--bslib-color-bg: #ef635c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #ef635c;color:#fff}.bg-gradient-orange-red{--bslib-color-fg: #fff;--bslib-color-bg: #e96c44;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #e96c44;color:#fff}.bg-gradient-orange-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #f89827;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #f89827;color:#fff}.bg-gradient-orange-green{--bslib-color-fg: #fff;--bslib-color-bg: #cd9942;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #cd9942;color:#fff}.bg-gradient-orange-teal{--bslib-color-fg: #fff;--bslib-color-bg: #9f9b60;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #9f9b60;color:#fff}.bg-gradient-orange-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #a38f7e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #a38f7e;color:#fff}.bg-gradient-yellow-blue{--bslib-color-fg: #fff;--bslib-color-bg: #ad993b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #ad993b;color:#fff}.bg-gradient-yellow-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #c27a65;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #c27a65;color:#fff}.bg-gradient-yellow-purple{--bslib-color-fg: #fff;--bslib-color-bg: #c58e51;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #c58e51;color:#fff}.bg-gradient-yellow-pink{--bslib-color-fg: #fff;--bslib-color-bg: #f68d3c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #f68d3c;color:#fff}.bg-gradient-yellow-red{--bslib-color-fg: #fff;--bslib-color-bg: #f09524;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #f09524;color:#fff}.bg-gradient-yellow-orange{--bslib-color-fg: #000;--bslib-color-bg: #fba51c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #fba51c;color:#000}.bg-gradient-yellow-green{--bslib-color-fg: #000;--bslib-color-bg: #d4c322;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #d4c322;color:#000}.bg-gradient-yellow-teal{--bslib-color-fg: #000;--bslib-color-bg: #a6c441;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #a6c441;color:#000}.bg-gradient-yellow-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #a9b85e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #a9b85e;color:#fff}.bg-gradient-green-blue{--bslib-color-fg: #fff;--bslib-color-bg: #6c9b63;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #6c9b63;color:#fff}.bg-gradient-green-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #817d8e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #817d8e;color:#fff}.bg-gradient-green-purple{--bslib-color-fg: #fff;--bslib-color-bg: #85917a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #85917a;color:#fff}.bg-gradient-green-pink{--bslib-color-fg: #fff;--bslib-color-bg: #b58f65;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #b58f65;color:#fff}.bg-gradient-green-red{--bslib-color-fg: #fff;--bslib-color-bg: #af974d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #af974d;color:#fff}.bg-gradient-green-orange{--bslib-color-fg: #fff;--bslib-color-bg: #baa845;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #baa845;color:#fff}.bg-gradient-green-yellow{--bslib-color-fg: #000;--bslib-color-bg: #bec330;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #bec330;color:#000}.bg-gradient-green-teal{--bslib-color-fg: #fff;--bslib-color-bg: #65c769;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #65c769;color:#fff}.bg-gradient-green-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #69bb87;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #69bb87;color:#fff}.bg-gradient-teal-blue{--bslib-color-fg: #fff;--bslib-color-bg: #279e91;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #279e91;color:#fff}.bg-gradient-teal-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #3c7fbb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #3c7fbb;color:#fff}.bg-gradient-teal-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4093a8;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4093a8;color:#fff}.bg-gradient-teal-pink{--bslib-color-fg: #fff;--bslib-color-bg: #709193;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #709193;color:#fff}.bg-gradient-teal-red{--bslib-color-fg: #fff;--bslib-color-bg: #6a9a7a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #6a9a7a;color:#fff}.bg-gradient-teal-orange{--bslib-color-fg: #fff;--bslib-color-bg: #75aa73;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #75aa73;color:#fff}.bg-gradient-teal-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #79c65d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #79c65d;color:#fff}.bg-gradient-teal-green{--bslib-color-fg: #fff;--bslib-color-bg: #4ec779;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #4ec779;color:#fff}.bg-gradient-teal-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #24bdb4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #24bdb4;color:#fff}.bg-gradient-cyan-blue{--bslib-color-fg: #fff;--bslib-color-bg: #2d8cbd;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #2d8cbd;color:#fff}.bg-gradient-cyan-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #416de7;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #416de7;color:#fff}.bg-gradient-cyan-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4581d4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4581d4;color:#fff}.bg-gradient-cyan-pink{--bslib-color-fg: #fff;--bslib-color-bg: #757fbe;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #757fbe;color:#fff}.bg-gradient-cyan-red{--bslib-color-fg: #fff;--bslib-color-bg: #6f88a6;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #6f88a6;color:#fff}.bg-gradient-cyan-orange{--bslib-color-fg: #fff;--bslib-color-bg: #7a989e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #7a989e;color:#fff}.bg-gradient-cyan-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #7fb489;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #7fb489;color:#fff}.bg-gradient-cyan-green{--bslib-color-fg: #fff;--bslib-color-bg: #53b5a4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #53b5a4;color:#fff}.bg-gradient-cyan-teal{--bslib-color-fg: #fff;--bslib-color-bg: #25b7c3;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #25b7c3;color:#fff}.bg-blue{--bslib-color-bg: #325d88;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-blue{--bslib-color-fg: #325d88;color:var(--bslib-color-fg)}.bg-indigo{--bslib-color-bg: #6610f2;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-indigo{--bslib-color-fg: #6610f2;color:var(--bslib-color-fg)}.bg-purple{--bslib-color-bg: #6f42c1;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-purple{--bslib-color-fg: #6f42c1;color:var(--bslib-color-fg)}.bg-pink{--bslib-color-bg: #e83e8c;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-pink{--bslib-color-fg: #e83e8c;color:var(--bslib-color-fg)}.bg-red{--bslib-color-bg: #d9534f;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-red{--bslib-color-fg: #d9534f;color:var(--bslib-color-fg)}.bg-orange{--bslib-color-bg: #f47c3c;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-orange{--bslib-color-fg: #f47c3c;color:var(--bslib-color-fg)}.bg-yellow{--bslib-color-bg: #ffc107;--bslib-color-fg: #000;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-yellow{--bslib-color-fg: #ffc107;color:var(--bslib-color-fg)}.bg-green{--bslib-color-bg: #93c54b;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-green{--bslib-color-fg: #93c54b;color:var(--bslib-color-fg)}.bg-teal{--bslib-color-bg: #20c997;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-teal{--bslib-color-fg: #20c997;color:var(--bslib-color-fg)}.bg-cyan{--bslib-color-bg: #29abe0;--bslib-color-fg: #fff;background-color:var(--bslib-color-bg);color:var(--bslib-color-fg)}.text-cyan{--bslib-color-fg: #29abe0;color:var(--bslib-color-fg)}.text-default{--bslib-color-fg: #6c757d}.bg-default{--bslib-color-bg: #6c757d;--bslib-color-fg: #fff}.text-primary{--bslib-color-fg: #325d88}.bg-primary{--bslib-color-bg: #325d88;--bslib-color-fg: #fff}.text-secondary{--bslib-color-fg: #6c757d}.bg-secondary{--bslib-color-bg: #6c757d;--bslib-color-fg: #fff}.text-success{--bslib-color-fg: #93c54b}.bg-success{--bslib-color-bg: #93c54b;--bslib-color-fg: #fff}.text-info{--bslib-color-fg: #29abe0}.bg-info{--bslib-color-bg: #29abe0;--bslib-color-fg: #fff}.text-warning{--bslib-color-fg: #f47c3c}.bg-warning{--bslib-color-bg: #f47c3c;--bslib-color-fg: #fff}.text-danger{--bslib-color-fg: #d9534f}.bg-danger{--bslib-color-bg: #d9534f;--bslib-color-fg: #fff}.text-light{--bslib-color-fg: #f8f5f0}.bg-light{--bslib-color-bg: #f8f5f0;--bslib-color-fg: #000}.text-dark{--bslib-color-fg: #343a40}.bg-dark{--bslib-color-bg: #343a40;--bslib-color-fg: #fff}.bg-gradient-blue-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #473eb2;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #473eb2;color:#fff}.bg-gradient-blue-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4a529f;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4a529f;color:#fff}.bg-gradient-blue-pink{--bslib-color-fg: #fff;--bslib-color-bg: #7b518a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #7b518a;color:#fff}.bg-gradient-blue-red{--bslib-color-fg: #fff;--bslib-color-bg: #755971;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #755971;color:#fff}.bg-gradient-blue-orange{--bslib-color-fg: #fff;--bslib-color-bg: #80696a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #80696a;color:#fff}.bg-gradient-blue-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #848554;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #848554;color:#fff}.bg-gradient-blue-green{--bslib-color-fg: #fff;--bslib-color-bg: #598770;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #598770;color:#fff}.bg-gradient-blue-teal{--bslib-color-fg: #fff;--bslib-color-bg: #2b888e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #2b888e;color:#fff}.bg-gradient-blue-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #2e7cab;background:linear-gradient(var(--bg-gradient-deg, 140deg), #325d88 var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #2e7cab;color:#fff}.bg-gradient-indigo-blue{--bslib-color-fg: #fff;--bslib-color-bg: #512fc8;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #512fc8;color:#fff}.bg-gradient-indigo-purple{--bslib-color-fg: #fff;--bslib-color-bg: #6a24de;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #6a24de;color:#fff}.bg-gradient-indigo-pink{--bslib-color-fg: #fff;--bslib-color-bg: #9a22c9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #9a22c9;color:#fff}.bg-gradient-indigo-red{--bslib-color-fg: #fff;--bslib-color-bg: #942bb1;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #942bb1;color:#fff}.bg-gradient-indigo-orange{--bslib-color-fg: #fff;--bslib-color-bg: #9f3ba9;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #9f3ba9;color:#fff}.bg-gradient-indigo-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #a35794;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #a35794;color:#fff}.bg-gradient-indigo-green{--bslib-color-fg: #fff;--bslib-color-bg: #7858af;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #7858af;color:#fff}.bg-gradient-indigo-teal{--bslib-color-fg: #fff;--bslib-color-bg: #4a5ace;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #4a5ace;color:#fff}.bg-gradient-indigo-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #4e4eeb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6610f2 var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #4e4eeb;color:#fff}.bg-gradient-purple-blue{--bslib-color-fg: #fff;--bslib-color-bg: #574daa;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #574daa;color:#fff}.bg-gradient-purple-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #6b2ed5;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #6b2ed5;color:#fff}.bg-gradient-purple-pink{--bslib-color-fg: #fff;--bslib-color-bg: #9f40ac;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #9f40ac;color:#fff}.bg-gradient-purple-red{--bslib-color-fg: #fff;--bslib-color-bg: #994993;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #994993;color:#fff}.bg-gradient-purple-orange{--bslib-color-fg: #fff;--bslib-color-bg: #a4598c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #a4598c;color:#fff}.bg-gradient-purple-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #a97577;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #a97577;color:#fff}.bg-gradient-purple-green{--bslib-color-fg: #fff;--bslib-color-bg: #7d7692;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #7d7692;color:#fff}.bg-gradient-purple-teal{--bslib-color-fg: #fff;--bslib-color-bg: #4f78b0;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #4f78b0;color:#fff}.bg-gradient-purple-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #536ccd;background:linear-gradient(var(--bg-gradient-deg, 140deg), #6f42c1 var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #536ccd;color:#fff}.bg-gradient-pink-blue{--bslib-color-fg: #fff;--bslib-color-bg: #9f4a8a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #9f4a8a;color:#fff}.bg-gradient-pink-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #b42cb5;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #b42cb5;color:#fff}.bg-gradient-pink-purple{--bslib-color-fg: #fff;--bslib-color-bg: #b840a1;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #b840a1;color:#fff}.bg-gradient-pink-red{--bslib-color-fg: #fff;--bslib-color-bg: #e24674;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #e24674;color:#fff}.bg-gradient-pink-orange{--bslib-color-fg: #fff;--bslib-color-bg: #ed576c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #ed576c;color:#fff}.bg-gradient-pink-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #f17257;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #f17257;color:#fff}.bg-gradient-pink-green{--bslib-color-fg: #fff;--bslib-color-bg: #c67472;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #c67472;color:#fff}.bg-gradient-pink-teal{--bslib-color-fg: #fff;--bslib-color-bg: #987690;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #987690;color:#fff}.bg-gradient-pink-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #9c6aae;background:linear-gradient(var(--bg-gradient-deg, 140deg), #e83e8c var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #9c6aae;color:#fff}.bg-gradient-red-blue{--bslib-color-fg: #fff;--bslib-color-bg: #965766;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #965766;color:#fff}.bg-gradient-red-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #ab3890;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #ab3890;color:#fff}.bg-gradient-red-purple{--bslib-color-fg: #fff;--bslib-color-bg: #af4c7d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #af4c7d;color:#fff}.bg-gradient-red-pink{--bslib-color-fg: #fff;--bslib-color-bg: #df4b67;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #df4b67;color:#fff}.bg-gradient-red-orange{--bslib-color-fg: #fff;--bslib-color-bg: #e46347;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #e46347;color:#fff}.bg-gradient-red-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #e87f32;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #e87f32;color:#fff}.bg-gradient-red-green{--bslib-color-fg: #fff;--bslib-color-bg: #bd814d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #bd814d;color:#fff}.bg-gradient-red-teal{--bslib-color-fg: #fff;--bslib-color-bg: #8f826c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #8f826c;color:#fff}.bg-gradient-red-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #937689;background:linear-gradient(var(--bg-gradient-deg, 140deg), #d9534f var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #937689;color:#fff}.bg-gradient-orange-blue{--bslib-color-fg: #fff;--bslib-color-bg: #a6705a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #a6705a;color:#fff}.bg-gradient-orange-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #bb5185;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #bb5185;color:#fff}.bg-gradient-orange-purple{--bslib-color-fg: #fff;--bslib-color-bg: #bf6571;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #bf6571;color:#fff}.bg-gradient-orange-pink{--bslib-color-fg: #fff;--bslib-color-bg: #ef635c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #ef635c;color:#fff}.bg-gradient-orange-red{--bslib-color-fg: #fff;--bslib-color-bg: #e96c44;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #e96c44;color:#fff}.bg-gradient-orange-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #f89827;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #f89827;color:#fff}.bg-gradient-orange-green{--bslib-color-fg: #fff;--bslib-color-bg: #cd9942;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #cd9942;color:#fff}.bg-gradient-orange-teal{--bslib-color-fg: #fff;--bslib-color-bg: #9f9b60;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #9f9b60;color:#fff}.bg-gradient-orange-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #a38f7e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #f47c3c var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #a38f7e;color:#fff}.bg-gradient-yellow-blue{--bslib-color-fg: #fff;--bslib-color-bg: #ad993b;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #ad993b;color:#fff}.bg-gradient-yellow-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #c27a65;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #c27a65;color:#fff}.bg-gradient-yellow-purple{--bslib-color-fg: #fff;--bslib-color-bg: #c58e51;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #c58e51;color:#fff}.bg-gradient-yellow-pink{--bslib-color-fg: #fff;--bslib-color-bg: #f68d3c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #f68d3c;color:#fff}.bg-gradient-yellow-red{--bslib-color-fg: #fff;--bslib-color-bg: #f09524;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #f09524;color:#fff}.bg-gradient-yellow-orange{--bslib-color-fg: #000;--bslib-color-bg: #fba51c;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #fba51c;color:#000}.bg-gradient-yellow-green{--bslib-color-fg: #000;--bslib-color-bg: #d4c322;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #d4c322;color:#000}.bg-gradient-yellow-teal{--bslib-color-fg: #000;--bslib-color-bg: #a6c441;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #a6c441;color:#000}.bg-gradient-yellow-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #a9b85e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #ffc107 var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #a9b85e;color:#fff}.bg-gradient-green-blue{--bslib-color-fg: #fff;--bslib-color-bg: #6c9b63;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #6c9b63;color:#fff}.bg-gradient-green-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #817d8e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #817d8e;color:#fff}.bg-gradient-green-purple{--bslib-color-fg: #fff;--bslib-color-bg: #85917a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #85917a;color:#fff}.bg-gradient-green-pink{--bslib-color-fg: #fff;--bslib-color-bg: #b58f65;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #b58f65;color:#fff}.bg-gradient-green-red{--bslib-color-fg: #fff;--bslib-color-bg: #af974d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #af974d;color:#fff}.bg-gradient-green-orange{--bslib-color-fg: #fff;--bslib-color-bg: #baa845;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #baa845;color:#fff}.bg-gradient-green-yellow{--bslib-color-fg: #000;--bslib-color-bg: #bec330;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #bec330;color:#000}.bg-gradient-green-teal{--bslib-color-fg: #fff;--bslib-color-bg: #65c769;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #65c769;color:#fff}.bg-gradient-green-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #69bb87;background:linear-gradient(var(--bg-gradient-deg, 140deg), #93c54b var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #69bb87;color:#fff}.bg-gradient-teal-blue{--bslib-color-fg: #fff;--bslib-color-bg: #279e91;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #279e91;color:#fff}.bg-gradient-teal-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #3c7fbb;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #3c7fbb;color:#fff}.bg-gradient-teal-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4093a8;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4093a8;color:#fff}.bg-gradient-teal-pink{--bslib-color-fg: #fff;--bslib-color-bg: #709193;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #709193;color:#fff}.bg-gradient-teal-red{--bslib-color-fg: #fff;--bslib-color-bg: #6a9a7a;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #6a9a7a;color:#fff}.bg-gradient-teal-orange{--bslib-color-fg: #fff;--bslib-color-bg: #75aa73;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #75aa73;color:#fff}.bg-gradient-teal-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #79c65d;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #79c65d;color:#fff}.bg-gradient-teal-green{--bslib-color-fg: #fff;--bslib-color-bg: #4ec779;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #4ec779;color:#fff}.bg-gradient-teal-cyan{--bslib-color-fg: #fff;--bslib-color-bg: #24bdb4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #20c997 var(--bg-gradient-start, 36%), #29abe0 var(--bg-gradient-end, 180%)) #24bdb4;color:#fff}.bg-gradient-cyan-blue{--bslib-color-fg: #fff;--bslib-color-bg: #2d8cbd;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #325d88 var(--bg-gradient-end, 180%)) #2d8cbd;color:#fff}.bg-gradient-cyan-indigo{--bslib-color-fg: #fff;--bslib-color-bg: #416de7;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #6610f2 var(--bg-gradient-end, 180%)) #416de7;color:#fff}.bg-gradient-cyan-purple{--bslib-color-fg: #fff;--bslib-color-bg: #4581d4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #6f42c1 var(--bg-gradient-end, 180%)) #4581d4;color:#fff}.bg-gradient-cyan-pink{--bslib-color-fg: #fff;--bslib-color-bg: #757fbe;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #e83e8c var(--bg-gradient-end, 180%)) #757fbe;color:#fff}.bg-gradient-cyan-red{--bslib-color-fg: #fff;--bslib-color-bg: #6f88a6;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #d9534f var(--bg-gradient-end, 180%)) #6f88a6;color:#fff}.bg-gradient-cyan-orange{--bslib-color-fg: #fff;--bslib-color-bg: #7a989e;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #f47c3c var(--bg-gradient-end, 180%)) #7a989e;color:#fff}.bg-gradient-cyan-yellow{--bslib-color-fg: #fff;--bslib-color-bg: #7fb489;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #ffc107 var(--bg-gradient-end, 180%)) #7fb489;color:#fff}.bg-gradient-cyan-green{--bslib-color-fg: #fff;--bslib-color-bg: #53b5a4;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #93c54b var(--bg-gradient-end, 180%)) #53b5a4;color:#fff}.bg-gradient-cyan-teal{--bslib-color-fg: #fff;--bslib-color-bg: #25b7c3;background:linear-gradient(var(--bg-gradient-deg, 140deg), #29abe0 var(--bg-gradient-start, 36%), #20c997 var(--bg-gradient-end, 180%)) #25b7c3;color:#fff}:root{--bslib-spacer: 1rem;--bslib-mb-spacer: var(--bslib-spacer, 1rem)}.bslib-mb-spacing{margin-bottom:var(--bslib-mb-spacer)}.bslib-gap-spacing{gap:var(--bslib-mb-spacer)}.bslib-gap-spacing>.bslib-mb-spacing,.bslib-gap-spacing>.form-group,.bslib-gap-spacing>p,.bslib-gap-spacing>pre{margin-bottom:0}.html-fill-container>.html-fill-item.bslib-mb-spacing{margin-bottom:0}.tab-content>.tab-pane.html-fill-container{display:none}.tab-content>.active.html-fill-container{display:flex}.tab-content.html-fill-container{padding:0}.accordion .accordion-header{font-size:calc(1.29rem + 0.48vw);margin-top:0;margin-bottom:.5rem;font-weight:400;line-height:1.2;color:var(--bs-heading-color);margin-bottom:0}@media(min-width: 1200px){.accordion .accordion-header{font-size:1.65rem}}.accordion .accordion-icon:not(:empty){margin-right:.75rem;display:flex}.accordion .accordion-button:not(.collapsed){box-shadow:none}.accordion .accordion-button:not(.collapsed):focus{box-shadow:var(--bs-accordion-btn-focus-box-shadow)}.bslib-card{overflow:auto}.bslib-card .card-body+.card-body{padding-top:0}.bslib-card .card-body{overflow:auto}.bslib-card .card-body p{margin-top:0}.bslib-card .card-body p:last-child{margin-bottom:0}.bslib-card .card-body{max-height:var(--bslib-card-body-max-height, none)}.bslib-card[data-full-screen=true]>.card-body{max-height:var(--bslib-card-body-max-height-full-screen, none)}.bslib-card .card-header .form-group{margin-bottom:0}.bslib-card .card-header .selectize-control{margin-bottom:0}.bslib-card .card-header .selectize-control .item{margin-right:1.15rem}.bslib-card .card-footer{margin-top:auto}.bslib-card .bslib-navs-card-title{display:flex;flex-wrap:wrap;justify-content:space-between;align-items:center}.bslib-card .bslib-navs-card-title .nav{margin-left:auto}.bslib-card .bslib-sidebar-layout:not([data-bslib-sidebar-border=true]){border:none}.bslib-card .bslib-sidebar-layout:not([data-bslib-sidebar-border-radius=true]){border-top-left-radius:0;border-top-right-radius:0}[data-full-screen=true]{position:fixed;inset:3.5rem 1rem 1rem;height:auto !important;max-height:none !important;width:auto !important;z-index:1070}.bslib-full-screen-enter{display:none;position:absolute;bottom:var(--bslib-full-screen-enter-bottom, 0.2rem);right:var(--bslib-full-screen-enter-right, 0);top:var(--bslib-full-screen-enter-top);left:var(--bslib-full-screen-enter-left);color:var(--bslib-color-fg, var(--bs-card-color));background-color:var(--bslib-color-bg, var(--bs-card-bg, var(--bs-body-bg)));border:var(--bs-card-border-width) solid var(--bslib-color-fg, var(--bs-card-border-color));box-shadow:0 2px 4px rgba(0,0,0,.15);margin:.2rem .4rem;padding:.55rem !important;font-size:.8rem;cursor:pointer;opacity:.7;z-index:1070}.bslib-full-screen-enter:hover{opacity:1}.card[data-full-screen=false]:hover>*>.bslib-full-screen-enter{display:block}.bslib-has-full-screen .card:hover>*>.bslib-full-screen-enter{display:none}@media(max-width: 575.98px){.bslib-full-screen-enter{display:none !important}}.bslib-full-screen-exit{position:relative;top:1.35rem;font-size:.9rem;cursor:pointer;text-decoration:none;display:flex;float:right;margin-right:2.15rem;align-items:center;color:rgba(var(--bs-body-bg-rgb), 0.8)}.bslib-full-screen-exit:hover{color:rgba(var(--bs-body-bg-rgb), 1)}.bslib-full-screen-exit svg{margin-left:.5rem;font-size:1.5rem}#bslib-full-screen-overlay{position:fixed;inset:0;background-color:rgba(var(--bs-body-color-rgb), 0.6);backdrop-filter:blur(2px);-webkit-backdrop-filter:blur(2px);z-index:1069;animation:bslib-full-screen-overlay-enter 400ms cubic-bezier(0.6, 0.02, 0.65, 1) forwards}@keyframes bslib-full-screen-overlay-enter{0%{opacity:0}100%{opacity:1}}.bslib-grid{display:grid !important;gap:var(--bslib-spacer, 1rem);height:var(--bslib-grid-height)}.bslib-grid.grid{grid-template-columns:repeat(var(--bs-columns, 12), minmax(0, 1fr));grid-template-rows:unset;grid-auto-rows:var(--bslib-grid--row-heights);--bslib-grid--row-heights--xs: unset;--bslib-grid--row-heights--sm: unset;--bslib-grid--row-heights--md: unset;--bslib-grid--row-heights--lg: unset;--bslib-grid--row-heights--xl: unset;--bslib-grid--row-heights--xxl: unset}.bslib-grid.grid.bslib-grid--row-heights--xs{--bslib-grid--row-heights: var(--bslib-grid--row-heights--xs)}@media(min-width: 576px){.bslib-grid.grid.bslib-grid--row-heights--sm{--bslib-grid--row-heights: var(--bslib-grid--row-heights--sm)}}@media(min-width: 768px){.bslib-grid.grid.bslib-grid--row-heights--md{--bslib-grid--row-heights: var(--bslib-grid--row-heights--md)}}@media(min-width: 992px){.bslib-grid.grid.bslib-grid--row-heights--lg{--bslib-grid--row-heights: var(--bslib-grid--row-heights--lg)}}@media(min-width: 1200px){.bslib-grid.grid.bslib-grid--row-heights--xl{--bslib-grid--row-heights: var(--bslib-grid--row-heights--xl)}}@media(min-width: 1400px){.bslib-grid.grid.bslib-grid--row-heights--xxl{--bslib-grid--row-heights: var(--bslib-grid--row-heights--xxl)}}.bslib-grid>*>.shiny-input-container{width:100%}.bslib-grid-item{grid-column:auto/span 1}@media(max-width: 767.98px){.bslib-grid-item{grid-column:1/-1}}@media(max-width: 575.98px){.bslib-grid{grid-template-columns:1fr !important;height:var(--bslib-grid-height-mobile)}.bslib-grid.grid{height:unset !important;grid-auto-rows:var(--bslib-grid--row-heights--xs, auto)}}@media(min-width: 576px){.nav:not(.nav-hidden){display:flex !important;display:-webkit-flex !important}.nav:not(.nav-hidden):not(.nav-stacked):not(.flex-column){float:none !important}.nav:not(.nav-hidden):not(.nav-stacked):not(.flex-column)>.bslib-nav-spacer{margin-left:auto !important}.nav:not(.nav-hidden):not(.nav-stacked):not(.flex-column)>.form-inline{margin-top:auto;margin-bottom:auto}.nav:not(.nav-hidden).nav-stacked{flex-direction:column;-webkit-flex-direction:column;height:100%}.nav:not(.nav-hidden).nav-stacked>.bslib-nav-spacer{margin-top:auto !important}}html{height:100%}.bslib-page-fill{width:100%;height:100%;margin:0;padding:var(--bslib-spacer, 1rem);gap:var(--bslib-spacer, 1rem)}@media(max-width: 575.98px){.bslib-page-fill{height:var(--bslib-page-fill-mobile-height, auto)}}.navbar+.container-fluid:has(>.tab-content>.tab-pane.active.html-fill-container),.navbar+.container-sm:has(>.tab-content>.tab-pane.active.html-fill-container),.navbar+.container-md:has(>.tab-content>.tab-pane.active.html-fill-container),.navbar+.container-lg:has(>.tab-content>.tab-pane.active.html-fill-container),.navbar+.container-xl:has(>.tab-content>.tab-pane.active.html-fill-container),.navbar+.container-xxl:has(>.tab-content>.tab-pane.active.html-fill-container){padding-left:0;padding-right:0}.navbar+.container-fluid>.tab-content>.tab-pane.active.html-fill-container,.navbar+.container-sm>.tab-content>.tab-pane.active.html-fill-container,.navbar+.container-md>.tab-content>.tab-pane.active.html-fill-container,.navbar+.container-lg>.tab-content>.tab-pane.active.html-fill-container,.navbar+.container-xl>.tab-content>.tab-pane.active.html-fill-container,.navbar+.container-xxl>.tab-content>.tab-pane.active.html-fill-container{padding:var(--bslib-spacer, 1rem);gap:var(--bslib-spacer, 1rem)}.navbar+.container-fluid>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child),.navbar+.container-sm>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child),.navbar+.container-md>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child),.navbar+.container-lg>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child),.navbar+.container-xl>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child),.navbar+.container-xxl>.tab-content>.tab-pane.active.html-fill-container:has(>.bslib-sidebar-layout:only-child){padding:0}.navbar+.container-fluid>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]),.navbar+.container-sm>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]),.navbar+.container-md>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]),.navbar+.container-lg>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]),.navbar+.container-xl>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]),.navbar+.container-xxl>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border=true]){border-left:none;border-right:none;border-bottom:none}.navbar+.container-fluid>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]),.navbar+.container-sm>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]),.navbar+.container-md>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]),.navbar+.container-lg>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]),.navbar+.container-xl>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]),.navbar+.container-xxl>.tab-content>.tab-pane.active.html-fill-container>.bslib-sidebar-layout:only-child:not([data-bslib-sidebar-border-radius=true]){border-radius:0}.navbar+div>.bslib-sidebar-layout{border-top:var(--bslib-sidebar-border)}:root{--bslib-page-sidebar-title-bg: #325d88;--bslib-page-sidebar-title-color: #fff}.bslib-page-title{background-color:var(--bslib-page-sidebar-title-bg);color:var(--bslib-page-sidebar-title-color);font-size:1.25rem;font-weight:300;padding:var(--bslib-spacer, 1rem);padding-left:1.5rem;margin-bottom:0;border-bottom:1px solid #dee2e6}.bslib-sidebar-layout{--bslib-sidebar-transition-duration: 500ms;--bslib-sidebar-transition-easing-x: cubic-bezier(0.8, 0.78, 0.22, 1.07);--bslib-sidebar-border: var(--bs-card-border-width, 1px) solid var(--bs-card-border-color, rgba(222, 226, 230, 0.75));--bslib-sidebar-border-radius: var(--bs-border-radius);--bslib-sidebar-vert-border: var(--bs-card-border-width, 1px) solid var(--bs-card-border-color, rgba(222, 226, 230, 0.75));--bslib-sidebar-bg: rgba(var(--bs-emphasis-color-rgb, 0, 0, 0), 0.05);--bslib-sidebar-fg: var(--bs-emphasis-color, black);--bslib-sidebar-main-fg: var(--bs-card-color, var(--bs-body-color));--bslib-sidebar-main-bg: var(--bs-card-bg, var(--bs-body-bg));--bslib-sidebar-toggle-bg: rgba(var(--bs-emphasis-color-rgb, 0, 0, 0), 0.1);--bslib-sidebar-padding: calc(var(--bslib-spacer) * 1.5);--bslib-sidebar-icon-size: var(--bslib-spacer, 1rem);--bslib-sidebar-icon-button-size: calc(var(--bslib-sidebar-icon-size, 1rem) * 2);--bslib-sidebar-padding-icon: calc(var(--bslib-sidebar-icon-button-size, 2rem) * 1.5);--bslib-collapse-toggle-border-radius: var(--bs-border-radius, 0.25rem);--bslib-collapse-toggle-transform: 0deg;--bslib-sidebar-toggle-transition-easing: cubic-bezier(1, 0, 0, 1);--bslib-collapse-toggle-right-transform: 180deg;--bslib-sidebar-column-main: minmax(0, 1fr);display:grid !important;grid-template-columns:min(100% - var(--bslib-sidebar-icon-size),var(--bslib-sidebar-width, 250px)) var(--bslib-sidebar-column-main);position:relative;transition:grid-template-columns ease-in-out var(--bslib-sidebar-transition-duration);border:var(--bslib-sidebar-border);border-radius:var(--bslib-sidebar-border-radius)}@media(prefers-reduced-motion: reduce){.bslib-sidebar-layout{transition:none}}.bslib-sidebar-layout[data-bslib-sidebar-border=false]{border:none}.bslib-sidebar-layout[data-bslib-sidebar-border-radius=false]{border-radius:initial}.bslib-sidebar-layout>.main,.bslib-sidebar-layout>.sidebar{grid-row:1/2;border-radius:inherit;overflow:auto}.bslib-sidebar-layout>.main{grid-column:2/3;border-top-left-radius:0;border-bottom-left-radius:0;padding:var(--bslib-sidebar-padding);transition:padding var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration);color:var(--bslib-sidebar-main-fg);background-color:var(--bslib-sidebar-main-bg)}.bslib-sidebar-layout>.sidebar{grid-column:1/2;width:100%;height:100%;border-right:var(--bslib-sidebar-vert-border);border-top-right-radius:0;border-bottom-right-radius:0;color:var(--bslib-sidebar-fg);background-color:var(--bslib-sidebar-bg);backdrop-filter:blur(5px)}.bslib-sidebar-layout>.sidebar>.sidebar-content{display:flex;flex-direction:column;gap:var(--bslib-spacer, 1rem);padding:var(--bslib-sidebar-padding);padding-top:var(--bslib-sidebar-padding-icon)}.bslib-sidebar-layout>.sidebar>.sidebar-content>:last-child:not(.sidebar-title){margin-bottom:0}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion{margin-left:calc(-1*var(--bslib-sidebar-padding));margin-right:calc(-1*var(--bslib-sidebar-padding))}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion:last-child{margin-bottom:calc(-1*var(--bslib-sidebar-padding))}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion:not(:last-child){margin-bottom:1rem}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion .accordion-body{display:flex;flex-direction:column}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion:not(:first-child) .accordion-item:first-child{border-top:var(--bs-accordion-border-width) solid var(--bs-accordion-border-color)}.bslib-sidebar-layout>.sidebar>.sidebar-content>.accordion:not(:last-child) .accordion-item:last-child{border-bottom:var(--bs-accordion-border-width) solid var(--bs-accordion-border-color)}.bslib-sidebar-layout>.sidebar>.sidebar-content.has-accordion>.sidebar-title{border-bottom:none;padding-bottom:0}.bslib-sidebar-layout>.sidebar .shiny-input-container{width:100%}.bslib-sidebar-layout[data-bslib-sidebar-open=always]>.sidebar>.sidebar-content{padding-top:var(--bslib-sidebar-padding)}.bslib-sidebar-layout>.collapse-toggle{grid-row:1/2;grid-column:1/2;display:inline-flex;align-items:center;position:absolute;right:calc(var(--bslib-sidebar-icon-size));top:calc(var(--bslib-sidebar-icon-size, 1rem)/2);border:none;border-radius:var(--bslib-collapse-toggle-border-radius);height:var(--bslib-sidebar-icon-button-size, 2rem);width:var(--bslib-sidebar-icon-button-size, 2rem);display:flex;align-items:center;justify-content:center;padding:0;color:var(--bslib-sidebar-fg);background-color:unset;transition:color var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration),top var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration),right var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration),left var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration)}.bslib-sidebar-layout>.collapse-toggle:hover{background-color:var(--bslib-sidebar-toggle-bg)}.bslib-sidebar-layout>.collapse-toggle>.collapse-icon{opacity:.8;width:var(--bslib-sidebar-icon-size);height:var(--bslib-sidebar-icon-size);transform:rotateY(var(--bslib-collapse-toggle-transform));transition:transform var(--bslib-sidebar-toggle-transition-easing) var(--bslib-sidebar-transition-duration)}.bslib-sidebar-layout>.collapse-toggle:hover>.collapse-icon{opacity:1}.bslib-sidebar-layout .sidebar-title{font-size:1.25rem;line-height:1.25;margin-top:0;margin-bottom:1rem;padding-bottom:1rem;border-bottom:var(--bslib-sidebar-border)}.bslib-sidebar-layout.sidebar-right{grid-template-columns:var(--bslib-sidebar-column-main) min(100% - var(--bslib-sidebar-icon-size),var(--bslib-sidebar-width, 250px))}.bslib-sidebar-layout.sidebar-right>.main{grid-column:1/2;border-top-right-radius:0;border-bottom-right-radius:0;border-top-left-radius:inherit;border-bottom-left-radius:inherit}.bslib-sidebar-layout.sidebar-right>.sidebar{grid-column:2/3;border-right:none;border-left:var(--bslib-sidebar-vert-border);border-top-left-radius:0;border-bottom-left-radius:0}.bslib-sidebar-layout.sidebar-right>.collapse-toggle{grid-column:2/3;left:var(--bslib-sidebar-icon-size);right:unset;border:var(--bslib-collapse-toggle-border)}.bslib-sidebar-layout.sidebar-right>.collapse-toggle>.collapse-icon{transform:rotateY(var(--bslib-collapse-toggle-right-transform))}.bslib-sidebar-layout.sidebar-collapsed{--bslib-collapse-toggle-transform: 180deg;--bslib-collapse-toggle-right-transform: 0deg;--bslib-sidebar-vert-border: none;grid-template-columns:0 minmax(0, 1fr)}.bslib-sidebar-layout.sidebar-collapsed.sidebar-right{grid-template-columns:minmax(0, 1fr) 0}.bslib-sidebar-layout.sidebar-collapsed:not(.transitioning)>.sidebar>*{display:none}.bslib-sidebar-layout.sidebar-collapsed>.main{border-radius:inherit}.bslib-sidebar-layout.sidebar-collapsed:not(.sidebar-right)>.main{padding-left:var(--bslib-sidebar-padding-icon)}.bslib-sidebar-layout.sidebar-collapsed.sidebar-right>.main{padding-right:var(--bslib-sidebar-padding-icon)}.bslib-sidebar-layout.sidebar-collapsed>.collapse-toggle{color:var(--bslib-sidebar-main-fg);top:calc(var(--bslib-sidebar-overlap-counter, 0)*(var(--bslib-sidebar-icon-size) + var(--bslib-sidebar-padding)) + var(--bslib-sidebar-icon-size, 1rem)/2);right:calc(-2.5*var(--bslib-sidebar-icon-size) - var(--bs-card-border-width, 1px))}.bslib-sidebar-layout.sidebar-collapsed.sidebar-right>.collapse-toggle{left:calc(-2.5*var(--bslib-sidebar-icon-size) - var(--bs-card-border-width, 1px));right:unset}@media(min-width: 576px){.bslib-sidebar-layout.transitioning>.sidebar>.sidebar-content{display:none}}@media(max-width: 575.98px){.bslib-sidebar-layout[data-bslib-sidebar-open=desktop]{--bslib-sidebar-js-init-collapsed: true}.bslib-sidebar-layout>.sidebar,.bslib-sidebar-layout.sidebar-right>.sidebar{border:none}.bslib-sidebar-layout>.main,.bslib-sidebar-layout.sidebar-right>.main{grid-column:1/3}.bslib-sidebar-layout[data-bslib-sidebar-open=always]{display:block !important}.bslib-sidebar-layout[data-bslib-sidebar-open=always]>.sidebar{max-height:var(--bslib-sidebar-max-height-mobile);overflow-y:auto;border-top:var(--bslib-sidebar-vert-border)}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]){grid-template-columns:100% 0}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]):not(.sidebar-collapsed)>.sidebar{z-index:1}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]):not(.sidebar-collapsed)>.collapse-toggle{z-index:1}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]).sidebar-right{grid-template-columns:0 100%}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]).sidebar-collapsed{grid-template-columns:0 100%}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]).sidebar-collapsed.sidebar-right{grid-template-columns:100% 0}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]):not(.sidebar-right)>.main{padding-left:var(--bslib-sidebar-padding-icon)}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]).sidebar-right>.main{padding-right:var(--bslib-sidebar-padding-icon)}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always])>.main{opacity:0;transition:opacity var(--bslib-sidebar-transition-easing-x) var(--bslib-sidebar-transition-duration)}.bslib-sidebar-layout:not([data-bslib-sidebar-open=always]).sidebar-collapsed>.main{opacity:1}}:root{--bslib-value-box-shadow: none;--bslib-value-box-border-width-auto-yes: var(--bslib-value-box-border-width-baseline);--bslib-value-box-border-width-auto-no: 0;--bslib-value-box-border-width-baseline: 1px}.bslib-value-box{border-width:var(--bslib-value-box-border-width-auto-no, var(--bslib-value-box-border-width-baseline));container-name:bslib-value-box;container-type:inline-size}.bslib-value-box.card{box-shadow:var(--bslib-value-box-shadow)}.bslib-value-box.border-auto{border-width:var(--bslib-value-box-border-width-auto-yes, var(--bslib-value-box-border-width-baseline))}.bslib-value-box.default{--bslib-value-box-bg-default: var(--bs-card-bg, #fff);--bslib-value-box-border-color-default: var(--bs-card-border-color, rgba(222, 226, 230, 0.75));color:var(--bslib-value-box-color);background-color:var(--bslib-value-box-bg, var(--bslib-value-box-bg-default));border-color:var(--bslib-value-box-border-color, var(--bslib-value-box-border-color-default))}.bslib-value-box .value-box-grid{display:grid;grid-template-areas:"left right";align-items:center;overflow:hidden}.bslib-value-box .value-box-showcase{height:100%;max-height:var(---bslib-value-box-showcase-max-h, 100%)}.bslib-value-box .value-box-showcase,.bslib-value-box .value-box-showcase>.html-fill-item{width:100%}.bslib-value-box[data-full-screen=true] .value-box-showcase{max-height:var(---bslib-value-box-showcase-max-h-fs, 100%)}@media screen and (min-width: 575.98px){@container bslib-value-box (max-width: 300px){.bslib-value-box:not(.showcase-bottom) .value-box-grid{grid-template-columns:1fr !important;grid-template-rows:auto auto;grid-template-areas:"top" "bottom"}.bslib-value-box:not(.showcase-bottom) .value-box-grid .value-box-showcase{grid-area:top !important}.bslib-value-box:not(.showcase-bottom) .value-box-grid .value-box-area{grid-area:bottom !important;justify-content:end}}}.bslib-value-box .value-box-area{justify-content:center;padding:1.5rem 1rem;font-size:.9rem;font-weight:500}.bslib-value-box .value-box-area *{margin-bottom:0;margin-top:0}.bslib-value-box .value-box-title{font-size:1rem;margin-top:0;margin-bottom:.5rem;font-weight:400;line-height:1.2}.bslib-value-box .value-box-title:empty::after{content:" "}.bslib-value-box .value-box-value{font-size:calc(1.29rem + 0.48vw);margin-top:0;margin-bottom:.5rem;font-weight:400;line-height:1.2}@media(min-width: 1200px){.bslib-value-box .value-box-value{font-size:1.65rem}}.bslib-value-box .value-box-value:empty::after{content:" "}.bslib-value-box .value-box-showcase{align-items:center;justify-content:center;margin-top:auto;margin-bottom:auto;padding:1rem}.bslib-value-box .value-box-showcase .bi,.bslib-value-box .value-box-showcase .fa,.bslib-value-box .value-box-showcase .fab,.bslib-value-box .value-box-showcase .fas,.bslib-value-box .value-box-showcase .far{opacity:.85;min-width:50px;max-width:125%}.bslib-value-box .value-box-showcase .bi,.bslib-value-box .value-box-showcase .fa,.bslib-value-box .value-box-showcase .fab,.bslib-value-box .value-box-showcase .fas,.bslib-value-box .value-box-showcase .far{font-size:4rem}.bslib-value-box.showcase-top-right .value-box-grid{grid-template-columns:1fr var(---bslib-value-box-showcase-w, 50%)}.bslib-value-box.showcase-top-right .value-box-grid .value-box-showcase{grid-area:right;margin-left:auto;align-self:start;align-items:end;padding-left:0;padding-bottom:0}.bslib-value-box.showcase-top-right .value-box-grid .value-box-area{grid-area:left;align-self:end}.bslib-value-box.showcase-top-right[data-full-screen=true] .value-box-grid{grid-template-columns:auto var(---bslib-value-box-showcase-w-fs, 1fr)}.bslib-value-box.showcase-top-right[data-full-screen=true] .value-box-grid>div{align-self:center}.bslib-value-box.showcase-top-right:not([data-full-screen=true]) .value-box-showcase{margin-top:0}@container bslib-value-box (max-width: 300px){.bslib-value-box.showcase-top-right:not([data-full-screen=true]) .value-box-grid .value-box-showcase{padding-left:1rem}}.bslib-value-box.showcase-left-center .value-box-grid{grid-template-columns:var(---bslib-value-box-showcase-w, 30%) auto}.bslib-value-box.showcase-left-center[data-full-screen=true] .value-box-grid{grid-template-columns:var(---bslib-value-box-showcase-w-fs, 1fr) auto}.bslib-value-box.showcase-left-center:not([data-fill-screen=true]) .value-box-grid .value-box-showcase{grid-area:left}.bslib-value-box.showcase-left-center:not([data-fill-screen=true]) .value-box-grid .value-box-area{grid-area:right}.bslib-value-box.showcase-bottom .value-box-grid{grid-template-columns:1fr;grid-template-rows:1fr var(---bslib-value-box-showcase-h, auto);grid-template-areas:"top" "bottom";overflow:hidden}.bslib-value-box.showcase-bottom .value-box-grid .value-box-showcase{grid-area:bottom;padding:0;margin:0}.bslib-value-box.showcase-bottom .value-box-grid .value-box-area{grid-area:top}.bslib-value-box.showcase-bottom[data-full-screen=true] .value-box-grid{grid-template-rows:1fr var(---bslib-value-box-showcase-h-fs, 2fr)}.bslib-value-box.showcase-bottom[data-full-screen=true] .value-box-grid .value-box-showcase{padding:1rem}[data-bs-theme=dark] .bslib-value-box{--bslib-value-box-shadow: 0 0.5rem 1rem rgb(0 0 0 / 50%)}.html-fill-container{display:flex;flex-direction:column;min-height:0;min-width:0}.html-fill-container>.html-fill-item{flex:1 1 auto;min-height:0;min-width:0}.html-fill-container>:not(.html-fill-item){flex:0 0 auto}.quarto-container{min-height:calc(100vh - 132px)}body.hypothesis-enabled #quarto-header{margin-right:16px}footer.footer .nav-footer,#quarto-header>nav{padding-left:1em;padding-right:1em}footer.footer div.nav-footer p:first-child{margin-top:0}footer.footer div.nav-footer p:last-child{margin-bottom:0}#quarto-content>*{padding-top:14px}#quarto-content>#quarto-sidebar-glass{padding-top:0px}@media(max-width: 991.98px){#quarto-content>*{padding-top:0}#quarto-content .subtitle{padding-top:14px}#quarto-content section:first-of-type h2:first-of-type,#quarto-content section:first-of-type .h2:first-of-type{margin-top:1rem}}.headroom-target,header.headroom{will-change:transform;transition:position 200ms linear;transition:all 200ms linear}header.headroom--pinned{transform:translateY(0%)}header.headroom--unpinned{transform:translateY(-100%)}.navbar-container{width:100%}.navbar-brand{overflow:hidden;text-overflow:ellipsis}.navbar-brand-container{max-width:calc(100% - 115px);min-width:0;display:flex;align-items:center}@media(min-width: 992px){.navbar-brand-container{margin-right:1em}}.navbar-brand.navbar-brand-logo{margin-right:4px;display:inline-flex}.navbar-toggler{flex-basis:content;flex-shrink:0}.navbar .navbar-brand-container{order:2}.navbar .navbar-toggler{order:1}.navbar .navbar-container>.navbar-nav{order:20}.navbar .navbar-container>.navbar-brand-container{margin-left:0 !important;margin-right:0 !important}.navbar .navbar-collapse{order:20}.navbar #quarto-search{order:4;margin-left:auto}.navbar .navbar-toggler{margin-right:.5em}.navbar-collapse .quarto-navbar-tools{margin-left:.5em}.navbar-logo{max-height:24px;width:auto;padding-right:4px}nav .nav-item:not(.compact){padding-top:1px}nav .nav-link i,nav .dropdown-item i{padding-right:1px}.navbar-expand-lg .navbar-nav .nav-link{padding-left:.6rem;padding-right:.6rem}nav .nav-item.compact .nav-link{padding-left:.5rem;padding-right:.5rem;font-size:1.1rem}.navbar .quarto-navbar-tools{order:3}.navbar .quarto-navbar-tools div.dropdown{display:inline-block}.navbar .quarto-navbar-tools .quarto-navigation-tool{color:rgba(255,255,255,.7)}.navbar .quarto-navbar-tools .quarto-navigation-tool:hover{color:#fefefd}.navbar-nav .dropdown-menu{min-width:220px;font-size:.9rem}.navbar .navbar-nav .nav-link.dropdown-toggle::after{opacity:.75;vertical-align:.175em}.navbar ul.dropdown-menu{padding-top:0;padding-bottom:0}.navbar .dropdown-header{text-transform:uppercase;font-size:.8rem;padding:0 .5rem}.navbar .dropdown-item{padding:.4rem .5rem}.navbar .dropdown-item>i.bi{margin-left:.1rem;margin-right:.25em}.sidebar #quarto-search{margin-top:-1px}.sidebar #quarto-search svg.aa-SubmitIcon{width:16px;height:16px}.sidebar-navigation a{color:inherit}.sidebar-title{margin-top:.25rem;padding-bottom:.5rem;font-size:1.3rem;line-height:1.6rem;visibility:visible}.sidebar-title>a{font-size:inherit;text-decoration:none}.sidebar-title .sidebar-tools-main{margin-top:-6px}@media(max-width: 991.98px){#quarto-sidebar div.sidebar-header{padding-top:.2em}}.sidebar-header-stacked .sidebar-title{margin-top:.6rem}.sidebar-logo{max-width:90%;padding-bottom:.5rem}.sidebar-logo-link{text-decoration:none}.sidebar-navigation li a{text-decoration:none}.sidebar-navigation .quarto-navigation-tool{opacity:.7;font-size:.875rem}#quarto-sidebar>nav>.sidebar-tools-main{margin-left:14px}.sidebar-tools-main{display:inline-flex;margin-left:0px;order:2}.sidebar-tools-main:not(.tools-wide){vertical-align:middle}.sidebar-navigation .quarto-navigation-tool.dropdown-toggle::after{display:none}.sidebar.sidebar-navigation>*{padding-top:1em}.sidebar-item{margin-bottom:.2em;line-height:1rem;margin-top:.4rem}.sidebar-section{padding-left:.5em;padding-bottom:.2em}.sidebar-item .sidebar-item-container{display:flex;justify-content:space-between;cursor:pointer}.sidebar-item-toggle:hover{cursor:pointer}.sidebar-item .sidebar-item-toggle .bi{font-size:.7rem;text-align:center}.sidebar-item .sidebar-item-toggle .bi-chevron-right::before{transition:transform 200ms ease}.sidebar-item .sidebar-item-toggle[aria-expanded=false] .bi-chevron-right::before{transform:none}.sidebar-item .sidebar-item-toggle[aria-expanded=true] .bi-chevron-right::before{transform:rotate(90deg)}.sidebar-item-text{width:100%}.sidebar-navigation .sidebar-divider{margin-left:0;margin-right:0;margin-top:.5rem;margin-bottom:.5rem}@media(max-width: 991.98px){.quarto-secondary-nav{display:block}.quarto-secondary-nav button.quarto-search-button{padding-right:0em;padding-left:2em}.quarto-secondary-nav button.quarto-btn-toggle{margin-left:-0.75rem;margin-right:.15rem}.quarto-secondary-nav nav.quarto-title-breadcrumbs{display:none}.quarto-secondary-nav nav.quarto-page-breadcrumbs{display:flex;align-items:center;padding-right:1em;margin-left:-0.25em}.quarto-secondary-nav nav.quarto-page-breadcrumbs a{text-decoration:none}.quarto-secondary-nav nav.quarto-page-breadcrumbs ol.breadcrumb{margin-bottom:0}}@media(min-width: 992px){.quarto-secondary-nav{display:none}}.quarto-title-breadcrumbs .breadcrumb{margin-bottom:.5em;font-size:.9rem}.quarto-title-breadcrumbs .breadcrumb li:last-of-type a{color:#6c757d}.quarto-secondary-nav .quarto-btn-toggle{color:#595959}.quarto-secondary-nav[aria-expanded=false] .quarto-btn-toggle .bi-chevron-right::before{transform:none}.quarto-secondary-nav[aria-expanded=true] .quarto-btn-toggle .bi-chevron-right::before{transform:rotate(90deg)}.quarto-secondary-nav .quarto-btn-toggle .bi-chevron-right::before{transition:transform 200ms ease}.quarto-secondary-nav{cursor:pointer}.no-decor{text-decoration:none}.quarto-secondary-nav-title{margin-top:.3em;color:#595959;padding-top:4px}.quarto-secondary-nav nav.quarto-page-breadcrumbs{color:#595959}.quarto-secondary-nav nav.quarto-page-breadcrumbs a{color:#595959}.quarto-secondary-nav nav.quarto-page-breadcrumbs a:hover{color:rgba(71,95,36,.8)}.quarto-secondary-nav nav.quarto-page-breadcrumbs .breadcrumb-item::before{color:#8c8c8c}.breadcrumb-item{line-height:1.2rem}div.sidebar-item-container{color:#595959}div.sidebar-item-container:hover,div.sidebar-item-container:focus{color:rgba(71,95,36,.8)}div.sidebar-item-container.disabled{color:rgba(89,89,89,.75)}div.sidebar-item-container .active,div.sidebar-item-container .show>.nav-link,div.sidebar-item-container .sidebar-link>code{color:#475f24}div.sidebar.sidebar-navigation.rollup.quarto-sidebar-toggle-contents,nav.sidebar.sidebar-navigation:not(.rollup){background-color:#fff}@media(max-width: 991.98px){.sidebar-navigation .sidebar-item a,.nav-page .nav-page-text,.sidebar-navigation{font-size:1rem}.sidebar-navigation ul.sidebar-section.depth1 .sidebar-section-item{font-size:1.1rem}.sidebar-logo{display:none}.sidebar.sidebar-navigation{position:static;border-bottom:1px solid #dee2e6}.sidebar.sidebar-navigation.collapsing{position:fixed;z-index:1000}.sidebar.sidebar-navigation.show{position:fixed;z-index:1000}.sidebar.sidebar-navigation{min-height:100%}nav.quarto-secondary-nav{background-color:#fff;border-bottom:1px solid #dee2e6}.quarto-banner nav.quarto-secondary-nav{background-color:#325d88;color:rgba(255,255,255,.7);border-top:1px solid #dee2e6}.sidebar .sidebar-footer{visibility:visible;padding-top:1rem;position:inherit}.sidebar-tools-collapse{display:block}}#quarto-sidebar{transition:width .15s ease-in}#quarto-sidebar>*{padding-right:1em}@media(max-width: 991.98px){#quarto-sidebar .sidebar-menu-container{white-space:nowrap;min-width:225px}#quarto-sidebar.show{transition:width .15s ease-out}}@media(min-width: 992px){#quarto-sidebar{display:flex;flex-direction:column}.nav-page .nav-page-text,.sidebar-navigation .sidebar-section .sidebar-item{font-size:.875rem}.sidebar-navigation .sidebar-item{font-size:.925rem}.sidebar.sidebar-navigation{display:block;position:sticky}.sidebar-search{width:100%}.sidebar .sidebar-footer{visibility:visible}}@media(min-width: 992px){#quarto-sidebar-glass{display:none}}@media(max-width: 991.98px){#quarto-sidebar-glass{position:fixed;top:0;bottom:0;left:0;right:0;background-color:rgba(255,255,255,0);transition:background-color .15s ease-in;z-index:-1}#quarto-sidebar-glass.collapsing{z-index:1000}#quarto-sidebar-glass.show{transition:background-color .15s ease-out;background-color:rgba(102,102,102,.4);z-index:1000}}.sidebar .sidebar-footer{padding:.5rem 1rem;align-self:flex-end;color:#6c757d;width:100%}.quarto-page-breadcrumbs .breadcrumb-item+.breadcrumb-item,.quarto-page-breadcrumbs .breadcrumb-item{padding-right:.33em;padding-left:0}.quarto-page-breadcrumbs .breadcrumb-item::before{padding-right:.33em}.quarto-sidebar-footer{font-size:.875em}.sidebar-section .bi-chevron-right{vertical-align:middle}.sidebar-section .bi-chevron-right::before{font-size:.9em}.notransition{-webkit-transition:none !important;-moz-transition:none !important;-o-transition:none !important;transition:none !important}.btn:focus:not(:focus-visible){box-shadow:none}.page-navigation{display:flex;justify-content:space-between}.nav-page{padding-bottom:.75em}.nav-page .bi{font-size:1.8rem;vertical-align:middle}.nav-page .nav-page-text{padding-left:.25em;padding-right:.25em}.nav-page a{color:#6c757d;text-decoration:none;display:flex;align-items:center}.nav-page a:hover{color:#769e3c}.nav-footer .toc-actions{padding-bottom:.5em;padding-top:.5em}.nav-footer .toc-actions a,.nav-footer .toc-actions a:hover{text-decoration:none}.nav-footer .toc-actions ul{display:flex;list-style:none}.nav-footer .toc-actions ul :first-child{margin-left:auto}.nav-footer .toc-actions ul :last-child{margin-right:auto}.nav-footer .toc-actions ul li{padding-right:1.5em}.nav-footer .toc-actions ul li i.bi{padding-right:.4em}.nav-footer .toc-actions ul li:last-of-type{padding-right:0}.nav-footer{display:flex;flex-direction:row;flex-wrap:wrap;justify-content:space-between;align-items:baseline;text-align:center;padding-top:.5rem;padding-bottom:.5rem;background-color:#fff}body.nav-fixed{padding-top:63px}.nav-footer-contents{color:#6c757d;margin-top:.25rem}.nav-footer{min-height:3.5em;color:#757575}.nav-footer a{color:#757575}.nav-footer .nav-footer-left{font-size:.825em}.nav-footer .nav-footer-center{font-size:.825em}.nav-footer .nav-footer-right{font-size:.825em}.nav-footer-left .footer-items,.nav-footer-center .footer-items,.nav-footer-right .footer-items{display:inline-flex;padding-top:.3em;padding-bottom:.3em;margin-bottom:0em}.nav-footer-left .footer-items .nav-link,.nav-footer-center .footer-items .nav-link,.nav-footer-right .footer-items .nav-link{padding-left:.6em;padding-right:.6em}@media(min-width: 768px){.nav-footer-left{flex:1 1 0px;text-align:left}}@media(max-width: 575.98px){.nav-footer-left{margin-bottom:1em;flex:100%}}@media(min-width: 768px){.nav-footer-right{flex:1 1 0px;text-align:right}}@media(max-width: 575.98px){.nav-footer-right{margin-bottom:1em;flex:100%}}.nav-footer-center{text-align:center;min-height:3em}@media(min-width: 768px){.nav-footer-center{flex:1 1 0px}}.nav-footer-center .footer-items{justify-content:center}@media(max-width: 767.98px){.nav-footer-center{margin-bottom:1em;flex:100%}}@media(max-width: 767.98px){.nav-footer-center{margin-top:3em;order:10}}.navbar .quarto-reader-toggle.reader .quarto-reader-toggle-btn{background-color:rgba(255,255,255,.7);border-radius:3px}@media(max-width: 991.98px){.quarto-reader-toggle{display:none}}.quarto-reader-toggle.reader.quarto-navigation-tool .quarto-reader-toggle-btn{background-color:#595959;border-radius:3px}.quarto-reader-toggle .quarto-reader-toggle-btn{display:inline-flex;padding-left:.2em;padding-right:.2em;margin-left:-0.2em;margin-right:-0.2em;text-align:center}.navbar .quarto-reader-toggle:not(.reader) .bi::before{background-image:url('data:image/svg+xml,')}.navbar .quarto-reader-toggle.reader .bi::before{background-image:url('data:image/svg+xml,')}.sidebar-navigation .quarto-reader-toggle:not(.reader) .bi::before{background-image:url('data:image/svg+xml,')}.sidebar-navigation .quarto-reader-toggle.reader .bi::before{background-image:url('data:image/svg+xml,')}#quarto-back-to-top{display:none;position:fixed;bottom:50px;background-color:#fff;border-radius:.25rem;box-shadow:0 .2rem .5rem #6c757d,0 0 .05rem #6c757d;color:#6c757d;text-decoration:none;font-size:.9em;text-align:center;left:50%;padding:.4rem .8rem;transform:translate(-50%, 0)}#quarto-announcement{padding:.5em;display:flex;justify-content:space-between;margin-bottom:0;font-size:.9em}#quarto-announcement .quarto-announcement-content{margin-right:auto}#quarto-announcement .quarto-announcement-content p{margin-bottom:0}#quarto-announcement .quarto-announcement-icon{margin-right:.5em;font-size:1.2em;margin-top:-0.15em}#quarto-announcement .quarto-announcement-action{cursor:pointer}.aa-DetachedSearchButtonQuery{display:none}.aa-DetachedOverlay ul.aa-List,#quarto-search-results ul.aa-List{list-style:none;padding-left:0}.aa-DetachedOverlay .aa-Panel,#quarto-search-results .aa-Panel{background-color:#fff;position:absolute;z-index:2000}#quarto-search-results .aa-Panel{max-width:400px}#quarto-search input{font-size:.925rem}@media(min-width: 992px){.navbar #quarto-search{margin-left:.25rem;order:999}}.navbar.navbar-expand-sm #quarto-search,.navbar.navbar-expand-md #quarto-search{order:999}@media(min-width: 992px){.navbar .quarto-navbar-tools{order:900}}@media(min-width: 992px){.navbar .quarto-navbar-tools.tools-end{margin-left:auto !important}}@media(max-width: 991.98px){#quarto-sidebar .sidebar-search{display:none}}#quarto-sidebar .sidebar-search .aa-Autocomplete{width:100%}.navbar .aa-Autocomplete .aa-Form{width:180px}.navbar #quarto-search.type-overlay .aa-Autocomplete{width:40px}.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form{background-color:inherit;border:none}.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form:focus-within{box-shadow:none;outline:none}.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form .aa-InputWrapper{display:none}.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form .aa-InputWrapper:focus-within{display:inherit}.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form .aa-Label svg,.navbar #quarto-search.type-overlay .aa-Autocomplete .aa-Form .aa-LoadingIndicator svg{width:26px;height:26px;color:rgba(255,255,255,.7);opacity:1}.navbar #quarto-search.type-overlay .aa-Autocomplete svg.aa-SubmitIcon{width:26px;height:26px;color:rgba(255,255,255,.7);opacity:1}.aa-Autocomplete .aa-Form,.aa-DetachedFormContainer .aa-Form{align-items:center;background-color:#fff;border:1px solid #dee2e6;border-radius:.25rem;color:#343a40;display:flex;line-height:1em;margin:0;position:relative;width:100%}.aa-Autocomplete .aa-Form:focus-within,.aa-DetachedFormContainer .aa-Form:focus-within{box-shadow:rgba(50,93,136,.6) 0 0 0 1px;outline:currentColor none medium}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix{align-items:center;display:flex;flex-shrink:0;order:1}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-Label,.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-Label,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator{cursor:initial;flex-shrink:0;padding:0;text-align:left}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-Label svg,.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator svg,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-Label svg,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator svg{color:#343a40;opacity:.5}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-SubmitButton,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-SubmitButton{appearance:none;background:none;border:0;margin:0}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator{align-items:center;display:flex;justify-content:center}.aa-Autocomplete .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator[hidden],.aa-DetachedFormContainer .aa-Form .aa-InputWrapperPrefix .aa-LoadingIndicator[hidden]{display:none}.aa-Autocomplete .aa-Form .aa-InputWrapper,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper{order:3;position:relative;width:100%}.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input{appearance:none;background:none;border:0;color:#343a40;font:inherit;height:calc(1.5em + .1rem + 2px);padding:0;width:100%}.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input::placeholder,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input::placeholder{color:#343a40;opacity:.8}.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input:focus,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input:focus{border-color:none;box-shadow:none;outline:none}.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-decoration,.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-cancel-button,.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-results-button,.aa-Autocomplete .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-results-decoration,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-decoration,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-cancel-button,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-results-button,.aa-DetachedFormContainer .aa-Form .aa-InputWrapper .aa-Input::-webkit-search-results-decoration{display:none}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix{align-items:center;display:flex;order:4}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-ClearButton,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-ClearButton{align-items:center;background:none;border:0;color:#343a40;opacity:.8;cursor:pointer;display:flex;margin:0;width:calc(1.5em + .1rem + 2px)}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-ClearButton:hover,.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-ClearButton:focus,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-ClearButton:hover,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-ClearButton:focus{color:#343a40;opacity:.8}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-ClearButton[hidden],.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-ClearButton[hidden]{display:none}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-ClearButton svg,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-ClearButton svg{width:calc(1.5em + 0.75rem + calc(1px * 2))}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-CopyButton,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-CopyButton{border:none;align-items:center;background:none;color:#343a40;opacity:.4;font-size:.7rem;cursor:pointer;display:none;margin:0;width:calc(1em + .1rem + 2px)}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-CopyButton:hover,.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-CopyButton:focus,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-CopyButton:hover,.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-CopyButton:focus{color:#343a40;opacity:.8}.aa-Autocomplete .aa-Form .aa-InputWrapperSuffix .aa-CopyButton[hidden],.aa-DetachedFormContainer .aa-Form .aa-InputWrapperSuffix .aa-CopyButton[hidden]{display:none}.aa-PanelLayout:empty{display:none}.quarto-search-no-results.no-query{display:none}.aa-Source:has(.no-query){display:none}#quarto-search-results .aa-Panel{border:solid #dee2e6 1px}#quarto-search-results .aa-SourceNoResults{width:398px}.aa-DetachedOverlay .aa-Panel,#quarto-search-results .aa-Panel{max-height:65vh;overflow-y:auto;font-size:.925rem}.aa-DetachedOverlay .aa-SourceNoResults,#quarto-search-results .aa-SourceNoResults{height:60px;display:flex;justify-content:center;align-items:center}.aa-DetachedOverlay .search-error,#quarto-search-results .search-error{padding-top:10px;padding-left:20px;padding-right:20px;cursor:default}.aa-DetachedOverlay .search-error .search-error-title,#quarto-search-results .search-error .search-error-title{font-size:1.1rem;margin-bottom:.5rem}.aa-DetachedOverlay .search-error .search-error-title .search-error-icon,#quarto-search-results .search-error .search-error-title .search-error-icon{margin-right:8px}.aa-DetachedOverlay .search-error .search-error-text,#quarto-search-results .search-error .search-error-text{font-weight:300}.aa-DetachedOverlay .search-result-text,#quarto-search-results .search-result-text{font-weight:300;overflow:hidden;text-overflow:ellipsis;display:-webkit-box;-webkit-line-clamp:2;-webkit-box-orient:vertical;line-height:1.2rem;max-height:2.4rem}.aa-DetachedOverlay .aa-SourceHeader .search-result-header,#quarto-search-results .aa-SourceHeader .search-result-header{font-size:.875rem;background-color:#f2f2f2;padding-left:14px;padding-bottom:4px;padding-top:4px}.aa-DetachedOverlay .aa-SourceHeader .search-result-header-no-results,#quarto-search-results .aa-SourceHeader .search-result-header-no-results{display:none}.aa-DetachedOverlay .aa-SourceFooter .algolia-search-logo,#quarto-search-results .aa-SourceFooter .algolia-search-logo{width:110px;opacity:.85;margin:8px;float:right}.aa-DetachedOverlay .search-result-section,#quarto-search-results .search-result-section{font-size:.925em}.aa-DetachedOverlay a.search-result-link,#quarto-search-results a.search-result-link{color:inherit;text-decoration:none}.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item,#quarto-search-results li.aa-Item[aria-selected=true] .search-item{background-color:#325d88}.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item.search-result-more,.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item .search-result-section,.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item .search-result-text,.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item .search-result-title-container,.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item .search-result-text-container,#quarto-search-results li.aa-Item[aria-selected=true] .search-item.search-result-more,#quarto-search-results li.aa-Item[aria-selected=true] .search-item .search-result-section,#quarto-search-results li.aa-Item[aria-selected=true] .search-item .search-result-text,#quarto-search-results li.aa-Item[aria-selected=true] .search-item .search-result-title-container,#quarto-search-results li.aa-Item[aria-selected=true] .search-item .search-result-text-container{color:#fff;background-color:#325d88}.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item mark.search-match,.aa-DetachedOverlay li.aa-Item[aria-selected=true] .search-item .search-match.mark,#quarto-search-results li.aa-Item[aria-selected=true] .search-item mark.search-match,#quarto-search-results li.aa-Item[aria-selected=true] .search-item .search-match.mark{color:#fff;background-color:#3d71a6}.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item,#quarto-search-results li.aa-Item[aria-selected=false] .search-item{background-color:#fff}.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item.search-result-more,.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item .search-result-section,.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item .search-result-text,.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item .search-result-title-container,.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item .search-result-text-container,#quarto-search-results li.aa-Item[aria-selected=false] .search-item.search-result-more,#quarto-search-results li.aa-Item[aria-selected=false] .search-item .search-result-section,#quarto-search-results li.aa-Item[aria-selected=false] .search-item .search-result-text,#quarto-search-results li.aa-Item[aria-selected=false] .search-item .search-result-title-container,#quarto-search-results li.aa-Item[aria-selected=false] .search-item .search-result-text-container{color:#343a40}.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item mark.search-match,.aa-DetachedOverlay li.aa-Item[aria-selected=false] .search-item .search-match.mark,#quarto-search-results li.aa-Item[aria-selected=false] .search-item mark.search-match,#quarto-search-results li.aa-Item[aria-selected=false] .search-item .search-match.mark{color:inherit;background-color:#afc8e1}.aa-DetachedOverlay .aa-Item .search-result-doc:not(.document-selectable) .search-result-title-container,#quarto-search-results .aa-Item .search-result-doc:not(.document-selectable) .search-result-title-container{background-color:#fff;color:#343a40}.aa-DetachedOverlay .aa-Item .search-result-doc:not(.document-selectable) .search-result-text-container,#quarto-search-results .aa-Item .search-result-doc:not(.document-selectable) .search-result-text-container{padding-top:0px}.aa-DetachedOverlay li.aa-Item .search-result-doc.document-selectable .search-result-text-container,#quarto-search-results li.aa-Item .search-result-doc.document-selectable .search-result-text-container{margin-top:-4px}.aa-DetachedOverlay .aa-Item,#quarto-search-results .aa-Item{cursor:pointer}.aa-DetachedOverlay .aa-Item .search-item,#quarto-search-results .aa-Item .search-item{border-left:none;border-right:none;border-top:none;background-color:#fff;border-color:#dee2e6;color:#343a40}.aa-DetachedOverlay .aa-Item .search-item p,#quarto-search-results .aa-Item .search-item p{margin-top:0;margin-bottom:0}.aa-DetachedOverlay .aa-Item .search-item i.bi,#quarto-search-results .aa-Item .search-item i.bi{padding-left:8px;padding-right:8px;font-size:1.3em}.aa-DetachedOverlay .aa-Item .search-item .search-result-title,#quarto-search-results .aa-Item .search-item .search-result-title{margin-top:.3em;margin-bottom:0em}.aa-DetachedOverlay .aa-Item .search-item .search-result-crumbs,#quarto-search-results .aa-Item .search-item .search-result-crumbs{white-space:nowrap;text-overflow:ellipsis;font-size:.8em;font-weight:300;margin-right:1em}.aa-DetachedOverlay .aa-Item .search-item .search-result-crumbs:not(.search-result-crumbs-wrap),#quarto-search-results .aa-Item .search-item .search-result-crumbs:not(.search-result-crumbs-wrap){max-width:30%;margin-left:auto;margin-top:.5em;margin-bottom:.1rem}.aa-DetachedOverlay .aa-Item .search-item .search-result-crumbs.search-result-crumbs-wrap,#quarto-search-results .aa-Item .search-item .search-result-crumbs.search-result-crumbs-wrap{flex-basis:100%;margin-top:0em;margin-bottom:.2em;margin-left:37px}.aa-DetachedOverlay .aa-Item .search-result-title-container,#quarto-search-results .aa-Item .search-result-title-container{font-size:1em;display:flex;flex-wrap:wrap;padding:6px 4px 6px 4px}.aa-DetachedOverlay .aa-Item .search-result-text-container,#quarto-search-results .aa-Item .search-result-text-container{padding-bottom:8px;padding-right:8px;margin-left:42px}.aa-DetachedOverlay .aa-Item .search-result-doc-section,.aa-DetachedOverlay .aa-Item .search-result-more,#quarto-search-results .aa-Item .search-result-doc-section,#quarto-search-results .aa-Item .search-result-more{padding-top:8px;padding-bottom:8px;padding-left:44px}.aa-DetachedOverlay .aa-Item .search-result-more,#quarto-search-results .aa-Item .search-result-more{font-size:.8em;font-weight:400}.aa-DetachedOverlay .aa-Item .search-result-doc,#quarto-search-results .aa-Item .search-result-doc{border-top:1px solid #dee2e6}.aa-DetachedSearchButton{background:none;border:none}.aa-DetachedSearchButton .aa-DetachedSearchButtonPlaceholder{display:none}.navbar .aa-DetachedSearchButton .aa-DetachedSearchButtonIcon{color:rgba(255,255,255,.7)}.sidebar-tools-collapse #quarto-search,.sidebar-tools-main #quarto-search{display:inline}.sidebar-tools-collapse #quarto-search .aa-Autocomplete,.sidebar-tools-main #quarto-search .aa-Autocomplete{display:inline}.sidebar-tools-collapse #quarto-search .aa-DetachedSearchButton,.sidebar-tools-main #quarto-search .aa-DetachedSearchButton{padding-left:4px;padding-right:4px}.sidebar-tools-collapse #quarto-search .aa-DetachedSearchButton .aa-DetachedSearchButtonIcon,.sidebar-tools-main #quarto-search .aa-DetachedSearchButton .aa-DetachedSearchButtonIcon{color:#595959}.sidebar-tools-collapse #quarto-search .aa-DetachedSearchButton .aa-DetachedSearchButtonIcon .aa-SubmitIcon,.sidebar-tools-main #quarto-search .aa-DetachedSearchButton .aa-DetachedSearchButtonIcon .aa-SubmitIcon{margin-top:-3px}.aa-DetachedContainer{background:rgba(255,255,255,.65);width:90%;bottom:0;box-shadow:rgba(222,226,230,.6) 0 0 0 1px;outline:currentColor none medium;display:flex;flex-direction:column;left:0;margin:0;overflow:hidden;padding:0;position:fixed;right:0;top:0;z-index:1101}.aa-DetachedContainer::after{height:32px}.aa-DetachedContainer .aa-SourceHeader{margin:var(--aa-spacing-half) 0 var(--aa-spacing-half) 2px}.aa-DetachedContainer .aa-Panel{background-color:#fff;border-radius:0;box-shadow:none;flex-grow:1;margin:0;padding:0;position:relative}.aa-DetachedContainer .aa-PanelLayout{bottom:0;box-shadow:none;left:0;margin:0;max-height:none;overflow-y:auto;position:absolute;right:0;top:0;width:100%}.aa-DetachedFormContainer{background-color:#fff;border-bottom:1px solid #dee2e6;display:flex;flex-direction:row;justify-content:space-between;margin:0;padding:.5em}.aa-DetachedCancelButton{background:none;font-size:.8em;border:0;border-radius:3px;color:#343a40;cursor:pointer;margin:0 0 0 .5em;padding:0 .5em}.aa-DetachedCancelButton:hover,.aa-DetachedCancelButton:focus{box-shadow:rgba(50,93,136,.6) 0 0 0 1px;outline:currentColor none medium}.aa-DetachedContainer--modal{bottom:inherit;height:auto;margin:0 auto;position:absolute;top:100px;border-radius:6px;max-width:850px}@media(max-width: 575.98px){.aa-DetachedContainer--modal{width:100%;top:0px;border-radius:0px;border:none}}.aa-DetachedContainer--modal .aa-PanelLayout{max-height:var(--aa-detached-modal-max-height);padding-bottom:var(--aa-spacing-half);position:static}.aa-Detached{height:100vh;overflow:hidden}.aa-DetachedOverlay{background-color:rgba(52,58,64,.4);position:fixed;left:0;right:0;top:0;margin:0;padding:0;height:100vh;z-index:1100}.quarto-dashboard.nav-fixed.dashboard-sidebar #quarto-content.quarto-dashboard-content{padding:0em}.quarto-dashboard #quarto-content.quarto-dashboard-content{padding:1em}.quarto-dashboard #quarto-content.quarto-dashboard-content>*{padding-top:0}@media(min-width: 576px){.quarto-dashboard{height:100%}}.quarto-dashboard .card.valuebox.bslib-card.bg-primary{background-color:#7b98ad !important}.quarto-dashboard .card.valuebox.bslib-card.bg-secondary{background-color:#6c757d !important}.quarto-dashboard .card.valuebox.bslib-card.bg-success{background-color:#60a545 !important}.quarto-dashboard .card.valuebox.bslib-card.bg-info{background-color:#3d9dd1 !important}.quarto-dashboard .card.valuebox.bslib-card.bg-warning{background-color:#af8e08 !important}.quarto-dashboard .card.valuebox.bslib-card.bg-danger{background-color:#ca8181 !important}.quarto-dashboard .card.valuebox.bslib-card.bg-light{background-color:#f8f5f0 !important}.quarto-dashboard .card.valuebox.bslib-card.bg-dark{background-color:#343a40 !important}.quarto-dashboard.dashboard-fill{display:flex;flex-direction:column}.quarto-dashboard #quarto-appendix{display:none}.quarto-dashboard #quarto-header #quarto-dashboard-header{border-top:solid 1px #4077ad;border-bottom:solid 1px #4077ad}.quarto-dashboard #quarto-header #quarto-dashboard-header>nav{padding-left:1em;padding-right:1em}.quarto-dashboard #quarto-header #quarto-dashboard-header>nav .navbar-brand-container{padding-left:0}.quarto-dashboard #quarto-header #quarto-dashboard-header .navbar-toggler{margin-right:0}.quarto-dashboard #quarto-header #quarto-dashboard-header .navbar-toggler-icon{height:1em;width:1em;background-image:url('data:image/svg+xml,')}.quarto-dashboard #quarto-header #quarto-dashboard-header .navbar-brand-container{padding-right:1em}.quarto-dashboard #quarto-header #quarto-dashboard-header .navbar-title{font-size:1.1em}.quarto-dashboard #quarto-header #quarto-dashboard-header .navbar-nav{font-size:.9em}.quarto-dashboard #quarto-dashboard-header .navbar{padding:0}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-container{padding-left:1em}.quarto-dashboard #quarto-dashboard-header .navbar.slim .navbar-brand-container .nav-link,.quarto-dashboard #quarto-dashboard-header .navbar.slim .navbar-nav .nav-link{padding:.7em}.quarto-dashboard #quarto-dashboard-header .navbar .quarto-color-scheme-toggle{order:9}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-toggler{margin-left:.5em;order:10}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-nav .nav-link{padding:.5em;height:100%;display:flex;align-items:center}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-nav .active{background-color:#3d71a6}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-brand-container{padding:.5em .5em .5em 0;display:flex;flex-direction:row;margin-right:2em;align-items:center}@media(max-width: 767.98px){.quarto-dashboard #quarto-dashboard-header .navbar .navbar-brand-container{margin-right:auto}}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-collapse{align-self:stretch}@media(min-width: 768px){.quarto-dashboard #quarto-dashboard-header .navbar .navbar-collapse{order:8}}@media(max-width: 767.98px){.quarto-dashboard #quarto-dashboard-header .navbar .navbar-collapse{order:1000;padding-bottom:.5em}}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-collapse .navbar-nav{align-self:stretch}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-title{font-size:1.25em;line-height:1.1em;display:flex;flex-direction:row;flex-wrap:wrap;align-items:baseline}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-title .navbar-title-text{margin-right:.4em}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-title a{text-decoration:none;color:inherit}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-subtitle,.quarto-dashboard #quarto-dashboard-header .navbar .navbar-author{font-size:.9rem;margin-right:.5em}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-author{margin-left:auto}.quarto-dashboard #quarto-dashboard-header .navbar .navbar-logo{max-height:48px;min-height:30px;object-fit:cover;margin-right:1em}.quarto-dashboard #quarto-dashboard-header .navbar .quarto-dashboard-links{order:9;padding-right:1em}.quarto-dashboard #quarto-dashboard-header .navbar .quarto-dashboard-link-text{margin-left:.25em}.quarto-dashboard #quarto-dashboard-header .navbar .quarto-dashboard-link{padding-right:0em;padding-left:.7em;text-decoration:none;color:rgba(255,255,255,.7)}.quarto-dashboard .page-layout-custom .tab-content{padding:0;border:none}.quarto-dashboard-img-contain{height:100%;width:100%;object-fit:contain}@media(max-width: 575.98px){.quarto-dashboard .bslib-grid{grid-template-rows:minmax(1em, max-content) !important}.quarto-dashboard .sidebar-content{height:inherit}.quarto-dashboard .page-layout-custom{min-height:100vh}}.quarto-dashboard.dashboard-toolbar>.page-layout-custom,.quarto-dashboard.dashboard-sidebar>.page-layout-custom{padding:0}.quarto-dashboard .quarto-dashboard-content.quarto-dashboard-pages{padding:0}.quarto-dashboard .callout{margin-bottom:0;margin-top:0}.quarto-dashboard .html-fill-container figure{overflow:hidden}.quarto-dashboard bslib-tooltip .rounded-pill{border:solid #6c757d 1px}.quarto-dashboard bslib-tooltip .rounded-pill .svg{fill:#343a40}.quarto-dashboard .tabset .dashboard-card-no-title .nav-tabs{margin-left:0;margin-right:auto}.quarto-dashboard .tabset .tab-content{border:none}.quarto-dashboard .tabset .card-header .nav-link[role=tab]{margin-top:-6px;padding-top:6px;padding-bottom:6px}.quarto-dashboard .card.valuebox,.quarto-dashboard .card.bslib-value-box{min-height:3rem}.quarto-dashboard .card.valuebox .card-body,.quarto-dashboard .card.bslib-value-box .card-body{padding:0}.quarto-dashboard .bslib-value-box .value-box-value{font-size:clamp(.1em,15cqw,5em)}.quarto-dashboard .bslib-value-box .value-box-showcase .bi{font-size:clamp(.1em,max(18cqw,5.2cqh),5em);text-align:center;height:1em}.quarto-dashboard .bslib-value-box .value-box-showcase .bi::before{vertical-align:1em}.quarto-dashboard .bslib-value-box .value-box-area{margin-top:auto;margin-bottom:auto}.quarto-dashboard .card figure.quarto-float{display:flex;flex-direction:column;align-items:center}.quarto-dashboard .dashboard-scrolling{padding:1em}.quarto-dashboard .full-height{height:100%}.quarto-dashboard .showcase-bottom .value-box-grid{display:grid;grid-template-columns:1fr;grid-template-rows:1fr auto;grid-template-areas:"top" "bottom"}.quarto-dashboard .showcase-bottom .value-box-grid .value-box-showcase{grid-area:bottom;padding:0;margin:0}.quarto-dashboard .showcase-bottom .value-box-grid .value-box-showcase i.bi{font-size:4rem}.quarto-dashboard .showcase-bottom .value-box-grid .value-box-area{grid-area:top}.quarto-dashboard .tab-content{margin-bottom:0}.quarto-dashboard .bslib-card .bslib-navs-card-title{justify-content:stretch;align-items:end}.quarto-dashboard .card-header{display:flex;flex-wrap:wrap;justify-content:space-between}.quarto-dashboard .card-header .card-title{display:flex;flex-direction:column;justify-content:center;margin-bottom:0}.quarto-dashboard .tabset .card-toolbar{margin-bottom:1em}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout{border:none;gap:var(--bslib-spacer, 1rem)}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout>.main{padding:0}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout>.sidebar{border-radius:.25rem;border:1px solid rgba(222,226,230,.75)}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout>.collapse-toggle{display:none}@media(max-width: 767.98px){.quarto-dashboard .bslib-grid>.bslib-sidebar-layout{grid-template-columns:1fr;grid-template-rows:max-content 1fr}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout>.main{grid-column:1;grid-row:2}.quarto-dashboard .bslib-grid>.bslib-sidebar-layout .sidebar{grid-column:1;grid-row:1}}.quarto-dashboard .sidebar-right .sidebar{padding-left:2.5em}.quarto-dashboard .sidebar-right .collapse-toggle{left:2px}.quarto-dashboard .quarto-dashboard .sidebar-right button.collapse-toggle:not(.transitioning){left:unset}.quarto-dashboard aside.sidebar{padding-left:1em;padding-right:1em;background-color:rgba(52,58,64,.25);color:#343a40}.quarto-dashboard .bslib-sidebar-layout>div.main{padding:.7em}.quarto-dashboard .bslib-sidebar-layout button.collapse-toggle{margin-top:.3em}.quarto-dashboard .bslib-sidebar-layout .collapse-toggle{top:0}.quarto-dashboard .bslib-sidebar-layout.sidebar-collapsed:not(.transitioning):not(.sidebar-right) .collapse-toggle{left:2px}.quarto-dashboard .sidebar>section>.h3:first-of-type{margin-top:0em}.quarto-dashboard .sidebar .h3,.quarto-dashboard .sidebar .h4,.quarto-dashboard .sidebar .h5,.quarto-dashboard .sidebar .h6{margin-top:.5em}.quarto-dashboard .sidebar form{flex-direction:column;align-items:start;margin-bottom:1em}.quarto-dashboard .sidebar form div[class*=oi-][class$=-input]{flex-direction:column}.quarto-dashboard .sidebar form[class*=oi-][class$=-toggle]{flex-direction:row-reverse;align-items:center;justify-content:start}.quarto-dashboard .sidebar form input[type=range]{margin-top:.5em;margin-right:.8em;margin-left:1em}.quarto-dashboard .sidebar label{width:fit-content}.quarto-dashboard .sidebar .card-body{margin-bottom:2em}.quarto-dashboard .sidebar .shiny-input-container{margin-bottom:1em}.quarto-dashboard .sidebar .shiny-options-group{margin-top:0}.quarto-dashboard .sidebar .control-label{margin-bottom:.3em}.quarto-dashboard .card .card-body .quarto-layout-row{align-items:stretch}.quarto-dashboard .toolbar{font-size:.9em;display:flex;flex-direction:row;border-top:solid 1px #c7c9cd;padding:1em;flex-wrap:wrap;background-color:rgba(52,58,64,.25)}.quarto-dashboard .toolbar .cell-output-display{display:flex}.quarto-dashboard .toolbar .shiny-input-container{padding-bottom:.5em;margin-bottom:.5em;width:inherit}.quarto-dashboard .toolbar .shiny-input-container>.checkbox:first-child{margin-top:6px}.quarto-dashboard .toolbar>*:last-child{margin-right:0}.quarto-dashboard .toolbar>*>*{margin-right:1em;align-items:baseline}.quarto-dashboard .toolbar>*>*>a{text-decoration:none;margin-top:auto;margin-bottom:auto}.quarto-dashboard .toolbar .shiny-input-container{padding-bottom:0;margin-bottom:0}.quarto-dashboard .toolbar .shiny-input-container>*{flex-shrink:0;flex-grow:0}.quarto-dashboard .toolbar .form-group.shiny-input-container:not([role=group])>label{margin-bottom:0}.quarto-dashboard .toolbar .shiny-input-container.no-baseline{align-items:start;padding-top:6px}.quarto-dashboard .toolbar .shiny-input-container{display:flex;align-items:baseline}.quarto-dashboard .toolbar .shiny-input-container label{padding-right:.4em}.quarto-dashboard .toolbar .shiny-input-container .bslib-input-switch{margin-top:6px}.quarto-dashboard .toolbar input[type=text]{line-height:1;width:inherit}.quarto-dashboard .toolbar .input-daterange{width:inherit}.quarto-dashboard .toolbar .input-daterange input[type=text]{height:2.4em;width:10em}.quarto-dashboard .toolbar .input-daterange .input-group-addon{height:auto;padding:0;margin-left:-5px !important;margin-right:-5px}.quarto-dashboard .toolbar .input-daterange .input-group-addon .input-group-text{padding-top:0;padding-bottom:0;height:100%}.quarto-dashboard .toolbar span.irs.irs--shiny{width:10em}.quarto-dashboard .toolbar span.irs.irs--shiny .irs-line{top:9px}.quarto-dashboard .toolbar span.irs.irs--shiny .irs-min,.quarto-dashboard .toolbar span.irs.irs--shiny .irs-max,.quarto-dashboard .toolbar span.irs.irs--shiny .irs-from,.quarto-dashboard .toolbar span.irs.irs--shiny .irs-to,.quarto-dashboard .toolbar span.irs.irs--shiny .irs-single{top:20px}.quarto-dashboard .toolbar span.irs.irs--shiny .irs-bar{top:8px}.quarto-dashboard .toolbar span.irs.irs--shiny .irs-handle{top:0px}.quarto-dashboard .toolbar .shiny-input-checkboxgroup>label{margin-top:6px}.quarto-dashboard .toolbar .shiny-input-checkboxgroup>.shiny-options-group{margin-top:0;align-items:baseline}.quarto-dashboard .toolbar .shiny-input-radiogroup>label{margin-top:6px}.quarto-dashboard .toolbar .shiny-input-radiogroup>.shiny-options-group{align-items:baseline;margin-top:0}.quarto-dashboard .toolbar .shiny-input-radiogroup>.shiny-options-group>.radio{margin-right:.3em}.quarto-dashboard .toolbar .form-select{padding-top:.2em;padding-bottom:.2em}.quarto-dashboard .toolbar .shiny-input-select{min-width:6em}.quarto-dashboard .toolbar div.checkbox{margin-bottom:0px}.quarto-dashboard .toolbar>.checkbox:first-child{margin-top:6px}.quarto-dashboard .toolbar form{width:fit-content}.quarto-dashboard .toolbar form label{padding-top:.2em;padding-bottom:.2em;width:fit-content}.quarto-dashboard .toolbar form input[type=date]{width:fit-content}.quarto-dashboard .toolbar form input[type=color]{width:3em}.quarto-dashboard .toolbar form button{padding:.4em}.quarto-dashboard .toolbar form select{width:fit-content}.quarto-dashboard .toolbar>*{font-size:.9em;flex-grow:0}.quarto-dashboard .toolbar .shiny-input-container label{margin-bottom:1px}.quarto-dashboard .toolbar-bottom{margin-top:1em;margin-bottom:0 !important;order:2}.quarto-dashboard .quarto-dashboard-content>.dashboard-toolbar-container>.toolbar-content>.tab-content>.tab-pane>*:not(.bslib-sidebar-layout){padding:1em}.quarto-dashboard .quarto-dashboard-content>.dashboard-toolbar-container>.toolbar-content>*:not(.tab-content){padding:1em}.quarto-dashboard .quarto-dashboard-content>.tab-content>.dashboard-page>.dashboard-toolbar-container>.toolbar-content,.quarto-dashboard .quarto-dashboard-content>.tab-content>.dashboard-page:not(.dashboard-sidebar-container)>*:not(.dashboard-toolbar-container){padding:1em}.quarto-dashboard .toolbar-content{padding:0}.quarto-dashboard .quarto-dashboard-content.quarto-dashboard-pages .tab-pane>.dashboard-toolbar-container .toolbar{border-radius:0;margin-bottom:0}.quarto-dashboard .dashboard-toolbar-container.toolbar-toplevel .toolbar{border-bottom:1px solid rgba(222,226,230,.75)}.quarto-dashboard .dashboard-toolbar-container.toolbar-toplevel .toolbar-bottom{margin-top:0}.quarto-dashboard .dashboard-toolbar-container:not(.toolbar-toplevel) .toolbar{margin-bottom:1em;border-top:none;border-radius:.25rem;border:1px solid rgba(222,226,230,.75)}.quarto-dashboard .vega-embed.has-actions details{width:1.7em;height:2em;position:absolute !important;top:0;right:0}.quarto-dashboard .dashboard-toolbar-container{padding:0}.quarto-dashboard .card .card-header p:last-child,.quarto-dashboard .card .card-footer p:last-child{margin-bottom:0}.quarto-dashboard .card .card-body>.h4:first-child{margin-top:0}.quarto-dashboard .card .card-body{z-index:4}@media(max-width: 767.98px){.quarto-dashboard .card .card-body .itables div.dataTables_wrapper div.dataTables_length,.quarto-dashboard .card .card-body .itables div.dataTables_wrapper div.dataTables_info,.quarto-dashboard .card .card-body .itables div.dataTables_wrapper div.dataTables_paginate{text-align:initial}.quarto-dashboard .card .card-body .itables div.dataTables_wrapper div.dataTables_filter{text-align:right}.quarto-dashboard .card .card-body .itables div.dataTables_wrapper div.dataTables_paginate ul.pagination{justify-content:initial}}.quarto-dashboard .card .card-body .itables .dataTables_wrapper{display:flex;flex-wrap:wrap;justify-content:space-between;align-items:center;padding-top:0}.quarto-dashboard .card .card-body .itables .dataTables_wrapper table{flex-shrink:0}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dt-buttons{margin-bottom:.5em;margin-left:auto;width:fit-content;float:right}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dt-buttons.btn-group{background:#fff;border:none}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dt-buttons .btn-secondary{background-color:#fff;background-image:none;border:solid #dee2e6 1px;padding:.2em .7em}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dt-buttons .btn span{font-size:.8em;color:#343a40}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_info{margin-left:.5em;margin-bottom:.5em;padding-top:0}@media(min-width: 768px){.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_info{font-size:.875em}}@media(max-width: 767.98px){.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_info{font-size:.8em}}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_filter{margin-bottom:.5em;font-size:.875em}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_filter input[type=search]{padding:1px 5px 1px 5px;font-size:.875em}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_length{flex-basis:1 1 50%;margin-bottom:.5em;font-size:.875em}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_length select{padding:.4em 3em .4em .5em;font-size:.875em;margin-left:.2em;margin-right:.2em}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_paginate{flex-shrink:0}@media(min-width: 768px){.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_paginate{margin-left:auto}}.quarto-dashboard .card .card-body .itables .dataTables_wrapper .dataTables_paginate ul.pagination .paginate_button .page-link{font-size:.8em}.quarto-dashboard .card .card-footer{font-size:.9em}.quarto-dashboard .card .card-toolbar{display:flex;flex-grow:1;flex-direction:row;width:100%;flex-wrap:wrap}.quarto-dashboard .card .card-toolbar>*{font-size:.8em;flex-grow:0}.quarto-dashboard .card .card-toolbar>.card-title{font-size:1em;flex-grow:1;align-self:flex-start;margin-top:.1em}.quarto-dashboard .card .card-toolbar .cell-output-display{display:flex}.quarto-dashboard .card .card-toolbar .shiny-input-container{padding-bottom:.5em;margin-bottom:.5em;width:inherit}.quarto-dashboard .card .card-toolbar .shiny-input-container>.checkbox:first-child{margin-top:6px}.quarto-dashboard .card .card-toolbar>*:last-child{margin-right:0}.quarto-dashboard .card .card-toolbar>*>*{margin-right:1em;align-items:baseline}.quarto-dashboard .card .card-toolbar>*>*>a{text-decoration:none;margin-top:auto;margin-bottom:auto}.quarto-dashboard .card .card-toolbar form{width:fit-content}.quarto-dashboard .card .card-toolbar form label{padding-top:.2em;padding-bottom:.2em;width:fit-content}.quarto-dashboard .card .card-toolbar form input[type=date]{width:fit-content}.quarto-dashboard .card .card-toolbar form input[type=color]{width:3em}.quarto-dashboard .card .card-toolbar form button{padding:.4em}.quarto-dashboard .card .card-toolbar form select{width:fit-content}.quarto-dashboard .card .card-toolbar .cell-output-display{display:flex}.quarto-dashboard .card .card-toolbar .shiny-input-container{padding-bottom:.5em;margin-bottom:.5em;width:inherit}.quarto-dashboard .card .card-toolbar .shiny-input-container>.checkbox:first-child{margin-top:6px}.quarto-dashboard .card .card-toolbar>*:last-child{margin-right:0}.quarto-dashboard .card .card-toolbar>*>*{margin-right:1em;align-items:baseline}.quarto-dashboard .card .card-toolbar>*>*>a{text-decoration:none;margin-top:auto;margin-bottom:auto}.quarto-dashboard .card .card-toolbar .shiny-input-container{padding-bottom:0;margin-bottom:0}.quarto-dashboard .card .card-toolbar .shiny-input-container>*{flex-shrink:0;flex-grow:0}.quarto-dashboard .card .card-toolbar .form-group.shiny-input-container:not([role=group])>label{margin-bottom:0}.quarto-dashboard .card .card-toolbar .shiny-input-container.no-baseline{align-items:start;padding-top:6px}.quarto-dashboard .card .card-toolbar .shiny-input-container{display:flex;align-items:baseline}.quarto-dashboard .card .card-toolbar .shiny-input-container label{padding-right:.4em}.quarto-dashboard .card .card-toolbar .shiny-input-container .bslib-input-switch{margin-top:6px}.quarto-dashboard .card .card-toolbar input[type=text]{line-height:1;width:inherit}.quarto-dashboard .card .card-toolbar .input-daterange{width:inherit}.quarto-dashboard .card .card-toolbar .input-daterange input[type=text]{height:2.4em;width:10em}.quarto-dashboard .card .card-toolbar .input-daterange .input-group-addon{height:auto;padding:0;margin-left:-5px !important;margin-right:-5px}.quarto-dashboard .card .card-toolbar .input-daterange .input-group-addon .input-group-text{padding-top:0;padding-bottom:0;height:100%}.quarto-dashboard .card .card-toolbar span.irs.irs--shiny{width:10em}.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-line{top:9px}.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-min,.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-max,.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-from,.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-to,.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-single{top:20px}.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-bar{top:8px}.quarto-dashboard .card .card-toolbar span.irs.irs--shiny .irs-handle{top:0px}.quarto-dashboard .card .card-toolbar .shiny-input-checkboxgroup>label{margin-top:6px}.quarto-dashboard .card .card-toolbar .shiny-input-checkboxgroup>.shiny-options-group{margin-top:0;align-items:baseline}.quarto-dashboard .card .card-toolbar .shiny-input-radiogroup>label{margin-top:6px}.quarto-dashboard .card .card-toolbar .shiny-input-radiogroup>.shiny-options-group{align-items:baseline;margin-top:0}.quarto-dashboard .card .card-toolbar .shiny-input-radiogroup>.shiny-options-group>.radio{margin-right:.3em}.quarto-dashboard .card .card-toolbar .form-select{padding-top:.2em;padding-bottom:.2em}.quarto-dashboard .card .card-toolbar .shiny-input-select{min-width:6em}.quarto-dashboard .card .card-toolbar div.checkbox{margin-bottom:0px}.quarto-dashboard .card .card-toolbar>.checkbox:first-child{margin-top:6px}.quarto-dashboard .card-body>table>thead{border-top:none}.quarto-dashboard .card-body>.table>:not(caption)>*>*{background-color:#fff}.tableFloatingHeaderOriginal{background-color:#fff;position:sticky !important;top:0 !important}.dashboard-data-table{margin-top:-1px}div.value-box-area span.observablehq--number{font-size:calc(clamp(.1em,15cqw,5em)*1.25);line-height:1.2;color:inherit;font-family:var(--bs-body-font-family)}.quarto-listing{padding-bottom:1em}.listing-pagination{padding-top:.5em}ul.pagination{float:right;padding-left:8px;padding-top:.5em}ul.pagination li{padding-right:.75em}ul.pagination li.disabled a,ul.pagination li.active a{color:#6c757d;text-decoration:none}ul.pagination li:last-of-type{padding-right:0}.listing-actions-group{display:flex}.quarto-listing-filter{margin-bottom:1em;width:200px;margin-left:auto}.quarto-listing-sort{margin-bottom:1em;margin-right:auto;width:auto}.quarto-listing-sort .input-group-text{font-size:.8em}.input-group-text{border-right:none}.quarto-listing-sort select.form-select{font-size:.8em}.listing-no-matching{text-align:center;padding-top:2em;padding-bottom:3em;font-size:1em}#quarto-margin-sidebar .quarto-listing-category{padding-top:0;font-size:1rem}#quarto-margin-sidebar .quarto-listing-category-title{cursor:pointer;font-weight:600;font-size:1rem}.quarto-listing-category .category{cursor:pointer}.quarto-listing-category .category.active{font-weight:600}.quarto-listing-category.category-cloud{display:flex;flex-wrap:wrap;align-items:baseline}.quarto-listing-category.category-cloud .category{padding-right:5px}.quarto-listing-category.category-cloud .category-cloud-1{font-size:.75em}.quarto-listing-category.category-cloud .category-cloud-2{font-size:.95em}.quarto-listing-category.category-cloud .category-cloud-3{font-size:1.15em}.quarto-listing-category.category-cloud .category-cloud-4{font-size:1.35em}.quarto-listing-category.category-cloud .category-cloud-5{font-size:1.55em}.quarto-listing-category.category-cloud .category-cloud-6{font-size:1.75em}.quarto-listing-category.category-cloud .category-cloud-7{font-size:1.95em}.quarto-listing-category.category-cloud .category-cloud-8{font-size:2.15em}.quarto-listing-category.category-cloud .category-cloud-9{font-size:2.35em}.quarto-listing-category.category-cloud .category-cloud-10{font-size:2.55em}.quarto-listing-cols-1{grid-template-columns:repeat(1, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-1{grid-template-columns:repeat(1, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-1{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-2{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-2{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-2{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-3{grid-template-columns:repeat(3, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-3{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-3{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-4{grid-template-columns:repeat(4, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-4{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-4{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-5{grid-template-columns:repeat(5, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-5{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-5{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-6{grid-template-columns:repeat(6, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-6{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-6{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-7{grid-template-columns:repeat(7, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-7{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-7{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-8{grid-template-columns:repeat(8, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-8{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-8{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-9{grid-template-columns:repeat(9, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-9{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-9{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-10{grid-template-columns:repeat(10, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-10{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-10{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-11{grid-template-columns:repeat(11, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-11{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-11{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-cols-12{grid-template-columns:repeat(12, minmax(0, 1fr));gap:1.5em}@media(max-width: 767.98px){.quarto-listing-cols-12{grid-template-columns:repeat(2, minmax(0, 1fr));gap:1.5em}}@media(max-width: 575.98px){.quarto-listing-cols-12{grid-template-columns:minmax(0, 1fr);gap:1.5em}}.quarto-listing-grid{gap:1.5em}.quarto-grid-item.borderless{border:none}.quarto-grid-item.borderless .listing-categories .listing-category:last-of-type,.quarto-grid-item.borderless .listing-categories .listing-category:first-of-type{padding-left:0}.quarto-grid-item.borderless .listing-categories .listing-category{border:0}.quarto-grid-link{text-decoration:none;color:inherit}.quarto-grid-link:hover{text-decoration:none;color:inherit}.quarto-grid-item h5.title,.quarto-grid-item .title.h5{margin-top:0;margin-bottom:0}.quarto-grid-item .card-footer{display:flex;justify-content:space-between;font-size:.8em}.quarto-grid-item .card-footer p{margin-bottom:0}.quarto-grid-item p.card-img-top{margin-bottom:0}.quarto-grid-item p.card-img-top>img{object-fit:cover}.quarto-grid-item .card-other-values{margin-top:.5em;font-size:.8em}.quarto-grid-item .card-other-values tr{margin-bottom:.5em}.quarto-grid-item .card-other-values tr>td:first-of-type{font-weight:600;padding-right:1em;padding-left:1em;vertical-align:top}.quarto-grid-item div.post-contents{display:flex;flex-direction:column;text-decoration:none;height:100%}.quarto-grid-item .listing-item-img-placeholder{background-color:rgba(52,58,64,.25);flex-shrink:0}.quarto-grid-item .card-attribution{padding-top:1em;display:flex;gap:1em;text-transform:uppercase;color:#6c757d;font-weight:500;flex-grow:10;align-items:flex-end}.quarto-grid-item .description{padding-bottom:1em}.quarto-grid-item .card-attribution .date{align-self:flex-end}.quarto-grid-item .card-attribution.justify{justify-content:space-between}.quarto-grid-item .card-attribution.start{justify-content:flex-start}.quarto-grid-item .card-attribution.end{justify-content:flex-end}.quarto-grid-item .card-title{margin-bottom:.1em}.quarto-grid-item .card-subtitle{padding-top:.25em}.quarto-grid-item .card-text{font-size:.9em}.quarto-grid-item .listing-reading-time{padding-bottom:.25em}.quarto-grid-item .card-text-small{font-size:.8em}.quarto-grid-item .card-subtitle.subtitle{font-size:.9em;font-weight:600;padding-bottom:.5em}.quarto-grid-item .listing-categories{display:flex;flex-wrap:wrap;padding-bottom:5px}.quarto-grid-item .listing-categories .listing-category{color:#6c757d;border:solid 1px #dee2e6;border-radius:.25rem;text-transform:uppercase;font-size:.65em;padding-left:.5em;padding-right:.5em;padding-top:.15em;padding-bottom:.15em;cursor:pointer;margin-right:4px;margin-bottom:4px}.quarto-grid-item.card-right{text-align:right}.quarto-grid-item.card-right .listing-categories{justify-content:flex-end}.quarto-grid-item.card-left{text-align:left}.quarto-grid-item.card-center{text-align:center}.quarto-grid-item.card-center .listing-description{text-align:justify}.quarto-grid-item.card-center .listing-categories{justify-content:center}table.quarto-listing-table td.image{padding:0px}table.quarto-listing-table td.image img{width:100%;max-width:50px;object-fit:contain}table.quarto-listing-table a{text-decoration:none;word-break:keep-all}table.quarto-listing-table th a{color:inherit}table.quarto-listing-table th a.asc:after{margin-bottom:-2px;margin-left:5px;display:inline-block;height:1rem;width:1rem;background-repeat:no-repeat;background-size:1rem 1rem;background-image:url('data:image/svg+xml,');content:""}table.quarto-listing-table th a.desc:after{margin-bottom:-2px;margin-left:5px;display:inline-block;height:1rem;width:1rem;background-repeat:no-repeat;background-size:1rem 1rem;background-image:url('data:image/svg+xml,');content:""}table.quarto-listing-table.table-hover td{cursor:pointer}.quarto-post.image-left{flex-direction:row}.quarto-post.image-right{flex-direction:row-reverse}@media(max-width: 767.98px){.quarto-post.image-right,.quarto-post.image-left{gap:0em;flex-direction:column}.quarto-post .metadata{padding-bottom:1em;order:2}.quarto-post .body{order:1}.quarto-post .thumbnail{order:3}}.list.quarto-listing-default div:last-of-type{border-bottom:none}@media(min-width: 992px){.quarto-listing-container-default{margin-right:2em}}div.quarto-post{display:flex;gap:2em;margin-bottom:1.5em;border-bottom:1px solid #dee2e6}@media(max-width: 767.98px){div.quarto-post{padding-bottom:1em}}div.quarto-post .metadata{flex-basis:20%;flex-grow:0;margin-top:.2em;flex-shrink:10}div.quarto-post .thumbnail{flex-basis:30%;flex-grow:0;flex-shrink:0}div.quarto-post .thumbnail img{margin-top:.4em;width:100%;object-fit:cover}div.quarto-post .body{flex-basis:45%;flex-grow:1;flex-shrink:0}div.quarto-post .body h3.listing-title,div.quarto-post .body .listing-title.h3{margin-top:0px;margin-bottom:0px;border-bottom:none}div.quarto-post .body .listing-subtitle{font-size:.875em;margin-bottom:.5em;margin-top:.2em}div.quarto-post .body .description{font-size:.9em}div.quarto-post .body pre code{white-space:pre-wrap}div.quarto-post a{color:#343a40;text-decoration:none}div.quarto-post .metadata{display:flex;flex-direction:column;font-size:.8em;font-family:Roboto,-apple-system,BlinkMacSystemFont,"Segoe UI","Helvetica Neue",Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol";flex-basis:33%}div.quarto-post .listing-categories{display:flex;flex-wrap:wrap;padding-bottom:5px}div.quarto-post .listing-categories .listing-category{color:#6c757d;border:solid 1px #dee2e6;border-radius:.25rem;text-transform:uppercase;font-size:.65em;padding-left:.5em;padding-right:.5em;padding-top:.15em;padding-bottom:.15em;cursor:pointer;margin-right:4px;margin-bottom:4px}div.quarto-post .listing-description{margin-bottom:.5em}div.quarto-about-jolla{display:flex !important;flex-direction:column;align-items:center;margin-top:10%;padding-bottom:1em}div.quarto-about-jolla .about-image{object-fit:cover;margin-left:auto;margin-right:auto;margin-bottom:1.5em}div.quarto-about-jolla img.round{border-radius:50%}div.quarto-about-jolla img.rounded{border-radius:10px}div.quarto-about-jolla .quarto-title h1.title,div.quarto-about-jolla .quarto-title .title.h1{text-align:center}div.quarto-about-jolla .quarto-title .description{text-align:center}div.quarto-about-jolla h2,div.quarto-about-jolla .h2{border-bottom:none}div.quarto-about-jolla .about-sep{width:60%}div.quarto-about-jolla main{text-align:center}div.quarto-about-jolla .about-links{display:flex}@media(min-width: 992px){div.quarto-about-jolla .about-links{flex-direction:row;column-gap:.8em;row-gap:15px;flex-wrap:wrap}}@media(max-width: 991.98px){div.quarto-about-jolla .about-links{flex-direction:column;row-gap:1em;width:100%;padding-bottom:1.5em}}div.quarto-about-jolla .about-link{color:#626d78;text-decoration:none;border:solid 1px}@media(min-width: 992px){div.quarto-about-jolla .about-link{font-size:.8em;padding:.25em .5em;border-radius:4px}}@media(max-width: 991.98px){div.quarto-about-jolla .about-link{font-size:1.1em;padding:.5em .5em;text-align:center;border-radius:6px}}div.quarto-about-jolla .about-link:hover{color:#93c54b}div.quarto-about-jolla .about-link i.bi{margin-right:.15em}div.quarto-about-solana{display:flex !important;flex-direction:column;padding-top:3em !important;padding-bottom:1em}div.quarto-about-solana .about-entity{display:flex !important;align-items:start;justify-content:space-between}@media(min-width: 992px){div.quarto-about-solana .about-entity{flex-direction:row}}@media(max-width: 991.98px){div.quarto-about-solana .about-entity{flex-direction:column-reverse;align-items:center;text-align:center}}div.quarto-about-solana .about-entity .entity-contents{display:flex;flex-direction:column}@media(max-width: 767.98px){div.quarto-about-solana .about-entity .entity-contents{width:100%}}div.quarto-about-solana .about-entity .about-image{object-fit:cover}@media(max-width: 991.98px){div.quarto-about-solana .about-entity .about-image{margin-bottom:1.5em}}div.quarto-about-solana .about-entity img.round{border-radius:50%}div.quarto-about-solana .about-entity img.rounded{border-radius:10px}div.quarto-about-solana .about-entity .about-links{display:flex;justify-content:left;padding-bottom:1.2em}@media(min-width: 992px){div.quarto-about-solana .about-entity .about-links{flex-direction:row;column-gap:.8em;row-gap:15px;flex-wrap:wrap}}@media(max-width: 991.98px){div.quarto-about-solana .about-entity .about-links{flex-direction:column;row-gap:1em;width:100%;padding-bottom:1.5em}}div.quarto-about-solana .about-entity .about-link{color:#626d78;text-decoration:none;border:solid 1px}@media(min-width: 992px){div.quarto-about-solana .about-entity .about-link{font-size:.8em;padding:.25em .5em;border-radius:4px}}@media(max-width: 991.98px){div.quarto-about-solana .about-entity .about-link{font-size:1.1em;padding:.5em .5em;text-align:center;border-radius:6px}}div.quarto-about-solana .about-entity .about-link:hover{color:#93c54b}div.quarto-about-solana .about-entity .about-link i.bi{margin-right:.15em}div.quarto-about-solana .about-contents{padding-right:1.5em;flex-basis:0;flex-grow:1}div.quarto-about-solana .about-contents main.content{margin-top:0}div.quarto-about-solana .about-contents h2,div.quarto-about-solana .about-contents .h2{border-bottom:none}div.quarto-about-trestles{display:flex !important;flex-direction:row;padding-top:3em !important;padding-bottom:1em}@media(max-width: 991.98px){div.quarto-about-trestles{flex-direction:column;padding-top:0em !important}}div.quarto-about-trestles .about-entity{display:flex !important;flex-direction:column;align-items:center;text-align:center;padding-right:1em}@media(min-width: 992px){div.quarto-about-trestles .about-entity{flex:0 0 42%}}div.quarto-about-trestles .about-entity .about-image{object-fit:cover;margin-bottom:1.5em}div.quarto-about-trestles .about-entity img.round{border-radius:50%}div.quarto-about-trestles .about-entity img.rounded{border-radius:10px}div.quarto-about-trestles .about-entity .about-links{display:flex;justify-content:center}@media(min-width: 992px){div.quarto-about-trestles .about-entity .about-links{flex-direction:row;column-gap:.8em;row-gap:15px;flex-wrap:wrap}}@media(max-width: 991.98px){div.quarto-about-trestles .about-entity .about-links{flex-direction:column;row-gap:1em;width:100%;padding-bottom:1.5em}}div.quarto-about-trestles .about-entity .about-link{color:#626d78;text-decoration:none;border:solid 1px}@media(min-width: 992px){div.quarto-about-trestles .about-entity .about-link{font-size:.8em;padding:.25em .5em;border-radius:4px}}@media(max-width: 991.98px){div.quarto-about-trestles .about-entity .about-link{font-size:1.1em;padding:.5em .5em;text-align:center;border-radius:6px}}div.quarto-about-trestles .about-entity .about-link:hover{color:#93c54b}div.quarto-about-trestles .about-entity .about-link i.bi{margin-right:.15em}div.quarto-about-trestles .about-contents{flex-basis:0;flex-grow:1}div.quarto-about-trestles .about-contents h2,div.quarto-about-trestles .about-contents .h2{border-bottom:none}@media(min-width: 992px){div.quarto-about-trestles .about-contents{border-left:solid 1px #dee2e6;padding-left:1.5em}}div.quarto-about-trestles .about-contents main.content{margin-top:0}div.quarto-about-marquee{padding-bottom:1em}div.quarto-about-marquee .about-contents{display:flex;flex-direction:column}div.quarto-about-marquee .about-image{max-height:550px;margin-bottom:1.5em;object-fit:cover}div.quarto-about-marquee img.round{border-radius:50%}div.quarto-about-marquee img.rounded{border-radius:10px}div.quarto-about-marquee h2,div.quarto-about-marquee .h2{border-bottom:none}div.quarto-about-marquee .about-links{display:flex;justify-content:center;padding-top:1.5em}@media(min-width: 992px){div.quarto-about-marquee .about-links{flex-direction:row;column-gap:.8em;row-gap:15px;flex-wrap:wrap}}@media(max-width: 991.98px){div.quarto-about-marquee .about-links{flex-direction:column;row-gap:1em;width:100%;padding-bottom:1.5em}}div.quarto-about-marquee .about-link{color:#626d78;text-decoration:none;border:solid 1px}@media(min-width: 992px){div.quarto-about-marquee .about-link{font-size:.8em;padding:.25em .5em;border-radius:4px}}@media(max-width: 991.98px){div.quarto-about-marquee .about-link{font-size:1.1em;padding:.5em .5em;text-align:center;border-radius:6px}}div.quarto-about-marquee .about-link:hover{color:#93c54b}div.quarto-about-marquee .about-link i.bi{margin-right:.15em}@media(min-width: 992px){div.quarto-about-marquee .about-link{border:none}}div.quarto-about-broadside{display:flex;flex-direction:column;padding-bottom:1em}div.quarto-about-broadside .about-main{display:flex !important;padding-top:0 !important}@media(min-width: 992px){div.quarto-about-broadside .about-main{flex-direction:row;align-items:flex-start}}@media(max-width: 991.98px){div.quarto-about-broadside .about-main{flex-direction:column}}@media(max-width: 991.98px){div.quarto-about-broadside .about-main .about-entity{flex-shrink:0;width:100%;height:450px;margin-bottom:1.5em;background-size:cover;background-repeat:no-repeat}}@media(min-width: 992px){div.quarto-about-broadside .about-main .about-entity{flex:0 10 50%;margin-right:1.5em;width:100%;height:100%;background-size:100%;background-repeat:no-repeat}}div.quarto-about-broadside .about-main .about-contents{padding-top:14px;flex:0 0 50%}div.quarto-about-broadside h2,div.quarto-about-broadside .h2{border-bottom:none}div.quarto-about-broadside .about-sep{margin-top:1.5em;width:60%;align-self:center}div.quarto-about-broadside .about-links{display:flex;justify-content:center;column-gap:20px;padding-top:1.5em}@media(min-width: 992px){div.quarto-about-broadside .about-links{flex-direction:row;column-gap:.8em;row-gap:15px;flex-wrap:wrap}}@media(max-width: 991.98px){div.quarto-about-broadside .about-links{flex-direction:column;row-gap:1em;width:100%;padding-bottom:1.5em}}div.quarto-about-broadside .about-link{color:#626d78;text-decoration:none;border:solid 1px}@media(min-width: 992px){div.quarto-about-broadside .about-link{font-size:.8em;padding:.25em .5em;border-radius:4px}}@media(max-width: 991.98px){div.quarto-about-broadside .about-link{font-size:1.1em;padding:.5em .5em;text-align:center;border-radius:6px}}div.quarto-about-broadside .about-link:hover{color:#93c54b}div.quarto-about-broadside .about-link i.bi{margin-right:.15em}@media(min-width: 992px){div.quarto-about-broadside .about-link{border:none}}.tippy-box[data-theme~=quarto]{background-color:#fff;border:solid 1px #dee2e6;border-radius:.25rem;color:#343a40;font-size:.875rem}.tippy-box[data-theme~=quarto]>.tippy-backdrop{background-color:#fff}.tippy-box[data-theme~=quarto]>.tippy-arrow:after,.tippy-box[data-theme~=quarto]>.tippy-svg-arrow:after{content:"";position:absolute;z-index:-1}.tippy-box[data-theme~=quarto]>.tippy-arrow:after{border-color:rgba(0,0,0,0);border-style:solid}.tippy-box[data-placement^=top]>.tippy-arrow:before{bottom:-6px}.tippy-box[data-placement^=bottom]>.tippy-arrow:before{top:-6px}.tippy-box[data-placement^=right]>.tippy-arrow:before{left:-6px}.tippy-box[data-placement^=left]>.tippy-arrow:before{right:-6px}.tippy-box[data-theme~=quarto][data-placement^=top]>.tippy-arrow:before{border-top-color:#fff}.tippy-box[data-theme~=quarto][data-placement^=top]>.tippy-arrow:after{border-top-color:#dee2e6;border-width:7px 7px 0;top:17px;left:1px}.tippy-box[data-theme~=quarto][data-placement^=top]>.tippy-svg-arrow>svg{top:16px}.tippy-box[data-theme~=quarto][data-placement^=top]>.tippy-svg-arrow:after{top:17px}.tippy-box[data-theme~=quarto][data-placement^=bottom]>.tippy-arrow:before{border-bottom-color:#fff;bottom:16px}.tippy-box[data-theme~=quarto][data-placement^=bottom]>.tippy-arrow:after{border-bottom-color:#dee2e6;border-width:0 7px 7px;bottom:17px;left:1px}.tippy-box[data-theme~=quarto][data-placement^=bottom]>.tippy-svg-arrow>svg{bottom:15px}.tippy-box[data-theme~=quarto][data-placement^=bottom]>.tippy-svg-arrow:after{bottom:17px}.tippy-box[data-theme~=quarto][data-placement^=left]>.tippy-arrow:before{border-left-color:#fff}.tippy-box[data-theme~=quarto][data-placement^=left]>.tippy-arrow:after{border-left-color:#dee2e6;border-width:7px 0 7px 7px;left:17px;top:1px}.tippy-box[data-theme~=quarto][data-placement^=left]>.tippy-svg-arrow>svg{left:11px}.tippy-box[data-theme~=quarto][data-placement^=left]>.tippy-svg-arrow:after{left:12px}.tippy-box[data-theme~=quarto][data-placement^=right]>.tippy-arrow:before{border-right-color:#fff;right:16px}.tippy-box[data-theme~=quarto][data-placement^=right]>.tippy-arrow:after{border-width:7px 7px 7px 0;right:17px;top:1px;border-right-color:#dee2e6}.tippy-box[data-theme~=quarto][data-placement^=right]>.tippy-svg-arrow>svg{right:11px}.tippy-box[data-theme~=quarto][data-placement^=right]>.tippy-svg-arrow:after{right:12px}.tippy-box[data-theme~=quarto]>.tippy-svg-arrow{fill:#343a40}.tippy-box[data-theme~=quarto]>.tippy-svg-arrow:after{background-image:url(data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMTYiIGhlaWdodD0iNiIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48cGF0aCBkPSJNMCA2czEuNzk2LS4wMTMgNC42Ny0zLjYxNUM1Ljg1MS45IDYuOTMuMDA2IDggMGMxLjA3LS4wMDYgMi4xNDguODg3IDMuMzQzIDIuMzg1QzE0LjIzMyA2LjAwNSAxNiA2IDE2IDZIMHoiIGZpbGw9InJnYmEoMCwgOCwgMTYsIDAuMikiLz48L3N2Zz4=);background-size:16px 6px;width:16px;height:6px}.top-right{position:absolute;top:1em;right:1em}.visually-hidden{border:0;clip:rect(0 0 0 0);height:auto;margin:0;overflow:hidden;padding:0;position:absolute;width:1px;white-space:nowrap}.hidden{display:none !important}.zindex-bottom{z-index:-1 !important}figure.figure{display:block}.quarto-layout-panel{margin-bottom:1em}.quarto-layout-panel>figure{width:100%}.quarto-layout-panel>figure>figcaption,.quarto-layout-panel>.panel-caption{margin-top:10pt}.quarto-layout-panel>.table-caption{margin-top:0px}.table-caption p{margin-bottom:.5em}.quarto-layout-row{display:flex;flex-direction:row;align-items:flex-start}.quarto-layout-valign-top{align-items:flex-start}.quarto-layout-valign-bottom{align-items:flex-end}.quarto-layout-valign-center{align-items:center}.quarto-layout-cell{position:relative;margin-right:20px}.quarto-layout-cell:last-child{margin-right:0}.quarto-layout-cell figure,.quarto-layout-cell>p{margin:.2em}.quarto-layout-cell img{max-width:100%}.quarto-layout-cell .html-widget{width:100% !important}.quarto-layout-cell div figure p{margin:0}.quarto-layout-cell figure{display:block;margin-inline-start:0;margin-inline-end:0}.quarto-layout-cell table{display:inline-table}.quarto-layout-cell-subref figcaption,figure .quarto-layout-row figure figcaption{text-align:center;font-style:italic}.quarto-figure{position:relative;margin-bottom:1em}.quarto-figure>figure{width:100%;margin-bottom:0}.quarto-figure-left>figure>p,.quarto-figure-left>figure>div{text-align:left}.quarto-figure-center>figure>p,.quarto-figure-center>figure>div{text-align:center}.quarto-figure-right>figure>p,.quarto-figure-right>figure>div{text-align:right}.quarto-figure>figure>div.cell-annotation,.quarto-figure>figure>div code{text-align:left}figure>p:empty{display:none}figure>p:first-child{margin-top:0;margin-bottom:0}figure>figcaption.quarto-float-caption-bottom{margin-bottom:.5em}figure>figcaption.quarto-float-caption-top{margin-top:.5em}div[id^=tbl-]{position:relative}.quarto-figure>.anchorjs-link{position:absolute;top:.6em;right:.5em}div[id^=tbl-]>.anchorjs-link{position:absolute;top:.7em;right:.3em}.quarto-figure:hover>.anchorjs-link,div[id^=tbl-]:hover>.anchorjs-link,h2:hover>.anchorjs-link,.h2:hover>.anchorjs-link,h3:hover>.anchorjs-link,.h3:hover>.anchorjs-link,h4:hover>.anchorjs-link,.h4:hover>.anchorjs-link,h5:hover>.anchorjs-link,.h5:hover>.anchorjs-link,h6:hover>.anchorjs-link,.h6:hover>.anchorjs-link,.reveal-anchorjs-link>.anchorjs-link{opacity:1}#title-block-header{margin-block-end:1rem;position:relative;margin-top:-1px}#title-block-header .abstract{margin-block-start:1rem}#title-block-header .abstract .abstract-title{font-weight:600}#title-block-header a{text-decoration:none}#title-block-header .author,#title-block-header .date,#title-block-header .doi{margin-block-end:.2rem}#title-block-header .quarto-title-block>div{display:flex}#title-block-header .quarto-title-block>div>h1,#title-block-header .quarto-title-block>div>.h1{flex-grow:1}#title-block-header .quarto-title-block>div>button{flex-shrink:0;height:2.25rem;margin-top:0}@media(min-width: 992px){#title-block-header .quarto-title-block>div>button{margin-top:5px}}tr.header>th>p:last-of-type{margin-bottom:0px}table,table.table{margin-top:.5rem;margin-bottom:.5rem}caption,.table-caption{padding-top:.5rem;padding-bottom:.5rem;text-align:center}figure.quarto-float-tbl figcaption.quarto-float-caption-top{margin-top:.5rem;margin-bottom:.25rem;text-align:center}figure.quarto-float-tbl figcaption.quarto-float-caption-bottom{padding-top:.25rem;margin-bottom:.5rem;text-align:center}.utterances{max-width:none;margin-left:-8px}iframe{margin-bottom:1em}details{margin-bottom:1em}details[show]{margin-bottom:0}details>summary{color:#6c757d}details>summary>p:only-child{display:inline}pre.sourceCode,code.sourceCode{position:relative}dd code:not(.sourceCode),p code:not(.sourceCode){white-space:pre-wrap}code{white-space:pre}@media print{code{white-space:pre-wrap}}pre>code{display:block}pre>code.sourceCode{white-space:pre}pre>code.sourceCode>span>a:first-child::before{text-decoration:none}pre.code-overflow-wrap>code.sourceCode{white-space:pre-wrap}pre.code-overflow-scroll>code.sourceCode{white-space:pre}code a:any-link{color:inherit;text-decoration:none}code a:hover{color:inherit;text-decoration:underline}ul.task-list{padding-left:1em}[data-tippy-root]{display:inline-block}.tippy-content .footnote-back{display:none}.footnote-back{margin-left:.2em}.tippy-content{overflow-x:auto}.quarto-embedded-source-code{display:none}.quarto-unresolved-ref{font-weight:600}.quarto-cover-image{max-width:35%;float:right;margin-left:30px}.cell-output-display .widget-subarea{margin-bottom:1em}.cell-output-display:not(.no-overflow-x),.knitsql-table:not(.no-overflow-x){overflow-x:auto}.panel-input{margin-bottom:1em}.panel-input>div,.panel-input>div>div{display:inline-block;vertical-align:top;padding-right:12px}.panel-input>p:last-child{margin-bottom:0}.layout-sidebar{margin-bottom:1em}.layout-sidebar .tab-content{border:none}.tab-content>.page-columns.active{display:grid}div.sourceCode>iframe{width:100%;height:300px;margin-bottom:-0.5em}a{text-underline-offset:3px}div.ansi-escaped-output{font-family:monospace;display:block}/*! +* +* ansi colors from IPython notebook's +* +* we also add `bright-[color]-` synonyms for the `-[color]-intense` classes since +* that seems to be what ansi_up emits +* +*/.ansi-black-fg{color:#3e424d}.ansi-black-bg{background-color:#3e424d}.ansi-black-intense-black,.ansi-bright-black-fg{color:#282c36}.ansi-black-intense-black,.ansi-bright-black-bg{background-color:#282c36}.ansi-red-fg{color:#e75c58}.ansi-red-bg{background-color:#e75c58}.ansi-red-intense-red,.ansi-bright-red-fg{color:#b22b31}.ansi-red-intense-red,.ansi-bright-red-bg{background-color:#b22b31}.ansi-green-fg{color:#00a250}.ansi-green-bg{background-color:#00a250}.ansi-green-intense-green,.ansi-bright-green-fg{color:#007427}.ansi-green-intense-green,.ansi-bright-green-bg{background-color:#007427}.ansi-yellow-fg{color:#ddb62b}.ansi-yellow-bg{background-color:#ddb62b}.ansi-yellow-intense-yellow,.ansi-bright-yellow-fg{color:#b27d12}.ansi-yellow-intense-yellow,.ansi-bright-yellow-bg{background-color:#b27d12}.ansi-blue-fg{color:#208ffb}.ansi-blue-bg{background-color:#208ffb}.ansi-blue-intense-blue,.ansi-bright-blue-fg{color:#0065ca}.ansi-blue-intense-blue,.ansi-bright-blue-bg{background-color:#0065ca}.ansi-magenta-fg{color:#d160c4}.ansi-magenta-bg{background-color:#d160c4}.ansi-magenta-intense-magenta,.ansi-bright-magenta-fg{color:#a03196}.ansi-magenta-intense-magenta,.ansi-bright-magenta-bg{background-color:#a03196}.ansi-cyan-fg{color:#60c6c8}.ansi-cyan-bg{background-color:#60c6c8}.ansi-cyan-intense-cyan,.ansi-bright-cyan-fg{color:#258f8f}.ansi-cyan-intense-cyan,.ansi-bright-cyan-bg{background-color:#258f8f}.ansi-white-fg{color:#c5c1b4}.ansi-white-bg{background-color:#c5c1b4}.ansi-white-intense-white,.ansi-bright-white-fg{color:#a1a6b2}.ansi-white-intense-white,.ansi-bright-white-bg{background-color:#a1a6b2}.ansi-default-inverse-fg{color:#fff}.ansi-default-inverse-bg{background-color:#000}.ansi-bold{font-weight:bold}.ansi-underline{text-decoration:underline}:root{--quarto-body-bg: #fff;--quarto-body-color: #343a40;--quarto-text-muted: #6c757d;--quarto-border-color: #dee2e6;--quarto-border-width: 1px;--quarto-border-radius: 0.25rem}table.gt_table{color:var(--quarto-body-color);font-size:1em;width:100%;background-color:rgba(0,0,0,0);border-top-width:inherit;border-bottom-width:inherit;border-color:var(--quarto-border-color)}table.gt_table th.gt_column_spanner_outer{color:var(--quarto-body-color);background-color:rgba(0,0,0,0);border-top-width:inherit;border-bottom-width:inherit;border-color:var(--quarto-border-color)}table.gt_table th.gt_col_heading{color:var(--quarto-body-color);font-weight:bold;background-color:rgba(0,0,0,0)}table.gt_table thead.gt_col_headings{border-bottom:1px solid currentColor;border-top-width:inherit;border-top-color:var(--quarto-border-color)}table.gt_table thead.gt_col_headings:not(:first-child){border-top-width:1px;border-top-color:var(--quarto-border-color)}table.gt_table td.gt_row{border-bottom-width:1px;border-bottom-color:var(--quarto-border-color);border-top-width:0px}table.gt_table tbody.gt_table_body{border-top-width:1px;border-bottom-width:1px;border-bottom-color:var(--quarto-border-color);border-top-color:currentColor}div.columns{display:initial;gap:initial}div.column{display:inline-block;overflow-x:initial;vertical-align:top;width:50%}.code-annotation-tip-content{word-wrap:break-word}.code-annotation-container-hidden{display:none !important}dl.code-annotation-container-grid{display:grid;grid-template-columns:min-content auto}dl.code-annotation-container-grid dt{grid-column:1}dl.code-annotation-container-grid dd{grid-column:2}pre.sourceCode.code-annotation-code{padding-right:0}code.sourceCode .code-annotation-anchor{z-index:100;position:relative;float:right;background-color:rgba(0,0,0,0)}input[type=checkbox]{margin-right:.5ch}:root{--mermaid-bg-color: #fff;--mermaid-edge-color: #6c757d;--mermaid-node-fg-color: #343a40;--mermaid-fg-color: #343a40;--mermaid-fg-color--lighter: #4b545c;--mermaid-fg-color--lightest: #626d78;--mermaid-font-family: Roboto, -apple-system, BlinkMacSystemFont, Segoe UI, Helvetica Neue, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji, Segoe UI Symbol;--mermaid-label-bg-color: #fff;--mermaid-label-fg-color: #325d88;--mermaid-node-bg-color: rgba(50, 93, 136, 0.1);--mermaid-node-fg-color: #343a40}@media print{:root{font-size:11pt}#quarto-sidebar,#TOC,.nav-page{display:none}.page-columns .content{grid-column-start:page-start}.fixed-top{position:relative}.panel-caption,.figure-caption,figcaption{color:#666}}.code-copy-button{position:absolute;top:0;right:0;border:0;margin-top:5px;margin-right:5px;background-color:rgba(0,0,0,0);z-index:3}.code-copy-button:focus{outline:none}.code-copy-button-tooltip{font-size:.75em}pre.sourceCode:hover>.code-copy-button>.bi::before{display:inline-block;height:1rem;width:1rem;content:"";vertical-align:-0.125em;background-image:url('data:image/svg+xml,');background-repeat:no-repeat;background-size:1rem 1rem}pre.sourceCode:hover>.code-copy-button-checked>.bi::before{background-image:url('data:image/svg+xml,')}pre.sourceCode:hover>.code-copy-button:hover>.bi::before{background-image:url('data:image/svg+xml,')}pre.sourceCode:hover>.code-copy-button-checked:hover>.bi::before{background-image:url('data:image/svg+xml,')}main ol ol,main ul ul,main ol ul,main ul ol{margin-bottom:1em}ul>li:not(:has(>p))>ul,ol>li:not(:has(>p))>ul,ul>li:not(:has(>p))>ol,ol>li:not(:has(>p))>ol{margin-bottom:0}ul>li:not(:has(>p))>ul>li:has(>p),ol>li:not(:has(>p))>ul>li:has(>p),ul>li:not(:has(>p))>ol>li:has(>p),ol>li:not(:has(>p))>ol>li:has(>p){margin-top:1rem}body{margin:0}main.page-columns>header>h1.title,main.page-columns>header>.title.h1{margin-bottom:0}@media(min-width: 992px){body .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset] 35px [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(500px, calc(850px - 3em)) [body-content-end] 1.5em [body-end] 35px [body-end-outset] minmax(75px, 145px) [page-end-inset] 35px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.fullcontent:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset] 35px [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(500px, calc(850px - 3em)) [body-content-end] 1.5em [body-end] 35px [body-end-outset] 35px [page-end-inset page-end] 5fr [screen-end-inset] 1.5em}body.slimcontent:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset] 35px [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(500px, calc(850px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(0px, 200px) [page-end-inset] 35px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.listing:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start] minmax(50px, 100px) [page-start-inset] 50px [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(500px, calc(850px - 3em)) [body-content-end] 3em [body-end] 50px [body-end-outset] minmax(0px, 250px) [page-end-inset] minmax(50px, 100px) [page-end] 1fr [screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 35px [page-start-inset] minmax(0px, 175px) [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(450px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(0px, 200px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 35px [page-start-inset] minmax(0px, 175px) [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(450px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(0px, 200px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] minmax(25px, 50px) [page-start-inset] minmax(50px, 150px) [body-start-outset] minmax(25px, 50px) [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] minmax(25px, 50px) [body-end-outset] minmax(50px, 150px) [page-end-inset] minmax(25px, 50px) [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start] minmax(50px, 100px) [page-start-inset] 50px [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(500px, calc(1000px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(50px, 100px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked.fullcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start] minmax(50px, 100px) [page-start-inset] 50px [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(500px, calc(1000px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating.fullcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 50px [page-start-inset] minmax(50px, 150px) [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked.slimcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start] minmax(50px, 100px) [page-start-inset] 50px [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(450px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(0px, 200px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked.listing .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start] minmax(50px, 100px) [page-start-inset] 50px [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(500px, calc(1000px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(0px, 200px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating.slimcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 50px [page-start-inset] minmax(50px, 150px) [body-start-outset] 50px [body-start] 1.5em [body-content-start] minmax(450px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(50px, 150px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating.listing .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] minmax(25px, 50px) [page-start-inset] minmax(50px, 150px) [body-start-outset] minmax(25px, 50px) [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] minmax(25px, 50px) [body-end-outset] minmax(50px, 150px) [page-end-inset] minmax(25px, 50px) [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}}@media(max-width: 991.98px){body .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset] 5fr [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] 35px [body-end-outset] minmax(75px, 145px) [page-end-inset] 35px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.fullcontent:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset] 5fr [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.slimcontent:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset] 5fr [body-start] 1.5em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end] 35px [body-end-outset] minmax(75px, 145px) [page-end-inset] 35px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.listing:not(.floating):not(.docked) .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset] 5fr [body-start] 1.5em [body-content-start] minmax(500px, calc(1250px - 3em)) [body-content-end body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 35px [page-start-inset] minmax(0px, 145px) [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(450px, calc(800px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start] 35px [page-start-inset] minmax(0px, 145px) [body-start-outset] 35px [body-start] 1.5em [body-content-start] minmax(450px, calc(800px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset body-start-outset body-start] 1.5em [body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(75px, 150px) [page-end-inset] 25px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(25px, 50px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked.fullcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(500px, calc(1000px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating.fullcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset body-start-outset body-start] 1em [body-content-start] minmax(500px, calc(800px - 3em)) [body-content-end] 1.5em [body-end body-end-outset page-end-inset page-end] 4fr [screen-end-inset] 1.5em [screen-end]}body.docked.slimcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(25px, 50px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.docked.listing .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(25px, 50px) [page-end-inset] 50px [page-end] 5fr [screen-end-inset] 1.5em [screen-end]}body.floating.slimcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset body-start-outset body-start] 1em [body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 35px [body-end-outset] minmax(75px, 145px) [page-end-inset] 35px [page-end] 4fr [screen-end-inset] 1.5em [screen-end]}body.floating.listing .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset] 5fr [page-start page-start-inset body-start-outset body-start] 1em [body-content-start] minmax(500px, calc(750px - 3em)) [body-content-end] 1.5em [body-end] 50px [body-end-outset] minmax(75px, 150px) [page-end-inset] 25px [page-end] 4fr [screen-end-inset] 1.5em [screen-end]}}@media(max-width: 767.98px){body .page-columns,body.fullcontent:not(.floating):not(.docked) .page-columns,body.slimcontent:not(.floating):not(.docked) .page-columns,body.docked .page-columns,body.docked.slimcontent .page-columns,body.docked.fullcontent .page-columns,body.floating .page-columns,body.floating.slimcontent .page-columns,body.floating.fullcontent .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(0px, 1fr) [body-content-end body-end body-end-outset page-end-inset page-end screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(0px, 1fr) [body-content-end body-end body-end-outset page-end-inset page-end screen-end-inset] 1.5em [screen-end]}body:not(.floating):not(.docked) .page-columns.toc-left .page-columns{display:grid;gap:0;grid-template-columns:[screen-start] 1.5em [screen-start-inset page-start page-start-inset body-start-outset body-start body-content-start] minmax(0px, 1fr) [body-content-end body-end body-end-outset page-end-inset page-end screen-end-inset] 1.5em [screen-end]}nav[role=doc-toc]{display:none}}body,.page-row-navigation{grid-template-rows:[page-top] max-content [contents-top] max-content [contents-bottom] max-content [page-bottom]}.page-rows-contents{grid-template-rows:[content-top] minmax(max-content, 1fr) [content-bottom] minmax(60px, max-content) [page-bottom]}.page-full{grid-column:screen-start/screen-end !important}.page-columns>*{grid-column:body-content-start/body-content-end}.page-columns.column-page>*{grid-column:page-start/page-end}.page-columns.column-page-left .page-columns.page-full>*,.page-columns.column-page-left>*{grid-column:page-start/body-content-end}.page-columns.column-page-right .page-columns.page-full>*,.page-columns.column-page-right>*{grid-column:body-content-start/page-end}.page-rows{grid-auto-rows:auto}.header{grid-column:screen-start/screen-end;grid-row:page-top/contents-top}#quarto-content{padding:0;grid-column:screen-start/screen-end;grid-row:contents-top/contents-bottom}body.floating .sidebar.sidebar-navigation{grid-column:page-start/body-start;grid-row:content-top/page-bottom}body.docked .sidebar.sidebar-navigation{grid-column:screen-start/body-start;grid-row:content-top/page-bottom}.sidebar.toc-left{grid-column:page-start/body-start;grid-row:content-top/page-bottom}.sidebar.margin-sidebar{grid-column:body-end/page-end;grid-row:content-top/page-bottom}.page-columns .content{grid-column:body-content-start/body-content-end;grid-row:content-top/content-bottom;align-content:flex-start}.page-columns .page-navigation{grid-column:body-content-start/body-content-end;grid-row:content-bottom/page-bottom}.page-columns .footer{grid-column:screen-start/screen-end;grid-row:contents-bottom/page-bottom}.page-columns .column-body{grid-column:body-content-start/body-content-end}.page-columns .column-body-fullbleed{grid-column:body-start/body-end}.page-columns .column-body-outset{grid-column:body-start-outset/body-end-outset;z-index:998;opacity:.999}.page-columns .column-body-outset table{background:#fff}.page-columns .column-body-outset-left{grid-column:body-start-outset/body-content-end;z-index:998;opacity:.999}.page-columns .column-body-outset-left table{background:#fff}.page-columns .column-body-outset-right{grid-column:body-content-start/body-end-outset;z-index:998;opacity:.999}.page-columns .column-body-outset-right table{background:#fff}.page-columns .column-page{grid-column:page-start/page-end;z-index:998;opacity:.999}.page-columns .column-page table{background:#fff}.page-columns .column-page-inset{grid-column:page-start-inset/page-end-inset;z-index:998;opacity:.999}.page-columns .column-page-inset table{background:#fff}.page-columns .column-page-inset-left{grid-column:page-start-inset/body-content-end;z-index:998;opacity:.999}.page-columns .column-page-inset-left table{background:#fff}.page-columns .column-page-inset-right{grid-column:body-content-start/page-end-inset;z-index:998;opacity:.999}.page-columns .column-page-inset-right figcaption table{background:#fff}.page-columns .column-page-left{grid-column:page-start/body-content-end;z-index:998;opacity:.999}.page-columns .column-page-left table{background:#fff}.page-columns .column-page-right{grid-column:body-content-start/page-end;z-index:998;opacity:.999}.page-columns .column-page-right figcaption table{background:#fff}#quarto-content.page-columns #quarto-margin-sidebar,#quarto-content.page-columns #quarto-sidebar{z-index:1}@media(max-width: 991.98px){#quarto-content.page-columns #quarto-margin-sidebar.collapse,#quarto-content.page-columns #quarto-sidebar.collapse,#quarto-content.page-columns #quarto-margin-sidebar.collapsing,#quarto-content.page-columns #quarto-sidebar.collapsing{z-index:1055}}#quarto-content.page-columns main.column-page,#quarto-content.page-columns main.column-page-right,#quarto-content.page-columns main.column-page-left{z-index:0}.page-columns .column-screen-inset{grid-column:screen-start-inset/screen-end-inset;z-index:998;opacity:.999}.page-columns .column-screen-inset table{background:#fff}.page-columns .column-screen-inset-left{grid-column:screen-start-inset/body-content-end;z-index:998;opacity:.999}.page-columns .column-screen-inset-left table{background:#fff}.page-columns .column-screen-inset-right{grid-column:body-content-start/screen-end-inset;z-index:998;opacity:.999}.page-columns .column-screen-inset-right table{background:#fff}.page-columns .column-screen{grid-column:screen-start/screen-end;z-index:998;opacity:.999}.page-columns .column-screen table{background:#fff}.page-columns .column-screen-left{grid-column:screen-start/body-content-end;z-index:998;opacity:.999}.page-columns .column-screen-left table{background:#fff}.page-columns .column-screen-right{grid-column:body-content-start/screen-end;z-index:998;opacity:.999}.page-columns .column-screen-right table{background:#fff}.page-columns .column-screen-inset-shaded{grid-column:screen-start/screen-end;padding:1em;background:#f8f5f0;z-index:998;opacity:.999;margin-bottom:1em}.zindex-content{z-index:998;opacity:.999}.zindex-modal{z-index:1055;opacity:.999}.zindex-over-content{z-index:999;opacity:.999}img.img-fluid.column-screen,img.img-fluid.column-screen-inset-shaded,img.img-fluid.column-screen-inset,img.img-fluid.column-screen-inset-left,img.img-fluid.column-screen-inset-right,img.img-fluid.column-screen-left,img.img-fluid.column-screen-right{width:100%}@media(min-width: 992px){.margin-caption,div.aside,aside:not(.footnotes):not(.sidebar),.column-margin{grid-column:body-end/page-end !important;z-index:998}.column-sidebar{grid-column:page-start/body-start !important;z-index:998}.column-leftmargin{grid-column:screen-start-inset/body-start !important;z-index:998}.no-row-height{height:1em;overflow:visible}}@media(max-width: 991.98px){.margin-caption,div.aside,aside:not(.footnotes):not(.sidebar),.column-margin{grid-column:body-end/page-end !important;z-index:998}.no-row-height{height:1em;overflow:visible}.page-columns.page-full{overflow:visible}.page-columns.toc-left .margin-caption,.page-columns.toc-left div.aside,.page-columns.toc-left aside:not(.footnotes):not(.sidebar),.page-columns.toc-left .column-margin{grid-column:body-content-start/body-content-end !important;z-index:998;opacity:.999}.page-columns.toc-left .no-row-height{height:initial;overflow:initial}}@media(max-width: 767.98px){.margin-caption,div.aside,aside:not(.footnotes):not(.sidebar),.column-margin{grid-column:body-content-start/body-content-end !important;z-index:998;opacity:.999}.no-row-height{height:initial;overflow:initial}#quarto-margin-sidebar{display:none}#quarto-sidebar-toc-left{display:none}.hidden-sm{display:none}}.panel-grid{display:grid;grid-template-rows:repeat(1, 1fr);grid-template-columns:repeat(24, 1fr);gap:1em}.panel-grid .g-col-1{grid-column:auto/span 1}.panel-grid .g-col-2{grid-column:auto/span 2}.panel-grid .g-col-3{grid-column:auto/span 3}.panel-grid .g-col-4{grid-column:auto/span 4}.panel-grid .g-col-5{grid-column:auto/span 5}.panel-grid .g-col-6{grid-column:auto/span 6}.panel-grid .g-col-7{grid-column:auto/span 7}.panel-grid .g-col-8{grid-column:auto/span 8}.panel-grid .g-col-9{grid-column:auto/span 9}.panel-grid .g-col-10{grid-column:auto/span 10}.panel-grid .g-col-11{grid-column:auto/span 11}.panel-grid .g-col-12{grid-column:auto/span 12}.panel-grid .g-col-13{grid-column:auto/span 13}.panel-grid .g-col-14{grid-column:auto/span 14}.panel-grid .g-col-15{grid-column:auto/span 15}.panel-grid .g-col-16{grid-column:auto/span 16}.panel-grid .g-col-17{grid-column:auto/span 17}.panel-grid .g-col-18{grid-column:auto/span 18}.panel-grid .g-col-19{grid-column:auto/span 19}.panel-grid .g-col-20{grid-column:auto/span 20}.panel-grid .g-col-21{grid-column:auto/span 21}.panel-grid .g-col-22{grid-column:auto/span 22}.panel-grid .g-col-23{grid-column:auto/span 23}.panel-grid .g-col-24{grid-column:auto/span 24}.panel-grid .g-start-1{grid-column-start:1}.panel-grid .g-start-2{grid-column-start:2}.panel-grid .g-start-3{grid-column-start:3}.panel-grid .g-start-4{grid-column-start:4}.panel-grid .g-start-5{grid-column-start:5}.panel-grid .g-start-6{grid-column-start:6}.panel-grid .g-start-7{grid-column-start:7}.panel-grid .g-start-8{grid-column-start:8}.panel-grid .g-start-9{grid-column-start:9}.panel-grid .g-start-10{grid-column-start:10}.panel-grid .g-start-11{grid-column-start:11}.panel-grid .g-start-12{grid-column-start:12}.panel-grid .g-start-13{grid-column-start:13}.panel-grid .g-start-14{grid-column-start:14}.panel-grid .g-start-15{grid-column-start:15}.panel-grid .g-start-16{grid-column-start:16}.panel-grid .g-start-17{grid-column-start:17}.panel-grid .g-start-18{grid-column-start:18}.panel-grid .g-start-19{grid-column-start:19}.panel-grid .g-start-20{grid-column-start:20}.panel-grid .g-start-21{grid-column-start:21}.panel-grid .g-start-22{grid-column-start:22}.panel-grid .g-start-23{grid-column-start:23}@media(min-width: 576px){.panel-grid .g-col-sm-1{grid-column:auto/span 1}.panel-grid .g-col-sm-2{grid-column:auto/span 2}.panel-grid .g-col-sm-3{grid-column:auto/span 3}.panel-grid .g-col-sm-4{grid-column:auto/span 4}.panel-grid .g-col-sm-5{grid-column:auto/span 5}.panel-grid .g-col-sm-6{grid-column:auto/span 6}.panel-grid .g-col-sm-7{grid-column:auto/span 7}.panel-grid .g-col-sm-8{grid-column:auto/span 8}.panel-grid .g-col-sm-9{grid-column:auto/span 9}.panel-grid .g-col-sm-10{grid-column:auto/span 10}.panel-grid .g-col-sm-11{grid-column:auto/span 11}.panel-grid .g-col-sm-12{grid-column:auto/span 12}.panel-grid .g-col-sm-13{grid-column:auto/span 13}.panel-grid .g-col-sm-14{grid-column:auto/span 14}.panel-grid .g-col-sm-15{grid-column:auto/span 15}.panel-grid .g-col-sm-16{grid-column:auto/span 16}.panel-grid .g-col-sm-17{grid-column:auto/span 17}.panel-grid .g-col-sm-18{grid-column:auto/span 18}.panel-grid .g-col-sm-19{grid-column:auto/span 19}.panel-grid .g-col-sm-20{grid-column:auto/span 20}.panel-grid .g-col-sm-21{grid-column:auto/span 21}.panel-grid .g-col-sm-22{grid-column:auto/span 22}.panel-grid .g-col-sm-23{grid-column:auto/span 23}.panel-grid .g-col-sm-24{grid-column:auto/span 24}.panel-grid .g-start-sm-1{grid-column-start:1}.panel-grid .g-start-sm-2{grid-column-start:2}.panel-grid .g-start-sm-3{grid-column-start:3}.panel-grid .g-start-sm-4{grid-column-start:4}.panel-grid .g-start-sm-5{grid-column-start:5}.panel-grid .g-start-sm-6{grid-column-start:6}.panel-grid .g-start-sm-7{grid-column-start:7}.panel-grid .g-start-sm-8{grid-column-start:8}.panel-grid .g-start-sm-9{grid-column-start:9}.panel-grid .g-start-sm-10{grid-column-start:10}.panel-grid .g-start-sm-11{grid-column-start:11}.panel-grid .g-start-sm-12{grid-column-start:12}.panel-grid .g-start-sm-13{grid-column-start:13}.panel-grid .g-start-sm-14{grid-column-start:14}.panel-grid .g-start-sm-15{grid-column-start:15}.panel-grid .g-start-sm-16{grid-column-start:16}.panel-grid .g-start-sm-17{grid-column-start:17}.panel-grid .g-start-sm-18{grid-column-start:18}.panel-grid .g-start-sm-19{grid-column-start:19}.panel-grid .g-start-sm-20{grid-column-start:20}.panel-grid .g-start-sm-21{grid-column-start:21}.panel-grid .g-start-sm-22{grid-column-start:22}.panel-grid .g-start-sm-23{grid-column-start:23}}@media(min-width: 768px){.panel-grid .g-col-md-1{grid-column:auto/span 1}.panel-grid .g-col-md-2{grid-column:auto/span 2}.panel-grid .g-col-md-3{grid-column:auto/span 3}.panel-grid .g-col-md-4{grid-column:auto/span 4}.panel-grid .g-col-md-5{grid-column:auto/span 5}.panel-grid .g-col-md-6{grid-column:auto/span 6}.panel-grid .g-col-md-7{grid-column:auto/span 7}.panel-grid .g-col-md-8{grid-column:auto/span 8}.panel-grid .g-col-md-9{grid-column:auto/span 9}.panel-grid .g-col-md-10{grid-column:auto/span 10}.panel-grid .g-col-md-11{grid-column:auto/span 11}.panel-grid .g-col-md-12{grid-column:auto/span 12}.panel-grid .g-col-md-13{grid-column:auto/span 13}.panel-grid .g-col-md-14{grid-column:auto/span 14}.panel-grid .g-col-md-15{grid-column:auto/span 15}.panel-grid .g-col-md-16{grid-column:auto/span 16}.panel-grid .g-col-md-17{grid-column:auto/span 17}.panel-grid .g-col-md-18{grid-column:auto/span 18}.panel-grid .g-col-md-19{grid-column:auto/span 19}.panel-grid .g-col-md-20{grid-column:auto/span 20}.panel-grid .g-col-md-21{grid-column:auto/span 21}.panel-grid .g-col-md-22{grid-column:auto/span 22}.panel-grid .g-col-md-23{grid-column:auto/span 23}.panel-grid .g-col-md-24{grid-column:auto/span 24}.panel-grid .g-start-md-1{grid-column-start:1}.panel-grid .g-start-md-2{grid-column-start:2}.panel-grid .g-start-md-3{grid-column-start:3}.panel-grid .g-start-md-4{grid-column-start:4}.panel-grid .g-start-md-5{grid-column-start:5}.panel-grid .g-start-md-6{grid-column-start:6}.panel-grid .g-start-md-7{grid-column-start:7}.panel-grid .g-start-md-8{grid-column-start:8}.panel-grid .g-start-md-9{grid-column-start:9}.panel-grid .g-start-md-10{grid-column-start:10}.panel-grid .g-start-md-11{grid-column-start:11}.panel-grid .g-start-md-12{grid-column-start:12}.panel-grid .g-start-md-13{grid-column-start:13}.panel-grid .g-start-md-14{grid-column-start:14}.panel-grid .g-start-md-15{grid-column-start:15}.panel-grid .g-start-md-16{grid-column-start:16}.panel-grid .g-start-md-17{grid-column-start:17}.panel-grid .g-start-md-18{grid-column-start:18}.panel-grid .g-start-md-19{grid-column-start:19}.panel-grid .g-start-md-20{grid-column-start:20}.panel-grid .g-start-md-21{grid-column-start:21}.panel-grid .g-start-md-22{grid-column-start:22}.panel-grid .g-start-md-23{grid-column-start:23}}@media(min-width: 992px){.panel-grid .g-col-lg-1{grid-column:auto/span 1}.panel-grid .g-col-lg-2{grid-column:auto/span 2}.panel-grid .g-col-lg-3{grid-column:auto/span 3}.panel-grid .g-col-lg-4{grid-column:auto/span 4}.panel-grid .g-col-lg-5{grid-column:auto/span 5}.panel-grid .g-col-lg-6{grid-column:auto/span 6}.panel-grid .g-col-lg-7{grid-column:auto/span 7}.panel-grid .g-col-lg-8{grid-column:auto/span 8}.panel-grid .g-col-lg-9{grid-column:auto/span 9}.panel-grid .g-col-lg-10{grid-column:auto/span 10}.panel-grid .g-col-lg-11{grid-column:auto/span 11}.panel-grid .g-col-lg-12{grid-column:auto/span 12}.panel-grid .g-col-lg-13{grid-column:auto/span 13}.panel-grid .g-col-lg-14{grid-column:auto/span 14}.panel-grid .g-col-lg-15{grid-column:auto/span 15}.panel-grid .g-col-lg-16{grid-column:auto/span 16}.panel-grid .g-col-lg-17{grid-column:auto/span 17}.panel-grid .g-col-lg-18{grid-column:auto/span 18}.panel-grid .g-col-lg-19{grid-column:auto/span 19}.panel-grid .g-col-lg-20{grid-column:auto/span 20}.panel-grid .g-col-lg-21{grid-column:auto/span 21}.panel-grid .g-col-lg-22{grid-column:auto/span 22}.panel-grid .g-col-lg-23{grid-column:auto/span 23}.panel-grid .g-col-lg-24{grid-column:auto/span 24}.panel-grid .g-start-lg-1{grid-column-start:1}.panel-grid .g-start-lg-2{grid-column-start:2}.panel-grid .g-start-lg-3{grid-column-start:3}.panel-grid .g-start-lg-4{grid-column-start:4}.panel-grid .g-start-lg-5{grid-column-start:5}.panel-grid .g-start-lg-6{grid-column-start:6}.panel-grid .g-start-lg-7{grid-column-start:7}.panel-grid .g-start-lg-8{grid-column-start:8}.panel-grid .g-start-lg-9{grid-column-start:9}.panel-grid .g-start-lg-10{grid-column-start:10}.panel-grid .g-start-lg-11{grid-column-start:11}.panel-grid .g-start-lg-12{grid-column-start:12}.panel-grid .g-start-lg-13{grid-column-start:13}.panel-grid .g-start-lg-14{grid-column-start:14}.panel-grid .g-start-lg-15{grid-column-start:15}.panel-grid .g-start-lg-16{grid-column-start:16}.panel-grid .g-start-lg-17{grid-column-start:17}.panel-grid .g-start-lg-18{grid-column-start:18}.panel-grid .g-start-lg-19{grid-column-start:19}.panel-grid .g-start-lg-20{grid-column-start:20}.panel-grid .g-start-lg-21{grid-column-start:21}.panel-grid .g-start-lg-22{grid-column-start:22}.panel-grid .g-start-lg-23{grid-column-start:23}}@media(min-width: 1200px){.panel-grid .g-col-xl-1{grid-column:auto/span 1}.panel-grid .g-col-xl-2{grid-column:auto/span 2}.panel-grid .g-col-xl-3{grid-column:auto/span 3}.panel-grid .g-col-xl-4{grid-column:auto/span 4}.panel-grid .g-col-xl-5{grid-column:auto/span 5}.panel-grid .g-col-xl-6{grid-column:auto/span 6}.panel-grid .g-col-xl-7{grid-column:auto/span 7}.panel-grid .g-col-xl-8{grid-column:auto/span 8}.panel-grid .g-col-xl-9{grid-column:auto/span 9}.panel-grid .g-col-xl-10{grid-column:auto/span 10}.panel-grid .g-col-xl-11{grid-column:auto/span 11}.panel-grid .g-col-xl-12{grid-column:auto/span 12}.panel-grid .g-col-xl-13{grid-column:auto/span 13}.panel-grid .g-col-xl-14{grid-column:auto/span 14}.panel-grid .g-col-xl-15{grid-column:auto/span 15}.panel-grid .g-col-xl-16{grid-column:auto/span 16}.panel-grid .g-col-xl-17{grid-column:auto/span 17}.panel-grid .g-col-xl-18{grid-column:auto/span 18}.panel-grid .g-col-xl-19{grid-column:auto/span 19}.panel-grid .g-col-xl-20{grid-column:auto/span 20}.panel-grid .g-col-xl-21{grid-column:auto/span 21}.panel-grid .g-col-xl-22{grid-column:auto/span 22}.panel-grid .g-col-xl-23{grid-column:auto/span 23}.panel-grid .g-col-xl-24{grid-column:auto/span 24}.panel-grid .g-start-xl-1{grid-column-start:1}.panel-grid .g-start-xl-2{grid-column-start:2}.panel-grid .g-start-xl-3{grid-column-start:3}.panel-grid .g-start-xl-4{grid-column-start:4}.panel-grid .g-start-xl-5{grid-column-start:5}.panel-grid .g-start-xl-6{grid-column-start:6}.panel-grid .g-start-xl-7{grid-column-start:7}.panel-grid .g-start-xl-8{grid-column-start:8}.panel-grid .g-start-xl-9{grid-column-start:9}.panel-grid .g-start-xl-10{grid-column-start:10}.panel-grid .g-start-xl-11{grid-column-start:11}.panel-grid .g-start-xl-12{grid-column-start:12}.panel-grid .g-start-xl-13{grid-column-start:13}.panel-grid .g-start-xl-14{grid-column-start:14}.panel-grid .g-start-xl-15{grid-column-start:15}.panel-grid .g-start-xl-16{grid-column-start:16}.panel-grid .g-start-xl-17{grid-column-start:17}.panel-grid .g-start-xl-18{grid-column-start:18}.panel-grid .g-start-xl-19{grid-column-start:19}.panel-grid .g-start-xl-20{grid-column-start:20}.panel-grid .g-start-xl-21{grid-column-start:21}.panel-grid .g-start-xl-22{grid-column-start:22}.panel-grid .g-start-xl-23{grid-column-start:23}}@media(min-width: 1400px){.panel-grid .g-col-xxl-1{grid-column:auto/span 1}.panel-grid .g-col-xxl-2{grid-column:auto/span 2}.panel-grid .g-col-xxl-3{grid-column:auto/span 3}.panel-grid .g-col-xxl-4{grid-column:auto/span 4}.panel-grid .g-col-xxl-5{grid-column:auto/span 5}.panel-grid .g-col-xxl-6{grid-column:auto/span 6}.panel-grid .g-col-xxl-7{grid-column:auto/span 7}.panel-grid .g-col-xxl-8{grid-column:auto/span 8}.panel-grid .g-col-xxl-9{grid-column:auto/span 9}.panel-grid .g-col-xxl-10{grid-column:auto/span 10}.panel-grid .g-col-xxl-11{grid-column:auto/span 11}.panel-grid .g-col-xxl-12{grid-column:auto/span 12}.panel-grid .g-col-xxl-13{grid-column:auto/span 13}.panel-grid .g-col-xxl-14{grid-column:auto/span 14}.panel-grid .g-col-xxl-15{grid-column:auto/span 15}.panel-grid .g-col-xxl-16{grid-column:auto/span 16}.panel-grid .g-col-xxl-17{grid-column:auto/span 17}.panel-grid .g-col-xxl-18{grid-column:auto/span 18}.panel-grid .g-col-xxl-19{grid-column:auto/span 19}.panel-grid .g-col-xxl-20{grid-column:auto/span 20}.panel-grid .g-col-xxl-21{grid-column:auto/span 21}.panel-grid .g-col-xxl-22{grid-column:auto/span 22}.panel-grid .g-col-xxl-23{grid-column:auto/span 23}.panel-grid .g-col-xxl-24{grid-column:auto/span 24}.panel-grid .g-start-xxl-1{grid-column-start:1}.panel-grid .g-start-xxl-2{grid-column-start:2}.panel-grid .g-start-xxl-3{grid-column-start:3}.panel-grid .g-start-xxl-4{grid-column-start:4}.panel-grid .g-start-xxl-5{grid-column-start:5}.panel-grid .g-start-xxl-6{grid-column-start:6}.panel-grid .g-start-xxl-7{grid-column-start:7}.panel-grid .g-start-xxl-8{grid-column-start:8}.panel-grid .g-start-xxl-9{grid-column-start:9}.panel-grid .g-start-xxl-10{grid-column-start:10}.panel-grid .g-start-xxl-11{grid-column-start:11}.panel-grid .g-start-xxl-12{grid-column-start:12}.panel-grid .g-start-xxl-13{grid-column-start:13}.panel-grid .g-start-xxl-14{grid-column-start:14}.panel-grid .g-start-xxl-15{grid-column-start:15}.panel-grid .g-start-xxl-16{grid-column-start:16}.panel-grid .g-start-xxl-17{grid-column-start:17}.panel-grid .g-start-xxl-18{grid-column-start:18}.panel-grid .g-start-xxl-19{grid-column-start:19}.panel-grid .g-start-xxl-20{grid-column-start:20}.panel-grid .g-start-xxl-21{grid-column-start:21}.panel-grid .g-start-xxl-22{grid-column-start:22}.panel-grid .g-start-xxl-23{grid-column-start:23}}main{margin-top:1em;margin-bottom:1em}h1,.h1,h2,.h2{color:inherit;margin-top:2rem;margin-bottom:1rem;font-weight:600}h1.title,.title.h1{margin-top:0}main.content>section:first-of-type>h2:first-child,main.content>section:first-of-type>.h2:first-child{margin-top:0}h2,.h2{border-bottom:1px solid #dee2e6;padding-bottom:.5rem}h3,.h3{font-weight:600}h3,.h3,h4,.h4{opacity:.9;margin-top:1.5rem}h5,.h5,h6,.h6{opacity:.9}.header-section-number{color:#6d7a86}.nav-link.active .header-section-number{color:inherit}mark,.mark{padding:0em}.panel-caption,.figure-caption,.subfigure-caption,.table-caption,figcaption,caption{font-size:.9rem;color:#6d7a86}.quarto-layout-cell[data-ref-parent] caption{color:#6d7a86}.column-margin figcaption,.margin-caption,div.aside,aside,.column-margin{color:#6d7a86;font-size:.825rem}.panel-caption.margin-caption{text-align:inherit}.column-margin.column-container p{margin-bottom:0}.column-margin.column-container>*:not(.collapse):first-child{padding-bottom:.5em;display:block}.column-margin.column-container>*:not(.collapse):not(:first-child){padding-top:.5em;padding-bottom:.5em;display:block}.column-margin.column-container>*.collapse:not(.show){display:none}@media(min-width: 768px){.column-margin.column-container .callout-margin-content:first-child{margin-top:4.5em}.column-margin.column-container .callout-margin-content-simple:first-child{margin-top:3.5em}}.margin-caption>*{padding-top:.5em;padding-bottom:.5em}@media(max-width: 767.98px){.quarto-layout-row{flex-direction:column}}.nav-tabs .nav-item{margin-top:1px;cursor:pointer}.tab-content{margin-top:0px;border-left:#dee2e6 1px solid;border-right:#dee2e6 1px solid;border-bottom:#dee2e6 1px solid;margin-left:0;padding:1em;margin-bottom:1em}@media(max-width: 767.98px){.layout-sidebar{margin-left:0;margin-right:0}}.panel-sidebar,.panel-sidebar .form-control,.panel-input,.panel-input .form-control,.selectize-dropdown{font-size:.9rem}.panel-sidebar .form-control,.panel-input .form-control{padding-top:.1rem}.tab-pane div.sourceCode{margin-top:0px}.tab-pane>p{padding-top:0}.tab-pane>p:nth-child(1){padding-top:0}.tab-pane>p:last-child{margin-bottom:0}.tab-pane>pre:last-child{margin-bottom:0}.tab-content>.tab-pane:not(.active){display:none !important}div.sourceCode{background-color:#f8f5f0;border:1px solid #f8f5f0;border-radius:.25rem}pre.sourceCode{background-color:rgba(0,0,0,0)}pre.sourceCode{border:none;font-size:.875em;overflow:visible !important;padding:.4em}.callout pre.sourceCode{padding-left:0}div.sourceCode{overflow-y:hidden}.callout div.sourceCode{margin-left:initial}.blockquote{font-size:inherit;padding-left:1rem;padding-right:1.5rem;color:#6d7a86}.blockquote h1:first-child,.blockquote .h1:first-child,.blockquote h2:first-child,.blockquote .h2:first-child,.blockquote h3:first-child,.blockquote .h3:first-child,.blockquote h4:first-child,.blockquote .h4:first-child,.blockquote h5:first-child,.blockquote .h5:first-child{margin-top:0}pre{background-color:initial;padding:initial;border:initial}p pre code:not(.sourceCode),li pre code:not(.sourceCode),pre code:not(.sourceCode){background-color:initial}p code:not(.sourceCode),li code:not(.sourceCode),td code:not(.sourceCode){background-color:#f8f5f0;padding:.2em}nav p code:not(.sourceCode),nav li code:not(.sourceCode),nav td code:not(.sourceCode){background-color:rgba(0,0,0,0);padding:0}td code:not(.sourceCode){white-space:pre-wrap}#quarto-embedded-source-code-modal>.modal-dialog{max-width:1000px;padding-left:1.75rem;padding-right:1.75rem}#quarto-embedded-source-code-modal>.modal-dialog>.modal-content>.modal-body{padding:0}#quarto-embedded-source-code-modal>.modal-dialog>.modal-content>.modal-body div.sourceCode{margin:0;padding:.2rem .2rem;border-radius:0px;border:none}#quarto-embedded-source-code-modal>.modal-dialog>.modal-content>.modal-header{padding:.7rem}.code-tools-button{font-size:1rem;padding:.15rem .15rem;margin-left:5px;color:#6c757d;background-color:rgba(0,0,0,0);transition:initial;cursor:pointer}.code-tools-button>.bi::before{display:inline-block;height:1rem;width:1rem;content:"";vertical-align:-0.125em;background-image:url('data:image/svg+xml,');background-repeat:no-repeat;background-size:1rem 1rem}.code-tools-button:hover>.bi::before{background-image:url('data:image/svg+xml,')}#quarto-embedded-source-code-modal .code-copy-button>.bi::before{background-image:url('data:image/svg+xml,')}#quarto-embedded-source-code-modal .code-copy-button-checked>.bi::before{background-image:url('data:image/svg+xml,')}.sidebar{will-change:top;transition:top 200ms linear;position:sticky;overflow-y:auto;padding-top:1.2em;max-height:100vh}.sidebar.toc-left,.sidebar.margin-sidebar{top:0px;padding-top:1em}.sidebar.quarto-banner-title-block-sidebar>*{padding-top:1.65em}figure .quarto-notebook-link{margin-top:.5em}.quarto-notebook-link{font-size:.75em;color:#6c757d;margin-bottom:1em;text-decoration:none;display:block}.quarto-notebook-link:hover{text-decoration:underline;color:#93c54b}.quarto-notebook-link::before{display:inline-block;height:.75rem;width:.75rem;margin-bottom:0em;margin-right:.25em;content:"";vertical-align:-0.125em;background-image:url('data:image/svg+xml,');background-repeat:no-repeat;background-size:.75rem .75rem}.toc-actions i.bi,.quarto-code-links i.bi,.quarto-other-links i.bi,.quarto-alternate-notebooks i.bi,.quarto-alternate-formats i.bi{margin-right:.4em;font-size:.8rem}.quarto-other-links-text-target .quarto-code-links i.bi,.quarto-other-links-text-target .quarto-other-links i.bi{margin-right:.2em}.quarto-other-formats-text-target .quarto-alternate-formats i.bi{margin-right:.1em}.toc-actions i.bi.empty,.quarto-code-links i.bi.empty,.quarto-other-links i.bi.empty,.quarto-alternate-notebooks i.bi.empty,.quarto-alternate-formats i.bi.empty{padding-left:1em}.quarto-notebook h2,.quarto-notebook .h2{border-bottom:none}.quarto-notebook .cell-container{display:flex}.quarto-notebook .cell-container .cell{flex-grow:4}.quarto-notebook .cell-container .cell-decorator{padding-top:1.5em;padding-right:1em;text-align:right}.quarto-notebook .cell-container.code-fold .cell-decorator{padding-top:3em}.quarto-notebook .cell-code code{white-space:pre-wrap}.quarto-notebook .cell .cell-output-stderr pre code,.quarto-notebook .cell .cell-output-stdout pre code{white-space:pre-wrap;overflow-wrap:anywhere}.toc-actions,.quarto-alternate-formats,.quarto-other-links,.quarto-code-links,.quarto-alternate-notebooks{padding-left:0em}.sidebar .toc-actions a,.sidebar .quarto-alternate-formats a,.sidebar .quarto-other-links a,.sidebar .quarto-code-links a,.sidebar .quarto-alternate-notebooks a,.sidebar nav[role=doc-toc] a{text-decoration:none}.sidebar .toc-actions a:hover,.sidebar .quarto-other-links a:hover,.sidebar .quarto-code-links a:hover,.sidebar .quarto-alternate-formats a:hover,.sidebar .quarto-alternate-notebooks a:hover{color:#93c54b}.sidebar .toc-actions h2,.sidebar .toc-actions .h2,.sidebar .quarto-code-links h2,.sidebar .quarto-code-links .h2,.sidebar .quarto-other-links h2,.sidebar .quarto-other-links .h2,.sidebar .quarto-alternate-notebooks h2,.sidebar .quarto-alternate-notebooks .h2,.sidebar .quarto-alternate-formats h2,.sidebar .quarto-alternate-formats .h2,.sidebar nav[role=doc-toc]>h2,.sidebar nav[role=doc-toc]>.h2{font-weight:500;margin-bottom:.2rem;margin-top:.3rem;font-family:inherit;border-bottom:0;padding-bottom:0;padding-top:0px}.sidebar .toc-actions>h2,.sidebar .toc-actions>.h2,.sidebar .quarto-code-links>h2,.sidebar .quarto-code-links>.h2,.sidebar .quarto-other-links>h2,.sidebar .quarto-other-links>.h2,.sidebar .quarto-alternate-notebooks>h2,.sidebar .quarto-alternate-notebooks>.h2,.sidebar .quarto-alternate-formats>h2,.sidebar .quarto-alternate-formats>.h2{font-size:.8rem}.sidebar nav[role=doc-toc]>h2,.sidebar nav[role=doc-toc]>.h2{font-size:.875rem}.sidebar nav[role=doc-toc]>ul a{border-left:1px solid #f8f5f0;padding-left:.6rem}.sidebar .toc-actions h2>ul a,.sidebar .toc-actions .h2>ul a,.sidebar .quarto-code-links h2>ul a,.sidebar .quarto-code-links .h2>ul a,.sidebar .quarto-other-links h2>ul a,.sidebar .quarto-other-links .h2>ul a,.sidebar .quarto-alternate-notebooks h2>ul a,.sidebar .quarto-alternate-notebooks .h2>ul a,.sidebar .quarto-alternate-formats h2>ul a,.sidebar .quarto-alternate-formats .h2>ul a{border-left:none;padding-left:.6rem}.sidebar .toc-actions ul a:empty,.sidebar .quarto-code-links ul a:empty,.sidebar .quarto-other-links ul a:empty,.sidebar .quarto-alternate-notebooks ul a:empty,.sidebar .quarto-alternate-formats ul a:empty,.sidebar nav[role=doc-toc]>ul a:empty{display:none}.sidebar .toc-actions ul,.sidebar .quarto-code-links ul,.sidebar .quarto-other-links ul,.sidebar .quarto-alternate-notebooks ul,.sidebar .quarto-alternate-formats ul{padding-left:0;list-style:none}.sidebar nav[role=doc-toc] ul{list-style:none;padding-left:0;list-style:none}.sidebar nav[role=doc-toc]>ul{margin-left:.45em}.quarto-margin-sidebar nav[role=doc-toc]{padding-left:.5em}.sidebar .toc-actions>ul,.sidebar .quarto-code-links>ul,.sidebar .quarto-other-links>ul,.sidebar .quarto-alternate-notebooks>ul,.sidebar .quarto-alternate-formats>ul{font-size:.8rem}.sidebar nav[role=doc-toc]>ul{font-size:.875rem}.sidebar .toc-actions ul li a,.sidebar .quarto-code-links ul li a,.sidebar .quarto-other-links ul li a,.sidebar .quarto-alternate-notebooks ul li a,.sidebar .quarto-alternate-formats ul li a,.sidebar nav[role=doc-toc]>ul li a{line-height:1.1rem;padding-bottom:.2rem;padding-top:.2rem;color:inherit}.sidebar nav[role=doc-toc] ul>li>ul>li>a{padding-left:1.2em}.sidebar nav[role=doc-toc] ul>li>ul>li>ul>li>a{padding-left:2.4em}.sidebar nav[role=doc-toc] ul>li>ul>li>ul>li>ul>li>a{padding-left:3.6em}.sidebar nav[role=doc-toc] ul>li>ul>li>ul>li>ul>li>ul>li>a{padding-left:4.8em}.sidebar nav[role=doc-toc] ul>li>ul>li>ul>li>ul>li>ul>li>ul>li>a{padding-left:6em}.sidebar nav[role=doc-toc] ul>li>a.active,.sidebar nav[role=doc-toc] ul>li>ul>li>a.active{border-left:1px solid #93c54b;color:#93c54b !important}.sidebar nav[role=doc-toc] ul>li>a:hover,.sidebar nav[role=doc-toc] ul>li>ul>li>a:hover{color:#93c54b !important}kbd,.kbd{color:#343a40;background-color:#f8f9fa;border:1px solid;border-radius:5px;border-color:#dee2e6}.quarto-appendix-contents div.hanging-indent{margin-left:0em}.quarto-appendix-contents div.hanging-indent div.csl-entry{margin-left:1em;text-indent:-1em}.citation a,.footnote-ref{text-decoration:none}.footnotes ol{padding-left:1em}.tippy-content>*{margin-bottom:.7em}.tippy-content>*:last-child{margin-bottom:0}.callout{margin-top:1.25rem;margin-bottom:1.25rem;border-radius:.25rem;overflow-wrap:break-word}.callout .callout-title-container{overflow-wrap:anywhere}.callout.callout-style-simple{padding:.4em .7em;border-left:5px solid;border-right:1px solid #dee2e6;border-top:1px solid #dee2e6;border-bottom:1px solid #dee2e6}.callout.callout-style-default{border-left:5px solid;border-right:1px solid #dee2e6;border-top:1px solid #dee2e6;border-bottom:1px solid #dee2e6}.callout .callout-body-container{flex-grow:1}.callout.callout-style-simple .callout-body{font-size:.9rem;font-weight:400}.callout.callout-style-default .callout-body{font-size:.9rem;font-weight:400}.callout:not(.no-icon).callout-titled.callout-style-simple .callout-body{padding-left:1.6em}.callout.callout-titled>.callout-header{padding-top:.2em;margin-bottom:-0.2em}.callout.callout-style-simple>div.callout-header{border-bottom:none;font-size:.9rem;font-weight:600;opacity:75%}.callout.callout-style-default>div.callout-header{border-bottom:none;font-weight:600;opacity:85%;font-size:.9rem;padding-left:.5em;padding-right:.5em}.callout.callout-style-default .callout-body{padding-left:.5em;padding-right:.5em}.callout.callout-style-default .callout-body>:first-child{padding-top:.5rem;margin-top:0}.callout>div.callout-header[data-bs-toggle=collapse]{cursor:pointer}.callout.callout-style-default .callout-header[aria-expanded=false],.callout.callout-style-default .callout-header[aria-expanded=true]{padding-top:0px;margin-bottom:0px;align-items:center}.callout.callout-titled .callout-body>:last-child:not(.sourceCode),.callout.callout-titled .callout-body>div>:last-child:not(.sourceCode){padding-bottom:.5rem;margin-bottom:0}.callout:not(.callout-titled) .callout-body>:first-child,.callout:not(.callout-titled) .callout-body>div>:first-child{margin-top:.25rem}.callout:not(.callout-titled) .callout-body>:last-child,.callout:not(.callout-titled) .callout-body>div>:last-child{margin-bottom:.2rem}.callout.callout-style-simple .callout-icon::before,.callout.callout-style-simple .callout-toggle::before{height:1rem;width:1rem;display:inline-block;content:"";background-repeat:no-repeat;background-size:1rem 1rem}.callout.callout-style-default .callout-icon::before,.callout.callout-style-default .callout-toggle::before{height:.9rem;width:.9rem;display:inline-block;content:"";background-repeat:no-repeat;background-size:.9rem .9rem}.callout.callout-style-default .callout-toggle::before{margin-top:5px}.callout .callout-btn-toggle .callout-toggle::before{transition:transform .2s linear}.callout .callout-header[aria-expanded=false] .callout-toggle::before{transform:rotate(-90deg)}.callout .callout-header[aria-expanded=true] .callout-toggle::before{transform:none}.callout.callout-style-simple:not(.no-icon) div.callout-icon-container{padding-top:.2em;padding-right:.55em}.callout.callout-style-default:not(.no-icon) div.callout-icon-container{padding-top:.1em;padding-right:.35em}.callout.callout-style-default:not(.no-icon) div.callout-title-container{margin-top:-1px}.callout.callout-style-default.callout-caution:not(.no-icon) div.callout-icon-container{padding-top:.3em;padding-right:.35em}.callout>.callout-body>.callout-icon-container>.no-icon,.callout>.callout-header>.callout-icon-container>.no-icon{display:none}div.callout.callout{border-left-color:#6c757d}div.callout.callout-style-default>.callout-header{background-color:#6c757d}div.callout-note.callout{border-left-color:#325d88}div.callout-note.callout-style-default>.callout-header{background-color:#ebeff3}div.callout-note:not(.callout-titled) .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-note.callout-titled .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-note .callout-toggle::before{background-image:url('data:image/svg+xml,')}div.callout-tip.callout{border-left-color:#93c54b}div.callout-tip.callout-style-default>.callout-header{background-color:#f4f9ed}div.callout-tip:not(.callout-titled) .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-tip.callout-titled .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-tip .callout-toggle::before{background-image:url('data:image/svg+xml,')}div.callout-warning.callout{border-left-color:#ffc107}div.callout-warning.callout-style-default>.callout-header{background-color:#fff9e6}div.callout-warning:not(.callout-titled) .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-warning.callout-titled .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-warning .callout-toggle::before{background-image:url('data:image/svg+xml,')}div.callout-caution.callout{border-left-color:#f47c3c}div.callout-caution.callout-style-default>.callout-header{background-color:#fef2ec}div.callout-caution:not(.callout-titled) .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-caution.callout-titled .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-caution .callout-toggle::before{background-image:url('data:image/svg+xml,')}div.callout-important.callout{border-left-color:#d9534f}div.callout-important.callout-style-default>.callout-header{background-color:#fbeeed}div.callout-important:not(.callout-titled) .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-important.callout-titled .callout-icon::before{background-image:url('data:image/svg+xml,');}div.callout-important .callout-toggle::before{background-image:url('data:image/svg+xml,')}.quarto-toggle-container{display:flex;align-items:center}.quarto-reader-toggle .bi::before,.quarto-color-scheme-toggle .bi::before{display:inline-block;height:1rem;width:1rem;content:"";background-repeat:no-repeat;background-size:1rem 1rem}.sidebar-navigation{padding-left:20px}.navbar{background-color:#325d88;color:rgba(255,255,255,.7)}.navbar .quarto-color-scheme-toggle:not(.alternate) .bi::before{background-image:url('data:image/svg+xml,')}.navbar .quarto-color-scheme-toggle.alternate .bi::before{background-image:url('data:image/svg+xml,')}.sidebar-navigation .quarto-color-scheme-toggle:not(.alternate) .bi::before{background-image:url('data:image/svg+xml,')}.sidebar-navigation .quarto-color-scheme-toggle.alternate .bi::before{background-image:url('data:image/svg+xml,')}.quarto-sidebar-toggle{border-color:#dee2e6;border-bottom-left-radius:.25rem;border-bottom-right-radius:.25rem;border-style:solid;border-width:1px;overflow:hidden;border-top-width:0px;padding-top:0px !important}.quarto-sidebar-toggle-title{cursor:pointer;padding-bottom:2px;margin-left:.25em;text-align:center;font-weight:400;font-size:.775em}#quarto-content .quarto-sidebar-toggle{background:#fafafa}#quarto-content .quarto-sidebar-toggle-title{color:#343a40}.quarto-sidebar-toggle-icon{color:#dee2e6;margin-right:.5em;float:right;transition:transform .2s ease}.quarto-sidebar-toggle-icon::before{padding-top:5px}.quarto-sidebar-toggle.expanded .quarto-sidebar-toggle-icon{transform:rotate(-180deg)}.quarto-sidebar-toggle.expanded .quarto-sidebar-toggle-title{border-bottom:solid #dee2e6 1px}.quarto-sidebar-toggle-contents{background-color:#fff;padding-right:10px;padding-left:10px;margin-top:0px !important;transition:max-height .5s ease}.quarto-sidebar-toggle.expanded .quarto-sidebar-toggle-contents{padding-top:1em;padding-bottom:10px}@media(max-width: 767.98px){.sidebar-menu-container{padding-bottom:5em}}.quarto-sidebar-toggle:not(.expanded) .quarto-sidebar-toggle-contents{padding-top:0px !important;padding-bottom:0px}nav[role=doc-toc]{z-index:1020}#quarto-sidebar>*,nav[role=doc-toc]>*{transition:opacity .1s ease,border .1s ease}#quarto-sidebar.slow>*,nav[role=doc-toc].slow>*{transition:opacity .4s ease,border .4s ease}.quarto-color-scheme-toggle:not(.alternate).top-right .bi::before{background-image:url('data:image/svg+xml,')}.quarto-color-scheme-toggle.alternate.top-right .bi::before{background-image:url('data:image/svg+xml,')}#quarto-appendix.default{border-top:1px solid #dee2e6}#quarto-appendix.default{background-color:#fff;padding-top:1.5em;margin-top:2em;z-index:998}#quarto-appendix.default .quarto-appendix-heading{margin-top:0;line-height:1.4em;font-weight:600;opacity:.9;border-bottom:none;margin-bottom:0}#quarto-appendix.default .footnotes ol,#quarto-appendix.default .footnotes ol li>p:last-of-type,#quarto-appendix.default .quarto-appendix-contents>p:last-of-type{margin-bottom:0}#quarto-appendix.default .footnotes ol{margin-left:.5em}#quarto-appendix.default .quarto-appendix-secondary-label{margin-bottom:.4em}#quarto-appendix.default .quarto-appendix-bibtex{font-size:.7em;padding:1em;border:solid 1px #dee2e6;margin-bottom:1em}#quarto-appendix.default .quarto-appendix-bibtex code.sourceCode{white-space:pre-wrap}#quarto-appendix.default .quarto-appendix-citeas{font-size:.9em;padding:1em;border:solid 1px #dee2e6;margin-bottom:1em}#quarto-appendix.default .quarto-appendix-heading{font-size:1em !important}#quarto-appendix.default *[role=doc-endnotes]>ol,#quarto-appendix.default .quarto-appendix-contents>*:not(h2):not(.h2){font-size:.9em}#quarto-appendix.default section{padding-bottom:1.5em}#quarto-appendix.default section *[role=doc-endnotes],#quarto-appendix.default section>*:not(a){opacity:.9;word-wrap:break-word}.btn.btn-quarto,div.cell-output-display .btn-quarto{--bs-btn-color: #fefefe;--bs-btn-bg: #6c757d;--bs-btn-border-color: #6c757d;--bs-btn-hover-color: #fefefe;--bs-btn-hover-bg: #828a91;--bs-btn-hover-border-color: #7b838a;--bs-btn-focus-shadow-rgb: 130, 138, 144;--bs-btn-active-color: #fff;--bs-btn-active-bg: #899197;--bs-btn-active-border-color: #7b838a;--bs-btn-active-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);--bs-btn-disabled-color: #fff;--bs-btn-disabled-bg: #6c757d;--bs-btn-disabled-border-color: #6c757d}nav.quarto-secondary-nav.color-navbar{background-color:#325d88;color:rgba(255,255,255,.7)}nav.quarto-secondary-nav.color-navbar h1,nav.quarto-secondary-nav.color-navbar .h1,nav.quarto-secondary-nav.color-navbar .quarto-btn-toggle{color:rgba(255,255,255,.7)}@media(max-width: 991.98px){body.nav-sidebar .quarto-title-banner{margin-bottom:0;padding-bottom:1em}body.nav-sidebar #title-block-header{margin-block-end:0}}p.subtitle{margin-top:.25em;margin-bottom:.5em}code a:any-link{color:inherit;text-decoration-color:#6c757d}/*! light */div.observablehq table thead tr th{background-color:var(--bs-body-bg)}input,button,select,optgroup,textarea{background-color:var(--bs-body-bg)}.code-annotated .code-copy-button{margin-right:1.25em;margin-top:0;padding-bottom:0;padding-top:3px}.code-annotation-gutter-bg{background-color:#fff}.code-annotation-gutter{background-color:#f8f5f0}.code-annotation-gutter,.code-annotation-gutter-bg{height:100%;width:calc(20px + .5em);position:absolute;top:0;right:0}dl.code-annotation-container-grid dt{margin-right:1em;margin-top:.25rem}dl.code-annotation-container-grid dt{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;color:#4b545c;border:solid #4b545c 1px;border-radius:50%;height:22px;width:22px;line-height:22px;font-size:11px;text-align:center;vertical-align:middle;text-decoration:none}dl.code-annotation-container-grid dt[data-target-cell]{cursor:pointer}dl.code-annotation-container-grid dt[data-target-cell].code-annotation-active{color:#fff;border:solid #aaa 1px;background-color:#aaa}pre.code-annotation-code{padding-top:0;padding-bottom:0}pre.code-annotation-code code{z-index:3}#code-annotation-line-highlight-gutter{width:100%;border-top:solid rgba(170,170,170,.2666666667) 1px;border-bottom:solid rgba(170,170,170,.2666666667) 1px;z-index:2;background-color:rgba(170,170,170,.1333333333)}#code-annotation-line-highlight{margin-left:-4em;width:calc(100% + 4em);border-top:solid rgba(170,170,170,.2666666667) 1px;border-bottom:solid rgba(170,170,170,.2666666667) 1px;z-index:2;background-color:rgba(170,170,170,.1333333333)}code.sourceCode .code-annotation-anchor.code-annotation-active{background-color:var(--quarto-hl-normal-color, #aaaaaa);border:solid var(--quarto-hl-normal-color, #aaaaaa) 1px;color:#f8f5f0;font-weight:bolder}code.sourceCode .code-annotation-anchor{font-family:SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",monospace;color:var(--quarto-hl-co-color);border:solid var(--quarto-hl-co-color) 1px;border-radius:50%;height:18px;width:18px;font-size:9px;margin-top:2px}code.sourceCode button.code-annotation-anchor{padding:2px;user-select:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;-o-user-select:none}code.sourceCode a.code-annotation-anchor{line-height:18px;text-align:center;vertical-align:middle;cursor:default;text-decoration:none}@media print{.page-columns .column-screen-inset{grid-column:page-start-inset/page-end-inset;z-index:998;opacity:.999}.page-columns .column-screen-inset table{background:#fff}.page-columns .column-screen-inset-left{grid-column:page-start-inset/body-content-end;z-index:998;opacity:.999}.page-columns .column-screen-inset-left table{background:#fff}.page-columns .column-screen-inset-right{grid-column:body-content-start/page-end-inset;z-index:998;opacity:.999}.page-columns .column-screen-inset-right table{background:#fff}.page-columns .column-screen{grid-column:page-start/page-end;z-index:998;opacity:.999}.page-columns .column-screen table{background:#fff}.page-columns .column-screen-left{grid-column:page-start/body-content-end;z-index:998;opacity:.999}.page-columns .column-screen-left table{background:#fff}.page-columns .column-screen-right{grid-column:body-content-start/page-end;z-index:998;opacity:.999}.page-columns .column-screen-right table{background:#fff}.page-columns .column-screen-inset-shaded{grid-column:page-start-inset/page-end-inset;padding:1em;background:#f8f5f0;z-index:998;opacity:.999;margin-bottom:1em}}.quarto-video{margin-bottom:1em}.table{border-top:1px solid #ebedee;border-bottom:1px solid #ebedee}.table>thead{border-top-width:0;border-bottom:1px solid #b2bac1}.table a{word-break:break-word}.table>:not(caption)>*>*{background-color:unset;color:unset}#quarto-document-content .crosstalk-input .checkbox input[type=checkbox],#quarto-document-content .crosstalk-input .checkbox-inline input[type=checkbox]{position:unset;margin-top:unset;margin-left:unset}#quarto-document-content .row{margin-left:unset;margin-right:unset}.quarto-xref{white-space:nowrap}#quarto-draft-alert{margin-top:0px;margin-bottom:0px;padding:.3em;text-align:center;font-size:.9em}#quarto-draft-alert i{margin-right:.3em}a.external:after{content:"";background-image:url('data:image/svg+xml,');background-size:contain;background-repeat:no-repeat;background-position:center center;margin-left:.2em;padding-right:.75em}div.sourceCode code a.external:after{content:none}a.external:after:hover{cursor:pointer}.quarto-ext-icon{display:inline-block;font-size:.75em;padding-left:.3em}.code-with-filename .code-with-filename-file{margin-bottom:0;padding-bottom:2px;padding-top:2px;padding-left:.7em;border:var(--quarto-border-width) solid var(--quarto-border-color);border-radius:var(--quarto-border-radius);border-bottom:0;border-bottom-left-radius:0%;border-bottom-right-radius:0%}.code-with-filename div.sourceCode,.reveal .code-with-filename div.sourceCode{margin-top:0;border-top-left-radius:0%;border-top-right-radius:0%}.code-with-filename .code-with-filename-file pre{margin-bottom:0}.code-with-filename .code-with-filename-file{background-color:rgba(219,219,219,.8)}.quarto-dark .code-with-filename .code-with-filename-file{background-color:#555}.code-with-filename .code-with-filename-file strong{font-weight:400}.quarto-title-banner{margin-bottom:1em;color:rgba(255,255,255,.7);background:#325d88}.quarto-title-banner a{color:rgba(255,255,255,.7)}.quarto-title-banner h1,.quarto-title-banner .h1,.quarto-title-banner h2,.quarto-title-banner .h2{color:rgba(255,255,255,.7)}.quarto-title-banner .code-tools-button{color:rgba(204,204,204,.7)}.quarto-title-banner .code-tools-button:hover{color:rgba(255,255,255,.7)}.quarto-title-banner .code-tools-button>.bi::before{background-image:url('data:image/svg+xml,')}.quarto-title-banner .code-tools-button:hover>.bi::before{background-image:url('data:image/svg+xml,')}.quarto-title-banner .quarto-title .title{font-weight:600}.quarto-title-banner .quarto-categories{margin-top:.75em}@media(min-width: 992px){.quarto-title-banner{padding-top:2.5em;padding-bottom:2.5em}}@media(max-width: 991.98px){.quarto-title-banner{padding-top:1em;padding-bottom:1em}}@media(max-width: 767.98px){body.hypothesis-enabled #title-block-header>*{padding-right:20px}}main.quarto-banner-title-block>section:first-child>h2,main.quarto-banner-title-block>section:first-child>.h2,main.quarto-banner-title-block>section:first-child>h3,main.quarto-banner-title-block>section:first-child>.h3,main.quarto-banner-title-block>section:first-child>h4,main.quarto-banner-title-block>section:first-child>.h4{margin-top:0}.quarto-title .quarto-categories{display:flex;flex-wrap:wrap;row-gap:.5em;column-gap:.4em;padding-bottom:.5em;margin-top:.75em}.quarto-title .quarto-categories .quarto-category{padding:.25em .75em;font-size:.65em;text-transform:uppercase;border:solid 1px;border-radius:.25rem;opacity:.6}.quarto-title .quarto-categories .quarto-category a{color:inherit}.quarto-title-meta-container{display:grid;grid-template-columns:1fr auto}.quarto-title-meta-column-end{display:flex;flex-direction:column;padding-left:1em}.quarto-title-meta-column-end a .bi{margin-right:.3em}#title-block-header.quarto-title-block.default .quarto-title-meta{display:grid;grid-template-columns:repeat(2, 1fr);grid-column-gap:1em}#title-block-header.quarto-title-block.default .quarto-title .title{margin-bottom:0}#title-block-header.quarto-title-block.default .quarto-title-author-orcid img{margin-top:-0.2em;height:.8em;width:.8em}#title-block-header.quarto-title-block.default .quarto-title-author-email{opacity:.7}#title-block-header.quarto-title-block.default .quarto-description p:last-of-type{margin-bottom:0}#title-block-header.quarto-title-block.default .quarto-title-meta-contents p,#title-block-header.quarto-title-block.default .quarto-title-authors p,#title-block-header.quarto-title-block.default .quarto-title-affiliations p{margin-bottom:.1em}#title-block-header.quarto-title-block.default .quarto-title-meta-heading{text-transform:uppercase;margin-top:1em;font-size:.8em;opacity:.8;font-weight:400}#title-block-header.quarto-title-block.default .quarto-title-meta-contents{font-size:.9em}#title-block-header.quarto-title-block.default .quarto-title-meta-contents p.affiliation:last-of-type{margin-bottom:.1em}#title-block-header.quarto-title-block.default p.affiliation{margin-bottom:.1em}#title-block-header.quarto-title-block.default .keywords,#title-block-header.quarto-title-block.default .description,#title-block-header.quarto-title-block.default .abstract{margin-top:0}#title-block-header.quarto-title-block.default .keywords>p,#title-block-header.quarto-title-block.default .description>p,#title-block-header.quarto-title-block.default .abstract>p{font-size:.9em}#title-block-header.quarto-title-block.default .keywords>p:last-of-type,#title-block-header.quarto-title-block.default .description>p:last-of-type,#title-block-header.quarto-title-block.default .abstract>p:last-of-type{margin-bottom:0}#title-block-header.quarto-title-block.default .keywords .block-title,#title-block-header.quarto-title-block.default .description .block-title,#title-block-header.quarto-title-block.default .abstract .block-title{margin-top:1em;text-transform:uppercase;font-size:.8em;opacity:.8;font-weight:400}#title-block-header.quarto-title-block.default .quarto-title-meta-author{display:grid;grid-template-columns:minmax(max-content, 1fr) 1fr;grid-column-gap:1em}.quarto-title-tools-only{display:flex;justify-content:right}.sandstone,.tooltip,.dropdown-menu .dropdown-item,.pagination,.breadcrumb,.nav-pills .nav-link,.nav-tabs .nav-link,.btn,.navbar .nav-link{font-size:13px;font-weight:500;line-height:22px;text-transform:uppercase}.navbar-form input,.navbar-form .form-control{border:none}.btn:hover{border-color:rgba(0,0,0,0)}.btn-success,.btn-warning{color:#fff}.table .thead-dark th{background-color:#343a40}.nav-tabs .nav-link{background-color:#f8f5f0;border-color:#dee2e6}.nav-tabs .nav-link,.nav-tabs .nav-link:hover,.nav-tabs .nav-link:focus{color:#6c757d}.nav-tabs .nav-link.disabled,.nav-tabs .nav-link.disabled:hover,.nav-tabs .nav-link.disabled:focus{color:#dee2e6;background-color:#f8f5f0;border-color:#dee2e6}.nav-pills .nav-link{color:#6c757d;border:1px solid rgba(0,0,0,0)}.nav-pills .nav-link.active,.nav-pills .nav-link:hover,.nav-pills .nav-link:focus{background-color:#f8f5f0;border-color:#dee2e6}.nav-pills .nav-link.disabled,.nav-pills .nav-link.disabled:hover{color:#dee2e6;background-color:rgba(0,0,0,0);border-color:rgba(0,0,0,0)}.breadcrumb{border:1px solid #dee2e6}.pagination a:hover{text-decoration:none}.alert{color:#fff}.alert a,.alert .alert-link{color:#fff;text-decoration:underline}.alert-primary,.alert-primary>th,.alert-primary>td{background-color:#325d88}.alert-secondary,.alert-secondary>th,.alert-secondary>td{background-color:#6c757d}.alert-success,.alert-success>th,.alert-success>td{background-color:#93c54b}.alert-info,.alert-info>th,.alert-info>td{background-color:#29abe0}.alert-danger,.alert-danger>th,.alert-danger>td{background-color:#d9534f}.alert-warning,.alert-warning>th,.alert-warning>td{background-color:#f47c3c}.alert-dark,.alert-dark>th,.alert-dark>td{background-color:#343a40}.alert-light,.alert-light>th,.alert-light>td{background-color:#f8f5f0}.alert-light,.alert-light a:not(.btn),.alert-light .alert-link{color:#343a40}.badge.bg-light{color:#343a40}.modal .btn-close,.toast .btn-close,.offcanvas .btn-close{background-image:url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16' fill='%23dee2e6'%3e%3cpath d='M.293.293a1 1 0 0 1 1.414 0L8 6.586 14.293.293a1 1 0 1 1 1.414 1.414L9.414 8l6.293 6.293a1 1 0 0 1-1.414 1.414L8 9.414l-6.293 6.293a1 1 0 0 1-1.414-1.414L6.586 8 .293 1.707a1 1 0 0 1 0-1.414z'/%3e%3c/svg%3e")} diff --git a/site_libs/bootstrap/bootstrap.min.js b/site_libs/bootstrap/bootstrap.min.js new file mode 100644 index 0000000..e8f21f7 --- /dev/null +++ b/site_libs/bootstrap/bootstrap.min.js @@ -0,0 +1,7 @@ +/*! + * Bootstrap v5.3.1 (https://getbootstrap.com/) + * Copyright 2011-2023 The Bootstrap Authors (https://github.com/twbs/bootstrap/graphs/contributors) + * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) + */ +!function(t,e){"object"==typeof exports&&"undefined"!=typeof module?module.exports=e():"function"==typeof define&&define.amd?define(e):(t="undefined"!=typeof globalThis?globalThis:t||self).bootstrap=e()}(this,(function(){"use strict";const t=new Map,e={set(e,i,n){t.has(e)||t.set(e,new Map);const s=t.get(e);s.has(i)||0===s.size?s.set(i,n):console.error(`Bootstrap doesn't allow more than one instance per element. Bound instance: ${Array.from(s.keys())[0]}.`)},get:(e,i)=>t.has(e)&&t.get(e).get(i)||null,remove(e,i){if(!t.has(e))return;const n=t.get(e);n.delete(i),0===n.size&&t.delete(e)}},i="transitionend",n=t=>(t&&window.CSS&&window.CSS.escape&&(t=t.replace(/#([^\s"#']+)/g,((t,e)=>`#${CSS.escape(e)}`))),t),s=t=>{t.dispatchEvent(new Event(i))},o=t=>!(!t||"object"!=typeof t)&&(void 0!==t.jquery&&(t=t[0]),void 0!==t.nodeType),r=t=>o(t)?t.jquery?t[0]:t:"string"==typeof t&&t.length>0?document.querySelector(n(t)):null,a=t=>{if(!o(t)||0===t.getClientRects().length)return!1;const e="visible"===getComputedStyle(t).getPropertyValue("visibility"),i=t.closest("details:not([open])");if(!i)return e;if(i!==t){const e=t.closest("summary");if(e&&e.parentNode!==i)return!1;if(null===e)return!1}return e},l=t=>!t||t.nodeType!==Node.ELEMENT_NODE||!!t.classList.contains("disabled")||(void 0!==t.disabled?t.disabled:t.hasAttribute("disabled")&&"false"!==t.getAttribute("disabled")),c=t=>{if(!document.documentElement.attachShadow)return null;if("function"==typeof t.getRootNode){const e=t.getRootNode();return e instanceof ShadowRoot?e:null}return t instanceof ShadowRoot?t:t.parentNode?c(t.parentNode):null},h=()=>{},d=t=>{t.offsetHeight},u=()=>window.jQuery&&!document.body.hasAttribute("data-bs-no-jquery")?window.jQuery:null,f=[],p=()=>"rtl"===document.documentElement.dir,m=t=>{var e;e=()=>{const e=u();if(e){const i=t.NAME,n=e.fn[i];e.fn[i]=t.jQueryInterface,e.fn[i].Constructor=t,e.fn[i].noConflict=()=>(e.fn[i]=n,t.jQueryInterface)}},"loading"===document.readyState?(f.length||document.addEventListener("DOMContentLoaded",(()=>{for(const t of f)t()})),f.push(e)):e()},g=(t,e=[],i=t)=>"function"==typeof t?t(...e):i,_=(t,e,n=!0)=>{if(!n)return void g(t);const o=(t=>{if(!t)return 0;let{transitionDuration:e,transitionDelay:i}=window.getComputedStyle(t);const n=Number.parseFloat(e),s=Number.parseFloat(i);return n||s?(e=e.split(",")[0],i=i.split(",")[0],1e3*(Number.parseFloat(e)+Number.parseFloat(i))):0})(e)+5;let r=!1;const a=({target:n})=>{n===e&&(r=!0,e.removeEventListener(i,a),g(t))};e.addEventListener(i,a),setTimeout((()=>{r||s(e)}),o)},b=(t,e,i,n)=>{const s=t.length;let o=t.indexOf(e);return-1===o?!i&&n?t[s-1]:t[0]:(o+=i?1:-1,n&&(o=(o+s)%s),t[Math.max(0,Math.min(o,s-1))])},v=/[^.]*(?=\..*)\.|.*/,y=/\..*/,w=/::\d+$/,A={};let E=1;const T={mouseenter:"mouseover",mouseleave:"mouseout"},C=new Set(["click","dblclick","mouseup","mousedown","contextmenu","mousewheel","DOMMouseScroll","mouseover","mouseout","mousemove","selectstart","selectend","keydown","keypress","keyup","orientationchange","touchstart","touchmove","touchend","touchcancel","pointerdown","pointermove","pointerup","pointerleave","pointercancel","gesturestart","gesturechange","gestureend","focus","blur","change","reset","select","submit","focusin","focusout","load","unload","beforeunload","resize","move","DOMContentLoaded","readystatechange","error","abort","scroll"]);function O(t,e){return e&&`${e}::${E++}`||t.uidEvent||E++}function x(t){const e=O(t);return t.uidEvent=e,A[e]=A[e]||{},A[e]}function k(t,e,i=null){return Object.values(t).find((t=>t.callable===e&&t.delegationSelector===i))}function L(t,e,i){const n="string"==typeof e,s=n?i:e||i;let o=I(t);return C.has(o)||(o=t),[n,s,o]}function S(t,e,i,n,s){if("string"!=typeof e||!t)return;let[o,r,a]=L(e,i,n);if(e in T){const t=t=>function(e){if(!e.relatedTarget||e.relatedTarget!==e.delegateTarget&&!e.delegateTarget.contains(e.relatedTarget))return t.call(this,e)};r=t(r)}const l=x(t),c=l[a]||(l[a]={}),h=k(c,r,o?i:null);if(h)return void(h.oneOff=h.oneOff&&s);const d=O(r,e.replace(v,"")),u=o?function(t,e,i){return function n(s){const o=t.querySelectorAll(e);for(let{target:r}=s;r&&r!==this;r=r.parentNode)for(const a of o)if(a===r)return P(s,{delegateTarget:r}),n.oneOff&&N.off(t,s.type,e,i),i.apply(r,[s])}}(t,i,r):function(t,e){return function i(n){return P(n,{delegateTarget:t}),i.oneOff&&N.off(t,n.type,e),e.apply(t,[n])}}(t,r);u.delegationSelector=o?i:null,u.callable=r,u.oneOff=s,u.uidEvent=d,c[d]=u,t.addEventListener(a,u,o)}function D(t,e,i,n,s){const o=k(e[i],n,s);o&&(t.removeEventListener(i,o,Boolean(s)),delete e[i][o.uidEvent])}function $(t,e,i,n){const s=e[i]||{};for(const[o,r]of Object.entries(s))o.includes(n)&&D(t,e,i,r.callable,r.delegationSelector)}function I(t){return t=t.replace(y,""),T[t]||t}const N={on(t,e,i,n){S(t,e,i,n,!1)},one(t,e,i,n){S(t,e,i,n,!0)},off(t,e,i,n){if("string"!=typeof e||!t)return;const[s,o,r]=L(e,i,n),a=r!==e,l=x(t),c=l[r]||{},h=e.startsWith(".");if(void 0===o){if(h)for(const i of Object.keys(l))$(t,l,i,e.slice(1));for(const[i,n]of Object.entries(c)){const s=i.replace(w,"");a&&!e.includes(s)||D(t,l,r,n.callable,n.delegationSelector)}}else{if(!Object.keys(c).length)return;D(t,l,r,o,s?i:null)}},trigger(t,e,i){if("string"!=typeof e||!t)return null;const n=u();let s=null,o=!0,r=!0,a=!1;e!==I(e)&&n&&(s=n.Event(e,i),n(t).trigger(s),o=!s.isPropagationStopped(),r=!s.isImmediatePropagationStopped(),a=s.isDefaultPrevented());const l=P(new Event(e,{bubbles:o,cancelable:!0}),i);return a&&l.preventDefault(),r&&t.dispatchEvent(l),l.defaultPrevented&&s&&s.preventDefault(),l}};function P(t,e={}){for(const[i,n]of Object.entries(e))try{t[i]=n}catch(e){Object.defineProperty(t,i,{configurable:!0,get:()=>n})}return t}function M(t){if("true"===t)return!0;if("false"===t)return!1;if(t===Number(t).toString())return Number(t);if(""===t||"null"===t)return null;if("string"!=typeof t)return t;try{return JSON.parse(decodeURIComponent(t))}catch(e){return t}}function j(t){return t.replace(/[A-Z]/g,(t=>`-${t.toLowerCase()}`))}const F={setDataAttribute(t,e,i){t.setAttribute(`data-bs-${j(e)}`,i)},removeDataAttribute(t,e){t.removeAttribute(`data-bs-${j(e)}`)},getDataAttributes(t){if(!t)return{};const e={},i=Object.keys(t.dataset).filter((t=>t.startsWith("bs")&&!t.startsWith("bsConfig")));for(const n of i){let i=n.replace(/^bs/,"");i=i.charAt(0).toLowerCase()+i.slice(1,i.length),e[i]=M(t.dataset[n])}return e},getDataAttribute:(t,e)=>M(t.getAttribute(`data-bs-${j(e)}`))};class H{static get Default(){return{}}static get DefaultType(){return{}}static get NAME(){throw new Error('You have to implement the static method "NAME", for each component!')}_getConfig(t){return t=this._mergeConfigObj(t),t=this._configAfterMerge(t),this._typeCheckConfig(t),t}_configAfterMerge(t){return t}_mergeConfigObj(t,e){const i=o(e)?F.getDataAttribute(e,"config"):{};return{...this.constructor.Default,..."object"==typeof i?i:{},...o(e)?F.getDataAttributes(e):{},..."object"==typeof t?t:{}}}_typeCheckConfig(t,e=this.constructor.DefaultType){for(const[n,s]of Object.entries(e)){const e=t[n],r=o(e)?"element":null==(i=e)?`${i}`:Object.prototype.toString.call(i).match(/\s([a-z]+)/i)[1].toLowerCase();if(!new RegExp(s).test(r))throw new TypeError(`${this.constructor.NAME.toUpperCase()}: Option "${n}" provided type "${r}" but expected type "${s}".`)}var i}}class W extends H{constructor(t,i){super(),(t=r(t))&&(this._element=t,this._config=this._getConfig(i),e.set(this._element,this.constructor.DATA_KEY,this))}dispose(){e.remove(this._element,this.constructor.DATA_KEY),N.off(this._element,this.constructor.EVENT_KEY);for(const t of Object.getOwnPropertyNames(this))this[t]=null}_queueCallback(t,e,i=!0){_(t,e,i)}_getConfig(t){return t=this._mergeConfigObj(t,this._element),t=this._configAfterMerge(t),this._typeCheckConfig(t),t}static getInstance(t){return e.get(r(t),this.DATA_KEY)}static getOrCreateInstance(t,e={}){return this.getInstance(t)||new this(t,"object"==typeof e?e:null)}static get VERSION(){return"5.3.1"}static get DATA_KEY(){return`bs.${this.NAME}`}static get EVENT_KEY(){return`.${this.DATA_KEY}`}static eventName(t){return`${t}${this.EVENT_KEY}`}}const B=t=>{let e=t.getAttribute("data-bs-target");if(!e||"#"===e){let i=t.getAttribute("href");if(!i||!i.includes("#")&&!i.startsWith("."))return null;i.includes("#")&&!i.startsWith("#")&&(i=`#${i.split("#")[1]}`),e=i&&"#"!==i?i.trim():null}return n(e)},z={find:(t,e=document.documentElement)=>[].concat(...Element.prototype.querySelectorAll.call(e,t)),findOne:(t,e=document.documentElement)=>Element.prototype.querySelector.call(e,t),children:(t,e)=>[].concat(...t.children).filter((t=>t.matches(e))),parents(t,e){const i=[];let n=t.parentNode.closest(e);for(;n;)i.push(n),n=n.parentNode.closest(e);return i},prev(t,e){let i=t.previousElementSibling;for(;i;){if(i.matches(e))return[i];i=i.previousElementSibling}return[]},next(t,e){let i=t.nextElementSibling;for(;i;){if(i.matches(e))return[i];i=i.nextElementSibling}return[]},focusableChildren(t){const e=["a","button","input","textarea","select","details","[tabindex]",'[contenteditable="true"]'].map((t=>`${t}:not([tabindex^="-"])`)).join(",");return this.find(e,t).filter((t=>!l(t)&&a(t)))},getSelectorFromElement(t){const e=B(t);return e&&z.findOne(e)?e:null},getElementFromSelector(t){const e=B(t);return e?z.findOne(e):null},getMultipleElementsFromSelector(t){const e=B(t);return e?z.find(e):[]}},R=(t,e="hide")=>{const i=`click.dismiss${t.EVENT_KEY}`,n=t.NAME;N.on(document,i,`[data-bs-dismiss="${n}"]`,(function(i){if(["A","AREA"].includes(this.tagName)&&i.preventDefault(),l(this))return;const s=z.getElementFromSelector(this)||this.closest(`.${n}`);t.getOrCreateInstance(s)[e]()}))},q=".bs.alert",V=`close${q}`,K=`closed${q}`;class Q extends W{static get NAME(){return"alert"}close(){if(N.trigger(this._element,V).defaultPrevented)return;this._element.classList.remove("show");const t=this._element.classList.contains("fade");this._queueCallback((()=>this._destroyElement()),this._element,t)}_destroyElement(){this._element.remove(),N.trigger(this._element,K),this.dispose()}static jQueryInterface(t){return this.each((function(){const e=Q.getOrCreateInstance(this);if("string"==typeof t){if(void 0===e[t]||t.startsWith("_")||"constructor"===t)throw new TypeError(`No method named "${t}"`);e[t](this)}}))}}R(Q,"close"),m(Q);const X='[data-bs-toggle="button"]';class Y extends W{static get NAME(){return"button"}toggle(){this._element.setAttribute("aria-pressed",this._element.classList.toggle("active"))}static jQueryInterface(t){return this.each((function(){const e=Y.getOrCreateInstance(this);"toggle"===t&&e[t]()}))}}N.on(document,"click.bs.button.data-api",X,(t=>{t.preventDefault();const e=t.target.closest(X);Y.getOrCreateInstance(e).toggle()})),m(Y);const U=".bs.swipe",G=`touchstart${U}`,J=`touchmove${U}`,Z=`touchend${U}`,tt=`pointerdown${U}`,et=`pointerup${U}`,it={endCallback:null,leftCallback:null,rightCallback:null},nt={endCallback:"(function|null)",leftCallback:"(function|null)",rightCallback:"(function|null)"};class st extends H{constructor(t,e){super(),this._element=t,t&&st.isSupported()&&(this._config=this._getConfig(e),this._deltaX=0,this._supportPointerEvents=Boolean(window.PointerEvent),this._initEvents())}static get Default(){return it}static get DefaultType(){return nt}static get NAME(){return"swipe"}dispose(){N.off(this._element,U)}_start(t){this._supportPointerEvents?this._eventIsPointerPenTouch(t)&&(this._deltaX=t.clientX):this._deltaX=t.touches[0].clientX}_end(t){this._eventIsPointerPenTouch(t)&&(this._deltaX=t.clientX-this._deltaX),this._handleSwipe(),g(this._config.endCallback)}_move(t){this._deltaX=t.touches&&t.touches.length>1?0:t.touches[0].clientX-this._deltaX}_handleSwipe(){const t=Math.abs(this._deltaX);if(t<=40)return;const e=t/this._deltaX;this._deltaX=0,e&&g(e>0?this._config.rightCallback:this._config.leftCallback)}_initEvents(){this._supportPointerEvents?(N.on(this._element,tt,(t=>this._start(t))),N.on(this._element,et,(t=>this._end(t))),this._element.classList.add("pointer-event")):(N.on(this._element,G,(t=>this._start(t))),N.on(this._element,J,(t=>this._move(t))),N.on(this._element,Z,(t=>this._end(t))))}_eventIsPointerPenTouch(t){return this._supportPointerEvents&&("pen"===t.pointerType||"touch"===t.pointerType)}static isSupported(){return"ontouchstart"in document.documentElement||navigator.maxTouchPoints>0}}const ot=".bs.carousel",rt=".data-api",at="next",lt="prev",ct="left",ht="right",dt=`slide${ot}`,ut=`slid${ot}`,ft=`keydown${ot}`,pt=`mouseenter${ot}`,mt=`mouseleave${ot}`,gt=`dragstart${ot}`,_t=`load${ot}${rt}`,bt=`click${ot}${rt}`,vt="carousel",yt="active",wt=".active",At=".carousel-item",Et=wt+At,Tt={ArrowLeft:ht,ArrowRight:ct},Ct={interval:5e3,keyboard:!0,pause:"hover",ride:!1,touch:!0,wrap:!0},Ot={interval:"(number|boolean)",keyboard:"boolean",pause:"(string|boolean)",ride:"(boolean|string)",touch:"boolean",wrap:"boolean"};class xt extends W{constructor(t,e){super(t,e),this._interval=null,this._activeElement=null,this._isSliding=!1,this.touchTimeout=null,this._swipeHelper=null,this._indicatorsElement=z.findOne(".carousel-indicators",this._element),this._addEventListeners(),this._config.ride===vt&&this.cycle()}static get Default(){return Ct}static get DefaultType(){return Ot}static get NAME(){return"carousel"}next(){this._slide(at)}nextWhenVisible(){!document.hidden&&a(this._element)&&this.next()}prev(){this._slide(lt)}pause(){this._isSliding&&s(this._element),this._clearInterval()}cycle(){this._clearInterval(),this._updateInterval(),this._interval=setInterval((()=>this.nextWhenVisible()),this._config.interval)}_maybeEnableCycle(){this._config.ride&&(this._isSliding?N.one(this._element,ut,(()=>this.cycle())):this.cycle())}to(t){const e=this._getItems();if(t>e.length-1||t<0)return;if(this._isSliding)return void N.one(this._element,ut,(()=>this.to(t)));const i=this._getItemIndex(this._getActive());if(i===t)return;const n=t>i?at:lt;this._slide(n,e[t])}dispose(){this._swipeHelper&&this._swipeHelper.dispose(),super.dispose()}_configAfterMerge(t){return t.defaultInterval=t.interval,t}_addEventListeners(){this._config.keyboard&&N.on(this._element,ft,(t=>this._keydown(t))),"hover"===this._config.pause&&(N.on(this._element,pt,(()=>this.pause())),N.on(this._element,mt,(()=>this._maybeEnableCycle()))),this._config.touch&&st.isSupported()&&this._addTouchEventListeners()}_addTouchEventListeners(){for(const t of z.find(".carousel-item img",this._element))N.on(t,gt,(t=>t.preventDefault()));const t={leftCallback:()=>this._slide(this._directionToOrder(ct)),rightCallback:()=>this._slide(this._directionToOrder(ht)),endCallback:()=>{"hover"===this._config.pause&&(this.pause(),this.touchTimeout&&clearTimeout(this.touchTimeout),this.touchTimeout=setTimeout((()=>this._maybeEnableCycle()),500+this._config.interval))}};this._swipeHelper=new st(this._element,t)}_keydown(t){if(/input|textarea/i.test(t.target.tagName))return;const e=Tt[t.key];e&&(t.preventDefault(),this._slide(this._directionToOrder(e)))}_getItemIndex(t){return this._getItems().indexOf(t)}_setActiveIndicatorElement(t){if(!this._indicatorsElement)return;const e=z.findOne(wt,this._indicatorsElement);e.classList.remove(yt),e.removeAttribute("aria-current");const i=z.findOne(`[data-bs-slide-to="${t}"]`,this._indicatorsElement);i&&(i.classList.add(yt),i.setAttribute("aria-current","true"))}_updateInterval(){const t=this._activeElement||this._getActive();if(!t)return;const e=Number.parseInt(t.getAttribute("data-bs-interval"),10);this._config.interval=e||this._config.defaultInterval}_slide(t,e=null){if(this._isSliding)return;const i=this._getActive(),n=t===at,s=e||b(this._getItems(),i,n,this._config.wrap);if(s===i)return;const o=this._getItemIndex(s),r=e=>N.trigger(this._element,e,{relatedTarget:s,direction:this._orderToDirection(t),from:this._getItemIndex(i),to:o});if(r(dt).defaultPrevented)return;if(!i||!s)return;const a=Boolean(this._interval);this.pause(),this._isSliding=!0,this._setActiveIndicatorElement(o),this._activeElement=s;const l=n?"carousel-item-start":"carousel-item-end",c=n?"carousel-item-next":"carousel-item-prev";s.classList.add(c),d(s),i.classList.add(l),s.classList.add(l),this._queueCallback((()=>{s.classList.remove(l,c),s.classList.add(yt),i.classList.remove(yt,c,l),this._isSliding=!1,r(ut)}),i,this._isAnimated()),a&&this.cycle()}_isAnimated(){return this._element.classList.contains("slide")}_getActive(){return z.findOne(Et,this._element)}_getItems(){return z.find(At,this._element)}_clearInterval(){this._interval&&(clearInterval(this._interval),this._interval=null)}_directionToOrder(t){return p()?t===ct?lt:at:t===ct?at:lt}_orderToDirection(t){return p()?t===lt?ct:ht:t===lt?ht:ct}static jQueryInterface(t){return this.each((function(){const e=xt.getOrCreateInstance(this,t);if("number"!=typeof t){if("string"==typeof t){if(void 0===e[t]||t.startsWith("_")||"constructor"===t)throw new TypeError(`No method named "${t}"`);e[t]()}}else e.to(t)}))}}N.on(document,bt,"[data-bs-slide], [data-bs-slide-to]",(function(t){const e=z.getElementFromSelector(this);if(!e||!e.classList.contains(vt))return;t.preventDefault();const i=xt.getOrCreateInstance(e),n=this.getAttribute("data-bs-slide-to");return n?(i.to(n),void i._maybeEnableCycle()):"next"===F.getDataAttribute(this,"slide")?(i.next(),void i._maybeEnableCycle()):(i.prev(),void i._maybeEnableCycle())})),N.on(window,_t,(()=>{const t=z.find('[data-bs-ride="carousel"]');for(const e of t)xt.getOrCreateInstance(e)})),m(xt);const kt=".bs.collapse",Lt=`show${kt}`,St=`shown${kt}`,Dt=`hide${kt}`,$t=`hidden${kt}`,It=`click${kt}.data-api`,Nt="show",Pt="collapse",Mt="collapsing",jt=`:scope .${Pt} .${Pt}`,Ft='[data-bs-toggle="collapse"]',Ht={parent:null,toggle:!0},Wt={parent:"(null|element)",toggle:"boolean"};class Bt extends W{constructor(t,e){super(t,e),this._isTransitioning=!1,this._triggerArray=[];const i=z.find(Ft);for(const t of i){const e=z.getSelectorFromElement(t),i=z.find(e).filter((t=>t===this._element));null!==e&&i.length&&this._triggerArray.push(t)}this._initializeChildren(),this._config.parent||this._addAriaAndCollapsedClass(this._triggerArray,this._isShown()),this._config.toggle&&this.toggle()}static get Default(){return Ht}static get DefaultType(){return Wt}static get NAME(){return"collapse"}toggle(){this._isShown()?this.hide():this.show()}show(){if(this._isTransitioning||this._isShown())return;let t=[];if(this._config.parent&&(t=this._getFirstLevelChildren(".collapse.show, .collapse.collapsing").filter((t=>t!==this._element)).map((t=>Bt.getOrCreateInstance(t,{toggle:!1})))),t.length&&t[0]._isTransitioning)return;if(N.trigger(this._element,Lt).defaultPrevented)return;for(const e of t)e.hide();const e=this._getDimension();this._element.classList.remove(Pt),this._element.classList.add(Mt),this._element.style[e]=0,this._addAriaAndCollapsedClass(this._triggerArray,!0),this._isTransitioning=!0;const i=`scroll${e[0].toUpperCase()+e.slice(1)}`;this._queueCallback((()=>{this._isTransitioning=!1,this._element.classList.remove(Mt),this._element.classList.add(Pt,Nt),this._element.style[e]="",N.trigger(this._element,St)}),this._element,!0),this._element.style[e]=`${this._element[i]}px`}hide(){if(this._isTransitioning||!this._isShown())return;if(N.trigger(this._element,Dt).defaultPrevented)return;const t=this._getDimension();this._element.style[t]=`${this._element.getBoundingClientRect()[t]}px`,d(this._element),this._element.classList.add(Mt),this._element.classList.remove(Pt,Nt);for(const t of this._triggerArray){const e=z.getElementFromSelector(t);e&&!this._isShown(e)&&this._addAriaAndCollapsedClass([t],!1)}this._isTransitioning=!0,this._element.style[t]="",this._queueCallback((()=>{this._isTransitioning=!1,this._element.classList.remove(Mt),this._element.classList.add(Pt),N.trigger(this._element,$t)}),this._element,!0)}_isShown(t=this._element){return t.classList.contains(Nt)}_configAfterMerge(t){return t.toggle=Boolean(t.toggle),t.parent=r(t.parent),t}_getDimension(){return this._element.classList.contains("collapse-horizontal")?"width":"height"}_initializeChildren(){if(!this._config.parent)return;const t=this._getFirstLevelChildren(Ft);for(const e of t){const t=z.getElementFromSelector(e);t&&this._addAriaAndCollapsedClass([e],this._isShown(t))}}_getFirstLevelChildren(t){const e=z.find(jt,this._config.parent);return z.find(t,this._config.parent).filter((t=>!e.includes(t)))}_addAriaAndCollapsedClass(t,e){if(t.length)for(const i of t)i.classList.toggle("collapsed",!e),i.setAttribute("aria-expanded",e)}static jQueryInterface(t){const e={};return"string"==typeof t&&/show|hide/.test(t)&&(e.toggle=!1),this.each((function(){const i=Bt.getOrCreateInstance(this,e);if("string"==typeof t){if(void 0===i[t])throw new TypeError(`No method named "${t}"`);i[t]()}}))}}N.on(document,It,Ft,(function(t){("A"===t.target.tagName||t.delegateTarget&&"A"===t.delegateTarget.tagName)&&t.preventDefault();for(const t of z.getMultipleElementsFromSelector(this))Bt.getOrCreateInstance(t,{toggle:!1}).toggle()})),m(Bt);var zt="top",Rt="bottom",qt="right",Vt="left",Kt="auto",Qt=[zt,Rt,qt,Vt],Xt="start",Yt="end",Ut="clippingParents",Gt="viewport",Jt="popper",Zt="reference",te=Qt.reduce((function(t,e){return t.concat([e+"-"+Xt,e+"-"+Yt])}),[]),ee=[].concat(Qt,[Kt]).reduce((function(t,e){return t.concat([e,e+"-"+Xt,e+"-"+Yt])}),[]),ie="beforeRead",ne="read",se="afterRead",oe="beforeMain",re="main",ae="afterMain",le="beforeWrite",ce="write",he="afterWrite",de=[ie,ne,se,oe,re,ae,le,ce,he];function ue(t){return t?(t.nodeName||"").toLowerCase():null}function fe(t){if(null==t)return window;if("[object Window]"!==t.toString()){var e=t.ownerDocument;return e&&e.defaultView||window}return t}function pe(t){return t instanceof fe(t).Element||t instanceof Element}function me(t){return t instanceof fe(t).HTMLElement||t instanceof HTMLElement}function ge(t){return"undefined"!=typeof ShadowRoot&&(t instanceof fe(t).ShadowRoot||t instanceof ShadowRoot)}const _e={name:"applyStyles",enabled:!0,phase:"write",fn:function(t){var e=t.state;Object.keys(e.elements).forEach((function(t){var i=e.styles[t]||{},n=e.attributes[t]||{},s=e.elements[t];me(s)&&ue(s)&&(Object.assign(s.style,i),Object.keys(n).forEach((function(t){var e=n[t];!1===e?s.removeAttribute(t):s.setAttribute(t,!0===e?"":e)})))}))},effect:function(t){var e=t.state,i={popper:{position:e.options.strategy,left:"0",top:"0",margin:"0"},arrow:{position:"absolute"},reference:{}};return Object.assign(e.elements.popper.style,i.popper),e.styles=i,e.elements.arrow&&Object.assign(e.elements.arrow.style,i.arrow),function(){Object.keys(e.elements).forEach((function(t){var n=e.elements[t],s=e.attributes[t]||{},o=Object.keys(e.styles.hasOwnProperty(t)?e.styles[t]:i[t]).reduce((function(t,e){return t[e]="",t}),{});me(n)&&ue(n)&&(Object.assign(n.style,o),Object.keys(s).forEach((function(t){n.removeAttribute(t)})))}))}},requires:["computeStyles"]};function be(t){return t.split("-")[0]}var ve=Math.max,ye=Math.min,we=Math.round;function Ae(){var t=navigator.userAgentData;return null!=t&&t.brands&&Array.isArray(t.brands)?t.brands.map((function(t){return t.brand+"/"+t.version})).join(" "):navigator.userAgent}function Ee(){return!/^((?!chrome|android).)*safari/i.test(Ae())}function Te(t,e,i){void 0===e&&(e=!1),void 0===i&&(i=!1);var n=t.getBoundingClientRect(),s=1,o=1;e&&me(t)&&(s=t.offsetWidth>0&&we(n.width)/t.offsetWidth||1,o=t.offsetHeight>0&&we(n.height)/t.offsetHeight||1);var r=(pe(t)?fe(t):window).visualViewport,a=!Ee()&&i,l=(n.left+(a&&r?r.offsetLeft:0))/s,c=(n.top+(a&&r?r.offsetTop:0))/o,h=n.width/s,d=n.height/o;return{width:h,height:d,top:c,right:l+h,bottom:c+d,left:l,x:l,y:c}}function Ce(t){var e=Te(t),i=t.offsetWidth,n=t.offsetHeight;return Math.abs(e.width-i)<=1&&(i=e.width),Math.abs(e.height-n)<=1&&(n=e.height),{x:t.offsetLeft,y:t.offsetTop,width:i,height:n}}function Oe(t,e){var i=e.getRootNode&&e.getRootNode();if(t.contains(e))return!0;if(i&&ge(i)){var n=e;do{if(n&&t.isSameNode(n))return!0;n=n.parentNode||n.host}while(n)}return!1}function xe(t){return fe(t).getComputedStyle(t)}function ke(t){return["table","td","th"].indexOf(ue(t))>=0}function Le(t){return((pe(t)?t.ownerDocument:t.document)||window.document).documentElement}function Se(t){return"html"===ue(t)?t:t.assignedSlot||t.parentNode||(ge(t)?t.host:null)||Le(t)}function De(t){return me(t)&&"fixed"!==xe(t).position?t.offsetParent:null}function $e(t){for(var e=fe(t),i=De(t);i&&ke(i)&&"static"===xe(i).position;)i=De(i);return i&&("html"===ue(i)||"body"===ue(i)&&"static"===xe(i).position)?e:i||function(t){var e=/firefox/i.test(Ae());if(/Trident/i.test(Ae())&&me(t)&&"fixed"===xe(t).position)return null;var i=Se(t);for(ge(i)&&(i=i.host);me(i)&&["html","body"].indexOf(ue(i))<0;){var n=xe(i);if("none"!==n.transform||"none"!==n.perspective||"paint"===n.contain||-1!==["transform","perspective"].indexOf(n.willChange)||e&&"filter"===n.willChange||e&&n.filter&&"none"!==n.filter)return i;i=i.parentNode}return null}(t)||e}function Ie(t){return["top","bottom"].indexOf(t)>=0?"x":"y"}function Ne(t,e,i){return ve(t,ye(e,i))}function Pe(t){return Object.assign({},{top:0,right:0,bottom:0,left:0},t)}function Me(t,e){return e.reduce((function(e,i){return e[i]=t,e}),{})}const je={name:"arrow",enabled:!0,phase:"main",fn:function(t){var e,i=t.state,n=t.name,s=t.options,o=i.elements.arrow,r=i.modifiersData.popperOffsets,a=be(i.placement),l=Ie(a),c=[Vt,qt].indexOf(a)>=0?"height":"width";if(o&&r){var h=function(t,e){return Pe("number"!=typeof(t="function"==typeof t?t(Object.assign({},e.rects,{placement:e.placement})):t)?t:Me(t,Qt))}(s.padding,i),d=Ce(o),u="y"===l?zt:Vt,f="y"===l?Rt:qt,p=i.rects.reference[c]+i.rects.reference[l]-r[l]-i.rects.popper[c],m=r[l]-i.rects.reference[l],g=$e(o),_=g?"y"===l?g.clientHeight||0:g.clientWidth||0:0,b=p/2-m/2,v=h[u],y=_-d[c]-h[f],w=_/2-d[c]/2+b,A=Ne(v,w,y),E=l;i.modifiersData[n]=((e={})[E]=A,e.centerOffset=A-w,e)}},effect:function(t){var e=t.state,i=t.options.element,n=void 0===i?"[data-popper-arrow]":i;null!=n&&("string"!=typeof n||(n=e.elements.popper.querySelector(n)))&&Oe(e.elements.popper,n)&&(e.elements.arrow=n)},requires:["popperOffsets"],requiresIfExists:["preventOverflow"]};function Fe(t){return t.split("-")[1]}var He={top:"auto",right:"auto",bottom:"auto",left:"auto"};function We(t){var e,i=t.popper,n=t.popperRect,s=t.placement,o=t.variation,r=t.offsets,a=t.position,l=t.gpuAcceleration,c=t.adaptive,h=t.roundOffsets,d=t.isFixed,u=r.x,f=void 0===u?0:u,p=r.y,m=void 0===p?0:p,g="function"==typeof h?h({x:f,y:m}):{x:f,y:m};f=g.x,m=g.y;var _=r.hasOwnProperty("x"),b=r.hasOwnProperty("y"),v=Vt,y=zt,w=window;if(c){var A=$e(i),E="clientHeight",T="clientWidth";A===fe(i)&&"static"!==xe(A=Le(i)).position&&"absolute"===a&&(E="scrollHeight",T="scrollWidth"),(s===zt||(s===Vt||s===qt)&&o===Yt)&&(y=Rt,m-=(d&&A===w&&w.visualViewport?w.visualViewport.height:A[E])-n.height,m*=l?1:-1),s!==Vt&&(s!==zt&&s!==Rt||o!==Yt)||(v=qt,f-=(d&&A===w&&w.visualViewport?w.visualViewport.width:A[T])-n.width,f*=l?1:-1)}var C,O=Object.assign({position:a},c&&He),x=!0===h?function(t,e){var i=t.x,n=t.y,s=e.devicePixelRatio||1;return{x:we(i*s)/s||0,y:we(n*s)/s||0}}({x:f,y:m},fe(i)):{x:f,y:m};return f=x.x,m=x.y,l?Object.assign({},O,((C={})[y]=b?"0":"",C[v]=_?"0":"",C.transform=(w.devicePixelRatio||1)<=1?"translate("+f+"px, "+m+"px)":"translate3d("+f+"px, "+m+"px, 0)",C)):Object.assign({},O,((e={})[y]=b?m+"px":"",e[v]=_?f+"px":"",e.transform="",e))}const Be={name:"computeStyles",enabled:!0,phase:"beforeWrite",fn:function(t){var e=t.state,i=t.options,n=i.gpuAcceleration,s=void 0===n||n,o=i.adaptive,r=void 0===o||o,a=i.roundOffsets,l=void 0===a||a,c={placement:be(e.placement),variation:Fe(e.placement),popper:e.elements.popper,popperRect:e.rects.popper,gpuAcceleration:s,isFixed:"fixed"===e.options.strategy};null!=e.modifiersData.popperOffsets&&(e.styles.popper=Object.assign({},e.styles.popper,We(Object.assign({},c,{offsets:e.modifiersData.popperOffsets,position:e.options.strategy,adaptive:r,roundOffsets:l})))),null!=e.modifiersData.arrow&&(e.styles.arrow=Object.assign({},e.styles.arrow,We(Object.assign({},c,{offsets:e.modifiersData.arrow,position:"absolute",adaptive:!1,roundOffsets:l})))),e.attributes.popper=Object.assign({},e.attributes.popper,{"data-popper-placement":e.placement})},data:{}};var ze={passive:!0};const Re={name:"eventListeners",enabled:!0,phase:"write",fn:function(){},effect:function(t){var e=t.state,i=t.instance,n=t.options,s=n.scroll,o=void 0===s||s,r=n.resize,a=void 0===r||r,l=fe(e.elements.popper),c=[].concat(e.scrollParents.reference,e.scrollParents.popper);return o&&c.forEach((function(t){t.addEventListener("scroll",i.update,ze)})),a&&l.addEventListener("resize",i.update,ze),function(){o&&c.forEach((function(t){t.removeEventListener("scroll",i.update,ze)})),a&&l.removeEventListener("resize",i.update,ze)}},data:{}};var qe={left:"right",right:"left",bottom:"top",top:"bottom"};function Ve(t){return t.replace(/left|right|bottom|top/g,(function(t){return qe[t]}))}var Ke={start:"end",end:"start"};function Qe(t){return t.replace(/start|end/g,(function(t){return Ke[t]}))}function Xe(t){var e=fe(t);return{scrollLeft:e.pageXOffset,scrollTop:e.pageYOffset}}function Ye(t){return Te(Le(t)).left+Xe(t).scrollLeft}function Ue(t){var e=xe(t),i=e.overflow,n=e.overflowX,s=e.overflowY;return/auto|scroll|overlay|hidden/.test(i+s+n)}function Ge(t){return["html","body","#document"].indexOf(ue(t))>=0?t.ownerDocument.body:me(t)&&Ue(t)?t:Ge(Se(t))}function Je(t,e){var i;void 0===e&&(e=[]);var n=Ge(t),s=n===(null==(i=t.ownerDocument)?void 0:i.body),o=fe(n),r=s?[o].concat(o.visualViewport||[],Ue(n)?n:[]):n,a=e.concat(r);return s?a:a.concat(Je(Se(r)))}function Ze(t){return Object.assign({},t,{left:t.x,top:t.y,right:t.x+t.width,bottom:t.y+t.height})}function ti(t,e,i){return e===Gt?Ze(function(t,e){var i=fe(t),n=Le(t),s=i.visualViewport,o=n.clientWidth,r=n.clientHeight,a=0,l=0;if(s){o=s.width,r=s.height;var c=Ee();(c||!c&&"fixed"===e)&&(a=s.offsetLeft,l=s.offsetTop)}return{width:o,height:r,x:a+Ye(t),y:l}}(t,i)):pe(e)?function(t,e){var i=Te(t,!1,"fixed"===e);return i.top=i.top+t.clientTop,i.left=i.left+t.clientLeft,i.bottom=i.top+t.clientHeight,i.right=i.left+t.clientWidth,i.width=t.clientWidth,i.height=t.clientHeight,i.x=i.left,i.y=i.top,i}(e,i):Ze(function(t){var e,i=Le(t),n=Xe(t),s=null==(e=t.ownerDocument)?void 0:e.body,o=ve(i.scrollWidth,i.clientWidth,s?s.scrollWidth:0,s?s.clientWidth:0),r=ve(i.scrollHeight,i.clientHeight,s?s.scrollHeight:0,s?s.clientHeight:0),a=-n.scrollLeft+Ye(t),l=-n.scrollTop;return"rtl"===xe(s||i).direction&&(a+=ve(i.clientWidth,s?s.clientWidth:0)-o),{width:o,height:r,x:a,y:l}}(Le(t)))}function ei(t){var e,i=t.reference,n=t.element,s=t.placement,o=s?be(s):null,r=s?Fe(s):null,a=i.x+i.width/2-n.width/2,l=i.y+i.height/2-n.height/2;switch(o){case zt:e={x:a,y:i.y-n.height};break;case Rt:e={x:a,y:i.y+i.height};break;case qt:e={x:i.x+i.width,y:l};break;case Vt:e={x:i.x-n.width,y:l};break;default:e={x:i.x,y:i.y}}var c=o?Ie(o):null;if(null!=c){var h="y"===c?"height":"width";switch(r){case Xt:e[c]=e[c]-(i[h]/2-n[h]/2);break;case Yt:e[c]=e[c]+(i[h]/2-n[h]/2)}}return e}function ii(t,e){void 0===e&&(e={});var i=e,n=i.placement,s=void 0===n?t.placement:n,o=i.strategy,r=void 0===o?t.strategy:o,a=i.boundary,l=void 0===a?Ut:a,c=i.rootBoundary,h=void 0===c?Gt:c,d=i.elementContext,u=void 0===d?Jt:d,f=i.altBoundary,p=void 0!==f&&f,m=i.padding,g=void 0===m?0:m,_=Pe("number"!=typeof g?g:Me(g,Qt)),b=u===Jt?Zt:Jt,v=t.rects.popper,y=t.elements[p?b:u],w=function(t,e,i,n){var s="clippingParents"===e?function(t){var e=Je(Se(t)),i=["absolute","fixed"].indexOf(xe(t).position)>=0&&me(t)?$e(t):t;return pe(i)?e.filter((function(t){return pe(t)&&Oe(t,i)&&"body"!==ue(t)})):[]}(t):[].concat(e),o=[].concat(s,[i]),r=o[0],a=o.reduce((function(e,i){var s=ti(t,i,n);return e.top=ve(s.top,e.top),e.right=ye(s.right,e.right),e.bottom=ye(s.bottom,e.bottom),e.left=ve(s.left,e.left),e}),ti(t,r,n));return a.width=a.right-a.left,a.height=a.bottom-a.top,a.x=a.left,a.y=a.top,a}(pe(y)?y:y.contextElement||Le(t.elements.popper),l,h,r),A=Te(t.elements.reference),E=ei({reference:A,element:v,strategy:"absolute",placement:s}),T=Ze(Object.assign({},v,E)),C=u===Jt?T:A,O={top:w.top-C.top+_.top,bottom:C.bottom-w.bottom+_.bottom,left:w.left-C.left+_.left,right:C.right-w.right+_.right},x=t.modifiersData.offset;if(u===Jt&&x){var k=x[s];Object.keys(O).forEach((function(t){var e=[qt,Rt].indexOf(t)>=0?1:-1,i=[zt,Rt].indexOf(t)>=0?"y":"x";O[t]+=k[i]*e}))}return O}function ni(t,e){void 0===e&&(e={});var i=e,n=i.placement,s=i.boundary,o=i.rootBoundary,r=i.padding,a=i.flipVariations,l=i.allowedAutoPlacements,c=void 0===l?ee:l,h=Fe(n),d=h?a?te:te.filter((function(t){return Fe(t)===h})):Qt,u=d.filter((function(t){return c.indexOf(t)>=0}));0===u.length&&(u=d);var f=u.reduce((function(e,i){return e[i]=ii(t,{placement:i,boundary:s,rootBoundary:o,padding:r})[be(i)],e}),{});return Object.keys(f).sort((function(t,e){return f[t]-f[e]}))}const si={name:"flip",enabled:!0,phase:"main",fn:function(t){var e=t.state,i=t.options,n=t.name;if(!e.modifiersData[n]._skip){for(var s=i.mainAxis,o=void 0===s||s,r=i.altAxis,a=void 0===r||r,l=i.fallbackPlacements,c=i.padding,h=i.boundary,d=i.rootBoundary,u=i.altBoundary,f=i.flipVariations,p=void 0===f||f,m=i.allowedAutoPlacements,g=e.options.placement,_=be(g),b=l||(_!==g&&p?function(t){if(be(t)===Kt)return[];var e=Ve(t);return[Qe(t),e,Qe(e)]}(g):[Ve(g)]),v=[g].concat(b).reduce((function(t,i){return t.concat(be(i)===Kt?ni(e,{placement:i,boundary:h,rootBoundary:d,padding:c,flipVariations:p,allowedAutoPlacements:m}):i)}),[]),y=e.rects.reference,w=e.rects.popper,A=new Map,E=!0,T=v[0],C=0;C=0,S=L?"width":"height",D=ii(e,{placement:O,boundary:h,rootBoundary:d,altBoundary:u,padding:c}),$=L?k?qt:Vt:k?Rt:zt;y[S]>w[S]&&($=Ve($));var I=Ve($),N=[];if(o&&N.push(D[x]<=0),a&&N.push(D[$]<=0,D[I]<=0),N.every((function(t){return t}))){T=O,E=!1;break}A.set(O,N)}if(E)for(var P=function(t){var e=v.find((function(e){var i=A.get(e);if(i)return i.slice(0,t).every((function(t){return t}))}));if(e)return T=e,"break"},M=p?3:1;M>0&&"break"!==P(M);M--);e.placement!==T&&(e.modifiersData[n]._skip=!0,e.placement=T,e.reset=!0)}},requiresIfExists:["offset"],data:{_skip:!1}};function oi(t,e,i){return void 0===i&&(i={x:0,y:0}),{top:t.top-e.height-i.y,right:t.right-e.width+i.x,bottom:t.bottom-e.height+i.y,left:t.left-e.width-i.x}}function ri(t){return[zt,qt,Rt,Vt].some((function(e){return t[e]>=0}))}const ai={name:"hide",enabled:!0,phase:"main",requiresIfExists:["preventOverflow"],fn:function(t){var e=t.state,i=t.name,n=e.rects.reference,s=e.rects.popper,o=e.modifiersData.preventOverflow,r=ii(e,{elementContext:"reference"}),a=ii(e,{altBoundary:!0}),l=oi(r,n),c=oi(a,s,o),h=ri(l),d=ri(c);e.modifiersData[i]={referenceClippingOffsets:l,popperEscapeOffsets:c,isReferenceHidden:h,hasPopperEscaped:d},e.attributes.popper=Object.assign({},e.attributes.popper,{"data-popper-reference-hidden":h,"data-popper-escaped":d})}},li={name:"offset",enabled:!0,phase:"main",requires:["popperOffsets"],fn:function(t){var e=t.state,i=t.options,n=t.name,s=i.offset,o=void 0===s?[0,0]:s,r=ee.reduce((function(t,i){return t[i]=function(t,e,i){var n=be(t),s=[Vt,zt].indexOf(n)>=0?-1:1,o="function"==typeof i?i(Object.assign({},e,{placement:t})):i,r=o[0],a=o[1];return r=r||0,a=(a||0)*s,[Vt,qt].indexOf(n)>=0?{x:a,y:r}:{x:r,y:a}}(i,e.rects,o),t}),{}),a=r[e.placement],l=a.x,c=a.y;null!=e.modifiersData.popperOffsets&&(e.modifiersData.popperOffsets.x+=l,e.modifiersData.popperOffsets.y+=c),e.modifiersData[n]=r}},ci={name:"popperOffsets",enabled:!0,phase:"read",fn:function(t){var e=t.state,i=t.name;e.modifiersData[i]=ei({reference:e.rects.reference,element:e.rects.popper,strategy:"absolute",placement:e.placement})},data:{}},hi={name:"preventOverflow",enabled:!0,phase:"main",fn:function(t){var e=t.state,i=t.options,n=t.name,s=i.mainAxis,o=void 0===s||s,r=i.altAxis,a=void 0!==r&&r,l=i.boundary,c=i.rootBoundary,h=i.altBoundary,d=i.padding,u=i.tether,f=void 0===u||u,p=i.tetherOffset,m=void 0===p?0:p,g=ii(e,{boundary:l,rootBoundary:c,padding:d,altBoundary:h}),_=be(e.placement),b=Fe(e.placement),v=!b,y=Ie(_),w="x"===y?"y":"x",A=e.modifiersData.popperOffsets,E=e.rects.reference,T=e.rects.popper,C="function"==typeof m?m(Object.assign({},e.rects,{placement:e.placement})):m,O="number"==typeof C?{mainAxis:C,altAxis:C}:Object.assign({mainAxis:0,altAxis:0},C),x=e.modifiersData.offset?e.modifiersData.offset[e.placement]:null,k={x:0,y:0};if(A){if(o){var L,S="y"===y?zt:Vt,D="y"===y?Rt:qt,$="y"===y?"height":"width",I=A[y],N=I+g[S],P=I-g[D],M=f?-T[$]/2:0,j=b===Xt?E[$]:T[$],F=b===Xt?-T[$]:-E[$],H=e.elements.arrow,W=f&&H?Ce(H):{width:0,height:0},B=e.modifiersData["arrow#persistent"]?e.modifiersData["arrow#persistent"].padding:{top:0,right:0,bottom:0,left:0},z=B[S],R=B[D],q=Ne(0,E[$],W[$]),V=v?E[$]/2-M-q-z-O.mainAxis:j-q-z-O.mainAxis,K=v?-E[$]/2+M+q+R+O.mainAxis:F+q+R+O.mainAxis,Q=e.elements.arrow&&$e(e.elements.arrow),X=Q?"y"===y?Q.clientTop||0:Q.clientLeft||0:0,Y=null!=(L=null==x?void 0:x[y])?L:0,U=I+K-Y,G=Ne(f?ye(N,I+V-Y-X):N,I,f?ve(P,U):P);A[y]=G,k[y]=G-I}if(a){var J,Z="x"===y?zt:Vt,tt="x"===y?Rt:qt,et=A[w],it="y"===w?"height":"width",nt=et+g[Z],st=et-g[tt],ot=-1!==[zt,Vt].indexOf(_),rt=null!=(J=null==x?void 0:x[w])?J:0,at=ot?nt:et-E[it]-T[it]-rt+O.altAxis,lt=ot?et+E[it]+T[it]-rt-O.altAxis:st,ct=f&&ot?function(t,e,i){var n=Ne(t,e,i);return n>i?i:n}(at,et,lt):Ne(f?at:nt,et,f?lt:st);A[w]=ct,k[w]=ct-et}e.modifiersData[n]=k}},requiresIfExists:["offset"]};function di(t,e,i){void 0===i&&(i=!1);var n,s,o=me(e),r=me(e)&&function(t){var e=t.getBoundingClientRect(),i=we(e.width)/t.offsetWidth||1,n=we(e.height)/t.offsetHeight||1;return 1!==i||1!==n}(e),a=Le(e),l=Te(t,r,i),c={scrollLeft:0,scrollTop:0},h={x:0,y:0};return(o||!o&&!i)&&(("body"!==ue(e)||Ue(a))&&(c=(n=e)!==fe(n)&&me(n)?{scrollLeft:(s=n).scrollLeft,scrollTop:s.scrollTop}:Xe(n)),me(e)?((h=Te(e,!0)).x+=e.clientLeft,h.y+=e.clientTop):a&&(h.x=Ye(a))),{x:l.left+c.scrollLeft-h.x,y:l.top+c.scrollTop-h.y,width:l.width,height:l.height}}function ui(t){var e=new Map,i=new Set,n=[];function s(t){i.add(t.name),[].concat(t.requires||[],t.requiresIfExists||[]).forEach((function(t){if(!i.has(t)){var n=e.get(t);n&&s(n)}})),n.push(t)}return t.forEach((function(t){e.set(t.name,t)})),t.forEach((function(t){i.has(t.name)||s(t)})),n}var fi={placement:"bottom",modifiers:[],strategy:"absolute"};function pi(){for(var t=arguments.length,e=new Array(t),i=0;iNumber.parseInt(t,10))):"function"==typeof t?e=>t(e,this._element):t}_getPopperConfig(){const t={placement:this._getPlacement(),modifiers:[{name:"preventOverflow",options:{boundary:this._config.boundary}},{name:"offset",options:{offset:this._getOffset()}}]};return(this._inNavbar||"static"===this._config.display)&&(F.setDataAttribute(this._menu,"popper","static"),t.modifiers=[{name:"applyStyles",enabled:!1}]),{...t,...g(this._config.popperConfig,[t])}}_selectMenuItem({key:t,target:e}){const i=z.find(".dropdown-menu .dropdown-item:not(.disabled):not(:disabled)",this._menu).filter((t=>a(t)));i.length&&b(i,e,t===Ti,!i.includes(e)).focus()}static jQueryInterface(t){return this.each((function(){const e=qi.getOrCreateInstance(this,t);if("string"==typeof t){if(void 0===e[t])throw new TypeError(`No method named "${t}"`);e[t]()}}))}static clearMenus(t){if(2===t.button||"keyup"===t.type&&"Tab"!==t.key)return;const e=z.find(Ni);for(const i of e){const e=qi.getInstance(i);if(!e||!1===e._config.autoClose)continue;const n=t.composedPath(),s=n.includes(e._menu);if(n.includes(e._element)||"inside"===e._config.autoClose&&!s||"outside"===e._config.autoClose&&s)continue;if(e._menu.contains(t.target)&&("keyup"===t.type&&"Tab"===t.key||/input|select|option|textarea|form/i.test(t.target.tagName)))continue;const o={relatedTarget:e._element};"click"===t.type&&(o.clickEvent=t),e._completeHide(o)}}static dataApiKeydownHandler(t){const e=/input|textarea/i.test(t.target.tagName),i="Escape"===t.key,n=[Ei,Ti].includes(t.key);if(!n&&!i)return;if(e&&!i)return;t.preventDefault();const s=this.matches(Ii)?this:z.prev(this,Ii)[0]||z.next(this,Ii)[0]||z.findOne(Ii,t.delegateTarget.parentNode),o=qi.getOrCreateInstance(s);if(n)return t.stopPropagation(),o.show(),void o._selectMenuItem(t);o._isShown()&&(t.stopPropagation(),o.hide(),s.focus())}}N.on(document,Si,Ii,qi.dataApiKeydownHandler),N.on(document,Si,Pi,qi.dataApiKeydownHandler),N.on(document,Li,qi.clearMenus),N.on(document,Di,qi.clearMenus),N.on(document,Li,Ii,(function(t){t.preventDefault(),qi.getOrCreateInstance(this).toggle()})),m(qi);const Vi="backdrop",Ki="show",Qi=`mousedown.bs.${Vi}`,Xi={className:"modal-backdrop",clickCallback:null,isAnimated:!1,isVisible:!0,rootElement:"body"},Yi={className:"string",clickCallback:"(function|null)",isAnimated:"boolean",isVisible:"boolean",rootElement:"(element|string)"};class Ui extends H{constructor(t){super(),this._config=this._getConfig(t),this._isAppended=!1,this._element=null}static get Default(){return Xi}static get DefaultType(){return Yi}static get NAME(){return Vi}show(t){if(!this._config.isVisible)return void g(t);this._append();const e=this._getElement();this._config.isAnimated&&d(e),e.classList.add(Ki),this._emulateAnimation((()=>{g(t)}))}hide(t){this._config.isVisible?(this._getElement().classList.remove(Ki),this._emulateAnimation((()=>{this.dispose(),g(t)}))):g(t)}dispose(){this._isAppended&&(N.off(this._element,Qi),this._element.remove(),this._isAppended=!1)}_getElement(){if(!this._element){const t=document.createElement("div");t.className=this._config.className,this._config.isAnimated&&t.classList.add("fade"),this._element=t}return this._element}_configAfterMerge(t){return t.rootElement=r(t.rootElement),t}_append(){if(this._isAppended)return;const t=this._getElement();this._config.rootElement.append(t),N.on(t,Qi,(()=>{g(this._config.clickCallback)})),this._isAppended=!0}_emulateAnimation(t){_(t,this._getElement(),this._config.isAnimated)}}const Gi=".bs.focustrap",Ji=`focusin${Gi}`,Zi=`keydown.tab${Gi}`,tn="backward",en={autofocus:!0,trapElement:null},nn={autofocus:"boolean",trapElement:"element"};class sn extends H{constructor(t){super(),this._config=this._getConfig(t),this._isActive=!1,this._lastTabNavDirection=null}static get Default(){return en}static get DefaultType(){return nn}static get NAME(){return"focustrap"}activate(){this._isActive||(this._config.autofocus&&this._config.trapElement.focus(),N.off(document,Gi),N.on(document,Ji,(t=>this._handleFocusin(t))),N.on(document,Zi,(t=>this._handleKeydown(t))),this._isActive=!0)}deactivate(){this._isActive&&(this._isActive=!1,N.off(document,Gi))}_handleFocusin(t){const{trapElement:e}=this._config;if(t.target===document||t.target===e||e.contains(t.target))return;const i=z.focusableChildren(e);0===i.length?e.focus():this._lastTabNavDirection===tn?i[i.length-1].focus():i[0].focus()}_handleKeydown(t){"Tab"===t.key&&(this._lastTabNavDirection=t.shiftKey?tn:"forward")}}const on=".fixed-top, .fixed-bottom, .is-fixed, .sticky-top",rn=".sticky-top",an="padding-right",ln="margin-right";class cn{constructor(){this._element=document.body}getWidth(){const t=document.documentElement.clientWidth;return Math.abs(window.innerWidth-t)}hide(){const t=this.getWidth();this._disableOverFlow(),this._setElementAttributes(this._element,an,(e=>e+t)),this._setElementAttributes(on,an,(e=>e+t)),this._setElementAttributes(rn,ln,(e=>e-t))}reset(){this._resetElementAttributes(this._element,"overflow"),this._resetElementAttributes(this._element,an),this._resetElementAttributes(on,an),this._resetElementAttributes(rn,ln)}isOverflowing(){return this.getWidth()>0}_disableOverFlow(){this._saveInitialAttribute(this._element,"overflow"),this._element.style.overflow="hidden"}_setElementAttributes(t,e,i){const n=this.getWidth();this._applyManipulationCallback(t,(t=>{if(t!==this._element&&window.innerWidth>t.clientWidth+n)return;this._saveInitialAttribute(t,e);const s=window.getComputedStyle(t).getPropertyValue(e);t.style.setProperty(e,`${i(Number.parseFloat(s))}px`)}))}_saveInitialAttribute(t,e){const i=t.style.getPropertyValue(e);i&&F.setDataAttribute(t,e,i)}_resetElementAttributes(t,e){this._applyManipulationCallback(t,(t=>{const i=F.getDataAttribute(t,e);null!==i?(F.removeDataAttribute(t,e),t.style.setProperty(e,i)):t.style.removeProperty(e)}))}_applyManipulationCallback(t,e){if(o(t))e(t);else for(const i of z.find(t,this._element))e(i)}}const hn=".bs.modal",dn=`hide${hn}`,un=`hidePrevented${hn}`,fn=`hidden${hn}`,pn=`show${hn}`,mn=`shown${hn}`,gn=`resize${hn}`,_n=`click.dismiss${hn}`,bn=`mousedown.dismiss${hn}`,vn=`keydown.dismiss${hn}`,yn=`click${hn}.data-api`,wn="modal-open",An="show",En="modal-static",Tn={backdrop:!0,focus:!0,keyboard:!0},Cn={backdrop:"(boolean|string)",focus:"boolean",keyboard:"boolean"};class On extends W{constructor(t,e){super(t,e),this._dialog=z.findOne(".modal-dialog",this._element),this._backdrop=this._initializeBackDrop(),this._focustrap=this._initializeFocusTrap(),this._isShown=!1,this._isTransitioning=!1,this._scrollBar=new cn,this._addEventListeners()}static get Default(){return Tn}static get DefaultType(){return Cn}static get NAME(){return"modal"}toggle(t){return this._isShown?this.hide():this.show(t)}show(t){this._isShown||this._isTransitioning||N.trigger(this._element,pn,{relatedTarget:t}).defaultPrevented||(this._isShown=!0,this._isTransitioning=!0,this._scrollBar.hide(),document.body.classList.add(wn),this._adjustDialog(),this._backdrop.show((()=>this._showElement(t))))}hide(){this._isShown&&!this._isTransitioning&&(N.trigger(this._element,dn).defaultPrevented||(this._isShown=!1,this._isTransitioning=!0,this._focustrap.deactivate(),this._element.classList.remove(An),this._queueCallback((()=>this._hideModal()),this._element,this._isAnimated())))}dispose(){N.off(window,hn),N.off(this._dialog,hn),this._backdrop.dispose(),this._focustrap.deactivate(),super.dispose()}handleUpdate(){this._adjustDialog()}_initializeBackDrop(){return new Ui({isVisible:Boolean(this._config.backdrop),isAnimated:this._isAnimated()})}_initializeFocusTrap(){return new sn({trapElement:this._element})}_showElement(t){document.body.contains(this._element)||document.body.append(this._element),this._element.style.display="block",this._element.removeAttribute("aria-hidden"),this._element.setAttribute("aria-modal",!0),this._element.setAttribute("role","dialog"),this._element.scrollTop=0;const e=z.findOne(".modal-body",this._dialog);e&&(e.scrollTop=0),d(this._element),this._element.classList.add(An),this._queueCallback((()=>{this._config.focus&&this._focustrap.activate(),this._isTransitioning=!1,N.trigger(this._element,mn,{relatedTarget:t})}),this._dialog,this._isAnimated())}_addEventListeners(){N.on(this._element,vn,(t=>{"Escape"===t.key&&(this._config.keyboard?this.hide():this._triggerBackdropTransition())})),N.on(window,gn,(()=>{this._isShown&&!this._isTransitioning&&this._adjustDialog()})),N.on(this._element,bn,(t=>{N.one(this._element,_n,(e=>{this._element===t.target&&this._element===e.target&&("static"!==this._config.backdrop?this._config.backdrop&&this.hide():this._triggerBackdropTransition())}))}))}_hideModal(){this._element.style.display="none",this._element.setAttribute("aria-hidden",!0),this._element.removeAttribute("aria-modal"),this._element.removeAttribute("role"),this._isTransitioning=!1,this._backdrop.hide((()=>{document.body.classList.remove(wn),this._resetAdjustments(),this._scrollBar.reset(),N.trigger(this._element,fn)}))}_isAnimated(){return this._element.classList.contains("fade")}_triggerBackdropTransition(){if(N.trigger(this._element,un).defaultPrevented)return;const t=this._element.scrollHeight>document.documentElement.clientHeight,e=this._element.style.overflowY;"hidden"===e||this._element.classList.contains(En)||(t||(this._element.style.overflowY="hidden"),this._element.classList.add(En),this._queueCallback((()=>{this._element.classList.remove(En),this._queueCallback((()=>{this._element.style.overflowY=e}),this._dialog)}),this._dialog),this._element.focus())}_adjustDialog(){const t=this._element.scrollHeight>document.documentElement.clientHeight,e=this._scrollBar.getWidth(),i=e>0;if(i&&!t){const t=p()?"paddingLeft":"paddingRight";this._element.style[t]=`${e}px`}if(!i&&t){const t=p()?"paddingRight":"paddingLeft";this._element.style[t]=`${e}px`}}_resetAdjustments(){this._element.style.paddingLeft="",this._element.style.paddingRight=""}static jQueryInterface(t,e){return this.each((function(){const i=On.getOrCreateInstance(this,t);if("string"==typeof t){if(void 0===i[t])throw new TypeError(`No method named "${t}"`);i[t](e)}}))}}N.on(document,yn,'[data-bs-toggle="modal"]',(function(t){const e=z.getElementFromSelector(this);["A","AREA"].includes(this.tagName)&&t.preventDefault(),N.one(e,pn,(t=>{t.defaultPrevented||N.one(e,fn,(()=>{a(this)&&this.focus()}))}));const i=z.findOne(".modal.show");i&&On.getInstance(i).hide(),On.getOrCreateInstance(e).toggle(this)})),R(On),m(On);const xn=".bs.offcanvas",kn=".data-api",Ln=`load${xn}${kn}`,Sn="show",Dn="showing",$n="hiding",In=".offcanvas.show",Nn=`show${xn}`,Pn=`shown${xn}`,Mn=`hide${xn}`,jn=`hidePrevented${xn}`,Fn=`hidden${xn}`,Hn=`resize${xn}`,Wn=`click${xn}${kn}`,Bn=`keydown.dismiss${xn}`,zn={backdrop:!0,keyboard:!0,scroll:!1},Rn={backdrop:"(boolean|string)",keyboard:"boolean",scroll:"boolean"};class qn extends W{constructor(t,e){super(t,e),this._isShown=!1,this._backdrop=this._initializeBackDrop(),this._focustrap=this._initializeFocusTrap(),this._addEventListeners()}static get Default(){return zn}static get DefaultType(){return Rn}static get NAME(){return"offcanvas"}toggle(t){return this._isShown?this.hide():this.show(t)}show(t){this._isShown||N.trigger(this._element,Nn,{relatedTarget:t}).defaultPrevented||(this._isShown=!0,this._backdrop.show(),this._config.scroll||(new cn).hide(),this._element.setAttribute("aria-modal",!0),this._element.setAttribute("role","dialog"),this._element.classList.add(Dn),this._queueCallback((()=>{this._config.scroll&&!this._config.backdrop||this._focustrap.activate(),this._element.classList.add(Sn),this._element.classList.remove(Dn),N.trigger(this._element,Pn,{relatedTarget:t})}),this._element,!0))}hide(){this._isShown&&(N.trigger(this._element,Mn).defaultPrevented||(this._focustrap.deactivate(),this._element.blur(),this._isShown=!1,this._element.classList.add($n),this._backdrop.hide(),this._queueCallback((()=>{this._element.classList.remove(Sn,$n),this._element.removeAttribute("aria-modal"),this._element.removeAttribute("role"),this._config.scroll||(new cn).reset(),N.trigger(this._element,Fn)}),this._element,!0)))}dispose(){this._backdrop.dispose(),this._focustrap.deactivate(),super.dispose()}_initializeBackDrop(){const t=Boolean(this._config.backdrop);return new Ui({className:"offcanvas-backdrop",isVisible:t,isAnimated:!0,rootElement:this._element.parentNode,clickCallback:t?()=>{"static"!==this._config.backdrop?this.hide():N.trigger(this._element,jn)}:null})}_initializeFocusTrap(){return new sn({trapElement:this._element})}_addEventListeners(){N.on(this._element,Bn,(t=>{"Escape"===t.key&&(this._config.keyboard?this.hide():N.trigger(this._element,jn))}))}static jQueryInterface(t){return this.each((function(){const e=qn.getOrCreateInstance(this,t);if("string"==typeof t){if(void 0===e[t]||t.startsWith("_")||"constructor"===t)throw new TypeError(`No method named "${t}"`);e[t](this)}}))}}N.on(document,Wn,'[data-bs-toggle="offcanvas"]',(function(t){const e=z.getElementFromSelector(this);if(["A","AREA"].includes(this.tagName)&&t.preventDefault(),l(this))return;N.one(e,Fn,(()=>{a(this)&&this.focus()}));const i=z.findOne(In);i&&i!==e&&qn.getInstance(i).hide(),qn.getOrCreateInstance(e).toggle(this)})),N.on(window,Ln,(()=>{for(const t of z.find(In))qn.getOrCreateInstance(t).show()})),N.on(window,Hn,(()=>{for(const t of z.find("[aria-modal][class*=show][class*=offcanvas-]"))"fixed"!==getComputedStyle(t).position&&qn.getOrCreateInstance(t).hide()})),R(qn),m(qn);const Vn={"*":["class","dir","id","lang","role",/^aria-[\w-]*$/i],a:["target","href","title","rel"],area:[],b:[],br:[],col:[],code:[],div:[],em:[],hr:[],h1:[],h2:[],h3:[],h4:[],h5:[],h6:[],i:[],img:["src","srcset","alt","title","width","height"],li:[],ol:[],p:[],pre:[],s:[],small:[],span:[],sub:[],sup:[],strong:[],u:[],ul:[]},Kn=new Set(["background","cite","href","itemtype","longdesc","poster","src","xlink:href"]),Qn=/^(?!javascript:)(?:[a-z0-9+.-]+:|[^&:/?#]*(?:[/?#]|$))/i,Xn=(t,e)=>{const i=t.nodeName.toLowerCase();return e.includes(i)?!Kn.has(i)||Boolean(Qn.test(t.nodeValue)):e.filter((t=>t instanceof RegExp)).some((t=>t.test(i)))},Yn={allowList:Vn,content:{},extraClass:"",html:!1,sanitize:!0,sanitizeFn:null,template:"
"},Un={allowList:"object",content:"object",extraClass:"(string|function)",html:"boolean",sanitize:"boolean",sanitizeFn:"(null|function)",template:"string"},Gn={entry:"(string|element|function|null)",selector:"(string|element)"};class Jn extends H{constructor(t){super(),this._config=this._getConfig(t)}static get Default(){return Yn}static get DefaultType(){return Un}static get NAME(){return"TemplateFactory"}getContent(){return Object.values(this._config.content).map((t=>this._resolvePossibleFunction(t))).filter(Boolean)}hasContent(){return this.getContent().length>0}changeContent(t){return this._checkContent(t),this._config.content={...this._config.content,...t},this}toHtml(){const t=document.createElement("div");t.innerHTML=this._maybeSanitize(this._config.template);for(const[e,i]of Object.entries(this._config.content))this._setContent(t,i,e);const e=t.children[0],i=this._resolvePossibleFunction(this._config.extraClass);return i&&e.classList.add(...i.split(" ")),e}_typeCheckConfig(t){super._typeCheckConfig(t),this._checkContent(t.content)}_checkContent(t){for(const[e,i]of Object.entries(t))super._typeCheckConfig({selector:e,entry:i},Gn)}_setContent(t,e,i){const n=z.findOne(i,t);n&&((e=this._resolvePossibleFunction(e))?o(e)?this._putElementInTemplate(r(e),n):this._config.html?n.innerHTML=this._maybeSanitize(e):n.textContent=e:n.remove())}_maybeSanitize(t){return this._config.sanitize?function(t,e,i){if(!t.length)return t;if(i&&"function"==typeof i)return i(t);const n=(new window.DOMParser).parseFromString(t,"text/html"),s=[].concat(...n.body.querySelectorAll("*"));for(const t of s){const i=t.nodeName.toLowerCase();if(!Object.keys(e).includes(i)){t.remove();continue}const n=[].concat(...t.attributes),s=[].concat(e["*"]||[],e[i]||[]);for(const e of n)Xn(e,s)||t.removeAttribute(e.nodeName)}return n.body.innerHTML}(t,this._config.allowList,this._config.sanitizeFn):t}_resolvePossibleFunction(t){return g(t,[this])}_putElementInTemplate(t,e){if(this._config.html)return e.innerHTML="",void e.append(t);e.textContent=t.textContent}}const Zn=new Set(["sanitize","allowList","sanitizeFn"]),ts="fade",es="show",is=".modal",ns="hide.bs.modal",ss="hover",os="focus",rs={AUTO:"auto",TOP:"top",RIGHT:p()?"left":"right",BOTTOM:"bottom",LEFT:p()?"right":"left"},as={allowList:Vn,animation:!0,boundary:"clippingParents",container:!1,customClass:"",delay:0,fallbackPlacements:["top","right","bottom","left"],html:!1,offset:[0,6],placement:"top",popperConfig:null,sanitize:!0,sanitizeFn:null,selector:!1,template:'',title:"",trigger:"hover focus"},ls={allowList:"object",animation:"boolean",boundary:"(string|element)",container:"(string|element|boolean)",customClass:"(string|function)",delay:"(number|object)",fallbackPlacements:"array",html:"boolean",offset:"(array|string|function)",placement:"(string|function)",popperConfig:"(null|object|function)",sanitize:"boolean",sanitizeFn:"(null|function)",selector:"(string|boolean)",template:"string",title:"(string|element|function)",trigger:"string"};class cs extends W{constructor(t,e){if(void 0===vi)throw new TypeError("Bootstrap's tooltips require Popper (https://popper.js.org)");super(t,e),this._isEnabled=!0,this._timeout=0,this._isHovered=null,this._activeTrigger={},this._popper=null,this._templateFactory=null,this._newContent=null,this.tip=null,this._setListeners(),this._config.selector||this._fixTitle()}static get Default(){return as}static get DefaultType(){return ls}static get NAME(){return"tooltip"}enable(){this._isEnabled=!0}disable(){this._isEnabled=!1}toggleEnabled(){this._isEnabled=!this._isEnabled}toggle(){this._isEnabled&&(this._activeTrigger.click=!this._activeTrigger.click,this._isShown()?this._leave():this._enter())}dispose(){clearTimeout(this._timeout),N.off(this._element.closest(is),ns,this._hideModalHandler),this._element.getAttribute("data-bs-original-title")&&this._element.setAttribute("title",this._element.getAttribute("data-bs-original-title")),this._disposePopper(),super.dispose()}show(){if("none"===this._element.style.display)throw new Error("Please use show on visible elements");if(!this._isWithContent()||!this._isEnabled)return;const t=N.trigger(this._element,this.constructor.eventName("show")),e=(c(this._element)||this._element.ownerDocument.documentElement).contains(this._element);if(t.defaultPrevented||!e)return;this._disposePopper();const i=this._getTipElement();this._element.setAttribute("aria-describedby",i.getAttribute("id"));const{container:n}=this._config;if(this._element.ownerDocument.documentElement.contains(this.tip)||(n.append(i),N.trigger(this._element,this.constructor.eventName("inserted"))),this._popper=this._createPopper(i),i.classList.add(es),"ontouchstart"in document.documentElement)for(const t of[].concat(...document.body.children))N.on(t,"mouseover",h);this._queueCallback((()=>{N.trigger(this._element,this.constructor.eventName("shown")),!1===this._isHovered&&this._leave(),this._isHovered=!1}),this.tip,this._isAnimated())}hide(){if(this._isShown()&&!N.trigger(this._element,this.constructor.eventName("hide")).defaultPrevented){if(this._getTipElement().classList.remove(es),"ontouchstart"in document.documentElement)for(const t of[].concat(...document.body.children))N.off(t,"mouseover",h);this._activeTrigger.click=!1,this._activeTrigger[os]=!1,this._activeTrigger[ss]=!1,this._isHovered=null,this._queueCallback((()=>{this._isWithActiveTrigger()||(this._isHovered||this._disposePopper(),this._element.removeAttribute("aria-describedby"),N.trigger(this._element,this.constructor.eventName("hidden")))}),this.tip,this._isAnimated())}}update(){this._popper&&this._popper.update()}_isWithContent(){return Boolean(this._getTitle())}_getTipElement(){return this.tip||(this.tip=this._createTipElement(this._newContent||this._getContentForTemplate())),this.tip}_createTipElement(t){const e=this._getTemplateFactory(t).toHtml();if(!e)return null;e.classList.remove(ts,es),e.classList.add(`bs-${this.constructor.NAME}-auto`);const i=(t=>{do{t+=Math.floor(1e6*Math.random())}while(document.getElementById(t));return t})(this.constructor.NAME).toString();return e.setAttribute("id",i),this._isAnimated()&&e.classList.add(ts),e}setContent(t){this._newContent=t,this._isShown()&&(this._disposePopper(),this.show())}_getTemplateFactory(t){return this._templateFactory?this._templateFactory.changeContent(t):this._templateFactory=new Jn({...this._config,content:t,extraClass:this._resolvePossibleFunction(this._config.customClass)}),this._templateFactory}_getContentForTemplate(){return{".tooltip-inner":this._getTitle()}}_getTitle(){return this._resolvePossibleFunction(this._config.title)||this._element.getAttribute("data-bs-original-title")}_initializeOnDelegatedTarget(t){return this.constructor.getOrCreateInstance(t.delegateTarget,this._getDelegateConfig())}_isAnimated(){return this._config.animation||this.tip&&this.tip.classList.contains(ts)}_isShown(){return this.tip&&this.tip.classList.contains(es)}_createPopper(t){const e=g(this._config.placement,[this,t,this._element]),i=rs[e.toUpperCase()];return bi(this._element,t,this._getPopperConfig(i))}_getOffset(){const{offset:t}=this._config;return"string"==typeof t?t.split(",").map((t=>Number.parseInt(t,10))):"function"==typeof t?e=>t(e,this._element):t}_resolvePossibleFunction(t){return g(t,[this._element])}_getPopperConfig(t){const e={placement:t,modifiers:[{name:"flip",options:{fallbackPlacements:this._config.fallbackPlacements}},{name:"offset",options:{offset:this._getOffset()}},{name:"preventOverflow",options:{boundary:this._config.boundary}},{name:"arrow",options:{element:`.${this.constructor.NAME}-arrow`}},{name:"preSetPlacement",enabled:!0,phase:"beforeMain",fn:t=>{this._getTipElement().setAttribute("data-popper-placement",t.state.placement)}}]};return{...e,...g(this._config.popperConfig,[e])}}_setListeners(){const t=this._config.trigger.split(" ");for(const e of t)if("click"===e)N.on(this._element,this.constructor.eventName("click"),this._config.selector,(t=>{this._initializeOnDelegatedTarget(t).toggle()}));else if("manual"!==e){const t=e===ss?this.constructor.eventName("mouseenter"):this.constructor.eventName("focusin"),i=e===ss?this.constructor.eventName("mouseleave"):this.constructor.eventName("focusout");N.on(this._element,t,this._config.selector,(t=>{const e=this._initializeOnDelegatedTarget(t);e._activeTrigger["focusin"===t.type?os:ss]=!0,e._enter()})),N.on(this._element,i,this._config.selector,(t=>{const e=this._initializeOnDelegatedTarget(t);e._activeTrigger["focusout"===t.type?os:ss]=e._element.contains(t.relatedTarget),e._leave()}))}this._hideModalHandler=()=>{this._element&&this.hide()},N.on(this._element.closest(is),ns,this._hideModalHandler)}_fixTitle(){const t=this._element.getAttribute("title");t&&(this._element.getAttribute("aria-label")||this._element.textContent.trim()||this._element.setAttribute("aria-label",t),this._element.setAttribute("data-bs-original-title",t),this._element.removeAttribute("title"))}_enter(){this._isShown()||this._isHovered?this._isHovered=!0:(this._isHovered=!0,this._setTimeout((()=>{this._isHovered&&this.show()}),this._config.delay.show))}_leave(){this._isWithActiveTrigger()||(this._isHovered=!1,this._setTimeout((()=>{this._isHovered||this.hide()}),this._config.delay.hide))}_setTimeout(t,e){clearTimeout(this._timeout),this._timeout=setTimeout(t,e)}_isWithActiveTrigger(){return Object.values(this._activeTrigger).includes(!0)}_getConfig(t){const e=F.getDataAttributes(this._element);for(const t of Object.keys(e))Zn.has(t)&&delete e[t];return t={...e,..."object"==typeof t&&t?t:{}},t=this._mergeConfigObj(t),t=this._configAfterMerge(t),this._typeCheckConfig(t),t}_configAfterMerge(t){return t.container=!1===t.container?document.body:r(t.container),"number"==typeof t.delay&&(t.delay={show:t.delay,hide:t.delay}),"number"==typeof t.title&&(t.title=t.title.toString()),"number"==typeof t.content&&(t.content=t.content.toString()),t}_getDelegateConfig(){const t={};for(const[e,i]of Object.entries(this._config))this.constructor.Default[e]!==i&&(t[e]=i);return t.selector=!1,t.trigger="manual",t}_disposePopper(){this._popper&&(this._popper.destroy(),this._popper=null),this.tip&&(this.tip.remove(),this.tip=null)}static jQueryInterface(t){return this.each((function(){const e=cs.getOrCreateInstance(this,t);if("string"==typeof t){if(void 0===e[t])throw new TypeError(`No method named "${t}"`);e[t]()}}))}}m(cs);const hs={...cs.Default,content:"",offset:[0,8],placement:"right",template:'',trigger:"click"},ds={...cs.DefaultType,content:"(null|string|element|function)"};class us extends cs{static get Default(){return hs}static get DefaultType(){return ds}static get NAME(){return"popover"}_isWithContent(){return this._getTitle()||this._getContent()}_getContentForTemplate(){return{".popover-header":this._getTitle(),".popover-body":this._getContent()}}_getContent(){return this._resolvePossibleFunction(this._config.content)}static jQueryInterface(t){return this.each((function(){const e=us.getOrCreateInstance(this,t);if("string"==typeof t){if(void 0===e[t])throw new TypeError(`No method named "${t}"`);e[t]()}}))}}m(us);const fs=".bs.scrollspy",ps=`activate${fs}`,ms=`click${fs}`,gs=`load${fs}.data-api`,_s="active",bs="[href]",vs=".nav-link",ys=`${vs}, .nav-item > ${vs}, .list-group-item`,ws={offset:null,rootMargin:"0px 0px -25%",smoothScroll:!1,target:null,threshold:[.1,.5,1]},As={offset:"(number|null)",rootMargin:"string",smoothScroll:"boolean",target:"element",threshold:"array"};class Es extends W{constructor(t,e){super(t,e),this._targetLinks=new Map,this._observableSections=new Map,this._rootElement="visible"===getComputedStyle(this._element).overflowY?null:this._element,this._activeTarget=null,this._observer=null,this._previousScrollData={visibleEntryTop:0,parentScrollTop:0},this.refresh()}static get Default(){return ws}static get DefaultType(){return As}static get NAME(){return"scrollspy"}refresh(){this._initializeTargetsAndObservables(),this._maybeEnableSmoothScroll(),this._observer?this._observer.disconnect():this._observer=this._getNewObserver();for(const t of this._observableSections.values())this._observer.observe(t)}dispose(){this._observer.disconnect(),super.dispose()}_configAfterMerge(t){return t.target=r(t.target)||document.body,t.rootMargin=t.offset?`${t.offset}px 0px -30%`:t.rootMargin,"string"==typeof t.threshold&&(t.threshold=t.threshold.split(",").map((t=>Number.parseFloat(t)))),t}_maybeEnableSmoothScroll(){this._config.smoothScroll&&(N.off(this._config.target,ms),N.on(this._config.target,ms,bs,(t=>{const e=this._observableSections.get(t.target.hash);if(e){t.preventDefault();const i=this._rootElement||window,n=e.offsetTop-this._element.offsetTop;if(i.scrollTo)return void i.scrollTo({top:n,behavior:"smooth"});i.scrollTop=n}})))}_getNewObserver(){const t={root:this._rootElement,threshold:this._config.threshold,rootMargin:this._config.rootMargin};return new IntersectionObserver((t=>this._observerCallback(t)),t)}_observerCallback(t){const e=t=>this._targetLinks.get(`#${t.target.id}`),i=t=>{this._previousScrollData.visibleEntryTop=t.target.offsetTop,this._process(e(t))},n=(this._rootElement||document.documentElement).scrollTop,s=n>=this._previousScrollData.parentScrollTop;this._previousScrollData.parentScrollTop=n;for(const o of t){if(!o.isIntersecting){this._activeTarget=null,this._clearActiveClass(e(o));continue}const t=o.target.offsetTop>=this._previousScrollData.visibleEntryTop;if(s&&t){if(i(o),!n)return}else s||t||i(o)}}_initializeTargetsAndObservables(){this._targetLinks=new Map,this._observableSections=new Map;const t=z.find(bs,this._config.target);for(const e of t){if(!e.hash||l(e))continue;const t=z.findOne(decodeURI(e.hash),this._element);a(t)&&(this._targetLinks.set(decodeURI(e.hash),e),this._observableSections.set(e.hash,t))}}_process(t){this._activeTarget!==t&&(this._clearActiveClass(this._config.target),this._activeTarget=t,t.classList.add(_s),this._activateParents(t),N.trigger(this._element,ps,{relatedTarget:t}))}_activateParents(t){if(t.classList.contains("dropdown-item"))z.findOne(".dropdown-toggle",t.closest(".dropdown")).classList.add(_s);else for(const e of z.parents(t,".nav, .list-group"))for(const t of z.prev(e,ys))t.classList.add(_s)}_clearActiveClass(t){t.classList.remove(_s);const e=z.find(`${bs}.${_s}`,t);for(const t of e)t.classList.remove(_s)}static jQueryInterface(t){return this.each((function(){const e=Es.getOrCreateInstance(this,t);if("string"==typeof t){if(void 0===e[t]||t.startsWith("_")||"constructor"===t)throw new TypeError(`No method named "${t}"`);e[t]()}}))}}N.on(window,gs,(()=>{for(const t of z.find('[data-bs-spy="scroll"]'))Es.getOrCreateInstance(t)})),m(Es);const Ts=".bs.tab",Cs=`hide${Ts}`,Os=`hidden${Ts}`,xs=`show${Ts}`,ks=`shown${Ts}`,Ls=`click${Ts}`,Ss=`keydown${Ts}`,Ds=`load${Ts}`,$s="ArrowLeft",Is="ArrowRight",Ns="ArrowUp",Ps="ArrowDown",Ms="Home",js="End",Fs="active",Hs="fade",Ws="show",Bs=":not(.dropdown-toggle)",zs='[data-bs-toggle="tab"], [data-bs-toggle="pill"], [data-bs-toggle="list"]',Rs=`.nav-link${Bs}, .list-group-item${Bs}, [role="tab"]${Bs}, ${zs}`,qs=`.${Fs}[data-bs-toggle="tab"], .${Fs}[data-bs-toggle="pill"], .${Fs}[data-bs-toggle="list"]`;class Vs extends W{constructor(t){super(t),this._parent=this._element.closest('.list-group, .nav, [role="tablist"]'),this._parent&&(this._setInitialAttributes(this._parent,this._getChildren()),N.on(this._element,Ss,(t=>this._keydown(t))))}static get NAME(){return"tab"}show(){const t=this._element;if(this._elemIsActive(t))return;const e=this._getActiveElem(),i=e?N.trigger(e,Cs,{relatedTarget:t}):null;N.trigger(t,xs,{relatedTarget:e}).defaultPrevented||i&&i.defaultPrevented||(this._deactivate(e,t),this._activate(t,e))}_activate(t,e){t&&(t.classList.add(Fs),this._activate(z.getElementFromSelector(t)),this._queueCallback((()=>{"tab"===t.getAttribute("role")?(t.removeAttribute("tabindex"),t.setAttribute("aria-selected",!0),this._toggleDropDown(t,!0),N.trigger(t,ks,{relatedTarget:e})):t.classList.add(Ws)}),t,t.classList.contains(Hs)))}_deactivate(t,e){t&&(t.classList.remove(Fs),t.blur(),this._deactivate(z.getElementFromSelector(t)),this._queueCallback((()=>{"tab"===t.getAttribute("role")?(t.setAttribute("aria-selected",!1),t.setAttribute("tabindex","-1"),this._toggleDropDown(t,!1),N.trigger(t,Os,{relatedTarget:e})):t.classList.remove(Ws)}),t,t.classList.contains(Hs)))}_keydown(t){if(![$s,Is,Ns,Ps,Ms,js].includes(t.key))return;t.stopPropagation(),t.preventDefault();const e=this._getChildren().filter((t=>!l(t)));let i;if([Ms,js].includes(t.key))i=e[t.key===Ms?0:e.length-1];else{const n=[Is,Ps].includes(t.key);i=b(e,t.target,n,!0)}i&&(i.focus({preventScroll:!0}),Vs.getOrCreateInstance(i).show())}_getChildren(){return z.find(Rs,this._parent)}_getActiveElem(){return this._getChildren().find((t=>this._elemIsActive(t)))||null}_setInitialAttributes(t,e){this._setAttributeIfNotExists(t,"role","tablist");for(const t of e)this._setInitialAttributesOnChild(t)}_setInitialAttributesOnChild(t){t=this._getInnerElement(t);const e=this._elemIsActive(t),i=this._getOuterElement(t);t.setAttribute("aria-selected",e),i!==t&&this._setAttributeIfNotExists(i,"role","presentation"),e||t.setAttribute("tabindex","-1"),this._setAttributeIfNotExists(t,"role","tab"),this._setInitialAttributesOnTargetPanel(t)}_setInitialAttributesOnTargetPanel(t){const e=z.getElementFromSelector(t);e&&(this._setAttributeIfNotExists(e,"role","tabpanel"),t.id&&this._setAttributeIfNotExists(e,"aria-labelledby",`${t.id}`))}_toggleDropDown(t,e){const i=this._getOuterElement(t);if(!i.classList.contains("dropdown"))return;const n=(t,n)=>{const s=z.findOne(t,i);s&&s.classList.toggle(n,e)};n(".dropdown-toggle",Fs),n(".dropdown-menu",Ws),i.setAttribute("aria-expanded",e)}_setAttributeIfNotExists(t,e,i){t.hasAttribute(e)||t.setAttribute(e,i)}_elemIsActive(t){return t.classList.contains(Fs)}_getInnerElement(t){return t.matches(Rs)?t:z.findOne(Rs,t)}_getOuterElement(t){return t.closest(".nav-item, .list-group-item")||t}static jQueryInterface(t){return this.each((function(){const e=Vs.getOrCreateInstance(this);if("string"==typeof t){if(void 0===e[t]||t.startsWith("_")||"constructor"===t)throw new TypeError(`No method named "${t}"`);e[t]()}}))}}N.on(document,Ls,zs,(function(t){["A","AREA"].includes(this.tagName)&&t.preventDefault(),l(this)||Vs.getOrCreateInstance(this).show()})),N.on(window,Ds,(()=>{for(const t of z.find(qs))Vs.getOrCreateInstance(t)})),m(Vs);const Ks=".bs.toast",Qs=`mouseover${Ks}`,Xs=`mouseout${Ks}`,Ys=`focusin${Ks}`,Us=`focusout${Ks}`,Gs=`hide${Ks}`,Js=`hidden${Ks}`,Zs=`show${Ks}`,to=`shown${Ks}`,eo="hide",io="show",no="showing",so={animation:"boolean",autohide:"boolean",delay:"number"},oo={animation:!0,autohide:!0,delay:5e3};class ro extends W{constructor(t,e){super(t,e),this._timeout=null,this._hasMouseInteraction=!1,this._hasKeyboardInteraction=!1,this._setListeners()}static get Default(){return oo}static get DefaultType(){return so}static get NAME(){return"toast"}show(){N.trigger(this._element,Zs).defaultPrevented||(this._clearTimeout(),this._config.animation&&this._element.classList.add("fade"),this._element.classList.remove(eo),d(this._element),this._element.classList.add(io,no),this._queueCallback((()=>{this._element.classList.remove(no),N.trigger(this._element,to),this._maybeScheduleHide()}),this._element,this._config.animation))}hide(){this.isShown()&&(N.trigger(this._element,Gs).defaultPrevented||(this._element.classList.add(no),this._queueCallback((()=>{this._element.classList.add(eo),this._element.classList.remove(no,io),N.trigger(this._element,Js)}),this._element,this._config.animation)))}dispose(){this._clearTimeout(),this.isShown()&&this._element.classList.remove(io),super.dispose()}isShown(){return this._element.classList.contains(io)}_maybeScheduleHide(){this._config.autohide&&(this._hasMouseInteraction||this._hasKeyboardInteraction||(this._timeout=setTimeout((()=>{this.hide()}),this._config.delay)))}_onInteraction(t,e){switch(t.type){case"mouseover":case"mouseout":this._hasMouseInteraction=e;break;case"focusin":case"focusout":this._hasKeyboardInteraction=e}if(e)return void this._clearTimeout();const i=t.relatedTarget;this._element===i||this._element.contains(i)||this._maybeScheduleHide()}_setListeners(){N.on(this._element,Qs,(t=>this._onInteraction(t,!0))),N.on(this._element,Xs,(t=>this._onInteraction(t,!1))),N.on(this._element,Ys,(t=>this._onInteraction(t,!0))),N.on(this._element,Us,(t=>this._onInteraction(t,!1)))}_clearTimeout(){clearTimeout(this._timeout),this._timeout=null}static jQueryInterface(t){return this.each((function(){const e=ro.getOrCreateInstance(this,t);if("string"==typeof t){if(void 0===e[t])throw new TypeError(`No method named "${t}"`);e[t](this)}}))}}return R(ro),m(ro),{Alert:Q,Button:Y,Carousel:xt,Collapse:Bt,Dropdown:qi,Modal:On,Offcanvas:qn,Popover:us,ScrollSpy:Es,Tab:Vs,Toast:ro,Tooltip:cs}})); +//# sourceMappingURL=bootstrap.bundle.min.js.map \ No newline at end of file diff --git a/site_libs/clipboard/clipboard.min.js b/site_libs/clipboard/clipboard.min.js new file mode 100644 index 0000000..1103f81 --- /dev/null +++ b/site_libs/clipboard/clipboard.min.js @@ -0,0 +1,7 @@ +/*! + * clipboard.js v2.0.11 + * https://clipboardjs.com/ + * + * Licensed MIT © Zeno Rocha + */ +!function(t,e){"object"==typeof exports&&"object"==typeof module?module.exports=e():"function"==typeof define&&define.amd?define([],e):"object"==typeof exports?exports.ClipboardJS=e():t.ClipboardJS=e()}(this,function(){return n={686:function(t,e,n){"use strict";n.d(e,{default:function(){return b}});var e=n(279),i=n.n(e),e=n(370),u=n.n(e),e=n(817),r=n.n(e);function c(t){try{return document.execCommand(t)}catch(t){return}}var a=function(t){t=r()(t);return c("cut"),t};function o(t,e){var n,o,t=(n=t,o="rtl"===document.documentElement.getAttribute("dir"),(t=document.createElement("textarea")).style.fontSize="12pt",t.style.border="0",t.style.padding="0",t.style.margin="0",t.style.position="absolute",t.style[o?"right":"left"]="-9999px",o=window.pageYOffset||document.documentElement.scrollTop,t.style.top="".concat(o,"px"),t.setAttribute("readonly",""),t.value=n,t);return e.container.appendChild(t),e=r()(t),c("copy"),t.remove(),e}var f=function(t){var e=1'); +} + +div#fullscreen-box pre .fullscreen-button .fullscreen-exit-icon { + content: url('data:image/svg+xml,'); +} + +/**************************** fullscreen buttons *************************/ + + + +/*************************** div#fullscreen-box **************************/ + +div#fullscreen-box { + margin: 0; + padding: 0; + z-index: 9999; + position: absolute; + top: 0; + right: 0; + width: 100vw; + height: 100svmax; + display: none; + background-color: rgba(233,236,239); +} + +div#fullscreen-box.visible{ + display: block; +} + +div#fullscreen-box pre.sourceCode { + background-color: rgba(233,236,239); +} + +div#fullscreen-box code { + padding-top: 0.3rem ; + padding-bottom: 1.5rem; +} + + +div#fullscreen-box pre code span a { + pointer-events: none; +} + +div#fullscreen-box code { + font-family: var(--r-code-font); +} + +div#fullscreen-box pre.numberSource code > span > a:first-child::before { + left: -1.2em !important; +} + +div#fullscreen-box pre.numberSource { + border-left: 1px solid #aaaaaa !important; + border-radius: 0; + margin-left: 4em; +} + + +/* using `auto` instead of `overlay` */ +body.reveal-viewport { + overflow: auto !important; +} + +/* for modal that open when clicking on Code button on html */ +body.modal-open { + overflow: auto !important; +} + + +.reveal-full-page div#fullscreen-box { + height: unset !important; +} + +.reveal-full-page div#fullscreen-box pre.numberSource { + border-left: 1px solid #aaaaaa !important; + border-radius: 0; + margin-left: 4em; +} + + +/********************** for mobile devices ****************************/ + +@media only screen and (max-width: 992px) { + div#fullscreen-box { + font-size: 1.5vw; + } +} + +@media only screen and (max-width: 500px) { + div#fullscreen-box pre.numberSource code > span { + left: -5em !important; + } +} + +@media only screen and (min-width: 501px) and (max-width: 768px) { + div#fullscreen-box pre.numberSource code > span { + left: -4.5em !important; + } + +} \ No newline at end of file diff --git a/site_libs/quarto-contrib/code-fullscreen-1.0.0/code-fullscreen.js b/site_libs/quarto-contrib/code-fullscreen-1.0.0/code-fullscreen.js new file mode 100644 index 0000000..054191f --- /dev/null +++ b/site_libs/quarto-contrib/code-fullscreen-1.0.0/code-fullscreen.js @@ -0,0 +1,61 @@ +/* +code modified from https://github.com/mkordulewski/reveal.js-fullscreen-code/blob/master/reveal.js-fullscreen-code.js +*/ + +/** + * code-fullscreen + * A filter that add a fullscreen button in code chunk. + * + * MIT License + * Copyright (c) 2023 Shafayet Khan Shafee. + */ + +function fullscreen_code() { + // creating the fullscreen box + $('body').append("
" ); + + // for revealjs do not link the button + if ($('div.reveal').length) { + $('pre code').before(""); + $('div#fullscreen-box button.fullscreen-button').replaceWith("") + } else { + // create buttons with links + $('pre code').before(""); + $('div#fullscreen-box button.fullscreen-button').replaceWith("") + } + + var fullscreenBoxVisible = false; + $('button.fullscreen-button').click(function(e){ + var code = this.parentElement.children[1].innerHTML; + if (fullscreenBoxVisible) { + $('div#fullscreen-box pre code').html(''); + $('div#fullscreen-box').toggleClass('visible'); + fullscreenBoxVisible = false; + } else { + $('div#fullscreen-box pre code').html(code); + $('div#fullscreen-box').toggleClass('visible'); + fullscreenBoxVisible = true; + var box_height = Math.max($('div#fullscreen-box pre').outerHeight(), $(document).height()); + $('div#fullscreen-box pre').height(box_height); + $('div#fullscreen-box').height(box_height); + // attach links for html (not revealjs) + if (!$('div.reveal').length) { + let id = this.parentElement.parentElement.id; + let id_string = `#${id}`; + $('div#fullscreen-box button.fullscreen-button a').attr("href", id_string); + }; + }; + // slow the link changing + if (!$('div.reveal').length) { + setTimeout(function() { + if (!fullscreenBoxVisible) { + $('div#fullscreen-box button.fullscreen-button a').attr("href", "#fullscreen-box"); + } + }, 100); + }; + }); +}; + +window.addEventListener("load", (event) => { + fullscreen_code(); +}); diff --git a/site_libs/quarto-contrib/jquery-3.6.3/jquery-3.6.3.min.js b/site_libs/quarto-contrib/jquery-3.6.3/jquery-3.6.3.min.js new file mode 100644 index 0000000..b5329e9 --- /dev/null +++ b/site_libs/quarto-contrib/jquery-3.6.3/jquery-3.6.3.min.js @@ -0,0 +1,2 @@ +/*! jQuery v3.6.3 | (c) OpenJS Foundation and other contributors | jquery.org/license */ +!function(e,t){"use strict";"object"==typeof module&&"object"==typeof module.exports?module.exports=e.document?t(e,!0):function(e){if(!e.document)throw new Error("jQuery requires a window with a document");return t(e)}:t(e)}("undefined"!=typeof window?window:this,function(C,e){"use strict";var t=[],r=Object.getPrototypeOf,s=t.slice,g=t.flat?function(e){return t.flat.call(e)}:function(e){return t.concat.apply([],e)},u=t.push,i=t.indexOf,n={},o=n.toString,y=n.hasOwnProperty,a=y.toString,l=a.call(Object),v={},m=function(e){return"function"==typeof e&&"number"!=typeof e.nodeType&&"function"!=typeof e.item},x=function(e){return null!=e&&e===e.window},S=C.document,c={type:!0,src:!0,nonce:!0,noModule:!0};function b(e,t,n){var r,i,o=(n=n||S).createElement("script");if(o.text=e,t)for(r in c)(i=t[r]||t.getAttribute&&t.getAttribute(r))&&o.setAttribute(r,i);n.head.appendChild(o).parentNode.removeChild(o)}function w(e){return null==e?e+"":"object"==typeof e||"function"==typeof e?n[o.call(e)]||"object":typeof e}var f="3.6.3",E=function(e,t){return new E.fn.init(e,t)};function p(e){var t=!!e&&"length"in e&&e.length,n=w(e);return!m(e)&&!x(e)&&("array"===n||0===t||"number"==typeof t&&0+~]|"+M+")"+M+"*"),U=new RegExp(M+"|>"),X=new RegExp(F),V=new RegExp("^"+I+"$"),G={ID:new RegExp("^#("+I+")"),CLASS:new RegExp("^\\.("+I+")"),TAG:new RegExp("^("+I+"|[*])"),ATTR:new RegExp("^"+W),PSEUDO:new RegExp("^"+F),CHILD:new RegExp("^:(only|first|last|nth|nth-last)-(child|of-type)(?:\\("+M+"*(even|odd|(([+-]|)(\\d*)n|)"+M+"*(?:([+-]|)"+M+"*(\\d+)|))"+M+"*\\)|)","i"),bool:new RegExp("^(?:"+R+")$","i"),needsContext:new RegExp("^"+M+"*[>+~]|:(even|odd|eq|gt|lt|nth|first|last)(?:\\("+M+"*((?:-\\d)?\\d*)"+M+"*\\)|)(?=[^-]|$)","i")},Y=/HTML$/i,Q=/^(?:input|select|textarea|button)$/i,J=/^h\d$/i,K=/^[^{]+\{\s*\[native \w/,Z=/^(?:#([\w-]+)|(\w+)|\.([\w-]+))$/,ee=/[+~]/,te=new RegExp("\\\\[\\da-fA-F]{1,6}"+M+"?|\\\\([^\\r\\n\\f])","g"),ne=function(e,t){var n="0x"+e.slice(1)-65536;return t||(n<0?String.fromCharCode(n+65536):String.fromCharCode(n>>10|55296,1023&n|56320))},re=/([\0-\x1f\x7f]|^-?\d)|^-$|[^\0-\x1f\x7f-\uFFFF\w-]/g,ie=function(e,t){return t?"\0"===e?"\ufffd":e.slice(0,-1)+"\\"+e.charCodeAt(e.length-1).toString(16)+" ":"\\"+e},oe=function(){T()},ae=be(function(e){return!0===e.disabled&&"fieldset"===e.nodeName.toLowerCase()},{dir:"parentNode",next:"legend"});try{H.apply(t=O.call(p.childNodes),p.childNodes),t[p.childNodes.length].nodeType}catch(e){H={apply:t.length?function(e,t){L.apply(e,O.call(t))}:function(e,t){var n=e.length,r=0;while(e[n++]=t[r++]);e.length=n-1}}}function se(t,e,n,r){var i,o,a,s,u,l,c,f=e&&e.ownerDocument,p=e?e.nodeType:9;if(n=n||[],"string"!=typeof t||!t||1!==p&&9!==p&&11!==p)return n;if(!r&&(T(e),e=e||C,S)){if(11!==p&&(u=Z.exec(t)))if(i=u[1]){if(9===p){if(!(a=e.getElementById(i)))return n;if(a.id===i)return n.push(a),n}else if(f&&(a=f.getElementById(i))&&v(e,a)&&a.id===i)return n.push(a),n}else{if(u[2])return H.apply(n,e.getElementsByTagName(t)),n;if((i=u[3])&&d.getElementsByClassName&&e.getElementsByClassName)return H.apply(n,e.getElementsByClassName(i)),n}if(d.qsa&&!N[t+" "]&&(!y||!y.test(t))&&(1!==p||"object"!==e.nodeName.toLowerCase())){if(c=t,f=e,1===p&&(U.test(t)||z.test(t))){(f=ee.test(t)&&ve(e.parentNode)||e)===e&&d.scope||((s=e.getAttribute("id"))?s=s.replace(re,ie):e.setAttribute("id",s=E)),o=(l=h(t)).length;while(o--)l[o]=(s?"#"+s:":scope")+" "+xe(l[o]);c=l.join(",")}try{if(d.cssSupportsSelector&&!CSS.supports("selector(:is("+c+"))"))throw new Error;return H.apply(n,f.querySelectorAll(c)),n}catch(e){N(t,!0)}finally{s===E&&e.removeAttribute("id")}}}return g(t.replace(B,"$1"),e,n,r)}function ue(){var r=[];return function e(t,n){return r.push(t+" ")>b.cacheLength&&delete e[r.shift()],e[t+" "]=n}}function le(e){return e[E]=!0,e}function ce(e){var t=C.createElement("fieldset");try{return!!e(t)}catch(e){return!1}finally{t.parentNode&&t.parentNode.removeChild(t),t=null}}function fe(e,t){var n=e.split("|"),r=n.length;while(r--)b.attrHandle[n[r]]=t}function pe(e,t){var n=t&&e,r=n&&1===e.nodeType&&1===t.nodeType&&e.sourceIndex-t.sourceIndex;if(r)return r;if(n)while(n=n.nextSibling)if(n===t)return-1;return e?1:-1}function de(t){return function(e){return"input"===e.nodeName.toLowerCase()&&e.type===t}}function he(n){return function(e){var t=e.nodeName.toLowerCase();return("input"===t||"button"===t)&&e.type===n}}function ge(t){return function(e){return"form"in e?e.parentNode&&!1===e.disabled?"label"in e?"label"in e.parentNode?e.parentNode.disabled===t:e.disabled===t:e.isDisabled===t||e.isDisabled!==!t&&ae(e)===t:e.disabled===t:"label"in e&&e.disabled===t}}function ye(a){return le(function(o){return o=+o,le(function(e,t){var n,r=a([],e.length,o),i=r.length;while(i--)e[n=r[i]]&&(e[n]=!(t[n]=e[n]))})})}function ve(e){return e&&"undefined"!=typeof e.getElementsByTagName&&e}for(e in d=se.support={},i=se.isXML=function(e){var t=e&&e.namespaceURI,n=e&&(e.ownerDocument||e).documentElement;return!Y.test(t||n&&n.nodeName||"HTML")},T=se.setDocument=function(e){var t,n,r=e?e.ownerDocument||e:p;return r!=C&&9===r.nodeType&&r.documentElement&&(a=(C=r).documentElement,S=!i(C),p!=C&&(n=C.defaultView)&&n.top!==n&&(n.addEventListener?n.addEventListener("unload",oe,!1):n.attachEvent&&n.attachEvent("onunload",oe)),d.scope=ce(function(e){return a.appendChild(e).appendChild(C.createElement("div")),"undefined"!=typeof e.querySelectorAll&&!e.querySelectorAll(":scope fieldset div").length}),d.cssSupportsSelector=ce(function(){return CSS.supports("selector(*)")&&C.querySelectorAll(":is(:jqfake)")&&!CSS.supports("selector(:is(*,:jqfake))")}),d.attributes=ce(function(e){return e.className="i",!e.getAttribute("className")}),d.getElementsByTagName=ce(function(e){return e.appendChild(C.createComment("")),!e.getElementsByTagName("*").length}),d.getElementsByClassName=K.test(C.getElementsByClassName),d.getById=ce(function(e){return a.appendChild(e).id=E,!C.getElementsByName||!C.getElementsByName(E).length}),d.getById?(b.filter.ID=function(e){var t=e.replace(te,ne);return function(e){return e.getAttribute("id")===t}},b.find.ID=function(e,t){if("undefined"!=typeof t.getElementById&&S){var n=t.getElementById(e);return n?[n]:[]}}):(b.filter.ID=function(e){var n=e.replace(te,ne);return function(e){var t="undefined"!=typeof e.getAttributeNode&&e.getAttributeNode("id");return t&&t.value===n}},b.find.ID=function(e,t){if("undefined"!=typeof t.getElementById&&S){var n,r,i,o=t.getElementById(e);if(o){if((n=o.getAttributeNode("id"))&&n.value===e)return[o];i=t.getElementsByName(e),r=0;while(o=i[r++])if((n=o.getAttributeNode("id"))&&n.value===e)return[o]}return[]}}),b.find.TAG=d.getElementsByTagName?function(e,t){return"undefined"!=typeof t.getElementsByTagName?t.getElementsByTagName(e):d.qsa?t.querySelectorAll(e):void 0}:function(e,t){var n,r=[],i=0,o=t.getElementsByTagName(e);if("*"===e){while(n=o[i++])1===n.nodeType&&r.push(n);return r}return o},b.find.CLASS=d.getElementsByClassName&&function(e,t){if("undefined"!=typeof t.getElementsByClassName&&S)return t.getElementsByClassName(e)},s=[],y=[],(d.qsa=K.test(C.querySelectorAll))&&(ce(function(e){var t;a.appendChild(e).innerHTML="",e.querySelectorAll("[msallowcapture^='']").length&&y.push("[*^$]="+M+"*(?:''|\"\")"),e.querySelectorAll("[selected]").length||y.push("\\["+M+"*(?:value|"+R+")"),e.querySelectorAll("[id~="+E+"-]").length||y.push("~="),(t=C.createElement("input")).setAttribute("name",""),e.appendChild(t),e.querySelectorAll("[name='']").length||y.push("\\["+M+"*name"+M+"*="+M+"*(?:''|\"\")"),e.querySelectorAll(":checked").length||y.push(":checked"),e.querySelectorAll("a#"+E+"+*").length||y.push(".#.+[+~]"),e.querySelectorAll("\\\f"),y.push("[\\r\\n\\f]")}),ce(function(e){e.innerHTML="";var t=C.createElement("input");t.setAttribute("type","hidden"),e.appendChild(t).setAttribute("name","D"),e.querySelectorAll("[name=d]").length&&y.push("name"+M+"*[*^$|!~]?="),2!==e.querySelectorAll(":enabled").length&&y.push(":enabled",":disabled"),a.appendChild(e).disabled=!0,2!==e.querySelectorAll(":disabled").length&&y.push(":enabled",":disabled"),e.querySelectorAll("*,:x"),y.push(",.*:")})),(d.matchesSelector=K.test(c=a.matches||a.webkitMatchesSelector||a.mozMatchesSelector||a.oMatchesSelector||a.msMatchesSelector))&&ce(function(e){d.disconnectedMatch=c.call(e,"*"),c.call(e,"[s!='']:x"),s.push("!=",F)}),d.cssSupportsSelector||y.push(":has"),y=y.length&&new RegExp(y.join("|")),s=s.length&&new RegExp(s.join("|")),t=K.test(a.compareDocumentPosition),v=t||K.test(a.contains)?function(e,t){var n=9===e.nodeType&&e.documentElement||e,r=t&&t.parentNode;return e===r||!(!r||1!==r.nodeType||!(n.contains?n.contains(r):e.compareDocumentPosition&&16&e.compareDocumentPosition(r)))}:function(e,t){if(t)while(t=t.parentNode)if(t===e)return!0;return!1},j=t?function(e,t){if(e===t)return l=!0,0;var n=!e.compareDocumentPosition-!t.compareDocumentPosition;return n||(1&(n=(e.ownerDocument||e)==(t.ownerDocument||t)?e.compareDocumentPosition(t):1)||!d.sortDetached&&t.compareDocumentPosition(e)===n?e==C||e.ownerDocument==p&&v(p,e)?-1:t==C||t.ownerDocument==p&&v(p,t)?1:u?P(u,e)-P(u,t):0:4&n?-1:1)}:function(e,t){if(e===t)return l=!0,0;var n,r=0,i=e.parentNode,o=t.parentNode,a=[e],s=[t];if(!i||!o)return e==C?-1:t==C?1:i?-1:o?1:u?P(u,e)-P(u,t):0;if(i===o)return pe(e,t);n=e;while(n=n.parentNode)a.unshift(n);n=t;while(n=n.parentNode)s.unshift(n);while(a[r]===s[r])r++;return r?pe(a[r],s[r]):a[r]==p?-1:s[r]==p?1:0}),C},se.matches=function(e,t){return se(e,null,null,t)},se.matchesSelector=function(e,t){if(T(e),d.matchesSelector&&S&&!N[t+" "]&&(!s||!s.test(t))&&(!y||!y.test(t)))try{var n=c.call(e,t);if(n||d.disconnectedMatch||e.document&&11!==e.document.nodeType)return n}catch(e){N(t,!0)}return 0":{dir:"parentNode",first:!0}," ":{dir:"parentNode"},"+":{dir:"previousSibling",first:!0},"~":{dir:"previousSibling"}},preFilter:{ATTR:function(e){return e[1]=e[1].replace(te,ne),e[3]=(e[3]||e[4]||e[5]||"").replace(te,ne),"~="===e[2]&&(e[3]=" "+e[3]+" "),e.slice(0,4)},CHILD:function(e){return e[1]=e[1].toLowerCase(),"nth"===e[1].slice(0,3)?(e[3]||se.error(e[0]),e[4]=+(e[4]?e[5]+(e[6]||1):2*("even"===e[3]||"odd"===e[3])),e[5]=+(e[7]+e[8]||"odd"===e[3])):e[3]&&se.error(e[0]),e},PSEUDO:function(e){var t,n=!e[6]&&e[2];return G.CHILD.test(e[0])?null:(e[3]?e[2]=e[4]||e[5]||"":n&&X.test(n)&&(t=h(n,!0))&&(t=n.indexOf(")",n.length-t)-n.length)&&(e[0]=e[0].slice(0,t),e[2]=n.slice(0,t)),e.slice(0,3))}},filter:{TAG:function(e){var t=e.replace(te,ne).toLowerCase();return"*"===e?function(){return!0}:function(e){return e.nodeName&&e.nodeName.toLowerCase()===t}},CLASS:function(e){var t=m[e+" "];return t||(t=new RegExp("(^|"+M+")"+e+"("+M+"|$)"))&&m(e,function(e){return t.test("string"==typeof e.className&&e.className||"undefined"!=typeof e.getAttribute&&e.getAttribute("class")||"")})},ATTR:function(n,r,i){return function(e){var t=se.attr(e,n);return null==t?"!="===r:!r||(t+="","="===r?t===i:"!="===r?t!==i:"^="===r?i&&0===t.indexOf(i):"*="===r?i&&-1:\x20\t\r\n\f]*)[\x20\t\r\n\f]*\/?>(?:<\/\1>|)$/i;function j(e,n,r){return m(n)?E.grep(e,function(e,t){return!!n.call(e,t,e)!==r}):n.nodeType?E.grep(e,function(e){return e===n!==r}):"string"!=typeof n?E.grep(e,function(e){return-1)[^>]*|#([\w-]+))$/;(E.fn.init=function(e,t,n){var r,i;if(!e)return this;if(n=n||D,"string"==typeof e){if(!(r="<"===e[0]&&">"===e[e.length-1]&&3<=e.length?[null,e,null]:q.exec(e))||!r[1]&&t)return!t||t.jquery?(t||n).find(e):this.constructor(t).find(e);if(r[1]){if(t=t instanceof E?t[0]:t,E.merge(this,E.parseHTML(r[1],t&&t.nodeType?t.ownerDocument||t:S,!0)),N.test(r[1])&&E.isPlainObject(t))for(r in t)m(this[r])?this[r](t[r]):this.attr(r,t[r]);return this}return(i=S.getElementById(r[2]))&&(this[0]=i,this.length=1),this}return e.nodeType?(this[0]=e,this.length=1,this):m(e)?void 0!==n.ready?n.ready(e):e(E):E.makeArray(e,this)}).prototype=E.fn,D=E(S);var L=/^(?:parents|prev(?:Until|All))/,H={children:!0,contents:!0,next:!0,prev:!0};function O(e,t){while((e=e[t])&&1!==e.nodeType);return e}E.fn.extend({has:function(e){var t=E(e,this),n=t.length;return this.filter(function(){for(var e=0;e\x20\t\r\n\f]*)/i,he=/^$|^module$|\/(?:java|ecma)script/i;ce=S.createDocumentFragment().appendChild(S.createElement("div")),(fe=S.createElement("input")).setAttribute("type","radio"),fe.setAttribute("checked","checked"),fe.setAttribute("name","t"),ce.appendChild(fe),v.checkClone=ce.cloneNode(!0).cloneNode(!0).lastChild.checked,ce.innerHTML="",v.noCloneChecked=!!ce.cloneNode(!0).lastChild.defaultValue,ce.innerHTML="",v.option=!!ce.lastChild;var ge={thead:[1,"","
"],col:[2,"","
"],tr:[2,"","
"],td:[3,"","
"],_default:[0,"",""]};function ye(e,t){var n;return n="undefined"!=typeof e.getElementsByTagName?e.getElementsByTagName(t||"*"):"undefined"!=typeof e.querySelectorAll?e.querySelectorAll(t||"*"):[],void 0===t||t&&A(e,t)?E.merge([e],n):n}function ve(e,t){for(var n=0,r=e.length;n",""]);var me=/<|&#?\w+;/;function xe(e,t,n,r,i){for(var o,a,s,u,l,c,f=t.createDocumentFragment(),p=[],d=0,h=e.length;d\s*$/g;function je(e,t){return A(e,"table")&&A(11!==t.nodeType?t:t.firstChild,"tr")&&E(e).children("tbody")[0]||e}function De(e){return e.type=(null!==e.getAttribute("type"))+"/"+e.type,e}function qe(e){return"true/"===(e.type||"").slice(0,5)?e.type=e.type.slice(5):e.removeAttribute("type"),e}function Le(e,t){var n,r,i,o,a,s;if(1===t.nodeType){if(Y.hasData(e)&&(s=Y.get(e).events))for(i in Y.remove(t,"handle events"),s)for(n=0,r=s[i].length;n").attr(n.scriptAttrs||{}).prop({charset:n.scriptCharset,src:n.url}).on("load error",i=function(e){r.remove(),i=null,e&&t("error"===e.type?404:200,e.type)}),S.head.appendChild(r[0])},abort:function(){i&&i()}}});var Ut,Xt=[],Vt=/(=)\?(?=&|$)|\?\?/;E.ajaxSetup({jsonp:"callback",jsonpCallback:function(){var e=Xt.pop()||E.expando+"_"+Ct.guid++;return this[e]=!0,e}}),E.ajaxPrefilter("json jsonp",function(e,t,n){var r,i,o,a=!1!==e.jsonp&&(Vt.test(e.url)?"url":"string"==typeof e.data&&0===(e.contentType||"").indexOf("application/x-www-form-urlencoded")&&Vt.test(e.data)&&"data");if(a||"jsonp"===e.dataTypes[0])return r=e.jsonpCallback=m(e.jsonpCallback)?e.jsonpCallback():e.jsonpCallback,a?e[a]=e[a].replace(Vt,"$1"+r):!1!==e.jsonp&&(e.url+=(St.test(e.url)?"&":"?")+e.jsonp+"="+r),e.converters["script json"]=function(){return o||E.error(r+" was not called"),o[0]},e.dataTypes[0]="json",i=C[r],C[r]=function(){o=arguments},n.always(function(){void 0===i?E(C).removeProp(r):C[r]=i,e[r]&&(e.jsonpCallback=t.jsonpCallback,Xt.push(r)),o&&m(i)&&i(o[0]),o=i=void 0}),"script"}),v.createHTMLDocument=((Ut=S.implementation.createHTMLDocument("").body).innerHTML="
",2===Ut.childNodes.length),E.parseHTML=function(e,t,n){return"string"!=typeof e?[]:("boolean"==typeof t&&(n=t,t=!1),t||(v.createHTMLDocument?((r=(t=S.implementation.createHTMLDocument("")).createElement("base")).href=S.location.href,t.head.appendChild(r)):t=S),o=!n&&[],(i=N.exec(e))?[t.createElement(i[1])]:(i=xe([e],t,o),o&&o.length&&E(o).remove(),E.merge([],i.childNodes)));var r,i,o},E.fn.load=function(e,t,n){var r,i,o,a=this,s=e.indexOf(" ");return-1").append(E.parseHTML(e)).find(r):e)}).always(n&&function(e,t){a.each(function(){n.apply(this,o||[e.responseText,t,e])})}),this},E.expr.pseudos.animated=function(t){return E.grep(E.timers,function(e){return t===e.elem}).length},E.offset={setOffset:function(e,t,n){var r,i,o,a,s,u,l=E.css(e,"position"),c=E(e),f={};"static"===l&&(e.style.position="relative"),s=c.offset(),o=E.css(e,"top"),u=E.css(e,"left"),("absolute"===l||"fixed"===l)&&-1<(o+u).indexOf("auto")?(a=(r=c.position()).top,i=r.left):(a=parseFloat(o)||0,i=parseFloat(u)||0),m(t)&&(t=t.call(e,n,E.extend({},s))),null!=t.top&&(f.top=t.top-s.top+a),null!=t.left&&(f.left=t.left-s.left+i),"using"in t?t.using.call(e,f):c.css(f)}},E.fn.extend({offset:function(t){if(arguments.length)return void 0===t?this:this.each(function(e){E.offset.setOffset(this,t,e)});var e,n,r=this[0];return r?r.getClientRects().length?(e=r.getBoundingClientRect(),n=r.ownerDocument.defaultView,{top:e.top+n.pageYOffset,left:e.left+n.pageXOffset}):{top:0,left:0}:void 0},position:function(){if(this[0]){var e,t,n,r=this[0],i={top:0,left:0};if("fixed"===E.css(r,"position"))t=r.getBoundingClientRect();else{t=this.offset(),n=r.ownerDocument,e=r.offsetParent||n.documentElement;while(e&&(e===n.body||e===n.documentElement)&&"static"===E.css(e,"position"))e=e.parentNode;e&&e!==r&&1===e.nodeType&&((i=E(e).offset()).top+=E.css(e,"borderTopWidth",!0),i.left+=E.css(e,"borderLeftWidth",!0))}return{top:t.top-i.top-E.css(r,"marginTop",!0),left:t.left-i.left-E.css(r,"marginLeft",!0)}}},offsetParent:function(){return this.map(function(){var e=this.offsetParent;while(e&&"static"===E.css(e,"position"))e=e.offsetParent;return e||re})}}),E.each({scrollLeft:"pageXOffset",scrollTop:"pageYOffset"},function(t,i){var o="pageYOffset"===i;E.fn[t]=function(e){return B(this,function(e,t,n){var r;if(x(e)?r=e:9===e.nodeType&&(r=e.defaultView),void 0===n)return r?r[i]:e[t];r?r.scrollTo(o?r.pageXOffset:n,o?n:r.pageYOffset):e[t]=n},t,e,arguments.length)}}),E.each(["top","left"],function(e,n){E.cssHooks[n]=_e(v.pixelPosition,function(e,t){if(t)return t=Be(e,n),Pe.test(t)?E(e).position()[n]+"px":t})}),E.each({Height:"height",Width:"width"},function(a,s){E.each({padding:"inner"+a,content:s,"":"outer"+a},function(r,o){E.fn[o]=function(e,t){var n=arguments.length&&(r||"boolean"!=typeof e),i=r||(!0===e||!0===t?"margin":"border");return B(this,function(e,t,n){var r;return x(e)?0===o.indexOf("outer")?e["inner"+a]:e.document.documentElement["client"+a]:9===e.nodeType?(r=e.documentElement,Math.max(e.body["scroll"+a],r["scroll"+a],e.body["offset"+a],r["offset"+a],r["client"+a])):void 0===n?E.css(e,t,i):E.style(e,t,n,i)},s,n?e:void 0,n)}})}),E.each(["ajaxStart","ajaxStop","ajaxComplete","ajaxError","ajaxSuccess","ajaxSend"],function(e,t){E.fn[t]=function(e){return this.on(t,e)}}),E.fn.extend({bind:function(e,t,n){return this.on(e,null,t,n)},unbind:function(e,t){return this.off(e,null,t)},delegate:function(e,t,n,r){return this.on(t,e,n,r)},undelegate:function(e,t,n){return 1===arguments.length?this.off(e,"**"):this.off(t,e||"**",n)},hover:function(e,t){return this.mouseenter(e).mouseleave(t||e)}}),E.each("blur focus focusin focusout resize scroll click dblclick mousedown mouseup mousemove mouseover mouseout mouseenter mouseleave change select submit keydown keypress keyup contextmenu".split(" "),function(e,n){E.fn[n]=function(e,t){return 0.anchorjs-link,.anchorjs-link:focus{opacity:1}",A.sheet.cssRules.length),A.sheet.insertRule("[data-anchorjs-icon]::after{content:attr(data-anchorjs-icon)}",A.sheet.cssRules.length),A.sheet.insertRule('@font-face{font-family:anchorjs-icons;src:url(data:n/a;base64,AAEAAAALAIAAAwAwT1MvMg8yG2cAAAE4AAAAYGNtYXDp3gC3AAABpAAAAExnYXNwAAAAEAAAA9wAAAAIZ2x5ZlQCcfwAAAH4AAABCGhlYWQHFvHyAAAAvAAAADZoaGVhBnACFwAAAPQAAAAkaG10eASAADEAAAGYAAAADGxvY2EACACEAAAB8AAAAAhtYXhwAAYAVwAAARgAAAAgbmFtZQGOH9cAAAMAAAAAunBvc3QAAwAAAAADvAAAACAAAQAAAAEAAHzE2p9fDzz1AAkEAAAAAADRecUWAAAAANQA6R8AAAAAAoACwAAAAAgAAgAAAAAAAAABAAADwP/AAAACgAAA/9MCrQABAAAAAAAAAAAAAAAAAAAAAwABAAAAAwBVAAIAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAMCQAGQAAUAAAKZAswAAACPApkCzAAAAesAMwEJAAAAAAAAAAAAAAAAAAAAARAAAAAAAAAAAAAAAAAAAAAAQAAg//0DwP/AAEADwABAAAAAAQAAAAAAAAAAAAAAIAAAAAAAAAIAAAACgAAxAAAAAwAAAAMAAAAcAAEAAwAAABwAAwABAAAAHAAEADAAAAAIAAgAAgAAACDpy//9//8AAAAg6cv//f///+EWNwADAAEAAAAAAAAAAAAAAAAACACEAAEAAAAAAAAAAAAAAAAxAAACAAQARAKAAsAAKwBUAAABIiYnJjQ3NzY2MzIWFxYUBwcGIicmNDc3NjQnJiYjIgYHBwYUFxYUBwYGIwciJicmNDc3NjIXFhQHBwYUFxYWMzI2Nzc2NCcmNDc2MhcWFAcHBgYjARQGDAUtLXoWOR8fORYtLTgKGwoKCjgaGg0gEhIgDXoaGgkJBQwHdR85Fi0tOAobCgoKOBoaDSASEiANehoaCQkKGwotLXoWOR8BMwUFLYEuehYXFxYugC44CQkKGwo4GkoaDQ0NDXoaShoKGwoFBe8XFi6ALjgJCQobCjgaShoNDQ0NehpKGgobCgoKLYEuehYXAAAADACWAAEAAAAAAAEACAAAAAEAAAAAAAIAAwAIAAEAAAAAAAMACAAAAAEAAAAAAAQACAAAAAEAAAAAAAUAAQALAAEAAAAAAAYACAAAAAMAAQQJAAEAEAAMAAMAAQQJAAIABgAcAAMAAQQJAAMAEAAMAAMAAQQJAAQAEAAMAAMAAQQJAAUAAgAiAAMAAQQJAAYAEAAMYW5jaG9yanM0MDBAAGEAbgBjAGgAbwByAGoAcwA0ADAAMABAAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAH//wAP) format("truetype")}',A.sheet.cssRules.length)),h=document.querySelectorAll("[id]"),t=[].map.call(h,function(A){return A.id}),i=0;i\]./()*\\\n\t\b\v\u00A0]/g,"-").replace(/-{2,}/g,"-").substring(0,this.options.truncate).replace(/^-+|-+$/gm,"").toLowerCase()},this.hasAnchorJSLink=function(A){var e=A.firstChild&&-1<(" "+A.firstChild.className+" ").indexOf(" anchorjs-link "),A=A.lastChild&&-1<(" "+A.lastChild.className+" ").indexOf(" anchorjs-link ");return e||A||!1}}}); +// @license-end \ No newline at end of file diff --git a/site_libs/quarto-html/popper.min.js b/site_libs/quarto-html/popper.min.js new file mode 100644 index 0000000..e3726d7 --- /dev/null +++ b/site_libs/quarto-html/popper.min.js @@ -0,0 +1,6 @@ +/** + * @popperjs/core v2.11.7 - MIT License + */ + +!function(e,t){"object"==typeof exports&&"undefined"!=typeof module?t(exports):"function"==typeof define&&define.amd?define(["exports"],t):t((e="undefined"!=typeof globalThis?globalThis:e||self).Popper={})}(this,(function(e){"use strict";function t(e){if(null==e)return window;if("[object Window]"!==e.toString()){var t=e.ownerDocument;return t&&t.defaultView||window}return e}function n(e){return e instanceof t(e).Element||e instanceof Element}function r(e){return e instanceof t(e).HTMLElement||e instanceof HTMLElement}function o(e){return"undefined"!=typeof ShadowRoot&&(e instanceof t(e).ShadowRoot||e instanceof ShadowRoot)}var i=Math.max,a=Math.min,s=Math.round;function f(){var e=navigator.userAgentData;return null!=e&&e.brands&&Array.isArray(e.brands)?e.brands.map((function(e){return e.brand+"/"+e.version})).join(" "):navigator.userAgent}function c(){return!/^((?!chrome|android).)*safari/i.test(f())}function p(e,o,i){void 0===o&&(o=!1),void 0===i&&(i=!1);var a=e.getBoundingClientRect(),f=1,p=1;o&&r(e)&&(f=e.offsetWidth>0&&s(a.width)/e.offsetWidth||1,p=e.offsetHeight>0&&s(a.height)/e.offsetHeight||1);var u=(n(e)?t(e):window).visualViewport,l=!c()&&i,d=(a.left+(l&&u?u.offsetLeft:0))/f,h=(a.top+(l&&u?u.offsetTop:0))/p,m=a.width/f,v=a.height/p;return{width:m,height:v,top:h,right:d+m,bottom:h+v,left:d,x:d,y:h}}function u(e){var n=t(e);return{scrollLeft:n.pageXOffset,scrollTop:n.pageYOffset}}function l(e){return e?(e.nodeName||"").toLowerCase():null}function d(e){return((n(e)?e.ownerDocument:e.document)||window.document).documentElement}function h(e){return p(d(e)).left+u(e).scrollLeft}function m(e){return t(e).getComputedStyle(e)}function v(e){var t=m(e),n=t.overflow,r=t.overflowX,o=t.overflowY;return/auto|scroll|overlay|hidden/.test(n+o+r)}function y(e,n,o){void 0===o&&(o=!1);var i,a,f=r(n),c=r(n)&&function(e){var t=e.getBoundingClientRect(),n=s(t.width)/e.offsetWidth||1,r=s(t.height)/e.offsetHeight||1;return 1!==n||1!==r}(n),m=d(n),y=p(e,c,o),g={scrollLeft:0,scrollTop:0},b={x:0,y:0};return(f||!f&&!o)&&(("body"!==l(n)||v(m))&&(g=(i=n)!==t(i)&&r(i)?{scrollLeft:(a=i).scrollLeft,scrollTop:a.scrollTop}:u(i)),r(n)?((b=p(n,!0)).x+=n.clientLeft,b.y+=n.clientTop):m&&(b.x=h(m))),{x:y.left+g.scrollLeft-b.x,y:y.top+g.scrollTop-b.y,width:y.width,height:y.height}}function g(e){var t=p(e),n=e.offsetWidth,r=e.offsetHeight;return Math.abs(t.width-n)<=1&&(n=t.width),Math.abs(t.height-r)<=1&&(r=t.height),{x:e.offsetLeft,y:e.offsetTop,width:n,height:r}}function b(e){return"html"===l(e)?e:e.assignedSlot||e.parentNode||(o(e)?e.host:null)||d(e)}function x(e){return["html","body","#document"].indexOf(l(e))>=0?e.ownerDocument.body:r(e)&&v(e)?e:x(b(e))}function w(e,n){var r;void 0===n&&(n=[]);var o=x(e),i=o===(null==(r=e.ownerDocument)?void 0:r.body),a=t(o),s=i?[a].concat(a.visualViewport||[],v(o)?o:[]):o,f=n.concat(s);return i?f:f.concat(w(b(s)))}function O(e){return["table","td","th"].indexOf(l(e))>=0}function j(e){return r(e)&&"fixed"!==m(e).position?e.offsetParent:null}function E(e){for(var n=t(e),i=j(e);i&&O(i)&&"static"===m(i).position;)i=j(i);return i&&("html"===l(i)||"body"===l(i)&&"static"===m(i).position)?n:i||function(e){var t=/firefox/i.test(f());if(/Trident/i.test(f())&&r(e)&&"fixed"===m(e).position)return null;var n=b(e);for(o(n)&&(n=n.host);r(n)&&["html","body"].indexOf(l(n))<0;){var i=m(n);if("none"!==i.transform||"none"!==i.perspective||"paint"===i.contain||-1!==["transform","perspective"].indexOf(i.willChange)||t&&"filter"===i.willChange||t&&i.filter&&"none"!==i.filter)return n;n=n.parentNode}return null}(e)||n}var D="top",A="bottom",L="right",P="left",M="auto",k=[D,A,L,P],W="start",B="end",H="viewport",T="popper",R=k.reduce((function(e,t){return e.concat([t+"-"+W,t+"-"+B])}),[]),S=[].concat(k,[M]).reduce((function(e,t){return e.concat([t,t+"-"+W,t+"-"+B])}),[]),V=["beforeRead","read","afterRead","beforeMain","main","afterMain","beforeWrite","write","afterWrite"];function q(e){var t=new Map,n=new Set,r=[];function o(e){n.add(e.name),[].concat(e.requires||[],e.requiresIfExists||[]).forEach((function(e){if(!n.has(e)){var r=t.get(e);r&&o(r)}})),r.push(e)}return e.forEach((function(e){t.set(e.name,e)})),e.forEach((function(e){n.has(e.name)||o(e)})),r}function C(e){return e.split("-")[0]}function N(e,t){var n=t.getRootNode&&t.getRootNode();if(e.contains(t))return!0;if(n&&o(n)){var r=t;do{if(r&&e.isSameNode(r))return!0;r=r.parentNode||r.host}while(r)}return!1}function I(e){return Object.assign({},e,{left:e.x,top:e.y,right:e.x+e.width,bottom:e.y+e.height})}function _(e,r,o){return r===H?I(function(e,n){var r=t(e),o=d(e),i=r.visualViewport,a=o.clientWidth,s=o.clientHeight,f=0,p=0;if(i){a=i.width,s=i.height;var u=c();(u||!u&&"fixed"===n)&&(f=i.offsetLeft,p=i.offsetTop)}return{width:a,height:s,x:f+h(e),y:p}}(e,o)):n(r)?function(e,t){var n=p(e,!1,"fixed"===t);return n.top=n.top+e.clientTop,n.left=n.left+e.clientLeft,n.bottom=n.top+e.clientHeight,n.right=n.left+e.clientWidth,n.width=e.clientWidth,n.height=e.clientHeight,n.x=n.left,n.y=n.top,n}(r,o):I(function(e){var t,n=d(e),r=u(e),o=null==(t=e.ownerDocument)?void 0:t.body,a=i(n.scrollWidth,n.clientWidth,o?o.scrollWidth:0,o?o.clientWidth:0),s=i(n.scrollHeight,n.clientHeight,o?o.scrollHeight:0,o?o.clientHeight:0),f=-r.scrollLeft+h(e),c=-r.scrollTop;return"rtl"===m(o||n).direction&&(f+=i(n.clientWidth,o?o.clientWidth:0)-a),{width:a,height:s,x:f,y:c}}(d(e)))}function F(e,t,o,s){var f="clippingParents"===t?function(e){var t=w(b(e)),o=["absolute","fixed"].indexOf(m(e).position)>=0&&r(e)?E(e):e;return n(o)?t.filter((function(e){return n(e)&&N(e,o)&&"body"!==l(e)})):[]}(e):[].concat(t),c=[].concat(f,[o]),p=c[0],u=c.reduce((function(t,n){var r=_(e,n,s);return t.top=i(r.top,t.top),t.right=a(r.right,t.right),t.bottom=a(r.bottom,t.bottom),t.left=i(r.left,t.left),t}),_(e,p,s));return u.width=u.right-u.left,u.height=u.bottom-u.top,u.x=u.left,u.y=u.top,u}function U(e){return e.split("-")[1]}function z(e){return["top","bottom"].indexOf(e)>=0?"x":"y"}function X(e){var t,n=e.reference,r=e.element,o=e.placement,i=o?C(o):null,a=o?U(o):null,s=n.x+n.width/2-r.width/2,f=n.y+n.height/2-r.height/2;switch(i){case D:t={x:s,y:n.y-r.height};break;case A:t={x:s,y:n.y+n.height};break;case L:t={x:n.x+n.width,y:f};break;case P:t={x:n.x-r.width,y:f};break;default:t={x:n.x,y:n.y}}var c=i?z(i):null;if(null!=c){var p="y"===c?"height":"width";switch(a){case W:t[c]=t[c]-(n[p]/2-r[p]/2);break;case B:t[c]=t[c]+(n[p]/2-r[p]/2)}}return t}function Y(e){return Object.assign({},{top:0,right:0,bottom:0,left:0},e)}function G(e,t){return t.reduce((function(t,n){return t[n]=e,t}),{})}function J(e,t){void 0===t&&(t={});var r=t,o=r.placement,i=void 0===o?e.placement:o,a=r.strategy,s=void 0===a?e.strategy:a,f=r.boundary,c=void 0===f?"clippingParents":f,u=r.rootBoundary,l=void 0===u?H:u,h=r.elementContext,m=void 0===h?T:h,v=r.altBoundary,y=void 0!==v&&v,g=r.padding,b=void 0===g?0:g,x=Y("number"!=typeof b?b:G(b,k)),w=m===T?"reference":T,O=e.rects.popper,j=e.elements[y?w:m],E=F(n(j)?j:j.contextElement||d(e.elements.popper),c,l,s),P=p(e.elements.reference),M=X({reference:P,element:O,strategy:"absolute",placement:i}),W=I(Object.assign({},O,M)),B=m===T?W:P,R={top:E.top-B.top+x.top,bottom:B.bottom-E.bottom+x.bottom,left:E.left-B.left+x.left,right:B.right-E.right+x.right},S=e.modifiersData.offset;if(m===T&&S){var V=S[i];Object.keys(R).forEach((function(e){var t=[L,A].indexOf(e)>=0?1:-1,n=[D,A].indexOf(e)>=0?"y":"x";R[e]+=V[n]*t}))}return R}var K={placement:"bottom",modifiers:[],strategy:"absolute"};function Q(){for(var e=arguments.length,t=new Array(e),n=0;n=0?-1:1,i="function"==typeof n?n(Object.assign({},t,{placement:e})):n,a=i[0],s=i[1];return a=a||0,s=(s||0)*o,[P,L].indexOf(r)>=0?{x:s,y:a}:{x:a,y:s}}(n,t.rects,i),e}),{}),s=a[t.placement],f=s.x,c=s.y;null!=t.modifiersData.popperOffsets&&(t.modifiersData.popperOffsets.x+=f,t.modifiersData.popperOffsets.y+=c),t.modifiersData[r]=a}},se={left:"right",right:"left",bottom:"top",top:"bottom"};function fe(e){return e.replace(/left|right|bottom|top/g,(function(e){return se[e]}))}var ce={start:"end",end:"start"};function pe(e){return e.replace(/start|end/g,(function(e){return ce[e]}))}function ue(e,t){void 0===t&&(t={});var n=t,r=n.placement,o=n.boundary,i=n.rootBoundary,a=n.padding,s=n.flipVariations,f=n.allowedAutoPlacements,c=void 0===f?S:f,p=U(r),u=p?s?R:R.filter((function(e){return U(e)===p})):k,l=u.filter((function(e){return c.indexOf(e)>=0}));0===l.length&&(l=u);var d=l.reduce((function(t,n){return t[n]=J(e,{placement:n,boundary:o,rootBoundary:i,padding:a})[C(n)],t}),{});return Object.keys(d).sort((function(e,t){return d[e]-d[t]}))}var le={name:"flip",enabled:!0,phase:"main",fn:function(e){var t=e.state,n=e.options,r=e.name;if(!t.modifiersData[r]._skip){for(var o=n.mainAxis,i=void 0===o||o,a=n.altAxis,s=void 0===a||a,f=n.fallbackPlacements,c=n.padding,p=n.boundary,u=n.rootBoundary,l=n.altBoundary,d=n.flipVariations,h=void 0===d||d,m=n.allowedAutoPlacements,v=t.options.placement,y=C(v),g=f||(y===v||!h?[fe(v)]:function(e){if(C(e)===M)return[];var t=fe(e);return[pe(e),t,pe(t)]}(v)),b=[v].concat(g).reduce((function(e,n){return e.concat(C(n)===M?ue(t,{placement:n,boundary:p,rootBoundary:u,padding:c,flipVariations:h,allowedAutoPlacements:m}):n)}),[]),x=t.rects.reference,w=t.rects.popper,O=new Map,j=!0,E=b[0],k=0;k=0,S=R?"width":"height",V=J(t,{placement:B,boundary:p,rootBoundary:u,altBoundary:l,padding:c}),q=R?T?L:P:T?A:D;x[S]>w[S]&&(q=fe(q));var N=fe(q),I=[];if(i&&I.push(V[H]<=0),s&&I.push(V[q]<=0,V[N]<=0),I.every((function(e){return e}))){E=B,j=!1;break}O.set(B,I)}if(j)for(var _=function(e){var t=b.find((function(t){var n=O.get(t);if(n)return n.slice(0,e).every((function(e){return e}))}));if(t)return E=t,"break"},F=h?3:1;F>0;F--){if("break"===_(F))break}t.placement!==E&&(t.modifiersData[r]._skip=!0,t.placement=E,t.reset=!0)}},requiresIfExists:["offset"],data:{_skip:!1}};function de(e,t,n){return i(e,a(t,n))}var he={name:"preventOverflow",enabled:!0,phase:"main",fn:function(e){var t=e.state,n=e.options,r=e.name,o=n.mainAxis,s=void 0===o||o,f=n.altAxis,c=void 0!==f&&f,p=n.boundary,u=n.rootBoundary,l=n.altBoundary,d=n.padding,h=n.tether,m=void 0===h||h,v=n.tetherOffset,y=void 0===v?0:v,b=J(t,{boundary:p,rootBoundary:u,padding:d,altBoundary:l}),x=C(t.placement),w=U(t.placement),O=!w,j=z(x),M="x"===j?"y":"x",k=t.modifiersData.popperOffsets,B=t.rects.reference,H=t.rects.popper,T="function"==typeof y?y(Object.assign({},t.rects,{placement:t.placement})):y,R="number"==typeof T?{mainAxis:T,altAxis:T}:Object.assign({mainAxis:0,altAxis:0},T),S=t.modifiersData.offset?t.modifiersData.offset[t.placement]:null,V={x:0,y:0};if(k){if(s){var q,N="y"===j?D:P,I="y"===j?A:L,_="y"===j?"height":"width",F=k[j],X=F+b[N],Y=F-b[I],G=m?-H[_]/2:0,K=w===W?B[_]:H[_],Q=w===W?-H[_]:-B[_],Z=t.elements.arrow,$=m&&Z?g(Z):{width:0,height:0},ee=t.modifiersData["arrow#persistent"]?t.modifiersData["arrow#persistent"].padding:{top:0,right:0,bottom:0,left:0},te=ee[N],ne=ee[I],re=de(0,B[_],$[_]),oe=O?B[_]/2-G-re-te-R.mainAxis:K-re-te-R.mainAxis,ie=O?-B[_]/2+G+re+ne+R.mainAxis:Q+re+ne+R.mainAxis,ae=t.elements.arrow&&E(t.elements.arrow),se=ae?"y"===j?ae.clientTop||0:ae.clientLeft||0:0,fe=null!=(q=null==S?void 0:S[j])?q:0,ce=F+ie-fe,pe=de(m?a(X,F+oe-fe-se):X,F,m?i(Y,ce):Y);k[j]=pe,V[j]=pe-F}if(c){var ue,le="x"===j?D:P,he="x"===j?A:L,me=k[M],ve="y"===M?"height":"width",ye=me+b[le],ge=me-b[he],be=-1!==[D,P].indexOf(x),xe=null!=(ue=null==S?void 0:S[M])?ue:0,we=be?ye:me-B[ve]-H[ve]-xe+R.altAxis,Oe=be?me+B[ve]+H[ve]-xe-R.altAxis:ge,je=m&&be?function(e,t,n){var r=de(e,t,n);return r>n?n:r}(we,me,Oe):de(m?we:ye,me,m?Oe:ge);k[M]=je,V[M]=je-me}t.modifiersData[r]=V}},requiresIfExists:["offset"]};var me={name:"arrow",enabled:!0,phase:"main",fn:function(e){var t,n=e.state,r=e.name,o=e.options,i=n.elements.arrow,a=n.modifiersData.popperOffsets,s=C(n.placement),f=z(s),c=[P,L].indexOf(s)>=0?"height":"width";if(i&&a){var p=function(e,t){return Y("number"!=typeof(e="function"==typeof e?e(Object.assign({},t.rects,{placement:t.placement})):e)?e:G(e,k))}(o.padding,n),u=g(i),l="y"===f?D:P,d="y"===f?A:L,h=n.rects.reference[c]+n.rects.reference[f]-a[f]-n.rects.popper[c],m=a[f]-n.rects.reference[f],v=E(i),y=v?"y"===f?v.clientHeight||0:v.clientWidth||0:0,b=h/2-m/2,x=p[l],w=y-u[c]-p[d],O=y/2-u[c]/2+b,j=de(x,O,w),M=f;n.modifiersData[r]=((t={})[M]=j,t.centerOffset=j-O,t)}},effect:function(e){var t=e.state,n=e.options.element,r=void 0===n?"[data-popper-arrow]":n;null!=r&&("string"!=typeof r||(r=t.elements.popper.querySelector(r)))&&N(t.elements.popper,r)&&(t.elements.arrow=r)},requires:["popperOffsets"],requiresIfExists:["preventOverflow"]};function ve(e,t,n){return void 0===n&&(n={x:0,y:0}),{top:e.top-t.height-n.y,right:e.right-t.width+n.x,bottom:e.bottom-t.height+n.y,left:e.left-t.width-n.x}}function ye(e){return[D,L,A,P].some((function(t){return e[t]>=0}))}var ge={name:"hide",enabled:!0,phase:"main",requiresIfExists:["preventOverflow"],fn:function(e){var t=e.state,n=e.name,r=t.rects.reference,o=t.rects.popper,i=t.modifiersData.preventOverflow,a=J(t,{elementContext:"reference"}),s=J(t,{altBoundary:!0}),f=ve(a,r),c=ve(s,o,i),p=ye(f),u=ye(c);t.modifiersData[n]={referenceClippingOffsets:f,popperEscapeOffsets:c,isReferenceHidden:p,hasPopperEscaped:u},t.attributes.popper=Object.assign({},t.attributes.popper,{"data-popper-reference-hidden":p,"data-popper-escaped":u})}},be=Z({defaultModifiers:[ee,te,oe,ie]}),xe=[ee,te,oe,ie,ae,le,he,me,ge],we=Z({defaultModifiers:xe});e.applyStyles=ie,e.arrow=me,e.computeStyles=oe,e.createPopper=we,e.createPopperLite=be,e.defaultModifiers=xe,e.detectOverflow=J,e.eventListeners=ee,e.flip=le,e.hide=ge,e.offset=ae,e.popperGenerator=Z,e.popperOffsets=te,e.preventOverflow=he,Object.defineProperty(e,"__esModule",{value:!0})})); + diff --git a/site_libs/quarto-html/quarto-syntax-highlighting-dark.css b/site_libs/quarto-html/quarto-syntax-highlighting-dark.css new file mode 100644 index 0000000..343d2f8 --- /dev/null +++ b/site_libs/quarto-html/quarto-syntax-highlighting-dark.css @@ -0,0 +1,189 @@ +/* quarto syntax highlight colors */ +:root { + --quarto-hl-al-color: #f07178; + --quarto-hl-an-color: #d4d0ab; + --quarto-hl-at-color: #00e0e0; + --quarto-hl-bn-color: #d4d0ab; + --quarto-hl-bu-color: #abe338; + --quarto-hl-ch-color: #abe338; + --quarto-hl-co-color: #f8f8f2; + --quarto-hl-cv-color: #ffd700; + --quarto-hl-cn-color: #ffd700; + --quarto-hl-cf-color: #ffa07a; + --quarto-hl-dt-color: #ffa07a; + --quarto-hl-dv-color: #d4d0ab; + --quarto-hl-do-color: #f8f8f2; + --quarto-hl-er-color: #f07178; + --quarto-hl-ex-color: #00e0e0; + --quarto-hl-fl-color: #d4d0ab; + --quarto-hl-fu-color: #ffa07a; + --quarto-hl-im-color: #abe338; + --quarto-hl-in-color: #d4d0ab; + --quarto-hl-kw-color: #ffa07a; + --quarto-hl-op-color: #ffa07a; + --quarto-hl-ot-color: #00e0e0; + --quarto-hl-pp-color: #dcc6e0; + --quarto-hl-re-color: #00e0e0; + --quarto-hl-sc-color: #abe338; + --quarto-hl-ss-color: #abe338; + --quarto-hl-st-color: #abe338; + --quarto-hl-va-color: #00e0e0; + --quarto-hl-vs-color: #abe338; + --quarto-hl-wa-color: #dcc6e0; +} + +/* other quarto variables */ +:root { + --quarto-font-monospace: SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace; +} + +code span.al { + background-color: #2a0f15; + font-weight: bold; + color: #f07178; +} + +code span.an { + color: #d4d0ab; +} + +code span.at { + color: #00e0e0; +} + +code span.bn { + color: #d4d0ab; +} + +code span.bu { + color: #abe338; +} + +code span.ch { + color: #abe338; +} + +code span.co { + font-style: italic; + color: #f8f8f2; +} + +code span.cv { + color: #ffd700; +} + +code span.cn { + color: #ffd700; +} + +code span.cf { + font-weight: bold; + color: #ffa07a; +} + +code span.dt { + color: #ffa07a; +} + +code span.dv { + color: #d4d0ab; +} + +code span.do { + color: #f8f8f2; +} + +code span.er { + color: #f07178; + text-decoration: underline; +} + +code span.ex { + font-weight: bold; + color: #00e0e0; +} + +code span.fl { + color: #d4d0ab; +} + +code span.fu { + color: #ffa07a; +} + +code span.im { + color: #abe338; +} + +code span.in { + color: #d4d0ab; +} + +code span.kw { + font-weight: bold; + color: #ffa07a; +} + +pre > code.sourceCode > span { + color: #f8f8f2; +} + +code span { + color: #f8f8f2; +} + +code.sourceCode > span { + color: #f8f8f2; +} + +div.sourceCode, +div.sourceCode pre.sourceCode { + color: #f8f8f2; +} + +code span.op { + color: #ffa07a; +} + +code span.ot { + color: #00e0e0; +} + +code span.pp { + color: #dcc6e0; +} + +code span.re { + background-color: #f8f8f2; + color: #00e0e0; +} + +code span.sc { + color: #abe338; +} + +code span.ss { + color: #abe338; +} + +code span.st { + color: #abe338; +} + +code span.va { + color: #00e0e0; +} + +code span.vs { + color: #abe338; +} + +code span.wa { + color: #dcc6e0; +} + +.prevent-inlining { + content: " code.sourceCode > span { + color: #003B4F; +} + +code span { + color: #003B4F; +} + +code.sourceCode > span { + color: #003B4F; +} + +div.sourceCode, +div.sourceCode pre.sourceCode { + color: #003B4F; +} + +code span.ot { + color: #003B4F; + font-style: inherit; +} + +code span.at { + color: #657422; + font-style: inherit; +} + +code span.ss { + color: #20794D; + font-style: inherit; +} + +code span.an { + color: #5E5E5E; + font-style: inherit; +} + +code span.fu { + color: #4758AB; + font-style: inherit; +} + +code span.st { + color: #20794D; + font-style: inherit; +} + +code span.cf { + color: #003B4F; + font-weight: bold; + font-style: inherit; +} + +code span.op { + color: #5E5E5E; + font-style: inherit; +} + +code span.er { + color: #AD0000; + font-style: inherit; +} + +code span.bn { + color: #AD0000; + font-style: inherit; +} + +code span.al { + color: #AD0000; + font-style: inherit; +} + +code span.va { + color: #111111; + font-style: inherit; +} + +code span.bu { + font-style: inherit; +} + +code span.ex { + font-style: inherit; +} + +code span.pp { + color: #AD0000; + font-style: inherit; +} + +code span.in { + color: #5E5E5E; + font-style: inherit; +} + +code span.vs { + color: #20794D; + font-style: inherit; +} + +code span.wa { + color: #5E5E5E; + font-style: italic; +} + +code span.do { + color: #5E5E5E; + font-style: italic; +} + +code span.im { + color: #00769E; + font-style: inherit; +} + +code span.ch { + color: #20794D; + font-style: inherit; +} + +code span.dt { + color: #AD0000; + font-style: inherit; +} + +code span.fl { + color: #AD0000; + font-style: inherit; +} + +code span.co { + color: #5E5E5E; + font-style: inherit; +} + +code span.cv { + color: #5E5E5E; + font-style: italic; +} + +code span.cn { + color: #8f5902; + font-style: inherit; +} + +code span.sc { + color: #5E5E5E; + font-style: inherit; +} + +code span.dv { + color: #AD0000; + font-style: inherit; +} + +code span.kw { + color: #003B4F; + font-weight: bold; + font-style: inherit; +} + +.prevent-inlining { + content: " { + // Find any conflicting margin elements and add margins to the + // top to prevent overlap + const marginChildren = window.document.querySelectorAll( + ".column-margin.column-container > *, .margin-caption, .aside" + ); + + let lastBottom = 0; + for (const marginChild of marginChildren) { + if (marginChild.offsetParent !== null) { + // clear the top margin so we recompute it + marginChild.style.marginTop = null; + const top = marginChild.getBoundingClientRect().top + window.scrollY; + if (top < lastBottom) { + const marginChildStyle = window.getComputedStyle(marginChild); + const marginBottom = parseFloat(marginChildStyle["marginBottom"]); + const margin = lastBottom - top + marginBottom; + marginChild.style.marginTop = `${margin}px`; + } + const styles = window.getComputedStyle(marginChild); + const marginTop = parseFloat(styles["marginTop"]); + lastBottom = top + marginChild.getBoundingClientRect().height + marginTop; + } + } +}; + +window.document.addEventListener("DOMContentLoaded", function (_event) { + // Recompute the position of margin elements anytime the body size changes + if (window.ResizeObserver) { + const resizeObserver = new window.ResizeObserver( + throttle(() => { + layoutMarginEls(); + if ( + window.document.body.getBoundingClientRect().width < 990 && + isReaderMode() + ) { + quartoToggleReader(); + } + }, 50) + ); + resizeObserver.observe(window.document.body); + } + + const tocEl = window.document.querySelector('nav.toc-active[role="doc-toc"]'); + const sidebarEl = window.document.getElementById("quarto-sidebar"); + const leftTocEl = window.document.getElementById("quarto-sidebar-toc-left"); + const marginSidebarEl = window.document.getElementById( + "quarto-margin-sidebar" + ); + // function to determine whether the element has a previous sibling that is active + const prevSiblingIsActiveLink = (el) => { + const sibling = el.previousElementSibling; + if (sibling && sibling.tagName === "A") { + return sibling.classList.contains("active"); + } else { + return false; + } + }; + + // fire slideEnter for bootstrap tab activations (for htmlwidget resize behavior) + function fireSlideEnter(e) { + const event = window.document.createEvent("Event"); + event.initEvent("slideenter", true, true); + window.document.dispatchEvent(event); + } + const tabs = window.document.querySelectorAll('a[data-bs-toggle="tab"]'); + tabs.forEach((tab) => { + tab.addEventListener("shown.bs.tab", fireSlideEnter); + }); + + // fire slideEnter for tabby tab activations (for htmlwidget resize behavior) + document.addEventListener("tabby", fireSlideEnter, false); + + // Track scrolling and mark TOC links as active + // get table of contents and sidebar (bail if we don't have at least one) + const tocLinks = tocEl + ? [...tocEl.querySelectorAll("a[data-scroll-target]")] + : []; + const makeActive = (link) => tocLinks[link].classList.add("active"); + const removeActive = (link) => tocLinks[link].classList.remove("active"); + const removeAllActive = () => + [...Array(tocLinks.length).keys()].forEach((link) => removeActive(link)); + + // activate the anchor for a section associated with this TOC entry + tocLinks.forEach((link) => { + link.addEventListener("click", () => { + if (link.href.indexOf("#") !== -1) { + const anchor = link.href.split("#")[1]; + const heading = window.document.querySelector( + `[data-anchor-id="${anchor}"]` + ); + if (heading) { + // Add the class + heading.classList.add("reveal-anchorjs-link"); + + // function to show the anchor + const handleMouseout = () => { + heading.classList.remove("reveal-anchorjs-link"); + heading.removeEventListener("mouseout", handleMouseout); + }; + + // add a function to clear the anchor when the user mouses out of it + heading.addEventListener("mouseout", handleMouseout); + } + } + }); + }); + + const sections = tocLinks.map((link) => { + const target = link.getAttribute("data-scroll-target"); + if (target.startsWith("#")) { + return window.document.getElementById(decodeURI(`${target.slice(1)}`)); + } else { + return window.document.querySelector(decodeURI(`${target}`)); + } + }); + + const sectionMargin = 200; + let currentActive = 0; + // track whether we've initialized state the first time + let init = false; + + const updateActiveLink = () => { + // The index from bottom to top (e.g. reversed list) + let sectionIndex = -1; + if ( + window.innerHeight + window.pageYOffset >= + window.document.body.offsetHeight + ) { + // This is the no-scroll case where last section should be the active one + sectionIndex = 0; + } else { + // This finds the last section visible on screen that should be made active + sectionIndex = [...sections].reverse().findIndex((section) => { + if (section) { + return window.pageYOffset >= section.offsetTop - sectionMargin; + } else { + return false; + } + }); + } + if (sectionIndex > -1) { + const current = sections.length - sectionIndex - 1; + if (current !== currentActive) { + removeAllActive(); + currentActive = current; + makeActive(current); + if (init) { + window.dispatchEvent(sectionChanged); + } + init = true; + } + } + }; + + const inHiddenRegion = (top, bottom, hiddenRegions) => { + for (const region of hiddenRegions) { + if (top <= region.bottom && bottom >= region.top) { + return true; + } + } + return false; + }; + + const categorySelector = "header.quarto-title-block .quarto-category"; + const activateCategories = (href) => { + // Find any categories + // Surround them with a link pointing back to: + // #category=Authoring + try { + const categoryEls = window.document.querySelectorAll(categorySelector); + for (const categoryEl of categoryEls) { + const categoryText = categoryEl.textContent; + if (categoryText) { + const link = `${href}#category=${encodeURIComponent(categoryText)}`; + const linkEl = window.document.createElement("a"); + linkEl.setAttribute("href", link); + for (const child of categoryEl.childNodes) { + linkEl.append(child); + } + categoryEl.appendChild(linkEl); + } + } + } catch { + // Ignore errors + } + }; + function hasTitleCategories() { + return window.document.querySelector(categorySelector) !== null; + } + + function offsetRelativeUrl(url) { + const offset = getMeta("quarto:offset"); + return offset ? offset + url : url; + } + + function offsetAbsoluteUrl(url) { + const offset = getMeta("quarto:offset"); + const baseUrl = new URL(offset, window.location); + + const projRelativeUrl = url.replace(baseUrl, ""); + if (projRelativeUrl.startsWith("/")) { + return projRelativeUrl; + } else { + return "/" + projRelativeUrl; + } + } + + // read a meta tag value + function getMeta(metaName) { + const metas = window.document.getElementsByTagName("meta"); + for (let i = 0; i < metas.length; i++) { + if (metas[i].getAttribute("name") === metaName) { + return metas[i].getAttribute("content"); + } + } + return ""; + } + + async function findAndActivateCategories() { + const currentPagePath = offsetAbsoluteUrl(window.location.href); + const response = await fetch(offsetRelativeUrl("listings.json")); + if (response.status == 200) { + return response.json().then(function (listingPaths) { + const listingHrefs = []; + for (const listingPath of listingPaths) { + const pathWithoutLeadingSlash = listingPath.listing.substring(1); + for (const item of listingPath.items) { + if ( + item === currentPagePath || + item === currentPagePath + "index.html" + ) { + // Resolve this path against the offset to be sure + // we already are using the correct path to the listing + // (this adjusts the listing urls to be rooted against + // whatever root the page is actually running against) + const relative = offsetRelativeUrl(pathWithoutLeadingSlash); + const baseUrl = window.location; + const resolvedPath = new URL(relative, baseUrl); + listingHrefs.push(resolvedPath.pathname); + break; + } + } + } + + // Look up the tree for a nearby linting and use that if we find one + const nearestListing = findNearestParentListing( + offsetAbsoluteUrl(window.location.pathname), + listingHrefs + ); + if (nearestListing) { + activateCategories(nearestListing); + } else { + // See if the referrer is a listing page for this item + const referredRelativePath = offsetAbsoluteUrl(document.referrer); + const referrerListing = listingHrefs.find((listingHref) => { + const isListingReferrer = + listingHref === referredRelativePath || + listingHref === referredRelativePath + "index.html"; + return isListingReferrer; + }); + + if (referrerListing) { + // Try to use the referrer if possible + activateCategories(referrerListing); + } else if (listingHrefs.length > 0) { + // Otherwise, just fall back to the first listing + activateCategories(listingHrefs[0]); + } + } + }); + } + } + if (hasTitleCategories()) { + findAndActivateCategories(); + } + + const findNearestParentListing = (href, listingHrefs) => { + if (!href || !listingHrefs) { + return undefined; + } + // Look up the tree for a nearby linting and use that if we find one + const relativeParts = href.substring(1).split("/"); + while (relativeParts.length > 0) { + const path = relativeParts.join("/"); + for (const listingHref of listingHrefs) { + if (listingHref.startsWith(path)) { + return listingHref; + } + } + relativeParts.pop(); + } + + return undefined; + }; + + const manageSidebarVisiblity = (el, placeholderDescriptor) => { + let isVisible = true; + let elRect; + + return (hiddenRegions) => { + if (el === null) { + return; + } + + // Find the last element of the TOC + const lastChildEl = el.lastElementChild; + + if (lastChildEl) { + // Converts the sidebar to a menu + const convertToMenu = () => { + for (const child of el.children) { + child.style.opacity = 0; + child.style.overflow = "hidden"; + child.style.pointerEvents = "none"; + } + + nexttick(() => { + const toggleContainer = window.document.createElement("div"); + toggleContainer.style.width = "100%"; + toggleContainer.classList.add("zindex-over-content"); + toggleContainer.classList.add("quarto-sidebar-toggle"); + toggleContainer.classList.add("headroom-target"); // Marks this to be managed by headeroom + toggleContainer.id = placeholderDescriptor.id; + toggleContainer.style.position = "fixed"; + + const toggleIcon = window.document.createElement("i"); + toggleIcon.classList.add("quarto-sidebar-toggle-icon"); + toggleIcon.classList.add("bi"); + toggleIcon.classList.add("bi-caret-down-fill"); + + const toggleTitle = window.document.createElement("div"); + const titleEl = window.document.body.querySelector( + placeholderDescriptor.titleSelector + ); + if (titleEl) { + toggleTitle.append( + titleEl.textContent || titleEl.innerText, + toggleIcon + ); + } + toggleTitle.classList.add("zindex-over-content"); + toggleTitle.classList.add("quarto-sidebar-toggle-title"); + toggleContainer.append(toggleTitle); + + const toggleContents = window.document.createElement("div"); + toggleContents.classList = el.classList; + toggleContents.classList.add("zindex-over-content"); + toggleContents.classList.add("quarto-sidebar-toggle-contents"); + for (const child of el.children) { + if (child.id === "toc-title") { + continue; + } + + const clone = child.cloneNode(true); + clone.style.opacity = 1; + clone.style.pointerEvents = null; + clone.style.display = null; + toggleContents.append(clone); + } + toggleContents.style.height = "0px"; + const positionToggle = () => { + // position the element (top left of parent, same width as parent) + if (!elRect) { + elRect = el.getBoundingClientRect(); + } + toggleContainer.style.left = `${elRect.left}px`; + toggleContainer.style.top = `${elRect.top}px`; + toggleContainer.style.width = `${elRect.width}px`; + }; + positionToggle(); + + toggleContainer.append(toggleContents); + el.parentElement.prepend(toggleContainer); + + // Process clicks + let tocShowing = false; + // Allow the caller to control whether this is dismissed + // when it is clicked (e.g. sidebar navigation supports + // opening and closing the nav tree, so don't dismiss on click) + const clickEl = placeholderDescriptor.dismissOnClick + ? toggleContainer + : toggleTitle; + + const closeToggle = () => { + if (tocShowing) { + toggleContainer.classList.remove("expanded"); + toggleContents.style.height = "0px"; + tocShowing = false; + } + }; + + // Get rid of any expanded toggle if the user scrolls + window.document.addEventListener( + "scroll", + throttle(() => { + closeToggle(); + }, 50) + ); + + // Handle positioning of the toggle + window.addEventListener( + "resize", + throttle(() => { + elRect = undefined; + positionToggle(); + }, 50) + ); + + window.addEventListener("quarto-hrChanged", () => { + elRect = undefined; + }); + + // Process the click + clickEl.onclick = () => { + if (!tocShowing) { + toggleContainer.classList.add("expanded"); + toggleContents.style.height = null; + tocShowing = true; + } else { + closeToggle(); + } + }; + }); + }; + + // Converts a sidebar from a menu back to a sidebar + const convertToSidebar = () => { + for (const child of el.children) { + child.style.opacity = 1; + child.style.overflow = null; + child.style.pointerEvents = null; + } + + const placeholderEl = window.document.getElementById( + placeholderDescriptor.id + ); + if (placeholderEl) { + placeholderEl.remove(); + } + + el.classList.remove("rollup"); + }; + + if (isReaderMode()) { + convertToMenu(); + isVisible = false; + } else { + // Find the top and bottom o the element that is being managed + const elTop = el.offsetTop; + const elBottom = + elTop + lastChildEl.offsetTop + lastChildEl.offsetHeight; + + if (!isVisible) { + // If the element is current not visible reveal if there are + // no conflicts with overlay regions + if (!inHiddenRegion(elTop, elBottom, hiddenRegions)) { + convertToSidebar(); + isVisible = true; + } + } else { + // If the element is visible, hide it if it conflicts with overlay regions + // and insert a placeholder toggle (or if we're in reader mode) + if (inHiddenRegion(elTop, elBottom, hiddenRegions)) { + convertToMenu(); + isVisible = false; + } + } + } + } + }; + }; + + const tabEls = document.querySelectorAll('a[data-bs-toggle="tab"]'); + for (const tabEl of tabEls) { + const id = tabEl.getAttribute("data-bs-target"); + if (id) { + const columnEl = document.querySelector( + `${id} .column-margin, .tabset-margin-content` + ); + if (columnEl) + tabEl.addEventListener("shown.bs.tab", function (event) { + const el = event.srcElement; + if (el) { + const visibleCls = `${el.id}-margin-content`; + // walk up until we find a parent tabset + let panelTabsetEl = el.parentElement; + while (panelTabsetEl) { + if (panelTabsetEl.classList.contains("panel-tabset")) { + break; + } + panelTabsetEl = panelTabsetEl.parentElement; + } + + if (panelTabsetEl) { + const prevSib = panelTabsetEl.previousElementSibling; + if ( + prevSib && + prevSib.classList.contains("tabset-margin-container") + ) { + const childNodes = prevSib.querySelectorAll( + ".tabset-margin-content" + ); + for (const childEl of childNodes) { + if (childEl.classList.contains(visibleCls)) { + childEl.classList.remove("collapse"); + } else { + childEl.classList.add("collapse"); + } + } + } + } + } + + layoutMarginEls(); + }); + } + } + + // Manage the visibility of the toc and the sidebar + const marginScrollVisibility = manageSidebarVisiblity(marginSidebarEl, { + id: "quarto-toc-toggle", + titleSelector: "#toc-title", + dismissOnClick: true, + }); + const sidebarScrollVisiblity = manageSidebarVisiblity(sidebarEl, { + id: "quarto-sidebarnav-toggle", + titleSelector: ".title", + dismissOnClick: false, + }); + let tocLeftScrollVisibility; + if (leftTocEl) { + tocLeftScrollVisibility = manageSidebarVisiblity(leftTocEl, { + id: "quarto-lefttoc-toggle", + titleSelector: "#toc-title", + dismissOnClick: true, + }); + } + + // Find the first element that uses formatting in special columns + const conflictingEls = window.document.body.querySelectorAll( + '[class^="column-"], [class*=" column-"], aside, [class*="margin-caption"], [class*=" margin-caption"], [class*="margin-ref"], [class*=" margin-ref"]' + ); + + // Filter all the possibly conflicting elements into ones + // the do conflict on the left or ride side + const arrConflictingEls = Array.from(conflictingEls); + const leftSideConflictEls = arrConflictingEls.filter((el) => { + if (el.tagName === "ASIDE") { + return false; + } + return Array.from(el.classList).find((className) => { + return ( + className !== "column-body" && + className.startsWith("column-") && + !className.endsWith("right") && + !className.endsWith("container") && + className !== "column-margin" + ); + }); + }); + const rightSideConflictEls = arrConflictingEls.filter((el) => { + if (el.tagName === "ASIDE") { + return true; + } + + const hasMarginCaption = Array.from(el.classList).find((className) => { + return className == "margin-caption"; + }); + if (hasMarginCaption) { + return true; + } + + return Array.from(el.classList).find((className) => { + return ( + className !== "column-body" && + !className.endsWith("container") && + className.startsWith("column-") && + !className.endsWith("left") + ); + }); + }); + + const kOverlapPaddingSize = 10; + function toRegions(els) { + return els.map((el) => { + const boundRect = el.getBoundingClientRect(); + const top = + boundRect.top + + document.documentElement.scrollTop - + kOverlapPaddingSize; + return { + top, + bottom: top + el.scrollHeight + 2 * kOverlapPaddingSize, + }; + }); + } + + let hasObserved = false; + const visibleItemObserver = (els) => { + let visibleElements = [...els]; + const intersectionObserver = new IntersectionObserver( + (entries, _observer) => { + entries.forEach((entry) => { + if (entry.isIntersecting) { + if (visibleElements.indexOf(entry.target) === -1) { + visibleElements.push(entry.target); + } + } else { + visibleElements = visibleElements.filter((visibleEntry) => { + return visibleEntry !== entry; + }); + } + }); + + if (!hasObserved) { + hideOverlappedSidebars(); + } + hasObserved = true; + }, + {} + ); + els.forEach((el) => { + intersectionObserver.observe(el); + }); + + return { + getVisibleEntries: () => { + return visibleElements; + }, + }; + }; + + const rightElementObserver = visibleItemObserver(rightSideConflictEls); + const leftElementObserver = visibleItemObserver(leftSideConflictEls); + + const hideOverlappedSidebars = () => { + marginScrollVisibility(toRegions(rightElementObserver.getVisibleEntries())); + sidebarScrollVisiblity(toRegions(leftElementObserver.getVisibleEntries())); + if (tocLeftScrollVisibility) { + tocLeftScrollVisibility( + toRegions(leftElementObserver.getVisibleEntries()) + ); + } + }; + + window.quartoToggleReader = () => { + // Applies a slow class (or removes it) + // to update the transition speed + const slowTransition = (slow) => { + const manageTransition = (id, slow) => { + const el = document.getElementById(id); + if (el) { + if (slow) { + el.classList.add("slow"); + } else { + el.classList.remove("slow"); + } + } + }; + + manageTransition("TOC", slow); + manageTransition("quarto-sidebar", slow); + }; + const readerMode = !isReaderMode(); + setReaderModeValue(readerMode); + + // If we're entering reader mode, slow the transition + if (readerMode) { + slowTransition(readerMode); + } + highlightReaderToggle(readerMode); + hideOverlappedSidebars(); + + // If we're exiting reader mode, restore the non-slow transition + if (!readerMode) { + slowTransition(!readerMode); + } + }; + + const highlightReaderToggle = (readerMode) => { + const els = document.querySelectorAll(".quarto-reader-toggle"); + if (els) { + els.forEach((el) => { + if (readerMode) { + el.classList.add("reader"); + } else { + el.classList.remove("reader"); + } + }); + } + }; + + const setReaderModeValue = (val) => { + if (window.location.protocol !== "file:") { + window.localStorage.setItem("quarto-reader-mode", val); + } else { + localReaderMode = val; + } + }; + + const isReaderMode = () => { + if (window.location.protocol !== "file:") { + return window.localStorage.getItem("quarto-reader-mode") === "true"; + } else { + return localReaderMode; + } + }; + let localReaderMode = null; + + const tocOpenDepthStr = tocEl?.getAttribute("data-toc-expanded"); + const tocOpenDepth = tocOpenDepthStr ? Number(tocOpenDepthStr) : 1; + + // Walk the TOC and collapse/expand nodes + // Nodes are expanded if: + // - they are top level + // - they have children that are 'active' links + // - they are directly below an link that is 'active' + const walk = (el, depth) => { + // Tick depth when we enter a UL + if (el.tagName === "UL") { + depth = depth + 1; + } + + // It this is active link + let isActiveNode = false; + if (el.tagName === "A" && el.classList.contains("active")) { + isActiveNode = true; + } + + // See if there is an active child to this element + let hasActiveChild = false; + for (child of el.children) { + hasActiveChild = walk(child, depth) || hasActiveChild; + } + + // Process the collapse state if this is an UL + if (el.tagName === "UL") { + if (tocOpenDepth === -1 && depth > 1) { + // toc-expand: false + el.classList.add("collapse"); + } else if ( + depth <= tocOpenDepth || + hasActiveChild || + prevSiblingIsActiveLink(el) + ) { + el.classList.remove("collapse"); + } else { + el.classList.add("collapse"); + } + + // untick depth when we leave a UL + depth = depth - 1; + } + return hasActiveChild || isActiveNode; + }; + + // walk the TOC and expand / collapse any items that should be shown + if (tocEl) { + updateActiveLink(); + walk(tocEl, 0); + } + + // Throttle the scroll event and walk peridiocally + window.document.addEventListener( + "scroll", + throttle(() => { + if (tocEl) { + updateActiveLink(); + walk(tocEl, 0); + } + if (!isReaderMode()) { + hideOverlappedSidebars(); + } + }, 5) + ); + window.addEventListener( + "resize", + throttle(() => { + if (tocEl) { + updateActiveLink(); + walk(tocEl, 0); + } + if (!isReaderMode()) { + hideOverlappedSidebars(); + } + }, 10) + ); + hideOverlappedSidebars(); + highlightReaderToggle(isReaderMode()); +}); + +// grouped tabsets +window.addEventListener("pageshow", (_event) => { + function getTabSettings() { + const data = localStorage.getItem("quarto-persistent-tabsets-data"); + if (!data) { + localStorage.setItem("quarto-persistent-tabsets-data", "{}"); + return {}; + } + if (data) { + return JSON.parse(data); + } + } + + function setTabSettings(data) { + localStorage.setItem( + "quarto-persistent-tabsets-data", + JSON.stringify(data) + ); + } + + function setTabState(groupName, groupValue) { + const data = getTabSettings(); + data[groupName] = groupValue; + setTabSettings(data); + } + + function toggleTab(tab, active) { + const tabPanelId = tab.getAttribute("aria-controls"); + const tabPanel = document.getElementById(tabPanelId); + if (active) { + tab.classList.add("active"); + tabPanel.classList.add("active"); + } else { + tab.classList.remove("active"); + tabPanel.classList.remove("active"); + } + } + + function toggleAll(selectedGroup, selectorsToSync) { + for (const [thisGroup, tabs] of Object.entries(selectorsToSync)) { + const active = selectedGroup === thisGroup; + for (const tab of tabs) { + toggleTab(tab, active); + } + } + } + + function findSelectorsToSyncByLanguage() { + const result = {}; + const tabs = Array.from( + document.querySelectorAll(`div[data-group] a[id^='tabset-']`) + ); + for (const item of tabs) { + const div = item.parentElement.parentElement.parentElement; + const group = div.getAttribute("data-group"); + if (!result[group]) { + result[group] = {}; + } + const selectorsToSync = result[group]; + const value = item.innerHTML; + if (!selectorsToSync[value]) { + selectorsToSync[value] = []; + } + selectorsToSync[value].push(item); + } + return result; + } + + function setupSelectorSync() { + const selectorsToSync = findSelectorsToSyncByLanguage(); + Object.entries(selectorsToSync).forEach(([group, tabSetsByValue]) => { + Object.entries(tabSetsByValue).forEach(([value, items]) => { + items.forEach((item) => { + item.addEventListener("click", (_event) => { + setTabState(group, value); + toggleAll(value, selectorsToSync[group]); + }); + }); + }); + }); + return selectorsToSync; + } + + const selectorsToSync = setupSelectorSync(); + for (const [group, selectedName] of Object.entries(getTabSettings())) { + const selectors = selectorsToSync[group]; + // it's possible that stale state gives us empty selections, so we explicitly check here. + if (selectors) { + toggleAll(selectedName, selectors); + } + } +}); + +function throttle(func, wait) { + let waiting = false; + return function () { + if (!waiting) { + func.apply(this, arguments); + waiting = true; + setTimeout(function () { + waiting = false; + }, wait); + } + }; +} + +function nexttick(func) { + return setTimeout(func, 0); +} diff --git a/site_libs/quarto-html/tippy.css b/site_libs/quarto-html/tippy.css new file mode 100644 index 0000000..e6ae635 --- /dev/null +++ b/site_libs/quarto-html/tippy.css @@ -0,0 +1 @@ +.tippy-box[data-animation=fade][data-state=hidden]{opacity:0}[data-tippy-root]{max-width:calc(100vw - 10px)}.tippy-box{position:relative;background-color:#333;color:#fff;border-radius:4px;font-size:14px;line-height:1.4;white-space:normal;outline:0;transition-property:transform,visibility,opacity}.tippy-box[data-placement^=top]>.tippy-arrow{bottom:0}.tippy-box[data-placement^=top]>.tippy-arrow:before{bottom:-7px;left:0;border-width:8px 8px 0;border-top-color:initial;transform-origin:center top}.tippy-box[data-placement^=bottom]>.tippy-arrow{top:0}.tippy-box[data-placement^=bottom]>.tippy-arrow:before{top:-7px;left:0;border-width:0 8px 8px;border-bottom-color:initial;transform-origin:center bottom}.tippy-box[data-placement^=left]>.tippy-arrow{right:0}.tippy-box[data-placement^=left]>.tippy-arrow:before{border-width:8px 0 8px 8px;border-left-color:initial;right:-7px;transform-origin:center left}.tippy-box[data-placement^=right]>.tippy-arrow{left:0}.tippy-box[data-placement^=right]>.tippy-arrow:before{left:-7px;border-width:8px 8px 8px 0;border-right-color:initial;transform-origin:center right}.tippy-box[data-inertia][data-state=visible]{transition-timing-function:cubic-bezier(.54,1.5,.38,1.11)}.tippy-arrow{width:16px;height:16px;color:#333}.tippy-arrow:before{content:"";position:absolute;border-color:transparent;border-style:solid}.tippy-content{position:relative;padding:5px 9px;z-index:1} \ No newline at end of file diff --git a/site_libs/quarto-html/tippy.umd.min.js b/site_libs/quarto-html/tippy.umd.min.js new file mode 100644 index 0000000..ca292be --- /dev/null +++ b/site_libs/quarto-html/tippy.umd.min.js @@ -0,0 +1,2 @@ +!function(e,t){"object"==typeof exports&&"undefined"!=typeof module?module.exports=t(require("@popperjs/core")):"function"==typeof define&&define.amd?define(["@popperjs/core"],t):(e=e||self).tippy=t(e.Popper)}(this,(function(e){"use strict";var t={passive:!0,capture:!0},n=function(){return document.body};function r(e,t,n){if(Array.isArray(e)){var r=e[t];return null==r?Array.isArray(n)?n[t]:n:r}return e}function o(e,t){var n={}.toString.call(e);return 0===n.indexOf("[object")&&n.indexOf(t+"]")>-1}function i(e,t){return"function"==typeof e?e.apply(void 0,t):e}function a(e,t){return 0===t?e:function(r){clearTimeout(n),n=setTimeout((function(){e(r)}),t)};var n}function s(e,t){var n=Object.assign({},e);return t.forEach((function(e){delete n[e]})),n}function u(e){return[].concat(e)}function c(e,t){-1===e.indexOf(t)&&e.push(t)}function p(e){return e.split("-")[0]}function f(e){return[].slice.call(e)}function l(e){return Object.keys(e).reduce((function(t,n){return void 0!==e[n]&&(t[n]=e[n]),t}),{})}function d(){return document.createElement("div")}function v(e){return["Element","Fragment"].some((function(t){return o(e,t)}))}function m(e){return o(e,"MouseEvent")}function g(e){return!(!e||!e._tippy||e._tippy.reference!==e)}function h(e){return v(e)?[e]:function(e){return o(e,"NodeList")}(e)?f(e):Array.isArray(e)?e:f(document.querySelectorAll(e))}function b(e,t){e.forEach((function(e){e&&(e.style.transitionDuration=t+"ms")}))}function y(e,t){e.forEach((function(e){e&&e.setAttribute("data-state",t)}))}function w(e){var t,n=u(e)[0];return null!=n&&null!=(t=n.ownerDocument)&&t.body?n.ownerDocument:document}function E(e,t,n){var r=t+"EventListener";["transitionend","webkitTransitionEnd"].forEach((function(t){e[r](t,n)}))}function O(e,t){for(var n=t;n;){var r;if(e.contains(n))return!0;n=null==n.getRootNode||null==(r=n.getRootNode())?void 0:r.host}return!1}var x={isTouch:!1},C=0;function T(){x.isTouch||(x.isTouch=!0,window.performance&&document.addEventListener("mousemove",A))}function A(){var e=performance.now();e-C<20&&(x.isTouch=!1,document.removeEventListener("mousemove",A)),C=e}function L(){var e=document.activeElement;if(g(e)){var t=e._tippy;e.blur&&!t.state.isVisible&&e.blur()}}var D=!!("undefined"!=typeof window&&"undefined"!=typeof document)&&!!window.msCrypto,R=Object.assign({appendTo:n,aria:{content:"auto",expanded:"auto"},delay:0,duration:[300,250],getReferenceClientRect:null,hideOnClick:!0,ignoreAttributes:!1,interactive:!1,interactiveBorder:2,interactiveDebounce:0,moveTransition:"",offset:[0,10],onAfterUpdate:function(){},onBeforeUpdate:function(){},onCreate:function(){},onDestroy:function(){},onHidden:function(){},onHide:function(){},onMount:function(){},onShow:function(){},onShown:function(){},onTrigger:function(){},onUntrigger:function(){},onClickOutside:function(){},placement:"top",plugins:[],popperOptions:{},render:null,showOnCreate:!1,touch:!0,trigger:"mouseenter focus",triggerTarget:null},{animateFill:!1,followCursor:!1,inlinePositioning:!1,sticky:!1},{allowHTML:!1,animation:"fade",arrow:!0,content:"",inertia:!1,maxWidth:350,role:"tooltip",theme:"",zIndex:9999}),k=Object.keys(R);function P(e){var t=(e.plugins||[]).reduce((function(t,n){var r,o=n.name,i=n.defaultValue;o&&(t[o]=void 0!==e[o]?e[o]:null!=(r=R[o])?r:i);return t}),{});return Object.assign({},e,t)}function j(e,t){var n=Object.assign({},t,{content:i(t.content,[e])},t.ignoreAttributes?{}:function(e,t){return(t?Object.keys(P(Object.assign({},R,{plugins:t}))):k).reduce((function(t,n){var r=(e.getAttribute("data-tippy-"+n)||"").trim();if(!r)return t;if("content"===n)t[n]=r;else try{t[n]=JSON.parse(r)}catch(e){t[n]=r}return t}),{})}(e,t.plugins));return n.aria=Object.assign({},R.aria,n.aria),n.aria={expanded:"auto"===n.aria.expanded?t.interactive:n.aria.expanded,content:"auto"===n.aria.content?t.interactive?null:"describedby":n.aria.content},n}function M(e,t){e.innerHTML=t}function V(e){var t=d();return!0===e?t.className="tippy-arrow":(t.className="tippy-svg-arrow",v(e)?t.appendChild(e):M(t,e)),t}function I(e,t){v(t.content)?(M(e,""),e.appendChild(t.content)):"function"!=typeof t.content&&(t.allowHTML?M(e,t.content):e.textContent=t.content)}function S(e){var t=e.firstElementChild,n=f(t.children);return{box:t,content:n.find((function(e){return e.classList.contains("tippy-content")})),arrow:n.find((function(e){return e.classList.contains("tippy-arrow")||e.classList.contains("tippy-svg-arrow")})),backdrop:n.find((function(e){return e.classList.contains("tippy-backdrop")}))}}function N(e){var t=d(),n=d();n.className="tippy-box",n.setAttribute("data-state","hidden"),n.setAttribute("tabindex","-1");var r=d();function o(n,r){var o=S(t),i=o.box,a=o.content,s=o.arrow;r.theme?i.setAttribute("data-theme",r.theme):i.removeAttribute("data-theme"),"string"==typeof r.animation?i.setAttribute("data-animation",r.animation):i.removeAttribute("data-animation"),r.inertia?i.setAttribute("data-inertia",""):i.removeAttribute("data-inertia"),i.style.maxWidth="number"==typeof r.maxWidth?r.maxWidth+"px":r.maxWidth,r.role?i.setAttribute("role",r.role):i.removeAttribute("role"),n.content===r.content&&n.allowHTML===r.allowHTML||I(a,e.props),r.arrow?s?n.arrow!==r.arrow&&(i.removeChild(s),i.appendChild(V(r.arrow))):i.appendChild(V(r.arrow)):s&&i.removeChild(s)}return r.className="tippy-content",r.setAttribute("data-state","hidden"),I(r,e.props),t.appendChild(n),n.appendChild(r),o(e.props,e.props),{popper:t,onUpdate:o}}N.$$tippy=!0;var B=1,H=[],U=[];function _(o,s){var v,g,h,C,T,A,L,k,M=j(o,Object.assign({},R,P(l(s)))),V=!1,I=!1,N=!1,_=!1,F=[],W=a(we,M.interactiveDebounce),X=B++,Y=(k=M.plugins).filter((function(e,t){return k.indexOf(e)===t})),$={id:X,reference:o,popper:d(),popperInstance:null,props:M,state:{isEnabled:!0,isVisible:!1,isDestroyed:!1,isMounted:!1,isShown:!1},plugins:Y,clearDelayTimeouts:function(){clearTimeout(v),clearTimeout(g),cancelAnimationFrame(h)},setProps:function(e){if($.state.isDestroyed)return;ae("onBeforeUpdate",[$,e]),be();var t=$.props,n=j(o,Object.assign({},t,l(e),{ignoreAttributes:!0}));$.props=n,he(),t.interactiveDebounce!==n.interactiveDebounce&&(ce(),W=a(we,n.interactiveDebounce));t.triggerTarget&&!n.triggerTarget?u(t.triggerTarget).forEach((function(e){e.removeAttribute("aria-expanded")})):n.triggerTarget&&o.removeAttribute("aria-expanded");ue(),ie(),J&&J(t,n);$.popperInstance&&(Ce(),Ae().forEach((function(e){requestAnimationFrame(e._tippy.popperInstance.forceUpdate)})));ae("onAfterUpdate",[$,e])},setContent:function(e){$.setProps({content:e})},show:function(){var e=$.state.isVisible,t=$.state.isDestroyed,o=!$.state.isEnabled,a=x.isTouch&&!$.props.touch,s=r($.props.duration,0,R.duration);if(e||t||o||a)return;if(te().hasAttribute("disabled"))return;if(ae("onShow",[$],!1),!1===$.props.onShow($))return;$.state.isVisible=!0,ee()&&(z.style.visibility="visible");ie(),de(),$.state.isMounted||(z.style.transition="none");if(ee()){var u=re(),p=u.box,f=u.content;b([p,f],0)}A=function(){var e;if($.state.isVisible&&!_){if(_=!0,z.offsetHeight,z.style.transition=$.props.moveTransition,ee()&&$.props.animation){var t=re(),n=t.box,r=t.content;b([n,r],s),y([n,r],"visible")}se(),ue(),c(U,$),null==(e=$.popperInstance)||e.forceUpdate(),ae("onMount",[$]),$.props.animation&&ee()&&function(e,t){me(e,t)}(s,(function(){$.state.isShown=!0,ae("onShown",[$])}))}},function(){var e,t=$.props.appendTo,r=te();e=$.props.interactive&&t===n||"parent"===t?r.parentNode:i(t,[r]);e.contains(z)||e.appendChild(z);$.state.isMounted=!0,Ce()}()},hide:function(){var e=!$.state.isVisible,t=$.state.isDestroyed,n=!$.state.isEnabled,o=r($.props.duration,1,R.duration);if(e||t||n)return;if(ae("onHide",[$],!1),!1===$.props.onHide($))return;$.state.isVisible=!1,$.state.isShown=!1,_=!1,V=!1,ee()&&(z.style.visibility="hidden");if(ce(),ve(),ie(!0),ee()){var i=re(),a=i.box,s=i.content;$.props.animation&&(b([a,s],o),y([a,s],"hidden"))}se(),ue(),$.props.animation?ee()&&function(e,t){me(e,(function(){!$.state.isVisible&&z.parentNode&&z.parentNode.contains(z)&&t()}))}(o,$.unmount):$.unmount()},hideWithInteractivity:function(e){ne().addEventListener("mousemove",W),c(H,W),W(e)},enable:function(){$.state.isEnabled=!0},disable:function(){$.hide(),$.state.isEnabled=!1},unmount:function(){$.state.isVisible&&$.hide();if(!$.state.isMounted)return;Te(),Ae().forEach((function(e){e._tippy.unmount()})),z.parentNode&&z.parentNode.removeChild(z);U=U.filter((function(e){return e!==$})),$.state.isMounted=!1,ae("onHidden",[$])},destroy:function(){if($.state.isDestroyed)return;$.clearDelayTimeouts(),$.unmount(),be(),delete o._tippy,$.state.isDestroyed=!0,ae("onDestroy",[$])}};if(!M.render)return $;var q=M.render($),z=q.popper,J=q.onUpdate;z.setAttribute("data-tippy-root",""),z.id="tippy-"+$.id,$.popper=z,o._tippy=$,z._tippy=$;var G=Y.map((function(e){return e.fn($)})),K=o.hasAttribute("aria-expanded");return he(),ue(),ie(),ae("onCreate",[$]),M.showOnCreate&&Le(),z.addEventListener("mouseenter",(function(){$.props.interactive&&$.state.isVisible&&$.clearDelayTimeouts()})),z.addEventListener("mouseleave",(function(){$.props.interactive&&$.props.trigger.indexOf("mouseenter")>=0&&ne().addEventListener("mousemove",W)})),$;function Q(){var e=$.props.touch;return Array.isArray(e)?e:[e,0]}function Z(){return"hold"===Q()[0]}function ee(){var e;return!(null==(e=$.props.render)||!e.$$tippy)}function te(){return L||o}function ne(){var e=te().parentNode;return e?w(e):document}function re(){return S(z)}function oe(e){return $.state.isMounted&&!$.state.isVisible||x.isTouch||C&&"focus"===C.type?0:r($.props.delay,e?0:1,R.delay)}function ie(e){void 0===e&&(e=!1),z.style.pointerEvents=$.props.interactive&&!e?"":"none",z.style.zIndex=""+$.props.zIndex}function ae(e,t,n){var r;(void 0===n&&(n=!0),G.forEach((function(n){n[e]&&n[e].apply(n,t)})),n)&&(r=$.props)[e].apply(r,t)}function se(){var e=$.props.aria;if(e.content){var t="aria-"+e.content,n=z.id;u($.props.triggerTarget||o).forEach((function(e){var r=e.getAttribute(t);if($.state.isVisible)e.setAttribute(t,r?r+" "+n:n);else{var o=r&&r.replace(n,"").trim();o?e.setAttribute(t,o):e.removeAttribute(t)}}))}}function ue(){!K&&$.props.aria.expanded&&u($.props.triggerTarget||o).forEach((function(e){$.props.interactive?e.setAttribute("aria-expanded",$.state.isVisible&&e===te()?"true":"false"):e.removeAttribute("aria-expanded")}))}function ce(){ne().removeEventListener("mousemove",W),H=H.filter((function(e){return e!==W}))}function pe(e){if(!x.isTouch||!N&&"mousedown"!==e.type){var t=e.composedPath&&e.composedPath()[0]||e.target;if(!$.props.interactive||!O(z,t)){if(u($.props.triggerTarget||o).some((function(e){return O(e,t)}))){if(x.isTouch)return;if($.state.isVisible&&$.props.trigger.indexOf("click")>=0)return}else ae("onClickOutside",[$,e]);!0===$.props.hideOnClick&&($.clearDelayTimeouts(),$.hide(),I=!0,setTimeout((function(){I=!1})),$.state.isMounted||ve())}}}function fe(){N=!0}function le(){N=!1}function de(){var e=ne();e.addEventListener("mousedown",pe,!0),e.addEventListener("touchend",pe,t),e.addEventListener("touchstart",le,t),e.addEventListener("touchmove",fe,t)}function ve(){var e=ne();e.removeEventListener("mousedown",pe,!0),e.removeEventListener("touchend",pe,t),e.removeEventListener("touchstart",le,t),e.removeEventListener("touchmove",fe,t)}function me(e,t){var n=re().box;function r(e){e.target===n&&(E(n,"remove",r),t())}if(0===e)return t();E(n,"remove",T),E(n,"add",r),T=r}function ge(e,t,n){void 0===n&&(n=!1),u($.props.triggerTarget||o).forEach((function(r){r.addEventListener(e,t,n),F.push({node:r,eventType:e,handler:t,options:n})}))}function he(){var e;Z()&&(ge("touchstart",ye,{passive:!0}),ge("touchend",Ee,{passive:!0})),(e=$.props.trigger,e.split(/\s+/).filter(Boolean)).forEach((function(e){if("manual"!==e)switch(ge(e,ye),e){case"mouseenter":ge("mouseleave",Ee);break;case"focus":ge(D?"focusout":"blur",Oe);break;case"focusin":ge("focusout",Oe)}}))}function be(){F.forEach((function(e){var t=e.node,n=e.eventType,r=e.handler,o=e.options;t.removeEventListener(n,r,o)})),F=[]}function ye(e){var t,n=!1;if($.state.isEnabled&&!xe(e)&&!I){var r="focus"===(null==(t=C)?void 0:t.type);C=e,L=e.currentTarget,ue(),!$.state.isVisible&&m(e)&&H.forEach((function(t){return t(e)})),"click"===e.type&&($.props.trigger.indexOf("mouseenter")<0||V)&&!1!==$.props.hideOnClick&&$.state.isVisible?n=!0:Le(e),"click"===e.type&&(V=!n),n&&!r&&De(e)}}function we(e){var t=e.target,n=te().contains(t)||z.contains(t);"mousemove"===e.type&&n||function(e,t){var n=t.clientX,r=t.clientY;return e.every((function(e){var t=e.popperRect,o=e.popperState,i=e.props.interactiveBorder,a=p(o.placement),s=o.modifiersData.offset;if(!s)return!0;var u="bottom"===a?s.top.y:0,c="top"===a?s.bottom.y:0,f="right"===a?s.left.x:0,l="left"===a?s.right.x:0,d=t.top-r+u>i,v=r-t.bottom-c>i,m=t.left-n+f>i,g=n-t.right-l>i;return d||v||m||g}))}(Ae().concat(z).map((function(e){var t,n=null==(t=e._tippy.popperInstance)?void 0:t.state;return n?{popperRect:e.getBoundingClientRect(),popperState:n,props:M}:null})).filter(Boolean),e)&&(ce(),De(e))}function Ee(e){xe(e)||$.props.trigger.indexOf("click")>=0&&V||($.props.interactive?$.hideWithInteractivity(e):De(e))}function Oe(e){$.props.trigger.indexOf("focusin")<0&&e.target!==te()||$.props.interactive&&e.relatedTarget&&z.contains(e.relatedTarget)||De(e)}function xe(e){return!!x.isTouch&&Z()!==e.type.indexOf("touch")>=0}function Ce(){Te();var t=$.props,n=t.popperOptions,r=t.placement,i=t.offset,a=t.getReferenceClientRect,s=t.moveTransition,u=ee()?S(z).arrow:null,c=a?{getBoundingClientRect:a,contextElement:a.contextElement||te()}:o,p=[{name:"offset",options:{offset:i}},{name:"preventOverflow",options:{padding:{top:2,bottom:2,left:5,right:5}}},{name:"flip",options:{padding:5}},{name:"computeStyles",options:{adaptive:!s}},{name:"$$tippy",enabled:!0,phase:"beforeWrite",requires:["computeStyles"],fn:function(e){var t=e.state;if(ee()){var n=re().box;["placement","reference-hidden","escaped"].forEach((function(e){"placement"===e?n.setAttribute("data-placement",t.placement):t.attributes.popper["data-popper-"+e]?n.setAttribute("data-"+e,""):n.removeAttribute("data-"+e)})),t.attributes.popper={}}}}];ee()&&u&&p.push({name:"arrow",options:{element:u,padding:3}}),p.push.apply(p,(null==n?void 0:n.modifiers)||[]),$.popperInstance=e.createPopper(c,z,Object.assign({},n,{placement:r,onFirstUpdate:A,modifiers:p}))}function Te(){$.popperInstance&&($.popperInstance.destroy(),$.popperInstance=null)}function Ae(){return f(z.querySelectorAll("[data-tippy-root]"))}function Le(e){$.clearDelayTimeouts(),e&&ae("onTrigger",[$,e]),de();var t=oe(!0),n=Q(),r=n[0],o=n[1];x.isTouch&&"hold"===r&&o&&(t=o),t?v=setTimeout((function(){$.show()}),t):$.show()}function De(e){if($.clearDelayTimeouts(),ae("onUntrigger",[$,e]),$.state.isVisible){if(!($.props.trigger.indexOf("mouseenter")>=0&&$.props.trigger.indexOf("click")>=0&&["mouseleave","mousemove"].indexOf(e.type)>=0&&V)){var t=oe(!1);t?g=setTimeout((function(){$.state.isVisible&&$.hide()}),t):h=requestAnimationFrame((function(){$.hide()}))}}else ve()}}function F(e,n){void 0===n&&(n={});var r=R.plugins.concat(n.plugins||[]);document.addEventListener("touchstart",T,t),window.addEventListener("blur",L);var o=Object.assign({},n,{plugins:r}),i=h(e).reduce((function(e,t){var n=t&&_(t,o);return n&&e.push(n),e}),[]);return v(e)?i[0]:i}F.defaultProps=R,F.setDefaultProps=function(e){Object.keys(e).forEach((function(t){R[t]=e[t]}))},F.currentInput=x;var W=Object.assign({},e.applyStyles,{effect:function(e){var t=e.state,n={popper:{position:t.options.strategy,left:"0",top:"0",margin:"0"},arrow:{position:"absolute"},reference:{}};Object.assign(t.elements.popper.style,n.popper),t.styles=n,t.elements.arrow&&Object.assign(t.elements.arrow.style,n.arrow)}}),X={mouseover:"mouseenter",focusin:"focus",click:"click"};var Y={name:"animateFill",defaultValue:!1,fn:function(e){var t;if(null==(t=e.props.render)||!t.$$tippy)return{};var n=S(e.popper),r=n.box,o=n.content,i=e.props.animateFill?function(){var e=d();return e.className="tippy-backdrop",y([e],"hidden"),e}():null;return{onCreate:function(){i&&(r.insertBefore(i,r.firstElementChild),r.setAttribute("data-animatefill",""),r.style.overflow="hidden",e.setProps({arrow:!1,animation:"shift-away"}))},onMount:function(){if(i){var e=r.style.transitionDuration,t=Number(e.replace("ms",""));o.style.transitionDelay=Math.round(t/10)+"ms",i.style.transitionDuration=e,y([i],"visible")}},onShow:function(){i&&(i.style.transitionDuration="0ms")},onHide:function(){i&&y([i],"hidden")}}}};var $={clientX:0,clientY:0},q=[];function z(e){var t=e.clientX,n=e.clientY;$={clientX:t,clientY:n}}var J={name:"followCursor",defaultValue:!1,fn:function(e){var t=e.reference,n=w(e.props.triggerTarget||t),r=!1,o=!1,i=!0,a=e.props;function s(){return"initial"===e.props.followCursor&&e.state.isVisible}function u(){n.addEventListener("mousemove",f)}function c(){n.removeEventListener("mousemove",f)}function p(){r=!0,e.setProps({getReferenceClientRect:null}),r=!1}function f(n){var r=!n.target||t.contains(n.target),o=e.props.followCursor,i=n.clientX,a=n.clientY,s=t.getBoundingClientRect(),u=i-s.left,c=a-s.top;!r&&e.props.interactive||e.setProps({getReferenceClientRect:function(){var e=t.getBoundingClientRect(),n=i,r=a;"initial"===o&&(n=e.left+u,r=e.top+c);var s="horizontal"===o?e.top:r,p="vertical"===o?e.right:n,f="horizontal"===o?e.bottom:r,l="vertical"===o?e.left:n;return{width:p-l,height:f-s,top:s,right:p,bottom:f,left:l}}})}function l(){e.props.followCursor&&(q.push({instance:e,doc:n}),function(e){e.addEventListener("mousemove",z)}(n))}function d(){0===(q=q.filter((function(t){return t.instance!==e}))).filter((function(e){return e.doc===n})).length&&function(e){e.removeEventListener("mousemove",z)}(n)}return{onCreate:l,onDestroy:d,onBeforeUpdate:function(){a=e.props},onAfterUpdate:function(t,n){var i=n.followCursor;r||void 0!==i&&a.followCursor!==i&&(d(),i?(l(),!e.state.isMounted||o||s()||u()):(c(),p()))},onMount:function(){e.props.followCursor&&!o&&(i&&(f($),i=!1),s()||u())},onTrigger:function(e,t){m(t)&&($={clientX:t.clientX,clientY:t.clientY}),o="focus"===t.type},onHidden:function(){e.props.followCursor&&(p(),c(),i=!0)}}}};var G={name:"inlinePositioning",defaultValue:!1,fn:function(e){var t,n=e.reference;var r=-1,o=!1,i=[],a={name:"tippyInlinePositioning",enabled:!0,phase:"afterWrite",fn:function(o){var a=o.state;e.props.inlinePositioning&&(-1!==i.indexOf(a.placement)&&(i=[]),t!==a.placement&&-1===i.indexOf(a.placement)&&(i.push(a.placement),e.setProps({getReferenceClientRect:function(){return function(e){return function(e,t,n,r){if(n.length<2||null===e)return t;if(2===n.length&&r>=0&&n[0].left>n[1].right)return n[r]||t;switch(e){case"top":case"bottom":var o=n[0],i=n[n.length-1],a="top"===e,s=o.top,u=i.bottom,c=a?o.left:i.left,p=a?o.right:i.right;return{top:s,bottom:u,left:c,right:p,width:p-c,height:u-s};case"left":case"right":var f=Math.min.apply(Math,n.map((function(e){return e.left}))),l=Math.max.apply(Math,n.map((function(e){return e.right}))),d=n.filter((function(t){return"left"===e?t.left===f:t.right===l})),v=d[0].top,m=d[d.length-1].bottom;return{top:v,bottom:m,left:f,right:l,width:l-f,height:m-v};default:return t}}(p(e),n.getBoundingClientRect(),f(n.getClientRects()),r)}(a.placement)}})),t=a.placement)}};function s(){var t;o||(t=function(e,t){var n;return{popperOptions:Object.assign({},e.popperOptions,{modifiers:[].concat(((null==(n=e.popperOptions)?void 0:n.modifiers)||[]).filter((function(e){return e.name!==t.name})),[t])})}}(e.props,a),o=!0,e.setProps(t),o=!1)}return{onCreate:s,onAfterUpdate:s,onTrigger:function(t,n){if(m(n)){var o=f(e.reference.getClientRects()),i=o.find((function(e){return e.left-2<=n.clientX&&e.right+2>=n.clientX&&e.top-2<=n.clientY&&e.bottom+2>=n.clientY})),a=o.indexOf(i);r=a>-1?a:r}},onHidden:function(){r=-1}}}};var K={name:"sticky",defaultValue:!1,fn:function(e){var t=e.reference,n=e.popper;function r(t){return!0===e.props.sticky||e.props.sticky===t}var o=null,i=null;function a(){var s=r("reference")?(e.popperInstance?e.popperInstance.state.elements.reference:t).getBoundingClientRect():null,u=r("popper")?n.getBoundingClientRect():null;(s&&Q(o,s)||u&&Q(i,u))&&e.popperInstance&&e.popperInstance.update(),o=s,i=u,e.state.isMounted&&requestAnimationFrame(a)}return{onMount:function(){e.props.sticky&&a()}}}};function Q(e,t){return!e||!t||(e.top!==t.top||e.right!==t.right||e.bottom!==t.bottom||e.left!==t.left)}return F.setDefaultProps({plugins:[Y,J,G,K],render:N}),F.createSingleton=function(e,t){var n;void 0===t&&(t={});var r,o=e,i=[],a=[],c=t.overrides,p=[],f=!1;function l(){a=o.map((function(e){return u(e.props.triggerTarget||e.reference)})).reduce((function(e,t){return e.concat(t)}),[])}function v(){i=o.map((function(e){return e.reference}))}function m(e){o.forEach((function(t){e?t.enable():t.disable()}))}function g(e){return o.map((function(t){var n=t.setProps;return t.setProps=function(o){n(o),t.reference===r&&e.setProps(o)},function(){t.setProps=n}}))}function h(e,t){var n=a.indexOf(t);if(t!==r){r=t;var s=(c||[]).concat("content").reduce((function(e,t){return e[t]=o[n].props[t],e}),{});e.setProps(Object.assign({},s,{getReferenceClientRect:"function"==typeof s.getReferenceClientRect?s.getReferenceClientRect:function(){var e;return null==(e=i[n])?void 0:e.getBoundingClientRect()}}))}}m(!1),v(),l();var b={fn:function(){return{onDestroy:function(){m(!0)},onHidden:function(){r=null},onClickOutside:function(e){e.props.showOnCreate&&!f&&(f=!0,r=null)},onShow:function(e){e.props.showOnCreate&&!f&&(f=!0,h(e,i[0]))},onTrigger:function(e,t){h(e,t.currentTarget)}}}},y=F(d(),Object.assign({},s(t,["overrides"]),{plugins:[b].concat(t.plugins||[]),triggerTarget:a,popperOptions:Object.assign({},t.popperOptions,{modifiers:[].concat((null==(n=t.popperOptions)?void 0:n.modifiers)||[],[W])})})),w=y.show;y.show=function(e){if(w(),!r&&null==e)return h(y,i[0]);if(!r||null!=e){if("number"==typeof e)return i[e]&&h(y,i[e]);if(o.indexOf(e)>=0){var t=e.reference;return h(y,t)}return i.indexOf(e)>=0?h(y,e):void 0}},y.showNext=function(){var e=i[0];if(!r)return y.show(0);var t=i.indexOf(r);y.show(i[t+1]||e)},y.showPrevious=function(){var e=i[i.length-1];if(!r)return y.show(e);var t=i.indexOf(r),n=i[t-1]||e;y.show(n)};var E=y.setProps;return y.setProps=function(e){c=e.overrides||c,E(e)},y.setInstances=function(e){m(!0),p.forEach((function(e){return e()})),o=e,m(!1),v(),l(),p=g(y),y.setProps({triggerTarget:a})},p=g(y),y},F.delegate=function(e,n){var r=[],o=[],i=!1,a=n.target,c=s(n,["target"]),p=Object.assign({},c,{trigger:"manual",touch:!1}),f=Object.assign({touch:R.touch},c,{showOnCreate:!0}),l=F(e,p);function d(e){if(e.target&&!i){var t=e.target.closest(a);if(t){var r=t.getAttribute("data-tippy-trigger")||n.trigger||R.trigger;if(!t._tippy&&!("touchstart"===e.type&&"boolean"==typeof f.touch||"touchstart"!==e.type&&r.indexOf(X[e.type])<0)){var s=F(t,f);s&&(o=o.concat(s))}}}}function v(e,t,n,o){void 0===o&&(o=!1),e.addEventListener(t,n,o),r.push({node:e,eventType:t,handler:n,options:o})}return u(l).forEach((function(e){var n=e.destroy,a=e.enable,s=e.disable;e.destroy=function(e){void 0===e&&(e=!0),e&&o.forEach((function(e){e.destroy()})),o=[],r.forEach((function(e){var t=e.node,n=e.eventType,r=e.handler,o=e.options;t.removeEventListener(n,r,o)})),r=[],n()},e.enable=function(){a(),o.forEach((function(e){return e.enable()})),i=!1},e.disable=function(){s(),o.forEach((function(e){return e.disable()})),i=!0},function(e){var n=e.reference;v(n,"touchstart",d,t),v(n,"mouseover",d),v(n,"focusin",d),v(n,"click",d)}(e)})),l},F.hideAll=function(e){var t=void 0===e?{}:e,n=t.exclude,r=t.duration;U.forEach((function(e){var t=!1;if(n&&(t=g(n)?e.reference===n:e.popper===n.popper),!t){var o=e.props.duration;e.setProps({duration:r}),e.hide(),e.state.isDestroyed||e.setProps({duration:o})}}))},F.roundArrow='',F})); + diff --git a/site_libs/quarto-nav/headroom.min.js b/site_libs/quarto-nav/headroom.min.js new file mode 100644 index 0000000..b08f1df --- /dev/null +++ b/site_libs/quarto-nav/headroom.min.js @@ -0,0 +1,7 @@ +/*! + * headroom.js v0.12.0 - Give your page some headroom. Hide your header until you need it + * Copyright (c) 2020 Nick Williams - http://wicky.nillia.ms/headroom.js + * License: MIT + */ + +!function(t,n){"object"==typeof exports&&"undefined"!=typeof module?module.exports=n():"function"==typeof define&&define.amd?define(n):(t=t||self).Headroom=n()}(this,function(){"use strict";function t(){return"undefined"!=typeof window}function d(t){return function(t){return t&&t.document&&function(t){return 9===t.nodeType}(t.document)}(t)?function(t){var n=t.document,o=n.body,s=n.documentElement;return{scrollHeight:function(){return Math.max(o.scrollHeight,s.scrollHeight,o.offsetHeight,s.offsetHeight,o.clientHeight,s.clientHeight)},height:function(){return t.innerHeight||s.clientHeight||o.clientHeight},scrollY:function(){return void 0!==t.pageYOffset?t.pageYOffset:(s||o.parentNode||o).scrollTop}}}(t):function(t){return{scrollHeight:function(){return Math.max(t.scrollHeight,t.offsetHeight,t.clientHeight)},height:function(){return Math.max(t.offsetHeight,t.clientHeight)},scrollY:function(){return t.scrollTop}}}(t)}function n(t,s,e){var n,o=function(){var n=!1;try{var t={get passive(){n=!0}};window.addEventListener("test",t,t),window.removeEventListener("test",t,t)}catch(t){n=!1}return n}(),i=!1,r=d(t),l=r.scrollY(),a={};function c(){var t=Math.round(r.scrollY()),n=r.height(),o=r.scrollHeight();a.scrollY=t,a.lastScrollY=l,a.direction=ls.tolerance[a.direction],e(a),l=t,i=!1}function h(){i||(i=!0,n=requestAnimationFrame(c))}var u=!!o&&{passive:!0,capture:!1};return t.addEventListener("scroll",h,u),c(),{destroy:function(){cancelAnimationFrame(n),t.removeEventListener("scroll",h,u)}}}function o(t){return t===Object(t)?t:{down:t,up:t}}function s(t,n){n=n||{},Object.assign(this,s.options,n),this.classes=Object.assign({},s.options.classes,n.classes),this.elem=t,this.tolerance=o(this.tolerance),this.offset=o(this.offset),this.initialised=!1,this.frozen=!1}return s.prototype={constructor:s,init:function(){return s.cutsTheMustard&&!this.initialised&&(this.addClass("initial"),this.initialised=!0,setTimeout(function(t){t.scrollTracker=n(t.scroller,{offset:t.offset,tolerance:t.tolerance},t.update.bind(t))},100,this)),this},destroy:function(){this.initialised=!1,Object.keys(this.classes).forEach(this.removeClass,this),this.scrollTracker.destroy()},unpin:function(){!this.hasClass("pinned")&&this.hasClass("unpinned")||(this.addClass("unpinned"),this.removeClass("pinned"),this.onUnpin&&this.onUnpin.call(this))},pin:function(){this.hasClass("unpinned")&&(this.addClass("pinned"),this.removeClass("unpinned"),this.onPin&&this.onPin.call(this))},freeze:function(){this.frozen=!0,this.addClass("frozen")},unfreeze:function(){this.frozen=!1,this.removeClass("frozen")},top:function(){this.hasClass("top")||(this.addClass("top"),this.removeClass("notTop"),this.onTop&&this.onTop.call(this))},notTop:function(){this.hasClass("notTop")||(this.addClass("notTop"),this.removeClass("top"),this.onNotTop&&this.onNotTop.call(this))},bottom:function(){this.hasClass("bottom")||(this.addClass("bottom"),this.removeClass("notBottom"),this.onBottom&&this.onBottom.call(this))},notBottom:function(){this.hasClass("notBottom")||(this.addClass("notBottom"),this.removeClass("bottom"),this.onNotBottom&&this.onNotBottom.call(this))},shouldUnpin:function(t){return"down"===t.direction&&!t.top&&t.toleranceExceeded},shouldPin:function(t){return"up"===t.direction&&t.toleranceExceeded||t.top},addClass:function(t){this.elem.classList.add.apply(this.elem.classList,this.classes[t].split(" "))},removeClass:function(t){this.elem.classList.remove.apply(this.elem.classList,this.classes[t].split(" "))},hasClass:function(t){return this.classes[t].split(" ").every(function(t){return this.classList.contains(t)},this.elem)},update:function(t){t.isOutOfBounds||!0!==this.frozen&&(t.top?this.top():this.notTop(),t.bottom?this.bottom():this.notBottom(),this.shouldUnpin(t)?this.unpin():this.shouldPin(t)&&this.pin())}},s.options={tolerance:{up:0,down:0},offset:0,scroller:t()?window:null,classes:{frozen:"headroom--frozen",pinned:"headroom--pinned",unpinned:"headroom--unpinned",top:"headroom--top",notTop:"headroom--not-top",bottom:"headroom--bottom",notBottom:"headroom--not-bottom",initial:"headroom"}},s.cutsTheMustard=!!(t()&&function(){}.bind&&"classList"in document.documentElement&&Object.assign&&Object.keys&&requestAnimationFrame),s}); diff --git a/site_libs/quarto-nav/quarto-nav.js b/site_libs/quarto-nav/quarto-nav.js new file mode 100644 index 0000000..38cc430 --- /dev/null +++ b/site_libs/quarto-nav/quarto-nav.js @@ -0,0 +1,325 @@ +const headroomChanged = new CustomEvent("quarto-hrChanged", { + detail: {}, + bubbles: true, + cancelable: false, + composed: false, +}); + +const announceDismiss = () => { + const annEl = window.document.getElementById("quarto-announcement"); + if (annEl) { + annEl.remove(); + + const annId = annEl.getAttribute("data-announcement-id"); + window.localStorage.setItem(`quarto-announce-${annId}`, "true"); + } +}; + +const announceRegister = () => { + const annEl = window.document.getElementById("quarto-announcement"); + if (annEl) { + const annId = annEl.getAttribute("data-announcement-id"); + const isDismissed = + window.localStorage.getItem(`quarto-announce-${annId}`) || false; + if (isDismissed) { + announceDismiss(); + return; + } else { + annEl.classList.remove("hidden"); + } + + const actionEl = annEl.querySelector(".quarto-announcement-action"); + if (actionEl) { + actionEl.addEventListener("click", function (e) { + e.preventDefault(); + // Hide the bar immediately + announceDismiss(); + }); + } + } +}; + +window.document.addEventListener("DOMContentLoaded", function () { + let init = false; + + announceRegister(); + + // Manage the back to top button, if one is present. + let lastScrollTop = window.pageYOffset || document.documentElement.scrollTop; + const scrollDownBuffer = 5; + const scrollUpBuffer = 35; + const btn = document.getElementById("quarto-back-to-top"); + const hideBackToTop = () => { + btn.style.display = "none"; + }; + const showBackToTop = () => { + btn.style.display = "inline-block"; + }; + if (btn) { + window.document.addEventListener( + "scroll", + function () { + const currentScrollTop = + window.pageYOffset || document.documentElement.scrollTop; + + // Shows and hides the button 'intelligently' as the user scrolls + if (currentScrollTop - scrollDownBuffer > lastScrollTop) { + hideBackToTop(); + lastScrollTop = currentScrollTop <= 0 ? 0 : currentScrollTop; + } else if (currentScrollTop < lastScrollTop - scrollUpBuffer) { + showBackToTop(); + lastScrollTop = currentScrollTop <= 0 ? 0 : currentScrollTop; + } + + // Show the button at the bottom, hides it at the top + if (currentScrollTop <= 0) { + hideBackToTop(); + } else if ( + window.innerHeight + currentScrollTop >= + document.body.offsetHeight + ) { + showBackToTop(); + } + }, + false + ); + } + + function throttle(func, wait) { + var timeout; + return function () { + const context = this; + const args = arguments; + const later = function () { + clearTimeout(timeout); + timeout = null; + func.apply(context, args); + }; + + if (!timeout) { + timeout = setTimeout(later, wait); + } + }; + } + + function headerOffset() { + // Set an offset if there is are fixed top navbar + const headerEl = window.document.querySelector("header.fixed-top"); + if (headerEl) { + return headerEl.clientHeight; + } else { + return 0; + } + } + + function footerOffset() { + const footerEl = window.document.querySelector("footer.footer"); + if (footerEl) { + return footerEl.clientHeight; + } else { + return 0; + } + } + + function dashboardOffset() { + const dashboardNavEl = window.document.getElementById( + "quarto-dashboard-header" + ); + if (dashboardNavEl !== null) { + return dashboardNavEl.clientHeight; + } else { + return 0; + } + } + + function updateDocumentOffsetWithoutAnimation() { + updateDocumentOffset(false); + } + + function updateDocumentOffset(animated) { + // set body offset + const topOffset = headerOffset(); + const bodyOffset = topOffset + footerOffset() + dashboardOffset(); + const bodyEl = window.document.body; + bodyEl.setAttribute("data-bs-offset", topOffset); + bodyEl.style.paddingTop = topOffset + "px"; + + // deal with sidebar offsets + const sidebars = window.document.querySelectorAll( + ".sidebar, .headroom-target" + ); + sidebars.forEach((sidebar) => { + if (!animated) { + sidebar.classList.add("notransition"); + // Remove the no transition class after the animation has time to complete + setTimeout(function () { + sidebar.classList.remove("notransition"); + }, 201); + } + + if (window.Headroom && sidebar.classList.contains("sidebar-unpinned")) { + sidebar.style.top = "0"; + sidebar.style.maxHeight = "100vh"; + } else { + sidebar.style.top = topOffset + "px"; + sidebar.style.maxHeight = "calc(100vh - " + topOffset + "px)"; + } + }); + + // allow space for footer + const mainContainer = window.document.querySelector(".quarto-container"); + if (mainContainer) { + mainContainer.style.minHeight = "calc(100vh - " + bodyOffset + "px)"; + } + + // link offset + let linkStyle = window.document.querySelector("#quarto-target-style"); + if (!linkStyle) { + linkStyle = window.document.createElement("style"); + linkStyle.setAttribute("id", "quarto-target-style"); + window.document.head.appendChild(linkStyle); + } + while (linkStyle.firstChild) { + linkStyle.removeChild(linkStyle.firstChild); + } + if (topOffset > 0) { + linkStyle.appendChild( + window.document.createTextNode(` + section:target::before { + content: ""; + display: block; + height: ${topOffset}px; + margin: -${topOffset}px 0 0; + }`) + ); + } + if (init) { + window.dispatchEvent(headroomChanged); + } + init = true; + } + + // initialize headroom + var header = window.document.querySelector("#quarto-header"); + if (header && window.Headroom) { + const headroom = new window.Headroom(header, { + tolerance: 5, + onPin: function () { + const sidebars = window.document.querySelectorAll( + ".sidebar, .headroom-target" + ); + sidebars.forEach((sidebar) => { + sidebar.classList.remove("sidebar-unpinned"); + }); + updateDocumentOffset(); + }, + onUnpin: function () { + const sidebars = window.document.querySelectorAll( + ".sidebar, .headroom-target" + ); + sidebars.forEach((sidebar) => { + sidebar.classList.add("sidebar-unpinned"); + }); + updateDocumentOffset(); + }, + }); + headroom.init(); + + let frozen = false; + window.quartoToggleHeadroom = function () { + if (frozen) { + headroom.unfreeze(); + frozen = false; + } else { + headroom.freeze(); + frozen = true; + } + }; + } + + window.addEventListener( + "hashchange", + function (e) { + if ( + getComputedStyle(document.documentElement).scrollBehavior !== "smooth" + ) { + window.scrollTo(0, window.pageYOffset - headerOffset()); + } + }, + false + ); + + // Observe size changed for the header + const headerEl = window.document.querySelector("header.fixed-top"); + if (headerEl && window.ResizeObserver) { + const observer = new window.ResizeObserver(() => { + setTimeout(updateDocumentOffsetWithoutAnimation, 0); + }); + observer.observe(headerEl, { + attributes: true, + childList: true, + characterData: true, + }); + } else { + window.addEventListener( + "resize", + throttle(updateDocumentOffsetWithoutAnimation, 50) + ); + } + setTimeout(updateDocumentOffsetWithoutAnimation, 250); + + // fixup index.html links if we aren't on the filesystem + if (window.location.protocol !== "file:") { + const links = window.document.querySelectorAll("a"); + for (let i = 0; i < links.length; i++) { + if (links[i].href) { + links[i].dataset.originalHref = links[i].href; + links[i].href = links[i].href.replace(/\/index\.html/, "/"); + } + } + + // Fixup any sharing links that require urls + // Append url to any sharing urls + const sharingLinks = window.document.querySelectorAll( + "a.sidebar-tools-main-item, a.quarto-navigation-tool, a.quarto-navbar-tools, a.quarto-navbar-tools-item" + ); + for (let i = 0; i < sharingLinks.length; i++) { + const sharingLink = sharingLinks[i]; + const href = sharingLink.getAttribute("href"); + if (href) { + sharingLink.setAttribute( + "href", + href.replace("|url|", window.location.href) + ); + } + } + + // Scroll the active navigation item into view, if necessary + const navSidebar = window.document.querySelector("nav#quarto-sidebar"); + if (navSidebar) { + // Find the active item + const activeItem = navSidebar.querySelector("li.sidebar-item a.active"); + if (activeItem) { + // Wait for the scroll height and height to resolve by observing size changes on the + // nav element that is scrollable + const resizeObserver = new ResizeObserver((_entries) => { + // The bottom of the element + const elBottom = activeItem.offsetTop; + const viewBottom = navSidebar.scrollTop + navSidebar.clientHeight; + + // The element height and scroll height are the same, then we are still loading + if (viewBottom !== navSidebar.scrollHeight) { + // Determine if the item isn't visible and scroll to it + if (elBottom >= viewBottom) { + navSidebar.scrollTop = elBottom; + } + + // stop observing now since we've completed the scroll + resizeObserver.unobserve(navSidebar); + } + }); + resizeObserver.observe(navSidebar); + } + } + } +}); diff --git a/site_libs/quarto-search/autocomplete.umd.js b/site_libs/quarto-search/autocomplete.umd.js new file mode 100644 index 0000000..ae0063a --- /dev/null +++ b/site_libs/quarto-search/autocomplete.umd.js @@ -0,0 +1,3 @@ +/*! @algolia/autocomplete-js 1.11.1 | MIT License | © Algolia, Inc. and contributors | https://github.com/algolia/autocomplete */ +!function(e,t){"object"==typeof exports&&"undefined"!=typeof module?t(exports):"function"==typeof define&&define.amd?define(["exports"],t):t((e="undefined"!=typeof globalThis?globalThis:e||self)["@algolia/autocomplete-js"]={})}(this,(function(e){"use strict";function t(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);t&&(r=r.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,r)}return n}function n(e){for(var n=1;n=0||(o[n]=e[n]);return o}(e,t);if(Object.getOwnPropertySymbols){var i=Object.getOwnPropertySymbols(e);for(r=0;r=0||Object.prototype.propertyIsEnumerable.call(e,n)&&(o[n]=e[n])}return o}function a(e,t){return function(e){if(Array.isArray(e))return e}(e)||function(e,t){var n=null==e?null:"undefined"!=typeof Symbol&&e[Symbol.iterator]||e["@@iterator"];if(null!=n){var r,o,i,u,a=[],l=!0,c=!1;try{if(i=(n=n.call(e)).next,0===t){if(Object(n)!==n)return;l=!1}else for(;!(l=(r=i.call(n)).done)&&(a.push(r.value),a.length!==t);l=!0);}catch(e){c=!0,o=e}finally{try{if(!l&&null!=n.return&&(u=n.return(),Object(u)!==u))return}finally{if(c)throw o}}return a}}(e,t)||c(e,t)||function(){throw new TypeError("Invalid attempt to destructure non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method.")}()}function l(e){return function(e){if(Array.isArray(e))return s(e)}(e)||function(e){if("undefined"!=typeof Symbol&&null!=e[Symbol.iterator]||null!=e["@@iterator"])return Array.from(e)}(e)||c(e)||function(){throw new TypeError("Invalid attempt to spread non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method.")}()}function c(e,t){if(e){if("string"==typeof e)return s(e,t);var n=Object.prototype.toString.call(e).slice(8,-1);return"Object"===n&&e.constructor&&(n=e.constructor.name),"Map"===n||"Set"===n?Array.from(e):"Arguments"===n||/^(?:Ui|I)nt(?:8|16|32)(?:Clamped)?Array$/.test(n)?s(e,t):void 0}}function s(e,t){(null==t||t>e.length)&&(t=e.length);for(var n=0,r=new Array(t);ne.length)&&(t=e.length);for(var n=0,r=new Array(t);ne.length)&&(t=e.length);for(var n=0,r=new Array(t);n=0||(o[n]=e[n]);return o}(e,t);if(Object.getOwnPropertySymbols){var i=Object.getOwnPropertySymbols(e);for(r=0;r=0||Object.prototype.propertyIsEnumerable.call(e,n)&&(o[n]=e[n])}return o}function x(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);t&&(r=r.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,r)}return n}function N(e){for(var t=1;t1&&void 0!==arguments[1]?arguments[1]:20,n=[],r=0;r=3||2===n&&r>=4||1===n&&r>=10);function i(t,n,r){if(o&&void 0!==r){var i=r[0].__autocomplete_algoliaCredentials,u={"X-Algolia-Application-Id":i.appId,"X-Algolia-API-Key":i.apiKey};e.apply(void 0,[t].concat(D(n),[{headers:u}]))}else e.apply(void 0,[t].concat(D(n)))}return{init:function(t,n){e("init",{appId:t,apiKey:n})},setUserToken:function(t){e("setUserToken",t)},clickedObjectIDsAfterSearch:function(){for(var e=arguments.length,t=new Array(e),n=0;n0&&i("clickedObjectIDsAfterSearch",B(t),t[0].items)},clickedObjectIDs:function(){for(var e=arguments.length,t=new Array(e),n=0;n0&&i("clickedObjectIDs",B(t),t[0].items)},clickedFilters:function(){for(var t=arguments.length,n=new Array(t),r=0;r0&&e.apply(void 0,["clickedFilters"].concat(n))},convertedObjectIDsAfterSearch:function(){for(var e=arguments.length,t=new Array(e),n=0;n0&&i("convertedObjectIDsAfterSearch",B(t),t[0].items)},convertedObjectIDs:function(){for(var e=arguments.length,t=new Array(e),n=0;n0&&i("convertedObjectIDs",B(t),t[0].items)},convertedFilters:function(){for(var t=arguments.length,n=new Array(t),r=0;r0&&e.apply(void 0,["convertedFilters"].concat(n))},viewedObjectIDs:function(){for(var e=arguments.length,t=new Array(e),n=0;n0&&t.reduce((function(e,t){var n=t.items,r=k(t,A);return[].concat(D(e),D(q(N(N({},r),{},{objectIDs:(null==n?void 0:n.map((function(e){return e.objectID})))||r.objectIDs})).map((function(e){return{items:n,payload:e}}))))}),[]).forEach((function(e){var t=e.items;return i("viewedObjectIDs",[e.payload],t)}))},viewedFilters:function(){for(var t=arguments.length,n=new Array(t),r=0;r0&&e.apply(void 0,["viewedFilters"].concat(n))}}}function F(e){var t=e.items.reduce((function(e,t){var n;return e[t.__autocomplete_indexName]=(null!==(n=e[t.__autocomplete_indexName])&&void 0!==n?n:[]).concat(t),e}),{});return Object.keys(t).map((function(e){return{index:e,items:t[e],algoliaSource:["autocomplete"]}}))}function L(e){return e.objectID&&e.__autocomplete_indexName&&e.__autocomplete_queryID}function U(e){return U="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(e){return typeof e}:function(e){return e&&"function"==typeof Symbol&&e.constructor===Symbol&&e!==Symbol.prototype?"symbol":typeof e},U(e)}function M(e){return function(e){if(Array.isArray(e))return H(e)}(e)||function(e){if("undefined"!=typeof Symbol&&null!=e[Symbol.iterator]||null!=e["@@iterator"])return Array.from(e)}(e)||function(e,t){if(!e)return;if("string"==typeof e)return H(e,t);var n=Object.prototype.toString.call(e).slice(8,-1);"Object"===n&&e.constructor&&(n=e.constructor.name);if("Map"===n||"Set"===n)return Array.from(e);if("Arguments"===n||/^(?:Ui|I)nt(?:8|16|32)(?:Clamped)?Array$/.test(n))return H(e,t)}(e)||function(){throw new TypeError("Invalid attempt to spread non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method.")}()}function H(e,t){(null==t||t>e.length)&&(t=e.length);for(var n=0,r=new Array(t);n0&&z({onItemsChange:r,items:n,insights:a,state:t}))}}),0);return{name:"aa.algoliaInsightsPlugin",subscribe:function(e){var t=e.setContext,n=e.onSelect,r=e.onActive;function l(e){t({algoliaInsightsPlugin:{__algoliaSearchParameters:W({clickAnalytics:!0},e?{userToken:e}:{}),insights:a}})}u("addAlgoliaAgent","insights-plugin"),l(),u("onUserTokenChange",l),u("getUserToken",null,(function(e,t){l(t)})),n((function(e){var t=e.item,n=e.state,r=e.event,i=e.source;L(t)&&o({state:n,event:r,insights:a,item:t,insightsEvents:[W({eventName:"Item Selected"},j({item:t,items:i.getItems().filter(L)}))]})})),r((function(e){var t=e.item,n=e.source,r=e.state,o=e.event;L(t)&&i({state:r,event:o,insights:a,item:t,insightsEvents:[W({eventName:"Item Active"},j({item:t,items:n.getItems().filter(L)}))]})}))},onStateChange:function(e){var t=e.state;c({state:t})},__autocomplete_pluginOptions:e}}function J(e,t){var n=t;return{then:function(t,r){return J(e.then(Y(t,n,e),Y(r,n,e)),n)},catch:function(t){return J(e.catch(Y(t,n,e)),n)},finally:function(t){return t&&n.onCancelList.push(t),J(e.finally(Y(t&&function(){return n.onCancelList=[],t()},n,e)),n)},cancel:function(){n.isCanceled=!0;var e=n.onCancelList;n.onCancelList=[],e.forEach((function(e){e()}))},isCanceled:function(){return!0===n.isCanceled}}}function X(e){return J(e,{isCanceled:!1,onCancelList:[]})}function Y(e,t,n){return e?function(n){return t.isCanceled?n:e(n)}:n}function Z(e,t,n,r){if(!n)return null;if(e<0&&(null===t||null!==r&&0===t))return n+e;var o=(null===t?-1:t)+e;return o<=-1||o>=n?null===r?null:0:o}function ee(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);t&&(r=r.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,r)}return n}function te(e){for(var t=1;te.length)&&(t=e.length);for(var n=0,r=new Array(t);n0},reshape:function(e){return e.sources}},e),{},{id:null!==(n=e.id)&&void 0!==n?n:d(),plugins:o,initialState:he({activeItemId:null,query:"",completion:null,collections:[],isOpen:!1,status:"idle",context:{}},e.initialState),onStateChange:function(t){var n;null===(n=e.onStateChange)||void 0===n||n.call(e,t),o.forEach((function(e){var n;return null===(n=e.onStateChange)||void 0===n?void 0:n.call(e,t)}))},onSubmit:function(t){var n;null===(n=e.onSubmit)||void 0===n||n.call(e,t),o.forEach((function(e){var n;return null===(n=e.onSubmit)||void 0===n?void 0:n.call(e,t)}))},onReset:function(t){var n;null===(n=e.onReset)||void 0===n||n.call(e,t),o.forEach((function(e){var n;return null===(n=e.onReset)||void 0===n?void 0:n.call(e,t)}))},getSources:function(n){return Promise.all([].concat(ye(o.map((function(e){return e.getSources}))),[e.getSources]).filter(Boolean).map((function(e){return function(e,t){var n=[];return Promise.resolve(e(t)).then((function(e){return Promise.all(e.filter((function(e){return Boolean(e)})).map((function(e){if(e.sourceId,n.includes(e.sourceId))throw new Error("[Autocomplete] The `sourceId` ".concat(JSON.stringify(e.sourceId)," is not unique."));n.push(e.sourceId);var t={getItemInputValue:function(e){return e.state.query},getItemUrl:function(){},onSelect:function(e){(0,e.setIsOpen)(!1)},onActive:O,onResolve:O};Object.keys(t).forEach((function(e){t[e].__default=!0}));var r=te(te({},t),e);return Promise.resolve(r)})))}))}(e,n)}))).then((function(e){return m(e)})).then((function(e){return e.map((function(e){return he(he({},e),{},{onSelect:function(n){e.onSelect(n),t.forEach((function(e){var t;return null===(t=e.onSelect)||void 0===t?void 0:t.call(e,n)}))},onActive:function(n){e.onActive(n),t.forEach((function(e){var t;return null===(t=e.onActive)||void 0===t?void 0:t.call(e,n)}))},onResolve:function(n){e.onResolve(n),t.forEach((function(e){var t;return null===(t=e.onResolve)||void 0===t?void 0:t.call(e,n)}))}})}))}))},navigator:he({navigate:function(e){var t=e.itemUrl;r.location.assign(t)},navigateNewTab:function(e){var t=e.itemUrl,n=r.open(t,"_blank","noopener");null==n||n.focus()},navigateNewWindow:function(e){var t=e.itemUrl;r.open(t,"_blank","noopener")}},e.navigator)})}function Se(e){return Se="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(e){return typeof e}:function(e){return e&&"function"==typeof Symbol&&e.constructor===Symbol&&e!==Symbol.prototype?"symbol":typeof e},Se(e)}function je(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);t&&(r=r.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,r)}return n}function Pe(e){for(var t=1;te.length)&&(t=e.length);for(var n=0,r=new Array(t);n=0||(o[n]=e[n]);return o}(e,t);if(Object.getOwnPropertySymbols){var i=Object.getOwnPropertySymbols(e);for(r=0;r=0||Object.prototype.propertyIsEnumerable.call(e,n)&&(o[n]=e[n])}return o}var He,Ve,We,Ke=null,Qe=(He=-1,Ve=-1,We=void 0,function(e){var t=++He;return Promise.resolve(e).then((function(e){return We&&t=0||(o[n]=e[n]);return o}(e,t);if(Object.getOwnPropertySymbols){var i=Object.getOwnPropertySymbols(e);for(r=0;r=0||Object.prototype.propertyIsEnumerable.call(e,n)&&(o[n]=e[n])}return o}function et(e){return et="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(e){return typeof e}:function(e){return e&&"function"==typeof Symbol&&e.constructor===Symbol&&e!==Symbol.prototype?"symbol":typeof e},et(e)}var tt=["props","refresh","store"],nt=["inputElement","formElement","panelElement"],rt=["inputElement"],ot=["inputElement","maxLength"],it=["source"],ut=["item","source"];function at(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);t&&(r=r.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,r)}return n}function lt(e){for(var t=1;t=0||(o[n]=e[n]);return o}(e,t);if(Object.getOwnPropertySymbols){var i=Object.getOwnPropertySymbols(e);for(r=0;r=0||Object.prototype.propertyIsEnumerable.call(e,n)&&(o[n]=e[n])}return o}function ft(e){var t=e.props,n=e.refresh,r=e.store,o=st(e,tt);return{getEnvironmentProps:function(e){var n=e.inputElement,o=e.formElement,i=e.panelElement;function u(e){!r.getState().isOpen&&r.pendingRequests.isEmpty()||e.target===n||!1===[o,i].some((function(t){return n=t,r=e.target,n===r||n.contains(r);var n,r}))&&(r.dispatch("blur",null),t.debug||r.pendingRequests.cancelAll())}return lt({onTouchStart:u,onMouseDown:u,onTouchMove:function(e){!1!==r.getState().isOpen&&n===t.environment.document.activeElement&&e.target!==n&&n.blur()}},st(e,nt))},getRootProps:function(e){return lt({role:"combobox","aria-expanded":r.getState().isOpen,"aria-haspopup":"listbox","aria-owns":r.getState().isOpen?r.getState().collections.map((function(e){var n=e.source;return ie(t.id,"list",n)})).join(" "):void 0,"aria-labelledby":ie(t.id,"label")},e)},getFormProps:function(e){return e.inputElement,lt({action:"",noValidate:!0,role:"search",onSubmit:function(i){var u;i.preventDefault(),t.onSubmit(lt({event:i,refresh:n,state:r.getState()},o)),r.dispatch("submit",null),null===(u=e.inputElement)||void 0===u||u.blur()},onReset:function(i){var u;i.preventDefault(),t.onReset(lt({event:i,refresh:n,state:r.getState()},o)),r.dispatch("reset",null),null===(u=e.inputElement)||void 0===u||u.focus()}},st(e,rt))},getLabelProps:function(e){return lt({htmlFor:ie(t.id,"input"),id:ie(t.id,"label")},e)},getInputProps:function(e){var i;function u(e){(t.openOnFocus||Boolean(r.getState().query))&&$e(lt({event:e,props:t,query:r.getState().completion||r.getState().query,refresh:n,store:r},o)),r.dispatch("focus",null)}var a=e||{};a.inputElement;var l=a.maxLength,c=void 0===l?512:l,s=st(a,ot),f=oe(r.getState()),p=function(e){return Boolean(e&&e.match(ue))}((null===(i=t.environment.navigator)||void 0===i?void 0:i.userAgent)||""),m=t.enterKeyHint||(null!=f&&f.itemUrl&&!p?"go":"search");return lt({"aria-autocomplete":"both","aria-activedescendant":r.getState().isOpen&&null!==r.getState().activeItemId?ie(t.id,"item-".concat(r.getState().activeItemId),null==f?void 0:f.source):void 0,"aria-controls":r.getState().isOpen?r.getState().collections.map((function(e){var n=e.source;return ie(t.id,"list",n)})).join(" "):void 0,"aria-labelledby":ie(t.id,"label"),value:r.getState().completion||r.getState().query,id:ie(t.id,"input"),autoComplete:"off",autoCorrect:"off",autoCapitalize:"off",enterKeyHint:m,spellCheck:"false",autoFocus:t.autoFocus,placeholder:t.placeholder,maxLength:c,type:"search",onChange:function(e){$e(lt({event:e,props:t,query:e.currentTarget.value.slice(0,c),refresh:n,store:r},o))},onKeyDown:function(e){!function(e){var t=e.event,n=e.props,r=e.refresh,o=e.store,i=Ze(e,Ge);if("ArrowUp"===t.key||"ArrowDown"===t.key){var u=function(){var e=oe(o.getState()),t=n.environment.document.getElementById(ie(n.id,"item-".concat(o.getState().activeItemId),null==e?void 0:e.source));t&&(t.scrollIntoViewIfNeeded?t.scrollIntoViewIfNeeded(!1):t.scrollIntoView(!1))},a=function(){var e=oe(o.getState());if(null!==o.getState().activeItemId&&e){var n=e.item,u=e.itemInputValue,a=e.itemUrl,l=e.source;l.onActive(Xe({event:t,item:n,itemInputValue:u,itemUrl:a,refresh:r,source:l,state:o.getState()},i))}};t.preventDefault(),!1===o.getState().isOpen&&(n.openOnFocus||Boolean(o.getState().query))?$e(Xe({event:t,props:n,query:o.getState().query,refresh:r,store:o},i)).then((function(){o.dispatch(t.key,{nextActiveItemId:n.defaultActiveItemId}),a(),setTimeout(u,0)})):(o.dispatch(t.key,{}),a(),u())}else if("Escape"===t.key)t.preventDefault(),o.dispatch(t.key,null),o.pendingRequests.cancelAll();else if("Tab"===t.key)o.dispatch("blur",null),o.pendingRequests.cancelAll();else if("Enter"===t.key){if(null===o.getState().activeItemId||o.getState().collections.every((function(e){return 0===e.items.length})))return void(n.debug||o.pendingRequests.cancelAll());t.preventDefault();var l=oe(o.getState()),c=l.item,s=l.itemInputValue,f=l.itemUrl,p=l.source;if(t.metaKey||t.ctrlKey)void 0!==f&&(p.onSelect(Xe({event:t,item:c,itemInputValue:s,itemUrl:f,refresh:r,source:p,state:o.getState()},i)),n.navigator.navigateNewTab({itemUrl:f,item:c,state:o.getState()}));else if(t.shiftKey)void 0!==f&&(p.onSelect(Xe({event:t,item:c,itemInputValue:s,itemUrl:f,refresh:r,source:p,state:o.getState()},i)),n.navigator.navigateNewWindow({itemUrl:f,item:c,state:o.getState()}));else if(t.altKey);else{if(void 0!==f)return p.onSelect(Xe({event:t,item:c,itemInputValue:s,itemUrl:f,refresh:r,source:p,state:o.getState()},i)),void n.navigator.navigate({itemUrl:f,item:c,state:o.getState()});$e(Xe({event:t,nextState:{isOpen:!1},props:n,query:s,refresh:r,store:o},i)).then((function(){p.onSelect(Xe({event:t,item:c,itemInputValue:s,itemUrl:f,refresh:r,source:p,state:o.getState()},i))}))}}}(lt({event:e,props:t,refresh:n,store:r},o))},onFocus:u,onBlur:O,onClick:function(n){e.inputElement!==t.environment.document.activeElement||r.getState().isOpen||u(n)}},s)},getPanelProps:function(e){return lt({onMouseDown:function(e){e.preventDefault()},onMouseLeave:function(){r.dispatch("mouseleave",null)}},e)},getListProps:function(e){var n=e||{},r=n.source,o=st(n,it);return lt({role:"listbox","aria-labelledby":ie(t.id,"label"),id:ie(t.id,"list",r)},o)},getItemProps:function(e){var i=e.item,u=e.source,a=st(e,ut);return lt({id:ie(t.id,"item-".concat(i.__autocomplete_id),u),role:"option","aria-selected":r.getState().activeItemId===i.__autocomplete_id,onMouseMove:function(e){if(i.__autocomplete_id!==r.getState().activeItemId){r.dispatch("mousemove",i.__autocomplete_id);var t=oe(r.getState());if(null!==r.getState().activeItemId&&t){var u=t.item,a=t.itemInputValue,l=t.itemUrl,c=t.source;c.onActive(lt({event:e,item:u,itemInputValue:a,itemUrl:l,refresh:n,source:c,state:r.getState()},o))}}},onMouseDown:function(e){e.preventDefault()},onClick:function(e){var a=u.getItemInputValue({item:i,state:r.getState()}),l=u.getItemUrl({item:i,state:r.getState()});(l?Promise.resolve():$e(lt({event:e,nextState:{isOpen:!1},props:t,query:a,refresh:n,store:r},o))).then((function(){u.onSelect(lt({event:e,item:i,itemInputValue:a,itemUrl:l,refresh:n,source:u,state:r.getState()},o))}))}},a)}}}function pt(e){return pt="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(e){return typeof e}:function(e){return e&&"function"==typeof Symbol&&e.constructor===Symbol&&e!==Symbol.prototype?"symbol":typeof e},pt(e)}function mt(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);t&&(r=r.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,r)}return n}function vt(e){for(var t=1;t=5&&((o||!e&&5===r)&&(u.push(r,0,o,n),r=6),e&&(u.push(r,e,0,n),r=6)),o=""},l=0;l"===t?(r=1,o=""):o=t+o[0]:i?t===i?i="":o+=t:'"'===t||"'"===t?i=t:">"===t?(a(),r=1):r&&("="===t?(r=5,n=o,o=""):"/"===t&&(r<5||">"===e[l][c+1])?(a(),3===r&&(u=u[0]),r=u,(u=u[0]).push(2,0,r),r=0):" "===t||"\t"===t||"\n"===t||"\r"===t?(a(),r=2):o+=t),3===r&&"!--"===o&&(r=4,u=u[0])}return a(),u}(e)),t),arguments,[])).length>1?t:t[0]}var kt=function(e){var t=e.environment,n=t.document.createElementNS("http://www.w3.org/2000/svg","svg");n.setAttribute("class","aa-ClearIcon"),n.setAttribute("viewBox","0 0 24 24"),n.setAttribute("width","18"),n.setAttribute("height","18"),n.setAttribute("fill","currentColor");var r=t.document.createElementNS("http://www.w3.org/2000/svg","path");return r.setAttribute("d","M5.293 6.707l5.293 5.293-5.293 5.293c-0.391 0.391-0.391 1.024 0 1.414s1.024 0.391 1.414 0l5.293-5.293 5.293 5.293c0.391 0.391 1.024 0.391 1.414 0s0.391-1.024 0-1.414l-5.293-5.293 5.293-5.293c0.391-0.391 0.391-1.024 0-1.414s-1.024-0.391-1.414 0l-5.293 5.293-5.293-5.293c-0.391-0.391-1.024-0.391-1.414 0s-0.391 1.024 0 1.414z"),n.appendChild(r),n};function xt(e,t){if("string"==typeof t){var n=e.document.querySelector(t);return"The element ".concat(JSON.stringify(t)," is not in the document."),n}return t}function Nt(){for(var e=arguments.length,t=new Array(e),n=0;n2&&(u.children=arguments.length>3?Jt.call(arguments,2):n),"function"==typeof e&&null!=e.defaultProps)for(i in e.defaultProps)void 0===u[i]&&(u[i]=e.defaultProps[i]);return sn(e,u,r,o,null)}function sn(e,t,n,r,o){var i={type:e,props:t,key:n,ref:r,__k:null,__:null,__b:0,__e:null,__d:void 0,__c:null,__h:null,constructor:void 0,__v:null==o?++Yt:o};return null==o&&null!=Xt.vnode&&Xt.vnode(i),i}function fn(e){return e.children}function pn(e,t){this.props=e,this.context=t}function mn(e,t){if(null==t)return e.__?mn(e.__,e.__.__k.indexOf(e)+1):null;for(var n;tt&&Zt.sort(nn));yn.__r=0}function bn(e,t,n,r,o,i,u,a,l,c){var s,f,p,m,v,d,y,b=r&&r.__k||on,g=b.length;for(n.__k=[],s=0;s0?sn(m.type,m.props,m.key,m.ref?m.ref:null,m.__v):m)){if(m.__=n,m.__b=n.__b+1,null===(p=b[s])||p&&m.key==p.key&&m.type===p.type)b[s]=void 0;else for(f=0;f=0;t--)if((n=e.__k[t])&&(r=On(n)))return r;return null}function _n(e,t,n){"-"===t[0]?e.setProperty(t,null==n?"":n):e[t]=null==n?"":"number"!=typeof n||un.test(t)?n:n+"px"}function Sn(e,t,n,r,o){var i;e:if("style"===t)if("string"==typeof n)e.style.cssText=n;else{if("string"==typeof r&&(e.style.cssText=r=""),r)for(t in r)n&&t in n||_n(e.style,t,"");if(n)for(t in n)r&&n[t]===r[t]||_n(e.style,t,n[t])}else if("o"===t[0]&&"n"===t[1])i=t!==(t=t.replace(/Capture$/,"")),t=t.toLowerCase()in e?t.toLowerCase().slice(2):t.slice(2),e.l||(e.l={}),e.l[t+i]=n,n?r||e.addEventListener(t,i?Pn:jn,i):e.removeEventListener(t,i?Pn:jn,i);else if("dangerouslySetInnerHTML"!==t){if(o)t=t.replace(/xlink(H|:h)/,"h").replace(/sName$/,"s");else if("width"!==t&&"height"!==t&&"href"!==t&&"list"!==t&&"form"!==t&&"tabIndex"!==t&&"download"!==t&&t in e)try{e[t]=null==n?"":n;break e}catch(e){}"function"==typeof n||(null==n||!1===n&&"-"!==t[4]?e.removeAttribute(t):e.setAttribute(t,n))}}function jn(e){return this.l[e.type+!1](Xt.event?Xt.event(e):e)}function Pn(e){return this.l[e.type+!0](Xt.event?Xt.event(e):e)}function wn(e,t,n,r,o,i,u,a,l){var c,s,f,p,m,v,d,y,b,g,h,O,_,S,j,P=t.type;if(void 0!==t.constructor)return null;null!=n.__h&&(l=n.__h,a=t.__e=n.__e,t.__h=null,i=[a]),(c=Xt.__b)&&c(t);try{e:if("function"==typeof P){if(y=t.props,b=(c=P.contextType)&&r[c.__c],g=c?b?b.props.value:c.__:r,n.__c?d=(s=t.__c=n.__c).__=s.__E:("prototype"in P&&P.prototype.render?t.__c=s=new P(y,g):(t.__c=s=new pn(y,g),s.constructor=P,s.render=Cn),b&&b.sub(s),s.props=y,s.state||(s.state={}),s.context=g,s.__n=r,f=s.__d=!0,s.__h=[],s._sb=[]),null==s.__s&&(s.__s=s.state),null!=P.getDerivedStateFromProps&&(s.__s==s.state&&(s.__s=an({},s.__s)),an(s.__s,P.getDerivedStateFromProps(y,s.__s))),p=s.props,m=s.state,s.__v=t,f)null==P.getDerivedStateFromProps&&null!=s.componentWillMount&&s.componentWillMount(),null!=s.componentDidMount&&s.__h.push(s.componentDidMount);else{if(null==P.getDerivedStateFromProps&&y!==p&&null!=s.componentWillReceiveProps&&s.componentWillReceiveProps(y,g),!s.__e&&null!=s.shouldComponentUpdate&&!1===s.shouldComponentUpdate(y,s.__s,g)||t.__v===n.__v){for(t.__v!==n.__v&&(s.props=y,s.state=s.__s,s.__d=!1),s.__e=!1,t.__e=n.__e,t.__k=n.__k,t.__k.forEach((function(e){e&&(e.__=t)})),h=0;h0&&void 0!==arguments[0]?arguments[0]:[];return{get:function(){return e},add:function(t){var n=e[e.length-1];(null==n?void 0:n.isHighlighted)===t.isHighlighted?e[e.length-1]={value:n.value+t.value,isHighlighted:n.isHighlighted}:e.push(t)}}}(n?[{value:n,isHighlighted:!1}]:[]);return t.forEach((function(e){var t=e.split(xn);r.add({value:t[0],isHighlighted:!0}),""!==t[1]&&r.add({value:t[1],isHighlighted:!1})})),r.get()}function Tn(e){return function(e){if(Array.isArray(e))return qn(e)}(e)||function(e){if("undefined"!=typeof Symbol&&null!=e[Symbol.iterator]||null!=e["@@iterator"])return Array.from(e)}(e)||function(e,t){if(!e)return;if("string"==typeof e)return qn(e,t);var n=Object.prototype.toString.call(e).slice(8,-1);"Object"===n&&e.constructor&&(n=e.constructor.name);if("Map"===n||"Set"===n)return Array.from(e);if("Arguments"===n||/^(?:Ui|I)nt(?:8|16|32)(?:Clamped)?Array$/.test(n))return qn(e,t)}(e)||function(){throw new TypeError("Invalid attempt to spread non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method.")}()}function qn(e,t){(null==t||t>e.length)&&(t=e.length);for(var n=0,r=new Array(t);n",""":'"',"'":"'"},Fn=new RegExp(/\w/i),Ln=/&(amp|quot|lt|gt|#39);/g,Un=RegExp(Ln.source);function Mn(e,t){var n,r,o,i=e[t],u=(null===(n=e[t+1])||void 0===n?void 0:n.isHighlighted)||!0,a=(null===(r=e[t-1])||void 0===r?void 0:r.isHighlighted)||!0;return Fn.test((o=i.value)&&Un.test(o)?o.replace(Ln,(function(e){return Rn[e]})):o)||a!==u?i.isHighlighted:a}function Hn(e){return Hn="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(e){return typeof e}:function(e){return e&&"function"==typeof Symbol&&e.constructor===Symbol&&e!==Symbol.prototype?"symbol":typeof e},Hn(e)}function Vn(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);t&&(r=r.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,r)}return n}function Wn(e){for(var t=1;te.length)&&(t=e.length);for(var n=0,r=new Array(t);n=0||(o[n]=e[n]);return o}(e,t);if(Object.getOwnPropertySymbols){var i=Object.getOwnPropertySymbols(e);for(r=0;r=0||Object.prototype.propertyIsEnumerable.call(e,n)&&(o[n]=e[n])}return o}function ur(e){return function(e){if(Array.isArray(e))return ar(e)}(e)||function(e){if("undefined"!=typeof Symbol&&null!=e[Symbol.iterator]||null!=e["@@iterator"])return Array.from(e)}(e)||function(e,t){if(!e)return;if("string"==typeof e)return ar(e,t);var n=Object.prototype.toString.call(e).slice(8,-1);"Object"===n&&e.constructor&&(n=e.constructor.name);if("Map"===n||"Set"===n)return Array.from(e);if("Arguments"===n||/^(?:Ui|I)nt(?:8|16|32)(?:Clamped)?Array$/.test(n))return ar(e,t)}(e)||function(){throw new TypeError("Invalid attempt to spread non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method.")}()}function ar(e,t){(null==t||t>e.length)&&(t=e.length);for(var n=0,r=new Array(t);n0;if(!O.value.core.openOnFocus&&!t.query)return n;var r=Boolean(y.current||O.value.renderer.renderNoResults);return!n&&r||n},__autocomplete_metadata:{userAgents:br,options:e}}))})),j=f(n({collections:[],completion:null,context:{},isOpen:!1,query:"",activeItemId:null,status:"idle"},O.value.core.initialState)),P={getEnvironmentProps:O.value.renderer.getEnvironmentProps,getFormProps:O.value.renderer.getFormProps,getInputProps:O.value.renderer.getInputProps,getItemProps:O.value.renderer.getItemProps,getLabelProps:O.value.renderer.getLabelProps,getListProps:O.value.renderer.getListProps,getPanelProps:O.value.renderer.getPanelProps,getRootProps:O.value.renderer.getRootProps},w={setActiveItemId:S.value.setActiveItemId,setQuery:S.value.setQuery,setCollections:S.value.setCollections,setIsOpen:S.value.setIsOpen,setStatus:S.value.setStatus,setContext:S.value.setContext,refresh:S.value.refresh,navigator:S.value.navigator},I=m((function(){return Ct.bind(O.value.renderer.renderer.createElement)})),A=m((function(){return Gt({autocomplete:S.value,autocompleteScopeApi:w,classNames:O.value.renderer.classNames,environment:O.value.core.environment,isDetached:_.value,placeholder:O.value.core.placeholder,propGetters:P,setIsModalOpen:k,state:j.current,translations:O.value.renderer.translations})}));function E(){Ht(A.value.panel,{style:_.value?{}:yr({panelPlacement:O.value.renderer.panelPlacement,container:A.value.root,form:A.value.form,environment:O.value.core.environment})})}function D(e){j.current=e;var t={autocomplete:S.value,autocompleteScopeApi:w,classNames:O.value.renderer.classNames,components:O.value.renderer.components,container:O.value.renderer.container,html:I.value,dom:A.value,panelContainer:_.value?A.value.detachedContainer:O.value.renderer.panelContainer,propGetters:P,state:j.current,renderer:O.value.renderer.renderer},r=!b(e)&&!y.current&&O.value.renderer.renderNoResults||O.value.renderer.render;!function(e){var t=e.autocomplete,r=e.autocompleteScopeApi,o=e.dom,i=e.propGetters,u=e.state;Vt(o.root,i.getRootProps(n({state:u,props:t.getRootProps({})},r))),Vt(o.input,i.getInputProps(n({state:u,props:t.getInputProps({inputElement:o.input}),inputElement:o.input},r))),Ht(o.label,{hidden:"stalled"===u.status}),Ht(o.loadingIndicator,{hidden:"stalled"!==u.status}),Ht(o.clearButton,{hidden:!u.query}),Ht(o.detachedSearchButtonQuery,{textContent:u.query}),Ht(o.detachedSearchButtonPlaceholder,{hidden:Boolean(u.query)})}(t),function(e,t){var r=t.autocomplete,o=t.autocompleteScopeApi,u=t.classNames,a=t.html,l=t.dom,c=t.panelContainer,s=t.propGetters,f=t.state,p=t.components,m=t.renderer;if(f.isOpen){c.contains(l.panel)||"loading"===f.status||c.appendChild(l.panel),l.panel.classList.toggle("aa-Panel--stalled","stalled"===f.status);var v=f.collections.filter((function(e){var t=e.source,n=e.items;return t.templates.noResults||n.length>0})).map((function(e,t){var l=e.source,c=e.items;return m.createElement("section",{key:t,className:u.source,"data-autocomplete-source-id":l.sourceId},l.templates.header&&m.createElement("div",{className:u.sourceHeader},l.templates.header({components:p,createElement:m.createElement,Fragment:m.Fragment,items:c,source:l,state:f,html:a})),l.templates.noResults&&0===c.length?m.createElement("div",{className:u.sourceNoResults},l.templates.noResults({components:p,createElement:m.createElement,Fragment:m.Fragment,source:l,state:f,html:a})):m.createElement("ul",i({className:u.list},s.getListProps(n({state:f,props:r.getListProps({source:l})},o))),c.map((function(e){var t=r.getItemProps({item:e,source:l});return m.createElement("li",i({key:t.id,className:u.item},s.getItemProps(n({state:f,props:t},o))),l.templates.item({components:p,createElement:m.createElement,Fragment:m.Fragment,item:e,state:f,html:a}))}))),l.templates.footer&&m.createElement("div",{className:u.sourceFooter},l.templates.footer({components:p,createElement:m.createElement,Fragment:m.Fragment,items:c,source:l,state:f,html:a})))})),d=m.createElement(m.Fragment,null,m.createElement("div",{className:u.panelLayout},v),m.createElement("div",{className:"aa-GradientBottom"})),y=v.reduce((function(e,t){return e[t.props["data-autocomplete-source-id"]]=t,e}),{});e(n(n({children:d,state:f,sections:v,elements:y},m),{},{components:p,html:a},o),l.panel)}else c.contains(l.panel)&&c.removeChild(l.panel)}(r,t)}function C(){var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{};l();var t=O.value.renderer,n=t.components,r=u(t,gr);g.current=qt(r,O.value.core,{components:Bt(n,(function(e){return!e.value.hasOwnProperty("__autocomplete_componentName")})),initialState:j.current},e),v(),c(),S.value.refresh().then((function(){D(j.current)}))}function k(e){requestAnimationFrame((function(){var t=O.value.core.environment.document.body.contains(A.value.detachedOverlay);e!==t&&(e?(O.value.core.environment.document.body.appendChild(A.value.detachedOverlay),O.value.core.environment.document.body.classList.add("aa-Detached"),A.value.input.focus()):(O.value.core.environment.document.body.removeChild(A.value.detachedOverlay),O.value.core.environment.document.body.classList.remove("aa-Detached")))}))}return a((function(){var e=S.value.getEnvironmentProps({formElement:A.value.form,panelElement:A.value.panel,inputElement:A.value.input});return Ht(O.value.core.environment,e),function(){Ht(O.value.core.environment,Object.keys(e).reduce((function(e,t){return n(n({},e),{},o({},t,void 0))}),{}))}})),a((function(){var e=_.value?O.value.core.environment.document.body:O.value.renderer.panelContainer,t=_.value?A.value.detachedOverlay:A.value.panel;return _.value&&j.current.isOpen&&k(!0),D(j.current),function(){e.contains(t)&&e.removeChild(t)}})),a((function(){var e=O.value.renderer.container;return e.appendChild(A.value.root),function(){e.removeChild(A.value.root)}})),a((function(){var e=p((function(e){D(e.state)}),0);return h.current=function(t){var n=t.state,r=t.prevState;(_.value&&r.isOpen!==n.isOpen&&k(n.isOpen),_.value||!n.isOpen||r.isOpen||E(),n.query!==r.query)&&O.value.core.environment.document.querySelectorAll(".aa-Panel--scrollable").forEach((function(e){0!==e.scrollTop&&(e.scrollTop=0)}));e({state:n})},function(){h.current=void 0}})),a((function(){var e=p((function(){var e=_.value;_.value=O.value.core.environment.matchMedia(O.value.renderer.detachedMediaQuery).matches,e!==_.value?C({}):requestAnimationFrame(E)}),20);return O.value.core.environment.addEventListener("resize",e),function(){O.value.core.environment.removeEventListener("resize",e)}})),a((function(){if(!_.value)return function(){};function e(e){A.value.detachedContainer.classList.toggle("aa-DetachedContainer--modal",e)}function t(t){e(t.matches)}var n=O.value.core.environment.matchMedia(getComputedStyle(O.value.core.environment.document.documentElement).getPropertyValue("--aa-detached-modal-media-query"));e(n.matches);var r=Boolean(n.addEventListener);return r?n.addEventListener("change",t):n.addListener(t),function(){r?n.removeEventListener("change",t):n.removeListener(t)}})),a((function(){return requestAnimationFrame(E),function(){}})),n(n({},w),{},{update:C,destroy:function(){l()}})},e.getAlgoliaFacets=function(e){var t=hr({transformResponse:function(e){return e.facetHits}}),r=e.queries.map((function(e){return n(n({},e),{},{type:"facet"})}));return t(n(n({},e),{},{queries:r}))},e.getAlgoliaResults=Or,Object.defineProperty(e,"__esModule",{value:!0})})); + diff --git a/site_libs/quarto-search/fuse.min.js b/site_libs/quarto-search/fuse.min.js new file mode 100644 index 0000000..adc2835 --- /dev/null +++ b/site_libs/quarto-search/fuse.min.js @@ -0,0 +1,9 @@ +/** + * Fuse.js v6.6.2 - Lightweight fuzzy-search (http://fusejs.io) + * + * Copyright (c) 2022 Kiro Risk (http://kiro.me) + * All Rights Reserved. Apache Software License 2.0 + * + * http://www.apache.org/licenses/LICENSE-2.0 + */ +var e,t;e=this,t=function(){"use strict";function e(e,t){var n=Object.keys(e);if(Object.getOwnPropertySymbols){var r=Object.getOwnPropertySymbols(e);t&&(r=r.filter((function(t){return Object.getOwnPropertyDescriptor(e,t).enumerable}))),n.push.apply(n,r)}return n}function t(t){for(var n=1;ne.length)&&(t=e.length);for(var n=0,r=new Array(t);n0&&void 0!==arguments[0]?arguments[0]:1,t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:3,n=new Map,r=Math.pow(10,t);return{get:function(t){var i=t.match(C).length;if(n.has(i))return n.get(i);var o=1/Math.pow(i,.5*e),c=parseFloat(Math.round(o*r)/r);return n.set(i,c),c},clear:function(){n.clear()}}}var $=function(){function e(){var t=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{},n=t.getFn,i=void 0===n?I.getFn:n,o=t.fieldNormWeight,c=void 0===o?I.fieldNormWeight:o;r(this,e),this.norm=E(c,3),this.getFn=i,this.isCreated=!1,this.setIndexRecords()}return o(e,[{key:"setSources",value:function(){var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:[];this.docs=e}},{key:"setIndexRecords",value:function(){var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:[];this.records=e}},{key:"setKeys",value:function(){var e=this,t=arguments.length>0&&void 0!==arguments[0]?arguments[0]:[];this.keys=t,this._keysMap={},t.forEach((function(t,n){e._keysMap[t.id]=n}))}},{key:"create",value:function(){var e=this;!this.isCreated&&this.docs.length&&(this.isCreated=!0,g(this.docs[0])?this.docs.forEach((function(t,n){e._addString(t,n)})):this.docs.forEach((function(t,n){e._addObject(t,n)})),this.norm.clear())}},{key:"add",value:function(e){var t=this.size();g(e)?this._addString(e,t):this._addObject(e,t)}},{key:"removeAt",value:function(e){this.records.splice(e,1);for(var t=e,n=this.size();t2&&void 0!==arguments[2]?arguments[2]:{},r=n.getFn,i=void 0===r?I.getFn:r,o=n.fieldNormWeight,c=void 0===o?I.fieldNormWeight:o,a=new $({getFn:i,fieldNormWeight:c});return a.setKeys(e.map(_)),a.setSources(t),a.create(),a}function R(e){var t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{},n=t.errors,r=void 0===n?0:n,i=t.currentLocation,o=void 0===i?0:i,c=t.expectedLocation,a=void 0===c?0:c,s=t.distance,u=void 0===s?I.distance:s,h=t.ignoreLocation,l=void 0===h?I.ignoreLocation:h,f=r/e.length;if(l)return f;var d=Math.abs(a-o);return u?f+d/u:d?1:f}function N(){for(var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:[],t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:I.minMatchCharLength,n=[],r=-1,i=-1,o=0,c=e.length;o=t&&n.push([r,i]),r=-1)}return e[o-1]&&o-r>=t&&n.push([r,o-1]),n}var P=32;function W(e){for(var t={},n=0,r=e.length;n1&&void 0!==arguments[1]?arguments[1]:{},o=i.location,c=void 0===o?I.location:o,a=i.threshold,s=void 0===a?I.threshold:a,u=i.distance,h=void 0===u?I.distance:u,l=i.includeMatches,f=void 0===l?I.includeMatches:l,d=i.findAllMatches,v=void 0===d?I.findAllMatches:d,g=i.minMatchCharLength,y=void 0===g?I.minMatchCharLength:g,p=i.isCaseSensitive,m=void 0===p?I.isCaseSensitive:p,k=i.ignoreLocation,M=void 0===k?I.ignoreLocation:k;if(r(this,e),this.options={location:c,threshold:s,distance:h,includeMatches:f,findAllMatches:v,minMatchCharLength:y,isCaseSensitive:m,ignoreLocation:M},this.pattern=m?t:t.toLowerCase(),this.chunks=[],this.pattern.length){var b=function(e,t){n.chunks.push({pattern:e,alphabet:W(e),startIndex:t})},x=this.pattern.length;if(x>P){for(var w=0,L=x%P,S=x-L;w3&&void 0!==arguments[3]?arguments[3]:{},i=r.location,o=void 0===i?I.location:i,c=r.distance,a=void 0===c?I.distance:c,s=r.threshold,u=void 0===s?I.threshold:s,h=r.findAllMatches,l=void 0===h?I.findAllMatches:h,f=r.minMatchCharLength,d=void 0===f?I.minMatchCharLength:f,v=r.includeMatches,g=void 0===v?I.includeMatches:v,y=r.ignoreLocation,p=void 0===y?I.ignoreLocation:y;if(t.length>P)throw new Error(w(P));for(var m,k=t.length,M=e.length,b=Math.max(0,Math.min(o,M)),x=u,L=b,S=d>1||g,_=S?Array(M):[];(m=e.indexOf(t,L))>-1;){var O=R(t,{currentLocation:m,expectedLocation:b,distance:a,ignoreLocation:p});if(x=Math.min(O,x),L=m+k,S)for(var j=0;j=z;q-=1){var B=q-1,J=n[e.charAt(B)];if(S&&(_[B]=+!!J),K[q]=(K[q+1]<<1|1)&J,F&&(K[q]|=(A[q+1]|A[q])<<1|1|A[q+1]),K[q]&$&&(C=R(t,{errors:F,currentLocation:B,expectedLocation:b,distance:a,ignoreLocation:p}))<=x){if(x=C,(L=B)<=b)break;z=Math.max(1,2*b-L)}}if(R(t,{errors:F+1,currentLocation:b,expectedLocation:b,distance:a,ignoreLocation:p})>x)break;A=K}var U={isMatch:L>=0,score:Math.max(.001,C)};if(S){var V=N(_,d);V.length?g&&(U.indices=V):U.isMatch=!1}return U}(e,n,i,{location:c+o,distance:a,threshold:s,findAllMatches:u,minMatchCharLength:h,includeMatches:r,ignoreLocation:l}),p=y.isMatch,m=y.score,k=y.indices;p&&(g=!0),v+=m,p&&k&&(d=[].concat(f(d),f(k)))}));var y={isMatch:g,score:g?v/this.chunks.length:1};return g&&r&&(y.indices=d),y}}]),e}(),z=function(){function e(t){r(this,e),this.pattern=t}return o(e,[{key:"search",value:function(){}}],[{key:"isMultiMatch",value:function(e){return D(e,this.multiRegex)}},{key:"isSingleMatch",value:function(e){return D(e,this.singleRegex)}}]),e}();function D(e,t){var n=e.match(t);return n?n[1]:null}var K=function(e){a(n,e);var t=l(n);function n(e){return r(this,n),t.call(this,e)}return o(n,[{key:"search",value:function(e){var t=e===this.pattern;return{isMatch:t,score:t?0:1,indices:[0,this.pattern.length-1]}}}],[{key:"type",get:function(){return"exact"}},{key:"multiRegex",get:function(){return/^="(.*)"$/}},{key:"singleRegex",get:function(){return/^=(.*)$/}}]),n}(z),q=function(e){a(n,e);var t=l(n);function n(e){return r(this,n),t.call(this,e)}return o(n,[{key:"search",value:function(e){var t=-1===e.indexOf(this.pattern);return{isMatch:t,score:t?0:1,indices:[0,e.length-1]}}}],[{key:"type",get:function(){return"inverse-exact"}},{key:"multiRegex",get:function(){return/^!"(.*)"$/}},{key:"singleRegex",get:function(){return/^!(.*)$/}}]),n}(z),B=function(e){a(n,e);var t=l(n);function n(e){return r(this,n),t.call(this,e)}return o(n,[{key:"search",value:function(e){var t=e.startsWith(this.pattern);return{isMatch:t,score:t?0:1,indices:[0,this.pattern.length-1]}}}],[{key:"type",get:function(){return"prefix-exact"}},{key:"multiRegex",get:function(){return/^\^"(.*)"$/}},{key:"singleRegex",get:function(){return/^\^(.*)$/}}]),n}(z),J=function(e){a(n,e);var t=l(n);function n(e){return r(this,n),t.call(this,e)}return o(n,[{key:"search",value:function(e){var t=!e.startsWith(this.pattern);return{isMatch:t,score:t?0:1,indices:[0,e.length-1]}}}],[{key:"type",get:function(){return"inverse-prefix-exact"}},{key:"multiRegex",get:function(){return/^!\^"(.*)"$/}},{key:"singleRegex",get:function(){return/^!\^(.*)$/}}]),n}(z),U=function(e){a(n,e);var t=l(n);function n(e){return r(this,n),t.call(this,e)}return o(n,[{key:"search",value:function(e){var t=e.endsWith(this.pattern);return{isMatch:t,score:t?0:1,indices:[e.length-this.pattern.length,e.length-1]}}}],[{key:"type",get:function(){return"suffix-exact"}},{key:"multiRegex",get:function(){return/^"(.*)"\$$/}},{key:"singleRegex",get:function(){return/^(.*)\$$/}}]),n}(z),V=function(e){a(n,e);var t=l(n);function n(e){return r(this,n),t.call(this,e)}return o(n,[{key:"search",value:function(e){var t=!e.endsWith(this.pattern);return{isMatch:t,score:t?0:1,indices:[0,e.length-1]}}}],[{key:"type",get:function(){return"inverse-suffix-exact"}},{key:"multiRegex",get:function(){return/^!"(.*)"\$$/}},{key:"singleRegex",get:function(){return/^!(.*)\$$/}}]),n}(z),G=function(e){a(n,e);var t=l(n);function n(e){var i,o=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{},c=o.location,a=void 0===c?I.location:c,s=o.threshold,u=void 0===s?I.threshold:s,h=o.distance,l=void 0===h?I.distance:h,f=o.includeMatches,d=void 0===f?I.includeMatches:f,v=o.findAllMatches,g=void 0===v?I.findAllMatches:v,y=o.minMatchCharLength,p=void 0===y?I.minMatchCharLength:y,m=o.isCaseSensitive,k=void 0===m?I.isCaseSensitive:m,M=o.ignoreLocation,b=void 0===M?I.ignoreLocation:M;return r(this,n),(i=t.call(this,e))._bitapSearch=new T(e,{location:a,threshold:u,distance:l,includeMatches:d,findAllMatches:g,minMatchCharLength:p,isCaseSensitive:k,ignoreLocation:b}),i}return o(n,[{key:"search",value:function(e){return this._bitapSearch.searchIn(e)}}],[{key:"type",get:function(){return"fuzzy"}},{key:"multiRegex",get:function(){return/^"(.*)"$/}},{key:"singleRegex",get:function(){return/^(.*)$/}}]),n}(z),H=function(e){a(n,e);var t=l(n);function n(e){return r(this,n),t.call(this,e)}return o(n,[{key:"search",value:function(e){for(var t,n=0,r=[],i=this.pattern.length;(t=e.indexOf(this.pattern,n))>-1;)n=t+i,r.push([t,n-1]);var o=!!r.length;return{isMatch:o,score:o?0:1,indices:r}}}],[{key:"type",get:function(){return"include"}},{key:"multiRegex",get:function(){return/^'"(.*)"$/}},{key:"singleRegex",get:function(){return/^'(.*)$/}}]),n}(z),Q=[K,H,B,J,V,U,q,G],X=Q.length,Y=/ +(?=(?:[^\"]*\"[^\"]*\")*[^\"]*$)/;function Z(e){var t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{};return e.split("|").map((function(e){for(var n=e.trim().split(Y).filter((function(e){return e&&!!e.trim()})),r=[],i=0,o=n.length;i1&&void 0!==arguments[1]?arguments[1]:{},i=n.isCaseSensitive,o=void 0===i?I.isCaseSensitive:i,c=n.includeMatches,a=void 0===c?I.includeMatches:c,s=n.minMatchCharLength,u=void 0===s?I.minMatchCharLength:s,h=n.ignoreLocation,l=void 0===h?I.ignoreLocation:h,f=n.findAllMatches,d=void 0===f?I.findAllMatches:f,v=n.location,g=void 0===v?I.location:v,y=n.threshold,p=void 0===y?I.threshold:y,m=n.distance,k=void 0===m?I.distance:m;r(this,e),this.query=null,this.options={isCaseSensitive:o,includeMatches:a,minMatchCharLength:u,findAllMatches:d,ignoreLocation:l,location:g,threshold:p,distance:k},this.pattern=o?t:t.toLowerCase(),this.query=Z(this.pattern,this.options)}return o(e,[{key:"searchIn",value:function(e){var t=this.query;if(!t)return{isMatch:!1,score:1};var n=this.options,r=n.includeMatches;e=n.isCaseSensitive?e:e.toLowerCase();for(var i=0,o=[],c=0,a=0,s=t.length;a-1&&(n.refIndex=e.idx),t.matches.push(n)}}))}function ve(e,t){t.score=e.score}function ge(e,t){var n=arguments.length>2&&void 0!==arguments[2]?arguments[2]:{},r=n.includeMatches,i=void 0===r?I.includeMatches:r,o=n.includeScore,c=void 0===o?I.includeScore:o,a=[];return i&&a.push(de),c&&a.push(ve),e.map((function(e){var n=e.idx,r={item:t[n],refIndex:n};return a.length&&a.forEach((function(t){t(e,r)})),r}))}var ye=function(){function e(n){var i=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{},o=arguments.length>2?arguments[2]:void 0;r(this,e),this.options=t(t({},I),i),this.options.useExtendedSearch,this._keyStore=new S(this.options.keys),this.setCollection(n,o)}return o(e,[{key:"setCollection",value:function(e,t){if(this._docs=e,t&&!(t instanceof $))throw new Error("Incorrect 'index' type");this._myIndex=t||F(this.options.keys,this._docs,{getFn:this.options.getFn,fieldNormWeight:this.options.fieldNormWeight})}},{key:"add",value:function(e){k(e)&&(this._docs.push(e),this._myIndex.add(e))}},{key:"remove",value:function(){for(var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:function(){return!1},t=[],n=0,r=this._docs.length;n1&&void 0!==arguments[1]?arguments[1]:{},n=t.limit,r=void 0===n?-1:n,i=this.options,o=i.includeMatches,c=i.includeScore,a=i.shouldSort,s=i.sortFn,u=i.ignoreFieldNorm,h=g(e)?g(this._docs[0])?this._searchStringList(e):this._searchObjectList(e):this._searchLogical(e);return fe(h,{ignoreFieldNorm:u}),a&&h.sort(s),y(r)&&r>-1&&(h=h.slice(0,r)),ge(h,this._docs,{includeMatches:o,includeScore:c})}},{key:"_searchStringList",value:function(e){var t=re(e,this.options),n=this._myIndex.records,r=[];return n.forEach((function(e){var n=e.v,i=e.i,o=e.n;if(k(n)){var c=t.searchIn(n),a=c.isMatch,s=c.score,u=c.indices;a&&r.push({item:n,idx:i,matches:[{score:s,value:n,norm:o,indices:u}]})}})),r}},{key:"_searchLogical",value:function(e){var t=this,n=function(e,t){var n=(arguments.length>2&&void 0!==arguments[2]?arguments[2]:{}).auto,r=void 0===n||n,i=function e(n){var i=Object.keys(n),o=ue(n);if(!o&&i.length>1&&!se(n))return e(le(n));if(he(n)){var c=o?n[ce]:i[0],a=o?n[ae]:n[c];if(!g(a))throw new Error(x(c));var s={keyId:j(c),pattern:a};return r&&(s.searcher=re(a,t)),s}var u={children:[],operator:i[0]};return i.forEach((function(t){var r=n[t];v(r)&&r.forEach((function(t){u.children.push(e(t))}))})),u};return se(e)||(e=le(e)),i(e)}(e,this.options),r=function e(n,r,i){if(!n.children){var o=n.keyId,c=n.searcher,a=t._findMatches({key:t._keyStore.get(o),value:t._myIndex.getValueForItemAtKeyId(r,o),searcher:c});return a&&a.length?[{idx:i,item:r,matches:a}]:[]}for(var s=[],u=0,h=n.children.length;u1&&void 0!==arguments[1]?arguments[1]:{},n=t.getFn,r=void 0===n?I.getFn:n,i=t.fieldNormWeight,o=void 0===i?I.fieldNormWeight:i,c=e.keys,a=e.records,s=new $({getFn:r,fieldNormWeight:o});return s.setKeys(c),s.setIndexRecords(a),s},ye.config=I,function(){ne.push.apply(ne,arguments)}(te),ye},"object"==typeof exports&&"undefined"!=typeof module?module.exports=t():"function"==typeof define&&define.amd?define(t):(e="undefined"!=typeof globalThis?globalThis:e||self).Fuse=t(); \ No newline at end of file diff --git a/site_libs/quarto-search/quarto-search.js b/site_libs/quarto-search/quarto-search.js new file mode 100644 index 0000000..d788a95 --- /dev/null +++ b/site_libs/quarto-search/quarto-search.js @@ -0,0 +1,1290 @@ +const kQueryArg = "q"; +const kResultsArg = "show-results"; + +// If items don't provide a URL, then both the navigator and the onSelect +// function aren't called (and therefore, the default implementation is used) +// +// We're using this sentinel URL to signal to those handlers that this +// item is a more item (along with the type) and can be handled appropriately +const kItemTypeMoreHref = "0767FDFD-0422-4E5A-BC8A-3BE11E5BBA05"; + +window.document.addEventListener("DOMContentLoaded", function (_event) { + // Ensure that search is available on this page. If it isn't, + // should return early and not do anything + var searchEl = window.document.getElementById("quarto-search"); + if (!searchEl) return; + + const { autocomplete } = window["@algolia/autocomplete-js"]; + + let quartoSearchOptions = {}; + let language = {}; + const searchOptionEl = window.document.getElementById( + "quarto-search-options" + ); + if (searchOptionEl) { + const jsonStr = searchOptionEl.textContent; + quartoSearchOptions = JSON.parse(jsonStr); + language = quartoSearchOptions.language; + } + + // note the search mode + if (quartoSearchOptions.type === "overlay") { + searchEl.classList.add("type-overlay"); + } else { + searchEl.classList.add("type-textbox"); + } + + // Used to determine highlighting behavior for this page + // A `q` query param is expected when the user follows a search + // to this page + const currentUrl = new URL(window.location); + const query = currentUrl.searchParams.get(kQueryArg); + const showSearchResults = currentUrl.searchParams.get(kResultsArg); + const mainEl = window.document.querySelector("main"); + + // highlight matches on the page + if (query && mainEl) { + // perform any highlighting + highlight(escapeRegExp(query), mainEl); + + // fix up the URL to remove the q query param + const replacementUrl = new URL(window.location); + replacementUrl.searchParams.delete(kQueryArg); + window.history.replaceState({}, "", replacementUrl); + } + + // function to clear highlighting on the page when the search query changes + // (e.g. if the user edits the query or clears it) + let highlighting = true; + const resetHighlighting = (searchTerm) => { + if (mainEl && highlighting && query && searchTerm !== query) { + clearHighlight(query, mainEl); + highlighting = false; + } + }; + + // Clear search highlighting when the user scrolls sufficiently + const resetFn = () => { + resetHighlighting(""); + window.removeEventListener("quarto-hrChanged", resetFn); + window.removeEventListener("quarto-sectionChanged", resetFn); + }; + + // Register this event after the initial scrolling and settling of events + // on the page + window.addEventListener("quarto-hrChanged", resetFn); + window.addEventListener("quarto-sectionChanged", resetFn); + + // Responsively switch to overlay mode if the search is present on the navbar + // Note that switching the sidebar to overlay mode requires more coordinate (not just + // the media query since we generate different HTML for sidebar overlays than we do + // for sidebar input UI) + const detachedMediaQuery = + quartoSearchOptions.type === "overlay" ? "all" : "(max-width: 991px)"; + + // If configured, include the analytics client to send insights + const plugins = configurePlugins(quartoSearchOptions); + + let lastState = null; + const { setIsOpen, setQuery, setCollections } = autocomplete({ + container: searchEl, + detachedMediaQuery: detachedMediaQuery, + defaultActiveItemId: 0, + panelContainer: "#quarto-search-results", + panelPlacement: quartoSearchOptions["panel-placement"], + debug: false, + openOnFocus: true, + plugins, + classNames: { + form: "d-flex", + }, + placeholder: language["search-text-placeholder"], + translations: { + clearButtonTitle: language["search-clear-button-title"], + detachedCancelButtonText: language["search-detached-cancel-button-title"], + submitButtonTitle: language["search-submit-button-title"], + }, + initialState: { + query, + }, + getItemUrl({ item }) { + return item.href; + }, + onStateChange({ state }) { + // If this is a file URL, note that + + // Perhaps reset highlighting + resetHighlighting(state.query); + + // If the panel just opened, ensure the panel is positioned properly + if (state.isOpen) { + if (lastState && !lastState.isOpen) { + setTimeout(() => { + positionPanel(quartoSearchOptions["panel-placement"]); + }, 150); + } + } + + // Perhaps show the copy link + showCopyLink(state.query, quartoSearchOptions); + + lastState = state; + }, + reshape({ sources, state }) { + return sources.map((source) => { + try { + const items = source.getItems(); + + // Validate the items + validateItems(items); + + // group the items by document + const groupedItems = new Map(); + items.forEach((item) => { + const hrefParts = item.href.split("#"); + const baseHref = hrefParts[0]; + const isDocumentItem = hrefParts.length === 1; + + const items = groupedItems.get(baseHref); + if (!items) { + groupedItems.set(baseHref, [item]); + } else { + // If the href for this item matches the document + // exactly, place this item first as it is the item that represents + // the document itself + if (isDocumentItem) { + items.unshift(item); + } else { + items.push(item); + } + groupedItems.set(baseHref, items); + } + }); + + const reshapedItems = []; + let count = 1; + for (const [_key, value] of groupedItems) { + const firstItem = value[0]; + reshapedItems.push({ + ...firstItem, + type: kItemTypeDoc, + }); + + const collapseMatches = quartoSearchOptions["collapse-after"]; + const collapseCount = + typeof collapseMatches === "number" ? collapseMatches : 1; + + if (value.length > 1) { + const target = `search-more-${count}`; + const isExpanded = + state.context.expanded && + state.context.expanded.includes(target); + + const remainingCount = value.length - collapseCount; + + for (let i = 1; i < value.length; i++) { + if (collapseMatches && i === collapseCount) { + reshapedItems.push({ + target, + title: isExpanded + ? language["search-hide-matches-text"] + : remainingCount === 1 + ? `${remainingCount} ${language["search-more-match-text"]}` + : `${remainingCount} ${language["search-more-matches-text"]}`, + type: kItemTypeMore, + href: kItemTypeMoreHref, + }); + } + + if (isExpanded || !collapseMatches || i < collapseCount) { + reshapedItems.push({ + ...value[i], + type: kItemTypeItem, + target, + }); + } + } + } + count += 1; + } + + return { + ...source, + getItems() { + return reshapedItems; + }, + }; + } catch (error) { + // Some form of error occurred + return { + ...source, + getItems() { + return [ + { + title: error.name || "An Error Occurred While Searching", + text: + error.message || + "An unknown error occurred while attempting to perform the requested search.", + type: kItemTypeError, + }, + ]; + }, + }; + } + }); + }, + navigator: { + navigate({ itemUrl }) { + if (itemUrl !== offsetURL(kItemTypeMoreHref)) { + window.location.assign(itemUrl); + } + }, + navigateNewTab({ itemUrl }) { + if (itemUrl !== offsetURL(kItemTypeMoreHref)) { + const windowReference = window.open(itemUrl, "_blank", "noopener"); + if (windowReference) { + windowReference.focus(); + } + } + }, + navigateNewWindow({ itemUrl }) { + if (itemUrl !== offsetURL(kItemTypeMoreHref)) { + window.open(itemUrl, "_blank", "noopener"); + } + }, + }, + getSources({ state, setContext, setActiveItemId, refresh }) { + return [ + { + sourceId: "documents", + getItemUrl({ item }) { + if (item.href) { + return offsetURL(item.href); + } else { + return undefined; + } + }, + onSelect({ + item, + state, + setContext, + setIsOpen, + setActiveItemId, + refresh, + }) { + if (item.type === kItemTypeMore) { + toggleExpanded(item, state, setContext, setActiveItemId, refresh); + + // Toggle more + setIsOpen(true); + } + }, + getItems({ query }) { + if (query === null || query === "") { + return []; + } + + const limit = quartoSearchOptions.limit; + if (quartoSearchOptions.algolia) { + return algoliaSearch(query, limit, quartoSearchOptions.algolia); + } else { + // Fuse search options + const fuseSearchOptions = { + isCaseSensitive: false, + shouldSort: true, + minMatchCharLength: 2, + limit: limit, + }; + + return readSearchData().then(function (fuse) { + return fuseSearch(query, fuse, fuseSearchOptions); + }); + } + }, + templates: { + noResults({ createElement }) { + const hasQuery = lastState.query; + + return createElement( + "div", + { + class: `quarto-search-no-results${ + hasQuery ? "" : " no-query" + }`, + }, + language["search-no-results-text"] + ); + }, + header({ items, createElement }) { + // count the documents + const count = items.filter((item) => { + return item.type === kItemTypeDoc; + }).length; + + if (count > 0) { + return createElement( + "div", + { class: "search-result-header" }, + `${count} ${language["search-matching-documents-text"]}` + ); + } else { + return createElement( + "div", + { class: "search-result-header-no-results" }, + `` + ); + } + }, + footer({ _items, createElement }) { + if ( + quartoSearchOptions.algolia && + quartoSearchOptions.algolia["show-logo"] + ) { + const libDir = quartoSearchOptions.algolia["libDir"]; + const logo = createElement("img", { + src: offsetURL( + `${libDir}/quarto-search/search-by-algolia.svg` + ), + class: "algolia-search-logo", + }); + return createElement( + "a", + { href: "http://www.algolia.com/" }, + logo + ); + } + }, + + item({ item, createElement }) { + return renderItem( + item, + createElement, + state, + setActiveItemId, + setContext, + refresh, + quartoSearchOptions + ); + }, + }, + }, + ]; + }, + }); + + window.quartoOpenSearch = () => { + setIsOpen(false); + setIsOpen(true); + focusSearchInput(); + }; + + document.addEventListener("keyup", (event) => { + const { key } = event; + const kbds = quartoSearchOptions["keyboard-shortcut"]; + const focusedEl = document.activeElement; + + const isFormElFocused = [ + "input", + "select", + "textarea", + "button", + "option", + ].find((tag) => { + return focusedEl.tagName.toLowerCase() === tag; + }); + + if ( + kbds && + kbds.includes(key) && + !isFormElFocused && + !document.activeElement.isContentEditable + ) { + event.preventDefault(); + window.quartoOpenSearch(); + } + }); + + // Remove the labeleledby attribute since it is pointing + // to a non-existent label + if (quartoSearchOptions.type === "overlay") { + const inputEl = window.document.querySelector( + "#quarto-search .aa-Autocomplete" + ); + if (inputEl) { + inputEl.removeAttribute("aria-labelledby"); + } + } + + function throttle(func, wait) { + let waiting = false; + return function () { + if (!waiting) { + func.apply(this, arguments); + waiting = true; + setTimeout(function () { + waiting = false; + }, wait); + } + }; + } + + // If the main document scrolls dismiss the search results + // (otherwise, since they're floating in the document they can scroll with the document) + window.document.body.onscroll = throttle(() => { + // Only do this if we're not detached + // Bug #7117 + // This will happen when the keyboard is shown on ios (resulting in a scroll) + // which then closed the search UI + if (!window.matchMedia(detachedMediaQuery).matches) { + setIsOpen(false); + } + }, 50); + + if (showSearchResults) { + setIsOpen(true); + focusSearchInput(); + } +}); + +function configurePlugins(quartoSearchOptions) { + const autocompletePlugins = []; + const algoliaOptions = quartoSearchOptions.algolia; + if ( + algoliaOptions && + algoliaOptions["analytics-events"] && + algoliaOptions["search-only-api-key"] && + algoliaOptions["application-id"] + ) { + const apiKey = algoliaOptions["search-only-api-key"]; + const appId = algoliaOptions["application-id"]; + + // Aloglia insights may not be loaded because they require cookie consent + // Use deferred loading so events will start being recorded when/if consent + // is granted. + const algoliaInsightsDeferredPlugin = deferredLoadPlugin(() => { + if ( + window.aa && + window["@algolia/autocomplete-plugin-algolia-insights"] + ) { + window.aa("init", { + appId, + apiKey, + useCookie: true, + }); + + const { createAlgoliaInsightsPlugin } = + window["@algolia/autocomplete-plugin-algolia-insights"]; + // Register the insights client + const algoliaInsightsPlugin = createAlgoliaInsightsPlugin({ + insightsClient: window.aa, + onItemsChange({ insights, insightsEvents }) { + const events = insightsEvents.flatMap((event) => { + // This API limits the number of items per event to 20 + const chunkSize = 20; + const itemChunks = []; + const eventItems = event.items; + for (let i = 0; i < eventItems.length; i += chunkSize) { + itemChunks.push(eventItems.slice(i, i + chunkSize)); + } + // Split the items into multiple events that can be sent + const events = itemChunks.map((items) => { + return { + ...event, + items, + }; + }); + return events; + }); + + for (const event of events) { + insights.viewedObjectIDs(event); + } + }, + }); + return algoliaInsightsPlugin; + } + }); + + // Add the plugin + autocompletePlugins.push(algoliaInsightsDeferredPlugin); + return autocompletePlugins; + } +} + +// For plugins that may not load immediately, create a wrapper +// plugin and forward events and plugin data once the plugin +// is initialized. This is useful for cases like cookie consent +// which may prevent the analytics insights event plugin from initializing +// immediately. +function deferredLoadPlugin(createPlugin) { + let plugin = undefined; + let subscribeObj = undefined; + const wrappedPlugin = () => { + if (!plugin && subscribeObj) { + plugin = createPlugin(); + if (plugin && plugin.subscribe) { + plugin.subscribe(subscribeObj); + } + } + return plugin; + }; + + return { + subscribe: (obj) => { + subscribeObj = obj; + }, + onStateChange: (obj) => { + const plugin = wrappedPlugin(); + if (plugin && plugin.onStateChange) { + plugin.onStateChange(obj); + } + }, + onSubmit: (obj) => { + const plugin = wrappedPlugin(); + if (plugin && plugin.onSubmit) { + plugin.onSubmit(obj); + } + }, + onReset: (obj) => { + const plugin = wrappedPlugin(); + if (plugin && plugin.onReset) { + plugin.onReset(obj); + } + }, + getSources: (obj) => { + const plugin = wrappedPlugin(); + if (plugin && plugin.getSources) { + return plugin.getSources(obj); + } else { + return Promise.resolve([]); + } + }, + data: (obj) => { + const plugin = wrappedPlugin(); + if (plugin && plugin.data) { + plugin.data(obj); + } + }, + }; +} + +function validateItems(items) { + // Validate the first item + if (items.length > 0) { + const item = items[0]; + const missingFields = []; + if (item.href == undefined) { + missingFields.push("href"); + } + if (!item.title == undefined) { + missingFields.push("title"); + } + if (!item.text == undefined) { + missingFields.push("text"); + } + + if (missingFields.length === 1) { + throw { + name: `Error: Search index is missing the ${missingFields[0]} field.`, + message: `The items being returned for this search do not include all the required fields. Please ensure that your index items include the ${missingFields[0]} field or use index-fields in your _quarto.yml file to specify the field names.`, + }; + } else if (missingFields.length > 1) { + const missingFieldList = missingFields + .map((field) => { + return `${field}`; + }) + .join(", "); + + throw { + name: `Error: Search index is missing the following fields: ${missingFieldList}.`, + message: `The items being returned for this search do not include all the required fields. Please ensure that your index items includes the following fields: ${missingFieldList}, or use index-fields in your _quarto.yml file to specify the field names.`, + }; + } + } +} + +let lastQuery = null; +function showCopyLink(query, options) { + const language = options.language; + lastQuery = query; + // Insert share icon + const inputSuffixEl = window.document.body.querySelector( + ".aa-Form .aa-InputWrapperSuffix" + ); + + if (inputSuffixEl) { + let copyButtonEl = window.document.body.querySelector( + ".aa-Form .aa-InputWrapperSuffix .aa-CopyButton" + ); + + if (copyButtonEl === null) { + copyButtonEl = window.document.createElement("button"); + copyButtonEl.setAttribute("class", "aa-CopyButton"); + copyButtonEl.setAttribute("type", "button"); + copyButtonEl.setAttribute("title", language["search-copy-link-title"]); + copyButtonEl.onmousedown = (e) => { + e.preventDefault(); + e.stopPropagation(); + }; + + const linkIcon = "bi-clipboard"; + const checkIcon = "bi-check2"; + + const shareIconEl = window.document.createElement("i"); + shareIconEl.setAttribute("class", `bi ${linkIcon}`); + copyButtonEl.appendChild(shareIconEl); + inputSuffixEl.prepend(copyButtonEl); + + const clipboard = new window.ClipboardJS(".aa-CopyButton", { + text: function (_trigger) { + const copyUrl = new URL(window.location); + copyUrl.searchParams.set(kQueryArg, lastQuery); + copyUrl.searchParams.set(kResultsArg, "1"); + return copyUrl.toString(); + }, + }); + clipboard.on("success", function (e) { + // Focus the input + + // button target + const button = e.trigger; + const icon = button.querySelector("i.bi"); + + // flash "checked" + icon.classList.add(checkIcon); + icon.classList.remove(linkIcon); + setTimeout(function () { + icon.classList.remove(checkIcon); + icon.classList.add(linkIcon); + }, 1000); + }); + } + + // If there is a query, show the link icon + if (copyButtonEl) { + if (lastQuery && options["copy-button"]) { + copyButtonEl.style.display = "flex"; + } else { + copyButtonEl.style.display = "none"; + } + } + } +} + +/* Search Index Handling */ +// create the index +var fuseIndex = undefined; +var shownWarning = false; + +// fuse index options +const kFuseIndexOptions = { + keys: [ + { name: "title", weight: 20 }, + { name: "section", weight: 20 }, + { name: "text", weight: 10 }, + ], + ignoreLocation: true, + threshold: 0.1, +}; + +async function readSearchData() { + // Initialize the search index on demand + if (fuseIndex === undefined) { + if (window.location.protocol === "file:" && !shownWarning) { + window.alert( + "Search requires JavaScript features disabled when running in file://... URLs. In order to use search, please run this document in a web server." + ); + shownWarning = true; + return; + } + const fuse = new window.Fuse([], kFuseIndexOptions); + + // fetch the main search.json + const response = await fetch(offsetURL("search.json")); + if (response.status == 200) { + return response.json().then(function (searchDocs) { + searchDocs.forEach(function (searchDoc) { + fuse.add(searchDoc); + }); + fuseIndex = fuse; + return fuseIndex; + }); + } else { + return Promise.reject( + new Error( + "Unexpected status from search index request: " + response.status + ) + ); + } + } + + return fuseIndex; +} + +function inputElement() { + return window.document.body.querySelector(".aa-Form .aa-Input"); +} + +function focusSearchInput() { + setTimeout(() => { + const inputEl = inputElement(); + if (inputEl) { + inputEl.focus(); + } + }, 50); +} + +/* Panels */ +const kItemTypeDoc = "document"; +const kItemTypeMore = "document-more"; +const kItemTypeItem = "document-item"; +const kItemTypeError = "error"; + +function renderItem( + item, + createElement, + state, + setActiveItemId, + setContext, + refresh, + quartoSearchOptions +) { + switch (item.type) { + case kItemTypeDoc: + return createDocumentCard( + createElement, + "file-richtext", + item.title, + item.section, + item.text, + item.href, + item.crumbs, + quartoSearchOptions + ); + case kItemTypeMore: + return createMoreCard( + createElement, + item, + state, + setActiveItemId, + setContext, + refresh + ); + case kItemTypeItem: + return createSectionCard( + createElement, + item.section, + item.text, + item.href + ); + case kItemTypeError: + return createErrorCard(createElement, item.title, item.text); + default: + return undefined; + } +} + +function createDocumentCard( + createElement, + icon, + title, + section, + text, + href, + crumbs, + quartoSearchOptions +) { + const iconEl = createElement("i", { + class: `bi bi-${icon} search-result-icon`, + }); + const titleEl = createElement("p", { class: "search-result-title" }, title); + const titleContents = [iconEl, titleEl]; + const showParent = quartoSearchOptions["show-item-context"]; + if (crumbs && showParent) { + let crumbsOut = undefined; + const crumbClz = ["search-result-crumbs"]; + if (showParent === "root") { + crumbsOut = crumbs.length > 1 ? crumbs[0] : undefined; + } else if (showParent === "parent") { + crumbsOut = crumbs.length > 1 ? crumbs[crumbs.length - 2] : undefined; + } else { + crumbsOut = crumbs.length > 1 ? crumbs.join(" > ") : undefined; + crumbClz.push("search-result-crumbs-wrap"); + } + + const crumbEl = createElement( + "p", + { class: crumbClz.join(" ") }, + crumbsOut + ); + titleContents.push(crumbEl); + } + + const titleContainerEl = createElement( + "div", + { class: "search-result-title-container" }, + titleContents + ); + + const textEls = []; + if (section) { + const sectionEl = createElement( + "p", + { class: "search-result-section" }, + section + ); + textEls.push(sectionEl); + } + const descEl = createElement("p", { + class: "search-result-text", + dangerouslySetInnerHTML: { + __html: text, + }, + }); + textEls.push(descEl); + + const textContainerEl = createElement( + "div", + { class: "search-result-text-container" }, + textEls + ); + + const containerEl = createElement( + "div", + { + class: "search-result-container", + }, + [titleContainerEl, textContainerEl] + ); + + const linkEl = createElement( + "a", + { + href: offsetURL(href), + class: "search-result-link", + }, + containerEl + ); + + const classes = ["search-result-doc", "search-item"]; + if (!section) { + classes.push("document-selectable"); + } + + return createElement( + "div", + { + class: classes.join(" "), + }, + linkEl + ); +} + +function createMoreCard( + createElement, + item, + state, + setActiveItemId, + setContext, + refresh +) { + const moreCardEl = createElement( + "div", + { + class: "search-result-more search-item", + onClick: (e) => { + // Handle expanding the sections by adding the expanded + // section to the list of expanded sections + toggleExpanded(item, state, setContext, setActiveItemId, refresh); + e.stopPropagation(); + }, + }, + item.title + ); + + return moreCardEl; +} + +function toggleExpanded(item, state, setContext, setActiveItemId, refresh) { + const expanded = state.context.expanded || []; + if (expanded.includes(item.target)) { + setContext({ + expanded: expanded.filter((target) => target !== item.target), + }); + } else { + setContext({ expanded: [...expanded, item.target] }); + } + + refresh(); + setActiveItemId(item.__autocomplete_id); +} + +function createSectionCard(createElement, section, text, href) { + const sectionEl = createSection(createElement, section, text, href); + return createElement( + "div", + { + class: "search-result-doc-section search-item", + }, + sectionEl + ); +} + +function createSection(createElement, title, text, href) { + const descEl = createElement("p", { + class: "search-result-text", + dangerouslySetInnerHTML: { + __html: text, + }, + }); + + const titleEl = createElement("p", { class: "search-result-section" }, title); + const linkEl = createElement( + "a", + { + href: offsetURL(href), + class: "search-result-link", + }, + [titleEl, descEl] + ); + return linkEl; +} + +function createErrorCard(createElement, title, text) { + const descEl = createElement("p", { + class: "search-error-text", + dangerouslySetInnerHTML: { + __html: text, + }, + }); + + const titleEl = createElement("p", { + class: "search-error-title", + dangerouslySetInnerHTML: { + __html: ` ${title}`, + }, + }); + const errorEl = createElement("div", { class: "search-error" }, [ + titleEl, + descEl, + ]); + return errorEl; +} + +function positionPanel(pos) { + const panelEl = window.document.querySelector( + "#quarto-search-results .aa-Panel" + ); + const inputEl = window.document.querySelector( + "#quarto-search .aa-Autocomplete" + ); + + if (panelEl && inputEl) { + panelEl.style.top = `${Math.round(panelEl.offsetTop)}px`; + if (pos === "start") { + panelEl.style.left = `${Math.round(inputEl.left)}px`; + } else { + panelEl.style.right = `${Math.round(inputEl.offsetRight)}px`; + } + } +} + +/* Highlighting */ +// highlighting functions +function highlightMatch(query, text) { + if (text) { + const start = text.toLowerCase().indexOf(query.toLowerCase()); + if (start !== -1) { + const startMark = ""; + const endMark = ""; + + const end = start + query.length; + text = + text.slice(0, start) + + startMark + + text.slice(start, end) + + endMark + + text.slice(end); + const startInfo = clipStart(text, start); + const endInfo = clipEnd( + text, + startInfo.position + startMark.length + endMark.length + ); + text = + startInfo.prefix + + text.slice(startInfo.position, endInfo.position) + + endInfo.suffix; + + return text; + } else { + return text; + } + } else { + return text; + } +} + +function clipStart(text, pos) { + const clipStart = pos - 50; + if (clipStart < 0) { + // This will just return the start of the string + return { + position: 0, + prefix: "", + }; + } else { + // We're clipping before the start of the string, walk backwards to the first space. + const spacePos = findSpace(text, pos, -1); + return { + position: spacePos.position, + prefix: "", + }; + } +} + +function clipEnd(text, pos) { + const clipEnd = pos + 200; + if (clipEnd > text.length) { + return { + position: text.length, + suffix: "", + }; + } else { + const spacePos = findSpace(text, clipEnd, 1); + return { + position: spacePos.position, + suffix: spacePos.clipped ? "…" : "", + }; + } +} + +function findSpace(text, start, step) { + let stepPos = start; + while (stepPos > -1 && stepPos < text.length) { + const char = text[stepPos]; + if (char === " " || char === "," || char === ":") { + return { + position: step === 1 ? stepPos : stepPos - step, + clipped: stepPos > 1 && stepPos < text.length, + }; + } + stepPos = stepPos + step; + } + + return { + position: stepPos - step, + clipped: false, + }; +} + +// removes highlighting as implemented by the mark tag +function clearHighlight(searchterm, el) { + const childNodes = el.childNodes; + for (let i = childNodes.length - 1; i >= 0; i--) { + const node = childNodes[i]; + if (node.nodeType === Node.ELEMENT_NODE) { + if ( + node.tagName === "MARK" && + node.innerText.toLowerCase() === searchterm.toLowerCase() + ) { + el.replaceChild(document.createTextNode(node.innerText), node); + } else { + clearHighlight(searchterm, node); + } + } + } +} + +function escapeRegExp(string) { + return string.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); // $& means the whole matched string +} + +// highlight matches +function highlight(term, el) { + const termRegex = new RegExp(term, "ig"); + const childNodes = el.childNodes; + + // walk back to front avoid mutating elements in front of us + for (let i = childNodes.length - 1; i >= 0; i--) { + const node = childNodes[i]; + + if (node.nodeType === Node.TEXT_NODE) { + // Search text nodes for text to highlight + const text = node.nodeValue; + + let startIndex = 0; + let matchIndex = text.search(termRegex); + if (matchIndex > -1) { + const markFragment = document.createDocumentFragment(); + while (matchIndex > -1) { + const prefix = text.slice(startIndex, matchIndex); + markFragment.appendChild(document.createTextNode(prefix)); + + const mark = document.createElement("mark"); + mark.appendChild( + document.createTextNode( + text.slice(matchIndex, matchIndex + term.length) + ) + ); + markFragment.appendChild(mark); + + startIndex = matchIndex + term.length; + matchIndex = text.slice(startIndex).search(new RegExp(term, "ig")); + if (matchIndex > -1) { + matchIndex = startIndex + matchIndex; + } + } + if (startIndex < text.length) { + markFragment.appendChild( + document.createTextNode(text.slice(startIndex, text.length)) + ); + } + + el.replaceChild(markFragment, node); + } + } else if (node.nodeType === Node.ELEMENT_NODE) { + // recurse through elements + highlight(term, node); + } + } +} + +/* Link Handling */ +// get the offset from this page for a given site root relative url +function offsetURL(url) { + var offset = getMeta("quarto:offset"); + return offset ? offset + url : url; +} + +// read a meta tag value +function getMeta(metaName) { + var metas = window.document.getElementsByTagName("meta"); + for (let i = 0; i < metas.length; i++) { + if (metas[i].getAttribute("name") === metaName) { + return metas[i].getAttribute("content"); + } + } + return ""; +} + +function algoliaSearch(query, limit, algoliaOptions) { + const { getAlgoliaResults } = window["@algolia/autocomplete-preset-algolia"]; + + const applicationId = algoliaOptions["application-id"]; + const searchOnlyApiKey = algoliaOptions["search-only-api-key"]; + const indexName = algoliaOptions["index-name"]; + const indexFields = algoliaOptions["index-fields"]; + const searchClient = window.algoliasearch(applicationId, searchOnlyApiKey); + const searchParams = algoliaOptions["params"]; + const searchAnalytics = !!algoliaOptions["analytics-events"]; + + return getAlgoliaResults({ + searchClient, + queries: [ + { + indexName: indexName, + query, + params: { + hitsPerPage: limit, + clickAnalytics: searchAnalytics, + ...searchParams, + }, + }, + ], + transformResponse: (response) => { + if (!indexFields) { + return response.hits.map((hit) => { + return hit.map((item) => { + return { + ...item, + text: highlightMatch(query, item.text), + }; + }); + }); + } else { + const remappedHits = response.hits.map((hit) => { + return hit.map((item) => { + const newItem = { ...item }; + ["href", "section", "title", "text", "crumbs"].forEach( + (keyName) => { + const mappedName = indexFields[keyName]; + if ( + mappedName && + item[mappedName] !== undefined && + mappedName !== keyName + ) { + newItem[keyName] = item[mappedName]; + delete newItem[mappedName]; + } + } + ); + newItem.text = highlightMatch(query, newItem.text); + return newItem; + }); + }); + return remappedHits; + } + }, + }); +} + +let subSearchTerm = undefined; +let subSearchFuse = undefined; +const kFuseMaxWait = 125; + +async function fuseSearch(query, fuse, fuseOptions) { + let index = fuse; + // Fuse.js using the Bitap algorithm for text matching which runs in + // O(nm) time (no matter the structure of the text). In our case this + // means that long search terms mixed with large index gets very slow + // + // This injects a subIndex that will be used once the terms get long enough + // Usually making this subindex is cheap since there will typically be + // a subset of results matching the existing query + if (subSearchFuse !== undefined && query.startsWith(subSearchTerm)) { + // Use the existing subSearchFuse + index = subSearchFuse; + } else if (subSearchFuse !== undefined) { + // The term changed, discard the existing fuse + subSearchFuse = undefined; + subSearchTerm = undefined; + } + + // Search using the active fuse + const then = performance.now(); + const resultsRaw = await index.search(query, fuseOptions); + const now = performance.now(); + + const results = resultsRaw.map((result) => { + const addParam = (url, name, value) => { + const anchorParts = url.split("#"); + const baseUrl = anchorParts[0]; + const sep = baseUrl.search("\\?") > 0 ? "&" : "?"; + anchorParts[0] = baseUrl + sep + name + "=" + value; + return anchorParts.join("#"); + }; + + return { + title: result.item.title, + section: result.item.section, + href: addParam(result.item.href, kQueryArg, query), + text: highlightMatch(query, result.item.text), + crumbs: result.item.crumbs, + }; + }); + + // If we don't have a subfuse and the query is long enough, go ahead + // and create a subfuse to use for subsequent queries + if ( + now - then > kFuseMaxWait && + subSearchFuse === undefined && + resultsRaw.length < fuseOptions.limit + ) { + subSearchTerm = query; + subSearchFuse = new window.Fuse([], kFuseIndexOptions); + resultsRaw.forEach((rr) => { + subSearchFuse.add(rr.item); + }); + } + return results; +} diff --git a/sitemap.xml b/sitemap.xml new file mode 100644 index 0000000..ad3cbef --- /dev/null +++ b/sitemap.xml @@ -0,0 +1,99 @@ + + + + https://RePsychLing.github.io/SMLP2024/fggk21.html + 2024-06-27T23:43:41.222Z + + + https://RePsychLing.github.io/SMLP2024/largescaledesigned.html + 2024-06-27T23:43:41.226Z + + + https://RePsychLing.github.io/SMLP2024/shrinkageplot.html + 2024-06-27T23:43:41.230Z + + + https://RePsychLing.github.io/SMLP2024/glmm.html + 2024-06-27T23:43:41.222Z + + + https://RePsychLing.github.io/SMLP2024/bootstrap.html + 2024-06-27T23:43:40.710Z + + + https://RePsychLing.github.io/SMLP2024/sleepstudy.html + 2024-06-27T23:43:41.230Z + + + https://RePsychLing.github.io/SMLP2024/profiling.html + 2024-06-27T23:43:41.230Z + + + https://RePsychLing.github.io/SMLP2024/contrasts_kwdyz11.html + 2024-06-27T23:43:40.710Z + + + https://RePsychLing.github.io/SMLP2024/check_emotikon_transform.html + 2024-06-27T23:43:40.710Z + + + https://RePsychLing.github.io/SMLP2024/AoGPlots.html + 2024-06-27T23:43:40.702Z + + + https://RePsychLing.github.io/SMLP2024/pkg.html + 2024-06-27T23:43:41.226Z + + + https://RePsychLing.github.io/SMLP2024/index.html + 2024-06-27T23:43:41.222Z + + + https://RePsychLing.github.io/SMLP2024/about.html + 2024-06-27T23:43:40.706Z + + + https://RePsychLing.github.io/SMLP2024/useful_packages.html + 2024-06-27T23:43:41.234Z + + + https://RePsychLing.github.io/SMLP2024/selection.html + 2024-06-27T23:43:41.230Z + + + https://RePsychLing.github.io/SMLP2024/singularity.html + 2024-06-27T23:43:41.230Z + + + https://RePsychLing.github.io/SMLP2024/sleepstudy_speed.html + 2024-06-27T23:43:41.230Z + + + https://RePsychLing.github.io/SMLP2024/kb07.html + 2024-06-27T23:43:41.222Z + + + https://RePsychLing.github.io/SMLP2024/arrow.html + 2024-06-27T23:43:40.706Z + + + https://RePsychLing.github.io/SMLP2024/partial_within.html + 2024-06-27T23:43:41.226Z + + + https://RePsychLing.github.io/SMLP2024/mrk17.html + 2024-06-27T23:43:41.226Z + + + https://RePsychLing.github.io/SMLP2024/kwdyz11.html + 2024-06-27T23:43:41.226Z + + + https://RePsychLing.github.io/SMLP2024/contrasts_fggk21.html + 2024-06-27T23:43:40.710Z + + + https://RePsychLing.github.io/SMLP2024/kkl15.html + 2024-06-27T23:43:41.226Z + + diff --git a/sleepstudy.html b/sleepstudy.html new file mode 100644 index 0000000..da8958f --- /dev/null +++ b/sleepstudy.html @@ -0,0 +1,1590 @@ + + + + + + + + + + + +Analysis of the sleepstudy data – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Analysis of the sleepstudy data

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

The sleepstudy data are from a study of the effects of sleep deprivation on response time reported in Balkin et al. (2000) and in Belenky et al. (2003). Eighteen subjects were allowed only 3 hours of time to sleep each night for 9 successive nights. Their reaction time was measured each day, starting the day before the first night of sleep deprivation, when the subjects were on their regular sleep schedule.

+
+
+
+ +
+
+Note +
+
+
+

This description is inaccurate. In fact the first two days were acclimatization, the third was a baseline and sleep deprivation was only enforced after day 2. To allow for comparison with earlier analyses of these data we retain the old data description for this notebook only.

+
+
+
+

1 Loading the data

+

First attach the MixedModels package and other packages for plotting. The CairoMakie package allows the Makie graphics system (Danisch & Krumbiegel, 2021) to generate high quality static images. Activate that package with the SVG (Scalable Vector Graphics) backend.

+
+
+Code +
using CairoMakie       # graphics back-end
+using DataFrames
+using KernelDensity    # density estimation
+using MixedModels
+using MixedModelsMakie # diagnostic plots
+using ProgressMeter
+using Random           # random number generators
+using RCall            # call R from Julia
+using SMLP2024
+using SMLP2024: dataset
+
+ProgressMeter.ijulia_behavior(:clear)
+CairoMakie.activate!(; type="svg")
+
+
+

The sleepstudy data are one of the datasets available with the MixedModels package. It is re-exported by the SMLP2024 package’s dataset function.

+
+
sleepstudy = dataset("sleepstudy")
+
+
Arrow.Table with 180 rows, 3 columns, and schema:
+ :subj      String
+ :days      Int8
+ :reaction  Float64
+
+
+

Figure 1 displays the data in a multi-panel plot created with the lattice package in R (Sarkar, 2008), using RCall.jl.

+
print(lattice::xyplot(reaction ~ days | subj,
+  $(DataFrame(sleepstudy)),
+  aspect="xy",
+  layout=c(9,2),
+  type=c("g", "p", "r"),
+  index.cond=function(x,y) coef(lm(y ~ x))[1],
+  xlab = "Days of sleep deprivation",
+  ylab = "Average reaction time (ms)"
+))
+NULL
+
+
+
+
+
+ +
+
+Figure 1: Average response time versus days of sleep deprivation by subject +
+
+
+
+
+

Each panel shows the data from one subject and a line fit by least squares to that subject’s data. Starting at the lower left panel and proceeding across rows, the panels are ordered by increasing intercept of the least squares line.

+

There are some deviations from linearity within the panels but the deviations are neither substantial nor systematic.

+
+
+

2 Fitting an initial model

+
+
contrasts = Dict{Symbol,Any}(:subj => Grouping())
+m1 = let f = @formula(reaction ~ 1 + days + (1 + days | subj))
+  fit(MixedModel, f, sleepstudy; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_subj
(Intercept)251.40516.632337.91<1e-9923.7805
days10.46731.50226.97<1e-115.7168
Residual25.5918
+
+
+

This model includes fixed effects for the intercept, representing the typical reaction time at the beginning of the experiment with zero days of sleep deprivation, and the slope w.r.t. days of sleep deprivation. The parameter estimates are about 250 ms. typical reaction time without deprivation and a typical increase of 10.5 ms. per day of sleep deprivation.

+

The random effects represent shifts from the typical behavior for each subject. The shift in the intercept has a standard deviation of about 24 ms. which would suggest a range of about 200 ms. to 300 ms. in the intercepts. Similarly within-subject slopes would be expected to have a range of about 0 ms./day up to 20 ms./day.

+

The random effects for the slope and for the intercept are allowed to be correlated within subject. The estimated correlation, 0.08, is small. This estimate is not shown in the default display above but is shown in the output from VarCorr (variance components and correlations).

+
+
VarCorr(m1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
subj(Intercept)565.5106723.78047
days32.682125.71683+0.08
Residual654.9414525.59182
+
+
+

Technically, the random effects for each subject are unobserved random variables and are not “parameters” in the model per se. Hence we do not report standard errors or confidence intervals for these deviations. However, we can produce prediction intervals on the random effects for each subject. Because the experimental design is balanced, these intervals will have the same width for all subjects.

+

A plot of the prediction intervals versus the level of the grouping factor (subj, in this case) is sometimes called a caterpillar plot because it can look like a fuzzy caterpillar if there are many levels of the grouping factor. By default, the levels of the grouping factor are sorted by increasing value of the first random effect.

+
+
+Code +
caterpillar(m1; vline_at_zero=true)
+
+
+
+
+
+ +
+
+Figure 2: Prediction intervals on random effects for model m1 +
+
+
+
+
+

Figure 2 reinforces the conclusion that there is little correlation between the random effect for intercept and the random effect for slope.

+
+
+

3 A model with uncorrelated random effects

+

The zerocorr function applied to a random-effects term creates uncorrelated vector-valued per-subject random effects.

+
+
m2 = let f = @formula reaction ~ 1 + days + zerocorr(1 + days | subj)
+  fit(MixedModel, f, sleepstudy; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_subj
(Intercept)251.40516.707737.48<1e-9924.1714
days10.46731.51936.89<1e-115.7994
Residual25.5561
+
+
+

Again, the default display doesn’t show that there is no correlation parameter to be estimated in this model, but the VarCorr display does.

+
+
VarCorr(m2)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
ColumnVarianceStd.DevCorr.
subj(Intercept)584.2589724.17145
days33.632815.79938.
Residual653.1157825.55613
+
+
+

This model has a slightly lower log-likelihood than does m1 and one fewer parameter than m1. A likelihood-ratio test can be used to compare these nested models.

+
+
MixedModels.likelihoodratiotest(m2, m1)
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
reaction ~ 1 + days + zerocorr(1 + days | subj)51752
reaction ~ 1 + days + (1 + days | subj)61752010.8004
+
+
+

Alternatively, the AIC or BIC values can be compared.

+
+
+Code +
let mods = [m2, m1]
+  Table(;
+    model=[:m2, :m1],
+    pars=dof.(mods),
+    geomdof=(sum ∘ leverage).(mods),
+    AIC=aic.(mods),
+    BIC=bic.(mods),
+    AICc=aicc.(mods),
+  )
+end
+
+
+
Table with 6 columns and 2 rows:
+     model  pars  geomdof  AIC      BIC      AICc
+   ┌────────────────────────────────────────────────
+ 1 │ m2     5     29.045   1762.0   1777.97  1762.35
+ 2 │ m1     6     28.6115  1763.94  1783.1   1764.42
+
+
+

The goodness of fit measures: AIC, BIC, and AICc, are all on a “smaller is better” scale and, hence, they all prefer m2.

+

The pars column, which is the same as the model-dof column in the likelihood ratio test output, is simply a count of the number of parameters to be estimated when fitting the model. For example, in m2 there are two fixed-effects parameters and three variance components (including the residual variance).

+

An alternative, more geometrically inspired definition of “degrees of freedom”, is the sum of the leverage values, called geomdof in this table.

+

Interestingly, the model with fewer parameters, m2, has a greater sum of the leverage values than the model with more parameters, m1. We’re not sure what to make of that.

+

In both cases the sum of the leverage values is toward the upper end of the range of possible values, which is the rank of the fixed-effects model matrix (2) up to the rank of the fixed-effects plus the random effects model matrix (2 + 36 = 38).

+
+
+
+ +
+
+Note +
+
+
+

I think that the upper bound may be 36, not 38, because the two columns of X lie in the column span of Z

+
+
+

This comparison does show, however, that a simple count of the parameters in a mixed-effects model can underestimate, sometimes drastically, the model complexity. This is because a single variance component or multiple components can add many dimensions to the linear predictor.

+
+
+

4 Some diagnostic plots

+

In mixed-effects models the linear predictor expression incorporates fixed-effects parameters, which summarize trends for the population or certain well-defined subpopulations, and random effects which represent deviations associated with the experimental units or observational units - individual subjects, in this case. The random effects are modeled as unobserved random variables.

+

The conditional means of these random variables, sometimes called the BLUPs or Best Linear Unbiased Predictors, are not simply the least squares estimates. They are attenuated or shrunk towards zero to reflect the fact that the individuals are assumed to come from a population. A shrinkage plot, Figure 3, shows the BLUPs from the model fit compared to the values without any shrinkage. If the BLUPs are similar to the unshrunk values then the more complicated model accounting for individual differences is supported. If the BLUPs are strongly shrunk towards zero then the additional complexity in the model to account for individual differences is not providing sufficient increase in fidelity to the data to warrant inclusion.

+
+
+Code +
shrinkageplot!(Figure(; resolution=(500, 500)), m1)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 3: Shrinkage plot of means of the random effects in model m1 +
+
+
+
+
+
+
+
+ +
+
+Note +
+
+
+

This plot could be drawn as shrinkageplot(m1). The reason for explicitly creating a Figure to be modified by shrinkageplot! is to control the resolution.

+
+
+

This plot shows an intermediate pattern. The random effects are somewhat shrunk toward the origin, a model simplification trend, but not completely shrunk - indicating that fidelity to the data is enhanced with these additional coefficients in the linear predictor.

+

If the shrinkage were primarily in one direction - for example, if the arrows from the unshrunk values to the shrunk values were mostly in the vertical direction - then we would get an indication that we could drop the random effect for slope and revert to a simpler model. This is not the case here.

+

As would be expected, the unshrunk values that are further from the origin tend to be shrunk more toward the origin. That is, the arrows that originate furthest from the origin are longer. However, that is not always the case. The arrow in the upper right corner, from S337, is relatively short. Examination of the panel for S337 in the data plot shows a strong linear trend, even though both the intercept and the slope are unusually large. The neighboring panels in the data plot, S330 and S331, have more variability around the least squares line and are subject to a greater amount of shrinkage in the model. (They correspond to the two arrows on the right hand side of the figure around -5 on the vertical scale.)

+
+
+

5 Assessing variability by bootstrapping

+

The speed of fitting linear mixed-effects models using MixedModels.jl allows for using simulation-based approaches to inference instead of relying on approximate standard errors. A parametric bootstrap sample for model m is a collection of models of the same form as m fit to data values simulated from m. That is, we pretend that m and its parameter values are the true parameter values, simulate data from these values, and estimate parameters from the simulated data.

+

Simulating and fitting a substantial number of model fits, 5000 in this case, takes only a few seconds, following which we extract a data frame of the parameter estimates and plot densities of some of these estimates.

+
+
rng = Random.seed!(42)    # initialize a random number generator
+m1bstp = parametricbootstrap(rng, 5000, m1)
+tbl = m1bstp.tbl
+
+
Table with 10 columns and 5000 rows:
+      obj      β1       β2       σ        σ1       σ2       ρ1          ⋯
+    ┌────────────────────────────────────────────────────────────────────
+ 1  │ 1717.29  260.712  9.84975  23.4092  15.3317  6.40296  -0.025958   ⋯
+ 2  │ 1744.06  262.253  12.3008  25.7047  16.3183  5.54688  0.552607    ⋯
+ 3  │ 1714.16  253.149  12.879   22.2753  25.4787  6.1444   0.0691545   ⋯
+ 4  │ 1711.54  263.376  11.5798  23.3128  18.8039  4.6557   0.103361    ⋯
+ 5  │ 1741.66  248.429  9.39444  25.4355  20.141   5.27357  -0.163599   ⋯
+ 6  │ 1754.81  256.794  8.024    26.5087  10.6781  7.14162  0.335471    ⋯
+ 7  │ 1777.73  253.388  8.83556  27.8623  17.8325  7.17386  0.00378979  ⋯
+ 8  │ 1768.59  254.441  11.4479  27.4033  16.2489  6.67042  0.725394    ⋯
+ 9  │ 1753.56  244.906  11.3423  25.6046  25.3607  5.98654  -0.171821   ⋯
+ 10 │ 1722.61  257.088  9.18397  23.3386  24.9274  5.18012  0.181143    ⋯
+ 11 │ 1738.16  251.262  11.6568  25.7823  17.6663  4.07218  0.25814     ⋯
+ 12 │ 1747.76  258.302  12.8015  26.1085  19.2398  5.06066  0.879712    ⋯
+ 13 │ 1745.91  254.57   11.8062  24.8863  24.2513  6.14642  0.0126996   ⋯
+ 14 │ 1738.8   251.179  10.3226  24.2672  23.7195  6.32645  0.368592    ⋯
+ 15 │ 1724.76  238.603  11.5045  25.23    19.0263  3.64038  -0.34657    ⋯
+ 16 │ 1777.7   254.133  8.26398  26.9846  26.3715  7.8283   -0.288773   ⋯
+ 17 │ 1748.33  251.571  9.5294   26.2927  21.9611  4.31316  -0.150104   ⋯
+ ⋮  │    ⋮        ⋮        ⋮        ⋮        ⋮        ⋮         ⋮       ⋱
+
+
+

An empirical density plot of the estimates for the fixed-effects coefficients, Figure 4, shows the normal distribution, “bell-curve”, shape as we might expect.

+
+
+Code +
begin
+  f1 = Figure(; resolution=(1000, 400))
+  CairoMakie.density!(
+    Axis(f1[1, 1]; xlabel="Intercept [ms]"), tbl.β1
+  )
+  CairoMakie.density!(
+    Axis(f1[1, 2]; xlabel="Coefficient of days [ms/day]"),
+    tbl.β2
+  )
+  f1
+end
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 4: Empirical density plots of bootstrap replications of fixed-effects parameter estimates +
+
+
+
+
+

It is also possible to create interval estimates of the parameters from the bootstrap replicates. We define the 1-α shortestcovint to be the shortest interval that contains a proportion 1-α (defaults to 95%) of the bootstrap estimates of the parameter.

+
+
Table(shortestcovint(m1bstp))
+
+
Table with 5 columns and 6 rows:
+     type  group     names              lower      upper
+   ┌──────────────────────────────────────────────────────
+ 1 │ β     missing   (Intercept)        239.64     265.228
+ 2 │ β     missing   days               7.42347    13.1607
+ 3 │ σ     subj      (Intercept)        10.1722    33.0876
+ 4 │ σ     subj      days               2.9948     7.66116
+ 5 │ ρ     subj      (Intercept), days  -0.401353  1.0
+ 6 │ σ     residual  missing            22.701     28.5016
+
+
+

The intervals look reasonable except that the upper end point of the interval for ρ1, the correlation coefficient, is 1.0 . It turns out that the estimates of ρ have a great deal of variability.

+

Because there are several values on the boundary (ρ = 1.0) and a pulse like this is not handled well by a density plot, we plot this sample as a histogram, Figure 5.

+
+
+Code +
hist(
+  tbl.ρ1;
+  bins=40,
+  axis=(; xlabel="Estimated correlation of the random effects"),
+  figure=(; resolution=(500, 500)),
+)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 5: Histogram of bootstrap replications of the within-subject correlation parameter +
+
+
+
+
+

Finally, density plots for the variance components (but on the scale of the standard deviation), Figure 6, show reasonable symmetry.

+
+
+Code +
begin
+  f2 = Figure(; resolution=(1000, 300))
+  CairoMakie.density!(
+    Axis(f2[1, 1]; xlabel="Residual σ"),
+    tbl.σ,
+  )
+  CairoMakie.density!(
+    Axis(f2[1, 2]; xlabel="subj-Intercept σ"),
+    tbl.σ1,
+  )
+  CairoMakie.density!(
+    Axis(f2[1, 3]; xlabel="subj-slope σ"),
+    tbl.σ2,
+  )
+  f2
+end
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 6: Empirical density plots of bootstrap replicates of standard deviation estimates +
+
+
+
+
+

The estimates of the coefficients, β₁ and β₂, are not highly correlated as shown in a scatterplot of the bootstrap estimates, Figure 7 .

+
+
vcov(m1; corr=true)  # correlation estimate from the model
+
+
2×2 Matrix{Float64}:
+  1.0       -0.137545
+ -0.137545   1.0
+
+
+
+
+Code +
let
+  scatter(
+    tbl.β1, tbl.β2,
+    color=(:blue, 0.20),
+    axis=(; xlabel="Intercept", ylabel="Coefficient of days"),
+    figure=(; resolution=(500, 500)),
+  )
+  contour!(kde((tbl.β1, tbl.β2)))
+  current_figure()
+end
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 7: Scatter-plot of bootstrap replicates of fixed-effects estimates with contours +
+
+
+
+
+
+
+

6 References

+
+
+Balkin, T., Thome, D., Sing, H., Thomas, M., Redmond, D., Wesensten, N., Williams, J., Hall, S., & Belenky, G. (2000). Effects of sleep schedules on commercial motor vehicle driver performance (DOT-MC-00-133). Federal Motor Carrier Safety Administration. https://doi.org/10.21949/1503015. +
+
+Belenky, G., Wesensten, N. J., Thorne, D. R., Thomas, M. L., Sing, H. C., Redmond, D. P., Russo, M. B., & Balkin, T. J. (2003). Patterns of performance degradation and restoration during sleep restriction and subsequent recovery: A sleep dose-response study. Journal of Sleep Research, 12(1), 1–12. https://doi.org/10.1046/j.1365-2869.2003.00337.x +
+
+Danisch, S., & Krumbiegel, J. (2021). Makie.jl: Flexible high-performance data visualization for julia. Journal of Open Source Software, 6(65), 3349. https://doi.org/10.21105/joss.03349 +
+
+Sarkar, D. (2008). Lattice: Mutivariate data visualization with r. Springer-Verlag GmbH. https://www.ebook.de/de/product/11429038/deepayan_sarkar_lattice.html +
+
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/sleepstudy_files/figure-html/cell-4-output-1.png b/sleepstudy_files/figure-html/cell-4-output-1.png new file mode 100644 index 0000000..7a0d8e7 Binary files /dev/null and b/sleepstudy_files/figure-html/cell-4-output-1.png differ diff --git a/sleepstudy_files/figure-html/fig-bsbetacontours-output-2.svg b/sleepstudy_files/figure-html/fig-bsbetacontours-output-2.svg new file mode 100644 index 0000000..e9cb5f8 --- /dev/null +++ b/sleepstudy_files/figure-html/fig-bsbetacontours-output-2.svg @@ -0,0 +1,5268 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/sleepstudy_files/figure-html/fig-bsbetadensity-output-2.svg b/sleepstudy_files/figure-html/fig-bsbetadensity-output-2.svg new file mode 100644 index 0000000..9331051 --- /dev/null +++ b/sleepstudy_files/figure-html/fig-bsbetadensity-output-2.svg @@ -0,0 +1,506 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/sleepstudy_files/figure-html/fig-bssigmadensity-output-2.svg b/sleepstudy_files/figure-html/fig-bssigmadensity-output-2.svg new file mode 100644 index 0000000..89aa7da --- /dev/null +++ b/sleepstudy_files/figure-html/fig-bssigmadensity-output-2.svg @@ -0,0 +1,503 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/sleepstudy_files/figure-html/fig-correlationhist-output-2.svg b/sleepstudy_files/figure-html/fig-correlationhist-output-2.svg new file mode 100644 index 0000000..f5cbaba --- /dev/null +++ b/sleepstudy_files/figure-html/fig-correlationhist-output-2.svg @@ -0,0 +1,381 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/sleepstudy_files/figure-html/fig-m1caterpillar-output-1.svg b/sleepstudy_files/figure-html/fig-m1caterpillar-output-1.svg new file mode 100644 index 0000000..9d86767 --- /dev/null +++ b/sleepstudy_files/figure-html/fig-m1caterpillar-output-1.svg @@ -0,0 +1,702 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/sleepstudy_files/figure-html/fig-m1shrinkage-output-2.svg b/sleepstudy_files/figure-html/fig-m1shrinkage-output-2.svg new file mode 100644 index 0000000..141b56f --- /dev/null +++ b/sleepstudy_files/figure-html/fig-m1shrinkage-output-2.svg @@ -0,0 +1,300 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/sleepstudy_files/figure-html/fig-reaction_vs_days_by_subject-output-1.png b/sleepstudy_files/figure-html/fig-reaction_vs_days_by_subject-output-1.png new file mode 100644 index 0000000..7a0d8e7 Binary files /dev/null and b/sleepstudy_files/figure-html/fig-reaction_vs_days_by_subject-output-1.png differ diff --git a/sleepstudy_files/figure-html/fig-reaction_vs_days_by_subject1-output-1.png b/sleepstudy_files/figure-html/fig-reaction_vs_days_by_subject1-output-1.png new file mode 100644 index 0000000..7a0d8e7 Binary files /dev/null and b/sleepstudy_files/figure-html/fig-reaction_vs_days_by_subject1-output-1.png differ diff --git a/sleepstudy_speed.html b/sleepstudy_speed.html new file mode 100644 index 0000000..1558c8c --- /dev/null +++ b/sleepstudy_speed.html @@ -0,0 +1,1756 @@ + + + + + + + + + + + +The sleepstudy: Speed - for a change … – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

The sleepstudy: Speed - for a change …

+
+ + + +
+ +
+
Author
+
+

Phillip Alday, Douglas Bates, and Reinhold Kliegl

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +
+

1 Background

+

Belenky et al. (2003) reported effects of sleep deprivation across a 14-day study of 30-to-40-year old men and women holding commercial vehicle driving licenses. Their analyses are based on a subset of tasks and ratings from very large and comprehensive test and questionnaire battery (Balkin et al., 2000).

+

Initially 66 subjects were assigned to one of four time-in-bed (TIB) groups with 9 hours (22:00-07:00) of sleep augmentation or 7 hours (24:00-07:00), 5 hours (02:00-07:00), and 3 hours (04:00-0:00) of sleep restrictions per night, respectively. The final sample comprised 56 subjects. The Psychomotor Vigilance Test (PVT) measures simple reaction time to a visual stimulus, presented approximately 10 times ⁄ minute (interstimulus interval varied from 2 to 10 s in 2-s increments) for 10 min and implemented in a thumb-operated, hand-held device (Dinges & Powell, 1985).

+
+

1.1 Design

+

The study comprised 2 training days (T1, T2), one day with baseline measures (B), seven days with sleep deprivation (E1 to E7), and four recovery days (R1 to R4). T1 and T2 were devoted to training on the performance tests and familiarization with study procedures. PVT baseline testing commenced on the morning of the third day (B) and testing continued for the duration of the study (E1–E7, R1–R3; no measures were taken on R4). Bed times during T, B, and R days were 8 hours (23:00-07:00).

+
+
+

1.2 Test schedule within days

+

The PVT (along with the Stanford Sleepiness Scale) was administered as a battery four times per day (09:00, 12:00, 15:00, and 21:00 h); the battery included other tests not reported here (see Balkin et al., 2000). The sleep latency test was administered at 09:40 and 15:30 h for all groups. Subjects in the 3- and 5-h TIB groups performed an additional battery at 00:00 h and 02:00 h to occupy their additional time awake. The PVT and SSS were administered in this battery; however, as data from the 00:00 and 02:00 h sessions were not common to all TIB groups, these data were not included in the statistical analyses reported in the paper.

+
+
+

1.3 Statistical analyses

+

The authors analyzed response speed, that is (1/RT)*1000 – completely warranted according to a Box-Cox check of the current data – with mixed-model ANOVAs using group as between- and day as within-subject factors. The ANOVA was followed up with simple tests of the design effects implemented over days for each of the four groups.

+
+
+

1.4 Current data

+

The current data distributed with the RData collection is attributed to the 3-hour TIB group, but the means do not agree at all with those reported for this group in (Belenky et al., 2003, fig. 3) where the 3-hour TIB group is also based on only 13 (not 18) subjects. Specifically, the current data show a much smaller slow-down of response speed across E1 to E7 and do not reflect the recovery during R1 to R3. The current data also cover only 10 not 11 days, but it looks like only R3 is missing. The closest match of the current means was with the average of the 3-hour and 7-hour TIB groups; if only males were included, this would amount to 18 subjects. (This conjecture is based only on visual inspection of graphs.)

+
+
+
+

2 Setup

+

First we attach the various packages needed, define a few helper functions, read the data, and get everything in the desired shape.

+
+
+Code +
using CairoMakie         # device driver for static (SVG, PDF, PNG) plots
+using Chain              # like pipes but cleaner
+using DataFrameMacros
+using DataFrames
+using MixedModels
+using MixedModelsMakie   # plots specific to mixed-effects models using Makie
+
+using ProgressMeter
+
+CairoMakie.activate!(; type="svg")
+ProgressMeter.ijulia_behavior(:clear);
+
+
+
+
+

3 Preprocessing

+

The sleepstudy data are one of the datasets available with recent versions of the MixedModels package. We carry out some preprocessing to have the dataframe in the desired shape:

+
    +
  • Capitalize random factor Subj
    • +
  • Compute speed as an alternative dependent variable from reaction, warranted by a ‘boxcox’ check of residuals.
    • +
  • Create a GroupedDataFrame by levels of Subj (the original dataframe is available as gdf.parent, which we name df)
    • +
+
+
gdf = @chain MixedModels.dataset(:sleepstudy) begin
+  DataFrame
+  rename!(:subj => :Subj, :days => :day)
+  @transform!(:speed = 1000 / :reaction)
+  groupby(:Subj)
+end
+
+

GroupedDataFrame with 18 groups based on key: Subj

First Group (10 rows): Subj = "S308"
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowSubjdayreactionspeed
StringInt8Float64Float64
1S3080249.564.00705
2S3081258.7053.86541
3S3082250.8013.98723
4S3083321.443.111
5S3084356.8522.80228
6S3085414.692.41144
7S3086382.2042.61641
8S3087290.1493.44651
9S3088430.5852.32242
10S3089466.3532.1443
+

⋮

Last Group (10 rows): Subj = "S372"
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowSubjdayreactionspeed
StringInt8Float64Float64
1S3720269.4123.71179
2S3721273.4743.65665
3S3722297.5973.36025
4S3723310.6323.21925
5S3724287.1733.48223
6S3725329.6083.03391
7S3726334.4822.9897
8S3727343.222.91358
9S3728369.1422.70899
10S3729364.1242.74632
+
+
+
+
+
df = gdf.parent
+describe(df)
+
+
4×7 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Rowvariablemeanminmedianmaxnmissingeltype
SymbolUnion…AnyUnion…AnyInt64DataType
1SubjS308S3720String
2day4.504.590Int8
3reaction298.508194.332288.651466.3530Float64
4speed3.466342.14433.464435.145830Float64
+
+
+
+
+
+

4 Estimates for pooled data

+

In the first analysis we ignore the dependency of observations due to repeated measures from the same subjects. We pool all the data and estimate the regression of 180 speed scores on the nine days of the experiment.

+
+
pooledcoef = simplelinreg(df.day, df.speed)  # produces a Tuple
+
+
(3.965811974783149, -0.11099359232199696)
+
+
+
+
+

5 Within-subject effects

+

In the second analysis we estimate coefficients for each Subj without regard of the information available from the complete set of data. We do not “borrow strength” to adjust for differences due to between-Subj variability and due to being far from the population mean.

+
+

5.1 Within-subject simple regressions

+

Applying combine to a grouped data frame like gdf produces a DataFrame with a row for each group. The permutation ord provides an ordering for the groups by increasing intercept (predicted response at day 0).

+
+
within = combine(gdf, [:day, :speed] => simplelinreg => :coef)
+
+
18×2 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowSubjcoef
StringTuple…
1S308(3.94806, -0.194812)
2S309(4.87022, -0.0475185)
3S310(4.90606, -0.120054)
4S330(3.4449, -0.0291309)
5S331(3.47647, -0.0498047)
6S332(3.84436, -0.105511)
7S333(3.60159, -0.0917378)
8S334(4.04528, -0.133527)
9S335(3.80451, 0.0455771)
10S337(3.34374, -0.137744)
11S349(4.46855, -0.170885)
12S350(4.21414, -0.20151)
13S351(3.80469, -0.0728582)
14S352(3.68634, -0.144957)
15S369(3.85384, -0.120531)
16S370(4.52679, -0.215965)
17S371(3.853, -0.0936243)
18S372(3.69208, -0.113292)
+
+
+
+

Figure 1 shows the reaction speed versus days of sleep deprivation by subject. The panels are arranged by increasing initial reaction speed starting at the lower left and proceeding across rows.

+
+
+Code +
let
+  ord = sortperm(first.(within.coef))
+  labs = values(only.(keys(gdf)))[ord]       # labels for panels
+  f = clevelandaxes!(Figure(; resolution=(1000, 750)), labs, (2, 9))
+  for (axs, sdf) in zip(f.content, gdf[ord]) # iterate over the panels and groups
+    scatter!(axs, sdf.day, sdf.speed)      # add the points
+    coef = simplelinreg(sdf.day, sdf.speed)
+    abline!(axs, first(coef), last(coef))  # add the regression line
+  end
+  f
+end
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+┌ Warning: abline! is deprecated and will be removed in the future. Use ablines / ablines! instead.
+│   caller = top-level scope at sleepstudy_speed.qmd:106
+└ @ Core ~/Work/SMLP2024/sleepstudy_speed.qmd:106
+
+
+
+
+
+ +
+
+Figure 1: Reaction speed (s⁻¹) versus days of sleep deprivation by subject +
+
+
+
+
+
+
+
+

6 Basic LMM

+
+
contrasts = Dict(:Subj => Grouping())
+m1 = let
+  form = @formula speed ~ 1 + day + (1 + day | Subj)
+  fit(MixedModel, form, df; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)3.96580.105637.55<1e-990.4190
day-0.11100.0151-7.37<1e-120.0566
Residual0.2698
+
+
+

This model includes fixed effects for the intercept which estimates the average speed on the baseline day of the experiment prior to sleep deprivation, and the slowing per day of sleep deprivation. In this case about -0.11/second.

+

The random effects represent shifts from the typical behavior for each subject.The shift in the intercept has a standard deviation of about 0.42/s.

+

The within-subject correlation of the random effects for intercept and slope is small, -0.18, indicating that a simpler model with a correlation parameter (CP) forced to/ assumed to be zero may be sufficient.

+
+
+

7 No correlation parameter: zcp LMM

+

The zerocorr function applied to a random-effects term estimates one parameter less than LMM m1– the CP is now fixed to zero.

+
+
m2 = let
+  form = @formula speed ~ 1 + day + zerocorr(1 + day | Subj)
+  fit(MixedModel, form, df; contrasts)
+end
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Est.SEzpσ_Subj
(Intercept)3.96580.103338.38<1e-990.4085
day-0.11100.0147-7.53<1e-130.0550
Residual0.2706
+
+
+

LMM m2 has a slghtly lower log-likelihood than LMM m1 but also one fewer parameters. A likelihood-ratio test is used to compare these nested models.

+
+
+Code +
MixedModels.likelihoodratiotest(m2, m1)
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
model-dofdevianceχ²χ²-dofP(>χ²)
speed ~ 1 + day + zerocorr(1 + day | Subj)5125
speed ~ 1 + day + (1 + day | Subj)6125010.5192
+
+
+

Alternatively, the AIC, AICc, and BIC values can be compared. They are on a scale where “smaller is better”. All three model-fit statistics prefer the zcpLMM m2.

+
+
+Code +
let
+  mods = [m2, m1]
+  DataFrame(;
+    dof=dof.(mods),
+    deviance=deviance.(mods),
+    AIC=aic.(mods),
+    AICc=aicc.(mods),
+    BIC=bic.(mods),
+  )
+end
+
+
+
2×5 DataFrame
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
RowdofdevianceAICAICcBIC
Int64Float64Float64Float64Float64
15125.379135.379135.724151.344
26124.964136.964137.45156.122
+
+
+
+
+
+

8 Conditional modes of the random effects

+

The third set of estimates are their conditional modes. They represent a compromise between their own data and the model parameters. When distributional assumptions hold, predictions based on these estimates are more accurate than either the pooled or the within-subject estimates. Here we “borrow strength” to improve the accuracy of prediction.

+
+
+

9 Caterpillar plots (effect profiles)

+
+
+Code +
caterpillar(m2)
+
+
+
+
+
+ +
+
+Figure 2: Prediction intervals on the random effects in model m2 +
+
+
+
+
+
+
+

10 Shrinkage plot

+
+
+Code +
shrinkageplot!(Figure(; resolution=(500, 500)), m2)
+
+
+
┌ Warning: Found `resolution` in the theme when creating a `Scene`. The `resolution` keyword for `Scene`s and `Figure`s has been deprecated. Use `Figure(; size = ...` or `Scene(; size = ...)` instead, which better reflects that this is a unitless size and not a pixel resolution. The key could also come from `set_theme!` calls or related theming functions.
+└ @ Makie ~/.julia/packages/Makie/qMluh/src/scenes.jl:227
+
+
+
+
+
+ +
+
+Figure 3: Shrinkage plot of the means of the random effects in model m2 +
+
+
+
+
+
+
+

11 References

+
+
+Balkin, T., Thome, D., Sing, H., Thomas, M., Redmond, D., Wesensten, N., Williams, J., Hall, S., & Belenky, G. (2000). Effects of sleep schedules on commercial motor vehicle driver performance (DOT-MC-00-133). Federal Motor Carrier Safety Administration. https://doi.org/10.21949/1503015. +
+
+Belenky, G., Wesensten, N. J., Thorne, D. R., Thomas, M. L., Sing, H. C., Redmond, D. P., Russo, M. B., & Balkin, T. J. (2003). Patterns of performance degradation and restoration during sleep restriction and subsequent recovery: A sleep dose-response study. Journal of Sleep Research, 12(1), 1–12. https://doi.org/10.1046/j.1365-2869.2003.00337.x +
+
+Dinges, D. F., & Powell, J. W. (1985). Microcomputer analyses of performance on a portable, simple visual RT task during sustained operations. Behavior Research Methods, Instruments, and Computers, 17(6), 652–655. https://doi.org/10.3758/bf03200977 +
+
+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file diff --git a/sleepstudy_speed_files/figure-html/fig-caterpillarm2-output-1.svg b/sleepstudy_speed_files/figure-html/fig-caterpillarm2-output-1.svg new file mode 100644 index 0000000..275ef1b --- /dev/null +++ b/sleepstudy_speed_files/figure-html/fig-caterpillarm2-output-1.svg @@ -0,0 +1,737 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/sleepstudy_speed_files/figure-html/fig-shrinkagem2-output-2.svg b/sleepstudy_speed_files/figure-html/fig-shrinkagem2-output-2.svg new file mode 100644 index 0000000..fe5a1c0 --- /dev/null +++ b/sleepstudy_speed_files/figure-html/fig-shrinkagem2-output-2.svg @@ -0,0 +1,288 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/sleepstudy_speed_files/figure-html/fig-xyplotsleepspeed-output-2.svg b/sleepstudy_speed_files/figure-html/fig-xyplotsleepspeed-output-2.svg new file mode 100644 index 0000000..ed782be --- /dev/null +++ b/sleepstudy_speed_files/figure-html/fig-xyplotsleepspeed-output-2.svg @@ -0,0 +1,903 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/useful_packages.html b/useful_packages.html new file mode 100644 index 0000000..89c2525 --- /dev/null +++ b/useful_packages.html @@ -0,0 +1,1084 @@ + + + + + + + + + + + +Useful packages – SMLP2024: Advanced Frequentist Track + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + +
+ + + +
+ +
+
+

Useful packages

+
+ + + +
+ +
+
Author
+
+

Phillip Alday

+
+
+ +
+
Published
+
+

2024-06-27

+
+
+ + +
+ + + +
+ + +

Unlike R, Julia does not immediately expose a huge number of functions, but instead requires loading packages (whether from the standard library or from the broader package ecosystem) for a lot of relevant functionality for statistical analysis. There are technical reasons for this, such as the ease of using the Julia package system. One further motivation is that Julia is aimed at a broader “technical computing” audience (like MATLAB or perhaps Python) and less at a “statistical analysis” audience.

+

This has two important implications:

+
    +
  1. Even relatively simple programs will often load several packages.
    2. +
  2. Packages are often focused on adding a relatively narrow set of functionality, which means that “basic” functionality (e.g. reading a CSV file and manipulating it as a DataFrame) is often split across multiple packages. In other words, see the the first point!
    3. +
+

This notebook is not intended to be an exhaustive list of packages, but rather to highlight a few packages that I suspect will be particularly useful. Before getting onto the packages, I have one final hint: take advantage of how easy and first-class package management in Julia is. Having good package management makes reproducible analyses much easier and avoids breaking old analyses when you start a new one. The package-manager REPL mode (activated by typing ] at the julia> prompt) is very useful.

+
+

1 Data wrangling

+
+

1.1 Reading data

+
    +

  • Arrow.jl a high performance format for data storage, accessible in R via the arrow package and in Python via pyarrow. (Confusingly, the function for reading and writing Arrow format files in R is called read_feather and write_feather, but the modern Arrow format is distinct from the older Feather format provided by the feather package.) This is the format that we store the example and test datasets in for MixedModels.jl.

    • +

  • CSV.jl useful for reading comma-separated values, tab-separated values and basically everything handled by the read.csv and read.table family of functions in R.

    • +
+

Note that by default both Arrow.jl and CSV.jl do not return a DataFrame, but rather “column tables” – named tuples of column vectors.

+
+
+

1.2 DataFrames

+

Unlike in R, DataFrames are not part of the base language, nor the standard library.

+

DataFrames.jl provides the basic infrastructure around DataFrames, as well as its own mini language for doing the split-apply-combine approach that underlies R’s dplyr and much of the tidyverse. The DataFrames.jl documentation is the place for looking at how to e.g. read in a CSV or Arrow file as a DataFrame. Note that DataFrames.jl by default depends on CategoricalArrays.jl to handle the equivalent of factor in the R world, but there is an alternative package for factor-like array type in Julia, PooledArrays.jl. PooledArrays are simpler, but more limited than CategoricalArrays and we (Phillip and Doug) sometimes use them in our examples and simulations. The tables produced by reading an Arrow file have their own representation of factor-like data as DictEncoded arrays.

+

DataFrame.jl’s mini language can be a bit daunting, if you’re used to manipulations in the style of base R or the tidyverse. For that, there are several options; recently, we’e had particularly nice experiences with DataFrameMacros.jl and Chain.jl for a convenient syntax to connect or “pipe” together successive operations. It’s your choice whether and which of these add-ons you want to use! Phillip tends to write his code using raw DataFrames.jl, but Doug really enjoys DataFrameMacros.jl.

+

The recently added Tidier collection of Julia packages is popular with those coming from the tidyverse.

+
+
+

1.3 Regression

+

Unlike in R, neither formula processing nor basic regression are part of the base language or the standard library.

+

The formula syntax and basic contrast-coding schemes in Julia is provided by StatsModels.jl. By default, MixedModels.jl re-exports the @formula macro and most commonly used contrast schemes from StatsModels.jl, so you often don’t have to worry about loading StatsModels.jl directly. The same is true for GLM.jl, which provides basic linear and generalized linear models, such as ordinary least squares (OLS) regression and logistic regression, i.e. the classical, non mixed regression models.

+

The basic functionality looks quite similar to R, e.g.

+
julia > lm(@formula(y ~ 1 + x), data)
+julia > glm(@formula(y ~ 1 + x), data, Binomial(), LogitLink())
+

but the more general modelling API (also used by MixedModels.jl) is also supported:

+
julia > fit(LinearModel, @formula(y ~ 1 + x), mydata)
+julia > fit(
+  GeneralizedLinearModel,
+  @formula(y ~ 1 + x),
+  data,
+  Binomial(),
+  LogitLink(),
+)
+

(You can also specify your model matrices directly and skip the formula interface, but we don’t recommend this as it’s easy to mess up in really subtle but very problematic ways.)

+
+
+

1.4 @formula, macros and domain-specific languages

+

As a sidebar: why is @formula a macro and not a normal function? Well, that’s because formulas are essentially their own domain-specific language (a variant of Wilkinson-Rogers notation) and macros are used for manipulating the language itself – or in this case, handling an entirely new, embedded language! This is also why macros are used by packages like Turing.jl and Soss.jl that define a language for Bayesian probabilistic programming like PyMC3 or Stan.

+
+
+

1.5 Extensions to the formula syntax

+

There are several ongoing efforts to extend the formula syntax to include some of the “extras” available in R, e.g. RegressionFormulae.jl to use the caret (^) notation to limit interactions to a certain order ((a+b+c)^2 generates a + b + c + a&b + a&c + b&c, but not a&b&c). Note also that Julia uses & to express interactions, not : like in R.

+
+
+

1.6 Standardizing Predictors

+

Although function calls such as log can be used within Julia formulae, they must act on a rowwise basis, i.e. on observations. Transformations such as z-scoring or centering (often done with scale in R) require knowledge of the entire column. StandardizedPredictors.jl provides functions for centering, scaling, and z-scoring within the formula. These are treated as pseudo-contrasts and computed on demand, meaning that predict and effects (see next) computations will handle these transformations on new data (e.g. centering new data around the mean computed during fitting the original data) correctly and automatically.

+
+
+

1.7 Effects

+

John Fox’s effects package in R (and the related ggeffects package for plotting these using ggplot2) provides a nice way to visualize a model’s overall view of the data. This functionality is provided by Effects.jl and works out-of-the-box with most regression model packages in Julia (including MixedModels.jl). Support for formulae with embedded functions (such as log) is not yet complete, but we’re working on it!

+
+
+

1.8 Estimated Marginal / Least Square Means

+

Effects.jl provides a subset of the functionality (basic estimated-marginal means and exhaustive pairwise comparisons) of the R package emmeans package. However, it is often better to use sensible, hypothesis-driven contrast coding than to compute all pairwise comparisons after the fact. 😃

+
+
+
+

2 Hypothesis Testing

+

Classical statistical tests such as the t-test can be found in the package HypothesisTests.jl.

+
+
+

3 Plotting ecosystem

+

Throughout this course, we have used the Makie ecosystem for plotting, but there are several alternatives in Julia.

+
+

3.1 Makie

+

The Makie ecosystem is a relatively new take on graphics that aims to be both powerful and easy to use. Makie.jl itself only provides abstract definitions for many components (and is used in e.g. MixedModelsMakie.jl to define plot types for MixedModels.jl). The actual plotting and rendering is handled by a backend package such as CairoMakie.jl (good for Quarto notebooks or rending static 2D images) and GLMakie.jl (good for dynamic, interactive visuals and 3D images). AlgebraOfGraphics.jl builds a grammar of graphics upon the Makie framework. It’s a great way to get good plots very quickly, but extensive customization is still best achieved by using Makie directly.

+
+
+

3.2 Plots.jl

+

Plots.jl is the original plotting package in Julia, but we often find it difficult to work with compared to some of the other alternatives. StatsPlots.jl builds on this, adding common statistical plots, while UnicodePlots.jl renders plots as Unicode characters directly in the REPL.

+

PGFPlotsX.jl is a very new package that writes directly to PGF (the format used by LaTeX’s tikz framework) and can stand alone or be used as a rendering backend for the Plots.jl ecosystem.

+
+
+

3.3 Gadfly

+

Gadfly.jl was the original attempt to create a plotting system in Julia based on the grammar of graphics (the “gg” in ggplot2). Development has largely stalled, but some functionality still exceeds AlgebraOfGraphics.jl, which has taken up the grammar of graphics mantle. Notably, the MixedModels.jl documentation still uses Gadfly as of this writing (early September 2021).

+
+
+

3.4 Others

+

There are many other graphics packages available in Julia, often wrapping well-established frameworks such as VegaLite.

+
+
+
+

4 Connecting to Other Languages

+

Using Julia doesn’t mean you have to leave all the packages you knew in other languages behind. In Julia, it’s often possible to even easily and quickly invoke code from other languages from within Julia.

+

RCall.jl provides a very convenient interface for interacting with R. JellyMe4.jl adds support for moving MixedModels.jl and lme4 models back and forth between the languages (which means that you can use emmeans, sjtools, DHARMa, car, etc. to examine MixedModels.jl models!). RData.jl provides support for reading .rds and .rda files from Julia, while RDatasets.jl provides convenient access to many of the standard datasets provided by R and various R packages.

+

PyCall.jl provides a very convenient way for interacting with Python code and packages. PyPlot.jl builds upon this foundation to provide support for Python’s matplotlib. Similarly, PyMNE.jl and PyFOOOF.jl provide some additional functionality to make interacting with MNE-Python and FOOOF from within Julia even easier than with vanilla PyCall. More recently, PythonCall.jl has proven to be a populat alternative to PyCall.jl.

+

For MATLAB users, there is also MATLAB.jl

+

Cxx.jl provides interoperability with C++. It also provides a C++ REPL mode, making it possible to treating C++ much more like a dynamic language than the traditional compiler toolchain would allow.

+

Support for calling C and Fortran is part of the Julia standard library.

+ + +
+ + Back to top
+ + +
+ + + + + \ No newline at end of file